mint install XCTestHTMLReport/XCTestHTMLReport@4.0.0rc1

4.0.0

4.0 ships as a release-candidate line

4.0 rewrites how result bundles are read, and the only way to learn whether
that rewrite holds against real projects is to put it in front of real
projects. So 4.0.0 is being cut as a series of release candidates —
4.0.0rc1, 4.0.0rc2, … — rather than one stable tag, and each is published
as a GitHub pre-release. brew install xctesthtmlreport and mint install XCTestHTMLReport/XCTestHTMLReport keep resolving to the 3.x stable line until
4.0.0 final, so a candidate is always opt-in — pin the candidate's own tag:

mint install XCTestHTMLReport/XCTestHTMLReport@<candidate-tag>

Each candidate's release page names its own tag and attaches a signed,
notarized universal binary built from it.

The first candidate is the migration plus a conservative refresh.
Everything below — the xcresulttool migration, the --json schema,
--result-reader, and the report changes both readers now share — is in it,
and is what we are asking for soak time on. The report's own markup and
styling are deliberately left close to 3.x there, so that any regression you
hit points at the reader rewrite rather than at a redesign landing on top of
it.

The Xcode-native report redesign lands later in the RC line, with its own
notes appended to this document as it arrives. The candidate line ends when
the reader rewrite has gone a full cycle without a parity or correctness
report against it.

Please file anything you find against
the 4.0 milestone;
a bug found in a candidate is the entire point of the exercise.

Why a major release

Every earlier version of xchtmlreport read result bundles through
xcresulttool's legacy API. Apple has deprecated that API and will remove
it; when that happens, every earlier version stops working entirely.

4.0 reads the current xcresulttool format directly. The new format is not
a superset of the old one — a handful of things the report used to show have
no source anymore — so this migration is unavoidably a breaking release. We
chose to take every consequence of that break once, deliberately, in one
version, rather than shipping a release that breaks again each time Apple
removes another piece: where the new format cannot supply something, the old
reader stops showing it too, and the two produce the same report by
construction. A differential test suite renders every fixture through both
readers on every CI run and holds the difference to a short, reviewed
allow-list — parity is enforced, not assumed.

Nothing changes in how you invoke the tool. Everything below is about what
comes out.

⚠️ Breaking change: --json is now our own documented schema

--json used to dump xcresulttool's legacy object graph verbatim — every
scalar wrapped in {"_value": ...}, keys named by Apple's internal types.
That graph is Apple's internal shape and disappears with the legacy
commands, so there is no version of the future in which it survives.

report.json is now our own schema: documented, versioned, and identical
whichever reader produced it.

{
  "runs" : [ ... ],
  "schemaVersion" : "1.0.0"
}

docs/json-schema.md
is the contract: field names and nesting with a complete worked example,
enum encodings, the null rule, duration and timestamp formats, array
ordering guarantees, and the version policy. Consumers should read
schemaVersion first; the top-level field exists so nobody ever has to
sniff keys to guess a version again.

If you parsed the old dump: there is no compatibility mode. The old output
could not be kept — the command producing it is being removed. Port your
consumer against the contract document once, and the schemaVersion policy
tells you exactly what may change afterwards and how it will be signalled.

New: --result-reader auto|legacy|modern

xchtmlreport now carries both readers and picks one per run:

• auto (the default) prefers the legacy reader while the toolchain still
  offers the legacy commands, and falls back to the modern reader once they
  are gone.
• modern forces the new-format reader — this is what CI uses to prove the
  future-proof path works end to end today.
• legacy forces the old reader; on a toolchain without the legacy
  commands this is an error, never a silent substitution.

The XCHR_RESULT_READER environment variable sets the default when the
flag is absent.

Report changes, on both readers

These are the once-only consequences of holding the two readers to one
output. Each was accepted by an explicit ruling during the migration, and
each applies identically to both readers.

Attachment files on disk are named by content, not by title

Exported attachment files are now named <payloadId>.<ext> — the payload's
content-addressed store id — instead of a prettified attachment title.
Display names in the report are unchanged; only the file names on disk
change. If your pipeline reaches into the report directory for attachment
files by name, it needs updating.

This is not cosmetic: Xcode 26.2 gives every automatic screen recording in
a session the same display name, so name-derived exports collapsed distinct
recordings onto one path — every video row played the same file — and
concurrent exports raced on it, intermittently degrading the report (#449).
Content-addressed names are unique per payload by construction, identical
byte-for-byte payloads deduplicate to one file, and the export is
idempotent.

The run log is named by run, not by internal reference

The exported run log is now <run-identifier-digest>.log (the same
path-digest scheme every element id in the report uses, #430) instead of a
backend-internal reference name. One log per run, identical on both
readers — a multi-action bundle no longer overwrites one action.log-style
file per action.

Parameterized Swift Testing cases render as one test

A Swift Testing @Test(arguments:) case used to render as N
pseudo-repetitions ("3 succeeded", rows titled Iteration 0) — retry
semantics invented for what are argument variations. It now renders as one
test case on both readers. True retries (-retry-tests-on-failure) are
unaffected and keep their per-iteration rows.

Skipped tests now show their reason

The skip reason (XCTSkip's message) was always in the result bundle; the
old reader simply never read it. Both readers now append it as a row on the
skipped test. Your reports gain a row they never had — that is a fix, not a
regression.

Failure rows render where the assertion fired

Assertion-failure rows are now interleaved into the activity timeline by
their timestamp on both readers, so a failure appears at the point in the
test where it happened rather than after everything else. JUnit reports
inherit this reposition: failure entries can appear at a different place
in a test case's output than in 3.x. Expected failures
(XCTExpectFailure) render nothing, exactly as before — on both readers.

Activity types and per-activity durations leave the report

The new format carries neither the legacy activity taxonomy (the five
activityType constants that drove row styling, including the
user-created-activity highlight) nor per-activity finish times. A field
only one reader could fill would poison the parity guarantee, so both
readers drop them: activity rows lose their type-specific styling and their
(1.23s) duration suffix. Suite and bundle rows show (0.00s) on the
modern reader for the same reason — the new format reports no duration for
groups.

What still differs between the two readers

Four differences could not be removed and are declared instead of hidden —
the differential suite fails if any other difference ever appears, and also
if one of these four quietly disappears:

--json output additionally carries testCase.arguments, which only the
modern reader can populate (the legacy format has no counterpart) — see the
contract document.

Fixes

Independent of the migration, and applying to both readers:

• A missing --output directory is created, not discovered at the end.
  xchtmlreport <bundle> --output /missing/dir used to parse the bundle,
  export attachments and render the whole report, then die on the final write
  with The file "index.html" doesn't exist. — a message that named neither
  the real problem nor anything you could act on, and arrived after the work
  was already done. The directory (with intermediates) is now created before
  any work starts. One that genuinely cannot be created is a usage error: it
  names the path and the reason and exits 64, like the ...

What's Changed

• Call the Homebrew bump from the release instead of an event trigger by @tylervick in #400
• 3.0.1-pre.69b32fe Version Bump by @github-actions[bot] in #399
• Don't bump Homebrew from the release; the formula is autobumped by @tylervick in #401
• Add Dependabot for GitHub Actions and SwiftPM by @tylervick in #402
• Add shellcheck, SwiftFormat and SwiftLint by @tylervick in #410
• Pin third-party actions to SHAs and scope workflow tokens by @tylervick in #413
• Use fputs for stderr writes in Logger by @tylervick in #416
• Add an opt-in pre-commit hook by @tylervick in #415
• Add Swift Testing (@test) fixture and structural assertions by @tylervick in #417
• Bump maxim-lobanov/setup-xcode from 1.6.0 to 1.7.0 in the actions-minor-and-patch group across 1 directory by @dependabot[bot] in #403
• Bump the swift-minor-and-patch group with 4 updates by @dependabot[bot] in #408
• Bump github.com/onevcat/rainbow from 3.1.5 to 4.2.1 by @dependabot[bot] in #409
• Bump actions/checkout from 2 to 7 by @dependabot[bot] in #407
• Bump actions/upload-artifact from 4 to 7 by @dependabot[bot] in #404
• Delete the GitHub Pages workflow by @tylervick in #418
• Bump codecov/codecov-action from 4.6.0 to 7.0.0 by @dependabot[bot] in #421
• Stop launching the app in sample UI tests that never use it by @tylervick in #423
• Drop the unused SwiftFormat package dependency by @tylervick in #424
• Share one build and one simulator across fixture generation by @tylervick in #425
• Publish releases with the runner's gh CLI instead of a third-party action by @tylervick in #427
• fix: derive report identifiers from bundle content so runs are reproducible by @tylervick in #430
• Download the release archive to a path the release step can find by @tylervick in #431

Full Changelog: 3.0.0...3.0.1rc1

3.0.0

⚠️ Breaking change: xchtmlreport now exits non-zero on a degraded report

Previously xchtmlreport exited 0 with "Report successfully created" even when it had failed to parse parts of the result bundle. There was no way for a CI pipeline — or a person — to tell a complete report from a silently incomplete one.

It now exits 3 when the report is degraded.

The report is still written in every case. This is purely an exit-code change.

If this breaks your pipeline

Pass --lenient to restore the previous always-zero behaviour:

xchtmlreport --lenient TestResults.xcresult

But before you do — a non-zero exit means some of your report really is missing. --lenient is an escape hatch for unblocking a pipeline today, not a fix. The printed fault list tells you what was lost.

Fixed: attachments could silently go missing from reports

Two attachments sharing a single payload id raced during export: both were written to the same temporary path, and the loser's file was deleted out from under it before it could be moved into place. The attachment was then absent from the generated report, with no error.

This affected ordinary UI test runs — screen recordings are captured automatically, and -retry-tests-on-failure is common in CI. If you have seen attachments intermittently missing from reports, this is a likely cause.

The test suite now runs without credentials

Test fixtures previously came from a private storage bucket via repository secrets, which GitHub does not expose to pull requests from forks. No outside contributor could ever get a green check — and on a fresh clone the test target did not even compile.

./prepareTestResults.sh now generates fixtures locally, and CI runs exactly that. Contributors can run:

./prepareTestResults.sh
swift test

This also fixes the script itself, which hardcoded an "iPhone 12" simulator that no longer exists in current Xcode, and removes a network fetch from the sample test suite.

Other changes

• Logger.error and Logger.warning now write to stderr rather than stdout
• Documented exit codes in the README and the build/test workflow in CONTRIBUTING
• CI tests against the latest Xcode plus one major version back
• Removed the long-dead .travis.yml

Known issues

• CoreTests.testRetryFunctionalityJunit is skipped pending #378 — its JUnit expectations drift on Xcode 26. The assertions are unchanged and will run again once resolved.
• Some XCResultKit decode failures are still not surfaced as faults, so exit 0 is not yet a complete guarantee of a perfect report. Tracked in #386.

Upgrading

brew upgrade xctesthtmlreport

If you pin a version in CI, review the exit-code change above before upgrading.

What's Changed

• Handle successful test with failed assertions during Junit report generation by @acecilia in #367
• Reduce the memory footprint when -z option is used by @aleksandar-angelov-smule in #372
• 2.5.2-pre.bc4e30e Version Bump by @github-actions[bot] in #375
• Make the tool report its own failures, and make the test suite runnable by anyone by @tylervick in #379
• Repair the release pipeline for 3.0 by @tylervick in #394
• Don't bump the version after a release candidate by @tylervick in #396
• Stop asserting simulator reliability in testResultStatusCount by @tylervick in #397

New Contributors

• @aleksandar-angelov-smule made their first contribution in #372

Full Changelog: 2.5.1...3.0.0

What's Changed

• Handle successful test with failed assertions during Junit report generation by @acecilia in #367
• Reduce the memory footprint when -z option is used by @aleksandar-angelov-smule in #372
• 2.5.2-pre.bc4e30e Version Bump by @github-actions[bot] in #375
• Make the tool report its own failures, and make the test suite runnable by anyone by @tylervick in #379
• Repair the release pipeline for 3.0 by @tylervick in #394

New Contributors

• @aleksandar-angelov-smule made their first contribution in #372

Full Changelog: 2.5.1...3.0.0rc1

What's Changed

• 2.5.1-pre.7bbc8db Version Bump by @github-actions in #360
• Bump actions/download-artifact from 2 to 4.1.7 in /.github/workflows by @dependabot in #363
• Fix skipped test cases that have no logs by @Thoromis in #362
• Bump dawidd6/action-download-artifact from 2 to 6 in /.github/workflows by @dependabot in #368
• Performance optimizations by @yusufozgul in #370
• upgrade upload-artifact@v3 by @tylervick in #374

New Contributors

• @Thoromis made their first contribution in #362
• @yusufozgul made their first contribution in #370

Full Changelog: 2.5.0...2.5.1

What's Changed

• 2.4.3-pre.4005e15 Version Bump by @github-actions in #352
• Update source for XCResultKit to support Xcode16 schema changes by @juanitos23 in #359

New Contributors

• @juanitos23 made their first contribution in #359

Full Changelog: 2.4.2...2.5.0

What's Changed

• Lazy load video and everything else by @kevin-brotcke in #350
• Support mixed children test types in test suites by @kevin-brotcke in #349

Full Changelog: 2.4.1...2.4.2

What's Changed

• 2.4.1-pre.ca8c215 Version Bump by @github-actions in #345
• bump xcresultkit to use xcresulttool version 3.44 by @tylervick in #346

Full Changelog: 2.4.0...2.4.1

What's Changed

• Create pages.yml by @tylervick in #327
• Update ci.yaml with xcode 15, 14, 13 and macos-13, replace xcode-select action by @tylervick in #335
• Lazy load screenshots by @kevin-brotcke in #339
• Update ci.yml by @tylervick in #340
• Add parallel test processing by @tothszabi in #334
• Update actions for xcode 15 by @tylervick in #343

New Contributors

• @kevin-brotcke made their first contribution in #339
• @tothszabi made their first contribution in #334

Full Changelog: 2.3.4...2.4.0

What's Changed

• 2.3.4-pre.27cc68 Version Bump by @github-actions in #312
• update XCResultKit sha for dateEncodingStrategy fix by @tylervick in #315
• Added gif attachment type support by @gokberkince in #314
• add test artifact upload job by @tylervick in #317
• add codecov action, small renames by @tylervick in #316
• fix outstanding sonar issues by @tylervick in #319
• Update README.md by @tylervick in #318
• Update readme to add guide for json flag by @cozzin in #320
• Preserve aspect ratio when downsizing images, add custom scaling factor flag by @tylervick in #325

New Contributors

• @gokberkince made their first contribution in #314
• @cozzin made their first contribution in #320

Full Changelog: 2.3.3...2.3.4