Fixture coverage for the redesign: expectedFailure + PNG-attachment cases (refs #439) - #453

The gate shared by `test.yml` and `toolchain-drift.yml` asked only whether each
`.xcresult` had an `Info.plist`. When a runner's simulator vanished mid-run
("Unable to find a device matching the provided destination specifier"), the
`|| true` in `prepareTestResults.sh` swallowed the failure and `xcodebuild`
still left a stub bundle behind — `Info.plist`, a `Data` directory, no tests.
That stub passed the gate, failed the suite later with a confusing error
instead of failing fast at generation, and came one step from being SAVED to
the fixture cache under a valid key, where it would have poisoned every
restore until the key rotated. The cache is the dangerous half.

`scripts/verify_fixtures.sh` now asserts real test data per bundle.
`xcresulttool get test-results summary` exits 0 on a stub and reports
`"totalTestCount": 0`, so the count is the assertion, not the exit status; the
command is the modern surface `ModernResultReader` already reads through, so
the gate needs no toolchain the tool does not. Both workflows call the one
script — the two inline copies could drift apart, and one of them is the gate
in front of the cache — and every bundle is checked before it exits, so a run
names every offender rather than the first.

Verified against a real bundle (accepted: `Real.xcresult: 2 tests`), against a
stub reproduced the way the incident produced one, by pointing `xcodebuild
test-without-building` at a device id that does not exist (rejected, exit 1,
naming the bundle), and against a corrupted bundle whose summary command fails
outright (rejected, with xcresulttool's own error echoed).

In `toolchain-drift.yml` verification becomes its own step. Generation is
allowed to fail tests, so folding the check into it conflated "generation blew
up" with "generation produced empty bundles" — distinct faults in the issue
that workflow files. The suite, the render and the cache save now hang off the
verify step instead of off generation. `test.yml` needs no reordering:
`actions/cache` saves post-job under `post-if: success()`, so a failing gate
already blocks the poisoned save.

Also from the #453 review: the `Expected Failure` doc comment is scoped to
`TestResults.xcresult`, since `RetryTests.testInUnknownState` records one too
and lands in `RetryResults.xcresult`; and the programmatic PNG pins its
renderer to scale 1, because `UIGraphicsImageRenderer(size:)` takes points and
would emit a 16×16 or 24×24 attachment on a Retina simulator instead of the
8×8 its comment promises.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Reject stub fixture bundles at the verify gate (fixes #454)

The gate shared by `test.yml` and `toolchain-drift.yml` asked only whether each
`.xcresult` had an `Info.plist`. When a runner's simulator vanished mid-run
("Unable to find a device matching the provided destination specifier"), the
`|| true` in `prepareTestResults.sh` swallowed the failure and `xcodebuild`
still left a stub bundle behind — `Info.plist`, a `Data` directory, no tests.
That stub passed the gate, failed the suite later with a confusing error
instead of failing fast at generation, and came one step from being SAVED to
the fixture cache under a valid key, where it would have poisoned every
restore until the key rotated. The cache is the dangerous half.

`scripts/verify_fixtures.sh` now asserts real test data per bundle.
`xcresulttool get test-results summary` exits 0 on a stub and reports
`"totalTestCount": 0`, so the count is the assertion, not the exit status; the
command is the modern surface `ModernResultReader` already reads through, so
the gate needs no toolchain the tool does not. Both workflows call the one
script — the two inline copies could drift apart, and one of them is the gate
in front of the cache — and every bundle is checked before it exits, so a run
names every offender rather than the first.

Verified against a real bundle (accepted: `Real.xcresult: 2 tests`), against a
stub reproduced the way the incident produced one, by pointing `xcodebuild
test-without-building` at a device id that does not exist (rejected, exit 1,
naming the bundle), and against a corrupted bundle whose summary command fails
outright (rejected, with xcresulttool's own error echoed).

In `toolchain-drift.yml` verification becomes its own step. Generation is
allowed to fail tests, so folding the check into it conflated "generation blew
up" with "generation produced empty bundles" — distinct faults in the issue
that workflow files. The suite, the render and the cache save now hang off the
verify step instead of off generation. `test.yml` needs no reordering:
`actions/cache` saves post-job under `post-if: success()`, so a failing gate
already blocks the poisoned save.

Also from the #453 review: the `Expected Failure` doc comment is scoped to
`TestResults.xcresult`, since `RetryTests.testInUnknownState` records one too
and lands in `RetryResults.xcresult`; and the programmatic PNG pins its
renderer to scale 1, because `UIGraphicsImageRenderer(size:)` takes points and
would emit a 16×16 or 24×24 attachment on a Retina simulator instead of the
8×8 its comment promises.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Route the last two fixture gates through the shared script (#470 review)

The verify gate this PR added covered test.yml and toolchain-drift.yml, but
main had four inline copies, not two. The two left behind were the ones that
mattered most.

pages.yml was the hole: it restores, generates and post-job saves all three
bundles under the same key test.yml restores from, on every merge to main,
while its inline gate checked only TestResults for an Info.plist. A stub from
an incident like #454 on a merge run passed that gate, published an empty demo,
and then poisoned the shared cache for every PR run until the key rotated. It
now runs the shared script over all three bundles — not just the one it
renders, because it writes all three. Failing there also blocks the save, since
the cache action's post step runs under `post-if: success()`.

pages-release.yml has no cache to poison, but it pushes an immutable per-tag
render, so a stub becomes a permanently hosted empty demo under /v/<tag>/. It
verifies just TestResults, the only bundle it publishes: failing a release over
a stub Sanity or Retry bundle nothing in the job reads would be a false
blocker. No version-skew risk — a tag push runs the workflow file as it existed
at the tag and that job checks out the tag with no ref: override, so the
workflow and the script always travel together.

Zero inline copies remain, so the script's own header no longer undercounts its
callers.

Also corrects the Expected-Failure doc comment, which the rider that rewrote it
left wrong in both halves. TestResults.xcresult carries two Expected Failure
rows, not one: testExpectedFailure and SwiftTestingSuite.knownIssue, both in the
SampleAppUnitTests target, so prepareTestResults.sh's
-skip-testing:SampleAppUITests/RetryTests does not exclude the second. Measured
with `xcresulttool get test-results tests` — 21 rows, {'Passed': 12, 'Failed':
6, 'Expected Failure': 2, 'Skipped': 1} — which is what CoreTests' `all -
skipped - 2` already encoded. The comment's second half was stale besides: #439
landed and gave the status its own Status.expectedFailure and glyph, so it no
longer renders as .unknown.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>