Cross-backend identifier normalizer and baseline render capture (#391) by tylervick · Pull Request #437 · XCTestHTMLReport/XCTestHTMLReport

tylervick · 2026-08-12T21:29:33Z

Implements Tasks 1 and 2 of the xcresulttool legacy migration plan (#391, milestone 4.0).

Task 1 — cross-backend identifier normalizer

Tests/XCTestHTMLReportTests/ReportNormalizer.swift: normalizeIdentifiers(_:) replaces every post-fix: derive report identifiers from bundle content so runs are reproducible #430 IdentifierPath digest (\b[0-9a-f]{32}\b, never RFC-4122 UUIDs) with the literal ID.
Two tests appended to the existing ReproducibilityTests.swift (shipped by fix: derive report identifiers from bundle content so runs are reproducible #430): one pins the replacement to digests and nothing else, one proves the pattern actually matches a rendered report so the Task 12 differential can't silently degrade to comparing raw identifiers.
The normalizer is not wired into any same-backend comparison — it exists solely for the future cross-backend differential (Task 12). The pre-existing fix: derive report identifiers from bundle content so runs are reproducible #430 reproducibility tests stay green and un-normalized.

Task 2 — opt-in baseline capture

Tests/XCTestHTMLReportTests/BaselineCaptureTests.swift: writes raw (un-normalized) renders of all three fixtures to $XCHR_BASELINE_DIR; skips with an explicit message when the variable is unset. Guards the Task 5a refactor: capture before, capture after, diff -r, within one fixture generation.

Plan amendments

The plan's snippets had never been compiled; three corrections were made and the plan document amended to match in the same commits:

sanityResultsUrl helper was missing — the Task 1 test snippet references it, but the shipped ReproducibilityTests.swift only had URL helpers for TestResults and RetryResults. Added beside them.
try! fails the pre-commit SwiftLint gate (force_try error) — the normalizer's constant regex now unwraps through a preconditionFailure closure instead. (Also swapped String(decoding:as:) for failable String(bytes:encoding:) to satisfy optional_data_string_conversion.)
Task 2's test was named testCaptureNormalizedRenders — a leftover from the pre-fix: derive report identifiers from bundle content so runs are reproducible #430 plan revision; it captures raw bytes, so it is named testCaptureRawRenders.

Verification

Step 0 gate: IdentifierPath.swift present (fix: derive report identifiers from bundle content so runs are reproducible #430 / a28b131); ReproducibilityTests green before any change.
TDD followed: normalizer test failed with cannot find 'normalizeIdentifiers' in scope before the implementation.
Baseline capture verified both ways: skips without XCHR_BASELINE_DIR, writes three non-empty .html files with it.
Full suite after regenerating fixtures: 31 tests, 0 failures, 2 skipped (the opt-in capture, plus the pre-existing skip).

🤖 Generated with Claude Code

Summary by CodeRabbit

Tests
- Added reproducibility checks for generated HTML reports using bundled result fixtures.
- Added validation that report output is non-empty, complete, and consistently normalized across runs.
- Added baseline capture coverage for comparing raw report output over time.
Documentation
- Updated the migration plan to reflect completed verification tasks and clarified report baseline terminology.

coderabbitai · 2026-08-12T21:29:54Z

📝 Walkthrough

Walkthrough

The test suite adds digest normalization, reproducibility checks for SanityResults.xcresult, and opt-in raw HTML baseline capture for three bundled xcresult fixtures. The migration plan records the completed implementation and verification steps.

Changes

Reproducibility and Baselines

Layer / File(s)	Summary
Identifier normalization and reproducibility checks `Tests/XCTestHTMLReportTests/ReportNormalizer.swift`, `Tests/XCTestHTMLReportTests/ReproducibilityTests.swift`, `docs/superpowers/plans/...`	A compiled regex replaces 32-character lowercase hexadecimal identifiers with `ID`. Tests validate isolated normalization and normalized rendered HTML from `SanityResults.xcresult`.
Raw HTML baseline capture `Tests/XCTestHTMLReportTests/BaselineCaptureTests.swift`, `docs/superpowers/plans/...`	An opt-in test renders three fixtures, writes non-empty UTF-8 HTML files, rejects missing fixtures, and checks the expected output filenames.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Mergeability Score: 🟡 Moderate · up to bb42d

The new cross-backend normalizer can leave rendered identifiers unchanged when digests follow hexadecimal identifier text, causing future differential comparisons to report false differences. Update the digest boundaries in the implementation and plan example before merging.

Possibly related PRs

XCTestHTMLReport/XCTestHTMLReport#426: Plans the baseline capture and identifier-normalization work implemented here.
XCTestHTMLReport/XCTestHTMLReport#435: Defines related reproducibility and deterministic identifier handling tests.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies both main changes: cross-backend identifier normalization and baseline render capture.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch tylervick/diff-normalizer-391

_{Comment @coderabbitai help to get the list of available commands.}

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@Tests/XCTestHTMLReportTests/ReportNormalizer.swift`:
- Around line 18-23: The identifierPattern in ReportNormalizer.swift must use
hexadecimal-character boundaries instead of \b so digests embedded in
device_<digest> are matched; replace its pattern with
(?<![0-9a-f])[0-9a-f]{32}(?![0-9a-f]). Apply the same pattern update in
docs/superpowers/plans/2026-08-10-xcresulttool-legacy-migration.md lines 198-208
for the copied plan example.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 52470804-8bbc-4aab-9e4a-ba174731e4a1

📥 Commits

Reviewing files that changed from the base of the PR and between e53ce0a and bb42d95.

📒 Files selected for processing (4)

Tests/XCTestHTMLReportTests/BaselineCaptureTests.swift
Tests/XCTestHTMLReportTests/ReportNormalizer.swift
Tests/XCTestHTMLReportTests/ReproducibilityTests.swift
docs/superpowers/plans/2026-08-10-xcresulttool-legacy-migration.md

coderabbitai · 2026-08-12T21:35:51Z

+private let identifierPattern: NSRegularExpression = {
+    guard let pattern = try? NSRegularExpression(pattern: "\\b[0-9a-f]{32}\\b") else {
+        preconditionFailure("The identifier pattern is a constant and must compile")
+    }
+    return pattern
+}()


🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash set -e printf '%s\n' '--- implementation ---' cat -n Tests/XCTestHTMLReportTests/ReportNormalizer.swift | sed -n '1,120p' printf '%s\n' '--- plan excerpt ---' cat -n docs/superpowers/plans/2026-08-10-xcresulttool-legacy-migration.md | sed -n '175,225p' printf '%s\n' '--- related tests and usages ---' rg -n -C 3 'ReportNormalizer|identifierPattern|device_|[0-9a-f]\{32\}' Tests docs printf '%s\n' '--- Swift availability ---' command -v swift || true

Repository: XCTestHTMLReport/XCTestHTMLReport

Length of output: 13266

🏁 Script executed:

#!/bin/bash set -e printf '%s\n' '--- IdentifierPath and identifier rendering ---' rg -n -C 5 'struct IdentifierPath|class IdentifierPath|enum IdentifierPath|IdentifierPath|device_' Classes Tests | head -n 240 printf '%s\n' '--- normalizer call sites ---' rg -n -C 5 'normalizeIdentifiers|identifierPattern' Tests docs printf '%s\n' '--- boundary probe ---' python3 - <<'PY' import re pattern = re.compile(r"\b[0-9a-f]{32}\b") hex_digest = "0123456789abcdef0123456789abcdef" for value in [ hex_digest, "device_" + hex_digest, "id=" + hex_digest, "x" + hex_digest, "_" + hex_digest, hex_digest + "_suffix", hex_digest + "0", "g" + hex_digest, ]: print(f"{value!r} -> {pattern.sub('ID', value)!r}") PY

Repository: XCTestHTMLReport/XCTestHTMLReport

Length of output: 21795

Use hexadecimal boundaries for IdentifierPath digests — \b does not match the digest in device_<digest>, so cross-backend normalization leaves rendered identifiers unchanged. Use (?<![0-9a-f])[0-9a-f]{32}(?![0-9a-f]) in both the implementation and the copied plan example.

📍 Affects 2 files

Tests/XCTestHTMLReportTests/ReportNormalizer.swift#L18-L23 (this comment)

docs/superpowers/plans/2026-08-10-xcresulttool-legacy-migration.md#L198-L208

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@Tests/XCTestHTMLReportTests/ReportNormalizer.swift` around lines 18 - 23, The identifierPattern in ReportNormalizer.swift must use hexadecimal-character boundaries instead of \b so digests embedded in device_<digest> are matched; replace its pattern with (?<![0-9a-f])[0-9a-f]{32}(?![0-9a-f]). Apply the same pattern update in docs/superpowers/plans/2026-08-10-xcresulttool-legacy-migration.md lines 198-208 for the copied plan example.

tylervick added 2 commits August 12, 2026 14:25

test: normalize IdentifierPath digests for cross-backend diffing

050df68

test: add opt-in baseline render capture for refactor guarding

bb42d95

tylervick added this to the 4.0 milestone Aug 12, 2026

coderabbitai Bot reviewed Aug 12, 2026

View reviewed changes

tylervick merged commit 6dbe749 into main Aug 12, 2026
8 checks passed

tylervick deleted the tylervick/diff-normalizer-391 branch August 12, 2026 21:44

Navigation Menu

Cross-backend identifier normalizer and baseline render capture (#391) - #437

Cross-backend identifier normalizer and baseline render capture (#391)#437
tylervick merged 2 commits into
mainfrom
tylervick/diff-normalizer-391

tylervick commented Aug 12, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Aug 12, 2026 •

edited

Loading

Walkthrough

Changes

Uh oh!

coderabbitai Bot left a comment

Uh oh!

coderabbitai Bot Aug 12, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

tylervick commented Aug 12, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Task 1 — cross-backend identifier normalizer

Task 2 — opt-in baseline capture

Plan amendments

Verification

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Aug 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

coderabbitai Bot Aug 12, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

tylervick commented Aug 12, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Aug 12, 2026 •

edited

Loading