Make npm run cov independent of node --test default discovery - #1512
Conversation
`cov` relied on `node --test` finding `fjs/emergent_testing/all.test.ts` through its default file patterns. Because that entrypoint is TypeScript, whether it is discovered depends on the Node version: measured with the previous command, v23.11.1 reports `tests 0` and prints no coverage report at all, while 22.22.2/24.18.1 report 2431 and 26.7.0 reports 2495, each with `all files 99.93`. Name the entrypoint explicitly so the signal no longer depends on discovery defaults, and drop the now-dead `**/module.f.ts` include glob (no authored `.f.ts` remains). The same dead extension goes from the Deno coverage filter, its regression proof, `deno.json` and the generated workflow. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017QjDaCd6HwQGoU2gwGyyGE
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
functionalscript | 9a15a99 | Commit Preview URL Branch Preview URL |
Aug 13 2026, 06:12 AM |
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017QjDaCd6HwQGoU2gwGyyGE
o2alexanderfedin
left a comment
There was a problem hiding this comment.
Approving. This fixes a real, reproducible defect, and I can confirm the fix works on the exact Node version where the old command was vacuous.
The headline result
My last several reviews carried a standing caveat: npm run cov reports a meaningless 100.00 over 0 tests in my environment, so coverage is unverifiable. That caveat is now closed. Node v23.11.0, same machine, same node_modules:
| tree | command | tests | report |
|---|---|---|---|
origin/main (3bc32ab9) |
npm run cov |
0 | vacuous |
this PR (65c886a7) |
npm run cov |
2431 / pass 2431 / fail 0 | all files 99.93 |
So naming fjs/emergent_testing/all.test.ts as the entrypoint does exactly what the TODO says it does, including on the one released Node line where default discovery finds nothing. I will stop excluding coverage from verification.
Verification battery (at 65c886a7, clean tree)
npx tsc --noEmit— exit 0.npm run prepackfrom a freshly cleaned tree — exit 0.npm test—pass: 2495, fail: 0, total: 2495, unchanged from main.npm run ci-updatethengit add -A && git diff --cached --exit-code— clean, so the generated.github/workflows/ci.ymlreally is the output of the editedcoverageIncludeconst.deno.jsonis not generated (it is only referenced in a comment), so its hand-edit is the right way to keep the third copy in sync.
Claims re-derived rather than assumed
- Dropping
**/module.f.tsand.f.tsfrom the Deno filter is safe.git ls-files | grep -cE '\.f\.ts$'is0at this head — no authored.f.tsremains, so neither filter can lose a module. - The unquoted glob in
package.jsonis not shell-expanded.--test-coverage-include=**/module.f.mjsis one word; sh would need a directory literally named--test-coverage-include=*to match, so it passes through intact. (There is a depth-1 match,fjs/module.f.mjs, so this was worth checking.) - The 2431-vs-2495 explanation in the TODO reproduces here. Of the 2431 top-level
✔lines on v23.11.0: 2376 carry the...inline marker and the remaining 55 are exactly the.throwtests, with no others in that set. That is the table the doc states, measured independently on a third Node version. - The rewritten
starcomment infjs/emergent_testing/module.f.mjsis accurate.usesInlineTestContextreturnstrueforbununconditionally and fornodebelow the26.0.0baseline, which is what the new comment says; the old comment also described the marker as' *'when the code has emitted' ...'(const star = o.inlineTestContext ? ' ...' : ''), so this corrects a stale comment as well.
CHANGELOG entry is present, sits in Unreleased, and links only /pull/1512 per §8.3. The TODO rewrite is honest about which part of the earlier diagnosis was right (discovery) and which framing was wrong («vacuous for several PRs» — it was vacuous only off-CI, on v23), and keeps both rather than quietly deleting the wrong one. I like that.
Nit, non-blocking: the CHANGELOG sentence «The dead .f.ts extension leaves the Node and Deno coverage filters» parses as departs from on the second read but as leaves behind on the first. «is dropped from» would be unambiguous.
Nothing blocking.
"leaves" read as "departs from" rather than "leaves behind". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017QjDaCd6HwQGoU2gwGyyGE
| }, | ||
| { | ||
| "run": "deno test --allow-read --allow-env --allow-sys --coverage && deno coverage --include='.*module\\.f\\.(ts|mjs)'" | ||
| "run": "deno test --allow-read --allow-env --allow-sys --coverage && deno coverage --include='.*module\\.f\\.mjs'" |
There was a problem hiding this comment.
Why do we run this line instead of deno task cov?
There was a problem hiding this comment.
No good reason — fixed in 959b765, the job now runs deno task cov.
The inlined form was pre-existing, and this PR made the cost of it visible: the coverage filter had three owners that had to be edited together — deno.json, coverageInclude in fjs/ci/deno/module.f.mjs, and the generated workflow. The Node jobs never had that problem because they run npm run cov and leave the filter to package.json; the Deno job just didn't follow the same rule.
So:
denoStepsemitsdeno task cov, andcoverageInclude/ thedenoTestfragment are gone.deno.jsonis now the only owner of both the permission set and the--includeregex.- The regression proof in
fjs/ci/deno/proof.f.mjsnow pins the delegation (assertEq(run, 'deno task cov')) instead of the filter string. Pinning the string was guarding the copy that could no longer drift once there is only one; what's worth guarding is that nobody re-inlines the command. fjs/ci/proof.f.mjs'sconfiguredPackageVersionassertion follows.
Verified with the pinned CI Deno (2.9.5) on a clean tree: deno install --frozen, then deno task cov → ok | 2431 passed (64 steps) | 0 failed, with the same filtered per-module report the inlined command produced. npx tsc clean, fjs t 2494 pass / 0 fail (2495 minus the deleted coverageInclude test), and fjs/ci/deno/module.f.mjs stays at 100% coverage.
Removing the export is a public API removal, so it carries a **BREAKING CHANGES:** CHANGELOG entry.
Generated by Claude Code
The Node jobs run `npm run cov` and let `package.json` own the coverage filter; the Deno job inlined `deno test … && deno coverage --include=…` instead, so the same filter had three owners — `deno.json`, the CI generator's `coverageInclude`, and the generated workflow. Delegate to `deno.json`'s `cov` task and drop `coverageInclude` along with the `denoTest` fragment. The regression proof now pins the delegation rather than the filter string, which is what can silently drift. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017QjDaCd6HwQGoU2gwGyyGE
|
@codex review |
o2alexanderfedin
left a comment
There was a problem hiding this comment.
Addendum — re-verified at the new head. My approval was posted at 65c886a7; the head has since moved to 9a15a993, which is not just a merge of main: it adds fddc22ac (CHANGELOG reword) and 959b7650 (run deno task cov in CI instead of inlining the command). I re-ran the battery at 9a15a993 and it still stands — re-approving in effect, nothing new to raise.
959b7650 is a genuine improvement over what I approved. Removing coverageInclude leaves deno.json as the single owner of the Deno filter, which is the same shape the Node side already has (package.json owns npm run cov), so the "keep these two semantically equal" comment I read as a maintenance hazard is now a one-owner invariant instead. The public-surface removal is correctly marked **BREAKING CHANGES:** per §8.4 and links only /pull/1512. The CHANGELOG reword also fixes the "leaves the … filters" ambiguity I nitted — thanks.
At 9a15a993:
npm run ci-updatethengit add -A && git diff --cached --exit-code— clean, so the regeneratedci.ymlstep really isdeno task cov.grep -rn coverageInclude fjs/outsidetodo/— no hits, so the export is gone with no dangling importer (fjs/ci/proof.f.mjsdropped its import too).npm test—pass: 2495, fail: 0, total: 2495against 2496 on currentmain(20cd6aa9). The −1 is exactly the retiredcoverageIncludeproof case; the merge of #1509 contributes +1, so 2495 + 1 − 1 = 2495. Accounted for, not a regression.npm run covon Node v23.11.0 —tests 2431 / pass 2431 / fail 0,all files 99.93. The headline result is unchanged: still a real coverage report wheremainreportstests 0.
|
Codex Review: Didn't find any major issues. 🚀 Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Addresses the Make
npm run covreport real coverage item underRemaining after stage 1 in
todo/migrate-typescript-to-mjs.md.What was actually wrong
The diagnosis recorded in the issue was right:
covpassed no patharguments, so
node --testfell back to its default file patterns(
*.test.*,test.*,*-test.*,test-*.*,*_test.*,test/**).The repo's proofs are
proof.f.mjs/module.f.mjsand match none ofthem; the only file in the tree that does is
fjs/emergent_testing/all.test.ts. Since that entrypoint is TypeScript,whether it is discovered depends on the Node version.
The "vacuous for several PRs" framing was the part that did not hold up.
Measured with the pre-fix command
(
node --test --experimental-test-coverage --test-coverage-include=**/module.f.ts --test-coverage-include=**/module.f.mjs),run against real Node tarballs:
all files 99.93all files 99.93all files 99.93So every version CI runs (
22.23.2,24.18.1,26.7.0) already reportedreal coverage. v23 — released, EOL, not in CI — is the version that reports
nothing, which is what the fragility looks like when it bites.
The change
package.json:covnamesfjs/emergent_testing/all.test.tsexplicitly,so the signal no longer depends on discovery defaults, and drops the dead
--test-coverage-include=**/module.f.tsglob — no authored.f.tsremains after stage 1.
deno.json: the same dead.f.tsalternative leaves the Deno coveragefilter.
fjs/ci/deno/module.f.mjs, its proof,fjs/ci/proof.f.mjsand thegenerated
.github/workflows/ci.yml: the Deno job now runsdeno task covinstead of inliningdeno test … && deno coverage --include=…, sodeno.jsonis the singleowner of the permission set and the coverage filter — the shape the Node
side already had via
npm run cov. BREAKING CHANGES: thecoverageIncludeexport is gone with it; the regression proof now pinsthe delegation, which is what can actually drift.
fjs/emergent_testing/module.f.mjs: thestarcomment said the marker is"non-empty only for Bun", which stopped being true when the inline
strategy was extended to Node below the 26 baseline.
todo/migrate-typescript-to-mjs.md: the item is checked off, with themeasurements above kept in place of the stale framing.
The 2431 vs 2495 gap
The issue asked for this to be understood rather than papered over. It is
not a second defect: it is exactly the 64 sub-tests reachable only through a
test function's return value.
usesInlineTestContext(
fjs/effects/node/module.f.mjs) selects the flattened registrationstrategy below the Node 26 baseline, where
inlineContextruns sub-testsinside their parent registration instead of declaring them to the framework,
so the framework counts the parent only. Node 26 uses native
expectFailurewith nested registration and counts all 2495, matching
npm test(node ./fjs/module.mjs t).Measured on 26.7.0: 2431 top-level plus 64 nested
✔lines. On 24.18.1:2431 top-level, 0 nested, of which 2376 carry the
...inline marker — theother 55 are exactly the throw-tests, which never produce sub-tests and so
never get the marker. Same work executed either way; only the reported count
differs.
Verification
npx tsc— clean.node ./fjs/module.mjs t— 0 fail. The count moves from 2495 to 2494 onthis branch alone, the retired
coverageIncludeproof case; mergingmain(Add reduceValue2DefaultBranch test case to btree remove proof #1509) brings it back to 2495.npm run covafter the change: 22.22.2 → 2431, 24.18.1 → 2431,26.7.0 → 2495, each
all files 99.93; 23.11.1 goes fromtests 0and noreport to
tests 2431with a full report.deno task covon the pinned CI Deno 2.9.5, afterdeno install --frozen—
ok | 2431 passed (64 steps) | 0 failed, same filtered per-modulereport the inlined command produced.
npm run ci-updateis idempotent and the workflow diff is the singleexpected line.
Rust is untouched, so
cargocommands were not run.🤖 Generated with Claude Code
https://claude.ai/code/session_017QjDaCd6HwQGoU2gwGyyGE