emergent_testing: an unreadable returned tree fails its leaf, not the run - #1758
Conversation
… run The shared walk enumerates whatever a proof returned, which runs user code: an enumerable getter or a proxy trap throws inside `collectTests`. Unguarded that unwinds the whole traversal, so one hostile value costs the results of every module that had already passed — including the modules that would have reported failures worth seeing. Adds the `catch` operation that `todo/hostile-proof-values.md` specified, with handlers in the real Node runner (`tryCatch`) and the virtual one (`ok(ok(f()))` — a pure runner still cannot catch, which is the bargain `sandbox` already makes). `sandbox` could not carry this: the virtual runner's is a deliberate pass-through whose thunk answers a `SandboxResult`, so routing a tree walk through it would break every fixture. The read now happens before the leaf is reported, so the failure is part of what gets reported rather than a correction after the fact, and the leaf keeps its own duration while the reporter receives the reading failure to describe. The module's *exported* tree is still read unguarded, deliberately: there is no leaf to attribute it to. That asymmetry is now written down in both the code and the issue. This is also the prerequisite for step 7 of share-browser-console-runner.md — the browser catches this today, so sharing the traversal without it would have lost a behaviour. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016PyLwDNkPQM1uD1Tg5ApBg
|
You have reached your Codex usage limits for security reviews. Please try again later. |
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016PyLwDNkPQM1uD1Tg5ApBg
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
functionalscript | 912c71f | Commit Preview URL Branch Preview URL |
Aug 28 2026, 05:32 PM |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 038bfbc9a5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Review found try/catch in proof.f.mjs — outside the language the file claims to be written in, and the same defect the repository has been removing rather than excusing. The irony is exact: the proof exists because FunctionalScript cannot catch, and it was written with a catch. `returnedTreeThrows` moves to catch.proof.mjs, where a host runner belongs. Its mock accumulates written output in the runner's own state rather than pushing into a captured array, which was the second finding and is the better shape anyway — the proof now reads like a virtual run. A second proof comes with the move: a readable returned tree is still walked, so the guard did not quietly replace recursion with a refusal to recurse. The changelog entry is marked breaking. Adding `Catch` to `NodeOp` makes a handler mandatory for any exhaustive `NodeOperationMap` — which this change proved by breaking the Node interpreter's typecheck and the mock map in `reporterWriteFailure` before supplying both. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016PyLwDNkPQM1uD1Tg5ApBg
CI's own lint caught what my local checks did not: no authored .mjs may carry a file-scope JSDoc @typedef, so the `_Ops` alias in the new proof failed the repository grep. The rule offers a sibling private.ts or inlining; a four-member union named at two annotations is the inlining case, and adding a file for it would be the heavier answer. The gap was in how I verified: tsc, fjs t and coverage all passed locally while the lint steps in the workflow went unrun. Both of those steps — the typedef grep and ci-update's no-diff check — now pass here before pushing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016PyLwDNkPQM1uD1Tg5ApBg
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b6acda8055
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
AGENTS.md reserves @module for a package's entry point — module.f.mjs or module.mjs — and the new proof had one. The leading block stays, as the same rule requires, because it holds the @import tags. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016PyLwDNkPQM1uD1Tg5ApBg
o2alexanderfedin
left a comment
There was a problem hiding this comment.
Approved.
The premise checks out on the merge-base: a proof returning a tree with a throwing getter crashes the run outright — no summary line is ever printed, so nothing after it is recorded. At this head the same fixture gives hostile(): error while its neighbours still report ok, the run completes, and the exit is a fail-count rather than a crash.
The check that mattered most — that "fail the leaf" has not quietly become "pass the leaf" — holds. Forcing a tree-read failure to keep status: 'passed' still fails npm test, caught by returnedTreeThrows's assertion on what the reporter actually wrote rather than by the status it was handed. The leaf is counted as a failure and the exit is non-zero.
Moving the catching runner into catch.proof.mjs costs no coverage: against the true merge-base, zero leaf names vanish and exactly two appear. .f.mjs stays free of try/catch repo-wide, so the split follows the rule rather than working around it, and the browser bundle skipping the file is structural — browser-prepare.mjs selects on the .f.mjs suffix — with existing browser-side coverage unchanged. The breaking catch member is real (removing it is a TS2741) and declared.
Also checked the last two commits, since the new file briefly broke #1750's guard: at a99607cdd, catch.proof.mjs:26 did carry a file-scope @typedef _Ops, and the guard grep now returns nothing at this head. Gates here: npm test 3523/3523, exit 0.
One stale line: fjs/emergent_testing/todo/hostile-proof-values.md around line 108 still places returnedTreeThrows in ../proof.f.mjs, which is no longer where it lives. The other ../proof.f.mjs reference in that file concerns the virtual runner's fixture and stays correct.
I did not run npm run cov, so the body's 100% claim is unverified here.
Review found the task list still placing the fjs t proof in proof.f.mjs, which is where it started and not where it ended up. It names catch.proof.mjs now, with the reason the file is .mjs at all. Also disambiguates the design section: the exportedTreeThrows / returnedTreeThrows it refers to are the browser's, and the fjs t proof deliberately reuses one of those names — one behaviour, named once, proven per runner. The other proof.f.mjs reference in the file is about the virtual runner's fixtures and stays as it was. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016PyLwDNkPQM1uD1Tg5ApBg
|
Thank you — and the mutation check is the one I'd have wanted most: "fail the leaf" quietly becoming "pass the leaf" is exactly the failure mode a guard like this can hide, and it's worth knowing that The stale line is fixed in 912c71f. The task list now names While there I disambiguated one thing my change introduced: the design section's On coverage, since you flagged the claim as unverified: Generated by Claude Code |
o2alexanderfedin
left a comment
There was a problem hiding this comment.
Approved. The stale line is fixed — returnedTreeThrows now points at ../catch.proof.mjs, where it and returnedTreeIsStillWalked both are, and the note now says why the file is .mjs. The shared naming with ../browser/proof.mjs is deliberate and stated, which is better than the silent collision it would otherwise look like.
Implements the
catchoperation thattodo/hostile-proof-values.mddesigned, and uses it where the shared traversal reads user values.The defect
The walk enumerates whatever a proof returned, and enumerating runs user code — an enumerable getter or a proxy trap throws inside
collectTests. Unguarded, that throw unwinds the entire traversal: one hostile value costs the results of every module that had already passed, including any failures worth seeing.returnedTreeThrowspins the fix — a leaf beside the hostile one still runs, is still reported, and the summary still counts both.returnedTreeIsStillWalkedpins the other half: a readable returned tree is still recursed into, so the guard did not quietly replace recursion with a refusal to recurse.Why not
sandboxThe obvious guard is the operation that already exists, and it cannot hold this — for a reason that is not the obvious one either. Timing isn't the problem: the sub-tree read happens after the leaf has settled, over a value already in hand. The problem is the virtual runner: its
sandboxis a deliberate pass-through,f => state => [state, ok(f())], with the fixture returning theSandboxResultit wants reported, becauseeffects/node/virtualis.f.mjsand FunctionalScript has notry/catchto implement a real one with. Routing a tree walk through it would hand that handler a thunk answering_TestAndPath[], and every fixture inproof.f.mjswould break.So
catchis a second, honest operation beside it — "run this pure thunk; a throw is theerrorbranch" — carrying no clock and no fixture convention, so each runner implements it truthfully:tryCatch(f).ok(ok(f()))— a pure runner still cannot catch, so a hostile fixture still panics there. That is the same bargainsandboxalready makes, and virtual proofs use benign fixtures.Ordering, and what stayed asymmetric
The read happens before the leaf is reported, so its failure is part of what gets reported rather than a correction issued afterwards. The leaf keeps its own
duration— that is what running it took — while the result handed to the reporter carries the reading failure, so a host that describes a thrown value describes this one.The module's exported tree is still read unguarded, and that is deliberate: there is no leaf to attribute it to, so an unreadable
proofexport belongs to whatever loaded the module.fjs tstill panics on one; the browser page still catches it and reports one failed module. Closing that is a report-shape question rather than a missing operation, and it is what remains open inhostile-proof-values.md.The proof lives in a
.mjs, and that is the pointcatch.proof.mjs, notproof.f.mjs. A runner that reports a throw instead of propagating it needstry/catchto write, and FunctionalScript has neither — which is the same fact that makes the virtual runner unable to demonstrate this at all. Review caught the first draft writing that runner inside a.f.mjs, which was precisely the defect the operation exists to work around. Its mock accumulates written output in the runner's own state rather than a captured array, so nothing reaches outside the runner to collect its results.Why this landed on its own
It is the prerequisite for step 7 of
share-browser-console-runner.md. The browser catches this case today and the shared walk did not, so sharing the traversal without it would have lost a behaviour — and a regression isn't something to record and defer. Landing it separately also meansfjs tgains the behaviour now, with its own proofs, rather than inside a port where the two arguments would be tangled.Verification
npx tscclean,fjs t3523/3523 (two new proofs),npm run covexit 0 (100% line/branch/function thresholds), plus the workflow's own lint steps — the file-scope@typedefgrep andci-update's no-diff check — which a CI failure on this branch taught me to run locally.Changelog:
effects: the newcatchoperation joinsNodeOp, soan exhaustive
NodeOperationMapmust supply a handler for itemergent_testing: a proof whose return value cannot be enumerated fails thatproof instead of the whole run
🤖 Generated with Claude Code
https://claude.ai/code/session_016PyLwDNkPQM1uD1Tg5ApBg