Skip to content

emergent_testing: an unreadable returned tree fails its leaf, not the run - #1758

Merged
sergey-shandar merged 6 commits into
mainfrom
claude/todo-unification-runners-7wvd1q
Aug 28, 2026
Merged

emergent_testing: an unreadable returned tree fails its leaf, not the run#1758
sergey-shandar merged 6 commits into
mainfrom
claude/todo-unification-runners-7wvd1q

Conversation

@sergey-shandar

@sergey-shandar sergey-shandar commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Implements the catch operation that todo/hostile-proof-values.md designed, and uses it where the shared traversal reads user values.

The defect

The walk enumerates whatever a proof returned, and enumerating runs user code — an enumerable getter or a proxy trap throws inside collectTests. Unguarded, that throw unwinds the entire traversal: one hostile value costs the results of every module that had already passed, including any failures worth seeing. returnedTreeThrows pins the fix — a leaf beside the hostile one still runs, is still reported, and the summary still counts both. returnedTreeIsStillWalked pins the other half: a readable returned tree is still recursed into, so the guard did not quietly replace recursion with a refusal to recurse.

Why not sandbox

The obvious guard is the operation that already exists, and it cannot hold this — for a reason that is not the obvious one either. Timing isn't the problem: the sub-tree read happens after the leaf has settled, over a value already in hand. The problem is the virtual runner: its sandbox is a deliberate pass-through, f => state => [state, ok(f())], with the fixture returning the SandboxResult it wants reported, because effects/node/virtual is .f.mjs and FunctionalScript has no try/catch to implement a real one with. Routing a tree walk through it would hand that handler a thunk answering _TestAndPath[], and every fixture in proof.f.mjs would break.

So catch is a second, honest operation beside it — "run this pure thunk; a throw is the error branch" — carrying no clock and no fixture convention, so each runner implements it truthfully:

  • real Node runner: tryCatch(f).
  • virtual runner: ok(ok(f())) — a pure runner still cannot catch, so a hostile fixture still panics there. That is the same bargain sandbox already makes, and virtual proofs use benign fixtures.
  • a browser interpreter gets one when it exists, which is the next step.

Ordering, and what stayed asymmetric

The read happens before the leaf is reported, so its failure is part of what gets reported rather than a correction issued afterwards. The leaf keeps its own duration — that is what running it took — while the result handed to the reporter carries the reading failure, so a host that describes a thrown value describes this one.

The module's exported tree is still read unguarded, and that is deliberate: there is no leaf to attribute it to, so an unreadable proof export belongs to whatever loaded the module. fjs t still panics on one; the browser page still catches it and reports one failed module. Closing that is a report-shape question rather than a missing operation, and it is what remains open in hostile-proof-values.md.

The proof lives in a .mjs, and that is the point

catch.proof.mjs, not proof.f.mjs. A runner that reports a throw instead of propagating it needs try/catch to write, and FunctionalScript has neither — which is the same fact that makes the virtual runner unable to demonstrate this at all. Review caught the first draft writing that runner inside a .f.mjs, which was precisely the defect the operation exists to work around. Its mock accumulates written output in the runner's own state rather than a captured array, so nothing reaches outside the runner to collect its results.

Why this landed on its own

It is the prerequisite for step 7 of share-browser-console-runner.md. The browser catches this case today and the shared walk did not, so sharing the traversal without it would have lost a behaviour — and a regression isn't something to record and defer. Landing it separately also means fjs t gains the behaviour now, with its own proofs, rather than inside a port where the two arguments would be tangled.

Verification

npx tsc clean, fjs t 3523/3523 (two new proofs), npm run cov exit 0 (100% line/branch/function thresholds), plus the workflow's own lint steps — the file-scope @typedef grep and ci-update's no-diff check — which a CI failure on this branch taught me to run locally.

Changelog:

  • BREAKING CHANGES: effects: the new catch operation joins NodeOp, so
    an exhaustive NodeOperationMap must supply a handler for it
  • emergent_testing: a proof whose return value cannot be enumerated fails that
    proof instead of the whole run

🤖 Generated with Claude Code

https://claude.ai/code/session_016PyLwDNkPQM1uD1Tg5ApBg

… run

The shared walk enumerates whatever a proof returned, which runs user
code: an enumerable getter or a proxy trap throws inside `collectTests`.
Unguarded that unwinds the whole traversal, so one hostile value costs
the results of every module that had already passed — including the
modules that would have reported failures worth seeing.

Adds the `catch` operation that `todo/hostile-proof-values.md`
specified, with handlers in the real Node runner (`tryCatch`) and the
virtual one (`ok(ok(f()))` — a pure runner still cannot catch, which is
the bargain `sandbox` already makes). `sandbox` could not carry this:
the virtual runner's is a deliberate pass-through whose thunk answers a
`SandboxResult`, so routing a tree walk through it would break every
fixture.

The read now happens before the leaf is reported, so the failure is part
of what gets reported rather than a correction after the fact, and the
leaf keeps its own duration while the reporter receives the reading
failure to describe.

The module's *exported* tree is still read unguarded, deliberately:
there is no leaf to attribute it to. That asymmetry is now written down
in both the code and the issue.

This is also the prerequisite for step 7 of
share-browser-console-runner.md — the browser catches this today, so
sharing the traversal without it would have lost a behaviour.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016PyLwDNkPQM1uD1Tg5ApBg
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for security reviews. Please try again later.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016PyLwDNkPQM1uD1Tg5ApBg
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
functionalscript 912c71f Commit Preview URL

Branch Preview URL
Aug 28 2026, 05:32 PM

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 038bfbc9a5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread fjs/emergent_testing/proof.f.mjs Outdated
Comment thread fjs/emergent_testing/proof.f.mjs Outdated
Comment thread changelog/unreleased/1758.md Outdated
claude added 2 commits August 28, 2026 17:15
Review found try/catch in proof.f.mjs — outside the language the file
claims to be written in, and the same defect the repository has been
removing rather than excusing. The irony is exact: the proof exists
because FunctionalScript cannot catch, and it was written with a catch.

`returnedTreeThrows` moves to catch.proof.mjs, where a host runner
belongs. Its mock accumulates written output in the runner's own state
rather than pushing into a captured array, which was the second finding
and is the better shape anyway — the proof now reads like a virtual run.

A second proof comes with the move: a readable returned tree is still
walked, so the guard did not quietly replace recursion with a refusal to
recurse.

The changelog entry is marked breaking. Adding `Catch` to `NodeOp` makes
a handler mandatory for any exhaustive `NodeOperationMap` — which this
change proved by breaking the Node interpreter's typecheck and the mock
map in `reporterWriteFailure` before supplying both.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016PyLwDNkPQM1uD1Tg5ApBg
CI's own lint caught what my local checks did not: no authored .mjs may
carry a file-scope JSDoc @typedef, so the `_Ops` alias in the new proof
failed the repository grep. The rule offers a sibling private.ts or
inlining; a four-member union named at two annotations is the inlining
case, and adding a file for it would be the heavier answer.

The gap was in how I verified: tsc, fjs t and coverage all passed
locally while the lint steps in the workflow went unrun. Both of those
steps — the typedef grep and ci-update's no-diff check — now pass here
before pushing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016PyLwDNkPQM1uD1Tg5ApBg

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b6acda8055

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread fjs/emergent_testing/catch.proof.mjs Outdated
AGENTS.md reserves @module for a package's entry point — module.f.mjs
or module.mjs — and the new proof had one. The leading block stays, as
the same rule requires, because it holds the @import tags.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016PyLwDNkPQM1uD1Tg5ApBg

@o2alexanderfedin o2alexanderfedin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved.

The premise checks out on the merge-base: a proof returning a tree with a throwing getter crashes the run outright — no summary line is ever printed, so nothing after it is recorded. At this head the same fixture gives hostile(): error while its neighbours still report ok, the run completes, and the exit is a fail-count rather than a crash.

The check that mattered most — that "fail the leaf" has not quietly become "pass the leaf" — holds. Forcing a tree-read failure to keep status: 'passed' still fails npm test, caught by returnedTreeThrows's assertion on what the reporter actually wrote rather than by the status it was handed. The leaf is counted as a failure and the exit is non-zero.

Moving the catching runner into catch.proof.mjs costs no coverage: against the true merge-base, zero leaf names vanish and exactly two appear. .f.mjs stays free of try/catch repo-wide, so the split follows the rule rather than working around it, and the browser bundle skipping the file is structural — browser-prepare.mjs selects on the .f.mjs suffix — with existing browser-side coverage unchanged. The breaking catch member is real (removing it is a TS2741) and declared.

Also checked the last two commits, since the new file briefly broke #1750's guard: at a99607cdd, catch.proof.mjs:26 did carry a file-scope @typedef _Ops, and the guard grep now returns nothing at this head. Gates here: npm test 3523/3523, exit 0.

One stale line: fjs/emergent_testing/todo/hostile-proof-values.md around line 108 still places returnedTreeThrows in ../proof.f.mjs, which is no longer where it lives. The other ../proof.f.mjs reference in that file concerns the virtual runner's fixture and stays correct.

I did not run npm run cov, so the body's 100% claim is unverified here.

Review found the task list still placing the fjs t proof in
proof.f.mjs, which is where it started and not where it ended up. It
names catch.proof.mjs now, with the reason the file is .mjs at all.

Also disambiguates the design section: the exportedTreeThrows /
returnedTreeThrows it refers to are the browser's, and the fjs t proof
deliberately reuses one of those names — one behaviour, named once,
proven per runner. The other proof.f.mjs reference in the file is about
the virtual runner's fixtures and stays as it was.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016PyLwDNkPQM1uD1Tg5ApBg

Copy link
Copy Markdown
Contributor Author

Thank you — and the mutation check is the one I'd have wanted most: "fail the leaf" quietly becoming "pass the leaf" is exactly the failure mode a guard like this can hide, and it's worth knowing that returnedTreeThrows catches it by asserting on what the reporter wrote rather than on the status it was handed.

The stale line is fixed in 912c71f. The task list now names ../catch.proof.mjs and says why the file is .mjs at all. You're right that the other ../proof.f.mjs reference is about the virtual runner's fixtures and stays correct — I left it.

While there I disambiguated one thing my change introduced: the design section's exportedTreeThrows / returnedTreeThrows are the browser's proofs, and my fjs t proof now reuses one of those names. That's deliberate — one behaviour, named once, proven per runner — but with the name existing twice it needed saying rather than leaving a reader to guess which file was meant.

On coverage, since you flagged the claim as unverified: npm run cov exits 0 with 100% line/branch/function here. Worth noting why that's compatible with moving a proof out of .f.mjs — the coverage run includes **/module.f.mjs only, so proof files never contribute coverage themselves; what matters is that the leaves still execute and still exercise module.f.mjs, which your zero-vanished/two-added leaf-name comparison establishes more directly than the threshold does.


Generated by Claude Code

@o2alexanderfedin o2alexanderfedin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. The stale line is fixed — returnedTreeThrows now points at ../catch.proof.mjs, where it and returnedTreeIsStillWalked both are, and the note now says why the file is .mjs. The shared naming with ../browser/proof.mjs is deliberate and stated, which is better than the silent collision it would otherwise look like.

@sergey-shandar
sergey-shandar added this pull request to the merge queue Aug 28, 2026
Merged via the queue into main with commit 5b677fe Aug 28, 2026
19 checks passed
@sergey-shandar
sergey-shandar deleted the claude/todo-unification-runners-7wvd1q branch August 28, 2026 18:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants