Skip to content

media/json/parser: hoist the repeated unexpected-token error - #1569

Merged
sergey-shandar merged 2 commits into
mainfrom
claude/todo-implementation-rifq4g
Aug 15, 2026
Merged

media/json/parser: hoist the repeated unexpected-token error#1569
sergey-shandar merged 2 commits into
mainfrom
claude/todo-implementation-rifq4g

Conversation

@sergey-shandar

Copy link
Copy Markdown
Contributor

Implements fjs/media/json/todo/parser-unexpected-token.md (deleted here).

The nine literals

{ status: 'error', message: 'unexpected token' } was built inline at nine sites. JSON errors carry no position or metadata, so the constructor the DJS side needs degenerates to a shared module-scope value:

/** @type {_JsonState} */
const unexpectedToken = { status: 'error', message: 'unexpected token' }

The case 'error': return { status: 'error', message: state.message } arm in parse is left alone — it forwards a message rather than raising this one.

The pushKey fallthrough

The TODO's second task was to decide whether pushKey's { status: 'error', message: 'error' } branch is a real condition or unreachable, and to restructure rather than relabel if it is unreachable. It is unreachable, and the module already said so in the proof it carried at the bottom:

pushKey is only ever invoked while state.top is an object (the state machine's '{'/'{,' statuses guarantee it), so its non-object guard is a defensive branch unreachable through parse. Call it directly to cover that branch.

So the guard is gone, and pushKey now reads exactly like its three siblings in the same module — endArray, endObject, and popStack's callers all state the construction guarantee in JSDoc and narrow with a cast:

/**
 * `pushKey` only ever runs while parsing the object `startObject` opened
 * (status `'{'`/`'{,'`), so `state.top` is always that object here — the same
 * construction guarantee `endArray` relies on below.
 */
const pushKey = state => value => ({
    status: '{k',
    top: addKeyToObject(/** @type {_JsonObject} */ (state.top))(value),
    stack: state.stack,
})

This is the §6.2-vs-§3.2 trade-off resolved the way the surrounding module already resolves it: §3.2 says restructure an unreachable branch away rather than keeping it, and the cast is the idiom endArray/endObject use for the identical situation. Consistency inside one state machine seemed worth more than avoiding a fourth instance of a cast the file already relies on three times.

Removing the branch also removes the only reason module.f.mjs exported a proof at all — a test that existed solely to reach the dead guard. The file is implementation-only again, which is what §3.2 asks for (module.f.mjs implements, proof.f.mjs proves).

Verification

  • npx tsc clean.
  • fjs test: 2724 pass, 0 fail, against 2725 on main — exactly the one deleted test, and nothing else moved.
  • npm run cov: fjs/media/json/parser/module.f.mjs stays at 100% lines/branches/functions, now without a test that had to call a private function directly to get there.

Changelog:

  • media/json/parser: the nine inline unexpected token error values are one shared constant, and pushKey drops a guard unreachable through parse

🤖 Generated with Claude Code

https://claude.ai/code/session_016HvbYkBMYWwQECL7myLhqs


Generated by Claude Code

Working notes; the PR title and description are the commit message.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HvbYkBMYWwQECL7myLhqs
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016HvbYkBMYWwQECL7myLhqs
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
functionalscript 07fd031 Commit Preview URL

Branch Preview URL
Aug 14 2026, 11:19 PM

@o2alexanderfedin o2alexanderfedin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified against origin/main at db76410 (types/byte_set: toRangeMap carries a boolean, not an FSM payload (#1566)), tested at head 07fd031a. Approving.

What I checked

Behavioural equivalence over a token corpus. I imported parse from both trees in one process and swept 111,111 token sequences — every sequence of length 0-5 over [ ] { } , : string number true null, each terminated with eof — comparing the whole serialized Result, not just the message text. 0 differences. Negative control (corrupting one of the 111,111 comparisons) reported exactly 1 diff, so the comparison was live. This covers the pushKey guard removal: no reachable token sequence distinguishes the old guarded pushKey from the new unchecked cast.

The cast itself is sound by construction: pushKey is called only from parseObjectStartOp (status '{') and parseObjectCommaOp (status '{,'); '{' is written only by startObject, and '{,' only by parseObjectNextOp from '{v', which pushValue writes only when top is a non-null non-array. So state.top is always _JsonObject there.

Mutation testing of the nine hoisted sites. The specific risk with hoisting a pinned error into one constant is that distinct branches become textually identical, so a mutation collapsing one into another leaves the suite green. I mutated each site in turn to return state (continue instead of erroring) and ran the parser's own proof:

site line result
parseValueOp case ']' 148 killed (47/0 → 45/2)
parseValueOp default 153 killed (→ 40/7)
parseArrayStartOp fallback 163 killed (→ 44/3)
parseArrayValueOp fallback 170 killed (→ 46/1)
parseObjectStartOp fallback 177 killed (→ 43/4)
parseObjectKeyOp fallback 183 killed (→ 46/1)
parseObjectNextOp fallback 190 killed (→ 46/1)
parseObjectCommaOp fallback 198 killed (→ 45/2)
foldOp case 'result' 207 killed (→ 44/3)

All nine are reached and discriminated. The hoist does not create an undetectable site.

Correcting my own first pass. My first mutant returned { status: 'result', value: 'MUT' } instead, and four sites came back green — as did four "return a sibling arm's transition" mutants (parseArrayValueOpendArray, parseObjectStartOpendObject, parseObjectKeyOp'{:', parseObjectNextOpendObject). That was my harness's fault, not a coverage hole: every proof input that reaches those arms has a trailing token, and foldOp's case 'result' arm re-errors on it with the same 'unexpected token' message, so the mutant's effect is swallowed before parse returns. Worth knowing that this masking exists, but it is pre-existingfoldOp's 'result' arm returns the identical message on main (line 192 there) and proof.f.mjs is untouched by this PR. Not a reason to block.

Public surface. npm run prepack exits 0 in both trees. I diffed the emitted declarations directly rather than trusting bin/extract.mjs: across all .d.mts in fjs/, the file set is identical and exactly one file differs — fjs/media/json/parser/module.f.d.mts, which loses export declare const proof: { pushKey: { nonObjectTop: () => void } } and nothing else. No new any, no /*elided*/, @module header still present in the emitted .d.mts. No new types, so §6.2 _-prefixing does not apply.

Tests. npm test from a cleaned tree: 2724 pass / 0 fail, against 2725 / 0 on main at db76410. The single-test delta is exactly the removed proof.pushKey.nonObjectTop, which tested the branch this PR deletes. npx tsc --noEmit exits 0.

Links. linkcheck broken-link sets compared, not counts: 129 on main → 122 here. The 7 that disappear are all outbound links from the deleted fjs/media/json/todo/parser-unexpected-token.md (its ../../djs/todo/… references were already broken — they were one ../ short). Nothing anywhere links to the deleted file. Strict improvement.

§8.3. changelog/unreleased/1569.md: list items only, no heading, no PR number or link inside, ~150 characters. Correct under the current convention. This is a code change, so an entry is required — and #1536's entry is the direct precedent for describing a defensive-branch removal in this same module without a **BREAKING CHANGES:** prefix.

One heads-up, not a review objection

#1567 modifies fjs/media/json/todo/parser-unexpected-token.md (refreshing its line numbers) while this PR deletes it. Whichever lands second will hit a modify/delete conflict; resolving in favour of the deletion is correct.

@sergey-shandar
sergey-shandar added this pull request to the merge queue Aug 15, 2026
Merged via the queue into main with commit 60d04cb Aug 15, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants