js/keywords: one source of truth for JavaScript keywords - #1562
Conversation
FunctionalScript is a strict subset of JavaScript — any FunctionalScript program must run the same on JavaScript — so every consumer that decides whether a name is a keyword has to agree with the JavaScript grammar. Three hand-maintained copies existed: the JavaScript tokenizer's entry list, the DJS tokenizer's Set, and the rtti TS printer's reserved list. The new fjs/js/keywords module holds the sets once, structured by the ECMA-262 categories: reservedWords (the ReservedWord production), strictModeReservedWords (every module is strict-mode code), restrictedNames (arguments/eval), and the aggregate keywords list that adds FunctionalScript's undefined. A type-level pin ties the aggregate's union to the groups' and the proof checks it is their sorted union. - fjs/js/tokenizer derives keywordEntries from `keywords`, and _KeywordToken's kind is now Exclude<typeof keywords[number], ...> — definitionally tied to the list instead of a parallel hand-written union. - fjs/djs/tokenizer builds its Set from `keywords`. - fjs/types/rtti/ts composes its type-alias screen from reservedWords and strictModeReservedWords plus the TypeScript-only names; the effective set is unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T8BR3aPUJDe5zZMDJKme7j
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T8BR3aPUJDe5zZMDJKme7j
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
functionalscript | fd09f75 | Commit Preview URL Branch Preview URL |
Aug 14 2026, 09:51 PM |
| 'public', 'return', 'static', 'super', 'switch', 'this', 'throw', 'try', | ||
| 'typeof', 'var', 'void', 'while', 'with', 'yield', | ||
| ])) | ||
| const keywordSet = /** @type {ReadonlySet<string>} */ (new Set(keywords)) |
There was a problem hiding this comment.
Avoid using /** @type {T} */(value). Use either /** @type {T} */const ... = ... or @satisfies. See AGENTS.md for that.
There was a problem hiding this comment.
Fixed in 65649da — declaration form, so the compiler checks new Set(keywords) against the annotation instead of overriding it.
For the record, one inline @type cast remains in this PR, on js/tokenizer's derived keywordEntries (/** @type {JsToken} */ ({ kind })). The declaration form (and @satisfies) is a compile error there: { kind: <49-literal union> } does not distribute over the JsToken discriminated union, which is exactly why the cast exists. It is the same boundary cast the file's djs counterpart toJsToken already uses for the same reason, and it is safe by construction — _KeywordToken's kinds derive from the same keywords list.
Generated by Claude Code
Per AGENTS.md, the declaration form lets the compiler check the initializer against the annotation instead of overriding it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T8BR3aPUJDe5zZMDJKme7j
o2alexanderfedin
left a comment
There was a problem hiding this comment.
Approving. Reviewed at 8441bd5, baseline origin/main = c3af487f ("Add task to split AGENTS.md into scoped documents (#1560)").
The risk in a DRY extraction like this is that the consumers were not actually identical — #1522's hex-codec extraction found DJS deliberately divergent, and #1533's sweep found a real divergence hiding as a latent bug. So I enumerated all three consumers' sets before and diffed them against the new shared lists.
Set diff — the three consumers, before vs after
Old sets parsed out of origin/main's sources; new sets taken from the exported lists (plus each consumer's own extras):
| consumer | old | new | removed | added |
|---|---|---|---|---|
js/tokenizer keywordEntries |
49 | 49 | — | — |
djs/tokenizer keywords |
49 | 49 | — | — |
types/rtti/ts reserved |
61 | 61 | — | — |
No set difference anywhere. No duplicates on either side. keywords is exactly reservedWords ∪ strictModeReservedWords ∪ restrictedNames ∪ {undefined}, sorted (38 + 8 + 2 + 1 = 49).
The rtti regrouping is the one that needed care and it lands correctly: false/null/true/void move out of its "predefined type names" comment-group into reservedWords, and yield moves out of its strict-mode group (the new strictModeReservedWords has 8 entries, no yield) into reservedWords — net zero. undefined correctly stays in rtti's own list, since reservedWords does not contain it and rtti does not spread restrictedNames. Re #1547: all nine TS strict-mode reserved words and intrinsic are present and accounted for.
Behavioural sweep across all three consumers
Set equality is not the same as behaviour equality, so I also drove each consumer's real entry point over 100 words — every keyword old and new, the TS predefined type names and type operators, TS contextual keywords that were never in any list (as, is, of, get, set, async, from, satisfies, asserts, declare, namespace, type, abstract, accessor, override, using, …), and near-miss identifiers (Await, AWAIT, awaits, yield2, let2, _, $, T0, …):
js/tokenizertokenize— full token stream, JSON-compareddjs/tokenizertokenizeJs— full token stream, JSON-comparedtypes/rtti/tsdataToTs— whether a rule of that name keeps its identifier or gets a generatedT<n>
Identical on origin/main and at this head for all 300 results. Negative control (perturbing yield's three results) diffs as expected, so the harness discriminates.
Type level
_KeywordToken went from a hand-written six-arm union to Exclude<typeof keywords[number], 'true' | 'false' | 'null' | 'undefined'>. I checked that against main's literal union with a [A] extends [B] ? [B] extends [A] equality inside the project's own tsc: exactly equal; adding one phantom kind to the old union makes it fail (TS2322), so the check is real.
_KeywordsPinned is not decorative either — appending 'zzz' to keywords gives module.f.mjs(65,21): error TS2344: Type 'false' does not satisfy the constraint 'true'. Good: it means the /** @type {JsToken} */ cast in the new keywords.map(…) cannot silently drift, because _KeywordToken is derived from the same list the cast reads.
Gates
npx tsc --noEmit→ 0.npm run prepackfrom a clean tree → 0.npm test→pass: 2709, fail: 0vs main's2708— exactly the one newjs/keywordsproof.- Emitted declarations read directly (not
bin/extract.mjs):js/tokenizer/module.f.d.mts,djs/tokenizer/module.f.d.mtsandtypes/rtti/ts/module.f.d.mtsare byte-identical to main. The only surface change is the newjs/keywordsmodule itself, whose consts emit as precisereadonly [...]tuples (notstring[]).@modulesurvives in the new declaration (1 occurrence, blank line after the header present); the single: anygrep hit is the word "any" in prose. bin/linkcheck.mjs— broken-link sets identical to main.
Proof quality
proof.aggregate mutation-tested rather than trusted by name:
| mutation | result |
|---|---|
drop 'yield' from reservedWords |
fail: 1 |
swap 'eval'/'export' in keywords (order only) |
fail: 1 |
drop 'class' from keywords |
fail: 3 — the aggregate proof plus two tokenizer proofs |
The third one is the useful one: it shows the DRY wiring is real, not just declared — a change to the shared list propagates into both tokenizers' behaviour. And pinning the sort order matters, since keywordEntries feeds ordered_map's fromEntries.
Minor
changelog/unreleased/1562.md ends with [#1562](…/pull/1562), but §8.3 now says entries carry "no PR number or link inside the file". Same drift I noted on #1564; most existing unreleased/ files still have links, so this follows the de-facto shape rather than the written rule.
Not an issue
types.tsimportingimport type { keywords }from a.f.mjsintroduces no cycle —js/keywordsimports nothing fromjs/tokenizer.reservedin rtti widening from aconsttuple toreadonly string[]is invisible: it is module-local and the emitted declaration is unchanged._KeywordsPinnedis_-prefixed, correct under §6.2 (a JSDoc@typedefhas no non-exported form).- No
README.mdfor the new module directory —fjs/js/tokenizerhas none either, and I find no rule requiring one.
The consumers really were identical this time, and the extraction is faithful to all three.
Summary
FunctionalScript is a strict subset of JavaScript — any FunctionalScript program must run the same on JavaScript — so every consumer that decides whether a name is a keyword has to agree with the JavaScript grammar. Three hand-maintained copies existed: the JavaScript tokenizer's entry list, the DJS tokenizer's
Set, and the rtti TypeScript printer's reserved-name screen (whose gaps took three review rounds on #1547 to close precisely because the list was hand-kept).This carries the follow-up work that landed on the #1547 branch after that PR merged.
Key changes
New
fjs/js/keywordsmodule — the one source of truth, structured by the ECMA-262 categories:reservedWords— theReservedWordproduction (§12.7.2), never usable as identifiers;strictModeReservedWords—implements,interface,let,package,private,protected,public,static; every module is strict-mode code, so FunctionalScript treats them like reserved words;restrictedNames—arguments/eval, not reserved words but unbindable in strict code;keywords— the alphabetical aggregate, adding FunctionalScript'sundefined(an ordinary global in JavaScript kept as a literal keyword).Two ties prevent drift: a type-level
Assert<Equal<…>>pins the aggregate's literal union to the union of the groups, and the proof verifies it is exactly their sorted, duplicate-free union.fjs/js/tokenizerderiveskeywordEntriesfromkeywords, and_KeywordToken's kind is nowExclude<typeof keywords[number], 'true' | 'false' | 'null' | 'undefined'>— definitionally tied to the list instead of a parallel hand-written union, so a keyword added to the source of truth flows into the token type automatically.fjs/djs/tokenizerbuilds its keywordSetfromkeywordsinstead of a copied literal.fjs/types/rtti/tscomposes its type-alias screen fromreservedWordsandstrictModeReservedWordsplus the TypeScript-only names (predefined type names,intrinsic, type-operator keywords), which correctly stay local — they are TypeScript facts, not JavaScript facts. The effective 61-name screen verified in RTTI TS printer: recursive schemas via the data form #1547's final review is unchanged.Checks
npx tsc— clean.fjs/js/keywordsat 100% line/branch/function coverage; the tokenizers' coverage is unchanged.🤖 Generated with Claude Code
https://claude.ai/code/session_01T8BR3aPUJDe5zZMDJKme7j
Generated by Claude Code