diff --git a/fjs/cas/todo/66g-cas-get-verify-option.md b/fjs/cas/todo/66g-cas-get-verify-option.md index 5b07cde30f..2033c3920b 100644 --- a/fjs/cas/todo/66g-cas-get-verify-option.md +++ b/fjs/cas/todo/66g-cas-get-verify-option.md @@ -6,13 +6,14 @@ ### Problem -`Cas.read` / `fileKvStore.read` (`fjs/cas/module.f.mjs:51-57, 85-92`) and the `cas get` -command (`fjs/cas/module.f.mjs:126-146`) return the bytes stored at an address without -recomputing their hash. If a blob was corrupted, truncated, or misnamed — for example by -disk corruption or by a copy-files synchronization that has not yet been verified — a reader -between the copy and a later scrub can consume invalid content under a hash that was signed -or referenced elsewhere. The reader has no way to ask "and prove these bytes actually hash -to the address I requested." +`fileCas(sha2)(path).read` (`fjs/cas/module.f.mjs`, the `read` method on the `FileCas` +returned by `fileCas`) streams the bytes stored at an address, chunk by chunk, without +recomputing their hash, and the `cas get` command (`fjs/cas/cli/module.f.mjs`) pipes that +stream straight to the output file via `writeFromStream`. If a blob was corrupted, truncated, +or misnamed — for example by disk corruption or by a copy-files synchronization that has not +yet been verified — a reader between the copy and a later scrub can consume invalid content +under a hash that was signed or referenced elsewhere. The reader has no way to ask "and prove +these bytes actually hash to the address I requested." A separate batch [`cas verify`](66g-cas-verify-command.md) command catches corruption eventually, but there is a window before it runs, and some callers want certainty at the diff --git a/fjs/cas/todo/66g-cas-verify-command.md b/fjs/cas/todo/66g-cas-verify-command.md index bc5825b014..cbe746f378 100644 --- a/fjs/cas/todo/66g-cas-verify-command.md +++ b/fjs/cas/todo/66g-cas-verify-command.md @@ -5,8 +5,9 @@ ### Problem -`fileKvStore.read` (`fjs/cas/module.f.mjs:51-57`) returns whatever bytes live at the -addressed path without recomputing the hash. After **synchronization by copying files** +`fileCas(sha2)(path).read` (`fjs/cas/module.f.mjs`, the `read` method on the `FileCas` +returned by `fileCas`) streams whatever bytes live at the addressed path without +recomputing the hash. After **synchronization by copying files** (see `issues/plan/vision.md`), or simply over time on a faulty disk, a blob can become corrupted, truncated, or misnamed and no longer hash to the address it sits under. Nothing in the store currently detects this, so the `same hash = same content` invariant the rest @@ -40,7 +41,7 @@ Open design points: ### Tasks - [ ] Add a `verify` function over `Cas`/`KvStore` that rehashes and reports mismatches -- [ ] Wire it as a `cas verify` CLI command in `fjs/cas/module.f.mjs` +- [ ] Wire it as a `cas verify` CLI command in `fjs/cas/cli/module.f.mjs` - [ ] Decide delete vs. quarantine for corrupted blobs and implement it - [ ] Tests: seed a store with a corrupted/truncated/misnamed blob and assert it is caught - [ ] Document the command in `fjs/cas/README.md` diff --git a/fjs/cas/todo/66k-cas-get-return-path.md b/fjs/cas/todo/66k-cas-get-return-path.md index 3202a796db..c974f8dce4 100644 --- a/fjs/cas/todo/66k-cas-get-return-path.md +++ b/fjs/cas/todo/66k-cas-get-return-path.md @@ -5,31 +5,25 @@ ### Problem -`cas get` currently reads the full file content through `fileKvStore.read` → -`readFile`, which is capped at `maxLengthBytes` (128 KiB). Files uploaded via the -streaming path (`cas upload`) can exceed this limit: the hash is stored and the -source is removed, but `cas get ` silently reports the file as missing -because `readFile` rejects oversized reads and `fileKvStore.read` maps that error -to `undefined`. - -More broadly, copying the full byte content through the effect layer is the wrong -model for large files: it requires a `Vec` allocation the size of the file, passes -it back to the caller, and forces the caller to write it out again — doubling peak -memory use. +`fileCas(sha2)(path).read` (`fjs/cas/module.f.mjs`) already streams the stored blob in +`<=128 KiB` chunks (`readBytes` in a loop, capped by `chunkBytes`/`maxLengthBytes` per +chunk, not per file), and `cas get` (`fjs/cas/cli/module.f.mjs`) pipes that stream to the +destination file via `writeFromStream`, so it no longer holds the whole file in memory and +is not limited to 128 KiB files. `FileCas` also already exposes a `url` method +(`fjs/cas/module.f.mjs`, and see `fjs/cas/types.ts`) that returns the path to a hash's +shard without reading its content. + +What is still missing: `cas get` always copies the blob's bytes into a fresh destination +file. There is no way to ask it to just print the existing shard path (or a `file://` URL) +instead, so a caller that only wants to know where the content lives — to hard-link, open +directly, or hand the path to another tool — still pays for a full copy. ### Proposal -Change `cas get` (and the underlying read path) to return the filesystem **path** -of the stored object rather than its contents. Callers that need the bytes can -open the file themselves, stream it, or hard-link / copy it at the OS level with -no size restriction. - -Two concrete forms to consider (may coexist): - -- **Path** — return the absolute path string to the `.cas/…` shard file; the - caller issues a system-level copy or `rename` as needed. -- **`file://` URL** — same information, useful when the result is consumed by a - web client or another tool that already speaks URLs. +Add a mode where `cas get` prints the filesystem **path** (or a `file://` URL) of the +stored object instead of copying it to a destination file. Callers that need a private +copy can still request the current copy-out behavior; callers that only need to locate +the content use the new mode and open/stream/hard-link it themselves. Additionally, mark the stored object **read-only** (e.g. `chmod 444`) immediately after the final `rename` in the upload pipeline. This: @@ -42,19 +36,16 @@ after the final `rename` in the upload pipeline. This: ### Tasks -- [ ] Add a `stat` / `lstat` primitive (or extend an existing one) to retrieve - file size without loading content, so callers can branch on size - [ ] Add a `chmod` (or `setReadOnly`) effect for marking files immutable after write -- [ ] Change `cas get` to print the shard path (and optionally a `file://` URL) - instead of copying bytes to a destination file -- [ ] Update `fileKvStore` (or add a parallel interface) with a `getPath` method - that returns the path for a given hash without reading content +- [ ] Add a `cas get` mode (flag or subcommand) that prints the shard path — via + the existing `FileCas.url` — instead of copying bytes to a destination file - [ ] Apply `setReadOnly` in the `cas upload` pipeline after the final `rename` - [ ] Update proof tests and documentation ### Related - `fileCas.write` / `casAddFile` (`fjs/cas/module.f.mjs`) — streaming upload - pipeline that stores files `cas get` cannot currently read back (design - formerly tracked as `66j-cas-large-file-support`, now implemented and deleted) + pipeline; `cas get` now reads uploaded files back via the streaming `read` + path (design formerly tracked as `66j-cas-large-file-support`, now + implemented and deleted) diff --git a/fjs/ci/todo/138.md b/fjs/ci/todo/138.md index 4f3734848b..bc8be9371c 100644 --- a/fjs/ci/todo/138.md +++ b/fjs/ci/todo/138.md @@ -2,6 +2,15 @@ **Priority:** P3 **Status:** open -**Blocked by:** i136 -Implement a script that will update the lock file by reading the latest versions of tools from the internet using the instructions from 136. +Implement a script that will update the lock file by reading the latest versions of tools from the internet. + +### Related + +This overlaps with the newer Nix-based tool-update proposals — check those first to avoid +duplicating effort: + +- [65Z-ci-nix](65z-ci-nix.md) — the Nixpkgs update command (`npm run ci-nix-update`) covers + updating pinned tool/package versions via Nix. +- [replace-npm-check-updates-with-an-internal-script](replace-npm-check-updates-with-an-internal-script.md) + — a broader internal-script proposal (`ci-lock.json`) covering the same lock-file-update idea. diff --git a/fjs/ci/todo/66h-ci-npm-global-install.md b/fjs/ci/todo/66h-ci-npm-global-install.md index 4f4347fcb1..54d8dafe2b 100644 --- a/fjs/ci/todo/66h-ci-npm-global-install.md +++ b/fjs/ci/todo/66h-ci-npm-global-install.md @@ -5,7 +5,18 @@ ### Problem -Two surviving CI step sites build the same `run`-based step for globally installing a +**Update:** the `@typescript/native-preview`/`tsgo` global-install call site described +below no longer exists in the repo (verified via +`grep -rn "native-preview\|tsgo" .github/workflows/ fjs/ci/`, which finds only this file's +own text). `fjs/ci/node/module.f.mjs` currently has a single global-install site, +`fjsGlobalInstall`, for `functionalscript`. With only one real consumer left, the +second-consumer threshold this proposal relies on (see "Why this still qualifies" below) +no longer holds, and a shared `npmGlobalInstall` factory may not be worth the abstraction +until a second consumer actually reappears. Leaving this open rather than closing it, +since a future global-install site (or the return of a tsgo-like tool) would revive the +case for it. + +Originally, two CI step sites built the same `run`-based step for globally installing a pinned npm package: ```ts @@ -16,8 +27,8 @@ const fjsGlobalInstall = (version: string): MetaStep => install({ run: `npm install -g @typescript/native-preview@${tsgo}` }) ``` -The shape `install({ run: `npm install -g ${pkg}@${version}` })` is duplicated; only the -package name and version differ. +The shape `install({ run: `npm install -g ${pkg}@${version}` })` was duplicated; only the +package name and version differed. The former `fjs/ci/playwright/module.f.mjs` call site is intentionally not a consumer of this proposal. That job and its global install have already been deleted, and this task @@ -49,10 +60,12 @@ acceptable implementation choice if those related APIs settle on that style. ### Why this still qualifies -- There are two real surviving consumers, meeting the second-consumer threshold. +- **Only one real surviving consumer remains** (`fjsGlobalInstall`); the + `@typescript/native-preview` site is gone, so the original second-consumer threshold no + longer holds — see the Problem update above. - The construction is identical and varies only by data. - The abstraction names one repository policy: install a pinned npm tool globally. -- A future third consumer can reuse the factory without restoring the deleted Playwright +- A future second consumer can reuse the factory without restoring the deleted Playwright job. This remains distinct from: @@ -66,8 +79,9 @@ This remains distinct from: - [ ] Add `npmGlobalInstall` to `fjs/ci/common/module.f.mjs`. - [ ] Rebind `fjsGlobalInstall` in `fjs/ci/node/module.f.mjs`. -- [ ] Replace the inline `@typescript/native-preview` global-install step. -- [ ] Confirm proof coverage for both surviving consumers and the generated step shape. +- [ ] ~~Replace the inline `@typescript/native-preview` global-install step.~~ (site no + longer exists — see Problem update) +- [ ] Confirm proof coverage for the surviving consumer and the generated step shape. - [ ] Verify generated workflow output is unchanged. - [ ] Run `npx tsc` and `fjs t`. diff --git a/fjs/ci/todo/ci-integration-tests.md b/fjs/ci/todo/ci-integration-tests.md index f28028b91b..0cf4881fd8 100644 --- a/fjs/ci/todo/ci-integration-tests.md +++ b/fjs/ci/todo/ci-integration-tests.md @@ -14,7 +14,7 @@ The key insight: it matters far more that the *published package* works on every Scenarios are expressed as FunctionalScript modules. A scenario module exports a `main` (a `NodeProgram`) that receives the environment and args and returns an effect. The CI generator reads a scenario list and emits one job per scenario. -See [669-scenario-testing.md](669-scenario-testing.md) for the scenario design — each scenario is a declarative description of initial state, an effect, and an expected result that can be run as either a unit test (mock interpreter) or a real CI job. +Each scenario is a declarative description of initial state, an effect, and an expected result that can be run as either a unit test (mock interpreter) or a real CI job. (The `669-scenario-testing.md` design doc this used to reference no longer exists; issue number 669 has since been reused for unrelated files, e.g. [669-ci-ubuntu-job-factory.md](669-ci-ubuntu-job-factory.md).) Open questions: - Where do scenario modules live? (`issues/demo/` style, or a dedicated `fjs/ci/scenarios/` directory?) diff --git a/fjs/ci/todo/deno-2-8-3-deno-install-frozen-breaks-deno-run-a-npm-functionalscript.md b/fjs/ci/todo/deno-2-8-3-deno-install-frozen-breaks-deno-run-a-npm-functionalscript.md index 473784b5f3..b3a446b5e5 100644 --- a/fjs/ci/todo/deno-2-8-3-deno-install-frozen-breaks-deno-run-a-npm-functionalscript.md +++ b/fjs/ci/todo/deno-2-8-3-deno-install-frozen-breaks-deno-run-a-npm-functionalscript.md @@ -1,7 +1,7 @@ ## Deno 2.8.3: `deno install --frozen` breaks `deno run -A npm:functionalscript` **Priority:** P1 -**Status:** investigate +**Status:** needs re-verification (see note below) With Deno 2.8.3, running `deno install --frozen` before `deno run -A npm:functionalscript@0.30.0` produces: @@ -10,3 +10,13 @@ error: Failed resolving binary export. '.../node_modules/.deno/functionalscript@ ``` The same command succeeds if `deno install --frozen` is **not** run beforehand. + +**Note (re-checked 2026-08-14):** CI now pins Deno `2.9.5` (`.github/workflows/ci.yml`, +`deno` in `fjs/ci/config/module.f.mjs`), not 2.8.3. The current Deno step order in +`fjs/ci/deno/module.f.mjs` already runs the smoke test (`deno run -A ... npm:functionalscript +... test`) *before* `deno install --frozen`, so the ordering that triggered this bug report +is no longer present in the pipeline as generated today. This has not been re-tested against +2.9.5 directly (e.g. running `deno install --frozen` immediately before `deno run -A +npm:functionalscript` by hand), so treat this as likely resolved by the current step order / +version bump, but unconfirmed — re-verify against the currently pinned Deno version before +closing, since Deno version pins change again. diff --git a/fjs/djs/todo/157.md b/fjs/djs/todo/157.md index 9b885fe25c..5b91154c88 100644 --- a/fjs/djs/todo/157.md +++ b/fjs/djs/todo/157.md @@ -117,9 +117,13 @@ The deltas: - `serializeWithConst` is `serializeWithoutConst` plus a ref-counter short-circuit prepended to `f`. -**Sub-task 2b (clearest, smallest):** the two DJS functions collapse into one -factory taking an optional ref-lookup callback — when absent, the const -short-circuit is skipped and you get `serializeWithoutConst`. +**Sub-task 2b (clearest, smallest, done):** the two DJS functions now collapse +into one `buildSerialize(refLookup)(sort)` factory in +`fjs/djs/serializer/module.f.mjs` taking an optional ref-lookup callback — +`serializeWithoutConst = buildSerialize(noRef)`, and `serializeWithConst` +supplies a ref-lookup closure that substitutes `c` references. What +remains of this section is extracting a shared walker between JSON's +`serialize` (`fjs/media/json/module.f.mjs:52`) and DJS's `buildSerialize`. A `serializeValue` factory (in `json/serializer`) parameterized by the extra `typeof` cases and an optional pre-`f` hook covers all three call sites. @@ -150,8 +154,10 @@ line numbers changed. Any extraction here must first re-measure the current code token tree. - [ ] Keep DJS module framing, refs, identifier keys, and metadata behavior DJS-specific. -- [ ] Extract the serializer walker independently where useful; collapse the two - DJS serializer variants through an optional ref hook. +- [x] Collapse the two DJS serializer variants through an optional ref hook — + landed as `buildSerialize` in `fjs/djs/serializer/module.f.mjs`. +- [ ] Extract the serializer walker independently, shared between JSON's + `serialize` and DJS's `buildSerialize`, where useful. - [ ] Re-measure the current tokenizer minus-folding duplication before extracting it; do not implement the stale line-number design blindly. - [ ] Preserve current behavior/proof coverage for both JSON and DJS. diff --git a/fjs/djs/todo/66e-parser-container-stack-bookkeeping.md b/fjs/djs/todo/66e-parser-container-stack-bookkeeping.md index 2c29be8c96..c9ff0248d0 100644 --- a/fjs/djs/todo/66e-parser-container-stack-bookkeeping.md +++ b/fjs/djs/todo/66e-parser-container-stack-bookkeeping.md @@ -7,15 +7,14 @@ Both `fjs/media/json/parser/module.f.mjs` and `fjs/djs/parser/module.f.mjs` build the container state machine out of four helpers — `startArray`, `startObject`, -`endArray`, `endObject` — and within each module the two `start*` helpers and -the two `end*` helpers share their *entire* stack-bookkeeping body. The only -thing that genuinely differs between array and object is the container kind: the -`status` label and the empty-container literal on the way in, and how the -finished container's value is extracted on the way out. Everything around that — -pushing the current `top` onto the stack, popping it back off, and threading the -result through `pushValue` — is repeated verbatim. +`endArray`, `endObject`. The pop side is already deduplicated in both modules +via a shared `popStack` helper (`fjs/media/json/parser/module.f.mjs:59`, +`fjs/djs/parser/module.f.mjs:272`), used by both `endArray` and `endObject`. +What remains is the push side: the two `start*` helpers in each module still +share their *entire* stack-push body verbatim — only the `status` label and +the empty-container literal differ between array and object. -#### JSON (`fjs/media/json/parser/module.f.mjs:79-111`) +#### JSON (`fjs/media/json/parser/module.f.mjs:46-49,79-82`) The stack-push line is byte-identical in both `start*` helpers: @@ -35,29 +34,28 @@ const startObject } ``` -and the pop-and-push-result body is identical in both `end*` helpers — only the -expression that turns `state.top` into a finished value changes: +The pop side (`endArray`/`endObject`) already shares its body through +`popStack`: ```ts -const endArray - : (state: StateParse) => JsonState - = state => { - const array = state.top !== null ? toArray(state.top.values) : null - const newState - : StateParse - = { status: '', top: first(null)(state.stack), stack: drop(1)(state.stack) } - return pushValue(newState)(array) - } +const popStack = stack => { + const ne = next(stack) + return ne === null + ? { status: '', top: null, stack: null } + : { status: '', top: ne.first, stack: ne.tail } +} -const endObject - : (state: StateParse) => JsonState - = state => { - const obj = state.top?.kind === 'object' ? fromMap(state.top.values) : null - const newState - : StateParse - = { status: '', top: first(null)(state.stack), stack: drop(1)(state.stack) } - return pushValue(newState)(obj) - } +const endArray = state => { + const array = toArray(state.top.values) + const newState = popStack(state.stack) + return pushValue(newState)(array) +} + +const endObject = state => { + const obj = fromMap(state.top.values) + const newState = popStack(state.stack) + return pushValue(newState)(obj) +} ``` #### DJS (`fjs/djs/parser/module.f.mjs:262-303`) @@ -74,32 +72,20 @@ const startObject = state => { const newStack = state.top === null ? null : { first: state.top, tail: state.stack } return { ... state, valueState: '{', top: ['object', null, ''], stack: newStack } } - -const endArray = state => { - const top = state.top; - const newState = { ... state, valueState: '', top: first(null)(state.stack), stack: drop(1)(state.stack) } - if (top !== null && top[0] === 'array') { - const array: AstArray = ['array', toArray(top[1])]; - return pushValue(newState)(array) - } - return pushValue(newState)(null) -} -const endObject = state => { - const obj = state?.top !== null && state?.top[0] === 'object' ? fromMap(state.top[1]) : null; - const newState = { ... state, valueState: '', top: first(null)(state.stack), stack: drop(1)(state.stack) } - return pushValue(newState)(obj) -} ``` -So the `newStack` push appears **four** times across the two modules and -`newState` pop appears **four** times, each one a verbatim copy of its sibling. -The repeated lines are not trivial one-liners: the push is a conditional -(`state.top === null ? null : { first, tail }`) and the pop combines -`first(null)(state.stack)` with `drop(1)(state.stack)` and resets the status. -This is exactly the case `AGENTS.md` calls out — "when two code branches share -most of their structure, refactor so the shared part appears once and only the -difference lives in the conditional" — and it is also a separation-of-concerns -point: *manipulating the container stack* is a distinct concern from *which +DJS's `endArray`/`endObject` also already share their pop body through a local +`popStack` helper (`fjs/djs/parser/module.f.mjs:272`), mirroring JSON's. + +So the `newStack` push appears **four** times across the two modules — twice +per module, byte-identical modulo the container-kind literal — while the pop +side is already down to one `popStack` per module. The repeated push is not a +trivial one-liner: it's a conditional (`state.top === null ? null : { first, +tail }`) that decides whether to grow the stack. This is exactly the case +`AGENTS.md` calls out — "when two code branches share most of their +structure, refactor so the shared part appears once and only the difference +lives in the conditional" — and it is also a separation-of-concerns point: +*manipulating the container stack* is a distinct concern from *which container kind* is being opened or closed. The DRY trigger is already met inside each module on its own: there are two real @@ -124,26 +110,21 @@ const startContainer = (status: '[' | '{') => (top: JsonStackElement) => (state: StateParse): JsonState => ({ status, top, stack: pushStack(state) }) -const endContainer = - (build: (top: JsonStackElement | null) => Unknown) => (state: StateParse): JsonState => - pushValue(popState(state))(build(state.top)) - const startArray = startContainer('[')({ kind: 'array', values: null }) const startObject = startContainer('{')({ kind: 'object', values: null, key: '' }) -const endArray = endContainer(top => top !== null ? toArray(top.values) : null) -const endObject = endContainer(top => top?.kind === 'object' ? fromMap(top.values) : null) ``` The empty-container literal is now evaluated once at module load and shared -across calls (sound, since the values are immutable), and the stack push/pop -lives in exactly one place. The four public helpers shrink to one-line -derivations whose body *is* the array-vs-object difference and nothing else. +across calls (sound, since the values are immutable), and the stack push +lives in exactly one place. `endArray`/`endObject` are unchanged — they +already share their pop body through the existing `popStack` helper — so only +`startArray`/`startObject` shrink to one-line derivations whose body *is* the +array-vs-object difference and nothing else. The DJS module gets the same treatment, keeping its `{ ...state, ... }` spread -inside `startContainer` / `popState` and its tuple containers / `['array', …]` -result in the `build` callbacks. `endArray`'s "top is not actually an array → -push `null`" fallback stays inside its `build` callback, so the shared -`pushValue(popState(state))(...)` skeleton is unchanged. +inside `startContainer` and its tuple containers in the `top` argument. +`endArray`/`endObject` need no change there either, since DJS's `popStack` +already covers the pop side. ### Why this is filed at P4 @@ -159,12 +140,13 @@ one and can land independently of 157. ### Tasks -- [ ] In `fjs/media/json/parser/module.f.mjs`, add `pushStack` / `popState` (or - equivalently named) and `startContainer` / `endContainer`; derive - `startArray` / `startObject` / `endArray` / `endObject` from them. +- [x] Pop side: both modules already share their pop body via a `popStack` + helper (`fjs/media/json/parser/module.f.mjs:59`, + `fjs/djs/parser/module.f.mjs:272`), used by `endArray`/`endObject`. +- [ ] In `fjs/media/json/parser/module.f.mjs`, add `pushStack` / `startContainer` + (or equivalently named); derive `startArray` / `startObject` from them. - [ ] Apply the same shape to `fjs/djs/parser/module.f.mjs`, preserving the - `{ ...state }` spread and the `endArray` non-array fallback inside the - `build` callback. + `{ ...state }` spread. - [ ] Run `npx tsc` and `fjs t`; confirm `fjs/media/json/parser/proof.f.mjs` and `fjs/djs/parser/proof.f.mjs` still pass with full line/branch coverage (behaviour is unchanged — this is a pure refactor). diff --git a/fjs/djs/todo/incremental.md b/fjs/djs/todo/incremental.md index f0872b7897..0bbe459f72 100644 --- a/fjs/djs/todo/incremental.md +++ b/fjs/djs/todo/incremental.md @@ -40,8 +40,8 @@ export default { ### 4. Next -- identifier properties -- trailing comma +- [x] identifier properties — shipped, `fjs/djs/parser/module.f.mjs` (`case 'id':`) +- [x] trailing comma — shipped, `fjs/djs/parser/module.f.mjs` (`parseObjectCommaOp`) ```js // import diff --git a/fjs/emergent_testing/todo/65y-proof-asserteq-adoption.md b/fjs/emergent_testing/todo/65y-proof-asserteq-adoption.md index 7c1989ac59..3d8f7037b9 100644 --- a/fjs/emergent_testing/todo/65y-proof-asserteq-adoption.md +++ b/fjs/emergent_testing/todo/65y-proof-asserteq-adoption.md @@ -23,16 +23,22 @@ if (cmp('apple')('banana') !== -1) { throw 3 } if (uint(s) !== 0x68656C6C_6F20776F_726C64n) { throw s } ``` -Counts in the current tree: - -- ~1,623 `if (...) { throw ... }` lines across `fjs/**/proof.f.mjs` — - the dominant assertion style. -- Only 4 files import `assertEq`: `fjs/sul/proof.f.mjs`, - `fjs/sul/level/hash/proof.f.mjs`, `fjs/sul/id/proof.f.mjs`, - `fjs/sul/id/module.f.mjs`. -- ~40 of the manual sites throw bare string literals - (`throw 'error'`, `throw 'Error'`) that carry no actual context — - worse than `assertEq`'s `[a, b]` pair on failure. +Counts in the current tree (re-verified 2026-08-14): + +- ~494 `if (...) { throw ... }` lines remain across `**/proof.f.mjs` — + down from the original count, but still a real chunk of the manual + pattern. +- 109 of 118 `proof.f.mjs` files now import `assertEq` — adoption is + well underway. The 9 remaining holdouts: `fjs/basen/base128/proof.f.mjs`, + `fjs/js/tokenizer/proof.f.mjs`, `fjs/media/json/tokenizer/proof.f.mjs`, + `fjs/types/nominal/proof.f.mjs`, + `fjs/types/object/structurally_same/proof.f.mjs`, + `fjs/types/range/proof.f.mjs`, `fjs/types/range_set/proof.f.mjs`, + `fjs/types/rtti/proof.f.mjs`, `todo/proof.f.mjs`. +- A number of files already using `assertEq` still carry leftover + manual `if (...) { throw ... }` sites alongside it (the 494 count + above is not confined to the 9 holdout files) — full adoption within + an already-migrated file is still incomplete in places. The mechanical translation is one-to-one: @@ -77,15 +83,15 @@ it's by far the most common and the lowest-judgement case. ### Why this qualifies -- **DRY at extreme volume.** ~1,623 spellings of the same three-token - conditional throw. Even partial adoption (e.g. the ~60% that are - exactly `if (x !== expected) { throw x }`) deletes hundreds of - redundant patterns and replaces them with a single call. +- **DRY at extreme volume.** Even after 109 of 118 files adopted + `assertEq`, ~494 spellings of the same three-token conditional throw + remain. Continuing adoption (both in the 9 holdout files and the + leftover manual sites within already-migrated files) keeps deleting + redundant patterns in favour of a single call. - **Failure-message quality goes up.** `throw [a, b]` always includes - both sides of the comparison. Today's `throw 0` / `throw 1` / + both sides of the comparison. Manual `throw 0` / `throw 1` / `throw 'error'` sites lose the actual value entirely, which forces - re-running with `console.log` to debug. The 40 bare-string throws - in particular are strictly worse than the helper. + re-running with `console.log` to debug. - **Separation of concerns.** "How a test asserts equality" is one decision and lives in one helper. Today each proof file re-makes that decision on every line. The helper already exists — it's just @@ -117,14 +123,14 @@ it's by far the most common and the lowest-judgement case. problem before mass-importing from `fjs/dev` into the `fjs/types` subtree. If there is, hoist `assert`/`assertEq` into a small `fjs/types/proof/module.f.mjs` (or co-located leaf) that `fjs/dev` can - re-export. The 4 existing `assertEq` consumers in `fjs/sul/` are a + re-export. The 109 existing `assertEq` consumers across the tree are a good existence proof that the import edge works from outside `fjs/types`. - **Land in small PRs.** AGENTS.md asks for "one feature/improvement - with minimal code changes" per PR; a single PR rewriting 1,600 lines - is not in the spirit of that rule even if each diff is trivial. - Folder-by-folder keeps reviews proportionate. No CHANGELOG entry per - PR — these are test-only changes. + with minimal code changes" per PR; a single PR rewriting hundreds of + lines is not in the spirit of that rule even if each diff is + trivial. Folder-by-folder keeps reviews proportionate. No CHANGELOG + entry per PR — these are test-only changes. - **Coverage delta = zero.** The helper does not change what is asserted, only how. Tests must continue to pass without any expected-result edits; if they don't, the rewrite caught a diff --git a/fjs/todo/132.md b/fjs/todo/132.md deleted file mode 100644 index 6b72d273eb..0000000000 --- a/fjs/todo/132.md +++ /dev/null @@ -1,7 +0,0 @@ -## 132. `exec` improvements. - -**Priority:** P3 -**Status:** open - -1. Keep most implementation code in `module.f.mjs` instead of `module.ts` -2. Use async functions and await instead of `.then` diff --git a/fjs/types/rtti/todo/proof-shared-asserts.md b/fjs/types/rtti/todo/proof-shared-asserts.md index c71d002b87..3656a646e1 100644 --- a/fjs/types/rtti/todo/proof-shared-asserts.md +++ b/fjs/types/rtti/todo/proof-shared-asserts.md @@ -24,12 +24,14 @@ In addition, `parse/proof.f.mjs` hand-rolls an `unwrap` that duplicates `unwrap` from `fjs/types/result/module.f.mjs:59` (assert `'ok'`, return the payload). -`assertDeepEqual` and `assertErrorPath`'s raw `if`/`throw` bodies are **done**: -`structurallySame` / `assertStructurallySame` landed, `assertDeepEqual` is -deleted in favour of `assertStructurallySame`, and `assertErrorPath` is now -`assertStructurallySame(e.path, expected, 'unexpected error path')`. What -remains below is the `unwrap` duplication, the `assertOk`/`assertError` move, -and sharing `assertErrorPath` itself between the two proofs. +`assertDeepEqual` is **done**: `structurallySame` / `assertStructurallySame` +landed and `assertDeepEqual` is deleted in favour of it. `assertErrorPath` is +only **half-migrated**: `parse/proof.f.mjs`'s copy is now +`assertStructurallySame(e.path, expected, 'unexpected error path')`, but +`validate/proof.f.mjs`'s copy still has the raw `if`/`throw` loop shown above. +What remains below is migrating `validate/proof.f.mjs`'s `assertErrorPath`, +the `unwrap` duplication, the `assertOk`/`assertError` move, and sharing +`assertErrorPath` itself between the two proofs. Beyond the helpers, roughly 80% of the two proof trees are copy-pasted verbatim modulo the checker name (`validate` vs `parse`): the `boolean` / @@ -70,8 +72,10 @@ Two steps; the first is the high-confidence part: - [ ] Move `assertOk`/`assertError` to `fjs/asserts/module.f.mjs` (with proof coverage) and update both rtti proofs. - [ ] Replace parse/proof's local `unwrap` with `fjs/types/result`'s `unwrap`. -- [x] Rewrite `assertErrorPath` and `assertDeepEqual` on top of the shared - assertion module — done via `assertStructurallySame`. +- [x] Delete `assertDeepEqual` in favour of `assertStructurallySame`. +- [ ] Rewrite `validate/proof.f.mjs`'s `assertErrorPath` on top of + `assertStructurallySame` — `parse/proof.f.mjs`'s copy already is, but + `validate/proof.f.mjs`'s still has the raw `if`/`throw` loop. - [ ] Share the rewritten `assertErrorPath` between the two proofs. - [ ] Evaluate the `commonSuite` factory; if adopted, keep the two proof files down to their genuinely divergent cases. diff --git a/fjs/types/todo/195.md b/fjs/types/todo/195.md deleted file mode 100644 index 4aa5f37ce1..0000000000 --- a/fjs/types/todo/195.md +++ /dev/null @@ -1,8 +0,0 @@ -## 195. Improve `listToVec` from `bit_vec` by changing concatenation order. - -**Priority:** P3 -**Status:** open - -Instead of -`(((((a + b) + c) + d) + e) + f)` which can be very slow for huge bigint, we can do -`(((a + b) + (c + d)) + (e + f))`. The number of operations that works with huge bigints is much smaller, $O(n)$ vs $O(\log n)$. We will still use the `fold` operation, but it will accumulate a binary tree branch. We can make this algorithm generic. diff --git a/fjs/website/todo/generate-website.md b/fjs/website/todo/generate-website.md index abf56d52ec..8548fdfd87 100644 --- a/fjs/website/todo/generate-website.md +++ b/fjs/website/todo/generate-website.md @@ -3,8 +3,8 @@ **Priority:** P3 **Status:** open -- [x] A minimal webpage -- [x] Generate Deno and Rust docs and publish them +- [x] A minimal webpage (`fjs/website/module.f.mjs` writes an `index.html` with a single GitHub link) +- [ ] Generate Deno and Rust docs and publish them - [ ] Convert `README.md` files into HTML and publish them - [ ] Source code highlighting - [ ] One `main.css` diff --git a/nanvm-lib/todo/86.md b/nanvm-lib/todo/86.md deleted file mode 100644 index 5c708ecf03..0000000000 --- a/nanvm-lib/todo/86.md +++ /dev/null @@ -1,29 +0,0 @@ -## 86. Operations for new VM implementation. - -**Priority:** P3 -**Status:** open - -```rust -// not all types require to implement these traits. -trait StringCoercion { - // link to MDN, optionally to ECMAScript - // https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String#string_coercion - fn string(self) -> String16 -} -// not types required to implement. -trait NumberCoercion { - // link to MDN, optionally to ECMAScript - fn unary_plus(self) -> Result -} -// ``` -// fn some() -> Result<(), Any> { -// let x = a.unary_plus()?; -// // let y = (-a)? // `-` never throws so we don't need `?`. -// let y = -a; -// } -// ``` -trait Js: StringCoercion + NumberCoercion + Neg {} - -impl Js for Any {} -impl Js for Unpacked {} -``` diff --git a/nanvm-lib/todo/89.md b/nanvm-lib/todo/89.md index 6641efa2d3..1d95976610 100644 --- a/nanvm-lib/todo/89.md +++ b/nanvm-lib/todo/89.md @@ -3,6 +3,31 @@ **Priority:** P3 **Status:** open +### Status (re-verified 2026-08-14) + +A real dispatch mechanism has since landed: `nanvm-lib/src/vm/dispatch.rs` +defines a `Dispatch` trait with one method per `Any` variant +(`nullish`/`bool`/`number`/`string`/`bigint`/`object`/`array`/`function`), +and it's used by the coercion visitors (`number_coercion.rs`, +`string_coercion.rs`, `primitive_coercion.rs`) to unpack an `Any` and +route to variant-specific logic. That satisfies the "match once on the +`Unpacked` variant, call out to a trait impl" half of this sketch. + +What it does *not* do is what the sketch below actually proposed: a +compile-time `Unary` tag (`UnaryPlus`, `UnaryMinus`, etc.) +where the *operation* is a generic parameter and each numeric type +implements `Unary` per operation, letting `Unpack` dispatch on both +axes (value variant × operation tag) through the type system. `Dispatch` +only dispatches on the value variant at runtime — there's no tag type, +and no generic `do::()`. For example, `nanvm-lib/src/vm/any/neg.rs` +still hand-implements `Neg for Any` with a direct `match self.to_numeric()` +over `Numeric::Number`/`Numeric::BigInt`, not through `Dispatch` or any +`Unary`-style trait. + +So: the runtime-dispatch-over-variants idea is done via `Dispatch`. +The compile-time operation-tag idea below is still an open proposal, +not implemented anywhere. + ```rust trait Unary { type Result; diff --git a/nanvm-lib/todo/numeric-operator-home.md b/nanvm-lib/todo/numeric-operator-home.md index 889edc52e6..5f14267351 100644 --- a/nanvm-lib/todo/numeric-operator-home.md +++ b/nanvm-lib/todo/numeric-operator-home.md @@ -47,6 +47,7 @@ home. ### Related -- [86](86.md) — coercion traits for a future VM, a different layer +- `src/vm/string_coercion.rs` / `src/vm/number_coercion.rs` — coercion + traits for a different layer, same "give the type its algebra" idea - [sign-algebra](sign-algebra.md) — the same "give the type its algebra" move for `Sign` diff --git a/todo/demo/README.md b/todo/demo/README.md index 6421859a88..fe4046cdd1 100644 --- a/todo/demo/README.md +++ b/todo/demo/README.md @@ -1,12 +1,16 @@ # FunctionalScript +> **Note:** This file is parked and stale; see [todo/samples.md](../samples.md) for +> the plan to migrate its usable content into a top-level `samples/` directory +> and delete this file. + ## 1. What is FunctionalScript? FunctionalScript is a **purely functional subset of JavaScript**. ### What does that mean? -* **Subset**: `.f.js` modules can be imported and used in JavaScript or TypeScript code without transpilation. +* **Subset**: `.f.mjs` modules can be imported and used in JavaScript or TypeScript code without transpilation. * **Functional**: Functions are first-class citizens and support composition and currying. * **Purely functional**: No side effects. Execution is deterministic and reproducible: @@ -18,7 +22,7 @@ FunctionalScript is a **purely functional subset of JavaScript**. FunctionalScript modules look like this: ```js -// math.f.js +// math.f.mjs export default { add: a => b => a + b, mul: a => b => a * b, @@ -29,7 +33,7 @@ You can use them from regular JavaScript: ```js // app.js -import math from "./math.f.js" +import math from "./math.f.mjs" const add2 = math.add(2) console.log("5 ===", add2(3)) @@ -78,23 +82,23 @@ Install the CLI: npm install --global functionalscript ``` -Convert `.f.js` files: +Convert `.f.mjs` files: ```sh # From JSON to FJS -fjs compile tree.json _tree.f.js +fjs compile tree.json _tree.f.mjs # To FunctionalScript (FJS) -fjs compile data.f.js _data.f.js +fjs compile data.f.mjs _data.f.mjs # To JSON -fjs compile data.f.js _data.json +fjs compile data.f.mjs _data.json ``` ## 5. Test Framework in FunctionalScript ```js -// test.f.js +// test.f.mjs const arrayOfTests = [ () => { if (2 + 2 !== 4) throw "It's the end of the world as we know it!" @@ -125,7 +129,6 @@ We are gradually adding more features: * Function support * Operators and control flow * Non-default exports - * `.f.ts` files (TypeScript type erasure) * Tooling: * BAST: Binary Abstract Syntax Tree for FunctionalScript @@ -141,6 +144,5 @@ We are gradually adding more features: * **Contribute**: We host **weekly contributor meetings**: everyone’s welcome. * **License**: - * Currently: AGPL (copyleft). - * Planning to adjust for broader adoption once we receive funding. + * Currently: MIT. * Need a custom license? Contact us: `sergey.oss@proton.me` diff --git a/todo/plan/capl.md b/todo/plan/capl.md index cf34f1a5b1..5dc984583b 100644 --- a/todo/plan/capl.md +++ b/todo/plan/capl.md @@ -22,7 +22,7 @@ This resolves several deep problems in modern software: **Normalization removes superficial differences.** The CA compiler normalizes code before hashing: it strips comments, whitespace, and renames internal variables to canonical forms. Two versions of a package that differ only in comments produce the same hash — they are the same package. This extends to dead code elimination: unused code that differs between versions does not affect the hash of the parts that are actually used. -Other CA languages exist — Unison is the most notable — but they require learning a new language and ecosystem from scratch. Most purely functional languages also impose a static type system (Haskell, Elm, PureScript). FunctionalScript takes a different approach: a dynamic type system at the core, with type validation as a separate, pluggable layer. TypeScript serves as the default validator today. Longer term, we plan to support additional type systems better suited to FunctionalScript's CA properties — including one based on `fjs/rtti` (runtime type information), which enables type-safe validation without requiring a compile-time type checker (tracked: [i141-universal-type-system](../141-universal-type-system.md), [i143-rtti-data](../143-rtti-data.md)). A pluggable type system means different communities can bring their own type discipline without forking the language. An RTTI-based type system has a further advantage: the same language is used for programming, for validating types, and for metaprogramming — one language, one mental model. This avoids the trap of TypeScript and similar systems, where the type layer is itself a separate, accidentally Turing-complete language (people have literally run DOOM inside the TypeScript type system). Types in FunctionalScript are ordinary FunctionalScript values and functions, not a second language bolted on top. Crucially, type annotations are erased during normalization — they do not affect the content hash of the logic. This means switching type systems never requires rewriting old algorithms: the normalized code is identical whether annotated with TypeScript types, RTTI validators, or no types at all. Old and new code remain fully compatible across type system changes. FunctionalScript is a strict subset of JavaScript: any software engineer who already knows JavaScript can read and write it immediately. The CA properties come from what FunctionalScript removes (mutation, side effects, identity-based equality) rather than from new syntax or concepts. This makes adoption frictionless for the world's largest developer community. +Other CA languages exist — Unison is the most notable — but they require learning a new language and ecosystem from scratch. Most purely functional languages also impose a static type system (Haskell, Elm, PureScript). FunctionalScript takes a different approach: a dynamic type system at the core, with type validation as a separate, pluggable layer. TypeScript serves as the default validator today. Longer term, we plan to support additional type systems better suited to FunctionalScript's CA properties — including one based on `fjs/rtti` (runtime type information), which enables type-safe validation without requiring a compile-time type checker. The RTTI data form is implemented at [`fjs/types/rtti/data/module.f.mjs`](../../fjs/types/rtti/data/module.f.mjs); the broader universal type system design is tracked in [i141](../../fjs/types/todo/141.md). A pluggable type system means different communities can bring their own type discipline without forking the language. An RTTI-based type system has a further advantage: the same language is used for programming, for validating types, and for metaprogramming — one language, one mental model. This avoids the trap of TypeScript and similar systems, where the type layer is itself a separate, accidentally Turing-complete language (people have literally run DOOM inside the TypeScript type system). Types in FunctionalScript are ordinary FunctionalScript values and functions, not a second language bolted on top. Crucially, type annotations are erased during normalization — they do not affect the content hash of the logic. This means switching type systems never requires rewriting old algorithms: the normalized code is identical whether annotated with TypeScript types, RTTI validators, or no types at all. Old and new code remain fully compatible across type system changes. FunctionalScript is a strict subset of JavaScript: any software engineer who already knows JavaScript can read and write it immediately. The CA properties come from what FunctionalScript removes (mutation, side effects, identity-based equality) rather than from new syntax or concepts. This makes adoption frictionless for the world's largest developer community. FunctionalScript's purely functional, side-effect-free design makes it an ideal foundation for a CA language: without mutation or identity-based equality, normalization is well-defined and deduplication is always safe.