Skip to content
15 changes: 8 additions & 7 deletions fjs/cas/todo/66g-cas-get-verify-option.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,14 @@

### Problem

`Cas.read` / `fileKvStore.read` (`fjs/cas/module.f.mjs:51-57, 85-92`) and the `cas get`
command (`fjs/cas/module.f.mjs:126-146`) return the bytes stored at an address without
recomputing their hash. If a blob was corrupted, truncated, or misnamed — for example by
disk corruption or by a copy-files synchronization that has not yet been verified — a reader
between the copy and a later scrub can consume invalid content under a hash that was signed
or referenced elsewhere. The reader has no way to ask "and prove these bytes actually hash
to the address I requested."
`fileCas(sha2)(path).read` (`fjs/cas/module.f.mjs`, the `read` method on the `FileCas`
returned by `fileCas`) streams the bytes stored at an address, chunk by chunk, without
recomputing their hash, and the `cas get` command (`fjs/cas/cli/module.f.mjs`) pipes that
stream straight to the output file via `writeFromStream`. If a blob was corrupted, truncated,
or misnamed — for example by disk corruption or by a copy-files synchronization that has not
yet been verified — a reader between the copy and a later scrub can consume invalid content
under a hash that was signed or referenced elsewhere. The reader has no way to ask "and prove
these bytes actually hash to the address I requested."

A separate batch [`cas verify`](66g-cas-verify-command.md) command catches corruption
eventually, but there is a window before it runs, and some callers want certainty at the
Expand Down
7 changes: 4 additions & 3 deletions fjs/cas/todo/66g-cas-verify-command.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,9 @@

### Problem

`fileKvStore.read` (`fjs/cas/module.f.mjs:51-57`) returns whatever bytes live at the
addressed path without recomputing the hash. After **synchronization by copying files**
`fileCas(sha2)(path).read` (`fjs/cas/module.f.mjs`, the `read` method on the `FileCas`
returned by `fileCas`) streams whatever bytes live at the addressed path without
recomputing the hash. After **synchronization by copying files**
(see `issues/plan/vision.md`), or simply over time on a faulty disk, a blob can become
corrupted, truncated, or misnamed and no longer hash to the address it sits under. Nothing
in the store currently detects this, so the `same hash = same content` invariant the rest
Expand Down Expand Up @@ -40,7 +41,7 @@ Open design points:
### Tasks

- [ ] Add a `verify` function over `Cas`/`KvStore` that rehashes and reports mismatches
- [ ] Wire it as a `cas verify` CLI command in `fjs/cas/module.f.mjs`
- [ ] Wire it as a `cas verify` CLI command in `fjs/cas/cli/module.f.mjs`
- [ ] Decide delete vs. quarantine for corrupted blobs and implement it
- [ ] Tests: seed a store with a corrupted/truncated/misnamed blob and assert it is caught
- [ ] Document the command in `fjs/cas/README.md`
Expand Down
51 changes: 21 additions & 30 deletions fjs/cas/todo/66k-cas-get-return-path.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,31 +5,25 @@

### Problem

`cas get` currently reads the full file content through `fileKvStore.read` →
`readFile`, which is capped at `maxLengthBytes` (128 KiB). Files uploaded via the
streaming path (`cas upload`) can exceed this limit: the hash is stored and the
source is removed, but `cas get <hash>` silently reports the file as missing
because `readFile` rejects oversized reads and `fileKvStore.read` maps that error
to `undefined`.

More broadly, copying the full byte content through the effect layer is the wrong
model for large files: it requires a `Vec` allocation the size of the file, passes
it back to the caller, and forces the caller to write it out again — doubling peak
memory use.
`fileCas(sha2)(path).read` (`fjs/cas/module.f.mjs`) already streams the stored blob in
`<=128 KiB` chunks (`readBytes` in a loop, capped by `chunkBytes`/`maxLengthBytes` per
chunk, not per file), and `cas get` (`fjs/cas/cli/module.f.mjs`) pipes that stream to the
destination file via `writeFromStream`, so it no longer holds the whole file in memory and
is not limited to 128 KiB files. `FileCas` also already exposes a `url` method
(`fjs/cas/module.f.mjs`, and see `fjs/cas/types.ts`) that returns the path to a hash's
shard without reading its content.

What is still missing: `cas get` always copies the blob's bytes into a fresh destination
file. There is no way to ask it to just print the existing shard path (or a `file://` URL)
instead, so a caller that only wants to know where the content lives — to hard-link, open
directly, or hand the path to another tool — still pays for a full copy.

### Proposal

Change `cas get` (and the underlying read path) to return the filesystem **path**
of the stored object rather than its contents. Callers that need the bytes can
open the file themselves, stream it, or hard-link / copy it at the OS level with
no size restriction.

Two concrete forms to consider (may coexist):

- **Path** — return the absolute path string to the `.cas/…` shard file; the
caller issues a system-level copy or `rename` as needed.
- **`file://` URL** — same information, useful when the result is consumed by a
web client or another tool that already speaks URLs.
Add a mode where `cas get` prints the filesystem **path** (or a `file://` URL) of the
stored object instead of copying it to a destination file. Callers that need a private
copy can still request the current copy-out behavior; callers that only need to locate
the content use the new mode and open/stream/hard-link it themselves.

Additionally, mark the stored object **read-only** (e.g. `chmod 444`) immediately
after the final `rename` in the upload pipeline. This:
Expand All @@ -42,19 +36,16 @@ after the final `rename` in the upload pipeline. This:

### Tasks

- [ ] Add a `stat` / `lstat` primitive (or extend an existing one) to retrieve
file size without loading content, so callers can branch on size
- [ ] Add a `chmod` (or `setReadOnly`) effect for marking files immutable after
write
- [ ] Change `cas get` to print the shard path (and optionally a `file://` URL)
instead of copying bytes to a destination file
- [ ] Update `fileKvStore` (or add a parallel interface) with a `getPath` method
that returns the path for a given hash without reading content
- [ ] Add a `cas get` mode (flag or subcommand) that prints the shard path — via
the existing `FileCas.url` — instead of copying bytes to a destination file
- [ ] Apply `setReadOnly` in the `cas upload` pipeline after the final `rename`
- [ ] Update proof tests and documentation

### Related

- `fileCas.write` / `casAddFile` (`fjs/cas/module.f.mjs`) — streaming upload
pipeline that stores files `cas get` cannot currently read back (design
formerly tracked as `66j-cas-large-file-support`, now implemented and deleted)
pipeline; `cas get` now reads uploaded files back via the streaming `read`
path (design formerly tracked as `66j-cas-large-file-support`, now
implemented and deleted)
13 changes: 11 additions & 2 deletions fjs/ci/todo/138.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,15 @@

**Priority:** P3
**Status:** open
**Blocked by:** i136

Implement a script that will update the lock file by reading the latest versions of tools from the internet using the instructions from 136.
Implement a script that will update the lock file by reading the latest versions of tools from the internet.

### Related

This overlaps with the newer Nix-based tool-update proposals — check those first to avoid
duplicating effort:

- [65Z-ci-nix](65z-ci-nix.md) — the Nixpkgs update command (`npm run ci-nix-update`) covers
updating pinned tool/package versions via Nix.
- [replace-npm-check-updates-with-an-internal-script](replace-npm-check-updates-with-an-internal-script.md)
— a broader internal-script proposal (`ci-lock.json`) covering the same lock-file-update idea.
28 changes: 21 additions & 7 deletions fjs/ci/todo/66h-ci-npm-global-install.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,18 @@

### Problem

Two surviving CI step sites build the same `run`-based step for globally installing a
**Update:** the `@typescript/native-preview`/`tsgo` global-install call site described
below no longer exists in the repo (verified via
`grep -rn "native-preview\|tsgo" .github/workflows/ fjs/ci/`, which finds only this file's
own text). `fjs/ci/node/module.f.mjs` currently has a single global-install site,
`fjsGlobalInstall`, for `functionalscript`. With only one real consumer left, the
second-consumer threshold this proposal relies on (see "Why this still qualifies" below)
no longer holds, and a shared `npmGlobalInstall` factory may not be worth the abstraction
until a second consumer actually reappears. Leaving this open rather than closing it,
since a future global-install site (or the return of a tsgo-like tool) would revive the
case for it.

Originally, two CI step sites built the same `run`-based step for globally installing a
pinned npm package:

```ts
Expand All @@ -16,8 +27,8 @@ const fjsGlobalInstall = (version: string): MetaStep =>
install({ run: `npm install -g @typescript/native-preview@${tsgo}` })
```

The shape `install({ run: `npm install -g ${pkg}@${version}` })` is duplicated; only the
package name and version differ.
The shape `install({ run: `npm install -g ${pkg}@${version}` })` was duplicated; only the
package name and version differed.

The former `fjs/ci/playwright/module.f.mjs` call site is intentionally not a consumer of
this proposal. That job and its global install have already been deleted, and this task
Expand Down Expand Up @@ -49,10 +60,12 @@ acceptable implementation choice if those related APIs settle on that style.

### Why this still qualifies

- There are two real surviving consumers, meeting the second-consumer threshold.
- **Only one real surviving consumer remains** (`fjsGlobalInstall`); the
`@typescript/native-preview` site is gone, so the original second-consumer threshold no
longer holds — see the Problem update above.
- The construction is identical and varies only by data.
- The abstraction names one repository policy: install a pinned npm tool globally.
- A future third consumer can reuse the factory without restoring the deleted Playwright
- A future second consumer can reuse the factory without restoring the deleted Playwright
job.

This remains distinct from:
Expand All @@ -66,8 +79,9 @@ This remains distinct from:

- [ ] Add `npmGlobalInstall` to `fjs/ci/common/module.f.mjs`.
- [ ] Rebind `fjsGlobalInstall` in `fjs/ci/node/module.f.mjs`.
- [ ] Replace the inline `@typescript/native-preview` global-install step.
- [ ] Confirm proof coverage for both surviving consumers and the generated step shape.
- [ ] ~~Replace the inline `@typescript/native-preview` global-install step.~~ (site no
longer exists — see Problem update)
- [ ] Confirm proof coverage for the surviving consumer and the generated step shape.
- [ ] Verify generated workflow output is unchanged.
- [ ] Run `npx tsc` and `fjs t`.

Expand Down
2 changes: 1 addition & 1 deletion fjs/ci/todo/ci-integration-tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ The key insight: it matters far more that the *published package* works on every

Scenarios are expressed as FunctionalScript modules. A scenario module exports a `main` (a `NodeProgram`) that receives the environment and args and returns an effect. The CI generator reads a scenario list and emits one job per scenario.

See [669-scenario-testing.md](669-scenario-testing.md) for the scenario design — each scenario is a declarative description of initial state, an effect, and an expected result that can be run as either a unit test (mock interpreter) or a real CI job.
Each scenario is a declarative description of initial state, an effect, and an expected result that can be run as either a unit test (mock interpreter) or a real CI job. (The `669-scenario-testing.md` design doc this used to reference no longer exists; issue number 669 has since been reused for unrelated files, e.g. [669-ci-ubuntu-job-factory.md](669-ci-ubuntu-job-factory.md).)

Open questions:
- Where do scenario modules live? (`issues/demo/` style, or a dedicated `fjs/ci/scenarios/` directory?)
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
## Deno 2.8.3: `deno install --frozen` breaks `deno run -A npm:functionalscript`

**Priority:** P1
**Status:** investigate
**Status:** needs re-verification (see note below)

With Deno 2.8.3, running `deno install --frozen` before `deno run -A npm:functionalscript@0.30.0` produces:

Expand All @@ -10,3 +10,13 @@ error: Failed resolving binary export. '.../node_modules/.deno/functionalscript@
```

The same command succeeds if `deno install --frozen` is **not** run beforehand.

**Note (re-checked 2026-08-14):** CI now pins Deno `2.9.5` (`.github/workflows/ci.yml`,
`deno` in `fjs/ci/config/module.f.mjs`), not 2.8.3. The current Deno step order in
`fjs/ci/deno/module.f.mjs` already runs the smoke test (`deno run -A ... npm:functionalscript
... test`) *before* `deno install --frozen`, so the ordering that triggered this bug report
is no longer present in the pipeline as generated today. This has not been re-tested against
2.9.5 directly (e.g. running `deno install --frozen` immediately before `deno run -A
npm:functionalscript` by hand), so treat this as likely resolved by the current step order /
version bump, but unconfirmed — re-verify against the currently pinned Deno version before
closing, since Deno version pins change again.
16 changes: 11 additions & 5 deletions fjs/djs/todo/157.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,9 +117,13 @@ The deltas:
- `serializeWithConst` is `serializeWithoutConst` plus a ref-counter
short-circuit prepended to `f`.

**Sub-task 2b (clearest, smallest):** the two DJS functions collapse into one
factory taking an optional ref-lookup callback — when absent, the const
short-circuit is skipped and you get `serializeWithoutConst`.
**Sub-task 2b (clearest, smallest, done):** the two DJS functions now collapse
into one `buildSerialize(refLookup)(sort)` factory in
`fjs/djs/serializer/module.f.mjs` taking an optional ref-lookup callback —
`serializeWithoutConst = buildSerialize(noRef)`, and `serializeWithConst`
supplies a ref-lookup closure that substitutes `c<N>` references. What
remains of this section is extracting a shared walker between JSON's
`serialize` (`fjs/media/json/module.f.mjs:52`) and DJS's `buildSerialize`.

A `serializeValue` factory (in `json/serializer`) parameterized by the extra
`typeof` cases and an optional pre-`f` hook covers all three call sites.
Expand Down Expand Up @@ -150,8 +154,10 @@ line numbers changed. Any extraction here must first re-measure the current code
token tree.
- [ ] Keep DJS module framing, refs, identifier keys, and metadata behavior
DJS-specific.
- [ ] Extract the serializer walker independently where useful; collapse the two
DJS serializer variants through an optional ref hook.
- [x] Collapse the two DJS serializer variants through an optional ref hook —
landed as `buildSerialize` in `fjs/djs/serializer/module.f.mjs`.
- [ ] Extract the serializer walker independently, shared between JSON's
`serialize` and DJS's `buildSerialize`, where useful.
- [ ] Re-measure the current tokenizer minus-folding duplication before extracting
it; do not implement the stale line-number design blindly.
- [ ] Preserve current behavior/proof coverage for both JSON and DJS.
Expand Down
Loading
Loading