diff --git a/CHANGELOG.md b/CHANGELOG.md index 87611de934..e70482e3b3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20,6 +20,30 @@ history. ## Unreleased +- **BREAKING CHANGES:** `fjs/media/type` migrates from authored + TypeScript (`.f.ts`) to JSDoc-typed JavaScript (`.f.mjs`), splitting + `DetectState`, `DetectMeta`, and the internal `_Signature`/`_MagicState`/ + `_Utf8Detect` types into a sibling `types.ts` — importers must use the + `.f.mjs` specifier for runtime values and the `types.ts` specifier for + types. `proof.f.ts` migrates alongside it + [#1493](https://github.com/functionalscript/functionalscript/pull/1493) +- **BREAKING CHANGES:** `fjs/ci/module.f.ts` migrates from authored + TypeScript (`.f.ts`) to JSDoc-typed JavaScript (`.f.mjs`), splitting + the `Setup` type into a sibling `types.ts` — importers must use the + `.f.mjs` specifier for runtime values and the `types.ts` specifier + for types. `proof.f.ts` stays TypeScript for now + [#1493](https://github.com/functionalscript/functionalscript/pull/1493) +- **BREAKING CHANGES:** `fjs/ci/node` migrates from authored + TypeScript (`.f.ts`) to JSDoc-typed JavaScript (`.f.mjs`) — no local + types to split. `proof.f.ts` migrates alongside it. Importers must + use the `.f.mjs` specifier + [#1493](https://github.com/functionalscript/functionalscript/pull/1493) +- **BREAKING CHANGES:** `fjs/ci/nix` migrates from authored TypeScript + (`.f.ts`) to JSDoc-typed JavaScript (`.f.mjs`), splitting the + `NixJob` type into a sibling `types.ts` — importers must use the + `.f.mjs` specifier for runtime values and the `types.ts` specifier + for types. `proof.f.ts` stays TypeScript for now + [#1493](https://github.com/functionalscript/functionalscript/pull/1493) - **BREAKING CHANGES:** `fjs/cli` migrates from authored TypeScript (`.f.ts`) to JSDoc-typed JavaScript (`.f.mjs`), splitting its type-level API into a sibling `types.ts` — importers must use the diff --git a/fjs/README.md b/fjs/README.md index 94a10c281a..822201d82a 100644 --- a/fjs/README.md +++ b/fjs/README.md @@ -53,7 +53,7 @@ cases. See [djs/README.md](djs/README.md) for the accepted subset. fjs ci ``` -`fjs ci` runs the built-in CI generator from `fjs/ci/module.f.ts`, writing +`fjs ci` runs the built-in CI generator from `fjs/ci/module.f.mjs`, writing `.github/workflows/ci.yml`. It is the standard entry point for projects that want FunctionalScript's default workflow. Projects with custom CI setup code should keep using `fjs run `, so their module can call `ci(setup)` with its own diff --git a/fjs/ci/README.md b/fjs/ci/README.md index 5d6ce733a1..20efacbc6c 100644 --- a/fjs/ci/README.md +++ b/fjs/ci/README.md @@ -7,20 +7,21 @@ canonical Node job under `nix/generated/`. ## Files -- `module.f.ts` — the top-level pipeline definition. Exports `ci(setup: Setup)` which - returns an `Effect` that writes the workflow file. Rust support is - detected automatically by checking for `Cargo.toml` at the repository root via the - `access` effect. +- `module.f.mjs` — the top-level pipeline definition. Exports `ci(setup: Setup)` + (`Setup` in `types.ts`) which returns an `Effect` that writes + the workflow file. Rust support is detected automatically by checking for + `Cargo.toml` at the repository root via the `access` effect. - `proof.f.ts` — property-based proofs for the CI generator (Rust/no-Rust job presence, per-OS extra steps). - `common/module.f.ts` — shared RTTI schemas and types (`Step`, `Job`, `Jobs`, `GitHubAction`, `MetaStep`, `Os`, `Architecture`), and step-builder helpers (`test`, `install`, `uses`). - `config/module.f.mjs` — runner image matrix (OS × architecture → GitHub-hosted image name) and pinned tool/package versions, including the FunctionalScript package version used by generated smoke tests and the exact Nixpkgs commit the generated flakes pin. -- `nix/module.f.ts` — writes one self-contained `nix/generated//flake.nix` - per declared job, using the Nix eDSL in `fjs/media/nix`. -- `node/module.f.ts` — Node.js job steps: platform smoke tests, canonical +- `nix/module.f.mjs` — writes one self-contained `nix/generated//flake.nix` + per declared job (`NixJob` in `types.ts`), using the Nix eDSL in `fjs/media/nix`. +- `node/module.f.mjs` — Node.js job steps: platform smoke tests, canonical per-version jobs, coverage, package checks, and the Node flake declarations. + `proof.f.mjs` — its property-based proofs. - `rust/module.f.mjs` — Rust toolchain setup and `cargo` build/test steps. - `deno/module.f.mjs` — Deno runtime steps. - `bun/module.f.mjs` — Bun runtime steps. @@ -42,7 +43,7 @@ plain text built from the pinned commit in `config/module.f.mjs`. ### Generated Nix environments Each canonical Node job declares a system and its Nixpkgs package attribute in -`node/module.f.ts` (`nodeNixJobs`), and `nix/module.f.ts` writes it out as one +`node/module.f.mjs` (`nodeNixJobs`), and `nix/module.f.mjs` writes it out as one static `flake.nix` exposing `devShells..default`. Node 22 also declares a job-local `shellHook` that points `npm install -g` at `$HOME/.npm-global`, so the installed `fjs` stays on `PATH` for the rest of the same `nix develop` invocation. diff --git a/fjs/ci/module.f.ts b/fjs/ci/module.f.mjs similarity index 66% rename from fjs/ci/module.f.ts rename to fjs/ci/module.f.mjs index e3de16584c..fbe1b6aaf7 100644 --- a/fjs/ci/module.f.ts +++ b/fjs/ci/module.f.mjs @@ -1,12 +1,15 @@ /** * Continuous integration helper commands for repository automation tasks. * + * See `./types.ts` for the `Setup` type-level API. + * * @module */ + import { mapStep, step } from '../effects/module.f.mjs' -import type { Effect } from '../effects/types.ts' +/** @import { Effect } from '../effects/types.ts' */ import { access, writeUtf8File } from '../effects/node/module.f.mjs' -import type { NodeOp } from '../effects/node/types.ts' +/** @import { NodeOp } from '../effects/node/types.ts' */ import { functionalscript, images } from './config/module.f.mjs' import { architecture, @@ -14,24 +17,17 @@ import { toSteps, ubuntuArm } from './common/module.f.mjs' -import type { - Architecture, - GitHubAction, - Job, - Jobs, - MetaStep, - Os, -} from './common/types.ts' +/** @import { Architecture, GitHubAction, Job, Jobs, MetaStep, Os } from './common/types.ts' */ import { rustPlatformSteps, rustWasmSteps } from './rust/module.f.mjs' -import { nodeMainSteps, nodeNixJobs, nodeNixVersionSteps, nodeVersionJobs } from './node/module.f.ts' -import { nixFlakes, nixInstall, type NixJob } from './nix/module.f.ts' +import { nodeMainSteps, nodeNixJobs, nodeNixVersionSteps, nodeVersionJobs } from './node/module.f.mjs' +import { nixFlakes, nixInstall } from './nix/module.f.mjs' +/** @import { NixJob } from './nix/types.ts' */ import { bunSteps } from './bun/module.f.mjs' import { denoSteps } from './deno/module.f.mjs' +/** @import { Setup } from './types.ts' */ -const job = ( - rust: boolean, - nodeExtra: readonly MetaStep[], -) => (o: Os) => (a: Architecture): readonly [string, Job] => { +/** @type {(rust: boolean, nodeExtra: readonly MetaStep[]) => (o: Os) => (a: Architecture) => readonly [string, Job]} */ +const job = (rust, nodeExtra) => o => a => { const id = `${o}-${a}` const image = images[o][a] const result = [ @@ -42,18 +38,17 @@ const job = ( return [id, { 'runs-on': image, steps: toSteps(result) }] } -export type Setup = { - readonly nodeExtra: (os: Os) => readonly MetaStep[], -} - // Every generated flake, across all job families that own one. -const nixJobs: readonly NixJob[] = nodeNixJobs +/** @type {readonly NixJob[]} */ +const nixJobs = nodeNixJobs // Temporary: proves the not-yet-migrated flakes still evaluate. Removed once // the canonical Node jobs check their own flake by running through it. -const nixFlakeJob: Job = ubuntuArm([nixInstall, ...nodeNixVersionSteps]) +/** @type {Job} */ +const nixFlakeJob = ubuntuArm([nixInstall, ...nodeNixVersionSteps]) -const canonicalJobs = (rust: boolean): Jobs => ({ +/** @type {(rust: boolean) => Jobs} */ +const canonicalJobs = rust => ({ ...(rust ? { wasm: ubuntuArm(rustWasmSteps) } : {}), deno: ubuntuArm(denoSteps(functionalscript)), bun: ubuntuArm(bunSteps(functionalscript)), @@ -61,15 +56,18 @@ const canonicalJobs = (rust: boolean): Jobs => ({ 'nix-flakes': nixFlakeJob, }) -export const ci = ({ nodeExtra }: Setup): Effect => step( +/** @type {(setup: Setup) => Effect} */ +export const ci = ({ nodeExtra }) => step( access('Cargo.toml'), result => { const rust = result[0] === 'ok' - const jobs: Jobs = { + /** @type {Jobs} */ + const jobs = { ...Object.fromEntries(os.flatMap(o => architecture.map(job(rust, nodeExtra(o))(o)))), ...canonicalJobs(rust), } - const gha: GitHubAction = { + /** @type {GitHubAction} */ + const gha = { name: 'CI', on: { pull_request: {}, diff --git a/fjs/ci/nix/module.f.ts b/fjs/ci/nix/module.f.mjs similarity index 68% rename from fjs/ci/nix/module.f.ts rename to fjs/ci/nix/module.f.mjs index bf5ac7bbfd..00e6bf00cd 100644 --- a/fjs/ci/nix/module.f.ts +++ b/fjs/ci/nix/module.f.mjs @@ -7,40 +7,33 @@ * readable on purpose: no job selection, no shared Nix modules, no helper * libraries. * + * See `./types.ts` for the `NixJob` type-level API. + * * @module */ + import { forEachStep, mapStep, pure, step } from '../../effects/module.f.mjs' -import type { Effect } from '../../effects/types.ts' +/** @import { Effect } from '../../effects/types.ts' */ import { mkdir, writeUtf8File } from '../../effects/node/module.f.mjs' -import type { Mkdir, WriteFile } from '../../effects/node/types.ts' +/** @import { Mkdir, WriteFile } from '../../effects/node/types.ts' */ import { nixToString } from '../../media/nix/module.f.mjs' -import type { Expression } from '../../media/nix/types.ts' +/** @import { Expression } from '../../media/nix/types.ts' */ import { fromUndefined, unwrap as unwrapNullable } from '../../types/nullable/module.f.mjs' import { unwrap } from '../../types/result/module.f.mjs' import { install, test, uses } from '../common/module.f.mjs' -import type { MetaStep } from '../common/types.ts' +/** @import { MetaStep } from '../common/types.ts' */ import { nixpkgs } from '../config/module.f.mjs' - -/** A CI job's development environment, one generated flake each. */ -export type NixJob = { - /** Generated directory name under `nix/generated`, matching the CI job id. */ - readonly id: string - /** Nix system of the job's runner, e.g. `aarch64-linux`. */ - readonly system: string - /** Nixpkgs attribute names made available in the job's shell. */ - readonly packages: readonly string[] - /** Job-local shell initialization, when the job needs one. */ - readonly shellHook?: string -} +/** @import { NixJob } from './types.ts' */ /** Directory owned by this generator. */ -export const generatedDirectory = 'nix/generated' as const +export const generatedDirectory = /** @type {const} */ ('nix/generated') const { commit } = nixpkgs const url = `github:NixOS/nixpkgs/${commit}` -const flake = ({ system, packages, shellHook }: NixJob): Expression => ['set', +/** @type {(job: NixJob) => Expression} */ +const flake = ({ system, packages, shellHook }) => ['set', ['=', ['inputs', 'nixpkgs', 'url'], url], ['=', ['outputs'], ['lambda', ['open-set-pattern', 'nixpkgs'], @@ -54,10 +47,10 @@ const flake = ({ system, packages, shellHook }: NixJob): Expression => ['set', ['apply', ['ref', 'pkgs', 'mkShell'], ['set', - ['=', ['packages'], ['list', ...packages.map(p => ['ref', 'pkgs', p] as const)]], + ['=', ['packages'], ['list', ...packages.map(p => /** @type {const} */ (['ref', 'pkgs', p]))]], ...(shellHook === undefined ? [] - : [['=', ['shellHook'], ['indented-string', shellHook]] as const]) + : [/** @type {const} */ (['=', ['shellHook'], ['indented-string', shellHook]])]) ] ] ]] @@ -72,11 +65,14 @@ const flake = ({ system, packages, shellHook }: NixJob): Expression => ['set', * is written here — a job only contributes attribute names and strings, which * are quoted when they are not identifiers. The unwrap is therefore a totality * assertion, not an input check. + * + * @type {(job: NixJob) => string} */ -export const flakeText = (job: NixJob): string => +export const flakeText = job => unwrapNullable(fromUndefined(nixToString(flake(job)))) -const writeFlake = (job: NixJob): Effect => { +/** @type {(job: NixJob) => Effect} */ +const writeFlake = job => { const directory = `${generatedDirectory}/${job.id}` const created = mapStep(mkdir(directory, { recursive: true }), unwrap) const written = step( @@ -85,26 +81,32 @@ const writeFlake = (job: NixJob): Effect => { return mapStep(written, unwrap) } -/** Writes one generated flake per job. */ -export const nixFlakes = (jobs: readonly NixJob[]): Effect => +/** + * Writes one generated flake per job. + * + * @type {(jobs: readonly NixJob[]) => Effect} + */ +export const nixFlakes = jobs => forEachStep(pure(jobs), writeFlake) /** Path a workflow passes to `nix develop`, for the job of the given id. */ -export const flakePath = (id: string): string => `./${generatedDirectory}/${id}` +export const flakePath = /** @type {(id: string) => string} */ (id => `./${generatedDirectory}/${id}`) /** Installs Nix, with `nix-command` and `flakes` enabled by the action's defaults. */ -export const nixInstall: MetaStep = install(uses('cachix/install-nix-action')) +export const nixInstall = install(uses('cachix/install-nix-action')) /** Runs one command inside a job's generated development shell. */ -export const nixDevelop = (id: string, command: string): string => - `nix develop ${flakePath(id)} --command ${command}` +export const nixDevelop = /** @type {(id: string, command: string) => string} */ + ((id, command) => `nix develop ${flakePath(id)} --command ${command}`) /** * Wraps a string so a POSIX shell reproduces it exactly. Single quotes protect * every other character, so only the quote itself needs handling: leave the * literal, reopen it, and escape the quote outside (`'` becomes `'\''`). + * + * @type {(value: string) => string} */ -const singleQuoted = (value: string): string => +const singleQuoted = value => `'${value.replaceAll("'", "'\\''")}'` /** @@ -114,8 +116,10 @@ const singleQuoted = (value: string): string => * * The commands are a shell script, joined so a failure stops the rest, and are * quoted as one argument — a command may contain quotes of its own. + * + * @type {(id: string, commands: readonly string[]) => string} */ -export const nixDevelopAll = (id: string, commands: readonly string[]): string => +export const nixDevelopAll = (id, commands) => nixDevelop(id, `bash -euo pipefail -c ${singleQuoted(commands.join(' && '))}`) /** @@ -124,6 +128,8 @@ export const nixDevelopAll = (id: string, commands: readonly string[]): string = * already determines the version, so this is the only place the expectation * is stated — the generated flakes stay declarative instead of carrying an * `assert` that restates the commit they pin. + * + * @type {(id: string, version: string) => MetaStep} */ -export const nixVersionCheckStep = (id: string, version: string): MetaStep => +export const nixVersionCheckStep = (id, version) => test({ run: `test "$(${nixDevelop(id, 'node --version')})" = v${version}` }) diff --git a/fjs/ci/nix/proof.f.ts b/fjs/ci/nix/proof.f.ts index 3b2058bd71..2df6cc48a3 100644 --- a/fjs/ci/nix/proof.f.ts +++ b/fjs/ci/nix/proof.f.ts @@ -8,7 +8,7 @@ import { step } from '../../effects/module.f.mjs' import { readUtf8File } from '../../effects/node/module.f.mjs' import { emptyState, virtual } from '../../effects/node/virtual/module.f.ts' import { nixpkgs } from '../config/module.f.mjs' -import { nodeNixJobs } from '../node/module.f.ts' +import { nodeNixJobs } from '../node/module.f.mjs' import { flakePath, flakeText, @@ -17,8 +17,8 @@ import { nixDevelopAll, nixFlakes, nixInstall, - type NixJob, -} from './module.f.ts' +} from './module.f.mjs' +import type { NixJob } from './types.ts' const { commit } = nixpkgs diff --git a/fjs/ci/nix/types.ts b/fjs/ci/nix/types.ts new file mode 100644 index 0000000000..7b36f2cd14 --- /dev/null +++ b/fjs/ci/nix/types.ts @@ -0,0 +1,17 @@ +/** + * Types for generated CI Nix flakes. + * + * @module + */ + +/** A CI job's development environment, one generated flake each. */ +export type NixJob = { + /** Generated directory name under `nix/generated`, matching the CI job id. */ + readonly id: string + /** Nix system of the job's runner, e.g. `aarch64-linux`. */ + readonly system: string + /** Nixpkgs attribute names made available in the job's shell. */ + readonly packages: readonly string[] + /** Job-local shell initialization, when the job needs one. */ + readonly shellHook?: string +} diff --git a/fjs/ci/node/module.f.ts b/fjs/ci/node/module.f.mjs similarity index 58% rename from fjs/ci/node/module.f.ts rename to fjs/ci/node/module.f.mjs index cfc9649b60..7cae01b798 100644 --- a/fjs/ci/node/module.f.ts +++ b/fjs/ci/node/module.f.mjs @@ -4,50 +4,62 @@ * * @module */ + import { node } from '../config/module.f.mjs' import { install, test, ubuntuArm, uses } from '../common/module.f.mjs' -import type { Job, Jobs, MetaStep } from '../common/types.ts' -import { nixInstall, nixVersionCheckStep, type NixJob } from '../nix/module.f.ts' +/** @import { Job, Jobs, MetaStep, Step } from '../common/types.ts' */ +import { nixInstall, nixVersionCheckStep } from '../nix/module.f.mjs' +/** @import { NixJob } from '../nix/types.ts' */ -export const major = (v: string): string => v.split('.')[0] +/** @type {(v: string) => string} */ +export const major = v => v.split('.')[0] -const jobId = (version: string): string => `node${major(version)}` +/** @type {(version: string) => string} */ +const jobId = version => `node${major(version)}` -const installNode = (version: string) => - uses('actions/setup-node', { 'node-version': version }) +/** @type {(v: string) => Step} */ +const installNode = v => + uses('actions/setup-node', { 'node-version': v }) -const nodeInstall = (v: string) => [ +/** @type {(v: string) => readonly MetaStep[]} */ +const nodeInstall = v => [ install(installNode(v)), test({ run: 'npm ci' }), ] -export const basicNode = (version: string) => (extra: readonly MetaStep[]): readonly MetaStep[] => [ +/** @type {(version: string) => (extra: readonly MetaStep[]) => readonly MetaStep[]} */ +export const basicNode = version => extra => [ ...nodeInstall(version), ...extra, ] -const fjsGlobalInstall = (version: string): MetaStep => +/** @type {(version: string) => MetaStep} */ +const fjsGlobalInstall = version => install({ run: `npm install -g functionalscript@${version}` }) -export const platformNodeSteps = (version: string): readonly MetaStep[] => [ +/** @type {(version: string) => readonly MetaStep[]} */ +export const platformNodeSteps = version => [ ...nodeInstall(node.default), fjsGlobalInstall(version), test({ run: 'fjs test' }), ] -const node22Steps = (version: string): readonly MetaStep[] => [ +/** @type {(version: string) => readonly MetaStep[]} */ +const node22Steps = version => [ ...nodeInstall(node.node22), fjsGlobalInstall(version), test({ run: 'fjs test' }), test({ run: 'node --test' }), ] -const node24Steps: readonly MetaStep[] = [ +/** @type {readonly MetaStep[]} */ +const node24Steps = [ ...nodeInstall(node.node24), test({ run: 'node --test' }), ] -const node26Steps: readonly MetaStep[] = [ +/** @type {readonly MetaStep[]} */ +const node26Steps = [ ...nodeInstall(node.default), test({ run: 'npm run ci-update' }), test({ run: 'git add -A && git diff --cached --exit-code' }), @@ -56,34 +68,40 @@ const node26Steps: readonly MetaStep[] = [ test({ run: 'npm pack' }), ] -const nodeJob = (steps: readonly MetaStep[]): Job => ubuntuArm(steps) +/** @type {(steps: readonly MetaStep[]) => Job} */ +const nodeJob = steps => ubuntuArm(steps) -export const nodeVersionJobs = (version: string): Jobs => ({ +/** @type {(version: string) => Jobs} */ +export const nodeVersionJobs = version => ({ [jobId(node.node22)]: nodeJob(node22Steps(version)), [jobId(node.node24)]: nodeJob(node24Steps), [jobId(node.default)]: nodeJob(node26Steps), }) // The canonical Node jobs run on the Ubuntu ARM runner. -export const nixSystem = 'aarch64-linux' as const +export const nixSystem = /** @type {const} */ ('aarch64-linux') // Keeps `npm install -g functionalscript` writable and puts the installed `fjs` // on `PATH` for the rest of the same `nix develop` invocation. -const npmGlobalShellHook = `export NPM_CONFIG_PREFIX="$HOME/.npm-global" +const npmGlobalShellHook = /** @type {const} */ (`export NPM_CONFIG_PREFIX="$HOME/.npm-global" export PATH="$NPM_CONFIG_PREFIX/bin:$PATH" -mkdir -p "$NPM_CONFIG_PREFIX"` as const +mkdir -p "$NPM_CONFIG_PREFIX"`) // Versions of the canonical Node jobs, in job order. -const nixVersions = [node.node22, node.node24, node.default] as const +const nixVersions = /** @type {const} */ ([node.node22, node.node24, node.default]) -const nixJob = (version: string): NixJob => ({ +/** @type {(version: string) => NixJob} */ +const nixJob = version => ({ id: jobId(version), system: nixSystem, packages: [`nodejs_${major(version)}`], }) -/** Generated development environments for the canonical Node jobs. */ -export const nodeNixJobs: readonly NixJob[] = [ +/** Generated development environments for the canonical Node jobs. + * + * @type {readonly NixJob[]} + */ +export const nodeNixJobs = [ { ...nixJob(node.node22), shellHook: npmGlobalShellHook }, nixJob(node.node24), nixJob(node.default), @@ -92,9 +110,11 @@ export const nodeNixJobs: readonly NixJob[] = [ /** * Version-check steps for the canonical Node jobs' generated flakes, one per * job. Collected into the shared temporary `nix-flakes` job in - * `fjs/ci/module.f.ts`. + * `fjs/ci/module.f.mjs`. + * + * @type {readonly MetaStep[]} */ -export const nodeNixVersionSteps: readonly MetaStep[] = +export const nodeNixVersionSteps = nixVersions.map(version => nixVersionCheckStep(jobId(version), version)) /** @@ -107,8 +127,10 @@ export const nodeNixVersionSteps: readonly MetaStep[] = * time. When the last one migrates and this job goes away, each migrated job * must check its own Node version inside the `nix develop` invocation, or the * guarantee is lost. + * + * @type {Job} */ -export const nodeNixFlakeJob: Job = ubuntuArm([ +export const nodeNixFlakeJob = ubuntuArm([ nixInstall, ...nodeNixVersionSteps, ]) diff --git a/fjs/ci/node/proof.f.ts b/fjs/ci/node/proof.f.mjs similarity index 94% rename from fjs/ci/node/proof.f.ts rename to fjs/ci/node/proof.f.mjs index 76d7b3422f..68059817fd 100644 --- a/fjs/ci/node/proof.f.ts +++ b/fjs/ci/node/proof.f.mjs @@ -1,4 +1,4 @@ -import { basicNode } from './module.f.ts' +import { basicNode } from './module.f.mjs' import { test } from '../common/module.f.mjs' import { assertEq } from '../../asserts/module.f.mjs' diff --git a/fjs/ci/proof.f.ts b/fjs/ci/proof.f.ts index 499d1c3832..7db23f0b7f 100644 --- a/fjs/ci/proof.f.ts +++ b/fjs/ci/proof.f.ts @@ -1,6 +1,6 @@ -import { ci, main } from './module.f.ts' +import { ci, main } from './module.f.mjs' import { functionalscript, node } from './config/module.f.mjs' -import { nodeNixJobs } from './node/module.f.ts' +import { nodeNixJobs } from './node/module.f.mjs' import { coverageInclude } from './deno/module.f.mjs' import { utf8, utf8ToString } from '../text/module.f.mjs' import { empty as emptyVec } from '../types/bit_vec/module.f.mjs' diff --git a/fjs/ci/todo/170.md b/fjs/ci/todo/170.md index 0813f6bb88..699454109c 100644 --- a/fjs/ci/todo/170.md +++ b/fjs/ci/todo/170.md @@ -26,7 +26,7 @@ export const denoSteps = (version: string): readonly MetaStep[] => [ test({ run: `${denoTest} --coverage && deno coverage --include='.*module\\.f\\.ts'` }), ] -// ci/node/module.f.ts:15-58 — same ingredients, split into composable pieces +// ci/node/module.f.mjs:25-53 — same ingredients, split into composable pieces const nodeInstall = (v: string) => [install(installNode(v)), test({ run: 'npm ci' })] export const basicNode = (version) => (extra) => [...nodeInstall(version), ...extra] const fjsGlobalInstall = (version) => install({ run: `npm install -g functionalscript@${version}` }) diff --git a/fjs/ci/todo/175.md b/fjs/ci/todo/175.md index d5d77febb7..233e8165b4 100644 --- a/fjs/ci/todo/175.md +++ b/fjs/ci/todo/175.md @@ -8,7 +8,7 @@ shape `install({ uses: '', with: { '-version': } })`, differing only in the action string and the version key/value: ```ts -// ci/node/module.f.ts:12 +// ci/node/module.f.mjs:21 const installNode = (version: string) => ({ uses: 'actions/setup-node@v6', with: { 'node-version': version } }) diff --git a/fjs/ci/todo/667-ci-self-test-script.md b/fjs/ci/todo/667-ci-self-test-script.md index e1a86cd5a3..0c93216d2a 100644 --- a/fjs/ci/todo/667-ci-self-test-script.md +++ b/fjs/ci/todo/667-ci-self-test-script.md @@ -60,6 +60,6 @@ npm uninstall -g - [ ] Choose and document the script name. - [ ] Move FunctionalScript's demo compile check into that package script. -- [ ] Update `fjs/ci/module.f.ts` to call the optional script instead of checking +- [ ] Update `fjs/ci/module.f.mjs` to call the optional script instead of checking for `package.json.name === "functionalscript"` for the demo compile step. - [ ] Update CI proofs for package-specific and absent-script behavior. diff --git a/fjs/ci/todo/669-ci-ubuntu-job-factory.md b/fjs/ci/todo/669-ci-ubuntu-job-factory.md index d49d88c89c..14bae4454c 100644 --- a/fjs/ci/todo/669-ci-ubuntu-job-factory.md +++ b/fjs/ci/todo/669-ci-ubuntu-job-factory.md @@ -23,7 +23,7 @@ export const ubuntuArm = (ms: readonly MetaStep[]): Job => ({ The entire body is duplicated. The only variation is `'runs-on'`. The same `{ 'runs-on': image, steps: toSteps(result) }` shape is also constructed in -`fjs/ci/module.f.ts`, so there are currently three surviving copies of the same Job +`fjs/ci/module.f.mjs`, so there are currently three surviving copies of the same Job construction pattern. The former `fjs/ci/playwright/module.f.ts` consumer is intentionally excluded: that @@ -44,7 +44,7 @@ export const ubuntu = job(images.ubuntu.intel) export const ubuntuArm = job(images.ubuntu.arm) ``` -Keep `job` exported because the surviving external construction in `fjs/ci/module.f.ts` +Keep `job` exported because the surviving external construction in `fjs/ci/module.f.mjs` can become: ```ts @@ -59,7 +59,7 @@ task must not add compatibility code for the deleted Playwright job. - [ ] Add the exported `job` factory in `fjs/ci/common/module.f.mjs`. - [ ] Re-express `ubuntu` and `ubuntuArm` in terms of `job`. -- [ ] Migrate the surviving external construction in `fjs/ci/module.f.ts`. +- [ ] Migrate the surviving external construction in `fjs/ci/module.f.mjs`. - [ ] Confirm `proof.f.ts` still covers `job`, `ubuntu`, and `ubuntuArm`. - [ ] Verify generated workflow output is unchanged. - [ ] Run `npx tsc` and `fjs t`. diff --git a/fjs/ci/todo/66b-dockerfile-nix-integration.md b/fjs/ci/todo/66b-dockerfile-nix-integration.md index 0b286cb026..61bb9d7d98 100644 --- a/fjs/ci/todo/66b-dockerfile-nix-integration.md +++ b/fjs/ci/todo/66b-dockerfile-nix-integration.md @@ -5,7 +5,7 @@ ### Progress -Phase 2 is done: `fjs/ci/nix/module.f.ts` generates +Phase 2 is done: `fjs/ci/nix/module.f.mjs` generates `nix/generated/node{22,24,26}/flake.nix` from the pinned Nixpkgs commit in `fjs/ci/config/module.f.mjs`, and `npm run ci-update` regenerates them without running Nix. `nodejs_22`, `nodejs_24`, and `nodejs_26` were verified to exist in diff --git a/fjs/ci/todo/66h-ci-npm-global-install.md b/fjs/ci/todo/66h-ci-npm-global-install.md index 4be945d55a..e6e6a96cc0 100644 --- a/fjs/ci/todo/66h-ci-npm-global-install.md +++ b/fjs/ci/todo/66h-ci-npm-global-install.md @@ -9,7 +9,7 @@ Two surviving CI step sites build the same `run`-based step for globally install pinned npm package: ```ts -// fjs/ci/node/module.f.ts +// fjs/ci/node/module.f.mjs const fjsGlobalInstall = (version: string): MetaStep => install({ run: `npm install -g functionalscript@${version}` }) @@ -65,7 +65,7 @@ This remains distinct from: ### Tasks - [ ] Add `npmGlobalInstall` to `fjs/ci/common/module.f.mjs`. -- [ ] Rebind `fjsGlobalInstall` in `fjs/ci/node/module.f.ts`. +- [ ] Rebind `fjsGlobalInstall` in `fjs/ci/node/module.f.mjs`. - [ ] Replace the inline `@typescript/native-preview` global-install step. - [ ] Confirm proof coverage for both surviving consumers and the generated step shape. - [ ] Verify generated workflow output is unchanged. diff --git a/fjs/ci/todo/ci-package-aware-deno-and-bun-steps.md b/fjs/ci/todo/ci-package-aware-deno-and-bun-steps.md index 0a05f83873..d800377dfe 100644 --- a/fjs/ci/todo/ci-package-aware-deno-and-bun-steps.md +++ b/fjs/ci/todo/ci-package-aware-deno-and-bun-steps.md @@ -26,7 +26,7 @@ test commands, runner image, and generated job identifier. ### Plan -- [ ] In `fjs/ci/module.f.ts`, read the repository root for `deno.lock` and `bun.lock` +- [ ] In `fjs/ci/module.f.mjs`, read the repository root for `deno.lock` and `bun.lock` through `access`, analogous to how `Cargo.toml` controls Rust jobs. - [ ] Construct the canonical job map so the complete Deno entry is conditionally added only when `deno.lock` exists. diff --git a/fjs/ci/todo/replace-npm-check-updates-with-an-internal-script.md b/fjs/ci/todo/replace-npm-check-updates-with-an-internal-script.md index 4aa5b03759..34a5f386a5 100644 --- a/fjs/ci/todo/replace-npm-check-updates-with-an-internal-script.md +++ b/fjs/ci/todo/replace-npm-check-updates-with-an-internal-script.md @@ -18,6 +18,6 @@ The internal update script would: - [ ] Define the `ci-lock.json` schema (tool versions + runner images + action versions). - [ ] Implement `fjs update` (or `fjs u`) subcommand that updates `package.json` deps and `ci-lock.json` tool versions. -- [ ] Update `fjs/ci/module.f.ts` to read `ci-lock.json` instead of importing `fjs/ci/config/module.f.mjs`. +- [ ] Update `fjs/ci/module.f.mjs` to read `ci-lock.json` instead of importing `fjs/ci/config/module.f.mjs`. - [ ] Bootstrap: generate a default `ci-lock.json` from the current `fjs/ci/config/module.f.mjs` values. - [ ] Wire `fjs u` (or equivalent) into the `update` script so dependency bumps are automated again. diff --git a/fjs/ci/todo/separate-ci-job-for-deno-coverage.md b/fjs/ci/todo/separate-ci-job-for-deno-coverage.md index be0512f273..3ad977bc4a 100644 --- a/fjs/ci/todo/separate-ci-job-for-deno-coverage.md +++ b/fjs/ci/todo/separate-ci-job-for-deno-coverage.md @@ -7,6 +7,6 @@ Add a dedicated CI job that runs `deno task cov` so coverage is tracked on every ### Plan -- [ ] Add a `deno-coverage` job to the CI generator (`fjs/ci/module.f.ts` or a new `fjs/ci/deno/module.f.mjs` variant) that runs `deno task cov` (runs `deno test --allow-read --allow-env && deno coverage --include='.*module\\.f\\.ts'`, matching the `npm run cov` scope). +- [ ] Add a `deno-coverage` job to the CI generator (`fjs/ci/module.f.mjs` or a new `fjs/ci/deno/module.f.mjs` variant) that runs `deno task cov` (runs `deno test --allow-read --allow-env && deno coverage --include='.*module\\.f\\.ts'`, matching the `npm run cov` scope). - [ ] Decide whether to upload the coverage report (e.g. to Codecov or as a GitHub artifact). - [ ] Run only on one platform (e.g. `ubuntu-intel`) to avoid redundancy. diff --git a/fjs/ci/types.ts b/fjs/ci/types.ts new file mode 100644 index 0000000000..26840f2780 --- /dev/null +++ b/fjs/ci/types.ts @@ -0,0 +1,11 @@ +/** + * Types for the CI workflow generator. + * + * @module + */ + +import type { MetaStep, Os } from './common/types.ts' + +export type Setup = { + readonly nodeExtra: (os: Os) => readonly MetaStep[], +} diff --git a/fjs/effects/todo/fold-stream-combinator.md b/fjs/effects/todo/fold-stream-combinator.md index d4c33d9c13..941d7c1033 100644 --- a/fjs/effects/todo/fold-stream-combinator.md +++ b/fjs/effects/todo/fold-stream-combinator.md @@ -13,7 +13,7 @@ Every consumer of a `List>` re-hand-writes the same three-case fold: *EOF → finalize; error item → propagate; chunk → fold and recurse on the tail.* The skeleton currently appears four times: -`detectStream` (`fjs/media/type/module.f.ts:290-299`) — pure fold: +`detectStream` (`fjs/media/type/module.f.mjs:268-281`) — pure fold: ```ts const loop = (s: DetectState) => (l: List>): Effect> => diff --git a/fjs/effects/todo/map-step-combinator.md b/fjs/effects/todo/map-step-combinator.md index db321be0e2..a6ec202dc9 100644 --- a/fjs/effects/todo/map-step-combinator.md +++ b/fjs/effects/todo/map-step-combinator.md @@ -50,7 +50,7 @@ const program = step( () => pure(0)) ``` -Also `fjs/djs/module.f.ts`, `fjs/module.f.ts`, `fjs/ci/module.f.ts`, +Also `fjs/djs/module.f.ts`, `fjs/module.f.ts`, `fjs/ci/module.f.mjs`, `fjs/cas/evo/module.f.ts`, `fjs/cas/module.f.ts`, `fjs/cas/cli/module.f.ts`, `fjs/mcp/cas/module.f.ts`, `fjs/protocol/mcp/module.f.ts`, `fjs/protocol/mcp/stdio/module.f.ts`, `fjs/emergent_testing/module.f.ts`. diff --git a/fjs/effects/todo/node-module-layering.md b/fjs/effects/todo/node-module-layering.md index 0b65948d7f..3d6b276000 100644 --- a/fjs/effects/todo/node-module-layering.md +++ b/fjs/effects/todo/node-module-layering.md @@ -83,7 +83,7 @@ Judgement calls worth deciding explicitly rather than by accident: Read the other way, `IoResult` is exactly a Node-layer contract and belongs beside the operations it describes. The fix for a **pure** consumer is to spell the underlying type, not to relocate the alias: - `fjs/media/type/module.f.ts:40` imports `type IoResult` from + `fjs/media/type/module.f.mjs:45` imports `IoResult` from `../../effects/node/types.ts` purely to write `IoResult` and `IoResult`; writing `Result` from `fjs/types/result` says the same thing and drops the `effects/node` import @@ -227,7 +227,7 @@ Judgement calls worth deciding explicitly rather than by accident: calls itself; that issue needs no change from this one. - [browser-testing](../../emergent_testing/todo/browser-testing.md) — owns the future Playwright adapter and browser-side test report. -- `fjs/media/type/module.f.ts:40`, `fjs/text/sgr/module.f.mjs:13`, +- `fjs/media/type/module.f.mjs:45`, `fjs/text/sgr/module.f.mjs:13`, `fjs/emergent_testing/module.f.ts:14-30` — importers that reach into the Node module for non-Node things. - [group-fs-subdirectories-by-concern](../../todo/group-fs-subdirectories-by-concern.md) diff --git a/fjs/mcp/README.md b/fjs/mcp/README.md index 9ff9aa202e..a76f13c4ca 100644 --- a/fjs/mcp/README.md +++ b/fjs/mcp/README.md @@ -135,7 +135,7 @@ such as tests. ### Metadata is size-independent (the default `content: false`) The metadata-only call **never buffers the blob**. It folds the CAS read stream -through [`fjs/media/type`](../media/type/module.f.ts) `detectStream` — a byte-accepting +through [`fjs/media/type`](../media/type/module.f.mjs) `detectStream` — a byte-accepting state machine (running byte count × magic-byte signature eliminator × UTF-8 validity DFA) that derives `{ length, mimeType, type }` in O(1) space. The detector stops decoding once the verdict is fixed — a magic match settles it @@ -154,7 +154,7 @@ valid UTF-8 until a trailing invalid byte is correctly classified as `base64` ### Content encoding (when `content: true`) Only the `content: true` path materializes the bytes (bounded by `maxLength`). It -classifies them with the **same** detector — [`fjs/media/type`](../media/type/module.f.ts) +classifies them with the **same** detector — [`fjs/media/type`](../media/type/module.f.mjs) `detectVec`, the single-`Vec` form of the `detectStream` machine above — so the three-way verdict is computed in exactly one place, never re-derived from a parallel `detect` + UTF-8 check. The `type` then selects whether the inline diff --git a/fjs/mcp/cas/module.f.ts b/fjs/mcp/cas/module.f.ts index 6ced6e27f7..0fb61df1d7 100644 --- a/fjs/mcp/cas/module.f.ts +++ b/fjs/mcp/cas/module.f.ts @@ -109,7 +109,7 @@ import type { MemOp } from '../../effects/memory/types.ts' import { cBase32ToVec, vecToCBase32 } from '../../basen/cbase32/module.f.mjs' import { decode as base64Decode, encode as base64Encode } from '../../basen/base64/module.f.mjs' import { tryUtf8 } from '../../text/module.f.mjs' -import { detectStream } from '../../media/type/module.f.ts' +import { detectStream } from '../../media/type/module.f.mjs' import { detect } from '../../media/module.f.ts' import { revisionDialect } from '../../media/revision/module.f.ts' import type { Vec } from '../../types/bit_vec/types.ts' diff --git a/fjs/media/json/todo/remove-native-json.md b/fjs/media/json/todo/remove-native-json.md index ba758af8e8..9426eed437 100644 --- a/fjs/media/json/todo/remove-native-json.md +++ b/fjs/media/json/todo/remove-native-json.md @@ -43,7 +43,7 @@ Three reasons to finish the job: | Assertion messages | 33 | `fjs/djs/tokenizer/proof.f.ts` (31), `fjs/types/rtti/ts/proof.f.mjs:8,12` (2) | pass the value, or `fjs/djs`'s `stringify` | | Source-text quoting | 5 | `fjs/emergent_testing/module.f.ts:305,322,335`, `fjs/types/ts/module.f.mjs:36,48` | `stringSerialize` — already designed in `66c-emit-literals-via-owner-modules.md` | | JSON line framing | 2 | `fjs/emergent_testing/proof.f.ts:42`, `fjs/mcp/proof.f.ts:128` | `stringify(identity)` | -| Pretty-printed file output | 1 | `fjs/ci/module.f.ts:81` | needs indentation support, which `serialize` does not have | +| Pretty-printed file output | 1 | `fjs/ci/module.f.mjs:83` | needs indentation support, which `serialize` does not have | Three semantic differences to respect while migrating, none of them blocking: @@ -113,7 +113,7 @@ reporter renders a failure payload with `String(v)` message. Either serialize with `fjs/djs`'s `stringify` (it handles the `bigint` token payloads) or improve the reporter's rendering first. -**4. Indentation for `fjs/ci/module.f.ts:81`**, the only site asking for +**4. Indentation for `fjs/ci/module.f.mjs:83`**, the only site asking for something `serialize` cannot do (`JSON.stringify(gha, null, ' ')`). Add an indenting variant to `fjs/media/json/serializer` — the natural shape is `serialize` parameterized by an indent unit, with today's behavior as the @@ -133,7 +133,7 @@ Consider a guard so it does not come back — the cheapest is a proof in - [ ] Phase 2: file the shortest-round-trip number-formatting issue under `fjs/types/bigfloat/todo/`, then implement `numberSerialize` on it. - [ ] Phase 3: migrate the write sites, row by row from the shape table. -- [ ] Phase 4: indenting serializer; migrate `fjs/ci/module.f.ts`. +- [ ] Phase 4: indenting serializer; migrate `fjs/ci/module.f.mjs`. - [ ] Per phase: `npx tsc`, `fjs t`, `npm run cov`, and a CHANGELOG entry. ### Related diff --git a/fjs/media/module.f.ts b/fjs/media/module.f.ts index 4ba8dd3044..a370703aca 100644 --- a/fjs/media/module.f.ts +++ b/fjs/media/module.f.ts @@ -43,7 +43,8 @@ */ import type { Vec } from '../types/bit_vec/types.ts' import { fromVec } from '../text/utf8/module.f.mjs' -import { detectVec, type DetectMeta } from './type/module.f.ts' +import { detectVec } from './type/module.f.mjs' +import type { DetectMeta } from './type/types.ts' import { parse } from './json/module.f.ts' import { assert, assertNotNullish } from '../asserts/module.f.mjs' import type { Struct } from '../types/rtti/types.ts' diff --git a/fjs/media/nix/todo/serializer-validation-split.md b/fjs/media/nix/todo/serializer-validation-split.md index 77b6ec49f0..a911af4c4a 100644 --- a/fjs/media/nix/todo/serializer-validation-split.md +++ b/fjs/media/nix/todo/serializer-validation-split.md @@ -127,7 +127,7 @@ collect. Do *not* add an `allOrNothing` / `traverse` helper to and abstracting it would preserve the plumbing this issue removes. **3. One public entry point that carries the reason.** There is exactly one -production importer — `fjs/ci/nix/module.f.ts` — so the contract is still cheap +production importer — `fjs/ci/nix/module.f.mjs` — so the contract is still cheap to fix, but the change is no longer confined to this module's own proof: ```ts @@ -169,7 +169,7 @@ make the reason reachable at all. One shape, decided here. The one production caller gets *simpler*, not harder. Today it launders `undefined` through the nullable convention to reach an assertion -(`fjs/ci/nix/module.f.ts:93-94`): +(`fjs/ci/nix/module.f.mjs:71-72`): ```ts export const flakeText = (job: NixJob): string => @@ -255,7 +255,7 @@ where it lands. `proof.f.ts:72-105` assertions must pass with only their `Result` wrapping changed, not their expected text. - [ ] Migrate the one production caller, `flakeText` - (`fjs/ci/nix/module.f.ts:93-94`): replace + (`fjs/ci/nix/module.f.mjs:71-72`): replace `unwrapNullable(fromUndefined(…))` with the `unwrap` from `fjs/types/result` that module already imports, and drop `fromUndefined`/`unwrapNullable` from its imports (`:15`). Land it in the diff --git a/fjs/media/todo/detect-cbor.md b/fjs/media/todo/detect-cbor.md index 536d4919f6..be3139db5a 100644 --- a/fjs/media/todo/detect-cbor.md +++ b/fjs/media/todo/detect-cbor.md @@ -71,7 +71,7 @@ matches. A canonical FS blob carries no wrapper (see the producer rule below), so its first byte is an ordinary CBOR header (e.g. a map header such as `0xA4`): no magic signature matches and the UTF-8 factor goes invalid almost immediately — exactly the state today's `isSettled` -(`fjs/media/type/module.f.ts`) treats as terminal `application/octet-stream`, +(`fjs/media/type/module.f.mjs`) treats as terminal `application/octet-stream`, which would freeze the verdict before this tier ever decodes the buffer. Tier 2 must add a detector factor that keeps such streams unsettled — buffering up to the 128 KiB cap — until the CBOR decode + schema validation succeeds or @@ -149,5 +149,5 @@ the honest answer. the same detector; tier 3 would be its CBOR sibling - [fjs/media/json streaming-recognizer](../json/todo/streaming-recognizer.md) — the payload-free recognizer pattern a tier-3 CBOR recognizer would follow -- `fjs/media/type/module.f.ts` — the magic table (tier 1) and `detectStream` (tiers 2–3) +- `fjs/media/type/module.f.mjs` — the magic table (tier 1) and `detectStream` (tiers 2–3) this lands in diff --git a/fjs/media/type/README.md b/fjs/media/type/README.md index a8cbb81487..427886b29f 100644 --- a/fjs/media/type/README.md +++ b/fjs/media/type/README.md @@ -4,7 +4,7 @@ Magic-byte MIME type detection: a pure table lookup over the leading bytes of a `Vec`. No I/O, no dependencies beyond [`fjs/types/bit_vec`](../../types/bit_vec/). ```ts -import { detect } from './module.f.ts' +import { detect } from './module.f.mjs' detect(pngBytes) // 'image/png' detect(textBytes) // null @@ -46,7 +46,7 @@ the module also exports `detectStream` — the streaming form of the **same byte-accepting state machine**: ```ts -import { detectStream, detectVec, push, finish, detectInit } from './module.f.ts' +import { detectStream, detectVec, push, finish, detectInit } from './module.f.mjs' // fold a CAS read stream (List>) into { length, mime_type, type } detectStream(stream) // Effect> diff --git a/fjs/media/type/module.f.ts b/fjs/media/type/module.f.mjs similarity index 70% rename from fjs/media/type/module.f.ts rename to fjs/media/type/module.f.mjs index 3a16475073..6cb0310c67 100644 --- a/fjs/media/type/module.f.ts +++ b/fjs/media/type/module.f.mjs @@ -30,20 +30,23 @@ * sits between the `RIFF` and `WEBP` markers, so it is matched as a prefix plus * a second marker at byte offset 8 rather than a single contiguous run. * + * See `./types.ts` for the type-level API. + * * @module */ -import type { Vec } from '../../types/bit_vec/types.ts' + import { msb, fromSentinel, length, u8List } from '../../types/bit_vec/module.f.mjs' +/** @import { Vec } from '../../types/bit_vec/types.ts' */ import { iterable } from '../../types/list/module.f.mjs' -import type { Nullable } from '../../types/nullable/types.ts' +/** @import { Nullable } from '../../types/nullable/types.ts' */ import { pure, step } from '../../effects/module.f.mjs' -import type { Effect, Operation } from '../../effects/types.ts' -import type { List } from '../../effects/list/types.ts' -import type { IoResult } from '../../effects/node/types.ts' +/** @import { Effect, Operation } from '../../effects/types.ts' */ +/** @import { List } from '../../effects/list/types.ts' */ +/** @import { IoResult } from '../../effects/node/types.ts' */ import { ok, error } from '../../types/result/module.f.mjs' import { isValidCodePoint, isTextCodePoint } from '../../text/code_point/module.f.mjs' -import type { Utf8State } from '../../text/utf8/types.ts' import { utf8ByteToCodePointOp } from '../../text/utf8/module.f.mjs' +/** @import { DetectMeta, DetectState, _MagicState, _Signature, _Utf8Detect } from './types.ts' */ const { startsWith, removeFront } = msb @@ -55,8 +58,10 @@ const sig = fromSentinel /** * Contiguous magic-byte signatures, checked in order; the first prefix match * wins. Ordering is irrelevant here — no signature is a prefix of another. + * + * @type {readonly (readonly [Vec, string])[]} */ -const table: readonly (readonly [Vec, string])[] = [ +const table = [ [sig(0x1_89_50_4e_47_0d_0a_1a_0an), 'image/png'], [sig(0x1_ff_d8_ffn), 'image/jpeg'], // Match the full GIF version headers ("GIF87a" / "GIF89a"), not just "GIF8", @@ -76,7 +81,8 @@ const table: readonly (readonly [Vec, string])[] = [ const riff = sig(0x1_52_49_46_46n) const webp = sig(0x1_57_45_42_50n) -const isWebp = (bytes: Vec): boolean => +/** @type {(bytes: Vec) => boolean} */ +const isWebp = bytes => length(bytes) >= 96n && startsWith(riff)(bytes) && startsWith(webp)(removeFront(64n)(bytes)) @@ -87,8 +93,10 @@ const isWebp = (bytes: Vec): boolean => * @returns the MIME type string for a recognized format, or `null` when the * leading bytes match no known signature (including any `Vec` shorter than * the signature it might otherwise match). + * + * @type {(bytes: Vec) => Nullable} */ -export const detect = (bytes: Vec): Nullable => { +export const detect = bytes => { if (isWebp(bytes)) { return 'image/webp' } for (const [s, m] of table) { if (startsWith(s)(bytes)) { return m } @@ -107,19 +115,11 @@ export const detect = (bytes: Vec): Nullable => { // of a large blob costs only length counting once the verdict is fixed (see // `isSettled`: a magic match settles it immediately, a dead magic once utf8 fails). -/** - * A magic-byte signature as a byte pattern. `null` entries are wildcards (the - * four little-endian size bytes of WebP, between its `RIFF` and `WEBP` markers). - */ -type Signature = { - readonly pattern: readonly Nullable[] - readonly mime: string -} - // The streaming counterpart of `table`/`isWebp`: the same signatures expressed as // byte patterns the eliminator can consume one byte at a time. WebP's gap is the // only wildcard run. -const signatures: readonly Signature[] = [ +/** @type {readonly _Signature[]} */ +const signatures = [ { pattern: [0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a], mime: 'image/png' }, { pattern: [0xff, 0xd8, 0xff], mime: 'image/jpeg' }, { pattern: [0x47, 0x49, 0x46, 0x38, 0x37, 0x61], mime: 'image/gif' }, @@ -134,19 +134,11 @@ const signatures: readonly Signature[] = [ }, ] -/** - * `A_magic`: signature elimination. `scan` holds the byte offset and the still-viable - * signatures; a fully matched signature absorbs into `matched`, an empty viable set - * into `dead`. Settles within 12 bytes — `matched`/`dead` are absorbing. - */ -type MagicState = - | { readonly tag: 'scan', readonly pos: number, readonly viable: readonly Signature[] } - | { readonly tag: 'matched', readonly mime: string } - | { readonly tag: 'dead' } - -const magicInit: MagicState = { tag: 'scan', pos: 0, viable: signatures } +/** @type {_MagicState} */ +const magicInit = { tag: 'scan', pos: 0, viable: signatures } -const magicStep = (m: MagicState, byte: number): MagicState => { +/** @type {(m: _MagicState, byte: number) => _MagicState} */ +const magicStep = (m, byte) => { if (m.tag !== 'scan') { return m } const { pos } = m const viable = m.viable.filter(s => { @@ -159,28 +151,14 @@ const magicStep = (m: MagicState, byte: number): MagicState => { return viable.length === 0 ? { tag: 'dead' } : { tag: 'scan', pos: pos + 1, viable } } -const magicMime = (m: MagicState): Nullable => m.tag === 'matched' ? m.mime : null +/** @type {(m: _MagicState) => Nullable} */ +const magicMime = m => m.tag === 'matched' ? m.mime : null -/** - * `A_utf8`: a streaming UTF-8 validity-and-text check riding the shared - * `utf8ByteToCodePointOp` decoder. `st` is the decoder's mid-sequence state; - * `valid` is `false` once an illegal byte, surrogate, or out-of-range code point - * is seen — `valid: false` is absorbing. A non-null `st` at EOF (a truncated - * multi-byte sequence) is invalid. `text` is the orthogonal text-ness verdict: it - * is `false` once a non-text (control) code point is decoded, even though that - * code point is perfectly well-formed UTF-8 — `text: false` is absorbing too. - * Keeping the two distinct lets a valid-but-control blob (e.g. NUL) decode - * cleanly yet still classify as binary. - */ -type Utf8Detect = { - readonly st: Utf8State - readonly valid: boolean - readonly text: boolean -} +/** @type {_Utf8Detect} */ +const utf8Init = { st: null, valid: true, text: true } -const utf8Init: Utf8Detect = { st: null, valid: true, text: true } - -const utf8Step = (u: Utf8Detect, byte: number): Utf8Detect => { +/** @type {(u: _Utf8Detect, byte: number) => _Utf8Detect} */ +const utf8Step = (u, byte) => { if (!u.valid) { return u } const [cps, st] = utf8ByteToCodePointOp(byte, u.st) let text = u.text @@ -191,24 +169,19 @@ const utf8Step = (u: Utf8Detect, byte: number): Utf8Detect => { return { st, valid: true, text } } -const utf8Valid = (u: Utf8Detect): boolean => u.valid && u.st === null +/** @type {(u: _Utf8Detect) => boolean} */ +const utf8Valid = u => u.valid && u.st === null // A blob is text only when it is whole-blob-valid UTF-8 *and* every decoded code // point is a text code point (no NUL/other controls). -const utf8Text = (u: Utf8Detect): boolean => utf8Valid(u) && u.text +/** @type {(u: _Utf8Detect) => boolean} */ +const utf8Text = u => utf8Valid(u) && u.text -/** - * The product state: running bit length × magic eliminator × UTF-8 validator. - * The factors never read each other; they meet only in {@link finish}. +/** The initial detector state `q₀`. + * + * @type {DetectState} */ -export type DetectState = { - readonly length: bigint - readonly magic: MagicState - readonly utf8: Utf8Detect -} - -/** The initial detector state `q₀`. */ -export const detectInit: DetectState = { +export const detectInit = { length: 0n, magic: magicInit, utf8: utf8Init, @@ -221,7 +194,8 @@ export const detectInit: DetectState = { // A magic `dead` leaves text-vs-octet open, so it settles only once utf8 can no // longer be text — either invalid or a control byte seen (both absorbing); `scan` // is never settled. -const isSettled = (magic: MagicState, utf8: Utf8Detect): boolean => { +/** @type {(magic: _MagicState, utf8: _Utf8Detect) => boolean} */ +const isSettled = (magic, utf8) => { switch (magic.tag) { case 'matched': return true case 'dead': return !utf8.valid || !utf8.text @@ -234,8 +208,10 @@ const isSettled = (magic: MagicState, utf8: Utf8Detect): boolean => { * always advances by the chunk's bit length; per-byte iteration stops as soon as * the verdict is fixed (see {@link isSettled}), so large blobs — including large * magic-matched ones — cost ≈ length counting. + * + * @type {(s: DetectState) => (chunk: Vec) => DetectState} */ -export const push = (s: DetectState) => (chunk: Vec): DetectState => { +export const push = s => chunk => { const bits = length(chunk) let magic = s.magic let utf8 = s.utf8 @@ -249,13 +225,6 @@ export const push = (s: DetectState) => (chunk: Vec): DetectState => { return { length: s.length + bits, magic, utf8 } } -/** The metadata read off the detector at end-of-stream. */ -export type DetectMeta = { - readonly length: bigint - readonly mime_type: string - readonly type: 'text' | 'base64' -} - /** * Reads the answer off the final state (`λ`). Reproduces the three-way result of * the pure path: magic hit → `base64` + detected mime; else whole-blob-valid UTF-8 @@ -263,8 +232,10 @@ export type DetectMeta = { * `text/plain`; else → `base64` + `application/octet-stream`. A valid-but-control * blob (NUL, other controls) is well-formed UTF-8 yet falls through to the binary * branch. + * + * @type {(s: DetectState) => DetectMeta} */ -export const finish = (s: DetectState): DetectMeta => { +export const finish = s => { const byteLength = s.length >> 3n const mime = magicMime(s.magic) if (mime !== null) { return { length: byteLength, mime_type: mime, type: 'base64' } } @@ -280,25 +251,31 @@ export const finish = (s: DetectState): DetectMeta => { * `cas_get` `content: true` path materializes the blob anyway): both paths read * the three-way `{ length, mime_type, type }` verdict from one machine instead of * re-deriving it from `detect` + a separate UTF-8 check. + * + * @type {(bytes: Vec) => DetectMeta} */ -export const detectVec = (bytes: Vec): DetectMeta => finish(push(detectInit)(bytes)) +export const detectVec = bytes => finish(push(detectInit)(bytes)) /** * Folds a CAS read stream through {@link push} and reads {@link finish} at EOF, * deriving `cas_get` metadata without ever materializing the blob. A read `error` * item short-circuits into the `IoResult` error. + * + * @template {Operation} O + * @param {List>} stream + * @returns {Effect>} */ -export const detectStream = - (stream: List>): Effect> => { - const loop = (s: DetectState) => (l: List>): Effect> => - step( - l, - (node): Effect> => { - if (node === undefined) { return pure(ok(finish(s))) } - const { first, tail } = node - const [t, v] = first - if (t === 'error') { return pure(error(v)) } - return loop(push(s)(v))(tail) - }) - return loop(detectInit)(stream) - } +export const detectStream = stream => { + /** @type {(s: DetectState) => (l: List>) => Effect>} */ + const loop = s => l => + step( + l, + node => { + if (node === undefined) { return pure(ok(finish(s))) } + const { first, tail } = node + const [t, v] = first + if (t === 'error') { return pure(error(v)) } + return loop(push(s)(v))(tail) + }) + return loop(detectInit)(stream) +} diff --git a/fjs/media/type/proof.f.ts b/fjs/media/type/proof.f.mjs similarity index 93% rename from fjs/media/type/proof.f.ts rename to fjs/media/type/proof.f.mjs index 1ce779f1bb..0bb8340743 100644 --- a/fjs/media/type/proof.f.ts +++ b/fjs/media/type/proof.f.mjs @@ -1,27 +1,31 @@ import { assert, assertEq } from '../../asserts/module.f.mjs' -import type { Vec } from '../../types/bit_vec/types.ts' import { msb, u8ListToVec, vec8, repeat, empty } from '../../types/bit_vec/module.f.mjs' +/** @import { Vec } from '../../types/bit_vec/types.ts' */ import { runPure } from '../../effects/module.f.mjs' import { nonEmpty, empty as emptyList } from '../../effects/list/module.f.mjs' -import type { List } from '../../effects/list/types.ts' -import type { Result } from '../../types/result/types.ts' +/** @import { List } from '../../effects/list/types.ts' */ +/** @import { Result } from '../../types/result/types.ts' */ import { ok } from '../../types/result/module.f.mjs' -import { detect, detectStream, detectVec, type DetectMeta } from './module.f.ts' +import { detect, detectStream, detectVec } from './module.f.mjs' +/** @import { DetectMeta } from './types.ts' */ // Builds a big-endian `Vec` from a list of byte values — mirrors how the CAS // store would hold the leading bytes of a stored blob. -const bytes = (...b: readonly number[]): Vec => u8ListToVec(msb)(b) +/** @type {(...b: readonly number[]) => Vec} */ +const bytes = (...b) => u8ListToVec(msb)(b) // ── Streaming detector helpers ────────────────────────────────────────────────── // Builds a CAS-style read stream from a sequence of ok(chunk) items. -const stream = (...chunks: readonly Vec[]): List> => - chunks.reduceRight>>( +/** @type {(...chunks: readonly Vec[]) => List>} */ +const stream = (...chunks) => + chunks.reduceRight( (tail, c) => nonEmpty(ok(c), tail), - emptyList>()) + /** @type {List>} */ (emptyList())) // Runs the streaming detector over the given chunks and unwraps the metadata. -const detectChunks = (...chunks: readonly Vec[]): DetectMeta => { +/** @type {(...chunks: readonly Vec[]) => DetectMeta} */ +const detectChunks = (...chunks) => { const o = runPure(detectStream(stream(...chunks))) assert(o.length === 1, 'effect is not pure') const [r] = o @@ -228,8 +232,9 @@ export const proof = { // A read `error` item short-circuits into the IoResult error. readErrorSurfaces: () => { - const errStream: List> = - nonEmpty(['error', 'boom'] as const, emptyList>()) + /** @type {List>} */ + const errStream = + nonEmpty(/** @type {const} */ (['error', 'boom']), emptyList()) const o = runPure(detectStream(errStream)) assert(o.length === 1, 'effect is not pure') assert(o[0][0] === 'error') diff --git a/fjs/media/type/todo/detect-json.md b/fjs/media/type/todo/detect-json.md index c0b107c34a..8601679fa9 100644 --- a/fjs/media/type/todo/detect-json.md +++ b/fjs/media/type/todo/detect-json.md @@ -9,7 +9,7 @@ The MCP server classifies stored content by content-sniffing, not by any stored type: `cas_get` folds the read stream through the `fjs/media/type` detector (`detectStream`) and reports `{ length, mime_type, type }`. The detector's -`finish` (`fjs/media/type/module.f.ts:264-272`) produces a three-way verdict: +`finish` (`fjs/media/type/module.f.mjs:238-246`) produces a three-way verdict: 1. magic-byte hit (PNG/JPEG/GIF/WebP/PDF/ZIP) → `base64` + the detected mime; 2. whole-blob-valid UTF-8 text → `text` + `text/plain`; @@ -31,7 +31,7 @@ future `fjs/media/type` consumer inherits it. Add JSON as a **refinement of the text branch**, keeping the single-classifier design (one machine, read off at EOF — no second, divergent copy of the rules) -that the module documents at `fjs/media/type/module.f.ts:96-105`. +that the module documents at `fjs/media/type/module.f.mjs:107-117`. #### 1. A fourth fold factor: a streaming JSON recognizer @@ -157,13 +157,13 @@ exactly the path `cas_get` uses. in `push`. - [ ] Refine `finish` to emit `application/json` for whole-blob-valid UTF-8 that is valid JSON **with an object/array top level** (§4 decision). -- [ ] Add `fjs/media/type/proof.f.ts` cases: `{"a":1}` and `[1,2,3]` (incl. split +- [ ] Add `fjs/media/type/proof.f.mjs` cases: `{"a":1}` and `[1,2,3]` (incl. split across chunks) → `application/json`/`text`; trailing garbage after valid JSON and truncated JSON → `text/plain`; non-JSON prose → `text/plain`; a raw TAB inside a string (`{"a":"⟨TAB⟩"}`) → `text/plain`, not `application/json`; bare scalars (`42`, `null`, `"hi"`, `true`) → `text/plain` (top-level object/array rule). -- [ ] Update `fjs/media/type/module.f.ts` module doc (recognised-types table) and the +- [ ] Update `fjs/media/type/module.f.mjs` module doc (recognised-types table) and the `cas_get` output section in `fjs/mcp/cas/module.f.ts` to list `application/json`. - [ ] `npx tsc` clean; `fjs t` green with both branches of the JSON verdict @@ -171,8 +171,8 @@ exactly the path `cas_get` uses. ### Related -- `fjs/media/type/module.f.ts:264-272` — `finish`, where the text→JSON refinement lands. -- `fjs/media/type/module.f.ts:180-195` — the UTF-8 factor whose decoded code points feed the JSON factor. +- `fjs/media/type/module.f.mjs:238-246` — `finish`, where the text→JSON refinement lands. +- `fjs/media/type/module.f.mjs:157-178` — the UTF-8 factor whose decoded code points feed the JSON factor. - `fjs/media/json/todo/streaming-recognizer.md` — **blocks this**; the payload-free, O(depth) validity recognizer `A_json` wraps. - `fjs/js/tokenizer/module.f.ts` — `parseStringStateOp`; already rejects raw U+0000–U+001F inside strings, so `A_json` inherits the correct verdict without re-deriving it. - `fjs/media/json/parser/module.f.ts:205-238` — `foldOp` / `parse`, the grammar the recognizer reuses value-free. diff --git a/fjs/media/type/todo/single-signature-table.md b/fjs/media/type/todo/single-signature-table.md index fbba8ee406..3fb9ea0e9b 100644 --- a/fjs/media/type/todo/single-signature-table.md +++ b/fjs/media/type/todo/single-signature-table.md @@ -6,24 +6,24 @@ ### Problem Every recognized signature (PNG, JPEG, 2×GIF, PDF, 3×ZIP, WebP) is declared -twice in `fjs/media/type/module.f.ts`, in two representations that must stay in +twice in `fjs/media/type/module.f.mjs`, in two representations that must stay in byte-for-byte lockstep: - sentinel-`Vec` form for the pure path — `table` at `:56-69` plus the WebP special case `riff`/`webp`/`isWebp` at `:73-79`, consumed by `detect` (`:88-94`); -- byte-pattern form for the streaming path — `signatures` at `:119-132` +- byte-pattern form for the streaming path — `signatures` at `:121-135` (WebP's gap expressed as `null` wildcards), consumed by `magicStep`/`detectVec`/`detectStream`. -The comment at `:116` admits it: *"The streaming counterpart of +The comment at `:118` admits it: *"The streaming counterpart of `table`/`isWebp`: the same signatures expressed as byte patterns…"*. Adding or correcting a signature means editing both lists, and WebP is special-cased in both (a bespoke `isWebp` here, a wildcard run there). Note the consumer situation: `detectStream` is the only export with a real -consumer (`fjs/mcp/cas/module.f.ts:88`); `detect` and `detectVec` are -exercised only by `fjs/media/type/proof.f.ts`. +consumer (`fjs/mcp/cas/module.f.ts:211`); `detect` and `detectVec` are +exercised only by `fjs/media/type/proof.f.mjs`. ### Proposal @@ -50,7 +50,7 @@ wildcard-free pattern into a `fromSentinel` bigint) so the byte values still appear once — but the fold-through-`magicStep` variant is smaller and needs no new helper. -Keep the doc-comment signature table (`:20-27`) as the human-readable +Keep the doc-comment signature table (`:18-27`) as the human-readable overview; it is documentation, not a third implementation. ### Tasks @@ -60,10 +60,10 @@ overview; it is documentation, not a third implementation. - [ ] Confirm `detect`'s "too short to match" behavior is preserved (a `scan`-state machine at EOF yields `null`, matching today's prefix semantics). -- [ ] `npx tsc`, `fjs t`; `fjs/media/type/proof.f.ts` must pass unchanged. +- [ ] `npx tsc`, `fjs t`; `fjs/media/type/proof.f.mjs` must pass unchanged. ### Related -- `fjs/media/type/module.f.ts:116` — the comment acknowledging the duplication. -- `fjs/mcp/cas/module.f.ts:88` — the sole external consumer +- `fjs/media/type/module.f.mjs:118` — the comment acknowledging the duplication. +- `fjs/mcp/cas/module.f.ts:211` — the sole external consumer (`detectStream`). diff --git a/fjs/media/type/types.ts b/fjs/media/type/types.ts new file mode 100644 index 0000000000..26d8923eef --- /dev/null +++ b/fjs/media/type/types.ts @@ -0,0 +1,67 @@ +/** + * Types for magic-byte MIME type detection. + * + * @module + */ + +import type { Nullable } from '../../types/nullable/types.ts' +import type { Utf8State } from '../../text/utf8/types.ts' + +/** + * A magic-byte signature as a byte pattern. `null` entries are wildcards (the + * four little-endian size bytes of WebP, between its `RIFF` and `WEBP` markers). + * + * @internal + */ +export type _Signature = { + readonly pattern: readonly Nullable[] + readonly mime: string +} + +/** + * `A_magic`: signature elimination. `scan` holds the byte offset and the still-viable + * signatures; a fully matched signature absorbs into `matched`, an empty viable set + * into `dead`. Settles within 12 bytes — `matched`/`dead` are absorbing. + * + * @internal + */ +export type _MagicState = + | { readonly tag: 'scan', readonly pos: number, readonly viable: readonly _Signature[] } + | { readonly tag: 'matched', readonly mime: string } + | { readonly tag: 'dead' } + +/** + * `A_utf8`: a streaming UTF-8 validity-and-text check riding the shared + * `utf8ByteToCodePointOp` decoder. `st` is the decoder's mid-sequence state; + * `valid` is `false` once an illegal byte, surrogate, or out-of-range code point + * is seen — `valid: false` is absorbing. A non-null `st` at EOF (a truncated + * multi-byte sequence) is invalid. `text` is the orthogonal text-ness verdict: it + * is `false` once a non-text (control) code point is decoded, even though that + * code point is perfectly well-formed UTF-8 — `text: false` is absorbing too. + * Keeping the two distinct lets a valid-but-control blob (e.g. NUL) decode + * cleanly yet still classify as binary. + * + * @internal + */ +export type _Utf8Detect = { + readonly st: Utf8State + readonly valid: boolean + readonly text: boolean +} + +/** + * The product state: running bit length × magic eliminator × UTF-8 validator. + * The factors never read each other; they meet only in `finish`. + */ +export type DetectState = { + readonly length: bigint + readonly magic: _MagicState + readonly utf8: _Utf8Detect +} + +/** The metadata read off the detector at end-of-stream. */ +export type DetectMeta = { + readonly length: bigint + readonly mime_type: string + readonly type: 'text' | 'base64' +} diff --git a/fjs/module.f.ts b/fjs/module.f.ts index 9d0333b8a2..c9d64ddc49 100644 --- a/fjs/module.f.ts +++ b/fjs/module.f.ts @@ -6,7 +6,7 @@ import { compile } from './djs/module.f.ts' import { main as testMain } from './emergent_testing/module.f.ts' import { commands as casCommands } from './cas/cli/module.f.ts' -import { main as ciMain } from './ci/module.f.ts' +import { main as ciMain } from './ci/module.f.mjs' import { import_ } from './effects/node/module.f.mjs' import type { NodeOp, NodeProgram } from './effects/node/types.ts' import { dispatch } from './cli/module.f.mjs' diff --git a/nix/README.md b/nix/README.md index 7ed4a4af1f..ed7c984a8b 100644 --- a/nix/README.md +++ b/nix/README.md @@ -1,6 +1,6 @@ # Nix environments -`generated//flake.nix` is **generated** by [`fjs/ci/nix`](../fjs/ci/nix/module.f.ts) +`generated//flake.nix` is **generated** by [`fjs/ci/nix`](../fjs/ci/nix/module.f.mjs) — one self-contained flake per CI job. Do not edit these files by hand: run `npm run ci-update` and commit the result. The Node 26 CI job fails when the committed files no longer match the generator's output. diff --git a/todo/plan/roadmap.md b/todo/plan/roadmap.md index ede9bc1162..9ca4f09ef3 100644 --- a/todo/plan/roadmap.md +++ b/todo/plan/roadmap.md @@ -41,7 +41,7 @@ **Layer 3 — Type detection (done)** - Detection via magic bytes: PNG, JPEG, GIF, WebP, PDF, ZIP → `null` for unrecognized bytes ✓ -- Pure logic in `fjs/media/type/module.f.ts` ✓ +- Pure logic in `fjs/media/type/module.f.mjs` ✓ - `cas_get`: when type is detected → returns `EmbeddedResource` with `mimeType`; when `null` → falls back to existing `textContent` response for backward compatibility ✓ - `fjs/protocol/mcp/module.f.ts` gained `blobResource` / `embeddedResource` schemas and a `contentItem` union ✓ - A separate on-demand `cas_type` tool is a possible extension; needs its own design issue before implementation