From 01f29ad7fe03934a0844206d1c89738def0709a4 Mon Sep 17 00:00:00 2001 From: Sergey Shandar Date: Thu, 27 Aug 2026 07:27:09 +0000 Subject: [PATCH 01/18] emergent_testing: one proof runner for `fjs t` and the browser MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The browser runner and `fjs t` implemented the same proof semantics twice — leaf discovery, tree walking, the structural `throw` expectation, promise resolution, path formatting, counting — in `emergent_testing/browser.mjs` and `emergent_testing/module.f.mjs`, and had begun to drift. They are now one runner. `runModuleMap` is the single source of truth; a host supplies a `Reporter` and an effect interpreter and nothing else. - `fjs/effects/common/` holds the operations no host owns — `all`, `await`, `fetch`, `import`, `now`, `sandbox` and the `IoError` helpers — moved out of `fjs/effects/node/`, which re-exports every one of them unchanged. - `fjs/effects/browser/module.mjs` interprets exactly that set against a browser realm, and takes the composed runner so a page can add operations of its own. - `emergent_testing`'s new `report`/`reported` operations and `recordingReporter` normalize each leaf into a `TestResult` carrying no terminal text and no DOM. - `emergent_testing/browser/module.f.mjs` is the pure browser application — link, run, report — provable from Node with a stand-in interpreter; `emergent_testing/browser/module.mjs` is left with the DOM, the published promise and the completion event. Verified end to end in Chromium: 3435 proofs, all passing, rendered and published from the generated page. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_016PyLwDNkPQM1uD1Tg5ApBg --- fjs/effects/README.md | 25 + fjs/effects/browser/module.mjs | 123 +++++ fjs/effects/common/module.f.mjs | 217 ++++++++ fjs/effects/common/proof.f.mjs | 168 ++++++ fjs/effects/common/types.ts | 143 ++++++ fjs/effects/memory/types.ts | 2 +- fjs/effects/node/module.f.mjs | 216 +------- fjs/effects/node/proof.f.mjs | 75 +-- fjs/effects/node/types.ts | 130 +---- fjs/emergent_testing/README.md | 44 ++ fjs/emergent_testing/browser.mjs | 455 ---------------- fjs/emergent_testing/browser/module.f.mjs | 140 +++++ fjs/emergent_testing/browser/module.mjs | 177 +++++++ fjs/emergent_testing/browser/proof.f.mjs | 162 ++++++ fjs/emergent_testing/browser/proof.mjs | 484 ++++++------------ .../browser/species.proof.mjs | 45 -- fjs/emergent_testing/browser/types.ts | 74 +++ fjs/emergent_testing/module.f.mjs | 97 +++- fjs/emergent_testing/proof.f.mjs | 62 ++- .../todo/browser-test-controls.md | 4 +- fjs/emergent_testing/todo/browser-testing.md | 9 +- .../todo/hostile-proof-values.md | 62 +++ .../todo/share-browser-console-runner.md | 146 ------ fjs/emergent_testing/types.ts | 43 +- fjs/website/module.f.mjs | 2 +- fjs/website/todo/generate-website.md | 2 +- .../todo/website-preparation-program.md | 69 +++ 27 files changed, 1809 insertions(+), 1367 deletions(-) create mode 100644 fjs/effects/browser/module.mjs create mode 100644 fjs/effects/common/module.f.mjs create mode 100644 fjs/effects/common/proof.f.mjs create mode 100644 fjs/effects/common/types.ts delete mode 100644 fjs/emergent_testing/browser.mjs create mode 100644 fjs/emergent_testing/browser/module.f.mjs create mode 100644 fjs/emergent_testing/browser/module.mjs create mode 100644 fjs/emergent_testing/browser/proof.f.mjs delete mode 100644 fjs/emergent_testing/browser/species.proof.mjs create mode 100644 fjs/emergent_testing/browser/types.ts create mode 100644 fjs/emergent_testing/todo/hostile-proof-values.md delete mode 100644 fjs/emergent_testing/todo/share-browser-console-runner.md create mode 100644 fjs/website/todo/website-preparation-program.md diff --git a/fjs/effects/README.md b/fjs/effects/README.md index 9cc4cd487..6049fb864 100644 --- a/fjs/effects/README.md +++ b/fjs/effects/README.md @@ -144,6 +144,31 @@ conflated in either direction — a capability the runner merely lacks is answer with `NotImplemented`, never by killing the program, and a refusal to continue is an interruption, never dressed up as `NotImplemented`. +## Where an operation lives + +An operation belongs to the host that alone can perform it, and to +[`./common/`](./common/module.f.mjs) when no host owns it. `all`, `await`, +`fetch`, `import`, `now` and `sandbox` describe what a JavaScript *realm* can do +— hold a value, wait for a promise, measure a call, link a module — so the Node +runner, the browser runner and the virtual runner each implement the same +command at the same contract. `readFile`, `write`, `exec`, `createServer` and +`test` describe what a *host* can do, and stay in [`./node/`](./node/types.ts). + +The line is not bookkeeping. It is what lets a program state that it needs +nothing host-specific and then be run by either host: the browser proof runner +(`fjs/emergent_testing/browser/module.f.mjs`) performs only `CommonOp` plus two +operations of its own, which is why it and `fjs t` can share every line of proof +semantics between them. `./node/` re-exports every common name, so a consumer +that already imports one module for `readFile` keeps importing it for `sandbox`. + +An interpreter lives beside the host it interprets — [`./node/module.mjs`](./node/module.mjs), +[`./browser/module.mjs`](./browser/module.mjs) — and the browser one implements +`CommonOp` and nothing else. There is no browser filesystem and no browser +stdout, and inventing spellings for them would describe a host that does not +exist; a page that needs an operation of its own composes its handlers on top of +that map, which is why `browserOperationMap` takes the composed runner rather +than closing over one of its own. + ## Leaving the layer Not every consumer is ready to compose. Two named policies exist so that a site diff --git a/fjs/effects/browser/module.mjs b/fjs/effects/browser/module.mjs new file mode 100644 index 000000000..9d8e1f2b2 --- /dev/null +++ b/fjs/effects/browser/module.mjs @@ -0,0 +1,123 @@ +/** + * Browser effect runner: interprets the host-independent operations + * (`../common/types.ts`) against a browser realm. + * + * It is the browser's counterpart of [`../node/module.mjs`](../node/module.mjs) + * and deliberately implements **only** `CommonOp`. There is no browser + * filesystem, no subprocess and no stdout to interpret, and inventing browser + * spellings for those would describe a host that does not exist; a page needing + * something of its own — a DOM to render into, a report to publish — composes + * its handlers on top of this map rather than finding them in it. + * + * The module has no Node dependency of any kind, so a page links it as an + * ordinary ES module with no bundling or transpilation. + * + * @module + * + * @import { Effect, ToAsyncOperationMap } from '../types.ts' + * @import { Result } from '../../types/result/types.ts' + * @import { CommonOp, Module, SandboxResult } from '../common/types.ts' + * @import { IoResult } from '../common/types.ts' + */ + +import { toIoError } from '../common/module.f.mjs' +import { error, ok } from '../../types/result/module.f.mjs' +import { asyncTryCatch } from '../../types/result/module.mjs' +import { toVec } from '../../types/uint8array/module.f.mjs' + +/** + * An effect runner over the operations this map is spread into. `all` runs its + * children through it rather than through a runner of its own, so an effect + * nested inside `all` reaches every handler the caller composed — not just the + * common ones. + * + * @typedef {(effect: Effect) => Promise>} CommonRun + */ + +/** + * Links a module in the page's realm. Injected so a caller can report loading + * progress, resolve a specifier against an application root, or drive the + * runner from a proof without a network; the default is the realm's own + * dynamic `import`. + * + * @typedef {(source: string) => Promise} BrowserImporter + */ + +/** + * Performs host IO, reporting a thrown failure as an {@link IoResult} error. + * + * The browser twin of the Node runner's `io`: the one place where an exception + * becomes ordinary effect data, normalized so nothing past it sees the thrown + * object. + * + * @template T + * @param {() => Promise} f + * @returns {Promise>} + */ +const io = async f => { + const r = await asyncTryCatch(f) + return r[0] === 'ok' ? r : error(toIoError(r[1])) +} + +/** + * Runs `f` and measures it, exactly as the Node runner does: a genuine + * `Promise` is awaited and a rejection is caught, and any other value — a proof + * tree carrying a `then` property included — is the result as it stands. + * + * That equality is the point. `fjs t` and this runner walk the same proof trees + * through the same shared semantics (`fjs/emergent_testing/module.f.mjs`), so + * the one operation that actually *executes* a proof body has to agree with its + * Node counterpart or the two runners disagree about what a suite means. + * + * @template T + * @param {() => T} f + * @returns {Promise>} + */ +const sandbox = async f => { + /** @type {Result} */ + let result + let after + const before = performance.now() + try { + let p = f() + after = performance.now() + if (p instanceof Promise) { + p = await p + after = performance.now() + } + result = ok(p) + } catch (e) { + after = performance.now() + result = error(e) + } + return { result, duration: after - before } +} + +/** + * The browser's handlers for the host-independent operations. + * + * `run` is the composed runner the caller builds — the one that also knows the + * caller's own operations — so `all` schedules its children through it. Passing + * it in rather than closing over a runner defined here is what keeps this map + * composable: a page adds handlers, and the effects nested inside `all` still + * reach them. + * + * @type {(run: CommonRun, importer?: BrowserImporter) => ToAsyncOperationMap} + */ +export const browserOperationMap = (run, importer = source => import(source)) => ({ + all: async (...effects) => ok(await Promise.all(effects.map(e => run(e)))), + await: async p => ok([p instanceof Promise ? await p : p]), + fetch: url => io(async () => { + const response = await globalThis.fetch(url) + if (!response.ok) { + throw new Error(`Fetch error: ${response.status} ${response.statusText}`) + } + return toVec(new Uint8Array(await response.arrayBuffer())) + }), + // A synchronous throw from the importer — a specifier the realm rejects + // before it ever starts loading — is a load failure like any other, so it + // is caught here rather than escaping the effect it belongs to. + import: path => io(async () => importer(path)), + now: async () => ok(Date.now()), + sandbox: async f => ok(await sandbox(f)), +}) diff --git a/fjs/effects/common/module.f.mjs b/fjs/effects/common/module.f.mjs new file mode 100644 index 000000000..a513e58db --- /dev/null +++ b/fjs/effects/common/module.f.mjs @@ -0,0 +1,217 @@ +/** + * The operations no host owns, and the helpers that read their error channel. + * + * `all` / `allOk` / `both` (concurrency), `await` (promise resolution), + * `fetch`, `import_`, `now` and `sandbox` each describe something a JavaScript + * realm can do on its own, so every runner implements them the same way: the + * Node runner in [`../node/module.mjs`](../node/module.mjs), the browser runner + * in [`../browser/module.mjs`](../browser/module.mjs), and the virtual one in + * [`../node/virtual/module.f.mjs`](../node/virtual/module.f.mjs). + * + * They lived in `../node/module.f.mjs`, which re-exports every name below so an + * existing importer keeps naming one module. What is genuinely Node's — the + * filesystem, streams, subprocesses, HTTP, an external test framework — stayed + * there. + * + * See [`./types.ts`](./types.ts) for the type-level API. + * + * @module + * + * @import { Effect, Func, NotImplemented, Operation } from '../types.ts' + * @import { Result } from '../../types/result/types.ts' + * @import { All, Await, Fetch, Import, IoChannel, IoError, IoErrorInfo, Now, Sandbox } from './types.ts' + */ + +import { do_, mapStep, pure, step } from '../module.f.mjs' +import { ok as resultOk, unwrap } from '../../types/result/module.f.mjs' + +/** + * Builds a normalized host error. The constructor exists so the shape is + * written once: every runner reports its failures through it, and a consumer + * matching on `'ioError'` knows what the payload holds. + * + * @type {(info: IoErrorInfo) => IoError} + */ +export const ioError = info => ['ioError', info] + +/** + * Normalizes a **thrown** value into an {@link IoError}: the OS error code when + * the host attached a string one, and a message that is the `Error`'s own or + * the value's string form. + * + * This is the boundary where an impure runner's `catch` becomes ordinary effect + * data. Nothing past it sees the thrown object, which is the point — a stack, a + * `cause`, and arbitrary own properties do not survive a wire hop, and a + * program that branched on them would be reading the host's implementation + * rather than the operation's contract. + * + * @type {(e: unknown) => IoError} + */ +export const toIoError = e => { + const message = e instanceof Error ? e.message : String(e) + if (typeof e !== 'object' || e === null || !('code' in e) || typeof e.code !== 'string') { + return ioError({ message }) + } + return ioError({ code: e.code, message }) +} + +/** + * True if `e` is a "file or directory does not exist" (`ENOENT`) error. + * + * Node's filesystem rejections are `Error`s carrying `code: 'ENOENT'`, which + * {@link toIoError} keeps; the virtual interpreter reports the same code for + * absent paths. Lets callers swallow only the missing-path case (e.g. a fresh + * store) while propagating genuine failures (permissions, corruption) rather + * than masking them. + * + * A {@link NotImplemented} is never "not found": a runner that cannot perform + * the operation has not looked for the path at all, so the two must not + * collapse into one benign branch — which is exactly what a bare `unknown` + * error channel used to allow. + * + * @type {(e: IoChannel) => boolean} + */ +export const isNotFound = ([tag, payload]) => + tag === 'ioError' && payload.code === 'ENOENT' + +/** + * Renders a channel error as a human line: an {@link IoError}'s own message, or + * the command name a runner could not dispatch. + * + * @type {(e: IoChannel) => string} + */ +export const errorMessage = ([tag, payload]) => + tag === 'notImplemented' ? `operation not implemented: ${payload}` : payload.message + +/** + * Renders a channel error for a **remote** caller: the command name for a + * {@link NotImplemented}, the OS error code for an `IoError`, and nothing else. + * + * {@link errorMessage} is for the operator of the program, who is entitled to + * the host's own words — including the path that failed. A protocol client is + * not, and the difference is not stylistic: `payload.message` is where the + * host puts the absolute path it could not read, so answering an MCP tool call + * with it publishes the server's filesystem layout to whoever is on the other + * end. The code (`ENOENT`, `EACCES`) says *what* went wrong without saying + * *where*, which is the part a client can act on anyway. + * + * A host that attached no code leaves nothing safe to forward, so the answer is + * the bare kind. That is deliberate: guessing which part of a free-text message + * is path-free is exactly the mistake this exists to prevent. + * + * @type {(e: IoChannel) => string} + */ +export const errorSummary = ([tag, payload]) => + tag === 'notImplemented' + ? `operation not implemented: ${payload}` + : payload.code === undefined ? 'io error' : `io error: ${payload.code}` + +// all + +/** + * To run the operation `O` should be known by the runner/engine. + * This is the reason why we merge `O` with `All` in the resulting effect. + */ +export const all = + // `Func` cannot express a variadic generic operation, so the declared type + // is written out here and `do_`'s is set aside. + /** @type {(...a: readonly Effect[]) => Effect[], NotImplemented>} */ + (/** @type {unknown} */ (do_('all'))) + +/** + * Collapses a list of results into a result of the list, keeping the **first** + * error in list order and discarding the later ones. + * + * Keeping one is what makes this a `Result` rather than a report: the callers + * that need it are chains, and a chain has one error channel. A site that wants + * every failure wants a different return type and should not reach for this. + * + * @type {(list: readonly Result[]) => Result} + */ +const okList = list => { + for (const r of list) { + if (r[0] === 'error') { return r } + } + return resultOk(list.map(unwrap)) +} + +/** + * {@link all} in the `ok` channel: collects the values when every effect + * succeeded, and answers with the first failure otherwise. + * + * `all` alone cannot serve a fallible chain. Its envelope is the runner's + * (`OpResult`, saying whether the *operation* could be dispatched), so handing + * it `Effect`s nests one `Result` inside another and the caller receives + * `readonly Result[]`. That has to be collapsed before the chain can + * `step` again, and a continuation that forgets to is the value-discarding + * hazard this migration exists to remove — one level in, where it is harder to + * see. + * + * **Every effect still runs.** The short-circuit is in the *result*, not in the + * execution: `all` performs them concurrently and this reads the answers once + * they are all in, so a failure does not cancel its siblings the way it stops + * the sequential `forEachStep` in `../module.f.mjs`. The error channel + * unions the runner's + * `NotImplemented` with the effects' own `E` for the same reason every other + * step does — either can be what went wrong. + * + * @type {(...a: readonly Effect[]) => Effect} + */ +export const allOk = (...a) => + step(all(...a), rs => pure(okList(rs))) + +/** + * @template {Operation} O0 + * @template T0 + * @template E0 + * @param {Effect} a + * @returns {(b: Effect) => Effect, Result], NotImplemented>} + */ +export const both = a => b => + /** @type {any} */ (all)(a, b) + +// fetch + +/** @type {Func} */ +export const fetch = do_('fetch') + +// import + +/** @type {Func} */ +export const import_ = do_('import') + +// now + +/** @type {Func} */ +export const now = do_('now') + +// sandbox + +/** + * Runs a plain synchronous function in an isolated, measured environment. + * + * Combines try/catch and high-resolution timing into a single atomic operation. + * Only plain synchronous functions are accepted — no effects, no promises. + * + * Using a single operation rather than separate `TryCatch` + `Perf` effects is + * necessary for correctness: effects execute as async tasks, so the scheduler + * can insert arbitrary work between two separate timing calls, making the + * measured delta inaccurate. Here the clock reads happen synchronously around + * the function call with nothing in between. + * + * Future parameters (time limit, memory limit) can be added to the payload + * without breaking the API. Worker-based implementations can enforce hard + * limits via worker termination. + * + * @see {@link SandboxResult} + * + * @type {Func} + */ +export const sandbox = do_('sandbox') + +/** @type {Func} */ +const awaitPromise = do_('await') + +/** @type {(p: unknown) => Effect} */ +export const awaitIfPromise = p => + mapStep(awaitPromise(p), ([x]) => x) diff --git a/fjs/effects/common/proof.f.mjs b/fjs/effects/common/proof.f.mjs new file mode 100644 index 000000000..1ea0435a9 --- /dev/null +++ b/fjs/effects/common/proof.f.mjs @@ -0,0 +1,168 @@ +/** + * Proofs for the host-independent operations and the helpers that read their + * error channel. + * + * The operations are proved against a stand-in interpreter declared here rather + * than against a host runner: what this module owns is the *constructors* and + * the `ok`-channel collapse, and a proof that reached for `../node/virtual` + * would be reading a Node runner's answers to decide whether `all` builds the + * right node. Each host runner proves its own handlers — `../node/proof.f.mjs` + * for the virtual and Node ones, `../../emergent_testing/browser/proof.mjs` for + * the browser one. + * + * @import { Effect } from '../types.ts' + * @import { Result } from '../../types/result/types.ts' + * @import { MemOperationMap, RunInstance } from '../mock/types.ts' + * @import { CommonOp, SandboxResult } from './types.ts' + */ + +import { assert, assertEq } from '../../asserts/module.f.mjs' +import { + all, allOk, awaitIfPromise, both, errorMessage, errorSummary, fetch, import_, + ioError, isNotFound, now, sandbox, toIoError, +} from './module.f.mjs' +import { run as mockRun } from '../mock/module.f.mjs' +import { error, ok, unwrap } from '../../types/result/module.f.mjs' +import { vec8 } from '../../types/bit_vec/module.f.mjs' + +/** The one number the stand-in clock ever answers. */ +const fixedNow = 1_700_000_000 + +/** @type {MemOperationMap} */ +const map = { + all: (...a) => state => [state, ok(a.map(i => common(state)(i)[1]))], + await: p => state => [state, ok([p])], + fetch: url => state => [ + state, + url === 'ok' ? ok(vec8(0x2An)) : error(ioError({ message: `cannot fetch ${url}` })), + ], + import: source => state => [ + state, + source === 'ok' ? ok({ value: 1 }) : error(ioError({ code: 'ENOENT', message: source })), + ], + now: () => state => [state, ok(fixedNow)], + // The same pass-through the virtual Node runner uses: a fixture returns the + // `SandboxResult` it wants reported, so an outcome is dictated rather than + // measured. + sandbox: f => state => [state, ok(/** @type {SandboxResult} */ (f()))], +} + +/** @type {RunInstance} */ +const common = mockRun(map) + +/** @type {(e: Effect) => Result} */ +const run = e => common(null)(e)[1] + +export const proof = { + // The one boundary where a runner's `catch` becomes effect data: whatever + // was thrown is reduced to a code (when the host attached a string one) + // and a message. + toIoError: { + error: () => { + assertEq(toIoError(new Error('boom'))[1].message, 'boom') + }, + withCode: () => { + const [, info] = toIoError(Object.assign(new Error('missing'), { code: 'ENOENT' })) + assertEq(info.code, 'ENOENT') + assertEq(info.message, 'missing') + }, + // A thrown non-`Error` still normalizes: the value's string form is the + // message, and there is no code to carry. + string: () => { + const [, info] = toIoError('plain') + assertEq(info.code, undefined) + assertEq(info.message, 'plain') + }, + null: () => { + assertEq(toIoError(null)[1].message, 'null') + }, + // An object whose `code` is not a string is not an OS error code, so it + // is dropped rather than carried as one. + nonStringCode: () => { + assertEq(toIoError({ code: 42 })[1].code, undefined) + }, + noCode: () => { + assertEq(toIoError({})[1].code, undefined) + }, + }, + isNotFound: { + enoent: () => { + assert(isNotFound(ioError({ code: 'ENOENT', message: 'no such file or directory' }))) + }, + otherCode: () => { + assert(!isNotFound(ioError({ code: 'EACCES', message: 'permission denied' }))) + }, + // A runner that cannot perform the operation has not looked for the + // path at all, so a missing handler is never "not found". + notImplemented: () => { + assert(!isNotFound(['notImplemented', 'readFile'])) + }, + }, + errorMessage: { + io: () => { + assertEq(errorMessage(ioError({ message: 'disk full' })), 'disk full') + }, + notImplemented: () => { + assertEq(errorMessage(['notImplemented', 'readFile']), 'operation not implemented: readFile') + }, + }, + errorSummary: { + // The distinction that matters: `errorMessage` hands back the host's + // words, which is where the path lives; `errorSummary` never does. + io: () => { + assertEq(errorSummary(ioError({ code: 'ENOENT', message: "no such file or directory, scandir '/home/u/.cas'" })), 'io error: ENOENT') + }, + ioWithoutCode: () => { + assertEq(errorSummary(ioError({ message: "cannot read '/home/u/.cas'" })), 'io error') + }, + notImplemented: () => { + assertEq(errorSummary(['notImplemented', 'readdir']), 'operation not implemented: readdir') + }, + }, + // `all` answers each effect's whole `Result`: its own envelope says only + // whether the operation could be dispatched. + all: () => { + const r = unwrap(run(all(fetch('ok'), fetch('no')))) + assertEq(r.length, 2) + assertEq(r[0]?.[0], 'ok') + assertEq(r[1]?.[0], 'error') + }, + allOk: { + // The collapse a fallible chain wants: values when every effect + // succeeded... + collects: () => { + assertEq(unwrap(run(allOk(now(), now()))).join(','), `${fixedNow},${fixedNow}`) + }, + // ...and the first failure in list order otherwise. + firstError: () => { + const r = run(allOk(fetch('no'), fetch('worse'))) + assert(r[0] === 'error', r) + assertEq(errorMessage(r[1]), 'cannot fetch no') + }, + }, + both: () => { + const [a, b] = unwrap(run(both(now())(import_('ok')))) + assertEq(unwrap(a ?? error(0)), fixedNow) + assertEq(unwrap(b ?? error(0)).value, 1) + }, + import: { + linked: () => { + assertEq(unwrap(run(import_('ok'))).value, 1) + }, + missing: () => { + const r = run(import_('nope')) + assert(r[0] === 'error', r) + assert(isNotFound(r[1]), r[1]) + }, + }, + sandbox: () => { + const { result, duration } = unwrap(run(sandbox(() => ({ result: ok(7), duration: 3 })))) + assertEq(unwrap(result), 7) + assertEq(duration, 3) + }, + // A promise is the runner's business, so what the constructor owns is + // unwrapping the one-element tuple the operation answers with. + awaitIfPromise: () => { + assertEq(unwrap(run(awaitIfPromise(5))), 5) + }, +} diff --git a/fjs/effects/common/types.ts b/fjs/effects/common/types.ts new file mode 100644 index 000000000..e4acbc550 --- /dev/null +++ b/fjs/effects/common/types.ts @@ -0,0 +1,143 @@ +/** + * Types for the operations no host owns. + * + * Every operation declared here describes something a JavaScript realm can do + * on its own — hold a value, wait for a promise, measure a call, link a module, + * fetch a URL — so a Node runner, a browser runner, and the virtual runner can + * each implement the same command with the same contract. What is genuinely + * Node's — streams, the filesystem, subprocesses, an external test framework — + * stays in [`../node/types.ts`](../node/types.ts), which re-exports these so an + * existing importer keeps naming one module. + * + * @module + */ + +import type { Vec } from '../../types/bit_vec/types.ts' +import type { Effect, NotImplemented } from '../types.ts' +import type { Result } from '../../types/result/types.ts' +import type { StringMap } from '../../types/object/types.ts' + +/** + * A host failure, normalized: whatever the runtime threw reduced to a + * serializable record. `code` is the OS error code when the host supplied one + * (`'ENOENT'`, `'EEXIST'`), absent otherwise. + * + * It is a tagged tuple for the same reason {@link NotImplemented} is — the two + * share an error channel, and the tag is what tells them apart. That + * distinction is the whole reason this type exists: with a bare `unknown` + * error, `NotImplemented | unknown` collapses to `unknown` and a program can no + * longer tell "this runner cannot do it" from "the host tried and failed". + * + * Normalizing also keeps the channel serializable. A thrown `Error` carries a + * stack, a `cause`, and arbitrary own properties; none of it survives a wire + * hop, and a runner in another process could not reproduce it. + */ +export type IoError = readonly['ioError', IoErrorInfo] + +export type IoErrorInfo = { + readonly code?: string + readonly message: string +} + +/** + * The result of an operation with no failures of its own: it either produces + * its value or reports that the runner does not implement it. + * + * Every operation's return type is a `Result`, including the ones that cannot + * fail on their own terms — an operation left on a raw contract would be a hole + * in the error channel, and a runner may omit a handler for any of them. + */ +export type OpResult = Result + +/** + * The error channel of anything that performs host IO: a normalized host + * failure, or the report that the runner does not implement the operation. + * + * It is one name rather than a union spelled at each site, and that is a + * migration property rather than brevity. An effect that does no IO *yet* is + * one added `readFile` away from doing some, and if each signature names its + * own errors, that one change walks up every enclosing signature — the failure + * mode that sank `throws` clauses elsewhere, where engineers eventually + * declared everything throwing rather than maintain the cascade. Declaring the + * standard channel once is that concession made deliberately: an IO-touching + * effect says it fails *the way node IO fails*, and gaining a new way to do so + * changes nothing above it. + * + * It is not a licence to widen. An operation with failures of its own extends + * the channel (`IoChannel | ParseError`), and a computation whose errors are + * genuinely narrower should say so — this is the default for IO, not a ceiling. + */ +export type IoChannel = NotImplemented | IoError + +/** + * The result of an operation that performs host IO: its value, a normalized + * host failure, or the missing-handler report. + */ +export type IoResult = Result + +// all + +/** + * Runs its effects concurrently and answers each one's whole `Result`. + * + * The nesting is deliberate and belongs to the runner: this envelope says + * whether `all` itself could be dispatched, and each inner `Result` is what + * that effect answered. `allOk` (`./module.f.mjs`) is the collapse a fallible + * chain wants. + */ +export type All = ['all', (...effects: Effect[]) => OpResult[]>] + +// fetch + +export type Fetch = ['fetch', (url: string) => IoResult] + +// import + +export type Module = StringMap + +export type Import = ['import', (path: string) => IoResult] + +// now + +export type Now = readonly['now', () => OpResult] + +// sandbox + +/** + * The outcome of a `Sandbox` operation. + * + * `result` carries either `['ok', value]` or `['error', thrown]`. `duration` + * is a floating-point millisecond count with up to microsecond precision, + * matching `performance.now()` directly. Additional fields (allocated memory, + * max stack depth, coverage) may be added in future without breaking consumers. + */ +export type SandboxResult = { + readonly result: Result + /** + * Elapsed time in milliseconds (microsecond precision via `performance.now()`). + * The virtual runner returns `0` for deterministic tests. + */ + readonly duration: number +} + +export type Sandbox = readonly['sandbox', (f: () => T) => OpResult>] + +/** + * Resolves the return value of a test function inside the effect runner. + * If `p` is a real `Promise`, it is awaited and rejections propagate as + * throws. If `p` is any other value it is returned as-is. Plain thenables + * (objects with a `.then` method that are not `instanceof Promise`) are + * treated as ordinary values — not awaited. See `fjs/dev/tf/README.md`. + */ +export type Await = readonly['await', (p: unknown) => OpResult] + +/** + * The operations every runner is expected to be able to implement. + * + * A host runner's operation set is this union plus whatever its host adds: + * `NodeOp` is `CommonOp | MemOp | Fs | Http | …`, and the browser interpreter + * in [`../browser/module.mjs`](../browser/module.mjs) implements exactly this + * set against the browser realm. Naming it once is what lets a program say it + * needs nothing host-specific, and be run by either. + */ +export type CommonOp = All | Await | Fetch | Import | Now | Sandbox diff --git a/fjs/effects/memory/types.ts b/fjs/effects/memory/types.ts index 844dbb80c..cc72052a8 100644 --- a/fjs/effects/memory/types.ts +++ b/fjs/effects/memory/types.ts @@ -6,7 +6,7 @@ import type { Phantom } from '../../types/phantom/types.ts' import type { Nominal } from '../../types/nominal/types.ts' -import type { OpResult } from '../node/types.ts' +import type { OpResult } from '../common/types.ts' /** Nominal brand version for memory keys. */ export type _MemKeyHash = '3f114fa6036a8da026b827f0c3e6d901f5e81ad9a320e431ccce31451892d286' diff --git a/fjs/effects/node/module.f.mjs b/fjs/effects/node/module.f.mjs index 5d1c6dddd..50344ea11 100644 --- a/fjs/effects/node/module.f.mjs +++ b/fjs/effects/node/module.f.mjs @@ -1,10 +1,14 @@ /** * Node.js effect operations: filesystem (`mkdir`, `readFile`, `readdir`, * `writeFile`, `rm`, `access`, plus the `readUtf8File`/`writeUtf8File` text - * helpers), networking (`fetch`, `createServer`, `listen`), - * subprocess `exec`, `log`/`error` (wrappers over `write`), `import_`, `now`, - * `sandbox`, `forever`, and `all`/`both` parallelism; defines the - * `NodeOp`/`NodeProgram` types used by the Node runner. + * helpers), HTTP (`createServer`, `listen`), subprocess `exec`, `log`/`error` + * (wrappers over `write`), `read`/`readLine`, `randomInt` and `forever`; defines + * the `NodeOp`/`NodeProgram` types used by the Node runner. + * + * The operations no host owns — `all`/`allOk`/`both`, `await`, `fetch`, + * `import_`, `now`, `sandbox`, and the `IoError` helpers — moved to + * [`../common/module.f.mjs`](../common/module.f.mjs) so the browser runner can + * link them, and are re-exported here unchanged. * * See `./types.ts` for the type-level API. * @@ -14,7 +18,8 @@ * @import { Result } from '../../types/result/types.ts' * @import { Commands, CommandSet, Effect, Func, NotImplemented, Operation } from '../types.ts' * @import { List } from '../list/types.ts' - * @import { All, Access, Await, Console, CreateExclusive, CreateServer, Dirent, Engine, Env, Exec, ExecResult, Fetch, FileStat, Forever, Fs, Headers, Http, IncomingMessage, Import, IoChannel, IoError, IoErrorInfo, Listen, MakeDirectoryOptions, Mkdir, Module, Now, NodeOp, NodeProgramOptions, RandomInt, Read, ReadBytes, ReadConsoles, ReadFile, Readdir, ReaddirOptions, RequestListener, Rename, Rm, Sandbox, SandboxResult, Server, ServerResponse, Stat, Test, TestContext, TestFn, Write, WriteBytes, WriteConsoles, WriteFile, _UtfList, _WriteLoop } from './types.ts' + * @import { IoError } from '../common/types.ts' + * @import { All, Access, Console, CreateExclusive, CreateServer, Dirent, Engine, Env, Exec, ExecResult, FileStat, Forever, Fs, Headers, Http, IncomingMessage, IoChannel, Listen, MakeDirectoryOptions, Mkdir, NodeOp, NodeProgramOptions, RandomInt, Read, ReadBytes, ReadConsoles, ReadFile, Readdir, ReaddirOptions, RequestListener, Rename, Rm, SandboxResult, Server, ServerResponse, Stat, Test, TestContext, TestFn, Write, WriteBytes, WriteConsoles, WriteFile, _UtfList, _WriteLoop } from './types.ts' */ import { utf8, utf8ToString } from '../../text/module.f.mjs' @@ -22,20 +27,23 @@ import { toCodePointList } from '../../text/utf8/module.f.mjs' import { codePointListToString } from '../../text/utf16/module.f.mjs' import { reverse } from '../../types/list/module.f.mjs' import { length } from '../../types/bit_vec/module.f.mjs' -import { error as resultError, ok as resultOk, unwrap } from '../../types/result/module.f.mjs' -import { do_, pure } from '../module.f.mjs' +import { error as resultError } from '../../types/result/module.f.mjs' +import { do_ } from '../module.f.mjs' import { mapStep as ioMapStep, pureError, pureOk, resultMapStep, resultStep, step as ioStep, } from '../module.f.mjs' +import { errorMessage, ioError } from '../common/module.f.mjs' /** - * Builds a normalized host error. The constructor exists so the shape is - * written once: every runner reports its failures through it, and a consumer - * matching on `'ioError'` knows what the payload holds. - * - * @type {(info: IoErrorInfo) => IoError} + * The host-independent operations, re-exported so a caller that already names + * this module for `readFile` keeps naming it for `sandbox` and `all` too. They + * are defined in [`../common/module.f.mjs`](../common/module.f.mjs), which the + * browser runner links without reaching a Node type. */ -export const ioError = info => ['ioError', info] +export { + all, allOk, awaitIfPromise, both, errorMessage, errorSummary, fetch, import_, + ioError, isNotFound, now, sandbox, toIoError, +} from '../common/module.f.mjs' /** * The host a {@link Listen} refuses. @@ -83,46 +91,6 @@ export const emptyHostError = ioError({ message: emptyHostMessage, }) -/** - * Normalizes a **thrown** value into an {@link IoError}: the OS error code when - * the host attached a string one, and a message that is the `Error`'s own or - * the value's string form. - * - * This is the boundary where an impure runner's `catch` becomes ordinary effect - * data. Nothing past it sees the thrown object, which is the point — a stack, a - * `cause`, and arbitrary own properties do not survive a wire hop, and a - * program that branched on them would be reading the host's implementation - * rather than the operation's contract. - * - * @type {(e: unknown) => IoError} - */ -export const toIoError = e => { - const message = e instanceof Error ? e.message : String(e) - if (typeof e !== 'object' || e === null || !('code' in e) || typeof e.code !== 'string') { - return ioError({ message }) - } - return ioError({ code: e.code, message }) -} - -/** - * True if `e` is a "file or directory does not exist" (`ENOENT`) error. - * - * Node's filesystem rejections are `Error`s carrying `code: 'ENOENT'`, which - * {@link toIoError} keeps; the virtual interpreter reports the same code for - * absent paths. Lets callers swallow only the missing-path case (e.g. a fresh - * store) while propagating genuine failures (permissions, corruption) rather - * than masking them. - * - * A {@link NotImplemented} is never "not found": a runner that cannot perform - * the operation has not looked for the path at all, so the two must not - * collapse into one benign branch — which is exactly what a bare `unknown` - * error channel used to allow. - * - * @type {(e: IoChannel) => boolean} - */ -export const isNotFound = ([tag, payload]) => - tag === 'ioError' && payload.code === 'ENOENT' - /** * `NodeOp`'s commands as data, so a runner that implements only part of them * can still tell an operation it lacks from a `Do` node whose `command` was @@ -155,75 +123,6 @@ const nodeCommandSet = { */ export const nodeCommands = /** @type {Commands} */ (Object.keys(nodeCommandSet)) -// all - -/** - * To run the operation `O` should be known by the runner/engine. - * This is the reason why we merge `O` with `All` in the resulting effect. - */ -export const all = - // `Func` cannot express a variadic generic operation, so the declared type - // is written out here and `do_`'s is set aside. - /** @type {(...a: readonly Effect[]) => Effect[], NotImplemented>} */ - (/** @type {unknown} */ (do_('all'))) - -/** - * Collapses a list of results into a result of the list, keeping the **first** - * error in list order and discarding the later ones. - * - * Keeping one is what makes this a `Result` rather than a report: the callers - * that need it are chains, and a chain has one error channel. A site that wants - * every failure wants a different return type and should not reach for this. - * - * @type {(list: readonly Result[]) => Result} - */ -const okList = list => { - for (const r of list) { - if (r[0] === 'error') { return r } - } - return resultOk(list.map(unwrap)) -} - -/** - * {@link all} in the `ok` channel: collects the values when every effect - * succeeded, and answers with the first failure otherwise. - * - * `all` alone cannot serve a fallible chain. Its envelope is the runner's - * (`OpResult`, saying whether the *operation* could be dispatched), so handing - * it `Effect`s nests one `Result` inside another and the caller receives - * `readonly Result[]`. That has to be collapsed before the chain can - * `step` again, and a continuation that forgets to is the value-discarding - * hazard this migration exists to remove — one level in, where it is harder to - * see. - * - * **Every effect still runs.** The short-circuit is in the *result*, not in the - * execution: `all` performs them concurrently and this reads the answers once - * they are all in, so a failure does not cancel its siblings the way it stops - * the sequential `forEachStep` in `./module.f.mjs`. The error channel - * unions the runner's - * `NotImplemented` with the effects' own `E` for the same reason every other - * step does — either can be what went wrong. - * - * @type {(...a: readonly Effect[]) => Effect} - */ -export const allOk = (...a) => - ioStep(all(...a), rs => pure(okList(rs))) - -/** - * @template {Operation} O0 - * @template T0 - * @template E0 - * @param {Effect} a - * @returns {(b: Effect) => Effect, Result], NotImplemented>} - */ -export const both = a => b => - /** @type {any} */ (all)(a, b) - -// fetch - -/** @type {Func} */ -export const fetch = do_('fetch') - // mkdir /** @type {Func} */ @@ -356,11 +255,6 @@ export const listen = do_('listen') /** @type {Func} */ export const forever = do_('forever') -// import - -/** @type {Func} */ -export const import_ = do_('import') - // write /** Emits a `Write` effect to the given named stream. */ @@ -430,42 +324,6 @@ export const readLine = stream => { return loop(null) } -// now - -/** @type {Func} */ -export const now = do_('now') - -// sandbox - -/** - * Runs a plain synchronous function in an isolated, measured environment. - * - * Combines try/catch and high-resolution timing into a single atomic operation. - * Only plain synchronous functions are accepted — no effects, no promises. - * - * Using a single operation rather than separate `TryCatch` + `Perf` effects is - * necessary for correctness: effects execute as async tasks, so the scheduler - * can insert arbitrary work between two separate timing calls, making the - * measured delta inaccurate. Here the clock reads happen synchronously around - * the function call with nothing in between. - * - * Future parameters (time limit, memory limit) can be added to the payload - * without breaking the API. Worker-based implementations can enforce hard - * limits via worker termination. - * - * @see {@link SandboxResult} - * - * @type {Func} - */ -export const sandbox = do_('sandbox') - -/** @type {Func} */ -const awaitPromise = do_('await') - -/** @type {(p: unknown) => Effect} */ -export const awaitIfPromise = p => - ioMapStep(awaitPromise(p), ([x]) => x) - // Test registration /** @type {Func} */ @@ -512,38 +370,6 @@ export const errorExit = s => */ export const exitCode = ([, code]) => code -/** - * Renders a channel error as a human line: an {@link IoError}'s own message, or - * the command name a runner could not dispatch. - * - * @type {(e: IoChannel) => string} - */ -export const errorMessage = ([tag, payload]) => - tag === 'notImplemented' ? `operation not implemented: ${payload}` : payload.message - -/** - * Renders a channel error for a **remote** caller: the command name for a - * {@link NotImplemented}, the OS error code for an `IoError`, and nothing else. - * - * {@link errorMessage} is for the operator of the program, who is entitled to - * the host's own words — including the path that failed. A protocol client is - * not, and the difference is not stylistic: `payload.message` is where the - * host puts the absolute path it could not read, so answering an MCP tool call - * with it publishes the server's filesystem layout to whoever is on the other - * end. The code (`ENOENT`, `EACCES`) says *what* went wrong without saying - * *where*, which is the part a client can act on anyway. - * - * A host that attached no code leaves nothing safe to forward, so the answer is - * the bare kind. That is deliberate: guessing which part of a free-text message - * is path-free is exactly the mistake this exists to prevent. - * - * @type {(e: IoChannel) => string} - */ -export const errorSummary = ([tag, payload]) => - tag === 'notImplemented' - ? `operation not implemented: ${payload}` - : payload.code === undefined ? 'io error' : `io error: ${payload.code}` - /** * Ends a program with an exit code that reflects `e`: `ok` yields `0`, and a * failure is reported on `stderr` and yields `1` ({@link errorExit}). diff --git a/fjs/effects/node/proof.f.mjs b/fjs/effects/node/proof.f.mjs index e04035da3..6c31b7f19 100644 --- a/fjs/effects/node/proof.f.mjs +++ b/fjs/effects/node/proof.f.mjs @@ -10,7 +10,7 @@ import { empty, isVec, uint, vec, vec8 } from "../../types/bit_vec/module.f.mjs" import { utf8, utf8ToString } from "../../text/module.f.mjs" import { match } from "../module.f.mjs" import { mapStep, step as ioStep } from "../module.f.mjs" -import { both, errorMessage, errorSummary, exitStep, fetch, ioError, isNotFound, mkdir, now, readdir, readFile, readUtf8File, rm, sandbox, toIoError, writeFile, writeUtf8File, rename, readBytes, randomInt, writeFromStream, usesInlineTestContext, versionLessThan } from "./module.f.mjs" +import { both, exitStep, fetch, mkdir, now, readdir, readFile, readUtf8File, rm, sandbox, writeFile, writeUtf8File, rename, readBytes, randomInt, writeFromStream, usesInlineTestContext, versionLessThan } from "./module.f.mjs" import { create as memCreate, read as memRead, write as memWrite } from "../memory/module.f.mjs" import { empty as listEmpty, nonEmpty as listNonEmpty } from "../list/module.f.mjs" import { emptyState, virtual } from "./virtual/module.f.mjs" @@ -50,77 +50,8 @@ const assertOk = (r, expected) => { } export const proof = { - // The one boundary where a runner's `catch` becomes effect data: whatever - // was thrown is reduced to a code (when the host attached a string one) - // and a message. - toIoError: { - error: () => { - assertIoMessage(toIoError(new Error('boom')), 'boom') - }, - withCode: () => { - const e = toIoError(Object.assign(new Error('missing'), { code: 'ENOENT' })) - assert(e[0] === 'ioError', e) - assertEq(e[1].code, 'ENOENT', e) - assertEq(e[1].message, 'missing', e) - }, - // A thrown non-`Error` still normalizes: the value's string form is the - // message, and there is no code to carry. - string: () => { - const e = toIoError('plain') - assert(e[0] === 'ioError', e) - assertEq(e[1].code, undefined, e) - assertEq(e[1].message, 'plain', e) - }, - null: () => { - assertIoMessage(toIoError(null), 'null') - }, - // An object whose `code` is not a string is not an OS error code, so it - // is dropped rather than carried as one. - nonStringCode: () => { - const e = toIoError({ code: 42 }) - assert(e[0] === 'ioError', e) - assertEq(e[1].code, undefined, e) - }, - noCode: () => { - const e = toIoError({}) - assert(e[0] === 'ioError', e) - assertEq(e[1].code, undefined, e) - }, - }, - isNotFound: { - enoent: () => { - assert(isNotFound(ioError({ code: 'ENOENT', message: 'no such file or directory' }))) - }, - otherCode: () => { - assert(!isNotFound(ioError({ code: 'EACCES', message: 'permission denied' }))) - }, - // A runner that cannot perform the operation has not looked for the - // path at all, so a missing handler is never "not found". - notImplemented: () => { - assert(!isNotFound(['notImplemented', 'readFile'])) - }, - }, - errorMessage: { - io: () => { - assertEq(errorMessage(ioError({ message: 'disk full' })), 'disk full') - }, - notImplemented: () => { - assertEq(errorMessage(['notImplemented', 'readFile']), 'operation not implemented: readFile') - }, - }, - errorSummary: { - // The distinction that matters: `errorMessage` hands back the host's - // words, which is where the path lives; `errorSummary` never does. - io: () => { - assertEq(errorSummary(ioError({ code: 'ENOENT', message: "no such file or directory, scandir '/home/u/.cas'" })), 'io error: ENOENT') - }, - ioWithoutCode: () => { - assertEq(errorSummary(ioError({ message: "cannot read '/home/u/.cas'" })), 'io error') - }, - notImplemented: () => { - assertEq(errorSummary(['notImplemented', 'readdir']), 'operation not implemented: readdir') - }, - }, + // `toIoError`, `isNotFound`, `errorMessage` and `errorSummary` are proved + // in `../common/proof.f.mjs`, beside the module that now defines them. exitStep: { // The exit-code policy a `NodeProgram` ends with: success is `0`... ok: () => { diff --git a/fjs/effects/node/types.ts b/fjs/effects/node/types.ts index 886a045f3..459c745d8 100644 --- a/fjs/effects/node/types.ts +++ b/fjs/effects/node/types.ts @@ -6,86 +6,23 @@ import type { List as EffectList } from '../../types/list/types.ts' import type { Vec } from '../../types/bit_vec/types.ts' +import type { All, Await, CommonOp, IoChannel, IoResult, OpResult } from '../common/types.ts' import type { MemOp } from '../memory/types.ts' import type { Nominal } from '../../types/nominal/types.ts' -import type { Result } from '../../types/result/types.ts' import type { StringMap } from '../../types/object/types.ts' -import type { Effect, NotImplemented, Operation, ToAsyncOperationMap } from '../types.ts' +import type { Effect, Operation, ToAsyncOperationMap } from '../types.ts' import type { List } from '../list/types.ts' /** - * A host failure, normalized: whatever the runtime threw reduced to a - * serializable record. `code` is the OS error code when the host supplied one - * (`'ENOENT'`, `'EEXIST'`), absent otherwise. - * - * It is a tagged tuple for the same reason {@link NotImplemented} is — the two - * share an error channel, and the tag is what tells them apart. That - * distinction is the whole reason this type exists: with a bare `unknown` - * error, `NotImplemented | unknown` collapses to `unknown` and a program can no - * longer tell "this runner cannot do it" from "the host tried and failed". - * - * Normalizing also keeps the channel serializable. A thrown `Error` carries a - * stack, a `cause`, and arbitrary own properties; none of it survives a wire - * hop, and a runner in another process could not reproduce it. - */ -export type IoError = readonly['ioError', IoErrorInfo] - -export type IoErrorInfo = { - readonly code?: string - readonly message: string -} - -/** - * The result of an operation with no failures of its own: it either produces - * its value or reports that the runner does not implement it. - * - * Every operation's return type is a `Result`, including the ones that cannot - * fail on their own terms — an operation left on a raw contract would be a hole - * in the error channel, and a runner may omit a handler for any of them. - */ -export type OpResult = Result - -/** - * The error channel of anything that performs host IO: a normalized host - * failure, or the report that the runner does not implement the operation. - * - * It is one name rather than a union spelled at each site, and that is a - * migration property rather than brevity. An effect that does no IO *yet* is - * one added `readFile` away from doing some, and if each signature names its - * own errors, that one change walks up every enclosing signature — the failure - * mode that sank `throws` clauses elsewhere, where engineers eventually - * declared everything throwing rather than maintain the cascade. Declaring the - * standard channel once is that concession made deliberately: an IO-touching - * effect says it fails *the way node IO fails*, and gaining a new way to do so - * changes nothing above it. - * - * It is not a licence to widen. An operation with failures of its own extends - * the channel (`IoChannel | ParseError`), and a computation whose errors are - * genuinely narrower should say so — this is the default for IO, not a ceiling. - */ -export type IoChannel = NotImplemented | IoError - -/** - * The result of an operation that performs host IO: its value, a normalized - * host failure, or the missing-handler report. - */ -export type IoResult = Result - -// all - -/** - * Runs its effects concurrently and answers each one's whole `Result`. - * - * The nesting is deliberate and belongs to the runner: this envelope says - * whether `all` itself could be dispatched, and each inner `Result` is what - * that effect answered. `allOk` (`./module.f.mjs`) is the collapse a fallible - * chain wants. + * The operations no host owns, re-exported so a consumer that already names + * this module for `ReadFile` keeps naming it for `Sandbox` and `All` too. They + * are declared in [`../common/types.ts`](../common/types.ts), which the browser + * runner reads without reaching a Node type. */ -export type All = ['all', (...effects: Effect[]) => OpResult[]>] - -// fetch - -export type Fetch = ['fetch', (url: string) => IoResult] +export type { + All, Await, Fetch, Import, IoChannel, IoError, IoErrorInfo, IoResult, Module, + Now, OpResult, Sandbox, SandboxResult, +} from '../common/types.ts' // mkdir @@ -261,12 +198,6 @@ export type Http = CreateServer | Listen export type Forever = ['forever', () => OpResult] -// import - -export type Module = StringMap - -export type Import = ['import', (path: string) => IoResult] - // write /** Named output streams accepted by the `Write` effect. */ @@ -299,40 +230,6 @@ export type Read = readonly['read', (stream: ReadConsoles) => OpResult -// now - -export type Now = readonly['now', () => OpResult] - -// sandbox - -/** - * The outcome of a `Sandbox` operation. - * - * `result` carries either `['ok', value]` or `['error', thrown]`. `duration` - * is a floating-point millisecond count with up to microsecond precision, - * matching `performance.now()` directly. Additional fields (allocated memory, - * max stack depth, coverage) may be added in future without breaking consumers. - */ -export type SandboxResult = { - readonly result: Result - /** - * Elapsed time in milliseconds (microsecond precision via `performance.now()`). - * The virtual runner returns `0` for deterministic tests. - */ - readonly duration: number -} - -export type Sandbox = readonly['sandbox', (f: () => T) => OpResult>] - -/** - * Resolves the return value of a test function inside the effect runner. - * If `p` is a real `Promise`, it is awaited and rejections propagate as - * throws. If `p` is any other value it is returned as-is. Plain thenables - * (objects with a `.then` method that are not `instanceof Promise`) are - * treated as ordinary values — not awaited. See `fjs/dev/tf/README.md`. - */ -export type Await = readonly['await', (p: unknown) => OpResult] - // Test registration /** @@ -371,18 +268,13 @@ export type Test = export type NodeOp = | Access - | All - | Await - | Fetch + | CommonOp | Fs | Http | Forever - | Import | MemOp - | Now | RandomInt | Read - | Sandbox | Write | Test diff --git a/fjs/emergent_testing/README.md b/fjs/emergent_testing/README.md index 883600e00..703a1c99d 100644 --- a/fjs/emergent_testing/README.md +++ b/fjs/emergent_testing/README.md @@ -83,9 +83,47 @@ Then invoke the runner: - `bun test` - `deno test --allow-read --allow-env --allow-sys` +### The browser + +[`browser/module.mjs`](./browser/module.mjs) runs the same proofs inside a +browser realm and answers a serializable report. The generated website hosts it; +see [`todo/browser-testing.md`](./todo/browser-testing.md) for the automated +runners still to come. + You can also implement your own runner, as long as it follows the proof-tree conventions described below. +## Design: one runner, several hosts + +`fjs t` and the browser runner are **the same runner**. Discovering +zero-argument leaves, walking the tree a proof returns, the structural `throw` +expectation, resolving real promises, formatting paths and counting results all +live once, in [`module.f.mjs`](./module.f.mjs); a host supplies only two things. + +- **A `Reporter`.** It receives semantic events — one normalized `TestResult` + per leaf, and the totals — and decides how they are shown. `defaultReporter` + writes coloured lines (or GitHub annotations); `recordingReporter` hands each + result to the `report` operation, and the browser adapter renders it into the + page. A `TestResult` carries no terminal text and no DOM, so neither reporter + can smuggle presentation back into the core. +- **An effect runner.** `sandbox` is the one operation that actually *executes* + a proof body, and each host implements it against its own realm — Node in + [`../effects/node/module.mjs`](../effects/node/module.mjs), the browser in + [`../effects/browser/module.mjs`](../effects/browser/module.mjs). Both + implement it identically, because a suite that meant different things in the + two would not be one suite. + +The two runners *used* to be two implementations of the same rules, in +`module.f.mjs` and a standalone `browser.mjs`, and the rules had begun to drift. +Consult that history before adding a rule to either host: it belongs in the +core, or it is not a rule about proofs. + +External runners (`node --test`, `bun test`, `deno test`) are the one genuine +exception, and `registerModule` is why: those frameworks own scheduling and +counting, so they are handed the tree rather than driven through it. The +differences that follow from that are documented in +[`todo/661-test-runner-behavior.md`](./todo/661-test-runner-behavior.md). + ## Design: dependency-free proofs Unlike most test frameworks (Jest, Mocha, Vitest, …), a proof does **not** import @@ -232,6 +270,12 @@ to decide whether to await it. Only genuine `Promise` instances are awaited; plain *thenables* — objects with a `.then` method that are not `instanceof Promise` — are treated as ordinary return values and walked as sub-trees. +Every runner asks the same question, in the same place — the `sandbox` +operation — so a suite means the same thing under `fjs t` and in a browser. One +consequence is that a promise built in *another* realm is not `instanceof +Promise` and so is not awaited; see +[`todo/hostile-proof-values.md`](./todo/hostile-proof-values.md). + This is intentional. FunctionalScript does not allow direct `Promise` construction; `Promise` objects only arise as the return value of `async` functions (an Effect). A plain `{ then: f }` object in FunctionalScript is almost diff --git a/fjs/emergent_testing/browser.mjs b/fjs/emergent_testing/browser.mjs deleted file mode 100644 index 3d280f3bf..000000000 --- a/fjs/emergent_testing/browser.mjs +++ /dev/null @@ -1,455 +0,0 @@ -/** - * Browser-native proof execution and report rendering. - * - * The module deliberately has no Node dependencies: generated applications - * import it directly as an ES module in the browser. - * Proof failures resolve the published report with `status: 'failed'`; an - * automated outer controller is responsible for consuming that status and - * choosing a nonzero process exit code. - * - * Every DOM entry point reaches the page through the `root` element it is - * given — `root.ownerDocument` and its `defaultView` — never through the - * runner realm's own `window`/`document`. A page embedding the suite in an - * iframe therefore renders into that frame, and a proof can drive the module - * with a stand-in root. - * - * @module - * - * @import { _TestAndPath } from './types.ts' - */ - -import { collectTests, fmtPath } from './module.f.mjs' - -/** @type {(value: unknown) => string} */ -const text = value => { - try { - return String(value) - } catch { - return 'Unknown thrown value' - } -} - -/** - * The message and stack to report a thrown value by. - * - * An Error thrown from another realm — an iframe, a worker — is not - * `instanceof Error` here, and its stack is the very thing the report exists to - * carry. What the fields say is therefore the test, not where the value was - * made: anything carrying `message` or `stack` is read as the failure it - * describes, and everything else by its own text. - * - * @type {(error: unknown) => readonly [string, string]} - */ -const errorDetails = error => { - try { - if (error !== null && (typeof error === 'object' || typeof error === 'function') - && ('message' in error || 'stack' in error)) { - const { message, stack } = /** @type {{ readonly message?: unknown, readonly stack?: unknown }} */ (error) - const described = text(message) - return [described, stack === undefined ? described : text(stack)] - } - } catch { - // Reading the fields, and asking whether they are there at all, are - // user-observable operations: revoked proxies and accessors can throw - // while the failure is inspected. - } - const fallback = text(error) - return [fallback, fallback] -} - -/** @typedef {{ readonly module: string, readonly path: string, readonly status: string, readonly duration: number, readonly message?: string, readonly stack?: string }} _BrowserTestResult */ -/** @typedef {{ readonly status: string, readonly browser: string, readonly totals: { readonly tests: number, readonly passed: number, readonly failed: number }, readonly duration: number, readonly results: readonly _BrowserTestResult[] }} BrowserTestReport */ - -/** - * Attaches the handlers with the intrinsic `then`, but answers with a promise - * of this realm instead of the one `then` returns. That result is built by - * `constructor[Symbol.species]`, which a promise can make an ordinary object: - * awaiting it would end the test before the promise it came from ever settled - * and put the species object itself in the report. - * - * The `then` call still throws — before either handler is attached — for a - * value that is not a promise or whose species construction fails, which is - * what `runPromise` reads. - * - * @type {(value: unknown, fulfilled: (value: unknown) => Promise | readonly _BrowserTestResult[], rejected: (error: unknown) => readonly _BrowserTestResult[]) => Promise} - */ -const subscribe = (value, fulfilled, rejected) => { - /** @type {(results: Promise | readonly _BrowserTestResult[]) => void} */ - let settle = () => undefined - /** @type {Promise} */ - const settled = new Promise(resolve => { settle = resolve }) - Reflect.apply(Promise.prototype.then, value, [ - /** @type {(value: unknown) => void} */ (resolved => settle(fulfilled(resolved))), - /** @type {(error: unknown) => void} */ (error => settle(rejected(error))), - ]) - return settled -} - -/** - * Reproduces the lookup `then` performs before it builds its result promise: - * `constructor`, then its `Symbol.species`. A genuine promise with a hostile - * species throws here too; an object that only claims to be a promise failed - * the brand check first and reads its `constructor` cleanly. That is what - * separates a promise nothing can subscribe to from an ordinary proof tree, - * once shadowing `constructor` has turned out to be impossible. - * - * @type {(value: unknown) => boolean} - */ -const speciesFails = value => { - try { - if (value === null || value === undefined) { return false } - const { constructor } = /** @type {{ readonly constructor?: unknown }} */ (value) - if (constructor === null || constructor === undefined) { return false } - // The species itself never matters, only whether reading it completes: - // that is the step `then` takes before it builds its result. - void /** @type {{ readonly [Symbol.species]?: unknown }} */ (constructor)[Symbol.species] - return false - } catch { - return true - } -} - -/** - * Runs the intrinsic Promise `then` only for genuine promises. The first call - * is both the native brand check and the normal await path, so arbitrary proof - * objects with a `then` key are never assimilated. - * - * A genuine Promise can still throw after passing the brand check if species - * construction fails. In that case, temporarily shadow `constructor` with the - * current realm's Promise and retry the same intrinsic call; the shadow is - * removed immediately after the handlers are attached. - * - * A promise that pins its own `constructor`, or is frozen, leaves nothing to - * shadow, so no subscription is possible at all. The species failure is then - * reported against the test that produced the promise — the same outcome - * `await` gives it in the Node runner — because a result nobody can observe is - * not a pass. A non-extensible object that merely claims to be a promise - * reaches the same dead end and is still walked as the proof tree it is. - * - * @type {(value: unknown, fulfilled: (value: unknown) => Promise | readonly _BrowserTestResult[], rejected: (error: unknown) => readonly _BrowserTestResult[]) => Promise | null} - */ -const runPromise = (value, fulfilled, rejected) => { - const call = () => subscribe(value, fulfilled, rejected) - try { - return call() - } catch (error) { - // Either `value` is not a promise and the brand check rejected it - // before any handler was attached, or it is a genuine promise that - // failed while constructing the result through Symbol.species. Only - // the second case is worth a retry, and `then` attaches nothing before - // it throws, so the retry cannot run the handlers twice. - try { - if (Object.prototype.toString.call(value) !== '[object Promise]') { return null } - } catch { - return null - } - if (value === null || (typeof value !== 'object' && typeof value !== 'function')) { return null } - /** @type {PropertyDescriptor | undefined} */ - let descriptor - try { - descriptor = Object.getOwnPropertyDescriptor(value, 'constructor') - Object.defineProperty(value, 'constructor', { value: Promise, configurable: true }) - } catch { - // Nothing to shadow, so the value is whatever its own lookup says: - // a promise that cannot be subscribed to fails on the species error - // rather than passing on a result that was never awaited, and a - // frozen spoof is an ordinary proof tree. - return speciesFails(value) ? Promise.resolve(rejected(error)) : null - } - try { - return call() - } catch { - // The intrinsic `constructor` cannot fail the retry, so the brand - // check did: `value` only claims to be a promise and is walked as - // an ordinary proof result. - return null - } finally { - try { - if (descriptor === undefined) { - Reflect.deleteProperty(value, 'constructor') - } else { - Object.defineProperty(value, 'constructor', descriptor) - } - } catch { - // The temporary property is configurable, so ordinary objects - // restore cleanly. A hostile Proxy can make restoration itself - // observable. - } - } - } -} - -/** @type {(module: string, path: readonly (string | null)[], throws: boolean, fn: () => unknown, result: (result: _BrowserTestResult) => void) => Promise} */ -const runOne = (module, path, throws, fn, result) => { - const start = performance.now() - /** @type {(value: unknown) => Promise | readonly _BrowserTestResult[]} */ - const passed = value => { - const duration = performance.now() - start - if (throws) { - const failure = { module, path: fmtPath(path), status: 'failed', duration, - message: 'Expected the proof to throw', stack: '' } - result(failure) - return [failure] - } - // Reading the returned tree runs user code: an enumerable getter - // or a proxy trap can throw. That is a failure of the test that - // produced the value, never of the run — a rejected run leaves the - // page in `running` with no report and no completion event. - /** @type {readonly _TestAndPath[]} */ - let children - try { - children = collectTests([...path, null], false, value) - } catch (error) { - return failed(error) - } - return Promise.all(children.map(([childPath, child]) => - runOne(module, childPath, child.throws, child.fn, result) - )).then(results => { - const success = { module, path: fmtPath(path), status: 'passed', duration } - result(success) - return [success, ...results.flat()] - }) - } - /** @type {(error: unknown) => readonly _BrowserTestResult[]} */ - const failed = error => { - const duration = performance.now() - start - if (throws) { - const success = { module, path: fmtPath(path), status: 'passed', duration } - result(success) - return [success] - } - const [message, stack] = errorDetails(error) - const failure = { module, path: fmtPath(path), status: 'failed', duration, message, stack } - result(failure) - return [failure] - } - // Wrap the raw return so Promise resolution does not assimilate arbitrary - // objects with a `then` proof property. The Node runner awaits only actual - // promises, and browser execution must preserve that same test-tree rule. - return Promise.resolve().then(() => [fn()]).then( - ([value]) => runPromise(value, passed, failed) ?? passed(value), - failed - ) -} - -/** @type {(status: string, duration: number, results: readonly _BrowserTestResult[]) => BrowserTestReport} */ -const reportOf = (status, duration, results) => { - const failed = results.filter(result => result.status === 'failed').length - return { - status, - browser: navigator.userAgent, - totals: { tests: results.length, passed: results.length - failed, failed }, - duration, - results, - } -} - -/** - * Runs named proof exports and returns the serializable browser report. - * - * @type {(modules: readonly (readonly [string, unknown])[], result?: (result: _BrowserTestResult) => void) => Promise} - */ -export const runBrowserProofs = (modules, result = () => undefined) => { - const start = performance.now() - // Reporting each result as it lands is the page's own code. A renderer that - // throws must not take the run down with it: the report it fails to show is - // the one thing the page is still waiting for. - /** @type {(result: _BrowserTestResult) => void} */ - const announce = value => { - try { - result(value) - } catch { - // The result stays in the report the run resolves with. - } - } - /** @type {(module: string, error: unknown) => () => Promise} */ - const unreadable = (module, error) => () => { - const [message, stack] = errorDetails(error) - const failure = { module, path: '', status: 'failed', duration: 0, message, stack } - announce(failure) - return Promise.resolve([failure]) - } - const tests = modules.flatMap(([module, proof]) => { - // Reading an exported tree runs user code just as reading a returned - // one does. A module that cannot be enumerated is one failed module, - // never a run that ends without a report. - try { - return collectTests([], false, proof).map(([path, entry]) => - () => runOne(module, path, entry.throws, entry.fn, announce) - ) - } catch (error) { - return [unreadable(module, error)] - } - }) - const batchSize = 25 - /** @type {(index: number, results: readonly _BrowserTestResult[]) => Promise} */ - const runBatch = (index, results) => { - const batch = tests.slice(index, index + batchSize) - if (batch.length === 0) { return Promise.resolve(results) } - return Promise.all(batch.map(test => test())).then(next => - new Promise(resolve => setTimeout(resolve, 0, [...results, ...next.flat()])) - ).then(next => runBatch(index + batchSize, next)) - } - const completed = runBatch(0, []) - return completed.then(results => reportOf( - results.some(result => result.status === 'failed') ? 'failed' : 'passed', - performance.now() - start, - results, - )) -} - -/** @typedef {(source: string) => Promise<{ readonly proof?: unknown }>} _BrowserImporter */ -/** @typedef {{ readonly status: 'loaded', readonly source: string, readonly proof: unknown } | { readonly status: 'error', readonly source: string, readonly error: unknown }} _LoadedModule */ -/** @typedef {Window & { fjsBrowserTestReport?: Promise }} _TestWindow */ - -/** @type {(root: Element) => _TestWindow | null} */ -const viewOf = root => root.ownerDocument.defaultView - -/** - * Renders the settled report into the page, publishes the run as - * `fjsBrowserTestReport` on the root's window, and announces it with - * `fjs-browser-test-complete`. - * - * @type {(root: Element, report: Promise) => Promise} - */ -const publish = (root, report) => { - const view = viewOf(root) - const done = report.then(value => { - renderBrowserReport(root, value) - view?.dispatchEvent(new CustomEvent('fjs-browser-test-complete', { detail: value })) - return value - }) - if (view !== null) { view.fjsBrowserTestReport = done } - return done -} - -/** - * Loads proof modules after the page has rendered, reporting module-loading - * progress before proof execution begins. - * - * @type {(root: Element, sources: readonly string[], importer: _BrowserImporter) => Promise} - */ -export const startBrowserTestSources = (root, sources, importer) => { - const start = performance.now() - setState(root, 'loading') - let loaded = 0 - const summary = root.querySelector('[data-test-summary]') - // Set synchronously, before any import settles: otherwise the page keeps - // showing its idle text throughout loading — indefinitely, if a module - // import never settles — even though the state and control already - // changed. - if (summary !== null) { summary.textContent = `Loading 0/${sources.length}` } - // The importer is supplied by the page, so obtaining the promise is itself - // a failure point: a synchronous throw becomes a rejection here and is - // reported as a loader failure, rather than escaping past a `loading` state - // that no report or completion event ever replaces. - /** @type {(source: string) => Promise<{ readonly proof?: unknown }>} */ - const load = source => { - try { - return importer(source) - } catch (error) { - return Promise.reject(error) - } - } - /** @type {Promise} */ - const modules = Promise.all(sources.map(source => load(source).then( - module => { - loaded += 1 - if (summary !== null) { summary.textContent = `Loading ${loaded}/${sources.length}: ${source}` } - return /** @type {const} */ ({ status: 'loaded', source, proof: module.proof }) - }, - error => /** @type {const} */ ({ status: 'error', source, error }) - ))) - const report = modules.then(loadedModules => { - const rejected = loadedModules.flatMap(module => - module.status === 'error' ? [module] : []) - if (rejected.length !== 0) { - // A module that never linked has no tests to run, so the run stops - // here. Each rejection is still counted as a failed result: totals - // that disagreed with `results` would tell an automated consumer - // the suite was empty rather than broken. - const duration = performance.now() - start - return publish(root, Promise.resolve(reportOf('infrastructure-error', duration, - rejected.map(({ source, error }) => { - const [message, stack] = errorDetails(error) - return { module: source, path: '', status: 'failed', duration, message, stack } - })))) - } - return startBrowserTests(root, loadedModules.flatMap(module => - module.status === 'loaded' - ? [/** @type {const} */ ([module.source, module.proof])] - : [])) - }) - const view = viewOf(root) - if (view !== null) { view.fjsBrowserTestReport = report } - return report -} - -/** - * Sets the runner state and keeps the `Run` control's real disabled state in - * sync with it: passive while a suite is loading or running, active in every - * other state (idle, or any terminal status). A disabled attribute is used - * rather than a click handler that silently ignores the action, so assistive - * technology sees the same unavailability a sighted user does. - * - * @type {(root: Element, state: string) => void} - */ -const setState = (root, state) => { - root.setAttribute('data-state', state) - const runButton = root.querySelector('[data-test-run]') - if (runButton !== null) { - if (state === 'loading' || state === 'running') { - runButton.setAttribute('disabled', '') - } else { - runButton.removeAttribute('disabled') - } - } -} - -/** - * Renders a completed report in the browser test page. - * - * @type {(root: Element, report: BrowserTestReport) => void} - */ -export const renderBrowserReport = (root, report) => { - setState(root, report.status) - const summary = root.querySelector('[data-test-summary]') - if (summary !== null) { - summary.textContent = report.status === 'infrastructure-error' - ? `Infrastructure error: ${report.totals.failed} failed to load (${report.duration.toFixed(1)} ms)` - : `${report.totals.passed} passed, ${report.totals.failed} failed (${report.duration.toFixed(1)} ms)` - } - const output = root.querySelector('[data-test-results]') - if (output !== null) { - output.replaceChildren(...report.results.map(result => - renderResult(root.ownerDocument, result))) - } -} - -/** @type {(document: Document, result: _BrowserTestResult) => HTMLLIElement} */ -const renderResult = (document, result) => { - const item = document.createElement('li') - item.setAttribute('data-status', result.status) - const detail = result.status === 'failed' ? `: ${result.message}\n${result.stack}` : '' - item.textContent = `${result.status === 'passed' ? 'PASS' : 'FAIL'} ${result.module} ${result.path} (${result.duration.toFixed(1)} ms)${detail}` - return item -} - -/** - * Runs the application, publishes its promise as `window.fjsBrowserTestReport`, - * and dispatches `fjs-browser-test-complete` with the report in `detail`. - * - * @type {(root: Element, modules: readonly (readonly [string, unknown])[]) => Promise} - */ -export const startBrowserTests = (root, modules) => { - setState(root, 'running') - const output = root.querySelector('[data-test-results]') - if (output !== null) { output.replaceChildren() } - let completed = 0 - return publish(root, runBrowserProofs(modules, result => { - completed += 1 - const summary = root.querySelector('[data-test-summary]') - if (summary !== null) { summary.textContent = `${completed} tests completed…` } - if (output !== null) { output.append(renderResult(root.ownerDocument, result)) } - })) -} diff --git a/fjs/emergent_testing/browser/module.f.mjs b/fjs/emergent_testing/browser/module.f.mjs new file mode 100644 index 000000000..45718d224 --- /dev/null +++ b/fjs/emergent_testing/browser/module.f.mjs @@ -0,0 +1,140 @@ +/** + * The browser proof application: link the proof modules, run them through the + * shared emergent-testing core, and answer one serializable report. + * + * **It performs no browser operation of its own.** Linking a module, reading + * the clock, executing a proof body and recording a result are all operations + * (`./types.ts`), so this program is exactly as runnable from a proof with a + * stand-in interpreter as it is from a page. What is genuinely the browser's — + * the DOM, the published promise, the completion event — lives in the impure + * adapter beside it, [`./module.mjs`](./module.mjs). + * + * **It owns no proof semantics either.** Discovering zero-argument leaves, + * walking a returned tree, the structural `throw` expectation, resolving real + * promises and counting results are `../module.f.mjs`'s, the same module `fjs t` + * runs through — this file only decides what a *run* is: load, run, report. + * + * @module + * + * @import { Effect } from '../../effects/types.ts' + * @import { Module } from '../../effects/common/types.ts' + * @import { IoChannel, Import } from '../../effects/common/types.ts' + * @import { TestResult } from '../types.ts' + * @import { BrowserOp, BrowserProgram, BrowserTestReport, ReportStatus, _Loaded } from './types.ts' + */ + +import { allOk, errorMessage, import_, now } from '../../effects/common/module.f.mjs' +import { history, historyStep, mapStep, pureOk, resultMapStep, step } from '../../effects/module.f.mjs' +import { recordingReporter, reported, runModuleMap } from '../module.f.mjs' +import { fromEntries } from '../../types/object/module.f.mjs' +import { ok } from '../../types/result/module.f.mjs' + +/** + * Builds the report from the results a run recorded. Totals are counted here + * rather than reported separately, so they cannot disagree with `results`. + * + * @type {(status: ReportStatus, browser: string, duration: number, results: readonly TestResult[]) => BrowserTestReport} + */ +export const reportOf = (status, browser, duration, results) => { + const failed = results.filter(result => result.status === 'failed').length + return { + status, + browser, + totals: { tests: results.length, passed: results.length - failed, failed }, + duration, + results, + } +} + +/** + * The result standing for something that went wrong outside any proof: a module + * that would not link, or an operation the runner does not implement. + * + * It is counted as a failed result rather than left out. Totals that disagreed + * with `results` would tell an automated consumer the suite was empty rather + * than broken. + * + * @type {(module: string, message: string) => TestResult} + */ +const infrastructureResult = (module, message) => + ({ module, path: '', status: 'failed', duration: 0, message, stack: '' }) + +/** + * Links one source, keeping the failure rather than propagating it: a run + * reports *every* module that would not link, and the first one would + * short-circuit the rest away. + * + * @type {(source: string) => Effect} + */ +const loadOne = source => resultMapStep(import_(source), r => { + /** @type {_Loaded} */ + const loaded = [source, r] + return ok(loaded) +}) + +/** @type {(results: readonly TestResult[]) => ReportStatus} */ +const statusOf = results => + results.some(result => result.status === 'failed') ? 'failed' : 'passed' + +/** @internal What a run answers before it is timed and packaged. */ +/** @typedef {readonly[ReportStatus, readonly TestResult[]]} _Outcome */ + +/** + * Runs the modules that linked, or reports the ones that did not. + * + * A module that never linked has no tests to run, so the run stops at the first + * broken graph rather than reporting a partial suite as a complete one. + * + * @type {(loaded: readonly _Loaded[]) => Effect} + */ +const runLoaded = loaded => { + const linked = loaded.flatMap(([source, r]) => + r[0] === 'ok' ? [/** @type {const} */ ([source, r[1]])] : []) + if (linked.length !== loaded.length) { + /** @type {_Outcome} */ + const broken = ['infrastructure-error', loaded.flatMap(([source, r]) => + r[0] === 'error' ? [infrastructureResult(source, errorMessage(r[1]))] : [])] + return pureOk(broken) + } + const ran = runModuleMap(recordingReporter)(fromEntries(linked)) + const collected = step(ran, () => reported()) + return mapStep(collected, results => { + /** @type {_Outcome} */ + const outcome = [statusOf(results), results] + return outcome + }) +} + +/** @type {(sources: readonly string[]) => Effect} */ +const runSources = sources => + step(allOk(...sources.map(loadOne)), runLoaded) + +/** + * A run that could not finish, reported as one infrastructure error against the + * run itself. + * + * This is what makes {@link BrowserProgram}'s empty error channel true: a + * runner that cannot dispatch `sandbox`, `now` or `report` leaves the program + * with nothing to answer, and a page waiting on the run has nowhere to put a + * failure it never receives. + * + * @type {(browser: string, message: string) => BrowserTestReport} + */ +const failedRun = (browser, message) => + reportOf('infrastructure-error', browser, 0, [infrastructureResult('', message)]) + +/** + * The application: link every source, run the proofs that linked, and answer + * the report. + * + * @type {BrowserProgram} + */ +export const main = ({ browser, sources }) => { + const started = history(now()) + const outcome = historyStep(started, () => runSources(sources)) + const ended = historyStep(outcome, () => now()) + const report = mapStep(ended, ([end, [status, results], start]) => + reportOf(status, browser, end - start, results)) + return resultMapStep(report, r => + ok(r[0] === 'error' ? failedRun(browser, errorMessage(r[1])) : r[1])) +} diff --git a/fjs/emergent_testing/browser/module.mjs b/fjs/emergent_testing/browser/module.mjs new file mode 100644 index 000000000..93b753afb --- /dev/null +++ b/fjs/emergent_testing/browser/module.mjs @@ -0,0 +1,177 @@ +/** + * The browser host adapter: capabilities, DOM rendering, and publication. + * + * It owns nothing about what a proof *means*. Walking proof trees, the + * structural `throw` expectation, resolving real promises, path formatting and + * the totals belong to `../module.f.mjs` — the module `fjs t` runs through — + * and what a *run* is belongs to the pure application in + * [`./module.f.mjs`](./module.f.mjs). What is left here is the browser: an + * interpreter for the operations that application performs, the DOM it is + * rendered into, and the promise and event a controller reads it from. + * + * The module deliberately has no Node dependency: generated applications import + * it directly as an ES module in the browser. + * Proof failures resolve the published report with `status: 'failed'`; an + * automated outer controller is responsible for consuming that status and + * choosing a nonzero process exit code. + * + * Every DOM entry point reaches the page through the `root` element it is + * given — `root.ownerDocument` and its `defaultView` — never through the + * runner realm's own `window`/`document`. A page embedding the suite in an + * iframe therefore renders into that frame, and a proof can drive the module + * with a stand-in root. + * + * @module + * + * @import { Effect } from '../../effects/types.ts' + * @import { Result } from '../../types/result/types.ts' + * @import { BrowserImporter } from '../../effects/browser/module.mjs' + * @import { TestResult } from '../types.ts' + * @import { BrowserOp, BrowserTestReport } from './types.ts' + */ + +import { asyncRun } from '../../effects/module.mjs' +import { browserOperationMap } from '../../effects/browser/module.mjs' +import { main } from './module.f.mjs' +import { ok } from '../../types/result/module.f.mjs' + +/** @typedef {Window & { fjsBrowserTestReport?: Promise }} _TestWindow */ + +/** @typedef {(effect: Effect) => Promise>} _Run */ + +/** + * How many results may be rendered before the runner hands the event loop back. + * + * Every operation resolves through a microtask, and microtasks do not let a + * browser paint: without a real task boundary the page would show its first + * frame again only once the whole suite had finished. Yielding per result would + * be the simpler rule and the wrong one — `setTimeout` clamps to 4 ms once + * nested, which is minutes across a few thousand proofs. + */ +const batchSize = 25 + +/** @type {() => Promise} */ +const macrotask = () => new Promise(resolve => { setTimeout(resolve, 0) }) + +/** @type {(root: Element) => _TestWindow | null} */ +const viewOf = root => root.ownerDocument.defaultView + +/** + * Sets the runner state and keeps the `Run` control's real disabled state in + * sync with it: passive while a suite is loading or running, active in every + * other state (idle, or any terminal status). A disabled attribute is used + * rather than a click handler that silently ignores the action, so assistive + * technology sees the same unavailability a sighted user does. + * + * @type {(root: Element, state: string) => void} + */ +const setState = (root, state) => { + root.setAttribute('data-state', state) + const runButton = root.querySelector('[data-test-run]') + if (runButton !== null) { + if (state === 'loading' || state === 'running') { + runButton.setAttribute('disabled', '') + } else { + runButton.removeAttribute('disabled') + } + } +} + +/** @type {(document: Document, result: TestResult) => HTMLLIElement} */ +const renderResult = (document, result) => { + const item = document.createElement('li') + item.setAttribute('data-status', result.status) + const detail = result.status === 'failed' ? `: ${result.message}\n${result.stack}` : '' + item.textContent = `${result.status === 'passed' ? 'PASS' : 'FAIL'} ${result.module} ${result.path} (${result.duration.toFixed(1)} ms)${detail}` + return item +} + +/** + * Renders a completed report in the browser test page. + * + * @type {(root: Element, report: BrowserTestReport) => void} + */ +export const renderBrowserReport = (root, report) => { + setState(root, report.status) + const summary = root.querySelector('[data-test-summary]') + if (summary !== null) { + summary.textContent = report.status === 'infrastructure-error' + ? `Infrastructure error: ${report.totals.failed} failed to load (${report.duration.toFixed(1)} ms)` + : `${report.totals.passed} passed, ${report.totals.failed} failed (${report.duration.toFixed(1)} ms)` + } + const output = root.querySelector('[data-test-results]') + if (output !== null) { + output.replaceChildren(...report.results.map(result => + renderResult(root.ownerDocument, result))) + } +} + +/** + * Runs the browser application against `root`, publishes its promise as + * `fjsBrowserTestReport` on the root's window, and dispatches + * `fjs-browser-test-complete` with the report in `detail`. + * + * `importer` is the seam a controller reaches for: an application root resolves + * its own specifiers, and a proof drives the whole runner without a network. + * The default is the realm's own dynamic `import`. + * + * @type {(root: Element, sources: readonly string[], importer?: BrowserImporter) => Promise} + */ +export const startBrowserTestSources = (root, sources, importer = source => import(source)) => { + setState(root, 'loading') + const summary = root.querySelector('[data-test-summary]') + const output = root.querySelector('[data-test-results]') + if (output !== null) { output.replaceChildren() } + // Set synchronously, before any import settles: otherwise the page keeps + // showing its idle text throughout loading — indefinitely, if a module + // import never settles — even though the state and control already changed. + if (summary !== null) { summary.textContent = `Loading 0/${sources.length}` } + let loaded = 0 + /** @type {(source: string) => void} */ + const linked = source => { + loaded += 1 + if (summary !== null) { summary.textContent = `Loading ${loaded}/${sources.length}: ${source}` } + // Whether the module linked or not, the loading phase is over once the + // last answer is in — a broken graph is reported by the run, not by + // leaving the page in `loading` forever. + if (loaded === sources.length) { setState(root, 'running') } + } + /** @type {BrowserImporter} */ + const load = source => importer(source).then( + module => { linked(source); return module }, + error => { linked(source); throw error }) + /** @type {readonly TestResult[]} */ + let results = [] + /** @type {_Run} */ + const run = asyncRun({ + ...browserOperationMap(effect => run(effect), load), + report: async result => { + results = [...results, result] + if (summary !== null) { summary.textContent = `${results.length} tests completed…` } + if (output !== null) { output.append(renderResult(root.ownerDocument, result)) } + if (results.length % batchSize === 0) { await macrotask() } + return ok(undefined) + }, + reported: async () => ok(results), + }) + const view = viewOf(root) + // The application's error channel is empty — every failure it can meet is + // reported — so the run's `Result` is always `ok` and `unwrap` would only + // add a panic path nothing can reach. + const report = run(main({ browser: navigatorName(root), sources })).then(([, value]) => { + renderBrowserReport(root, value) + view?.dispatchEvent(new CustomEvent('fjs-browser-test-complete', { detail: value })) + return value + }) + if (view !== null) { view.fjsBrowserTestReport = report } + return report +} + +/** + * The realm the run is recorded under, read through the root's own window so an + * embedded suite names the frame it actually runs in — and so a proof driving + * the runner with a stand-in root never needs a global `navigator`. + * + * @type {(root: Element) => string} + */ +const navigatorName = root => viewOf(root)?.navigator.userAgent ?? '' diff --git a/fjs/emergent_testing/browser/proof.f.mjs b/fjs/emergent_testing/browser/proof.f.mjs new file mode 100644 index 000000000..e46c4cf31 --- /dev/null +++ b/fjs/emergent_testing/browser/proof.f.mjs @@ -0,0 +1,162 @@ +/** + * Proofs for the browser proof application. + * + * The application performs only operations, so a state-threading stand-in + * interpreter is enough to drive every path from Node — no browser, no DOM, and + * no globals for these proofs to install and unset. `sandbox` is the same + * pass-through the virtual Node runner uses: a fixture returns the + * `SandboxResult` it wants reported, so outcomes are dictated rather than + * measured. + * + * @import { Result } from '../../types/result/types.ts' + * @import { MemOperationMap, RunInstance } from '../../effects/mock/types.ts' + * @import { Module, SandboxResult } from '../../effects/common/types.ts' + * @import { StringMap } from '../../types/object/types.ts' + * @import { TestResult } from '../types.ts' + * @import { BrowserOp, BrowserTestReport } from './types.ts' + */ + +import { assert, assertEq } from '../../asserts/module.f.mjs' +import { ioError } from '../../effects/common/module.f.mjs' +import { notImplemented } from '../../effects/module.f.mjs' +import { run as mockRun } from '../../effects/mock/module.f.mjs' +import { error, ok, unwrap } from '../../types/result/module.f.mjs' +import { main } from './module.f.mjs' + +/** + * @typedef {{ + * readonly time: number, + * readonly clock: boolean, + * readonly results: readonly TestResult[], + * readonly modules: StringMap, + * }} _State + */ + +/** @type {MemOperationMap} */ +const map = { + all: (...a) => state => { + /** @type {readonly Result[]} */ + let e = [] + for (const i of a) { + const [ns, ei] = browser(state)(i) + state = ns + e = [...e, ei] + } + return [state, ok(e)] + }, + await: p => state => [state, ok([p])], + fetch: () => state => [state, error(ioError({ message: 'no network' }))], + import: source => state => { + const module = state.modules[source] + return [ + state, + module === undefined + ? error(ioError({ code: 'ENOENT', message: `cannot link ${source}` })) + : ok(module), + ] + }, + // A clock that ticks once per read, so a run's duration is the number of + // reads between its ends and never a real elapsed time. + now: () => state => [ + { ...state, time: state.time + 1 }, + state.clock ? ok(state.time) : error(notImplemented('now')), + ], + sandbox: f => state => [state, ok(/** @type {SandboxResult} */ (f()))], + report: result => state => [{ ...state, results: [...state.results, result] }, ok(undefined)], + reported: () => state => [state, ok(state.results)], +} + +/** @type {RunInstance} */ +const browser = mockRun(map) + +/** @type {(sources: readonly string[], modules: StringMap, clock?: boolean) => BrowserTestReport} */ +const run = (sources, modules, clock = true) => { + /** @type {_State} */ + const state = { time: 100, clock, results: [], modules } + const [, report] = browser(state)(main({ browser: 'proof', sources })) + return unwrap(report) +} + +/** A leaf that passes, taking 2 ms. + * + * @type {() => unknown} + */ +const pass = () => ({ result: ok(undefined), duration: 2 }) + +/** A leaf that fails with an `Error`. + * + * @type {() => unknown} + */ +const fail = () => ({ result: error(new Error('oops')), duration: 3 }) + +export const proof = { + passing: () => { + const report = run(['a'], { a: { proof: { x: pass } } }) + assertEq(report.status, 'passed') + assertEq(report.browser, 'proof') + assertEq(report.totals.tests, 1) + assertEq(report.totals.passed, 1) + assertEq(report.totals.failed, 0) + // Two clock reads bracket the run, and the stand-in ticks once per read. + assertEq(report.duration, 1) + assertEq(report.results[0]?.module, 'a') + assertEq(report.results[0]?.path, '.x') + assertEq(report.results[0]?.duration, 2) + }, + failing: () => { + const report = run(['a'], { a: { proof: { x: pass, y: fail } } }) + assertEq(report.status, 'failed') + assertEq(report.totals.tests, 2) + assertEq(report.totals.failed, 1) + const failed = report.results.filter(r => r.status === 'failed') + assertEq(failed[0]?.path, '.y') + assertEq(failed[0]?.message, 'oops') + }, + // The proof tree a leaf returns is walked by the same shared core `fjs t` + // uses, so a sub-test is a result of its own with a call boundary in its + // path. + subTree: () => { + const report = run(['a'], { + a: { proof: { outer: () => ({ result: ok({ inner: pass }), duration: 0 }) } }, + }) + assertEq(report.totals.tests, 2) + assertEq(report.results[1]?.path, '.outer().inner') + }, + expectedThrow: () => { + const report = run(['a'], { a: { proof: { throw: { boom: fail, quiet: pass } } } }) + assertEq(report.totals.tests, 2) + assertEq(report.totals.failed, 1) + const failed = report.results.filter(r => r.status === 'failed') + assertEq(failed[0]?.path, '.throw.quiet') + assertEq(failed[0]?.message, 'Expected the proof to throw') + }, + // A module without a `proof` export contributes no tests, and an empty run + // still answers a report rather than nothing. + withoutProof: () => { + const report = run(['a'], { a: {} }) + assertEq(report.status, 'passed') + assertEq(report.totals.tests, 0) + }, + // One module that would not link stops the run: the suite never ran, so its + // status is not the one a failing suite gets, and every rejected source is + // still counted as a failed result. + unlinkable: () => { + const report = run(['a', 'missing'], { a: { proof: { x: pass } } }) + assertEq(report.status, 'infrastructure-error') + assertEq(report.totals.tests, 1) + assertEq(report.totals.failed, 1) + assertEq(report.results[0]?.module, 'missing') + assertEq(report.results[0]?.path, '') + assertEq(report.results[0]?.message, 'cannot link missing') + }, + // A runner missing an operation the application needs is reported the same + // way, which is what makes the program's empty error channel true: a page + // waiting on the run always receives a report. + incompleteRunner: () => { + const report = run(['a'], { a: { proof: { x: pass } } }, false) + assertEq(report.status, 'infrastructure-error') + assertEq(report.duration, 0) + assertEq(report.results[0]?.message, 'operation not implemented: now') + assert(report.results.length === 1, report.results) + }, +} diff --git a/fjs/emergent_testing/browser/proof.mjs b/fjs/emergent_testing/browser/proof.mjs index c4af06c77..fc7a7fcb7 100644 --- a/fjs/emergent_testing/browser/proof.mjs +++ b/fjs/emergent_testing/browser/proof.mjs @@ -1,20 +1,29 @@ /** - * Proofs for the browser runner. + * Proofs for the browser host adapter and the browser interpretation of the + * host-independent operations. * - * The runner reaches the page only through the root element it is handed, so + * The adapter reaches the page only through the root element it is handed, so * the DOM stand-in below is enough to drive every rendering branch from Node — * no headless browser, and no global `window`/`document` for these proofs to - * install and unset. + * install and unset. What each proof *means* is settled a layer down, by the + * shared core and its own proofs; what is checked here is that a browser run + * reaches it, renders it, and publishes it. + * + * @import { Module } from '../../effects/common/types.ts' + * @import { CommonRun } from '../../effects/browser/module.mjs' + * @import { BrowserTestReport } from './types.ts' */ -import { runInNewContext } from 'node:vm' - -import { assert, assertEq, assertNotNullish, assertStructurallySame } from '../../asserts/module.f.mjs' -import { renderBrowserReport, runBrowserProofs, startBrowserTests, startBrowserTestSources } from '../browser.mjs' +import { assert, assertEq, assertNotNullish } from '../../asserts/module.f.mjs' +import { browserOperationMap } from '../../effects/browser/module.mjs' +import { asyncRun } from '../../effects/module.mjs' +import { pureOk } from '../../effects/module.f.mjs' +import { renderBrowserReport, startBrowserTestSources } from './module.mjs' +import { unwrap } from '../../types/result/module.f.mjs' /** @typedef {{ readonly tag: string, attributes: ReadonlyMap, readonly ownerDocument: _Document, textContent: string, children: readonly _Element[], readonly setAttribute: (name: string, value: string) => void, readonly removeAttribute: (name: string) => void, readonly querySelector: (selector: string) => _Element | null, readonly replaceChildren: (...nodes: readonly _Element[]) => void, readonly append: (node: _Element) => void }} _Element */ /** @typedef {{ defaultView: _View | null, readonly createElement: (tag: string) => _Element }} _Document */ -/** @typedef {{ events: readonly CustomEvent[], readonly dispatchEvent: (event: Event) => boolean, fjsBrowserTestReport?: Promise }} _View */ +/** @typedef {{ events: readonly CustomEvent[], readonly navigator: { readonly userAgent: string }, readonly dispatchEvent: (event: Event) => boolean, fjsBrowserTestReport?: Promise }} _View */ /** @type {(node: _Element, name: string) => _Element | null} */ const find = (node, name) => @@ -40,7 +49,7 @@ const element = (document, tag, attributes, states) => { removeAttribute: name => { self.attributes = new Map([...self.attributes].filter(([key]) => key !== name)) }, - // The runner only ever queries an attribute selector of `[name]` form. + // The adapter only ever queries an attribute selector of `[name]` form. querySelector: selector => self.children.reduce( (/** @type {_Element | null} */ acc, child) => acc ?? find(child, selector.slice(1, -1)), @@ -52,7 +61,7 @@ const element = (document, tag, attributes, states) => { } /** - * Builds what the generated page gives the runner: a root carrying the summary + * Builds what the generated page gives the adapter: a root carrying the summary * paragraph and the result list. `states` records every `data-state` written, * so a proof can check the whole progression and not just its last step. * @@ -69,6 +78,7 @@ const page = (withView = true) => { /** @type {_View} */ const view = { events: [], + navigator: { userAgent: 'stand-in browser' }, dispatchEvent: event => { view.events = [...view.events, /** @type {CustomEvent} */ (event)] return true @@ -90,339 +100,179 @@ const page = (withView = true) => { } } -/** @type {(proof: unknown) => ReturnType} */ -const run = proof => runBrowserProofs([['proof', proof]]) +/** Runs one in-memory proof module through the whole browser stack. + * + * @type {(proof: unknown) => Promise} + */ +const run = proof => + startBrowserTestSources(page().root, ['proof'], async () => ({ proof })) /** @type {(element: _Element) => readonly (string | undefined)[]} */ const statuses = element => element.children.map(child => child.attributes.get('data-status')) +/** + * The browser handlers on their own, so the operations the proof application + * never reaches — `fetch`, `await`, a nested `all` — are still exercised. + */ +const operations = browserOperationMap( + effect => commonRun(effect), + async source => ({ source })) + +/** @type {CommonRun} */ +const commonRun = asyncRun(operations) + +const { all, await: awaitOp, fetch: fetchOp, import: importOp, now, sandbox } = operations + export const proof = { - namedThrow: async () => { - const named = { throw: () => { throw 'expected' } }.throw - const report = await run({ extracted: named }) + // The whole stack: a module is linked, its proofs run, each result is + // rendered as it lands, and the report is published and announced. + passing: async () => { + const { root, summary, results, view, states } = page() + const report = await startBrowserTestSources(root, ['a'], async () => ({ + proof: { x: () => undefined }, + })) assertEq(report.status, 'passed') + assertEq(report.browser, 'stand-in browser') + assertEq(report.totals.tests, 1) + assertEq(report.results[0]?.path, '.x') + assertEq(statuses(results).join(','), 'passed') + assert(summary.textContent.startsWith('1 passed, 0 failed'), summary.textContent) + assertEq(states.join(','), 'loading,running,passed') + assertEq(view.events.length, 1) + assertEq(/** @type {BrowserTestReport} */ (view.events[0]?.detail).status, 'passed') + assertEq(await view.fjsBrowserTestReport, report) }, - path: async () => { - const report = await run({ 'a.b': () => undefined }) - assertEq(report.results[0]?.path, '["a.b"]') - }, - arbitraryThrow: async () => { - const report = await run({ fail: () => { throw Object.create(null) } }) - assertEq(report.status, 'failed') - assertEq(report.results[0]?.message, 'Unknown thrown value') - }, - errorFields: async () => { - const error = new Proxy(new Error(), { - get: (target, property) => property === 'message' || property === 'stack' - ? Symbol(property) - : Reflect.get(target, property), - }) - const report = await run({ fail: () => { throw error } }) - assertEq(report.results[0]?.message, 'Symbol(message)') - assertEq(report.results[0]?.stack, 'Symbol(stack)') - }, - errorAccessorThrows: async () => { - const error = new Error('hidden') - Object.defineProperty(error, 'message', { - get: () => { throw new Error('message getter failed') }, - }) - const report = await run({ fail: () => { throw error } }) - assertEq(report.status, 'failed') - assertEq(report.results[0]?.message, 'Unknown thrown value') - assertEq(report.results[0]?.stack, 'Unknown thrown value') - }, - revokedErrorProxy: async () => { - const { proxy, revoke } = Proxy.revocable(new Error('revoked'), {}) - revoke() - const report = await run({ fail: () => { throw proxy } }) + failing: async () => { + const { root, results, runButton } = page() + const report = await startBrowserTestSources(root, ['a'], async () => ({ + proof: { boom: () => { throw new Error('bang') } }, + })) assertEq(report.status, 'failed') - assertEq(report.results[0]?.message, 'Unknown thrown value') - }, - crossRealmError: async () => { - // An Error from another realm is not `instanceof Error` here, and its - // stack is what the report exists to carry. - const other = runInNewContext( - '({ fail: () => { throw new Error(\'cross boom\') } })') - const report = await run({ fail: other.fail }) - assertEq(report.results[0]?.message, 'cross boom') - const stack = report.results[0]?.stack ?? '' - assert(stack !== 'cross boom', stack) - assert(stack.includes('cross boom'), stack) - }, - errorWithoutStack: async () => { - const error = new Error('no stack') - const report = await run({ fail: () => { throw Object.assign(error, { stack: undefined }) } }) - assertEq(report.results[0]?.message, 'no stack') - assertEq(report.results[0]?.stack, 'no stack') + assertEq(report.results[0]?.message, 'bang') + assert((report.results[0]?.stack ?? '').includes('bang')) + assertEq(statuses(results).join(','), 'failed') + // The control is available again the moment the run reaches a terminal + // state, and was not while it was loading or running. + assert(!runButton.attributes.has('disabled')) }, expectedThrow: async () => { - const report = await run({ throw: { silent: () => undefined } }) - assertEq(report.status, 'failed') - assertEq(report.results[0]?.message, 'Expected the proof to throw') - }, - crossRealmPromise: async () => { - // A promise built in another realm is not `instanceof Promise`. The - // runner has to await it anyway and walk the tree it resolves to, - // otherwise a rejected cross-realm promise is reported as a pass. - const other = runInNewContext('({ resolve: value => Promise.resolve(value) })') - const report = await run({ - nested: () => other.resolve({ child: () => { throw 'boom' } }), - }) - assertEq(report.totals.tests, 2) - assertEq(report.totals.failed, 1) - assertEq(report.results[1]?.path, '.nested().child') - }, - spoofedPromiseTag: async () => { - const report = await run({ - nested: () => ({ - [Symbol.toStringTag]: 'Promise', - then: /** @type {(...args: (() => void)[]) => void} */ ((...args) => { args[0]?.() }), - }), - }) - assertEq(report.totals.tests, 2) - assertEq(report.results[1]?.path, '.nested().then') + const report = await run({ throw: { boom: () => { throw 'expected' } } }) + assertEq(report.status, 'passed') }, - frozenPromiseTag: async () => { - // A non-extensible spoof leaves the runner nothing to shadow, the same - // dead end a pinned promise reaches. It is still an ordinary proof - // tree, so it is walked rather than reported as a brand-check failure. - const report = await run({ - nested: () => Object.freeze({ - [Symbol.toStringTag]: 'Promise', - then: () => undefined, - }), - }) + // Only a real promise is an asynchronous value, which is exactly the rule + // `fjs t` follows: the browser `sandbox` awaits one and reports what it + // resolves to. + promise: async () => { + const report = await run({ nested: () => Promise.resolve({ inner: () => undefined }) }) assertEq(report.totals.tests, 2) assertEq(report.totals.failed, 0) - assertEq(report.results[1]?.path, '.nested().then') - }, - exportedTreeThrows: async () => { - // The exported tree is read before any test runs, and reading it runs - // user code as well. The module fails; the page still gets its report. - const p = page() - const report = await startBrowserTests(p.root, - [['m', { get bad() { throw new Error('enumerating') } }]]) - assertEq(report.status, 'failed') - assertStructurallySame({ ...report.totals }, { tests: 1, passed: 0, failed: 1 }) - assertEq(report.results[0]?.module, 'm') - assertEq(report.results[0]?.message, 'enumerating') - assertStructurallySame([...p.states], ['running', 'failed']) - assertEq(p.view.events.length, 1) + assertEq(report.results[1]?.path, '.nested().inner') }, - returnedTreeThrows: async () => { - // Reading the returned tree runs user code. When it throws, the test - // that produced the value fails and the page still reaches a terminal - // state — a rejected run would leave it in `running` forever. - const p = page() - const report = await startBrowserTests(p.root, - [['m', { nested: () => ({ get bad() { throw new Error('getter') } }) }]]) + rejectedPromise: async () => { + const report = await run({ nested: () => Promise.reject(new Error('later')) }) assertEq(report.status, 'failed') - assertStructurallySame({ ...report.totals }, { tests: 1, passed: 0, failed: 1 }) - assertEq(report.results[0]?.message, 'getter') - assertStructurallySame([...p.states], ['running', 'failed']) - assertEq(p.view.events.length, 1) + assertEq(report.results[0]?.message, 'later') }, - speciesResultIsNotAPromise: async () => { - // `then` builds its result through `constructor[Symbol.species]`, and a - // promise can make that an ordinary object. The run has to answer with - // the promise it subscribed to, not with what `then` handed back, or - // the test ends before the promise settles and the species object - // itself lands in the report. - const species = function (/** @type {(...args: (() => void)[]) => void} */ executor) { - executor(() => undefined, () => undefined) - return { notAPromise: true } - } - const promised = new Promise(resolve => - setTimeout(resolve, 1, { child: () => { throw 'boom' } })) - Object.defineProperty(promised, 'constructor', - { value: { [Symbol.species]: species }, configurable: true }) - const report = await run({ nested: () => promised }) + // ...and an ordinary object carrying a `then` proof is a proof tree, never + // a thenable to assimilate. + thenIsATestName: async () => { + const report = await run({ nested: () => ({ then: () => undefined }) }) assertEq(report.totals.tests, 2) - assertEq(report.totals.failed, 1) - assertEq(report.results[1]?.path, '.nested().child') + assertEq(report.results[1]?.path, '.nested().then') }, - reportingThrows: async () => { - // Announcing a result as it lands is the page's own rendering. It must - // not take the run down with it: the report is what the page waits for. - const report = await runBrowserProofs([['m', { t: () => undefined }]], - () => { throw new Error('render') }) - assertEq(report.status, 'passed') - assertEq(report.totals.passed, 1) + // A module that will not link stops the run before any proof body, and says + // so with a status an automated consumer must not read as a failing suite. + unlinkable: async () => { + const { root, summary, states } = page() + const report = await startBrowserTestSources(root, ['a'], async () => { + throw new Error('404') + }) + assertEq(report.status, 'infrastructure-error') + assertEq(report.totals.failed, 1) + assertEq(report.results[0]?.module, 'a') + assertEq(report.results[0]?.message, '404') + assert(summary.textContent.startsWith('Infrastructure error: 1 failed to load'), summary.textContent) + assertEq(states.join(','), 'loading,running,infrastructure-error') }, - thenIsATestName: async () => { - // A `then` proof entry is a test called `then`, never a thenable for - // the runner to adopt. - const report = await run({ then: () => undefined }) - assertEq(report.totals.tests, 1) - assertEq(report.results[0]?.path, '.then') + // The importer is page code, so obtaining the promise is itself a failure + // point: a synchronous throw is a load failure, not an escape past a + // `loading` state no report ever replaces. + importerThrowsSynchronously: async () => { + const { root } = page() + const report = await startBrowserTestSources(root, ['a'], () => { + throw new Error('bad specifier') + }) + assertEq(report.status, 'infrastructure-error') + assertEq(report.results[0]?.message, 'bad specifier') }, + // Past the batch size the adapter hands the event loop back, so a long + // suite paints instead of freezing the page on its first frame. batches: async () => { - // More leaves than one batch holds, so the batch loop recurses. - const report = await run(Object.fromEntries( - Array.from({ length: 30 }, (_, index) => [`t${index}`, () => undefined]))) - assertEq(report.totals.tests, 30) - assertEq(report.totals.passed, 30) - }, - render: async () => { - const p = page() - const report = await startBrowserTests(p.root, - [['m', { ok: () => undefined, bad: () => { throw 'x' } }]]) - assertEq(report.status, 'failed') - assertStructurallySame([...p.states], ['running', 'failed']) - assertEq(p.summary.textContent, `1 passed, 1 failed (${report.duration.toFixed(1)} ms)`) - assertStructurallySame([...statuses(p.results)], ['passed', 'failed']) - const event = assertNotNullish(p.view.events[0]) - assertEq(event.type, 'fjs-browser-test-complete') - assertEq(event.detail, report) - assertEq(await p.view.fjsBrowserTestReport, report) + const proof = Object.fromEntries( + [...new Array(60).keys()].map(i => [`t${i}`, () => undefined])) + const report = await run(proof) + assertEq(report.totals.tests, 60) + assertEq(report.totals.passed, 60) }, - renderWithoutView: async () => { - // A detached document has no window: the run still renders, and - // nothing is published or announced. - const p = page(false) - const report = await startBrowserTests(p.root, [['m', { ok: () => undefined }]]) + // A root whose document has no window still runs and still answers: there + // is simply nowhere to publish the promise or dispatch the event. + withoutView: async () => { + const { root, view } = page(false) + const report = await startBrowserTestSources(root, ['a'], async () => ({ + proof: { x: () => undefined }, + })) assertEq(report.status, 'passed') - assertEq(p.summary.textContent, `1 passed, 0 failed (${report.duration.toFixed(1)} ms)`) - assertEq(p.view.events.length, 0) - assertEq(p.view.fjsBrowserTestReport, undefined) + assertEq(report.browser, '') + assertEq(view.events.length, 0) + assertEq(view.fjsBrowserTestReport, undefined) }, - renderReport: () => { - // The renderer is exported on its own for a controller that already - // holds a report. - const p = page() - renderBrowserReport(p.root, { + // A root with none of the page's elements is rendered into without a throw: + // an embedder may host the runner in a bare container. + renderWithoutElements: () => { + const { root, states } = page() + root.replaceChildren() + renderBrowserReport(root, { status: 'passed', - browser: 'test', - totals: { tests: 1, passed: 1, failed: 0 }, - duration: 1, - results: [{ module: 'm', path: '.t', status: 'passed', duration: 0.5 }], + browser: 'x', + totals: { tests: 0, passed: 0, failed: 0 }, + duration: 0, + results: [], }) - assertEq(p.summary.textContent, '1 passed, 0 failed (1.0 ms)') - assertEq(p.results.children[0]?.textContent, 'PASS m .t (0.5 ms)') + assertEq(states.join(','), 'passed') }, - sources: async () => { - const p = page() - const report = await startBrowserTestSources(p.root, ['a.mjs', 'b.mjs'], - source => Promise.resolve({ proof: { [source]: () => undefined } })) - assertEq(report.status, 'passed') - assertEq(report.totals.tests, 2) - assertStructurallySame([...p.states], ['loading', 'running', 'passed']) - assertEq(await p.view.fjsBrowserTestReport, report) - }, - sourcesLoadingSummaryIsSynchronous: () => { - // The summary must not keep showing idle text through loading: it is - // replaced the instant a run starts, before any import has had a - // chance to settle — even one that never does. - const p = page() - void startBrowserTestSources(p.root, ['a.mjs', 'b.mjs'], () => new Promise(() => undefined)) - assertEq(p.summary.textContent, 'Loading 0/2') - }, - sourcesProgress: async () => { - const p = page() - /** @type {(module: { readonly proof?: unknown }) => void} */ - let release = () => undefined - /** @type {Promise<{ readonly proof?: unknown }>} */ - const pending = new Promise(resolve => { release = resolve }) - const done = startBrowserTestSources(p.root, ['a.mjs', 'b.mjs'], - source => source === 'a.mjs' ? Promise.resolve({ proof: {} }) : pending) - await Promise.resolve() - await Promise.resolve() - assertEq(p.summary.textContent, 'Loading 1/2: a.mjs') - release({ proof: {} }) - assertEq((await done).status, 'passed') - }, - sourcesImporterThrows: async () => { - // An importer that throws before it returns a promise is a loader - // failure like any other: the page must not be left in `loading` with - // no report and no completion event. - const p = page() - const report = await startBrowserTestSources(p.root, ['bad.mjs'], - source => { throw new Error(`no loader for ${source}`) }) - assertEq(report.status, 'infrastructure-error') - assertStructurallySame({ ...report.totals }, { tests: 1, passed: 0, failed: 1 }) - assertEq(report.results[0]?.message, 'no loader for bad.mjs') - assertStructurallySame([...p.states], ['loading', 'infrastructure-error']) - assertEq(p.view.events.length, 1) - }, - runControlAbsentButtonIsIgnored: async () => { - // An embedding root with no `[data-test-run]` control is still - // supported: `setState` finds nothing to toggle and moves on rather - // than throwing. - /** @type {string[]} */ - const states = [] - /** @type {_Document} */ - const document = { - defaultView: null, - createElement: tag => element(document, tag, [], states), - } - const root = element(document, 'main', ['data-browser-tests'], states) - root.replaceChildren( - element(document, 'p', ['data-test-summary'], states), - element(document, 'ol', ['data-test-results'], states)) - const report = await startBrowserTests(/** @type {Element} */ (/** @type {unknown} */ (root)), - [['m', { ok: () => undefined }]]) - assertEq(report.status, 'passed') - }, - runControlDisabledWhileActive: async () => { - // `Run` must be passive — genuinely disabled, not just click-ignoring — - // for the whole span between a click and the next terminal state: - // through loading and through execution. - const p = page() - /** @type {(module: { readonly proof?: unknown }) => void} */ - let release = () => undefined - /** @type {Promise<{ readonly proof?: unknown }>} */ - const pending = new Promise(resolve => { release = resolve }) - const done = startBrowserTestSources(p.root, ['a.mjs'], () => pending) - await Promise.resolve() - assertEq(p.states[0], 'loading') - assertEq(p.runButton.attributes.has('disabled'), true) - release({ proof: { t: () => undefined } }) - await Promise.resolve() - await Promise.resolve() - assertEq(p.runButton.attributes.has('disabled'), true) - const report = await done - assertEq(report.status, 'passed') - // Terminal state hands control back: a new run can be started. - assertEq(p.runButton.attributes.has('disabled'), false) - }, - runControlReenabledAfterFailure: async () => { - // A failed or infrastructure-error run is just as terminal as a passed - // one: `Run` reactivates either way. - const p = page() - const report = await startBrowserTestSources(p.root, ['bad.mjs'], - source => Promise.reject(new Error(`offline: ${source}`))) - assertEq(report.status, 'infrastructure-error') - assertEq(p.runButton.attributes.has('disabled'), false) - }, - runControlNewRunAfterCompletion: async () => { - // The same action starts every run: nothing but the `Run` control's - // own state stands between a completed run and the next one. - const p = page() - await startBrowserTestSources(p.root, ['a.mjs'], - () => Promise.resolve({ proof: { t: () => undefined } })) - assertEq(p.runButton.attributes.has('disabled'), false) - const second = await startBrowserTestSources(p.root, ['a.mjs'], - () => Promise.resolve({ proof: { t: () => undefined } })) - assertEq(second.status, 'passed') - assertStructurallySame([...p.states], - ['loading', 'running', 'passed', 'loading', 'running', 'passed']) - }, - sourcesLoadFailure: async () => { - const p = page() - const report = await startBrowserTestSources(p.root, ['ok.mjs', 'bad.mjs'], - source => source === 'bad.mjs' - ? Promise.reject(new Error('offline')) - : Promise.resolve({ proof: { t: () => undefined } })) - assertEq(report.status, 'infrastructure-error') - // The totals have to agree with `results`: a consumer reading - // `0 of 0` would take a broken suite for an empty one. - assertStructurallySame({ ...report.totals }, { tests: 1, passed: 0, failed: 1 }) - assertEq(report.results[0]?.module, 'bad.mjs') - assertEq(report.results[0]?.message, 'offline') - assertStructurallySame([...p.states], ['loading', 'infrastructure-error']) - assert(p.summary.textContent.startsWith('Infrastructure error: 1 failed to load'), - p.summary.textContent) - assertStructurallySame([...statuses(p.results)], ['failed']) - assertEq(p.view.events.length, 1) + operations: { + // `fetch` reads a `data:` URL rather than a network one, so the proof + // stays offline while still going through the realm's own `fetch`. + fetch: async () => { + const r = await fetchOp('data:text/plain,ok') + assert(r[0] === 'ok', r) + }, + fetchFailure: async () => { + const r = await fetchOp('not-a-scheme://x') + assert(r[0] === 'error', r) + assertEq(r[1][0], 'ioError') + }, + import: async () => { + const r = await importOp('./x.mjs') + assertEq(/** @type {Module} */ (unwrap(r)).source, './x.mjs') + }, + awaitsPromise: async () => { + assertEq(unwrap(await awaitOp(Promise.resolve(7)))[0], 7) + }, + awaitsPlainValue: async () => { + assertEq(unwrap(await awaitOp(7))[0], 7) + }, + now: async () => { + assert(unwrap(await now()) > 0) + }, + sandboxMeasures: async () => { + const { result, duration } = unwrap(await sandbox(() => 1)) + assertEq(unwrap(result), 1) + assert(duration >= 0, duration) + }, + all: async () => { + const results = unwrap(await all(pureOk(1), pureOk(2))) + assertEq(results.map(unwrap).join(','), '1,2') + }, }, } diff --git a/fjs/emergent_testing/browser/species.proof.mjs b/fjs/emergent_testing/browser/species.proof.mjs deleted file mode 100644 index 11303e009..000000000 --- a/fjs/emergent_testing/browser/species.proof.mjs +++ /dev/null @@ -1,45 +0,0 @@ -import { assertEq } from '../../asserts/module.f.mjs' -import { runBrowserProofs } from '../browser.mjs' - -/** - * A genuine promise whose `then` always throws: the result promise is built - * through `constructor[Symbol.species]`, and this `constructor` has none to - * give. `configurable` decides whether the runner can shadow the property for - * the length of one subscription. - * - * @type {(configurable: boolean) => Promise} - */ -const throwingSpeciesPromise = configurable => { - const promised = Promise.resolve({ - child: () => { throw 'boom' }, - }) - const constructor = {} - Object.defineProperty(constructor, Symbol.species, { - get: () => { throw new Error('species') }, - }) - Object.defineProperty(promised, 'constructor', { value: constructor, configurable }) - return promised -} - -/** @type {(promised: Promise) => ReturnType} */ -const run = promised => runBrowserProofs([['proof', { nested: () => promised }]]) - -export const proof = { - throwingSpecies: async () => { - // The intrinsic Promise shadows the hostile `constructor` while the - // handlers are attached, so the resolved sub-tree still runs. - const report = await run(throwingSpeciesPromise(true)) - assertEq(report.totals.tests, 2) - assertEq(report.totals.failed, 1) - assertEq(report.results[1]?.path, '.nested().child') - }, - pinnedThrowingSpecies: async () => { - // Nothing to shadow, so the promise can never be subscribed to. The - // test that produced it fails, rather than passing on a result the - // runner never observed. - const report = await run(throwingSpeciesPromise(false)) - assertEq(report.totals.tests, 1) - assertEq(report.totals.failed, 1) - assertEq(report.results[0]?.message, 'species') - }, -} diff --git a/fjs/emergent_testing/browser/types.ts b/fjs/emergent_testing/browser/types.ts new file mode 100644 index 000000000..f947e00c7 --- /dev/null +++ b/fjs/emergent_testing/browser/types.ts @@ -0,0 +1,74 @@ +/** + * Types for the browser proof application. + * + * @module + */ + +import type { CommonOp, Module } from '../../effects/common/types.ts' +import type { Effect } from '../../effects/types.ts' +import type { IoResult } from '../../effects/common/types.ts' +import type { ReportOp, TestResult } from '../types.ts' + +/** + * The operations the browser application performs: the host-independent set + * every runner implements, plus the two that record normalized results. + * + * There is nothing browser-specific in it, and that is the design rather than + * an accident — the DOM is the *adapter's* business + * ([`./module.mjs`](./module.mjs)), never the application's. A page, a proof + * with a stand-in interpreter, and a future headless controller therefore run + * the very same program. + */ +export type BrowserOp = CommonOp | ReportOp + +/** + * How a whole run ended. `infrastructure-error` is not a third kind of test + * failure: it says the suite never got to run — a module that would not link, a + * runner missing an operation — which an automated consumer must not read as + * "the proofs failed". + */ +export type ReportStatus = 'passed' | 'failed' | 'infrastructure-error' + +/** + * The serializable answer of a run, independent of the runner that produced it + * and of the page that rendered it. + */ +export type BrowserTestReport = { + readonly status: ReportStatus + readonly browser: string + readonly totals: { + readonly tests: number + readonly passed: number + readonly failed: number + } + readonly duration: number + readonly results: readonly TestResult[] +} + +/** + * What the host supplies to a run: the proof modules to link, and the name to + * record the realm under. + * + * `browser` is data rather than a `navigator` read, for the reason every other + * capability here is an operation — the application must be runnable outside a + * browser, and a proof that had to install a global `navigator` to check a + * report would be testing the stub. + */ +export type BrowserOptions = { + readonly browser: string + readonly sources: readonly string[] +} + +/** + * A run: options in, a report out. + * + * **The error channel is `never`**, and it is earned rather than asserted: a + * module that will not link and an operation the runner lacks are both + * *reported*, as an `infrastructure-error` report. A page waiting on the run + * has nowhere to put a failure — leaving it in `running` with no report and no + * completion event is the one outcome an automated controller cannot act on. + */ +export type BrowserProgram = (options: BrowserOptions) => Effect + +/** @internal One source paired with what linking it answered. */ +export type _Loaded = readonly[string, IoResult] diff --git a/fjs/emergent_testing/module.f.mjs b/fjs/emergent_testing/module.f.mjs index a32868040..9c8d025c1 100644 --- a/fjs/emergent_testing/module.f.mjs +++ b/fjs/emergent_testing/module.f.mjs @@ -2,25 +2,34 @@ * Test-framework helpers for running and reporting FunctionalScript tests. * * Two parallel execution paths: - * - `runModule` / `Reporter` — self-hosted Effects runner used by `fjs t`; - * sandboxes each leaf call individually and accumulates `TestState`. + * - `runModule` / `Reporter` — self-hosted Effects runner; sandboxes each + * leaf call individually and accumulates `TestState`. **Both** `fjs t` and + * the browser runner (`./browser/module.f.mjs`) go through it: proof-tree + * walking, the structural `throw` expectation, promise resolution, path + * formatting and the totals are decided here once, and each host differs only + * in its `Reporter` and in the runner that interprets `sandbox`. * - `registerModule` / `TestContext` — registers tests with an external * framework (Node `--test`, Bun, Deno) at import time; the framework owns * scheduling and pass/fail counting. * + * `recordingReporter` is the host-independent reporter of the first path: it + * normalizes each leaf into a `TestResult` carrying no terminal text and no DOM + * and hands it to the `report` operation, leaving presentation to the host. + * * @module * * @import { Operation } from '../effects/types.ts' - * @import { Effect, NotImplemented } from '../effects/types.ts' + * @import { Effect, Func, NotImplemented } from '../effects/types.ts' * @import { LoadModuleOperations, ModuleMap } from '../dev/types.ts' - * @import { TestFn, TestEntry, TestSet, Path, Reporter, _TestState, _TestAndPath } from './types.ts' + * @import { Report, Reported, TestFn, TestEntry, TestResult, TestSet, Path, Reporter, _TestState, _TestAndPath } from './types.ts' * @import { All, Await, Env, IoChannel, NodeProgram, NodeProgramOptions, Program, Sandbox, SandboxResult, Test, TestContext, Write, WriteConsoles } from '../effects/node/types.ts' */ import { reset, fgGreen, fgRed, bold, csiWrite } from '../text/sgr/module.f.mjs' -import { allOk, awaitIfPromise, errorExit, errorMessage, errorSummary, exitStep, sandbox, test } from '../effects/node/module.f.mjs' +import { allOk, awaitIfPromise, sandbox } from '../effects/common/module.f.mjs' +import { errorExit, errorMessage, errorSummary, exitStep, test } from '../effects/node/module.f.mjs' import { - catchStep, history, historyStep, mapStep, pureError, pureOk, resultStep, step, + catchStep, do_, history, historyStep, mapStep, pureError, pureOk, resultStep, step, } from '../effects/module.f.mjs' import { loadModuleMap } from '../dev/module.f.mjs' import { invert } from '../types/result/module.f.mjs' @@ -369,6 +378,82 @@ export const ghEscape = s => export const defaultTest = (file, path, { fn, throws }) => mapStep(sandbox(fn), r => throws ? { ...r, result: invert(r.result) } : r) +/** What a `throws` leaf that returned cleanly is reported as. */ +const expectedThrow = 'Expected the proof to throw' + +/** + * The message and stack to report a thrown value by. + * + * An `Error` thrown from another realm — an iframe, a worker — is not + * `instanceof Error` here, and its stack is the very thing a report exists to + * carry. What the fields say is therefore the test, not where the value was + * made: anything carrying `message` or `stack` is read as the failure it + * describes, and everything else by its own text. + * + * @type {(error: unknown) => readonly[string, string]} + */ +export const errorDetails = error => { + if (error !== null && (typeof error === 'object' || typeof error === 'function') + && ('message' in error || 'stack' in error)) { + const { message, stack } = /** @type {{ readonly message?: unknown, readonly stack?: unknown }} */ (error) + const described = String(message) + return [described, stack === undefined ? described : String(stack)] + } + const fallback = String(error) + return [fallback, fallback] +} + +/** + * Normalizes one leaf outcome into the {@link TestResult} every reporter + * renders from. + * + * `r` is what {@link Reporter.test} answered, so a `throws` leaf has already + * been inverted by {@link defaultTest}: an `error` there means the proof + * returned when it was expected to throw, which is why that case is named + * rather than described by the value it returned. + * + * @type {(file: string, path: Path, r: SandboxResult, throws: boolean) => TestResult} + */ +export const testResult = (file, path, { result, duration }, throws) => { + const [status, value] = result + const common = { module: file, path: fmtPath(path), duration } + if (status === 'ok') { return { ...common, status: 'passed' } } + const [message, stack] = throws ? [expectedThrow, ''] : errorDetails(value) + return { ...common, status: 'failed', message, stack } +} + +/** Records one normalized leaf result as it lands. + * + * @type {Func} + */ +export const report = do_('report') + +/** Reads back every result {@link report} has recorded. + * + * @type {Func} + */ +export const reported = do_('reported') + +/** + * The reporter that answers in {@link TestResult}s instead of rendering: each + * leaf is normalized and handed to the {@link report} operation, and the run's + * consumer reads the sequence back with {@link reported}. + * + * **Its `summary` writes nothing**, and that is not an omission. Pass, fail and + * total are `results.length` and a count of the failed ones, so a summary event + * would restate what the recorded results already say — and a consumer that + * derives them cannot disagree with itself about how many tests ran. The + * terminal reporter keeps its own `summary` because a line of text is genuinely + * not derivable from the results a user has already scrolled past. + * + * @type {Reporter} + */ +export const recordingReporter = { + result: (file, path, r, throws) => report(testResult(file, path, r, throws)), + summary: () => pureOk(undefined), + test: defaultTest, +} + /** @type {(file: string, path: Path, color: string, label: string, duration: number) => string} */ const fmtResultLine = (file, path, color, label, duration) => `${fmtImport(file, path)}: ${color}${label}${reset}, ${timeFormat(duration)}` diff --git a/fjs/emergent_testing/proof.f.mjs b/fjs/emergent_testing/proof.f.mjs index 2e19f7668..ed5d8fcf3 100644 --- a/fjs/emergent_testing/proof.f.mjs +++ b/fjs/emergent_testing/proof.f.mjs @@ -13,8 +13,8 @@ import { log } from '../effects/node/module.f.mjs' import { defaultNodeProgramOptions, emptyState, virtual } from '../effects/node/virtual/module.f.mjs' import { assert, assertEq, todo } from '../asserts/module.f.mjs' import { - testAll, fmtPath, fmtTerm, fmtImport, ghEscape, isInteger, isIdentifier, - registerModule, parseTestSet, + testAll, errorDetails, fmtPath, fmtTerm, fmtImport, ghEscape, isInteger, isIdentifier, + registerModule, parseTestSet, testResult, defaultTest, main, register, } from './module.f.mjs' import { run as mockRun } from '../effects/mock/module.f.mjs' @@ -597,6 +597,64 @@ export const helpers = { assertEq(ghEscape('a\r\nb'), 'a%0D%0Ab') assertEq(ghEscape('a%b:c,d'), 'a%25b%3Ac%2Cd') }, + errorDetails: { + // Read structurally rather than by `instanceof Error`, so an error from + // another realm still reports its own stack. + messageAndStack: () => { + const [message, stack] = errorDetails({ message: 'boom', stack: 'boom\n at x' }) + assertEq(message, 'boom') + assertEq(stack, 'boom\n at x') + }, + withoutStack: () => { + const [message, stack] = errorDetails({ message: 'no stack' }) + assertEq(message, 'no stack') + assertEq(stack, 'no stack') + }, + // A value carrying only a stack is still a failure description; the + // message it does not have reads as the absent value it is. + stackOnly: () => { + const [message, stack] = errorDetails({ stack: 'trace' }) + assertEq(message, 'undefined') + assertEq(stack, 'trace') + }, + // A thrown *function* is an object as far as this reading goes. + callable: () => { + const [message] = errorDetails(Object.assign(() => undefined, { message: 'fn' })) + assertEq(message, 'fn') + }, + plainValue: () => { + const [message, stack] = errorDetails('just text') + assertEq(message, 'just text') + assertEq(stack, 'just text') + }, + nullValue: () => { + assertEq(errorDetails(null)[0], 'null') + }, + }, + testResult: { + passed: () => { + const r = testResult('a.f.mjs', ['x'], { result: ok(1), duration: 2 }, false) + assertEq(r.module, 'a.f.mjs') + assertEq(r.path, '.x') + assertEq(r.status, 'passed') + assertEq(r.duration, 2) + assertEq(r.message, undefined) + }, + failed: () => { + const r = testResult('a.f.mjs', ['x'], { result: error(new Error('bad')), duration: 0 }, false) + assertEq(r.status, 'failed') + assertEq(r.message, 'bad') + }, + // `defaultTest` has already inverted a `throws` leaf, so an `error` here + // means it returned when it was expected to throw — named rather than + // described by whatever it happened to return. + expectedToThrow: () => { + const r = testResult('a.f.mjs', ['throw', 'x'], { result: error(7), duration: 0 }, true) + assertEq(r.status, 'failed') + assertEq(r.message, 'Expected the proof to throw') + assertEq(r.stack, '') + }, + }, parseTestSet: { nullReturnsEmpty: () => { const result = parseTestSet(false, null) diff --git a/fjs/emergent_testing/todo/browser-test-controls.md b/fjs/emergent_testing/todo/browser-test-controls.md index 77c391782..41d4f2913 100644 --- a/fjs/emergent_testing/todo/browser-test-controls.md +++ b/fjs/emergent_testing/todo/browser-test-controls.md @@ -66,5 +66,5 @@ module or a default query parameter. - [Browser testing](browser-testing.md) — the shared browser application and report contract. -- [Shared browser/console runner core](share-browser-console-runner.md) — future - separation of pure runner state from DOM controls. +- [`emergent_testing/browser`](../browser/module.f.mjs) — the pure application + the controls drive; runner state is already separate from DOM presentation. diff --git a/fjs/emergent_testing/todo/browser-testing.md b/fjs/emergent_testing/todo/browser-testing.md index d81dc562b..81daec8fd 100644 --- a/fjs/emergent_testing/todo/browser-testing.md +++ b/fjs/emergent_testing/todo/browser-testing.md @@ -48,7 +48,7 @@ three independent test frameworks. eventual isolated browser-test application root ├── index.html ├── _browser-test-entry.mjs -├── fjs/emergent_testing/browser.mjs +├── fjs/emergent_testing/browser/module.mjs └── authored or copied .f.mjs / .mjs modules ``` @@ -147,8 +147,9 @@ workers, or visual regression testing. - [ ] Create the JavaScript-only application root with a generated entry module covering every accepted module. - [x] Implement the first browser-compatible emergent-test runner and report - API; follow up by sharing its pure semantics with `fjs t` in - [share-browser-console-runner](share-browser-console-runner.md). + API, and share its proof semantics with `fjs t`: both runners now walk + proof trees through `emergent_testing/module.f.mjs` and differ only in + their `Reporter` and their effect interpreter. - [x] Implement the HTML UI and integrate it into the FunctionalScript website. - [ ] Add shared controller code for preparation, serving, report validation, @@ -163,7 +164,7 @@ workers, or visual regression testing. ### Related - [`.f.mjs` proof discovery and coverage](f-mjs-test-and-coverage.md) -- [Shared browser/console runner core](share-browser-console-runner.md) +- [Hostile thrown values and cross-realm promises](hostile-proof-values.md) - [Explicit browser test controls](browser-test-controls.md) - [authored `.f.mjs` package support](../../ci/todo/f-mjs-package-support.md) - [project roadmap](../../../todo/plan/roadmap.md) diff --git a/fjs/emergent_testing/todo/hostile-proof-values.md b/fjs/emergent_testing/todo/hostile-proof-values.md new file mode 100644 index 000000000..8bdaf598a --- /dev/null +++ b/fjs/emergent_testing/todo/hostile-proof-values.md @@ -0,0 +1,62 @@ +## Hostile thrown values and cross-realm promises kill a run + +**Priority:** P3 +**Status:** open + +### Problem + +Both runners now share one core (`../module.f.mjs`), so they also share two +weaknesses the core cannot fix on its own. Neither is reachable from ordinary +FunctionalScript, and both were reachable — and covered — by the browser runner +before it and `fjs t` were unified; unifying adopted `fjs t`'s semantics +deliberately, so this file is where the difference went rather than being +silently dropped. + +**A thrown value that resists being read takes the run down.** `errorDetails` +reads `message` and `stack` and calls `String`, and a revoked `Proxy`, a +throwing accessor, or a `toString` that panics makes any of those throw. There +is no `try`/`catch` in FunctionalScript, so the shared core cannot guard it, and +the panic escapes the reporter — the run ends with no report at all rather than +one failed test. `fjs t` has always had this exposure (its reporter interpolates +the thrown value into a line); the browser runner used to defend against it in +impure code, and no longer does. + +**A promise from another realm is not awaited.** Both `sandbox` interpreters ask +`p instanceof Promise`, which is false for a promise built in an iframe, a +worker, or a `node:vm` context. Such a value is walked as an ordinary proof tree +instead, so a *rejected* cross-realm promise is reported as a pass. The obvious +repair — brand-checking with `Object.prototype.toString` — is not one: the tag +is settable through `Symbol.toStringTag`, and an object carrying a `then` proof +would then be assimilated, breaking the rule that only actual promises are +asynchronous values. + +### Preliminary design + +Both belong to the *operation*, not to the shared core, which is what makes one +fix serve every runner: + +- Normalization could move behind `sandbox`: the operation already runs user + code inside the host's `try`/`catch`, so it is the one place that can read a + hostile value safely and hand back a `message`/`stack` pair that is already + ordinary data. The shared `errorDetails` would then read a record rather than + an arbitrary thrown value, and stay total. +- The brand check needs a test that a page cannot forge and that no proof tree + can pass by accident. Candidates: `Promise.resolve(p) === p` on the value's + own constructor, or asking each realm the runner knows about. Whatever is + chosen must be one function both interpreters call, or the two drift again. + +Neither is worth doing speculatively. Do the first when a real proof loses a +run to it, and the second when proofs genuinely execute in more than one realm — +which is the point [browser-testing](browser-testing.md) reaches with iframes or +workers. + +### Constraints + +- Whatever is added must apply to `fjs t` and to the browser runner alike; + a defense in one runner only is what this repository just finished removing. +- An object carrying a `then` proof property must stay an ordinary proof tree. + +### Related + +- [Browser testing](browser-testing.md) +- [Test-runner behavior](661-test-runner-behavior.md) diff --git a/fjs/emergent_testing/todo/share-browser-console-runner.md b/fjs/emergent_testing/todo/share-browser-console-runner.md deleted file mode 100644 index 771806229..000000000 --- a/fjs/emergent_testing/todo/share-browser-console-runner.md +++ /dev/null @@ -1,146 +0,0 @@ -## Share the browser and console proof runners - -**Priority:** P3 -**Status:** open - -### Problem - -The browser runner and `fjs t` currently implement the same proof semantics in -different places. In particular, both must discover zero-argument leaves, walk -returned proof trees, propagate the structural `throw` expectation, await real -promises, format paths, count results, and distinguish proof failures from -runner failures. Keeping those rules in `emergent_testing/browser.mjs` and -`emergent_testing/module.f.mjs` independently invites behavioral drift. - -The current browser file also mixes three layers: - -1. pure proof-tree and result logic; -2. browser operations such as time, yielding, and module loading; -3. DOM rendering and global/event integration. - -That makes the reusable semantics harder to see and leaves the impure browser -entry much larger than it needs to be. - -### Preliminary design - -Share semantics, not host mechanics. The console runner should keep using the -Node Effects runner and the browser should keep executing proof bodies in the -browser realm; neither runner should call through the other host's adapter. - -The intended layout is: - -```text -fjs/emergent_testing/ -├── module.f.mjs shared proof semantics used by every runner -├── browser/ -│ ├── module.f.mjs pure browser application/effect composition -│ └── module.mjs minimal browser host runner and DOM integration -└── ... existing console/external-runner adapters - -fjs/effects/browser/ browser operations and interpreter, only if useful -├── module.f.mjs operation constructors/composition -├── module.mjs browser interpreter -└── types.ts operation types -``` - -Website preparation follows the same boundary. Restore the package command to -the FunctionalScript entry point: - -```json -"website": "node ./fjs/module.mjs r ./fjs/website/module.f.mjs" -``` - -`fjs/website/module.f.mjs` must own proof discovery, manifest generation, and -HTML/entry generation as one `NodeProgram`. Do not invoke a non-FunctionalScript -preparation script such as `website/browser-prepare.mjs` directly from an npm -script. If preparation needs a Node capability that the FunctionalScript -program cannot currently express, add the smallest operation to -`fjs/effects/node/` and its real and virtual interpreters instead of bypassing -Effects. Existing `readdir`, `readFile`, and `writeFile` operations should be -reused where sufficient. - -Move `emergent_testing/browser.mjs` to -`emergent_testing/browser/module.mjs`. It should become a thin impure shell: -provide browser capabilities, start the pure program, render semantic events, -publish `window.fjsBrowserTestReport`, and dispatch the completion event. Pure -code belongs in `emergent_testing/browser/module.f.mjs` or in the shared -`emergent_testing/module.f.mjs`, depending on whether console runners can use -it. - -Extract or reuse these host-independent concepts first: - -- proof-tree parsing and recursive path handling (`collectTests` already exists - and should be the source of truth rather than being copied); -- expected-throw semantics; -- normalized per-test results and total/result reducers; -- report status and infrastructure-error classification; -- semantic progress events, independent of terminal text or DOM elements. - -Keep host capabilities at the leaves. Candidate browser effects are module -import, monotonic time, event-loop yield, and report publication. DOM node -construction may instead remain in the small `module.mjs` adapter if making it -an effect adds an operation for every DOM detail without improving the shared -API. Add `fjs/effects/browser/` only after the required operation set is clear; -do not create a mirror of `effects/node` merely for directory symmetry. - -An executor boundary will still be necessary because the console runner uses -the Effects sandbox while a browser catches synchronous throws and awaits -native promises. That boundary should answer one normalized leaf result. Tree -walking, throw inversion, aggregation, and reporting policy stay above it and -are shared. - -### Constraints - -- Preserve the recursive proof semantics and totals of `fjs t` exactly, - including objects with a proof property named `then`; only actual promises - are asynchronous values. -- Browser modules must not import Node built-ins, the Node effect interpreter, - `node:test`, or Playwright. -- Website build-time filesystem access must be expressed by the FunctionalScript - `NodeProgram` through Node effects; npm scripts must not run an impure helper - as a second application entry point. -- The browser host runner must remain usable as native JavaScript with no - bundling or transpilation. -- Pure `.f.mjs` additions require co-located proofs with complete line, - function, and branch coverage. -- Keep the serializable browser report, documented promise, and completion - event compatible unless a simpler shared report API deliberately replaces - all callers in the same change. -- Do not move terminal formatting or DOM presentation into the shared semantic - core. - -### Tasks - -- [ ] Inventory duplicated semantics in `emergent_testing/module.f.mjs` and - `emergent_testing/browser.mjs`, and define the smallest shared API. -- [ ] Make the existing `collectTests`/path behavior the single source of truth - for console and browser execution. -- [ ] Define normalized leaf, progress, infrastructure-error, totals, and report - values without terminal or DOM fields. -- [ ] Decide whether browser import/time/yield/publication justify - `fjs/effects/browser/`; document the decision before adding operations. -- [ ] Move static proof discovery and `_browser-suite.mjs` generation into - `fjs/website/module.f.mjs`; extend `fjs/effects/node/` only for a concrete - missing capability and prove the real and virtual interpretations. -- [ ] Delete `fjs/website/browser-prepare.mjs` and make the sole `website` - command `node ./fjs/module.mjs r ./fjs/website/module.f.mjs` once the - FunctionalScript generator owns the complete build; do not restore the - removed `index-html` alias. -- [ ] Add `emergent_testing/browser/module.f.mjs` for pure browser application - composition and its complete proof. -- [ ] Move the current browser host code to - `emergent_testing/browser/module.mjs` and reduce it to capability - interpretation, DOM rendering, and browser publication. -- [ ] Update the generated website entry and browser-test application imports - to the new module paths. -- [ ] Prove both runners produce equivalent paths, throw outcomes, recursive - test counts, and normalized failures from the same fixtures. - -### Related - -- [Browser testing](browser-testing.md) — browser-native application and runner - requirements. -- [Test-runner behavior](661-test-runner-behavior.md) — documented differences - that must remain intentional after sharing the core. -- [Test tree walker](65z-tf-test-tree-walker.md) — earlier work around recursive - proof-tree traversal. diff --git a/fjs/emergent_testing/types.ts b/fjs/emergent_testing/types.ts index a3274ae6a..68d18bb58 100644 --- a/fjs/emergent_testing/types.ts +++ b/fjs/emergent_testing/types.ts @@ -5,7 +5,7 @@ */ import type { Effect, Operation } from '../effects/types.ts' -import type { IoChannel, SandboxResult } from '../effects/node/types.ts' +import type { IoChannel, OpResult, SandboxResult } from '../effects/common/types.ts' /** A zero-argument test function whose return value may contain sub-tests. */ export type TestFn = () => unknown @@ -68,6 +68,47 @@ export type Reporter = { readonly test: (file: string, path: Path, set: TestEntry) => Effect, IoChannel> } +/** How a leaf test ended. */ +export type TestStatus = 'passed' | 'failed' + +/** + * One leaf result, normalized: which module it came from, the property chain + * that names it, how it ended, and how long it took. A failure also carries the + * message and stack it should be reported by. + * + * **It carries no terminal text and no DOM.** This is what a runner *observes*, + * so every reporter can render it its own way — coloured lines on a TTY, a + * `::error` annotation on GitHub, a list item in a page — and an automated + * consumer can read it off the wire. `path` is already rendered by + * {@link fmtPath} rather than left as a `Path`, because the chain is what a + * reader identifies the test by and nothing downstream walks it again. + */ +export type TestResult = { + readonly module: string + readonly path: string + readonly status: TestStatus + readonly duration: number + readonly message?: string + readonly stack?: string +} + +/** + * Records one normalized leaf result the moment it lands. + * + * It is an *operation* rather than a value threaded through the run because the + * results arrive concurrently: `all` performs a module's leaves at once, so a + * read-modify-write over shared memory would interleave and lose them. A + * runner's handler appends in one step, and {@link Reported} reads the whole + * sequence back once the run is over. + */ +export type Report = readonly['report', (result: TestResult) => OpResult] + +/** Every result {@link Report} has recorded, in the order they landed. */ +export type Reported = readonly['reported', () => OpResult] + +/** The pair of operations a recording runner implements. */ +export type ReportOp = Report | Reported + /** @internal */ export type _TestState = { readonly time: number, diff --git a/fjs/website/module.f.mjs b/fjs/website/module.f.mjs index 24d569c1d..f18dd4437 100644 --- a/fjs/website/module.f.mjs +++ b/fjs/website/module.f.mjs @@ -49,7 +49,7 @@ pre { white-space: pre-wrap } ['script', { type: 'module', src: './_browser-test-entry.mjs' }] ) -const entry = utf8(`import { startBrowserTestSources } from './fjs/emergent_testing/browser.mjs' +const entry = utf8(`import { startBrowserTestSources } from './fjs/emergent_testing/browser/module.mjs' import { browserProofSources } from './fjs/emergent_testing/_browser-suite.mjs' const root = /** @type {Element} */ (document.querySelector('[data-browser-tests]')) diff --git a/fjs/website/todo/generate-website.md b/fjs/website/todo/generate-website.md index 959fd8231..59bb1817c 100644 --- a/fjs/website/todo/generate-website.md +++ b/fjs/website/todo/generate-website.md @@ -12,4 +12,4 @@ - [x] Browser test runner and proof-result UI - [ ] Move browser-manifest preparation into the website `NodeProgram` through Node effects, as designed in - [share-browser-console-runner](../../emergent_testing/todo/share-browser-console-runner.md) + [website-preparation-program](website-preparation-program.md) diff --git a/fjs/website/todo/website-preparation-program.md b/fjs/website/todo/website-preparation-program.md new file mode 100644 index 000000000..95abebc68 --- /dev/null +++ b/fjs/website/todo/website-preparation-program.md @@ -0,0 +1,69 @@ +## Own the browser-suite preparation from the website `NodeProgram` + +**Priority:** P3 +**Status:** open + +### Problem + +`npm run website` runs `fjs/website/browser-prepare.mjs`, an impure Node script +that is a second application entry point beside the FunctionalScript program in +`fjs/website/module.f.mjs`. It walks the source tree, decides which proof +modules a browser can link, writes `fjs/emergent_testing/_browser-suite.mjs`, +and only then calls `run(main)` to emit the page. Everything it does — reading +directories, reading files, writing generated source — is expressible as Node +effects, so the split exists for no reason other than history, and the +preparation half is proved only through `browser-source.proof.mjs`'s unit tests +of the token scanner rather than end to end against the virtual filesystem. + +The [shared browser/console runner](../../emergent_testing/README.md) work that +this issue was carved out of is done: the browser and `fjs t` now run the same +proof semantics, so what is left here is the *build*, not the runner. + +### Preliminary design + +Restore the package command to the FunctionalScript entry point: + +```json +"website": "node ./fjs/module.mjs r ./fjs/website/module.f.mjs" +``` + +`fjs/website/module.f.mjs` must own proof discovery, manifest generation, and +HTML/entry generation as one `NodeProgram`. If preparation needs a Node +capability that the FunctionalScript program cannot currently express, add the +smallest operation to `fjs/effects/node/` and its real and virtual interpreters +instead of bypassing Effects. Existing `readdir`, `readFile`, and `writeFile` +operations should be reused where sufficient. + +`fjs/website/browser-source.mjs` — the token scanner answering "does this +module export `proof`?" and "which modules does it import?" — is already pure +and has no `try`/`catch` or regular expressions. Renaming it to `.f.mjs` and +proving it as authored FunctionalScript is the first step; the graph walk and +the blocker classification then move into the program beside it. + +### Constraints + +- Website build-time filesystem access must be expressed by the FunctionalScript + `NodeProgram` through Node effects; npm scripts must not run an impure helper + as a second application entry point. +- The generated manifest and page must stay byte-identical across the move, so + the change is provably a refactor. +- Do not restore the removed `index-html` alias. + +### Tasks + +- [ ] Rename `fjs/website/browser-source.mjs` to authored `.f.mjs` with a + co-located proof at full coverage. +- [ ] Move static proof discovery and `_browser-suite.mjs` generation into + `fjs/website/module.f.mjs`; extend `fjs/effects/node/` only for a concrete + missing capability and prove the real and virtual interpretations. +- [ ] Delete `fjs/website/browser-prepare.mjs` and make the sole `website` + command `node ./fjs/module.mjs r ./fjs/website/module.f.mjs`. +- [ ] Prove the generator end to end against the virtual filesystem: a module + whose graph reaches `node:` is skipped with its reason, one that does not + is emitted. + +### Related + +- [Generate website](generate-website.md) — the parent issue. +- [Browser testing](../../emergent_testing/todo/browser-testing.md) — the + browser-native application the manifest feeds. From 62ba9d16f7e4b6e2f61b1c11c6c0dcae188c0cd2 Mon Sep 17 00:00:00 2001 From: Sergey Shandar Date: Thu, 27 Aug 2026 07:29:19 +0000 Subject: [PATCH 02/18] changelog: entry for #1737 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_016PyLwDNkPQM1uD1Tg5ApBg --- changelog/unreleased/1737.md | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 changelog/unreleased/1737.md diff --git a/changelog/unreleased/1737.md b/changelog/unreleased/1737.md new file mode 100644 index 000000000..ab9258d98 --- /dev/null +++ b/changelog/unreleased/1737.md @@ -0,0 +1,7 @@ +- **BREAKING CHANGES:** `emergent_testing`: `browser.mjs` moves to + `browser/module.mjs` and now shares `fjs t`'s proof semantics; + `runBrowserProofs` and `startBrowserTests` are gone, `startBrowserTestSources` + remains. Only `instanceof Promise` values are awaited, matching `fjs t` +- `effects`: the host-independent operations (`all`, `await`, `fetch`, `import`, + `now`, `sandbox`) move to `effects/common`, re-exported unchanged from + `effects/node`; `effects/browser` interprets them in a browser realm From 0aac2255ff755d3336a8be37f206e7a25a55483c Mon Sep 17 00:00:00 2001 From: Sergey Shandar Date: Thu, 27 Aug 2026 07:43:43 +0000 Subject: [PATCH 03/18] emergent_testing/browser: yield between batches, and never end a run without a report MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two defects found by review, both reproduced first and both regressions against the deleted runner. `all` started every child in the same turn, so the macrotask the `report` handler awaited paused only that result's own continuation while its siblings ran on: no task boundary, and a page frozen on its first frame until the last proof body finished. The slicing belongs to the browser interpreter, where the scheduling is, so `all` now runs its children 25 at a time and yields between slices; the adapter's counter is gone. Reading the tree a proof returns runs user code, and the shared traversal has no `try`/`catch` to give it, so a throwing getter panicked through the run: the promise rejected, the page stayed in `running`, and no report or completion event ever arrived — the one outcome an automated controller cannot act on. The adapter now turns that panic into an `infrastructure-error` report. Attributing such a failure to the leaf that caused it, rather than to the run, stays in `todo/hostile-proof-values.md`, which is updated to say what is now handled and what is not. Verified in Chromium: loading progress and result rows advance throughout the run (349 → 3435) instead of appearing only at the end. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_016PyLwDNkPQM1uD1Tg5ApBg --- fjs/effects/browser/module.mjs | 48 ++++++++++++++++++- fjs/emergent_testing/browser/module.mjs | 41 ++++++++-------- fjs/emergent_testing/browser/proof.mjs | 29 +++++++++++ .../todo/hostile-proof-values.md | 32 ++++++++----- 4 files changed, 116 insertions(+), 34 deletions(-) diff --git a/fjs/effects/browser/module.mjs b/fjs/effects/browser/module.mjs index 9d8e1f2b2..ab75cb8a4 100644 --- a/fjs/effects/browser/module.mjs +++ b/fjs/effects/browser/module.mjs @@ -43,6 +43,52 @@ import { toVec } from '../../types/uint8array/module.f.mjs' * @typedef {(source: string) => Promise} BrowserImporter */ +/** + * How many effects one `all` starts before it hands the event loop back. + * + * **A browser needs a real task boundary to paint, and only `all` can give it + * one.** Every operation resolves through a microtask, so a page running a + * suite of any size would show its first frame until the last proof body had + * finished — `all` starts every child in the same turn, and a child that yielded + * inside its own continuation would pause only itself while its siblings ran on. + * Slicing the children is what bounds the work between two frames. + * + * `all` promises that its effects run concurrently and that it answers each + * one's whole `Result`. Neither says they start simultaneously, so the slicing + * is the runner's business — the Node runner has no frame to paint and starts + * them all at once. + * + * Yielding per effect would be the simpler rule and the wrong one: `setTimeout` + * clamps to 4 ms once nested, which is minutes across a few thousand proofs. + */ +const batchSize = 25 + +/** @type {() => Promise} */ +const macrotask = () => new Promise(resolve => { setTimeout(resolve, 0) }) + +/** + * Runs `effects` in slices of {@link batchSize}, yielding to the event loop + * between them, and answers every `Result` in the order the effects were given. + * + * @template T + * @template E + * @param {CommonRun} run + * @param {readonly Effect[]} effects + * @returns {Promise[]>} + */ +const runBatched = async (run, effects) => { + /** @type {readonly Result[]} */ + let done = [] + let index = 0 + while (index < effects.length) { + const batch = await Promise.all(effects.slice(index, index + batchSize).map(e => run(e))) + done = [...done, ...batch] + index += batchSize + if (index < effects.length) { await macrotask() } + } + return done +} + /** * Performs host IO, reporting a thrown failure as an {@link IoResult} error. * @@ -105,7 +151,7 @@ const sandbox = async f => { * @type {(run: CommonRun, importer?: BrowserImporter) => ToAsyncOperationMap} */ export const browserOperationMap = (run, importer = source => import(source)) => ({ - all: async (...effects) => ok(await Promise.all(effects.map(e => run(e)))), + all: async (...effects) => ok(await runBatched(run, effects)), await: async p => ok([p instanceof Promise ? await p : p]), fetch: url => io(async () => { const response = await globalThis.fetch(url) diff --git a/fjs/emergent_testing/browser/module.mjs b/fjs/emergent_testing/browser/module.mjs index 93b753afb..d6064639f 100644 --- a/fjs/emergent_testing/browser/module.mjs +++ b/fjs/emergent_testing/browser/module.mjs @@ -32,27 +32,14 @@ import { asyncRun } from '../../effects/module.mjs' import { browserOperationMap } from '../../effects/browser/module.mjs' -import { main } from './module.f.mjs' +import { errorDetails } from '../module.f.mjs' +import { main, reportOf } from './module.f.mjs' import { ok } from '../../types/result/module.f.mjs' /** @typedef {Window & { fjsBrowserTestReport?: Promise }} _TestWindow */ /** @typedef {(effect: Effect) => Promise>} _Run */ -/** - * How many results may be rendered before the runner hands the event loop back. - * - * Every operation resolves through a microtask, and microtasks do not let a - * browser paint: without a real task boundary the page would show its first - * frame again only once the whole suite had finished. Yielding per result would - * be the simpler rule and the wrong one — `setTimeout` clamps to 4 ms once - * nested, which is minutes across a few thousand proofs. - */ -const batchSize = 25 - -/** @type {() => Promise} */ -const macrotask = () => new Promise(resolve => { setTimeout(resolve, 0) }) - /** @type {(root: Element) => _TestWindow | null} */ const viewOf = root => root.ownerDocument.defaultView @@ -149,16 +136,30 @@ export const startBrowserTestSources = (root, sources, importer = source => impo results = [...results, result] if (summary !== null) { summary.textContent = `${results.length} tests completed…` } if (output !== null) { output.append(renderResult(root.ownerDocument, result)) } - if (results.length % batchSize === 0) { await macrotask() } return ok(undefined) }, reported: async () => ok(results), }) const view = viewOf(root) - // The application's error channel is empty — every failure it can meet is - // reported — so the run's `Result` is always `ok` and `unwrap` would only - // add a panic path nothing can reach. - const report = run(main({ browser: navigatorName(root), sources })).then(([, value]) => { + const browser = navigatorName(root) + // The application's error channel is empty — every failure it can *answer* + // is reported — so the run's `Result` is always `ok`. A **panic** is the + // other thing, and it is what `never` cannot promise away: reading a proof + // tree runs user code, so an enumerable getter or a proxy trap throws + // through the shared traversal, which has no `try`/`catch` to give it. That + // must not be where the page stops. A rejected run with the suite left in + // `running` is the one outcome an automated controller cannot act on, so + // the panic becomes the report it could not produce — see + // `../todo/hostile-proof-values.md` for attributing it to the test that + // caused it. + const settled = run(main({ browser, sources })).then( + ([, value]) => value, + error => { + const [message, stack] = errorDetails(error) + return reportOf('infrastructure-error', browser, 0, [ + { module: '', path: '', status: 'failed', duration: 0, message, stack }]) + }) + const report = settled.then(value => { renderBrowserReport(root, value) view?.dispatchEvent(new CustomEvent('fjs-browser-test-complete', { detail: value })) return value diff --git a/fjs/emergent_testing/browser/proof.mjs b/fjs/emergent_testing/browser/proof.mjs index fc7a7fcb7..1e919bceb 100644 --- a/fjs/emergent_testing/browser/proof.mjs +++ b/fjs/emergent_testing/browser/proof.mjs @@ -214,6 +214,21 @@ export const proof = { assertEq(report.totals.tests, 60) assertEq(report.totals.passed, 60) }, + // Reading the tree a proof returns runs user code, and the shared traversal + // has no `try`/`catch` to give it — so a throwing getter panics *through* + // the run. The page must still reach a terminal state and still publish a + // report: a rejected run left in `running` is the one outcome an automated + // controller cannot act on. + hostileProofTree: async () => { + const { root, view, states } = page() + const report = await startBrowserTestSources(root, ['a'], async () => ({ + proof: { hostile: () => ({ get boom() { throw new Error('trap') } }) }, + })) + assertEq(report.status, 'infrastructure-error') + assertEq(report.results[0]?.message, 'trap') + assertEq(states.join(','), 'loading,running,infrastructure-error') + assertEq(view.events.length, 1) + }, // A root whose document has no window still runs and still answers: there // is simply nowhere to publish the promise or dispatch the event. withoutView: async () => { @@ -274,5 +289,19 @@ export const proof = { const results = unwrap(await all(pureOk(1), pureOk(2))) assertEq(results.map(unwrap).join(','), '1,2') }, + // Past the batch size `all` hands the event loop back, which is the only + // thing that lets a page paint mid-suite: a timer queued before the call + // has to run before it resolves. Without the slicing every child settles + // on microtasks and no timer gets a turn — which is what this asserts, + // since the effects below perform nothing. + allYieldsBetweenBatches: async () => { + let fired = false + setTimeout(() => { fired = true }, 0) + const many = [...new Array(60).keys()].map(i => pureOk(i)) + const results = unwrap(await all(...many)) + assertEq(results.length, 60) + assertEq(results.map(unwrap).join(','), many.map((_, i) => i).join(',')) + assert(fired, 'all resolved without yielding to the event loop') + }, }, } diff --git a/fjs/emergent_testing/todo/hostile-proof-values.md b/fjs/emergent_testing/todo/hostile-proof-values.md index 8bdaf598a..faf815c1f 100644 --- a/fjs/emergent_testing/todo/hostile-proof-values.md +++ b/fjs/emergent_testing/todo/hostile-proof-values.md @@ -12,14 +12,18 @@ before it and `fjs t` were unified; unifying adopted `fjs t`'s semantics deliberately, so this file is where the difference went rather than being silently dropped. -**A thrown value that resists being read takes the run down.** `errorDetails` -reads `message` and `stack` and calls `String`, and a revoked `Proxy`, a -throwing accessor, or a `toString` that panics makes any of those throw. There -is no `try`/`catch` in FunctionalScript, so the shared core cannot guard it, and -the panic escapes the reporter — the run ends with no report at all rather than -one failed test. `fjs t` has always had this exposure (its reporter interpolates -the thrown value into a line); the browser runner used to defend against it in -impure code, and no longer does. +**A value that resists being read is not attributed to the test that produced +it.** Two shared functions read user-supplied values without a guard: the +`collectTests` traversal enumerates a returned proof tree, and `errorDetails` +reads `message`/`stack` and calls `String` on a thrown value. A throwing +accessor, a revoked `Proxy`, or a hostile `toString` panics through either, and +there is no `try`/`catch` in FunctionalScript for the core to catch it with. + +The browser adapter turns that panic into an `infrastructure-error` report +rather than leaving the page in `running`, so a run always terminates — but the +whole run is lost where the deleted runner lost one test, and `fjs t` still ends +with a stack trace and no summary. What is missing is *attribution*: naming the +leaf whose value could not be read, and continuing with the rest. **A promise from another realm is not awaited.** Both `sandbox` interpreters ask `p instanceof Promise`, which is false for a promise built in an iframe, a @@ -35,11 +39,13 @@ asynchronous values. Both belong to the *operation*, not to the shared core, which is what makes one fix serve every runner: -- Normalization could move behind `sandbox`: the operation already runs user - code inside the host's `try`/`catch`, so it is the one place that can read a - hostile value safely and hand back a `message`/`stack` pair that is already - ordinary data. The shared `errorDetails` would then read a record rather than - an arbitrary thrown value, and stay total. +- Reading a user value could move behind `sandbox`: the operation already runs + user code inside the host's `try`/`catch`, so it is the one place that can + enumerate a returned tree, or read a thrown value's `message`/`stack`, and + hand back something that is already ordinary data. The shared `errorDetails` + and `collectTests` would then read a record rather than an arbitrary value, + and stay total — which also lets the failure be reported against the leaf that + caused it instead of against the run. - The brand check needs a test that a page cannot forge and that no proof tree can pass by accident. Candidates: `Promise.resolve(p) === p` on the value's own constructor, or asking each realm the runner knows about. Whatever is From dc10dcf1605fff0c49844ebcb962f057e13f69d1 Mon Sep 17 00:00:00 2001 From: Sergey Shandar Date: Thu, 27 Aug 2026 08:05:16 +0000 Subject: [PATCH 04/18] effects: share the impure sandbox, and restore the browser proofs review found missing MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `sandbox`, `io` and the `await` body were byte-identical in the Node and browser interpreters, with a comment in one saying it matched the other — the drift this PR exists to remove, surviving where there was no impure module to hold it. `effects/common/module.mjs` holds them now, with its own proofs: nothing in them touches a host, and `sandbox` in particular is the operation that executes a proof body, so a runner spelling it its own way would make one suite mean different things in two hosts. Four proof leaves that the move dropped are back, and two of them pin live code that had gone unpinned: `loadingSummaryIsSynchronous` and `loadingProgress` defend the synchronous `Loading n/m` write, and `newRunAfterCompletion` covers re-running on the same root. `renderingThrows` came back with the guard it proves — showing a result is the page's own code, and a renderer that throws must not cost the report every consumer is waiting for; the result is recorded before it is rendered. `browser-testing.md` records what the review measured: the "demonstrably execute inside browsers" gate for a CI job is met, the controller is what still blocks one, and no runner asserts a floor on the number of proofs it discovers. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_016PyLwDNkPQM1uD1Tg5ApBg --- fjs/effects/README.md | 10 +++ fjs/effects/browser/module.mjs | 60 +------------ fjs/effects/common/module.mjs | 89 ++++++++++++++++++++ fjs/effects/common/proof.mjs | 75 +++++++++++++++++ fjs/effects/node/module.mjs | 48 +---------- fjs/emergent_testing/browser/module.mjs | 11 ++- fjs/emergent_testing/browser/proof.mjs | 53 ++++++++++++ fjs/emergent_testing/todo/browser-testing.md | 12 ++- 8 files changed, 253 insertions(+), 105 deletions(-) create mode 100644 fjs/effects/common/module.mjs create mode 100644 fjs/effects/common/proof.mjs diff --git a/fjs/effects/README.md b/fjs/effects/README.md index 6049fb864..5d14a0005 100644 --- a/fjs/effects/README.md +++ b/fjs/effects/README.md @@ -161,6 +161,16 @@ operations of its own, which is why it and `fjs t` can share every line of proof semantics between them. `./node/` re-exports every common name, so a consumer that already imports one module for `readFile` keeps importing it for `sandbox`. +**Part of the interpretation is common too**, and +[`./common/module.mjs`](./common/module.mjs) holds it: `sandbox`'s +`try`/`catch`-and-measure, `await`'s promise test, and the `io` wrapper that +turns a thrown value into an `IoError`. None of them touches a host — a bare +JavaScript realm has `Promise`, a clock and a `catch` — and `sandbox` in +particular is the operation that actually *executes* a proof body, so a runner +that spelled it its own way would make a test suite mean different things in +different hosts. The two runners did have it byte-identical, with a comment in +one saying it matched the other; a comment is not a mechanism. + An interpreter lives beside the host it interprets — [`./node/module.mjs`](./node/module.mjs), [`./browser/module.mjs`](./browser/module.mjs) — and the browser one implements `CommonOp` and nothing else. There is no browser filesystem and no browser diff --git a/fjs/effects/browser/module.mjs b/fjs/effects/browser/module.mjs index ab75cb8a4..d9ffbfbdd 100644 --- a/fjs/effects/browser/module.mjs +++ b/fjs/effects/browser/module.mjs @@ -16,13 +16,11 @@ * * @import { Effect, ToAsyncOperationMap } from '../types.ts' * @import { Result } from '../../types/result/types.ts' - * @import { CommonOp, Module, SandboxResult } from '../common/types.ts' - * @import { IoResult } from '../common/types.ts' + * @import { CommonOp, Module } from '../common/types.ts' */ -import { toIoError } from '../common/module.f.mjs' -import { error, ok } from '../../types/result/module.f.mjs' -import { asyncTryCatch } from '../../types/result/module.mjs' +import { awaitPromise, io, sandbox } from '../common/module.mjs' +import { ok } from '../../types/result/module.f.mjs' import { toVec } from '../../types/uint8array/module.f.mjs' /** @@ -89,56 +87,6 @@ const runBatched = async (run, effects) => { return done } -/** - * Performs host IO, reporting a thrown failure as an {@link IoResult} error. - * - * The browser twin of the Node runner's `io`: the one place where an exception - * becomes ordinary effect data, normalized so nothing past it sees the thrown - * object. - * - * @template T - * @param {() => Promise} f - * @returns {Promise>} - */ -const io = async f => { - const r = await asyncTryCatch(f) - return r[0] === 'ok' ? r : error(toIoError(r[1])) -} - -/** - * Runs `f` and measures it, exactly as the Node runner does: a genuine - * `Promise` is awaited and a rejection is caught, and any other value — a proof - * tree carrying a `then` property included — is the result as it stands. - * - * That equality is the point. `fjs t` and this runner walk the same proof trees - * through the same shared semantics (`fjs/emergent_testing/module.f.mjs`), so - * the one operation that actually *executes* a proof body has to agree with its - * Node counterpart or the two runners disagree about what a suite means. - * - * @template T - * @param {() => T} f - * @returns {Promise>} - */ -const sandbox = async f => { - /** @type {Result} */ - let result - let after - const before = performance.now() - try { - let p = f() - after = performance.now() - if (p instanceof Promise) { - p = await p - after = performance.now() - } - result = ok(p) - } catch (e) { - after = performance.now() - result = error(e) - } - return { result, duration: after - before } -} - /** * The browser's handlers for the host-independent operations. * @@ -152,7 +100,7 @@ const sandbox = async f => { */ export const browserOperationMap = (run, importer = source => import(source)) => ({ all: async (...effects) => ok(await runBatched(run, effects)), - await: async p => ok([p instanceof Promise ? await p : p]), + await: async p => ok(await awaitPromise(p)), fetch: url => io(async () => { const response = await globalThis.fetch(url) if (!response.ok) { diff --git a/fjs/effects/common/module.mjs b/fjs/effects/common/module.mjs new file mode 100644 index 000000000..61240f061 --- /dev/null +++ b/fjs/effects/common/module.mjs @@ -0,0 +1,89 @@ +/** + * The impure half of the host-independent operations: the three handlers every + * runner would otherwise write for itself. + * + * `../common/module.f.mjs` holds the *constructors* for `all`, `await`, `fetch`, + * `import`, `now` and `sandbox`; this holds the parts of their *interpretation* + * that are the same wherever they run. Nothing here touches a host: `sandbox` + * needs a `try`/`catch`, a clock and `Promise`, `await` needs `Promise`, and + * `io` needs a `catch` and the normalizer — all of which a bare JavaScript realm + * has. What differs between hosts is `fetch`, `import`, the clock's epoch and + * the concurrency policy, and those stay in each runner. + * + * It exists because the two runners had `sandbox`, `io` and the `await` body + * byte-identical, with a comment in one saying it matched the other. That is the + * drift this layer is meant to remove, and a comment is not a mechanism. + * + * @module + * + * @import { IoResult, SandboxResult } from './types.ts' + * @import { Result } from '../../types/result/types.ts' + */ + +import { toIoError } from './module.f.mjs' +import { error, ok } from '../../types/result/module.f.mjs' +import { asyncTryCatch } from '../../types/result/module.mjs' + +/** + * Performs host IO, reporting a thrown failure as an {@link IoResult} error. + * + * The one place where an exception becomes ordinary effect data, normalized so + * that nothing past it sees the thrown object — a stack, a `cause` and + * arbitrary own properties do not survive a wire hop. + * + * @template T + * @param {() => Promise} f + * @returns {Promise>} + */ +export const io = async f => { + const r = await asyncTryCatch(f) + return r[0] === 'ok' ? r : error(toIoError(r[1])) +} + +/** + * Runs `f` and measures it: a genuine `Promise` is awaited and a rejection is + * caught, and any other value — a proof tree carrying a `then` property + * included — is the result as it stands. + * + * **This is the operation that actually executes a proof body**, so every runner + * has to agree on it exactly or a test suite means different things in different + * hosts. That is why it is here rather than written once per runner: the two + * copies it replaces were identical, and nothing but a review would have caught + * them drifting apart. + * + * The clock is read either side of the call with nothing in between, which is + * the whole reason `sandbox` is one operation rather than a `tryCatch` and a + * `now` a scheduler could interleave. + * + * @template T + * @param {() => T} f + * @returns {Promise>} + */ +export const sandbox = async f => { + /** @type {Result} */ + let result + let after + const before = performance.now() + try { + let p = f() + after = performance.now() + if (p instanceof Promise) { + p = await p + after = performance.now() + } + result = ok(p) + } catch (e) { + after = performance.now() + result = error(e) + } + return { result, duration: after - before } +} + +/** + * Resolves a real `Promise` and hands anything else back untouched, in the + * one-element tuple the `await` operation answers with. + * + * @type {(p: unknown) => Promise} + */ +export const awaitPromise = async p => + [p instanceof Promise ? await p : p] diff --git a/fjs/effects/common/proof.mjs b/fjs/effects/common/proof.mjs new file mode 100644 index 000000000..a2b3a4f30 --- /dev/null +++ b/fjs/effects/common/proof.mjs @@ -0,0 +1,75 @@ +/** + * Proofs for the impure half of the host-independent operations. + * + * These three handlers are what every runner would otherwise write for itself, + * so they are proved here rather than only through whichever runner happens to + * call them — the duplication this module removed was invisible precisely + * because each copy was covered by its own host's proofs. + * + * @import { Result } from '../../types/result/types.ts' + */ + +import { assert, assertEq } from '../../asserts/module.f.mjs' +import { awaitPromise, io, sandbox } from './module.mjs' +import { errorMessage } from './module.f.mjs' +import { unwrap } from '../../types/result/module.f.mjs' + +export const proof = { + io: { + value: async () => { + assertEq(unwrap(await io(async () => 7)), 7) + }, + // The one boundary where an exception becomes ordinary effect data. + thrown: async () => { + const r = await io(async () => { throw Object.assign(new Error('nope'), { code: 'ENOENT' }) }) + assert(r[0] === 'error', r) + assertEq(errorMessage(r[1]), 'nope') + assertEq(r[1][0], 'ioError') + }, + }, + sandbox: { + value: async () => { + const { result, duration } = await sandbox(() => 1) + assertEq(unwrap(result), 1) + assert(duration >= 0, duration) + }, + thrown: async () => { + const { result } = await sandbox(() => { throw new Error('boom') }) + assert(result[0] === 'error', result) + assertEq(/** @type {Error} */ (result[1]).message, 'boom') + }, + // A real promise is awaited, and its rejection is the failure — which is + // the rule every runner has to agree on, since this is the operation + // that executes a proof body. + promise: async () => { + // The thunk is annotated because `Sandbox` declares + // `SandboxResult` while every runner resolves a real promise + // before answering, so the declared value type is `Promise` + // where the runtime value is `2`. + /** @type {() => unknown} */ + const resolves = () => Promise.resolve(2) + const { result } = await sandbox(resolves) + assertEq(unwrap(result), 2) + }, + rejected: async () => { + const { result } = await sandbox(() => Promise.reject(new Error('later'))) + assert(result[0] === 'error', result) + assertEq(/** @type {Error} */ (result[1]).message, 'later') + }, + // ...and an ordinary object carrying a `then` is a value, never a + // thenable to adopt. + thenable: async () => { + const value = { then: () => undefined } + const { result } = await sandbox(() => value) + assertEq(unwrap(result), value) + }, + }, + awaitPromise: { + promise: async () => { + assertEq((await awaitPromise(Promise.resolve(3)))[0], 3) + }, + plainValue: async () => { + assertEq((await awaitPromise(3))[0], 3) + }, + }, +} diff --git a/fjs/effects/node/module.mjs b/fjs/effects/node/module.mjs index b523f9b3f..6386bae9b 100644 --- a/fjs/effects/node/module.mjs +++ b/fjs/effects/node/module.mjs @@ -13,7 +13,7 @@ * @module * * @import { Effect } from '../types.ts' - * @import { IoResult, Server as EffectServer, Headers, Module, NodeOp, RequestListener as Erl, NodeProgram, NodeProgramOptions, WriteConsoles, TestContext, TestFn, } from './types.ts' + * @import { Server as EffectServer, Headers, Module, NodeOp, RequestListener as Erl, NodeProgram, NodeProgramOptions, WriteConsoles, TestContext, TestFn, } from './types.ts' * @import { Result } from '../../types/result/types.ts' * @import { StringMap } from '../../types/object/types.ts' * @import { Nullable } from '../../types/nullable/types.ts' @@ -30,6 +30,7 @@ import * as testContext from 'node:test' import { concat, normalize, toPosix } from '../../path/module.f.mjs' import { asyncRun } from '../module.mjs' +import { awaitPromise, io, sandbox } from '../common/module.mjs' import { memoryOperationMap } from './memory/module.mjs' import { emptyHost, emptyHostCode, emptyHostMessage, exitCode, toIoError, usesInlineTestContext, @@ -85,22 +86,6 @@ const createServer = http.createServer /** @typedef {(effect: Effect) => Promise>} _EffectToPromise */ -/** - * Performs host IO, reporting a thrown failure as an {@link IoResult} error. - * - * Every filesystem, network, and subprocess handler below goes through it, so - * the `catch` that turns an exception into effect data — and the normalization - * that keeps the channel serializable — happens in exactly one place. - * - * @template T - * @param {() => Promise} f - * @returns {Promise>} - */ -const io = async f => { - const r = await asyncTryCatch(f) - return r[0] === 'ok' ? r : error(toIoError(r[1])) -} - /** * Reads a request body, giving up at the `Vec` cap rather than at the point * where converting it would throw. @@ -246,35 +231,6 @@ const asyncImport = v => { return import(s1) } -/** - * @template T - * @param {() => T} f - * @returns {Promise<{ readonly result: Result, readonly duration: number }>} - */ -const sandbox = async f => { - /** @type {Result} */ - let result - let after - const before = performance.now() - try { - let p = f() - after = performance.now() - if (p instanceof Promise) { - p = await p - after = performance.now() - } - result = ok(p) - } catch (e) { - after = performance.now() - result = error(e) - } - return { result, duration: after - before } -} - -/** @type {(p: unknown) => Promise} */ -const awaitPromise = async p => - [p instanceof Promise ? await p : p] - const { now } = Date /** Maps `WriteConsoles` names to the corresponding Node.js writable streams. diff --git a/fjs/emergent_testing/browser/module.mjs b/fjs/emergent_testing/browser/module.mjs index d6064639f..3fd1b8ca5 100644 --- a/fjs/emergent_testing/browser/module.mjs +++ b/fjs/emergent_testing/browser/module.mjs @@ -134,8 +134,15 @@ export const startBrowserTestSources = (root, sources, importer = source => impo ...browserOperationMap(effect => run(effect), load), report: async result => { results = [...results, result] - if (summary !== null) { summary.textContent = `${results.length} tests completed…` } - if (output !== null) { output.append(renderResult(root.ownerDocument, result)) } + // Showing a result as it lands is the page's own rendering, and it + // must not take the run down with it: the report is the one thing + // the page is still waiting for, and it is already recorded above. + try { + if (summary !== null) { summary.textContent = `${results.length} tests completed…` } + if (output !== null) { output.append(renderResult(root.ownerDocument, result)) } + } catch { + // The result stays in the report the run resolves with. + } return ok(undefined) }, reported: async () => ok(results), diff --git a/fjs/emergent_testing/browser/proof.mjs b/fjs/emergent_testing/browser/proof.mjs index 1e919bceb..2d449aa46 100644 --- a/fjs/emergent_testing/browser/proof.mjs +++ b/fjs/emergent_testing/browser/proof.mjs @@ -229,6 +229,59 @@ export const proof = { assertEq(states.join(','), 'loading,running,infrastructure-error') assertEq(view.events.length, 1) }, + // The summary must not keep showing idle text through loading: it is + // replaced the instant a run starts, before any import has had a chance to + // settle — even one that never does. + loadingSummaryIsSynchronous: () => { + const { root, summary } = page() + void startBrowserTestSources(root, ['a.mjs', 'b.mjs'], () => new Promise(() => undefined)) + assertEq(summary.textContent, 'Loading 0/2') + }, + // ...and it counts up as modules link, so a slow graph shows progress + // rather than one frozen line. + loadingProgress: async () => { + const { root, summary } = page() + /** @type {(module: Module) => void} */ + let release = () => undefined + /** @type {Promise} */ + const pending = new Promise(resolve => { release = resolve }) + const done = startBrowserTestSources(root, ['a.mjs', 'b.mjs'], + source => source === 'a.mjs' ? Promise.resolve({ proof: {} }) : pending) + await Promise.resolve() + await Promise.resolve() + assertEq(summary.textContent, 'Loading 1/2: a.mjs') + release({ proof: {} }) + assertEq((await done).status, 'passed') + }, + // The same action starts every run: nothing but the `Run` control's own + // state stands between a completed run and the next one. + newRunAfterCompletion: async () => { + const { root, runButton, states } = page() + /** @type {() => Promise} */ + const load = () => Promise.resolve({ proof: { t: () => undefined } }) + await startBrowserTestSources(root, ['a.mjs'], load) + assert(!runButton.attributes.has('disabled')) + const second = await startBrowserTestSources(root, ['a.mjs'], load) + assertEq(second.status, 'passed') + assertEq(second.totals.tests, 1) + assertEq(states.join(','), 'loading,running,passed,loading,running,passed') + }, + // Rendering a result is the page's own code, so it is a failure point of + // the page and not of the run: a renderer that throws must not cost the + // report every consumer is waiting for. + renderingThrows: async () => { + const { root, results } = page() + const append = results.append + const report = await startBrowserTestSources(root, ['a.mjs'], async () => { + // Break rendering only once the run is under way, so the page is + // built normally and only the per-result append fails. + Object.assign(results, { append: () => { throw new Error('render') } }) + return { proof: { t: () => undefined } } + }) + Object.assign(results, { append }) + assertEq(report.status, 'passed') + assertEq(report.totals.passed, 1) + }, // A root whose document has no window still runs and still answers: there // is simply nowhere to publish the promise or dispatch the event. withoutView: async () => { diff --git a/fjs/emergent_testing/todo/browser-testing.md b/fjs/emergent_testing/todo/browser-testing.md index 81daec8fd..3cc9c3781 100644 --- a/fjs/emergent_testing/todo/browser-testing.md +++ b/fjs/emergent_testing/todo/browser-testing.md @@ -159,7 +159,17 @@ workers, or visual regression testing. `playwright/test` and reuses the shared controller. - [ ] Run the same application in Chromium, Firefox, and WebKit. - [ ] Add the validation fixtures above; add CI only after proof bodies - demonstrably execute inside browsers. + demonstrably execute inside browsers. **That gate is now met** — the + unified runner was driven in Chromium over the generated page, 3435 proofs + linked and executed, so what still blocks a CI job is the controller + below, not evidence. Nothing in `.github/workflows/` starts a browser + today, and `npm run website` only *generates* the suite: it exits `0` with + a failing proof in the manifest, so the browser suite is not a gate + anywhere yet. +- [ ] Assert a floor on the number of proofs a run discovers. Nothing does + today, in any runner: a `collectTests` that silently skipped most leaves + would keep `fjs t` at exit `0`, and a suite that loses coverage cannot + report that it has. ### Related From 57e295bd75102a46f35e9ddeb268ef8bbeed2fc7 Mon Sep 17 00:00:00 2001 From: Sergey Shandar Date: Thu, 27 Aug 2026 08:15:37 +0000 Subject: [PATCH 05/18] emergent_testing/browser: make the panic guard total, and stop diagnosing every infrastructure error as a load failure MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The guard added for a panicking proof tree was not itself total: describing the panic reads the value that caused it, so a proxy whose traps throw *itself* made `errorDetails` panic in turn and the page stuck in `running` again — the guard becoming the thing it was meant to prevent. It is the last handler there is, so it now says what it cannot describe rather than being thrown by it. `infrastructure-error` covers a run that panicked and a runner missing an operation as well as a module that would not link, so the summary no longer claims they all "failed to load" — a false diagnosis sends a reader to debug their imports. Each result still carries its own module and message. The browser clock reads `performance.timeOrigin + performance.now()` rather than `Date.now()`. The operation means the same thing — milliseconds since the epoch, as the Node runner answers — but a suite runs for minutes, and a report's duration is the difference between two reads: with wall-clock time an NTP correction inside a run makes that negative or inflated, which is what the deleted runner avoided by measuring in `performance.now()`. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_016PyLwDNkPQM1uD1Tg5ApBg --- fjs/effects/browser/module.mjs | 9 +++++- fjs/emergent_testing/browser/module.mjs | 28 ++++++++++++++-- fjs/emergent_testing/browser/proof.mjs | 43 +++++++++++++++++++++++-- 3 files changed, 75 insertions(+), 5 deletions(-) diff --git a/fjs/effects/browser/module.mjs b/fjs/effects/browser/module.mjs index d9ffbfbdd..871355e50 100644 --- a/fjs/effects/browser/module.mjs +++ b/fjs/effects/browser/module.mjs @@ -112,6 +112,13 @@ export const browserOperationMap = (run, importer = source => import(source)) => // before it ever starts loading — is a load failure like any other, so it // is caught here rather than escaping the effect it belongs to. import: path => io(async () => importer(path)), - now: async () => ok(Date.now()), + // `performance.timeOrigin + performance.now()`, not `Date.now()`. The + // operation means the same thing either way — milliseconds since the epoch, + // as the Node runner answers — but this one cannot go backwards. A suite + // runs for minutes, an NTP correction lands inside one, and the report's + // duration is the difference between two of these reads: with wall-clock + // time that difference can come out negative or inflated, which is what the + // deleted browser runner avoided by measuring in `performance.now()`. + now: async () => ok(performance.timeOrigin + performance.now()), sandbox: async f => ok(await sandbox(f)), }) diff --git a/fjs/emergent_testing/browser/module.mjs b/fjs/emergent_testing/browser/module.mjs index 3fd1b8ca5..a278bdfcd 100644 --- a/fjs/emergent_testing/browser/module.mjs +++ b/fjs/emergent_testing/browser/module.mjs @@ -35,11 +35,21 @@ import { browserOperationMap } from '../../effects/browser/module.mjs' import { errorDetails } from '../module.f.mjs' import { main, reportOf } from './module.f.mjs' import { ok } from '../../types/result/module.f.mjs' +import { tryCatch } from '../../types/result/module.mjs' /** @typedef {Window & { fjsBrowserTestReport?: Promise }} _TestWindow */ /** @typedef {(effect: Effect) => Promise>} _Run */ +/** + * What a run is reported as when even *describing* its panic panicked. + * + * There is nothing left to say about the value at that point — every way of + * reading it is a way of being thrown by it — so the report says exactly that + * rather than inventing a message. + */ +const unreadableFailure = 'The run failed with a value that cannot be read' + /** @type {(root: Element) => _TestWindow | null} */ const viewOf = root => root.ownerDocument.defaultView @@ -83,7 +93,11 @@ export const renderBrowserReport = (root, report) => { const summary = root.querySelector('[data-test-summary]') if (summary !== null) { summary.textContent = report.status === 'infrastructure-error' - ? `Infrastructure error: ${report.totals.failed} failed to load (${report.duration.toFixed(1)} ms)` + // Not "failed to load": this status also covers a run that panicked + // and a runner missing an operation, and naming the wrong cause + // sends a reader to debug their imports. Each result below carries + // its own module and message, so the detail is not lost. + ? `Infrastructure error: ${report.totals.failed} failed (${report.duration.toFixed(1)} ms)` : `${report.totals.passed} passed, ${report.totals.failed} failed (${report.duration.toFixed(1)} ms)` } const output = root.querySelector('[data-test-results]') @@ -162,7 +176,17 @@ export const startBrowserTestSources = (root, sources, importer = source => impo const settled = run(main({ browser, sources })).then( ([, value]) => value, error => { - const [message, stack] = errorDetails(error) + // Describing the panic reads the value that caused it, and the + // value is the reason there was one: a proxy whose traps throw + // *itself* makes `errorDetails` panic in turn. This is the last + // handler there is, so it is the one that may not fail — a second + // failure here is the page stuck in `running` again, with the + // guard that was supposed to prevent it. What it cannot describe, + // it says it cannot describe. + const described = tryCatch(() => errorDetails(error)) + const [message, stack] = described[0] === 'ok' + ? described[1] + : [unreadableFailure, ''] return reportOf('infrastructure-error', browser, 0, [ { module: '', path: '', status: 'failed', duration: 0, message, stack }]) }) diff --git a/fjs/emergent_testing/browser/proof.mjs b/fjs/emergent_testing/browser/proof.mjs index 2d449aa46..0bdf102ce 100644 --- a/fjs/emergent_testing/browser/proof.mjs +++ b/fjs/emergent_testing/browser/proof.mjs @@ -191,7 +191,7 @@ export const proof = { assertEq(report.totals.failed, 1) assertEq(report.results[0]?.module, 'a') assertEq(report.results[0]?.message, '404') - assert(summary.textContent.startsWith('Infrastructure error: 1 failed to load'), summary.textContent) + assert(summary.textContent.startsWith('Infrastructure error: 1 failed'), summary.textContent) assertEq(states.join(','), 'loading,running,infrastructure-error') }, // The importer is page code, so obtaining the promise is itself a failure @@ -282,6 +282,39 @@ export const proof = { assertEq(report.status, 'passed') assertEq(report.totals.passed, 1) }, + // Describing a panic reads the value that caused it, so a value every trap + // of which throws *itself* makes the description panic in turn. That is the + // last handler there is: it may not fail, or the guard against a stuck page + // becomes the thing that sticks it. + unreadableFailure: async () => { + /** @type {ProxyHandler} */ + const handler = {} + const hostile = new Proxy({}, handler) + const rethrow = () => { throw hostile } + Object.assign(handler, { has: rethrow, get: rethrow, ownKeys: rethrow }) + const { root, states } = page() + const report = await startBrowserTestSources(root, ['a'], async () => ({ + proof: { boom: () => { throw hostile } }, + })) + assertEq(report.status, 'infrastructure-error') + assertEq(report.results[0]?.message, 'The run failed with a value that cannot be read') + assertEq(states.join(','), 'loading,running,infrastructure-error') + }, + // `infrastructure-error` covers a panic and a runner missing an operation as + // well as a module that would not link, so the summary must not diagnose + // every one of them as a loading failure. + infrastructureSummaryNamesNoCause: () => { + const { root, summary } = page() + renderBrowserReport(root, { + status: 'infrastructure-error', + browser: 'x', + totals: { tests: 1, passed: 0, failed: 1 }, + duration: 0, + results: [{ module: '', path: '', status: 'failed', duration: 0, message: 'no sandbox', stack: '' }], + }) + assert(!summary.textContent.includes('to load'), summary.textContent) + assert(summary.textContent.startsWith('Infrastructure error: 1 failed'), summary.textContent) + }, // A root whose document has no window still runs and still answers: there // is simply nowhere to publish the promise or dispatch the event. withoutView: async () => { @@ -330,8 +363,14 @@ export const proof = { awaitsPlainValue: async () => { assertEq(unwrap(await awaitOp(7))[0], 7) }, + // Epoch milliseconds, as the Node runner answers — but read through + // `performance`, so two reads never come out in the wrong order however + // the system clock is adjusted between them. now: async () => { - assert(unwrap(await now()) > 0) + const before = unwrap(await now()) + const after = unwrap(await now()) + assert(before > Date.UTC(2020, 0, 1), before) + assert(after >= before, [before, after]) }, sandboxMeasures: async () => { const { result, duration } = unwrap(await sandbox(() => 1)) From 68fc983723d7c7f0475b32b17c9723f19c9e8372 Mon Sep 17 00:00:00 2001 From: Sergey Shandar Date: Thu, 27 Aug 2026 08:36:48 +0000 Subject: [PATCH 06/18] emergent_testing/todo: design the `catch` operation that guards the traversal MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review pressed on the one gap left, and correctly: since the two runners are now one, guarding the proof-tree traversal once covers `fjs t` and the browser together. The todo carried a vague "read user values behind `sandbox`" and one wrong reason for why that does not work. Timing is not the obstacle — the sub-tree walk runs after the leaf's promise has resolved, so a synchronous thunk over a settled value would be fine. The obstacle is the virtual runner: its `sandbox` is a deliberate pass-through, because a `.f.mjs` runner has no `try`/`catch` to implement a real one with, and routing the traversal through it would break every fixture. So the design is a second, honest operation beside it — `catch`, "run this pure thunk; a throw is the `error` branch" — with the file-by-file work and the proofs it restores written down. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_016PyLwDNkPQM1uD1Tg5ApBg --- .../todo/hostile-proof-values.md | 81 ++++++++++++++----- 1 file changed, 60 insertions(+), 21 deletions(-) diff --git a/fjs/emergent_testing/todo/hostile-proof-values.md b/fjs/emergent_testing/todo/hostile-proof-values.md index faf815c1f..7b40e2591 100644 --- a/fjs/emergent_testing/todo/hostile-proof-values.md +++ b/fjs/emergent_testing/todo/hostile-proof-values.md @@ -34,27 +34,66 @@ is settable through `Symbol.toStringTag`, and an object carrying a `then` proof would then be assimilated, breaking the rule that only actual promises are asynchronous values. -### Preliminary design - -Both belong to the *operation*, not to the shared core, which is what makes one -fix serve every runner: - -- Reading a user value could move behind `sandbox`: the operation already runs - user code inside the host's `try`/`catch`, so it is the one place that can - enumerate a returned tree, or read a thrown value's `message`/`stack`, and - hand back something that is already ordinary data. The shared `errorDetails` - and `collectTests` would then read a record rather than an arbitrary value, - and stay total — which also lets the failure be reported against the leaf that - caused it instead of against the run. -- The brand check needs a test that a page cannot forge and that no proof tree - can pass by accident. Candidates: `Promise.resolve(p) === p` on the value's - own constructor, or asking each realm the runner knows about. Whatever is - chosen must be one function both interpreters call, or the two drift again. - -Neither is worth doing speculatively. Do the first when a real proof loses a -run to it, and the second when proofs genuinely execute in more than one realm — -which is the point [browser-testing](browser-testing.md) reaches with iframes or -workers. +### Design: a `catch` operation + +Reading a user value belongs to the *operation*, not to the shared core, which +is what makes one fix serve every runner. Since the two runners are now one, +guarding the traversal once covers `fjs t` and the browser together. + +**`sandbox` cannot hold it, and the reason is not the one it looks like.** +Timing is not the obstacle: the sub-tree walk in `runModule` happens *after* the +runner has resolved the leaf's promise, so `sandbox(() => collectTests(path, +false, r))` would run a pure synchronous thunk over an already-settled value. +The obstacle is the **virtual runner**. Its `sandbox` is a deliberate +pass-through — `f => state => [state, ok(f())]`, with the fixture returning the +`SandboxResult` it wants reported — because `../../effects/node/virtual` is +`.f.mjs` and FunctionalScript has no `try`/`catch` to implement a real one with. +Routing the traversal through `sandbox` would hand that handler a thunk +answering `_TestAndPath[]`, which it would cast to `SandboxResult` and every +fixture in `../proof.f.mjs` would break. + +So add a second, honest operation beside it: + +```ts +export type Catch = readonly['catch', (f: () => T) => OpResult>] +``` + +"Run this pure thunk; a throw is the `error` branch." It carries no clock and no +fixture convention, so each runner implements it truthfully: + +- `effects/node/module.mjs` and `effects/browser/module.mjs`: `tryCatch(f)`, one + line each, from `types/result/module.mjs`. +- `effects/node/virtual/module.f.mjs`: `ok(ok(f()))` — a pure runner still + cannot catch, and a hostile fixture still panics there, which is the same + bargain `sandbox` already makes. Virtual proofs use benign fixtures. + +`walk` then reads a sub-tree through `catch` and, on the `error` branch, reports +one failed result at that path instead of panicking — which is what restores +`exportedTreeThrows` / `returnedTreeThrows`, and gives `fjs t` a behaviour it +never had. `errorDetails` gets the same treatment at its one call site. + +The work is roughly: the operation and its constructor in `effects/common`, one +handler in each of the three runners, the `CommandSet` entries, the `walk` +change and its new result shape, and the mock maps in +`effects/common/proof.f.mjs` and `emergent_testing/browser/proof.f.mjs`. + +**The brand check** for cross-realm promises needs a test that a page cannot +forge and that no proof tree can pass by accident. Candidates: +`Promise.resolve(p) === p` on the value's own constructor, or asking each realm +the runner knows about. Whatever is chosen must be one function both +interpreters call, or the two drift again. Do it when proofs genuinely execute +in more than one realm — the point [browser-testing](browser-testing.md) reaches +with iframes or workers. + +### Tasks + +- [ ] Add the `catch` operation, its constructor, and a handler in each of the + Node, browser and virtual runners. +- [ ] Read sub-trees through it in `walk`, reporting an unreadable tree as one + failed result at its path rather than a panic. +- [ ] Restore `exportedTreeThrows` and `returnedTreeThrows`, and add the `fjs t` + counterparts the browser-only versions never had. +- [ ] Read a thrown value through it at `errorDetails`' call site. ### Constraints From 505766d8a6403d4e501232c46643df53decb1e7e Mon Sep 17 00:00:00 2001 From: Sergey Shandar Date: Thu, 27 Aug 2026 09:22:09 +0000 Subject: [PATCH 07/18] emergent_testing: one spelling for a test's name, and the three TODOs review asked for MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The page rendered `./a.proof.f.mjs .x` where the terminal rendered `import("./a.proof.f.mjs").proof.x()` — one identifier in two spellings, months after the semantics were shared, because rendering was still per host. The format now lives once, in `fmtCall`, which takes a module and an already-rendered key chain so a reporter holding a `TestResult` names a test exactly as `fjs t` does; `fmtImport` is that function over an unrendered `Path`. `passing` asserts the rendered line, so the two cannot drift again. The browser's `all` yields every ten effects rather than every twenty-five. A count measures the wrong thing — proofs differ in cost by orders of magnitude, so ten fast ones waste a boundary and one slow one stalls the page anyway — so this is a mitigation and is labelled as one, with the elapsed-time design in `todo/report-scheduling.md`. Three TODOs, none of them changes here: - `share-the-whole-runner.md` — the semantics are shared but the runner around them is written per host: discovery, reporting and the outcome. Compares an artificial effect per capability against injecting the host's verbs, with the formatting drift above as the symptom to keep in mind. - `report-scheduling.md` — yield on a time budget instead of a count. - `imports-promises-realms.md` — a study, not a design: a module namespace adopts a `then`, a proof tree refuses to, and `instanceof Promise` does not survive a realm. Three mechanisms whose interaction nobody has written down, which is why it keeps being rediscovered. `hostile-proof-values.md` hands the cross-realm brand check to the last of those rather than sketching it twice. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_016PyLwDNkPQM1uD1Tg5ApBg --- fjs/effects/browser/module.mjs | 8 +- fjs/emergent_testing/browser/module.mjs | 6 +- fjs/emergent_testing/browser/proof.mjs | 6 ++ fjs/emergent_testing/module.f.mjs | 23 ++++- .../todo/hostile-proof-values.md | 13 ++- .../todo/imports-promises-realms.md | 73 ++++++++++++++++ .../todo/report-scheduling.md | 63 ++++++++++++++ .../todo/share-the-whole-runner.md | 84 +++++++++++++++++++ 8 files changed, 263 insertions(+), 13 deletions(-) create mode 100644 fjs/emergent_testing/todo/imports-promises-realms.md create mode 100644 fjs/emergent_testing/todo/report-scheduling.md create mode 100644 fjs/emergent_testing/todo/share-the-whole-runner.md diff --git a/fjs/effects/browser/module.mjs b/fjs/effects/browser/module.mjs index 871355e50..46a5df12c 100644 --- a/fjs/effects/browser/module.mjs +++ b/fjs/effects/browser/module.mjs @@ -58,8 +58,14 @@ import { toVec } from '../../types/uint8array/module.f.mjs' * * Yielding per effect would be the simpler rule and the wrong one: `setTimeout` * clamps to 4 ms once nested, which is minutes across a few thousand proofs. + * + * A count is the wrong measure and this number is a mitigation, not a design: + * proofs differ in cost by orders of magnitude, so a slice of ten fast ones + * yields immediately while a slice holding one slow one stalls the page for as + * long as that proof runs. Yielding on elapsed time instead is + * `fjs/emergent_testing/todo/report-scheduling.md`. */ -const batchSize = 25 +const batchSize = 10 /** @type {() => Promise} */ const macrotask = () => new Promise(resolve => { setTimeout(resolve, 0) }) diff --git a/fjs/emergent_testing/browser/module.mjs b/fjs/emergent_testing/browser/module.mjs index a278bdfcd..d64f162ac 100644 --- a/fjs/emergent_testing/browser/module.mjs +++ b/fjs/emergent_testing/browser/module.mjs @@ -32,7 +32,7 @@ import { asyncRun } from '../../effects/module.mjs' import { browserOperationMap } from '../../effects/browser/module.mjs' -import { errorDetails } from '../module.f.mjs' +import { errorDetails, fmtCall } from '../module.f.mjs' import { main, reportOf } from './module.f.mjs' import { ok } from '../../types/result/module.f.mjs' import { tryCatch } from '../../types/result/module.mjs' @@ -79,7 +79,9 @@ const renderResult = (document, result) => { const item = document.createElement('li') item.setAttribute('data-status', result.status) const detail = result.status === 'failed' ? `: ${result.message}\n${result.stack}` : '' - item.textContent = `${result.status === 'passed' ? 'PASS' : 'FAIL'} ${result.module} ${result.path} (${result.duration.toFixed(1)} ms)${detail}` + // `fmtCall`, so a test is named here exactly as `fjs t` names it — one + // identifier, one spelling, whichever runner is reporting. + item.textContent = `${result.status === 'passed' ? 'PASS' : 'FAIL'} ${fmtCall(result.module, result.path)} (${result.duration.toFixed(1)} ms)${detail}` return item } diff --git a/fjs/emergent_testing/browser/proof.mjs b/fjs/emergent_testing/browser/proof.mjs index 0bdf102ce..e6d3633fb 100644 --- a/fjs/emergent_testing/browser/proof.mjs +++ b/fjs/emergent_testing/browser/proof.mjs @@ -136,6 +136,12 @@ export const proof = { assertEq(report.totals.tests, 1) assertEq(report.results[0]?.path, '.x') assertEq(statuses(results).join(','), 'passed') + // The page names a test exactly as `fjs t` names it. The two spellings + // had drifted — `./a .x` here against the call expression there — which + // is the thing a shared runner is supposed to make impossible. + assert( + results.children[0]?.textContent.startsWith('PASS import("a").proof.x()'), + results.children[0]?.textContent) assert(summary.textContent.startsWith('1 passed, 0 failed'), summary.textContent) assertEq(states.join(','), 'loading,running,passed') assertEq(view.events.length, 1) diff --git a/fjs/emergent_testing/module.f.mjs b/fjs/emergent_testing/module.f.mjs index 9c8d025c1..142f8ab12 100644 --- a/fjs/emergent_testing/module.f.mjs +++ b/fjs/emergent_testing/module.f.mjs @@ -330,14 +330,31 @@ export const fmtPath = path => path.reduce((/** @type {string} */ acc, k) => acc + fmtKey(k), '') /** - * Formats a fully-qualified test identifier as a JS-like expression, e.g. - * `import("./math.proof.f.ts").add()` or `import("./a.proof.f.ts").users[3].name()`. + * A fully-qualified test identifier, from a module and an **already-rendered** + * key chain: `import("./math.proof.f.mjs").proof.add()`. + * + * This is the one place the format lives, and it takes the rendered chain + * rather than a {@link Path} so that a reporter holding a {@link TestResult} — + * whose `path` is already a string — names a test exactly as `fjs t` does. It + * did not, and the browser page rendered `./math.proof.f.mjs .add` while the + * terminal rendered the call expression: one identifier in two spellings, which + * is the drift a shared runner is supposed to make impossible. + * + * @type {(file: string, path: string) => string} + */ +export const fmtCall = (file, path) => + `import(${JSON.stringify(file)}).proof${path}()` + +/** + * {@link fmtCall} over a {@link Path} that has not been rendered yet, e.g. + * `import("./math.proof.f.ts").proof.add()` or + * `import("./a.proof.f.ts").proof.users[3].name()`. * Self-contained per line — suitable for parallel output and as a CLI filter argument. * * @type {(file: string, path: Path) => string} */ export const fmtImport = (file, path) => - `import(${JSON.stringify(file)}).proof${fmtPath(path)}()` + fmtCall(file, fmtPath(path)) /** * Renders a key chain for terminal output: `| ` per level of depth, followed diff --git a/fjs/emergent_testing/todo/hostile-proof-values.md b/fjs/emergent_testing/todo/hostile-proof-values.md index 7b40e2591..1bda0e573 100644 --- a/fjs/emergent_testing/todo/hostile-proof-values.md +++ b/fjs/emergent_testing/todo/hostile-proof-values.md @@ -77,13 +77,10 @@ handler in each of the three runners, the `CommandSet` entries, the `walk` change and its new result shape, and the mock maps in `effects/common/proof.f.mjs` and `emergent_testing/browser/proof.f.mjs`. -**The brand check** for cross-realm promises needs a test that a page cannot -forge and that no proof tree can pass by accident. Candidates: -`Promise.resolve(p) === p` on the value's own constructor, or asking each realm -the runner knows about. Whatever is chosen must be one function both -interpreters call, or the two drift again. Do it when proofs genuinely execute -in more than one realm — the point [browser-testing](browser-testing.md) reaches -with iframes or workers. +**The brand check** for cross-realm promises is not designed here. It belongs +with the two mechanisms it keeps being confused with — a module namespace +adopting a `then`, and a proof tree refusing to — which are studied together in +[imports, promises and realms](imports-promises-realms.md). ### Tasks @@ -103,5 +100,7 @@ with iframes or workers. ### Related +- [Imports, promises and realms](imports-promises-realms.md) — where the + cross-realm brand check is studied. - [Browser testing](browser-testing.md) - [Test-runner behavior](661-test-runner-behavior.md) diff --git a/fjs/emergent_testing/todo/imports-promises-realms.md b/fjs/emergent_testing/todo/imports-promises-realms.md new file mode 100644 index 000000000..10b24bf8b --- /dev/null +++ b/fjs/emergent_testing/todo/imports-promises-realms.md @@ -0,0 +1,73 @@ +## Investigate imports, promises and realms + +**Priority:** P3 +**Status:** open — investigation, not yet actionable + +### Problem + +Three mechanisms meet in the runner, none of them is written down as a rule, and +the code where they meet reads as a pile of special cases rather than a design. +They are separate mechanisms that happen to interact, and the interaction is +what nobody has stated: + +**A module namespace object is a thenable.** `import()` resolves by *adopting* +what a module exports, so a module exporting a function named `then` corrupts +its own dynamic import. That is why exporting `then` from a proof module is +forbidden ([`spec/todo/3240-export.md`](../../../spec/todo/3240-export.md)) — +but the rule lives in a spec issue and a README paragraph, and nothing checks +it. The proof discovery in `../../dev/module.f.mjs` imports whatever it finds. + +**A proof tree is not a thenable, even when it has a `then`.** The runner's rule +is that only an actual `Promise` is an asynchronous value, so `{ then: f }` +returned from a proof is a sub-tree with a test called `then` in it. This is the +opposite reading of the same property name, one layer down, and both readings +are correct in their own layer. Nothing says so in one place. + +**`instanceof Promise` is realm-local.** A promise built in an iframe, a worker +or a `node:vm` context is not `instanceof Promise` here, so it is walked as a +proof tree and a *rejected* one is reported as a pass. The deleted browser +runner defended against this with `Symbol.species` shadowing and an intrinsic +`then` — about 150 lines that were, fairly, called a magic mess; they were +removed when the runners were unified, on the grounds that `fjs t` never had +them. The defence is gone and the exposure is not. + +The three are usually discussed one at a time, which is why the interaction +keeps being rediscovered: the thing that makes a namespace dangerous (`then` is +adopted) is the thing the runner deliberately refuses to do (`then` is a name), +and the check that separates them (`instanceof`) is the one that does not +survive a realm boundary. + +### What to investigate + +This is a study, not a design. It is worth doing before +[browser-testing](browser-testing.md) puts proofs in iframes or workers, because +that is the point at which cross-realm promises stop being hypothetical. + +- **State the layering.** One document saying which layer adopts a `then` and + which layer refuses to, and why both are right. Until that exists, every fix + to one looks like a bug in the other. +- **Find a brand check that survives a realm and cannot be forged.** + `Object.prototype.toString` is forgeable through `Symbol.toStringTag`. + `Promise.resolve(p) === p` against the value's own constructor is a candidate. + Whatever is chosen must be one function every interpreter calls. +- **Decide whether the runner should see namespace objects at all.** If + discovery handed the runner a plain record of proofs rather than the module + namespace, the `then` export hazard would not reach it — and the `then`-export + ban could become a check rather than a convention. +- **Establish what the removed 150 lines actually bought**, from the proofs that + covered them (`species.proof.mjs` in this PR's history), so that whatever + replaces them is measured against the same cases rather than against a memory. + +### Constraints + +- An object carrying a `then` proof property must stay an ordinary proof tree. +- Whatever is added must apply to every runner. A defence in one host only is + what unifying the runners just finished removing. + +### Related + +- [Hostile proof values](hostile-proof-values.md) — the cross-realm promise + exposure, and the traversal guard it shares a cause with. +- [Browser testing](browser-testing.md) — iframes and workers. +- [`spec/todo/3240-export.md`](../../../spec/todo/3240-export.md) — the `then` + export ban. diff --git a/fjs/emergent_testing/todo/report-scheduling.md b/fjs/emergent_testing/todo/report-scheduling.md new file mode 100644 index 000000000..8176f8221 --- /dev/null +++ b/fjs/emergent_testing/todo/report-scheduling.md @@ -0,0 +1,63 @@ +## Yield on elapsed time, not on a count of proofs + +**Priority:** P3 +**Status:** open + +### Problem + +The browser runner's `all` starts its children in slices of ten and yields to +the event loop between slices +([`fjs/effects/browser/module.mjs`](../../effects/browser/module.mjs)). Ten is a +mitigation, not a design. + +A count measures the wrong thing. Proofs differ in cost by orders of magnitude — +most are microseconds, a few run for a second or more — so a slice of ten fast +proofs yields almost immediately and wastes a task boundary, while a slice +holding one slow proof stalls the page for as long as that proof runs and no +count would have helped. The page freezes in bursts, and the reported progress +stops with it, which is exactly when a reader most wants to see it move. The +number was 25 and is now 10 for that reason; the next person to notice a stall +will have the same argument for 5. + +### Preliminary design + +Yield on a **time budget** rather than a count: keep starting children while the +slice has spent less than some milliseconds — a frame's worth, or a small +multiple of one — and hand the loop back when it has. That bounds the *stall*, +which is the thing a reader actually experiences, and it self-tunes: a thousand +trivial proofs run in one slice and one slow proof yields after itself. + +The clock read has to be cheap and monotonic; `performance.now()` is both, and +the shared `sandbox` already measures each proof with it, so the elapsed time +may be available without a second read. + +Two questions to settle with measurement rather than by argument: + +- **Where the budget belongs.** In `all` alongside the slicing, or in the + reporting handler that renders? `all` is where the work is started, which is + what made the slicing correct in the first place. +- **Whether a slow proof can yield at all.** A single proof body is synchronous + from the runner's point of view; nothing can interrupt it. A budget bounds how + many *more* are started after one, not the stall the slow one itself causes. + Reporting the slow proof's *start* — not only its result — may matter more + than any scheduling change, and is the cheaper experiment. + +### Constraints + +- The Node runner has no frame to paint and must keep starting its children at + once; this is the browser interpreter's policy, as the slicing already is. +- `all` must keep answering every `Result` in the order its effects were given. + +### Tasks + +- [ ] Measure where the page actually stalls on the real suite, per slice, and + whether the cause is proof cost or rendering. +- [ ] Replace the count with an elapsed-time budget, and prove the boundary the + way `operations.allYieldsBetweenBatches` proves the current one. +- [ ] Consider reporting a proof's start as well as its result, so a stall is + visible rather than silent. + +### Related + +- [Browser testing](browser-testing.md) +- [Explicit browser test controls](browser-test-controls.md) diff --git a/fjs/emergent_testing/todo/share-the-whole-runner.md b/fjs/emergent_testing/todo/share-the-whole-runner.md new file mode 100644 index 000000000..485d4af92 --- /dev/null +++ b/fjs/emergent_testing/todo/share-the-whole-runner.md @@ -0,0 +1,84 @@ +## Share the whole runner, not just its proof semantics + +**Priority:** P2 +**Status:** open + +### Problem + +The proof *semantics* are shared: `../module.f.mjs` decides what a leaf is, how +a returned tree is walked, what `throw` means, which values are asynchronous, +how a path is spelled and how results are counted, and `fjs t` and the browser +both go through it. What is **not** shared is the runner around them. Each host +still writes its own program: + +| | `fjs t` | browser | +| --- | --- | --- | +| entry | `main` → `testAll` → `runModuleMap` | `browser/module.f.mjs`'s `main` | +| discovery | `loadModuleMap` over `readdir` + `import` | a generated manifest, linked one specifier at a time | +| reporting | `defaultReporter` → `Write` | `recordingReporter` → `report` | +| outcome | an exit code through `exitCodeStep` | a `BrowserTestReport` | + +Two of those four differ for a real reason and two do not. A browser has no +`stdout` and no exit code, so `Write` and `Program` genuinely cannot cross — +but "load these modules, run them, answer an outcome" is one program written +twice, and every future host writes it a third time. + +The formatting drift this issue was raised over is the symptom worth keeping in +mind: the page rendered `./a.proof.f.mjs .x` where the terminal rendered +`import("./a.proof.f.mjs").proof.x()`. One identifier, two spellings, months +after the semantics were shared — because *rendering* was still per host. That +is fixed (`fmtCall`), but only that instance of it. + +### Preliminary design + +Lift the host difference into the program's parameters instead of into a +separate program per host. Two shapes are worth comparing before either is +built: + +**An artificial effect per host capability.** Where a host lacks an operation, +replace it with one every host can implement at the semantic level: `log` is not +available in a browser, but `testReport` is — a page renders it, a terminal +formats it, an MCP server serializes it. `report`/`reported` already exist and +are exactly this move made once; the question is whether the *whole* set can be +expressed that way, including discovery and the run's outcome. + +**Dependency injection of the effect-producing functions.** The runner is +generic in its operation set and takes the host's verbs as a record: + +```ts +type Host = { + readonly log: (message: string) => Effect + readonly load: () => Effect + readonly import: (source: string) => Effect +} +``` + +`Reporter` is already this shape for one third of the job, so the question +is whether extending it beats adding operations, or whether the two are the +same thing written differently. + +Whichever is chosen, the test is concrete: adding a third host — an MCP server, +a worker, `fjs browser-test` — must not mean writing a fourth `main`. + +### Constraints + +- The shared semantics must not acquire terminal text or DOM: a `TestResult` + carries neither today and that is what lets both reporters render it. +- A browser must not gain a `Write` or a `Program` it cannot honour. Lifting the + abstraction means finding the operation both hosts *can* implement, not giving + one a stub. + +### Tasks + +- [ ] Inventory what each host's `main` does that is not host-specific. +- [ ] Choose between artificial effects and injected verbs, and write down why. +- [ ] Express discovery once, so a manifest and a `readdir` walk are two + implementations of one operation rather than two programs. +- [ ] Express the outcome once, so an exit code and a report are two renderings + of one value. + +### Related + +- [Browser testing](browser-testing.md) — the hosts that are still to come. +- [Test-runner behavior](661-test-runner-behavior.md) — the differences between + runners that are intentional, and must stay intentional. From 569636be1c6e58191ac732fddf00fdc69b3011d4 Mon Sep 17 00:00:00 2001 From: Sergey Shandar Date: Thu, 27 Aug 2026 09:31:11 +0000 Subject: [PATCH 08/18] emergent_testing/todo: put the browser batching on probation rather than designing it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The batching is premature optimization. It was not added because anyone found the suite slow — `fjs t` schedules nothing at all and nobody has complained — but because a page that renders nothing until the run finishes looked wrong in review. Observing a behaviour is not the same as someone having a problem with it, and the count moved from 25 to 10 with no measurement on either side. So the scheduling todo now asks for the opposite of a design: remove the batching entirely, run the real suite in a browser, and look at what actually happens — the page may paint anyway, since module loading is network-bound and dominates the first seconds. Only if that produces a stall someone objects to is there a problem to solve, and the elapsed-time argument is kept for that case rather than presented as the plan. The code says the same thing where the constant is defined, so a reader meets the probation before the rationale. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_016PyLwDNkPQM1uD1Tg5ApBg --- fjs/effects/browser/module.mjs | 12 ++- .../todo/report-scheduling.md | 96 ++++++++++--------- 2 files changed, 58 insertions(+), 50 deletions(-) diff --git a/fjs/effects/browser/module.mjs b/fjs/effects/browser/module.mjs index 46a5df12c..8b9224bf4 100644 --- a/fjs/effects/browser/module.mjs +++ b/fjs/effects/browser/module.mjs @@ -59,11 +59,13 @@ import { toVec } from '../../types/uint8array/module.f.mjs' * Yielding per effect would be the simpler rule and the wrong one: `setTimeout` * clamps to 4 ms once nested, which is minutes across a few thousand proofs. * - * A count is the wrong measure and this number is a mitigation, not a design: - * proofs differ in cost by orders of magnitude, so a slice of ten fast ones - * yields immediately while a slice holding one slow one stalls the page for as - * long as that proof runs. Yielding on elapsed time instead is - * `fjs/emergent_testing/todo/report-scheduling.md`. + * **This whole mechanism is on probation.** It was not added because anyone + * found the suite slow — `fjs t` schedules nothing at all and no one has + * complained — but because a page that renders nothing until the run finishes + * looked wrong. That is an observation, not a problem someone has, and the + * count has already moved from 25 to 10 with no measurement on either side. + * `fjs/emergent_testing/todo/report-scheduling.md` asks for the batching to be + * removed and the real suite watched before any of this is treated as a design. */ const batchSize = 10 diff --git a/fjs/emergent_testing/todo/report-scheduling.md b/fjs/emergent_testing/todo/report-scheduling.md index 8176f8221..69eddb14d 100644 --- a/fjs/emergent_testing/todo/report-scheduling.md +++ b/fjs/emergent_testing/todo/report-scheduling.md @@ -1,63 +1,69 @@ -## Yield on elapsed time, not on a count of proofs +## Try removing the browser runner's batching entirely **Priority:** P3 -**Status:** open +**Status:** open — experiment first, design only if the experiment says so ### Problem The browser runner's `all` starts its children in slices of ten and yields to the event loop between slices -([`fjs/effects/browser/module.mjs`](../../effects/browser/module.mjs)). Ten is a -mitigation, not a design. - -A count measures the wrong thing. Proofs differ in cost by orders of magnitude — -most are microseconds, a few run for a second or more — so a slice of ten fast -proofs yields almost immediately and wastes a task boundary, while a slice -holding one slow proof stalls the page for as long as that proof runs and no -count would have helped. The page freezes in bursts, and the reported progress -stops with it, which is exactly when a reader most wants to see it move. The -number was 25 and is now 10 for that reason; the next person to notice a stall -will have the same argument for 5. - -### Preliminary design - -Yield on a **time budget** rather than a count: keep starting children while the -slice has spent less than some milliseconds — a frame's worth, or a small -multiple of one — and hand the loop back when it has. That bounds the *stall*, -which is the thing a reader actually experiences, and it self-tunes: a thousand -trivial proofs run in one slice and one slow proof yields after itself. - -The clock read has to be cheap and monotonic; `performance.now()` is both, and -the shared `sandbox` already measures each proof with it, so the elapsed time -may be available without a second read. - -Two questions to settle with measurement rather than by argument: - -- **Where the budget belongs.** In `all` alongside the slicing, or in the - reporting handler that renders? `all` is where the work is started, which is - what made the slicing correct in the first place. -- **Whether a slow proof can yield at all.** A single proof body is synchronous - from the runner's point of view; nothing can interrupt it. A budget bounds how - many *more* are started after one, not the stall the slow one itself causes. - Reporting the slow proof's *start* — not only its result — may matter more - than any scheduling change, and is the cheaper experiment. +([`fjs/effects/browser/module.mjs`](../../effects/browser/module.mjs)). **That +batching is premature optimization and should probably not exist.** + +It was not added because anyone found the suite slow. It was added in a review +round, because without it the page renders nothing until the run finishes, and +that *looked* wrong. Observing a behaviour is not the same as someone having a +problem with it: nobody has reported a stall, and the number has already been +argued down from 25 to 10 with no measurement on either side of the change — +which is the shape of an optimization nobody can evaluate. + +`fjs t` is the reference and it schedules nothing at all. It starts every leaf +of a module at once, prints results as they land, and no one has complained. The +browser runner sharing its semantics but not its scheduling is a difference that +has to justify itself, and so far it has not. + +### The experiment, before any design + +Remove the batching completely — `all` back to `Promise.all` over every child, +no `macrotask`, no `batchSize` — and run the real suite in a browser. Then look: + +- Does the page actually stay blank until the end, or does the browser paint + anyway? Module loading is network-bound and dominates the first seconds, which + may be all the yielding a page needs. +- If it does stay blank, for how long, and does that matter to anyone reading a + passing run? A suite that finishes in two seconds with no intermediate frames + is not a problem; one that finishes in two minutes might be. +- Does anything except rendering depend on the yielding? + +Only if that produces a stall someone objects to is there a problem to solve, +and only then is the shape of a solution worth arguing about. If it comes to +that, the argument against a count still holds — proofs differ in cost by orders +of magnitude, so a slice of ten fast ones wastes a boundary while a slice +holding one slow one stalls anyway — and an elapsed-time budget bounds the thing +a reader actually experiences. Reporting a proof's *start* as well as its result +may serve better than any scheduling change, and is cheaper to try. ### Constraints -- The Node runner has no frame to paint and must keep starting its children at - once; this is the browser interpreter's policy, as the slicing already is. -- `all` must keep answering every `Result` in the order its effects were given. +- Whatever the answer, it is the browser interpreter's policy. The Node runner + has no frame to paint and must keep starting its children at once. +- `all` must keep answering every `Result` in the order its effects were given, + batching or not. +- `operations.allYieldsBetweenBatches` pins the current behaviour. Removing the + batching means removing that proof, not weakening it. ### Tasks -- [ ] Measure where the page actually stalls on the real suite, per slice, and - whether the cause is proof cost or rendering. -- [ ] Replace the count with an elapsed-time budget, and prove the boundary the - way `operations.allYieldsBetweenBatches` proves the current one. -- [ ] Consider reporting a proof's start as well as its result, so a stall is - visible rather than silent. +- [ ] Remove the batching and the yield; run the real suite in a browser and + write down what actually happens. +- [ ] Decide from that whether there is a problem at all. +- [ ] Only then, if there is: bound the stall by elapsed time rather than by a + count, and prove the boundary the way the current one is proved. ### Related - [Browser testing](browser-testing.md) - [Explicit browser test controls](browser-test-controls.md) +- [Share the whole runner](share-the-whole-runner.md) — the CLI runner's + scheduling is one more thing the two hosts do differently for no stated + reason. From 4923ddbf2b4e168264ee90c94e64d1588e8b6dfa Mon Sep 17 00:00:00 2001 From: Sergey Shandar Date: Thu, 27 Aug 2026 09:40:52 +0000 Subject: [PATCH 09/18] emergent_testing/todo: measure the browser scheduling instead of arguing about it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Ran the real suite in headless Chromium four ways, 3435 proofs each, sampling how often the page visibly moves. Two findings, and the first contradicts what "remove the batching" expected. Removing the yield removes the progress: with plain `Promise.all` the first row appears at 39.8 s of a 39.7 s run — nothing at all until the end. A browser cannot paint between microtasks and every operation resolves through one, so the whole suite is a single task. It is not faster for it. The grouping was never the point; the clamp was. Yielding after every result — what a reader actually wants, and what `fjs t` does — costs 2% over no yielding at all when the yield is a `MessageChannel`. It cost 45% only through `setTimeout`, which clamps to 4 ms once nested: 3435 results times 4 ms is the whole difference. So batching was a workaround for a bad yield primitive, and the workaround is what made grouping look necessary. The todo carries the table and the change it indicates: delete `batchSize` and `runBatched`, yield after each result over an unclamped primitive, and re-point the proof at that. Not made here, per the request to keep it out of this PR. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_016PyLwDNkPQM1uD1Tg5ApBg --- .../todo/report-scheduling.md | 68 ++++++++++++------- 1 file changed, 42 insertions(+), 26 deletions(-) diff --git a/fjs/emergent_testing/todo/report-scheduling.md b/fjs/emergent_testing/todo/report-scheduling.md index 69eddb14d..ada9b0f21 100644 --- a/fjs/emergent_testing/todo/report-scheduling.md +++ b/fjs/emergent_testing/todo/report-scheduling.md @@ -1,7 +1,7 @@ ## Try removing the browser runner's batching entirely **Priority:** P3 -**Status:** open — experiment first, design only if the experiment says so +**Status:** open — the experiment is done; the change it indicates is not ### Problem @@ -22,26 +22,50 @@ of a module at once, prints results as they land, and no one has complained. The browser runner sharing its semantics but not its scheduling is a difference that has to justify itself, and so far it has not. -### The experiment, before any design +### The experiment, and what it measured -Remove the batching completely — `all` back to `Promise.all` over every child, -no `macrotask`, no `batchSize` — and run the real suite in a browser. Then look: +Run in headless Chromium against the generated page, 3435 proofs each time, +sampling the run's state every 150 ms. "Progress steps" counts how many distinct +result-counts a reader ever sees — how often the page visibly moves. -- Does the page actually stay blank until the end, or does the browser paint - anyway? Module loading is network-bound and dominates the first seconds, which - may be all the yielding a page needs. -- If it does stay blank, for how long, and does that matter to anyone reading a - passing run? A suite that finishes in two seconds with no intermediate frames - is not a problem; one that finishes in two minutes might be. -- Does anything except rendering depend on the yielding? +| `all` schedules | run | first row | progress steps | +| --- | ---: | ---: | ---: | +| no yielding at all (`Promise.all`) | 39.7 s | **39.8 s** | 2 | +| slices of 10, `setTimeout` yield | 40.2 s | 3.5 s | 49 | +| every result, `setTimeout` yield | **58.1 s** | 3.5 s | 229 | +| **every result, `MessageChannel` yield** | **40.4 s** | 3.6 s | 123 | -Only if that produces a stall someone objects to is there a problem to solve, -and only then is the shape of a solution worth arguing about. If it comes to -that, the argument against a count still holds — proofs differ in cost by orders -of magnitude, so a slice of ten fast ones wastes a boundary while a slice -holding one slow one stalls anyway — and an elapsed-time budget bounds the thing -a reader actually experiences. Reporting a proof's *start* as well as its result -may serve better than any scheduling change, and is cheaper to try. +Two things fall out, and the first is not what "remove the batching" expected: + +**Removing the yield removes the progress.** With no yielding the page shows +nothing at all until the run ends — first row at 39.8 s of a 39.7 s run. It is +not that the browser paints anyway; every operation resolves through a microtask +and a browser cannot paint between microtasks, so the whole suite is one task. +The run is not faster for it either (39.7 s against 40.2 s). + +**The grouping was never the point — the clamp was.** Yielding after *every* +result is what a reader actually wants, and it costs nothing: 40.4 s against +39.7 s with no yielding, about 2%. It cost 45% only through `setTimeout`, which +clamps to 4 ms once nested — 3435 results × 4 ms is the entire difference. +`MessageChannel` (or `scheduler.yield()` where available) has no clamp. + +So batching was the wrong mechanism, as suspected, but not because scheduling is +unnecessary: it was a workaround for a bad yield primitive, and the workaround is +what made grouping look necessary. `fjs t` prints each result as it lands and +that is the behaviour to match — one result, one update — which per-result +yielding gives and grouping only approximates. + +### Tasks + +- [x] Remove the batching and the yield; run the real suite in a browser and + write down what actually happens. — table above. +- [ ] Replace `batchSize`/`runBatched` with a yield after every result, over a + yield primitive with no clamp; delete the batch-size constant entirely. +- [ ] Re-point `operations.allYieldsBetweenBatches` at the new behaviour: it + pins that a boundary exists between children, and should pin that one + exists after *each* child. +- [ ] Check the yield primitive across browsers, and whether `scheduler.yield()` + is worth preferring where it exists. ### Constraints @@ -52,14 +76,6 @@ may serve better than any scheduling change, and is cheaper to try. - `operations.allYieldsBetweenBatches` pins the current behaviour. Removing the batching means removing that proof, not weakening it. -### Tasks - -- [ ] Remove the batching and the yield; run the real suite in a browser and - write down what actually happens. -- [ ] Decide from that whether there is a problem at all. -- [ ] Only then, if there is: bound the stall by elapsed time rather than by a - count, and prove the boundary the way the current one is proved. - ### Related - [Browser testing](browser-testing.md) From cbc445420466cc072551773593f5158cd87b15a6 Mon Sep 17 00:00:00 2001 From: Sergey Shandar Date: Thu, 27 Aug 2026 09:48:52 +0000 Subject: [PATCH 10/18] effects/browser: `all` must start every child before it awaits any MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The slice loop awaited each slice before starting the next, which broke `all`'s concurrency promise rather than delaying it: a child waiting on something a later sibling produces waited for a sibling that was never started, and the run hung with no report and no completion event — on a graph the Node runner completes. Reproduced with a proof that waits on a gate its eleventh sibling opens; it hangs before the fix and passes after. The loop now starts every effect, yielding between one slice's launch and the next, and awaits them all at the end. That keeps the paint boundary — what a slice does when it starts is exactly the work worth bounding, since a proof body runs synchronously inside `sandbox` before that handler's first `await` — while restoring the concurrency. `allStartsEveryChildBeforeAwaiting` pins it. It also corrects the scheduling todo, whose measurements were taken against the serializing loop. "Yield after every result" was the wrong way to describe the target: per-result yielding *is* sequential execution, a bigger break than the batching it was meant to remove. Per-*launch* yielding is the right shape, costs about 3% over no yielding at all, and reaches the first row sooner. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_016PyLwDNkPQM1uD1Tg5ApBg --- fjs/effects/browser/module.mjs | 24 ++++++++---- fjs/emergent_testing/browser/proof.mjs | 25 +++++++++++++ .../todo/report-scheduling.md | 37 ++++++++++++++----- 3 files changed, 69 insertions(+), 17 deletions(-) diff --git a/fjs/effects/browser/module.mjs b/fjs/effects/browser/module.mjs index 8b9224bf4..8e43d0f51 100644 --- a/fjs/effects/browser/module.mjs +++ b/fjs/effects/browser/module.mjs @@ -73,8 +73,19 @@ const batchSize = 10 const macrotask = () => new Promise(resolve => { setTimeout(resolve, 0) }) /** - * Runs `effects` in slices of {@link batchSize}, yielding to the event loop - * between them, and answers every `Result` in the order the effects were given. + * Starts `effects` in slices of {@link batchSize}, yielding to the event loop + * between one slice's *launch* and the next, and answers every `Result` in the + * order the effects were given. + * + * **Every effect is started before any is awaited**, which is not a detail. + * `all` promises its children run concurrently, and a runner that awaited each + * slice before starting the next would break that promise rather than merely + * delay it: a child waiting on something a later sibling produces would wait + * for a sibling that is never started, and the run would hang with no report — + * on a graph the Node runner completes. Yielding between launches costs + * nothing, because what a slice does when it starts is exactly the work worth + * bounding: a proof body runs synchronously inside `sandbox` before that + * handler's first `await`. * * @template T * @template E @@ -83,16 +94,15 @@ const macrotask = () => new Promise(resolve => { setTimeout(resolve, 0) }) * @returns {Promise[]>} */ const runBatched = async (run, effects) => { - /** @type {readonly Result[]} */ - let done = [] + /** @type {readonly Promise>[]} */ + let started = [] let index = 0 while (index < effects.length) { - const batch = await Promise.all(effects.slice(index, index + batchSize).map(e => run(e))) - done = [...done, ...batch] + started = [...started, ...effects.slice(index, index + batchSize).map(e => run(e))] index += batchSize if (index < effects.length) { await macrotask() } } - return done + return Promise.all(started) } /** diff --git a/fjs/emergent_testing/browser/proof.mjs b/fjs/emergent_testing/browser/proof.mjs index e6d3633fb..91be4fb49 100644 --- a/fjs/emergent_testing/browser/proof.mjs +++ b/fjs/emergent_testing/browser/proof.mjs @@ -9,6 +9,7 @@ * shared core and its own proofs; what is checked here is that a browser run * reaches it, renders it, and publishes it. * + * @import { Result } from '../../types/result/types.ts' * @import { Module } from '../../effects/common/types.ts' * @import { CommonRun } from '../../effects/browser/module.mjs' * @import { BrowserTestReport } from './types.ts' @@ -18,6 +19,7 @@ import { assert, assertEq, assertNotNullish } from '../../asserts/module.f.mjs' import { browserOperationMap } from '../../effects/browser/module.mjs' import { asyncRun } from '../../effects/module.mjs' import { pureOk } from '../../effects/module.f.mjs' +import { all as allEffect, sandbox as sandboxEffect } from '../../effects/common/module.f.mjs' import { renderBrowserReport, startBrowserTestSources } from './module.mjs' import { unwrap } from '../../types/result/module.f.mjs' @@ -387,6 +389,29 @@ export const proof = { const results = unwrap(await all(pureOk(1), pureOk(2))) assertEq(results.map(unwrap).join(','), '1,2') }, + // Slicing must not serialize: every child is *started* before any is + // awaited, so a child waiting on something a later sibling produces + // still sees that sibling run. Awaiting each slice before starting the + // next hangs this — the releaser sits in the second slice, which is + // never reached — on a graph the Node runner completes. + allStartsEveryChildBeforeAwaiting: async () => { + /** @type {(value: unknown) => void} */ + let release = () => undefined + /** @type {Promise} */ + const gate = new Promise(resolve => { release = resolve }) + const filler = sandboxEffect(() => 0) + const waits = sandboxEffect(() => gate) + const releases = sandboxEffect(() => { release(1); return 0 }) + const many = [waits, ...[...new Array(9).keys()].map(() => filler), releases] + /** @type {'hung'} */ + const hung = 'hung' + const outcome = await Promise.race([ + commonRun(allEffect(...many)), + new Promise(resolve => { setTimeout(resolve, 1000, hung) }), + ]) + assert(outcome !== hung, 'all serialized its slices') + assertEq(unwrap(/** @type {Result} */ (outcome)).length, 11) + }, // Past the batch size `all` hands the event loop back, which is the only // thing that lets a page paint mid-suite: a timer queued before the call // has to run before it resolves. Without the slicing every child settles diff --git a/fjs/emergent_testing/todo/report-scheduling.md b/fjs/emergent_testing/todo/report-scheduling.md index ada9b0f21..183514a63 100644 --- a/fjs/emergent_testing/todo/report-scheduling.md +++ b/fjs/emergent_testing/todo/report-scheduling.md @@ -31,9 +31,12 @@ result-counts a reader ever sees — how often the page visibly moves. | `all` schedules | run | first row | progress steps | | --- | ---: | ---: | ---: | | no yielding at all (`Promise.all`) | 39.7 s | **39.8 s** | 2 | -| slices of 10, `setTimeout` yield | 40.2 s | 3.5 s | 49 | -| every result, `setTimeout` yield | **58.1 s** | 3.5 s | 229 | -| **every result, `MessageChannel` yield** | **40.4 s** | 3.6 s | 123 | +| slices of 10, `setTimeout` yield | 39.8 s | 4.0 s | 29 | +| every launch, `setTimeout` yield | **58.1 s** | 3.5 s | 229 | +| **every launch, `MessageChannel` yield** | **41.1 s** | 3.1 s | 91 | + +(Sampled at 150 ms, so "progress steps" is a floor and varies a little run to +run; the shape is what matters, not the digits.) Two things fall out, and the first is not what "remove the batching" expected: @@ -44,10 +47,22 @@ and a browser cannot paint between microtasks, so the whole suite is one task. The run is not faster for it either (39.7 s against 40.2 s). **The grouping was never the point — the clamp was.** Yielding after *every* -result is what a reader actually wants, and it costs nothing: 40.4 s against -39.7 s with no yielding, about 2%. It cost 45% only through `setTimeout`, which -clamps to 4 ms once nested — 3435 results × 4 ms is the entire difference. -`MessageChannel` (or `scheduler.yield()` where available) has no clamp. +launch is what a reader actually wants, and it costs about 3%: 41.1 s against +39.7 s with no yielding, and it reaches the first row sooner. It cost 45% only +through `setTimeout`, which clamps to 4 ms once nested — 3435 × 4 ms is the +entire difference. `MessageChannel` (or `scheduler.yield()` where available) has +no clamp. + +**A yield between *launches*, never between a launch and its result.** `all` +promises its children run concurrently, and a slice loop that awaited each slice +before starting the next broke that promise rather than delaying it: a child +waiting on something a later sibling produces waited for a sibling that was +never started, and the run hung with no report — on a graph the Node runner +completes. That is fixed and pinned by +`operations.allStartsEveryChildBeforeAwaiting`; the numbers above are from the +fixed loop. It is also why "yield after every result" is the wrong way to say +this: per-*result* yielding is sequential execution, which is a bigger break +than the batching it was meant to remove. So batching was the wrong mechanism, as suspected, but not because scheduling is unnecessary: it was a workaround for a bad yield primitive, and the workaround is @@ -59,11 +74,13 @@ yielding gives and grouping only approximates. - [x] Remove the batching and the yield; run the real suite in a browser and write down what actually happens. — table above. -- [ ] Replace `batchSize`/`runBatched` with a yield after every result, over a - yield primitive with no clamp; delete the batch-size constant entirely. +- [ ] Replace `batchSize` with a yield after every *launch*, over a yield + primitive with no clamp; delete the batch-size constant entirely. Keep + the launch-then-await shape — the concurrency, not just the boundary. - [ ] Re-point `operations.allYieldsBetweenBatches` at the new behaviour: it pins that a boundary exists between children, and should pin that one - exists after *each* child. + exists after *each* child. `allStartsEveryChildBeforeAwaiting` stays as + it is — it pins the concurrency, which no scheduling change may cost. - [ ] Check the yield primitive across browsers, and whether `scheduler.yield()` is worth preferring where it exists. From 07736137cdc66c07bb5f29d1837c8692859fc431 Mon Sep 17 00:00:00 2001 From: Sergey Shandar Date: Thu, 27 Aug 2026 09:50:51 +0000 Subject: [PATCH 11/18] emergent_testing/todo: record the lost loader stack MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Linking is an `Import` effect now and its failure is an `IoError`, which is `{ code?, message }` — so a module-loading failure reports `stack: ''` where the deleted runner reported the loader's own frames. Those frames are what name the importing module and line for a broken dependency graph, and this section already requires failures to retain a stack, so it is a regression rather than a declared trade-off. The task carries the fix: one additive optional field on `IoErrorInfo`. The rationale that made `IoError` drop it — a stack does not survive a wire hop — is right about `cause` and own properties and wrong about a stack, which is a string. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_016PyLwDNkPQM1uD1Tg5ApBg --- fjs/emergent_testing/todo/browser-testing.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/fjs/emergent_testing/todo/browser-testing.md b/fjs/emergent_testing/todo/browser-testing.md index 3cc9c3781..6c32a7f47 100644 --- a/fjs/emergent_testing/todo/browser-testing.md +++ b/fjs/emergent_testing/todo/browser-testing.md @@ -166,6 +166,20 @@ workers, or visual regression testing. today, and `npm run website` only *generates* the suite: it exits `0` with a failing proof in the manifest, so the browser suite is not a gate anywhere yet. +- [ ] Keep a module-loading failure's stack. This section requires failures to + retain "module path, test path, message, and stack", and a *proof* failure + does — but a **load** failure no longer does. Linking is an `Import` + effect now, and its failure is an `IoError`, which is `{ code?, message }`: + `toIoError` drops the stack, so the report shows `stack: ''` where the + deleted runner showed the loader's own frames, which are what name the + importing module and line for a broken graph. The fix is one additive + optional field, `stack?: string` on `IoErrorInfo` in + `../../effects/common/types.ts`, filled by `toIoError` and read by + `infrastructureResult`. `IoError`'s rationale for dropping it — "a stack, a + `cause`, and arbitrary own properties do not survive a wire hop" — is right + about the last two and wrong about a stack, which is a string. Note that + reading `.stack` is a user-observable operation on a hostile value, the + same exposure `toIoError` already has reading `.message`. - [ ] Assert a floor on the number of proofs a run discovers. Nothing does today, in any runner: a `collectTests` that silently skipped most leaves would keep `fjs t` at exit `0`, and a suite that loses coverage cannot From 401f1917c33947da06b73994bac0bc4cd8863269 Mon Sep 17 00:00:00 2001 From: Sergey Shandar Date: Thu, 27 Aug 2026 10:02:41 +0000 Subject: [PATCH 12/18] effects/browser: delete the batch size; yield after every launch MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit You checked `batchSize = 1` and found it much more responsive, at almost twice the run time. It does not have to cost that: the 2x was `setTimeout`'s 4 ms clamp, not the yielding. Over a `MessageChannel` the same per-launch yield costs about 3%. So there is no batch size any more. `all` starts every effect, hands the event loop back between one launch and the next, and awaits them all at the end. Measured on the real suite in Chromium: 40.7 s against 39.7 s with no yielding at all, first row at 2.7 s (against 4.0 s at slices of ten), and roughly three times as many visible progress steps. No constant left to tune, which was the complaint. `todo/report-scheduling.md` is deleted — its experiment is done and its answer is the code above. The one part that outlives it, checking the yield primitive across browsers and whether `scheduler.yield()` is worth preferring, moves to the cross-browser task in `browser-testing.md`. `allStartsEveryChildBeforeAwaiting` is rewritten. It raced a 1 s wall clock, which in a suite of 3499 concurrent proofs measures how loaded the machine is — it failed once under coverage. It now counts turns of the event loop and records which opener reached the gate first, so it fails rather than hangs, and cannot flake. Checked in both directions against a serializing `all`. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_016PyLwDNkPQM1uD1Tg5ApBg --- fjs/effects/browser/module.mjs | 85 +++++++-------- fjs/emergent_testing/browser/proof.mjs | 52 ++++++--- fjs/emergent_testing/todo/browser-testing.md | 5 +- .../todo/report-scheduling.md | 102 ------------------ 4 files changed, 82 insertions(+), 162 deletions(-) delete mode 100644 fjs/emergent_testing/todo/report-scheduling.md diff --git a/fjs/effects/browser/module.mjs b/fjs/effects/browser/module.mjs index 8e43d0f51..799da2792 100644 --- a/fjs/effects/browser/module.mjs +++ b/fjs/effects/browser/module.mjs @@ -42,50 +42,49 @@ import { toVec } from '../../types/uint8array/module.f.mjs' */ /** - * How many effects one `all` starts before it hands the event loop back. + * Hands the event loop back, so the browser gets a turn. * - * **A browser needs a real task boundary to paint, and only `all` can give it - * one.** Every operation resolves through a microtask, so a page running a - * suite of any size would show its first frame until the last proof body had - * finished — `all` starts every child in the same turn, and a child that yielded - * inside its own continuation would pause only itself while its siblings ran on. - * Slicing the children is what bounds the work between two frames. - * - * `all` promises that its effects run concurrently and that it answers each - * one's whole `Result`. Neither says they start simultaneously, so the slicing - * is the runner's business — the Node runner has no frame to paint and starts - * them all at once. + * **Not `setTimeout`.** It clamps to 4 ms once nested, and a yield between every + * launch across a few thousand proofs is then minutes of pure clamp — measured + * at 58 s against 40 s on the real suite. That cost is what once made grouping + * the launches look necessary. A `MessageChannel` message is an ordinary task + * with no clamp, so the same per-launch yield costs about 3%. * - * Yielding per effect would be the simpler rule and the wrong one: `setTimeout` - * clamps to 4 ms once nested, which is minutes across a few thousand proofs. - * - * **This whole mechanism is on probation.** It was not added because anyone - * found the suite slow — `fjs t` schedules nothing at all and no one has - * complained — but because a page that renders nothing until the run finishes - * looked wrong. That is an observation, not a problem someone has, and the - * count has already moved from 25 to 10 with no measurement on either side. - * `fjs/emergent_testing/todo/report-scheduling.md` asks for the batching to be - * removed and the real suite watched before any of this is treated as a design. + * @type {() => Promise} */ -const batchSize = 10 - -/** @type {() => Promise} */ -const macrotask = () => new Promise(resolve => { setTimeout(resolve, 0) }) +const yieldToLoop = () => new Promise(resolve => { + const { port1, port2 } = new MessageChannel() + port1.onmessage = () => { port1.close(); resolve(undefined) } + port2.postMessage(0) +}) /** - * Starts `effects` in slices of {@link batchSize}, yielding to the event loop - * between one slice's *launch* and the next, and answers every `Result` in the - * order the effects were given. + * Starts every effect, handing the event loop back between one launch and the + * next, and answers each `Result` in the order the effects were given. + * + * **A browser needs a real task boundary to paint, and only `all` can give it + * one.** Every operation resolves through a microtask, and a browser cannot + * paint between microtasks, so without this the whole suite is a single task: + * measured on the real suite, the first result appears at 39.8 s of a 39.7 s + * run — nothing at all until the end, and no faster for it. What a launch does + * is exactly the work worth bounding, because a proof body runs synchronously + * inside `sandbox` before that handler's first `await`. * * **Every effect is started before any is awaited**, which is not a detail. - * `all` promises its children run concurrently, and a runner that awaited each - * slice before starting the next would break that promise rather than merely - * delay it: a child waiting on something a later sibling produces would wait - * for a sibling that is never started, and the run would hang with no report — - * on a graph the Node runner completes. Yielding between launches costs - * nothing, because what a slice does when it starts is exactly the work worth - * bounding: a proof body runs synchronously inside `sandbox` before that - * handler's first `await`. + * `all` promises its children run concurrently, and awaiting one before + * starting the next would break that promise rather than delay it: a child + * waiting on something a later sibling produces would wait for a sibling that + * is never started, and the run would hang with no report — on a graph the Node + * runner completes. `all` says its children run concurrently and that it + * answers every `Result`; it does not say they start in the same task, which is + * what leaves the scheduling to the runner. The Node runner has no frame to + * paint and starts them all at once. + * + * There is deliberately **no batch size**. Grouping launches was a workaround + * for `setTimeout`'s clamp, and a count is the wrong measure anyway — proofs + * differ in cost by orders of magnitude, so a group of ten fast ones wastes a + * boundary while a group holding one slow one stalls regardless. With an + * unclamped yield there is no constant left to tune. * * @template T * @template E @@ -93,14 +92,12 @@ const macrotask = () => new Promise(resolve => { setTimeout(resolve, 0) }) * @param {readonly Effect[]} effects * @returns {Promise[]>} */ -const runBatched = async (run, effects) => { +const runYielding = async (run, effects) => { /** @type {readonly Promise>[]} */ let started = [] - let index = 0 - while (index < effects.length) { - started = [...started, ...effects.slice(index, index + batchSize).map(e => run(e))] - index += batchSize - if (index < effects.length) { await macrotask() } + for (const effect of effects) { + if (started.length !== 0) { await yieldToLoop() } + started = [...started, run(effect)] } return Promise.all(started) } @@ -117,7 +114,7 @@ const runBatched = async (run, effects) => { * @type {(run: CommonRun, importer?: BrowserImporter) => ToAsyncOperationMap} */ export const browserOperationMap = (run, importer = source => import(source)) => ({ - all: async (...effects) => ok(await runBatched(run, effects)), + all: async (...effects) => ok(await runYielding(run, effects)), await: async p => ok(await awaitPromise(p)), fetch: url => io(async () => { const response = await globalThis.fetch(url) diff --git a/fjs/emergent_testing/browser/proof.mjs b/fjs/emergent_testing/browser/proof.mjs index 91be4fb49..004af30b1 100644 --- a/fjs/emergent_testing/browser/proof.mjs +++ b/fjs/emergent_testing/browser/proof.mjs @@ -213,9 +213,8 @@ export const proof = { assertEq(report.status, 'infrastructure-error') assertEq(report.results[0]?.message, 'bad specifier') }, - // Past the batch size the adapter hands the event loop back, so a long - // suite paints instead of freezing the page on its first frame. - batches: async () => { + // A long suite runs to completion with a yield between every launch. + manyLeaves: async () => { const proof = Object.fromEntries( [...new Array(60).keys()].map(i => [`t${i}`, () => undefined])) const report = await run(proof) @@ -395,29 +394,52 @@ export const proof = { // next hangs this — the releaser sits in the second slice, which is // never reached — on a graph the Node runner completes. allStartsEveryChildBeforeAwaiting: async () => { + // The gate is opened either by the eleventh child — which is the + // property under test — or, after far more turns of the event loop + // than every launch can need, by the fallback below. Which one + // opened it is the assertion. + // + // Counting turns rather than milliseconds is deliberate: this proof + // runs concurrently with the rest of the suite, so a wall-clock + // deadline measures how loaded the machine is, not what `all` did. + // The fallback exists so a serializing `all` *fails* here instead of + // hanging the run. + /** @type {string | null} */ + let openedBy = null /** @type {(value: unknown) => void} */ let release = () => undefined /** @type {Promise} */ const gate = new Promise(resolve => { release = resolve }) + // Whoever opens the gate *first* is recorded. A later opener must + // not overwrite it: a serializing `all` still reaches the sibling + // eventually, just far too late to have been what unblocked the + // first child. + /** @type {(who: string) => void} */ + const open = who => { + if (openedBy === null) { openedBy = who } + release(0) + } + const fallback = async () => { + for (let turn = 0; turn < 50 && openedBy === null; turn += 1) { + await new Promise(resolve => { setTimeout(resolve, 0) }) + } + open('the fallback') + } + void fallback() const filler = sandboxEffect(() => 0) const waits = sandboxEffect(() => gate) - const releases = sandboxEffect(() => { release(1); return 0 }) + const releases = sandboxEffect(() => { open('a later sibling'); return 0 }) const many = [waits, ...[...new Array(9).keys()].map(() => filler), releases] - /** @type {'hung'} */ - const hung = 'hung' - const outcome = await Promise.race([ - commonRun(allEffect(...many)), - new Promise(resolve => { setTimeout(resolve, 1000, hung) }), - ]) - assert(outcome !== hung, 'all serialized its slices') - assertEq(unwrap(/** @type {Result} */ (outcome)).length, 11) + const results = unwrap(await commonRun(allEffect(...many))) + assertEq(openedBy, 'a later sibling') + assertEq(results.length, 11) }, - // Past the batch size `all` hands the event loop back, which is the only + // `all` hands the event loop back between launches, which is the only // thing that lets a page paint mid-suite: a timer queued before the call - // has to run before it resolves. Without the slicing every child settles + // has to run before it resolves. Without the yield every child settles // on microtasks and no timer gets a turn — which is what this asserts, // since the effects below perform nothing. - allYieldsBetweenBatches: async () => { + allYieldsBetweenLaunches: async () => { let fired = false setTimeout(() => { fired = true }, 0) const many = [...new Array(60).keys()].map(i => pureOk(i)) diff --git a/fjs/emergent_testing/todo/browser-testing.md b/fjs/emergent_testing/todo/browser-testing.md index 6c32a7f47..0e6eeece8 100644 --- a/fjs/emergent_testing/todo/browser-testing.md +++ b/fjs/emergent_testing/todo/browser-testing.md @@ -157,7 +157,10 @@ workers, or visual regression testing. - [ ] Implement `fjs browser-test` without any Playwright dependency. - [ ] Implement a Playwright Test adapter that dynamically resolves external `playwright/test` and reuses the shared controller. -- [ ] Run the same application in Chromium, Firefox, and WebKit. +- [ ] Run the same application in Chromium, Firefox, and WebKit. Check the + yield `all` uses to give the page a turn while it runs + (`MessageChannel`, `../../effects/browser/module.mjs`) behaves in each, + and whether `scheduler.yield()` is worth preferring where it exists. - [ ] Add the validation fixtures above; add CI only after proof bodies demonstrably execute inside browsers. **That gate is now met** — the unified runner was driven in Chromium over the generated page, 3435 proofs diff --git a/fjs/emergent_testing/todo/report-scheduling.md b/fjs/emergent_testing/todo/report-scheduling.md deleted file mode 100644 index 183514a63..000000000 --- a/fjs/emergent_testing/todo/report-scheduling.md +++ /dev/null @@ -1,102 +0,0 @@ -## Try removing the browser runner's batching entirely - -**Priority:** P3 -**Status:** open — the experiment is done; the change it indicates is not - -### Problem - -The browser runner's `all` starts its children in slices of ten and yields to -the event loop between slices -([`fjs/effects/browser/module.mjs`](../../effects/browser/module.mjs)). **That -batching is premature optimization and should probably not exist.** - -It was not added because anyone found the suite slow. It was added in a review -round, because without it the page renders nothing until the run finishes, and -that *looked* wrong. Observing a behaviour is not the same as someone having a -problem with it: nobody has reported a stall, and the number has already been -argued down from 25 to 10 with no measurement on either side of the change — -which is the shape of an optimization nobody can evaluate. - -`fjs t` is the reference and it schedules nothing at all. It starts every leaf -of a module at once, prints results as they land, and no one has complained. The -browser runner sharing its semantics but not its scheduling is a difference that -has to justify itself, and so far it has not. - -### The experiment, and what it measured - -Run in headless Chromium against the generated page, 3435 proofs each time, -sampling the run's state every 150 ms. "Progress steps" counts how many distinct -result-counts a reader ever sees — how often the page visibly moves. - -| `all` schedules | run | first row | progress steps | -| --- | ---: | ---: | ---: | -| no yielding at all (`Promise.all`) | 39.7 s | **39.8 s** | 2 | -| slices of 10, `setTimeout` yield | 39.8 s | 4.0 s | 29 | -| every launch, `setTimeout` yield | **58.1 s** | 3.5 s | 229 | -| **every launch, `MessageChannel` yield** | **41.1 s** | 3.1 s | 91 | - -(Sampled at 150 ms, so "progress steps" is a floor and varies a little run to -run; the shape is what matters, not the digits.) - -Two things fall out, and the first is not what "remove the batching" expected: - -**Removing the yield removes the progress.** With no yielding the page shows -nothing at all until the run ends — first row at 39.8 s of a 39.7 s run. It is -not that the browser paints anyway; every operation resolves through a microtask -and a browser cannot paint between microtasks, so the whole suite is one task. -The run is not faster for it either (39.7 s against 40.2 s). - -**The grouping was never the point — the clamp was.** Yielding after *every* -launch is what a reader actually wants, and it costs about 3%: 41.1 s against -39.7 s with no yielding, and it reaches the first row sooner. It cost 45% only -through `setTimeout`, which clamps to 4 ms once nested — 3435 × 4 ms is the -entire difference. `MessageChannel` (or `scheduler.yield()` where available) has -no clamp. - -**A yield between *launches*, never between a launch and its result.** `all` -promises its children run concurrently, and a slice loop that awaited each slice -before starting the next broke that promise rather than delaying it: a child -waiting on something a later sibling produces waited for a sibling that was -never started, and the run hung with no report — on a graph the Node runner -completes. That is fixed and pinned by -`operations.allStartsEveryChildBeforeAwaiting`; the numbers above are from the -fixed loop. It is also why "yield after every result" is the wrong way to say -this: per-*result* yielding is sequential execution, which is a bigger break -than the batching it was meant to remove. - -So batching was the wrong mechanism, as suspected, but not because scheduling is -unnecessary: it was a workaround for a bad yield primitive, and the workaround is -what made grouping look necessary. `fjs t` prints each result as it lands and -that is the behaviour to match — one result, one update — which per-result -yielding gives and grouping only approximates. - -### Tasks - -- [x] Remove the batching and the yield; run the real suite in a browser and - write down what actually happens. — table above. -- [ ] Replace `batchSize` with a yield after every *launch*, over a yield - primitive with no clamp; delete the batch-size constant entirely. Keep - the launch-then-await shape — the concurrency, not just the boundary. -- [ ] Re-point `operations.allYieldsBetweenBatches` at the new behaviour: it - pins that a boundary exists between children, and should pin that one - exists after *each* child. `allStartsEveryChildBeforeAwaiting` stays as - it is — it pins the concurrency, which no scheduling change may cost. -- [ ] Check the yield primitive across browsers, and whether `scheduler.yield()` - is worth preferring where it exists. - -### Constraints - -- Whatever the answer, it is the browser interpreter's policy. The Node runner - has no frame to paint and must keep starting its children at once. -- `all` must keep answering every `Result` in the order its effects were given, - batching or not. -- `operations.allYieldsBetweenBatches` pins the current behaviour. Removing the - batching means removing that proof, not weakening it. - -### Related - -- [Browser testing](browser-testing.md) -- [Explicit browser test controls](browser-test-controls.md) -- [Share the whole runner](share-the-whole-runner.md) — the CLI runner's - scheduling is one more thing the two hosts do differently for no stated - reason. From 5a74d4d654a6c947d6dd7a7a3dae40358e0c2ea1 Mon Sep 17 00:00:00 2001 From: Sergey Shandar Date: Thu, 27 Aug 2026 10:04:02 +0000 Subject: [PATCH 13/18] emergent_testing/todo: report a test's name before running it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Every runner reports a test only once it has finished, so a running test is invisible: a slow proof looks like a hung runner, the browser page counts completions rather than naming where it is, and — the case that matters — when a proof takes the process down, the last line printed is the last test that *succeeded* and the one that broke is never named. `Reporter` has no event for it: `result` takes a `SandboxResult`, so it cannot be called before there is one. The todo adds a start event and notes that the easy part is the event; the real question is terminal output under concurrency, which is the same question in both hosts. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_016PyLwDNkPQM1uD1Tg5ApBg --- .../todo/report-before-running.md | 75 +++++++++++++++++++ 1 file changed, 75 insertions(+) create mode 100644 fjs/emergent_testing/todo/report-before-running.md diff --git a/fjs/emergent_testing/todo/report-before-running.md b/fjs/emergent_testing/todo/report-before-running.md new file mode 100644 index 000000000..ba1eba76a --- /dev/null +++ b/fjs/emergent_testing/todo/report-before-running.md @@ -0,0 +1,75 @@ +## Report a test's name before running it, not only after + +**Priority:** P2 +**Status:** open + +### Problem + +Every runner reports a test only once it has finished. `fjs t` writes +`import("./a.proof.f.mjs").proof.x(): ok, 0.3 ms` after the fact, and the +browser page appends `PASS import("a").proof.x() (0.3 ms)` the same way. A test +that is *running* is invisible. + +Three things follow from that, and the third is the one that matters: + +- **A slow test looks like a hung runner.** Nothing distinguishes "this proof + has been going for ten seconds" from "the runner stopped", so the only way to + find the slow one is to wait for it to finish and read the duration. +- **Progress is a count, not a place.** The browser page says "1247 tests + completed…" while a reader wants to know *which* one it is on. +- **A crash loses the one fact worth having.** When a proof takes the process + down — a panic through the shared traversal, an out-of-memory, a stack + overflow, a runner bug — the last line printed is the last test that + *succeeded*, and the one that actually broke is never named. That is exactly + the case where a name is worth more than a result, and it is the case where + the current design has none. + +`Reporter` has no event for it: `result` is called with a `SandboxResult`, so it +cannot be called before there is one. + +### Preliminary design + +Add a `start` (or `begin`) event to `Reporter`, called with the file and path +before the leaf is sandboxed, and let each host decide what to do with it: + +- **`fjs t`** prints the name, then completes the line with `ok`/`error` and the + duration when the result lands — the standard runner shape, and the format + `fmtImport` already produces. Interleaving is the thing to get right: leaves + run concurrently, so a half-written line cannot be left open across another + test's output. Either the name and its outcome are one deferred line with the + name shown live elsewhere, or output is a two-column log that names the start + and closes it by identifier. +- **The browser page** renders a row in a pending state and settles it in place, + which is the same list it renders now with one more state per row. +- **`TestResult`** may not need to change at all: a start is an event, not a + result. Whether `report`/`reported` grow a sibling operation or the existing + one gains a status is part of the design. + +The `Reporter` change is small; the interleaving question is the real one, and +it is the same question in both hosts, which is an argument for settling it in +the shared core rather than twice. + +### Constraints + +- A start event must not cost a `sandbox` call or a clock read of its own: the + duration reported is still the sandboxed one. +- Concurrency stays. Naming a test before running it must not serialize the + suite to keep the output tidy. +- Whatever is emitted has to be as useful to an automated consumer as to a + reader — a start with no matching result is precisely the signal a crashed + run leaves behind, and a controller should be able to read it. + +### Tasks + +- [ ] Add the start event to `Reporter` and call it from `runModule` before the + leaf is sandboxed. +- [ ] Decide the terminal format for concurrent output, and prove it. +- [ ] Render a pending row in the browser page and settle it in place. +- [ ] Prove that a run killed mid-test leaves the running test's name behind. + +### Related + +- [Share the whole runner](share-the-whole-runner.md) — reporting is one of the + things each host still does its own way. +- [Hostile proof values](hostile-proof-values.md) — the crash case this would + make diagnosable, where today the run ends with no summary and no name. From 09bac1ff5cc6dceb87a5664a10d71ea96660c888 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 27 Aug 2026 10:14:35 +0000 Subject: [PATCH 14/18] emergent_testing/browser: assert the yield on the queue all posts to MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `allYieldsBetweenLaunches` queued a `setTimeout(0)` before calling `all` and asserted the timer had fired by the time `all` resolved. That reads as engine-neutral and is not: bun delivers every pending `MessageChannel` message before it runs a due timer, so the 59 yields `all` performs keep a `setTimeout(0)` queued behind them indefinitely. Confirmed directly — 59 yields, then 100000 more, and the timer never fires under bun 1.3.11, while node fires it on the first. The proof was calling a yielding `all` a non-yielding one, and bun CI failed on it. Queue a `MessageChannel` message instead, which is the same queue `all` posts to, so the assertion states the property every engine agrees on: a launch ends the task, so anything already queued runs before `all` resolves. Still load-bearing — deleting the yield from `runYielding` fails it under bun. effects/browser/module.mjs is unchanged; the browser behaviour measured in Chromium is what it was. Also files a todo for browser timer precision: `performance.now()` is coarsened to 100 us in Chromium and rounded and jittered to 1 ms in Firefox, which is at or above what a typical proof takes, so the page's per-proof durations are largely the clamp rather than a measurement. Changelog: no user-visible change. --- fjs/emergent_testing/browser/proof.mjs | 21 +++- fjs/emergent_testing/todo/browser-testing.md | 1 + fjs/emergent_testing/todo/timer-precision.md | 103 +++++++++++++++++++ 3 files changed, 120 insertions(+), 5 deletions(-) create mode 100644 fjs/emergent_testing/todo/timer-precision.md diff --git a/fjs/emergent_testing/browser/proof.mjs b/fjs/emergent_testing/browser/proof.mjs index 004af30b1..8e9041fdb 100644 --- a/fjs/emergent_testing/browser/proof.mjs +++ b/fjs/emergent_testing/browser/proof.mjs @@ -435,18 +435,29 @@ export const proof = { assertEq(results.length, 11) }, // `all` hands the event loop back between launches, which is the only - // thing that lets a page paint mid-suite: a timer queued before the call + // thing that lets a page paint mid-suite: a task queued before the call // has to run before it resolves. Without the yield every child settles - // on microtasks and no timer gets a turn — which is what this asserts, + // on microtasks and no task gets a turn — which is what this asserts, // since the effects below perform nothing. + // + // The task queued here is a `MessageChannel` message rather than a + // `setTimeout`, because the two are not interchangeable across engines. + // Bun delivers port messages until none are left before it runs a due + // timer, so 59 yields there leave a `setTimeout(0)` queued behind them + // and this proof would report a yielding `all` as a non-yielding one. + // Asserting on the queue `all` actually posts to states the property + // — that a launch ends the task, so anything already queued runs — in + // terms every engine agrees on. allYieldsBetweenLaunches: async () => { - let fired = false - setTimeout(() => { fired = true }, 0) + let delivered = false + const { port1, port2 } = new MessageChannel() + port1.onmessage = () => { port1.close(); delivered = true } + port2.postMessage(0) const many = [...new Array(60).keys()].map(i => pureOk(i)) const results = unwrap(await all(...many)) assertEq(results.length, 60) assertEq(results.map(unwrap).join(','), many.map((_, i) => i).join(',')) - assert(fired, 'all resolved without yielding to the event loop') + assert(delivered, 'all resolved without yielding to the event loop') }, }, } diff --git a/fjs/emergent_testing/todo/browser-testing.md b/fjs/emergent_testing/todo/browser-testing.md index 0e6eeece8..a3f008bc0 100644 --- a/fjs/emergent_testing/todo/browser-testing.md +++ b/fjs/emergent_testing/todo/browser-testing.md @@ -192,6 +192,7 @@ workers, or visual regression testing. - [`.f.mjs` proof discovery and coverage](f-mjs-test-and-coverage.md) - [Hostile thrown values and cross-realm promises](hostile-proof-values.md) +- [Browser timer precision](timer-precision.md) - [Explicit browser test controls](browser-test-controls.md) - [authored `.f.mjs` package support](../../ci/todo/f-mjs-package-support.md) - [project roadmap](../../../todo/plan/roadmap.md) diff --git a/fjs/emergent_testing/todo/timer-precision.md b/fjs/emergent_testing/todo/timer-precision.md new file mode 100644 index 000000000..b09ff1eb2 --- /dev/null +++ b/fjs/emergent_testing/todo/timer-precision.md @@ -0,0 +1,103 @@ +## Browser timer precision makes per-proof durations mostly noise + +**Priority:** P2 +**Status:** open + +### Problem + +`sandbox` measures every proof the same way in every host — read the clock, +run the body, read it again: + +```js +const before = performance.now() +// ... +return { result, duration: after - before } +``` + +That is right for `fjs t`, where `performance.now()` resolves to well under a +microsecond. It is not right in a browser, where the same call is deliberately +degraded as a Spectre and fingerprinting mitigation: + +- Chromium coarsens `performance.now()` to **100 µs** on an ordinary page, and + to 5 µs only when the page is cross-origin isolated (`COOP`/`COEP`). +- Firefox rounds to **1 ms** by default (`privacy.reduceTimerPrecision`) and + additionally *jitters* the value, so successive reads are not merely coarse + but non-deterministic. +- WebKit coarsens as well, and the exact figure has moved between releases. + +The numbers our own suite produces put almost every proof under those clamps: +a typical leaf in the CLI report is 0.03–0.2 ms. On an ordinary Chromium page +that is one clock tick or zero, and on Firefox it is zero or one whole +millisecond of jitter. So the browser page's `(0.3 ms)` column is not a +measurement of anything — it is the clamp, rendered per row. Worse, a *total* +built by summing thousands of such rows accumulates the rounding rather than +cancelling it, so the sum can be off by a large multiple in either direction +depending on which way each read rounded. + +Note this is not the same concern as +[`now`'s monotonicity](../../effects/browser/module.mjs), which is already +handled: `performance.timeOrigin + performance.now()` cannot go backwards. A +monotonic clock can still be a coarse one, and this is about the resolution. + +### Preliminary design + +Nothing here is decided; the point of the todo is to establish what is true +before changing the measurement. + +- **Measure the clamp rather than assume it.** A proof that reads the clock in + a tight loop and reports the smallest non-zero difference tells us the real + resolution in whatever browser is running, which is a fact the report could + carry alongside the durations. It is also the honest precondition for every + option below. +- **Report a resolution, not just a duration.** If the host clock ticks at + 100 µs, a row saying `0.1 ms` is claiming precision it does not have. The + report is serializable and consumed by controllers, so a `resolution` field + would let a consumer decide what is significant instead of guessing. +- **Accumulate over a group.** The idea raised when this was filed: time a + batch of leaves with one pair of reads and divide, so the clamp is amortized + across many proofs instead of applied to each. This is speculation — it + trades a per-test number for an average, it cannot attribute a slow proof, + and it interacts with concurrency, since `all` interleaves launches and a + group's wall time would then include siblings' work. Worth prototyping, + not worth assuming. +- **Cross-origin isolation.** Serving the eventual application root with + `COOP: same-origin` and `COEP: require-corp` buys Chromium's 5 µs clock and + is a header change in the shared controller, not a design change. It does + nothing for Firefox's jitter, and it constrains what the page may embed. +- **Consider not reporting a per-proof duration in the browser at all** if + none of the above yields a number worth printing. A column that is always + the clamp is worse than no column. + +### Constraints + +- `sandbox` is the operation that executes a proof body, and both runners must + agree on it exactly or a suite means different things in different hosts. + Any change to how it measures is a change to the shared contract, not a + browser-local tweak. +- The clock must stay monotonic. Whatever replaces or supplements + `performance.now()` cannot reintroduce wall-clock time. +- A duration must not cost a second `sandbox` call or an extra scheduling + boundary: the reads are adjacent today precisely so a scheduler cannot + interleave between them. +- Whatever the browser reports has to stay serializable and comparable to what + `fjs t` reports, or the two reports cannot be diffed. + +### Tasks + +- [ ] Measure the actual `performance.now()` resolution in Chromium, Firefox + and WebKit from inside the runner, and record the figures here. +- [ ] Decide whether the report carries the resolution, and whether a row + below it renders a duration at all. +- [ ] Prototype accumulated timing over a group of leaves and check what it + costs in attribution and what concurrency does to it. +- [ ] Check whether cross-origin isolation is worth the headers in the shared + controller. + +### Related + +- [Run FunctionalScript proofs inside real browsers](browser-testing.md) — the + report contract these durations belong to. +- [Report a test's name before running it](report-before-running.md) — the + other thing wrong with what a row shows. +- [Share the whole runner](share-the-whole-runner.md) — `sandbox` is shared, + so this is one decision, not two. From 6f5dd5977ad1222ed396751dc8bcec7caa7ed3df Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 27 Aug 2026 10:30:56 +0000 Subject: [PATCH 15/18] DESIGN: follow the example when porting a capability to a second context "Reuse code" is satisfiable while still getting the important half wrong: share a module, then give the new context its own rules, and the result looks unified but is two behaviours behind one name. Records that the existing implementation is the specification for a port, that a difference has to be justified rather than merely noticed, and that a problem the new context reveals is fixed for the shared code or recorded as an issue -- never worked around in one host. --- DESIGN.md | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/DESIGN.md b/DESIGN.md index 2ee621b5a..8567a7b96 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -107,8 +107,47 @@ on top of the weaker design. belongs in `fjs/path`, not inline in a loader). First search for an appropriate existing module; create a new one only if no good fit exists. This is different from DRY extraction: it is always appropriate. +- **Follow the example** — when the same capability already exists elsewhere, + match it before improving on it. See below. - **Avoid side effects and mutability.** +### Follow the example + +When a capability already exists somewhere in the repository and is being +brought to a second context — another host, another backend, another runner — +**the existing one is the specification.** Reproduce its behaviour first, +including the simplifications it made and the things it does not do. Only once +the second context matches the first is it worth asking whether either should +change. + +This is not the same as reusing code, and it is the part that is easy to skip +while believing the principle is satisfied. Sharing a module and then giving the +new context its own rules produces something that *looks* unified and is not: +two behaviours behind one name, which is worse than two implementations behind +two names, because nothing signals the difference. + +The rule has three consequences worth stating outright. + +**A difference has to be justified, not merely noticed.** "The new context can +do better here" is a reason to file an issue, not a reason to diverge inside a +port. The example may be simple *for a reason* that is not visible from inside +the new context — `fjs t` runs proofs one after another, and its report is +readable, attributable and reproducible because of it. + +**A problem the new context reveals is everyone's problem.** If porting exposes +that a measurement is inaccurate, that an error loses attribution, or that an +ordering is unspecified, then it was very likely already true of the example and +merely easier to see now. Fix it once, for both, as its own change — or record +it as an issue. Fixing it only in the new context leaves the two out of step and +hides the finding from the place that has had the defect longest. + +**Solve it for the shared code or not at all.** A workaround that lives in one +host is a fork with extra steps. Either the shared layer learns the answer, or +the issue stays open and honest. + +The order, then, is: reuse and match the example; land that; *then* take the +new problems one at a time, as changes that apply everywhere. + ### Exception to DRY: performance measurement Time measurement must capture immediately after an operation completes to avoid From 565f4cee73d8592ba608b563f6b0dc7b86a647d0 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 27 Aug 2026 10:44:03 +0000 Subject: [PATCH 16/18] Revert the shared runner; keep what it taught Restores every code file to main. The unified runner worked -- one `runModuleMap`, an `effects/common` layer, a browser interpreter, 100% coverage, green CI, 3435 proofs executed in Chromium -- but the way it got there is not a way worth repeating, and the record of why is worth more than the code was. What is left is the knowledge: - `share-browser-console-runner.md` is restored and now carries a "How to do this" section: `fjs t` is the specification including the things it does not do; sharing modules while keeping per-host rules looks like success and is not; the browser is sequential for a reason; a problem the port reveals is everyone's problem and belongs in an issue, not in a workaround inside the port. It also traces the six review rounds that a single unrequested batch-size constant produced, all the way to the bun failure, and ends where copying the example would have started. - `DESIGN.md` section 4 gains "Follow the example", the general form of the same rule. - Four issues the attempt surfaced stay, rewritten to describe the code as it is on main rather than as the branch left it: `hostile-proof-values.md`, `imports-promises-realms.md`, `report-before-running.md`, `timer-precision.md`. No code changes: the diff against main is documentation only. --- changelog/unreleased/1737.md | 7 - fjs/effects/README.md | 35 - fjs/effects/browser/module.mjs | 139 ---- fjs/effects/common/module.f.mjs | 217 ------ fjs/effects/common/module.mjs | 89 --- fjs/effects/common/proof.f.mjs | 168 ----- fjs/effects/common/proof.mjs | 75 -- fjs/effects/common/types.ts | 143 ---- fjs/effects/memory/types.ts | 2 +- fjs/effects/node/module.f.mjs | 216 +++++- fjs/effects/node/module.mjs | 48 +- fjs/effects/node/proof.f.mjs | 75 +- fjs/effects/node/types.ts | 130 +++- fjs/emergent_testing/README.md | 44 -- fjs/emergent_testing/browser.mjs | 455 ++++++++++++ fjs/emergent_testing/browser/module.f.mjs | 140 ---- fjs/emergent_testing/browser/module.mjs | 211 ------ fjs/emergent_testing/browser/proof.f.mjs | 162 ----- fjs/emergent_testing/browser/proof.mjs | 649 +++++++++--------- .../browser/species.proof.mjs | 45 ++ fjs/emergent_testing/browser/types.ts | 74 -- fjs/emergent_testing/module.f.mjs | 120 +--- fjs/emergent_testing/proof.f.mjs | 62 +- .../todo/browser-test-controls.md | 4 +- fjs/emergent_testing/todo/browser-testing.md | 41 +- .../todo/hostile-proof-values.md | 79 ++- .../todo/imports-promises-realms.md | 25 +- .../todo/report-before-running.md | 23 +- .../todo/share-browser-console-runner.md | 246 +++++++ .../todo/share-the-whole-runner.md | 84 --- fjs/emergent_testing/todo/timer-precision.md | 23 +- fjs/emergent_testing/types.ts | 43 +- fjs/website/module.f.mjs | 2 +- fjs/website/todo/generate-website.md | 2 +- .../todo/website-preparation-program.md | 69 -- 35 files changed, 1590 insertions(+), 2357 deletions(-) delete mode 100644 changelog/unreleased/1737.md delete mode 100644 fjs/effects/browser/module.mjs delete mode 100644 fjs/effects/common/module.f.mjs delete mode 100644 fjs/effects/common/module.mjs delete mode 100644 fjs/effects/common/proof.f.mjs delete mode 100644 fjs/effects/common/proof.mjs delete mode 100644 fjs/effects/common/types.ts create mode 100644 fjs/emergent_testing/browser.mjs delete mode 100644 fjs/emergent_testing/browser/module.f.mjs delete mode 100644 fjs/emergent_testing/browser/module.mjs delete mode 100644 fjs/emergent_testing/browser/proof.f.mjs create mode 100644 fjs/emergent_testing/browser/species.proof.mjs delete mode 100644 fjs/emergent_testing/browser/types.ts create mode 100644 fjs/emergent_testing/todo/share-browser-console-runner.md delete mode 100644 fjs/emergent_testing/todo/share-the-whole-runner.md delete mode 100644 fjs/website/todo/website-preparation-program.md diff --git a/changelog/unreleased/1737.md b/changelog/unreleased/1737.md deleted file mode 100644 index ab9258d98..000000000 --- a/changelog/unreleased/1737.md +++ /dev/null @@ -1,7 +0,0 @@ -- **BREAKING CHANGES:** `emergent_testing`: `browser.mjs` moves to - `browser/module.mjs` and now shares `fjs t`'s proof semantics; - `runBrowserProofs` and `startBrowserTests` are gone, `startBrowserTestSources` - remains. Only `instanceof Promise` values are awaited, matching `fjs t` -- `effects`: the host-independent operations (`all`, `await`, `fetch`, `import`, - `now`, `sandbox`) move to `effects/common`, re-exported unchanged from - `effects/node`; `effects/browser` interprets them in a browser realm diff --git a/fjs/effects/README.md b/fjs/effects/README.md index 5d14a0005..9cc4cd487 100644 --- a/fjs/effects/README.md +++ b/fjs/effects/README.md @@ -144,41 +144,6 @@ conflated in either direction — a capability the runner merely lacks is answer with `NotImplemented`, never by killing the program, and a refusal to continue is an interruption, never dressed up as `NotImplemented`. -## Where an operation lives - -An operation belongs to the host that alone can perform it, and to -[`./common/`](./common/module.f.mjs) when no host owns it. `all`, `await`, -`fetch`, `import`, `now` and `sandbox` describe what a JavaScript *realm* can do -— hold a value, wait for a promise, measure a call, link a module — so the Node -runner, the browser runner and the virtual runner each implement the same -command at the same contract. `readFile`, `write`, `exec`, `createServer` and -`test` describe what a *host* can do, and stay in [`./node/`](./node/types.ts). - -The line is not bookkeeping. It is what lets a program state that it needs -nothing host-specific and then be run by either host: the browser proof runner -(`fjs/emergent_testing/browser/module.f.mjs`) performs only `CommonOp` plus two -operations of its own, which is why it and `fjs t` can share every line of proof -semantics between them. `./node/` re-exports every common name, so a consumer -that already imports one module for `readFile` keeps importing it for `sandbox`. - -**Part of the interpretation is common too**, and -[`./common/module.mjs`](./common/module.mjs) holds it: `sandbox`'s -`try`/`catch`-and-measure, `await`'s promise test, and the `io` wrapper that -turns a thrown value into an `IoError`. None of them touches a host — a bare -JavaScript realm has `Promise`, a clock and a `catch` — and `sandbox` in -particular is the operation that actually *executes* a proof body, so a runner -that spelled it its own way would make a test suite mean different things in -different hosts. The two runners did have it byte-identical, with a comment in -one saying it matched the other; a comment is not a mechanism. - -An interpreter lives beside the host it interprets — [`./node/module.mjs`](./node/module.mjs), -[`./browser/module.mjs`](./browser/module.mjs) — and the browser one implements -`CommonOp` and nothing else. There is no browser filesystem and no browser -stdout, and inventing spellings for them would describe a host that does not -exist; a page that needs an operation of its own composes its handlers on top of -that map, which is why `browserOperationMap` takes the composed runner rather -than closing over one of its own. - ## Leaving the layer Not every consumer is ready to compose. Two named policies exist so that a site diff --git a/fjs/effects/browser/module.mjs b/fjs/effects/browser/module.mjs deleted file mode 100644 index 799da2792..000000000 --- a/fjs/effects/browser/module.mjs +++ /dev/null @@ -1,139 +0,0 @@ -/** - * Browser effect runner: interprets the host-independent operations - * (`../common/types.ts`) against a browser realm. - * - * It is the browser's counterpart of [`../node/module.mjs`](../node/module.mjs) - * and deliberately implements **only** `CommonOp`. There is no browser - * filesystem, no subprocess and no stdout to interpret, and inventing browser - * spellings for those would describe a host that does not exist; a page needing - * something of its own — a DOM to render into, a report to publish — composes - * its handlers on top of this map rather than finding them in it. - * - * The module has no Node dependency of any kind, so a page links it as an - * ordinary ES module with no bundling or transpilation. - * - * @module - * - * @import { Effect, ToAsyncOperationMap } from '../types.ts' - * @import { Result } from '../../types/result/types.ts' - * @import { CommonOp, Module } from '../common/types.ts' - */ - -import { awaitPromise, io, sandbox } from '../common/module.mjs' -import { ok } from '../../types/result/module.f.mjs' -import { toVec } from '../../types/uint8array/module.f.mjs' - -/** - * An effect runner over the operations this map is spread into. `all` runs its - * children through it rather than through a runner of its own, so an effect - * nested inside `all` reaches every handler the caller composed — not just the - * common ones. - * - * @typedef {(effect: Effect) => Promise>} CommonRun - */ - -/** - * Links a module in the page's realm. Injected so a caller can report loading - * progress, resolve a specifier against an application root, or drive the - * runner from a proof without a network; the default is the realm's own - * dynamic `import`. - * - * @typedef {(source: string) => Promise} BrowserImporter - */ - -/** - * Hands the event loop back, so the browser gets a turn. - * - * **Not `setTimeout`.** It clamps to 4 ms once nested, and a yield between every - * launch across a few thousand proofs is then minutes of pure clamp — measured - * at 58 s against 40 s on the real suite. That cost is what once made grouping - * the launches look necessary. A `MessageChannel` message is an ordinary task - * with no clamp, so the same per-launch yield costs about 3%. - * - * @type {() => Promise} - */ -const yieldToLoop = () => new Promise(resolve => { - const { port1, port2 } = new MessageChannel() - port1.onmessage = () => { port1.close(); resolve(undefined) } - port2.postMessage(0) -}) - -/** - * Starts every effect, handing the event loop back between one launch and the - * next, and answers each `Result` in the order the effects were given. - * - * **A browser needs a real task boundary to paint, and only `all` can give it - * one.** Every operation resolves through a microtask, and a browser cannot - * paint between microtasks, so without this the whole suite is a single task: - * measured on the real suite, the first result appears at 39.8 s of a 39.7 s - * run — nothing at all until the end, and no faster for it. What a launch does - * is exactly the work worth bounding, because a proof body runs synchronously - * inside `sandbox` before that handler's first `await`. - * - * **Every effect is started before any is awaited**, which is not a detail. - * `all` promises its children run concurrently, and awaiting one before - * starting the next would break that promise rather than delay it: a child - * waiting on something a later sibling produces would wait for a sibling that - * is never started, and the run would hang with no report — on a graph the Node - * runner completes. `all` says its children run concurrently and that it - * answers every `Result`; it does not say they start in the same task, which is - * what leaves the scheduling to the runner. The Node runner has no frame to - * paint and starts them all at once. - * - * There is deliberately **no batch size**. Grouping launches was a workaround - * for `setTimeout`'s clamp, and a count is the wrong measure anyway — proofs - * differ in cost by orders of magnitude, so a group of ten fast ones wastes a - * boundary while a group holding one slow one stalls regardless. With an - * unclamped yield there is no constant left to tune. - * - * @template T - * @template E - * @param {CommonRun} run - * @param {readonly Effect[]} effects - * @returns {Promise[]>} - */ -const runYielding = async (run, effects) => { - /** @type {readonly Promise>[]} */ - let started = [] - for (const effect of effects) { - if (started.length !== 0) { await yieldToLoop() } - started = [...started, run(effect)] - } - return Promise.all(started) -} - -/** - * The browser's handlers for the host-independent operations. - * - * `run` is the composed runner the caller builds — the one that also knows the - * caller's own operations — so `all` schedules its children through it. Passing - * it in rather than closing over a runner defined here is what keeps this map - * composable: a page adds handlers, and the effects nested inside `all` still - * reach them. - * - * @type {(run: CommonRun, importer?: BrowserImporter) => ToAsyncOperationMap} - */ -export const browserOperationMap = (run, importer = source => import(source)) => ({ - all: async (...effects) => ok(await runYielding(run, effects)), - await: async p => ok(await awaitPromise(p)), - fetch: url => io(async () => { - const response = await globalThis.fetch(url) - if (!response.ok) { - throw new Error(`Fetch error: ${response.status} ${response.statusText}`) - } - return toVec(new Uint8Array(await response.arrayBuffer())) - }), - // A synchronous throw from the importer — a specifier the realm rejects - // before it ever starts loading — is a load failure like any other, so it - // is caught here rather than escaping the effect it belongs to. - import: path => io(async () => importer(path)), - // `performance.timeOrigin + performance.now()`, not `Date.now()`. The - // operation means the same thing either way — milliseconds since the epoch, - // as the Node runner answers — but this one cannot go backwards. A suite - // runs for minutes, an NTP correction lands inside one, and the report's - // duration is the difference between two of these reads: with wall-clock - // time that difference can come out negative or inflated, which is what the - // deleted browser runner avoided by measuring in `performance.now()`. - now: async () => ok(performance.timeOrigin + performance.now()), - sandbox: async f => ok(await sandbox(f)), -}) diff --git a/fjs/effects/common/module.f.mjs b/fjs/effects/common/module.f.mjs deleted file mode 100644 index a513e58db..000000000 --- a/fjs/effects/common/module.f.mjs +++ /dev/null @@ -1,217 +0,0 @@ -/** - * The operations no host owns, and the helpers that read their error channel. - * - * `all` / `allOk` / `both` (concurrency), `await` (promise resolution), - * `fetch`, `import_`, `now` and `sandbox` each describe something a JavaScript - * realm can do on its own, so every runner implements them the same way: the - * Node runner in [`../node/module.mjs`](../node/module.mjs), the browser runner - * in [`../browser/module.mjs`](../browser/module.mjs), and the virtual one in - * [`../node/virtual/module.f.mjs`](../node/virtual/module.f.mjs). - * - * They lived in `../node/module.f.mjs`, which re-exports every name below so an - * existing importer keeps naming one module. What is genuinely Node's — the - * filesystem, streams, subprocesses, HTTP, an external test framework — stayed - * there. - * - * See [`./types.ts`](./types.ts) for the type-level API. - * - * @module - * - * @import { Effect, Func, NotImplemented, Operation } from '../types.ts' - * @import { Result } from '../../types/result/types.ts' - * @import { All, Await, Fetch, Import, IoChannel, IoError, IoErrorInfo, Now, Sandbox } from './types.ts' - */ - -import { do_, mapStep, pure, step } from '../module.f.mjs' -import { ok as resultOk, unwrap } from '../../types/result/module.f.mjs' - -/** - * Builds a normalized host error. The constructor exists so the shape is - * written once: every runner reports its failures through it, and a consumer - * matching on `'ioError'` knows what the payload holds. - * - * @type {(info: IoErrorInfo) => IoError} - */ -export const ioError = info => ['ioError', info] - -/** - * Normalizes a **thrown** value into an {@link IoError}: the OS error code when - * the host attached a string one, and a message that is the `Error`'s own or - * the value's string form. - * - * This is the boundary where an impure runner's `catch` becomes ordinary effect - * data. Nothing past it sees the thrown object, which is the point — a stack, a - * `cause`, and arbitrary own properties do not survive a wire hop, and a - * program that branched on them would be reading the host's implementation - * rather than the operation's contract. - * - * @type {(e: unknown) => IoError} - */ -export const toIoError = e => { - const message = e instanceof Error ? e.message : String(e) - if (typeof e !== 'object' || e === null || !('code' in e) || typeof e.code !== 'string') { - return ioError({ message }) - } - return ioError({ code: e.code, message }) -} - -/** - * True if `e` is a "file or directory does not exist" (`ENOENT`) error. - * - * Node's filesystem rejections are `Error`s carrying `code: 'ENOENT'`, which - * {@link toIoError} keeps; the virtual interpreter reports the same code for - * absent paths. Lets callers swallow only the missing-path case (e.g. a fresh - * store) while propagating genuine failures (permissions, corruption) rather - * than masking them. - * - * A {@link NotImplemented} is never "not found": a runner that cannot perform - * the operation has not looked for the path at all, so the two must not - * collapse into one benign branch — which is exactly what a bare `unknown` - * error channel used to allow. - * - * @type {(e: IoChannel) => boolean} - */ -export const isNotFound = ([tag, payload]) => - tag === 'ioError' && payload.code === 'ENOENT' - -/** - * Renders a channel error as a human line: an {@link IoError}'s own message, or - * the command name a runner could not dispatch. - * - * @type {(e: IoChannel) => string} - */ -export const errorMessage = ([tag, payload]) => - tag === 'notImplemented' ? `operation not implemented: ${payload}` : payload.message - -/** - * Renders a channel error for a **remote** caller: the command name for a - * {@link NotImplemented}, the OS error code for an `IoError`, and nothing else. - * - * {@link errorMessage} is for the operator of the program, who is entitled to - * the host's own words — including the path that failed. A protocol client is - * not, and the difference is not stylistic: `payload.message` is where the - * host puts the absolute path it could not read, so answering an MCP tool call - * with it publishes the server's filesystem layout to whoever is on the other - * end. The code (`ENOENT`, `EACCES`) says *what* went wrong without saying - * *where*, which is the part a client can act on anyway. - * - * A host that attached no code leaves nothing safe to forward, so the answer is - * the bare kind. That is deliberate: guessing which part of a free-text message - * is path-free is exactly the mistake this exists to prevent. - * - * @type {(e: IoChannel) => string} - */ -export const errorSummary = ([tag, payload]) => - tag === 'notImplemented' - ? `operation not implemented: ${payload}` - : payload.code === undefined ? 'io error' : `io error: ${payload.code}` - -// all - -/** - * To run the operation `O` should be known by the runner/engine. - * This is the reason why we merge `O` with `All` in the resulting effect. - */ -export const all = - // `Func` cannot express a variadic generic operation, so the declared type - // is written out here and `do_`'s is set aside. - /** @type {(...a: readonly Effect[]) => Effect[], NotImplemented>} */ - (/** @type {unknown} */ (do_('all'))) - -/** - * Collapses a list of results into a result of the list, keeping the **first** - * error in list order and discarding the later ones. - * - * Keeping one is what makes this a `Result` rather than a report: the callers - * that need it are chains, and a chain has one error channel. A site that wants - * every failure wants a different return type and should not reach for this. - * - * @type {(list: readonly Result[]) => Result} - */ -const okList = list => { - for (const r of list) { - if (r[0] === 'error') { return r } - } - return resultOk(list.map(unwrap)) -} - -/** - * {@link all} in the `ok` channel: collects the values when every effect - * succeeded, and answers with the first failure otherwise. - * - * `all` alone cannot serve a fallible chain. Its envelope is the runner's - * (`OpResult`, saying whether the *operation* could be dispatched), so handing - * it `Effect`s nests one `Result` inside another and the caller receives - * `readonly Result[]`. That has to be collapsed before the chain can - * `step` again, and a continuation that forgets to is the value-discarding - * hazard this migration exists to remove — one level in, where it is harder to - * see. - * - * **Every effect still runs.** The short-circuit is in the *result*, not in the - * execution: `all` performs them concurrently and this reads the answers once - * they are all in, so a failure does not cancel its siblings the way it stops - * the sequential `forEachStep` in `../module.f.mjs`. The error channel - * unions the runner's - * `NotImplemented` with the effects' own `E` for the same reason every other - * step does — either can be what went wrong. - * - * @type {(...a: readonly Effect[]) => Effect} - */ -export const allOk = (...a) => - step(all(...a), rs => pure(okList(rs))) - -/** - * @template {Operation} O0 - * @template T0 - * @template E0 - * @param {Effect} a - * @returns {(b: Effect) => Effect, Result], NotImplemented>} - */ -export const both = a => b => - /** @type {any} */ (all)(a, b) - -// fetch - -/** @type {Func} */ -export const fetch = do_('fetch') - -// import - -/** @type {Func} */ -export const import_ = do_('import') - -// now - -/** @type {Func} */ -export const now = do_('now') - -// sandbox - -/** - * Runs a plain synchronous function in an isolated, measured environment. - * - * Combines try/catch and high-resolution timing into a single atomic operation. - * Only plain synchronous functions are accepted — no effects, no promises. - * - * Using a single operation rather than separate `TryCatch` + `Perf` effects is - * necessary for correctness: effects execute as async tasks, so the scheduler - * can insert arbitrary work between two separate timing calls, making the - * measured delta inaccurate. Here the clock reads happen synchronously around - * the function call with nothing in between. - * - * Future parameters (time limit, memory limit) can be added to the payload - * without breaking the API. Worker-based implementations can enforce hard - * limits via worker termination. - * - * @see {@link SandboxResult} - * - * @type {Func} - */ -export const sandbox = do_('sandbox') - -/** @type {Func} */ -const awaitPromise = do_('await') - -/** @type {(p: unknown) => Effect} */ -export const awaitIfPromise = p => - mapStep(awaitPromise(p), ([x]) => x) diff --git a/fjs/effects/common/module.mjs b/fjs/effects/common/module.mjs deleted file mode 100644 index 61240f061..000000000 --- a/fjs/effects/common/module.mjs +++ /dev/null @@ -1,89 +0,0 @@ -/** - * The impure half of the host-independent operations: the three handlers every - * runner would otherwise write for itself. - * - * `../common/module.f.mjs` holds the *constructors* for `all`, `await`, `fetch`, - * `import`, `now` and `sandbox`; this holds the parts of their *interpretation* - * that are the same wherever they run. Nothing here touches a host: `sandbox` - * needs a `try`/`catch`, a clock and `Promise`, `await` needs `Promise`, and - * `io` needs a `catch` and the normalizer — all of which a bare JavaScript realm - * has. What differs between hosts is `fetch`, `import`, the clock's epoch and - * the concurrency policy, and those stay in each runner. - * - * It exists because the two runners had `sandbox`, `io` and the `await` body - * byte-identical, with a comment in one saying it matched the other. That is the - * drift this layer is meant to remove, and a comment is not a mechanism. - * - * @module - * - * @import { IoResult, SandboxResult } from './types.ts' - * @import { Result } from '../../types/result/types.ts' - */ - -import { toIoError } from './module.f.mjs' -import { error, ok } from '../../types/result/module.f.mjs' -import { asyncTryCatch } from '../../types/result/module.mjs' - -/** - * Performs host IO, reporting a thrown failure as an {@link IoResult} error. - * - * The one place where an exception becomes ordinary effect data, normalized so - * that nothing past it sees the thrown object — a stack, a `cause` and - * arbitrary own properties do not survive a wire hop. - * - * @template T - * @param {() => Promise} f - * @returns {Promise>} - */ -export const io = async f => { - const r = await asyncTryCatch(f) - return r[0] === 'ok' ? r : error(toIoError(r[1])) -} - -/** - * Runs `f` and measures it: a genuine `Promise` is awaited and a rejection is - * caught, and any other value — a proof tree carrying a `then` property - * included — is the result as it stands. - * - * **This is the operation that actually executes a proof body**, so every runner - * has to agree on it exactly or a test suite means different things in different - * hosts. That is why it is here rather than written once per runner: the two - * copies it replaces were identical, and nothing but a review would have caught - * them drifting apart. - * - * The clock is read either side of the call with nothing in between, which is - * the whole reason `sandbox` is one operation rather than a `tryCatch` and a - * `now` a scheduler could interleave. - * - * @template T - * @param {() => T} f - * @returns {Promise>} - */ -export const sandbox = async f => { - /** @type {Result} */ - let result - let after - const before = performance.now() - try { - let p = f() - after = performance.now() - if (p instanceof Promise) { - p = await p - after = performance.now() - } - result = ok(p) - } catch (e) { - after = performance.now() - result = error(e) - } - return { result, duration: after - before } -} - -/** - * Resolves a real `Promise` and hands anything else back untouched, in the - * one-element tuple the `await` operation answers with. - * - * @type {(p: unknown) => Promise} - */ -export const awaitPromise = async p => - [p instanceof Promise ? await p : p] diff --git a/fjs/effects/common/proof.f.mjs b/fjs/effects/common/proof.f.mjs deleted file mode 100644 index 1ea0435a9..000000000 --- a/fjs/effects/common/proof.f.mjs +++ /dev/null @@ -1,168 +0,0 @@ -/** - * Proofs for the host-independent operations and the helpers that read their - * error channel. - * - * The operations are proved against a stand-in interpreter declared here rather - * than against a host runner: what this module owns is the *constructors* and - * the `ok`-channel collapse, and a proof that reached for `../node/virtual` - * would be reading a Node runner's answers to decide whether `all` builds the - * right node. Each host runner proves its own handlers — `../node/proof.f.mjs` - * for the virtual and Node ones, `../../emergent_testing/browser/proof.mjs` for - * the browser one. - * - * @import { Effect } from '../types.ts' - * @import { Result } from '../../types/result/types.ts' - * @import { MemOperationMap, RunInstance } from '../mock/types.ts' - * @import { CommonOp, SandboxResult } from './types.ts' - */ - -import { assert, assertEq } from '../../asserts/module.f.mjs' -import { - all, allOk, awaitIfPromise, both, errorMessage, errorSummary, fetch, import_, - ioError, isNotFound, now, sandbox, toIoError, -} from './module.f.mjs' -import { run as mockRun } from '../mock/module.f.mjs' -import { error, ok, unwrap } from '../../types/result/module.f.mjs' -import { vec8 } from '../../types/bit_vec/module.f.mjs' - -/** The one number the stand-in clock ever answers. */ -const fixedNow = 1_700_000_000 - -/** @type {MemOperationMap} */ -const map = { - all: (...a) => state => [state, ok(a.map(i => common(state)(i)[1]))], - await: p => state => [state, ok([p])], - fetch: url => state => [ - state, - url === 'ok' ? ok(vec8(0x2An)) : error(ioError({ message: `cannot fetch ${url}` })), - ], - import: source => state => [ - state, - source === 'ok' ? ok({ value: 1 }) : error(ioError({ code: 'ENOENT', message: source })), - ], - now: () => state => [state, ok(fixedNow)], - // The same pass-through the virtual Node runner uses: a fixture returns the - // `SandboxResult` it wants reported, so an outcome is dictated rather than - // measured. - sandbox: f => state => [state, ok(/** @type {SandboxResult} */ (f()))], -} - -/** @type {RunInstance} */ -const common = mockRun(map) - -/** @type {(e: Effect) => Result} */ -const run = e => common(null)(e)[1] - -export const proof = { - // The one boundary where a runner's `catch` becomes effect data: whatever - // was thrown is reduced to a code (when the host attached a string one) - // and a message. - toIoError: { - error: () => { - assertEq(toIoError(new Error('boom'))[1].message, 'boom') - }, - withCode: () => { - const [, info] = toIoError(Object.assign(new Error('missing'), { code: 'ENOENT' })) - assertEq(info.code, 'ENOENT') - assertEq(info.message, 'missing') - }, - // A thrown non-`Error` still normalizes: the value's string form is the - // message, and there is no code to carry. - string: () => { - const [, info] = toIoError('plain') - assertEq(info.code, undefined) - assertEq(info.message, 'plain') - }, - null: () => { - assertEq(toIoError(null)[1].message, 'null') - }, - // An object whose `code` is not a string is not an OS error code, so it - // is dropped rather than carried as one. - nonStringCode: () => { - assertEq(toIoError({ code: 42 })[1].code, undefined) - }, - noCode: () => { - assertEq(toIoError({})[1].code, undefined) - }, - }, - isNotFound: { - enoent: () => { - assert(isNotFound(ioError({ code: 'ENOENT', message: 'no such file or directory' }))) - }, - otherCode: () => { - assert(!isNotFound(ioError({ code: 'EACCES', message: 'permission denied' }))) - }, - // A runner that cannot perform the operation has not looked for the - // path at all, so a missing handler is never "not found". - notImplemented: () => { - assert(!isNotFound(['notImplemented', 'readFile'])) - }, - }, - errorMessage: { - io: () => { - assertEq(errorMessage(ioError({ message: 'disk full' })), 'disk full') - }, - notImplemented: () => { - assertEq(errorMessage(['notImplemented', 'readFile']), 'operation not implemented: readFile') - }, - }, - errorSummary: { - // The distinction that matters: `errorMessage` hands back the host's - // words, which is where the path lives; `errorSummary` never does. - io: () => { - assertEq(errorSummary(ioError({ code: 'ENOENT', message: "no such file or directory, scandir '/home/u/.cas'" })), 'io error: ENOENT') - }, - ioWithoutCode: () => { - assertEq(errorSummary(ioError({ message: "cannot read '/home/u/.cas'" })), 'io error') - }, - notImplemented: () => { - assertEq(errorSummary(['notImplemented', 'readdir']), 'operation not implemented: readdir') - }, - }, - // `all` answers each effect's whole `Result`: its own envelope says only - // whether the operation could be dispatched. - all: () => { - const r = unwrap(run(all(fetch('ok'), fetch('no')))) - assertEq(r.length, 2) - assertEq(r[0]?.[0], 'ok') - assertEq(r[1]?.[0], 'error') - }, - allOk: { - // The collapse a fallible chain wants: values when every effect - // succeeded... - collects: () => { - assertEq(unwrap(run(allOk(now(), now()))).join(','), `${fixedNow},${fixedNow}`) - }, - // ...and the first failure in list order otherwise. - firstError: () => { - const r = run(allOk(fetch('no'), fetch('worse'))) - assert(r[0] === 'error', r) - assertEq(errorMessage(r[1]), 'cannot fetch no') - }, - }, - both: () => { - const [a, b] = unwrap(run(both(now())(import_('ok')))) - assertEq(unwrap(a ?? error(0)), fixedNow) - assertEq(unwrap(b ?? error(0)).value, 1) - }, - import: { - linked: () => { - assertEq(unwrap(run(import_('ok'))).value, 1) - }, - missing: () => { - const r = run(import_('nope')) - assert(r[0] === 'error', r) - assert(isNotFound(r[1]), r[1]) - }, - }, - sandbox: () => { - const { result, duration } = unwrap(run(sandbox(() => ({ result: ok(7), duration: 3 })))) - assertEq(unwrap(result), 7) - assertEq(duration, 3) - }, - // A promise is the runner's business, so what the constructor owns is - // unwrapping the one-element tuple the operation answers with. - awaitIfPromise: () => { - assertEq(unwrap(run(awaitIfPromise(5))), 5) - }, -} diff --git a/fjs/effects/common/proof.mjs b/fjs/effects/common/proof.mjs deleted file mode 100644 index a2b3a4f30..000000000 --- a/fjs/effects/common/proof.mjs +++ /dev/null @@ -1,75 +0,0 @@ -/** - * Proofs for the impure half of the host-independent operations. - * - * These three handlers are what every runner would otherwise write for itself, - * so they are proved here rather than only through whichever runner happens to - * call them — the duplication this module removed was invisible precisely - * because each copy was covered by its own host's proofs. - * - * @import { Result } from '../../types/result/types.ts' - */ - -import { assert, assertEq } from '../../asserts/module.f.mjs' -import { awaitPromise, io, sandbox } from './module.mjs' -import { errorMessage } from './module.f.mjs' -import { unwrap } from '../../types/result/module.f.mjs' - -export const proof = { - io: { - value: async () => { - assertEq(unwrap(await io(async () => 7)), 7) - }, - // The one boundary where an exception becomes ordinary effect data. - thrown: async () => { - const r = await io(async () => { throw Object.assign(new Error('nope'), { code: 'ENOENT' }) }) - assert(r[0] === 'error', r) - assertEq(errorMessage(r[1]), 'nope') - assertEq(r[1][0], 'ioError') - }, - }, - sandbox: { - value: async () => { - const { result, duration } = await sandbox(() => 1) - assertEq(unwrap(result), 1) - assert(duration >= 0, duration) - }, - thrown: async () => { - const { result } = await sandbox(() => { throw new Error('boom') }) - assert(result[0] === 'error', result) - assertEq(/** @type {Error} */ (result[1]).message, 'boom') - }, - // A real promise is awaited, and its rejection is the failure — which is - // the rule every runner has to agree on, since this is the operation - // that executes a proof body. - promise: async () => { - // The thunk is annotated because `Sandbox` declares - // `SandboxResult` while every runner resolves a real promise - // before answering, so the declared value type is `Promise` - // where the runtime value is `2`. - /** @type {() => unknown} */ - const resolves = () => Promise.resolve(2) - const { result } = await sandbox(resolves) - assertEq(unwrap(result), 2) - }, - rejected: async () => { - const { result } = await sandbox(() => Promise.reject(new Error('later'))) - assert(result[0] === 'error', result) - assertEq(/** @type {Error} */ (result[1]).message, 'later') - }, - // ...and an ordinary object carrying a `then` is a value, never a - // thenable to adopt. - thenable: async () => { - const value = { then: () => undefined } - const { result } = await sandbox(() => value) - assertEq(unwrap(result), value) - }, - }, - awaitPromise: { - promise: async () => { - assertEq((await awaitPromise(Promise.resolve(3)))[0], 3) - }, - plainValue: async () => { - assertEq((await awaitPromise(3))[0], 3) - }, - }, -} diff --git a/fjs/effects/common/types.ts b/fjs/effects/common/types.ts deleted file mode 100644 index e4acbc550..000000000 --- a/fjs/effects/common/types.ts +++ /dev/null @@ -1,143 +0,0 @@ -/** - * Types for the operations no host owns. - * - * Every operation declared here describes something a JavaScript realm can do - * on its own — hold a value, wait for a promise, measure a call, link a module, - * fetch a URL — so a Node runner, a browser runner, and the virtual runner can - * each implement the same command with the same contract. What is genuinely - * Node's — streams, the filesystem, subprocesses, an external test framework — - * stays in [`../node/types.ts`](../node/types.ts), which re-exports these so an - * existing importer keeps naming one module. - * - * @module - */ - -import type { Vec } from '../../types/bit_vec/types.ts' -import type { Effect, NotImplemented } from '../types.ts' -import type { Result } from '../../types/result/types.ts' -import type { StringMap } from '../../types/object/types.ts' - -/** - * A host failure, normalized: whatever the runtime threw reduced to a - * serializable record. `code` is the OS error code when the host supplied one - * (`'ENOENT'`, `'EEXIST'`), absent otherwise. - * - * It is a tagged tuple for the same reason {@link NotImplemented} is — the two - * share an error channel, and the tag is what tells them apart. That - * distinction is the whole reason this type exists: with a bare `unknown` - * error, `NotImplemented | unknown` collapses to `unknown` and a program can no - * longer tell "this runner cannot do it" from "the host tried and failed". - * - * Normalizing also keeps the channel serializable. A thrown `Error` carries a - * stack, a `cause`, and arbitrary own properties; none of it survives a wire - * hop, and a runner in another process could not reproduce it. - */ -export type IoError = readonly['ioError', IoErrorInfo] - -export type IoErrorInfo = { - readonly code?: string - readonly message: string -} - -/** - * The result of an operation with no failures of its own: it either produces - * its value or reports that the runner does not implement it. - * - * Every operation's return type is a `Result`, including the ones that cannot - * fail on their own terms — an operation left on a raw contract would be a hole - * in the error channel, and a runner may omit a handler for any of them. - */ -export type OpResult = Result - -/** - * The error channel of anything that performs host IO: a normalized host - * failure, or the report that the runner does not implement the operation. - * - * It is one name rather than a union spelled at each site, and that is a - * migration property rather than brevity. An effect that does no IO *yet* is - * one added `readFile` away from doing some, and if each signature names its - * own errors, that one change walks up every enclosing signature — the failure - * mode that sank `throws` clauses elsewhere, where engineers eventually - * declared everything throwing rather than maintain the cascade. Declaring the - * standard channel once is that concession made deliberately: an IO-touching - * effect says it fails *the way node IO fails*, and gaining a new way to do so - * changes nothing above it. - * - * It is not a licence to widen. An operation with failures of its own extends - * the channel (`IoChannel | ParseError`), and a computation whose errors are - * genuinely narrower should say so — this is the default for IO, not a ceiling. - */ -export type IoChannel = NotImplemented | IoError - -/** - * The result of an operation that performs host IO: its value, a normalized - * host failure, or the missing-handler report. - */ -export type IoResult = Result - -// all - -/** - * Runs its effects concurrently and answers each one's whole `Result`. - * - * The nesting is deliberate and belongs to the runner: this envelope says - * whether `all` itself could be dispatched, and each inner `Result` is what - * that effect answered. `allOk` (`./module.f.mjs`) is the collapse a fallible - * chain wants. - */ -export type All = ['all', (...effects: Effect[]) => OpResult[]>] - -// fetch - -export type Fetch = ['fetch', (url: string) => IoResult] - -// import - -export type Module = StringMap - -export type Import = ['import', (path: string) => IoResult] - -// now - -export type Now = readonly['now', () => OpResult] - -// sandbox - -/** - * The outcome of a `Sandbox` operation. - * - * `result` carries either `['ok', value]` or `['error', thrown]`. `duration` - * is a floating-point millisecond count with up to microsecond precision, - * matching `performance.now()` directly. Additional fields (allocated memory, - * max stack depth, coverage) may be added in future without breaking consumers. - */ -export type SandboxResult = { - readonly result: Result - /** - * Elapsed time in milliseconds (microsecond precision via `performance.now()`). - * The virtual runner returns `0` for deterministic tests. - */ - readonly duration: number -} - -export type Sandbox = readonly['sandbox', (f: () => T) => OpResult>] - -/** - * Resolves the return value of a test function inside the effect runner. - * If `p` is a real `Promise`, it is awaited and rejections propagate as - * throws. If `p` is any other value it is returned as-is. Plain thenables - * (objects with a `.then` method that are not `instanceof Promise`) are - * treated as ordinary values — not awaited. See `fjs/dev/tf/README.md`. - */ -export type Await = readonly['await', (p: unknown) => OpResult] - -/** - * The operations every runner is expected to be able to implement. - * - * A host runner's operation set is this union plus whatever its host adds: - * `NodeOp` is `CommonOp | MemOp | Fs | Http | …`, and the browser interpreter - * in [`../browser/module.mjs`](../browser/module.mjs) implements exactly this - * set against the browser realm. Naming it once is what lets a program say it - * needs nothing host-specific, and be run by either. - */ -export type CommonOp = All | Await | Fetch | Import | Now | Sandbox diff --git a/fjs/effects/memory/types.ts b/fjs/effects/memory/types.ts index cc72052a8..844dbb80c 100644 --- a/fjs/effects/memory/types.ts +++ b/fjs/effects/memory/types.ts @@ -6,7 +6,7 @@ import type { Phantom } from '../../types/phantom/types.ts' import type { Nominal } from '../../types/nominal/types.ts' -import type { OpResult } from '../common/types.ts' +import type { OpResult } from '../node/types.ts' /** Nominal brand version for memory keys. */ export type _MemKeyHash = '3f114fa6036a8da026b827f0c3e6d901f5e81ad9a320e431ccce31451892d286' diff --git a/fjs/effects/node/module.f.mjs b/fjs/effects/node/module.f.mjs index 50344ea11..5d1c6dddd 100644 --- a/fjs/effects/node/module.f.mjs +++ b/fjs/effects/node/module.f.mjs @@ -1,14 +1,10 @@ /** * Node.js effect operations: filesystem (`mkdir`, `readFile`, `readdir`, * `writeFile`, `rm`, `access`, plus the `readUtf8File`/`writeUtf8File` text - * helpers), HTTP (`createServer`, `listen`), subprocess `exec`, `log`/`error` - * (wrappers over `write`), `read`/`readLine`, `randomInt` and `forever`; defines - * the `NodeOp`/`NodeProgram` types used by the Node runner. - * - * The operations no host owns — `all`/`allOk`/`both`, `await`, `fetch`, - * `import_`, `now`, `sandbox`, and the `IoError` helpers — moved to - * [`../common/module.f.mjs`](../common/module.f.mjs) so the browser runner can - * link them, and are re-exported here unchanged. + * helpers), networking (`fetch`, `createServer`, `listen`), + * subprocess `exec`, `log`/`error` (wrappers over `write`), `import_`, `now`, + * `sandbox`, `forever`, and `all`/`both` parallelism; defines the + * `NodeOp`/`NodeProgram` types used by the Node runner. * * See `./types.ts` for the type-level API. * @@ -18,8 +14,7 @@ * @import { Result } from '../../types/result/types.ts' * @import { Commands, CommandSet, Effect, Func, NotImplemented, Operation } from '../types.ts' * @import { List } from '../list/types.ts' - * @import { IoError } from '../common/types.ts' - * @import { All, Access, Console, CreateExclusive, CreateServer, Dirent, Engine, Env, Exec, ExecResult, FileStat, Forever, Fs, Headers, Http, IncomingMessage, IoChannel, Listen, MakeDirectoryOptions, Mkdir, NodeOp, NodeProgramOptions, RandomInt, Read, ReadBytes, ReadConsoles, ReadFile, Readdir, ReaddirOptions, RequestListener, Rename, Rm, SandboxResult, Server, ServerResponse, Stat, Test, TestContext, TestFn, Write, WriteBytes, WriteConsoles, WriteFile, _UtfList, _WriteLoop } from './types.ts' + * @import { All, Access, Await, Console, CreateExclusive, CreateServer, Dirent, Engine, Env, Exec, ExecResult, Fetch, FileStat, Forever, Fs, Headers, Http, IncomingMessage, Import, IoChannel, IoError, IoErrorInfo, Listen, MakeDirectoryOptions, Mkdir, Module, Now, NodeOp, NodeProgramOptions, RandomInt, Read, ReadBytes, ReadConsoles, ReadFile, Readdir, ReaddirOptions, RequestListener, Rename, Rm, Sandbox, SandboxResult, Server, ServerResponse, Stat, Test, TestContext, TestFn, Write, WriteBytes, WriteConsoles, WriteFile, _UtfList, _WriteLoop } from './types.ts' */ import { utf8, utf8ToString } from '../../text/module.f.mjs' @@ -27,23 +22,20 @@ import { toCodePointList } from '../../text/utf8/module.f.mjs' import { codePointListToString } from '../../text/utf16/module.f.mjs' import { reverse } from '../../types/list/module.f.mjs' import { length } from '../../types/bit_vec/module.f.mjs' -import { error as resultError } from '../../types/result/module.f.mjs' -import { do_ } from '../module.f.mjs' +import { error as resultError, ok as resultOk, unwrap } from '../../types/result/module.f.mjs' +import { do_, pure } from '../module.f.mjs' import { mapStep as ioMapStep, pureError, pureOk, resultMapStep, resultStep, step as ioStep, } from '../module.f.mjs' -import { errorMessage, ioError } from '../common/module.f.mjs' /** - * The host-independent operations, re-exported so a caller that already names - * this module for `readFile` keeps naming it for `sandbox` and `all` too. They - * are defined in [`../common/module.f.mjs`](../common/module.f.mjs), which the - * browser runner links without reaching a Node type. + * Builds a normalized host error. The constructor exists so the shape is + * written once: every runner reports its failures through it, and a consumer + * matching on `'ioError'` knows what the payload holds. + * + * @type {(info: IoErrorInfo) => IoError} */ -export { - all, allOk, awaitIfPromise, both, errorMessage, errorSummary, fetch, import_, - ioError, isNotFound, now, sandbox, toIoError, -} from '../common/module.f.mjs' +export const ioError = info => ['ioError', info] /** * The host a {@link Listen} refuses. @@ -91,6 +83,46 @@ export const emptyHostError = ioError({ message: emptyHostMessage, }) +/** + * Normalizes a **thrown** value into an {@link IoError}: the OS error code when + * the host attached a string one, and a message that is the `Error`'s own or + * the value's string form. + * + * This is the boundary where an impure runner's `catch` becomes ordinary effect + * data. Nothing past it sees the thrown object, which is the point — a stack, a + * `cause`, and arbitrary own properties do not survive a wire hop, and a + * program that branched on them would be reading the host's implementation + * rather than the operation's contract. + * + * @type {(e: unknown) => IoError} + */ +export const toIoError = e => { + const message = e instanceof Error ? e.message : String(e) + if (typeof e !== 'object' || e === null || !('code' in e) || typeof e.code !== 'string') { + return ioError({ message }) + } + return ioError({ code: e.code, message }) +} + +/** + * True if `e` is a "file or directory does not exist" (`ENOENT`) error. + * + * Node's filesystem rejections are `Error`s carrying `code: 'ENOENT'`, which + * {@link toIoError} keeps; the virtual interpreter reports the same code for + * absent paths. Lets callers swallow only the missing-path case (e.g. a fresh + * store) while propagating genuine failures (permissions, corruption) rather + * than masking them. + * + * A {@link NotImplemented} is never "not found": a runner that cannot perform + * the operation has not looked for the path at all, so the two must not + * collapse into one benign branch — which is exactly what a bare `unknown` + * error channel used to allow. + * + * @type {(e: IoChannel) => boolean} + */ +export const isNotFound = ([tag, payload]) => + tag === 'ioError' && payload.code === 'ENOENT' + /** * `NodeOp`'s commands as data, so a runner that implements only part of them * can still tell an operation it lacks from a `Do` node whose `command` was @@ -123,6 +155,75 @@ const nodeCommandSet = { */ export const nodeCommands = /** @type {Commands} */ (Object.keys(nodeCommandSet)) +// all + +/** + * To run the operation `O` should be known by the runner/engine. + * This is the reason why we merge `O` with `All` in the resulting effect. + */ +export const all = + // `Func` cannot express a variadic generic operation, so the declared type + // is written out here and `do_`'s is set aside. + /** @type {(...a: readonly Effect[]) => Effect[], NotImplemented>} */ + (/** @type {unknown} */ (do_('all'))) + +/** + * Collapses a list of results into a result of the list, keeping the **first** + * error in list order and discarding the later ones. + * + * Keeping one is what makes this a `Result` rather than a report: the callers + * that need it are chains, and a chain has one error channel. A site that wants + * every failure wants a different return type and should not reach for this. + * + * @type {(list: readonly Result[]) => Result} + */ +const okList = list => { + for (const r of list) { + if (r[0] === 'error') { return r } + } + return resultOk(list.map(unwrap)) +} + +/** + * {@link all} in the `ok` channel: collects the values when every effect + * succeeded, and answers with the first failure otherwise. + * + * `all` alone cannot serve a fallible chain. Its envelope is the runner's + * (`OpResult`, saying whether the *operation* could be dispatched), so handing + * it `Effect`s nests one `Result` inside another and the caller receives + * `readonly Result[]`. That has to be collapsed before the chain can + * `step` again, and a continuation that forgets to is the value-discarding + * hazard this migration exists to remove — one level in, where it is harder to + * see. + * + * **Every effect still runs.** The short-circuit is in the *result*, not in the + * execution: `all` performs them concurrently and this reads the answers once + * they are all in, so a failure does not cancel its siblings the way it stops + * the sequential `forEachStep` in `./module.f.mjs`. The error channel + * unions the runner's + * `NotImplemented` with the effects' own `E` for the same reason every other + * step does — either can be what went wrong. + * + * @type {(...a: readonly Effect[]) => Effect} + */ +export const allOk = (...a) => + ioStep(all(...a), rs => pure(okList(rs))) + +/** + * @template {Operation} O0 + * @template T0 + * @template E0 + * @param {Effect} a + * @returns {(b: Effect) => Effect, Result], NotImplemented>} + */ +export const both = a => b => + /** @type {any} */ (all)(a, b) + +// fetch + +/** @type {Func} */ +export const fetch = do_('fetch') + // mkdir /** @type {Func} */ @@ -255,6 +356,11 @@ export const listen = do_('listen') /** @type {Func} */ export const forever = do_('forever') +// import + +/** @type {Func} */ +export const import_ = do_('import') + // write /** Emits a `Write` effect to the given named stream. */ @@ -324,6 +430,42 @@ export const readLine = stream => { return loop(null) } +// now + +/** @type {Func} */ +export const now = do_('now') + +// sandbox + +/** + * Runs a plain synchronous function in an isolated, measured environment. + * + * Combines try/catch and high-resolution timing into a single atomic operation. + * Only plain synchronous functions are accepted — no effects, no promises. + * + * Using a single operation rather than separate `TryCatch` + `Perf` effects is + * necessary for correctness: effects execute as async tasks, so the scheduler + * can insert arbitrary work between two separate timing calls, making the + * measured delta inaccurate. Here the clock reads happen synchronously around + * the function call with nothing in between. + * + * Future parameters (time limit, memory limit) can be added to the payload + * without breaking the API. Worker-based implementations can enforce hard + * limits via worker termination. + * + * @see {@link SandboxResult} + * + * @type {Func} + */ +export const sandbox = do_('sandbox') + +/** @type {Func} */ +const awaitPromise = do_('await') + +/** @type {(p: unknown) => Effect} */ +export const awaitIfPromise = p => + ioMapStep(awaitPromise(p), ([x]) => x) + // Test registration /** @type {Func} */ @@ -370,6 +512,38 @@ export const errorExit = s => */ export const exitCode = ([, code]) => code +/** + * Renders a channel error as a human line: an {@link IoError}'s own message, or + * the command name a runner could not dispatch. + * + * @type {(e: IoChannel) => string} + */ +export const errorMessage = ([tag, payload]) => + tag === 'notImplemented' ? `operation not implemented: ${payload}` : payload.message + +/** + * Renders a channel error for a **remote** caller: the command name for a + * {@link NotImplemented}, the OS error code for an `IoError`, and nothing else. + * + * {@link errorMessage} is for the operator of the program, who is entitled to + * the host's own words — including the path that failed. A protocol client is + * not, and the difference is not stylistic: `payload.message` is where the + * host puts the absolute path it could not read, so answering an MCP tool call + * with it publishes the server's filesystem layout to whoever is on the other + * end. The code (`ENOENT`, `EACCES`) says *what* went wrong without saying + * *where*, which is the part a client can act on anyway. + * + * A host that attached no code leaves nothing safe to forward, so the answer is + * the bare kind. That is deliberate: guessing which part of a free-text message + * is path-free is exactly the mistake this exists to prevent. + * + * @type {(e: IoChannel) => string} + */ +export const errorSummary = ([tag, payload]) => + tag === 'notImplemented' + ? `operation not implemented: ${payload}` + : payload.code === undefined ? 'io error' : `io error: ${payload.code}` + /** * Ends a program with an exit code that reflects `e`: `ok` yields `0`, and a * failure is reported on `stderr` and yields `1` ({@link errorExit}). diff --git a/fjs/effects/node/module.mjs b/fjs/effects/node/module.mjs index 6386bae9b..b523f9b3f 100644 --- a/fjs/effects/node/module.mjs +++ b/fjs/effects/node/module.mjs @@ -13,7 +13,7 @@ * @module * * @import { Effect } from '../types.ts' - * @import { Server as EffectServer, Headers, Module, NodeOp, RequestListener as Erl, NodeProgram, NodeProgramOptions, WriteConsoles, TestContext, TestFn, } from './types.ts' + * @import { IoResult, Server as EffectServer, Headers, Module, NodeOp, RequestListener as Erl, NodeProgram, NodeProgramOptions, WriteConsoles, TestContext, TestFn, } from './types.ts' * @import { Result } from '../../types/result/types.ts' * @import { StringMap } from '../../types/object/types.ts' * @import { Nullable } from '../../types/nullable/types.ts' @@ -30,7 +30,6 @@ import * as testContext from 'node:test' import { concat, normalize, toPosix } from '../../path/module.f.mjs' import { asyncRun } from '../module.mjs' -import { awaitPromise, io, sandbox } from '../common/module.mjs' import { memoryOperationMap } from './memory/module.mjs' import { emptyHost, emptyHostCode, emptyHostMessage, exitCode, toIoError, usesInlineTestContext, @@ -86,6 +85,22 @@ const createServer = http.createServer /** @typedef {(effect: Effect) => Promise>} _EffectToPromise */ +/** + * Performs host IO, reporting a thrown failure as an {@link IoResult} error. + * + * Every filesystem, network, and subprocess handler below goes through it, so + * the `catch` that turns an exception into effect data — and the normalization + * that keeps the channel serializable — happens in exactly one place. + * + * @template T + * @param {() => Promise} f + * @returns {Promise>} + */ +const io = async f => { + const r = await asyncTryCatch(f) + return r[0] === 'ok' ? r : error(toIoError(r[1])) +} + /** * Reads a request body, giving up at the `Vec` cap rather than at the point * where converting it would throw. @@ -231,6 +246,35 @@ const asyncImport = v => { return import(s1) } +/** + * @template T + * @param {() => T} f + * @returns {Promise<{ readonly result: Result, readonly duration: number }>} + */ +const sandbox = async f => { + /** @type {Result} */ + let result + let after + const before = performance.now() + try { + let p = f() + after = performance.now() + if (p instanceof Promise) { + p = await p + after = performance.now() + } + result = ok(p) + } catch (e) { + after = performance.now() + result = error(e) + } + return { result, duration: after - before } +} + +/** @type {(p: unknown) => Promise} */ +const awaitPromise = async p => + [p instanceof Promise ? await p : p] + const { now } = Date /** Maps `WriteConsoles` names to the corresponding Node.js writable streams. diff --git a/fjs/effects/node/proof.f.mjs b/fjs/effects/node/proof.f.mjs index 6c31b7f19..e04035da3 100644 --- a/fjs/effects/node/proof.f.mjs +++ b/fjs/effects/node/proof.f.mjs @@ -10,7 +10,7 @@ import { empty, isVec, uint, vec, vec8 } from "../../types/bit_vec/module.f.mjs" import { utf8, utf8ToString } from "../../text/module.f.mjs" import { match } from "../module.f.mjs" import { mapStep, step as ioStep } from "../module.f.mjs" -import { both, exitStep, fetch, mkdir, now, readdir, readFile, readUtf8File, rm, sandbox, writeFile, writeUtf8File, rename, readBytes, randomInt, writeFromStream, usesInlineTestContext, versionLessThan } from "./module.f.mjs" +import { both, errorMessage, errorSummary, exitStep, fetch, ioError, isNotFound, mkdir, now, readdir, readFile, readUtf8File, rm, sandbox, toIoError, writeFile, writeUtf8File, rename, readBytes, randomInt, writeFromStream, usesInlineTestContext, versionLessThan } from "./module.f.mjs" import { create as memCreate, read as memRead, write as memWrite } from "../memory/module.f.mjs" import { empty as listEmpty, nonEmpty as listNonEmpty } from "../list/module.f.mjs" import { emptyState, virtual } from "./virtual/module.f.mjs" @@ -50,8 +50,77 @@ const assertOk = (r, expected) => { } export const proof = { - // `toIoError`, `isNotFound`, `errorMessage` and `errorSummary` are proved - // in `../common/proof.f.mjs`, beside the module that now defines them. + // The one boundary where a runner's `catch` becomes effect data: whatever + // was thrown is reduced to a code (when the host attached a string one) + // and a message. + toIoError: { + error: () => { + assertIoMessage(toIoError(new Error('boom')), 'boom') + }, + withCode: () => { + const e = toIoError(Object.assign(new Error('missing'), { code: 'ENOENT' })) + assert(e[0] === 'ioError', e) + assertEq(e[1].code, 'ENOENT', e) + assertEq(e[1].message, 'missing', e) + }, + // A thrown non-`Error` still normalizes: the value's string form is the + // message, and there is no code to carry. + string: () => { + const e = toIoError('plain') + assert(e[0] === 'ioError', e) + assertEq(e[1].code, undefined, e) + assertEq(e[1].message, 'plain', e) + }, + null: () => { + assertIoMessage(toIoError(null), 'null') + }, + // An object whose `code` is not a string is not an OS error code, so it + // is dropped rather than carried as one. + nonStringCode: () => { + const e = toIoError({ code: 42 }) + assert(e[0] === 'ioError', e) + assertEq(e[1].code, undefined, e) + }, + noCode: () => { + const e = toIoError({}) + assert(e[0] === 'ioError', e) + assertEq(e[1].code, undefined, e) + }, + }, + isNotFound: { + enoent: () => { + assert(isNotFound(ioError({ code: 'ENOENT', message: 'no such file or directory' }))) + }, + otherCode: () => { + assert(!isNotFound(ioError({ code: 'EACCES', message: 'permission denied' }))) + }, + // A runner that cannot perform the operation has not looked for the + // path at all, so a missing handler is never "not found". + notImplemented: () => { + assert(!isNotFound(['notImplemented', 'readFile'])) + }, + }, + errorMessage: { + io: () => { + assertEq(errorMessage(ioError({ message: 'disk full' })), 'disk full') + }, + notImplemented: () => { + assertEq(errorMessage(['notImplemented', 'readFile']), 'operation not implemented: readFile') + }, + }, + errorSummary: { + // The distinction that matters: `errorMessage` hands back the host's + // words, which is where the path lives; `errorSummary` never does. + io: () => { + assertEq(errorSummary(ioError({ code: 'ENOENT', message: "no such file or directory, scandir '/home/u/.cas'" })), 'io error: ENOENT') + }, + ioWithoutCode: () => { + assertEq(errorSummary(ioError({ message: "cannot read '/home/u/.cas'" })), 'io error') + }, + notImplemented: () => { + assertEq(errorSummary(['notImplemented', 'readdir']), 'operation not implemented: readdir') + }, + }, exitStep: { // The exit-code policy a `NodeProgram` ends with: success is `0`... ok: () => { diff --git a/fjs/effects/node/types.ts b/fjs/effects/node/types.ts index 459c745d8..886a045f3 100644 --- a/fjs/effects/node/types.ts +++ b/fjs/effects/node/types.ts @@ -6,23 +6,86 @@ import type { List as EffectList } from '../../types/list/types.ts' import type { Vec } from '../../types/bit_vec/types.ts' -import type { All, Await, CommonOp, IoChannel, IoResult, OpResult } from '../common/types.ts' import type { MemOp } from '../memory/types.ts' import type { Nominal } from '../../types/nominal/types.ts' +import type { Result } from '../../types/result/types.ts' import type { StringMap } from '../../types/object/types.ts' -import type { Effect, Operation, ToAsyncOperationMap } from '../types.ts' +import type { Effect, NotImplemented, Operation, ToAsyncOperationMap } from '../types.ts' import type { List } from '../list/types.ts' /** - * The operations no host owns, re-exported so a consumer that already names - * this module for `ReadFile` keeps naming it for `Sandbox` and `All` too. They - * are declared in [`../common/types.ts`](../common/types.ts), which the browser - * runner reads without reaching a Node type. + * A host failure, normalized: whatever the runtime threw reduced to a + * serializable record. `code` is the OS error code when the host supplied one + * (`'ENOENT'`, `'EEXIST'`), absent otherwise. + * + * It is a tagged tuple for the same reason {@link NotImplemented} is — the two + * share an error channel, and the tag is what tells them apart. That + * distinction is the whole reason this type exists: with a bare `unknown` + * error, `NotImplemented | unknown` collapses to `unknown` and a program can no + * longer tell "this runner cannot do it" from "the host tried and failed". + * + * Normalizing also keeps the channel serializable. A thrown `Error` carries a + * stack, a `cause`, and arbitrary own properties; none of it survives a wire + * hop, and a runner in another process could not reproduce it. + */ +export type IoError = readonly['ioError', IoErrorInfo] + +export type IoErrorInfo = { + readonly code?: string + readonly message: string +} + +/** + * The result of an operation with no failures of its own: it either produces + * its value or reports that the runner does not implement it. + * + * Every operation's return type is a `Result`, including the ones that cannot + * fail on their own terms — an operation left on a raw contract would be a hole + * in the error channel, and a runner may omit a handler for any of them. + */ +export type OpResult = Result + +/** + * The error channel of anything that performs host IO: a normalized host + * failure, or the report that the runner does not implement the operation. + * + * It is one name rather than a union spelled at each site, and that is a + * migration property rather than brevity. An effect that does no IO *yet* is + * one added `readFile` away from doing some, and if each signature names its + * own errors, that one change walks up every enclosing signature — the failure + * mode that sank `throws` clauses elsewhere, where engineers eventually + * declared everything throwing rather than maintain the cascade. Declaring the + * standard channel once is that concession made deliberately: an IO-touching + * effect says it fails *the way node IO fails*, and gaining a new way to do so + * changes nothing above it. + * + * It is not a licence to widen. An operation with failures of its own extends + * the channel (`IoChannel | ParseError`), and a computation whose errors are + * genuinely narrower should say so — this is the default for IO, not a ceiling. + */ +export type IoChannel = NotImplemented | IoError + +/** + * The result of an operation that performs host IO: its value, a normalized + * host failure, or the missing-handler report. + */ +export type IoResult = Result + +// all + +/** + * Runs its effects concurrently and answers each one's whole `Result`. + * + * The nesting is deliberate and belongs to the runner: this envelope says + * whether `all` itself could be dispatched, and each inner `Result` is what + * that effect answered. `allOk` (`./module.f.mjs`) is the collapse a fallible + * chain wants. */ -export type { - All, Await, Fetch, Import, IoChannel, IoError, IoErrorInfo, IoResult, Module, - Now, OpResult, Sandbox, SandboxResult, -} from '../common/types.ts' +export type All = ['all', (...effects: Effect[]) => OpResult[]>] + +// fetch + +export type Fetch = ['fetch', (url: string) => IoResult] // mkdir @@ -198,6 +261,12 @@ export type Http = CreateServer | Listen export type Forever = ['forever', () => OpResult] +// import + +export type Module = StringMap + +export type Import = ['import', (path: string) => IoResult] + // write /** Named output streams accepted by the `Write` effect. */ @@ -230,6 +299,40 @@ export type Read = readonly['read', (stream: ReadConsoles) => OpResult +// now + +export type Now = readonly['now', () => OpResult] + +// sandbox + +/** + * The outcome of a `Sandbox` operation. + * + * `result` carries either `['ok', value]` or `['error', thrown]`. `duration` + * is a floating-point millisecond count with up to microsecond precision, + * matching `performance.now()` directly. Additional fields (allocated memory, + * max stack depth, coverage) may be added in future without breaking consumers. + */ +export type SandboxResult = { + readonly result: Result + /** + * Elapsed time in milliseconds (microsecond precision via `performance.now()`). + * The virtual runner returns `0` for deterministic tests. + */ + readonly duration: number +} + +export type Sandbox = readonly['sandbox', (f: () => T) => OpResult>] + +/** + * Resolves the return value of a test function inside the effect runner. + * If `p` is a real `Promise`, it is awaited and rejections propagate as + * throws. If `p` is any other value it is returned as-is. Plain thenables + * (objects with a `.then` method that are not `instanceof Promise`) are + * treated as ordinary values — not awaited. See `fjs/dev/tf/README.md`. + */ +export type Await = readonly['await', (p: unknown) => OpResult] + // Test registration /** @@ -268,13 +371,18 @@ export type Test = export type NodeOp = | Access - | CommonOp + | All + | Await + | Fetch | Fs | Http | Forever + | Import | MemOp + | Now | RandomInt | Read + | Sandbox | Write | Test diff --git a/fjs/emergent_testing/README.md b/fjs/emergent_testing/README.md index 703a1c99d..883600e00 100644 --- a/fjs/emergent_testing/README.md +++ b/fjs/emergent_testing/README.md @@ -83,47 +83,9 @@ Then invoke the runner: - `bun test` - `deno test --allow-read --allow-env --allow-sys` -### The browser - -[`browser/module.mjs`](./browser/module.mjs) runs the same proofs inside a -browser realm and answers a serializable report. The generated website hosts it; -see [`todo/browser-testing.md`](./todo/browser-testing.md) for the automated -runners still to come. - You can also implement your own runner, as long as it follows the proof-tree conventions described below. -## Design: one runner, several hosts - -`fjs t` and the browser runner are **the same runner**. Discovering -zero-argument leaves, walking the tree a proof returns, the structural `throw` -expectation, resolving real promises, formatting paths and counting results all -live once, in [`module.f.mjs`](./module.f.mjs); a host supplies only two things. - -- **A `Reporter`.** It receives semantic events — one normalized `TestResult` - per leaf, and the totals — and decides how they are shown. `defaultReporter` - writes coloured lines (or GitHub annotations); `recordingReporter` hands each - result to the `report` operation, and the browser adapter renders it into the - page. A `TestResult` carries no terminal text and no DOM, so neither reporter - can smuggle presentation back into the core. -- **An effect runner.** `sandbox` is the one operation that actually *executes* - a proof body, and each host implements it against its own realm — Node in - [`../effects/node/module.mjs`](../effects/node/module.mjs), the browser in - [`../effects/browser/module.mjs`](../effects/browser/module.mjs). Both - implement it identically, because a suite that meant different things in the - two would not be one suite. - -The two runners *used* to be two implementations of the same rules, in -`module.f.mjs` and a standalone `browser.mjs`, and the rules had begun to drift. -Consult that history before adding a rule to either host: it belongs in the -core, or it is not a rule about proofs. - -External runners (`node --test`, `bun test`, `deno test`) are the one genuine -exception, and `registerModule` is why: those frameworks own scheduling and -counting, so they are handed the tree rather than driven through it. The -differences that follow from that are documented in -[`todo/661-test-runner-behavior.md`](./todo/661-test-runner-behavior.md). - ## Design: dependency-free proofs Unlike most test frameworks (Jest, Mocha, Vitest, …), a proof does **not** import @@ -270,12 +232,6 @@ to decide whether to await it. Only genuine `Promise` instances are awaited; plain *thenables* — objects with a `.then` method that are not `instanceof Promise` — are treated as ordinary return values and walked as sub-trees. -Every runner asks the same question, in the same place — the `sandbox` -operation — so a suite means the same thing under `fjs t` and in a browser. One -consequence is that a promise built in *another* realm is not `instanceof -Promise` and so is not awaited; see -[`todo/hostile-proof-values.md`](./todo/hostile-proof-values.md). - This is intentional. FunctionalScript does not allow direct `Promise` construction; `Promise` objects only arise as the return value of `async` functions (an Effect). A plain `{ then: f }` object in FunctionalScript is almost diff --git a/fjs/emergent_testing/browser.mjs b/fjs/emergent_testing/browser.mjs new file mode 100644 index 000000000..3d280f3bf --- /dev/null +++ b/fjs/emergent_testing/browser.mjs @@ -0,0 +1,455 @@ +/** + * Browser-native proof execution and report rendering. + * + * The module deliberately has no Node dependencies: generated applications + * import it directly as an ES module in the browser. + * Proof failures resolve the published report with `status: 'failed'`; an + * automated outer controller is responsible for consuming that status and + * choosing a nonzero process exit code. + * + * Every DOM entry point reaches the page through the `root` element it is + * given — `root.ownerDocument` and its `defaultView` — never through the + * runner realm's own `window`/`document`. A page embedding the suite in an + * iframe therefore renders into that frame, and a proof can drive the module + * with a stand-in root. + * + * @module + * + * @import { _TestAndPath } from './types.ts' + */ + +import { collectTests, fmtPath } from './module.f.mjs' + +/** @type {(value: unknown) => string} */ +const text = value => { + try { + return String(value) + } catch { + return 'Unknown thrown value' + } +} + +/** + * The message and stack to report a thrown value by. + * + * An Error thrown from another realm — an iframe, a worker — is not + * `instanceof Error` here, and its stack is the very thing the report exists to + * carry. What the fields say is therefore the test, not where the value was + * made: anything carrying `message` or `stack` is read as the failure it + * describes, and everything else by its own text. + * + * @type {(error: unknown) => readonly [string, string]} + */ +const errorDetails = error => { + try { + if (error !== null && (typeof error === 'object' || typeof error === 'function') + && ('message' in error || 'stack' in error)) { + const { message, stack } = /** @type {{ readonly message?: unknown, readonly stack?: unknown }} */ (error) + const described = text(message) + return [described, stack === undefined ? described : text(stack)] + } + } catch { + // Reading the fields, and asking whether they are there at all, are + // user-observable operations: revoked proxies and accessors can throw + // while the failure is inspected. + } + const fallback = text(error) + return [fallback, fallback] +} + +/** @typedef {{ readonly module: string, readonly path: string, readonly status: string, readonly duration: number, readonly message?: string, readonly stack?: string }} _BrowserTestResult */ +/** @typedef {{ readonly status: string, readonly browser: string, readonly totals: { readonly tests: number, readonly passed: number, readonly failed: number }, readonly duration: number, readonly results: readonly _BrowserTestResult[] }} BrowserTestReport */ + +/** + * Attaches the handlers with the intrinsic `then`, but answers with a promise + * of this realm instead of the one `then` returns. That result is built by + * `constructor[Symbol.species]`, which a promise can make an ordinary object: + * awaiting it would end the test before the promise it came from ever settled + * and put the species object itself in the report. + * + * The `then` call still throws — before either handler is attached — for a + * value that is not a promise or whose species construction fails, which is + * what `runPromise` reads. + * + * @type {(value: unknown, fulfilled: (value: unknown) => Promise | readonly _BrowserTestResult[], rejected: (error: unknown) => readonly _BrowserTestResult[]) => Promise} + */ +const subscribe = (value, fulfilled, rejected) => { + /** @type {(results: Promise | readonly _BrowserTestResult[]) => void} */ + let settle = () => undefined + /** @type {Promise} */ + const settled = new Promise(resolve => { settle = resolve }) + Reflect.apply(Promise.prototype.then, value, [ + /** @type {(value: unknown) => void} */ (resolved => settle(fulfilled(resolved))), + /** @type {(error: unknown) => void} */ (error => settle(rejected(error))), + ]) + return settled +} + +/** + * Reproduces the lookup `then` performs before it builds its result promise: + * `constructor`, then its `Symbol.species`. A genuine promise with a hostile + * species throws here too; an object that only claims to be a promise failed + * the brand check first and reads its `constructor` cleanly. That is what + * separates a promise nothing can subscribe to from an ordinary proof tree, + * once shadowing `constructor` has turned out to be impossible. + * + * @type {(value: unknown) => boolean} + */ +const speciesFails = value => { + try { + if (value === null || value === undefined) { return false } + const { constructor } = /** @type {{ readonly constructor?: unknown }} */ (value) + if (constructor === null || constructor === undefined) { return false } + // The species itself never matters, only whether reading it completes: + // that is the step `then` takes before it builds its result. + void /** @type {{ readonly [Symbol.species]?: unknown }} */ (constructor)[Symbol.species] + return false + } catch { + return true + } +} + +/** + * Runs the intrinsic Promise `then` only for genuine promises. The first call + * is both the native brand check and the normal await path, so arbitrary proof + * objects with a `then` key are never assimilated. + * + * A genuine Promise can still throw after passing the brand check if species + * construction fails. In that case, temporarily shadow `constructor` with the + * current realm's Promise and retry the same intrinsic call; the shadow is + * removed immediately after the handlers are attached. + * + * A promise that pins its own `constructor`, or is frozen, leaves nothing to + * shadow, so no subscription is possible at all. The species failure is then + * reported against the test that produced the promise — the same outcome + * `await` gives it in the Node runner — because a result nobody can observe is + * not a pass. A non-extensible object that merely claims to be a promise + * reaches the same dead end and is still walked as the proof tree it is. + * + * @type {(value: unknown, fulfilled: (value: unknown) => Promise | readonly _BrowserTestResult[], rejected: (error: unknown) => readonly _BrowserTestResult[]) => Promise | null} + */ +const runPromise = (value, fulfilled, rejected) => { + const call = () => subscribe(value, fulfilled, rejected) + try { + return call() + } catch (error) { + // Either `value` is not a promise and the brand check rejected it + // before any handler was attached, or it is a genuine promise that + // failed while constructing the result through Symbol.species. Only + // the second case is worth a retry, and `then` attaches nothing before + // it throws, so the retry cannot run the handlers twice. + try { + if (Object.prototype.toString.call(value) !== '[object Promise]') { return null } + } catch { + return null + } + if (value === null || (typeof value !== 'object' && typeof value !== 'function')) { return null } + /** @type {PropertyDescriptor | undefined} */ + let descriptor + try { + descriptor = Object.getOwnPropertyDescriptor(value, 'constructor') + Object.defineProperty(value, 'constructor', { value: Promise, configurable: true }) + } catch { + // Nothing to shadow, so the value is whatever its own lookup says: + // a promise that cannot be subscribed to fails on the species error + // rather than passing on a result that was never awaited, and a + // frozen spoof is an ordinary proof tree. + return speciesFails(value) ? Promise.resolve(rejected(error)) : null + } + try { + return call() + } catch { + // The intrinsic `constructor` cannot fail the retry, so the brand + // check did: `value` only claims to be a promise and is walked as + // an ordinary proof result. + return null + } finally { + try { + if (descriptor === undefined) { + Reflect.deleteProperty(value, 'constructor') + } else { + Object.defineProperty(value, 'constructor', descriptor) + } + } catch { + // The temporary property is configurable, so ordinary objects + // restore cleanly. A hostile Proxy can make restoration itself + // observable. + } + } + } +} + +/** @type {(module: string, path: readonly (string | null)[], throws: boolean, fn: () => unknown, result: (result: _BrowserTestResult) => void) => Promise} */ +const runOne = (module, path, throws, fn, result) => { + const start = performance.now() + /** @type {(value: unknown) => Promise | readonly _BrowserTestResult[]} */ + const passed = value => { + const duration = performance.now() - start + if (throws) { + const failure = { module, path: fmtPath(path), status: 'failed', duration, + message: 'Expected the proof to throw', stack: '' } + result(failure) + return [failure] + } + // Reading the returned tree runs user code: an enumerable getter + // or a proxy trap can throw. That is a failure of the test that + // produced the value, never of the run — a rejected run leaves the + // page in `running` with no report and no completion event. + /** @type {readonly _TestAndPath[]} */ + let children + try { + children = collectTests([...path, null], false, value) + } catch (error) { + return failed(error) + } + return Promise.all(children.map(([childPath, child]) => + runOne(module, childPath, child.throws, child.fn, result) + )).then(results => { + const success = { module, path: fmtPath(path), status: 'passed', duration } + result(success) + return [success, ...results.flat()] + }) + } + /** @type {(error: unknown) => readonly _BrowserTestResult[]} */ + const failed = error => { + const duration = performance.now() - start + if (throws) { + const success = { module, path: fmtPath(path), status: 'passed', duration } + result(success) + return [success] + } + const [message, stack] = errorDetails(error) + const failure = { module, path: fmtPath(path), status: 'failed', duration, message, stack } + result(failure) + return [failure] + } + // Wrap the raw return so Promise resolution does not assimilate arbitrary + // objects with a `then` proof property. The Node runner awaits only actual + // promises, and browser execution must preserve that same test-tree rule. + return Promise.resolve().then(() => [fn()]).then( + ([value]) => runPromise(value, passed, failed) ?? passed(value), + failed + ) +} + +/** @type {(status: string, duration: number, results: readonly _BrowserTestResult[]) => BrowserTestReport} */ +const reportOf = (status, duration, results) => { + const failed = results.filter(result => result.status === 'failed').length + return { + status, + browser: navigator.userAgent, + totals: { tests: results.length, passed: results.length - failed, failed }, + duration, + results, + } +} + +/** + * Runs named proof exports and returns the serializable browser report. + * + * @type {(modules: readonly (readonly [string, unknown])[], result?: (result: _BrowserTestResult) => void) => Promise} + */ +export const runBrowserProofs = (modules, result = () => undefined) => { + const start = performance.now() + // Reporting each result as it lands is the page's own code. A renderer that + // throws must not take the run down with it: the report it fails to show is + // the one thing the page is still waiting for. + /** @type {(result: _BrowserTestResult) => void} */ + const announce = value => { + try { + result(value) + } catch { + // The result stays in the report the run resolves with. + } + } + /** @type {(module: string, error: unknown) => () => Promise} */ + const unreadable = (module, error) => () => { + const [message, stack] = errorDetails(error) + const failure = { module, path: '', status: 'failed', duration: 0, message, stack } + announce(failure) + return Promise.resolve([failure]) + } + const tests = modules.flatMap(([module, proof]) => { + // Reading an exported tree runs user code just as reading a returned + // one does. A module that cannot be enumerated is one failed module, + // never a run that ends without a report. + try { + return collectTests([], false, proof).map(([path, entry]) => + () => runOne(module, path, entry.throws, entry.fn, announce) + ) + } catch (error) { + return [unreadable(module, error)] + } + }) + const batchSize = 25 + /** @type {(index: number, results: readonly _BrowserTestResult[]) => Promise} */ + const runBatch = (index, results) => { + const batch = tests.slice(index, index + batchSize) + if (batch.length === 0) { return Promise.resolve(results) } + return Promise.all(batch.map(test => test())).then(next => + new Promise(resolve => setTimeout(resolve, 0, [...results, ...next.flat()])) + ).then(next => runBatch(index + batchSize, next)) + } + const completed = runBatch(0, []) + return completed.then(results => reportOf( + results.some(result => result.status === 'failed') ? 'failed' : 'passed', + performance.now() - start, + results, + )) +} + +/** @typedef {(source: string) => Promise<{ readonly proof?: unknown }>} _BrowserImporter */ +/** @typedef {{ readonly status: 'loaded', readonly source: string, readonly proof: unknown } | { readonly status: 'error', readonly source: string, readonly error: unknown }} _LoadedModule */ +/** @typedef {Window & { fjsBrowserTestReport?: Promise }} _TestWindow */ + +/** @type {(root: Element) => _TestWindow | null} */ +const viewOf = root => root.ownerDocument.defaultView + +/** + * Renders the settled report into the page, publishes the run as + * `fjsBrowserTestReport` on the root's window, and announces it with + * `fjs-browser-test-complete`. + * + * @type {(root: Element, report: Promise) => Promise} + */ +const publish = (root, report) => { + const view = viewOf(root) + const done = report.then(value => { + renderBrowserReport(root, value) + view?.dispatchEvent(new CustomEvent('fjs-browser-test-complete', { detail: value })) + return value + }) + if (view !== null) { view.fjsBrowserTestReport = done } + return done +} + +/** + * Loads proof modules after the page has rendered, reporting module-loading + * progress before proof execution begins. + * + * @type {(root: Element, sources: readonly string[], importer: _BrowserImporter) => Promise} + */ +export const startBrowserTestSources = (root, sources, importer) => { + const start = performance.now() + setState(root, 'loading') + let loaded = 0 + const summary = root.querySelector('[data-test-summary]') + // Set synchronously, before any import settles: otherwise the page keeps + // showing its idle text throughout loading — indefinitely, if a module + // import never settles — even though the state and control already + // changed. + if (summary !== null) { summary.textContent = `Loading 0/${sources.length}` } + // The importer is supplied by the page, so obtaining the promise is itself + // a failure point: a synchronous throw becomes a rejection here and is + // reported as a loader failure, rather than escaping past a `loading` state + // that no report or completion event ever replaces. + /** @type {(source: string) => Promise<{ readonly proof?: unknown }>} */ + const load = source => { + try { + return importer(source) + } catch (error) { + return Promise.reject(error) + } + } + /** @type {Promise} */ + const modules = Promise.all(sources.map(source => load(source).then( + module => { + loaded += 1 + if (summary !== null) { summary.textContent = `Loading ${loaded}/${sources.length}: ${source}` } + return /** @type {const} */ ({ status: 'loaded', source, proof: module.proof }) + }, + error => /** @type {const} */ ({ status: 'error', source, error }) + ))) + const report = modules.then(loadedModules => { + const rejected = loadedModules.flatMap(module => + module.status === 'error' ? [module] : []) + if (rejected.length !== 0) { + // A module that never linked has no tests to run, so the run stops + // here. Each rejection is still counted as a failed result: totals + // that disagreed with `results` would tell an automated consumer + // the suite was empty rather than broken. + const duration = performance.now() - start + return publish(root, Promise.resolve(reportOf('infrastructure-error', duration, + rejected.map(({ source, error }) => { + const [message, stack] = errorDetails(error) + return { module: source, path: '', status: 'failed', duration, message, stack } + })))) + } + return startBrowserTests(root, loadedModules.flatMap(module => + module.status === 'loaded' + ? [/** @type {const} */ ([module.source, module.proof])] + : [])) + }) + const view = viewOf(root) + if (view !== null) { view.fjsBrowserTestReport = report } + return report +} + +/** + * Sets the runner state and keeps the `Run` control's real disabled state in + * sync with it: passive while a suite is loading or running, active in every + * other state (idle, or any terminal status). A disabled attribute is used + * rather than a click handler that silently ignores the action, so assistive + * technology sees the same unavailability a sighted user does. + * + * @type {(root: Element, state: string) => void} + */ +const setState = (root, state) => { + root.setAttribute('data-state', state) + const runButton = root.querySelector('[data-test-run]') + if (runButton !== null) { + if (state === 'loading' || state === 'running') { + runButton.setAttribute('disabled', '') + } else { + runButton.removeAttribute('disabled') + } + } +} + +/** + * Renders a completed report in the browser test page. + * + * @type {(root: Element, report: BrowserTestReport) => void} + */ +export const renderBrowserReport = (root, report) => { + setState(root, report.status) + const summary = root.querySelector('[data-test-summary]') + if (summary !== null) { + summary.textContent = report.status === 'infrastructure-error' + ? `Infrastructure error: ${report.totals.failed} failed to load (${report.duration.toFixed(1)} ms)` + : `${report.totals.passed} passed, ${report.totals.failed} failed (${report.duration.toFixed(1)} ms)` + } + const output = root.querySelector('[data-test-results]') + if (output !== null) { + output.replaceChildren(...report.results.map(result => + renderResult(root.ownerDocument, result))) + } +} + +/** @type {(document: Document, result: _BrowserTestResult) => HTMLLIElement} */ +const renderResult = (document, result) => { + const item = document.createElement('li') + item.setAttribute('data-status', result.status) + const detail = result.status === 'failed' ? `: ${result.message}\n${result.stack}` : '' + item.textContent = `${result.status === 'passed' ? 'PASS' : 'FAIL'} ${result.module} ${result.path} (${result.duration.toFixed(1)} ms)${detail}` + return item +} + +/** + * Runs the application, publishes its promise as `window.fjsBrowserTestReport`, + * and dispatches `fjs-browser-test-complete` with the report in `detail`. + * + * @type {(root: Element, modules: readonly (readonly [string, unknown])[]) => Promise} + */ +export const startBrowserTests = (root, modules) => { + setState(root, 'running') + const output = root.querySelector('[data-test-results]') + if (output !== null) { output.replaceChildren() } + let completed = 0 + return publish(root, runBrowserProofs(modules, result => { + completed += 1 + const summary = root.querySelector('[data-test-summary]') + if (summary !== null) { summary.textContent = `${completed} tests completed…` } + if (output !== null) { output.append(renderResult(root.ownerDocument, result)) } + })) +} diff --git a/fjs/emergent_testing/browser/module.f.mjs b/fjs/emergent_testing/browser/module.f.mjs deleted file mode 100644 index 45718d224..000000000 --- a/fjs/emergent_testing/browser/module.f.mjs +++ /dev/null @@ -1,140 +0,0 @@ -/** - * The browser proof application: link the proof modules, run them through the - * shared emergent-testing core, and answer one serializable report. - * - * **It performs no browser operation of its own.** Linking a module, reading - * the clock, executing a proof body and recording a result are all operations - * (`./types.ts`), so this program is exactly as runnable from a proof with a - * stand-in interpreter as it is from a page. What is genuinely the browser's — - * the DOM, the published promise, the completion event — lives in the impure - * adapter beside it, [`./module.mjs`](./module.mjs). - * - * **It owns no proof semantics either.** Discovering zero-argument leaves, - * walking a returned tree, the structural `throw` expectation, resolving real - * promises and counting results are `../module.f.mjs`'s, the same module `fjs t` - * runs through — this file only decides what a *run* is: load, run, report. - * - * @module - * - * @import { Effect } from '../../effects/types.ts' - * @import { Module } from '../../effects/common/types.ts' - * @import { IoChannel, Import } from '../../effects/common/types.ts' - * @import { TestResult } from '../types.ts' - * @import { BrowserOp, BrowserProgram, BrowserTestReport, ReportStatus, _Loaded } from './types.ts' - */ - -import { allOk, errorMessage, import_, now } from '../../effects/common/module.f.mjs' -import { history, historyStep, mapStep, pureOk, resultMapStep, step } from '../../effects/module.f.mjs' -import { recordingReporter, reported, runModuleMap } from '../module.f.mjs' -import { fromEntries } from '../../types/object/module.f.mjs' -import { ok } from '../../types/result/module.f.mjs' - -/** - * Builds the report from the results a run recorded. Totals are counted here - * rather than reported separately, so they cannot disagree with `results`. - * - * @type {(status: ReportStatus, browser: string, duration: number, results: readonly TestResult[]) => BrowserTestReport} - */ -export const reportOf = (status, browser, duration, results) => { - const failed = results.filter(result => result.status === 'failed').length - return { - status, - browser, - totals: { tests: results.length, passed: results.length - failed, failed }, - duration, - results, - } -} - -/** - * The result standing for something that went wrong outside any proof: a module - * that would not link, or an operation the runner does not implement. - * - * It is counted as a failed result rather than left out. Totals that disagreed - * with `results` would tell an automated consumer the suite was empty rather - * than broken. - * - * @type {(module: string, message: string) => TestResult} - */ -const infrastructureResult = (module, message) => - ({ module, path: '', status: 'failed', duration: 0, message, stack: '' }) - -/** - * Links one source, keeping the failure rather than propagating it: a run - * reports *every* module that would not link, and the first one would - * short-circuit the rest away. - * - * @type {(source: string) => Effect} - */ -const loadOne = source => resultMapStep(import_(source), r => { - /** @type {_Loaded} */ - const loaded = [source, r] - return ok(loaded) -}) - -/** @type {(results: readonly TestResult[]) => ReportStatus} */ -const statusOf = results => - results.some(result => result.status === 'failed') ? 'failed' : 'passed' - -/** @internal What a run answers before it is timed and packaged. */ -/** @typedef {readonly[ReportStatus, readonly TestResult[]]} _Outcome */ - -/** - * Runs the modules that linked, or reports the ones that did not. - * - * A module that never linked has no tests to run, so the run stops at the first - * broken graph rather than reporting a partial suite as a complete one. - * - * @type {(loaded: readonly _Loaded[]) => Effect} - */ -const runLoaded = loaded => { - const linked = loaded.flatMap(([source, r]) => - r[0] === 'ok' ? [/** @type {const} */ ([source, r[1]])] : []) - if (linked.length !== loaded.length) { - /** @type {_Outcome} */ - const broken = ['infrastructure-error', loaded.flatMap(([source, r]) => - r[0] === 'error' ? [infrastructureResult(source, errorMessage(r[1]))] : [])] - return pureOk(broken) - } - const ran = runModuleMap(recordingReporter)(fromEntries(linked)) - const collected = step(ran, () => reported()) - return mapStep(collected, results => { - /** @type {_Outcome} */ - const outcome = [statusOf(results), results] - return outcome - }) -} - -/** @type {(sources: readonly string[]) => Effect} */ -const runSources = sources => - step(allOk(...sources.map(loadOne)), runLoaded) - -/** - * A run that could not finish, reported as one infrastructure error against the - * run itself. - * - * This is what makes {@link BrowserProgram}'s empty error channel true: a - * runner that cannot dispatch `sandbox`, `now` or `report` leaves the program - * with nothing to answer, and a page waiting on the run has nowhere to put a - * failure it never receives. - * - * @type {(browser: string, message: string) => BrowserTestReport} - */ -const failedRun = (browser, message) => - reportOf('infrastructure-error', browser, 0, [infrastructureResult('', message)]) - -/** - * The application: link every source, run the proofs that linked, and answer - * the report. - * - * @type {BrowserProgram} - */ -export const main = ({ browser, sources }) => { - const started = history(now()) - const outcome = historyStep(started, () => runSources(sources)) - const ended = historyStep(outcome, () => now()) - const report = mapStep(ended, ([end, [status, results], start]) => - reportOf(status, browser, end - start, results)) - return resultMapStep(report, r => - ok(r[0] === 'error' ? failedRun(browser, errorMessage(r[1])) : r[1])) -} diff --git a/fjs/emergent_testing/browser/module.mjs b/fjs/emergent_testing/browser/module.mjs deleted file mode 100644 index d64f162ac..000000000 --- a/fjs/emergent_testing/browser/module.mjs +++ /dev/null @@ -1,211 +0,0 @@ -/** - * The browser host adapter: capabilities, DOM rendering, and publication. - * - * It owns nothing about what a proof *means*. Walking proof trees, the - * structural `throw` expectation, resolving real promises, path formatting and - * the totals belong to `../module.f.mjs` — the module `fjs t` runs through — - * and what a *run* is belongs to the pure application in - * [`./module.f.mjs`](./module.f.mjs). What is left here is the browser: an - * interpreter for the operations that application performs, the DOM it is - * rendered into, and the promise and event a controller reads it from. - * - * The module deliberately has no Node dependency: generated applications import - * it directly as an ES module in the browser. - * Proof failures resolve the published report with `status: 'failed'`; an - * automated outer controller is responsible for consuming that status and - * choosing a nonzero process exit code. - * - * Every DOM entry point reaches the page through the `root` element it is - * given — `root.ownerDocument` and its `defaultView` — never through the - * runner realm's own `window`/`document`. A page embedding the suite in an - * iframe therefore renders into that frame, and a proof can drive the module - * with a stand-in root. - * - * @module - * - * @import { Effect } from '../../effects/types.ts' - * @import { Result } from '../../types/result/types.ts' - * @import { BrowserImporter } from '../../effects/browser/module.mjs' - * @import { TestResult } from '../types.ts' - * @import { BrowserOp, BrowserTestReport } from './types.ts' - */ - -import { asyncRun } from '../../effects/module.mjs' -import { browserOperationMap } from '../../effects/browser/module.mjs' -import { errorDetails, fmtCall } from '../module.f.mjs' -import { main, reportOf } from './module.f.mjs' -import { ok } from '../../types/result/module.f.mjs' -import { tryCatch } from '../../types/result/module.mjs' - -/** @typedef {Window & { fjsBrowserTestReport?: Promise }} _TestWindow */ - -/** @typedef {(effect: Effect) => Promise>} _Run */ - -/** - * What a run is reported as when even *describing* its panic panicked. - * - * There is nothing left to say about the value at that point — every way of - * reading it is a way of being thrown by it — so the report says exactly that - * rather than inventing a message. - */ -const unreadableFailure = 'The run failed with a value that cannot be read' - -/** @type {(root: Element) => _TestWindow | null} */ -const viewOf = root => root.ownerDocument.defaultView - -/** - * Sets the runner state and keeps the `Run` control's real disabled state in - * sync with it: passive while a suite is loading or running, active in every - * other state (idle, or any terminal status). A disabled attribute is used - * rather than a click handler that silently ignores the action, so assistive - * technology sees the same unavailability a sighted user does. - * - * @type {(root: Element, state: string) => void} - */ -const setState = (root, state) => { - root.setAttribute('data-state', state) - const runButton = root.querySelector('[data-test-run]') - if (runButton !== null) { - if (state === 'loading' || state === 'running') { - runButton.setAttribute('disabled', '') - } else { - runButton.removeAttribute('disabled') - } - } -} - -/** @type {(document: Document, result: TestResult) => HTMLLIElement} */ -const renderResult = (document, result) => { - const item = document.createElement('li') - item.setAttribute('data-status', result.status) - const detail = result.status === 'failed' ? `: ${result.message}\n${result.stack}` : '' - // `fmtCall`, so a test is named here exactly as `fjs t` names it — one - // identifier, one spelling, whichever runner is reporting. - item.textContent = `${result.status === 'passed' ? 'PASS' : 'FAIL'} ${fmtCall(result.module, result.path)} (${result.duration.toFixed(1)} ms)${detail}` - return item -} - -/** - * Renders a completed report in the browser test page. - * - * @type {(root: Element, report: BrowserTestReport) => void} - */ -export const renderBrowserReport = (root, report) => { - setState(root, report.status) - const summary = root.querySelector('[data-test-summary]') - if (summary !== null) { - summary.textContent = report.status === 'infrastructure-error' - // Not "failed to load": this status also covers a run that panicked - // and a runner missing an operation, and naming the wrong cause - // sends a reader to debug their imports. Each result below carries - // its own module and message, so the detail is not lost. - ? `Infrastructure error: ${report.totals.failed} failed (${report.duration.toFixed(1)} ms)` - : `${report.totals.passed} passed, ${report.totals.failed} failed (${report.duration.toFixed(1)} ms)` - } - const output = root.querySelector('[data-test-results]') - if (output !== null) { - output.replaceChildren(...report.results.map(result => - renderResult(root.ownerDocument, result))) - } -} - -/** - * Runs the browser application against `root`, publishes its promise as - * `fjsBrowserTestReport` on the root's window, and dispatches - * `fjs-browser-test-complete` with the report in `detail`. - * - * `importer` is the seam a controller reaches for: an application root resolves - * its own specifiers, and a proof drives the whole runner without a network. - * The default is the realm's own dynamic `import`. - * - * @type {(root: Element, sources: readonly string[], importer?: BrowserImporter) => Promise} - */ -export const startBrowserTestSources = (root, sources, importer = source => import(source)) => { - setState(root, 'loading') - const summary = root.querySelector('[data-test-summary]') - const output = root.querySelector('[data-test-results]') - if (output !== null) { output.replaceChildren() } - // Set synchronously, before any import settles: otherwise the page keeps - // showing its idle text throughout loading — indefinitely, if a module - // import never settles — even though the state and control already changed. - if (summary !== null) { summary.textContent = `Loading 0/${sources.length}` } - let loaded = 0 - /** @type {(source: string) => void} */ - const linked = source => { - loaded += 1 - if (summary !== null) { summary.textContent = `Loading ${loaded}/${sources.length}: ${source}` } - // Whether the module linked or not, the loading phase is over once the - // last answer is in — a broken graph is reported by the run, not by - // leaving the page in `loading` forever. - if (loaded === sources.length) { setState(root, 'running') } - } - /** @type {BrowserImporter} */ - const load = source => importer(source).then( - module => { linked(source); return module }, - error => { linked(source); throw error }) - /** @type {readonly TestResult[]} */ - let results = [] - /** @type {_Run} */ - const run = asyncRun({ - ...browserOperationMap(effect => run(effect), load), - report: async result => { - results = [...results, result] - // Showing a result as it lands is the page's own rendering, and it - // must not take the run down with it: the report is the one thing - // the page is still waiting for, and it is already recorded above. - try { - if (summary !== null) { summary.textContent = `${results.length} tests completed…` } - if (output !== null) { output.append(renderResult(root.ownerDocument, result)) } - } catch { - // The result stays in the report the run resolves with. - } - return ok(undefined) - }, - reported: async () => ok(results), - }) - const view = viewOf(root) - const browser = navigatorName(root) - // The application's error channel is empty — every failure it can *answer* - // is reported — so the run's `Result` is always `ok`. A **panic** is the - // other thing, and it is what `never` cannot promise away: reading a proof - // tree runs user code, so an enumerable getter or a proxy trap throws - // through the shared traversal, which has no `try`/`catch` to give it. That - // must not be where the page stops. A rejected run with the suite left in - // `running` is the one outcome an automated controller cannot act on, so - // the panic becomes the report it could not produce — see - // `../todo/hostile-proof-values.md` for attributing it to the test that - // caused it. - const settled = run(main({ browser, sources })).then( - ([, value]) => value, - error => { - // Describing the panic reads the value that caused it, and the - // value is the reason there was one: a proxy whose traps throw - // *itself* makes `errorDetails` panic in turn. This is the last - // handler there is, so it is the one that may not fail — a second - // failure here is the page stuck in `running` again, with the - // guard that was supposed to prevent it. What it cannot describe, - // it says it cannot describe. - const described = tryCatch(() => errorDetails(error)) - const [message, stack] = described[0] === 'ok' - ? described[1] - : [unreadableFailure, ''] - return reportOf('infrastructure-error', browser, 0, [ - { module: '', path: '', status: 'failed', duration: 0, message, stack }]) - }) - const report = settled.then(value => { - renderBrowserReport(root, value) - view?.dispatchEvent(new CustomEvent('fjs-browser-test-complete', { detail: value })) - return value - }) - if (view !== null) { view.fjsBrowserTestReport = report } - return report -} - -/** - * The realm the run is recorded under, read through the root's own window so an - * embedded suite names the frame it actually runs in — and so a proof driving - * the runner with a stand-in root never needs a global `navigator`. - * - * @type {(root: Element) => string} - */ -const navigatorName = root => viewOf(root)?.navigator.userAgent ?? '' diff --git a/fjs/emergent_testing/browser/proof.f.mjs b/fjs/emergent_testing/browser/proof.f.mjs deleted file mode 100644 index e46c4cf31..000000000 --- a/fjs/emergent_testing/browser/proof.f.mjs +++ /dev/null @@ -1,162 +0,0 @@ -/** - * Proofs for the browser proof application. - * - * The application performs only operations, so a state-threading stand-in - * interpreter is enough to drive every path from Node — no browser, no DOM, and - * no globals for these proofs to install and unset. `sandbox` is the same - * pass-through the virtual Node runner uses: a fixture returns the - * `SandboxResult` it wants reported, so outcomes are dictated rather than - * measured. - * - * @import { Result } from '../../types/result/types.ts' - * @import { MemOperationMap, RunInstance } from '../../effects/mock/types.ts' - * @import { Module, SandboxResult } from '../../effects/common/types.ts' - * @import { StringMap } from '../../types/object/types.ts' - * @import { TestResult } from '../types.ts' - * @import { BrowserOp, BrowserTestReport } from './types.ts' - */ - -import { assert, assertEq } from '../../asserts/module.f.mjs' -import { ioError } from '../../effects/common/module.f.mjs' -import { notImplemented } from '../../effects/module.f.mjs' -import { run as mockRun } from '../../effects/mock/module.f.mjs' -import { error, ok, unwrap } from '../../types/result/module.f.mjs' -import { main } from './module.f.mjs' - -/** - * @typedef {{ - * readonly time: number, - * readonly clock: boolean, - * readonly results: readonly TestResult[], - * readonly modules: StringMap, - * }} _State - */ - -/** @type {MemOperationMap} */ -const map = { - all: (...a) => state => { - /** @type {readonly Result[]} */ - let e = [] - for (const i of a) { - const [ns, ei] = browser(state)(i) - state = ns - e = [...e, ei] - } - return [state, ok(e)] - }, - await: p => state => [state, ok([p])], - fetch: () => state => [state, error(ioError({ message: 'no network' }))], - import: source => state => { - const module = state.modules[source] - return [ - state, - module === undefined - ? error(ioError({ code: 'ENOENT', message: `cannot link ${source}` })) - : ok(module), - ] - }, - // A clock that ticks once per read, so a run's duration is the number of - // reads between its ends and never a real elapsed time. - now: () => state => [ - { ...state, time: state.time + 1 }, - state.clock ? ok(state.time) : error(notImplemented('now')), - ], - sandbox: f => state => [state, ok(/** @type {SandboxResult} */ (f()))], - report: result => state => [{ ...state, results: [...state.results, result] }, ok(undefined)], - reported: () => state => [state, ok(state.results)], -} - -/** @type {RunInstance} */ -const browser = mockRun(map) - -/** @type {(sources: readonly string[], modules: StringMap, clock?: boolean) => BrowserTestReport} */ -const run = (sources, modules, clock = true) => { - /** @type {_State} */ - const state = { time: 100, clock, results: [], modules } - const [, report] = browser(state)(main({ browser: 'proof', sources })) - return unwrap(report) -} - -/** A leaf that passes, taking 2 ms. - * - * @type {() => unknown} - */ -const pass = () => ({ result: ok(undefined), duration: 2 }) - -/** A leaf that fails with an `Error`. - * - * @type {() => unknown} - */ -const fail = () => ({ result: error(new Error('oops')), duration: 3 }) - -export const proof = { - passing: () => { - const report = run(['a'], { a: { proof: { x: pass } } }) - assertEq(report.status, 'passed') - assertEq(report.browser, 'proof') - assertEq(report.totals.tests, 1) - assertEq(report.totals.passed, 1) - assertEq(report.totals.failed, 0) - // Two clock reads bracket the run, and the stand-in ticks once per read. - assertEq(report.duration, 1) - assertEq(report.results[0]?.module, 'a') - assertEq(report.results[0]?.path, '.x') - assertEq(report.results[0]?.duration, 2) - }, - failing: () => { - const report = run(['a'], { a: { proof: { x: pass, y: fail } } }) - assertEq(report.status, 'failed') - assertEq(report.totals.tests, 2) - assertEq(report.totals.failed, 1) - const failed = report.results.filter(r => r.status === 'failed') - assertEq(failed[0]?.path, '.y') - assertEq(failed[0]?.message, 'oops') - }, - // The proof tree a leaf returns is walked by the same shared core `fjs t` - // uses, so a sub-test is a result of its own with a call boundary in its - // path. - subTree: () => { - const report = run(['a'], { - a: { proof: { outer: () => ({ result: ok({ inner: pass }), duration: 0 }) } }, - }) - assertEq(report.totals.tests, 2) - assertEq(report.results[1]?.path, '.outer().inner') - }, - expectedThrow: () => { - const report = run(['a'], { a: { proof: { throw: { boom: fail, quiet: pass } } } }) - assertEq(report.totals.tests, 2) - assertEq(report.totals.failed, 1) - const failed = report.results.filter(r => r.status === 'failed') - assertEq(failed[0]?.path, '.throw.quiet') - assertEq(failed[0]?.message, 'Expected the proof to throw') - }, - // A module without a `proof` export contributes no tests, and an empty run - // still answers a report rather than nothing. - withoutProof: () => { - const report = run(['a'], { a: {} }) - assertEq(report.status, 'passed') - assertEq(report.totals.tests, 0) - }, - // One module that would not link stops the run: the suite never ran, so its - // status is not the one a failing suite gets, and every rejected source is - // still counted as a failed result. - unlinkable: () => { - const report = run(['a', 'missing'], { a: { proof: { x: pass } } }) - assertEq(report.status, 'infrastructure-error') - assertEq(report.totals.tests, 1) - assertEq(report.totals.failed, 1) - assertEq(report.results[0]?.module, 'missing') - assertEq(report.results[0]?.path, '') - assertEq(report.results[0]?.message, 'cannot link missing') - }, - // A runner missing an operation the application needs is reported the same - // way, which is what makes the program's empty error channel true: a page - // waiting on the run always receives a report. - incompleteRunner: () => { - const report = run(['a'], { a: { proof: { x: pass } } }, false) - assertEq(report.status, 'infrastructure-error') - assertEq(report.duration, 0) - assertEq(report.results[0]?.message, 'operation not implemented: now') - assert(report.results.length === 1, report.results) - }, -} diff --git a/fjs/emergent_testing/browser/proof.mjs b/fjs/emergent_testing/browser/proof.mjs index 8e9041fdb..c4af06c77 100644 --- a/fjs/emergent_testing/browser/proof.mjs +++ b/fjs/emergent_testing/browser/proof.mjs @@ -1,31 +1,20 @@ /** - * Proofs for the browser host adapter and the browser interpretation of the - * host-independent operations. + * Proofs for the browser runner. * - * The adapter reaches the page only through the root element it is handed, so + * The runner reaches the page only through the root element it is handed, so * the DOM stand-in below is enough to drive every rendering branch from Node — * no headless browser, and no global `window`/`document` for these proofs to - * install and unset. What each proof *means* is settled a layer down, by the - * shared core and its own proofs; what is checked here is that a browser run - * reaches it, renders it, and publishes it. - * - * @import { Result } from '../../types/result/types.ts' - * @import { Module } from '../../effects/common/types.ts' - * @import { CommonRun } from '../../effects/browser/module.mjs' - * @import { BrowserTestReport } from './types.ts' + * install and unset. */ -import { assert, assertEq, assertNotNullish } from '../../asserts/module.f.mjs' -import { browserOperationMap } from '../../effects/browser/module.mjs' -import { asyncRun } from '../../effects/module.mjs' -import { pureOk } from '../../effects/module.f.mjs' -import { all as allEffect, sandbox as sandboxEffect } from '../../effects/common/module.f.mjs' -import { renderBrowserReport, startBrowserTestSources } from './module.mjs' -import { unwrap } from '../../types/result/module.f.mjs' +import { runInNewContext } from 'node:vm' + +import { assert, assertEq, assertNotNullish, assertStructurallySame } from '../../asserts/module.f.mjs' +import { renderBrowserReport, runBrowserProofs, startBrowserTests, startBrowserTestSources } from '../browser.mjs' /** @typedef {{ readonly tag: string, attributes: ReadonlyMap, readonly ownerDocument: _Document, textContent: string, children: readonly _Element[], readonly setAttribute: (name: string, value: string) => void, readonly removeAttribute: (name: string) => void, readonly querySelector: (selector: string) => _Element | null, readonly replaceChildren: (...nodes: readonly _Element[]) => void, readonly append: (node: _Element) => void }} _Element */ /** @typedef {{ defaultView: _View | null, readonly createElement: (tag: string) => _Element }} _Document */ -/** @typedef {{ events: readonly CustomEvent[], readonly navigator: { readonly userAgent: string }, readonly dispatchEvent: (event: Event) => boolean, fjsBrowserTestReport?: Promise }} _View */ +/** @typedef {{ events: readonly CustomEvent[], readonly dispatchEvent: (event: Event) => boolean, fjsBrowserTestReport?: Promise }} _View */ /** @type {(node: _Element, name: string) => _Element | null} */ const find = (node, name) => @@ -51,7 +40,7 @@ const element = (document, tag, attributes, states) => { removeAttribute: name => { self.attributes = new Map([...self.attributes].filter(([key]) => key !== name)) }, - // The adapter only ever queries an attribute selector of `[name]` form. + // The runner only ever queries an attribute selector of `[name]` form. querySelector: selector => self.children.reduce( (/** @type {_Element | null} */ acc, child) => acc ?? find(child, selector.slice(1, -1)), @@ -63,7 +52,7 @@ const element = (document, tag, attributes, states) => { } /** - * Builds what the generated page gives the adapter: a root carrying the summary + * Builds what the generated page gives the runner: a root carrying the summary * paragraph and the result list. `states` records every `data-state` written, * so a proof can check the whole progression and not just its last step. * @@ -80,7 +69,6 @@ const page = (withView = true) => { /** @type {_View} */ const view = { events: [], - navigator: { userAgent: 'stand-in browser' }, dispatchEvent: event => { view.events = [...view.events, /** @type {CustomEvent} */ (event)] return true @@ -102,362 +90,339 @@ const page = (withView = true) => { } } -/** Runs one in-memory proof module through the whole browser stack. - * - * @type {(proof: unknown) => Promise} - */ -const run = proof => - startBrowserTestSources(page().root, ['proof'], async () => ({ proof })) +/** @type {(proof: unknown) => ReturnType} */ +const run = proof => runBrowserProofs([['proof', proof]]) /** @type {(element: _Element) => readonly (string | undefined)[]} */ const statuses = element => element.children.map(child => child.attributes.get('data-status')) -/** - * The browser handlers on their own, so the operations the proof application - * never reaches — `fetch`, `await`, a nested `all` — are still exercised. - */ -const operations = browserOperationMap( - effect => commonRun(effect), - async source => ({ source })) - -/** @type {CommonRun} */ -const commonRun = asyncRun(operations) - -const { all, await: awaitOp, fetch: fetchOp, import: importOp, now, sandbox } = operations - export const proof = { - // The whole stack: a module is linked, its proofs run, each result is - // rendered as it lands, and the report is published and announced. - passing: async () => { - const { root, summary, results, view, states } = page() - const report = await startBrowserTestSources(root, ['a'], async () => ({ - proof: { x: () => undefined }, - })) + namedThrow: async () => { + const named = { throw: () => { throw 'expected' } }.throw + const report = await run({ extracted: named }) assertEq(report.status, 'passed') - assertEq(report.browser, 'stand-in browser') - assertEq(report.totals.tests, 1) - assertEq(report.results[0]?.path, '.x') - assertEq(statuses(results).join(','), 'passed') - // The page names a test exactly as `fjs t` names it. The two spellings - // had drifted — `./a .x` here against the call expression there — which - // is the thing a shared runner is supposed to make impossible. - assert( - results.children[0]?.textContent.startsWith('PASS import("a").proof.x()'), - results.children[0]?.textContent) - assert(summary.textContent.startsWith('1 passed, 0 failed'), summary.textContent) - assertEq(states.join(','), 'loading,running,passed') - assertEq(view.events.length, 1) - assertEq(/** @type {BrowserTestReport} */ (view.events[0]?.detail).status, 'passed') - assertEq(await view.fjsBrowserTestReport, report) }, - failing: async () => { - const { root, results, runButton } = page() - const report = await startBrowserTestSources(root, ['a'], async () => ({ - proof: { boom: () => { throw new Error('bang') } }, - })) + path: async () => { + const report = await run({ 'a.b': () => undefined }) + assertEq(report.results[0]?.path, '["a.b"]') + }, + arbitraryThrow: async () => { + const report = await run({ fail: () => { throw Object.create(null) } }) assertEq(report.status, 'failed') - assertEq(report.results[0]?.message, 'bang') - assert((report.results[0]?.stack ?? '').includes('bang')) - assertEq(statuses(results).join(','), 'failed') - // The control is available again the moment the run reaches a terminal - // state, and was not while it was loading or running. - assert(!runButton.attributes.has('disabled')) + assertEq(report.results[0]?.message, 'Unknown thrown value') }, - expectedThrow: async () => { - const report = await run({ throw: { boom: () => { throw 'expected' } } }) - assertEq(report.status, 'passed') + errorFields: async () => { + const error = new Proxy(new Error(), { + get: (target, property) => property === 'message' || property === 'stack' + ? Symbol(property) + : Reflect.get(target, property), + }) + const report = await run({ fail: () => { throw error } }) + assertEq(report.results[0]?.message, 'Symbol(message)') + assertEq(report.results[0]?.stack, 'Symbol(stack)') }, - // Only a real promise is an asynchronous value, which is exactly the rule - // `fjs t` follows: the browser `sandbox` awaits one and reports what it - // resolves to. - promise: async () => { - const report = await run({ nested: () => Promise.resolve({ inner: () => undefined }) }) - assertEq(report.totals.tests, 2) - assertEq(report.totals.failed, 0) - assertEq(report.results[1]?.path, '.nested().inner') + errorAccessorThrows: async () => { + const error = new Error('hidden') + Object.defineProperty(error, 'message', { + get: () => { throw new Error('message getter failed') }, + }) + const report = await run({ fail: () => { throw error } }) + assertEq(report.status, 'failed') + assertEq(report.results[0]?.message, 'Unknown thrown value') + assertEq(report.results[0]?.stack, 'Unknown thrown value') }, - rejectedPromise: async () => { - const report = await run({ nested: () => Promise.reject(new Error('later')) }) + revokedErrorProxy: async () => { + const { proxy, revoke } = Proxy.revocable(new Error('revoked'), {}) + revoke() + const report = await run({ fail: () => { throw proxy } }) assertEq(report.status, 'failed') - assertEq(report.results[0]?.message, 'later') + assertEq(report.results[0]?.message, 'Unknown thrown value') }, - // ...and an ordinary object carrying a `then` proof is a proof tree, never - // a thenable to assimilate. - thenIsATestName: async () => { - const report = await run({ nested: () => ({ then: () => undefined }) }) + crossRealmError: async () => { + // An Error from another realm is not `instanceof Error` here, and its + // stack is what the report exists to carry. + const other = runInNewContext( + '({ fail: () => { throw new Error(\'cross boom\') } })') + const report = await run({ fail: other.fail }) + assertEq(report.results[0]?.message, 'cross boom') + const stack = report.results[0]?.stack ?? '' + assert(stack !== 'cross boom', stack) + assert(stack.includes('cross boom'), stack) + }, + errorWithoutStack: async () => { + const error = new Error('no stack') + const report = await run({ fail: () => { throw Object.assign(error, { stack: undefined }) } }) + assertEq(report.results[0]?.message, 'no stack') + assertEq(report.results[0]?.stack, 'no stack') + }, + expectedThrow: async () => { + const report = await run({ throw: { silent: () => undefined } }) + assertEq(report.status, 'failed') + assertEq(report.results[0]?.message, 'Expected the proof to throw') + }, + crossRealmPromise: async () => { + // A promise built in another realm is not `instanceof Promise`. The + // runner has to await it anyway and walk the tree it resolves to, + // otherwise a rejected cross-realm promise is reported as a pass. + const other = runInNewContext('({ resolve: value => Promise.resolve(value) })') + const report = await run({ + nested: () => other.resolve({ child: () => { throw 'boom' } }), + }) + assertEq(report.totals.tests, 2) + assertEq(report.totals.failed, 1) + assertEq(report.results[1]?.path, '.nested().child') + }, + spoofedPromiseTag: async () => { + const report = await run({ + nested: () => ({ + [Symbol.toStringTag]: 'Promise', + then: /** @type {(...args: (() => void)[]) => void} */ ((...args) => { args[0]?.() }), + }), + }) assertEq(report.totals.tests, 2) assertEq(report.results[1]?.path, '.nested().then') }, - // A module that will not link stops the run before any proof body, and says - // so with a status an automated consumer must not read as a failing suite. - unlinkable: async () => { - const { root, summary, states } = page() - const report = await startBrowserTestSources(root, ['a'], async () => { - throw new Error('404') + frozenPromiseTag: async () => { + // A non-extensible spoof leaves the runner nothing to shadow, the same + // dead end a pinned promise reaches. It is still an ordinary proof + // tree, so it is walked rather than reported as a brand-check failure. + const report = await run({ + nested: () => Object.freeze({ + [Symbol.toStringTag]: 'Promise', + then: () => undefined, + }), }) - assertEq(report.status, 'infrastructure-error') + assertEq(report.totals.tests, 2) + assertEq(report.totals.failed, 0) + assertEq(report.results[1]?.path, '.nested().then') + }, + exportedTreeThrows: async () => { + // The exported tree is read before any test runs, and reading it runs + // user code as well. The module fails; the page still gets its report. + const p = page() + const report = await startBrowserTests(p.root, + [['m', { get bad() { throw new Error('enumerating') } }]]) + assertEq(report.status, 'failed') + assertStructurallySame({ ...report.totals }, { tests: 1, passed: 0, failed: 1 }) + assertEq(report.results[0]?.module, 'm') + assertEq(report.results[0]?.message, 'enumerating') + assertStructurallySame([...p.states], ['running', 'failed']) + assertEq(p.view.events.length, 1) + }, + returnedTreeThrows: async () => { + // Reading the returned tree runs user code. When it throws, the test + // that produced the value fails and the page still reaches a terminal + // state — a rejected run would leave it in `running` forever. + const p = page() + const report = await startBrowserTests(p.root, + [['m', { nested: () => ({ get bad() { throw new Error('getter') } }) }]]) + assertEq(report.status, 'failed') + assertStructurallySame({ ...report.totals }, { tests: 1, passed: 0, failed: 1 }) + assertEq(report.results[0]?.message, 'getter') + assertStructurallySame([...p.states], ['running', 'failed']) + assertEq(p.view.events.length, 1) + }, + speciesResultIsNotAPromise: async () => { + // `then` builds its result through `constructor[Symbol.species]`, and a + // promise can make that an ordinary object. The run has to answer with + // the promise it subscribed to, not with what `then` handed back, or + // the test ends before the promise settles and the species object + // itself lands in the report. + const species = function (/** @type {(...args: (() => void)[]) => void} */ executor) { + executor(() => undefined, () => undefined) + return { notAPromise: true } + } + const promised = new Promise(resolve => + setTimeout(resolve, 1, { child: () => { throw 'boom' } })) + Object.defineProperty(promised, 'constructor', + { value: { [Symbol.species]: species }, configurable: true }) + const report = await run({ nested: () => promised }) + assertEq(report.totals.tests, 2) assertEq(report.totals.failed, 1) - assertEq(report.results[0]?.module, 'a') - assertEq(report.results[0]?.message, '404') - assert(summary.textContent.startsWith('Infrastructure error: 1 failed'), summary.textContent) - assertEq(states.join(','), 'loading,running,infrastructure-error') + assertEq(report.results[1]?.path, '.nested().child') }, - // The importer is page code, so obtaining the promise is itself a failure - // point: a synchronous throw is a load failure, not an escape past a - // `loading` state no report ever replaces. - importerThrowsSynchronously: async () => { - const { root } = page() - const report = await startBrowserTestSources(root, ['a'], () => { - throw new Error('bad specifier') - }) - assertEq(report.status, 'infrastructure-error') - assertEq(report.results[0]?.message, 'bad specifier') + reportingThrows: async () => { + // Announcing a result as it lands is the page's own rendering. It must + // not take the run down with it: the report is what the page waits for. + const report = await runBrowserProofs([['m', { t: () => undefined }]], + () => { throw new Error('render') }) + assertEq(report.status, 'passed') + assertEq(report.totals.passed, 1) }, - // A long suite runs to completion with a yield between every launch. - manyLeaves: async () => { - const proof = Object.fromEntries( - [...new Array(60).keys()].map(i => [`t${i}`, () => undefined])) - const report = await run(proof) - assertEq(report.totals.tests, 60) - assertEq(report.totals.passed, 60) + thenIsATestName: async () => { + // A `then` proof entry is a test called `then`, never a thenable for + // the runner to adopt. + const report = await run({ then: () => undefined }) + assertEq(report.totals.tests, 1) + assertEq(report.results[0]?.path, '.then') }, - // Reading the tree a proof returns runs user code, and the shared traversal - // has no `try`/`catch` to give it — so a throwing getter panics *through* - // the run. The page must still reach a terminal state and still publish a - // report: a rejected run left in `running` is the one outcome an automated - // controller cannot act on. - hostileProofTree: async () => { - const { root, view, states } = page() - const report = await startBrowserTestSources(root, ['a'], async () => ({ - proof: { hostile: () => ({ get boom() { throw new Error('trap') } }) }, - })) - assertEq(report.status, 'infrastructure-error') - assertEq(report.results[0]?.message, 'trap') - assertEq(states.join(','), 'loading,running,infrastructure-error') - assertEq(view.events.length, 1) + batches: async () => { + // More leaves than one batch holds, so the batch loop recurses. + const report = await run(Object.fromEntries( + Array.from({ length: 30 }, (_, index) => [`t${index}`, () => undefined]))) + assertEq(report.totals.tests, 30) + assertEq(report.totals.passed, 30) }, - // The summary must not keep showing idle text through loading: it is - // replaced the instant a run starts, before any import has had a chance to - // settle — even one that never does. - loadingSummaryIsSynchronous: () => { - const { root, summary } = page() - void startBrowserTestSources(root, ['a.mjs', 'b.mjs'], () => new Promise(() => undefined)) - assertEq(summary.textContent, 'Loading 0/2') + render: async () => { + const p = page() + const report = await startBrowserTests(p.root, + [['m', { ok: () => undefined, bad: () => { throw 'x' } }]]) + assertEq(report.status, 'failed') + assertStructurallySame([...p.states], ['running', 'failed']) + assertEq(p.summary.textContent, `1 passed, 1 failed (${report.duration.toFixed(1)} ms)`) + assertStructurallySame([...statuses(p.results)], ['passed', 'failed']) + const event = assertNotNullish(p.view.events[0]) + assertEq(event.type, 'fjs-browser-test-complete') + assertEq(event.detail, report) + assertEq(await p.view.fjsBrowserTestReport, report) + }, + renderWithoutView: async () => { + // A detached document has no window: the run still renders, and + // nothing is published or announced. + const p = page(false) + const report = await startBrowserTests(p.root, [['m', { ok: () => undefined }]]) + assertEq(report.status, 'passed') + assertEq(p.summary.textContent, `1 passed, 0 failed (${report.duration.toFixed(1)} ms)`) + assertEq(p.view.events.length, 0) + assertEq(p.view.fjsBrowserTestReport, undefined) }, - // ...and it counts up as modules link, so a slow graph shows progress - // rather than one frozen line. - loadingProgress: async () => { - const { root, summary } = page() - /** @type {(module: Module) => void} */ + renderReport: () => { + // The renderer is exported on its own for a controller that already + // holds a report. + const p = page() + renderBrowserReport(p.root, { + status: 'passed', + browser: 'test', + totals: { tests: 1, passed: 1, failed: 0 }, + duration: 1, + results: [{ module: 'm', path: '.t', status: 'passed', duration: 0.5 }], + }) + assertEq(p.summary.textContent, '1 passed, 0 failed (1.0 ms)') + assertEq(p.results.children[0]?.textContent, 'PASS m .t (0.5 ms)') + }, + sources: async () => { + const p = page() + const report = await startBrowserTestSources(p.root, ['a.mjs', 'b.mjs'], + source => Promise.resolve({ proof: { [source]: () => undefined } })) + assertEq(report.status, 'passed') + assertEq(report.totals.tests, 2) + assertStructurallySame([...p.states], ['loading', 'running', 'passed']) + assertEq(await p.view.fjsBrowserTestReport, report) + }, + sourcesLoadingSummaryIsSynchronous: () => { + // The summary must not keep showing idle text through loading: it is + // replaced the instant a run starts, before any import has had a + // chance to settle — even one that never does. + const p = page() + void startBrowserTestSources(p.root, ['a.mjs', 'b.mjs'], () => new Promise(() => undefined)) + assertEq(p.summary.textContent, 'Loading 0/2') + }, + sourcesProgress: async () => { + const p = page() + /** @type {(module: { readonly proof?: unknown }) => void} */ let release = () => undefined - /** @type {Promise} */ + /** @type {Promise<{ readonly proof?: unknown }>} */ const pending = new Promise(resolve => { release = resolve }) - const done = startBrowserTestSources(root, ['a.mjs', 'b.mjs'], + const done = startBrowserTestSources(p.root, ['a.mjs', 'b.mjs'], source => source === 'a.mjs' ? Promise.resolve({ proof: {} }) : pending) await Promise.resolve() await Promise.resolve() - assertEq(summary.textContent, 'Loading 1/2: a.mjs') + assertEq(p.summary.textContent, 'Loading 1/2: a.mjs') release({ proof: {} }) assertEq((await done).status, 'passed') }, - // The same action starts every run: nothing but the `Run` control's own - // state stands between a completed run and the next one. - newRunAfterCompletion: async () => { - const { root, runButton, states } = page() - /** @type {() => Promise} */ - const load = () => Promise.resolve({ proof: { t: () => undefined } }) - await startBrowserTestSources(root, ['a.mjs'], load) - assert(!runButton.attributes.has('disabled')) - const second = await startBrowserTestSources(root, ['a.mjs'], load) - assertEq(second.status, 'passed') - assertEq(second.totals.tests, 1) - assertEq(states.join(','), 'loading,running,passed,loading,running,passed') - }, - // Rendering a result is the page's own code, so it is a failure point of - // the page and not of the run: a renderer that throws must not cost the - // report every consumer is waiting for. - renderingThrows: async () => { - const { root, results } = page() - const append = results.append - const report = await startBrowserTestSources(root, ['a.mjs'], async () => { - // Break rendering only once the run is under way, so the page is - // built normally and only the per-result append fails. - Object.assign(results, { append: () => { throw new Error('render') } }) - return { proof: { t: () => undefined } } - }) - Object.assign(results, { append }) - assertEq(report.status, 'passed') - assertEq(report.totals.passed, 1) - }, - // Describing a panic reads the value that caused it, so a value every trap - // of which throws *itself* makes the description panic in turn. That is the - // last handler there is: it may not fail, or the guard against a stuck page - // becomes the thing that sticks it. - unreadableFailure: async () => { - /** @type {ProxyHandler} */ - const handler = {} - const hostile = new Proxy({}, handler) - const rethrow = () => { throw hostile } - Object.assign(handler, { has: rethrow, get: rethrow, ownKeys: rethrow }) - const { root, states } = page() - const report = await startBrowserTestSources(root, ['a'], async () => ({ - proof: { boom: () => { throw hostile } }, - })) + sourcesImporterThrows: async () => { + // An importer that throws before it returns a promise is a loader + // failure like any other: the page must not be left in `loading` with + // no report and no completion event. + const p = page() + const report = await startBrowserTestSources(p.root, ['bad.mjs'], + source => { throw new Error(`no loader for ${source}`) }) assertEq(report.status, 'infrastructure-error') - assertEq(report.results[0]?.message, 'The run failed with a value that cannot be read') - assertEq(states.join(','), 'loading,running,infrastructure-error') + assertStructurallySame({ ...report.totals }, { tests: 1, passed: 0, failed: 1 }) + assertEq(report.results[0]?.message, 'no loader for bad.mjs') + assertStructurallySame([...p.states], ['loading', 'infrastructure-error']) + assertEq(p.view.events.length, 1) }, - // `infrastructure-error` covers a panic and a runner missing an operation as - // well as a module that would not link, so the summary must not diagnose - // every one of them as a loading failure. - infrastructureSummaryNamesNoCause: () => { - const { root, summary } = page() - renderBrowserReport(root, { - status: 'infrastructure-error', - browser: 'x', - totals: { tests: 1, passed: 0, failed: 1 }, - duration: 0, - results: [{ module: '', path: '', status: 'failed', duration: 0, message: 'no sandbox', stack: '' }], - }) - assert(!summary.textContent.includes('to load'), summary.textContent) - assert(summary.textContent.startsWith('Infrastructure error: 1 failed'), summary.textContent) + runControlAbsentButtonIsIgnored: async () => { + // An embedding root with no `[data-test-run]` control is still + // supported: `setState` finds nothing to toggle and moves on rather + // than throwing. + /** @type {string[]} */ + const states = [] + /** @type {_Document} */ + const document = { + defaultView: null, + createElement: tag => element(document, tag, [], states), + } + const root = element(document, 'main', ['data-browser-tests'], states) + root.replaceChildren( + element(document, 'p', ['data-test-summary'], states), + element(document, 'ol', ['data-test-results'], states)) + const report = await startBrowserTests(/** @type {Element} */ (/** @type {unknown} */ (root)), + [['m', { ok: () => undefined }]]) + assertEq(report.status, 'passed') }, - // A root whose document has no window still runs and still answers: there - // is simply nowhere to publish the promise or dispatch the event. - withoutView: async () => { - const { root, view } = page(false) - const report = await startBrowserTestSources(root, ['a'], async () => ({ - proof: { x: () => undefined }, - })) + runControlDisabledWhileActive: async () => { + // `Run` must be passive — genuinely disabled, not just click-ignoring — + // for the whole span between a click and the next terminal state: + // through loading and through execution. + const p = page() + /** @type {(module: { readonly proof?: unknown }) => void} */ + let release = () => undefined + /** @type {Promise<{ readonly proof?: unknown }>} */ + const pending = new Promise(resolve => { release = resolve }) + const done = startBrowserTestSources(p.root, ['a.mjs'], () => pending) + await Promise.resolve() + assertEq(p.states[0], 'loading') + assertEq(p.runButton.attributes.has('disabled'), true) + release({ proof: { t: () => undefined } }) + await Promise.resolve() + await Promise.resolve() + assertEq(p.runButton.attributes.has('disabled'), true) + const report = await done assertEq(report.status, 'passed') - assertEq(report.browser, '') - assertEq(view.events.length, 0) - assertEq(view.fjsBrowserTestReport, undefined) + // Terminal state hands control back: a new run can be started. + assertEq(p.runButton.attributes.has('disabled'), false) }, - // A root with none of the page's elements is rendered into without a throw: - // an embedder may host the runner in a bare container. - renderWithoutElements: () => { - const { root, states } = page() - root.replaceChildren() - renderBrowserReport(root, { - status: 'passed', - browser: 'x', - totals: { tests: 0, passed: 0, failed: 0 }, - duration: 0, - results: [], - }) - assertEq(states.join(','), 'passed') + runControlReenabledAfterFailure: async () => { + // A failed or infrastructure-error run is just as terminal as a passed + // one: `Run` reactivates either way. + const p = page() + const report = await startBrowserTestSources(p.root, ['bad.mjs'], + source => Promise.reject(new Error(`offline: ${source}`))) + assertEq(report.status, 'infrastructure-error') + assertEq(p.runButton.attributes.has('disabled'), false) }, - operations: { - // `fetch` reads a `data:` URL rather than a network one, so the proof - // stays offline while still going through the realm's own `fetch`. - fetch: async () => { - const r = await fetchOp('data:text/plain,ok') - assert(r[0] === 'ok', r) - }, - fetchFailure: async () => { - const r = await fetchOp('not-a-scheme://x') - assert(r[0] === 'error', r) - assertEq(r[1][0], 'ioError') - }, - import: async () => { - const r = await importOp('./x.mjs') - assertEq(/** @type {Module} */ (unwrap(r)).source, './x.mjs') - }, - awaitsPromise: async () => { - assertEq(unwrap(await awaitOp(Promise.resolve(7)))[0], 7) - }, - awaitsPlainValue: async () => { - assertEq(unwrap(await awaitOp(7))[0], 7) - }, - // Epoch milliseconds, as the Node runner answers — but read through - // `performance`, so two reads never come out in the wrong order however - // the system clock is adjusted between them. - now: async () => { - const before = unwrap(await now()) - const after = unwrap(await now()) - assert(before > Date.UTC(2020, 0, 1), before) - assert(after >= before, [before, after]) - }, - sandboxMeasures: async () => { - const { result, duration } = unwrap(await sandbox(() => 1)) - assertEq(unwrap(result), 1) - assert(duration >= 0, duration) - }, - all: async () => { - const results = unwrap(await all(pureOk(1), pureOk(2))) - assertEq(results.map(unwrap).join(','), '1,2') - }, - // Slicing must not serialize: every child is *started* before any is - // awaited, so a child waiting on something a later sibling produces - // still sees that sibling run. Awaiting each slice before starting the - // next hangs this — the releaser sits in the second slice, which is - // never reached — on a graph the Node runner completes. - allStartsEveryChildBeforeAwaiting: async () => { - // The gate is opened either by the eleventh child — which is the - // property under test — or, after far more turns of the event loop - // than every launch can need, by the fallback below. Which one - // opened it is the assertion. - // - // Counting turns rather than milliseconds is deliberate: this proof - // runs concurrently with the rest of the suite, so a wall-clock - // deadline measures how loaded the machine is, not what `all` did. - // The fallback exists so a serializing `all` *fails* here instead of - // hanging the run. - /** @type {string | null} */ - let openedBy = null - /** @type {(value: unknown) => void} */ - let release = () => undefined - /** @type {Promise} */ - const gate = new Promise(resolve => { release = resolve }) - // Whoever opens the gate *first* is recorded. A later opener must - // not overwrite it: a serializing `all` still reaches the sibling - // eventually, just far too late to have been what unblocked the - // first child. - /** @type {(who: string) => void} */ - const open = who => { - if (openedBy === null) { openedBy = who } - release(0) - } - const fallback = async () => { - for (let turn = 0; turn < 50 && openedBy === null; turn += 1) { - await new Promise(resolve => { setTimeout(resolve, 0) }) - } - open('the fallback') - } - void fallback() - const filler = sandboxEffect(() => 0) - const waits = sandboxEffect(() => gate) - const releases = sandboxEffect(() => { open('a later sibling'); return 0 }) - const many = [waits, ...[...new Array(9).keys()].map(() => filler), releases] - const results = unwrap(await commonRun(allEffect(...many))) - assertEq(openedBy, 'a later sibling') - assertEq(results.length, 11) - }, - // `all` hands the event loop back between launches, which is the only - // thing that lets a page paint mid-suite: a task queued before the call - // has to run before it resolves. Without the yield every child settles - // on microtasks and no task gets a turn — which is what this asserts, - // since the effects below perform nothing. - // - // The task queued here is a `MessageChannel` message rather than a - // `setTimeout`, because the two are not interchangeable across engines. - // Bun delivers port messages until none are left before it runs a due - // timer, so 59 yields there leave a `setTimeout(0)` queued behind them - // and this proof would report a yielding `all` as a non-yielding one. - // Asserting on the queue `all` actually posts to states the property - // — that a launch ends the task, so anything already queued runs — in - // terms every engine agrees on. - allYieldsBetweenLaunches: async () => { - let delivered = false - const { port1, port2 } = new MessageChannel() - port1.onmessage = () => { port1.close(); delivered = true } - port2.postMessage(0) - const many = [...new Array(60).keys()].map(i => pureOk(i)) - const results = unwrap(await all(...many)) - assertEq(results.length, 60) - assertEq(results.map(unwrap).join(','), many.map((_, i) => i).join(',')) - assert(delivered, 'all resolved without yielding to the event loop') - }, + runControlNewRunAfterCompletion: async () => { + // The same action starts every run: nothing but the `Run` control's + // own state stands between a completed run and the next one. + const p = page() + await startBrowserTestSources(p.root, ['a.mjs'], + () => Promise.resolve({ proof: { t: () => undefined } })) + assertEq(p.runButton.attributes.has('disabled'), false) + const second = await startBrowserTestSources(p.root, ['a.mjs'], + () => Promise.resolve({ proof: { t: () => undefined } })) + assertEq(second.status, 'passed') + assertStructurallySame([...p.states], + ['loading', 'running', 'passed', 'loading', 'running', 'passed']) + }, + sourcesLoadFailure: async () => { + const p = page() + const report = await startBrowserTestSources(p.root, ['ok.mjs', 'bad.mjs'], + source => source === 'bad.mjs' + ? Promise.reject(new Error('offline')) + : Promise.resolve({ proof: { t: () => undefined } })) + assertEq(report.status, 'infrastructure-error') + // The totals have to agree with `results`: a consumer reading + // `0 of 0` would take a broken suite for an empty one. + assertStructurallySame({ ...report.totals }, { tests: 1, passed: 0, failed: 1 }) + assertEq(report.results[0]?.module, 'bad.mjs') + assertEq(report.results[0]?.message, 'offline') + assertStructurallySame([...p.states], ['loading', 'infrastructure-error']) + assert(p.summary.textContent.startsWith('Infrastructure error: 1 failed to load'), + p.summary.textContent) + assertStructurallySame([...statuses(p.results)], ['failed']) + assertEq(p.view.events.length, 1) }, } diff --git a/fjs/emergent_testing/browser/species.proof.mjs b/fjs/emergent_testing/browser/species.proof.mjs new file mode 100644 index 000000000..11303e009 --- /dev/null +++ b/fjs/emergent_testing/browser/species.proof.mjs @@ -0,0 +1,45 @@ +import { assertEq } from '../../asserts/module.f.mjs' +import { runBrowserProofs } from '../browser.mjs' + +/** + * A genuine promise whose `then` always throws: the result promise is built + * through `constructor[Symbol.species]`, and this `constructor` has none to + * give. `configurable` decides whether the runner can shadow the property for + * the length of one subscription. + * + * @type {(configurable: boolean) => Promise} + */ +const throwingSpeciesPromise = configurable => { + const promised = Promise.resolve({ + child: () => { throw 'boom' }, + }) + const constructor = {} + Object.defineProperty(constructor, Symbol.species, { + get: () => { throw new Error('species') }, + }) + Object.defineProperty(promised, 'constructor', { value: constructor, configurable }) + return promised +} + +/** @type {(promised: Promise) => ReturnType} */ +const run = promised => runBrowserProofs([['proof', { nested: () => promised }]]) + +export const proof = { + throwingSpecies: async () => { + // The intrinsic Promise shadows the hostile `constructor` while the + // handlers are attached, so the resolved sub-tree still runs. + const report = await run(throwingSpeciesPromise(true)) + assertEq(report.totals.tests, 2) + assertEq(report.totals.failed, 1) + assertEq(report.results[1]?.path, '.nested().child') + }, + pinnedThrowingSpecies: async () => { + // Nothing to shadow, so the promise can never be subscribed to. The + // test that produced it fails, rather than passing on a result the + // runner never observed. + const report = await run(throwingSpeciesPromise(false)) + assertEq(report.totals.tests, 1) + assertEq(report.totals.failed, 1) + assertEq(report.results[0]?.message, 'species') + }, +} diff --git a/fjs/emergent_testing/browser/types.ts b/fjs/emergent_testing/browser/types.ts deleted file mode 100644 index f947e00c7..000000000 --- a/fjs/emergent_testing/browser/types.ts +++ /dev/null @@ -1,74 +0,0 @@ -/** - * Types for the browser proof application. - * - * @module - */ - -import type { CommonOp, Module } from '../../effects/common/types.ts' -import type { Effect } from '../../effects/types.ts' -import type { IoResult } from '../../effects/common/types.ts' -import type { ReportOp, TestResult } from '../types.ts' - -/** - * The operations the browser application performs: the host-independent set - * every runner implements, plus the two that record normalized results. - * - * There is nothing browser-specific in it, and that is the design rather than - * an accident — the DOM is the *adapter's* business - * ([`./module.mjs`](./module.mjs)), never the application's. A page, a proof - * with a stand-in interpreter, and a future headless controller therefore run - * the very same program. - */ -export type BrowserOp = CommonOp | ReportOp - -/** - * How a whole run ended. `infrastructure-error` is not a third kind of test - * failure: it says the suite never got to run — a module that would not link, a - * runner missing an operation — which an automated consumer must not read as - * "the proofs failed". - */ -export type ReportStatus = 'passed' | 'failed' | 'infrastructure-error' - -/** - * The serializable answer of a run, independent of the runner that produced it - * and of the page that rendered it. - */ -export type BrowserTestReport = { - readonly status: ReportStatus - readonly browser: string - readonly totals: { - readonly tests: number - readonly passed: number - readonly failed: number - } - readonly duration: number - readonly results: readonly TestResult[] -} - -/** - * What the host supplies to a run: the proof modules to link, and the name to - * record the realm under. - * - * `browser` is data rather than a `navigator` read, for the reason every other - * capability here is an operation — the application must be runnable outside a - * browser, and a proof that had to install a global `navigator` to check a - * report would be testing the stub. - */ -export type BrowserOptions = { - readonly browser: string - readonly sources: readonly string[] -} - -/** - * A run: options in, a report out. - * - * **The error channel is `never`**, and it is earned rather than asserted: a - * module that will not link and an operation the runner lacks are both - * *reported*, as an `infrastructure-error` report. A page waiting on the run - * has nowhere to put a failure — leaving it in `running` with no report and no - * completion event is the one outcome an automated controller cannot act on. - */ -export type BrowserProgram = (options: BrowserOptions) => Effect - -/** @internal One source paired with what linking it answered. */ -export type _Loaded = readonly[string, IoResult] diff --git a/fjs/emergent_testing/module.f.mjs b/fjs/emergent_testing/module.f.mjs index 142f8ab12..a32868040 100644 --- a/fjs/emergent_testing/module.f.mjs +++ b/fjs/emergent_testing/module.f.mjs @@ -2,34 +2,25 @@ * Test-framework helpers for running and reporting FunctionalScript tests. * * Two parallel execution paths: - * - `runModule` / `Reporter` — self-hosted Effects runner; sandboxes each - * leaf call individually and accumulates `TestState`. **Both** `fjs t` and - * the browser runner (`./browser/module.f.mjs`) go through it: proof-tree - * walking, the structural `throw` expectation, promise resolution, path - * formatting and the totals are decided here once, and each host differs only - * in its `Reporter` and in the runner that interprets `sandbox`. + * - `runModule` / `Reporter` — self-hosted Effects runner used by `fjs t`; + * sandboxes each leaf call individually and accumulates `TestState`. * - `registerModule` / `TestContext` — registers tests with an external * framework (Node `--test`, Bun, Deno) at import time; the framework owns * scheduling and pass/fail counting. * - * `recordingReporter` is the host-independent reporter of the first path: it - * normalizes each leaf into a `TestResult` carrying no terminal text and no DOM - * and hands it to the `report` operation, leaving presentation to the host. - * * @module * * @import { Operation } from '../effects/types.ts' - * @import { Effect, Func, NotImplemented } from '../effects/types.ts' + * @import { Effect, NotImplemented } from '../effects/types.ts' * @import { LoadModuleOperations, ModuleMap } from '../dev/types.ts' - * @import { Report, Reported, TestFn, TestEntry, TestResult, TestSet, Path, Reporter, _TestState, _TestAndPath } from './types.ts' + * @import { TestFn, TestEntry, TestSet, Path, Reporter, _TestState, _TestAndPath } from './types.ts' * @import { All, Await, Env, IoChannel, NodeProgram, NodeProgramOptions, Program, Sandbox, SandboxResult, Test, TestContext, Write, WriteConsoles } from '../effects/node/types.ts' */ import { reset, fgGreen, fgRed, bold, csiWrite } from '../text/sgr/module.f.mjs' -import { allOk, awaitIfPromise, sandbox } from '../effects/common/module.f.mjs' -import { errorExit, errorMessage, errorSummary, exitStep, test } from '../effects/node/module.f.mjs' +import { allOk, awaitIfPromise, errorExit, errorMessage, errorSummary, exitStep, sandbox, test } from '../effects/node/module.f.mjs' import { - catchStep, do_, history, historyStep, mapStep, pureError, pureOk, resultStep, step, + catchStep, history, historyStep, mapStep, pureError, pureOk, resultStep, step, } from '../effects/module.f.mjs' import { loadModuleMap } from '../dev/module.f.mjs' import { invert } from '../types/result/module.f.mjs' @@ -330,31 +321,14 @@ export const fmtPath = path => path.reduce((/** @type {string} */ acc, k) => acc + fmtKey(k), '') /** - * A fully-qualified test identifier, from a module and an **already-rendered** - * key chain: `import("./math.proof.f.mjs").proof.add()`. - * - * This is the one place the format lives, and it takes the rendered chain - * rather than a {@link Path} so that a reporter holding a {@link TestResult} — - * whose `path` is already a string — names a test exactly as `fjs t` does. It - * did not, and the browser page rendered `./math.proof.f.mjs .add` while the - * terminal rendered the call expression: one identifier in two spellings, which - * is the drift a shared runner is supposed to make impossible. - * - * @type {(file: string, path: string) => string} - */ -export const fmtCall = (file, path) => - `import(${JSON.stringify(file)}).proof${path}()` - -/** - * {@link fmtCall} over a {@link Path} that has not been rendered yet, e.g. - * `import("./math.proof.f.ts").proof.add()` or - * `import("./a.proof.f.ts").proof.users[3].name()`. + * Formats a fully-qualified test identifier as a JS-like expression, e.g. + * `import("./math.proof.f.ts").add()` or `import("./a.proof.f.ts").users[3].name()`. * Self-contained per line — suitable for parallel output and as a CLI filter argument. * * @type {(file: string, path: Path) => string} */ export const fmtImport = (file, path) => - fmtCall(file, fmtPath(path)) + `import(${JSON.stringify(file)}).proof${fmtPath(path)}()` /** * Renders a key chain for terminal output: `| ` per level of depth, followed @@ -395,82 +369,6 @@ export const ghEscape = s => export const defaultTest = (file, path, { fn, throws }) => mapStep(sandbox(fn), r => throws ? { ...r, result: invert(r.result) } : r) -/** What a `throws` leaf that returned cleanly is reported as. */ -const expectedThrow = 'Expected the proof to throw' - -/** - * The message and stack to report a thrown value by. - * - * An `Error` thrown from another realm — an iframe, a worker — is not - * `instanceof Error` here, and its stack is the very thing a report exists to - * carry. What the fields say is therefore the test, not where the value was - * made: anything carrying `message` or `stack` is read as the failure it - * describes, and everything else by its own text. - * - * @type {(error: unknown) => readonly[string, string]} - */ -export const errorDetails = error => { - if (error !== null && (typeof error === 'object' || typeof error === 'function') - && ('message' in error || 'stack' in error)) { - const { message, stack } = /** @type {{ readonly message?: unknown, readonly stack?: unknown }} */ (error) - const described = String(message) - return [described, stack === undefined ? described : String(stack)] - } - const fallback = String(error) - return [fallback, fallback] -} - -/** - * Normalizes one leaf outcome into the {@link TestResult} every reporter - * renders from. - * - * `r` is what {@link Reporter.test} answered, so a `throws` leaf has already - * been inverted by {@link defaultTest}: an `error` there means the proof - * returned when it was expected to throw, which is why that case is named - * rather than described by the value it returned. - * - * @type {(file: string, path: Path, r: SandboxResult, throws: boolean) => TestResult} - */ -export const testResult = (file, path, { result, duration }, throws) => { - const [status, value] = result - const common = { module: file, path: fmtPath(path), duration } - if (status === 'ok') { return { ...common, status: 'passed' } } - const [message, stack] = throws ? [expectedThrow, ''] : errorDetails(value) - return { ...common, status: 'failed', message, stack } -} - -/** Records one normalized leaf result as it lands. - * - * @type {Func} - */ -export const report = do_('report') - -/** Reads back every result {@link report} has recorded. - * - * @type {Func} - */ -export const reported = do_('reported') - -/** - * The reporter that answers in {@link TestResult}s instead of rendering: each - * leaf is normalized and handed to the {@link report} operation, and the run's - * consumer reads the sequence back with {@link reported}. - * - * **Its `summary` writes nothing**, and that is not an omission. Pass, fail and - * total are `results.length` and a count of the failed ones, so a summary event - * would restate what the recorded results already say — and a consumer that - * derives them cannot disagree with itself about how many tests ran. The - * terminal reporter keeps its own `summary` because a line of text is genuinely - * not derivable from the results a user has already scrolled past. - * - * @type {Reporter} - */ -export const recordingReporter = { - result: (file, path, r, throws) => report(testResult(file, path, r, throws)), - summary: () => pureOk(undefined), - test: defaultTest, -} - /** @type {(file: string, path: Path, color: string, label: string, duration: number) => string} */ const fmtResultLine = (file, path, color, label, duration) => `${fmtImport(file, path)}: ${color}${label}${reset}, ${timeFormat(duration)}` diff --git a/fjs/emergent_testing/proof.f.mjs b/fjs/emergent_testing/proof.f.mjs index ed96a5f7d..ae074d667 100644 --- a/fjs/emergent_testing/proof.f.mjs +++ b/fjs/emergent_testing/proof.f.mjs @@ -13,8 +13,8 @@ import { log } from '../effects/node/module.f.mjs' import { defaultNodeProgramOptions, emptyState, virtual } from '../effects/node/virtual/module.f.mjs' import { assert, assertEq, todo } from '../asserts/module.f.mjs' import { - testAll, errorDetails, fmtPath, fmtTerm, fmtImport, ghEscape, isInteger, isIdentifier, - registerModule, parseTestSet, testResult, + testAll, fmtPath, fmtTerm, fmtImport, ghEscape, isInteger, isIdentifier, + registerModule, parseTestSet, defaultTest, main, register, } from './module.f.mjs' import { run as mockRun } from '../effects/mock/module.f.mjs' @@ -597,64 +597,6 @@ export const helpers = { assertEq(ghEscape('a\r\nb'), 'a%0D%0Ab') assertEq(ghEscape('a%b:c,d'), 'a%25b%3Ac%2Cd') }, - errorDetails: { - // Read structurally rather than by `instanceof Error`, so an error from - // another realm still reports its own stack. - messageAndStack: () => { - const [message, stack] = errorDetails({ message: 'boom', stack: 'boom\n at x' }) - assertEq(message, 'boom') - assertEq(stack, 'boom\n at x') - }, - withoutStack: () => { - const [message, stack] = errorDetails({ message: 'no stack' }) - assertEq(message, 'no stack') - assertEq(stack, 'no stack') - }, - // A value carrying only a stack is still a failure description; the - // message it does not have reads as the absent value it is. - stackOnly: () => { - const [message, stack] = errorDetails({ stack: 'trace' }) - assertEq(message, 'undefined') - assertEq(stack, 'trace') - }, - // A thrown *function* is an object as far as this reading goes. - callable: () => { - const [message] = errorDetails(Object.assign(() => undefined, { message: 'fn' })) - assertEq(message, 'fn') - }, - plainValue: () => { - const [message, stack] = errorDetails('just text') - assertEq(message, 'just text') - assertEq(stack, 'just text') - }, - nullValue: () => { - assertEq(errorDetails(null)[0], 'null') - }, - }, - testResult: { - passed: () => { - const r = testResult('a.f.mjs', ['x'], { result: ok(1), duration: 2 }, false) - assertEq(r.module, 'a.f.mjs') - assertEq(r.path, '.x') - assertEq(r.status, 'passed') - assertEq(r.duration, 2) - assertEq(r.message, undefined) - }, - failed: () => { - const r = testResult('a.f.mjs', ['x'], { result: error(new Error('bad')), duration: 0 }, false) - assertEq(r.status, 'failed') - assertEq(r.message, 'bad') - }, - // `defaultTest` has already inverted a `throws` leaf, so an `error` here - // means it returned when it was expected to throw — named rather than - // described by whatever it happened to return. - expectedToThrow: () => { - const r = testResult('a.f.mjs', ['throw', 'x'], { result: error(7), duration: 0 }, true) - assertEq(r.status, 'failed') - assertEq(r.message, 'Expected the proof to throw') - assertEq(r.stack, '') - }, - }, parseTestSet: { nullReturnsEmpty: () => { const result = parseTestSet(false, null) diff --git a/fjs/emergent_testing/todo/browser-test-controls.md b/fjs/emergent_testing/todo/browser-test-controls.md index 41d4f2913..77c391782 100644 --- a/fjs/emergent_testing/todo/browser-test-controls.md +++ b/fjs/emergent_testing/todo/browser-test-controls.md @@ -66,5 +66,5 @@ module or a default query parameter. - [Browser testing](browser-testing.md) — the shared browser application and report contract. -- [`emergent_testing/browser`](../browser/module.f.mjs) — the pure application - the controls drive; runner state is already separate from DOM presentation. +- [Shared browser/console runner core](share-browser-console-runner.md) — future + separation of pure runner state from DOM controls. diff --git a/fjs/emergent_testing/todo/browser-testing.md b/fjs/emergent_testing/todo/browser-testing.md index a3f008bc0..d81dc562b 100644 --- a/fjs/emergent_testing/todo/browser-testing.md +++ b/fjs/emergent_testing/todo/browser-testing.md @@ -48,7 +48,7 @@ three independent test frameworks. eventual isolated browser-test application root ├── index.html ├── _browser-test-entry.mjs -├── fjs/emergent_testing/browser/module.mjs +├── fjs/emergent_testing/browser.mjs └── authored or copied .f.mjs / .mjs modules ``` @@ -147,9 +147,8 @@ workers, or visual regression testing. - [ ] Create the JavaScript-only application root with a generated entry module covering every accepted module. - [x] Implement the first browser-compatible emergent-test runner and report - API, and share its proof semantics with `fjs t`: both runners now walk - proof trees through `emergent_testing/module.f.mjs` and differ only in - their `Reporter` and their effect interpreter. + API; follow up by sharing its pure semantics with `fjs t` in + [share-browser-console-runner](share-browser-console-runner.md). - [x] Implement the HTML UI and integrate it into the FunctionalScript website. - [ ] Add shared controller code for preparation, serving, report validation, @@ -157,42 +156,14 @@ workers, or visual regression testing. - [ ] Implement `fjs browser-test` without any Playwright dependency. - [ ] Implement a Playwright Test adapter that dynamically resolves external `playwright/test` and reuses the shared controller. -- [ ] Run the same application in Chromium, Firefox, and WebKit. Check the - yield `all` uses to give the page a turn while it runs - (`MessageChannel`, `../../effects/browser/module.mjs`) behaves in each, - and whether `scheduler.yield()` is worth preferring where it exists. +- [ ] Run the same application in Chromium, Firefox, and WebKit. - [ ] Add the validation fixtures above; add CI only after proof bodies - demonstrably execute inside browsers. **That gate is now met** — the - unified runner was driven in Chromium over the generated page, 3435 proofs - linked and executed, so what still blocks a CI job is the controller - below, not evidence. Nothing in `.github/workflows/` starts a browser - today, and `npm run website` only *generates* the suite: it exits `0` with - a failing proof in the manifest, so the browser suite is not a gate - anywhere yet. -- [ ] Keep a module-loading failure's stack. This section requires failures to - retain "module path, test path, message, and stack", and a *proof* failure - does — but a **load** failure no longer does. Linking is an `Import` - effect now, and its failure is an `IoError`, which is `{ code?, message }`: - `toIoError` drops the stack, so the report shows `stack: ''` where the - deleted runner showed the loader's own frames, which are what name the - importing module and line for a broken graph. The fix is one additive - optional field, `stack?: string` on `IoErrorInfo` in - `../../effects/common/types.ts`, filled by `toIoError` and read by - `infrastructureResult`. `IoError`'s rationale for dropping it — "a stack, a - `cause`, and arbitrary own properties do not survive a wire hop" — is right - about the last two and wrong about a stack, which is a string. Note that - reading `.stack` is a user-observable operation on a hostile value, the - same exposure `toIoError` already has reading `.message`. -- [ ] Assert a floor on the number of proofs a run discovers. Nothing does - today, in any runner: a `collectTests` that silently skipped most leaves - would keep `fjs t` at exit `0`, and a suite that loses coverage cannot - report that it has. + demonstrably execute inside browsers. ### Related - [`.f.mjs` proof discovery and coverage](f-mjs-test-and-coverage.md) -- [Hostile thrown values and cross-realm promises](hostile-proof-values.md) -- [Browser timer precision](timer-precision.md) +- [Shared browser/console runner core](share-browser-console-runner.md) - [Explicit browser test controls](browser-test-controls.md) - [authored `.f.mjs` package support](../../ci/todo/f-mjs-package-support.md) - [project roadmap](../../../todo/plan/roadmap.md) diff --git a/fjs/emergent_testing/todo/hostile-proof-values.md b/fjs/emergent_testing/todo/hostile-proof-values.md index 1bda0e573..3bc46b646 100644 --- a/fjs/emergent_testing/todo/hostile-proof-values.md +++ b/fjs/emergent_testing/todo/hostile-proof-values.md @@ -5,40 +5,46 @@ ### Problem -Both runners now share one core (`../module.f.mjs`), so they also share two -weaknesses the core cannot fix on its own. Neither is reachable from ordinary -FunctionalScript, and both were reachable — and covered — by the browser runner -before it and `fjs t` were unified; unifying adopted `fjs t`'s semantics -deliberately, so this file is where the difference went rather than being -silently dropped. +The browser runner (`../browser.mjs`) defends against two things `fjs t` does +not, and neither is reachable from ordinary FunctionalScript. That asymmetry is +the point of this file: when the two runners are unified +([share the browser and console proof runners](share-browser-console-runner.md)), +the shared core has to have *one* answer for each of them, decided rather than +inherited twice. `fjs t` is the reference, so the honest reading is that these +are gaps in `fjs t` which the browser happened to cover — and closing them in +the shared core is the way to keep that coverage instead of losing it to a port. **A value that resists being read is not attributed to the test that produced -it.** Two shared functions read user-supplied values without a guard: the -`collectTests` traversal enumerates a returned proof tree, and `errorDetails` -reads `message`/`stack` and calls `String` on a thrown value. A throwing -accessor, a revoked `Proxy`, or a hostile `toString` panics through either, and -there is no `try`/`catch` in FunctionalScript for the core to catch it with. - -The browser adapter turns that panic into an `infrastructure-error` report -rather than leaving the page in `running`, so a run always terminates — but the -whole run is lost where the deleted runner lost one test, and `fjs t` still ends -with a stack trace and no summary. What is missing is *attribution*: naming the -leaf whose value could not be read, and continuing with the rest. - -**A promise from another realm is not awaited.** Both `sandbox` interpreters ask -`p instanceof Promise`, which is false for a promise built in an iframe, a -worker, or a `node:vm` context. Such a value is walked as an ordinary proof tree -instead, so a *rejected* cross-realm promise is reported as a pass. The obvious -repair — brand-checking with `Object.prototype.toString` — is not one: the tag -is settable through `Symbol.toStringTag`, and an object carrying a `then` proof +it.** Two functions in the shared core read user-supplied values without a +guard: the `collectTests` traversal enumerates a returned proof tree, and +`errorDetails` reads `message`/`stack` and calls `String` on a thrown value. A +throwing accessor, a revoked `Proxy`, or a hostile `toString` panics through +either, and there is no `try`/`catch` in FunctionalScript for the core to catch +it with. `fjs t` ends with a stack trace and no summary; the browser runner +today loses one test and carries on. What is missing from the core is +*attribution*: naming the leaf whose value could not be read, and continuing +with the rest. Whichever runner ends up on top of it, a page left in `running` +or a process that exits with no summary is the outcome an automated controller +cannot act on. + +**A promise from another realm is not awaited.** `fjs t`'s `sandbox` asks `p +instanceof Promise`, which is false for a promise built in an iframe, a worker, +or a `node:vm` context. Such a value is walked as an ordinary proof tree +instead, so a *rejected* cross-realm promise is reported as a pass. The browser +runner carries `Symbol.species` machinery against this, which is a second answer +to the same question and is studied in +[imports, promises and realms](imports-promises-realms.md). The obvious repair — +brand-checking with `Object.prototype.toString` — is not one: the tag is +settable through `Symbol.toStringTag`, and an object carrying a `then` proof would then be assimilated, breaking the rule that only actual promises are asynchronous values. ### Design: a `catch` operation Reading a user value belongs to the *operation*, not to the shared core, which -is what makes one fix serve every runner. Since the two runners are now one, -guarding the traversal once covers `fjs t` and the browser together. +is what makes one fix serve every runner. Once the two runners share a core, +guarding the traversal once covers `fjs t` and the browser together — which is +an argument for doing this *with* the sharing change rather than before it. **`sandbox` cannot hold it, and the reason is not the one it looks like.** Timing is not the obstacle: the sub-tree walk in `runModule` happens *after* the @@ -61,8 +67,8 @@ export type Catch = readonly['catch', (f: () => T) => OpResult = { - readonly log: (message: string) => Effect - readonly load: () => Effect - readonly import: (source: string) => Effect -} -``` - -`Reporter` is already this shape for one third of the job, so the question -is whether extending it beats adding operations, or whether the two are the -same thing written differently. - -Whichever is chosen, the test is concrete: adding a third host — an MCP server, -a worker, `fjs browser-test` — must not mean writing a fourth `main`. - -### Constraints - -- The shared semantics must not acquire terminal text or DOM: a `TestResult` - carries neither today and that is what lets both reporters render it. -- A browser must not gain a `Write` or a `Program` it cannot honour. Lifting the - abstraction means finding the operation both hosts *can* implement, not giving - one a stub. - -### Tasks - -- [ ] Inventory what each host's `main` does that is not host-specific. -- [ ] Choose between artificial effects and injected verbs, and write down why. -- [ ] Express discovery once, so a manifest and a `readdir` walk are two - implementations of one operation rather than two programs. -- [ ] Express the outcome once, so an exit code and a report are two renderings - of one value. - -### Related - -- [Browser testing](browser-testing.md) — the hosts that are still to come. -- [Test-runner behavior](661-test-runner-behavior.md) — the differences between - runners that are intentional, and must stay intentional. diff --git a/fjs/emergent_testing/todo/timer-precision.md b/fjs/emergent_testing/todo/timer-precision.md index b09ff1eb2..edb330b27 100644 --- a/fjs/emergent_testing/todo/timer-precision.md +++ b/fjs/emergent_testing/todo/timer-precision.md @@ -5,8 +5,7 @@ ### Problem -`sandbox` measures every proof the same way in every host — read the clock, -run the body, read it again: +`sandbox` measures a proof by reading the clock either side of the body: ```js const before = performance.now() @@ -34,10 +33,10 @@ built by summing thousands of such rows accumulates the rounding rather than cancelling it, so the sum can be off by a large multiple in either direction depending on which way each read rounded. -Note this is not the same concern as -[`now`'s monotonicity](../../effects/browser/module.mjs), which is already -handled: `performance.timeOrigin + performance.now()` cannot go backwards. A -monotonic clock can still be a coarse one, and this is about the resolution. +Note this is not the same concern as monotonicity. `performance.now()` cannot +go backwards, which is why it is the right clock for a duration; a wall clock +would be worse. A monotonic clock can still be a coarse one, and this is about +the resolution. ### Preliminary design @@ -71,9 +70,10 @@ before changing the measurement. ### Constraints - `sandbox` is the operation that executes a proof body, and both runners must - agree on it exactly or a suite means different things in different hosts. - Any change to how it measures is a change to the shared contract, not a - browser-local tweak. + agree on it exactly or a suite means different things in different hosts. Any + change to how it measures is a change for both, not a browser-local tweak — + and it is very likely `fjs t` has a milder version of the same problem, since + a coarse clock is only easier to notice in a browser. - The clock must stay monotonic. Whatever replaces or supplements `performance.now()` cannot reintroduce wall-clock time. - A duration must not cost a second `sandbox` call or an extra scheduling @@ -99,5 +99,6 @@ before changing the measurement. report contract these durations belong to. - [Report a test's name before running it](report-before-running.md) — the other thing wrong with what a row shows. -- [Share the whole runner](share-the-whole-runner.md) — `sandbox` is shared, - so this is one decision, not two. +- [Share the browser and console proof runners](share-browser-console-runner.md) + — `sandbox` is the operation that executes a proof body in both hosts, so its + measurement is one decision, not two. diff --git a/fjs/emergent_testing/types.ts b/fjs/emergent_testing/types.ts index 68d18bb58..a3274ae6a 100644 --- a/fjs/emergent_testing/types.ts +++ b/fjs/emergent_testing/types.ts @@ -5,7 +5,7 @@ */ import type { Effect, Operation } from '../effects/types.ts' -import type { IoChannel, OpResult, SandboxResult } from '../effects/common/types.ts' +import type { IoChannel, SandboxResult } from '../effects/node/types.ts' /** A zero-argument test function whose return value may contain sub-tests. */ export type TestFn = () => unknown @@ -68,47 +68,6 @@ export type Reporter = { readonly test: (file: string, path: Path, set: TestEntry) => Effect, IoChannel> } -/** How a leaf test ended. */ -export type TestStatus = 'passed' | 'failed' - -/** - * One leaf result, normalized: which module it came from, the property chain - * that names it, how it ended, and how long it took. A failure also carries the - * message and stack it should be reported by. - * - * **It carries no terminal text and no DOM.** This is what a runner *observes*, - * so every reporter can render it its own way — coloured lines on a TTY, a - * `::error` annotation on GitHub, a list item in a page — and an automated - * consumer can read it off the wire. `path` is already rendered by - * {@link fmtPath} rather than left as a `Path`, because the chain is what a - * reader identifies the test by and nothing downstream walks it again. - */ -export type TestResult = { - readonly module: string - readonly path: string - readonly status: TestStatus - readonly duration: number - readonly message?: string - readonly stack?: string -} - -/** - * Records one normalized leaf result the moment it lands. - * - * It is an *operation* rather than a value threaded through the run because the - * results arrive concurrently: `all` performs a module's leaves at once, so a - * read-modify-write over shared memory would interleave and lose them. A - * runner's handler appends in one step, and {@link Reported} reads the whole - * sequence back once the run is over. - */ -export type Report = readonly['report', (result: TestResult) => OpResult] - -/** Every result {@link Report} has recorded, in the order they landed. */ -export type Reported = readonly['reported', () => OpResult] - -/** The pair of operations a recording runner implements. */ -export type ReportOp = Report | Reported - /** @internal */ export type _TestState = { readonly time: number, diff --git a/fjs/website/module.f.mjs b/fjs/website/module.f.mjs index f18dd4437..24d569c1d 100644 --- a/fjs/website/module.f.mjs +++ b/fjs/website/module.f.mjs @@ -49,7 +49,7 @@ pre { white-space: pre-wrap } ['script', { type: 'module', src: './_browser-test-entry.mjs' }] ) -const entry = utf8(`import { startBrowserTestSources } from './fjs/emergent_testing/browser/module.mjs' +const entry = utf8(`import { startBrowserTestSources } from './fjs/emergent_testing/browser.mjs' import { browserProofSources } from './fjs/emergent_testing/_browser-suite.mjs' const root = /** @type {Element} */ (document.querySelector('[data-browser-tests]')) diff --git a/fjs/website/todo/generate-website.md b/fjs/website/todo/generate-website.md index 59bb1817c..959fd8231 100644 --- a/fjs/website/todo/generate-website.md +++ b/fjs/website/todo/generate-website.md @@ -12,4 +12,4 @@ - [x] Browser test runner and proof-result UI - [ ] Move browser-manifest preparation into the website `NodeProgram` through Node effects, as designed in - [website-preparation-program](website-preparation-program.md) + [share-browser-console-runner](../../emergent_testing/todo/share-browser-console-runner.md) diff --git a/fjs/website/todo/website-preparation-program.md b/fjs/website/todo/website-preparation-program.md deleted file mode 100644 index 95abebc68..000000000 --- a/fjs/website/todo/website-preparation-program.md +++ /dev/null @@ -1,69 +0,0 @@ -## Own the browser-suite preparation from the website `NodeProgram` - -**Priority:** P3 -**Status:** open - -### Problem - -`npm run website` runs `fjs/website/browser-prepare.mjs`, an impure Node script -that is a second application entry point beside the FunctionalScript program in -`fjs/website/module.f.mjs`. It walks the source tree, decides which proof -modules a browser can link, writes `fjs/emergent_testing/_browser-suite.mjs`, -and only then calls `run(main)` to emit the page. Everything it does — reading -directories, reading files, writing generated source — is expressible as Node -effects, so the split exists for no reason other than history, and the -preparation half is proved only through `browser-source.proof.mjs`'s unit tests -of the token scanner rather than end to end against the virtual filesystem. - -The [shared browser/console runner](../../emergent_testing/README.md) work that -this issue was carved out of is done: the browser and `fjs t` now run the same -proof semantics, so what is left here is the *build*, not the runner. - -### Preliminary design - -Restore the package command to the FunctionalScript entry point: - -```json -"website": "node ./fjs/module.mjs r ./fjs/website/module.f.mjs" -``` - -`fjs/website/module.f.mjs` must own proof discovery, manifest generation, and -HTML/entry generation as one `NodeProgram`. If preparation needs a Node -capability that the FunctionalScript program cannot currently express, add the -smallest operation to `fjs/effects/node/` and its real and virtual interpreters -instead of bypassing Effects. Existing `readdir`, `readFile`, and `writeFile` -operations should be reused where sufficient. - -`fjs/website/browser-source.mjs` — the token scanner answering "does this -module export `proof`?" and "which modules does it import?" — is already pure -and has no `try`/`catch` or regular expressions. Renaming it to `.f.mjs` and -proving it as authored FunctionalScript is the first step; the graph walk and -the blocker classification then move into the program beside it. - -### Constraints - -- Website build-time filesystem access must be expressed by the FunctionalScript - `NodeProgram` through Node effects; npm scripts must not run an impure helper - as a second application entry point. -- The generated manifest and page must stay byte-identical across the move, so - the change is provably a refactor. -- Do not restore the removed `index-html` alias. - -### Tasks - -- [ ] Rename `fjs/website/browser-source.mjs` to authored `.f.mjs` with a - co-located proof at full coverage. -- [ ] Move static proof discovery and `_browser-suite.mjs` generation into - `fjs/website/module.f.mjs`; extend `fjs/effects/node/` only for a concrete - missing capability and prove the real and virtual interpretations. -- [ ] Delete `fjs/website/browser-prepare.mjs` and make the sole `website` - command `node ./fjs/module.mjs r ./fjs/website/module.f.mjs`. -- [ ] Prove the generator end to end against the virtual filesystem: a module - whose graph reaches `node:` is skipped with its reason, one that does not - is emitted. - -### Related - -- [Generate website](generate-website.md) — the parent issue. -- [Browser testing](../../emergent_testing/todo/browser-testing.md) — the - browser-native application the manifest feeds. From fe112e84641d0aa6a605f32a225621f00a62afbc Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 27 Aug 2026 10:52:47 +0000 Subject: [PATCH 17/18] DESIGN: state the sharing cycle as five steps, ending in "solve for both" The first draft of "Follow the example" read as a demand for identical behaviour, which is the wrong target: a browser has no stdout and a terminal has no DOM, so different APIs and wrappers around a shared core are the normal shape. It also understated the obligation that actually keeps two contexts together. Restates the principle as the cycle: share the code; adjust where the host requires it; document every difference that remains; open an issue for each problem the port revealed; solve each of those issues for every context at once. Differences are allowed -- undocumented ones are not, and a fix that lands in one context only is how the two drift apart again while hiding the finding from the older one. `share-browser-console-runner.md` follows the same wording, and its constraints now say that host APIs may differ freely, behaviour only for a written reason, and a fix for either runner lands in both. --- DESIGN.md | 77 +++++++++++-------- .../todo/report-before-running.md | 3 + .../todo/share-browser-console-runner.md | 44 +++++++---- 3 files changed, 75 insertions(+), 49 deletions(-) diff --git a/DESIGN.md b/DESIGN.md index 8567a7b96..46a3ea467 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -108,45 +108,56 @@ on top of the weaker design. existing module; create a new one only if no good fit exists. This is different from DRY extraction: it is always appropriate. - **Follow the example** — when the same capability already exists elsewhere, - match it before improving on it. See below. + share it first, document what still differs, and fix what you find in every + context at once. See below. - **Avoid side effects and mutability.** ### Follow the example When a capability already exists somewhere in the repository and is being brought to a second context — another host, another backend, another runner — -**the existing one is the specification.** Reproduce its behaviour first, -including the simplifications it made and the things it does not do. Only once -the second context matches the first is it worth asking whether either should -change. - -This is not the same as reusing code, and it is the part that is easy to skip -while believing the principle is satisfied. Sharing a module and then giving the -new context its own rules produces something that *looks* unified and is not: -two behaviours behind one name, which is worse than two implementations behind -two names, because nothing signals the difference. - -The rule has three consequences worth stating outright. - -**A difference has to be justified, not merely noticed.** "The new context can -do better here" is a reason to file an issue, not a reason to diverge inside a -port. The example may be simple *for a reason* that is not visible from inside -the new context — `fjs t` runs proofs one after another, and its report is -readable, attributable and reproducible because of it. - -**A problem the new context reveals is everyone's problem.** If porting exposes -that a measurement is inaccurate, that an error loses attribution, or that an -ordering is unspecified, then it was very likely already true of the example and -merely easier to see now. Fix it once, for both, as its own change — or record -it as an issue. Fixing it only in the new context leaves the two out of step and -hides the finding from the place that has had the defect longest. - -**Solve it for the shared code or not at all.** A workaround that lives in one -host is a fork with extra steps. Either the shared layer learns the answer, or -the issue stays open and honest. - -The order, then, is: reuse and match the example; land that; *then* take the -new problems one at a time, as changes that apply everywhere. +**the existing one is the specification.** The order of work is: + +1. **Share the code.** Take the existing implementation as the shared core. +2. **Adjust where the second context genuinely requires it.** Different hosts + have different APIs, and a wrapper or an adapter around the shared core is + the normal, expected shape. Two contexts may end up behaving slightly + differently for reasons their hosts impose. +3. **Document every difference that remains,** at the point where it is made. +4. **Open an issue for each problem the port revealed,** rather than fixing it + inside the port. +5. **Solve each issue for every context at once,** so they stay in sync. + +The cost of skipping a step is not paid where it is skipped. Steps 1–2 without +3–5 give something that *looks* unified and is not: two behaviours behind one +name, which is worse than two implementations behind two names, because nothing +signals the difference. + +The parts worth stating outright: + +**Differences are allowed; undocumented differences are not.** The goal is not +one identical behaviour — a browser has no stdout and a terminal has no DOM, and +pretending otherwise invents a host that does not exist. The goal is that every +difference is deliberate, written down, and traceable to the host that forced +it. "This context can do better here" is not such a reason: that is an +improvement, and improvements go through step 4. + +**Solve it for both, or for neither.** Once an issue from step 4 is picked up, +the fix lands in every context, in the same change. A fix in one context only is +how the two drift back apart, and it hides the finding from the place that has +had the defect longest — which is usually the older one. This is the step that +keeps the contexts in sync, and it is the one under time pressure to skip. + +**The example may be simple for a reason.** What looks like a gap from inside +the new context is often a decision made in the old one. Copy it first; if it +turns out to be wrong, it is wrong in both places and worth an issue that says +so. + +**Keep the port separate from everything it inspires.** Land the sharing change +on its own, with behaviour unchanged. Anything new — a different scheduling +policy, a better measurement, an extra guard — is its own change afterwards. +Combined, they cannot be reviewed: an argument about the new idea becomes an +argument about the port. ### Exception to DRY: performance measurement diff --git a/fjs/emergent_testing/todo/report-before-running.md b/fjs/emergent_testing/todo/report-before-running.md index 9d5cded82..37d8fff98 100644 --- a/fjs/emergent_testing/todo/report-before-running.md +++ b/fjs/emergent_testing/todo/report-before-running.md @@ -58,6 +58,9 @@ the shared core rather than twice. - Whatever is emitted has to be as useful to an automated consumer as to a reader — a start with no matching result is precisely the signal a crashed run leaves behind, and a controller should be able to read it. +- The start event lands in both runners in the same change. Their output differs + — a terminal line and a DOM row — but a runner that names a running test and + one that does not are two different tools. ### Tasks diff --git a/fjs/emergent_testing/todo/share-browser-console-runner.md b/fjs/emergent_testing/todo/share-browser-console-runner.md index f61d69fc7..d200a3c93 100644 --- a/fjs/emergent_testing/todo/share-browser-console-runner.md +++ b/fjs/emergent_testing/todo/share-browser-console-runner.md @@ -31,14 +31,20 @@ repository does not want to pay again, and the record of why is worth more than the code was. **The order of work is the deliverable here, not just the final shape.** See [DESIGN.md §4, "Follow the example"](../../../DESIGN.md). -**`fjs t` is the specification, including the things it does not do.** The -attempt shared the modules and then let the browser keep its own rules: its own -test-name format, its own scheduling policy, its own clock. That is the failure -mode to avoid, and it is easy to miss because it *looks* like success — one -module, one name, two behaviours behind it. Two implementations behind two names -are more honest than that, because nothing about the shared name signals the -difference. Sharing code and sharing behaviour are different achievements, and -only the second one is this issue. +**Share the code, then keep the two in sync.** The order is: share; adjust +where the host genuinely requires it; document every difference that remains; +open an issue for each problem the port revealed; and solve each of those issues +for **both** runners at once. The last step is the one that matters and the one +under pressure to skip. + +Differences are fine — a browser has no stdout, a terminal has no DOM, and the +two will use different APIs and wrappers around the same core. *Undocumented* +differences are not. The attempt shared the modules and then let the browser +keep its own test-name format, its own scheduling policy and its own clock, none +of which the host forced. That is the failure mode: it *looks* like success — +one module, one name — while two behaviours hide behind it, and two +implementations behind two names would have been more honest, because nothing +about the shared name signals the difference. **`fjs t` is sequential, and that is a decision to copy, not a gap to fill.** The attempt gave the browser a batch size — proofs launched in groups with a @@ -69,11 +75,11 @@ both are properly issues rather than fixes inside a port: [Hostile thrown values and cross-realm promises](hostile-proof-values.md) and [Imports, promises and realms](imports-promises-realms.md). -The rule that follows: **land the shared core matching `fjs t` exactly, then -take each new problem as its own change that applies everywhere.** A difference -between hosts is something to justify in an issue, not to introduce inside a -port. If the port cannot preserve a behaviour, that is a finding to record -before the port merges, not a silent divergence to explain in review. +The rule that follows: **land the shared core with behaviour unchanged, then +take each new problem as its own change that lands in both runners together.** +An improvement the browser could have is an issue, not something to introduce +inside a port. A behaviour the port cannot preserve is a finding to record +before it merges, not a silent divergence to explain in review. **Keep the change reviewable.** The attempt was 2646 insertions and 1408 deletions across 35 files in one PR — a move, a rewrite, a new effects layer, a @@ -174,9 +180,13 @@ are shared. - Preserve the recursive proof semantics and totals of `fjs t` exactly, including objects with a proof property named `then`; only actual promises are asynchronous values. -- Both runners must produce the same test name for the same leaf. -- Introduce no host-specific behaviour that `fjs t` does not already have. A - difference the port cannot avoid is recorded as an issue before it merges. +- Both runners must produce the same test name for the same leaf. This one is + not a host difference: nothing about a browser prevents it, and a divergence + here is the visible sign that the semantics underneath were never unified. +- Every remaining difference between the two runners is documented where it is + made, and traceable to something the host forced. Host APIs and wrappers may + differ freely; behaviour may differ only for a written reason. +- A fix for a problem either runner has lands in both, in the same change. - Browser modules must not import Node built-ins, the Node effect interpreter, `node:test`, or Playwright. - Website build-time filesystem access must be expressed by the FunctionalScript @@ -226,6 +236,8 @@ are shared. test counts, and normalized failures from the same fixtures. - [ ] Record every behaviour the browser file has today and the shared core will not keep, as an issue, before the sharing change merges. +- [ ] Close each of those issues for both runners at once, so the two stay in + sync rather than drifting from the day the core is shared. ### Related From b9b273c7234b751097ada1467a7631bfe96cb88f Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 27 Aug 2026 10:55:13 +0000 Subject: [PATCH 18/18] DESIGN: the shared thing is the skeleton, not just the outcome "Share the code" left open where the sharing stops, which is the question that actually decides whether two runners stay one. States it: what is shared is the skeleton -- the control flow, the order of operations, the decisions and their names. Every context runs that same skeleton, and differs only by supplying a different part at a place the skeleton names. That leaves exactly two ways to accommodate a context, both additive: change that context's part, or improve the skeleton so every context benefits. There is no third. A branch inside the skeleton asking which host it is running on is a fork wearing a shared name, and a need no existing part can express is a missing extension point rather than a special case. `share-browser-console-runner.md` names the runner's skeleton and its parts in those terms -- execute a leaf, report a result, link a module -- and gains a task to check that nothing host-specific is left outside one of them. --- DESIGN.md | 72 ++++++++++++------- .../todo/share-browser-console-runner.md | 57 +++++++++------ 2 files changed, 83 insertions(+), 46 deletions(-) diff --git a/DESIGN.md b/DESIGN.md index 46a3ea467..c224dbc3a 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -107,46 +107,66 @@ on top of the weaker design. belongs in `fjs/path`, not inline in a loader). First search for an appropriate existing module; create a new one only if no good fit exists. This is different from DRY extraction: it is always appropriate. -- **Follow the example** — when the same capability already exists elsewhere, - share it first, document what still differs, and fix what you find in every - context at once. See below. +- **Follow the example** — one skeleton for every context; differences live in + the parts it calls, and improvements go into the skeleton so everyone gets + them. See below. - **Avoid side effects and mutability.** ### Follow the example When a capability already exists somewhere in the repository and is being brought to a second context — another host, another backend, another runner — -**the existing one is the specification.** The order of work is: - -1. **Share the code.** Take the existing implementation as the shared core. -2. **Adjust where the second context genuinely requires it.** Different hosts - have different APIs, and a wrapper or an adapter around the shared core is - the normal, expected shape. Two contexts may end up behaving slightly - differently for reasons their hosts impose. -3. **Document every difference that remains,** at the point where it is made. +**the existing one is the specification.** + +What is shared is the **skeleton**: the control flow, the order of operations, +the decisions and their names — the shape of the whole thing. Every context runs +that same skeleton. Where a context differs, it differs by supplying a different +**part** that the skeleton calls out to, at a place the skeleton names. It does +not differ by having a skeleton of its own. + +So there are exactly two ways to accommodate a context, and both are additive: + +- **Adjust that context's part.** A browser writes rows into a DOM where a + terminal writes lines to stdout; those are two implementations of one named + part, and the skeleton above them cannot tell which it has. +- **Improve the skeleton, for everyone.** If what the new context needs is + something the skeleton should have had, put it there. Every context gets it, + and that is a feature of the change rather than a side effect to apologize + for. + +There is no third way. A branch inside the skeleton that asks which host it is +running on is a fork wearing a shared name, and it is worse than two honest +implementations, because nothing about the shared name signals the difference. A +context that cannot be served by any existing part means the skeleton is missing +an extension point: add the point — one more named part that every context then +supplies — rather than a special case. + +The order of work follows from that: + +1. **Share the skeleton.** Take the existing implementation as the core, with + its behaviour unchanged. +2. **Adjust the parts** the new context genuinely requires, or extend the + skeleton so it can express what the new context needs. +3. **Document every difference that remains,** at the part where it is made. 4. **Open an issue for each problem the port revealed,** rather than fixing it inside the port. -5. **Solve each issue for every context at once,** so they stay in sync. - -The cost of skipping a step is not paid where it is skipped. Steps 1–2 without -3–5 give something that *looks* unified and is not: two behaviours behind one -name, which is worse than two implementations behind two names, because nothing -signals the difference. +5. **Solve each issue in the skeleton or in every part at once,** so the + contexts stay in sync. The parts worth stating outright: **Differences are allowed; undocumented differences are not.** The goal is not one identical behaviour — a browser has no stdout and a terminal has no DOM, and pretending otherwise invents a host that does not exist. The goal is that every -difference is deliberate, written down, and traceable to the host that forced -it. "This context can do better here" is not such a reason: that is an -improvement, and improvements go through step 4. - -**Solve it for both, or for neither.** Once an issue from step 4 is picked up, -the fix lands in every context, in the same change. A fix in one context only is -how the two drift back apart, and it hides the finding from the place that has -had the defect longest — which is usually the older one. This is the step that -keeps the contexts in sync, and it is the one under time pressure to skip. +difference lives in a named part, is deliberate, and is traceable to something +the host forced. "This context could do better here" is not such a reason: that +is an improvement, and an improvement belongs in the skeleton, where everyone +gets it. + +**Solve it for every context, or for none.** Once an issue from step 4 is picked +up, the fix lands everywhere in the same change. A fix in one context only is how +the contexts drift back apart, and it hides the finding from the place that has +had the defect longest — usually the older one. **The example may be simple for a reason.** What looks like a gap from inside the new context is often a decision made in the old one. Copy it first; if it diff --git a/fjs/emergent_testing/todo/share-browser-console-runner.md b/fjs/emergent_testing/todo/share-browser-console-runner.md index d200a3c93..5e3cc0e56 100644 --- a/fjs/emergent_testing/todo/share-browser-console-runner.md +++ b/fjs/emergent_testing/todo/share-browser-console-runner.md @@ -31,17 +31,26 @@ repository does not want to pay again, and the record of why is worth more than the code was. **The order of work is the deliverable here, not just the final shape.** See [DESIGN.md §4, "Follow the example"](../../../DESIGN.md). -**Share the code, then keep the two in sync.** The order is: share; adjust -where the host genuinely requires it; document every difference that remains; -open an issue for each problem the port revealed; and solve each of those issues -for **both** runners at once. The last step is the one that matters and the one -under pressure to skip. - -Differences are fine — a browser has no stdout, a terminal has no DOM, and the -two will use different APIs and wrappers around the same core. *Undocumented* -differences are not. The attempt shared the modules and then let the browser -keep its own test-name format, its own scheduling policy and its own clock, none -of which the host forced. That is the failure mode: it *looks* like success — +**One skeleton, with named parts.** The thing to share is the *runner itself*: +the order in which modules are linked, leaves discovered, bodies executed, +throws inverted, results counted and the run concluded. Both hosts run that same +skeleton. Everything host-specific is a **part** the skeleton calls at a place it +names — where the leaf body is executed, where a result is reported, where a +module is linked — and a part is where a browser is allowed to be a browser. + +That gives exactly two ways to accommodate a host, both additive: change *that +host's part*, or *improve the skeleton so every host benefits*. There is no +third. A branch inside the skeleton asking which host it is running on is a fork +wearing a shared name. A host need that no existing part can express means the +skeleton is missing an extension point — add the point, which every host then +supplies, rather than a special case. + +Differences between the parts are fine and expected: a DOM row and a terminal +line are two implementations of the same named part, and the skeleton above them +cannot tell which it has. *Undocumented* differences are not. The attempt shared +the modules and then let the browser keep its own test-name format, its own +scheduling policy and its own clock — none of which its host forced, and none of +which belonged in a part. That is the failure mode: it *looks* like success — one module, one name — while two behaviours hide behind it, and two implementations behind two names would have been more honest, because nothing about the shared name signals the difference. @@ -75,11 +84,12 @@ both are properly issues rather than fixes inside a port: [Hostile thrown values and cross-realm promises](hostile-proof-values.md) and [Imports, promises and realms](imports-promises-realms.md). -The rule that follows: **land the shared core with behaviour unchanged, then -take each new problem as its own change that lands in both runners together.** -An improvement the browser could have is an issue, not something to introduce -inside a port. A behaviour the port cannot preserve is a finding to record -before it merges, not a silent divergence to explain in review. +The rule that follows: **land the shared skeleton with behaviour unchanged, then +take each new problem as its own change — in the skeleton where it belongs +there, so both runners get it, or in every part at once.** An improvement the +browser could have is an issue, not something to introduce inside a port. A +behaviour the port cannot preserve is a finding to record before it merges, not +a silent divergence to explain in review. **Keep the change reviewable.** The attempt was 2646 insertions and 1408 deletions across 35 files in one PR — a move, a rewrite, a new effects layer, a @@ -183,10 +193,14 @@ are shared. - Both runners must produce the same test name for the same leaf. This one is not a host difference: nothing about a browser prevents it, and a divergence here is the visible sign that the semantics underneath were never unified. -- Every remaining difference between the two runners is documented where it is - made, and traceable to something the host forced. Host APIs and wrappers may - differ freely; behaviour may differ only for a written reason. -- A fix for a problem either runner has lands in both, in the same change. +- The skeleton never asks which host it is running on. Anything host-specific is + a part it calls; anything it cannot express through a part is a missing + extension point, not a special case. +- Every remaining difference between the two runners lives in a part, is + documented there, and is traceable to something the host forced. Host APIs and + wrappers may differ freely; behaviour may differ only for a written reason. +- A fix for a problem either runner has lands in the skeleton, or in every part + at once — in the same change. - Browser modules must not import Node built-ins, the Node effect interpreter, `node:test`, or Playwright. - Website build-time filesystem access must be expressed by the FunctionalScript @@ -210,6 +224,9 @@ are shared. - [ ] Inventory duplicated semantics in `emergent_testing/module.f.mjs` and `emergent_testing/browser.mjs`, and define the smallest shared API. +- [ ] Name the skeleton's parts explicitly — execute a leaf, report a result, + link a module — and check that nothing host-specific is left outside one + of them. - [ ] Make the existing `collectTests`/path behavior the single source of truth for console and browser execution. - [ ] Share the test-name format, and prove both runners name the same leaf