Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 8 additions & 6 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -306,12 +306,14 @@ import type ... from '../other/types.ts'
import type ... from './module.f.mjs'
```

The one exception is `fjs/emergent_testing/scenarios/*.ts`, `scenarios/all.ts`
and `all.test.ts`, which do have runtime imports. Their `.ts` extension is
load-bearing — `run.sh` dispatches on it to prove that Node, Bun and Deno execute
a **TypeScript** proof natively — so they are deliberately not `types.ts` and
must not be ported to `.mjs`. See the scenario-fixture item in
[`todo/migrate-typescript-to-mjs.md`](./todo/migrate-typescript-to-mjs.md).
The one exception is `fjs/emergent_testing/all.test.ts`, the entry point that
external runners load, which does have runtime imports and is therefore
deliberately not `types.ts`. It is the only authored non-`types.ts` TypeScript
left in the repository; `prepack` compiles it to the published
`all.test.js` that consumers import. The scenario fixtures that used to share
this exception are gone — see
[`fjs/emergent_testing/scenarios.md`](./fjs/emergent_testing/scenarios.md) for
what they covered and how to rebuild them.

The runtime-import grouping applies to repository-owned relative imports, not to
external or built-in modules: a FunctionalScript module may depend at runtime on
Expand Down
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,10 @@ history.

## Unreleased

- `fjs/types/bigfloat`: `decToBin` no longer returns a 54-bit mantissa when
rounding carries out of 53 bits; the result is always a binary64 significand
(`abs(m) < 2^53`)
[#1524](https://github.com/functionalscript/functionalscript/pull/1524)
- `fjs/text/ascii` owns the hex-digit codec: `hexDigitValue`,
`hexDigitCodePoint`, and the `a-f` / `A-F` ranges. The JSON serializer and
both tokenizers use it instead of rederiving the offsets; the DJS tokenizer
Expand Down
4 changes: 4 additions & 0 deletions fjs/emergent_testing/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,10 @@ Then invoke the runner:
You can also implement your own runner, as long as it follows the proof-tree
conventions described below.

This repository used to check the external runners against fixtures with known
pass/fail outcomes. That harness has been removed;
[scenarios.md](./scenarios.md) records what it covered and how to rebuild it.

## Design: dependency-free proofs

Unlike most test frameworks (Jest, Mocha, Vitest, …), a proof does **not** import
Expand Down
259 changes: 259 additions & 0 deletions fjs/emergent_testing/scenarios.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,259 @@
# Scenario fixtures

External-runner scenario fixtures used to live in
`fjs/emergent_testing/scenarios/`. They have been **removed**. This file records
what they were and how to rebuild them, so the capability can be recreated
deliberately rather than reconstructed from git archaeology.

## What they were

Nine one-module fixtures plus a shell harness. Each fixture exported a `proof`
whose outcome under an external runner (`node --test`, `bun test`,
`deno test`) was known in advance from its filename: `*.pass.ts` had to exit
`0`, `*.fail.ts` had to exit `1`. The harness ran one fixture at a time and
compared the runner's exit status against that expectation.

They tested the framework end to end — registration, async handling, sub-tests,
throw tests, thenable handling — through a real runner in a real process, which
is the one thing the in-process proofs in
[`proof.f.mjs`](./proof.f.mjs) cannot do for themselves.

## Why they were removed

- **Nothing ran them.** No CI job and no generated workflow invoked
`scenarios/run.sh`; it was a manual, undocumented step.
- **Part of it had already rotted.** `run.sh`'s `fjs` runner branch ran
`npm run fst`, a script that no longer exists in `package.json`, so that
quarter of the matrix reported `FAIL` for every fixture regardless of the
fixture. Nobody noticed, which is the clearest evidence they were unrun.
- **They blocked a migration.** `scenarios/*.ts` and `scenarios/all.ts` were,
with [`all.test.ts`](./all.test.ts), the last authored non-`types.ts`
TypeScript in the repository — see
[`todo/migrate-typescript-to-mjs.md`](../../todo/migrate-typescript-to-mjs.md).

Removing them deletes real coverage. That is the trade being accepted: the
coverage was not being collected anyway. Recreate it — from this file — if it
is wanted back, ideally wired into CI in the same change.

## What they covered

| Fixture | Expected exit | What it proved |
| --- | --- | --- |
| `fail.fail.ts` | `1` | A throwing test case fails the run. |
| `async.pass.ts` | `0` | An `async` test case is awaited and passes. |
| `async.fail.ts` | `1` | An `async` test case that rejects fails the run. |
| `async-subtests.pass.ts` | `0` | An `async` case returning an object of test cases has them run as sub-tests. |
| `async-subtests.fail.ts` | `1` | One failing sub-test of an `async` case fails the run. |
| `return-value.pass.ts` | `0` | A case returning an object is walked as a sub-tree. |
| `throw.pass.ts` | `0` | A case under a `throw` key passes *because* it throws. |
| `thenable.pass.ts` | `0` | A thenable is treated as a plain value, not awaited: its only key `then` is a function *with parameters*, so no leaf test is found and the run trivially passes. |
| `thenable2.pass.ts` | `0` | Same, for a zero-parameter `then` returning a value. |

The two thenable cases are the subtle ones and the reason to keep the set if it
is ever rebuilt: they pin FunctionalScript's decision that thenables are *not*
awaited, a rule no other test states.

## How to recreate

### 1. The fixtures

Each is a standalone module exporting `proof`, with no imports. Recreate them
verbatim:

```ts
// fail.fail.ts
export const proof = {
failing: () => { throw 'intentional failure' }
}
```

```ts
// async.pass.ts
export const proof = {
sleep: async () => {
await new Promise<void>(resolve => setTimeout(resolve, 10))
}
}
```

```ts
// async.fail.ts
export const proof = {
sleep_fail: async () => {
await new Promise<void>(resolve => setTimeout(resolve, 10))
throw 'async failure'
}
}
```

```ts
// async-subtests.pass.ts
export const proof = {
withSubtests: async () => {
await new Promise<void>(resolve => setTimeout(resolve, 10))
return {
sub1: () => {},
sub2: () => {},
}
}
}
```

```ts
// async-subtests.fail.ts
export const proof = {
withSubtests: async () => {
await new Promise<void>(resolve => setTimeout(resolve, 10))
return {
sub1: () => {},
sub2: () => { throw 'sub-test failure' },
}
}
}
```

```ts
// return-value.pass.ts
const inner = () => {}

export const proof = {
outer: (): unknown => ({ inner })
}
```

```ts
// throw.pass.ts
export const proof = {
throw: { a: () => { throw 'expected' } }
}
```

```ts
// thenable.pass.ts
export const proof = {
thenableResolves: () => ({
then(resolve: (v: undefined) => void) { resolve(undefined) }
})
}
```

```ts
// thenable2.pass.ts
export const proof = {
shouldPass: () => ({ then: () => 'ok' })
}
```

### 2. The entry-point shim

```ts
// all.ts
import '../all.test.ts'
```

This one-line file is **not** redundant with `all.test.ts`, and the reason is
the single most easily lost piece of this design — see
[Two traps](#two-traps) below.

### 3. The harness

```sh
#!/bin/sh
# Usage: run.sh <runner> <scenario>
# runner: fjs | bun | node | deno
# scenario: path to a *.pass.ts or *.fail.ts file
set -e

runner=$1
scenario=$(realpath "$2")

scendir=$(cd "$(dirname "$0")" && pwd)

case "$scenario" in
*.pass.ts) expected=0; scenfile="$scendir/_scenario.proof.ts" ;;
*.fail.ts) expected=1; scenfile="$scendir/_scenario.proof.ts" ;;
*) echo "unknown suffix: $scenario" >&2; exit 2 ;;
esac
allfile="$scendir/_all.test.ts"

ln "$scenario" "$scenfile"
ln "$scendir/all.ts" "$allfile"

cleanup() { rm -f "$scenfile" "$allfile"; }
trap cleanup EXIT

case "$runner" in
fjs) cmd="npm run fst" ;;
bun) cmd="bun test" ;;
node) cmd="node --test" ;;
deno) cmd="deno test --allow-read --allow-env --allow-sys" ;;
*) echo "unknown runner: $runner" >&2; exit 2 ;;
esac

actual=0
(cd "$scendir" && $cmd) > /dev/null 2>&1 || actual=$?

if [ "$actual" -eq "$expected" ]; then
echo "pass: $(basename "$scenario") [exit $actual]"
exit 0
else
echo "FAIL: $(basename "$scenario") [expected $expected, got $actual]"
exit 1
fi
```

Invoked as `sh run.sh node ./fail.fail.ts`. Fix the `fjs` branch before
reusing it: `npm run fst` does not exist. Whatever replaces it must run the
built-in runner over the scenario directory and exit non-zero on failure.

The harness hard-links exactly two files into the scenario directory,
`_scenario.proof.ts` (the fixture, renamed so the built-in runner's
`proof`-module discovery finds it) and `_all.test.ts` (the shim, renamed so the
external runner's `*.test.*` discovery finds it), runs the runner with the
directory as its working directory, and removes both links on exit.

## Two traps

Anyone rebuilding this will hit both.

### The shim cannot be replaced by hard-linking `all.test.ts`

A hard link has no "original": both names are equal directory entries to one
inode, and Node resolves a module's relative specifiers from whichever path it
was reached through. `all.test.ts` imports `../effects/node/module.mjs`, which
is correct at `fjs/emergent_testing/` and wrong one level deeper. Hard-linking
it into `scenarios/` fails at load:

```
Error [ERR_MODULE_NOT_FOUND]: Cannot find module
'.../fjs/emergent_testing/effects/node/module.mjs'
imported from .../scenarios/_all.test.ts
```

The shim works because it *lives in the directory it is linked into*, so its
own `../all.test.ts` stays correct at both paths. A symlink would resolve to
its realpath and avoid the shim entirely — a legitimate simplification if the
harness is rewritten, and one worth taking, since it collapses two files into
one.

### The shim must not be named `*.test.*` at rest

External runners scan the directory. If the at-rest shim also matched
`*.test.*`, the runner would discover both it and the `_all.test.ts` hard link
and register the whole suite twice — Node caches modules by resolved URL, not
by inode. That is why the file is `all.ts` and the link is `_all.test.ts`.
See [`todo/65z-singleton-effect.md`](./todo/65z-singleton-effect.md), which
proposes a general fix for duplicate proof execution under multiple paths.

## If you rebuild it

- **Wire it into CI in the same change.** An unrun harness rots silently, which
is how this one ended up with a permanently failing runner branch.
- **Decide the language deliberately.** These fixtures were TypeScript on
purpose: they proved Node, Bun and Deno execute a *TypeScript* proof
natively. If that property no longer needs testing, `.mjs` fixtures are
simpler and drop the fixtures from the `prepack` emit pass.
- **Prefer a symlink or a generated file** over the hard link, and the shim
disappears.
- **Keep the thenable cases.** They are the only statement of the
not-awaited rule.
1 change: 0 additions & 1 deletion fjs/emergent_testing/scenarios/all.ts

This file was deleted.

9 changes: 0 additions & 9 deletions fjs/emergent_testing/scenarios/async-subtests.fail.ts

This file was deleted.

9 changes: 0 additions & 9 deletions fjs/emergent_testing/scenarios/async-subtests.pass.ts

This file was deleted.

6 changes: 0 additions & 6 deletions fjs/emergent_testing/scenarios/async.fail.ts

This file was deleted.

5 changes: 0 additions & 5 deletions fjs/emergent_testing/scenarios/async.pass.ts

This file was deleted.

3 changes: 0 additions & 3 deletions fjs/emergent_testing/scenarios/fail.fail.ts

This file was deleted.

5 changes: 0 additions & 5 deletions fjs/emergent_testing/scenarios/return-value.pass.ts

This file was deleted.

42 changes: 0 additions & 42 deletions fjs/emergent_testing/scenarios/run.sh

This file was deleted.

Loading