diff --git a/AGENTS.md b/AGENTS.md index 4a2d4ea441..4f4944fd1d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -158,33 +158,38 @@ acceptable. If a line or branch genuinely cannot be reached, restructure the cod so it isn't there rather than leaving it uncovered. The implementation and proof extensions are independent during the incremental -`.f.mjs` migration: +TypeScript-to-JavaScript migration: | Implementation | Proof | When | |---|---|---| -| `module.f.ts` | `proof.f.ts` | Default. | -| `module.f.mjs` | `proof.f.ts` | A module migrated to `.f.mjs` keeps its TypeScript proof, which may still import `.f.ts` test helpers such as `fjs/asserts/module.f.ts`. | -| `module.f.mjs` | `proof.f.mjs` | Once the proof's own syntax and relative FunctionalScript dependencies are compiler-ready. | +| `module.f.ts` | `proof.f.ts` | Both files are still authored TypeScript. | +| `module.f.mjs` | `proof.f.ts` | The implementation has migrated, while the proof temporarily remains TypeScript. | +| `module.f.mjs` | `proof.f.mjs` | The proof is valid JavaScript/JSDoc and all of its authored FunctionalScript runtime/type dependencies outside the migration group are already `.f.mjs`. Compiler readiness is not required. | Renaming an implementation to `.f.mjs` therefore never requires renaming its -proof, and never removes it from proof discovery or from Node and Deno coverage: -`shouldLoad` in [`fjs/dev/module.f.ts`](./fjs/dev/module.f.ts) matches both -authored extensions, and both `npm run cov` and `deno task cov` include -`module.f.ts` and `module.f.mjs`. Ordinary (non-FunctionalScript) `.mjs` files -stay opt-in through the `proof.mjs` filename convention. +proof in the same change, and never removes it from proof discovery or from Node +and Deno coverage: `shouldLoad` in [`fjs/dev/module.f.ts`](./fjs/dev/module.f.ts) +matches both authored extensions, and both `npm run cov` and `deno task cov` +include `module.f.ts` and `module.f.mjs`. Ordinary (non-FunctionalScript) `.mjs` +files stay opt-in through the `proof.mjs` filename convention. Stage 1 still +ends with no authored `.f.ts`, so every remaining `proof.f.ts` must eventually +migrate to `proof.f.mjs`. A `proof.f.mjs` is authored `.f.mjs` like any other, so it must satisfy the same -dependency-closure rule as any other migrated file — its relative imports and -type references may target `.f.mjs` modules only. That rule is what makes -`proof.f.ts` the default layout for a migrated module: a TypeScript proof can -keep using the existing `.f.ts` test helpers. See +source dependency-closure rule as any other migrated file: its relative runtime +imports and JSDoc type references may target `.f.mjs` modules only. This is a +JavaScript/JSDoc and dependency-readiness rule, not a FunctionalScript compiler +feature gate. A proof that still imports an unmigrated helper such as +`fjs/asserts/module.f.ts` remains `proof.f.ts` until that helper migrates. See [`fjs/fsc/README.md`](./fjs/fsc/README.md) for the migration order and the module-import policy it implies. ### 3.3 Use `assert` / `assertEq`, never a hand-written `if`/`throw` -Assert results in `proof` code with `assert`/`assertEq` from -`fjs/asserts/module.f.ts`, not a hand-written `if (cond) { throw ... }`. +Assert results in `proof` code with `assert`/`assertEq` from the current authored +`fjs/asserts/module.f.*` source, not a hand-written `if (cond) { throw ... }`. +During Stage 1 that helper is `.f.ts` until it migrates to `.f.mjs`; a +`proof.f.mjs` that needs it must wait for that dependency migration. A local `if`/`throw` in a test is itself a new branch for the coverage tool to track, and its failure side is normally never exercised (the test is expected to @@ -429,6 +434,53 @@ cases remain explicit and independently testable. ### 6.2 Types +#### JavaScript/JSDoc type declarations + +Authored `.mjs` / `.f.mjs` files must remain valid JavaScript. Put named and +generic static types in JSDoc rather than TypeScript syntax, and preserve the +same public assignability and declaration-emission behavior when translating a +`.ts` / `.f.ts` file. + +Use `@typedef` for a named type and `@template` for its type parameters. A +constraint goes in braces before the parameter name: + +```js +/** + * @template {Operation} O + * @template T + * @typedef {(_: Pr[1]) => Effect} Cont + */ +``` + +TypeScript 7 also supports variance modifiers on JSDoc type-alias parameters. +Translate TypeScript `in` / `out` directly on `@template` instead of dropping +the variance annotation. For example: + +```ts +export type Cont = + (_: Pr[1]) => Effect +``` + +becomes: + +```js +/** + * @template {Operation} out O + * @template T + * @typedef {(_: Pr[1]) => Effect} Cont + */ +``` + +The supported forms are `@template out T`, `@template in T`, and constrained +forms such as `@template {Operation} out O`. Variance modifiers belong to type +parameters of a JSDoc type alias (`@typedef`); do not put `in` / `out` on an +ordinary function's `@template`, where TypeScript rejects them. + +When translating a public type, verify both normal type checking and emitted +`.d.ts` / `.d.mts` declarations. The JSDoc spelling may differ, but the public +type contract must not become weaker just because the source moved to +JavaScript. + #### Prefer inference Let TypeScript infer the type of private constants, local variables, and return @@ -803,9 +855,17 @@ anywhere else as the rule being broken. ### 6.5 FunctionalScript module rules -- Only import other `.f.ts` files from FunctionalScript modules. Avoid references - to built-in or external Node modules such as `node:path` in `.f.ts` files. -- No `try`/`catch` — see [§3.5](#35-never-use-trycatch-test-throwing-with-the-throw-key). +During Stage 1 of the TypeScript-to-JavaScript migration, relative authored +FunctionalScript dependencies follow the asymmetric source rule: + +- `.f.ts` may import `.f.ts` or already migrated `.f.mjs`; +- `.f.mjs` may import only `.f.mjs` authored FunctionalScript dependencies; +- compiler support does not gate an `.f.ts` / `proof.f.ts` -> `.f.mjs` / + `proof.f.mjs` rename; JavaScript/JSDoc validity and dependency closure do. + +Avoid references to built-in or external Node modules such as `node:path` in +FunctionalScript source. No `try`/`catch` — see +[§3.5](#35-never-use-trycatch-test-throwing-with-the-throw-key). ### 6.6 Formatting @@ -942,4 +1002,4 @@ Only add CHANGELOG entries for code changes — PRs that only touch `todo/`, - Releasing is its own commit: the version lives in `package.json` (`"version"`) — `deno.json` holds tasks and formatting only. When it's bumped, create a new `## X.Y.Z` section in `CHANGELOG.md` immediately after `## Unreleased` and move - all entries from `## Unreleased` into it, leaving `## Unreleased` empty. + all entries from `## Unreleased` into it, leaving `## Unreleased` empty. \ No newline at end of file diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index e05f414603..7a7de6ee58 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -66,9 +66,12 @@ Bun, and published-CLI equivalents are listed in [AGENTS.md §1.4](./AGENTS.md#14-ways-to-run-the-functionalscript-test-suite). New `.f.ts` and `.f.mjs` modules need a co-located proof with 100% proof -coverage — see [AGENTS.md §3](./AGENTS.md#3-testing-and-proof-coverage). A -`module.f.mjs` may keep a `proof.f.ts` during the incremental `.f.mjs` -migration, or use a `proof.f.mjs` once that proof is itself compiler-ready. +coverage — see [AGENTS.md §3](./AGENTS.md#3-testing-and-proof-coverage). During +the TypeScript-to-JavaScript migration, a `module.f.mjs` may keep a +`proof.f.ts`, or the proof may migrate to `proof.f.mjs` as soon as the proof +itself is valid JavaScript/JSDoc and its authored FunctionalScript dependencies +are already `.f.mjs`. Current FunctionalScript compiler support is not required +for that proof rename. ### Updating dependencies diff --git a/fjs/ci/todo/f-js-package-support.md b/fjs/ci/todo/f-js-package-support.md new file mode 100644 index 0000000000..25573bad70 --- /dev/null +++ b/fjs/ci/todo/f-js-package-support.md @@ -0,0 +1,100 @@ +## Package support for authored `.f.js` + +**Priority:** P1 +**Status:** blocked +**Blocked by:** [`todo/migrate-typescript-to-mjs.md`](../../../todo/migrate-typescript-to-mjs.md) + +### Problem + +After authored TypeScript is removed, `.f.js` will become the marker for +FunctionalScript source accepted by the current parser/compiler. The existing +package design, however, is built around authored `.ts` / `.mjs` roots and does +not yet guarantee that a standalone authored `.f.js` module is directly checked, +gets a generated `.d.ts`, is included in the NPM package, or type-checks from a +clean consumer. + +Compiler compatibility alone is therefore not enough to rename `.f.mjs` to +`.f.js`. Without explicit package/tooling support, a compiler-ready module could +ship without the declaration and validation guarantees expected from the rest of +the package. + +This work cannot begin until +[`migrate-typescript-to-mjs.md`](../../../todo/migrate-typescript-to-mjs.md) +finishes: while `.f.ts` exists, TypeScript can generate `.f.js`, and the +repository intentionally ignores `**/*.js`. + +### Proposal + +After stage 1 removes authored TypeScript, TypeScript-to-JavaScript emission, and +the blanket `.js` ignore, make authored `.f.js` a first-class checked and +packable source extension before the first compiler-compatibility rename. + +The stage-2 invariant is: + +```text +source.f.mjs -> source.f.js + source.f.d.ts +``` + +The rename from `.f.mjs` to `.f.js` means the source is accepted by the current +FunctionalScript compiler. TypeScript still checks the authored JavaScript via +`allowJs` / `checkJs` and emits its declaration. + +Validation must include `.f.js` as an explicit root/source pattern, not merely +rely on another `.mjs` module importing it. Declaration emission must likewise +cover standalone `.f.js` modules and produce `.d.ts` files for package +consumers. + +Package and publish jobs continue to run in CI from a clean checkout. Stage 2 +therefore does not need generated-output cleanup or repeated local-pack safety; +it only needs to make sure authored `.f.js` is included as source and never +mistaken for generated TypeScript output. + +NPM packaging must include the authored `.f.js` runtime and its `.d.ts` +declaration, and clean-consumer tests must verify runtime and type resolution. + +Do not add package-time import or declaration-specifier rewriting. A migrated +`.f.js` group must remain dependency-closed according to the compiler migration +rules in [`todo/fjs-nanvm-integration.md`](../../../todo/fjs-nanvm-integration.md). + +### Tasks + +- [ ] After stage 1, include authored `.f.js` directly in the root TypeScript + checked source set with `allowJs` / `checkJs`. +- [ ] Ensure declaration emission produces `.d.ts` for standalone authored + `.f.js` modules. +- [ ] Verify NPM package rules include authored `.f.js` and its `.d.ts`. +- [ ] Add an authored `.f.js` package fixture that is not reachable only through + an `.mjs` root, proving direct source discovery. +- [ ] Verify the fixture is type-checked in the repository. +- [ ] Verify the clean CI package build contains the authored `.f.js` and its + generated declaration. +- [ ] Verify a clean consumer can import the `.f.js` runtime and type-check + against its generated `.d.ts` without repository source files. +- [ ] Update package/contributor documentation for the stage-2 authored `.f.js` + meaning. + +### Acceptance criteria + +- A standalone authored `.f.js` is directly included in repository TypeScript + checking. +- Declaration emission produces a corresponding `.d.ts`. +- The packed NPM artifact contains the authored `.f.js` and all declarations + required by its public/transitive type graph. +- Package/publish runs start from a clean CI checkout; no local generated-output + cleanup or repeated-pack guarantee is required. +- A clean consumer can execute/import the `.f.js` runtime and type-check it. +- No staging tree or package-time runtime/declaration specifier rewrite is + required. +- The first `.f.mjs` -> `.f.js` compiler-compatibility rename is **blocked by** + completion of this task. + +### Related + +- [`todo/migrate-typescript-to-mjs.md`](../../../todo/migrate-typescript-to-mjs.md) + — stage-1 prerequisite that removes TypeScript and makes `.js` authorable. +- [`f-mjs-package-support.md`](./f-mjs-package-support.md) — stage-1 authored + `.mjs` package support. +- [`publishing-packages.md`](./publishing-packages.md) — broader package plan. +- [`fjs/fsc/README.md`](../../fsc/README.md) — extension contract. +- [`todo/fjs-nanvm-integration.md`](../../../todo/fjs-nanvm-integration.md) — + compiler-compatibility migration blocked by this package prerequisite. diff --git a/fjs/ci/todo/f-mjs-package-support.md b/fjs/ci/todo/f-mjs-package-support.md index c7a5d7e813..a7a101454a 100644 --- a/fjs/ci/todo/f-mjs-package-support.md +++ b/fjs/ci/todo/f-mjs-package-support.md @@ -1,157 +1,183 @@ -## Package support for authored `.f.mjs` +## Package support for authored `.mjs` **Priority:** P1 **Status:** open ### Problem -The first existing repository module cannot move from `.f.ts` to authored -`.f.mjs` until the TypeScript and NPM package pipeline understands that source -extension. The current package configuration validates and publishes generated -`.js` and `.d.ts`, but does not yet provide the corresponding authored `.mjs` -and generated `.d.mts` path. - -The repository's authoritative FunctionalScript module rule also currently -allows imports only from `.f.ts`. Incremental migration requires unmigrated -`.f.ts` importers to target modules after those dependencies move to `.f.mjs`, -while migrated `.f.mjs` modules must remain closed over migrated dependencies. - -Emission must also be repeatable in an ordinary working tree. A declaration -produced by an earlier pack can be resolved as an input by a later declaration -pass, even when generated declarations are excluded from root discovery. The -second pass can then fail with TS5055 while trying to overwrite that input. - -This work blocks the P1 incremental compiler migration and fjs–nanvm integration. -Keeping it only inside the broader P3 package-publishing roadmap makes the -ordering unclear and allows a prerequisite to be deferred behind the work it -blocks. +The repository-wide +[TypeScript-to-`.mjs` migration](../../../todo/migrate-typescript-to-mjs.md) +cannot convert its first package-owned `.ts` / `.f.ts` source until the +TypeScript and NPM pipeline treats authored `.mjs` as first-class source. + +The current package configuration validates authored TypeScript and publishes +its generated `.js` / `.d.ts`, but does not yet provide the corresponding +checked authored `.mjs` plus generated `.d.mts` path. Turning on `allowJs` / +`checkJs` while keeping the current one-pass emit makes authored `.mjs` both an +input and an output target. + +Stage 1 is dependency-first. Remaining `.ts` / `.f.ts` may import already +migrated `.mjs` / `.f.mjs`, but migrated JavaScript must not retain runtime or +declaration references to remaining TypeScript. The package pipeline does not +rewrite module specifiers, so this invariant must work directly in a clean +checkout and packed artifact. + +Packaging and publishing run in CI from a clean checkout. Generated `.js`, +`.d.ts`, and `.d.mts` from an earlier commit or package build therefore do not +survive into the next package job. `prepack` is part of packaging, not a normal +development command, so the design does not need a local working-tree cleanup +protocol or separately exposed emission scripts. ### Proposal -Implement the minimum validation, repeatable emission, package-content, -repository-policy, and consumer tests needed before the first real `.f.mjs` -module enters the repository's published runtime graph. The broader package -strategy and other package targets remain in +Implement the minimum validation, emission, package-content, repository-policy, +and clean-consumer support required before the first stage-1 source migration. +The broader package roadmap remains in [`publishing-packages.md`](./publishing-packages.md). -Use the authored/generated extension invariant: +Use the stage-1 authored/generated invariant: ```text source.ts -> source.js + source.d.ts source.mjs -> source.mjs + source.d.mts ``` -TypeScript validates both authored extensions. Declaration emission covers both -`.ts` and `.mjs`, while JavaScript emission runs only for `.ts`; authored `.mjs` -is copied unchanged into the package. - -Run a repository-owned generated-output cleanup before declaration emission. -The cleanup derives output paths from authored `.ts` and `.mjs` inputs and -removes only their generated `.js`, `.d.ts`, and `.d.mts` artifacts. It must not -remove authored `.mjs`, unrelated files, or use a broad working-tree cleanup such -as `git clean`. Starting every `prepack` from this known emission state prevents -an earlier `.d.mts` from becoming an input to the next declaration pass. +Enable `allowJs` and `checkJs` before the first source conversion so TypeScript +validates both authored extensions. -Do not introduce a staging tree or rewrite module specifiers during packaging. -A migrated `.f.mjs` group must therefore be dependency-closed across both -runtime imports and references retained in emitted `.d.mts` declarations. -Relative runtime or type references from authored `.f.mjs` must resolve to -`.f.mjs` modules already migrated or converted in the same group. Authored -`.ts` may import `.mjs`, and its generated outputs preserve the `.mjs` -specifier. +Use one packaging lifecycle command with two ordered TypeScript passes while +TypeScript source remains: -Update the FunctionalScript module rules in `AGENTS.md` with the same asymmetric -policy: - -- authored `.f.ts` may import relative `.f.ts` or `.f.mjs` modules; -- authored `.f.mjs` may import or reference relative `.f.mjs` modules only; -- neither extension may use built-in or external Node modules. +```json +"prepack": "tsc --noEmit false --emitDeclarationOnly && tsc --noEmit false --declaration false" +``` -This permits unmigrated callers to follow a renamed dependency without weakening -the dependency-closure invariant for compiler-ready `.f.mjs` source. +The first pass emits declarations for both `.ts` and `.mjs`. With those +declarations present, the second TypeScript invocation resolves the generated +`.d.mts` declarations for authored `.mjs` modules, so it emits runtime +JavaScript for the remaining TypeScript sources without overwriting authored +`.mjs`. + +This exact configuration is already exercised by +[PR #1451](https://github.com/functionalscript/functionalscript/pull/1451): it +enables `allowJs` / `checkJs`, keeps `benchmark.mjs` in the repository, uses the +same two-pass `prepack`, and its Node 26 CI `npm pack` step succeeds. Keep this +simple ordering rather than adding a separate runtime-emission configuration +unless a real repository case demonstrates that it is needed. + +Keep both passes inline in `prepack`; do not add public `emit:*` scripts for +users to run independently. Normal development should type-check and test the +source tree without generating package artifacts. + +Because the CI package job starts from a clean checkout, a renamed +`source.ts -> source.mjs` does not carry ignored `source.js` / `source.d.ts` +artifacts from an earlier revision into the package job. Those files never need +to be discovered or deleted by the new `.mjs` input. + +Do not introduce a staging tree or rewrite runtime/declaration specifiers. An +authored `.mjs` / `.f.mjs` group must therefore be closed over authored +JavaScript dependencies outside the group. Remaining TypeScript may import +already migrated `.mjs`; the reverse direction is rejected. + +For FunctionalScript modules during stage 1: + +- `.f.ts` is remaining authored TypeScript; +- `.f.mjs` is authored FunctionalScript-intent JavaScript, whether or not the + current FunctionalScript compiler accepts all of its syntax; +- `.f.ts` may depend on `.f.ts` or already migrated `.f.mjs`; +- `.f.mjs` may depend on `.f.mjs`, not remaining `.f.ts` or generated `.f.js`. + +Update `AGENTS.md` with that asymmetric source-migration policy. Compiler +compatibility is a later `.f.mjs` -> `.f.js` migration and is not part of this +package prerequisite. + +Package selection does not need to distinguish every authored `.mjs` by public +API status during this transition. Incidental authored files such as +`fjs/types/bigint/benchmark.mjs` may be present in the archive; because they are +not part of the documented public API, their presence does not block this task. +They can be removed separately when no longer useful. + +As soon as no authored `.ts` / `.f.ts` source remains, remove the second +TypeScript runtime-emission pass. `prepack` then needs only declaration emission: + +```json +"prepack": "tsc --noEmit false --emitDeclarationOnly" +``` ### Tasks -- [ ] Make `fjs/types/bigint/benchmark.mjs` pass TypeScript validation or delete - it if it is no longer needed. -- [ ] Enable `allowJs` and `checkJs` in the main TypeScript configuration. -- [ ] Include authored `.ts` and `.mjs` source while excluding generated - `.d.ts` and `.d.mts` declarations from source validation. -- [ ] Update NPM package rules to include package-owned `.mjs` and `.d.mts` - files while excluding unrelated `.mjs` files. -- [ ] Add a cross-platform repository script that derives and removes only the - generated `.js`, `.d.ts`, and `.d.mts` outputs for authored source. -- [ ] Replace the current one-pass `prepack` script with generated-output - cleanup, declaration emission for `.ts` and `.mjs`, and JavaScript - emission from `.ts` only, in that order. -- [ ] Add a package fixture containing an authored `.ts` module and an authored - JSDoc `.mjs` module. -- [ ] Test the supported mixed-source direction, authored `.ts` importing - authored `.mjs`, in a clean checkout and from the packed archive. -- [ ] Run `npm pack` twice consecutively without manual cleanup and verify that - both runs succeed and contain the same package file set. -- [ ] Reject authored `.mjs` runtime imports that reference relative `.ts` or - generated `.js` files. -- [ ] Reject JSDoc or declaration-retained references from authored `.mjs` to - relative `.ts` or generated `.js` files. -- [ ] Verify emitted `.d.mts` files contain no references to files omitted from - the packed archive. -- [ ] Type-check a clean consumer against the packed archive, including an - exported type from the authored `.mjs` fixture and its transitive - declaration dependencies. -- [ ] Verify the packed archive contains authored `.mjs`, generated `.js`, - `.d.ts`, and `.d.mts` files in the expected package paths. -- [ ] Update `AGENTS.md` so `.f.ts` FunctionalScript modules may import relative - `.f.ts` or `.f.mjs`, while authored `.f.mjs` runtime and type dependencies - remain restricted to relative `.f.mjs`. -- [ ] Add validation or proofs for the allowed `.f.ts` → `.f.mjs` direction and - the rejected `.f.mjs` → `.f.ts` / generated `.f.js` directions. +- [ ] Keep `fjs/types/bigint/benchmark.mjs` type-checked with the rest of authored + JavaScript; removing the benchmark is a separate cleanup and is not a + prerequisite for this task. +- [ ] Enable `allowJs` and `checkJs` in the root TypeScript configuration before + the first `.ts` / `.f.ts` migration. +- [ ] Update NPM package rules to include authored `.mjs` and generated `.d.mts`. + Do not add special exclusions merely for non-public authored `.mjs` files. +- [ ] Replace one-pass package emission with the two ordered `tsc` commands + directly in `prepack`: declarations first, then JavaScript emission. +- [ ] Do not expose separate `emit:*` package scripts; packaging owns generated + outputs. +- [ ] Keep package/publish jobs on a clean CI checkout; do not add generated + output tracking or cleanup for artifacts from previous revisions. +- [ ] Add a mixed authored `.ts` + JSDoc `.mjs` package fixture. +- [ ] Test the allowed `.ts` -> `.mjs` dependency direction in a clean checkout + and CI-built package archive. +- [ ] Reject authored `.mjs` runtime imports and declaration-retained references + to remaining relative `.ts` or generated `.js`. +- [ ] Verify emitted `.d.mts` contains no references to omitted package files. +- [ ] Type-check a clean consumer using exported/transitive types from the + authored `.mjs` fixture. +- [ ] Verify the CI-built archive contains authored `.mjs`, generated `.js`, + `.d.ts`, and `.d.mts` in the expected paths during stage 1. +- [ ] Update `AGENTS.md` to the asymmetric `.f.ts` / `.f.mjs` migration policy. +- [ ] Add validation/proofs for the allowed TypeScript -> migrated-JavaScript + direction and rejected migrated-JavaScript -> TypeScript direction. ### Acceptance criteria -- The main TypeScript check validates authored `.ts` and `.mjs` source without - treating generated declarations as source inputs. -- Every pack begins by removing only known generated outputs derived from - authored source. -- Packing emits `.d.ts` for `.ts`, emits `.d.mts` for `.mjs`, emits `.js` only +- `allowJs` and `checkJs` are enabled before the first source conversion. +- The main TypeScript check validates authored `.ts` and `.mjs`. +- `prepack` contains the two ordered `tsc` passes directly while TypeScript + remains, with declaration emission first and JavaScript emission second. +- The exact two-pass command succeeds under `npm pack` with authored `.mjs` + present; PR #1451 provides the initial repository validation of this behavior. +- Package emission produces `.d.ts` for `.ts`, `.d.mts` for `.mjs`, `.js` only for `.ts`, and preserves authored `.mjs` unchanged. -- Two consecutive `npm pack` runs succeed without manual cleanup, do not produce - TS5055, and contain the same package file set. -- The package contains every runtime and declaration file needed by migrated - `.f.mjs` modules and excludes unrelated `.mjs` files. -- Authored `.ts` importing authored `.mjs` works before and after packing. -- Authored `.mjs` cannot introduce relative runtime or declaration references - to unmigrated `.ts` or generated `.js` files. -- A clean consumer can import the packed `.mjs` runtime and type-check against - its emitted `.d.mts` declarations without access to repository source files. -- `AGENTS.md` explicitly permits `.f.ts` modules to import `.f.mjs` and preserves - the dependency-closed import and type-reference rule for authored `.f.mjs`. -- No staging tree or package-time runtime-import or declaration-specifier - rewriting is needed. +- Non-public authored `.mjs` files do not require special package exclusions. +- No separate user-facing `emit:*` scripts are required. +- Package/publish runs start from a clean CI checkout, so ignored generated + outputs from previous revisions cannot leak into a package build. +- No repository-owned cleanup or legacy generated-output tracking is required + for the stage-1 migration. +- Remaining `.ts` may import migrated `.mjs`; migrated `.mjs` cannot retain + runtime/declaration references to remaining `.ts` or generated `.js`. +- A clean consumer can import the CI-built `.mjs` runtime and type-check against + its `.d.mts` declarations. +- `.f.mjs` carries no current-compiler compatibility promise during stage 1. +- No staging tree or package-time specifier rewrite is needed. ### Ordering -Complete this task, including repeatable emission and the `AGENTS.md` -module-import policy update, before converting the first existing repository -module from `.f.ts` to `.f.mjs`. A synthetic compiler fixture that does not -enter the published runtime graph may be used earlier. - -This task now comes **first** among the two `.f.mjs` prerequisites. The tooling -half of -[`.f.mjs` test and coverage support](../../emergent_testing/todo/f-mjs-test-and-coverage.md) -— proof discovery, Node and Deno coverage, the generated CI step, and the -`AGENTS.md`/`CONTRIBUTING.md` proof policy — has shipped; what remains there are -the runtime fixtures, which cannot land until authored `.mjs` is a type-checked, -packable source extension. Enabling `allowJs`/`checkJs` on its own is not enough: -`npm run prepack` (`tsc --NoEmit false`) then fails with TS5055 because authored -`.mjs` becomes both an input and a JavaScript emit target, which is what the -cleanup and split-emission tasks above fix. +Complete this task before the first package-owned `.ts` / `.f.ts` -> `.mjs` / +`.f.mjs` conversion in +[`todo/migrate-typescript-to-mjs.md`](../../../todo/migrate-typescript-to-mjs.md). +The migration then proceeds gradually from dependency leaves. + +After the last authored `.ts` / `.f.ts` source is removed, simplify `prepack` to +its declaration-only form and remove the TypeScript-to-JavaScript emit path. +Then the separate [`f-js-package-support.md`](./f-js-package-support.md) task +prepares authored `.f.js` before compiler-compatibility migration starts. ### Related -- [`publishing-packages.md`](./publishing-packages.md) — broader P3 package - publishing roadmap and authored/generated JavaScript convention. -- [`fjs/fsc/README.md`](../../fsc/README.md) — FunctionalScript extension - contract and incremental migration strategy. -- [`todo/fjs-nanvm-integration.md`](../../../todo/fjs-nanvm-integration.md) — P1 - integration plan blocked by this package prerequisite. +- [PR #1451](https://github.com/functionalscript/functionalscript/pull/1451) — + initial implementation and CI validation of `allowJs` / `checkJs` plus the + two-pass `prepack`. +- [`todo/migrate-typescript-to-mjs.md`](../../../todo/migrate-typescript-to-mjs.md) + — repository-wide stage-1 source migration. +- [`publishing-packages.md`](./publishing-packages.md) — broader package roadmap. +- [`f-js-package-support.md`](./f-js-package-support.md) — stage-2 authored + `.f.js` package prerequisite. +- [`fjs/fsc/README.md`](../../fsc/README.md) — authoritative extension contract. +- [`.f.mjs` test and coverage support](../../emergent_testing/todo/f-mjs-test-and-coverage.md) + — runtime proof/coverage fixtures for authored `.f.mjs`. diff --git a/fjs/ci/todo/publishing-packages.md b/fjs/ci/todo/publishing-packages.md index 6d4cd0bc3d..c9c7276b24 100644 --- a/fjs/ci/todo/publishing-packages.md +++ b/fjs/ci/todo/publishing-packages.md @@ -23,120 +23,171 @@ FunctionalScript can't currently be installed from Git using NPM. - [x] Check if the version is new, then publish. +Package and publish jobs run in CI from a clean checkout. We do not rely on +packing from a developer working tree, and ignored generated outputs from an +earlier revision are not part of the package-build state. `prepack` belongs to +this packaging path; normal development should type-check and test without +producing package artifacts. + ### Authored and generated JavaScript extensions -During the transition from TypeScript to JavaScript, use different extensions for authored and generated JavaScript: +The repository source migration is split into two stages; see +[`todo/migrate-typescript-to-mjs.md`](../../../todo/migrate-typescript-to-mjs.md) +and the authoritative FunctionalScript extension contract in +[`fjs/fsc/README.md`](../../fsc/README.md). + +During stage 1, use different extensions for authored JavaScript and generated +TypeScript output: ```text source.ts -> source.js + source.d.ts source.mjs -> source.mjs + source.d.mts ``` -The extension invariant is: +The stage-1 invariant is: -- `.ts` is authored TypeScript; +- `.ts` is authored TypeScript still awaiting migration; - `.mjs` is authored ESM JavaScript with JSDoc types; -- `.js` is generated JavaScript and is never authored; +- `.js` is generated from `.ts` and is not authored; - `.d.ts` and `.d.mts` are generated declarations. -For FunctionalScript modules, [`fjs/fsc/README.md`](../../fsc/README.md) adds a -stronger capability convention: `.f.mjs` is authored JavaScript that the -current FunctionalScript parser and compiler must accept, while `.f.ts` may -still use unsupported parser features or TypeScript syntax. The blocking P1 -implementation work required before the first real `.f.mjs` migration is -tracked in [`f-mjs-package-support.md`](./f-mjs-package-support.md). This P3 -document remains the broader package-publishing roadmap and records the shared -package-emission convention. +For FunctionalScript specifically, `.f.mjs` means authored +FunctionalScript-intent JavaScript during stage 1; it does **not** promise that +the current FunctionalScript parser/compiler accepts the whole module. The +focused P1 prerequisite before the first package-owned migration is +[`f-mjs-package-support.md`](./f-mjs-package-support.md). + +After all authored TypeScript is removed, the TypeScript-to-JavaScript emit path +is removed and the blanket `**/*.js` ignore is removed. Stage 2 can then use +`.f.js` as authored compiler-compatible FunctionalScript. Before the first +`.f.mjs` -> `.f.js` rename, complete +[`f-js-package-support.md`](./f-js-package-support.md) so standalone authored +`.f.js` source is directly checked, gets `.d.ts`, is packed, and resolves for a +clean consumer. -The main `tsconfig.json` should validate authored TypeScript and JavaScript while excluding generated declarations: +### Stage-1 TypeScript configuration + +Before the first `.ts` -> `.mjs` conversion, the main `tsconfig.json` should +validate both authored TypeScript and JavaScript by enabling: ```jsonc { "compilerOptions": { "allowJs": true, - "checkJs": true, - "noEmit": true, - "declaration": true - }, - "include": [ - "**/*.ts", - "**/*.mjs" - ], - "exclude": [ - "target", - "**/*.d.ts", - "**/*.d.mts" - ] + "checkJs": true + } } ``` -`**/*.d.ts` must be excluded because it also matches `**/*.ts`. +Enabling `checkJs` includes `fjs/types/bigint/benchmark.mjs`; keep it checked +like any other authored JavaScript. Its eventual removal is independent cleanup +and is not a prerequisite for the migration. -### Existing benchmark +NPM must include the stage-1 runtime and declaration extensions. It is not +necessary to special-case incidental non-public authored `.mjs` files: for +example, packing `benchmark.mjs` is harmless because it exposes no documented +public API. Such files can be removed later when no longer useful. -Enabling `checkJs` includes `fjs/types/bigint/benchmark.mjs`. Before enabling it, either make the benchmark pass TypeScript validation or delete the benchmark if it is no longer needed. The benchmark should not be excluded from `tsconfig.json`. +### Stage-1 emission -NPM must include both runtime extensions and both declaration extensions. Non-package `.mjs` files must remain excluded from the packed archive. - -Publishing requires a repository-owned `clean:generated` script followed by two TypeScript emission passes: +Keep packaging simple and keep emission as an implementation detail of the NPM +lifecycle. Use the two ordered TypeScript passes directly in `prepack` while any +`.ts` / `.f.ts` source remains: ```json { "scripts": { - "emit:declarations": "tsc --noEmit false --emitDeclarationOnly", - "emit:typescript": "tsc --noEmit false --allowJs false --checkJs false --declaration false", - "prepack": "npm run clean:generated && npm run emit:declarations && npm run emit:typescript" + "prepack": "tsc --noEmit false --emitDeclarationOnly && tsc --noEmit false --declaration false" } } ``` -The repository-owned cleanup derives output paths from authored `.ts` and -`.mjs` inputs and removes only their generated `.js`, `.d.ts`, and `.d.mts` -artifacts. It must preserve authored `.mjs` and unrelated files and must not use -a broad working-tree cleanup. This makes repeated local `prepack` and `npm pack` -runs independent of ignored outputs left by an earlier run. The exact script -location and implementation are left to the focused P1 task. +Do not expose separate `emit:declarations` or `emit:typescript` scripts. Users +should not need to invoke individual package-emission phases during normal +development. -The first emission pass emits declarations for both authored source extensions: +The first pass emits declarations for both authored extensions: ```text source.ts -> source.d.ts source.mjs -> source.d.mts ``` -The second emission pass excludes JavaScript inputs and emits JavaScript only from TypeScript: +The generated declarations are then present for the second invocation. For the +repository configuration, TypeScript resolves those `.d.mts` declarations for +the authored `.mjs` modules, so the second pass emits the remaining TypeScript +runtime JavaScript without trying to overwrite the authored `.mjs` files: ```text source.ts -> source.js ``` -Changing a module from `.ts` to `.mjs` also changes its import extension. Importers must be updated from the TypeScript or generated `.js` path to the authored `.mjs` path. +This exact setup is validated by +[PR #1451](https://github.com/functionalscript/functionalscript/pull/1451), +which enables `allowJs` / `checkJs`, keeps an authored `benchmark.mjs`, uses this +`prepack`, and passes the Node 26 CI `npm pack` step. A separate runtime-emission +configuration is therefore unnecessary unless a real repository case shows the +simple two-pass command is insufficient. + +No generated-output cleanup is needed before packaging because the CI package +job starts from a clean checkout. In particular, after `source.ts` is renamed +to `source.mjs`, an ignored `source.js` / `source.d.ts` from a developer's older +working tree cannot appear in the CI package job. + +Authored `.mjs` is copied without rewriting runtime imports, and emitted +`.d.mts` specifiers are not rewritten. Stage-1 migration is therefore +asymmetric and dependency-first: authored `.ts` may import already migrated +`.mjs`, while authored `.mjs` must not retain relative runtime or declaration +references to remaining `.ts` or generated `.js`. + +As soon as the last authored `.ts` / `.f.ts` source is removed, the runtime +JavaScript emission pass has no purpose and should be removed from +`package.json`. `prepack` then becomes declaration-only: + +```json +{ + "scripts": { + "prepack": "tsc --noEmit false --emitDeclarationOnly" + } +} +``` + +### Stage-2 authored `.f.js` + +Once stage 1 is complete, `.js` is no longer generated from repository +TypeScript and becomes authorable again. The stage-2 package invariant for a +compiler-compatible FunctionalScript module is: -Authored `.mjs` is copied to the package without rewriting runtime import -specifiers, and this plan does not rewrite module specifiers retained in emitted -`.d.mts` declarations. Therefore migration must be dependency-closed in both -graphs: an authored `.mjs` module must not reference an unmigrated relative -`.ts` source or generated `.js` sibling through executable imports, JSDoc type -imports, or any other declaration-retained reference. Its relative runtime and -type dependencies must already be authored `.mjs` or be converted in the same -coherent group. +```text +source.f.js -> source.f.js + source.f.d.ts +``` -An authored `.ts` module may import `.mjs`; its generated `.js` and `.d.ts` -outputs preserve that `.mjs` specifier, which works both in a checkout and in -the packed artifact. No staging tree, package-time runtime-import rewrite, or -declaration-specifier rewrite is planned. +TypeScript with `allowJs` / `checkJs` must include authored `.f.js` directly in +its checked source roots and declaration emission. NPM and clean-consumer tests +must cover both runtime and declarations. These requirements are owned by +[`f-js-package-support.md`](./f-js-package-support.md). ### Tasks -- [ ] Complete the blocking P1 authored-`.f.mjs` package work in - [`f-mjs-package-support.md`](./f-mjs-package-support.md), including the - repeatable cleanup-and-emission sequence and consecutive-pack regression. +- [ ] Complete [`f-mjs-package-support.md`](./f-mjs-package-support.md) before + the first stage-1 source conversion. +- [ ] After the last `.ts` / `.f.ts` source is removed, simplify `prepack` to the + declaration-only `tsc --noEmit false --emitDeclarationOnly` command. +- [ ] Complete [`f-js-package-support.md`](./f-js-package-support.md) after + stage 1 and before the first authored `.f.js` compiler-compatibility + conversion. ### Related -- [`f-mjs-package-support.md`](./f-mjs-package-support.md) — focused P1 package - prerequisite for the first real `.f.mjs` migration. -- [`fjs/fsc/README.md`](../../fsc/README.md) — FunctionalScript source - extensions and incremental repository migration. +- [PR #1451](https://github.com/functionalscript/functionalscript/pull/1451) — + initial implementation and CI validation of authored `.mjs` package support. +- [`todo/migrate-typescript-to-mjs.md`](../../../todo/migrate-typescript-to-mjs.md) + — repository-wide two-stage ordering. +- [`f-mjs-package-support.md`](./f-mjs-package-support.md) — focused stage-1 + authored `.mjs` prerequisite. +- [`f-js-package-support.md`](./f-js-package-support.md) — focused stage-2 + authored `.f.js` prerequisite. +- [`fjs/fsc/README.md`](../../fsc/README.md) — authoritative FunctionalScript + extension and migration contract. - [GitHub issue #398](https://github.com/functionalscript/functionalscript/issues/398) - — the original report. + — the original package report. diff --git a/fjs/emergent_testing/todo/f-mjs-test-and-coverage.md b/fjs/emergent_testing/todo/f-mjs-test-and-coverage.md index d60c0912a5..6b69e60512 100644 --- a/fjs/emergent_testing/todo/f-mjs-test-and-coverage.md +++ b/fjs/emergent_testing/todo/f-mjs-test-and-coverage.md @@ -8,91 +8,92 @@ Tooling recognition of `.f.mjs` has landed: `shouldLoad` in `fjs/dev/module.f.ts` matches `.f.mjs`, `npm run cov` and `deno task cov` -include `module.f.mjs`, the canonical Deno CI generator -(`fjs/ci/deno/module.f.ts`) exports `coverageInclude` with a regression proof, -the checked-in workflow is regenerated from it, and `AGENTS.md` §3.2 plus the -`CONTRIBUTING.md` summary state the proof policy for both authored extensions -including the mixed `module.f.mjs` / `proof.f.ts` layout. - -What is still missing is the end-to-end evidence: no `.f.mjs` file exists in the -repository, so nothing yet proves at runtime that a migrated module keeps its -proofs and its coverage rows. The current proofs cover discovery and the -generated command; they do not cover an actual loaded `.f.mjs` module. - -Adding those fixtures is not just a matter of writing two files. Two concrete -obstacles were found while implementing the tooling half: - -1. **A `proof.f.ts` cannot import a `module.f.mjs` today.** With `allowJs` - off, `npx tsc` reports `TS7016: Could not find a declaration file for module - './module.f.mjs'`. Turning `allowJs`/`checkJs` on makes `npx tsc` pass, but - then `npm run prepack` (`tsc --NoEmit false`) fails with - `TS5055: Cannot write file '…/benchmark.mjs' because it would overwrite input - file` — authored `.mjs` becomes both an input and a JavaScript emit target. - Making that work is exactly the repeatable-emission and package-content work - owned by - [`f-mjs-package-support.md`](../../ci/todo/f-mjs-package-support.md); it must - not be duplicated here. -2. **An internal `proof` export inside a `.f.mjs` module has no assert helper - it is allowed to import.** `AGENTS.md` §3.3 requires `assert`/`assertEq` - from `fjs/asserts/module.f.ts` rather than a hand-written `if`/`throw`, but - the dependency-closure rule forbids authored `.f.mjs` from importing a - relative `.f.ts` module. A fixture written with `if`/`throw` would also leave - a permanently-uncovered branch in a `module.f.mjs` that the new coverage - filter now includes. +include `module.f.mjs`, and the canonical Deno CI generator +(`fjs/ci/deno/module.f.ts`) exports `coverageInclude` with a regression proof. + +What is still missing is end-to-end evidence from an actual `.f.mjs` runtime +fixture. No repository fixture currently proves that a migrated +`module.f.mjs` is loaded through its proof and retained in both Node and Deno +coverage output. + +The fixture depends on authored `.mjs` package support because a TypeScript proof +must be able to import `module.f.mjs` while `allowJs` / `checkJs` remain enabled, +and package emission must preserve that authored `.mjs` rather than treating it +as generated JavaScript. + +This task does **not** require the first real repository module migration or a +compiler-ready `proof.f.mjs`. Stage 1 is independent of FunctionalScript compiler +coverage. A mixed synthetic fixture using `module.f.mjs` + `proof.f.ts` is enough +to prove the first source-migration layout end to end. + +`proof.f.mjs` is nevertheless an allowed Stage-1 source extension. A real proof +may migrate from `proof.f.ts` to `proof.f.mjs` as soon as it can be expressed as +JavaScript with JSDoc and its authored runtime and declaration-retained type +dependencies are already `.f.mjs`; current FunctionalScript compiler support is +not a migration gate. A proof that still depends on an unmigrated `.f.ts` helper, +such as `fjs/asserts/module.f.ts`, remains `proof.f.ts` until that dependency +moves. ### Proposal -Land the fixtures once package support makes authored `.f.mjs` a first-class, -type-checked, packable source extension. +After [`f-mjs-package-support.md`](../../ci/todo/f-mjs-package-support.md) +completes, add the smallest synthetic `.f.mjs` runtime fixture that proves the +mixed Stage-1 layout: -Decide obstacle 2 explicitly before writing the fixture, and record the decision -in `AGENTS.md` §3.3. The options are: +```text +module.f.mjs +proof.f.ts +``` -- migrate `fjs/asserts/module.f.ts` to `fjs/asserts/module.f.mjs` as the first - real conversion, so every `.f.mjs` proof — fixture or not — has a compliant - assert helper (`.f.ts` callers may keep importing it under the asymmetric - import policy); or -- state that a `.f.mjs` module keeps its proofs in a co-located `proof.f.ts` - until `fjs/asserts` migrates, and drop the internal-`proof`-in-`.f.mjs` - fixture in favour of the mixed-layout one. +The fixture should be outside the published runtime API and should exercise the +normal test discovery and coverage commands. It exists only to prove the tooling +boundary before the first real repository `.f.ts` -> `.f.mjs` conversion. -The first option is preferable: it is a real migration step the plan needs -anyway, and it removes the fixture's special case instead of documenting one. +Keep proof-extension migration separate from compiler readiness. Update +`AGENTS.md` and `CONTRIBUTING.md` so `proof.f.mjs` is explicitly allowed during +Stage 1 whenever its JavaScript/JSDoc and dependency closure is ready. + +A dedicated `proof.f.mjs` fixture may be added when useful, but it is not a +prerequisite for the first real module conversion and must not create a circular +dependency on migrating assertion helpers first. ### Tasks -- [ ] Decide and document how an internal `proof` inside a `.f.mjs` module - asserts (see the two options above), updating `AGENTS.md` §3.3. -- [ ] Add a fixture proving that an internal `proof` export from a `.f.mjs` - module is executed by the normal test command. -- [ ] Add a fixture proving that a co-located `proof.f.ts` can import and test - `module.f.mjs`. -- [ ] Verify that `.f.mjs` implementations appear in both Node and Deno coverage - output for the supported proof layouts. +- [ ] Add a synthetic `module.f.mjs` fixture with a co-located `proof.f.ts` that + imports and tests it through the normal test command. +- [ ] Verify the fixture type-checks under `npx tsc` with the Stage-1 + `allowJs` / `checkJs` configuration. +- [ ] Verify the `.f.mjs` implementation appears in both Node and Deno coverage + output. +- [ ] Update `AGENTS.md` and `CONTRIBUTING.md` so `proof.f.mjs` migration is + gated by JavaScript/JSDoc plus dependency readiness, not compiler support. ### Acceptance criteria -- An internal exported `proof` from a `.f.mjs` module is executed by the normal - test command. - A `proof.f.ts` importing `module.f.mjs` is executed by the normal test command and type-checks under `npx tsc`. - The `.f.mjs` fixture appears as a covered file in `npm run cov` and in `deno task cov`. +- `proof.f.mjs` is explicitly allowed during Stage 1 when its authored + dependencies are already migrated and the proof is valid JavaScript/JSDoc. - Existing `.f.ts`, generated `.f.js`, `proof.f.ts`, and standalone `proof.mjs` behavior is unchanged. ### Ordering -Complete this task before converting the first repository module from `.f.ts` to -`.f.mjs`, and after -[`f-mjs-package-support.md`](../../ci/todo/f-mjs-package-support.md). The -tooling half already shipped, so a synthetic compiler fixture that does not -enter the published runtime graph is not blocked by this issue. +This task is **blocked by** +[`f-mjs-package-support.md`](../../ci/todo/f-mjs-package-support.md) and must +complete before converting the first real repository module from `.f.ts` to +`.f.mjs`. The synthetic fixture itself is the prerequisite evidence and does not +count as a production/source migration. A synthetic compiler fixture that does +not enter the published runtime graph is likewise not blocked by this issue. ### Related - [`fjs/fsc/README.md`](../../fsc/README.md) — source-extension convention and - incremental repository migration. + two-stage repository migration. +- [`../../../todo/migrate-typescript-to-mjs.md`](../../../todo/migrate-typescript-to-mjs.md) + — repository-wide Stage-1 migration that is blocked by this fixture task. - [`664-emergent-testing-module-files.md`](./664-emergent-testing-module-files.md) — separate proposal to bulk-load ordinary `module.*` files for white-box testing. Ordinary `.mjs` files stay opt-in through the `proof.mjs` convention diff --git a/fjs/fsc/README.md b/fjs/fsc/README.md index c392e1a109..cc21bd82c8 100644 --- a/fjs/fsc/README.md +++ b/fjs/fsc/README.md @@ -1,85 +1,91 @@ # FunctionalScript Compiler -## Source files and incremental repository migration +## Source files and repository migration -The FunctionalScript repository uses file extensions to distinguish authored -source, generated output, and the subset accepted by the current -FunctionalScript compiler. +The FunctionalScript repository uses extensions to separate source-language +migration from compatibility with the current FunctionalScript compiler. | Extension | Meaning | |---|---| -| `.f.ts` | Authored FunctionalScript-intent TypeScript. It may use TypeScript syntax or FunctionalScript features that the current parser does not support yet. | -| `.f.mjs` | Authored FunctionalScript ESM JavaScript with JSDoc types. The complete module must be accepted by the current FunctionalScript parser and compiler. | -| `.f.js` | Generated JavaScript emitted from `.f.ts`; never authored directly. | +| `.f.ts` | Authored FunctionalScript-intent TypeScript that has not yet completed the repository TypeScript-to-JavaScript migration. | +| `.f.mjs` | Authored FunctionalScript-intent ESM JavaScript with JSDoc types. It may use FunctionalScript features the current parser/compiler does not support yet. | +| `.f.js` | During stage 1, generated JavaScript emitted from `.f.ts` and never authored. After stage 1 and authored-`.f.js` package support are complete, authored FunctionalScript that the current parser/compiler must accept. | | `.d.ts`, `.d.mts` | Generated TypeScript declarations. | -The general authored/generated JavaScript convention is described in +The migration is deliberately split into two stages. The repository-wide plan is +[`todo/migrate-typescript-to-mjs.md`](../../todo/migrate-typescript-to-mjs.md), +and the package conventions are documented in [`fjs/ci/todo/publishing-packages.md`](../ci/todo/publishing-packages.md). -The `.f.mjs` extension adds a stronger FunctionalScript-specific promise: it -marks a module that is ready for translation by the compiler available in the -same revision of the repository. - -Repository migration is incremental, not a single task or pull request. A -synthetic `.f.mjs` compiler fixture may be added as soon as the parser supports -it. Before converting the first existing repository module, complete both: - -- [authored `.f.mjs` package support](../ci/todo/f-mjs-package-support.md), - including TypeScript checking, `.mjs`/`.d.mts` package inclusion, repeatable - cleanup and declaration emission, consecutive-pack validation, runtime import - tests, packed-package type-resolution tests, and the repository module-import - policy update, so published runtime and declaration imports cannot reference - omitted files and unmigrated callers may follow renamed dependencies; -- the remaining - [`.f.mjs` test and coverage fixtures](../emergent_testing/todo/f-mjs-test-and-coverage.md), - which prove end to end that a migrated module keeps its proofs and its - coverage rows. The tooling half already ships: proof discovery - (`shouldLoad`), `npm run cov`, `deno task cov`, the generated Deno CI step, - and the proof policy in `AGENTS.md` and `CONTRIBUTING.md` all cover both - authored extensions today. - -The repository import policy is asymmetric during migration: authored `.f.ts` -may import relative `.f.ts` or `.f.mjs` modules, while authored `.f.mjs` runtime -imports and type references may target relative `.f.mjs` modules only. Update -`AGENTS.md` with this rule as part of the package-support prerequisite before the -first real rename. - -Migration uses a dependency-closed order. An existing module is eligible only -when every relative FunctionalScript dependency referenced by its runtime code -or retained in its emitted `.d.mts` declaration is already `.f.mjs` or is -converted in the same coherent group. Authored `.f.mjs` runtime imports and -JSDoc type references must not point to an unmigrated `.f.ts` module or generated -`.f.js` output. Packaging copies `.mjs` source and emits declarations without a -specifier-rewrite step, so the runtime and declaration graphs must both resolve -in a clean checkout and in the packed NPM artifact. If either dependency closure -is not yet eligible, leave the module as `.f.ts`; this plan does not introduce a -staging, package-time import-rewrite, or declaration-rewrite mechanism. - -1. As soon as the parser supports the first useful function modules, select an - existing dependency-closed `.f.ts` module or coherent group whose complete - syntax, runtime dependencies, and declaration-retained type dependencies are - supported. -2. Rename the selected files to `.f.mjs` and replace TypeScript-only syntax with - JSDoc types. -3. Update runtime imports and JSDoc type references within the group, plus all - `.f.ts` importers of renamed modules, to the authored `.f.mjs` paths. -4. Add the group to parser/compiler validation and preserve its existing proof, - coverage, type-checking, repeatable package-runtime, and package-type-resolution - expectations. -5. Repeat as each new parser feature makes more dependency-closed groups - eligible. - -A file stays `.f.ts` until all syntax it uses and both required dependency -closures are supported. Migration must not require implementing unrelated -language features merely to convert a file. Likewise, `.f.mjs` must not be used -as an aspirational label: once a module has that extension, accepting and -compiling it, resolving its runtime imports, and resolving its emitted types are -compatibility requirements. - -The migration grows real-repository compiler coverage alongside the parser and -code generator. It does not wait for the complete FunctionalScript feature set, -and compiler progress does not wait for the entire repository to migrate. See -the [project roadmap](../../todo/plan/roadmap.md) and the -[fjs–nanvm integration plan](../../todo/fjs-nanvm-integration.md). + +### Stage 1: remove authored TypeScript + +Before the first real repository source conversion, complete both prerequisites +in order: + +1. [authored `.mjs` package support](../ci/todo/f-mjs-package-support.md), + including `allowJs` / `checkJs`, split declaration/runtime emission, package + inclusion, and clean-consumer tests; +2. [`.f.mjs` test and coverage fixtures](../emergent_testing/todo/f-mjs-test-and-coverage.md), + which are **blocked by** package support and prove with an actual `.f.mjs` + runtime fixture that proofs execute and Node/Deno coverage retains the + migrated module. + +Package and publish jobs run from a clean CI checkout, so the package prerequisite +does not require developer-worktree cleanup or tracking ignored outputs from +earlier revisions. + +Then migrate dependency leaves first: + +```text +module.ts -> module.mjs +module.f.ts -> module.f.mjs +``` + +This stage is independent of FunctionalScript parser coverage. `.f.mjs` means +FunctionalScript-intent JavaScript; it is not a compiler-compatibility promise. +A `.f.ts` module should move once its authored TypeScript runtime and +declaration-retained type dependencies can move, even if the current compiler +cannot parse every feature it uses. + +The transition is asymmetric: remaining `.f.ts` may depend on already migrated +`.f.mjs`, while migrated `.f.mjs` must not depend on remaining `.f.ts`. Cycles +may migrate as a coherent group. Packaging does not rewrite runtime or +declaration specifiers, so both dependency graphs must resolve directly. + +Proofs follow the same source-language rule. `proof.f.ts` may remain temporarily +beside a migrated `module.f.mjs`, but it may move to `proof.f.mjs` as soon as the +proof itself is valid JavaScript with JSDoc and its authored runtime and type +dependencies are already `.f.mjs`. Current FunctionalScript compiler support is +not a condition for that rename. + +When the last authored `.ts` / `.f.ts` file is gone, remove the +TypeScript-to-JavaScript emit path, remove obsolete generated `.js` from the +working tree for that transition, and remove the blanket `**/*.js` rule from +`.gitignore`. Only then is `.js` available as an authored extension. + +### Stage 2: mark compiler-compatible FunctionalScript + +The repository compiler-compatibility migration in +[`todo/fjs-nanvm-integration.md`](../../todo/fjs-nanvm-integration.md) is +**blocked by** stage 1. Before its first rename, also complete +[authored `.f.js` package support](../ci/todo/f-js-package-support.md), so a +standalone `.f.js` is directly type-checked, receives a `.d.ts`, is packed in +the clean CI package build, and resolves for a clean consumer. + +Then migrate compiler-supported dependency-closed groups incrementally: + +```text +module.f.mjs -> module.f.js +``` + +An authored `.f.js` is a compatibility commitment: the FunctionalScript parser +and compiler in the same repository revision must accept the complete module, +and its runtime and declaration dependencies must satisfy the compiler migration +rules. Unsupported modules remain `.f.mjs` until the required compiler features +land. + +A synthetic JavaScript compiler fixture may be used before repository migration; +it does not change the extension contract for repository source. ## Tokenizer diff --git a/nanvm-lib/todo/mvp-roadmap.md b/nanvm-lib/todo/mvp-roadmap.md index a626b563ee..8c956aa9ad 100644 --- a/nanvm-lib/todo/mvp-roadmap.md +++ b/nanvm-lib/todo/mvp-roadmap.md @@ -100,18 +100,22 @@ testing, self-hosting, and AOT embedding. #### Effects: the `nanvm-effects-node` runner crate (decided) The compiler CLI is pure FJS that *returns* effect descriptions -(`Effect`); all actual impurity lives in thin `.ts` runner -modules (e.g. [`fjs/effects/node/module.ts`](../../fjs/effects/node/module.ts)), -which are not FJS and never pass through the code generator. During compiler -bootstrap, `.f.mjs` is the compiled source marker: every `.f.mjs` module must -compile to Rust, while every impure `.ts` runner needs a hand-written Rust twin -interpreting the same operation vocabulary against the OS (`std::fs`, -`std::process`, stdio) instead of Node built-ins. Existing `.f.ts` modules are -the broader FunctionalScript-intent source set; they move to `.f.mjs` -incrementally when the parser supports their complete syntax and their -TypeScript types have been moved to JSDoc. The extension contract and migration +(`Effect`); all actual impurity lives in thin runner modules +(e.g. [`fjs/effects/node/module.ts`](../../fjs/effects/node/module.ts)), +which are not FJS and never pass through the code generator. Stage 1 of the +repository migration moves authored `.ts` / `.f.ts` to `.mjs` / `.f.mjs` with +JSDoc independently of compiler support, so `.f.mjs` is **not** the compiled +source marker. After Stage 1 and authored `.f.js` package support are complete, +compiler-supported `.f.mjs` modules may move to authored `.f.js`; `.f.js` is the +repository compiler-compatibility marker. The extension contract and migration strategy are documented in [`fjs/fsc/README.md`](../../fjs/fsc/README.md). +Impure runner modules remain outside the FJS compiler regardless of whether their +authored JavaScript extension is `.ts` during migration or `.mjs` afterward. A +native build therefore still needs a hand-written Rust twin interpreting the +same operation vocabulary against the OS (`std::fs`, `std::process`, stdio) +instead of Node built-ins. + The Rust twin of `fjs/effects/node` is the **`nanvm-effects-node`** library crate — named to mirror the effect directory structure: this specific effect set is the CLI-on-Node vocabulary, and future sets (e.g. a browser @@ -154,10 +158,12 @@ Scoping notes: - **Testing comes cheap.** The pure in-memory interpreters ([`fjs/effects/mock`](../../fjs/effects/mock), [`fjs/effects/node/virtual`](../../fjs/effects/node/virtual)) are currently - `.f.ts` code. As the parser reaches the syntax each module uses, migrate it - to `.f.mjs`; from that point it compiles through the code generator unchanged, - so the compiled CLI can run against in-memory effects with no Rust twins. - The `nanvm-effects-node` runner can then be cross-checked against the pure + `.f.ts` code. Stage 1 migrates them to `.f.mjs` when their JavaScript/JSDoc + and dependency closure are ready, independently of parser support. Once the + compiler supports their complete syntax in Stage 2, rename them to `.f.js`; + from that point they compile through the code generator unchanged, so the + compiled CLI can run against in-memory effects with no Rust twins. The + `nanvm-effects-node` runner can then be cross-checked against the pure interpreter operation by operation. The exception is [`fjs/effects/node/memory`](../../fjs/effects/node/memory) — the runner for the mutable memory effects (`MemOp`) — which is an impure `.ts` module @@ -274,9 +280,11 @@ as a generic `Any` facility, post-MVP. - [ ] **Harness + walking skeleton** — a harness crate (or generated tests in `nanvm-lib`) whose `main` evaluates a generated module's `export default` and prints the result as JSON; wire the pipeline - end-to-end early with a minimal `.f.mjs` subset (e.g. a constant default - export), driven by `cargo test` in CI, so every later feature lands - into a working pipeline. See + end-to-end early with a minimal synthetic FunctionalScript JavaScript + fixture (e.g. a constant default export), driven by `cargo test` in CI, + so every later feature lands into a working pipeline. This synthetic + fixture may use `.f.mjs`; it does not define the repository extension + contract. See [fjs-nanvm-integration](../../todo/fjs-nanvm-integration.md). - [ ] **Test generation for operators** — one test-data module drives both the FJS proof (JS engine reference) and the generated Rust tests. @@ -290,11 +298,11 @@ as a generic `Any` facility, post-MVP. Current status: [operator tables in `nanvm-lib/README.md`](../README.md). Spec: [operators](../../todo/lang/2340-operators.md). - [ ] **Parser**, using [`fjs/bnf/`](../../fjs/bnf/README.md) (FJS). -- [ ] **Incremental repository coverage** — after the parser supports the first - useful function modules, convert the first eligible `.f.ts` module to - `.f.mjs`, move its types to JSDoc, and keep it in the end-to-end compiler - test set. Continue in separate changes as language coverage grows; do not - make whole-repository migration an MVP gate. +- [ ] **Incremental repository compiler coverage** — this is not an MVP gate. + First complete the repository TypeScript-to-JavaScript Stage 1 and authored + `.f.js` package support. Then, as compiler coverage grows, rename eligible + dependency-closed `.f.mjs` modules to `.f.js` and keep them in the + end-to-end compiler test set. Unsupported modules remain `.f.mjs`. #### P2 @@ -336,16 +344,16 @@ as a generic `Any` facility, post-MVP. Compile the compiler itself (written in FJS) to Rust with the code generator and ship it as the `nanvm` crate ([console-program](./console-program.md)): a single native executable that -parses and runs `.f.mjs` directly — no Node/Deno, no rustc at the user's run -time — executing code via the interpreter behind the `Function` constructor, -with its I/O interpreted by the `nanvm-effects-node` runner (see the effects -section above). - -Reached incrementally: the code generator's language coverage grows with the -operator/function/control tasks above until it covers everything the -compiler's own code uses. The corresponding compiler modules move from -`.f.ts` to `.f.mjs` as they become supported, so self-hosting is the completion -of the same repository-migration strategy rather than a separate rewrite. +parses and runs FunctionalScript JavaScript directly — no Node/Deno, no rustc at +the user's run time — executing code via the interpreter behind the `Function` +constructor, with its I/O interpreted by the `nanvm-effects-node` runner (see the +effects section above). + +Reached incrementally: Stage 1 first removes authored TypeScript from the +compiler source into `.f.mjs` independently of parser coverage. As the code +generator's language coverage grows, compiler-supported modules move from +`.f.mjs` to `.f.js`; self-hosting is the completion of that same Stage-2 +compiler-compatibility migration rather than a separate rewrite. ### Open questions diff --git a/todo/fjs-nanvm-integration.md b/todo/fjs-nanvm-integration.md index feffa743dd..c092a9b0c0 100644 --- a/todo/fjs-nanvm-integration.md +++ b/todo/fjs-nanvm-integration.md @@ -22,44 +22,54 @@ prove much. The entry point is the module's `export default`: the harness evaluates it, runs it if it is a function, and prints the result to stdout as JSON. -### Repository source selection - -The first integration does not imply that every existing `.f.ts` module is -accepted by the new parser. `.f.ts` is the broader authored -FunctionalScript-intent source set and may contain TypeScript syntax or -FunctionalScript features that are not implemented yet. - -Use `.f.mjs` for authored modules whose complete syntax is accepted by the -current parser and compiler. Types in these modules are expressed with JSDoc, -not TypeScript syntax. The initial walking skeleton may start with a minimal -synthetic `.f.mjs` fixture before repository-wide `.mjs` infrastructure is -complete, because that fixture does not enter the published runtime graph. - -Before converting the first existing repository module, complete both: - -- [`.f.mjs` test and coverage support](../fjs/emergent_testing/todo/f-mjs-test-and-coverage.md), - so the rename cannot silently remove internal proofs or coverage and the - proof guidance in `AGENTS.md` and `CONTRIBUTING.md` remains consistent; -- [authored `.f.mjs` package support](../fjs/ci/todo/f-mjs-package-support.md), - including TypeScript checking, package inclusion for `.mjs` and `.d.mts`, - repeatable generated-output cleanup and declaration emission, consecutive-pack - validation, runtime import tests, and packed-package type-resolution tests. - -After those prerequisites, select a dependency-closed module or coherent group. -Every relative FunctionalScript dependency used by executable code or retained -in emitted `.d.mts` declarations must already be `.f.mjs` or be converted in the -same change. Authored `.f.mjs` runtime imports and JSDoc type references must not -point to unmigrated `.f.ts` or generated `.f.js`; package emission does not -rewrite authored `.mjs` imports or declaration specifiers. Rename the group, -update its runtime and type references plus all importers, and use it as an -end-to-end compiler input. If either required dependency closure is not yet -supported, postpone that group and choose a smaller eligible leaf. - -Repository migration then continues independently, one dependency-closed group -at a time, as parser features land. It is neither part of one large PR nor a -gate on the initial synthetic walking skeleton. See -[`fjs/fsc/README.md`](../fjs/fsc/README.md) for the extension contract and -migration strategy. +### Repository compiler-compatibility migration + +**Blocked by:** + +- [migrate authored TypeScript to `.mjs`](./migrate-typescript-to-mjs.md) +- [package support for authored `.f.js`](../fjs/ci/todo/f-js-package-support.md) + +The initial compiler walking skeleton does not require repository source +migration and may use a small synthetic JavaScript fixture. The extension-based +compiler-compatibility migration of existing repository modules is separate and +cannot begin while authored TypeScript remains. + +Stage 1 first converts the repository gradually and dependency-first: + +```text +module.ts -> module.mjs +module.f.ts -> module.f.mjs +``` + +During stage 1, `.f.mjs` means authored FunctionalScript-intent JavaScript; it +does not promise current compiler support. The stage also removes the +TypeScript-to-JavaScript emit path after the last TypeScript source is gone, +cleans obsolete generated `.js`, and removes the blanket `**/*.js` ignore so +`.js` becomes authorable and trackable again. + +Before stage 2 renames any repository source, complete the focused +[`f-js-package-support.md`](../fjs/ci/todo/f-js-package-support.md) prerequisite. +A standalone authored `.f.js` must be directly included in TypeScript checking, +receive a generated `.d.ts`, be included in the packed NPM artifact, and work +from a clean consumer. Stage-2 package and publish validation runs from a clean +CI checkout, so generated-output cleanup or repeated-pack safety is not part of +this prerequisite. Compiler acceptance alone is not a sufficient rename gate. + +Only then does the repository compiler-compatibility migration use: + +```text +module.f.mjs -> module.f.js +``` + +An authored `.f.js` must be accepted by the FunctionalScript parser and compiler +in the same repository revision. Migration remains incremental: select a +compiler-supported dependency-closed `.f.mjs` module or coherent group, rename +it to `.f.js`, update runtime and type references plus callers, and keep it as a +permanent end-to-end compiler regression input. Unsupported FunctionalScript +modules remain `.f.mjs` until the required compiler features land. + +See [`fjs/fsc/README.md`](../fjs/fsc/README.md) for the authoritative extension +contract and migration strategy. ### CLI: an output target, not a command group (decided) @@ -91,33 +101,37 @@ via the `Function` constructor — no rustc at the user's run time. - [ ] Define the convention for generated module imports (`use` paths, file/directory layout — see the open question in [mvp-roadmap](../nanvm-lib/todo/mvp-roadmap.md#open-questions)). -- [ ] Prove the pipeline with a minimal synthetic `.f.mjs` subset: a constant - default export compiled by `fjs` to `.rs`, built and run by cargo, with - the result printed to stdout as JSON. -- [ ] Complete - [`.f.mjs` test and coverage support](../fjs/emergent_testing/todo/f-mjs-test-and-coverage.md). +- [ ] Prove the pipeline with a minimal synthetic JavaScript FunctionalScript + subset: a constant default export compiled by `fjs` to `.rs`, built and + run by cargo, with the result printed to stdout as JSON. +- [ ] Complete [migrate authored TypeScript to `.mjs`](./migrate-typescript-to-mjs.md), + including removal of TypeScript JavaScript emission and the blanket + `**/*.js` `.gitignore` rule. - [ ] Complete - [authored `.f.mjs` package support](../fjs/ci/todo/f-mjs-package-support.md), - including the consecutive-pack repeatability regression. -- [ ] Convert the first eligible repository module or group that is closed over - both runtime and declaration-retained type dependencies from `.f.ts` to - `.f.mjs`, and keep it in the end-to-end compiler, proof, coverage, - type-checking, repeatable package-runtime, and package-type-resolution test + [package support for authored `.f.js`](../fjs/ci/todo/f-js-package-support.md), + including direct type-checking, declaration emission, packing, and + clean-consumer runtime/type tests. +- [ ] Verify `.js` is trackable and authored `.f.js` is a first-class package + source before the first compiler-compatibility rename. +- [ ] Convert the first eligible dependency-closed repository module or group + from `.f.mjs` to `.f.js` and keep it in the end-to-end compiler, proof, + coverage, type-checking, package-runtime, and package-type-resolution test sets. +- [ ] Continue `.f.mjs` -> `.f.js` incrementally as compiler support grows. ### Related +- [migrate authored TypeScript to `.mjs`](./migrate-typescript-to-mjs.md) — + **blocked-by prerequisite** for repository compiler-compatibility migration. +- [package support for authored `.f.js`](../fjs/ci/todo/f-js-package-support.md) + — **blocked-by prerequisite** before the first stage-2 rename. - [nanvm-lib/todo/mvp-roadmap.md](../nanvm-lib/todo/mvp-roadmap.md) — MVP definition and task list. - [nanvm-lib/todo/console-program.md](../nanvm-lib/todo/console-program.md) — the self-hosted `nanvm` crate (post-MVP). -- [`.f.mjs` test and coverage support](../fjs/emergent_testing/todo/f-mjs-test-and-coverage.md) - — proof-discovery, cross-runner coverage, and contributor-policy prerequisite - for the first repository conversion. -- [authored `.f.mjs` package support](../fjs/ci/todo/f-mjs-package-support.md) — - focused P1 validation, repeatable emission, package-content, dependency, and - consumer type-resolution prerequisite. -- [`publishing-packages.md`](../fjs/ci/todo/publishing-packages.md) — broader P3 - package-publishing roadmap and shared authored/generated extension convention. +- [authored `.mjs` package support](../fjs/ci/todo/f-mjs-package-support.md) — + stage-1 validation, declaration, and package prerequisite. +- [`publishing-packages.md`](../fjs/ci/todo/publishing-packages.md) — broader + package-publishing roadmap. - [ast-spec](./ast-spec.md) — the schema of the code-describing `Any`; the `Function` constructor contract. diff --git a/todo/lang/README.md b/todo/lang/README.md index 77f7f9f790..70facc3b76 100644 --- a/todo/lang/README.md +++ b/todo/lang/README.md @@ -14,16 +14,23 @@ File Types: |File Type|Extension|Notes| |---------|---------|-----| |JSON|`.json`|Tree.| -|FJS source|`.f.ts`, `.f.mjs`|Graph with functions.| -|Generated FJS|`.f.js`|Generated from `.f.ts`; never authored directly.| +|FJS source|`.f.ts`, `.f.mjs`, `.f.js`|Graph with functions; extension meaning follows the repository migration stage below.| -`.f.ts` is authored FunctionalScript-intent TypeScript and may contain syntax -that the current FunctionalScript parser does not support yet. `.f.mjs` is -authored ESM JavaScript with JSDoc types whose complete syntax must be accepted -by the current parser and compiler. Repository modules move from `.f.ts` to -`.f.mjs` incrementally as language support grows; this is not a single migration -step. See [`fjs/fsc/README.md`](../../fjs/fsc/README.md) for the extension -contract and migration strategy. +Repository source migration has two separate stages: + +1. `.f.ts -> .f.mjs` removes authored TypeScript. `.f.mjs` is authored ESM + JavaScript with JSDoc types and does **not** imply that the current + FunctionalScript parser/compiler accepts the module. +2. After authored TypeScript is gone and authored `.f.js` package support is + complete, compiler-supported `.f.mjs` modules may move to authored `.f.js`. + `.f.js` is then the compiler-compatibility marker. + +During Stage 1, `.f.js` remains generated output from `.f.ts` and must not be +authored. Stage 1 is independent of parser coverage; Stage 2 grows incrementally +as compiler support grows. See [`fjs/fsc/README.md`](../../fjs/fsc/README.md) for +the authoritative extension contract and +[`todo/migrate-typescript-to-mjs.md`](../migrate-typescript-to-mjs.md) for the +repository migration plan. **Note**: An FJS value can't be serialized without additional run-time infrastructure. @@ -37,7 +44,6 @@ spellings are a deferred feature, see [js-string-literals](./2460-js-string-literals.md). **VM**: - We are introducing new commands in such a way that every new command depends only on previous commands. |format|any |Tag| | @@ -437,7 +443,6 @@ FJS value (`Any`) using the tag tables above. Code is data: the `Function` const **CBOR** ([RFC 8949](https://www.rfc-editor.org/rfc/rfc8949)), chosen because it represents numbers as exact IEEE 754 doubles, avoiding the ambiguous binary↔decimal number conversion of text formats. - There are two execution paths, observably identical except in performance: - **Interpretation** — the `Function` constructor executes the `Any` code description directly: diff --git a/todo/migrate-typescript-to-mjs.md b/todo/migrate-typescript-to-mjs.md new file mode 100644 index 0000000000..a3170155b7 --- /dev/null +++ b/todo/migrate-typescript-to-mjs.md @@ -0,0 +1,277 @@ +## Migrate authored TypeScript to `.mjs` + +**Priority:** P1 +**Status:** open + +### Problem + +FunctionalScript currently uses authored `.ts` / `.f.ts` source and generated +`.js` output. The compiler migration also used `.f.mjs` as a marker for modules +accepted by the current FunctionalScript compiler. Those two migrations should +not be coupled: removing TypeScript is a repository-wide source-language +migration, while compiler compatibility depends on the feature set implemented +by the FunctionalScript parser/compiler. + +Keeping unsupported modules as `.f.ts` until the FunctionalScript compiler can +parse them would unnecessarily block the TypeScript removal. It also prevents us +from eventually using `.f.js` as the simple compiler-compatibility marker, +because TypeScript currently generates `.f.js` from `.f.ts` and `.gitignore` +blanket-ignores `**/*.js`. + +The repository therefore needs two ordered stages: + +1. migrate all authored TypeScript to JavaScript with JSDoc, independently of + FunctionalScript compiler support; +2. after TypeScript is gone, migrate compiler-supported FunctionalScript modules + from `.f.mjs` to authored `.f.js`. + +The existing compiler-compatibility migration in +[`fjs-nanvm-integration.md`](./fjs-nanvm-integration.md) is **blocked by** this +stage-1 task. + +### Proposal + +#### Stage 1 extension meaning + +During this task: + +```text +module.ts -> module.mjs +module.f.ts -> module.f.mjs +``` + +- `.ts` / `.f.ts` are authored TypeScript that still remains to migrate; +- `.mjs` is authored ESM JavaScript with JSDoc types; +- `.f.mjs` is authored FunctionalScript-intent JavaScript with JSDoc types; +- `.f.mjs` does **not** promise that the current FunctionalScript compiler can + parse the module; +- `.js` remains generated output and must not be authored while any TypeScript + source remains; +- `.d.ts` / `.d.mts` remain generated declarations. + +The authoritative extension contract in [`../fjs/fsc/README.md`](../fjs/fsc/README.md) +and the package plans must use these meanings throughout stage 1. + +#### Enable JavaScript checking and `.f.mjs` validation first + +Before the first `.ts` / `.f.ts` source file moves to `.mjs` / `.f.mjs`, enable +`allowJs` and `checkJs` in the root `tsconfig.json`. TypeScript remains the +repository type checker during this migration; JSDoc replaces TypeScript syntax +without creating an unchecked intermediate source set. + +Stage 1 is **blocked by** both of these prerequisites before the first real +repository `.f.ts` -> `.f.mjs` conversion: + +- [`../fjs/ci/todo/f-mjs-package-support.md`](../fjs/ci/todo/f-mjs-package-support.md) + makes authored `.mjs` a checked, declaration-emitting, packable source + extension; +- [`../fjs/emergent_testing/todo/f-mjs-test-and-coverage.md`](../fjs/emergent_testing/todo/f-mjs-test-and-coverage.md) + is **blocked by** that package-support task and adds an actual `.f.mjs` + runtime fixture proving proof execution plus Node and Deno coverage. + +Package and publish jobs run only in CI from a clean checkout. The migration does +not need to preserve packability of arbitrary developer working trees or track +ignored generated outputs across source renames; a later CI package job starts +without those stale files. + +#### Migrate gradually from dependency leaves + +Stage 1 is incremental, not a repository-wide atomic rename. Start with authored +`.ts` / `.f.ts` files that do not depend on other authored TypeScript files, then +migrate their callers and continue upward through the dependency graph. + +A file or coherent group is eligible when every relative authored runtime +source dependency and every declaration-retained type dependency outside the +group is already JavaScript (`.mjs` / `.f.mjs`). Cycles may migrate as one +coherent group. + +The transition is intentionally asymmetric: + +- remaining `.ts` / `.f.ts` may depend on already migrated `.mjs` / `.f.mjs`; +- migrated `.mjs` / `.f.mjs` must not depend on remaining authored `.ts` / + `.f.ts`. + +FunctionalScript parser support is not an eligibility condition. A `.f.ts` file +may move to `.f.mjs` even if the current FunctionalScript compiler does not yet +support all syntax in that file. + +Proof files follow the same source-language rule. A migrated `module.f.mjs` may +keep its existing `proof.f.ts` temporarily, but `proof.f.mjs` is allowed as soon +as that proof can be expressed as JavaScript with JSDoc and every authored +runtime or declaration-retained type dependency outside its migration group is +already `.f.mjs`. Compiler support for the proof is not required. By the end of +stage 1, all remaining `proof.f.ts` files must therefore have migrated to +`proof.f.mjs` along with the rest of authored TypeScript. + +Preserve TypeScript type semantics when translating to JSDoc. TypeScript 7 +supports variance annotations on JSDoc type aliases through modifiers on +`@template`. For example: + +```ts +export type Cont = + (_: Pr[1]) => Effect +``` + +becomes: + +```js +/** + * @template {Operation} out O + * @template T + * @typedef {(_: Pr[1]) => Effect} Cont + */ +``` + +Use `@template out T`, `@template in T`, or constrained forms such as +`@template {Operation} out O`. Variance modifiers belong to a JSDoc type alias +(`@typedef`), not to an ordinary function's `@template`. + +#### Known TypeScript-to-JSDoc hard cases + +Do not require the migration plan to pre-design every TypeScript-only type +construct before Stage 1 starts. Instead, identify hard cases as they are found, +record them explicitly, and block only the affected migration group until its +focused design is resolved. Unrelated dependency leaves should continue to +migrate. + +One known case is `fjs/types/phantom/module.f.ts`, whose public `Phantom` type +uses a type-only `declare const phantomKey: unique symbol`. `declare` is not +valid JavaScript, and replacing it with a runtime `Symbol()` would change the +module's current zero-runtime-representation design. The exact JSDoc/runtime +representation is intentionally deferred; that module and dependents that need +its declaration identity must not migrate until the focused design is decided. + +The same rule applies to future TypeScript constructs without an obvious +semantics-preserving JSDoc translation: identify the issue, keep the affected +module in TypeScript temporarily, and resolve it before that group crosses the +Stage-1 boundary. Stage 1 still ends only when every such case has been resolved +and no authored `.ts` / `.f.ts` remains. + +For each migration group: + +- replace TypeScript-only syntax with equivalent JavaScript plus JSDoc types; +- preserve public assignability semantics, not only runtime behavior; +- if a TypeScript-only construct has no established semantics-preserving JSDoc + translation, record it as a focused hard case and postpone that group rather + than inventing a redesign inside the mechanical migration; +- update runtime imports and JSDoc type imports to the new source paths; +- update proofs, tests, scripts, generated CI configuration, documentation, and + other path-sensitive tooling; +- preserve type checking, declaration generation, runtime behavior, proofs, + coverage, and package behavior. + +#### End of stage 1 + +Keep `**/*.js` ignored while TypeScript can still generate `.js`. After the last +authored `.ts` / `.f.ts` source file is removed: + +1. simplify `prepack` from + `tsc --noEmit false --emitDeclarationOnly && tsc --noEmit false --declaration false` + to declaration-only `tsc --noEmit false --emitDeclarationOnly`; +2. remove the TypeScript-to-JavaScript emission path; +3. remove obsolete generated `.js` output from the working tree when performing + that transition; +4. remove the blanket `**/*.js` rule from `.gitignore` so authored `.js` can be + tracked again. + +Generated declaration ignores are independent and may remain. + +Only after this boundary may stage 2 use: + +```text +module.f.mjs -> module.f.js +``` + +Stage 2 additionally requires +[`../fjs/ci/todo/f-js-package-support.md`](../fjs/ci/todo/f-js-package-support.md) +so authored `.f.js` is directly type-checked, receives `.d.ts` declarations, is +packed, and works for a clean package consumer before the first +compiler-compatibility rename. + +### Tasks + +- [ ] Complete + [`f-mjs-package-support.md`](../fjs/ci/todo/f-mjs-package-support.md), + including `allowJs` / `checkJs`. +- [ ] Then complete + [`f-mjs-test-and-coverage.md`](../fjs/emergent_testing/todo/f-mjs-test-and-coverage.md) + before the first real repository `.f.ts` -> `.f.mjs` conversion. +- [ ] Update contributor, compiler, language, package, test, and roadmap + documentation to the stage-1 extension meanings. +- [ ] Identify dependency-leaf `.ts` / `.f.ts` files and migrate those first. +- [ ] Identify TypeScript-only type constructs that do not yet have a proven + semantics-preserving JSDoc translation; record them as focused hard cases + and postpone only the affected migration groups. +- [ ] Resolve the known `Phantom` / `unique symbol` hard case before migrating + `fjs/types/phantom/module.f.ts` or dependent groups that require it. +- [ ] Migrate `proof.f.ts` to `proof.f.mjs` when the proof is JavaScript/JSDoc + ready and its authored dependencies are migrated; do not gate this on + compiler support. +- [ ] Translate TypeScript generic constraints and `in` / `out` variance to + JSDoc `@template` syntax without changing assignability. +- [ ] Continue upward through the dependency graph in reviewable groups until no + authored TypeScript remains. +- [ ] Translate `.ts` to `.mjs` and `.f.ts` to `.f.mjs`, moving static type + information to JSDoc without weakening public type semantics. +- [ ] Keep migrated JavaScript free of runtime and declaration-retained + dependencies on remaining authored TypeScript. +- [ ] Update imports, proofs, tests, coverage globs, scripts, generated CI, and + documentation for every migrated group. +- [ ] Preserve Node, Deno, Bun, proof, coverage, type-checking, declaration, and + CI package behavior throughout the migration. +- [ ] Add required `**BREAKING CHANGES:**` changelog entries for public runtime + import paths that change. +- [ ] After the last authored TypeScript file is gone, simplify `prepack` to its + declaration-only command and remove the TS-to-JS emit path and obsolete + generated `.js` outputs. +- [ ] Then remove `**/*.js` from `.gitignore` so authored `.js` is trackable. +- [ ] Keep the compiler-compatibility migration explicitly **blocked by** this + task. + +### Acceptance criteria + +- `allowJs` and `checkJs` are enabled before the first authored TypeScript source + is converted to JavaScript. +- The `.f.mjs` runtime test/coverage fixture is complete before the first real + repository `.f.ts` -> `.f.mjs` conversion. +- No authored `.ts` or `.f.ts` source files remain in the repository, including + proof files. +- Migration can proceed incrementally from dependency leaves toward callers. +- Authored JavaScript uses `.mjs` / `.f.mjs` with JSDoc where static type + information is needed. +- Known TypeScript-to-JSDoc hard cases are explicitly identified; each affected + group remains in TypeScript until its focused design preserves the required + public/runtime semantics, without blocking unrelated migration groups. +- `proof.f.mjs` migration is gated by JavaScript/JSDoc and dependency readiness, + never by current FunctionalScript compiler support. +- TypeScript generic constraints and variance annotations are preserved with + their JSDoc `@template` equivalents; public assignability is not weakened. +- `.f.mjs` means FunctionalScript-intent JavaScript, not current-compiler + compatibility. +- Migrated JavaScript never depends on remaining authored TypeScript during the + transition. +- Package-owned `.mjs` and generated declarations work from a clean CI package + build and clean NPM consumer. +- Tests, proofs, coverage, supported runtimes, and type checking continue to + pass. +- After the last authored TypeScript source is removed, `prepack` performs only + declaration emission and no TypeScript-to-JavaScript emission remains. +- `.gitignore` no longer blanket-ignores `.js` at the end of this task. +- The compiler-compatibility migration starts only after this task and the + authored-`.f.js` package/tooling prerequisite are complete. + +### Related + +- [`../fjs/ci/todo/f-mjs-package-support.md`](../fjs/ci/todo/f-mjs-package-support.md) + — stage-1 authored `.mjs` validation, declarations, and package support. +- [`../fjs/emergent_testing/todo/f-mjs-test-and-coverage.md`](../fjs/emergent_testing/todo/f-mjs-test-and-coverage.md) + — stage-1 end-to-end `.f.mjs` proof and coverage prerequisite. +- [`../fjs/ci/todo/f-js-package-support.md`](../fjs/ci/todo/f-js-package-support.md) + — stage-2 authored `.f.js` package/tooling prerequisite. +- [`../fjs/ci/todo/publishing-packages.md`](../fjs/ci/todo/publishing-packages.md) + — broader package-publishing plan. +- [`../fjs/fsc/README.md`](../fjs/fsc/README.md) — authoritative FunctionalScript + extension and migration contract. +- [`fjs-nanvm-integration.md`](./fjs-nanvm-integration.md) — existing compiler + integration and compiler-compatibility migration. +- [`plan/roadmap.md`](./plan/roadmap.md) — project roadmap. diff --git a/todo/plan/roadmap.md b/todo/plan/roadmap.md index 742302f61d..f7c601708a 100644 --- a/todo/plan/roadmap.md +++ b/todo/plan/roadmap.md @@ -108,24 +108,29 @@ See [architecture.md §Human-readable paths](./architecture.md). bytecode is an optional, VM-internal, performance-oriented representation 5. Generic `Any` serialization (CBOR) in `nanvm-lib` — covers code as data; needed for CAS/CAVM -**Incremental repository migration:** - -The compiler will begin compiling the FunctionalScript repository as soon as it -can parse the first useful function modules; it will not wait for the complete -language feature set. Repository coverage grows together with parser and code -generator coverage: - -1. Select an existing `.f.ts` module whose complete syntax is supported. -2. Rename it to `.f.mjs`, move TypeScript types to JSDoc, and update imports. -3. Make that module a permanent parser/compiler regression input. -4. Repeat in separate, reviewable changes as more syntax becomes supported. - -`.f.mjs` is the marker for authored FunctionalScript that the compiler in the -same repository revision must accept. Unsupported modules remain `.f.ts`; no -migration should pull unrelated language features into scope merely to convert -a file. This is a continuing strategy rather than a one-time migration task or -a prerequisite for the compiler MVP. The extension contract and detailed -workflow are documented in [`fjs/fsc/README.md`](../../fjs/fsc/README.md). +**Repository source migration and compiler coverage:** + +The repository source-language migration is independent of compiler feature +coverage and is tracked in +[`todo/migrate-typescript-to-mjs.md`](../migrate-typescript-to-mjs.md): + +1. Stage 1 migrates authored `.f.ts` to `.f.mjs` dependency-first, moving types + to JSDoc. `.f.mjs` means FunctionalScript-intent JavaScript and may contain + syntax the current compiler does not support. +2. The compiler may validate any migrated `.f.mjs` module it already supports, + and synthetic compiler fixtures may land earlier, but compiler readiness does + not decide whether Stage-1 source or proof files migrate. +3. After all authored TypeScript is gone, Stage 2 migrates compiler-supported + dependency-closed groups from `.f.mjs` to `.f.js`. +4. An authored `.f.js` is the compiler-compatibility marker: the parser/compiler + in the same repository revision must accept it. Unsupported modules remain + `.f.mjs` until their compiler features land. + +This lets TypeScript removal and compiler implementation proceed independently +without either one blocking unrelated progress. The authoritative extension +contract and detailed workflow are documented in +[`fjs/fsc/README.md`](../../fjs/fsc/README.md), and the Stage-2 compiler migration +is tracked in [`todo/fjs-nanvm-integration.md`](../fjs-nanvm-integration.md). This is the longest dependency chain. Everything after it depends on it. @@ -164,7 +169,7 @@ Prerequisite: compiler + CA FunctionalScript complete. | SUL deduplication | `fjs/sul/` L1–L4 ✓ | CAS integration layer | | Compiler (parsing) | `fjs/djs/` data pipeline ✓, `fjs/bnf/` framework ✓ | Function support, FS grammar | | Compiler (codegen) | — | Rust code generator (FJS), `Function` constructor + interpreter in `nanvm-lib` | -| Compiler (repository coverage) | Extension and migration strategy defined | First eligible `.f.ts` → `.f.mjs` conversion, then incremental expansion | +| Compiler (repository coverage) | Stage-1 `.f.mjs` source migration is compiler-independent | Validate supported `.f.mjs` as coverage grows; after Stage 1, rename supported groups `.f.mjs` → `.f.js` | | CA FunctionalScript | — | Depends on VM + AST canonicalization | | Sandboxed execution | — | Depends on CA FS | | Hybrid intelligence | — | Depends on all above |