From 0dc41692fef29307f90a878fd04db26f367439c4 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 27 Aug 2026 18:34:51 +0000 Subject: [PATCH] private_types: prepare the migration and add the first private.ts Establishes the rule, the documents, and the packaging step that fjs/todo/separate-private-types.md needs, then migrates one module to prove the shape end to end. Rule: no authored .mjs anywhere in the repository carries a file-scope JSDoc @typedef, since declaration emit turns one into an exported type alias. A typedef inside a function is unaffected. Root AGENTS.md carries the repository-wide rule; fjs/AGENTS.md gains a "Private types" section with the public declaration closure, the optional private.ts, the optional meta/module.f.mjs, and the intra-directory dependency order; fjs/fsc/README.md replaces its `_`-leak-tolerance policy with the private-type contract. todo/blocked/jsdoc-typedef-strip-internal.md is deleted rather than narrowed - splitting private types out closes the leak, so the repository no longer waits on @internal/stripInternal or keeps two conflicting strategies - and its referrers now point at the migration. Packaging: prepack ends with node ./fjs/ci/prepack.mjs, which deletes every private.d.ts generated from an authored private.ts and then fails packaging if a remaining declaration still imports a private module. The check is semantic, not textual: it reads static module specifiers as tokens with `specifiers` from fjs/website/browser-source.mjs, so a JSDoc @import comment TypeScript kept in a declaration is read as a comment and no emitted text is rewritten. First private.ts: fjs/djs/tokenizer. _Token, _FlatToken, _TokenScanState, _StringDecodeState and _DjsScanState are reached only by module-private constants, so they move out of the public surface entirely - the emitted module.f.d.mts names none of them, and _StringDecodeState loses the spurious `any |` arm the multi-line @typedef form used to emit. Verified: npx tsc, the full proof suite (3477 pass), npm pack (no private artifact in the tarball), and a clean TypeScript consumer of fjs/djs/tokenizer installed from that tarball, with its negative control still failing TS2322. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01BHDDUXrAnbCeGBuGgbuPuw --- AGENTS.md | 8 ++ fjs/AGENTS.md | 102 ++++++++++++++-- fjs/ci/README.md | 13 ++ fjs/ci/prepack.mjs | 89 ++++++++++++++ fjs/ci/todo/f-mjs-package-support.md | 20 ++-- fjs/djs/tokenizer/module.f.mjs | 22 ++-- fjs/djs/tokenizer/private.ts | 35 ++++++ fjs/fsc/README.md | 78 ++++++------ fjs/todo/separate-private-types.md | 75 ++++++++---- package.json | 2 +- .../jsdoc-typedef-doc-declaration-emit.md | 5 +- todo/blocked/jsdoc-typedef-strip-internal.md | 113 ------------------ todo/migrate-typescript-to-mjs.md | 17 +-- 13 files changed, 365 insertions(+), 214 deletions(-) create mode 100644 fjs/ci/prepack.mjs create mode 100644 fjs/djs/tokenizer/private.ts delete mode 100644 todo/blocked/jsdoc-typedef-strip-internal.md diff --git a/AGENTS.md b/AGENTS.md index 0ac48b15a..cba8f1be6 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -73,6 +73,14 @@ Values are immutable (no in-place mutation, no `.push`/`Map#set`/index assignment), there is no `try`/`catch` and no regular expressions, and types are written in JSDoc with a sibling `types.ts` for a type-level API. +**No authored `.mjs` anywhere in the repository — inside `fjs/` or not — contains +a file-scope JSDoc `@typedef`.** Declaration emit turns one into an exported type +alias, so a file-scope typedef publishes an implementation detail. A typedef +written *inside a function* is fine and is how a compile-time proof states its +claim. A type a public declaration needs goes in `types.ts`; one nothing public +reaches goes in an optional sibling `private.ts`, whose generated declaration +`prepack` deletes before packaging. Private names keep their leading `_`. + Testing, documentation, and the full coding style: [fjs/AGENTS.md](./fjs/AGENTS.md). ## 4. Rust (`nanvm-lib/`) diff --git a/fjs/AGENTS.md b/fjs/AGENTS.md index eb11b882b..340dbb8ec 100644 --- a/fjs/AGENTS.md +++ b/fjs/AGENTS.md @@ -247,13 +247,13 @@ changes. A separately useful type-level API may live in an authored sibling `types.ts`; that file remains TypeScript type source and holds no runtime implementation. -Name implementation-only JSDoc typedefs with a leading `_` -(`/** @typedef {number} _Type */`). Declaration emit cannot strip them yet, so -the underscore — not the emitted `.d.ts` — is what marks a name private, -and renaming or removing a `_`-prefixed alias is not by itself a breaking -change. The public contract still governs transitive effects. See -[Private JSDoc typedefs](./fsc/README.md#private-jsdoc-typedefs) for the -full rule and examples. +Name private types and private runtime constants with a leading `_` — `_Type`, +`_framingKeywords`. The underscore, not the emitted `.d.ts`, is what marks a +name private: renaming or removing one is not by itself a breaking change, +though the public contract still governs transitive effects. Where a private +type is *written* is [Private types](#private-types) below; +[Private types](./fsc/README.md#private-types) in `fjs/fsc/README.md` has the +breaking-change examples. Use `@typedef` for a named type and `@template` for its type parameters. A constraint goes in braces before the parameter name: @@ -355,6 +355,88 @@ TypeScript one, but the public type contract must not become weaker for being written in JavaScript. Types authored in `types.ts` use ordinary TypeScript syntax and declaration emit. +#### Private types + +Authored `.mjs` files carry **no file-scope JSDoc `@typedef`** — this rule is +repository-wide, not `fjs/`-specific, and holds for `module.f.mjs`, +`proof.f.mjs`, descriptive companions such as `testlib.f.mjs`, and host `.mjs` +alike. Declaration emit turns a file-scope typedef into an exported type alias, +so writing one publishes it whether or not that was the intent. The rule holds +for everything you write; the files that predate it are being migrated under +[`fjs/todo/separate-private-types.md`](./todo/separate-private-types.md), so +finding one is not a licence to add another. + +A typedef **inside a function** is unaffected and is the right tool for a +compile-time proof, which often needs a lexical or downstream runtime value: + +```js +const signatures = () => { + /** @typedef {Assert>, Effect<...>>>} _Step */ + /** @typedef {Assert>, Effect<...>>>} _CatchStep */ +} +``` + +Everything else moves out of the implementation, by who needs it: + +- `types.ts` holds the **public declaration closure**: the public types, plus + every private `_` helper a shipped public declaration reaches — including the + declaration of an exported runtime function. If `find`'s emitted declaration + names `_SortedArray`, then `_SortedArray` belongs in `types.ts` or is + inlined; moving it to an unshipped module would leave the public declaration + incomplete. `types.ts` never depends on `private.ts`. +- `private.ts` is an **optional** sibling for implementation-private types + outside that closure. Use it where separating them makes the design cleaner, + not mechanically for every `_` name; a module with one local alias is usually + clearer with it inlined. It is authored type-only TypeScript like `types.ts`, + reached from JavaScript through JSDoc `@import { _X } from './private.ts'`. + +Within one module directory, preserve the dependency direction for whichever of +these roles exist — the arrow points from dependency to dependent: + +```text +types.ts <- private.ts <- module.f.mjs <- proof.f.mjs <- module.mjs <- proof.mjs +``` + +This is a layering guide, not a requirement that every file exists. Move +verification downstream rather than implementation upstream: an +`Assert, …>>` that checks `module.f.mjs` belongs in a +proof function in `proof.f.mjs`, not in `types.ts` where it would reverse the +arrow. Recursive RTTI whose annotation depends on the module's own public types +stays in `module.f.mjs` for the same reason. + +A subordinate `meta/module.f.mjs` is the other optional tool: an ordinary +lower-level module for declarative constants both TypeScript and the runtime +read — RTTI/schema constants, `as const` literal data, lookup tables whose +literal shape defines a type. `meta` is *metaprogramming*, not a file role: it +is discovered, tested, and covered as the `module.f.mjs` it is, with no +metadata-specific tooling rule. A constant exported from it only for +sibling-module linkage keeps its `_`, since exportability is linkage, not API +status: + +```js +// meta/module.f.mjs +export const _framingKeywords = + /** @type {const} */ (['import', 'const', 'export', 'default', 'from']) +``` + +Do not create either file because a `_` name or a runtime value exists. Ordinary +implementation functions stay in `module.f.mjs`. + +Moving an existing public type out of an `.mjs` declaration surface, or a public +runtime constant into `meta/module.f.mjs`, changes an import path: treat it as an +intentional breaking change — update every importer and the changelog, and add +no compatibility re-export. + +`private.ts` stays in the normal TypeScript program so source consumers are +checked, so declaration emit produces a `private.d.ts`. That file is not +shipped: `fjs/ci/prepack.mjs` runs as the final `prepack` step, after +declaration emit and the round-trip check, and deletes it before the package +file list is read. Emitted declarations are never text-postprocessed — TypeScript +may keep the source's `/** @import { _X } from './private.ts' */` comment, which +is a comment in a `.d.ts` and no dependency at all — so the same step checks the +*semantic* dependency instead, failing packaging if any shipped declaration +actually imports a private module. + #### Prefer inference Let TypeScript infer the type of private constants, local variables, and return @@ -963,10 +1045,14 @@ repository-owned dependencies follow these source rules: - `.f.mjs` is authored FunctionalScript implementation/proof source, and its relative runtime imports target `.f.mjs`; - `types.ts` is authored type-only TypeScript source and carries no runtime - implementation; + implementation; an optional `private.ts` beside it is the same kind of file + for the private types no public declaration reaches + ([§3.2](#private-types)); - `.f.mjs` — and later `.f.js` — consumes `types.ts` through JSDoc `@import`, and TypeScript consumes it through `import type`, both always naming the real `types.ts` file; +- no authored `.mjs` carries a file-scope JSDoc `@typedef`; a typedef inside a + function is fine; - a declaration-only module belongs in `types.ts` rather than acquiring an artificial runtime representation; - never add a runtime import/export or runtime value solely to represent a diff --git a/fjs/ci/README.md b/fjs/ci/README.md index d22b521a7..69f7e6eec 100644 --- a/fjs/ci/README.md +++ b/fjs/ci/README.md @@ -151,6 +151,19 @@ export type Setup = { `nodeExtra` receives the target OS so callers can conditionally add OS-specific steps. Rust steps are included automatically when `Cargo.toml` is present; no flag is needed. +## Packaging + +`prepack.mjs` is not part of the workflow generator: it is the last step of the +package's own `prepack`, run by `npm pack` and `npm publish` after declaration +emit and the round-trip check. It deletes every `private.d.ts` that declaration +emit produced from an authored `private.ts` — implementation-private types are +checked with the rest of the program but never shipped — and then fails +packaging if any remaining declaration still *imports* a private module. That +check reads static module specifiers as tokens, so a JSDoc `@import` comment +TypeScript kept in a declaration is correctly read as a comment and nothing in +the emitted text is rewritten. The rule it enforces is "Private types" in +[`fjs/AGENTS.md`](../AGENTS.md#private-types). + ## Related - [`packed-consumer-validation.md`](./packed-consumer-validation.md) — manual diff --git a/fjs/ci/prepack.mjs b/fjs/ci/prepack.mjs new file mode 100644 index 000000000..60cced813 --- /dev/null +++ b/fjs/ci/prepack.mjs @@ -0,0 +1,89 @@ +/** + * The final `prepack` step: drops the declarations generated for authored + * `private.ts` modules, then proves that nothing left for packaging depends on + * one. + * + * A `private.ts` holds implementation-private types that are outside the public + * declaration closure (see `fjs/AGENTS.md`), so it stays in the TypeScript + * program — source consumers are checked — while its `private.d.ts` is never + * shipped. Declaration emit runs first, this runs last, and the package file + * list is read after both. + * + * The dependency check is semantic, not textual: emitted declarations may keep + * a source JSDoc `@import { _X } from './private.ts'` comment, which is a + * comment in a `.d.ts` and no dependency at all. `specifiers` reads static + * module specifiers as tokens, so it sees the `import`/`export` statements and + * not what a comment says. + */ + +import { readdir, readFile, rm } from 'node:fs/promises' +import { relative } from 'node:path' +import { fileURLToPath } from 'node:url' + +import { local, specifiers } from '../website/browser-source.mjs' + +const sourceRoot = new URL('../../', import.meta.url) + +/** @type {(url: URL) => string} */ +const repoPath = url => relative(fileURLToPath(sourceRoot), fileURLToPath(url)) + +/** @type {(directory: URL) => Promise} */ +const files = async directory => { + const entries = await readdir(directory, { withFileTypes: true }) + return (await Promise.all(entries.map(entry => { + if (entry.name.startsWith('.') || entry.name === 'node_modules' || entry.name === 'target') { return [] } + const url = new URL(entry.isDirectory() ? `${entry.name}/` : entry.name, directory) + return entry.isDirectory() ? files(url) : [url] + }))).flat() +} + +/** @type {(url: URL, name: string) => boolean} */ +const named = (url, name) => url.pathname.endsWith(`/${name}`) + +/** + * Whether a static module specifier names a private type module. Only a + * repository-relative one can: a bare specifier names a package, and a package + * has no private module to reach. + * + * @type {(specifier: string) => boolean} + */ +const privateSpecifier = specifier => { + if (!local(specifier)) { return false } + const name = specifier.slice(specifier.lastIndexOf('/') + 1) + return name === 'private.ts' || name === 'private.js' + || name === 'private.mjs' || name === 'private.d.ts' +} + +const all = await files(sourceRoot) + +// Only a declaration with an authored `private.ts` beside it was generated from +// one; anything else named `private.d.ts` is not this step's to delete. +const authored = new Set(all.filter(url => named(url, 'private.ts')).map(url => url.href)) +const generated = all.filter(url => + named(url, 'private.d.ts') && authored.has(new URL('private.ts', url).href)) + +await Promise.all(generated.map(url => rm(url))) +for (const url of generated) { + console.log(`removed ${repoPath(url)}`) +} + +const removed = new Set(generated.map(url => url.href)) +const declarations = all.filter(url => + (url.pathname.endsWith('.d.ts') || url.pathname.endsWith('.d.mts')) && !removed.has(url.href)) + +const dependents = (await Promise.all(declarations.map(async url => + specifiers(await readFile(url, 'utf8')) + .filter(privateSpecifier) + .map(specifier => /** @type {const} */ ([url, specifier])) +))).flat() + +for (const [url, specifier] of dependents) { + console.error(`${repoPath(url)} depends on the unshipped private type module ${specifier}`) +} + +if (dependents.length === 0) { + console.log(`private type modules removed: ${generated.length}; declarations checked: ${declarations.length}`) +} else { + console.error('move the types a public declaration needs into types.ts, or inline them') + process.exitCode = 1 +} diff --git a/fjs/ci/todo/f-mjs-package-support.md b/fjs/ci/todo/f-mjs-package-support.md index eea1a0991..bff9a50a8 100644 --- a/fjs/ci/todo/f-mjs-package-support.md +++ b/fjs/ci/todo/f-mjs-package-support.md @@ -170,12 +170,11 @@ must not turn `_`-prefixed declaration artifacts into supported API merely because TypeScript emitted them. Types intentionally moved to `types.ts` use ordinary TypeScript syntax and do -not need the JSDoc-emission workaround merely to remain expressible. The eventual -replacement for private JSDoc typedefs is still `@internal` plus `stripInternal`, -blocked on -[microsoft/TypeScript#46407](https://github.com/microsoft/TypeScript/issues/46407) -and tracked in -[`todo/blocked/jsdoc-typedef-strip-internal.md`](../../../todo/blocked/jsdoc-typedef-strip-internal.md). +not need the JSDoc-emission workaround merely to remain expressible. That +workaround is being retired outright rather than waiting for `@internal` plus +`stripInternal`: no authored `.mjs` carries a file-scope `@typedef`, so nothing +is left for declaration emit to leak. See +[`fjs/todo/separate-private-types.md`](../../todo/separate-private-types.md). Package selection does not need to distinguish every authored `.mjs` by public API status during this transition. Incidental authored files such as @@ -327,11 +326,10 @@ not, and the pipeline is simplified accordingly. two-pass `prepack`. - [`todo/migrate-typescript-to-mjs.md`](../../../todo/migrate-typescript-to-mjs.md) — repository-wide stage-1 implementation source migration. -- [`todo/blocked/jsdoc-typedef-strip-internal.md`](../../../todo/blocked/jsdoc-typedef-strip-internal.md) - — replace the temporary `_` convention with `@internal` when declaration emit - supports it. -- [microsoft/TypeScript#46407](https://github.com/microsoft/TypeScript/issues/46407) - — upstream blocker for stripping private JSDoc typedefs. +- [`fjs/todo/separate-private-types.md`](../../todo/separate-private-types.md) + — keep private types out of public declarations; the final `prepack` step that + drops generated `private.d.ts` and checks the packed declarations for a + semantic dependency on one. - [`publishing-packages.md`](./publishing-packages.md) — broader package roadmap. - [`f-js-package-support.md`](./f-js-package-support.md) — stage-2 authored `.f.js` package prerequisite. diff --git a/fjs/djs/tokenizer/module.f.mjs b/fjs/djs/tokenizer/module.f.mjs index 1d628563f..3afed8976 100644 --- a/fjs/djs/tokenizer/module.f.mjs +++ b/fjs/djs/tokenizer/module.f.mjs @@ -28,6 +28,13 @@ * @import { StateScan } from '../../types/function/operator/types.ts' * @import { List } from '../../types/list/types.ts' * @import { DjsToken, DjsTokenWithMetadata } from './types.ts' + * @import { + * _DjsScanState, + * _FlatToken, + * _StringDecodeState, + * _Token, + * _TokenScanState, + * } from './private.ts' * @import { TriviaKind } from '../../js/tokenizer/types.ts' * @import { Nullable } from '../../types/nullable/types.ts' */ @@ -314,13 +321,6 @@ const metadataScan = (cp, metadata) => [[[cp, metadata]], advanceMetadata(cp)(me /** @type {(path: string) => (cp: readonly number[]) => readonly CodePointMeta[]} */ const codePointsWithMetadata = path => cp => toArray(flat(stateScan(metadataScan)({ path, line: 1, column: 1 })(cp))) -// tag, the metadata of the token's first code point, and its code points. -/** @typedef {[string, TokenMetadata, readonly number[]]} _Token */ - -/** @typedef {string | CodePointMeta} _FlatToken */ - -/** @typedef {[string, TokenMetadata | null, List]} _TokenScanState */ - /** * The grammar tag of a trivia code point, as the kind `mergeTrivia` speaks in; * `null` for every other tag. @@ -396,12 +396,6 @@ const filterFunc = tk => { */ const unwrapHexDigitValue = mapUnwrap(hexDigitValue) -/** @typedef { - * | { readonly kind: 'normal' } - * | { readonly kind: 'escape' } - * | { readonly kind: 'unicode', readonly acc: number, readonly count: number } - * } _StringDecodeState */ - /** @type {StateScan>} */ const stringDecodeScan = (cp, state) => { switch (state.kind) { @@ -592,8 +586,6 @@ export const tokenizeJs = input => path => { return withMetadata([{ token: { kind: 'eof' }, metadata: finalMetadata }]) } -/** @typedef {{ readonly kind: 'def' | '-' }} _DjsScanState */ - /** @type {(input: JsToken) => List} */ const mapDjsToken = input => { switch (input.kind) { diff --git a/fjs/djs/tokenizer/private.ts b/fjs/djs/tokenizer/private.ts new file mode 100644 index 000000000..570093831 --- /dev/null +++ b/fjs/djs/tokenizer/private.ts @@ -0,0 +1,35 @@ +/** + * Implementation-private types for `fjs/djs/tokenizer/module.f.mjs`. + * + * Nothing here belongs to the public declaration closure: no exported + * declaration of the module names any of these types, so keeping them out of + * `./types.ts` keeps them out of the shipped declarations too. They are + * exported only so the implementation can `@import` them — the leading `_` is + * what marks them private, and renaming or removing one is not a breaking + * change. + */ + +import type { CodePointMeta } from '../../bnf/descent/types.ts' +import type { TokenMetadata } from '../../js/tokenizer/types.ts' +import type { List } from '../../types/list/types.ts' + +/** + * A token as `scanFunc` emits it: its tag, the metadata of its first code + * point, and its code points. + */ +export type _Token = [string, TokenMetadata, readonly number[]] + +/** Either a bare grammar tag or one code point paired with its metadata. */ +export type _FlatToken = string | CodePointMeta + +/** The token `scanFunc` is still accumulating: `null` metadata until its first code point arrives. */ +export type _TokenScanState = [string, TokenMetadata | null, List] + +/** Where `stringDecodeScan` is inside a string literal's escape sequences. */ +export type _StringDecodeState = + | { readonly kind: 'normal' } + | { readonly kind: 'escape' } + | { readonly kind: 'unicode', readonly acc: number, readonly count: number } + +/** Whether `scanDjsToken` has an unconsumed `-` to fold into the next number. */ +export type _DjsScanState = { readonly kind: 'def' | '-' } diff --git a/fjs/fsc/README.md b/fjs/fsc/README.md index 2ea737ab0..a69ac9a54 100644 --- a/fjs/fsc/README.md +++ b/fjs/fsc/README.md @@ -126,28 +126,38 @@ move, so a `module.f.mjs` is accompanied by a `proof.f.mjs`. Type-only APIs may remain in `types.ts`. Current FunctionalScript compiler support was never a condition for that rename. -#### Private JSDoc typedefs - -TypeScript declaration emit currently turns JSDoc `@typedef`s into exported type -aliases, including typedefs that exist only as implementation details. This is -tracked upstream by -[microsoft/TypeScript#46407](https://github.com/microsoft/TypeScript/issues/46407). - -Until JSDoc typedefs can be stripped with `@internal` and `stripInternal`, use a -leading `_` for implementation-only typedefs created during the migration: - -```js -/** @typedef {number} _Type */ -``` - -The underscore is an API contract, not declaration-level visibility. Generated -`.d.ts` / `.d.mts` may still contain `export type _Type = number`, but names that -begin with `_` are private FunctionalScript implementation details. Consumers -must not rely on those names directly, so renaming or removing a `_`-prefixed -alias is not a breaking change solely because TypeScript emitted it. The public -contract still governs transitive effects: if a public type depends on `_Type`, -changing `_Type` in a way that changes that public type's assignability is a -breaking change and requires the normal `**BREAKING CHANGES:**` treatment. +#### Private types + +A private type is one consumers must not depend on, and its name begins with +`_`. Where it is written follows from who reaches it: + +- a type some shipped public declaration names — including the declaration of an + exported runtime function — is part of the **public declaration closure** and + belongs in `types.ts`, or is inlined. It ships, `_` and all; moving it + somewhere unshipped would only leave the public declaration incomplete. +- a type nothing public reaches belongs in an optional sibling `private.ts`. It + is checked with the rest of the program, but the `private.d.ts` that + declaration emit produces for it is deleted by the final `prepack` step and + never packed. +- a type that exists only to state a compile-time proof lives inside the + function that proves it. + +No authored `.mjs` carries a file-scope `@typedef` at all, because declaration +emit turns one into an exported type alias — the leak this section used to +describe as unavoidable. A typedef inside a function is unaffected. The full +rule, the dependency order between `types.ts`, `private.ts` and the +implementation, and the optional metaprogramming submodule are in +[`fjs/AGENTS.md`](../AGENTS.md#private-types). + +The underscore is an API contract, not declaration-level visibility: a shipped +`types.d.ts` may well contain `export type _Type = number`, and a `.d.mts` may +keep a source `/** @import { _T } from './private.ts' */` comment naming a module +that was never packed — a comment in a declaration file is not a dependency. +Consumers must not rely on a `_` name, so renaming or removing one is not a +breaking change solely because TypeScript emitted it. The public contract still +governs transitive effects: if a public type depends on `_Type`, changing +`_Type` in a way that changes that public type's assignability is a breaking +change and requires the normal `**BREAKING CHANGES:**` treatment. For example, suppose the generated declaration initially contains: @@ -176,24 +186,24 @@ export type Public = readonly [_Internal] The emitted private alias is still private, but the expanded public contract of `Public` changed from `readonly [number]` to `readonly [string]`. -Public JSDoc typedefs keep ordinary names without the `_` prefix. Which JSDoc -typedefs are public is an API design decision, not a mechanical restatement of -what the pre-migration `.f.ts` file happened to export: a helper that belongs to -the module's public vocabulary may be published under an ordinary name even -though its TypeScript alias was module-private, and a former export may become -`_` when it only ever described an implementation detail. Types intentionally -separated into `types.ts` use ordinary TypeScript source visibility instead of -this JSDoc-emission workaround. +Public types keep ordinary names without the `_` prefix. Which of them are public +is an API design decision, not a mechanical restatement of what the +pre-migration `.f.ts` file happened to export: a helper that belongs to the +module's public vocabulary may be published under an ordinary name even though +its TypeScript alias was module-private, and a former export may become `_` when +it only ever described an implementation detail. -When upstream support is ready, replace this workaround with `@internal`; that -cleanup is tracked by -[`todo/blocked/jsdoc-typedef-strip-internal.md`](../../todo/blocked/jsdoc-typedef-strip-internal.md). +Splitting the private types out is what makes the boundary real, so nothing here +waits on `@internal` / `stripInternal` any more. The migration of the files +written before this rule is +[`fjs/todo/separate-private-types.md`](../todo/separate-private-types.md). When the last authored implementation/proof `.ts` / `.f.ts` file is gone, authored `types.ts` files may remain. The TypeScript runtime-emission pass is removed ([#1520](https://github.com/functionalscript/functionalscript/pull/1520) measured that package resolution does not require a generated `types.js`), while -`prepack` keeps a no-emit re-check with declarations present. Remove the blanket +`prepack` keeps a no-emit re-check with declarations present and then drops the +generated `private.d.ts` files. Remove the blanket `**/*.js` rule from `.gitignore` only when generated implementation `.js` no longer conflicts with authored `.js`. diff --git a/fjs/todo/separate-private-types.md b/fjs/todo/separate-private-types.md index 8110d887f..f0fb807ed 100644 --- a/fjs/todo/separate-private-types.md +++ b/fjs/todo/separate-private-types.md @@ -1,7 +1,7 @@ ## Keep private types out of public declarations **Priority:** P2 -**Status:** open +**Status:** wip ### Problem @@ -13,6 +13,35 @@ The requirement is a clean, self-contained public declaration/API boundary. `private.ts` and subordinate modules such as `meta/module.f.mjs` are **tools** for reaching that result, not required companion files. +### Progress + +The rules, the policy documents, and the packaging step are in place; what +remains is migrating the authored `.mjs` files written before them. + +Done: + +- root `AGENTS.md`, `fjs/AGENTS.md` ("Private types" under §3.2 and §3.5), and + `fjs/fsc/README.md` state the rule, the public declaration closure, the + optional `private.ts` and `meta/module.f.mjs`, and the dependency order; +- `todo/blocked/jsdoc-typedef-strip-internal.md` is deleted — the `@internal` / + `stripInternal` wait is superseded, not merely narrowed — and its referrers + point here; +- `prepack` ends with `node ./fjs/ci/prepack.mjs`, which deletes every generated + `private.d.ts` and then fails packaging if a remaining declaration has a + *semantic* dependency on a private module. The check reads static module + specifiers as tokens (`specifiers` in `fjs/website/browser-source.mjs`), so a + retained JSDoc `@import` comment is not mistaken for one and no emitted text is + rewritten; +- `fjs/djs/tokenizer` is the first module with a `private.ts`: `_Token`, + `_FlatToken`, `_TokenScanState`, `_StringDecodeState` and `_DjsScanState` left + its declaration, which now names none of them. Validated against the packed + tarball — no private artifact in it, and a clean TypeScript consumer of + `fjs/djs/tokenizer` type-checks with `bad.ts` still failing TS2322. + +Remaining: the other authored `.mjs` files that still hold a file-scope +`@typedef`, including the individually-analyzed cases below and `todo/proof.f.mjs` +outside `fjs/`. + ### Rules #### No file-scope typedefs in authored `.mjs` @@ -194,30 +223,26 @@ Package validation must check semantic dependencies, not raw text: ### Repository policy -When this TODO is implemented: - -- update root `AGENTS.md` with the repository-wide rule that authored `.mjs` files - may not contain file-scope JSDoc `@typedef`; -- update `fjs/AGENTS.md` with the public-declaration-closure rule, optional - `private.ts`, optional subordinate metaprogramming modules such as - `meta/module.f.mjs`, and the dependency-order guidance; -- update `fjs/fsc/README.md` and delete or narrow - `todo/blocked/jsdoc-typedef-strip-internal.md` so the repository does not keep - two conflicting private-type strategies. +Done, see Progress above: root `AGENTS.md` carries the repository-wide rule, +`fjs/AGENTS.md` the public-declaration-closure rule with the optional +`private.ts`, the optional `meta/module.f.mjs`, and the dependency order, and +`fjs/fsc/README.md` the private-type contract that replaced its `_`-leak-tolerance +policy. `todo/blocked/jsdoc-typedef-strip-internal.md` is deleted, so the +repository no longer keeps two conflicting private-type strategies. Authored TypeScript type modules (`types.ts`, and `private.ts` when present) remain type-only and use named `import type { ... }` imports. ### Tasks -- [ ] Document the repository-wide prohibition on file-scope JSDoc `@typedef` in +- [x] Document the repository-wide prohibition on file-scope JSDoc `@typedef` in authored `.mjs`; allow function-local typedefs. - [ ] Migrate existing violations, including authored `.mjs` outside `fjs/` such as `todo/proof.f.mjs`. - [ ] Keep `types.ts` as the public declaration closure; retain/in-line private helpers required by public declarations. -- [ ] Use `private.ts` only where separating implementation-private file-scope - types improves the design. +- [x] Use `private.ts` only where separating implementation-private file-scope + types improves the design — first one: `fjs/djs/tokenizer/private.ts`. - [ ] Preserve the intra-directory dependency direction shown above; move verification downstream when that is cleaner. - [ ] Move the `fjs/effects/types.ts` implementation-signature asserts into proof @@ -230,15 +255,15 @@ type-only and use named `import type { ... }` imports. - [ ] Preserve leading `_` for private types and private runtime constants. - [ ] Treat chosen public import-path moves as breaking changes with no compatibility re-exports. -- [ ] If `private.ts` is used, delete generated `private.d.ts` as the final +- [x] If `private.ts` is used, delete generated `private.d.ts` as the final `prepack` step. -- [ ] Do not text-postprocess emitted declarations; validate semantic private +- [x] Do not text-postprocess emitted declarations; validate semantic private dependencies and clean-consumer type checking instead. - [ ] Add fixtures/examples covering: public-declaration helpers, optional `private.ts`, function-local proof typedefs, recursive RTTI kept in `module.f.mjs`, optional `meta/module.f.mjs`, retained non-semantic JSDoc comments, and authored `.mjs` outside `fjs/`. -- [ ] Update root/fjs policy documentation and reconcile the old `_` leak policy. +- [x] Update root/fjs policy documentation and reconcile the old `_` leak policy. ### Acceptance criteria @@ -269,11 +294,17 @@ type-only and use named `import type { ... }` imports. ### Related -- [`../fsc/README.md`](../fsc/README.md) — current `_` leak-tolerance policy. -- [`../../AGENTS.md`](../../AGENTS.md) — root repository policy to update. -- [`../AGENTS.md`](../AGENTS.md) — `fjs/`-specific file/dependency policy. -- [`../../todo/blocked/jsdoc-typedef-strip-internal.md`](../../todo/blocked/jsdoc-typedef-strip-internal.md) - — current wait-for-`@internal`/`stripInternal` strategy. +- [`../fsc/README.md`](../fsc/README.md) — the private-type contract that + replaced the `_` leak-tolerance policy. +- [`../../AGENTS.md`](../../AGENTS.md) — root repository policy; carries the + repository-wide no-file-scope-`@typedef` rule. +- [`../AGENTS.md`](../AGENTS.md) — `fjs/`-specific file/dependency policy; + "Private types" holds the closure, `private.ts`, `meta/` and dependency order. +- [`../ci/prepack.mjs`](../ci/prepack.mjs) — the final `prepack` step: drops + generated `private.d.ts` and checks the remaining declarations for a semantic + dependency on a private module. +- [`../djs/tokenizer/private.ts`](../djs/tokenizer/private.ts) — the first + `private.ts`. - [microsoft/TypeScript#46407](https://github.com/microsoft/TypeScript/issues/46407) — upstream JSDoc typedef stripping limitation. - [`detect-unexported-types-referenced-by-exported-types.md`](./detect-unexported-types-referenced-by-exported-types.md) diff --git a/package.json b/package.json index c06f77097..3316c6aa9 100644 --- a/package.json +++ b/package.json @@ -10,7 +10,7 @@ ], "description": "FunctionalScript is a purely functional subset of JavaScript", "scripts": { - "prepack": "tsc --noEmit false --emitDeclarationOnly && tsc", + "prepack": "tsc --noEmit false --emitDeclarationOnly && tsc && node ./fjs/ci/prepack.mjs", "test": "tsc && node ./fjs/module.mjs t", "cov": "node --test --experimental-test-coverage --test-coverage-include=**/module.f.mjs --test-coverage-lines=100 --test-coverage-branches=100 --test-coverage-functions=100", "start": "node ./fjs/module.mjs", diff --git a/todo/blocked/jsdoc-typedef-doc-declaration-emit.md b/todo/blocked/jsdoc-typedef-doc-declaration-emit.md index c141d98d4..92d5f6d31 100644 --- a/todo/blocked/jsdoc-typedef-doc-declaration-emit.md +++ b/todo/blocked/jsdoc-typedef-doc-declaration-emit.md @@ -208,8 +208,9 @@ Body: - [`todo/migrate-typescript-to-mjs.md`](../migrate-typescript-to-mjs.md) — "Typedef documentation does not survive declaration emit". -- [`jsdoc-typedef-strip-internal.md`](./jsdoc-typedef-strip-internal.md) — - the adjacent `@internal` + `stripInternal` gap for JSDoc typedefs. +- [`../fjs/todo/separate-private-types.md`](../../fjs/todo/separate-private-types.md) + — the adjacent private-typedef leak, closed by removing file-scope typedefs + from authored `.mjs` rather than by `@internal` + `stripInternal`. - [microsoft/TypeScript#43534](https://github.com/microsoft/TypeScript/issues/43534), [microsoft/TypeScript#61664](https://github.com/microsoft/TypeScript/issues/61664) — adjacent strada behaviors. diff --git a/todo/blocked/jsdoc-typedef-strip-internal.md b/todo/blocked/jsdoc-typedef-strip-internal.md deleted file mode 100644 index fadfd6d1f..000000000 --- a/todo/blocked/jsdoc-typedef-strip-internal.md +++ /dev/null @@ -1,113 +0,0 @@ -# Use `@internal` for private JSDoc typedefs - -**Priority:** P3 -**Status:** blocked - -### Problem - -During the TypeScript-to-JavaScript migration, implementation-only TypeScript -types become JSDoc `@typedef`s. TypeScript currently emits those typedefs as -exported type aliases in generated declarations even when they are not intended -to be public API. - -Until the declaration emitter can strip private JSDoc typedefs, the repository -uses a leading `_` as an API convention: a typedef such as `_Node` is private by -contract even if the generated `.d.ts` / `.d.mts` contains `export type _Node`. -Consumers must not depend on that emitted name directly, so renaming or removing -the alias is not a breaking change solely because it was emitted. This does not -exempt changes propagated into public types: if a public declaration depends on -`_Node`, any change that alters that public declaration's assignability remains -a breaking API change. - -The desired long-term representation is `@internal` plus `stripInternal`, so the -generated declaration does not expose the private type at all. - -### Trigger - -Unblocked when the TypeScript compiler used by this repository supports applying -`@internal` to JSDoc `@typedef` declarations and `stripInternal` reliably omits -those typedefs from generated `.d.ts` / `.d.mts` files. - -The canonical blocker is -[microsoft/TypeScript#46407](https://github.com/microsoft/TypeScript/issues/46407), -which is still open and specifically requests `stripInternal` support for types -defined with JSDoc. - -Another open TypeScript declaration/comment-emission issue, -[microsoft/TypeScript#62453](https://github.com/microsoft/TypeScript/issues/62453), -demonstrates the same JSDoc typedef-to-`export type` emission path while tracking -duplicated typedef comments. It is related context, not the visibility blocker. - -A separate equivalent TypeScript 7 / Go issue was not found in -`microsoft/typescript-go`. The native compiler does implement `stripInternal` -in general, but the known JSDoc declaration-emission reports still show -`@typedef`s becoming exported aliases. Related TypeScript-Go issues are: - -- [microsoft/typescript-go#4363](https://github.com/microsoft/typescript-go/issues/4363) - — open; emitted JSDoc typedef aliases and their documentation ordering. -- [microsoft/typescript-go#4235](https://github.com/microsoft/typescript-go/issues/4235) - — closed; JSDoc typedef/property documentation in declaration emit. -- [microsoft/typescript-go#4011](https://github.com/microsoft/typescript-go/issues/4011) - — closed; correctness of generated declaration syntax for JSDoc typedefs. - -These TypeScript-Go issues are adjacent declaration-emitter bugs, not substitutes -for #46407. Re-check the current TypeScript tracker when this task is unblocked, -especially as the native compiler work is consolidated with the main TypeScript -project. - -### Proposal - -Once the trigger is satisfied: - -1. enable or retain `stripInternal` for declaration emission; -2. mark implementation-only JSDoc typedefs with `@internal`; -3. remove leading `_` from private typedef names where the prefix exists only as - the current visibility workaround; -4. add a package/declaration fixture proving that private typedefs are absent - from emitted declarations while public declarations remain valid; -5. update migration, compiler, package, and contributor documentation to remove - the underscore workaround. - -Do not strip a private typedef if a public declaration still depends on its name; -refactor the public declaration first so emitted declarations remain -self-contained and preserve the same public assignability contract. - -### Tasks - -- [ ] Enable or retain `stripInternal` for declaration emission once the trigger - is satisfied. -- [ ] Mark implementation-only JSDoc typedefs with `@internal`. -- [ ] Remove leading `_` from private typedef names where the prefix exists only - as the temporary visibility workaround. -- [ ] Refactor public declarations that refer to private typedef names so they - remain self-contained and preserve the same public assignability contract - before those private typedefs are stripped. -- [ ] Add a package/declaration fixture proving that private typedefs are absent - from emitted declarations while public declarations remain valid. -- [ ] Update migration, compiler, package, and contributor documentation to - remove the underscore workaround. - -### Acceptance criteria - -- `@internal` on a JSDoc `@typedef` is honored by the repository's TypeScript - declaration emitter when `stripInternal` is enabled. -- Generated `.d.ts` / `.d.mts` files omit implementation-only typedefs. -- Public emitted declarations never reference a stripped private type. -- Removing the `_` workaround does not weaken or otherwise change public - assignability unless that change is explicitly treated as breaking. -- The `_`-prefix workaround is removed from repository documentation and from - private typedefs that used it solely for visibility. -- Clean package-consumer type checking still passes. - -### Related - -- [`../migrate-typescript-to-mjs.md`](../migrate-typescript-to-mjs.md) — Stage 1 - TypeScript-to-JSDoc migration and the temporary `_` convention. -- [`../../fjs/fsc/README.md`](../../fjs/fsc/README.md) — source migration and - JSDoc visibility contract. -- [`../../fjs/ci/todo/f-mjs-package-support.md`](../../fjs/ci/todo/f-mjs-package-support.md) - — declaration-emission and clean-consumer validation. -- [microsoft/TypeScript#46407](https://github.com/microsoft/TypeScript/issues/46407) - — canonical upstream feature request. -- [microsoft/TypeScript#62453](https://github.com/microsoft/TypeScript/issues/62453) - — related JSDoc typedef declaration/comment emission bug. diff --git a/todo/migrate-typescript-to-mjs.md b/todo/migrate-typescript-to-mjs.md index 0065dce91..a88af61fc 100644 --- a/todo/migrate-typescript-to-mjs.md +++ b/todo/migrate-typescript-to-mjs.md @@ -402,9 +402,12 @@ plans to remove both. Hiding a type behind `_` to make its eventual removal cheaper gives up a real present-day API in exchange for a discount on a breaking change that should simply be documented when it happens. -This convention is temporary. Once TypeScript can strip `@internal` JSDoc -typedefs correctly, replace the underscore workaround as tracked by -[`blocked/jsdoc-typedef-strip-internal.md`](./blocked/jsdoc-typedef-strip-internal.md). +The `_` convention itself stays; what was temporary is the leak. Rather than +waiting for TypeScript to strip `@internal` JSDoc typedefs, authored `.mjs` no +longer carries a file-scope `@typedef` at all — a private type a public +declaration needs lives in `types.ts`, one nothing public reaches in an optional +`private.ts`, and a proof's own aliases inside the proof function. See +[`../fjs/todo/separate-private-types.md`](../fjs/todo/separate-private-types.md). #### Typedef documentation does not survive declaration emit @@ -1250,11 +1253,9 @@ person can re-check rather than re-derive. Counts are as of — broader package-publishing plan. - [`../fjs/fsc/README.md`](../fjs/fsc/README.md) — authoritative FunctionalScript extension and migration contract. -- [`blocked/jsdoc-typedef-strip-internal.md`](./blocked/jsdoc-typedef-strip-internal.md) - — replace the temporary `_` convention with `@internal` when upstream - declaration emit supports it. -- [microsoft/TypeScript#46407](https://github.com/microsoft/TypeScript/issues/46407) - — upstream request for `stripInternal` support on JSDoc typedefs. +- [`../fjs/todo/separate-private-types.md`](../fjs/todo/separate-private-types.md) + — keep private types out of public declarations, replacing the `@internal` / + `stripInternal` wait. - [`fjs-nanvm-integration.md`](./fjs-nanvm-integration.md) — existing compiler integration and compiler-compatibility migration. - [`plan/roadmap.md`](./plan/roadmap.md) — project roadmap.