Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,14 @@ Values are immutable (no in-place mutation, no `.push`/`Map#set`/index
assignment), there is no `try`/`catch` and no regular expressions, and types are
written in JSDoc with a sibling `types.ts` for a type-level API.

**No authored `.mjs` anywhere in the repository — inside `fjs/` or not — contains
a file-scope JSDoc `@typedef`.** Declaration emit turns one into an exported type
alias, so a file-scope typedef publishes an implementation detail. A typedef
written *inside a function* is fine and is how a compile-time proof states its
claim. A type a public declaration needs goes in `types.ts`; one nothing public
reaches goes in an optional sibling `private.ts`, whose generated declaration
`prepack` deletes before packaging. Private names keep their leading `_`.

Testing, documentation, and the full coding style: [fjs/AGENTS.md](./fjs/AGENTS.md).

## 4. Rust (`nanvm-lib/`)
Expand Down
102 changes: 94 additions & 8 deletions fjs/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -247,13 +247,13 @@ changes. A separately useful type-level API may live in an authored sibling
`types.ts`; that file remains TypeScript type source and holds no runtime
implementation.

Name implementation-only JSDoc typedefs with a leading `_`
(`/** @typedef {number} _Type */`). Declaration emit cannot strip them yet, so
the underscore — not the emitted `.d.ts` — is what marks a name private,
and renaming or removing a `_`-prefixed alias is not by itself a breaking
change. The public contract still governs transitive effects. See
[Private JSDoc typedefs](./fsc/README.md#private-jsdoc-typedefs) for the
full rule and examples.
Name private types and private runtime constants with a leading `_` — `_Type`,
`_framingKeywords`. The underscore, not the emitted `.d.ts`, is what marks a
name private: renaming or removing one is not by itself a breaking change,
though the public contract still governs transitive effects. Where a private
type is *written* is [Private types](#private-types) below;
[Private types](./fsc/README.md#private-types) in `fjs/fsc/README.md` has the
breaking-change examples.

Use `@typedef` for a named type and `@template` for its type parameters. A
constraint goes in braces before the parameter name:
Expand Down Expand Up @@ -355,6 +355,88 @@ TypeScript one, but the public type contract must not become weaker for being
written in JavaScript. Types authored in `types.ts` use ordinary TypeScript
syntax and declaration emit.

#### Private types

Authored `.mjs` files carry **no file-scope JSDoc `@typedef`** — this rule is
repository-wide, not `fjs/`-specific, and holds for `module.f.mjs`,
`proof.f.mjs`, descriptive companions such as `testlib.f.mjs`, and host `.mjs`
alike. Declaration emit turns a file-scope typedef into an exported type alias,
so writing one publishes it whether or not that was the intent. The rule holds
for everything you write; the files that predate it are being migrated under
[`fjs/todo/separate-private-types.md`](./todo/separate-private-types.md), so
finding one is not a licence to add another.

A typedef **inside a function** is unaffected and is the right tool for a
compile-time proof, which often needs a lexical or downstream runtime value:

```js
const signatures = () => {
/** @typedef {Assert<Equal<ReturnType<typeof step<...>>, Effect<...>>>} _Step */
/** @typedef {Assert<Equal<ReturnType<typeof catchStep<...>>, Effect<...>>>} _CatchStep */
}
```

Everything else moves out of the implementation, by who needs it:

- `types.ts` holds the **public declaration closure**: the public types, plus
every private `_` helper a shipped public declaration reaches — including the
declaration of an exported runtime function. If `find`'s emitted declaration
names `_SortedArray<T>`, then `_SortedArray` belongs in `types.ts` or is
inlined; moving it to an unshipped module would leave the public declaration
incomplete. `types.ts` never depends on `private.ts`.
- `private.ts` is an **optional** sibling for implementation-private types
outside that closure. Use it where separating them makes the design cleaner,
not mechanically for every `_` name; a module with one local alias is usually
clearer with it inlined. It is authored type-only TypeScript like `types.ts`,
reached from JavaScript through JSDoc `@import { _X } from './private.ts'`.

Within one module directory, preserve the dependency direction for whichever of
these roles exist — the arrow points from dependency to dependent:

```text
types.ts <- private.ts <- module.f.mjs <- proof.f.mjs <- module.mjs <- proof.mjs
```

This is a layering guide, not a requirement that every file exists. Move
verification downstream rather than implementation upstream: an
`Assert<Equal<ReturnType<typeof …>, …>>` that checks `module.f.mjs` belongs in a
proof function in `proof.f.mjs`, not in `types.ts` where it would reverse the
arrow. Recursive RTTI whose annotation depends on the module's own public types
stays in `module.f.mjs` for the same reason.

A subordinate `meta/module.f.mjs` is the other optional tool: an ordinary
lower-level module for declarative constants both TypeScript and the runtime
read — RTTI/schema constants, `as const` literal data, lookup tables whose
literal shape defines a type. `meta` is *metaprogramming*, not a file role: it
is discovered, tested, and covered as the `module.f.mjs` it is, with no
metadata-specific tooling rule. A constant exported from it only for
sibling-module linkage keeps its `_`, since exportability is linkage, not API
status:

```js
// meta/module.f.mjs
export const _framingKeywords =
/** @type {const} */ (['import', 'const', 'export', 'default', 'from'])
```

Do not create either file because a `_` name or a runtime value exists. Ordinary
implementation functions stay in `module.f.mjs`.

Moving an existing public type out of an `.mjs` declaration surface, or a public
runtime constant into `meta/module.f.mjs`, changes an import path: treat it as an
intentional breaking change — update every importer and the changelog, and add
no compatibility re-export.

`private.ts` stays in the normal TypeScript program so source consumers are
checked, so declaration emit produces a `private.d.ts`. That file is not
shipped: `fjs/ci/prepack.mjs` runs as the final `prepack` step, after
declaration emit and the round-trip check, and deletes it before the package
file list is read. Emitted declarations are never text-postprocessed — TypeScript
may keep the source's `/** @import { _X } from './private.ts' */` comment, which
is a comment in a `.d.ts` and no dependency at all — so the same step checks the
*semantic* dependency instead, failing packaging if any shipped declaration
actually imports a private module.

#### Prefer inference

Let TypeScript infer the type of private constants, local variables, and return
Expand Down Expand Up @@ -963,10 +1045,14 @@ repository-owned dependencies follow these source rules:
- `.f.mjs` is authored FunctionalScript implementation/proof source, and its
relative runtime imports target `.f.mjs`;
- `types.ts` is authored type-only TypeScript source and carries no runtime
implementation;
implementation; an optional `private.ts` beside it is the same kind of file
for the private types no public declaration reaches
([§3.2](#private-types));
- `.f.mjs` — and later `.f.js` — consumes `types.ts` through JSDoc `@import`,
and TypeScript consumes it through `import type`, both always naming the real
`types.ts` file;
- no authored `.mjs` carries a file-scope JSDoc `@typedef`; a typedef inside a
function is fine;
- a declaration-only module belongs in `types.ts` rather than acquiring an
artificial runtime representation;
- never add a runtime import/export or runtime value solely to represent a
Expand Down
13 changes: 13 additions & 0 deletions fjs/ci/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,19 @@ export type Setup = {
`nodeExtra` receives the target OS so callers can conditionally add OS-specific steps.
Rust steps are included automatically when `Cargo.toml` is present; no flag is needed.

## Packaging

`prepack.mjs` is not part of the workflow generator: it is the last step of the
package's own `prepack`, run by `npm pack` and `npm publish` after declaration
emit and the round-trip check. It deletes every `private.d.ts` that declaration
emit produced from an authored `private.ts` — implementation-private types are
checked with the rest of the program but never shipped — and then fails
packaging if any remaining declaration still *imports* a private module. That
check reads static module specifiers as tokens, so a JSDoc `@import` comment
TypeScript kept in a declaration is correctly read as a comment and nothing in
the emitted text is rewritten. The rule it enforces is "Private types" in
[`fjs/AGENTS.md`](../AGENTS.md#private-types).

## Related

- [`packed-consumer-validation.md`](./packed-consumer-validation.md) — manual
Expand Down
89 changes: 89 additions & 0 deletions fjs/ci/prepack.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
/**
* The final `prepack` step: drops the declarations generated for authored
* `private.ts` modules, then proves that nothing left for packaging depends on
* one.
*
* A `private.ts` holds implementation-private types that are outside the public
* declaration closure (see `fjs/AGENTS.md`), so it stays in the TypeScript
* program — source consumers are checked — while its `private.d.ts` is never
* shipped. Declaration emit runs first, this runs last, and the package file
* list is read after both.
*
* The dependency check is semantic, not textual: emitted declarations may keep
* a source JSDoc `@import { _X } from './private.ts'` comment, which is a
* comment in a `.d.ts` and no dependency at all. `specifiers` reads static
* module specifiers as tokens, so it sees the `import`/`export` statements and
* not what a comment says.
*/

import { readdir, readFile, rm } from 'node:fs/promises'
import { relative } from 'node:path'
import { fileURLToPath } from 'node:url'

import { local, specifiers } from '../website/browser-source.mjs'

const sourceRoot = new URL('../../', import.meta.url)

/** @type {(url: URL) => string} */
const repoPath = url => relative(fileURLToPath(sourceRoot), fileURLToPath(url))

/** @type {(directory: URL) => Promise<readonly URL[]>} */
const files = async directory => {
const entries = await readdir(directory, { withFileTypes: true })
return (await Promise.all(entries.map(entry => {
if (entry.name.startsWith('.') || entry.name === 'node_modules' || entry.name === 'target') { return [] }
const url = new URL(entry.isDirectory() ? `${entry.name}/` : entry.name, directory)
return entry.isDirectory() ? files(url) : [url]
}))).flat()
}

/** @type {(url: URL, name: string) => boolean} */
const named = (url, name) => url.pathname.endsWith(`/${name}`)

/**
* Whether a static module specifier names a private type module. Only a
* repository-relative one can: a bare specifier names a package, and a package
* has no private module to reach.
*
* @type {(specifier: string) => boolean}
*/
const privateSpecifier = specifier => {
if (!local(specifier)) { return false }
const name = specifier.slice(specifier.lastIndexOf('/') + 1)
return name === 'private.ts' || name === 'private.js'
|| name === 'private.mjs' || name === 'private.d.ts'
}

const all = await files(sourceRoot)

// Only a declaration with an authored `private.ts` beside it was generated from
// one; anything else named `private.d.ts` is not this step's to delete.
const authored = new Set(all.filter(url => named(url, 'private.ts')).map(url => url.href))
const generated = all.filter(url =>
named(url, 'private.d.ts') && authored.has(new URL('private.ts', url).href))

await Promise.all(generated.map(url => rm(url)))
for (const url of generated) {
console.log(`removed ${repoPath(url)}`)
}

const removed = new Set(generated.map(url => url.href))
const declarations = all.filter(url =>
(url.pathname.endsWith('.d.ts') || url.pathname.endsWith('.d.mts')) && !removed.has(url.href))

const dependents = (await Promise.all(declarations.map(async url =>
specifiers(await readFile(url, 'utf8'))
.filter(privateSpecifier)
.map(specifier => /** @type {const} */ ([url, specifier]))
))).flat()

for (const [url, specifier] of dependents) {
console.error(`${repoPath(url)} depends on the unshipped private type module ${specifier}`)
}

if (dependents.length === 0) {
console.log(`private type modules removed: ${generated.length}; declarations checked: ${declarations.length}`)
} else {
console.error('move the types a public declaration needs into types.ts, or inline them')
process.exitCode = 1
}
20 changes: 9 additions & 11 deletions fjs/ci/todo/f-mjs-package-support.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,12 +170,11 @@ must not turn `_`-prefixed declaration artifacts into supported API merely
because TypeScript emitted them.

Types intentionally moved to `types.ts` use ordinary TypeScript syntax and do
not need the JSDoc-emission workaround merely to remain expressible. The eventual
replacement for private JSDoc typedefs is still `@internal` plus `stripInternal`,
blocked on
[microsoft/TypeScript#46407](https://github.com/microsoft/TypeScript/issues/46407)
and tracked in
[`todo/blocked/jsdoc-typedef-strip-internal.md`](../../../todo/blocked/jsdoc-typedef-strip-internal.md).
not need the JSDoc-emission workaround merely to remain expressible. That
workaround is being retired outright rather than waiting for `@internal` plus
`stripInternal`: no authored `.mjs` carries a file-scope `@typedef`, so nothing
is left for declaration emit to leak. See
[`fjs/todo/separate-private-types.md`](../../todo/separate-private-types.md).

Package selection does not need to distinguish every authored `.mjs` by public
API status during this transition. Incidental authored files such as
Expand Down Expand Up @@ -327,11 +326,10 @@ not, and the pipeline is simplified accordingly.
two-pass `prepack`.
- [`todo/migrate-typescript-to-mjs.md`](../../../todo/migrate-typescript-to-mjs.md)
— repository-wide stage-1 implementation source migration.
- [`todo/blocked/jsdoc-typedef-strip-internal.md`](../../../todo/blocked/jsdoc-typedef-strip-internal.md)
— replace the temporary `_` convention with `@internal` when declaration emit
supports it.
- [microsoft/TypeScript#46407](https://github.com/microsoft/TypeScript/issues/46407)
— upstream blocker for stripping private JSDoc typedefs.
- [`fjs/todo/separate-private-types.md`](../../todo/separate-private-types.md)
— keep private types out of public declarations; the final `prepack` step that
drops generated `private.d.ts` and checks the packed declarations for a
semantic dependency on one.
- [`publishing-packages.md`](./publishing-packages.md) — broader package roadmap.
- [`f-js-package-support.md`](./f-js-package-support.md) — stage-2 authored
`.f.js` package prerequisite.
Expand Down
22 changes: 7 additions & 15 deletions fjs/djs/tokenizer/module.f.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,13 @@
* @import { StateScan } from '../../types/function/operator/types.ts'
* @import { List } from '../../types/list/types.ts'
* @import { DjsToken, DjsTokenWithMetadata } from './types.ts'
* @import {
* _DjsScanState,
* _FlatToken,
* _StringDecodeState,
* _Token,
* _TokenScanState,
* } from './private.ts'
* @import { TriviaKind } from '../../js/tokenizer/types.ts'
* @import { Nullable } from '../../types/nullable/types.ts'
*/
Expand Down Expand Up @@ -314,13 +321,6 @@ const metadataScan = (cp, metadata) => [[[cp, metadata]], advanceMetadata(cp)(me
/** @type {(path: string) => (cp: readonly number[]) => readonly CodePointMeta<TokenMetadata>[]} */
const codePointsWithMetadata = path => cp => toArray(flat(stateScan(metadataScan)({ path, line: 1, column: 1 })(cp)))

// tag, the metadata of the token's first code point, and its code points.
/** @typedef {[string, TokenMetadata, readonly number[]]} _Token */

/** @typedef {string | CodePointMeta<TokenMetadata>} _FlatToken */

/** @typedef {[string, TokenMetadata | null, List<number>]} _TokenScanState */

/**
* The grammar tag of a trivia code point, as the kind `mergeTrivia` speaks in;
* `null` for every other tag.
Expand Down Expand Up @@ -396,12 +396,6 @@ const filterFunc = tk => {
*/
const unwrapHexDigitValue = mapUnwrap(hexDigitValue)

/** @typedef {
* | { readonly kind: 'normal' }
* | { readonly kind: 'escape' }
* | { readonly kind: 'unicode', readonly acc: number, readonly count: number }
* } _StringDecodeState */

/** @type {StateScan<number, _StringDecodeState, List<number>>} */
const stringDecodeScan = (cp, state) => {
switch (state.kind) {
Expand Down Expand Up @@ -592,8 +586,6 @@ export const tokenizeJs = input => path => {
return withMetadata([{ token: { kind: 'eof' }, metadata: finalMetadata }])
}

/** @typedef {{ readonly kind: 'def' | '-' }} _DjsScanState */

/** @type {(input: JsToken) => List<DjsToken>} */
const mapDjsToken = input => {
switch (input.kind) {
Expand Down
35 changes: 35 additions & 0 deletions fjs/djs/tokenizer/private.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
/**
* Implementation-private types for `fjs/djs/tokenizer/module.f.mjs`.
*
* Nothing here belongs to the public declaration closure: no exported
* declaration of the module names any of these types, so keeping them out of
* `./types.ts` keeps them out of the shipped declarations too. They are
* exported only so the implementation can `@import` them — the leading `_` is
* what marks them private, and renaming or removing one is not a breaking
* change.
*/

import type { CodePointMeta } from '../../bnf/descent/types.ts'
import type { TokenMetadata } from '../../js/tokenizer/types.ts'
import type { List } from '../../types/list/types.ts'

/**
* A token as `scanFunc` emits it: its tag, the metadata of its first code
* point, and its code points.
*/
export type _Token = [string, TokenMetadata, readonly number[]]

/** Either a bare grammar tag or one code point paired with its metadata. */
export type _FlatToken = string | CodePointMeta<TokenMetadata>

/** The token `scanFunc` is still accumulating: `null` metadata until its first code point arrives. */
export type _TokenScanState = [string, TokenMetadata | null, List<number>]

/** Where `stringDecodeScan` is inside a string literal's escape sequences. */
export type _StringDecodeState =
| { readonly kind: 'normal' }
| { readonly kind: 'escape' }
| { readonly kind: 'unicode', readonly acc: number, readonly count: number }

/** Whether `scanDjsToken` has an unconsumed `-` to fold into the next number. */
export type _DjsScanState = { readonly kind: 'def' | '-' }
Loading
Loading