fjs/web: file an issue for the missing-index message - #1714

Avoid claiming a non-regular index is absent

When a directory-form URL resolves to an existing non-regular index.html (for example, a directory, FIFO, device, or socket), readBounded returns notRegular and fileResponse deliberately maps that to the same 404 as a missing path. Replacing every directory-form 404 with no index.html in … would therefore make a false claim; limiting the change to missing-file errors would instead disclose the distinction this proposal says to preserve. Specify neutral wording such as “no index page available” so the design covers both cases before implementation.

AGENTS.md reference: AGENTS.md:L47-L49

Useful? React with 👍 / 👎.

Keep ENOTDIR handling local to web

If the implementer chooses the proposed shared isNotFound change, an ENOTDIR from fileCas.list's access(storePrefix)—for example, when the configured store path or one of its components is a regular file—will be converted into an empty store. fjs/cas/module.f.mjs explicitly uses this predicate to treat only a genuinely absent .cas as empty while surfacing corruption and other failures, so broadening it would silently mask a malformed store. Specify the web-local check here, or concretely include auditing and updating every existing caller, rather than leaving this behavior-changing decision open.

AGENTS.md reference: AGENTS.md:L47-L49

Useful? React with 👍 / 👎.

Link the existing virtual-filesystem issue

This task is already tracked in fjs/effects/node/virtual/todo/lexical-path-resolution.md lines 27–42, including the exact ENOTDIR behavior and the requirement to make that operation-wide change in its own PR. Leaving the implementation task here creates duplicate owners and encourages bundling a cross-cutting virtual-runner change into the web fix; mark this issue as blocked by that existing issue (or use the partialRun injection pattern from fjs/web/proof.f.mjs lines 321–325) and retain only the web-specific mapping and proof here.

AGENTS.md reference: AGENTS.md:L10-L11

Useful? React with 👍 / 👎.

Define the widened Resolve result before implementation

The preferred option changes the exported Resolve API, but the issue never specifies the new success payload or how respond consumes it; the task even leaves the transport decision open. Because types.ts currently promises Result<string, Refusal> and callers unwrap a string, an implementer must invent a public API rather than follow a concrete design. Record the exact result shape and its handling before this issue is ready for implementation.

AGENTS.md reference: AGENTS.md:L47-L49

Useful? React with 👍 / 👎.

Carry directory-form metadata through hidden refusals

The preferred flow only carries the routing fact alongside a successfully resolved path, but resolve returns the hidden-path 404 at fjs/web/module.f.mjs:283 before computing isDirectory at line 284. Consequently, /.git/ can never receive this metadata and would remain not found while /no-such-dir/ gets the new index-specific sentence, contradicting the proposal to handle directory-form 404s consistently. Specify how the fact is attached to this refusal too, and include a hidden-directory proof.

AGENTS.md reference: AGENTS.md:L47-L49

Useful? React with 👍 / 👎.

Preserve ENOTDIR failures caused by the configured root

When the configured root is itself a regular file or has a non-directory ancestor (for example, fjs web README.md), stat('README.md/index.html') also returns ENOTDIR; a check inside fileResponse cannot distinguish that operator/configuration failure from /README.md/ descending through a file beneath a valid root. The proposed mapping would therefore turn every request to such a misconfigured server from the current 500 into a misleading 404. Specify a separate root validation or another way to retain 500 when the offending component is at or above the served root before implementing this task.

AGENTS.md reference: AGENTS.md:L47-L49

Useful? React with 👍 / 👎.

Confirmed and fixed in 08e106c — this is a genuine flaw in the design I committed to two rounds ago, not just an underspecified corner.

join('README.md', 'index.html') is README.md/index.html, so with a regular file as root every request stats a path descending through a file. Same errno, and nothing in fileResponse can separate it from /README.md/ under a good root. The blanket mapping would have answered 404 not found to every request against a misconfigured server — hiding the one case where the 500 was telling the operator something true.

Fixed by validating the root in main, before listen, the way an out-of-range port already is. Better than comparing the offending component against the root per request: no extra stat on the hot path, it fails at the moment the mistake was made rather than on some later visitor's request, and afterwards every ENOTDIR reaching fileResponse is client-caused by construction — which is exactly what the mapping assumes. It shares the replace-underneath window with stat-then-read and does not widen it, and that is now cross-linked.

One thing that fell out while checking: Windows already answers 404 to everything for such a root, silently. fjs web README.md starts happily there and 404s every request, because stat reports ENOENT rather than ENOTDIR — verified, after I first fooled myself with a stale server on a reused port and had to redo it against the serving README.md on … log line. So today the two platforms give two different misleading answers per request; the startup check makes both say the same true thing once.

Escape control characters before echoing URL paths

For a request such as /%1B%5B31m/, percentDecode turns the escape into a literal control character that the proposed response would reflect. text/plain and nosniff constrain browser interpretation only; terminal clients such as curl or wget can still pass ANSI/OSC sequences to the user's terminal, altering its state or triggering terminal-specific actions. Specify an escaped or percent-encoded representation rather than declaring these characters harmless, so this non-trivial issue has a safe concrete design.

AGENTS.md reference: AGENTS.md:L47-L49

Useful? React with 👍 / 👎.

Fixed in f306a7c, and you're right that "harmless" was wrong — I had reasoned about browsers and stopped there. text/plain and nosniff bound what a browser does with a body; curl and wget write it to a terminal, where the sequence /%1B%5B31m/ decodes to is acted on rather than displayed. A message quoting the client would hand attacker-chosen escapes to whoever ran the request.

Fixed by choosing a representation rather than declaring the characters safe: echo the percent-encoded spelling, never percentDecode's output.

What makes that more than a rule to remember is that the encoded form cannot carry the problem. A raw control byte never reaches this module — Node's parser answers 400 Bad Request on the request line before the listener runs, which I checked over a socket rather than assuming, since curl would not send one. So in the target such a character exists only as the printable text %1B. Echoing the target's path as received therefore needs no escaping pass and no list of dangerous characters — the version of this that cannot rot as the list ages. A normalized path is equally fine on the same terms provided it is re-encoded on the way out, so the /fjs/.. wording question stays open on its own merits rather than being decided by this.

Two things worth recording alongside it:

• Nothing echoes anything today. The current answer is the constant not found — confirmed against a live server for /%1B%5B31m/ — so this is a property to build in, not a bug to fix. The issue now says so, so a reader does not go looking for a live vulnerability.
• Added the proof task: /%1B%5B31m/ must echo %1B%5B31m and not the escape it names, so no answer this server writes can drive a terminal.

Gates: tsc clean, ci-update no diff. Docs-only.

Keep the ENOTDIR proof platform-independent

The proposal explicitly adds ENOTDIR to the virtual filesystem, and the existing web proofs execute respond through that virtual runner, so a Windows test run can exercise the distinction without relying on the host filesystem. Requiring “a host whose stat distinguishes them” may lead the implementation to skip this proof on Windows and leave the new response branch uncovered there; specify a platform-independent virtual/injected ENOTDIR proof, with any POSIX host check kept as separate integration validation.

AGENTS.md reference: fjs/AGENTS.md:L27-L35

Useful? React with 👍 / 👎.

Right, and the task contradicted this file's own plan — fixed in 08e106c.

The proposal already teaches the virtual file system to report ENOTDIR, and for exactly this reason: fjs/AGENTS.md §1.2 rejects an unreachable branch, so the error has to be producible without a host. fjs/web/proof.f.mjs already drives respond through virtual(...). So requiring "a host whose stat distinguishes them" would have made the branch uncovered on Windows while the file was simultaneously arranging for it to be coverable anywhere — my wording, carried over from the platform-split paragraph where it did belong.

The task is now virtual and unconditioned on the host. I kept a one-off POSIX check as a separate task rather than dropping it, and said why: the virtual file system models what a host does, and this is the issue where that model turned out to be wrong on two platforms at once — ENOTDIR on POSIX, ENOENT on Windows. A virtual proof pins the branch; it cannot notice the model drifting from either host. Keeping them as two tasks says which one answers which question.

Define a directory-capable root check

The proposed validation cannot distinguish a directory with the existing stat API: FileStat in fjs/effects/node/types.ts exposes only size and isFile, so isFile === false also covers FIFOs, devices, sockets, and the virtual runner's JsModule. Implementing this task therefore requires an unspecified API change or a different operation such as readdir, with different error and permission behavior. Specify that mechanism and its failure handling so the non-trivial issue has a concrete implementable design.

AGENTS.md reference: AGENTS.md:L47-L49

Useful? React with 👍 / 👎.

Correct, and it lands squarely on the fix from the previous commit — that root check was not implementable as written. Specified in 14e9fdf.

FileStat is { size, isFile }, so isFile === false is not "is a directory": it covers a FIFO, a device, a socket, and the virtual runner's JsModule, whose _Entity is readonly Vec[] | Dir | JsModule. Serving any of those as a root is the same operator error as serving a regular file, so the check has to name what it wants rather than infer it from a negation.

Specified: add isDirectory to FileStat, in the node runner and the virtual one together, and reject a root that is not one — including the case where both flags are false. Failure handling covers two cases and uses the same errorExit for both: stat failing outright (a root that does not exist), and a root that exists and is not a directory. Both are the command line being wrong, which main already reports that way for an out-of-range port.

Not readdir(root), which is the obvious alternative and needs no new API — I recorded why, since a later reader will reach for it. It answers a different question: a directory may be traversable without being listable, so mode --x permits opening a known path under it while readdir fails EACCES, and a root this server can serve perfectly well would be refused at startup. Reading a whole directory only to discard it is the smaller objection.

The task list now separates the API change from its use, and FileStat is cross-referenced under Related.

Revalidate the root before mapping ENOTDIR to 404

When the served directory is renamed or replaced with a regular file after startup, this one-time check no longer establishes the stated invariant: every subsequent stat(root/...) can return ENOTDIR, which fileResponse would misclassify as a client-caused 404 instead of the operator-facing 500. Unlike the request-local stat/read race, this window lasts for the server's remaining lifetime; recheck the root when handling ENOTDIR or otherwise preserve the error in this case. The fresh scenario beyond the earlier configured-root comment is replacement after the startup validation has succeeded.

AGENTS.md reference: AGENTS.md:L47-L49

Useful? React with 👍 / 👎.

Right, and the part worth conceding explicitly is the sentence you're contradicting: the previous commit said the startup check "leaves the same replace-underneath window as stat-then-read, and no more". That was wrong, and not by a little. stat-then-read races two calls microseconds apart; this window opens once and stays open for the life of the process, turning the operator's mistake into a lie told to every visitor. Fixed in 84f3386, and the file now says the earlier claim was wrong rather than quietly dropping it.

The mapping re-checks the root on ENOTDIR and answers 404 only if it is still a directory, otherwise 500 — which is again the true answer. Your placement is the right one: the cost sits on the rare path, and the serving path is untouched, so it does not reintroduce the per-request stat that made the component-comparison alternative unattractive.

What remains after that is a genuine race between the re-check and the stat that produced the error — but it is request-local, so it degrades to exactly the class stat-then-read already covers: a wrong status in a vanishing window rather than a wrong status forever. The Related entry now says that instead of claiming equivalence.

The startup check stays, for the reason it was added: it turns a mistyped root into immediate feedback rather than a 500 that waits for a visitor to discover. The two do different jobs — one reports the mistake, the other keeps the mapping honest afterwards — and the task list now separates them.

Not reproducible on this host, for the standing reason: Windows reports ENOENT rather than ENOTDIR, so the misclassification never arises there. Reasoned from the code, not measured.

Recheck vanished roots on ENOENT

When the served directory is renamed or deleted after startup, stat(root/…) returns ENOENT, not ENOTDIR; the existing fileResponse maps that directly to 404, while the proposed root recheck runs only for ENOTDIR. The server would therefore still report a permanent operator failure as client-caused absence. The fresh evidence beyond the earlier revalidation comment is this distinct errno for root disappearance, so the design must also validate the root before accepting ENOENT as a 404 (or otherwise retain a stable root reference).

AGENTS.md reference: AGENTS.md:L47-L49

Useful? React with 👍 / 👎.

Confirmed: a deleted or renamed root fails ENOENT, takes the existing isNotFound branch, and reports the same permanent operator failure as a client-caused 404 — the ENOTDIR re-check does not see it. Addressed in 3496001, though by scoping rather than by extending the re-check, and I want the reasoning on the record since it is a decline.

Validating the root before accepting ENOENT is declined on cost. An ENOENT 404 is the most common answer a static server gives; adding a root stat there puts a second syscall on the hot path to improve a diagnostic. The ENOTDIR re-check pays nothing on that path precisely because ENOTDIR is rare. The file now says this is a trade rather than a principle, so nobody later reads the asymmetry as a claim that the two cases differ in kind.

One thing worth being precise about, which sharpens what is actually lost: 404 is not false in either case. With the root gone or replaced by a file, nothing beneath it exists, so "not found" is a true statement about the requested resource. What the 500 buys is telling the operator which mistake they made. So the asymmetry costs diagnostic reach, not correctness — that framing is now in the file, and it is the honest version of the argument I was making two rounds ago for ENOTDIR.

The design that answers both, with no re-check at all, is holding the root open and resolving beneath the handle, so it cannot be swapped underneath the server at any point. That is exactly the effect stat-then-read is blocked on, and it is now cross-referenced as a second reason to want it — which is a better outcome than this issue growing a third mitigation.

Escape raw controls for direct respond calls

When exported respond is invoked directly, through the virtual runner, or by a non-Node runner with a URL such as '/\u001b[31m/', Node's request parser is never involved; IncomingMessage.url is typed as an unrestricted string. Echoing target.path as received would therefore place the terminal control sequence in the response despite this claim. The fresh evidence beyond the earlier control-character thread is this supported direct-call path, so raw controls still need escaping or an enforced input invariant even while percent-encoded spelling is preserved.

AGENTS.md reference: AGENTS.md:L47-L49

Useful? React with 👍 / 👎.

Correct — the invariant was one caller's property, not respond's. Fixed in 3496001.

respond is exported, IncomingMessage.url is an unrestricted string, and fjs/web/proof.f.mjs's answer already calls respond('.')(request(method, url)) straight through the virtual runner with whatever URL a case names. So "echoing the target as received needs no escaping pass" held only behind Node's parser, and put the invariant outside the function that depends on it — the arrangement that quietly stops holding when someone adds a caller.

The message now percent-encodes what it echoes rather than trusting what it was handed. That is total and still needs no list of dangerous characters: encode everything outside the safe set, and %1B survives as %1B whether it arrived encoded or raw. The parser fact stays in the file, demoted to what it is — true of the HTTP path, and not a licence to skip the pass.

It also has a side benefit worth noting: with an encoding pass on the way out, the raw-versus-normalized choice becomes free of safety consequences, so that question stays open purely on wording grounds, which is where it belonged.

The proof task now covers the direct-call path: a raw control character passed to respond the way proof.f.mjs already calls it, alongside the %1B%5B31m case. Without that, the proof would only ever exercise inputs the parser had already filtered — which is precisely how this would have regressed unnoticed.

Normalize the empty root before validating it

When fjs web '' is used, the supported empty-root argument is normalized by served to .; the existing emptyRoot proof covers this behavior. If either proposed root check applies stat to the raw argument as written here, stat('') returns ENOENT, so startup rejects a valid invocation and the request-time recheck misclassifies ENOTDIR beneath respond(''). Specify that both checks use served(root) (or an equivalently normalized root) before this design is implemented.

AGENTS.md reference: AGENTS.md:L47-L49

Useful? React with 👍 / 👎.

Confirmed and fixed in 2b01b11. Both checks said "stat the root", and the raw argument is not always the served directory.

served maps '' to ., and fjs web '' is supported with two emptyRoot proofs behind it — one on resolve(''), one driving main with args: [''] and asserting it prints serving . on http://127.0.0.1:8080/. statSync('') is ENOENT, verified, so the startup check would have rejected a valid invocation and the re-check would have misjudged every ENOTDIR under respond('').

Both now say served(root) explicitly, and the task points at the proof that pins it — the existing emptyRoot case would have caught the startup half, but only for an implementer who ran it before wondering why, and the re-check half it would not have caught at all.

Worth noting this is the second time served's normalization has mattered to a design that did not mention it: its own JSDoc records that fjs web '' would otherwise serve /etc/passwd, because join('', 'etc') reads as absolute. An empty root is a value the argument default cannot catch, and evidently one a proposal has to name rather than assume away.

Specify how validated escapes survive encoding

For an already encoded request such as /%1B%5B31m/, ordinary URI encoders encode the percent signs and produce /%251B%255B31m/, contradicting both this claim and the required proof at lines 155–159; preserving % indiscriminately is not a complete encoding rule either. The fresh problem after adding the encoding pass is that the design still needs to define the safe set and explicitly say how validated %XX triplets are preserved while raw controls are encoded, rather than leaving the implementer to invent the security-sensitive transformation.

AGENTS.md reference: AGENTS.md:L47-L49

Useful? React with 👍 / 👎.

Right — "encode everything outside the safe set" left the security-sensitive half to the implementer, and the obvious reading of it contradicts this file's own proof task. Fixed in 6529b19.

Your counterexample is the decisive one: a general-purpose URI encoder escapes the % as well, so /%1B%5B31m/ comes back /%251B%255B31m/ — unreadable, and not what the proof asks for. And preserving every % is not a rule, since a lone one is not an escape. The pass is now three cases:

• a %XX triplet passes through verbatim — isEscape, the predicate percentDecode already uses, so the rule reuses the module's own notion rather than inventing a second one;
• unreserved characters and / pass through;
• every other byte is percent-encoded, which is what catches a raw control character.

Preserving triplets is total rather than a special case, which is the part that makes this safe to state so briefly: by the time the message is built, resolve has accepted the path and percentDecode rejects a malformed escape, so every % present already begins a valid triplet. The direct-call path you raised earlier does not weaken that — it takes the same route through resolve, so it cannot smuggle a lone % past; what it can carry is a raw control byte, and that is exactly what the third rule encodes. So the two findings close against each other rather than needing separate guards.

Put the root recheck in an implementable effect chain

The fresh issue introduced by the revalidation requirement is that the current fileResponse is a synchronous (path) => (Result) => ServerResponse mapper with neither root nor the ability to execute stat, while respond invokes it inside resultMapStep (fjs/web/module.f.mjs:483-535). Thus the later instruction to perform this check “from fileResponse” cannot be implemented as written without redesigning the effect chain; specify whether the ENOTDIR branch moves into respond, or exactly how fileResponse becomes effectful and its nested effect is flattened.

AGENTS.md reference: AGENTS.md:L47-L49

Useful? React with 👍 / 👎.

Confirmed against the code, and fixed in 6529b19 — "re-check from fileResponse" was not implementable as written.

fileResponse is (path) => (Result) => ServerResponse: pure, and holding neither root nor any way to run a stat. I had carried "keep it local to fileResponse" forward from the round where the question was which predicate to widen, where it was the right answer, without re-checking that it still fit once the decision needed an effect.

The branch goes in respond, which has both root and the effect chain. The change is smaller than redesigning anything: respond already ends in resultMapStep(bytes, r => ok(fileResponse(path)(r))), and resultMapStep is by definition resultStep over a pure function. Dropping to resultStep is the whole edit — the ENOTDIR case becomes a step(stat(served(root)), …) deciding 404 or 500, every other case stays fileResponse(path)(r) exactly as today, and fileResponse keeps its signature and its purity.

That also preserves what the original "local, not isNotFound" decision was actually protecting: the reading stays inside fjs/web, and the two fjs/cas callers are untouched. Only the placement within the module moves.