perf: memoize URL and route resolution in link codecs - #1951
Conversation
More templates
@orpc/ai-sdk
@orpc/arktype
@orpc/bun
@orpc/client
@orpc/cloudflare
@orpc/contract
@orpc/experimental-effect
@orpc/evlog
@orpc/hibernation
@orpc/json-schema
@orpc/experimental-msw
@orpc/nest
@orpc/next
@orpc/node
@orpc/openapi
@orpc/opentelemetry
@orpc/pinia-colada
@orpc/pino
@orpc/publisher
@orpc/ratelimit
@orpc/server
@orpc/shared
@orpc/swr
@orpc/tanstack-query
@orpc/trpc
@orpc/valibot
@orpc/zod
commit: |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Merging this PR will improve performance by 10.97%
|
| Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|
| ⚡ | octet stream |
912.2 µs | 822 µs | +10.97% |
| 🆕 | encodeInput GET with 50 query params |
N/A | 760.5 µs | N/A |
| 🆕 | encodeInput with 5 dynamic path params |
N/A | 235.4 µs | N/A |
| 🆕 | encodeInput at path depth 10 |
N/A | 43.4 µs | N/A |
| 🆕 | encodeInput at path depth 20 |
N/A | 45.9 µs | N/A |
| 🆕 | encodeInput on a 1000-procedure router |
N/A | 42.6 µs | N/A |
| 🆕 | encodeInput at path depth 10 |
N/A | 37.9 µs | N/A |
| 🆕 | encodeInput at path depth 20 |
N/A | 39 µs | N/A |
| 🆕 | encodeInput at path depth 20 (fresh array per call) |
N/A | 109.5 µs | N/A |
Tip
Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.
Comparing spa5k:perf/client-codec-memoization (06effe7) with main (2991776)
There was a problem hiding this comment.
ℹ️ No critical issues — one minor suggestion inline.
This PR is stacked on #1950; I reviewed only the last two commits (41e3985 perf change and f2a3e54 bench fold). The memoization is well-engineered and well-tested: the WeakMap identity cache in pathToHttpPath, the single-slot base-url parse caches, and the OpenAPILinkCodec route memoization (with in-flight dedupe via routePromises cleared in finally, so failed resolves are retried) are all correct. The dotted-vs-nested cache-key test genuinely pins the stringifyJSON keying, and the lazy-load tests meaningfully exercise the dedupe/not-cached semantics. I ran the affected suites (191 tests) — all pass. The untraced StandardLink fast path preserves behavior: without a configured tracer neither the outer span nor the three inner runWithSpan wrappers were ever observable, so skipping them changes nothing but cost. The one item worth flagging is the interceptors snapshot in createORPCClient, inlined below.
Reviewed changes
- Added a module-level
WeakMapidentity cache topathToHttpPathso memoized path arrays skip re-encoding. - Added single-slot base-URL parse caches to the RPC and OpenAPI link codecs, keyed by the resolved URL value.
- Memoized
OpenAPILinkCodecroute resolution (meta, method, structures, merged pathname, dynamic params), shared betweenencodeInputanddecodeResponse, resolving once instead of twice with in-flight dedupe and no caching of failed lazy loads. - Added an untraced fast path to
StandardLink.callthat skips span-name construction and therunWithSpanwrappers when no tracer is configured. - Hoisted the client
interceptorsarray out of the per-call closure increateClient. - Folded codec depth/params/query scenarios into the existing bench files.
DeepSeek Flash (free via Pullfrog for OSS) | 𝕏
| link: ClientLink<InferClientContext<T>>, | ||
| { path = [], ...options }: NoInfer<ORPCClientOptions<T>> = {}, | ||
| ): T { | ||
| const clientInterceptors: ORPCClientInterceptor<InferClientContext<T>, unknown, unknown, InferClientError<T>>[] = [ |
There was a problem hiding this comment.
This is a silent semantic change to a public API: interceptors and scoped.interceptors are now snapshotted at client creation, so an in-place mutation of the array afterwards (e.g. pushing a later-added interceptor) is no longer observed by subsequent calls. The PR body documents this, but nothing in the code or the option's JSDoc telegraphs it, and the inner ...toArray(options.interceptors) reads previously made the live-read semantics reasonably discoverable. Worth a short comment here noting the snapshot — nothing blocking, the change itself is a good perf win.
There was a problem hiding this comment.
ℹ️ Re-review of the new commit (
73fc524"fix: preserve cache and promise semantics") — no new issues found.
This run re-reviews 73fc524, which sits on top of the previously-reviewed 41e3985 + f2a3e54. It hardens the memoization from the first review, and each fix is genuinely pinned by a test that fails without the change. I verified the affected suites — http.test.ts, openapi-link-codec.test.ts, and link.test.ts — 144 tests pass. The prior minor thread on the client.ts interceptors snapshot (a requested inline note) is unchanged by this commit and stays open below the approval gate; it is non-blocking.
Reviewed changes
- Changed
pathToHttpPathto store[path.slice(), httpPath]in itsWeakMapand validate on every hit (length +.every(===)), so reusing a mutated array can no longer return a stale encoded path. The new mutation-invalidation test locks this in. - Wrapped the untraced
StandardLink.callinterceptor invocation inPromise.resolve/Promise.rejectso a synchronously-throwing interceptor yields a rejected promise instead of a synchronous throw, matching the traced path and thePromisereturn type. New test asserts the promise rejects with the thrown error. - Deferred
OpenAPILinkCodec.computeRoutebehindPromise.resolve().then(...)and snapshotted the path, so a synchronously-throwing lazy loader self-cleans fromroutePromisesinfinally(never cached) and an in-flight lazy path array mutated mid-load cannot corrupt resolution. Two tests pin the not-cached (loads === 2) and stable-path behaviors.
ℹ️ Nitpicks
pathToHttpPathnow pays O(depth) for the length check +.everyon every cache hit — the previously O(1) fast path this PR optimizes. The trade-off is fair (still far cheaper thanencodeURIComponent+join), but since this is a perf PR it's worth re-running the depth benchmarks after this commit to confirm the memoized-array hot path didn't regress materially; trusting the documented stable-array contract is an alternative if it did.
Technical details
# Snapshot-validation cost on pathToHttpPath
## Affected sites
- packages/shared/src/http.ts:12-18 — per-hit length + `.every(===)` scan.
## Required outcome
- Confirm the added O(depth) snapshot check does not materially regress the memoized-array hot path measured in `benches/rpc-link-handler.bench.ts` / `openapi-link-handler.bench.ts`. If it does, prefer trusting the documented stable-array contract (mutation-miss) over validating every hit.
## Open questions for the human
- Is in-place mutation of a shared client path array a real scenario, or is the mutation guard theoretical? If theoretical, the snapshot may not be worth its per-hit cost.DeepSeek Flash (free via Pullfrog for OSS) | 𝕏
73ff524 to
92a40d2
Compare
- cache pathToHttpPath results by path identity (paths are stable arrays)
- single-slot cache for base URL parsing in RPC and OpenAPI link codecs
- memoize OpenAPILinkCodec route resolution (meta, path template, dynamic
params) with in-flight dedupe, keyed collision-free; was resolved twice
per call, once in encodeInput and once in decodeResponse
- untraced fast path in StandardLink.call: build span names and wrap
encode/send/decode with runWithSpan only when a tracer is configured
- hoist the invariant client interceptors array out of the per-call closure
Includes adversarial tests: dotted-vs-nested route key isolation (the
join('.') cache key collided), lazy contract resolution with failures
not cached, alternating per-call base urls, untraced StandardLink
round-trip, and pathToHttpPath cache consistency.
…sting files - rpc-link-handler: codec path-encoding depth (10/20, memoized paths) plus the fresh-array cache-miss worst case - openapi-link-handler: codec route-resolution depth (10/20), 1000-procedure router, 5 dynamic path params, 50-param query encoding
92a40d2 to
ce45443
Compare
| * Procedure paths are usually stable arrays reused across calls. Keep a | ||
| * snapshot so mutable arrays cannot return a stale encoded path. | ||
| */ | ||
| const HTTP_PATH_CACHE = new WeakMap<readonly string[], [readonly string[], `/${string}`]>() |
There was a problem hiding this comment.
Not worth to introducing a global WEAKMAP
| const baseUrl = await value(this.baseUrl, options, path, input) | ||
|
|
||
| const [pathname, search, hash] = parseStandardUrl(baseUrl) | ||
| const [pathname, search, hash] = this.parseBaseUrl(baseUrl) |
There was a problem hiding this comment.
parseStandardUrl is fast enough
| private readonly serializer: Exclude<OpenAPILinkCodecOptions<T>['serializer'], undefined> | ||
| private readonly customErrorResponseBodyDecoder: OpenAPILinkCodecOptions<T>['customErrorResponseBodyDecoder'] | ||
| private parsedBaseUrl: [StandardUrl, `/${string}`, `?${string}` | undefined, `#${string}` | undefined] | undefined | ||
| private readonly routes = new Map<string, ResolvedOpenAPIRoute>() |
There was a problem hiding this comment.
TO support this pattern, we shouldn't compute routes: https://orpc.dev/docs/contract/client-factory

What this PR does
The client made the same URL and route data again for each call. This PR caches that data. Profiling showed about 8.6 percent of each call in this work.
What changed
pathToHttpPathcaches by path array identity and validates a snapshot of its segments. Stable procedure paths hit the fast path. A mutated array is recomputed correctly.StandardLink.callreads the tracer config one time. When no tracer is set, the code skips the span names and the threerunWithSpanwrappers. Tracing behavior does not change when a tracer is set.How much faster
Test setup: Apple M4, Node v25.9.0, Vitest v4.1.11, tracing off. The base is current
main(29917766).The codec cost now stays almost flat when the path gets deeper. End-to-end timings vary more, so they are not used as headline results.
Known trade-off
The path cache uses array identity and validates a segment snapshot. A caller that makes a new array for each call does not hit the cache. That caller pays the WeakMap lookup on top of the encoding. At depth 20, a fresh array is about 23 percent slower than
main. Clients made withcreateORPCClientreuse path arrays. That case is 3.44 times faster.Benchmark scenarios
The codec scenarios live in the existing benchmark files. This PR adds no new files.
rpc-link-handler.bench.ts: codec path encoding at depth 10 and 20. One scenario uses a fresh array per call. This is the cache-miss worst case.openapi-link-handler.bench.ts: codec route resolution at depth 10 and 20, a router with 1000 procedures, 5 dynamic path params, and a GET query with 50 params.Behavior notes
interceptorsarray after creation are not seen by later calls. Before, the code read the array again for each call.Tests
mainand this branch.eslintand package diagnostics pass with no errors.