perf(server): reduce middleware pipeline overhead - #1950
Conversation
- precompute per-procedure execution plans (schema slices, flags) in a WeakMap - skip async runWithSpan wrappers and eager span names when tracing is disabled - skip context spreads when middleware passes an empty context (string- and symbol-keyed aware) - skip unlazy promise allocation for non-lazy procedures - cache per-procedure error constructor map and reconcileError closure - skip the prototype-chain-walking instanceof ORPCError check for primitive outputs
- procedure-call: middleware scaling (10/100 passthrough, 10/50 context-adding, 10/50 with stacked input schemas) - rpc-link-handler: tiny-payload fixed overhead scenarios (plain, middlewares x3, error thrown, 404) - openapi-link-handler: tiny dynamic-path GET/POST scenarios - rpc-serializer: pure-JSON payload scenarios
|
Stacked follow-up: #1951 (client codec memoization). Review it after this one. Its last three commits are the new work. The first four commits belong to this PR. |
More templates
@orpc/ai-sdk
@orpc/arktype
@orpc/bun
@orpc/client
@orpc/cloudflare
@orpc/contract
@orpc/experimental-effect
@orpc/evlog
@orpc/hibernation
@orpc/json-schema
@orpc/experimental-msw
@orpc/nest
@orpc/next
@orpc/node
@orpc/openapi
@orpc/opentelemetry
@orpc/pinia-colada
@orpc/pino
@orpc/publisher
@orpc/ratelimit
@orpc/server
@orpc/shared
@orpc/swr
@orpc/tanstack-query
@orpc/trpc
@orpc/valibot
@orpc/zod
commit: |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Merging this PR will improve performance by 12.78%
Performance Changes
Tip Curious why this is faster? Comment Comparing |
There was a problem hiding this comment.
✅ No new issues found.
I reviewed the whole diff and traced the pipeline logic line-by-line. The change is a clean, well-scoped refactor with no functional regressions.
Reviewed changes
- Per-procedure caching: Two
WeakMaps keyed on the procedure —getProcedureCallArtifactscaches the error-constructor map (which allocates aProxy) andreconcileError;getExecutionPlanprecomputes the per-level schema slice boundaries, the stacked-object snapshot, and validation flags once instead of per request. - Untraced fast path:
getOpenTelemetryConfig()?.traceris read once per call; when no tracer is configured the middleware/validators/handler run directly, skipping therunWithSpanwrapper. Span names and attribute side effects are unchanged when traced. - Conditional context copy:
hasEnumerablePropertiesdecides whether to spread-merge or reuse the same context object when a middleware adds no data; symbol keys are handled. Syntactic equivalence to the old always-copy path holds for the merged content. - Micro-opts: primitives skip the
instanceof ORPCErrorbranch, non-lazy procedures skip theunlazyresolved-promise, andnextcontext extraction readsrest[0]?.contextdirectly. - Tests & benches: new tests meaningfully exercise the empty-
{}/next(undefined)/symbol-key context cases, thecontext: undefinedresult, primitive outputs, and the exact traced span-name sequence. Benches extend existing files. Full test file passes (44/44).
ℹ️ Middleware context now aliases across pass-through frames
This is a documented, deliberate tradeoff, not a bug — but it's slightly subtler than "the next level gets the same object". Previously every next() performed { ...context, ...nextContext }, which gave each middleware frame an isolated copy. On the new path, when a middleware passes no context data the identical object reference flows down, so an in-place mutation of options.context inside a middleware is now visible to ancestor frames that previously received their own copy. The content-equality contract is intact; only in-place mutation aliasing changes. Given use() snapshots schemas into immutable OrderedMiddleware entries and contexts are conventionally read-only, this looks safe — just flagging it as the one caveat reviewers should keep in mind rather than a change to request.
Technical details
# Context object aliasing across pass-through middleware frames
## Affected sites
- packages/server/src/procedure-client.ts:379-389 (the middleware `next` callback: `hasEnumerableProperties` decides merge-copy vs. reuse)
- packages/server/src/procedure-client.ts:406-409 (result context reuse)
## Required outcome
- No change required. Content-merge semantics are preserved; only object identity differs from the previous always-copy behavior.
- Document, if not already, that middlewares must not mutate `options.context` in place and rely on isolation, since passthrough frames now share the same object.DeepSeek Flash (free via Pullfrog for OSS) | 𝕏
|
I'm still not convinced by this approach. It only saves <1µs per request while using a |
|
I don't agree with the bundle size tbf since it runs in the backend not frontend, but for the rest, let me think. |
|
but you are still right, the performance improvement isnt that much, want me to close it? |
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes
Since the prior pullfrog review (466cff4), this run reviewed commit 5eead6e:
- Added a test covering the untraced input/output validation fast path — a transformed
.input(...)/.output(...)pipeline is exercised with the no-op tracer disabled, so the test would fail if the fast path skipped transform application. - Simplified
hasEnumerablePropertiesby dropping the now-redundantundefinedearly-return and narrowing its parameter toContext; both call sites (nextContext !== undefined && …andresultContext !== undefined && …) already guard againstundefinedbefore invoking.
The narrowed signature is type-safe and behavior-neutral, and the new test meaningfully exercises the branch it claims to cover. The full test file passes (45/45).
DeepSeek Flash (free via Pullfrog for OSS) | 𝕏
|
Closing this after considering the review feedback. On normal request paths, the measured savings are about 0.2–0.9µs per request. The larger gains require unusually deep middleware chains. The implementation adds two Thanks for the review. #1951 has been rebased onto |

What this PR does
The middleware pipeline does less work per request. The API does not change. Runtime code changes are limited to
packages/server/src/procedure-client.ts.What changed
WeakMapholds it. Before, the code computed it again for each request.runWithSpanwrapper. It also skips the span name strings. Span names and attributes do not change when a tracer is set.instanceof ORPCErrorcheck.unlazypromise.How much faster
Test setup: Apple M4, Node v25.9.0, Vitest v4.1.11, tracing off. The base is current
main.The largest gain is on deep pass-through middleware chains.
Benchmark scenarios
The PR adds scenarios to the existing benchmark files. It adds no new files.
procedure-call.bench.ts: middleware counts 10 and 100 (passthrough), 10 and 50 (context-adding), 10 and 50 with stacked input schemas.rpc-link-handler.bench.ts: small payloads. The scenarios cover plain, middleware x3, error, and 404.openapi-link-handler.bench.ts: small payloads with GET and POST on a dynamic path.rpc-serializer.bench.ts: payloads with plain JSON only.The old scenario names stay the same. CodSpeed history stays comparable.
Behavior notes
Tests
procedure-client.tshas 100 percent line coverage.next(undefined), symbol keys, primitive outputs, and the span name sequence with a mock tracer.eslintand package diagnostics pass with no errors.A second PR is stacked on this one. It speeds up the client codecs. I will link it here.