feat(node): add batch response compression plugin - #1882
Conversation
Compresses batch responses with node:zlib, flushing each message as it is produced so a streaming batch keeps its streaming semantics.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
More templates
@orpc/ai-sdk
@orpc/arktype
@orpc/bun
@orpc/client
@orpc/cloudflare
@orpc/contract
@orpc/experimental-effect
@orpc/evlog
@orpc/hibernation
@orpc/json-schema
@orpc/nest
@orpc/next
@orpc/node
@orpc/openapi
@orpc/opentelemetry
@orpc/pinia-colada
@orpc/pino
@orpc/publisher
@orpc/ratelimit
@orpc/server
@orpc/shared
@orpc/swr
@orpc/tanstack-query
@orpc/trpc
@orpc/valibot
@orpc/zod
commit: |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Merging this PR will degrade performance by 10.35%
|
| Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|
| ❌ | octet stream |
817.5 µs | 911.8 µs | -10.35% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing dinwwwh:claude/compress-batch-response-node-05e0b2 (461c18d) with main (5f023ca)
Asserts on the wire that a compressed json batch carries no body hint, and states plainly how permissive the message test is.
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes — I reviewed the full single-commit diff (16 files, head e748553d) for this new opt-in plugin that compresses every successful batch response with node:zlib + sync flush so streaming batches stay streaming.
- New plugin (
packages/node/src/batch-response-compression-handler-plugin.ts): compresses framed streaming/blob and plain json-array buffered batch responses, guarded by requestorpc-batchheader,status>=400, existing content-encoding,206/content-range, andno-transform; negotiates gzip/deflate/deflate-raw and keys caches withVary: accept-encoding. - Shared-helper extraction:
varyByAcceptEncodingandisNoTransformCacheControlmoved to@orpc/shared(with tests); the Response Compression Plugin's five existing call sites now pre-flatten the vary header — behavior-preserving. - Tests: unit tests for every guard and both buffered shapes; raw-socket streaming tests proving per-message and keep-alive flush with gzip-magic and wire-size assertions; a compressed client/server variant of the batch e2e suite plus a json-only buffered round-trip.
- Docs: new plugin page, tips cross-linked from the batch and response-compression pages, api-reference table row, and two stale batch source links corrected to files that exist.
I traced the plugin body shapes against batch.ts (streaming→ReadableStream, buffered+binary→Blob, buffered json→Array) and the logic covers each exactly, mirroring the client's own isServerPeerSendMessage decode check so nothing the client can read back is left uncompressed and nothing else is touched. The coexistence tests confirm neither compression plugin double-compresses in either registration order.
I verified the one load-bearing novelty — the streaming claim — against Node's own zlib source: flush: Z_SYNC_FLUSH in the createGzip/createDeflate options becomes the per-write flush flag (_defaultFlushFlag), so each message and keep-alive frame is emitted promptly while the batch is still open, and the terminal chunk still upgrades to Z_FINISH so the trailer is written. Duplex.toWeb propagates close/error to the served web stream. The raw-socket tests would genuinely fail if the compressor buffered, so the coverage is real.
No blocking or actionable concerns found.
DeepSeek Flash (free via Pullfrog for OSS) | 𝕏
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes — This increments the prior clean review (head e748553d) and covers only the new commit 31f8c279 (test(node): pin the json batch headers at the socket), which is additive test/docs coverage with no production behavior change.
isBatchMessagedoc comment (packages/node/src/batch-response-compression-handler-plugin.ts): now explicitly documents that the deliberatekind-only match is intentional — compressing an ordinary array of objects as json is the same thing the response compression plugin would have done to it. No code change.- New raw-socket test (
packages/node/tests/streams-compressed-batches.test.ts):sends a buffered json batch compressed, as json the client can still parsedrives the buffered-json path over a real HTTP server (thesendBatchRequesthelper gains amodeparameter) and pins the wire bytes — gzip magic,content-type: application/json, absentstandard-server/content-length, andvary: accept-encoding— thengunzipSyncs the body back to two messages.
The new test genuinely fails if the buffered-json branch were broken (magic check + successful gunzip parse are exact assertions, not loose ones), and it closes the coverage gap where that path previously had only unit-level coverage. No issues introduced.
DeepSeek Flash (free via Pullfrog for OSS) | 𝕏
The batch header and success status already establish the envelope, so the per-message test bought nothing and cost a dependency.
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes — I reviewed commit b2ab40bc (refactor(node): recognise a json batch by its array alone), the delta since the prior clean review (head 31f8c279).
- Simplified json-batch recognition (
packages/node/src/batch-response-compression-handler-plugin.ts):if (Array.isArray(body) && body.every(isBatchMessage))is now justif (Array.isArray(body)), dropping theisBatchMessage/isServerPeerSendMessage/isTypescriptObjectimports and the helper. Theorpc-batchrequest-header guard plus thestatus < 400/ content-encoding / 206 / no-transform checks already establish this is a batch envelope, and a json-only buffered batch response is always an array of messages, so thekind-only check added no real discrimination. - Flipped corner-case test: the "leaves an array response ... alone" test became "compresses an array response that holds something other than messages", now asserting
content-encoding: gzip,content-type: application/json, and an exact[null, null, largeValue]round-trip through the decompressor (pinning thatundefinedserializes tonull, as the adapter would emit). - Docs tweak:
batch-response-compression.mdxwording ("compressible" instead of "mostly compressible").
The only behavior change is for the "client sends orpc-batch but the server forgot the batch plugin" corner case: an ordinary array response is now compressed as the json the adapter would have sent anyway, which is transparent to clients that decompress on arrival. This is a justified simplification — the removed check never protected a real batch response. The new test genuinely fails without the array-only branch (exact decompressed-equality assertion). No issues found.
DeepSeek Flash (free via Pullfrog for OSS) | 𝕏
Runs the batch suite through the fetch adapter without a socket, so the response compression link plugin does the decoding instead of fetch.
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes — I reviewed commit 7a720496 (test(batch): cover the fetch adapter and client-side decompression), the delta since the prior clean review (head b2ab40bc). This commit is test/docs-only with no production behavior change.
- New fetch-adapter test variant (
tests/batch/__shared__/client-server.compression-fetch.ts): afetchSpyreturns the handler's raw compressedResponsewith no socket in between, so no fetch implementation auto-decompresses — the client'sResponseCompressionLinkPluginis the only thing standing between the gzip bytes and the batch decoder. This covers the half of the contract the node-http variant cannot reach (wherefetchalready decoded the body). The spy assertscontent-encoding: gzipon every207, and the suite's round-trip assertions genuinely fail if either the server compression or the client decompression were broken. - Wired into
batch-plugin.test.tsas the fourthcompression-fetchvariant, running the whole batch e2e suite (streaming-per-message timing, keep-alive, buffered json-only round-trip) through the fetch adapter plus client decompression. - Docs note (
apps/content/docs/plugins/batch.mdx): clarifies the handler is an RPCHandler and that either the Node.js or Fetch adapter serves it on Node.js wherezliblives.
I confirmed the new variant conforms to the shared CreateBatchClientServerTest contract (client + fetchSpy), and ran the suite: all 17 tests pass (4 variants). The fetch variant's timing/flush assertions would genuinely fail against a buffering compressor, so the coverage is real. No issues introduced.
DeepSeek Flash (free via Pullfrog for OSS) | 𝕏
Replaces the socketless variant, so the client is an ordinary RPCLink whose fetch decompresses the response on its own.
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes — I reviewed commit 2b98a8fe (test(batch): cover compressed batches on the fetch adapter), the delta since the prior clean review (head 7a720496). Test-only, no production change.
- Fetch-adapter compression variant reworked (
tests/batch/__shared__/client-server.compression-fetch.ts→client-server.compression-hono-fetch.ts): now serves the handler through a real@hono/node-server(serve({ port: 0 })) and has thefetchSpycall globalfetchrather than replaying the handler's Response by hand. Because real fetch auto-decompresses, theResponseCompressionLinkPluginwas dropped from the link (with a comment noting fetch already decodes). Default method flipped toGET(node-http keepsPOST), withoriginpointing at the ephemeral server port. - Wired into
batch-plugin.test.tsas the renamedcompression-hono-fetchvariant, running the full batch e2e suite (streaming per-message timing, keep-alive, buffered json-only round-trip) over a real socket + client.
I verified the load-bearing details of the rework: the spy's content-encoding: gzip assertion on every 207 still holds after undici auto-decompresses (undici preserves the header); the assertion genuinely fails if the guard were removed; the @hono/node-server dependency is already declared at the repo root; and no stale references to the old filename remain anywhere. Client-side decompression coverage is retained too — tests/plugins/all-plugins.test.ts still exercises ResponseCompressionLinkPlugin decompressing a compressed batch response (its mock fetch returns the raw compressed Response). I ran the suite: all 17 tests pass across the 4 variants.
DeepSeek Flash (free via Pullfrog for OSS) | 𝕏
The end-to-end variants already drive both adapters over real sockets. Keeps the keep-alive frames case, which nothing else covered, and trims the unit file down to one request helper.
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes — I reviewed the delta since the prior clean review (head 2b98a8f): the sole new commit 4323705d (test(node): drop the raw socket batch compression file). Test-only, no production behavior change.
- Deleted the raw-socket test file (
packages/node/tests/streams-compressed-batches.test.ts, 214 lines) whose coverage duplicated what the real-socket e2e variants and the unit file already provide over both adapters. - Folded its uniquely-owned coverage into
batch-response-compression-handler-plugin.test.ts: the streaming per-message flush test (readUntilthrough aDecompressionStream) and the keep-alive-frames-flush test (the only place keep-alive flushing is exercised — the e2e suites don't cover it). - Trimmed the unit file to one request helper: the
urls[]→ request-messagecreateBatchRequestnow builds subrequests itself, dropping the separatemakePeerRequestMessageand the now-unusedFetchHandlerPluginimport.
I verified the retention claims rather than assuming them: the real-socket flush guarantee still lives in the compression e2e variants' per-message timing assertions (slow subrequest sleeps 200ms while the fast responses must resolve in <100ms), and gzip validity is still proven because the handler's Response body is the raw gzip stream the tests must successfully pipe through DecompressionStream('gzip'). The two wire-byte assertions unique to the raw file (gzip magic 0x1F 0x8B, wire-size ratio) are not carried over, but neither guards behavior the remaining assertions don't already pin. No stale references to the deleted filename remain. I ran the reorganized suite: packages/node unit (25 tests) and tests/batch e2e (17 tests) all pass.
The deletion is well-justified and the rearranged coverage is as real as what it replaced.
DeepSeek Flash (free via Pullfrog for OSS) | 𝕏
Adds the unmatched request, the non-batch body behind a batch header, and a batch content type with no bytes, taking the plugin to full coverage.
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes — I reviewed the delta since the prior clean review (head 4323705d): the sole new commit d58e2369 (test(node): cover the remaining bail-out paths). Test-only; no production behavior change.
- Three new unit tests (
packages/node/src/batch-response-compression-handler-plugin.test.ts) covering guard branches not previously exercised directly: a request that matches no procedure (early return on!result.matched), a body that is neither framed nor an array when the misconfigured client pairs carry theorpc-batchheader (content-type mismatch guard), and a response that carries the batch content-type yet is neitherBlobnorReadableStream(the final fall-through guard).
I traced each test against the plugin's interceptor logic and each lands on a distinct bail-out branch: the first asserts matched === false / response undefined through the !matched early return; the second and third both assert content-encoding stays null while the body round-trips intact, which would fail if the plugin compressed. I ran the suite: all 28 tests pass. The new coverage is real and fills the last uncovered default branches without redundancy.
DeepSeek Flash (free via Pullfrog for OSS) | 𝕏
Titles now read as what a caller would recognise rather than as the branch each one lands in.
The content type check said what the request header already established, and dropping it leaves the plugin with no import from the server plugins entry.
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes — I reviewed the delta since the prior clean review (head 3f1a340): the sole new commit 15f7f2b9 (refactor(node): trust the batch header for the framed body too), a production-behavior refactor of the plugin's framed-body branch.
- Dropped the content-type re-check on the framed path (
packages/node/src/batch-response-compression-handler-plugin.ts): the Blob/ReadableStreambranch previously required the responsecontent-typeto equal the batch content type before compressing; it now trusts theorpc-batchrequest-header guard plusstatus < 400,content-encoding,206/content-range, andno-transformto identify a batch envelope. This extends theb2ab40bcarray-path simplification to the framed path, so the plugin now recognizes a batch by the request header alone. - New
standard-server: octet-streamhint for the streaming case: theReadableStreambranch now sets the octet-stream body hint where it previously set none (only the Blob branch did). Correct, since the compressed body is octet-stream regardless of origin, and it's pinned by the "compresses streaming batch responses" test. - Cleanup: the now-unused
BATCH_CONTENT_TYPEimport is removed; the fall-through test is renamed toleaves a batch response that carries neither bytes nor messages alone(string body, still exercises theelsebranch).
I traced the implication of removing the guard: the only behavior change is the misconfigured "client batches but server forgot the batch plugin" corner, where a non-batch Blob/stream response is now compressed where it previously wasn't (when its content type differed from the batch type). That is transparent to any conforming client — auto-decompression restores the bytes and the batch decoder fails on a non-batch body either way — and it is the same already-approved precedent established by b2ab40bc. Within a correctly configured handler (this plugin registered alongside the batch plugin, after = ['~batch']), the response to a batch-header request is always a batch envelope, so the removed check never protected a real batch. The streaming standard-server addition is a genuine correctness improvement. No blocking or actionable concerns found.
DeepSeek Flash (free via Pullfrog for OSS) | 𝕏
There was a problem hiding this comment.
ℹ️ No critical issues — one minor typo inline.
Reviewed changes — I reviewed the sole new commit since the prior clean review: 9e9eba43 ("improve"), a docs-only edit to apps/content/docs/plugins/batch-response-compression.mdx.
- Removed the
:::infocallout that called out batching being RPC-specific and the handler running on Node.js wherezliblives. Fine editorially — the page's "Why a Node.js Plugin" section still carries the zlib/Node requirement. - Accidentally introduced a stray
Gcharacter on line 68 (see inline comment).
Everything else is unchanged from the previously approved scope: the plugin, its tests, and the shared-helper extraction were all reviewed clean across prior runs.
DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

Batch responses were never compressed. Their body frames several subresponses of mixed content types, so the envelope has no single content type for the runtime-agnostic Response Compression Plugin to judge, and a batch of JSON went out uncompressed even though it shrinks about tenfold. Compressing it with the web
CompressionStreamis not an option either: it cannot flush, so every early subresponse would sit in the compressor until the slowest one resolved, trading streaming for compression.BatchResponseCompressionHandlerPluginin@orpc/nodecompresses them withnode:zlibinstead, ending every write with a sync flush. Each message reaches the client as soon as its procedure resolves, keep-alive frames included, so a streaming batch stays streaming while compressed. Registering the plugin is the explicit statement that your batches are mostly compressible, which is why it ships as an opt-in in the Node.js package rather than as default behaviour.Resolves #1645, which reached for the same goal by probing Node builtins from the shared fetch plugin via
process.getBuiltinModule.Behaviour
status >= 400), bodies already content-encoded, partial206responses whoseContent-Rangewould stop describing the body, andCache-Control: no-transform.Accept-Encoding, keys shared caches withVary: accept-encoding, and applies a size threshold where the size is known.Testing
Also
varyByAcceptEncodingandisNoTransformCacheControlmoved from the server plugin into@orpc/shared, with tests, so both compression plugins share one implementation. Two stale source links on the batch docs page now point at files that exist.