Skip to content

feat(node): add batch response compression plugin - #1882

Merged
dinwwwh merged 12 commits into
middleapi:mainfrom
dinwwwh:claude/compress-batch-response-node-05e0b2
Aug 18, 2026
Merged

feat(node): add batch response compression plugin#1882
dinwwwh merged 12 commits into
middleapi:mainfrom
dinwwwh:claude/compress-batch-response-node-05e0b2

Conversation

@dinwwwh

@dinwwwh dinwwwh commented Aug 17, 2026

Copy link
Copy Markdown
Member

Batch responses were never compressed. Their body frames several subresponses of mixed content types, so the envelope has no single content type for the runtime-agnostic Response Compression Plugin to judge, and a batch of JSON went out uncompressed even though it shrinks about tenfold. Compressing it with the web CompressionStream is not an option either: it cannot flush, so every early subresponse would sit in the compressor until the slowest one resolved, trading streaming for compression.

BatchResponseCompressionHandlerPlugin in @orpc/node compresses them with node:zlib instead, ending every write with a sync flush. Each message reaches the client as soon as its procedure resolves, keep-alive frames included, so a streaming batch stays streaming while compressed. Registering the plugin is the explicit statement that your batches are mostly compressible, which is why it ships as an opt-in in the Node.js package rather than as default behaviour.

Resolves #1645, which reached for the same goal by probing Node builtins from the shared fetch plugin via process.getBuiltinModule.

Behaviour

  • Covers every successful batch response: the length-prefixed framing of a streaming batch or a buffered batch carrying binary, and the plain JSON array a buffered batch of JSON-only subresponses produces.
  • Leaves everything else alone: non-batch responses, whole-batch failures (status >= 400), bodies already content-encoded, partial 206 responses whose Content-Range would stop describing the body, and Cache-Control: no-transform.
  • Negotiates gzip, deflate, or deflate-raw from Accept-Encoding, keys shared caches with Vary: accept-encoding, and applies a size threshold where the size is known.
  • Composes with the Response Compression Plugin in either registration order, neither compressing what the other already did.
  • Nothing changes on the client: fetch implementations decompress as the stream arrives, so each message is decoded the moment it lands.

Testing

  • A batch response measured 8.2 KB uncompressed against under 600 bytes on the wire.
  • The batch end-to-end suite gained a compressed variant over real HTTP with a real client, so its existing per-subresponse timing assertions now prove the compressor flushes rather than buffers.
  • A raw-socket test reads gzip bytes off the wire, and confirms keep-alive frames keep flowing while the batch is idle.
  • Every guard is covered by a test that fails when the guard is removed.

Also

varyByAcceptEncoding and isNoTransformCacheControl moved from the server plugin into @orpc/shared, with tests, so both compression plugins share one implementation. Two stale source links on the batch docs page now point at files that exist.

Compresses batch responses with node:zlib, flushing each message as it is
produced so a streaming batch keeps its streaming semantics.
@vercel

vercel Bot commented Aug 17, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
orpc Ready Ready Preview Aug 18, 2026 2:13am

@pkg-pr-new

pkg-pr-new Bot commented Aug 17, 2026

Copy link
Copy Markdown
More templates

@orpc/ai-sdk

npm i https://pkg.pr.new/@orpc/ai-sdk@1882

@orpc/arktype

npm i https://pkg.pr.new/@orpc/arktype@1882

@orpc/bun

npm i https://pkg.pr.new/@orpc/bun@1882

@orpc/client

npm i https://pkg.pr.new/@orpc/client@1882

@orpc/cloudflare

npm i https://pkg.pr.new/@orpc/cloudflare@1882

@orpc/contract

npm i https://pkg.pr.new/@orpc/contract@1882

@orpc/experimental-effect

npm i https://pkg.pr.new/@orpc/experimental-effect@1882

@orpc/evlog

npm i https://pkg.pr.new/@orpc/evlog@1882

@orpc/hibernation

npm i https://pkg.pr.new/@orpc/hibernation@1882

@orpc/json-schema

npm i https://pkg.pr.new/@orpc/json-schema@1882

@orpc/nest

npm i https://pkg.pr.new/@orpc/nest@1882

@orpc/next

npm i https://pkg.pr.new/@orpc/next@1882

@orpc/node

npm i https://pkg.pr.new/@orpc/node@1882

@orpc/openapi

npm i https://pkg.pr.new/@orpc/openapi@1882

@orpc/opentelemetry

npm i https://pkg.pr.new/@orpc/opentelemetry@1882

@orpc/pinia-colada

npm i https://pkg.pr.new/@orpc/pinia-colada@1882

@orpc/pino

npm i https://pkg.pr.new/@orpc/pino@1882

@orpc/publisher

npm i https://pkg.pr.new/@orpc/publisher@1882

@orpc/ratelimit

npm i https://pkg.pr.new/@orpc/ratelimit@1882

@orpc/server

npm i https://pkg.pr.new/@orpc/server@1882

@orpc/shared

npm i https://pkg.pr.new/@orpc/shared@1882

@orpc/swr

npm i https://pkg.pr.new/@orpc/swr@1882

@orpc/tanstack-query

npm i https://pkg.pr.new/@orpc/tanstack-query@1882

@orpc/trpc

npm i https://pkg.pr.new/@orpc/trpc@1882

@orpc/valibot

npm i https://pkg.pr.new/@orpc/valibot@1882

@orpc/zod

npm i https://pkg.pr.new/@orpc/zod@1882

commit: 461c18d

@codecov

codecov Bot commented Aug 17, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@codspeed-hq

codspeed-hq Bot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will degrade performance by 10.35%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

❌ 1 regressed benchmark
✅ 29 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
octet stream 817.5 µs 911.8 µs -10.35%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing dinwwwh:claude/compress-batch-response-node-05e0b2 (461c18d) with main (5f023ca)

Open in CodSpeed

Asserts on the wire that a compressed json batch carries no body hint,
and states plainly how permissive the message test is.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — I reviewed the full single-commit diff (16 files, head e748553d) for this new opt-in plugin that compresses every successful batch response with node:zlib + sync flush so streaming batches stay streaming.

  • New plugin (packages/node/src/batch-response-compression-handler-plugin.ts): compresses framed streaming/blob and plain json-array buffered batch responses, guarded by request orpc-batch header, status>=400, existing content-encoding, 206/content-range, and no-transform; negotiates gzip/deflate/deflate-raw and keys caches with Vary: accept-encoding.
  • Shared-helper extraction: varyByAcceptEncoding and isNoTransformCacheControl moved to @orpc/shared (with tests); the Response Compression Plugin's five existing call sites now pre-flatten the vary header — behavior-preserving.
  • Tests: unit tests for every guard and both buffered shapes; raw-socket streaming tests proving per-message and keep-alive flush with gzip-magic and wire-size assertions; a compressed client/server variant of the batch e2e suite plus a json-only buffered round-trip.
  • Docs: new plugin page, tips cross-linked from the batch and response-compression pages, api-reference table row, and two stale batch source links corrected to files that exist.

I traced the plugin body shapes against batch.ts (streaming→ReadableStream, buffered+binary→Blob, buffered json→Array) and the logic covers each exactly, mirroring the client's own isServerPeerSendMessage decode check so nothing the client can read back is left uncompressed and nothing else is touched. The coexistence tests confirm neither compression plugin double-compresses in either registration order.

I verified the one load-bearing novelty — the streaming claim — against Node's own zlib source: flush: Z_SYNC_FLUSH in the createGzip/createDeflate options becomes the per-write flush flag (_defaultFlushFlag), so each message and keep-alive frame is emitted promptly while the batch is still open, and the terminal chunk still upgrades to Z_FINISH so the trailer is written. Duplex.toWeb propagates close/error to the served web stream. The raw-socket tests would genuinely fail if the compressor buffered, so the coverage is real.

No blocking or actionable concerns found.

Pullfrog  | View workflow run | Using DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — This increments the prior clean review (head e748553d) and covers only the new commit 31f8c279 (test(node): pin the json batch headers at the socket), which is additive test/docs coverage with no production behavior change.

  • isBatchMessage doc comment (packages/node/src/batch-response-compression-handler-plugin.ts): now explicitly documents that the deliberate kind-only match is intentional — compressing an ordinary array of objects as json is the same thing the response compression plugin would have done to it. No code change.
  • New raw-socket test (packages/node/tests/streams-compressed-batches.test.ts): sends a buffered json batch compressed, as json the client can still parse drives the buffered-json path over a real HTTP server (the sendBatchRequest helper gains a mode parameter) and pins the wire bytes — gzip magic, content-type: application/json, absent standard-server/content-length, and vary: accept-encoding — then gunzipSyncs the body back to two messages.

The new test genuinely fails if the buffered-json branch were broken (magic check + successful gunzip parse are exact assertions, not loose ones), and it closes the coverage gap where that path previously had only unit-level coverage. No issues introduced.

Pullfrog  | View workflow run | Using DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

The batch header and success status already establish the envelope, so the
per-message test bought nothing and cost a dependency.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — I reviewed commit b2ab40bc (refactor(node): recognise a json batch by its array alone), the delta since the prior clean review (head 31f8c279).

  • Simplified json-batch recognition (packages/node/src/batch-response-compression-handler-plugin.ts): if (Array.isArray(body) && body.every(isBatchMessage)) is now just if (Array.isArray(body)), dropping the isBatchMessage / isServerPeerSendMessage / isTypescriptObject imports and the helper. The orpc-batch request-header guard plus the status < 400 / content-encoding / 206 / no-transform checks already establish this is a batch envelope, and a json-only buffered batch response is always an array of messages, so the kind-only check added no real discrimination.
  • Flipped corner-case test: the "leaves an array response ... alone" test became "compresses an array response that holds something other than messages", now asserting content-encoding: gzip, content-type: application/json, and an exact [null, null, largeValue] round-trip through the decompressor (pinning that undefined serializes to null, as the adapter would emit).
  • Docs tweak: batch-response-compression.mdx wording ("compressible" instead of "mostly compressible").

The only behavior change is for the "client sends orpc-batch but the server forgot the batch plugin" corner case: an ordinary array response is now compressed as the json the adapter would have sent anyway, which is transparent to clients that decompress on arrival. This is a justified simplification — the removed check never protected a real batch response. The new test genuinely fails without the array-only branch (exact decompressed-equality assertion). No issues found.

Pullfrog  | View workflow run | Using DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

Runs the batch suite through the fetch adapter without a socket, so the
response compression link plugin does the decoding instead of fetch.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — I reviewed commit 7a720496 (test(batch): cover the fetch adapter and client-side decompression), the delta since the prior clean review (head b2ab40bc). This commit is test/docs-only with no production behavior change.

  • New fetch-adapter test variant (tests/batch/__shared__/client-server.compression-fetch.ts): a fetchSpy returns the handler's raw compressed Response with no socket in between, so no fetch implementation auto-decompresses — the client's ResponseCompressionLinkPlugin is the only thing standing between the gzip bytes and the batch decoder. This covers the half of the contract the node-http variant cannot reach (where fetch already decoded the body). The spy asserts content-encoding: gzip on every 207, and the suite's round-trip assertions genuinely fail if either the server compression or the client decompression were broken.
  • Wired into batch-plugin.test.ts as the fourth compression-fetch variant, running the whole batch e2e suite (streaming-per-message timing, keep-alive, buffered json-only round-trip) through the fetch adapter plus client decompression.
  • Docs note (apps/content/docs/plugins/batch.mdx): clarifies the handler is an RPCHandler and that either the Node.js or Fetch adapter serves it on Node.js where zlib lives.

I confirmed the new variant conforms to the shared CreateBatchClientServerTest contract (client + fetchSpy), and ran the suite: all 17 tests pass (4 variants). The fetch variant's timing/flush assertions would genuinely fail against a buffering compressor, so the coverage is real. No issues introduced.

Pullfrog  | View workflow run | Using DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

Replaces the socketless variant, so the client is an ordinary RPCLink whose
fetch decompresses the response on its own.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — I reviewed commit 2b98a8fe (test(batch): cover compressed batches on the fetch adapter), the delta since the prior clean review (head 7a720496). Test-only, no production change.

  • Fetch-adapter compression variant reworked (tests/batch/__shared__/client-server.compression-fetch.tsclient-server.compression-hono-fetch.ts): now serves the handler through a real @hono/node-server (serve({ port: 0 })) and has the fetchSpy call global fetch rather than replaying the handler's Response by hand. Because real fetch auto-decompresses, the ResponseCompressionLinkPlugin was dropped from the link (with a comment noting fetch already decodes). Default method flipped to GET (node-http keeps POST), with origin pointing at the ephemeral server port.
  • Wired into batch-plugin.test.ts as the renamed compression-hono-fetch variant, running the full batch e2e suite (streaming per-message timing, keep-alive, buffered json-only round-trip) over a real socket + client.

I verified the load-bearing details of the rework: the spy's content-encoding: gzip assertion on every 207 still holds after undici auto-decompresses (undici preserves the header); the assertion genuinely fails if the guard were removed; the @hono/node-server dependency is already declared at the repo root; and no stale references to the old filename remain anywhere. Client-side decompression coverage is retained too — tests/plugins/all-plugins.test.ts still exercises ResponseCompressionLinkPlugin decompressing a compressed batch response (its mock fetch returns the raw compressed Response). I ran the suite: all 17 tests pass across the 4 variants.

Pullfrog  | View workflow run | Using DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

The end-to-end variants already drive both adapters over real sockets. Keeps
the keep-alive frames case, which nothing else covered, and trims the unit
file down to one request helper.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — I reviewed the delta since the prior clean review (head 2b98a8f): the sole new commit 4323705d (test(node): drop the raw socket batch compression file). Test-only, no production behavior change.

  • Deleted the raw-socket test file (packages/node/tests/streams-compressed-batches.test.ts, 214 lines) whose coverage duplicated what the real-socket e2e variants and the unit file already provide over both adapters.
  • Folded its uniquely-owned coverage into batch-response-compression-handler-plugin.test.ts: the streaming per-message flush test (readUntil through a DecompressionStream) and the keep-alive-frames-flush test (the only place keep-alive flushing is exercised — the e2e suites don't cover it).
  • Trimmed the unit file to one request helper: the urls[] → request-message createBatchRequest now builds subrequests itself, dropping the separate makePeerRequestMessage and the now-unused FetchHandlerPlugin import.

I verified the retention claims rather than assuming them: the real-socket flush guarantee still lives in the compression e2e variants' per-message timing assertions (slow subrequest sleeps 200ms while the fast responses must resolve in <100ms), and gzip validity is still proven because the handler's Response body is the raw gzip stream the tests must successfully pipe through DecompressionStream('gzip'). The two wire-byte assertions unique to the raw file (gzip magic 0x1F 0x8B, wire-size ratio) are not carried over, but neither guards behavior the remaining assertions don't already pin. No stale references to the deleted filename remain. I ran the reorganized suite: packages/node unit (25 tests) and tests/batch e2e (17 tests) all pass.

The deletion is well-justified and the rearranged coverage is as real as what it replaced.

Pullfrog  | View workflow run | Using DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

Adds the unmatched request, the non-batch body behind a batch header, and a
batch content type with no bytes, taking the plugin to full coverage.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — I reviewed the delta since the prior clean review (head 4323705d): the sole new commit d58e2369 (test(node): cover the remaining bail-out paths). Test-only; no production behavior change.

  • Three new unit tests (packages/node/src/batch-response-compression-handler-plugin.test.ts) covering guard branches not previously exercised directly: a request that matches no procedure (early return on !result.matched), a body that is neither framed nor an array when the misconfigured client pairs carry the orpc-batch header (content-type mismatch guard), and a response that carries the batch content-type yet is neither Blob nor ReadableStream (the final fall-through guard).

I traced each test against the plugin's interceptor logic and each lands on a distinct bail-out branch: the first asserts matched === false / response undefined through the !matched early return; the second and third both assert content-encoding stays null while the body round-trips intact, which would fail if the plugin compressed. I ran the suite: all 28 tests pass. The new coverage is real and fills the last uncovered default branches without redundancy.

Pullfrog  | View workflow run | Using DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

Titles now read as what a caller would recognise rather than as the branch
each one lands in.
The content type check said what the request header already established, and
dropping it leaves the plugin with no import from the server plugins entry.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — I reviewed the delta since the prior clean review (head 3f1a340): the sole new commit 15f7f2b9 (refactor(node): trust the batch header for the framed body too), a production-behavior refactor of the plugin's framed-body branch.

  • Dropped the content-type re-check on the framed path (packages/node/src/batch-response-compression-handler-plugin.ts): the Blob/ReadableStream branch previously required the response content-type to equal the batch content type before compressing; it now trusts the orpc-batch request-header guard plus status < 400, content-encoding, 206/content-range, and no-transform to identify a batch envelope. This extends the b2ab40bc array-path simplification to the framed path, so the plugin now recognizes a batch by the request header alone.
  • New standard-server: octet-stream hint for the streaming case: the ReadableStream branch now sets the octet-stream body hint where it previously set none (only the Blob branch did). Correct, since the compressed body is octet-stream regardless of origin, and it's pinned by the "compresses streaming batch responses" test.
  • Cleanup: the now-unused BATCH_CONTENT_TYPE import is removed; the fall-through test is renamed to leaves a batch response that carries neither bytes nor messages alone (string body, still exercises the else branch).

I traced the implication of removing the guard: the only behavior change is the misconfigured "client batches but server forgot the batch plugin" corner, where a non-batch Blob/stream response is now compressed where it previously wasn't (when its content type differed from the batch type). That is transparent to any conforming client — auto-decompression restores the bytes and the batch decoder fails on a non-batch body either way — and it is the same already-approved precedent established by b2ab40bc. Within a correctly configured handler (this plugin registered alongside the batch plugin, after = ['~batch']), the response to a batch-header request is always a batch envelope, so the removed check never protected a real batch. The streaming standard-server addition is a genuine correctness improvement. No blocking or actionable concerns found.

Pullfrog  | View workflow run | Using DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No critical issues — one minor typo inline.

Reviewed changes — I reviewed the sole new commit since the prior clean review: 9e9eba43 ("improve"), a docs-only edit to apps/content/docs/plugins/batch-response-compression.mdx.

  • Removed the :::info callout that called out batching being RPC-specific and the handler running on Node.js where zlib lives. Fine editorially — the page's "Why a Node.js Plugin" section still carries the zlib/Node requirement.
  • Accidentally introduced a stray G character on line 68 (see inline comment).

Everything else is unchanged from the previously approved scope: the plugin, its tests, and the shared-helper extraction were all reviewed clean across prior runs.

Pullfrog  | Fix all ➔Fix 👍s ➔View workflow run | Using DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

Comment thread apps/content/docs/plugins/batch-response-compression.mdx Outdated
@dinwwwh
dinwwwh merged commit b6a88ad into middleapi:main Aug 18, 2026
11 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant