Skip to content

πŸ› Bug Report β€” Native streams: Non-byte chunks in a JS-backed body stream are silently dropped when the body is sentΒ #8063

Open

Description

@vicb

When a Response or subrequest body is a JS ReadableStream, and a non-byte chunk (a number,
{}, null, …) is produced asynchronously, the default streams implementation skips the
chunk and sends the rest. The client gets a complete 200 with data missing, and nothing is
logged. If the same chunk is already buffered when the pump reads it, the stream errors
correctly. typescript_implemented_streams does not have this bug.

Repro (workerd 1.20261028.1, compat date 2026-10-20)

// worker.js
const enc = (s) => new TextEncoder().encode(s);
const CHUNKS = () => [enc("a"), 1, enc("b")];

const asyncBody = (chunks) => new ReadableStream({
  async pull(c) {
    await new Promise((r) => setTimeout(r, 1));
    chunks.length ? c.enqueue(chunks.shift()) : c.close();
  },
}, { highWaterMark: 0 });

const syncBody = (chunks) => new ReadableStream({
  start(c) { chunks.forEach((x) => c.enqueue(x)); c.close(); },
});

export default {
  async fetch(req, env) {
    switch (new URL(req.url).pathname) {
      case "/async": return new Response(asyncBody(CHUNKS()));
      case "/sync": return new Response(syncBody(CHUNKS()));
      case "/upload": {
        const r = await env.SELF.fetch("http://x/echo", { method: "POST", body: asyncBody(CHUNKS()) });
        return new Response(`echo: ${JSON.stringify(await r.text())}`);
      }
      case "/echo": return new Response(await req.text());
    }
  },
};
# config.capnp
using Workerd = import "/workerd/workerd.capnp";
const config :Workerd.Config = (
  services = [(name = "main", worker = (
    modules = [(name = "worker", esModule = embed "worker.js")],
    compatibilityDate = "2026-10-20",
    bindings = [(name = "SELF", service = "main")],
  ))],
  sockets = [(name = "http", address = "127.0.0.1:8080", http = (), service = "main")],
);
npx workerd@1.20261028.1 serve config.capnp &
for p in async sync upload; do
  printf '/%-7s ' $p; curl -s -m 5 -w ' [HTTP %{http_code}, curl exit ' http://127.0.0.1:8080/$p; echo "$?]"
done
Route Default streams typescript_implemented_streams
/async ab, HTTP 200, curl exit 0, nothing logged a, then aborted (curl exit 18), error logged
/sync aborted (curl exit 52), error logged aborted (curl exit 52), error logged
/upload echo: "ab", no error on either side echo: "a" (see note)
  • await new Response(asyncBody(CHUNKS())).text() correctly rejects with
    TypeError: This ReadableStream did not return bytes. in both implementations.
  • An async generator body behaves the same as /async.
  • A body made only of non-byte chunks ([1, 2]) produces an empty 200.

For comparison, Node 24 rejects the same stream in Response.text()
("Received non-Uint8Array chunk"). Its fetch() upload hangs rather than dropping the chunk.

Expected: error the stream as /sync does: abort the response or request, cancel the source
and log the TypeError. Per the Fetch spec, a body chunk that is not a Uint8Array is a
TypeError. It must never be skipped silently.

Root cause

ValueQueue::Consumer::drainingRead() (src/workerd/api/streams/queue.c++), used by
pumpToImpl() through DrainingReader:

  • Buffered values go through drainBuffer(), which errors the consumer when
    valueToBytes() fails.
  • Values delivered to a pending read are handled in the continuation shown below. It drops
    the value and returns done = false, so the pump keeps going:
KJ_IF_SOME(bytes, valueToBytes(js, val.addRef(js))) {
  chunks.add(kj::mv(bytes));
}
// If valueToBytes returned kj::none, we just return empty chunks.
// The error case should have been caught earlier.

This was introduced in b73532d (#5838) and first shipped in v1.20260115.0.
queue-test.c++ only covers the buffered case. The TypeScript streams pump
(src/workerd/api/js-readable-stream.c++) rejects every non-byte chunk, so it isn't affected.

Suggested fix

In that continuation, mirror drainBuffer(): error the consumer and reject with the same
TypeError. Add a queue-test.c++ case for a pending read and an integration test for a served
Response. Shipping without a compat flag seems reasonable, because this turns silent data loss
into an error and the behavior only dates from v1.20260115.0.

Note: with either implementation, an /upload receiver whose request body is aborted reads
the partial body ("a" with TS streams, "" for a buffered bad chunk with default streams)
without seeing an error. That looks like a separate service-binding issue.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions