Skip to content

worker: chunked-prefill interleaving — +25-35% batched throughput, up to 60% lower TTFT under load - #37

Merged
hvasconcelos merged 2 commits into
masterfrom
chunked-prefill-experiment
Jun 11, 2026
Merged

hvasconcelos merged 2 commits into
masterfrom
chunked-prefill-experiment

doc: whitepaper covers quantized KV storage and the prefix cache

eb44b71
Select commit
Loading
Failed to load commit list.
Sign in for the full log view

Annotations

1 error and 1 warning

The logs for this run have expired and are no longer available.