worker: chunked-prefill interleaving — +25-35% batched throughput, up to 60% lower TTFT under load - #37

worker: chunked-prefill interleaving — +25-35% batched throughput, up to 60% lower TTFT under load#37

hvasconcelos merged 2 commits into

masterhvasconcelos/libmlxforge:masterfrom

chunked-prefill-experimenthvasconcelos/libmlxforge:chunked-prefill-experimentCopy head branch name to clipboard