Repository navigation

All pull requests

Add --prompt-split-any-experts: allow the prompt split for non-NVFP4 expert builds
#250omskye-cloud opened Nov 5, 2026
prefill: --prefill-timing, a prompt read's time split by stage
#249xyehya opened Nov 5, 2026Contributor
pool: --pool-timing, the CPU expert pool's critical path per request in decode
#248xyehya opened Nov 5, 2026Contributor
serve: --decode-timing adds the decode loop's host phases and the swaps' bytes
#247xyehya opened Nov 5, 2026Contributor
decode: the draft wait, JobRunner::join and the adapt go-spin yield their core after 50 us
#246xyehya opened Nov 5, 2026Contributor
serve: disable runtime RAM guard
#244pngen opened Nov 5, 2026
serve: keep --vision dynamic usable under --lazy (only a sidecar encoder is dropped)
#243polygizer opened Nov 5, 2026Contributor
benchmarks: NVFP4, Q4_K_XL, Q5_K_XL and IQ3_S rows for Qwen3.8-Flash-Next on one RTX 5090 + Windows
#241polygizer opened Nov 4, 2026Contributor3 tasks done
community: 4x RTX 5070 Ti NVFP4 run, 250 W each, two cards at 4.0 x4 …
#238cloudfinapps opened Nov 4, 2026
Report the elastic KV's ceiling at startup and in /v1/status (#148, idea 2)
#237f1zz1ec0ke opened Nov 4, 2026Contributor
docs: add RTX 5060 Ti IQ3_S community benchmark
#236CarstenRuecker opened Nov 4, 2026
serve: log where a request's time to its first token goes
#232xyehya opened Nov 4, 2026Contributor
engine: --adapt-margin sets the count margin of an adaptive swap
#231xyehya opened Nov 4, 2026Contributor
Windows on Arm64: native build for the RTX Spark N1X (sm_121, NEON CPU kernels)
#230SolettaSolaris opened Nov 4, 2026
serve: the stats page reports state, context, per-request detail and the cache
#225giveen opened Nov 3, 2026Contributor
2
tests: file_expert_repin_test checks the alias by its bytes and skips where the driver refuses the sleep (#221)
#223easonLiangWorldedtech opened Nov 3, 2026
tests: iq_avx512_test hashes the dot rows only, so MSVC and glibc agree (#140)
#222easonLiangWorldedtech opened Nov 3, 2026
serve: session-affinity headers pick the slot - a conversation stays where its K/V sits
#214gtrak opened Nov 2, 2026
serve: a big prompt read on a layer split owns the cards - decode windows wait
#213gtrak opened Nov 2, 2026
program: the layer split and its cutoff points are command-line arguments
#212gtrak opened Nov 2, 2026
program: enable peer access between the cards a run uses
#211gtrak opened Nov 2, 2026
feature: /v1/decision & /v1/systemone - Simulate Jev with local inference
#206CommanderTurtle opened Nov 1, 2026
8
Batch slots stream their K/V with --kv-resident (#148, idea 1)
#186f1zz1ec0ke opened Nov 1, 2026Contributor
3
kernels: BF16 rows in the per-layer token embedding table
#185azido-azide-azide opened Oct 31, 2026Contributor
Add support for adapter files
#184jmgao opened Oct 31, 2026Contributor

Navigation Menu

Repository navigation

All pull requests

Add --prompt-split-any-experts: allow the prompt split for non-NVFP4 expert builds

prefill: --prefill-timing, a prompt read's time split by stage

pool: --pool-timing, the CPU expert pool's critical path per request in decode

serve: --decode-timing adds the decode loop's host phases and the swaps' bytes

decode: the draft wait, JobRunner::join and the adapt go-spin yield their core after 50 us

serve: disable runtime RAM guard

serve: keep --vision dynamic usable under --lazy (only a sidecar encoder is dropped)

benchmarks: NVFP4, Q4_K_XL, Q5_K_XL and IQ3_S rows for Qwen3.8-Flash-Next on one RTX 5090 + Windows

community: 4x RTX 5070 Ti NVFP4 run, 250 W each, two cards at 4.0 x4 …

Report the elastic KV's ceiling at startup and in /v1/status (#148, idea 2)

docs: add RTX 5060 Ti IQ3_S community benchmark

serve: log where a request's time to its first token goes

engine: --adapt-margin sets the count margin of an adaptive swap

Windows on Arm64: native build for the RTX Spark N1X (sm_121, NEON CPU kernels)

serve: the stats page reports state, context, per-request detail and the cache

tests: file_expert_repin_test checks the alias by its bytes and skips where the driver refuses the sleep (#221)

tests: iq_avx512_test hashes the dot rows only, so MSVC and glibc agree (#140)

serve: session-affinity headers pick the slot - a conversation stays where its K/V sits

serve: a big prompt read on a layer split owns the cards - decode windows wait

program: the layer split and its cutoff points are command-line arguments

program: enable peer access between the cards a run uses

feature: /v1/decision & /v1/systemone - Simulate Jev with local inference

Batch slots stream their K/V with `--kv-resident` (#148, idea 1)

kernels: BF16 rows in the per-layer token embedding table

Add support for adapter files

Pull requests