Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
4be35df
feat(kev): add pinned CPU export and FP32 pointer head
psv666 Oct 24, 2026
76b764a
Merge current main into Kev export and head preparation
psv666 Oct 27, 2026
8f765fd
fix(kev): validate every pinned LoRA projection target
psv666 Oct 27, 2026
0102113
feat(kev): add eager native Rust CUDA text worker and parity harness
psv666 Oct 27, 2026
57e239b
test(kev): reject invalid parity evidence and report boundary flips
psv666 Oct 27, 2026
1d44e4f
docs(kev): record eager numerical and serving acceptance limits
psv666 Oct 27, 2026
490d80d
fix(kev): make reference model lifetime explicit for lint checks
psv666 Oct 27, 2026
33e1848
Kev: qualify against default BF16 service and fix Unicode tokenization
psv666 Oct 27, 2026
b446d6e
docs: mark Kev BF16 numerical qualification as failing
psv666 Oct 27, 2026
224fa3a
test(kev): verify rendered prompts across the frozen service corpus
psv666 Oct 27, 2026
0efa1d9
fix(kev): align eager native numerics with the pinned BF16 service
psv666 Oct 28, 2026
d850153
test(kev): qualify Noul boundaries using official service outputs
psv666 Oct 28, 2026
b7c94b3
docs(kev): record BF16 acceptance and shared Qwen regression evidence
psv666 Oct 28, 2026
2892608
Merge remote-tracking branch 'origin/main' into codex/issue-27-kev-ex…
psv666 Oct 31, 2026
85119d0
test(kev): cover complete export and late-write cleanup
psv666 Oct 31, 2026
82751ae
Merge refreshed Kev export slice and main prefix-cache support
psv666 Oct 31, 2026
9501ede
test(qwen): guard fixed-prefix entry points against Kev numerics
psv666 Oct 31, 2026
125251c
docs(kev): pin the qualified CUDA reference wheel
psv666 Nov 1, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,21 @@ jobs:
python -m pip install pytest
PYTHONPATH=src python -m pytest tests/clef -q

kev-export:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v5
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: CPU-only export dependencies
run: |
python -m pip install torch==2.8.0 --index-url https://download.pytorch.org/whl/cpu
python -m pip install -r recipe/kev/requirements.txt
- name: Export contract tests (no weights or GPU)
run: python -m unittest discover -s tests/kev -p 'test_*.py' -v

cuda-contract:
# The CUDA backend contract and its checker. No CUDA toolkit and no GPU: both
# read files as text.
Expand Down
17 changes: 17 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
[workspace]
members = ["src/frontend", "src/runtime", "src/models/clm", "src/models/cua_s1/native", "src/models/qwen3_5/native", "src/models/open_jev/native", "src/models/jev_vl/native", "src/models/omnijev/native", "src/models/laya", "src/backends/cuda", "src/models/decider/native", "src/models/jemm/native", "src/models/startlux/native", "src/models/clef/native"]
members = ["src/frontend", "src/runtime", "src/models/clm", "src/models/cua_s1/native", "src/models/qwen3_5/native", "src/models/open_jev/native", "src/models/jev_vl/native", "src/models/omnijev/native", "src/models/laya", "src/backends/cuda", "src/models/decider/native", "src/models/jemm/native", "src/models/startlux/native", "src/models/clef/native", "src/models/kev/native"]
resolver = "3"
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -191,6 +191,7 @@ checkpoint. Public231 concurrency-1 latency for heads `1ba07d1`, `f968e576`,
| CLM-v0.1-8B | [External worker with a CPU stub encoder](recipe/clm/README.md); contract checks only, real Qwen3-8B decisions unverified by this recipe |
| LFM2.5-350M | [Transformers worker](recipe/lfm2/README.md); text state and independent choice questions |
| OmniJev-4B v1.1 | [Native Rust/CUDA worker](recipe/omnijev/native.md), eager: one image, Choice, Noul and Score; [reference comparison on sm_89](recipe/omnijev/native.md#comparison-with-the-reference) |
| Kev-4B | [Eager native Rust/CUDA text worker](recipe/kev/native.md); Choice/Noul/Score; [BF16 acceptance passes, including 36 Noul boundaries](recipe/kev/noul-boundary.md); author-run; review pending ([#27](https://github.com/InferMatrixLab/system1-omni/issues/27)) |
| D3 (Decision 3.0) | [Python reference worker](recipe/d3/reference.md) on the checkpoint's pinned runtime; `choice`/`noul`/`score` over text, image and video state; author-run on A100 for text and media serving (see the integration and media PRs) |
| Vela 2.0 0.3B | [Python reference worker](recipe/vela/reference.md) on the checkpoint's pinned runtime; all five kinds (`choice`/`noul`/`score`/`span`/`set`) for text state; CPU FP32 validated (author-run, evidence in the integration PR) |
| Vela 2.0 0.8B hybrid | [Python reference worker](recipe/vela/reference.md) on the checkpoint's pinned runtime; same surface plus `span_heads`; CPU FP32 parity recorded in the integration PR |
Expand Down
18 changes: 16 additions & 2 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ design. Concrete input/output types follow each executor's supported layout.
## Implementation status

The [Rust frontend](../src/frontend/README.md) currently forwards HTTP requests
to separately running workers. Cua-S1, Open-Jev, Decider, and StartLux Instant
to separately running workers. Cua-S1, Open-Jev, Decider, Kev-4B, and StartLux Instant
4B have native Rust/CUDA workers that share the
[Qwen3.5/3.8 executor](../src/models/qwen3_5/native/), which accepts
single prompts and bounded packed prefill. StartLux Instant uses a
Expand Down Expand Up @@ -81,6 +81,14 @@ FP32 adapters and framework-reference arithmetic. It computes selected BF16
label-head logits on CUDA, then calibrates typed answers on the CPU. JEMM admits
one complete text or 1–4-image request and synchronizes both model streams before
releasing its resources. See its [model contract](../src/models/jemm/README.md).
Kev uses merged BF16 weights and an optional Torch 2.8 numerical profile. It
recomputes state then question in eager mode, matching the pinned official
padding and projection shapes, and downloads selected option/decision hidden
rows for its FP32 pointer head. Temporary GDN/KV state belongs to one question
call and is drained before release; it is not shared between requests. Its
processor owns prompt tokens and question geometry, while complete-question
calibration and typed response reconstruction remain in postprocessing.
The existing Qwen default, fixed and reference profiles keep their own paths.
Decider projects selected tied-embedding rows on CUDA with BF16 output rounding
before FP32 calibration and CPU response assembly. Laya computes its scorer and
action head on CUDA. Its fixed-shape
Expand Down Expand Up @@ -138,6 +146,7 @@ and candidate identity, usage, and response metadata outside the executor.
| Cua-S1 | One unpadded token-ID vector and option count per question, in request order. | One FP32 answer-letter logit vector per question. | Per-question softmax, choice/confidence, ordered answers, and token usage. |
| Open-Jev | Token-ID vectors grouped by question, then independent candidate, in request order. | One FP32 learned scalar per candidate in the same grouping. | Add the `noul` false logit of zero, calibrate across each complete question, and restore typed answers, usage, and metadata. |
| Decider | Complete independent token-ID rows in question order; Score expands one no/yes row per level. | One BF16-projection-rounded FP32 selected-label logit vector per row. | Per-type calibration, whole-Score normalization, ordered typed answers and unique-prefix token usage. |
| Kev-4B | Complete text rows, consecutive positions and ordered option/decision readouts, one row per question. | One FP32 pointer-logit vector per complete question. | Saved-temperature softmax over all candidates, ordered Choice/Noul/Score answers and official token billing. |
| Laya | One padded request: token IDs, true lengths, question types and ordered option markers; at most 16 questions, 512 tokens per row and 2048 markers. | Per-question FP32 option logits and two action logits copied back after GPU heads. | Calibrate and decode ordered `choice`, `score` and `noul` answers, usage and metadata. |
| JEMM | Ordered token IDs, three-axis positions, candidate counts and optional shared image patches for one complete request. | Per-question selected BF16 label logits copied back as FP32. | Apply checksum-covered text/image calibration and reconstruct ordered `choice`, `score` and `noul` answers and usage. |
| StartLux Instant 4B | One unpadded token-ID prompt per question, or per knockout group and then a final prompt, in request order. | FP32 letter logits from a CPU dot of the tied embedding rows at the last position. | Per-type temperature, none-check split, wide-choice residual, and ordered `choice`, `noul`, and `score` answers with token usage. |
Expand All @@ -149,7 +158,7 @@ Open-Jev's model-specific batch adapter packs up to 16 independent candidates an
execute alone. It restores question/candidate grouping before normalization.
Laya batches questions within one request. Shared runtime
admission precedes blocking dispatch: Cua-S1 admits one question forward at a
time; Open-Jev, Decider, Laya, StartLux Instant, and Clef admit one complete request. Clef's admitted work is the frame exchange with its executor process. StartLux runs each prompt inside that admission one at a time. Cua-S1's CPU letter projection
time; Open-Jev, Decider, Kev, Laya, StartLux Instant, and Clef admit one complete request. Kev runs all question rows and its FP32 head inside that admission; listener readiness follows a real mixed-question warmup. Clef's admitted work is the frame exchange with its executor process. StartLux runs each prompt inside that admission one at a time. Cua-S1's CPU letter projection
stays outside admission; Open-Jev's scalar heads and Laya's GPU heads and
synchronized readback remain inside their request unit. Decider includes every
backbone row and CUDA head projection, synchronizes both streams and retires
Expand Down Expand Up @@ -212,6 +221,11 @@ semantics.
Preserve whole-request failure behavior, error status/body, and usage fields
when splitting or regrouping work.

Kev prepares one complete causal row per question, reads every candidate closing
marker plus the final decision marker through the shared Qwen selected-position
readout, and applies its FP32 pointer head. Its eager executor recomputes the state
for each question and admits one whole request through the serial scheduler.

Cua-S1 prepares one prompt per question and reads option-letter logits.
Open-Jev prepares independent candidate prompts and normalizes across the
complete question's candidates. Laya pads prepared questions into one request
Expand Down
10 changes: 10 additions & 0 deletions docs/supported-models.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,16 @@ Models that are being added are also tracked in issues labeled [new model](https
| Clef (27B) | Same `omni-clef` server and text contract as Clef-flash, selected with `--variant clef` | Contract tests only; no inference | Not run | Not supported | Same stack; pinned `Cloudflare/clef` snapshot (~55.0 GB). Weights and joint-head config differ from flash |
| pplx-decider-v1.1-27b | [Python reference worker](../recipe/pplx_decider/README.md) on the checkpoint's own runtime. Text and up to four images; `choice`/`noul`/`score`. Native CUDA executor is not this worker. Lab L20X-labeled GPUs are the H200 target. The causal-conv1d build keeps sm_90 and sm_89. Capability 8.9 is ready on FLA chunk. Capability 9.0 uses all-sequence FlashInfer. The kept primary rotates 48 `cu_seqlens` buffers per device and length, passes `use_cp` false, skips the q/k repeat, and packs q and k before the L2-norm view. `801ef7b` fused the 48 linear gated RMSNorms and missed the median (p50 89.172 ms, p95 274.790 ms), so that kernel stays off. `2bbbe4e` replaced `silu(gate) * up` with FLA `swiglu_fwd` and the owner closeout rejected it (p50 97.613 ms, p95 292.795 ms), so that kernel stays off. Gate and up GEMMs stay separate. Residual RMSNorm and the gated output norm stay eager. An unknown `PPLX_GDN_PREFILL` fails closed on every device, including capability 8.9. `3ecdf5a` crashed in warmup (0/231). Always-on fusion `3afb472` and the 512-token gate `705ae089` missed the median. The 48-slot FlashInfer state ring `17a7b120` missed the gate. `PPLX_GDN_PREFILL=fla` reproduces the prior `f968e576` chunk. `PPLX_GDN_PREFILL=short` is the rejected ≤256 dispatch. SDPA stays on the checkpoint order. The fused gate-up GEMM is refused. The forward stays eager. `PPLX_COMPILE=dynamic` and `PPLX_COMPILE=dynamic-tail` are rejected opt-ins | Contract checks only; no inference | Reported load on 1× lab L20X (sm_89, ~140 GiB) at head `1ba07d1`: BF16 resident ~50333 MiB, no OOM. Historical primary: 231/231, p50 98.777 ms, p95 341.202 ms. Prior primary is the `f968e576` remeasure on the lab L20X-labeled GPU, `compute_capability=9.0`, `fla-chunk`, `flashinfer` null: 231/231, p50 94.077 ms, p95 300.200 ms (p50 −4.76%, p95 −12.02%). Prior primary `ffae182`: 231/231, p50 92.844 ms, p95 296.458 ms (p50 −1.31%, p95 −1.25%). Prior primary `4b99b73`: 231/231, p50 88.942 ms, p95 295.797 ms (p50 −4.20%, p95 −0.22%), 48-slot `cu_seqlens`, quality `choice_acc=0.9065` (139), `noul_acc=0.8784` (74), `score_mae=0.2035` (18). Kept primary is `fcf5e2b`: 231/231, p50 87.782 ms, p95 291.183 ms (p50 −1.30%, p95 −1.56%), grouped-value FlashInfer with q and k packed, same quality digits. Uncached all-sequence FlashInfer `3c72f02`: p50 88.808 ms, p95 300.751 ms, gate failed. Short-sequence FlashInfer `a50b9aa`: p50 95.446 ms, p95 300.871 ms, gate failed. cuDNN-first SDPA `0dcaa0ac`: p50 138.501 ms, p95 332.859 ms, gate failed. Fused gate-up `8b4ee326`: p50 102.437 ms, p95 304.970 ms, gate failed. Always-on dynamic compile `e36b32ed`: p50 97.751 ms, p95 263.124 ms, gate failed (p95 improved). 512-token compile `7e3b1be`: p50 100.037 ms, p95 257.084 ms, gate failed. Isolated-median compile `34ba5de`: p50 97.084 ms, p95 256.679 ms, gate failed (median probe was eager). FlashInfer state-workspace ring `17a7b120`: p50 89.165 ms, p95 295.723 ms, gate failed. Always-on FLA residual RMSNorm `3afb472`: p50 93.935 ms, p95 274.902 ms, gate failed. Length-gated FLA residual RMSNorm `705ae089`: p50 94.604 ms, p95 274.835 ms, gate failed. `use_cp=auto` `a6f4d9a`: p50 112.294 ms, p95 299.419 ms, gate failed. Grouped-value `3ecdf5a` crashed in warmup (0/231, no p50 or p95). The fused linear gated-norm candidate has no sample. Does not enter the H800 board | Not supported | Python 3.12, pinned checkpoint revision `14c9d90137962d300f3de95633f7b6535e80c5ce`, checkpoint `requirements.txt`, `causal-conv1d` 1.7.0, `flash-linear-attention` 0.5.2, optional `flashinfer-python` 0.6.16.post3. Shard sizes in the release manifest sum with the readout to about 48.6 GiB before activations |

Kev-4B has an [eager native Rust/CUDA text worker](../recipe/kev/native.md) for
Choice, Noul and Score. All frozen gates pass over 5,484 requests / 5,662
questions from the fixed official BF16 default service: maximum probability error 0.0024709404 <=0.01 and Score error
0.00000305218 <=0.1 level. Near-boundary Noul coverage is 36/10 (Choice 28/10).
Processing, independent head, 896-question task-quality gates, real warmup and
55 HTTP response comparisons pass. Author-run evidence; maintainer review remains pending.
See the [boundary qualification and limitations](../recipe/kev/noul-boundary.md)
for [#27](https://github.com/InferMatrixLab/system1-omni/issues/27). Other sizes
and backends remain unverified.

- **Validated:** covered by the recipe on `main` or by the checks in the linked merged pull request.
- **Author-run:** recorded by the recipe author on the stated hardware and not independently reproduced; the row names its owner.
- **Unverified:** the worker accepts this device, but no recipe or merged pull request covers it.
Expand Down
5 changes: 5 additions & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,11 @@ nav:
- JEV-27B-VL experimental worker: recipe/jev_vl/README.md
- JEV-27B-VL validation status: recipe/jev_vl/validation.md
- OmniJev preparation: recipe/omnijev/README.md
- Kev-4B preparation: recipe/kev/README.md
- Kev-4B native worker: recipe/kev/native.md
- Kev-4B service acceptance: recipe/kev/service-acceptance.md
- Kev-4B numerical repair: recipe/kev/numerical-diagnosis.md
- Kev-4B Noul boundary qualification: recipe/kev/noul-boundary.md
- OmniJev native worker: recipe/omnijev/native.md
- CLM stub-encoder contract: recipe/clm/README.md
- LFM2.5-350M choice worker: recipe/lfm2/README.md
Expand Down
3 changes: 3 additions & 0 deletions recipe/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,9 @@ For a first real decision, follow the [complete CPU walkthrough](../docs/getting
- [OmniJev-4B v1.1 preparation](omnijev/README.md): export the checkpoint and
regenerate the CPU fixtures; [native worker](omnijev/native.md): serve it on CUDA
and compare it with the reference.
- [Kev-4B preparation](kev/README.md): export all LoRA projections and validate
the Rust FP32 pointer head on CPU; [native worker](kev/native.md) serves eager
text Choice/Noul/Score with explicit numerical acceptance gates.
- [CLM behind the frontend](clm/README.md): run CLM's own server behind the frontend on
CPU with a stub encoder, and what the response comparison has to allow for.
- [pplx-decider reference worker](pplx_decider/README.md): pin
Expand Down
Loading
Loading