Repository navigation
Conversation
- recipe/vela/export_native.py: gate-verified export into a native bundle — 320 backbone tensors re-keyed backbone. -> model. with per-tensor dtypes preserved (204 BF16 Linear/conv1d + 116 F32), the 37 F32 heads incl. the side-loaded broad span head in one file, backbone_config at the config root (the family Config::load shape), and a manifest pinning the source revision, the gate table, the tensor inventory and every bundle file's SHA-256. - src/models/vela/native (omni-vela-native): the bundle manifest gate and the wire admission mirroring protocol.py and the pinned runtime's preamble with byte-equal 422/400 bodies. - recipe/vela/export_token_plan.py and five frozen token plans under tests/vela/fixtures/: the predict mapping, the row decomposition and the rendered records (p_ids, blocks, readout positions, truncation, a windowed long-span case); example-request totals 1,041 tokens, matching the oracle's usage. Checks: cargo test (15 tests, no weights, no GPU), fmt/clippy clean, mkdocs --strict.
The Rust processing of the native path, mirrored from the pinned runtime and checked field-for-field against the five frozen token plans (opt-in render parity with the exported bundle's tokenizer): - state.rs: the state -> typed-parts mapping (role grouping, field ranges, canonical DEV2.0 JSON) with the pinned rejection triples; - questions.rs: the SystemOne -> predict mapping — the validation walk and order, Noul folded into a two-option choice (choice renders [ABS], the reference's abstain default), over resolution, threshold/head carried, the set-label collision check; - words.rs: the GLiNER2/CJK _WORD splitter and the first-token word mapping; - render.rs: the encoder (char-offset tokens, memoized), the six-rule span-head dispatch with the hallucination-alias label replacement, the truncation budget, the DEV2.0 rendering (parts pass, question blocks with option ends and query positions, span layout a/c with word positions) and the window rows (head only, as the reference); - processing.rs: the Processor over the exported bundle. Also: the envelope error locs are flat arrays, matching the reference triples exactly. Checks: cargo test (26 tests) + fmt/clippy clean + opt-in render parity (5/5 fixtures).
The Rust processing of the native path, mirrored from the pinned runtime and checked field-for-field against the five frozen token plans (opt-in render parity with the exported bundle's tokenizer): - state.rs: the state -> typed-parts mapping (role grouping, field ranges, canonical DEV2.0 JSON) with the pinned rejection triples; - questions.rs: the SystemOne -> predict mapping — the validation walk and order, Noul folded into a two-option choice (choice renders [ABS], the reference's abstain default), over resolution, threshold/head carried, the set-label collision check; - words.rs: the GLiNER2/CJK _WORD splitter and the first-token word mapping; - render.rs: the encoder (char-offset tokens, memoized), the six-rule span-head dispatch with the hallucination-alias label replacement, the truncation budget, the DEV2.0 rendering (parts pass, question blocks with option ends and query positions, span layout a/c with word positions) and the window rows (head only, as the reference); - processing.rs: the Processor over the exported bundle. Also: the envelope error locs are flat arrays, matching the reference triples exactly. Checks: cargo test (26 tests) + fmt/clippy clean + opt-in render parity (5/5 fixtures).
- heads.rs: CandidateHead (option-end + query hiddens; bilinear + scalar-MLP, [ABS] last), both SpanHeadV2 instances (router and the side-loaded broad head) and set_bias, loaded from the exported bundle's heads.safetensors — plain float32, the reference's own autocast-off setting (erf-based GELU via libm). - scoring.rs: the calibration mirror — per-type temperatures, set thresholds, the PII length rule (log-space interp) and sparse gate, span thresholds by id/label set, softmax/sigmoid, span_decode_r4 (word units, unit vote, fill, edge/URL trim), label aliasing, score value normalization (i/(n-1)), and the window aggregation (mean over covering windows, -30 uncovered). - recipe/vela/export_head_inputs.py and the recorded head-input fixtures: the reference's consumed hidden states, its head logits and its calibrated predict answers for the five frozen cases. Opt-in parity (VELA_NATIVE_BUNDLE): five cases, head-logit drift <= 3.8e-5, predict-answer drift <= 7.3e-7 (the windowed long-span case exact), discrete picks, span structures and threshold rules equal.
- heads.rs: CandidateHead (option-end + query hiddens; bilinear + scalar-MLP, [ABS] last), both SpanHeadV2 instances (router and the side-loaded broad head) and set_bias, loaded from the exported bundle's heads.safetensors — plain float32, the reference's own autocast-off setting (erf-based GELU via libm). - scoring.rs: the calibration mirror — per-type temperatures, set thresholds, the PII length rule (log-space interp) and sparse gate, span thresholds by id/label set, softmax/sigmoid, span_decode_r4 (word units, unit vote, fill, edge/URL trim), label aliasing, score value normalization (i/(n-1)), and the window aggregation (mean over covering windows, -30 uncovered). - recipe/vela/export_head_inputs.py and the recorded head-input fixtures: the reference's consumed hidden states, its head logits and its calibrated predict answers for the five frozen cases. Opt-in parity (VELA_NATIVE_BUNDLE): five cases, head-logit drift <= 3.8e-5, predict-answer drift <= 7.3e-7 (the windowed long-span case exact), discrete picks, span structures and threshold rules equal.
scoring.rs system_one_response: the SystemOne envelope from the predict answers and the response plan — Noul views (span = the range-filtered max word probability, set fan-out <qid>.<label>), span clipping to the plan's field ranges with re-trim, span_heads, thresholds, the choice top-two margin and the score variance confidence, and usage. Replaying the recorded reference logits into the frozen oracle responses matches within 1.5e-6 on the four oracle cases (opt-in with VELA_NATIVE_BUNDLE).
executor.rs: the backbone through the shared qwen3_5 crate. The family Model exposes only last-position hidden states, so every readout position runs as its own full sequence [parts .. position] — causality makes that hidden identical to the reference's in-tree readout — packed into forward_batch. The backbone runs from the uniform-BF16 export (export_native.py --bf16-out, the shared loader's format); the heads stay F32 on the faithful tensors. Opt-in GPU parity (VELA_NATIVE_BUNDLE + VELA_CUDA_LIB): four cases replay the recorded reference logits, worst drift 0.285 (the bf16-vs-FP32 scale; gate 0.35). First recorded execution of these kernels on sm_120 (RTX 5060, CUDA 13.1).
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
…processing # Conflicts: # Cargo.toml # README.md # docs/supported-models.md # recipe/README.md
…heads # Conflicts: # Cargo.toml # README.md # docs/supported-models.md # recipe/README.md
…executor # Conflicts: # Cargo.toml # README.md # docs/supported-models.md # recipe/README.md
382d46d to
b6dc583
Compare
…ows alongside the Clef/pplx-decider additions
… add the Clef/pplx rows
…native-processing # Conflicts: # recipe/README.md
…heads # Conflicts: # recipe/vela/native.md
# Conflicts: # recipe/vela/native.md # src/models/vela/native/Cargo.toml # src/models/vela/native/src/bundle.rs # tests/vela/native_heads.rs # tests/vela/native_processing.rs
psv666
left a comment
psv666
left a comment
There was a problem hiding this comment.
The latest export repair is present, but the earlier slices still need their correctness fixes propagated here.
- [P1] Span decoding joins across rejected words. The current decode loop returns one
PERSONspan coveringAlice met Bobfor probabilities[0.9,0.1,0.9]at threshold0.5. A standalone CPU probe linked against this head's independently built library returned[0,13]and failed the expected two-span assertion. #195's active-span reset and regression should be backported to this slice as well as #176. - [P2] The executor logit gate can accept NaN/missing output. native_executor.rs:140–143 and the candidate/head reductions zip arrays and use
f32::maxwithout first asserting finite values and exact dimensions. All-NaN output reduces to zero in a CPU reproduction; a truncated array can omit comparisons. The tighten-only tolerance change does not close this. Carry over #195's finite/cardinality checks. - The actual token-plan exporter also creates a bytecode cache that makes the next package verification fail, as reported on #175. The head exporter needs the same import/initialization policy.
eadb785 restores the manifest construction and puts BF16 output inside the bundle's single staging tree. However, #175/#176 have since advanced their anchored load signatures; this head does not contain those newer commits. Reconcile those prerequisites explicitly, then rerun the integrated tests and refresh the PR's older head/check descriptions.
Personally run with a dedicated build directory: 28 Rust tests passed / 4 ignored, 135 Python tests passed, strict MkDocs passed. The span probe fails as described. No exported-bundle replay or GPU executor check was run. GitHub currently reports main conflicts.
Review scope and provenance
Head eadb78535fae5c9a5d9cf0b6a54f7d4c3a5a8e93; PR API base 05075d0f36e399e3a001ffedea400c666990cda9; diff merge-base 05075d0f36e399e3a001ffedea400c666990cda9. Inspected scope: executor/response integration and parity validation, actual export repair, and predecessor follow-up gaps. Complete changed-file inventory: 62 files, 6,594 changed authored-code lines / 7,845 total text lines (binary fixtures counted as files, not text lines).
Loaded trusted InferMatrixLab/system1-omni@5fa5179d57f9267bb0d20162004d8ce26352aa9d:.agents/skills/system1-omni-review/SKILL.md, SHA256 3a4b876162da14eee8db56af86664576aa5755b8f3e80b3072e944018647c3bc, plus architecture, repository contracts, CONTRIBUTING, self-review artifact requirements and CI/Docs workflows. Personally executed checks are distinguished above from author-reported GPU evidence. No GPU inference was run in this review; no approval or readiness transition is implied.
Importing vela2_inference and initializing the runtime with bytecode writes enabled leaves a __pycache__ in the package, which the verify_files gate then rejects at the next verification or worker startup. Disable bytecode writes across import and initialization — from_pretrained imports package modules lazily — and add a repeat-use regression covering both entry points. Backport of the PR InferMatrixLab#195 fix to this prerequisite slice.
…ve-processing # Conflicts: # Cargo.toml # README.md # docs/supported-models.md # recipe/README.md
…heads # Conflicts: # README.md # docs/supported-models.md # recipe/vela/native.md
A rejected valid word no longer merges into the surrounding span: the decoder tracks the current label separately from the last stored span, so probabilities [0.9, 0.1, 0.9] decode as two spans, not one covering span. Logit parity now goes through max_logit_drift, which rejects non-finite values and mismatched lengths before reducing, and the head-input exporter imports and initializes with bytecode writing off. Regressions cover the rejected-word decode, empty offsets, non-finite and misshaped logits, and repeat exporter use. Backport of the PR InferMatrixLab#195 fixes to this prerequisite slice.
…he executor layer Reconcile the prerequisite slices: Bundle::load, Heads::load, Calibration::load and Processor::load now take the manifest anchor, so Executor::load threads it through as well and the executor/response opt-in checks read VELA_NATIVE_MANIFEST_SHA256 like the processing and heads checks. Keep both sides' bundle-gate regressions and the Clef/Vela doc rows.
The executor replay's drift reductions zipped arrays without first asserting finiteness and exact cardinality, so an all-NaN or truncated readout could pass: an all-NaN forward reduces to zero drift and a shorter array silently omits comparisons. Route the comparisons through max_logit_drift (rejecting non-finite values and length mismatches), assert the recorded question, padded-candidate, candidate-readout and span-word counts, require every recorded q/span call to be consumed, and drop the silently-skipped missing-query branch. Backport of the PR InferMatrixLab#195 executor gate to this slice.
|
Fixed in bc0fb9c. Executor drift comparisons go through max_logit_drift and the replay asserts question, padded candidate, candidate readout and span word counts, with every recorded q and span call required to be consumed. 175 and 176 merged, the anchored load signatures reconciled, Executor threads the manifest anchor and reads VELA_NATIVE_MANIFEST_SHA256. Real main merged. 139 Python and 40 Rust tests pass, CI green. Opt in GPU replay not rerun, no bundle on this host. |
Levius-Fubuki
left a comment
Levius-Fubuki
left a comment
There was a problem hiding this comment.
Independent review, real-package CPU replay and CUDA execution at bc0fb9cb27857ae94198e3f79d8acd660736ef0d. Requesting changes: the current 0.8B GPU acceptance gate fails on the tested SM89 configuration.
Environment: RTX 4090 24GB, SM89, driver 580.105.08, nvcc 13.0.88; 16 available CPUs / 120 GiB container RAM; Rust 1.98.0, Python 3.12, Torch 2.11.0+cu130, Transformers 5.19.0. Each head had its own source-verified build target; tracked source files, committed JSON expectations and acceptance ceilings were unchanged.
GPU gate failure, reproduced without changes
The original opt-in vela_native_executor test was run twice with the trusted export's manifest anchor and the freshly built SM89 CUDA library. Both runs produced:
| Case | Worst logit drift | Result against 0.35 |
|---|---|---|
| example-request | 0.2296 | pass |
| example-request-set-threshold | 0.0114 | pass |
| example-request-cjk | 0.39117718 | fail |
| example-request-score | — | not reached after the assertion |
example-request-cjk: span logits drift 0.3912
example-request-cjk: worst logit drift 0.3912
example-request-cjk: drift 0.39117718 > 0.35
test result: FAILED. 0 passed; 1 failed
Reproduction uses the original targets and default ceiling:
NVCC=/usr/local/cuda-13.0/bin/nvcc \
bash src/backends/cuda/qwen3_5/build.sh "$CUDA_OUT" 89
VELA_NATIVE_BUNDLE="$BUNDLE" \
VELA_NATIVE_MANIFEST_SHA256="$EXPORT_MANIFEST_SHA256" \
VELA_CUDA_LIB="$CUDA_OUT/libqwen3_5_cuda.so" \
cargo test --release --locked --offline -p omni-vela-native \
--test vela_native_executor -- --ignored --nocapture --test-threads=1EXPORT_MANIFEST_SHA256 is the trusted exporter-produced external anchor. No gate override, source patch, quantization or offload was used. The original uniform-BF16 export/F32 heads profile was retained. Device-memory sampling peaked at 2,955 MiB; this failure was not an OOM.
All 7 selected shared CUDA kernel checks passed: device copy, flash attention vs float64, fused attention gate, GDN vs recurrent reference, residual RMS norm, SiLU multiply, and windowed attention vs full pass. Separately, the pinned Python GPU reference completed all four small cases with FP32 parameters/BF16 autocast (Torch allocated peak 3.95 GiB). An external diagnostic comparing unchanged native execution against those fresh GPU logits still failed CJK at 0.4371 > 0.35. That diagnostic did not replace the committed CPU-FP32 oracle or turn the original failure into a pass.
Please investigate the full-path numerical difference on this configuration and retain the 0.35 ceiling and finite/shape/call-count assertions. I have not isolated a single faulty kernel; the passing selected kernels do not establish the full executor's parity.
CPU and prerequisite status
- Vela Python: 139 passed; Rust crate: 40 passed, 0 failed, 4 ignored in the ordinary run.
- Ignored processing/head/response CPU replays were explicitly executed and passed on the real 0.8B bundle; the inherited long-span replay coverage bug is reported on #176.
- fmt, strict crate clippy, strict MkDocs and the source-specific CUDA build: passed.
- The prior span-reset, finite/dimension, consumed-call, manifest-anchor and nested-export fixes are present. The newly found canonical-state issue in #175 and window attribution issue in #176 still need propagation.
Package: vllm-sr/Vela-2.0-0.8B@326b01d81f61fff7fd4ae570e54b87a49119e490, independently hash-verified. Scope: base/merge-base 5fa5179d; 60 changed files, 6,065 code / 6,661 total changed lines; own delta over #176: 1,168 code / 1,444 total lines. CI/Docs success was observed at this exact head; this does not cover the failed opt-in GPU replay. Historical SM120 results are a different hardware/toolchain observation.
Advance conditions: propagate the #175/#176 fixes and obtain a passing original GPU replay for the intended supported profile, or explicitly resolve/document the supported hardware/profile boundary with evidence. Long-span/broad-head full GPU execution, HTTP serving and performance were not validated here.
Full logs, source/library/test-binary identities and probe sources are retained locally in the sealed evidence archive, SHA-256 c3e490e39b7e234e0bc6f666f7513226aada481cf5fc202ab82892f7efcc349c; raw archive is not attached here. The command and failure excerpt above are the reviewer-run reproduction.
Review method: the repository's system1-omni-review skill at trusted main 5fa5179d, SHA-256 3a4b876162da14eee8db56af86664576aa5755b8f3e80b3072e944018647c3bc; contributor self-review/split rationale, component interfaces and complete changed-file inventory checked. Results below are reviewer-run; previous author measurements and green weight-free CI are not substituted for model/GPU replay.
| "{case}: span calls" | ||
| ); | ||
| println!("{case}: worst logit drift {worst_logit:.4}"); | ||
| assert!( |
There was a problem hiding this comment.
[P2] This acceptance check fails on the reviewed head with RTX4090 24GB/SM89, driver580.105.08 and nvcc13.0.88. Two unchanged original replays both fail CJK at 0.39117718 > the default0.35; earlier cases are0.2296 and0.0114, score is not reached. Seven selected shared-kernel checks pass, but the full path remains unqualified here. Please investigate/pin the supported numerical profile and rerun this original gate without widening it. The review body records the command and diagnostic limits.
The recorder imported the package without running the worker's own verify_package gate, so a tampered package could execute code under the export path. Run the gate before sys.path gains the package, add a tampered-package regression, and note the ordering in native.md.
…heads # Conflicts: # recipe/vela/native.md
The head-input recorder shared the token-plan recorder's ungated import path; run verify_package before sys.path gains the package and extend the tampered-package regression to both exporters.
# Conflicts: # recipe/vela/native.md
Blank checks use Python str.strip's set (Rust whitespace plus the
\u{1c}-\u{1f} separators) so pathological controls answer the same
triples as the reference, including the state JSON-object sniff.
canonical() renders floats through the shared Python-parity formatter
(1e+20, 1.5e-07) instead of serde_json's exponent forms, so structured
state tokenizes identically to the pinned runtime, whose json.dumps
emits 1e-07 where serde_json emits 1e-7.
Carried from 83b2dfab; its scoring.rs gate-fallback hunk lands with the
heads slice, where the scoring module lives.
The canonical() unit tests pin Python json.dumps rendering on the review probe (1e+20, 1e-07) and the repr edge cases (two-digit signed exponents, decimal notation down to 1e-4, integral-float .0, bare integers). The opt-in bundle test walks the actual state -> parts -> tokenizer path and requires the pinned runtime's exact 17 ids, so a rendering drift fails instead of silently re-tokenizing.
The long-span replay grouped span calls by recorded row but selected the window logits from the parent row's calls after the first one. The recorder stores the windows as their own rows after the main rows, so the parent row 0 carried just the main call and the aggregation received an empty set, filling every word with -30 - masked only because the 0.8B long-span answer happens to contain no spans. Attribute each window by its recorded row, require exactly one span call on the parent row and on every window row, and assert at the end of each case that every recorded span-call row was consumed: the empty-window regression now fails structurally instead of passing with a -30 fill.
The manifest's own digest table is self-referential, so a bundle rewritten together with its manifest satisfied it; calibration.json, tokenizer.json and tokenizer_config.json are now also checked against the repository's pin table (mirrored from hybrid.py), which such a rewrite cannot satisfy. The directory walk refuses symlinks, verifies pinned directories one level deep instead of excusing them wholesale, and propagates entry errors instead of panicking. Drop the redundant onig package key.
VELA_GPU_LOGIT_GATE now rejects unparseable values and clamps wider ones back to 0.35, matching the heads test's documented ceiling rule. The executor asserts each block's query is its last token instead of debug-asserting the list is non-empty, so a render change that moves the query fails loudly instead of decoding the wrong position.
… tests assemble_budget is pure token arithmetic, so four unit tests (via the contract-module #[path] wiring) now pin the reference's assemble order: shrink floor 64 then 1, protected parts cut straight toward 1, and the Err that answers the whole-request 413 with the too_long triple. Also state the tokenizer-failure abort's actual reachability in its comment instead of claiming the error is surfaced.
Blank checks use Python str.strip's set (Rust whitespace plus the
\u{1c}-\u{1f} separators) so pathological controls answer the same
triples as the reference, including the state JSON-object sniff.
canonical() renders floats through the shared Python-parity formatter
(1e+20, 1.5e-07) instead of serde_json's exponent forms. A sparse gate
without t_sparse disables itself instead of renaming the rule under an
undefined threshold, and an unknown plan type fails loudly instead of
finishing as a span.
The response test feeds usage from the plan's rendered rows so the oracle's own usage must survive the drift comparison instead of being read back, and compares each envelope section's keys in insertion order, not just their count. The hybrid gate test names the hybrid pin revision it checks.
The runtime windows the un-truncated labelled part (overlap 264 at stride 1536, not 512 over the kept length), protected parts cut straight toward the 1-token floor, the threshold digits are labeled per member with the hybrid's own values, the byte-equal admission claim now carries the non-finite residual, the windowed long-span case's empty-spans coverage limit is stated, and the always-run test count is back in step (35). Drop the duplicated capabilities sentence in supported-models.md.
The sm_120 evidence is refreshed on the anchored, pin-table stack (worst logit drift 0.2845, all four cases inside the unchanged 0.35 gate). An independent sm_89 review reproduced a full-path CJK drift of 0.3912 against the same gate with every selected kernel check passing, so the supported hardware boundary is documented: sm_89 stays outside it until a passing replay there, and the gate and the ceilings are unchanged.
Stacked on #151 + #174 + #175 + #176 — incremental review: the anchored-load merge
5301e6f5and the gate fixbc0fb9cb.Required human contributor answers
What outcome are you trying to achieve?
Complete the native library: the SystemOne response envelope from predict answers (Noul views, set fan-out, span clipping, confidence, usage), and backbone execution on CUDA through the shared qwen3_5 crate. (Source: issue [New Model]: Support the Vela 2.0 series #137.)
What is the highest-leverage variable?
Backbone readout parity — the causality argument (full-sequence-per-position ≡ in-tree readout) must hold, and the bf16-vs-FP32 gap must stay within declared gates.
How will this work move that variable?
The opt-in GPU test replays the recorded reference logits through the Rust heads fed by the CUDA backbone — four cases, worst drift 0.285, gate 0.35 — independently reproduced by the precheck.
Who is the customer or user?
Contributors wiring the HTTP worker around Processor + Executor; ultimately users needing GPU-accelerated Vela decisions.
Is a PR needed? Yes — completes the native library.
For a bug fix: N/A.
Purpose
Two commits:
scoring.rs system_one_response— Noul views (span = range-filtered max word probability via per-label sigmoid, set fan-out), span clipping with re-trim,span_heads, thresholds, confidence formulas, usage. Replays recorded logits into frozen oracle responses.executor.rs— full-sequence-per-readout-position throughforward_batch(causality makes each hidden identical to the reference's in-tree readout; layout-a/c both handled). Uniform-BF16 backbone (pinned in the manifest); heads stay F32.Test Plan
System1-Omni Version / Commit:
model/vela2-native-executor@bc0fb9cbcargo test -p omni-vela-nativecargo clippy -p omni-vela-native --all-targets -- -D warningscargo clippy --workspace --locked --all-targets --features omni-laya/serve -- -D warningsmkdocs build --strictrecipe/vela/test_compare_reference.pyVELA_NATIVE_BUNDLE=… VELA_NATIVE_MANIFEST_SHA256=… --test vela_native_response -- --ignoredVELA_NATIVE_BUNDLE=… VELA_NATIVE_MANIFEST_SHA256=… VELA_CUDA_LIB=… --test vela_native_executor -- --ignored149a9bab)Test Result
All passed. GPU parity is opt-in (~5.7 min on tested hardware).
bc0fb9cb; vela: add the 0.8B native bundle export, wire contract and token-level processing #175, vela: add the 0.8B native F32 heads, scoring and span decoding #176 andinfermatrix/mainmerged. Every check in the table plusPYTHONPATH=src pytest tests/vela(139 passed),cargo fmt --all --checkand strict mkdocs rerun and green at this head; the opt-in checks now also requireVELA_NATIVE_MANIFEST_SHA256. GPU replay not rerun on this host.Demo / evidence
Author-run, x86 WSL2, RTX 5060 (sm_120), CUDA 13.1, CPU-FP32 fixtures:
Response parity (4/4 oracle, drift ≤1.5e-6):
GPU executor parity (4/4, logit drift vs CPU-FP32 reference, gate 0.35):
Independently reproduced by the precheck (212.6s vs author's 206s, identical drifts). First recorded execution of the shared qwen3_5 CUDA kernels on sm_120 (Blackwell).
Precision note: the uniform-BF16 backbone is a slightly lower-precision profile than the reference's GPU autocast (which keeps norms and GDN gate math in FP32); drift concentrates in span heads (0.05–0.28), q blocks at 0.0066. Long-span and broad-head GPU cases not measured.
Usage note:
input_tokenspasses through from the caller; the reference'ssum(n_input)is not reimplemented in native code. The response test feeds the oracle's value — usage parity rests on the reference until engine wiring.Not measured: serving latency, throughput, memory.
Self-review