knowledge: Drama-3 decode graphs on Ascend 910B1

docs/drama3-ascend-910b1.md

@@ -0,0 +1,97 @@
+# Drama-3 (fish_omni) on two Ascend 910B1
+
+Status: **INCONCLUSIVE** for the requested RTF targets. The decode FULL-graph
+change is author-reported, including a second measurement by the same author.
+The RTF < 0.1 (C1) and RTF < 1 (C64) bars are not met on this hardware.
+These figures are not independently reproducible validation until exact
+revision and evidence references are retained.
+
+## Provenance
+
+Measurements below are author-reported by amy-why-3459 on a shared 8x910B1
+host (cards 3/4), 2026-10-23/24. They are not independently reproducible
+validation until exact revision and evidence references are retained.
+`plan/plan.json` stores `<baseline-sha-from-acceptance-issue>` and
+`<candidate-sha-from-acceptance-issue>` as placeholders, not pinned SHAs.
+The upstream change is described as vLLM-Omni work "Replay Drama 3 decode
+steps as FULL graphs" on the mutable branch
+`perf/ascend-drama3-full-decode-graphs` (described as based on `c829af4`).
+That branch name is not an immutable candidate revision. The Drama-3 NPU
+runner path it builds on is
+[vllm-omni#9106](https://github.com/vllm-project/vllm-omni/pull/9106) (merged).
+Raw logs and the sweep JSON are external. They were retained under
+`<ARTIFACT_ROOT>/evidence/` and `<ARTIFACT_ROOT>/runs/` and are not in
+this repository, so they are unavailable for review from this checkout.
+This document stores the summary, not those artifacts. A field marking the
+summary locally verified would not make those external artifacts reviewable.
+
+## RunSpec
+
+| Field | Value |
+| --- | --- |
+| repo | vllm-project/vllm-omni, author-described as `c829af4` plus an unpinned FULL-graph change on mutable branch `perf/ascend-drama3-full-decode-graphs` |
+| backend | vLLM-Omni 0.32.0 nightly image `quay.io/ascend/vllm-omni:nightly-20261022-6b35bca`, vLLM-Ascend 0.19.1rc2, CANN 9.1.0, torch_npu 2.10.0.post4 |
+| device | 2 x Ascend 910B1 (A2), stage 0 TP2 + expert parallel (devices 0,1), stage 1 DAC on device 1 |
+| model | `fishaudio/drama-3` (Qwen3.5 MoE 40 layers, 256 experts / 8 active, block-FP8; 13-layer codebook tower; 44.1 kHz DAC) |
+| scenario | streaming `/v1/audio/speech`, Seed-TTS-Eval `en`, one fixed pre-encoded reference (85 frames x 10 codebooks), `max_tokens=500`, 64 prompts, 1 warmup |
+
+Metrics: audio seconds counted from SSE PCM14 payloads at 44.1 kHz mono;
+`frames/s = audio_s x 21.5 / wall`; per-request `RTF = (end - start) / audio_s`
+so queueing and prefill are inside the number, as the target definition asks.
+
+## Kept
+
+| Change | C1 frames/s | C1 RTF mean | C1 RTF p95 | C1 TTFA p50 | Source |
+| --- | ---: | ---: | ---: | ---: | --- |
+| baseline: stage-0 PIECEWISE capture | 17.31 | 1.237 | 1.301 | 0.91 s | upstream `fish_omni.yaml` |
+| decode `FULL_AND_PIECEWISE`, sizes to `max_num_seqs` | 22.4 | 0.962 | 1.141 | 0.80 s | this work, run 1 |
+| same, repeat | 22.91 | 0.947 | 1.036 | 0.78 s | this work, run 2 |
+| same, final sweep (host loaded) | 18.77 | 1.147 | 1.353 | 0.89 s | `runs/final/final-v2.json` |
+
+The host is shared (load average 15–20 during runs); the two clean repeats
+are the attribution, the loaded sweep is the conservative bound. The gain is
+one graph replay per decode step instead of ~53 layers of eager launches.
+
+Batching on the same build, best of repeats:
+16 streams 133 frames/s, 32 streams 137, 64 streams 305 (best of three:
+137 / 197 / 188, one later run 305); final sweep under load: 65 / 137 / 269.
+Spreading inside the sweep is large enough that single readings of C8–C32
+should not be compared across tables.
+
+## Rejected
+
+| Change | Result | Why it stays out |
+| --- | --- | --- |
+| bind `fla_npu.ops.ascendc.causal_conv1d_update` into vLLM's GDN update | C1 22.4–22.6 frames/s, RTF 0.95 — identical to the unbound build; FULL capture succeeds with the portable `causal_conv1d_update` bound | the vLLM-Ascend warning claims the fallback stalls decode-FULL capture, but on this stack it does not; no measurable effect |
+| keep the checkpoint's block-FP8 experts as FP8 | not attempted as a change: vLLM-Ascend builds `AscendW8A8MXFP8Dynamic*` only on 950 (`is_950()`), so A2 expands every expert to bf16 at load | would need a native A2 block-FP8 grouped GEMM; out of scope here |
+| `mamba_ssm_cache_dtype: bfloat16` | already the default in this deploy config | not a lever |
+
+## Where the time goes
+
+Phase timers in `fish_omni_model.forward` (eager, one stream, one request):
+backbone ~195 ms, tower + sampling ~72 ms, writeback ~0.1 ms per frame. With
+FULL graphs the same step is ~44 ms, so the graph removes most of the launch
+overhead, not the arithmetic.
+
+The C1 RTF < 0.1 target was not reached in this measured configuration.
+The kept rows above are the recorded outcome. Resident expert-weight
+capacity does not establish that the target's 4.65 ms/frame wall budget
+(0.1 s / 21.5 frames) is physically impossible: 8 of 256 experts are
+active, so that resident footprint is not the bytes read per generated
+frame. No bandwidth lower bound is claimed. Retaining one would require
+active-byte traffic, the applicable bandwidth, and an explicit calculation,
+which this record does not contain.
+
+## Operating notes learned the hard way
+
+- CANN 9.1.0 does not ship `aclnnAddRmsNormBias`; vLLM-Ascend's custom-op
+  vendor package must be registered (`${ASCEND_OPP_PATH}/vendors/config.ini`
+  `load_priority`, plus readable directory modes). A `750 root:root` vendor
+  directory makes `dlopen` fail with "Permission denied" and surfaces as a
+  missing-operator error.
+- `fla_npu`'s `.pth` rewrites `ASCEND_CUSTOM_OPP_PATH` at interpreter start;
+  set `FLA_NPU_DISABLE_PTH=1` when a specific vendor list is required.
+- `/health` answers before stage 0 finishes warming. A readiness probe must
+  issue one real synthesis request, or a sweep starts against a dead port.
+- Stage 1 (DAC) initialises after stage 0; the first request pays its engine
+  startup unless the probe absorbs it.

evidence/drama3-ascend-20261023/README.md

@@ -0,0 +1,16 @@
+# Drama-3 Ascend campaign (2026-10-23/24)
+
+Verdict: **INCONCLUSIVE**. One optimization is author-reported and kept (decode
+FULL graphs), two candidate changes were rejected. RTF targets are recorded
+as unmet. The numbers are author-reported, not independently reproducible
+validation: plan revisions are placeholders, the patch names a mutable
+branch, and raw logs and benchmark outputs are external and unavailable in
+this checkout. Registered in `tools/evidence_inventory.json` as a campaign
+bundle with no freeze manifest; unavailable external artifacts are listed
+there, and no freeze hashes were added. Details, RunSpec and operating notes:
+[docs/drama3-ascend-910b1.md](../../docs/drama3-ascend-910b1.md).
+
+- [plan/plan.json](plan/plan.json) — recorded protocol and success/stop rules; revision fields are placeholders
+- [results/summary.json](results/summary.json) — author-reported kept/rejected numbers
+- [patch/README.md](patch/README.md) — mutable branch pointer; the patch is not vendored
+- [validator/README.md](validator/README.md) — re-run notes; raw benchmark outputs are external

evidence/drama3-ascend-20261023/patch/README.md

@@ -0,0 +1,10 @@
+# Patch
+
+Candidate pointer: mutable branch `perf/ascend-drama3-full-decode-graphs`
+(`[Perf][Ascend] Replay Drama 3 decode steps as FULL graphs`), described as
+based on `c829af4`. Two files are named: `vllm_omni/deploy/fish_omni.yaml`
+and a CPU test in
+`tests/model_executor/models/fish_omni/test_fish_omni_config.py`.
+
+The branch name is mutable and is not an immutable revision. The patch is
+not vendored here, and no exact candidate SHA is retained in this bundle.

evidence/drama3-ascend-20261023/plan/plan.json

@@ -0,0 +1,16 @@
+{
+  "purpose": "Decode-graph optimization campaign for Drama-3 on Ascend 910B1; RTF targets are recorded, not claimed",
+  "hypothesis": "Drama-3 stage 0 samples one whole frame per stream per step, so decode is launch-bound; replaying decode as a single FULL ACL graph should cut single-stream RTF and raise frames/s without changing outputs",
+  "isolated_variable": "stage-0 NPU compilation_config.cudagraph_mode: PIECEWISE vs FULL_AND_PIECEWISE",
+  "control_revision": "<baseline-sha-from-acceptance-issue>",
+  "candidate_revision": "<candidate-sha-from-acceptance-issue>",
+  "revision_status": "control_revision and candidate_revision are placeholders. Exact baseline and candidate SHAs are not retained in this bundle.",
+  "engine_revision": "vLLM-Omni 0.32.0 nightly 2026-10-22 image; vLLM-Ascend 0.19.1rc2; CANN 9.1.0",
+  "devices": "2x Ascend 910B1 (A2), stage 0 TP2 + EP on devices 0,1; DAC stage 1 on device 1",
+  "protocol": "fishaudio/drama-3, streaming /v1/audio/speech, Seed-TTS-Eval en, one fixed pre-encoded reference (85 frames x 10 codebooks), max_tokens=500, 64 prompts, 1 warmup, concurrency 1..64",
+  "metrics": "frames/s = audio_s x 21.5 / wall; RTF = (end - start) / audio_s including queueing and prefill; TTFA from first SSE audio event",
+  "targets": {"rtf_mean_c1_lt": 0.1, "rtf_mean_c64_lt": 1.0},
+  "success": "C1 RTF mean improves by at least 20% with no request failures and no change to sampling semantics",
+  "stop": "Stop on stage-0 readiness failure, request failures above 1 percent, or host load above twice the quiet baseline",
+  "notes": "Host is shared (load average 15-20); single readings of C8-C32 are not comparable across tables. Only the FULL-graph change is in the candidate; the causal_conv1d_update binding was tested and rejected."
+}

evidence/drama3-ascend-20261023/results/summary.json

@@ -0,0 +1,19 @@
+{
+  "campaign": "drama3-ascend-20261023",
+  "verdict": "INCONCLUSIVE",
+  "verdict_reason": "Targets unmet (C1 RTF 0.95-1.15 vs <0.1; C64 RTF 2-5 vs <1); the kept change is measured but the acceptance bars for this campaign are not.",
+  "kept_change": {
+    "name": "decode FULL graphs for Drama-3 stage 0 on Ascend",
+    "c1_frames_per_s": {"control": 17.31, "candidate": 22.4, "repeat": 22.91, "loaded": 18.77},
+    "c1_rtf_mean": {"control": 1.237, "candidate": 0.962, "repeat": 0.947, "loaded": 1.147},
+    "c1_ttfa_p50_s": {"control": 0.91, "candidate": 0.80, "repeat": 0.78, "loaded": 0.89},
+    "batching": {"c16_best": 133, "c32_best": 137, "c64_best": 305, "c64_repeats": [137, 197, 188]}
+  },
+  "rejected": [
+    {"change": "bind fla_npu causal_conv1d_update into the GDN decode path", "result": "no measurable effect; FULL capture succeeds with the portable update", "evidence": "serve-full-without-conv1d-control.log"},
+    {"change": "native block-FP8 experts on A2", "result": "not available: MXFP8 method is 950-only; A2 expands experts to bf16", "evidence": "vllm-ascend fp8_block.py is_950() branch"}
+  ],
+  "measurement_status": "author-reported; not independently reproducible validation until exact revision and evidence references are retained",
+  "artifacts_retained_at": "<ARTIFACT_ROOT>/evidence and <ARTIFACT_ROOT>/runs",
+  "artifacts_availability": "external; unavailable in this checkout"
+}

evidence/drama3-ascend-20261023/validator/README.md

@@ -0,0 +1,13 @@
+# Validator
+
+Re-run: start the service with the candidate deploy config on two 910B1 cards,
+then run the streaming benchmark (`bench/bench_stream.py` in the campaign
+workspace) over concurrency 1, 2, 4, 8, 16, 32, 64 with 64 prompts, 1 warmup.
+
+Pass requires: no request failures, C1 RTF mean at least 20% below the control
+build measured the same day, and identical sampling parameters. Measure the
+control and candidate back to back; this host is shared and single readings
+move by tens of percent.
+
+The benchmark script and raw outputs are external and are not in this
+checkout, so these notes are not an independently replayable validator.

tests/test_evidence_inventory.py

@@ -145,6 +145,6 @@ def test_checked_in_inventory():
     repo = Path(__file__).resolve().parents[1]
     result = validate_inventory(repo)
     assert result["errors"] == []
-    assert len(result["bundles"]) == 21
+    assert len(result["bundles"]) == 22
     assert result["checked_hashes"] == 430
     assert result["unavailable"]  # Historical raw artifacts are not in this checkout.

tools/evidence_inventory.json

@@ -105,6 +105,13 @@
       ],
       "external": ["Owner-approved 8K search protocol activated prospectively in Issue67; campaign remains INCONCLUSIVE. Private source wrappers, operational logs, model outputs and runtime identities remain in controlled task storage."]
     },
+    "drama3-ascend-20261023": {
+      "kind": "campaign",
+      "manifests": [],
+      "external": [
+        "Author-reported summary only. Raw logs, benchmark/sweep outputs, and the candidate patch are external and unavailable in this checkout. No freeze manifest is recorded, and none is invented."
+      ]
+    },
     "drama3-b300-scale150": {
       "kind": "campaign",
       "manifests": [{"path": "plan/freeze.json", "schema": "bundle-sha256"}],