Skip to content

recipe(wav2vec2): add Polish Wav2Vec2 ASR functional Eval - #1343

Draft
ssss141414 wants to merge 9 commits into
microsoft:mainfrom
ssss141414:ssss141414/add-polish-wav2vec2-asr-eval
Draft

ssss141414 wants to merge 9 commits into
microsoft:mainfrom
ssss141414:ssss141414/add-polish-wav2vec2-asr-eval

Conversation

@ssss141414

@ssss141414 ssss141414 commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

Summary

This follow-up refreshes CPU FP32 and FP16 recipes for jonatasgrosman/wav2vec2-large-xlsr-53-polish, a Polish Wav2Vec2 CTC automatic-speech-recognition checkpoint originally added in #1207. The model-owned contribution is Effort L0 / Outcome L0 and reaches Goal L3 PASS_FUNCTIONAL_SMOKE: both CPU precisions pass build, performance, and PyTorch parity, while one representative FP32 CPU run proves end-to-end Polish ASR Eval operability. Exact C6 3d089489b96b913b49fcbe2210aa8bbeb2e43c3f is one commit directly atop approved D6 4b528133fef8cbeed4e9a3aba9be42d22a3c0221 on current main e28b128f5c2f69ecb2d73b63d2aea0a5ee8bddd0; merge Draft #1335 first, then update this Draft so its upstream diff is exactly the two recipes.

Model metadata

What the model does

This Apache-2.0 Wav2Vec2 XLSR-53 checkpoint transcribes Polish speech. It accepts a normalized 16 kHz waveform and emits frame-level logits over a 40-token Polish CTC vocabulary, which a CTC tokenizer collapses into text. This is verified from the pinned checkpoint/model card/config/tokenizer at 6b1cea36bd8bc5f65ec8081667cd9c0207d51970 and the WinML inspection contract.

Primary user stories

  • A user supplies Polish speech audio to obtain a Polish text transcription for captions, indexing, or downstream speech workflows. Model-card evidence; verified.
  • A developer supplies fixed 16 kHz waveform windows to obtain CTC frame logits for custom greedy or language-model-assisted decoding. Config/tokenizer/export-contract evidence; mapped.

Supported tasks

Task Support surface Evidence Confidence
automatic-speech-recognition checkpoint, Transformers, Optimum ONNX, WinML Pipeline tag; Wav2Vec2ForCTC; vendor Wav2Vec2OnnxConfig; WinML AutoModelForCTC inspection verified

Model architecture

Wav2Vec2ForCTC
├── Wav2Vec2Model
│   ├── Convolutional feature encoder x 7 (512 channels; total stride 320)
│   ├── Feature projection (512 -> 1024) + LayerNorm
│   └── Stable-LayerNorm encoder x 24
│       ├── Self-attention (16 heads, hidden size 1024)
│       └── Feed-forward (1024 -> 4096 -> 1024)
├── Dropout
└── Linear CTC head (1024 -> 40 logits; blank/pad token 0)
  • Source/confidence: pinned checkpoint config, tokenizer, model card, Wav2Vec2ForCTC source, and WinML inspection (verified; developer frame-logit story mapped).

Validation and support evidence

1. Baseline

  • Historical frozen baseline: microsoft/winml-cli main 0876e5ae1c98a169a6137e092e0d7b30bf9cee33, WinML 0.3.0.
  • Recipe-free FP32 baseline: PASS; build complete in 141.4 s (171.743236 s wall), AutoModelForCTC, input input_values float32 [1,16000], output CTC logits, 1,132/1,132 tagged export nodes, and a 1,261,923,716-byte external-data sidecar. That baseline spent 35.6 s in Optimize despite --no-optimize.
  • Starting auto-config selected task automatic-speech-recognition, model class AutoModelForCTC, model type wav2vec2, opset 17, eager Transformers attention compatibility, fixed float32 waveform input [1,16000], and no Eval section.
  • Baseline Perf was not run because post-build free space was 36.54 GB, below the planner's 40 GB optional-expansion target; no checkpoint baseline latency is claimed.
  • Baseline Eval was unavailable because main had no ASR evaluator. The initial dependency also selected optional-LM machinery without pyctcdecode and rejected the valid ordinary-Wav2Vec2 tokenizer's target_lang=null.
  • Optimum already supported automatic-speech-recognition; WinML added no new Optimum task (VENDOR-ONLY). Goal floor: L0.

2. Goal

  • Effort: L0; the model-owned delta is recipe-only.
  • Goal ceiling: L3; CPU FP32 and FP16 L0 build/structure, L1 bounded performance, L2 PyTorch parity, static Analyze, and one representative FP32 CPU two-row Polish FLEURS functional smoke with exact source identity and accounting.
  • Outcome: L0; ship only the two recipe JSON files. Shared evaluator behavior remains owned by feat(eval): add shared CTC ASR support for facebook/mms-1b-all #1335.
  • Charter r3 re-issued the selection contract after evidence falsified sort-by-semantic-ID. Charter r6 retained the model-stage contract while moving the exact stack to current main; it required composite source-row identity, seeded selection, honest mixed provenance, targeted current-main VED quality, and replacement public authority.

3. Outcome

  • Highest reached Goal: L3 PASS_FUNCTIONAL_SMOKE.
  • Coverage: cpu/cpu/fp32 and cpu/cpu/fp16 passed L0, L1, and L2. One representative cpu/cpu/fp32 run passed L3. No tuples are deferred.
  • Final stack: e28b128f5c2f69ecb2d73b63d2aea0a5ee8bddd0 -> approved D6 4b528133fef8cbeed4e9a3aba9be42d22a3c0221 (8 commits) -> C6 3d089489b96b913b49fcbe2210aa8bbeb2e43c3f (1 commit). C6 parent is exactly D6; C6 tree is 21ad6d9a44a2773ae033fcb5d82297a4168d3971; merge-base is current main.
  • Merge order: merge Draft feat(eval): add shared CTC ASR support for facebook/mms-1b-all #1335 first, then update/rebase recipe(wav2vec2): add Polish Wav2Vec2 ASR functional Eval #1343 so the upstream diff is exactly the two model recipes. The historical merged recipe(wav2vec2): add jonatasgrosman/wav2vec2-large-xlsr-53-polish ASR recipes #1207 is unchanged.
  • Shipped paths: examples/recipes/jonatasgrosman_wav2vec2-large-xlsr-53-polish/cpu/cpu/automatic-speech-recognition_fp32_config.json and automatic-speech-recognition_fp16_config.json in the same directory.
  • Lane A: gim-home/ModelKitArtifacts#254 remains Open/Draft at b7efe2b77c59eafbd13c48c4cb9beb9336df8305, labeled model-scale-by-skill, mergeable, with GitOps/AdvancedSecurity terminal SUCCESS. It retains wav2vec2-028, wav2vec2-027, and _meta-112. Final C6 review found no new methodology friction, so no content change or duplicate finding was warranted.

4. Per-EP/device/precision results and Functional smoke Eval

The full required CPU tuple matrix passed. FP16 reduced recorded RSS delta but was slower than FP32 on this CPU; no speedup is claimed.

Tier EP / Device Precision Verdict Structure / parity Mean p50 Throughput RAM Δ
L0 CPUExecutionProvider / cpu fp32 PASS IR 8; input_values FLOAT [1,16000]; logits FLOAT [1,49,40]; 424 FLOAT + 11 INT64 initializers
L0 CPUExecutionProvider / cpu fp16 PASS IR 8; FP32 I/O; logits [1,49,40]; 424 FLOAT16 + 11 INT64 initializers
L1 CPUExecutionProvider / cpu fp32 PASS 129.693 ms 130.705 ms 7.71 samples/s +91.01 MB
L1 CPUExecutionProvider / cpu fp16 PASS 154.715 ms 154.522 ms 6.46 samples/s +43.56 MB
L2 CPUExecutionProvider / cpu fp32 PASS cosine 0.9999999999982816; max abs 0.00007200241088867188; frame argmax agreement 1.0
L2 CPUExecutionProvider / cpu fp16 PASS cosine 0.9999972943487657; max abs 0.08947086334228516; frame argmax agreement 1.0

Historical provenance is explicit: L0 and L2 executed at cac2526b620df08fdbcc26ef60fe590479e55a0e; L1 Perf executed at C5 ac8acd690b48828eb7dfdd38c4aa5de9e3bcb14d using artifacts from cac2526b620df08fdbcc26ef60fe590479e55a0e; Analyze executed at C5 ac8acd690b48828eb7dfdd38c4aa5de9e3bcb14d over the same retained artifacts. All retained evidence was rehashed successfully and is adopted for C6, never relabeled as fresh C6 execution. VED/export/models quality and public acquisition/identity are fresh on exact C6.

Functional smoke Eval

CPU FP32 only; functional operability evidence only. This is not representative checkpoint accuracy, benchmark quality, or a claim that FP16 accuracy was measured. The retained artifact execution is from cac2526b620df08fdbcc26ef60fe590479e55a0e; evaluator execution is from 3440c5401d0111f06456b00662469daad2a6f0f6; the sealed result is adopted for exact C6 after D6 evaluator-series equality and full rehash. It is not fresh C6 model-stage execution.

  • Dataset: google/fleurs@70bb2e84b976b7e960aa89f1c648e09c59f894dd, pl_pl, validation, deterministic source order, no shuffle, non-streaming.
  • Schema/semantics: audio -> normalized 16 kHz float32 waveform -> fixed [1,16000] windows -> logits [1,T,40]; transcription references; argmax CTC collapse with blank/pad token 0; NFKC and whitespace normalization.
  • Fan-out caps: 2 selected utterances; batch size 1; at most 64 windows per utterance; 31 selected windows total; 488,640 waveform frames / 30.54 seconds total; one CTC decode per utterance; no beams, prompts, crops, or autoregressive generation.
  • Raw metrics: WER 0.7115384615384616; CER 0.17794486215538846.
Source index Dataset ID Audio Accounting Reference Prediction
0 1525 10018492969996036091.wav selected and processed szerokie bulwary przeszklone fronty budynków i nowoczesne galerie handlowe współistnieją z tradycyjnymi dachami krytymi czerwonymi dachówkami rynkiem z xviii wieku i starymi meczetami i kościołami choć w mieście panuje atmosfera przywołująca bardziej śródziemnomorską europę niż tradycyjną turcję szerokie bundwary przeszklone frąty budynków iowoczesne galeriehadlowe spór cnieją strady cyjnymi dachamii krytymi czerwonymi dachówkami rym kiedyś osiemnastegłowieku i starymi miaczetami i kościołami choć w mieście panujej atmosfera przy wołująca bardziej strudni ciemnał morską europewnistr ady cyjną turc ę
1 1657 10288018704489549018.wav selected and processed jakiekolwiek korekty lub żądania powinny trafiać najpierw do biura podróży a nie bezpośrednio do hotelu jekli końwek korekt yrub żądania powinle trafiać najpierdo biora podróży ani bezpośblednio do hotelu

Accounting closes exactly: requested 2, selected 2, processed 2, rejected 0, skipped 0, rejection reasons {}, predictions 2, references 2. Streaming and non-streaming probes selected the same ordered (source index, dataset ID, audio path) rows. D6 validates the composite identity before inference: missing/malformed index, dataset ID, or audio path and duplicate row/media identity fail with zero inference calls; repeated dataset IDs remain valid when source index and audio identity are distinct.

5. Delta

The model-owned C6 delta is one commit, exactly two files, 42 insertions(+), 2 deletions(-), with stable patch ID 4cb75500a8c98244c5e159ed93b3ba60c66b4575. It is patch-, subtree-, blob-, byte-, and semantic-equivalent to the accepted C5 recipe commit. There are no source, test, README, dependency, lock, workflow, or generated-report changes; the production recipe README is untouched.

Recipe JSON pointer Baseline C6
fp32 /eval absent task automatic-speech-recognition; google/fleurs / pl_pl / validation; revision 70bb2e84b976b7e960aa89f1c648e09c59f894dd; samples 2; shuffle false; seed 42; streaming false; audio -> input; transcription -> label
fp16 /eval absent identical to fp32
fp16 /quant null mode fp16; fp16_keep_io_types=true; remaining generated quant fields retained

Recipe loader values remain automatic-speech-recognition / AutoModelForCTC / wav2vec2; input remains float32 [1,16000]; output remains logits; fp32 quant remains null; fp16 retains FP32 I/O.

Bug fix explanation for the stacked dependency

  1. Symptom/minimal trigger: the prior dependency accepted selected rows with a missing dataset ID, missing audio path, boolean dataset ID, or duplicate audio path and made inference calls, allowing metrics over ambiguous row/media identity.
  2. Root cause: _selected_row_provenance and compute validated only integer source indices and duplicate indices, not the frozen composite (source index, dataset ID, audio path) identity.
  3. Changed mechanism: D6's Validate CTC selected row identities commit validates every composite field before inference, rejects duplicate row/media identity, preserves repeated semantic IDs when source and audio identities differ, and emits the exact selected IDs, indices, and audio paths. The final Honor seeded CTC evaluation shuffle commit closes deterministic seeded selection without changing that identity contract.
  4. General rule: validation is data-driven from row provenance and processor/tokenizer capability; no checkpoint-ID branch is used.
  5. Compatibility/blast radius: non-CTC ASR still fails closed; MMS configured-language behavior remains; fixed-window limits and processor-owned decode remain. Intentional changes are ordinary-Wav2Vec2 fallback, source-order/seed equivalence, composite pre-inference validation, and complete accounting.
  6. Regression evidence: exact public D6: focused CTC 50 passed, including malformed composite identity, duplicate audio identity, repeated dataset ID with distinct row/media identity, and pinned FLEURS identity tests; build compatibility 116 passed; VED 15 passed; five non-hardware CI partitions 8,543 passed, 78 skipped, 3 xfailed, 1 marker-deselected; all nine exact-head checks succeeded; independent review posted APPROVE with no blocker.

6. Analyze summary — component level and op level

Static rule analysis completed for both retained built artifacts with no errors or warnings. These classifications describe static rule coverage, not runtime execution on accelerator EPs; CPUExecutionProvider is the only runtime exercised here. Analyze executed at C5 ac8acd690b48828eb7dfdd38c4aa5de9e3bcb14d over artifacts from cac2526b620df08fdbcc26ef60fe590479e55a0e and was adopted for C6 after exact equivalence and rehash.

Component-level summary

Artifact Architecture coverage Mapping Actionable EP findings
fp32 7x convolutional feature encoder; projection; 24x stable-LayerNorm attention/FFN encoder; CTC head HTP export trace 1,132/1,132 tagged nodes; final graph 769 nodes No partial or unsupported operator types in populated rules
fp16 Same regions; FP16 weights with FP32 boundary I/O Same complete export hierarchy; final graph 771 nodes No partial or unsupported operator types in populated rules

Op-level summary

Artifact Graph Dominant ops Static EP roll-up
fp32 769 ops / 12 types Reshape 244; Gemm 146; Transpose 111; Add 73; LayerNormalization 57; MatMul 48 All 12 types supported by populated NvTensorRTRTX GPU, QNN NPU/GPU, and OpenVINO NPU/GPU/CPU rules
fp16 771 ops / 13 types Reshape 244; Gemm 146; Transpose 111; Add 73; LayerNormalization 57; MatMul 48; plus 2 Cast All 13 types supported by the same populated rule targets

CUDA/GPU, MIGraphX/GPU, TensorRT/GPU, DML/GPU, CPU/CPU, and VitisAI/NPU had no populated runtime-support classification in this rules set. Across 12 provider/device records per artifact there were zero analysis errors and zero warnings.

7. Reproduce commands

Fresh public acquisition and closure on exact C6 is 11/11 CLOSED: authority seals; public identity/parent/tree/merge-base/recipes; frozen initialization; self-contained public commands; clean candidate/dependency; authorized reuse rehash; prior r6 integrity rewalk; Polish L3 authority/provenance; exact read-only PR identity; terminal PR CI; successful PR CI. The fresh command rows were source binding, VED export, models partition, full Ruff, scoped Ruff format on C6 and D6, mypy, license, and both recipe config resolutions. The scoped format commands intentionally returned 1 for the same two inherited VED paths on D6 and C6; the sets are equal with no C6-only path, so this is PASS_NO_REGRESSION, not a clean format claim.

$Remote = 'https://github.com/ssss141414/winml-cli.git'
$Ref = 'refs/heads/ssss141414/add-polish-wav2vec2-asr-eval'
$Head = '3d089489b96b913b49fcbe2210aa8bbeb2e43c3f'
$Parent = '4b528133fef8cbeed4e9a3aba9be42d22a3c0221'
$Tree = '21ad6d9a44a2773ae033fcb5d82297a4168d3971'
$Repo = Join-Path $OUT 'candidate'

git clone --no-checkout $Remote $Repo
git -C $Repo fetch origin "$Ref`:$Ref"
git -C $Repo checkout --detach $Head
if ((git -C $Repo rev-parse HEAD).Trim() -ne $Head) { throw 'HEAD mismatch' }
if ((git -C $Repo rev-parse 'HEAD^').Trim() -ne $Parent) { throw 'Parent mismatch' }
if ((git -C $Repo show -s --format=%T HEAD).Trim() -ne $Tree) { throw 'Tree mismatch' }

Push-Location $Repo
try {
  Remove-Item Env:PYTHONPATH -ErrorAction SilentlyContinue
  uv sync --locked --extra audio --all-groups
  uv run --no-sync python -c "import pathlib,winml.modelkit; from winml.modelkit.models.hf import vision_encoder_decoder; from winml.modelkit.eval import ctc_asr_evaluator; root=pathlib.Path.cwd().resolve(); paths=[pathlib.Path(winml.modelkit.__file__).resolve(),pathlib.Path(vision_encoder_decoder.__file__).resolve(),pathlib.Path(ctc_asr_evaluator.__file__).resolve()]; assert all(root in path.parents for path in paths)"
  uv run --no-sync pytest tests/unit/export/test_vision_encoder_decoder_onnx_config.py --tb=short --no-cov
  uv run --no-sync pytest tests/unit/models tests/unit/loader tests/unit/datasets tests/unit/export --tb=short --no-cov -m 'not e2e and not npu and not gpu'
  uv run --no-sync ruff check src/ tests/
  uv run --no-sync ruff format --check src/winml/modelkit/models/hf/vision_encoder_decoder.py tests/unit/export/test_vision_encoder_decoder_onnx_config.py
  uv run --no-sync mypy -p winml.modelkit
  uv run --no-sync pre-commit run insert-license --all-files
  uv run --no-sync winml config -m jonatasgrosman/wav2vec2-large-xlsr-53-polish -c examples/recipes/jonatasgrosman_wav2vec2-large-xlsr-53-polish/cpu/cpu/automatic-speech-recognition_fp16_config.json -o (Join-Path $OUT 'resolved-fp16.json')
  uv run --no-sync winml config -m jonatasgrosman/wav2vec2-large-xlsr-53-polish -c examples/recipes/jonatasgrosman_wav2vec2-large-xlsr-53-polish/cpu/cpu/automatic-speech-recognition_fp32_config.json -o (Join-Path $OUT 'resolved-fp32.json')
} finally {
  Pop-Location
}

Fresh exact-C6 results: VED export 15 passed; models partition 1,538 passed, 6 skipped, 2 xfailed; Ruff check passed; mypy found no issues in 440 source files; license passed; both config resolutions passed and remained byte/semantic-equivalent to C5. All nine exact-head GitHub checks are terminal SUCCESS.

@ssss141414 ssss141414 added the model-scale-by-skill Model support PR created or maintained by the adding-model-support skill label Aug 23, 2026
@ssss141414

Copy link
Copy Markdown
Contributor Author

Independent review of Draft #1343 at exact head 78e0713b0ac19142cc6abf65ec93ff359d4d2749 found one blocking provenance defect in the stacked dependency.

P1 - Enforce the frozen selected-row identity contract before inference. _selected_row_provenance requires only an integer source index, and compute rejects only duplicate source indices. Charter r3 freezes each selected identity as source index + dataset ID + audio path, requires missing identity fields and duplicate audio paths/row identities to fail before inference, and requires the pinned result to match the exact ID/path sequence. The current focused regression only duplicates _winml_source_index, so it does not cover that contract (test).

An exact-head discriminator supplied distinct valid source indices but (separately) a missing dataset ID, missing audio path, duplicate audio path, and boolean dataset ID. Every case was accepted and each made two inference calls; for example, duplicate same.wav rows were emitted as two selected rows. This falsifies the PR body's claim that missing/malformed/duplicate selected identities fail before inference with zero calls and can certify metrics over an ambiguous denominator. Preserve repeated semantic IDs at different source indices; the defect is incomplete composite row/media identity validation, not repeated semantic IDs themselves.

Required handoff:

  1. Dependency producer for feat(eval): add shared CTC ASR support for facebook/mms-1b-all #1335: implement charter-compliant pre-inference identity validation and focused tests for missing/malformed identity fields, duplicate composite row/media identity, exact emitted IDs/indices/audio paths, and zero inference calls on rejection.
  2. Tester: rerun focused CTC regressions, selection/provenance discriminators, final-SHA L3, affected static/CI gates, and regenerate public closure/integrity evidence.
  3. Model producer: restack the unchanged two-recipe patch on the accepted repaired dependency and re-prove the exact two-file model-owned range.
  4. Explainer: update recipe(wav2vec2): add Polish Wav2Vec2 ASR functional Eval #1343's head/body/evidence bindings and public commands to the replacement exact head.

Verified independently: exact stack and two-recipe range; recipe schema and FP16/FP32 semantics; all 81 sealed paths/bytes/SHA-256 values; pinned FLEURS source rows; exact corpus WER 0.7115384615384616 and CER 0.17794486215538846; focused CTC 35 passed; build tests 116 passed; Ruff, mypy, targeted format; clean checkout; nine exact-head GitHub checks; zero review threads; live Lane A findings/check.

REQUEST_CHANGES

@ssss141414
ssss141414 force-pushed the ssss141414/add-polish-wav2vec2-asr-eval branch from 78e0713 to 3d08948 Compare August 24, 2026 13:52
@ssss141414

Copy link
Copy Markdown
Contributor Author

Resolved the provenance defect reported in comment 5388804450 on the replacement dependency/model stack.

Dependency fix on #1335: approved D6 is 4b528133fef8cbeed4e9a3aba9be42d22a3c0221, an unchanged 8/8 replay on current main e28b128f5c2f69ecb2d73b63d2aea0a5ee8bddd0. The Validate CTC selected row identities change now validates the complete (source index, dataset ID, audio path) identity before inference, rejects missing/malformed fields and duplicate row/media identity with zero inference calls, preserves repeated dataset IDs when source index and audio identity are distinct, and emits exact selected IDs/indices/audio paths. The final Honor seeded CTC evaluation shuffle change closes deterministic seeded selection without weakening that composite identity contract.

Fresh exact-D6 public evidence: focused CTC 50 passed, including malformed composite identity, duplicate audio identity, repeated dataset IDs with distinct row/media identity, and pinned FLEURS identity coverage; build compatibility 116 passed; VED 15 passed; all five non-hardware CI partitions 8,543 passed, 78 skipped, 3 xfailed, 1 marker-deselected; Ruff, mypy (440 source files), license, and changed-file format gates passed. The independently reviewed public D6 has all 9 exact-head checks terminal SUCCESS, zero unresolved threads, and an APPROVE opinion.

Model restack on #1343: exact C6 3d089489b96b913b49fcbe2210aa8bbeb2e43c3f has direct parent D6 and is exactly one unchanged recipe commit: 2 files, 42 insertions(+), 2 deletions(-), stable patch ID 4cb75500a8c98244c5e159ed93b3ba60c66b4575, with matching recipe subtree, blobs, bytes, SHA-256 values, and resolved semantics. No source, test, README, dependency, lock, workflow, or generated-report path is added by C6.

The pinned Polish two-row result remains WER 0.7115384615384616, CER 0.17794486215538846, source indices [0,1], dataset IDs [1525,1657], audio paths 10018492969996036091.wav and 10288018704489549018.wav, with requested/selected/processed 2/2/2, rejected/skipped 0/0, predictions/references 2/2. Provenance is explicit: artifact execution cac2526b620df08fdbcc26ef60fe590479e55a0e, evaluator execution 3440c5401d0111f06456b00662469daad2a6f0f6, adopted for C6 after exact D6 equality and rehash; it is not represented as fresh C6 model-stage execution.

Fresh exact-C6 public closure is 11/11 CLOSED, including authority seals, identity/parent/tree/merge-base/recipes, clean public acquisition, self-contained commands, reuse rehash, L3 provenance, and terminal-success PR CI. VED 15 passed; models 1,538 passed, 6 skipped, 2 xfailed; Ruff check, mypy (440 source files), license, and both config resolutions passed. Scoped format returns the same two inherited VED paths on D6 and C6 with no C6-only path: PASS_NO_REGRESSION, not a clean global-format claim.

The PR body and public commands now bind exact C6/D6/current-main identity and preserve the mixed historical execution provenance. Draft state and model-scale-by-skill remain unchanged. All 9 checks on exact C6 are terminal SUCCESS; complete review-thread pagination currently contains 0 threads and therefore 0 unresolved threads.

@ssss141414

Copy link
Copy Markdown
Contributor Author

APPROVE

  • Reviewed exact C6 head 3d089489b96b913b49fcbe2210aa8bbeb2e43c3f against exact D6 parent 4b528133fef8cbeed4e9a3aba9be42d22a3c0221 and current main e28b128f5c2f69ecb2d73b63d2aea0a5ee8bddd0. C6 is one unchanged recipe commit: exactly the Polish FP16 and FP32 CPU recipes, stable patch ID 4cb75500a8c98244c5e159ed93b3ba60c66b4575, recipe subtree 86d5e77896c9742b33a6e2638a58b39659584de7, exact frozen blobs/bytes/SHA-256 values, and clean source-bound checkout state.
  • The prior selected-row provenance blocker is closed by approved D6: complete (source index, dataset ID, audio path) validation occurs before inference, malformed or duplicate row/media identity is rejected with zero calls, repeated semantic IDs remain valid when row/media identity differs, and deterministic seeded selection preserves exact emitted provenance. Independent C6 replay passed CTC 50, build 116, and VED 15 focused tests.
  • Every non-hardware workflow partition passed on exact C6: Analyze 1,529; Models 1,538; Optim 848; Commands 3,692; Remaining 936, totaling 8,543 passed, 78 skipped, 3 xfailed, and 1 marker-deselected. Full Ruff, package mypy (440 source files), and license passed. The two scoped format findings are inherited D6/main VED files; C6 introduces zero format-failure paths.
  • Both config resolutions match the frozen outputs: FP16 1513b371ddd68ab678f2a121a69557600fd6d325323ffc9d91a6f860262735c3; FP32 1199a5704a4f39fbfe00f8d05b410e9ae5d04ccbe31818cad7d68705ed26204b.
  • Reachable coverage is complete for both required CPU tuples (cpu/cpu/fp32, cpu/cpu/fp16) through build and performance; there are no deferred tuples. The representative FP32 CPU L3 Polish ASR smoke remains honestly attributed to its original artifact/evaluator execution, with WER 0.7115384615384616, CER 0.17794486215538846, source indices [0,1], dataset IDs [1525,1657], exact audio identities, and 2/2 processed with no skips or rejects.
  • Public authority is closed: the tester verdict is PASS/READY (11/11), its sealed integrity manifest independently rehashed 3,356/3,356 entries with zero mismatch, and all four authority sidecars rehash exactly. Live feat(eval): add shared CTC ASR support for facebook/mms-1b-all #1335 and recipe(wav2vec2): add Polish Wav2Vec2 ASR functional Eval #1343 heads remain exact; both PRs are Open/Draft with model-scale-by-skill, all 9/9 exact-head checks are terminal-success, and unresolved review threads are 0 on each.

No actionable blocker remains. This is the single skill-level terminal reviewer opinion as a normal PR comment; it does not submit GitHub Review state, change Draft/readiness or labels, edit or resolve threads, push, or merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

model-scale-by-skill Model support PR created or maintained by the adding-model-support skill

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant