Skip to content

recipe(vitpose): add unvalidated QNN NPU candidates - #1419

Draft
Qiong Wu (qiowu) (DingmaomaoBJTU) wants to merge 1 commit into
microsoft:mainfrom
DingmaomaoBJTU:dingmaomaobjtu/add-usyd-community-vitpose-plus-base-recipe
Draft

Qiong Wu (qiowu) (DingmaomaoBJTU) wants to merge 1 commit into
microsoft:mainfrom
DingmaomaoBJTU:dingmaomaobjtu/add-usyd-community-vitpose-plus-base-recipe

Conversation

@DingmaomaoBJTU

@DingmaomaoBJTU Qiong Wu (qiowu) (DingmaomaoBJTU) commented Sep 15, 2026

Copy link
Copy Markdown
Collaborator

Summary

UNVALIDATED DEPENDENT DRAFT — candidate files only. All current model validation is NOT_RUN; no model Goal or support acceptance is claimed. The publication exception does not authorize merge, ready-for-review status, or validated-support claims.

This proposes two QNN/NPU recipes for usyd-community/vitpose-plus-base (keypoint-detection): an optimization-enabled FP32 fallback candidate and a W8A8 candidate with retained historical quality failures. The scope is recipe-only, with no CLI, source, test, or production-index changes. The user-selected revision-2 “先发待验证 Draft” exception permits publication before model acceptance, not acceptance itself. A trustworthy canonical no-optimize baseline remains pending separate #1401; that is not a proven runtime dependency of these optimization-enabled recipe builds.

Model metadata

Frozen source-only profile for checkpoint usyd-community/vitpose-plus-base, historical revision 92be54d7a29e42fad47b6e2ca01dd9e685a61e0d, model type vitpose, class VitPoseForPoseEstimation. The four fields below retain their stored order, claims, and confidence labels. These labels concern checkpoint/Transformers source facts, not WinML target support; no ONNX graph was loaded for the profile.

What the model does

A top-down RGB image and person-box model that processes normalized person crops through a ViT-style backbone and pose decoder to produce 17-keypoint heatmaps, which the processor maps to image coordinates. The person detector is external and is not part of this model.

Primary user stories

Supported tasks

Task Support surfaces Confidence
keypoint-detection checkpoint, transformers verified

Evidence: pinned model card, pinned checkpoint configuration, and Transformers v4.57.6 VitPose. These support surfaces are not QNN/NPU validation results.

Model architecture

The model uses a patch-embedded ViT-style encoder with twelve MoE-augmented encoder layers, followed by a final backbone normalization/stage output and a classic deconvolutional pose decoder that produces 17 heatmaps.

RGB image + person boxes
Normalized person crops
Patch Conv2d + position embedding
Encoder x12
  Attention: 12 heads + LayerNorm + residual paths
  MoE within each encoder layer: fc1 768->3072 + GELU
    Shared 3072->576 plus 6 experts 3072->192, concatenated to 768 with dataset_index
Final backbone LayerNorm / stage 12
Classic decoder: deconv 768->256, deconv 256->256, BatchNorm + ReLU
Conv 256->17 keypoint heatmaps
Processor maps heatmaps to coordinates

Frozen-profile provenance caveat: the outline includes external image/box preprocessing and coordinate postprocessing. Within each encoder layer, dataset_index selects the 192-wide expert contribution, combined with 576 shared features to make 768; the six-expert line does not mean concatenating all six expert outputs. MoE is inside the encoder, not the decoder; final backbone normalization follows the encoder stack. This is the original profile's clarification, not new architecture or runtime analysis.

Validation and support evidence

Baseline

  • Tested source base: da5dbcd5812daaabb64b89b47c24db2da2d4e1c7. The tester verified imported WinML source against that local tree with the two candidate additions present. Current committed candidate: 1fd43e7abd3914620f090255c0a7546694d9b205, whose parent is the tested source base; the committed recipe bytes match the tester's frozen candidates. Shipment verified local HEAD and the fork branch at that exact candidate SHA. This is Git/byte provenance, not rerun tests, model validation, or validation of newer main. Earlier provenance separately recorded source project version 0.3.1 and installed distribution metadata 0.3.0; those are distinct identifiers, not proof of environment equivalence.
  • Current canonical baseline: NOT_RUN / CLI-BLOCKED pending fix(build): honor CLI optimization bypass #1401, also under the no-model-work resource hold. The frozen handoff describes fix(build): honor CLI optimization bypass #1401 as open, Draft, and unmerged; its live state was not queried here. It repairs the CLI skip-optimize baseline path, and no dependency code is included in this recipe change. Both candidate builds are optimization-enabled (CLI default optimize=true, fusion settings retained, no authored skip_optimize); neither FP32 nor W8A8 is the canonical raw/no-optimize baseline.
  • Starting auto-config anchor is historical, last compared on dcad2f4a29cb34af71d7b138bcafc4276a70a4ed, not regenerated or validated on the recorded test base. It omitted /auto (default true), had /optim={}, static uint8-weight/uint16-activation quantization, and /compile=null. That earlier unqualified config probe also triggered QNN catalog acquisition; it was not repeated or represented as side-effect-free evidence.
  • Historical diagnosis only: on that earlier comparison commit, the Optimum probe returned VENDOR+OVERRIDE, exit 0: vendor and post-WinML task lists both contained keypoint-detection, with added_by_winml=[]. The vendor exporter was optimum.exporters.onnx.model_configs.VitPoseOnnxConfig; the existing WinML override was winml.modelkit.models.hf.vitpose.VitPoseIOConfig. This is not a new exporter/task implementation or a current model test.
  • No accepted current build/perf/Eval floor. An older CPU build attributed by the original charter to cf437687d1c629299df3260a81f02395dc4743cf reported completion in 71.0 s, including 52.7 s export and 17.1 s optimize. Its exact original argv is unrecovered: an Optimize stage is not proof that a no-optimize request was honored. The separate historical CPU smoke is reported below. Current build time, graph/node counts, latency, and task metrics are not established.

Goal

Revision 2 explicitly supersedes the provisional revision-1 L3 aspiration for Draft delivery only. Effort is limited to copying two historical per-checkpoint/target configurations with warnings and the portable W8A8 processor identifier: L0-shaped recipe scope, not an accepted support tier. The amended delivery goal is two candidates plus candid dependent-Draft publication and an independent limited scope/honesty review. This PR publishes the committed candidates under that exception; the completed limited review is recorded in Outcome and is not model-support approval or a normal-workflow APPROVE.

validation_goal_status=PENDING_NOT_ACHIEVED; highest_achieved_goal=null. L0, L1, L2, and L3 remain NOT_RUN for all four required QNN/NPU precisions. Schema checks do not establish model build acceptance. The reported free-space figure, 18.4 GiB versus the 25 GiB model-stage minimum, was not re-probed; no model work or cleanup was authorized. Missing model evidence intentionally does not block this Draft-only handoff. Future merge, ready status, or validated-support claims still require their own acceptance evidence; no such gate is waived.

Outcome

This Draft proposes commit 1fd43e7abd3914620f090255c0a7546694d9b205 from fork/branch DingmaomaoBJTU/winml-cli:dingmaomaobjtu/add-usyd-community-vitpose-plus-base-recipe, targeting microsoft/winml-cli main. The commit adds exactly two candidate files (+57 / +77, no deletions):

Both remain CANDIDATE / UNVALIDATED; coverage is UNVALIDATED, not full or partial accepted support. Deferred tuples are qnn/npu/fp32, qnn/npu/fp16, qnn/npu/w8a8, and qnn/npu/w8a16. No QNN/NPU FP16 or W8A16 recipe is added. No source, CLI, tracked test, API, generic recipe, or production-index change is included, and nothing is deleted. The generic FP16 recipe, generic W8A8 recipe, and recipes README/index remain unchanged.

Learner findings are an internal scratch-only handoff, not appended model knowledge or a completed methodology update. Under the explicit exception, no new Lane A/source/skill/profile changes are included or required for this Draft-only step. Ordinary support approval is not claimed. The separate limited scope/honesty review is recorded below; no GitHub approval, ready-for-review transition, or merge is authorized. GitHub check status is separate from the tester's bounded local checks and does not establish model acceptance.

Publication closure — 2026-09-15 17:27 UTC: head 1fd43e7abd3914620f090255c0a7546694d9b205; independent limited review, transparently relayed: LIMITED_SCOPE_HONESTY_CHECK: NO_FINDINGS. Not model approval: candidates remain UNVALIDATED, current L0–L3 NOT_RUN. Latest API snapshot (2026-09-15 17:25 UTC): 9/9 visible CI checks SUCCESS, not model-support validation.

Independent bounded offline checks, as recorded by the tester (not rerun by the explainer):

Check / attempt Recorded result Scope
Initial bootstrap Exit 4; 0 tests executed Tester guard incorrectly rejected Windows NUL during dependency import
Intermediate attempt Exit 1; 91 passed, 6 failed of 97 Four scratch tuple/JSON comparisons, one blocked alias-policy resolver, and its guard rollup
Final bounded run Exit 0; 97 passed, no failures/skips 34 config + 26 discovery + 9 static export-policy + 28 scratch candidate/history checks
Repository-only replay Exit 0; 60 passed Same 34 config + 26 discovery tests: a subset replay, not 157 unique checks
Final integrity audit 8 passed Separate metadata/handoff/preservation checks, no additional model coverage

Actual local environment: CPython 3.11.16, AMD64, 64-bit, existing Transformers 4.57.6 / Hugging Face Hub 0.36.2 compatibility overlay, pytest 9.1.1, pytest-cov 7.1.0, and pytest-timeout 2.4.0. The editor-selected Python 3.14.3 was not used for these imports/tests. No packages were installed and no fresh public environment was certified.

Earlier harness mistakes were corrected scratch-only: recognize Windows NUL before path normalization; compare tuple-backed serializer shapes after actual JSON encoding while retaining strict historical JSON comparisons; isolate only the explicit alias-policy test with a pure-intent resolver mock. No production/candidate/tracked-test fix or skipped failing test was used. Actual candidate loaders, serialization, completeness validation, and discovery were not mocked. The final guarded run recorded zero forbidden attempts; saved preservation checks covered 62 protected text hashes and three repository states, not a recursive model-payload inventory.

Retained native ORT import warning: Init provider bridge failed. from CreateOrtEnv. It is not a target-runtime verdict or a pytest warning record; the run is not described as warning-free. No model payload I/O, model/config generation, export, inference, explicit provider discovery/acquisition/registration, network/download, installation, or cleanup work was performed in the bounded tester phase. Its sole CLI invocation was help-only to verify option spelling. The tester did not run full suites or repository-wide static checks and did not verify CI; CI is reported separately in the API snapshot above.

Per-EP/device/precision results and Functional smoke Eval

Current results on the recorded local base — no historical acceptance carried forward:

QNN/NPU precision Recipe disposition L0 build L1 perf L2 numeric L3 task metric Mean / p50 / throughput / RAM / VRAM
FP32 CANDIDATE / UNVALIDATED NOT_RUN NOT_RUN NOT_RUN NOT_RUN NOT_RUN
FP16 NO_TARGET_RECIPE / UNVALIDATED NOT_RUN NOT_RUN NOT_RUN NOT_RUN NOT_RUN
W8A8 CANDIDATE / UNVALIDATED NOT_RUN NOT_RUN NOT_RUN NOT_RUN NOT_RUN
W8A16 NO_TARGET_RECIPE / UNVALIDATED NOT_RUN NOT_RUN NOT_RUN NOT_RUN NOT_RUN

Current per-tuple datasets/revisions, task metrics, runtime artifacts, and memory values are not established. FP16 has no exact QNN/NPU historical attempt; a generic filename proves neither support nor unsupported status. W8A16 retains its historical failure below, with no new recipe, repair/search, or performance ranking.

Historical numeric evidence only

Historical QNN source: 169a6f0122d1e1dd7d60c9605b5f19add26a209b. Historical checkpoint revision: 92be54d7a29e42fad47b6e2ca01dd9e685a61e0d. Historical environment metadata records ONNX Runtime Windows ML 1.24.5.202604171637 and QNN EP package 2.2480.53.0. These identify old evidence, not execution of the new portable configurations; model payloads were not reopened or rehashed for these checks.

Historical comparison Samples Recorded disposition Minimum / single cosine Overall cosine Exact heatmap peaks
FP32 three-person 3 Historical numeric pass only 0.9999722983142216 0.9999878550245817 51/51
W8A8 full three-person 3 FAIL 0.982657340445321 0.9908252702248718 31/51
W8A8 wallpaper 1 FAIL 0.6957623048200067 0.6957623048200067 Not recorded
W8A8 posthoc two-person core 2 Historical subset PASS only; not a rescue 0.9937934557606259 0.9947364296310016 23/34
W8A16 three-person 3 FIDELITY_FAIL 0.4100480666185648 0.6330862896099927 4/51

The recorded threshold is 0.99. Full-set W8A8 sample cosines are 0.9957561025512272, 0.9937934557606259, 0.982657340445321; the posthoc core retains only the first two. Neither its higher minimum nor the full-set overall cosine overturns the original full-set/wallpaper failures. These comparisons are not representative W8A8 task accuracy.

Historical paired latency only — no new performance measurement

The recorded protocol was 8 alternating A/B and B/A pairs, 16 sessions, 20 warmups and 100 measured iterations per session, QNN/NPU with burst mode and graph-finalization mode 3. Tester checked the 16 stored child results, each with 100 raw samples; no benchmark was rerun. Here “baseline” means the historical optimization-enabled FP32 comparator, not the pending canonical raw baseline.

Historical variant Mean of session means Mean of session p50s QNN memory
FP32 comparator 9.386 ms 9.30875 ms NOT_MEASURED
W8A8 experiment 3.634625 ms 3.537875 ms NOT_MEASURED

Recorded mean paired speedup: 2.5840062576234804x, 95% CI [2.539734340098991, 2.620416130903574]. Recorded mean gain fraction: 0.6127990435719093, 95% CI [0.6059867221376429, 0.6183396084021368]. Recorded p50 paired speedup: 2.631350525801282x, 95% CI [2.6149376770791664, 2.648977705483742]; p50 gain fraction 0.6199336171460872, 95% CI [0.6175667739774706, 0.6224590182454179]. These are stored paired statistics, not a newly computed ratio of rounded means; averages of session p50s are not pooled percentiles. No aggregate throughput is invented from latency.

Historical no-memory scope is retained: no child result has a memory section, and CPU RSS cannot substitute for QNN memory. Latency does not cancel failed W8A8 fidelity or establish correctness-gated support for either portable candidate.

Functional smoke Eval — current FP32 CPU NOT_RUN

There is no current final-artifact FP32 CPU evaluation, current sample/crop-cap verification, or current task metric. The separate historical one-image CPU result is end-to-end operability evidence only, not representative accuracy or W8A8 accuracy. Its source attribution, cf437687d1c629299df3260a81f02395dc4743cf, comes from the original charter, not an embedded source SHA in the result.

The saved COCO17 functional subset used the validation label, no shuffle, seed 42, and one image, one prediction, one ground truth; the result does not record a dataset revision. Raw metrics: map=0.9999999999999998, map_large=0.9999999999999998, map_50=0.9999999999999999, map_75=0.9999999999999999, map_medium=-1.0, mar=1.0, mar_50=1.0, mar_75=1.0. The medium-size metric has no denominator; it is not negative task accuracy. AP approximately 1 on this one image does not establish general quality.

The separate old CPU FP32 performance smoke used 1 warmup, 3 iterations, with mean 180.186 ms, p50 179.868 ms, 5.55 samples/s, and total RSS delta 30.6 MB. None of these values closes current model acceptance or measures QNN memory.

Delta

The tester independently verified complete recursive JSON comparisons: object order/whitespace ignored, array order/scalar types preserved, and absent distinct from null. Only /_note is excluded from the semantic counts; its full new values are disclosed below. FP32/W8A8 have 0/1 semantic changes against their respective historical configurations, and 5/7 against the historical auto-config anchor. These are not byte-equality or runtime-equivalence claims.

Public-display redaction: the sole historical machine-local processor string is represented below as "<historical-local-processor>", not printed as an inaccessible absolute path. This is a documentation token, not a literal configuration value or a pinned revision. The tester compared the exact original string; all other displayed values are literal, with every object field retained.

Historical configuration → candidate JSON pointer Old value New value
FP32 → FP32 None, excluding /_note All semantic fields Identical; 0 changes
W8A8 → W8A8 /quant/model_id "<historical-local-processor>" (redacted string) "usyd-community/vitpose-plus-base"; 1 change

The following table gives the full anchor delta. “Both” rows apply separately to FP32 and W8A8, yielding exactly five and seven changed pointers respectively. Replacing an object with null is one enclosing-pointer change, not a count of its children.

Candidate JSON pointer Anchor old value Candidate new value
Both /auto absent false
Both /optim/gelu_fusion absent true
Both /optim/matmul_add_fusion absent true
Both /compile null {"execution_provider":"qnn","provider_options":{"device_type":"NPU","htp_performance_mode":"burst","htp_graph_finalization_optimization_mode":"3"},"provider_option_file_keys":[],"enable_ep_context":true,"embed_context":false,"compiler":"ort","qnn_sdk_root":null,"device":"npu","ep_device":null,"validate":true}
FP32 /quant {"mode":"static","samples":10,"calibration_method":"minmax","weight_type":"uint8","activation_type":"uint16","per_channel":false,"symmetric":false,"weight_symmetric":null,"activation_symmetric":null,"save_calibration":false,"distribution":"uniform","seed":null,"calibration_load_path":null,"calibration_save_path":null,"op_types_to_quantize":null,"nodes_to_exclude":null,"task":"keypoint-detection","model_id":"<historical-local-processor>","model_type":"vitpose"} null
W8A8 /quant/activation_type "uint16" "uint8"
W8A8 /quant/seed null 42
W8A8 /quant/model_id "<historical-local-processor>" (redacted string) "usyd-community/vitpose-plus-base"

For W8A8, the full old /quant object above is retained except for the three listed scalar replacements; no null/default/exclusion fields are dropped. It remains static minmax, 10 samples, uint8 weights/activations, per_channel=false, and symmetric=false. Both candidates also retain the following complete historical blocks:

JSON pointer Preserved value
/loader {"task":"keypoint-detection","model_class":"VitPoseForPoseEstimation","model_type":"vitpose"}
/export {"opset_version":17,"batch_size":1,"export_params":true,"do_constant_folding":true,"verbose":false,"dynamo":false,"enable_hierarchy_tags":true,"clean_onnx":false,"hierarchy_tag_format":"full","input_tensors":[{"name":"pixel_values","dtype":"float32","shape":[1,3,256,192],"value_range":[0,1]}],"output_tensors":[{"name":"heatmaps"}],"compatibility":{"transformers_attention":"eager"}}

Additional first-field warnings, excluded from counts but not hidden: /_note was absent in both historical configurations and the anchor.

Candidate / pointer Old value Full new string value
FP32 /_note absent "UNVALIDATED CANDIDATE - DRAFT_ONLY_UNVALIDATED. Current-main model verification NOT_RUN; historical evidence only, not a model-support claim. #1401 is needed for a trustworthy canonical raw/no-optimize baseline. This optimization-enabled FP32 recipe is a fallback candidate, not that canonical raw baseline. The companion W8A8 candidate failed historical full-three-person and wallpaper parity; its posthoc two-person pass is not general quality evidence. Portable calibration processor/revision binding for W8A8 is not yet verified."
W8A8 /_note absent "UNVALIDATED CANDIDATE - DRAFT_ONLY_UNVALIDATED. Current-main model verification NOT_RUN; historical evidence only, not a model-support claim. #1401 is needed for a trustworthy canonical raw/no-optimize baseline. Historical W8A8 full-three-person and wallpaper parity FAILED; posthoc two-person PASS does not establish general quality or overturn those failures. Portable calibration processor/revision binding and exact portable-config equivalence are not yet verified."

WinMLBuildConfig.from_dict() ignores /_note, serialization omits it, and the alternate loader retains the raw dictionary. The warning is advisory, not an execution barrier. Likewise, compile.validate=true requests future validation; it is not evidence that compilation/validation ran. No eval block or skip_optimize field was authored.

Intentional discovery change: qnn/npu and QNNExecutionProvider/NPU lookup selects only the two target candidates. The target directory replaces generic-directory lookup; it does not union in generic FP16. Untargeted and missing QNN/GPU lookup still uses the unchanged generic FP16/W8A8 recipes. Automated consumers can select these unvalidated candidates despite the warning.

Calibration and reducibility: historical calibration records real images, not uniform noise inferred from distribution=uniform. The export dummy range [0,1] is not the actual processor-normalized input range. seed=42 proves neither effective seeded row selection nor pinned calibration rows. The portable processor ID is not revision-pinned; immutable processor/revision binding, calibration-row provenance, and exact portable-config runtime equivalence remain UNVALIDATED. No revision field was invented. These are frozen per-checkpoint/target graph, precision, power/latency, and external-context packaging choices—not a family-wide optimum, a default recommendation, or a metadata-derived CLI repair. No code bug fix or recipe-free acceptance is claimed; the no-recipe gate is deferred only for Draft publication. The production recipes README remains unchanged.

Analyze summary — component level and op level

Current analysis: NOT_RUN. No current built graph exists for this validation scope; the source-only architecture profile is not component mapping, static operator coverage, or runtime execution evidence.

Analysis layer Status Unavailable evidence
Component level NOT_RUN No current architecture-region mapping, mapped/partial/unmapped node totals, mapping confidence, or component-level EP findings
Op level NOT_RUN No current total/unique operator counts, dominant-op counts, or per-EP classifications

The unresolved gap is the absence of a current built graph and its analysis. Missing counts are not zero counts; none are generated from the source outline or carried forward from older artifacts.

Reproduce commands

Command provenance: exact historical build/perf/Eval argv is unrecovered; stored command templates and the future commands below are not historical reproductions. Shipment is limited to publication, Git/GitHub metadata, and preservation checks; no tests or model commands were rerun. The following public commands are transcribed without modification from the tester's handoff.

Portable spelling of the completed 60-test repository subset: run from the winml-cli repository root in an already prepared compatible CPython 3.11 environment with repository dependencies, pytest, pytest-cov, and pytest-timeout. The actual tester replay used the existing Transformers 4.57.6 / Hub 0.36.2 overlay; this does not certify a newly provisioned environment. This selection covers only the same 34 config + 26 discovery tests—not the additional 37 guarded checks, eight integrity checks, or any model run. It needs no unpublished harness. This replay was performed by the tester, not repeated for shipment.

$env:PYTHONPATH = 'src;.'
$env:HF_HUB_OFFLINE = '1'; $env:HF_DATASETS_OFFLINE = '1'; $env:HF_HUB_DISABLE_TELEMETRY = '1'; $env:DO_NOT_TRACK = '1'
$env:PYTHONUTF8 = '1'; $env:PYTHONDONTWRITEBYTECODE = '1'; $env:NO_COLOR = '1'; $env:PYTEST_DISABLE_PLUGIN_AUTOLOAD = '1'; $env:GIT_OPTIONAL_LOCKS = '0'
$OUT = Join-Path (Get-Location) ('temp/offline-vitpose-checks-' + [guid]::NewGuid().ToString('N'))
New-Item -ItemType Directory -Path (Join-Path $OUT 'os-temp') -ErrorAction Stop | Out-Null
$env:TEMP = Join-Path $OUT 'os-temp'; $env:TMP = $env:TEMP; $env:TMPDIR = $env:TEMP
python -m pytest -p pytest_cov -p timeout -o addopts= --strict-markers --strict-config --import-mode=importlib --no-cov -p no:cacheprovider --timeout=45 --tb=short --color=no -ra -v --basetemp="$OUT/pytest-temp" --junitxml="$OUT/junit.xml" tests/unit/config/test_build.py::TestValidate tests/unit/config/test_build.py::TestQuantModelId tests/unit/config/test_build.py::TestInt16QuantTypes tests/unit/config/test_build.py::TestExportCompatibilityBuildConfig tests/unit/config/test_build.py::TestPatchInputTensors tests/unit/config/test_build.py::TestMergeExportOverrides tests/unit/eval/test_recipes.py

Separate future recipe-authoritative builds — NOT_EXECUTED. These are not part of the offline replay or authorization to start model work. Use a separately authorized, resource-ready validation session after reviewing the canonical-baseline dependency. Only option spelling was checked through CLI help; there is no build result. The HF ID in these proposals does not pin the historical checkpoint/processor revision.

$OUT = Join-Path 'out' ('vitpose-candidates-' + [guid]::NewGuid().ToString('N'))
python -m winml.modelkit build -c examples/recipes/usyd-community_vitpose-plus-base/qnn/npu/keypoint-detection_fp32_config.json -m usyd-community/vitpose-plus-base -o "$OUT/fp32"
python -m winml.modelkit build -c examples/recipes/usyd-community_vitpose-plus-base/qnn/npu/keypoint-detection_w8a8_config.json -m usyd-community/vitpose-plus-base -o "$OUT/w8a8"

Recipe authority is preserved: -c, -m, and -o only, with no precision, EP/device, shape, optimization, or calibration rescue overrides and no quantization/compile bypass. Both builds remain optimization-enabled; the FP32 proposal is not the pending canonical no-optimize baseline. No current support, performance, memory, fidelity, or task-accuracy result is implied.

@DingmaomaoBJTU Qiong Wu (qiowu) (DingmaomaoBJTU) added the model-scale-by-skill Model support PR created or maintained by the adding-model-support skill label Sep 15, 2026
@DingmaomaoBJTU

Copy link
Copy Markdown
Collaborator Author

Independent agent limited review, relayed by the publishing agent.

LIMITED_SCOPE_HONESTY_CHECK: NO_FINDINGS

Independent post-publication review under charter revision 2 and the explicit “先发待验证 Draft” exception. This is not model-support approval, Goal acceptance, ordinary workflow approval, a GitHub Review, or permission to merge or mark ready.

Reviewed the actual published body and complete diff at head 1fd43e7, base/parent da5dbcd.

  • Scope and publication: OPEN/Draft, author DingmaomaoBJTU, label model-scale-by-skill; exactly two added QNN/NPU FP32/W8A8 JSON candidates, +57/+77, no deletions or CLI/source/test/index changes. Remote blobs, committed blobs, local bytes, and the tester's frozen SHA256s match.
  • Reporting: the three required headings and seven evidence sections are present. Both first-field _note warnings are candid. The body admits that _note is ignored/advisory, not an execution gate, and exact-target discovery selects FP32/W8A8 instead of unioning generic FP16. Full historical deltas are 0/1; anchor deltas 5/7. The portable processor ID is not revision-pinned; dummy range, seed, and distribution do not prove normalized inputs or calibration provenance. The frozen source-only profile retains its selected-expert caveat, and public processor-path redaction is explicit.
  • Saved evidence, not rerun tests: logs, JUnit identities, and seals corroborate 97 = 34 config + 26 discovery + 9 policy + 28 scratch checks, the 60-test subset replay (not additive), and 8 metadata checks. The earlier bootstrap failure, six harness failures, scratch-only corrections, and native ORT import warning are disclosed. Protected small-text hashes, including the lock and transcript, remain unchanged.
  • Unvalidated boundaries: all current L0–L3 rows for FP32/FP16/W8A8/W8A16, FP32 CPU functional evaluation, and component/operator analysis remain NOT_RUN. No highest Goal, current performance, or support result is established. Historical FP32 minimum cosine 0.9999722983142216 is old numeric evidence only. W8A8 full-set 0.982657340445321 and wallpaper 0.6957623048200067 fail 0.99; posthoc 0.9937934557606259 does not rescue them. W8A16 0.4100480666185648 remains a failure, with no target FP16/W8A16 recipe. Historical 2.5840062576234804x, 8 pairs/16 sessions/20 warmups/100 measurements, is not new or correctness-gated performance. QNN memory remains NOT_MEASURED; the older one-image CPU AP is operability-only, not representative or W8A8 accuracy.
  • Dependency and commands: fix(build): honor CLI optimization bypass #1401 remains a separate canonical raw/no-optimize baseline dependency, not a proven optimization-enabled runtime dependency. Proposed builds are NOT_EXECUTED; the public 60-test command is distinguished from both the guarded checks and model work.

At 2026-09-15 17:09 UTC, all 9 visible GitHub checks were successful; the earlier publication snapshot was pending. Neither state validates these model candidates. Before this comment, there were 0 conversation comments, 0 review threads, and 0 GitHub reviews.

Link-check limitation: seven linked GitHub files and #1401 resolved through live metadata. The two pinned Hugging Face link patterns and existing source-file hashes match the frozen identity; a live Hugging Face metadata request failed at the network connection, so remote reachability is not claimed.

Review verification used read-only Git/GitHub and small text/JSON/XML/hash inspection. No Python/project imports, tests, CLI help, model commands, dependency/provider operations, payload downloads, cleanup, source/recipe edits, or auth changes were performed. No fixes are requested within this limited review; intentional missing model validation is not a finding against the authorized unvalidated Draft. This opinion may be delivered only as a normal comment, never a GitHub Review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

model-scale-by-skill Model support PR created or maintained by the adding-model-support skill

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant