recipe(swin2sr): add QNN NPU configs

[Qiong Wu (qiowu) (DingmaomaoBJTU)]

Summary

Adds QNN/NPU recipes for caidas/swin2SR-classical-sr-x2-64, a 2x image-super-resolution checkpoint, in FP32, W8A8, and W8A16. This is an L0★ effort with an L0 recipe-only outcome; the three shipped tuples reached L2 PASS for fixed-input performance and tensor parity. The required FP16 tuple is terminal EXHAUSTED-FAIL at L0, so coverage is required-tuples-contain-exhausted-failures, neither full nor partial. W8A8 completed L2 but has explicitly low numerical quality.

Model metadata

What the model does

This Swin2SR checkpoint accepts a 64 x 64 RGB image tensor and reconstructs an RGB image at twice the spatial resolution (128 x 128) for image super-resolution.

• Evidence: pinned caidas/swin2SR-classical-sr-x2-64 model card and config (image_size=64, num_channels=3, upscale=2, upsampler=pixelshuffle); recipe-free ONNX I/O pixel_values[1,3,64,64] -> reconstruction[1,3,128,128].
• Confidence: verified.

Primary user stories

• A user supplies a low-resolution RGB image to obtain a two-times larger reconstructed image for image super-resolution.
• Evidence: the pinned model card identifies image super resolution as the intended use, and Swin2SRForImageSuperResolution.forward returns ImageSuperResolutionOutput.reconstruction.
• Confidence: verified.

Supported tasks

• image-to-image across the checkpoint, Transformers, Optimum ONNX, and WinML support surfaces.
• Evidence: the checkpoint architecture is Swin2SRForImageSuperResolution; Optimum 2.1.0 registers swin2sr/image-to-image; WinML inspect resolves image-to-image, AutoModelForImageToImage, and Swin2srOnnxConfig.
• Confidence: verified.

Model architecture

The concrete task model normalizes and embeds the image, runs six residual Swin Transformer stages with six shifted-window layers per stage at width 180, restores the spatial feature map through a residual body convolution, and applies a pixel-shuffle x2 reconstruction head.

Swin2SRForImageSuperResolution
|-- Swin2SRModel backbone (RGB 64 x 64, embed_dim 180)
|   |-- Reflect padding + RGB mean/range normalization
|   |-- Initial 3 x 3 convolution (3 -> 180)
|   |-- Patch embedding (patch 1) + LayerNorm
|   |-- Residual Swin Transformer stages x 6
|   |   |-- Swin2SRLayer x 6 (window 8; shifts alternate 0 and 4)
|   |   |   |-- 6-head cosine self-attention + continuous relative-position bias
|   |   |   `-- MLP (180 -> 360 -> 180, GELU) + residual LayerNorm
|   |   `-- 3 x 3 convolution + stage residual
|   `-- LayerNorm + patch unembedding + 3 x 3 body residual
`-- PixelShuffleUpsampler x2
    |-- 3 x 3 convolution (180 -> 64) + LeakyReLU
    |-- 3 x 3 convolution (64 -> 256) + PixelShuffle(2)
    `-- 3 x 3 convolution (64 -> 3) -> RGB reconstruction 128 x 128

• Source/confidence: pinned checkpoint config plus Transformers 4.57.6 Swin2SRForImageSuperResolution and PixelShuffleUpsampler source (verified). The recipe-free export reported 884 modules, 278 traced modules, and 12.1M parameters, with encoder and pixel-shuffle scopes present in ONNX.

Validation and support evidence

Baseline

• Base: current main commit 169a6f0122d1e1dd7d60c9605b5f19add26a209b, WinML 0.3.0, model revision cee1c923c6a37361c6e5650b65dcf4be821e5d52.
• Recipe-free CPU build: PASS in 52.5 seconds. The opset-17 artifact passed full ONNX checking and CPU Runtime loading with 2,240 nodes and 695 initializers; I/O is pixel_values float32 [1,3,64,64] to reconstruction float32 [1,3,128,128].
• CPU performance: 10 measured iterations after 3 warmups; mean/p50/p95 959.390/957.538/1020.707 ms, throughput 1.04 samples/s.
• Eval floor: winml eval --schema --task image-to-image exited 2 because image-to-image is not registered; no task metric was produced.
• Starting auto-config: WinML resolved AutoModelForImageToImage, image-to-image, QNN/NPU, and static uint8-weight/uint16-activation quantization. The contribution freezes measured choices instead of leaving autoconf active.
• Optimum probe: VENDOR-ONLY; Optimum already registered swin2sr for feature-extraction and image-to-image, and WinML added no task registration.

Goal

• Effort: L0★.
• Goal ceiling: L2, unchanged. Success required every QNN/NPU precision to build recipe-only with exact effective semantics, run fixed-input QNN performance, and complete fixed-input PyTorch parity.
• Outcome target: L0 recipe-only; no source implementation, evaluator implementation, or optimizer work belongs to this contribution.

Outcome

The shipped outcome is L0 with three exact-tuple recipes: qnn/npu/fp32, qnn/npu/w8a8, and qnn/npu/w8a16. Each reached L0/L1/L2 PASS. The reachable required qnn/npu/fp16 tuple is terminal EXHAUSTED-FAIL, with no candidate shipped and no deferred tuple; therefore the exact coverage classification is required-tuples-contain-exhausted-failures, neither full nor partial.

No source code, tests, dependencies, lockfiles, or examples/recipes/README.md changed. Model knowledge was captured only in run-local evidence and is not committed in this model PR. Methodology friction was observed; the proposed Lane A follow-up remains separate and no skill commit or skill PR is claimed here.

Repository gates passed: license insertion check, Ruff, mypy across 442 source files, and five non-hardware pytest partitions. Aggregate pytest results were 8,607 passed, 77 skipped, 2 xfailed, 1 deselected, and no failures. The Node-reached partition passed with Node 20.20.0 while the workflow pins Node 22; this is not a Node 22 claim.

Per-EP/device/precision results

Performance used 20 warmups and 30 measured iterations per candidate.

EP / device / precision	L0	L1	L2	Mean / p50 / p95	Throughput	RSS delta	Local / shared VRAM delta
QNN / NPU / FP32	PASS	PASS	PASS	276.5 / 275.83 / 279.163 ms	3.62 samples/s	105.89 MB	35.43 / 49.75 MB
QNN / NPU / FP16	EXHAUSTED-FAIL	INVALIDATED-BY-L0-FAIL (not run)	INVALIDATED-BY-L0-FAIL (not run)	Not run	Not run	Not run	Not run
QNN / NPU / W8A8	PASS	PASS	PASS	154.015 / 153.436 / 158.573 ms	6.49 samples/s	104.69 MB	41.2 / 55.52 MB
QNN / NPU / W8A16	PASS	PASS	PASS	269.012 / 269.073 / 273.194 ms	3.72 samples/s	104.02 MB	39.27 / 53.59 MB

EP / device / precision	Fixed-input parity verdict	Cosine	NRMSE	PSNR	Max absolute error	Eval support
QNN / NPU / FP32	PASS	0.9999999114198923	0.00023490583378868581	72.58212395131255 dB	0.0008974671363830566	CLI-BLOCKED, exit 2; no task metric
QNN / NPU / FP16	INVALIDATED-BY-L0-FAIL (not run)	Not run	Not run	Not run	Not run	CLI-BLOCKED, exit 2; no task metric
QNN / NPU / W8A8	PASS with low numerical quality	0.8576177223755574	0.3223373992552116	9.833786049364113 dB	1.3874439895153046	CLI-BLOCKED, exit 2; no task metric
QNN / NPU / W8A16	PASS	0.9962657138814027	0.04731689809712455	26.499674673721422 dB	0.22148799896240234	CLI-BLOCKED, exit 2; no task metric

W8A8 is not a high-quality result: its L2 command completed, but the low cosine and high NRMSE above are retained without upgrade or softening. These are single fixed-input tensor comparisons, not task-level image-quality measurements.

W8A16 also reproduced the separately sealed A1/B1/B2/A2 paired result: 17.975235769464405% gain with bootstrap 95% CI [17.889904562599636%, 18.04282472170429%]. That paired result used 20 warmups and 100 measured iterations per session; it is distinct from the fresh 30-iteration performance row above.

FP16 exhausted two real attempts. Both completed FLOAT16 conversion and then crashed during native QNN HTP compilation with exit -1073741819 (0xC0000005). The recipe-owned attempt used auto=false, quant.mode=fp16, FP32 public I/O, and the complete QNN provider options, but produced no Build complete marker, final model.onnx, wrapper, or EPContext. The failed candidate is absent; switching from QNN HTP/NPU to QNN GPU or adding shared runtime capability would change scope.

Functional smoke Eval

All four tuples are CLI-BLOCKED: winml eval --schema --task image-to-image exits 2 because WinML 0.3.0 has no image-to-image evaluator registration. The missing capability includes a paired low/high-resolution dataset adapter, Swin2SR prediction decoder, and super-resolution task metric. No dataset was processed and no task metric, representative accuracy, or benchmark-quality image claim is made.

Delta

The final diff contains exactly these checked-in recipes:

• examples/recipes/caidas_swin2SR-classical-sr-x2-64/qnn/npu/image-to-image_fp32_config.json
• examples/recipes/caidas_swin2SR-classical-sr-x2-64/qnn/npu/image-to-image_w8a8_config.json
• examples/recipes/caidas_swin2SR-classical-sr-x2-64/qnn/npu/image-to-image_w8a16_config.json

All three recipes change /auto from the schema default true to false; change /optim from {} to {"gelu_fusion":true,"highdimRTR_lowdimRTR":true,"matmul_add_fusion":false}; add /compile/provider_options/htp_performance_mode="burst"; and add /compile/provider_options/htp_graph_finalization_optimization_mode="3". Effective QNN options are device_type=NPU, htp_performance_mode=burst, and finalization mode 3. Canonical line-ending normalization does not change runtime configuration semantics: the committed recipes materialize to the tested effective configs.

Recipe	Precision-specific change from starting auto-config
FP32	/quant: static uint8-weight/uint16-activation object -> null
W8A8	/quant/activation_type: uint16 -> uint8; /quant/model_id: planner-local source -> caidas/swin2SR-classical-sr-x2-64; /quant/seed: null -> 42
W8A16	/quant/model_id: planner-local source -> caidas/swin2SR-classical-sr-x2-64; /quant/seed: null -> 42

The failed FP16 candidate was removed. This remains consistent with the L0★ per-model recipe resolution; recipe-free acceptance is not required. There are no code changes, and the production recipe README remains untouched.

Analyze summary - component and op levels

All four static scans are ANALYZE-PARTIAL-SUCCESS: each all-EP command exited 1, but each emitted all 12 requested EP/device rows with complete classification arrays. This is static compatibility analysis, not runtime execution or provider-attributed hotspot evidence; runtime claims come only from the independent build, performance, and parity results above.

Component-level summary

Artifact	Architecture regions	Mapping	Actionable EP findings
FP32	Image normalization; shallow/patch embedding; 6x6 shifted-window attention and MLP; residual body; pixel-shuffle x2 head	2,023 mapped; 2 partial; 73 unmapped; mapped with explicit gaps	OpenVINO/NPU partial: Slice; unsupported: none
FP16	Same regions	2,025 mapped; 74 partial; 73 unmapped; mapped with explicit gaps	QNN/NPU and QNN/GPU partial: Expand; OpenVINO/NPU partial: Slice; unsupported: none
W8A8	Same regions	6,708 mapped; 0 partial; 37 unmapped; mapped with explicit gaps	QNN/NPU has no partial or unsupported types
W8A16	Same regions	6,708 mapped; 72 partial; 37 unmapped; mapped with explicit gaps	QNN/NPU partial: MatMul QDQ; unsupported: none

Partial-node counts are subsets of mapped nodes and are not added to mapped plus unmapped totals. The remaining 73, 73, 37, and 37 optimizer-generated nodes are explicitly unmapped because their mapped neighbors disagree or provide no unique component owner.

Op-level summary

Artifact	Graph	Dominant operator counts	EP roll-up
FP32	2,096 operators / 19 types	Reshape 392; Transpose 338; Add 332; MatMul 288; Slice 146; LayerNormalization 74	OpenVINO/NPU partial: Slice; unsupported: none
FP16	2,098 operators / 20 types	Reshape 392; Transpose 338; Add 332; MatMul 288; Slice 146; LayerNormalization 74	QNN/NPU and QNN/GPU partial: Expand; OpenVINO/NPU partial: Slice; unsupported: none
W8A8	6,745 operators / 20 types	DequantizeLinear 2,696; QuantizeLinear 2,025; Reshape 392; Transpose 338; Add 332; MatMul 288	QNN/NPU has no partial or unsupported types
W8A16	6,745 operators / 20 types	DequantizeLinear 2,696; QuantizeLinear 2,025; Reshape 392; Transpose 338; Add 332; MatMul 288	QNN/NPU partial: MatMul QDQ; unsupported: none

CUDA/GPU, MIGraphX/GPU, TensorRT/GPU, DML/GPU, CPU/CPU, and VitisAI/NPU are rule-less groups in these static results. Their operator classifications are unknown, not unsupported. Static operator frequency and partial-node counts are not runtime hotspots.

Reproduce commands

These are the tester-normalized public commands. Resolve the durable model ID at the pinned revision to a local snapshot, and provide an equivalent fixed performance input NPZ.

$OUT='temp/swin2sr-qnn-validation'
$MODEL_SOURCE='<local snapshot resolved from caidas/swin2SR-classical-sr-x2-64@cee1c923c6a37361c6e5650b65dcf4be821e5d52>'
$RECIPE_ROOT='examples/recipes/caidas_swin2SR-classical-sr-x2-64/qnn/npu'
python -m winml.modelkit.cli build -c $RECIPE_ROOT/image-to-image_fp32_config.json -m $MODEL_SOURCE -o $OUT/fp32 --rebuild
python -m winml.modelkit.cli analyze -m $OUT/fp32/optimized.onnx --ep all --device all -o $OUT/fp32/analyze_all.json
python -m winml.modelkit.cli perf -m $OUT/fp32/model.onnx --skip-build --device npu --ep qnn --ep-options device_type=NPU --ep-options htp_performance_mode=burst --ep-options htp_graph_finalization_optimization_mode=3 --input-data <fixed-perf-input.npz> --warmup 20 --iterations 30 --memory --format json -o $OUT/fp32/perf.json
python -m winml.modelkit.cli eval --schema --task image-to-image
python -m winml.modelkit.cli build -c $RECIPE_ROOT/image-to-image_w8a8_config.json -m $MODEL_SOURCE -o $OUT/w8a8 --rebuild
python -m winml.modelkit.cli analyze -m $OUT/w8a8/quantized.onnx --ep all --device all -o $OUT/w8a8/analyze_all.json
python -m winml.modelkit.cli perf -m $OUT/w8a8/model.onnx --skip-build --device npu --ep qnn --ep-options device_type=NPU --ep-options htp_performance_mode=burst --ep-options htp_graph_finalization_optimization_mode=3 --input-data <fixed-perf-input.npz> --warmup 20 --iterations 30 --memory --format json -o $OUT/w8a8/perf.json
python -m winml.modelkit.cli eval --schema --task image-to-image
python -m winml.modelkit.cli build -c $RECIPE_ROOT/image-to-image_w8a16_config.json -m $MODEL_SOURCE -o $OUT/w8a16 --rebuild
python -m winml.modelkit.cli analyze -m $OUT/w8a16/quantized.onnx --ep all --device all -o $OUT/w8a16/analyze_all.json
python -m winml.modelkit.cli perf -m $OUT/w8a16/model.onnx --skip-build --device npu --ep qnn --ep-options device_type=NPU --ep-options htp_performance_mode=burst --ep-options htp_graph_finalization_optimization_mode=3 --input-data <fixed-perf-input.npz> --warmup 20 --iterations 30 --memory --format json -o $OUT/w8a16/perf.json
python -m winml.modelkit.cli eval --schema --task image-to-image

[Qiong Wu (qiowu) (DingmaomaoBJTU)]

Independent reviewer verdict: APPROVE

• Reviewed SHA: ef2eeb3e95c356b04f7a123fe8e2d1e0e1a7a4f5 (stable through the final gate).
• Current-main impact: NOT-MOVED; fetched origin/main is still the charter base and PR parent 169a6f0122d1e1dd7d60c9605b5f19add26a209b.
• Merge state: conflict-free (MERGEABLE); PR remains DRAFT and retains model-scale-by-skill.
• Discussion: 0 issue comments, 0 pull-review comments, 0 review threads, 0 open threads; no dispositions required.
• Checks: all 9 visible checks are COMPLETED/SUCCESS: Analyze (Python), lint, five test partitions, CodeQL, and license/cla.

Scope and evidence

The diff is one commit containing exactly three canonical LF recipe JSON files under qnn/npu, with no FP16 recipe and no source, test, workflow, README, dependency, or lockfile changes. git diff --check passes. The committed SHA-256 values are:

• FP32: af1c7296393f121b793d85ef160f283038a123bf75543f69b5a0f20fe5c03123
• W8A8: e034bcad2b384f500554a859a0926e020d90c6d8cda7c18c3dbf7f6b8a1b55a5
• W8A16: c60530499eb003b18fdc8a604cdb643bf9fc8737edecd795bde68a7a9e93f267

All three use auto=false, the sealed optimizer settings, and the exact QNN/NPU provider options. Independent WinMLBuildConfig parsing and validation passed; each committed config materializes identically to its tested effective config. Reconstructing the tested CRLF/no-final-newline bytes from each committed LF blob produced exact byte and hash matches.

Tuple disposition

Tuple	Ship	Verdict	Key evidence
qnn/npu/fp32	yes	L0/L1/L2 PASS	p50 275.83 ms; cosine 0.9999999114
qnn/npu/w8a8	yes	L0/L1/L2 PASS	p50 153.436 ms; cosine 0.8576177224, explicitly low quality
qnn/npu/w8a16	yes	L0/L1/L2 PASS	p50 269.073 ms; cosine 0.9962657139; sealed gain 17.9752357695% (95% CI 17.8899045626%-18.0428247217%)
qnn/npu/fp16	no	L0 EXHAUSTED-FAIL; L1/L2 invalidated and not run	two true-FLOAT16 attempts reached native HTP compile and exited 0xC0000005; no final model or EPContext

Coverage is correctly reported as required-tuples-contain-exhausted-failures, neither full nor partial. Eval is CLI-blocked for image-to-image with no metric claim. Component/op analysis is complete as static evidence and is not presented as runtime hotspot data.

Quality and residual risk

The frozen license/Ruff/mypy and five-partition results total 8,607 passed with zero failures. Reuse is valid because source, workflow, test, and pyproject.toml identities at the PR head exactly match the tested base; exact-head GitHub CI is also green. The local remaining partition used Node 20.20.0, while Node 22 is established only by the successful GitHub workflow.

Residuals are explicit: W8A8 has low fixed-input quality; FP16 is absent after exhausted native HTP failures; image-to-image Eval has no task metric; analyzer results are static; and proposed methodology findings _meta-115 through _meta-117 still need separate Lane A disposition. Under the explicit direct-recipe scope, the missing Lane A PR is not a model-PR blocker and skill files do not belong in this Lane B diff.

Orchestrator follow-up is required only to record terminal APPROVE, perform the separately owned promotion-handoff update, and disposition the Lane A proposals. Keep this PR in DRAFT.