Skip to content

Default TensorRT transformer trunks to NHWC - #1218

Merged
lightvector merged 1 commit into
lightvector:masterfrom
zsqdx:agent/auto-trt-nhwc-sm12
Jul 29, 2026
Merged

Default TensorRT transformer trunks to NHWC#1218
lightvector merged 1 commit into
lightvector:masterfrom
zsqdx:agent/auto-trt-nhwc-sm12

Conversation

@zsqdx

@zsqdx zsqdx commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Default TensorRT transformer trunks to NHWC.
  • Keep trtTransformerNHWC=false as an explicit per-machine fallback.
  • Normalize non-transformer models to NCHW so convnets do not create redundant plan/timing cache entries.
  • Document the default and override in gtp_example.cfg.

Motivation

The old NCHW default was based on earlier TensorRT results. Current results point the other way: my RTX PRO 6000 Blackwell test with TensorRT 10.16.1 showed a large NHWC win, and follow-up testing by lightvector on a weak laptop GPU and an A5000 found NHWC equal or faster on both TensorRT 10.9 and 10.16.

Official katago benchmark results for b11c768h12nbt3tflrs-fson-silu, FP16, fixed symmetry and seed:

Backend/layout Recommended threads visits/s nnEvals/s
CUDA 64 2814.27 2716.93
TensorRT NCHW 40 1625.87 1531.59
TensorRT NHWC 32 2836.89 2612.18

Equal-thread NHWC gains over NCHW were +29.1% at 5 threads, +55.8% at 16, +81.2% at 32, and +78.6% at 64.

The resolved layout is already part of the timing-cache hash and the onnx / onnxnh plan-cache tag, so the two engine layouts cannot collide.

Validation

  • Fresh TensorRT build at this PR's exact commit against TensorRT 10.16.1 / CUDA 13.
  • With no trtTransformerNHWC key, runtime logged transformerNHWC=true: 2852.22 visits/s, 2622.04 nnEvals/s at 32 threads.
  • With explicit trtTransformerNHWC=false, runtime logged transformerNHWC=false: 1578.90 visits/s, 1453.44 nnEvals/s at 32 threads.
  • OpenCL full build, katago runtests, and katago runoutputtests.

@zsqdx
zsqdx marked this pull request as ready for review July 28, 2026 17:18
@zsqdx
zsqdx force-pushed the agent/auto-trt-nhwc-sm12 branch from 19de8b9 to 680d37e Compare July 28, 2026 18:39
@zsqdx zsqdx changed the title Auto-select NHWC transformer layout on SM 12.x TensorRT GPUs Default TensorRT transformer trunks to NHWC Jul 28, 2026
@lightvector
lightvector merged commit d0c49c6 into lightvector:master Jul 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants