Skip to content

[Bug]: CuTe DSL NVFP4 SwiGLU / FP4Out dense GEMM has no SM107 kernel #19362

Description

@farazkh80

Summary

The CuTe DSL NVFP4 dense GEMM with fused SwiGLU (and the FP4-output variant) is Blackwell-only. On SM107 the ops raise at init:

CuteDSL NVFP4 SwiGLU backend requires SM 100 (B200) or SM 103 (B300), but got SM 107
CuteDSL NVFP4 SwiGLU FP4Out requires SM 100 or SM 103

Gates: tensorrt_llm/_torch/custom_ops/cute_dsl_custom_ops.py around L1716 and L2248.

Impact

TestDeepSeekV3Lite::test_cute_dsl_nvfp4[...] cannot run on SM107 (found while sweeping llm_function_core.txt on 4x SM107 against main). The dense BF16/FP8/MXFP8/NVFP4 SM107 GEMMs landed in #18761/#18765, but the SwiGLU act-fusion variants did not.

Ask

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Customized kernels<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions