[asherliu]

Summary

This adds HunyuanVideo 1.5 text-to-video on CUDA for
hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-480p_t2v: one GPU, BF16, through
difflet run, the Python API and difflet serve, eager or --torch-compile. On one RTX PRO
6000, its output meets a relative accuracy bar against vLLM-Omni in both modes, and its warm
latency is within 10% of vLLM-Omni's.

The model code is in difflet/backends/cuda/hunyuan_video_15/ with its GPU spec in
difflet/models/hunyuan_video_15/spec.py, and "cuda" is added to the HunyuanVideo 1.5
registry entry. The Trainium and Neuron HunyuanVideo 1.5 paths are unchanged.

• Transformer: Diffusers' HunyuanVideo 1.5 blocks with Difflet's projections and
  attention, loaded strictly from the checkpoint.
• Eager attention: packed QKV (one GEMM per stream), the text context trimmed to its
  valid tokens, Q/K RMSNorm and RoPE in one fused Triton op written for Difflet
  (difflet_hv15::fused_joint_qk_norm_rope), and cuDNN SDPA. Whether a model uses this path
  is decided once at load.
• Prefix attention (compiled blocks, and eager blocks the path above doesn't apply to):
  each sample's valid token prefix, so one static graph serves every prompt length, with
  deterministic Inductor reductions.
• Application: Diffusers' driver and native VAE tiles. Prompts are encoded with
  GPU-computed Qwen rotary frequencies and cuDNN SDPA off, which matches vLLM-Omni's
  conditioning bit for bit.

Shared code

• serving/model_registry.py: on GPUs, serving takes a model's metadata from its GPU spec
  and keeps CFG-parallel off unless asked, as the GPU CLI does.
• The rest is registration: the GPU spec module list, the serve model list and the registry
  entry's backends. The compile options and CFG-parallel refusal this model uses came in
  with feat(ltx2): add single-GPU CUDA audiovisual generation #147.

Behaviour changes for other models

• On a GPU, an unknown checkpoint of a GPU-flow model is now refused by difflet serve
  with a one-line 400 serving error. Before, it raised a ValueError with a traceback.
• On Trainium and Neuron, difflet serve with the HunyuanVideo 1.5 ID passes the CLI's
  model list and then gets a one-line "not enabled for serving" error. Before, the CLI
  said "Unknown model-id".
• On a GPU, difflet serve now leaves CFG-parallel off unless --cfg-parallel is passed, as
  difflet run does. No other model changes: HunyuanVideo 1.5's is the only registry layout
  that turns CFG-parallel on.

Results

Measured against vLLM-Omni 08177216 (vLLM 0.32.0) on one RTX PRO 6000 Blackwell (96 GB),
with the same Python 3.12 / Torch 2.13 / Diffusers 0.40 runtime for both engines. Each engine
and mode ran in a fresh process, and the whole set ran twice, the second time in reverse
order. G is vLLM-Omni's own test request (832×480×33, 50 steps, seed 42); P is 848×480×121.
Details are in the guide.

Output (ffmpeg SSIM / PSNR through vLLM-Omni's accuracy-test helper). The bar is relative:
on G, Difflet against vLLM-Omni must score at least what vLLM-Omni's own eager and compiled
outputs score against each other. P is reported. vLLM-Omni's own accuracy test uses a fixed
bar of SSIM ≥ 0.94 and PSNR ≥ 28 dB: Difflet's eager G meets it, and compiled G does not.

Request	Mode	Difflet vs vLLM-Omni	vLLM-Omni eager vs compiled	Gate
G	eager	0.9711 / 39.95 dB	0.8412 / 27.51 dB	pass
G	compiled	0.9181 / 31.65 dB	0.8412 / 27.51 dB	pass
P	eager	0.9555 / 35.46 dB	0.4999 / 18.04 dB	reported
P	compiled	0.6465 / 21.16 dB	0.4999 / 18.04 dB	reported

In eager mode the denoiser is bit-identical to vLLM-Omni's (all 54 blocks, both guidance
branches); the remaining difference is the VAE decode. The fused Q/K op is bit-identical to
vLLM-Omni's in 12 of 12 cases. Every engine and mode reproduced its denoiser digests exactly in
the second pass. An earlier run of the same code gave byte-identical Difflet videos in both
modes; only vLLM-Omni's compiled videos changed, and the bar passed there too (compiled G
0.9041 / 30.20 dB against 0.8343 / 27.46 dB).

The two engines compile differently, so their compiled outputs differ from the first block on:
vLLM-Omni compiles regions with dynamic shapes, and Difflet compiles each block with static
shapes and deterministic reductions. On G, each engine's eager and compiled videos differ by a
similar amount (Difflet 0.8615 / 28.35 dB, vLLM-Omni 0.8412 / 27.51 dB). At 121 frames
compilation moves the output further still: vLLM-Omni's own eager and compiled videos score
0.4999 / 18.04 dB against each other, below Difflet's compiled P score against vLLM-Omni.

Warm latency (s, pass 1; G is the median of three requests, P one request after a
warm-up; the bar is Difflet / vLLM-Omni ≤ 1.10):

Mode	Request	Difflet	vLLM-Omni	Ratio, pass 1 / 2
eager	G	58.83	58.81	1.000 / 1.000
eager	P	412.2	418.5	0.985 / 0.985
compiled	G	58.58	54.11	1.083 / 1.083
compiled	P	408.1	398.4	1.024 / 1.026

Peak device memory is 62.9 GiB for Difflet and 92.9 GiB for vLLM-Omni. Load takes 15–26 s
against 25–28 s.

Testing

• CPU unit suite: CI runs it on the three stacks. Locally on macOS, Torch 2.14 / Python 3.14:
  5123 passed. The 20 failures are all in tests this PR does not change, and come from the local
  machine: Inductor's CPU C++ compile, torchrun rendezvous on its hostname, no /proc on
  macOS, too little free disk for a video-service test, and a Qwen CPU golden test.
• BF16 bounds on CPU follow what x86 runners actually do. The tiny-transformer parity test
  allows one BF16 ulp per element: one runner's BF16 GEMM kernels rounded 2 of 48 elements one
  ulp apart, while a real attention-math difference moves elements by 2 to 4 ulps. The driver
  test's BF16 case bounds differences by the peak magnitude. Its FP32 case checks the CFG scale
  and negative-prompt wiring; a wrong scale or a dropped negative prompt misses its 1e-4 bound
  by two orders of magnitude.
• On the GPU, in the pinned requirements-cuda.lock environment (Torch 2.11):
  tests/unit/backends/test_cuda_hunyuan15_joint_attention.py passed 20 of 20, including the
  fused kernel against its reference; difflet run and difflet serve each generated a G
  video in eager mode; warm latency was within 3% of the Torch 2.13 runs.
• The GPU comparison in Results. It and the pinned-environment GPU run above used this
  branch at baf10e46, before the rebase onto main. The rebase leaves the HunyuanVideo 1.5
  code, the GPU application, the ops and serving byte-identical; on this model's path it brings
  in only feat(ltx2): add single-GPU CUDA audiovisual generation #147's --negative-prompt handling in the GPU CLI, which the comparison did not use.
  Nothing was re-run on a GPU after that: since the rebase, the branch has changed only CPU
  tests, docs and one code comment.
• ruff check passes on the changed files. black --check passes on them except for lines in
  registry.py and cli/main.py that are unformatted on main too; this PR leaves those lines
  alone.

🤖 Generated with Claude Code

[dongjiezou-arch]

Test placement / naming — nothing blocking, all mechanical:

1. tests/unit/models/hunyuan_video/test_cuda_application15.py tests difflet/backends/cuda/hunyuan_video_15/, but sits in the 1.0 model's test dir. The Trainium 1.5 tests live there because that code is in difflet/models/hunyuan_video/ (model_version="1.5"); the CUDA 1.5 code has its own package. Every other CUDA model keeps its application test at tests/unit/models/<model>/test_cuda_application.py (hunyuan_video, ltx_2, qwen_image, flux, wan), so this one belongs at tests/unit/models/hunyuan_video_15/test_cuda_application.py, mirroring difflet/models/hunyuan_video_15/.

2. tests/unit/models/hunyuan_video/test_native_vae_tiling.py: generic name, one HV1.5-only case, and what it asserts (256 px tiles, 16 latent, 0.25 overlap, no slicing) are Diffusers' own enable_tiling() defaults. The Difflet-owned behaviour is only that load_companions turns tiling on. It also imports _application/_pipeline from test_cuda_application15.py, which imports _vae back from it. Suggest one assert app.pipe.vae.use_tiling inside the application test and dropping the file (−43 lines, and the circular import goes with it).

3. tests/unit/backends/test_hunyuan15_conditioning.py: the CUDA tests in that dir are all test_cuda_* (including this PR's test_cuda_hunyuan15_joint_attention.py / test_cuda_hunyuan15_transformer.py), and this one is CUDA-only too (cuDNN SDPA policy), so test_cuda_hunyuan15_conditioning.py.

4. tests/unit/pipeline/test_pipeline_hunyuan15_cuda.py is a per-model public-factory/spec test. feat(ltx2): add single-GPU CUDA audiovisual generation #147 put the same kind of test for LTX-2 inside tests/unit/models/ltx_2/test_cuda_application.py (torch-free spec import, DiffletPipeline.from_pretrained(..., backend="cuda")), and tests/unit/pipeline/ otherwise holds the generic GPU policy/manifest tests. Suggest folding it into the model dir's test_cuda_application.py the same way.

[asherliu]

Thanks. All four are done in 20241c9 (test files only, no product code):

1. Moved to tests/unit/models/hunyuan_video_15/test_cuda_application.py.
2. Dropped test_native_vae_tiling.py. The application test's load_companions case now asserts app.pipe.vae.use_tiling, which also removes the circular import. Removing enable_tiling() from the application makes that assert fail.
3. Renamed to tests/unit/backends/test_cuda_hunyuan15_conditioning.py.
4. Folded the spec, public-factory, CFG-refusal and torch-free import tests into the same test_cuda_application.py, as LTX-2 does. tests/unit/pipeline/test_pipeline_hunyuan15_cuda.py is gone.

The moved tests and the rest of tests/unit/models/hunyuan_video and tests/unit/pipeline pass locally on CPU.