diff --git a/README.md b/README.md index 48014422..7f742595 100644 --- a/README.md +++ b/README.md @@ -6,9 +6,7 @@ # -**Run diffusion transformers on NVIDIA GPUs, AWS Trainium and (experimentally) Cloud TPU: FLUX, Qwen-Image, Wan, HunyuanVideo, LTX-2 etc., from one CLI, one Python API, and one OpenAI-compatible server.** - -MiniMax-H3 also supports text-to-video with audio through the staged CLI. +**Run diffusion transformers on NVIDIA GPUs, AWS Trainium and (experimentally) Cloud TPU: FLUX, Qwen-Image, Wan, HunyuanVideo, LTX-2 and MiniMax-H3 (video with audio), from one CLI, one Python API, and one OpenAI-compatible server.** [![License](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE) ![Status](https://img.shields.io/badge/status-pre--alpha-orange.svg) @@ -31,11 +29,11 @@ artifact, and later runs skip straight to load and denoise. [Developer guide](DEVELOPER.md) ## Latest News -- [11/02] **HunyuanVideo 1.5 generates video end to end on Trainium.** The 480p text-to-video model runs on both the AOT and the native backend with true CFG, TP, CFG-parallel, SP, CP (Ulysses, all-gather), DP, JSONL batching and TeaCache, at the default 480×848×121 shape; the VAE decodes on the host by default. See the [AOT verification](docs/verification/hv15-aot-evidence.md). -- [10/31] **The CUDA backend grows into a shared GPU platform.** Every GPU model now goes through one application, serving and CLI path, with data parallelism (`--dp`, `serve --dp-degree`; two replicas on 2 × A100, output identical to one card), CFG-parallel Wan 2.2 on two GPUs, `--torch-compile`, and experimental single-GPU HunyuanVideo. See the [GPU backend results](docs/verification/gpu-backend-results.md). -- [10/29] **Difflet runs on Trainium through native PyTorch.** The new `neuron` backend executes models with torch-neuronx, eagerly or through `torch.compile`, without ahead-of-time tracing, for all seven Trainium models (FLUX.1-dev, Qwen-Image, Wan 2.1/2.2, HunyuanVideo 1.0/1.5, LTX-2) with TP, SP, CP, CFG-parallel and DP layouts, on Trainium2 and Trainium3. See the [native backend results](docs/verification/neuron-backend-results.md). -- [09/30] **MiniMax-H3 text-to-video with audio is verified on Trainium2.** The four-stage CLI generated a 256×448, 124-frame MP4 with stereo audio on `trn2.3xlarge` using TP4. See the [end-to-end verification](docs/design/minimax_h3/01_e2e_reverification_2026-09-30.md). -- [09/10] **Difflet 1.0 is released.** +- [2026-11-02] **HunyuanVideo 1.5 generates video end to end on Trainium.** The 480p text-to-video model runs on both the AOT and the native backend with true CFG, TP, CFG-parallel, SP, CP (Ulysses, all-gather), DP, JSONL batching and TeaCache, at the default 480×848×121 shape; the VAE decodes on the host by default. See the [AOT verification](docs/verification/hv15-aot-evidence.md). +- [2026-10-31] **The CUDA backend grows into a shared GPU platform.** Every GPU model now goes through one application, serving and CLI path, with data parallelism (`--dp`, `serve --dp-degree`; two replicas on 2 × A100, output identical to one card), CFG-parallel Wan 2.2 on two GPUs, `--torch-compile`, and experimental single-GPU HunyuanVideo. See the [GPU backend results](docs/verification/gpu-backend-results.md). +- [2026-10-29] **Difflet runs on Trainium through native PyTorch.** The new `neuron` backend executes models with torch-neuronx, eagerly or through `torch.compile`, without ahead-of-time tracing, for all seven Trainium models (FLUX.1-dev, Qwen-Image, Wan 2.1/2.2, HunyuanVideo 1.0/1.5, LTX-2) with TP, SP, CP, CFG-parallel and DP layouts, on Trainium2 and Trainium3. See the [native backend results](docs/verification/neuron-backend-results.md). +- [2026-09-30] **MiniMax-H3 text-to-video with audio is verified on Trainium2.** The four-stage CLI generated a 256×448, 124-frame MP4 with stereo audio on `trn2.3xlarge` using TP4. See the [end-to-end verification](docs/design/minimax_h3/01_e2e_reverification_2026-09-30.md). +- [2026-09-10] **First public release.** ## What Difflet does @@ -62,15 +60,19 @@ Three entry points share one engine: | Backend | `DIFFLET_BACKEND` | Status | Models | Parallelism | Details | |---|---|---|---|---|---| -| NVIDIA GPUs | `cuda` | ✅ Supported (subset) | FLUX.1-dev, Qwen-Image, Wan 2.2 | DP, CFG-parallel (Wan) | [GPU features](#nvidia-gpus) | -| AWS Trainium2 | `trainium` | ✅ Supported | All [supported models](#supported-models) | TP, CP, SP, CFG-parallel, DP | [Trainium features](#trainium) | +| NVIDIA GPUs | `cuda` | ✅ Supported (subset) | FLUX.1-dev, Qwen-Image, Wan 2.2; HunyuanVideo and LTX-2 experimental | DP, CFG-parallel (Wan) | [GPU features](#nvidia-gpus) | +| AWS Trainium2 (AOT) | `trainium` | ✅ Supported | All [supported models](#supported-models) | TP, CP, SP, CFG-parallel, DP | [Trainium features](#trainium) | +| AWS Trainium2 / Trainium3 (native PyTorch) | `neuron` | ✅ Supported | FLUX.1-dev, Qwen-Image, Wan 2.1/2.2, HunyuanVideo 1.0/1.5, LTX-2 | TP, CP, SP, CFG-parallel, DP; FP8 (`--quant mxfp8`) on Trainium3 | [Native backend results](docs/verification/neuron-backend-results.md) | | Cloud TPU | `tpu` | 🧪 Experimental | FLUX.1-dev, Qwen-Image, Wan, HunyuanVideo, LTX-2 (`difflet serve` and the benchmark runners) | TP | [v5e benchmarks](benchmark/v5e/RESULTS.md) | | AMD ROCm | `rocm` | Planned | — | — | — | -Without `DIFFLET_BACKEND`, Difflet picks the backend it detects on the host. It checks Trainium, -then TPU, then CUDA, then ROCm, which only matters on a host with more than one accelerator. -Set `DIFFLET_BACKEND` to choose one explicitly. A CPU reference backend (`cpu`) exists for -numerical tests and CI. It is not meant for generation. +Without `DIFFLET_BACKEND` (or `--backend`), Difflet picks the backend it detects on the host. It +checks the native Neuron stack (`torch_neuronx` without `torch_xla`), then Trainium (AOT), then +TPU, then CUDA, then ROCm, which only matters on a host with more than one accelerator. With +no accelerator detected it falls back to `trainium`, so on a GPU host whose torch build lacks +CUDA, set `DIFFLET_BACKEND=cuda` to get a clear error instead of a Neuron import failure. +A CPU reference backend (`cpu`) exists for numerical tests and CI. It is not meant for +generation. On Cloud TPU the runtime is TorchTPU (`torch_tpu`), one process per chip, eager or `torch.compile`. Probe-free TeaCache (`--teacache-cadence` / `--teacache-online-delta`) works in serving there. @@ -85,9 +87,9 @@ Probe-free TeaCache (`--teacache-cadence` / `--teacache-online-delta`) works in | [Wan-AI/Wan2.2-T2V-A14B-Diffusers](https://huggingface.co/Wan-AI/Wan2.2-T2V-A14B-Diffusers) | Text-to-video | 480×832×9 | 2-stage (transformer → vae); true CFG | | [Wan-AI/Wan2.1-T2V-14B-Diffusers](https://huggingface.co/Wan-AI/Wan2.1-T2V-14B-Diffusers) | Text-to-video | 480×832×9 | Same runtime as Wan 2.2 | | [hunyuanvideo-community/HunyuanVideo](https://huggingface.co/hunyuanvideo-community/HunyuanVideo) | Text-to-video | 320×512×61 | 3-stage (clip → llama → generate) | -| [hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-480p_t2v](https://huggingface.co/hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-480p_t2v) | Text-to-video | 480×848×121 | 3-stage (encode → transformer → vae); true CFG (6.0), default `--tp-degree 2 --cfg-parallel`; on Trainium the VAE decodes on the host ¹⁴ | +| [hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-480p_t2v](https://huggingface.co/hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-480p_t2v) | Text-to-video | 480×848×121 | 3-stage (encode → transformer → vae); true CFG (6.0), default `--tp-degree 2 --cfg-parallel`; on Trainium the VAE decodes on the host ¹⁶ | | [Lightricks/LTX-2](https://huggingface.co/Lightricks/LTX-2) | Text-to-video | 512×768×121 | Single-process; TP only; exports `.mp4` | -| [MiniMaxAI/MiniMax-H3](https://huggingface.co/MiniMaxAI/MiniMax-H3) | Text-to-video with audio | 768×1344×124 ⚠️ ¹¹ | Verified at **256×448×124**; TP4; 4-stage CLI (text → generate → video_vae → audio_vae); `.mp4` with stereo audio | +| [MiniMaxAI/MiniMax-H3](https://huggingface.co/MiniMaxAI/MiniMax-H3) | Text-to-video with audio | 768×1344×124 ⚠️ ¹³ | Verified at **256×448×124**; TP4; 4-stage CLI (text → generate → video_vae → audio_vae); `.mp4` with stereo audio | ## Feature support @@ -142,7 +144,6 @@ TeaCache (adaptive), TAEF1, and `--host-vae` are refused on GPUs for these model Eager and compiled Python, CLI and HTTP generation were verified at 320×512×61, 50 steps on a 96 GB GPU. See the [HunyuanVideo CUDA guide](docs/verification/hunyuanvideo-cuda.md) for usage, accuracy and benchmark limits. HunyuanVideo 1.5 is a separate model. - 8. LTX-2 single-GPU BF16 Python audiovisual generation passes eager/compiled correctness and performance checks on B200. CLI/HTTP video-only paths have CPU coverage; GPU frontend validation is not claimed. The opt-in profile requires explicit TP=1 and a pre-CUDA startup @@ -157,13 +158,13 @@ Measured timings against stock diffusers, and the full pass/limit matrix, are in | Model | TP | CP — all-gather | CP — ring | CP — ulysses | SP | CFG-parallel | DP | |---|:---:|:---:|:---:|:---:|:---:|:---:|:---:| -| FLUX.1-dev | ✅ | ✅ | ✅ | ✅ | ✅ | — ⁷ | ✅ | -| Qwen-Image | ✅ | ✅ | ✅ | ✅ | ✅ | — ⁷ | ✅ | +| FLUX.1-dev | ✅ | ✅ | ✅ | ✅ | ✅ | — ⁹ | ✅ | +| Qwen-Image | ✅ | ✅ | ✅ | ✅ | ✅ | — ⁹ | ✅ | | Wan 2.2 / 2.1 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -| HunyuanVideo | ✅ | ⚠️ ⁸ | — ¹² | ✅ | ✅ | — ⁷ | ⚠️ ⁹ | -| HunyuanVideo 1.5 | ✅ | ✅ | — ¹² | ✅ | ⚠️ ¹³ | ✅ | ✅ | +| HunyuanVideo | ✅ | ⚠️ ¹⁰ | — ¹⁴ | ✅ | ✅ | — ⁹ | ⚠️ ¹¹ | +| HunyuanVideo 1.5 | ✅ | ✅ | — ¹⁴ | ✅ | ⚠️ ¹⁵ | ✅ | ✅ | | LTX-2 | ✅ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | -| MiniMax-H3 | ✅ TP4 | ❌ | ❌ | ❌ | ❌ | — ⁷ | ❌ | +| MiniMax-H3 | ✅ TP4 | ❌ | ❌ | ❌ | ❌ | — ⁹ | ❌ | **Runtime features** @@ -171,9 +172,9 @@ Measured timings against stock diffusers, and the full pass/limit matrix, are in |---|:---:|:---:|:---:|:---:|:---:|:---:| | FLUX.1-dev | ✅ image | ✅ | ✅ | ✅ | ✅ image | ✅ | | Qwen-Image | ✅ image | ✅ | ✅ | ✅ | ✅ image | ✅ | -| Wan 2.2 / 2.1 | ✅ video | ✅ | ✅ | ✅ | ✅ video ¹⁰ | ✅ | +| Wan 2.2 / 2.1 | ✅ video | ✅ | ✅ | ✅ | ✅ video ¹² | ✅ | | HunyuanVideo | ✅ video | ✅ | ✅ | ✅ | ✅ video | ✅ | -| HunyuanVideo 1.5 | ⚠️ video ¹⁴ | ❌ | ⚠️ ¹⁵ | ✅ | ❌ | ✅ | +| HunyuanVideo 1.5 | ⚠️ video ¹⁶ | ❌ | ⚠️ ¹⁷ | ✅ | ❌ | ✅ | | LTX-2 | ✅ video | ❌ | ✅ | ✅ | ✅ video | ✅ | | MiniMax-H3 | ✅ video + stereo audio | ❌ | ❌ | ❌ | ❌ | ❌ | @@ -191,15 +192,15 @@ and the evidence in **Notes** -7. Guidance-distilled model (single forward pass with the guidance scale baked into the timestep embedding) — there is no second CFG branch to split. -8. `tp2 cp2` with gather-KV hits a `neuronx-cc` internal error (`NCC_INLA001` / `NCC_IBIR243`) on the CP-degree-2 DiT graph; ulysses CP and `tp4 --sp` are the working multi-core paths. See [DEVELOPER.md](DEVELOPER.md). -9. DP works, but on a 4-core `trn2.3xlarge` each 2-core replica runs out of HBM loading the compiled VAE at the default 320×512×61 shape. Use a smaller shape or a host with more cores per replica. -10. Wan 2.1 is the qualified serving checkpoint. Wan 2.2 can be started for experiments but its dual-transformer path has not passed resident-serving acceptance. -11. MiniMax-H3 supports the Trainium four-stage CLI at TP4 and CP1: text and DiT use four cores, and each VAE uses one core. The verified end-to-end shape is **256×448×124**; pass `--height 256 --width 448 --num-frames 124`. The registry default **768×1344×124** is not qualified on `trn2.3xlarge`: the standard DiT graph exceeds device memory, and compilation with precomputed AdaLN exceeded host memory. The 640×1152×124 DiT compiled successfully but has not been verified end to end. Python API generation, resident serving, DP, JSONL batching, multi-shape compilation, and TeaCache are not supported for H3. See the [verification](docs/design/minimax_h3/01_e2e_reverification_2026-09-30.md) and [bring-up handoff](docs/design/minimax_h3/00_trainium_bringup_handoff.md). -12. Not applicable: the model always carries a padded-text attention mask, and the Trainium ring-attention kernel has no text-mask path. `--cp-mode ring` is refused (for HunyuanVideo 1.5 before any stage launches); use `--cp-mode ulysses` (or `gather_kv` for HunyuanVideo 1.5). -13. `--tp-degree 4 --sp` needs the image token count divisible by 4. It works at 320×512×61; at the default 480×848×121 it is refused before compile, so use the default `--tp-degree 2 --cfg-parallel` there. -14. The DiT runs on the NeuronCores; the VAE decodes on the host by default (diffusers' tiled decoder in fp32), because the compiled decoder does not build on neuronx-cc 2.26 (`--device-vae` opts into it). The host decode takes about 7 minutes at 320×512×61 and about 45 minutes at 480×848×121. Verified on device: TP2, TP2 × CFG2, TP4, TP4 + SP, TP2 × CP2 (ulysses and gather_kv), DP, JSONL batches and the default shape; see [docs/verification/hv15-aot-evidence.md](docs/verification/hv15-aot-evidence.md). -15. Available but not recommended for HunyuanVideo 1.5: the calibration signal is weak and the output is visibly degraded, and the probe loads a second copy of the DiT weights, so it does not fit at the default shape. Use `--teacache-cadence 2` (20 of 50 steps skipped, about 40% less denoise time). +9. Guidance-distilled model (single forward pass with the guidance scale baked into the timestep embedding) — there is no second CFG branch to split. +10. `tp2 cp2` with gather-KV hits a `neuronx-cc` internal error (`NCC_INLA001` / `NCC_IBIR243`) on the CP-degree-2 DiT graph; ulysses CP and `tp4 --sp` are the working multi-core paths. See [DEVELOPER.md](DEVELOPER.md). +11. DP works, but on a 4-core `trn2.3xlarge` each 2-core replica runs out of HBM loading the compiled VAE at the default 320×512×61 shape. Use a smaller shape or a host with more cores per replica. +12. Wan 2.1 is the qualified serving checkpoint. Wan 2.2 can be started for experiments but its dual-transformer path has not passed resident-serving acceptance. +13. MiniMax-H3 supports the Trainium four-stage CLI at TP4 and CP1: text and DiT use four cores, and each VAE uses one core. The verified end-to-end shape is **256×448×124**; pass `--height 256 --width 448 --num-frames 124`. The registry default **768×1344×124** is not qualified on `trn2.3xlarge`: the standard DiT graph exceeds device memory, and compilation with precomputed AdaLN exceeded host memory. The 640×1152×124 DiT compiled successfully but has not been verified end to end. Python API generation, resident serving, DP, JSONL batching, multi-shape compilation, and TeaCache are not supported for H3. See the [verification](docs/design/minimax_h3/01_e2e_reverification_2026-09-30.md) and [bring-up handoff](docs/design/minimax_h3/00_trainium_bringup_handoff.md). +14. Not applicable: the model always carries a padded-text attention mask, and the Trainium ring-attention kernel has no text-mask path. `--cp-mode ring` is refused (for HunyuanVideo 1.5 before any stage launches); use `--cp-mode ulysses` (or `gather_kv` for HunyuanVideo 1.5). +15. `--tp-degree 4 --sp` needs the image token count divisible by 4. It works at 320×512×61; at the default 480×848×121 it is refused before compile, so use the default `--tp-degree 2 --cfg-parallel` there. +16. The DiT runs on the NeuronCores; the VAE decodes on the host by default (diffusers' tiled decoder in fp32), because the compiled decoder does not build on neuronx-cc 2.26 (`--device-vae` opts into it). The host decode takes about 7 minutes at 320×512×61 and about 45 minutes at 480×848×121. Verified on device: TP2, TP2 × CFG2, TP4, TP4 + SP, TP2 × CP2 (ulysses and gather_kv), DP, JSONL batches and the default shape; see [docs/verification/hv15-aot-evidence.md](docs/verification/hv15-aot-evidence.md). +17. Available but not recommended for HunyuanVideo 1.5: the calibration signal is weak and the output is visibly degraded, and the probe loads a second copy of the DiT weights, so it does not fit at the default shape. Use `--teacache-cadence 2` (20 of 50 steps skipped, about 40% less denoise time). Context parallelism (`--cp-degree > 1`) and CFG-parallel both consume the data-parallel lanes, so they are mutually exclusive (and each is mutually exclusive with `--sp`). `world_size = dp × (2 if cfg-parallel else 1) × cp_degree × tp_degree`; `--sp` leaves it unchanged. `difflet plan --model-id ` lists the combinations your host can run. @@ -214,6 +215,9 @@ Context parallelism (`--cp-degree > 1`) and CFG-parallel both consume the data-p - **AWS Trainium2.** Validated on `trn2.3xlarge`: one Trainium2 chip presented as 4 logical NeuronCores under the default `LNC=2`. Other Trn2 shapes should work; the tensor-parallel degree must divide the number of visible cores. + - **AWS Trainium3** through the native PyTorch backend (`--backend neuron`), which also + runs on Trainium2; FP8 (`--quant mxfp8`) is Trainium3-only. The AOT `trainium` backend + is qualified on Trainium2. - Python 3.11 or newer. - A Hugging Face account with access to the gated FLUX.1-dev repo, if you want Flux. - Disk: weights are large (LTX-2 alone is 86 GB). On Trainium, compiled artifacts add tens of @@ -227,7 +231,8 @@ Context parallelism (`--cp-degree > 1`) and CFG-parallel both consume the data-p git clone git@github.com:ai-decentralized/Difflet.git cd Difflet python3 -m venv .venv && source .venv/bin/activate -# torch must come from the CUDA 12.8 index (Blackwell needs a 12.8+ wheel). +# torch must come from the CUDA 12.8 index (Blackwell needs a 12.8+ wheel); +# the version is the one in requirements-cuda.lock. pip install --index-url https://download.pytorch.org/whl/cu128 torch==2.11.0 pip install -e ".[cuda]" huggingface-cli login # for gated repos such as FLUX.1-dev @@ -455,10 +460,8 @@ Stop the server with `Ctrl+C` or `SIGTERM`. ### Observability The server logs each stage's start and completion with its duration (`stage.start`, -`stage.complete`), not the device memory each stage keeps resident. A per-request stage -breakdown and per-stage memory (from those logs plus `neuron-monitor` on Trainium or -`nvidia-smi` on GPUs) are planned as a skill under `.claude/skills/`, so the serving code -itself carries no instrumentation. +`stage.complete`). It does not report device memory; read that from `neuron-monitor` on +Trainium or `nvidia-smi` on GPUs while a request runs. ### Hardening @@ -529,6 +532,7 @@ paths, and stage core counts are in [docs/cli-staged-commands.md](docs/cli-stage |---|---|---| | `--model-id` | all | Hugging Face model id (see [Supported models](#supported-models)) | | `--revision REV` | download, compile, generate, run, serve | Pin a Hub revision | +| `--backend NAME` | compile, generate, run | Hardware backend, one of the `DIFFLET_BACKEND` values (default: `$DIFFLET_BACKEND`, else auto-detected); `serve` reads `DIFFLET_BACKEND` | | `--tp-degree N` | compile, generate, run, serve | Tensor-parallel degree (default: registry default) | | `--cp-degree N` | compile, generate, run, serve | Context-parallel degree (default 1) | | `--cp-mode {gather_kv,ring,ulysses}` | compile, generate, run, serve | Context-parallel attention strategy | @@ -536,6 +540,7 @@ paths, and stage core counts are in [docs/cli-staged-commands.md](docs/cli-stage | `--sp` | compile, generate, run, serve | Megatron-style sequence parallelism over the TP group (Flux, Qwen-Image, Wan, HunyuanVideo, HunyuanVideo 1.5) | | `--dp N` | compile, generate, run | Data-parallel replicas; a router spreads requests across them | | `--dp-schedule {round_robin,least_loaded}` | compile, generate, run | Request-to-replica schedule for `--dp > 1` | +| `--dp-degree N` | serve | CUDA backend only: resident replicas, one GPU each; requests go to the least loaded | | `--mode {latency,throughput,mixed}` | compile, generate, run | Preset that picks `dp`, `cfg`, and `cp` for the model class | | `--total-cores N` | compile, generate, run | Core budget for validation (default: `NEURON_RT_NUM_CORES`) | | `--height/--width/--num-frames` | compile, generate, run, serve | Output shape (default: the model's registry shape) | @@ -543,13 +548,17 @@ paths, and stage core counts are in [docs/cli-staged-commands.md](docs/cli-stage | `--teacache-cadence N` | generate, run | Skip every N-th denoise step | | `--teacache-speedup F`, `--teacache-calibration PATH` | generate, run, serve | Adaptive TeaCache target and calibration file | | `--teacache-online-delta ALPHA` | generate, run | Probe-free online-delta TeaCache | +| `--torch-compile` | generate, run, serve | CUDA backend only: `torch.compile` the DiT blocks (eager is the default) | +| `--quant mxfp8` | compile, generate, run | `--backend neuron` on Trainium3: MXFP8 attention q/k/v and MLP linears (Qwen-Image, FLUX.1-dev) | | `--prompt` | generate, run | Text prompt | +| `--negative-prompt TEXT`, `--true-cfg-scale F` | generate, run | Qwen-Image on `--backend neuron`: opt-in true classifier-free guidance | | `--requests FILE` | generate, run | JSONL batch file, one request per line | | `--output PATH` | generate, run | Output file (`.png` or `.mp4`) | | `--steps N`, `--guidance-scale F`, `--seed N` | generate, run | Sampler settings (seed default 42) | | `--cache-dir PATH` | compile, generate, run, serve, cache | Compiled-artifact cache root (default `~/.cache/difflet/`) | | `--force` | compile, generate, run, serve | Recompile even if a valid cache entry exists | | `--work-dir PATH`, `--keep-work-dir` | generate, run | Inter-stage tensor directory for staged models | +| `--host-vae`, `--device-vae` | compile, generate, run; `--host-vae` also serve | VAE placement: decode on the host CPU, or in a device VAE stage of its own (model-specific; see `--help`) | | `--host`, `--port`, `--api-key` | serve | Bind address and optional shared API key | | `--clip-placement {host,neuron}` | serve | HunyuanVideo CLIP placement | diff --git a/pyproject.toml b/pyproject.toml index dc6041da..ac85e1ba 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -15,6 +15,7 @@ classifiers = [ "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", + "Programming Language :: Python :: 3.13", "Programming Language :: Python :: 3.14", ]