rtx-pro-6000

Optimized SGLang runtime for Qwen3.8-27B FP8 with DFlash2 and Qwen3.8 Flash-Next NVFP4 with FR-Spec on one NVIDIA RTX PRO 6000 Blackwell 96 GB GPU (SM120): 524K context, HiCache and NIXL.

GLM-5.2-NVFP4-REAP-469B serving on SM120 (4× RTX PRO 6000 Blackwell) — one-command vLLM launch recipe, 250K context, DeepSeek Sparse Attention + MTP speculative decode

Validated GLM-5.3 Flash recipe for 2x NVIDIA RTX PRO 6000 Blackwell 96GB: 262K context, EXL3/TR3, adaptive MTP, tools, and vision.

Sglang LLM Inference: RTX Pro 6000 vs DGX Spark

Reproducible SGLang recipe + public prebuilt image (ghcr.io) for DeepSeek-V4-Flash-0731 on 4x RTX PRO 6000 Blackwell (SM120): TP4/DP4/EP4, 1M ctx, benchmarks, and the DSPARK draft-depth corruption boundary

SGLang 0.5.21 for Qwen3.8-Flash-Next on one RTX PRO 6000: 264.5 tok/s decode, W4A16 routed experts (KLD 0.0111 vs BF16), 262K context, exact speculative rejection sampling, reproducible benchmarks.

GLM-5.3-Flash EXL3 on 4x RTX PRO 6000 with TensorFold: one-command recipe and engine patches. By Aevonix Research in collaboration with Mia's AI Lab.

Reproducible 262K GLM-5.3 Flash Uncensored EXL3+DFlash2 deployment for 2x RTX PRO 6000 Blackwell

Systematic 24-hour benchmark study of Qwen3.6-27B inference on dual NVIDIA RTX PRO 6000 Blackwell SM120 (TP=2). 8 experiments comparing repne/vllm fork vs upstream vLLM across FP8/BF16/NVFP4/Q8_0 quants and MTP/DFlash speculative decoding. Peak: 2,083 tok/s at c=32. Quality: KLD vs BF16 = 0.0018 (noise floor).

Fish Audio OpenAudio S2-Pro on vLLM-Omni. low-latency ~100ms TTFA, OpenAI-compatible, runs on NVIDIA Blackwell (RTX 5090 / RTX PRO 6000). Self-hosted streaming TTS & voice cloning.

GLM-5.3 EXL3 on 4x RTX PRO 6000 with TensorFold: one-command recipe and engine patches. By Aevonix Research in collaboration with Mia's AI Lab.

GLM-5.3-Flash EXL3 on 2x RTX PRO 6000 with TensorFold: one-command recipe and engine patches. By Aevonix Research in collaboration with Mia's AI Lab.

Abliterated Qwen3.8-27B in AWQ-INT4 with SGLang and an FP8 DFlash 2 drafter on half an NVIDIA RTX PRO 6000: 124 tok/s, 237k context, 2 slots, vision

Prebuilt spconv v2.3.8 wheels for CUDA 12.8 / 13.0 with native Blackwell (RTX 50-series, sm_120) kernels — for default PyPI torch (cu130) or torch +cu128

MiniMax-M3 (428B MoE) running on 3× RTX PRO 6000 Blackwell at TP=3 with 240K context, FP8 KV cache, and working multimodal vision input. Includes dist_utils.py patch for non-divisible attention heads.

Image-to-3D-Video-Asset-Generator is an all-in-one generative 3D pipeline that transitions smoothly from textual concepts or reference images into fully realized 3D mesh assets (.glb), dynamic camera movements in 5-second MP4 videos, and clean bundle exports (.zip).

An opinionated single-GPU vLLM recipe for Qwen3.8-Flash-Next on an RTX PRO 6000 Blackwell: vendored Qwen-Sharp chat template, MTP-3 speculative decoding, and a bundled local Prometheus + Grafana stack. Every non-default flag benchmarked.

GLM-5.3-Flash (320B) on one RTX PRO 6000: EXL3 3.05bpw with a GPU expert cache, 61-68 tok/s decode, 1.9-2.9K tok/s prefill, 512K context, vision, OpenAI + Anthropic APIs. Legacy llama.cpp recipe included.

Taming Qwen3.8-Flash-Next on a single RTX 6000 Pro 96GB — KV pool 374K → 1,111,168, 1M ctx, native NVFP4 MTP | 单卡服役实录：96G 显存 + 62G 内存，SSD-Stream 双通 PLE，768K 长文实测过闸，全数字带日志出处

The shared toolchain for ComfyUI workflow packages on any NVIDIA GPU host: one installer, pinned packs, declared models from Hugging Face or GitHub, the boot, the driver, and Comfy MCP so an agent can drive the machine. MIT, maintained by SoulCraft.