Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions MANIFEST.sha256
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
af891a1c6f3f3958b43ea1f97a079ff898394ac72b2f49f88f3db66276f94eaf base.sh
577a83f8e3e3c4d69e04f4cbcee7ead42c49981c76f9bb07f8f78fd076876e01 comfyui-base-script.sh
105fe4ca913db6115a333785d3d0e57dc6856ee2985ca263890d262f106dc1cd comfyui-base-handbook.md
84cce5af664e3dd0fb0305d3a3484c37d0d95e0ef7ce0d168b9c03f7dd89631e VERSION
17e09f0c32655044fdc038fb046f87a5d0548ed9601f59fcb97fb3ea379c6d30 comfyui-base-handbook.md
6ed25779fb19fdab1aa720b59c911750e3c4c9208e0496bb72e2605302947616 VERSION
939c10b34e782bb77792eb876d8a51d74b38192598483c8c628f5da1b877397d pytest.ini
8aa67b7331d052dbbd1dcf9b357f088891e0fa0f4ed56e6669c105046fe881b2 LICENSE
75cf532f7718f7b546d033810c2aa623abd788605c42df4fa25e5f57108dddb3 lib/00-env.sh
Expand Down Expand Up @@ -36,7 +36,7 @@ e530f7d17ce5e9ac85a6b3bea83c788df4dfdfb9b81a1c7ff6a5f5f55b22d337 py/brand.py
d0528cf40dc376d3190158aeb8469d749c93d2ff5987d553aa7b90d67d5262ba py/comfy_settings.py
d50119de88f21134ba96830e8594228fdacac7c1654efd2b8a4ea1c710ea29cd py/derive.py
70445de5ed9a3236511ffe13aaa4372d93dfee3f415a542120511b0d1a1b0e3c py/gpu_facts.py
3a50989f855e75ed35ab64bfe660c1dc5a35c5bad3b44541ba4a40c1611ba0ae py/gpu_fit.py
8eeef1fcf8cabf3fe76da0e0a1be9293cde64c788b127b9421675606670f6029 py/gpu_fit.py
43d810bbc7694d5282b15b8e121d0e9fafa5be210b20a088697ab95195e8b379 py/holders.py
dfc2b8d5f75f9b0a4aec5fc2cc9585a76a9d9e9c1b8cf232937d70c49b831d61 py/import_scan.py
d6d4e3c0bc0acfe42bdc8d59b634587d1c8f27bef11ffe690c17b28b024a73db py/ledger.py
Expand Down Expand Up @@ -78,7 +78,7 @@ e4840d044d8bc334e9a816815bada3beb95882d41c8f30c55a0622d1e9415a06 suite/test_run
786deecfaa096c2ab77a15c924b8f5b032257fc3923b4c952a6ec067b6a663d9 suite/test_runtime.py
70b1fe09d400958c2e3680fade6750ea6f9639ee99f325f4abc0e8035b99b220 suite/test_speed.py
dbc1df30af0c04ecbc0eef14dc08e3d756dc4a4dfa934be96f71c49d423c9c93 suite/test_staged.py
f2f571a6de780d6f2ef16927402a4a65e37e9ae0871d498d194c348a1a27511a suite/test_unit.py
a799fa8528007764ac2cbdafe208e7ff64779bb1baec9e9eee3906155cf521d4 suite/test_unit.py
56dbcab4c427fbdc13a05c0eb4626ba464c84bced1486fb459877f25690835cb suite/test_verda.py
44edb78dcde7390a35b626b7b7a37b8568ccca0d2959683988e2d61b9887bb5f hosts/README.md
38feee74e9b4fc39aeded6a69c6f194ffc866d23d0dac7b4632fcdd27a28137e hosts/__init__.py
Expand Down
2 changes: 1 addition & 1 deletion VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
3.20.3
3.21.0
15 changes: 14 additions & 1 deletion comfyui-base-handbook.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# ComfyUI Base — Handbook

**Version 3.20.3.** The shared toolchain every workflow package on a machine sources, on **any host with an NVIDIA
**Version 3.21.0.** The shared toolchain every workflow package on a machine sources, on **any host with an NVIDIA
GPU** (RunPod, Verda, Crusoe, an owned box) and on any image or OS that gives it a driver and Python 3. One command,
run once per machine as **step one**; then each workflow package is **step two**, still one command. From a Mac,
`podctl` reaches the machine and hands its boot to the base (§1); after that every boot is the base's.
Expand Down Expand Up @@ -304,6 +304,15 @@ create reads it live):
`h264_nvenc` save fails there; encode on the CPU (`libx264`). "RTX" features (RTX Video Super Resolution) name an
RTX GPU, which these are not.

**Which test lane (3.21.0).** A smoke is not a pod run: it goes to the cheapest GPU that holds it. `py/gpu_fit.py`'s
`LANES` lists them cheapest first (Kaggle T4 and L4, HF ZeroGPU `large` and `xlarge`, Modal's monthly credits from a T4
to a B200, then RunPod, which is paid), and `pick_lane(peak_gib, minutes, needs)` returns the first whose card holds the
measured peak plus 10 % and whose own limit holds the run: Kaggle's 12 h session, ZeroGPU's 30-minute cap (15 on
`xlarge`, which costs the quota twice), the minutes $30 buys at Modal's list price within its 24 h per call. `needs`
names fast dtypes (`fp16 bf16 fp8 fp4`, from the compute capability), `server` (ZeroGPU runs per-call functions, so
never), or GPU classes, judged by `fits()` as `GPU_PREFER` is. `python3 py/gpu_fit.py --peak-gib 35.7 --minutes 10
--needs bf16` prints `lane=zerogpu-large` and its card; `lane=none` exits 1.

**Past B200 and B300, and no volume needed (3.18.0, Paul, 2026-10-16: rendering speed first).** `podctl create` with no
`--gpu` asks for `NVIDIA B200`, `NVIDIA B300 SXM6 AC`, then `NVIDIA H200`, `NVIDIA H100 80GB HBM3` and `NVIDIA RTX PRO 6000
Blackwell Server Edition` (Paul's order; `DEFAULT_GPUS`, `hosts/runpod/provider.py`). The GPU order comes first: for each GPU in turn, the network volume `--volume-name` where
Expand Down Expand Up @@ -822,6 +831,10 @@ for any invocation that somehow arrives without the environment. The suite asser

## 10. Record

- 3.21.0: which test lane a smoke takes (2026-10-17). `py/gpu_fit.py` gains `LANES`, `fast_dtypes()` and
`pick_lane(peak_gib, minutes, needs)`, which builds on `fits()` for GPU classes instead of a second rule, and a
`--peak-gib/--minutes/--needs` mode on the command line (§1, "Which test lane"). The unit tier covers the order, the
headroom, the limits, the dtypes and the CLI. Nothing else changed; no pod used.
- 3.20.3: the CPU-quota and launch-environment tests hold on any machine (2026-10-17). On the free `soulcraft-hf` runner
(a 2-CPU Hugging Face CPU Space that exports `OMP_NUM_THREADS=2`, workspace integration run 39049424709) three unit
tests failed while the code was right. The cap to the CPUs online turned the fake 250000/100000 quota into 2, and the
Expand Down
Binary file modified comfyui-base.zip
Binary file not shown.
99 changes: 99 additions & 0 deletions py/gpu_fit.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,21 @@
oldest preferred class AND has at least 90 % of the smallest preferred class's memory (a newer chip with the memory,
before this table names it). Everything else is below. A MIG slice is never its parent class: RunPod sells
"NVIDIA B300 SXM6 AC MIG 1g.34gb", 34 GB of a B300.

Which test lane (3.21.0). LANES lists the GPUs a smoke can run on, cheapest first (Kaggle, HF ZeroGPU, Modal's
credits, then RunPod, which is paid), and pick_lane(peak_gib, minutes, needs) returns the first whose card holds the
measured peak plus HEADROOM and whose own limit holds the run:

python3 gpu_fit.py --peak-gib 35.7 --minutes 10 --needs "bf16" # lane=zerogpu-large ... paid=no
python3 gpu_fit.py --peak-gib 69 --minutes 60 --needs "B200" # a GPU class is judged by fits(), as GPU_PREFER is

Exit 0 with a lane, 1 with lane=none, 2 for an unknown need.
"""
from __future__ import annotations

import argparse
import re
from collections import namedtuple
import subprocess
import sys

Expand Down Expand Up @@ -90,6 +100,86 @@ def fits(prefer: list[str], cc: str | None, mib: int | None, name: str = "") ->
return "below"


# 3.21.0: the test lanes, cheapest first. A smoke takes the first lane whose card holds its measured peak with HEADROOM
# to spare and whose own limit holds the run (pick_lane). gb is the memory the vendor states; minutes is the longest
# one run may take there, None for no limit of the lane's own; server says a ComfyUI server can stay up (ZeroGPU runs
# per-call Gradio functions, so no). First-party: Modal's pricing page (per-second GPU prices, $30 a month of credits)
# and timeouts guide (1 s to 24 h per call); HF's ZeroGPU docs (`large` is half an RTX PRO 6000, `xlarge` the whole card
# at twice the quota rate). A paid lane (RunPod) comes last and still needs its owner's yes.
HEADROOM = 0.10 # the Kaggle harness calls a peak past 95 % of the card "tight"; 10 % keeps clear of that
KAGGLE_SESSION_MIN = 12 * 60
ZEROGPU_CAP_MIN = 30 # our own cap on the PRO quota's 40 daily minutes; past the quota HF bills credits
MODAL_CREDITS_USD = 30.0
MODAL_MAX_MIN = 24 * 60
DTYPES = ("fp16", "bf16", "fp8", "fp4")

Lane = namedtuple("Lane", "name gpu cc gb minutes server paid")


def _modal_min(usd_s: float) -> int:
"""The minutes the month's credits buy at this GPU's list price, within Modal's 24 h per call."""
return min(MODAL_MAX_MIN, int(MODAL_CREDITS_USD / usd_s / 60))


LANES = (
Lane("kaggle-t4", "NVIDIA T4", "7.5", 16, KAGGLE_SESSION_MIN, True, False),
Lane("kaggle-l4", "NVIDIA L4", "8.9", 24, KAGGLE_SESSION_MIN, True, False),
Lane("zerogpu-large", "half an NVIDIA RTX PRO 6000", "12.0", 48, ZEROGPU_CAP_MIN, False, False),
Lane("zerogpu-xlarge", "NVIDIA RTX PRO 6000", "12.0", 96, ZEROGPU_CAP_MIN // 2, False, False),
Lane("modal-t4", "NVIDIA T4", "7.5", 16, _modal_min(0.000164), True, False),
Lane("modal-l4", "NVIDIA L4", "8.9", 24, _modal_min(0.000222), True, False),
Lane("modal-a10", "NVIDIA A10", "8.6", 24, _modal_min(0.000306), True, False),
Lane("modal-l40s", "NVIDIA L40S", "8.9", 48, _modal_min(0.000542), True, False),
Lane("modal-a100-80gb", "NVIDIA A100 80GB", "8.0", 80, _modal_min(0.000694), True, False),
Lane("modal-rtx-pro-6000", "NVIDIA RTX PRO 6000", "12.0", 96, _modal_min(0.000842), True, False),
Lane("modal-b200", "NVIDIA B200", CLASSES["B200"][0], CLASSES["B200"][1], _modal_min(0.001736), True, False),
Lane("runpod-rtx-pro-6000", "NVIDIA RTX PRO 6000", "12.0", 96, None, True, True),
Lane("runpod-b200", "NVIDIA B200", CLASSES["B200"][0], CLASSES["B200"][1], None, True, True),
Lane("runpod-b300", "NVIDIA B300", CLASSES["B300"][0], CLASSES["B300"][1], None, True, True),
)


def fast_dtypes(cc: str) -> list[str]:
"""The dtypes this architecture's tensor cores run natively: bf16 from Ampere (8.0), fp8 from Ada (8.9), fp4 from
Blackwell (10.0). A T4 (7.5) has fp16 alone."""
k = _cc_key(cc)
return [d for d, floor in zip(DTYPES, ((0, 0), (8, 0), (8, 9), (10, 0))) if k >= floor]


def pick_lane(peak_gib: float, minutes: float, needs=(), lanes=LANES) -> Lane | None:
"""The cheapest lane whose card holds `peak_gib` (measured, e.g. torch.cuda.max_memory_reserved) plus HEADROOM,
whose limit holds a run of `minutes`, and which has every need: a fast dtype (fp16 bf16 fp8 fp4), `server`, or GPU
classes from CLASSES, judged by fits() exactly as a package's GPU_PREFER is. None when no lane holds it.
`needs` is a list or a string in GPU_PREFER's form ("bf16 server"); an unknown need is a ValueError."""
needs = parse_prefer(needs) if isinstance(needs, str) else list(needs)
bad = [n for n in needs if n not in DTYPES and n != "server" and n not in CLASSES]
if bad:
raise ValueError("unknown need %s (known: %s, server, %s)" % (", ".join(bad), " ".join(DTYPES), " ".join(CLASSES)))
if peak_gib < 0 or minutes < 0:
raise ValueError("peak_gib and minutes are measured amounts, never negative")
classes = [n for n in needs if n in CLASSES]
for lane in lanes:
if peak_gib * (1 + HEADROOM) > lane.gb * 1e9 / 2 ** 30:
continue
if lane.minutes is not None and minutes > lane.minutes:
continue
if "server" in needs and not lane.server:
continue
if not {n for n in needs if n in DTYPES} <= set(fast_dtypes(lane.cc)):
continue
if classes and fits(classes, lane.cc, int(lane.gb * 1e9 / 2 ** 20), lane.gpu) != "ok":
continue
return lane
return None


def lane_line(lane: Lane | None, peak_gib: float, minutes: float, needs: str) -> str:
if lane is None:
return "\t".join(("lane=none", "peak_gib=%g" % peak_gib, "minutes=%g" % minutes, "needs=%s" % needs))
return "\t".join(("lane=%s" % lane.name, "gpu=%s" % lane.gpu, "sm=%s" % sm_of(lane.cc), "gb=%d" % lane.gb,
"minutes=%s" % ("none" if lane.minutes is None else lane.minutes), "paid=%s" % ("yes" if lane.paid else "no")))


def _smi():
"""(name, cc, total MiB) of the first GPU, or None when nvidia-smi does not answer."""
try:
Expand Down Expand Up @@ -117,7 +207,16 @@ def main(argv=None):
ap.add_argument("--prefer", default="")
ap.add_argument("--check", metavar="PREFER")
ap.add_argument("--name"); ap.add_argument("--cc"); ap.add_argument("--mib", type=int)
ap.add_argument("--peak-gib", type=float); ap.add_argument("--minutes", type=float, default=0.0)
ap.add_argument("--needs", default="")
a = ap.parse_args(argv)
if a.peak_gib is not None:
try:
lane = pick_lane(a.peak_gib, a.minutes, a.needs)
except ValueError as e:
print(e); return 2
print(lane_line(lane, a.peak_gib, a.minutes, a.needs))
return 0 if lane else 1
if a.check is not None:
bad = unknown_classes(parse_prefer(a.check))
if bad:
Expand Down
45 changes: 45 additions & 0 deletions suite/test_unit.py
Original file line number Diff line number Diff line change
Expand Up @@ -257,6 +257,51 @@ def test_unit_gpu_fit_names_b200_and_b300_and_judges_every_other_gpu_below(tmp_p
assert r.returncode == 1 and "H9000" in r.stdout


def test_unit_pick_lane_takes_the_cheapest_lane_that_holds_the_peak_and_the_run(tmp_path):
"""3.21.0: pick_lane(peak_gib, minutes, needs) walks LANES cheapest first (Kaggle, ZeroGPU, Modal's credits, then
paid RunPod) and returns the first whose card holds the measured peak plus 10 % and whose own limit holds the run.
The peaks are measured ones: Photo Auditor's 17.72 GB reserved on a Modal L4, Krea 2's 35.7 GB in bf16, Flux 2 Dev's
fp8 transformer and encoder together. A GPU class in `needs` is judged by fits(), the GPU_PREFER rule, never a copy."""
sys.path.insert(0, str(BASE / "py")); import gpu_fit

def pick(*a):
lane = gpu_fit.pick_lane(*a)
return lane.name if lane else None
assert pick(12, 20) == "kaggle-t4"
assert pick(14, 20) == "kaggle-l4" # 14 GiB plus 10 % is past a T4's 14.9 GiB
assert pick(12, 20, "bf16") == "kaggle-l4" # a T4 (sm_75) has fp16 alone
assert pick(12, 20, ["bf16", "server"]) == "kaggle-l4"
assert pick(17.72, 10) == "kaggle-l4"
assert pick(35.7, 10) == "zerogpu-large"
assert pick(35.7, 10, "server") == "modal-l40s" # ZeroGPU runs per-call functions, no server
assert pick(35.7, 45) == "modal-l40s" # past the 30-minute ZeroGPU cap
assert pick(71.1, 10, "fp8") == "zerogpu-xlarge"
assert pick(71.1, 20, "fp8") == "modal-rtx-pro-6000" # xlarge costs the quota twice: 15 minutes
assert pick(60, 60) == "modal-a100-80gb" and pick(60, 60, "fp8") == "modal-rtx-pro-6000" # an A100 has no fp8
assert pick(69, 60, "B200") == "modal-b200" # an RTX PRO 6000 holds it but is below B200
assert pick(69, 600, "B200") == "runpod-b200" and gpu_fit.pick_lane(69, 600, "B200").paid # past the $30
assert pick(150, 60) == "modal-b200" and pick(240, 60) == "runpod-b300"
assert pick(300, 1) is None
lanes = {lane.name: lane for lane in gpu_fit.LANES}
assert lanes["modal-b200"].minutes == 288 and lanes["modal-rtx-pro-6000"].minutes == 593 # $30 at list price
assert lanes["modal-l4"].minutes == 24 * 60 # Modal's own 24 h per call
assert [lane.paid for lane in gpu_fit.LANES] == sorted(lane.paid for lane in gpu_fit.LANES) # paid lanes last
assert gpu_fit.fast_dtypes("7.5") == ["fp16"] and gpu_fit.fast_dtypes("8.0") == ["fp16", "bf16"]
assert gpu_fit.fast_dtypes("8.9") == ["fp16", "bf16", "fp8"] and gpu_fit.fast_dtypes("12.0") == list(gpu_fit.DTYPES)
with pytest.raises(ValueError):
gpu_fit.pick_lane(12, 20, "bf17")
with pytest.raises(ValueError):
gpu_fit.pick_lane(-1, 20)
cli = [sys.executable, str(BASE / "py" / "gpu_fit.py")]
r = subprocess.run(cli + ["--peak-gib", "35.7", "--minutes", "10", "--needs", "bf16"], capture_output=True, text=True)
assert r.returncode == 0 and r.stdout.rstrip("\n").split("\t") == [
"lane=zerogpu-large", "gpu=half an NVIDIA RTX PRO 6000", "sm=sm_120", "gb=48", "minutes=30", "paid=no"], r.stdout
r = subprocess.run(cli + ["--peak-gib", "300", "--minutes", "1"], capture_output=True, text=True)
assert r.returncode == 1 and r.stdout.startswith("lane=none\t"), r.stdout
r = subprocess.run(cli + ["--peak-gib", "12", "--needs", "bf17"], capture_output=True, text=True)
assert r.returncode == 2 and "bf17" in r.stdout


def test_unit_a_gpu_below_gpu_prefer_is_a_warning_in_the_banner_and_a_typo_is_refused(tmp_path):
"""The pod's own run (preflight and the real run share base_init → base_banner) says, once, whether this GPU is the one
the package is tuned for: a warning that lands in the SUMMARY (podctl prints it), never a failure. An unknown class
Expand Down
Loading