From 90f7dea4cbb1612637acce41376d6769af31f6c0 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sun, 1 Nov 2026 17:50:51 -0800 Subject: [PATCH 1/3] chore(quest): claim quest/m0/ci/mbx-arg-max.md Quest-Claim: 823a51bc1a53f93e From 9a7d5cbc93a24e4571f913e8b92075f2c24c39b0 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sun, 1 Nov 2026 17:54:55 -0800 Subject: [PATCH 2/3] docs(quest): isolate mbx E2BIG to upstream session metadata Co-Authored-By: GPT-6.1 Sol --- quest/m0/ci/mbx-arg-max.md | 110 ++++++++++++++++++++++++++++++++----- 1 file changed, 95 insertions(+), 15 deletions(-) diff --git a/quest/m0/ci/mbx-arg-max.md b/quest/m0/ci/mbx-arg-max.md index 294ea2d6cf9..6a3816945fe 100644 --- a/quest/m0/ci/mbx-arg-max.md +++ b/quest/m0/ci/mbx-arg-max.md @@ -1,21 +1,101 @@ -# [S] mbx launches Cargo with any argument length +# [S] Pin mbx after its launch metadata stops causing E2BIG ## Goal -mbx launches Cargo however long its argument list or environment is, and -`RecipeTest.test_unexecuted_bench_is_not_compiled` (`sh/rs/select.test.py`) -passes in `just ci`. +Pin an mbx release whose launch metadata does not turn an OS-valid Cargo +invocation into `E2BIG`. `RecipeTest.test_unexecuted_bench_is_not_compiled` +(`sh/rs/select.test.py`) passes in `just ci`, including with a long inherited +PATH that plain Cargo accepts. ## Plan -Planned 2026-11-01 (maintainer, follow-up planning). A `just ci` run during -[moq-dev/moq#7567](https://github.com/moq-dev/moq/pull/7567) failed in that -test, because mbx couldn't launch Cargo: "Argument list too long" (`E2BIG`). - -- First find which part goes over the limit: the argv, or the environment - the test passes (`os.environ` plus its overrides). -- Then fix it at the cause, for example by passing long lists through a file - or by trimming what mbx forwards, rather than skipping the test. -- If the limit is hit inside mbx itself rather than in this repository's - wrapper, the fix belongs upstream, and this quest pins the release that - carries it. +Investigated 2026-11-01. The fix belongs in +[jdx/mr-boxington](https://github.com/jdx/mr-boxington), outside this +repository. No upstream post or patch was submitted. An upstream release +with the fix is the condition for completing this quest; then update the +`version` pin in `.github/actions/rust-cache/action.yml` (currently 1.28.0) +and verify the focused test with that release. + +The original failure was reported during +[moq-dev/moq#7567](https://github.com/moq-dev/moq/pull/7567). +The focused test passes in a smaller environment. Appending repeated +`:/usr/bin` entries to PATH reproduces its exact mbx error for both its +library and binary fixtures, without changing its Cargo arguments: +`Argument list too long (os error 7)`. + +A separate dependency-free Cargo fixture isolates the cause. With only +HOME, PATH, CARGO, RUSTC, MBX_SCHEDULER=0, and MBX_SUMMARY=off in the incoming +environment, direct Cargo succeeds with both PATH lengths. mbx 1.25.0 and +the latest release, [1.29.1](https://github.com/jdx/mr-boxington/releases/tag/v1.29.1), +succeed at 1,080 bytes and fail at 40,077 bytes. The incoming environment +in the failing case is only 40,340 bytes, and the mbx argv is under 200 bytes. +`strace -f -v -s 1000000 -e trace=execve` shows mbx's Cargo exec returning +`E2BIG`, with these environment entry sizes including name, `=`, and NUL: + +| Entry | mbx 1.25.0 | mbx 1.29.1 | +| --- | ---: | ---: | +| PATH | 40,083 | 40,083 | +| MBX_SESSION_PATH | 142,525 | 142,525 | +| MBX_SESSION_RESTORE | 145,702 | 145,870 | + +This hits Linux's per-string limit (131,072 bytes on this 4 KiB-page host), +not its 2,097,152-byte aggregate `ARG_MAX`. + +The source is +[`record_overlay` in 1.29.1](https://github.com/jdx/mr-boxington/blob/v1.29.1/crates/mbx/src/cli/launch.rs#L523). +It JSON-serializes an `OsString` PATH into `MBX_SESSION_PATH`, and includes +PATH again in the serialized restoration overlay, `MBX_SESSION_RESTORE`. +Serde represents Unix `OsString` values as byte arrays, so both entries +expand beyond the per-string limit even though PATH itself is valid. +The same implementation remains in +[upstream main at 42c019c](https://github.com/jdx/mr-boxington/blob/42c019cd656409ab403472efe16804bc6cbfe9b1/crates/mbx/src/cli/launch.rs#L523). +No matching open upstream issue or PR was found during the investigation. + +Recommended next action: obtain authorization to submit this reproduction +upstream, recommending file-backed session restoration and PATH metadata, +with lifetime/ownership shared by the build session and its descendants. +Merely using a more compact environment encoding moves the failure to a +longer valid PATH; trimming this repository's environment or bypassing mbx +would hide the upstream bug. + +To reproduce in the Nix dev shell, create a dependency-free package under +`.tmp/` and compare real Cargo with mbx. This script inherits no credentials; +run tracing only around this controlled environment if inspecting envp: + +```python +import os +import shutil +import subprocess +from pathlib import Path + +scratch = Path(".tmp/mbx-arg-max").resolve() +(scratch / "src").mkdir(parents=True, exist_ok=True) +(scratch / "home").mkdir(exist_ok=True) +(scratch / "Cargo.toml").write_text( + '[package]\nname="arg-max-probe"\nversion="0.0.0"\nedition="2024"\n[workspace]\n' +) +(scratch / "src/lib.rs").write_text("pub fn probe() {}\n") +rustbin = Path(shutil.which("rustc")).parent +mbx = shutil.which("mbx") +env = { + "HOME": str(scratch / "home"), + "PATH": f"{rustbin}:/usr/bin:/bin" + ":/usr/bin" * 4444, + "CARGO": str(rustbin / "cargo"), + "RUSTC": str(rustbin / "rustc"), + "MBX_SCHEDULER": "0", + "MBX_SUMMARY": "off", +} +for program in (str(rustbin / "cargo"), mbx): + result = subprocess.run( + [program, "build", "--offline", "--manifest-path", str(scratch / "Cargo.toml")], + cwd=scratch, + env=env, + ) + print(program, result.returncode) +``` + +Verify both the reproduction and the original focused test when a fix is +available. Keep the benchmark exclusion assertion and production mbx path +intact. The original goal's "however long" is bounded by the OS; this goal +now states the actionable boundary: mbx must not reject caller-valid input +because of its own metadata expansion. From 21f8dbb70d16a5723d090227a2a487af13502a26 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sun, 1 Nov 2026 21:09:28 -0800 Subject: [PATCH 3/3] fix(nix): dedupe the dev shell's PATH so nested shells stay under mbx's E2BIG limit Each nested dev shell prepended another ~10 KB copy of its store paths. mbx 1.25-1.29 copies PATH into MBX_SESSION_PATH and MBX_SESSION_RESTORE at ~3.6x its size, so four nested shells pushed both past Linux's 131,072-byte per-string limit and every Cargo launch failed with E2BIG. Keeping each directory's first entry leaves lookups unchanged and holds PATH at ~10.8 KB. The repo-side goal is met and the maintainer chose not to report upstream, so the quest is deleted. Co-Authored-By: Claude Opus 5.5 --- flake.nix | 8 +++ quest/m0/ci/README.md | 1 - quest/m0/ci/mbx-arg-max.md | 101 ------------------------------------- 3 files changed, 8 insertions(+), 102 deletions(-) delete mode 100644 quest/m0/ci/mbx-arg-max.md diff --git a/flake.nix b/flake.nix index 2a6b998f86d..fc5453de07e 100644 --- a/flake.nix +++ b/flake.nix @@ -672,6 +672,14 @@ if [ -x "$shim" ]; then export PATH="$(dirname "$shim"):$PATH" fi + + # Each nested shell (direnv, then `nix develop`, then another + # inside a gate) prepends this shell's ~160 store paths again, + # about 10 KB. mbx copies PATH into two variables at ~3.6 times + # its size, and Linux rejects any one variable over 128 KiB, so + # four nested shells failed every Cargo launch with E2BIG. Keep + # each directory's first entry only: lookups resolve as before. + PATH=$(printf '%s' "$PATH" | awk -v RS=: '!seen[$0]++ { printf "%s%s", sep, $0; sep = ":" }') ''; env = { diff --git a/quest/m0/ci/README.md b/quest/m0/ci/README.md index 4c3cb465c0e..80062ce7fe1 100644 --- a/quest/m0/ci/README.md +++ b/quest/m0/ci/README.md @@ -166,4 +166,3 @@ The historical samples locate candidates; they are not controlled benchmarks. - [The queue host keeps the queue's disk](/quest/m0/ci/queue-host-disk.md) - agent sessions can't starve the queue runner of disk, and a crashed runner restarts - [The next test-CPU cuts](/quest/m0/ci/test-cost-followups.md) - the slow-test leftovers from #6987, cut at their source - [Biome checks every file it can parse](/quest/m0/ci/biome-whole-repo.md) - Biome runs over the whole repository and main is clean under it -- [mbx launches Cargo with any argument length](/quest/m0/ci/mbx-arg-max.md) - no "Argument list too long" in `sh/rs/select.test.py`'s recipe test diff --git a/quest/m0/ci/mbx-arg-max.md b/quest/m0/ci/mbx-arg-max.md deleted file mode 100644 index 6a3816945fe..00000000000 --- a/quest/m0/ci/mbx-arg-max.md +++ /dev/null @@ -1,101 +0,0 @@ -# [S] Pin mbx after its launch metadata stops causing E2BIG - -## Goal - -Pin an mbx release whose launch metadata does not turn an OS-valid Cargo -invocation into `E2BIG`. `RecipeTest.test_unexecuted_bench_is_not_compiled` -(`sh/rs/select.test.py`) passes in `just ci`, including with a long inherited -PATH that plain Cargo accepts. - -## Plan - -Investigated 2026-11-01. The fix belongs in -[jdx/mr-boxington](https://github.com/jdx/mr-boxington), outside this -repository. No upstream post or patch was submitted. An upstream release -with the fix is the condition for completing this quest; then update the -`version` pin in `.github/actions/rust-cache/action.yml` (currently 1.28.0) -and verify the focused test with that release. - -The original failure was reported during -[moq-dev/moq#7567](https://github.com/moq-dev/moq/pull/7567). -The focused test passes in a smaller environment. Appending repeated -`:/usr/bin` entries to PATH reproduces its exact mbx error for both its -library and binary fixtures, without changing its Cargo arguments: -`Argument list too long (os error 7)`. - -A separate dependency-free Cargo fixture isolates the cause. With only -HOME, PATH, CARGO, RUSTC, MBX_SCHEDULER=0, and MBX_SUMMARY=off in the incoming -environment, direct Cargo succeeds with both PATH lengths. mbx 1.25.0 and -the latest release, [1.29.1](https://github.com/jdx/mr-boxington/releases/tag/v1.29.1), -succeed at 1,080 bytes and fail at 40,077 bytes. The incoming environment -in the failing case is only 40,340 bytes, and the mbx argv is under 200 bytes. -`strace -f -v -s 1000000 -e trace=execve` shows mbx's Cargo exec returning -`E2BIG`, with these environment entry sizes including name, `=`, and NUL: - -| Entry | mbx 1.25.0 | mbx 1.29.1 | -| --- | ---: | ---: | -| PATH | 40,083 | 40,083 | -| MBX_SESSION_PATH | 142,525 | 142,525 | -| MBX_SESSION_RESTORE | 145,702 | 145,870 | - -This hits Linux's per-string limit (131,072 bytes on this 4 KiB-page host), -not its 2,097,152-byte aggregate `ARG_MAX`. - -The source is -[`record_overlay` in 1.29.1](https://github.com/jdx/mr-boxington/blob/v1.29.1/crates/mbx/src/cli/launch.rs#L523). -It JSON-serializes an `OsString` PATH into `MBX_SESSION_PATH`, and includes -PATH again in the serialized restoration overlay, `MBX_SESSION_RESTORE`. -Serde represents Unix `OsString` values as byte arrays, so both entries -expand beyond the per-string limit even though PATH itself is valid. -The same implementation remains in -[upstream main at 42c019c](https://github.com/jdx/mr-boxington/blob/42c019cd656409ab403472efe16804bc6cbfe9b1/crates/mbx/src/cli/launch.rs#L523). -No matching open upstream issue or PR was found during the investigation. - -Recommended next action: obtain authorization to submit this reproduction -upstream, recommending file-backed session restoration and PATH metadata, -with lifetime/ownership shared by the build session and its descendants. -Merely using a more compact environment encoding moves the failure to a -longer valid PATH; trimming this repository's environment or bypassing mbx -would hide the upstream bug. - -To reproduce in the Nix dev shell, create a dependency-free package under -`.tmp/` and compare real Cargo with mbx. This script inherits no credentials; -run tracing only around this controlled environment if inspecting envp: - -```python -import os -import shutil -import subprocess -from pathlib import Path - -scratch = Path(".tmp/mbx-arg-max").resolve() -(scratch / "src").mkdir(parents=True, exist_ok=True) -(scratch / "home").mkdir(exist_ok=True) -(scratch / "Cargo.toml").write_text( - '[package]\nname="arg-max-probe"\nversion="0.0.0"\nedition="2024"\n[workspace]\n' -) -(scratch / "src/lib.rs").write_text("pub fn probe() {}\n") -rustbin = Path(shutil.which("rustc")).parent -mbx = shutil.which("mbx") -env = { - "HOME": str(scratch / "home"), - "PATH": f"{rustbin}:/usr/bin:/bin" + ":/usr/bin" * 4444, - "CARGO": str(rustbin / "cargo"), - "RUSTC": str(rustbin / "rustc"), - "MBX_SCHEDULER": "0", - "MBX_SUMMARY": "off", -} -for program in (str(rustbin / "cargo"), mbx): - result = subprocess.run( - [program, "build", "--offline", "--manifest-path", str(scratch / "Cargo.toml")], - cwd=scratch, - env=env, - ) - print(program, result.returncode) -``` - -Verify both the reproduction and the original focused test when a fix is -available. Keep the benchmark exclusion assertion and production mbx path -intact. The original goal's "however long" is bounded by the OS; this goal -now states the actionable boundary: mbx must not reject caller-valid input -because of its own metadata expansion.