Make the dev image usable as a VS Code devcontainer, with Rust - #80
Make the dev image usable as a VS Code devcontainer, with Rust#80alycda wants to merge 8 commits into
Conversation
modules/dev/rust.nix carried lldb, so importing it cost 1.6 GiB of closure (measured: lldb 1.6 GiB, rustup 94 MiB, bacon 63 MiB). That priced the container profiles out of importing it at all - which is why alyssa@dev has no Rust today despite the module existing since forever. lldb is not a Rust dependency: cargo and bacon never invoke it, and VS Code's CodeLLDB extension bundles its own. It belongs to the one profile that wants a system debugger, so it moves to work.nix rather than being dropped. No profile's package set changes: work.nix was the only importer. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`programs.zsh.initExtra = "rustup update"` ran a network call on every shell start. Tolerable with one terminal open; in a devcontainer every VS Code terminal pays it before showing a prompt. It was also the wrong command. `rustup update` refreshes installed toolchains and installs nothing when there are none, so a fresh profile got rustup's shims with nothing behind them - `cargo` on PATH, failing with "no default toolchain configured". `rustup default stable` is what makes the profile usable and no-ops afterwards. Same pattern profiles/code.nix already uses. Guarded with `|| true`: this is the only network call in activation, and activation runs at container start. A failed `run` aborts activation after linkGeneration has already written the dotfiles - the documented route to a container with a perfect prompt and no packages. Starting offline should mean "no toolchain yet", not a broken profile. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
alyssa@dev shipped no rustup, cargo or bacon, so the image that exists to be a devcontainer could not build a Rust project. modules/dev/rust.nix has been there all along; work.nix was its only importer. Affordable now that lldb's 1.6 GiB lives in work.nix. rustup is 94 MiB and fetches toolchains at runtime into ~/.rustup - the devhome volume, not an image layer - so alyssa@dev-x86 on the 2012 MBP gains ~160 MiB, well inside the headroom PR #34 exhausted. Verified: alyssa@dev and alyssa@dev-x86 evaluate, and dev's package set gains exactly rustup + bacon while work-dev keeps rustup + bacon + lldb. Note: alyssa@work-dev does not evaluate on aarch64-linux, and did not before this branch either - work.nix carries cocoapods, which is aarch64-darwin only. Left alone as an unrelated bug. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The nixos/nix base image ships /etc/{passwd,group,shadow} as absolute symlinks
into the store. Resolvable from inside the container, so `docker run` and every
shell in it are fine - but `docker exec` under OrbStack fails:
Shell server terminated (code: 126, signal: null)
openat etc/passwd: path escapes from parent
That takes VS Code Dev Containers with it, which is what this image exists for:
the CLI's first act after starting the container is to exec a shell server, and
it dies before any lifecycle command runs.
The exec path resolves the user entry host-side, rooted at the container
rootfs, with openat2 RESOLVE_BENEATH semantics - absolute symlinks are rejected
as escaping their parent rather than re-rooted as RESOLVE_IN_ROOT would.
`docker run` uses a different path, hence the split behaviour and the
misleading error.
Copying the content into place is inert: same bytes, same lookups, store paths
untouched, and nothing in this image adds users afterwards.
Independent of the Rust commits on this branch - touches only the Dockerfile,
so it cherry-picks to main on its own.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
With docker exec working, the Dev Containers flow got as far as installing the server and then died: check-requirements.sh: line 48: getconf: command not found getconf is the symptom, not the disease. The server ships a 121 MB glibc-linked node built for FHS - it wants /lib/ld-linux-aarch64.so.1 as its ELF interpreter, and this image has no /lib at all. Verified against the real artifact: the unpatched binary fails with a bare "no such file or directory" naming the binary rather than the missing loader. Microsoft ships an escape hatch for exactly this. Given all three variables, bin/code-server patchelfs its own node on first launch, and check-requirements.sh exits 0 the moment it sees the LINKER one - before any FHS probing. Verified end to end against the exact server build VS Code 1.131.0 pulls: patch via these paths, then `node -e` prints v24.18.0. --inputs-from resolves nixpkgs through the flake's lock rather than the ambient registry, so this reuses the glibc the image already links against; marginal cost is patchelf and gcc-lib. -o roots them for the GC, as /opt/hm-activation already does. The unsuffixed symlinks exist so ENV encodes no store path, architecture, or nix output-naming rule - `-o foo` on a non-default output lands at foo-lib, which cost a debugging round here. The tests turn that class of mistake into a build failure. Not fixed via /etc/os-release ID=nixos, the other bypass in that script: it parses the file with sed, which is absent from a docker exec PATH here, under set -e. That trades missing-getconf for missing-sed. The env-var check runs first and shells out to nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ning Two defects, one root: the image is a Nix rootfs while Dev Containers and the VS Code Server assume FHS. Both reported something other than themselves - an exec failure that reads as a volume bug, and a missing getconf that is really a missing dynamic linker - so the write-up leads with the misdirection and records both wrong turns rather than only the fix. Also defines "Dev image" in CONCEPTS.md. Three entries in the Dev container cluster already leaned on the term without it being defined anywhere, and the non-FHS property is what makes this whole class of failure predictable rather than surprising. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The VSCODE_SERVER_CUSTOM_GLIBC_* mechanism was reverse-engineered from the server's own shell scripts, but it is documented - under "older Linux distributions", which is why it does not surface when searching for NixOS or FHS. Same mechanism: "older distro" and "no FHS at all" fail its glibc assumptions identically. Recording the link so the next person reads the docs instead of the tarball. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
⊕ Entity-level changesCONCEPTS.md
Dockerfile
docs/solutions/integration-issues/vscode-dev-containers-on-a-nix-rootfs.md
home-manager/modules/dev/rust.nix
home-manager/profiles/dev.nix
home-manager/profiles/work.nix
Summary: 19 added, 7 modified, 3 deleted across 6 files (1 added orphan, 2 modified orphans, 1 deleted orphan) functions and classes, not lines · sem |
Two gaps that only show up once someone actually builds in the container. `linker \`cc\` not found` on the first dependency with a build script. rustup ships rustc and cargo and deliberately stops there - linking is the system's job - so a profile with no C toolchain cannot compile anything with a -sys crate or a build.rs. stdenv.cc provides cc, ld and binutils wired to this nixpkgs. pkg-config rides along because the -sys crates shell out to it and its absence fails just as indirectly; it does not supply the libraries, so a crate needing openssl still needs openssl in the profile or its vendored feature. rust-analyzer as a rustup component, plus an arch-independent symlink for editors to name. VS Code's extension ships its own prebuilt server, which is an unpatched FHS binary that cannot run here - it fails as `spawn ... ENOENT`, which reads as "file missing" when the file is present and only its ELF interpreter is absent. Same root cause as the VS Code Server node binary, and the same reason the standing "rustup provides rust-analyzer" rule is load bearing rather than stylistic: rustup's copy is patched by nixpkgs and runs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
P1 —
That's outside what this PR set out to do: the commit fixes Suggested fix — scope both to Linux: home.packages = with pkgs; [
rustup
bacon
] ++ lib.optionals pkgs.stdenv.hostPlatform.isLinux [
stdenv.cc
pkg-config
];The mechanism is verified; the practical breakage on macOS is inferred — I couldn't test it on the actual machine. |
|
P2 — this size justification is now off by ~4x, on the machine it names The comment says rustup is 94 MiB and that the import "stays cheap for Measured against
Both container images carry it, If the |
Makes the
devimage actually work as a VS Code devcontainer, and gives it a Rust toolchain. The two are independent — the Rust commits are the original goal, the Docker commits are what it took to get a devcontainer to open at all.Two defects, one root cause
The image is a Nix rootfs; Dev Containers and the VS Code Server assume FHS. Both failures reported something other than themselves, which is most of why they took a while:
docker execwas broken. The base image ships/etc/{passwd,group,shadow}as absolute symlinks into the store. They resolve fine inside the container, sodocker runworks — butdocker execresolves the user entry host-side withopenat2 RESOLVE_BENEATHsemantics, which rejects absolute symlinks as escaping their parent. Dev Containers execs a shell server as its first act, so it died before any lifecycle command ran:The server's
nodecouldn't run. It ships a 121 MB glibc-linked binary wanting/lib/ld-linux-aarch64.so.1; this rootfs has no/lib. The visible error wascheck-requirements.sh: getconf: command not found— a symptom, not the disease. Fixed with the mechanism Microsoft documents under "Can I run VS Code Server on older Linux distributions?": givenVSCODE_SERVER_CUSTOM_GLIBC_LINKER,VSCODE_SERVER_CUSTOM_GLIBC_PATHandVSCODE_SERVER_PATCHELF_PATH, the server patchelfs its ownnodeon first launch and the requirements check short-circuits.--inputs-fromresolves nixpkgs through this flake's lock rather than the ambient registry, so it reuses the glibc the image already links against — marginal cost is patchelf and gcc-lib, not a second libc.Rust in the dev profile
alyssa@devhad norustup,cargoorbacon, so the image that exists to be a devcontainer couldn't build a Rust project.modules/dev/rust.nixhad been there all along withwork.nixas its only importer — because importing it cost 1.6 GiB of closure forlldb(measured; rustup is 94 MiB, bacon 63 MiB). That would have landed onalyssa@dev-x86too, on the 2012 MBP, which is the disk headroom PR #34 exhausted.lldbisn't a Rust dependency and CodeLLDB bundles its own, so it moves towork.nixand the shared module becomes importable.Also replaces
programs.zsh.initExtra = "rustup update"with an activation-timerustup default stable. The old form ran a network call on every shell — miserable when every VS Code terminal pays it — and was the wrong command besides:rustup updateinstalls nothing when no toolchain exists, leavingcargoas a shim that errors. Guarded with|| true, because activation runs at container start and a failedrunaborts it after dotfiles are linked, which is the documented route to a container with a perfect prompt and no packages.Verification
alyssa@devandalyssa@dev-x86evaluate;devgains exactlyrustup + bacon,workkeepsrustup + bacon + lldb.docker execondevreturned 126, and 0 once the three files were materialized.nodefails with a bare "no such file or directory"; patched through these paths,node -eprints v24.18.0.Notes
alyssa@work-devdoes not evaluate on aarch64-linux —work.nixcarriescocoapods, which isaarch64-darwinonly. Verified pre-existing onmain; left alone as unrelated.mainon their own. Worth landing first if the Rust work needs discussion — until they're in, every freshdevbuild breaks Dev Containers for any folder."postStartCommand": "/opt/dotfiles/docker/entrypoint.sh true"in their devcontainer config: the CLI starts containers with--entrypoint /bin/sh, so the image ENTRYPOINT and its home-manager activation never run on their own.Full write-up in
docs/solutions/integration-issues/vscode-dev-containers-on-a-nix-rootfs.md, including the two wrong turns.🤖 Generated with Claude Code
https://claude.ai/code/session_01KBhs11SMRHwp2CGkoGRzRj