dsh-layerfs 0.1.0 on OrbStack (kernel 7.0.14): 8/10 scripted scenarios stop at workspace.unmount (stage Join); Docker Desktop 4.22.0 cannot create Sandboxes (API 1.43 < 1.54) #466

Activity

1. yifanxuaaa commented on Oct 27, 2026

   

   yifanxuaaa

   on Oct 27, 2026

   Contributor

   More actions

   Thanks for the detailed matrix. The three runs and the exact TeardownCustody output are very useful.

   A lot has changed since v1.1.0 (9f97a7ec95). Could you update to the latest main (currently ea11b00408) and rebuild the sealed bundle, then run Run all once more on OrbStack?

   • Docker Desktop 4.22.0: fix(sandbox): negotiate the Docker Engine API version (#457) #476 now negotiates the Docker API version instead of always sending /v1.54. Engines below API 1.48 (Docker Engine 28.0) are refused with a clear Unsupported error, because Sandbox upload fails on 23.0–27.5. So 4.22 still won't run Sandboxes, but you should see that error instead of the raw 400. Please confirm that's what you get.
   • Join wedge: this isn't fixed on main yet. Workspace mounted while its Sandbox starts cannot be unmounted (TeardownCustody { stage: Join }) and stays wedged #426 and FUSE readers remain blocked after successful unmount: independent reproducer and Linux comparison (related to #426) #460 are still open. Note that D8 · Refactor core/ into core, engine and api layers #463 moved the daemon to engine/crates/layerfs-daemon/ (DRAIN_WAIT is now engine/crates/layerfs-daemon/src/application/filesystem.rs:54).

   If it still fails at Join on the latest main, could you open a PR with the fix? We don't have OrbStack (or a 7.0.14 kernel) to reproduce this, so you're in the best position to write and check it. Making detach wake or abort the receive loops (your second option, per #460) is the direction we'd prefer over only making the deadline configurable, since a longer wait just delays the same wedge. Please link #426, #460 and this issue. Include a before/after from your scenario matrix, plus a test that fails without the fix where that's practical. AGENTS.md has the repo rules for tests and commit messages.
2. yifanxuaaa

   added

   orbstackSeen on OrbStack (macOS Docker engine)Seen on OrbStack (macOS Docker engine)

   on Oct 27, 2026
3. mengmengmengqiang commented on Oct 28, 2026

   

   mengmengmengqiang

   on Oct 28, 2026

   ContributorAuthor

   More actions

   Updated to the latest main and rebuilt the sealed bundle — the Join stall is still here on a newer OrbStack kernel, but #481 now recovers it.

   Answering the request to retest on latest main. Environment below; probe uses the public SDK only.

   Host	macOS 26.5.1 (Darwin 25.5.0, arm64)
   Docker Engine	OrbStack 29.8.2, API 1.54, kernel 7.2.9-orbstack-00391-g2b298d05fbd4 (newer than 7.0.14 in the report)
   Node	22.23.2
   LayerFS	9f97a7ec95 (v1.1.0) → 582bc786f9 (main; includes #481, #488, #476, #463 layout)
   Bundle	rebuilt from that checkout with tools/worktree.py build --platform linux --profile release --jobs 6, sealed as build-linux-aa6c5132eb224caaa85dbbfb26a4b010
   Runtime image	unchanged, sha256:d93570c6… (node/rust/chromium only; the daemon comes from the bundle)
   dsh-layerfs	baac1b9 → b9bf3bc (Follow LayerFS's #463 layout; pins fabefdf87a)

   Probe

   Same shape as the earlier #426 comment: for each iteration sandbox.create() → workspace.mount(sandbox, branch) immediately → sleep 1500 → workspace.unmount(ws); on refusal workspace.unmount(ws, true, true); then sandbox.close().

   Results

   Build	Mount after create	Plain unmount	Then	Outcome
   v1.1.0 9f97a7ec	62 ms	Failed (Join), 5 064 ms	sandbox.close → MountsActive	wedge, only docker rm -f clears it (1/1)
   main 582bc786 ×6	13–79 ms	Failed (Join), 5 044–5 504 ms — 6/6	--force → ForceUnmounted	close → Closed 6/6, 0/6 wedged

   Same refusal signature as before the upgrade:

   Failed: OperationFailure { request: Unmount(… discard: false, force: false …),
     cause: Retained(TeardownCustody { stage: Join, detached: true,
       … loops_exited: 0, joined: 0 … }) }
   

   Control on the same upgraded build: delay 2 500 ms between sandbox.create() and the mount → plain Unmounted 3/3 (21–27 ms). On v1.1.0 the delayed loop was 7/7, and six Sandboxes in parallel with the 2.5 s delay were 6/6. So the stall tracks the early-mount window, not load; on this kernel it is deterministic there.

   What this says

   • The root cause behind FUSE readers remain blocked after successful unmount: independent reproducer and Linux comparison (related to #426) #460 is unchanged on 7.2.9: plain unmount still stops at Join after a successful detach.
   • fix(daemon): end a normal unmount stalled at Join with one forced unmount (#426, #460) #481 does exactly what it describes: one forced unmount ends the Retained entry, and the Sandbox then closes. A caller that does not pass force still sees the failure — dsh-layerfs's scenario runner does not retry with --force, so I did not expect Run all to turn green and did not run it (the scripted-model plugin is not installed in this profile). The deterministic probe above is the path those scenarios take.
   • fix(sandbox): negotiate the Docker Engine API version (#457) #476 was not exercised: OrbStack is API 1.54, so the Unsupported message for Docker Desktop 4.22 is still unverified.

   Since the early-mount stall reproduces 6/6 here, I can also test a candidate fix for #460 (waking / aborting the receive loops) against these runs, and run Run all once the scripted model is enabled.
4. mengmengmengqiang commented on Oct 28, 2026

   

   mengmengmengqiang

   on Oct 28, 2026

   ContributorAuthor

   More actions

   Follow-up: I wrote the fix in the direction you suggested — the daemon now wakes the stalled connection itself instead of adding to the deadline.

   PR: #496 (fix(daemon): end a stalled normal unmount at Join with its own abort write).

   On a Join stop the normal unmount makes one abort_detached write on the already-detached connection and runs the same forced drain, then revokes and closes as usual. Only a refused or short write, or a second stop, keeps custody.

   Same probe as my previous comment, same machine (kernel 7.2.9-orbstack-00391), this branch's daemon:

   	plain unmount	sandbox.close
   582bc786	Failed (Join), 5.0–5.5 s — 6/6	only after --force (6/6)
   PR #496	Unmounted 6/6 (5.05–5.08 s: the drain deadline, then the abort)	Closed 6/6

   The new stalled_unmount test fails on the parent (which keeps Retained { stage: Join }) and passes with the change; the docs say what --force is still for. Happy to change the design (opt-in flag, or a different abort timing) if you'd prefer that over doing it inside a normal unmount.