Skip to content

feat(sandbox): 统一 Sandbox provider 接口,收编 Docker / Linux 原生 / macOS 原生(B1/B2/B3,叠在 #427 上) - #498

Draft
jiangkoumo wants to merge 85 commits into
Ephemeral-AI-Lab:mainfrom
jiangkoumo:feat/sandbox-provider-b1
Draft

jiangkoumo wants to merge 85 commits into
Ephemeral-AI-Lab:mainfrom
jiangkoumo:feat/sandbox-provider-b1

Conversation

@jiangkoumo

Copy link
Copy Markdown
Contributor

概要

对应 #446 B 区的 B1(统一 Sandbox provider 接口)、B2(Linux 原生) 和 B3(macOS 原生) 的接口部分。这里把 Docker、Linux 原生(root daemon + FUSE)、macOS 原生(loopback NFSv3)三种后端收进同一套 SandboxProvider / SandboxInstance 生命周期接口,layerfs-api 和 layerfs-node 只走 provider 这一条路径。

本 PR 叠在 #427 上面:分支从 #427 的头 d55acc15f 开始,在它后面加了 11 个提交(其中 1 个是 merge)。评审时只看 d55acc15f.. 之后的提交就行。#427 合并后,这部分 diff 会只剩下本 PR 自己的改动。main 目前是 582bc786f,不需要再 merge。

提交

提交 内容
559bff5 定义 provider 生命周期 trait、命令类型、ExecRequest
8ce4697 macOS 原生改成 NativeMacosProvider / NativeMacosInstance
59c0da2 抽出共用的 daemon 启动、握手、Hello、Store 安装流程
c82a903 Linux 原生 provider,复用 NativeProvider<const LINUX: bool>
7a6952c Node 的 Linux native 部署接到 provider 流水线
e5004d6 DockerProvider / DockerInstance(create/inspect/run/log/stop/remove,只尝试一次,typed refusal)
2e4e3cf node 测试里的假 Docker Engine 补上 GET /version 应答(上游 73d974d 加版本协商之后,夹具的回复错位,测试报 ConnectionRefused)
abed25d merge 上面的夹具修复
7922044 ProviderLaunchOptions::Docker;用 SandboxRecord 枚举统一 local/docker 记录(磁盘格式不变);去掉 reopen_docker
70ec59e SandboxApi 只走 provider
8cffb6b node 的 Docker 路径走 DockerProvider,删掉旧入口(ManagedSandbox、SandboxCreate、SandboxApi::new(Docker)、internal::split/exec 等)

保持不变的行为

验证(本地;上游 CI 结果以本 PR 页面为准)

macOS(M2,macOS 27.2,rustc 1.85.1)

  • cargo fmt --all -- --check、cargo clippy --locked --all-targets -- -D warnings:通过
  • layerfs-sandbox、受影响的 layerfs-api 测试目标、layerfs-node:通过(跳过 10 万文件规模的 sdk_merge_scale)
  • node --test api/cli/test/cli.test.mjs:75/75
  • release(CARGO_PROFILE_RELEASE_STRIP=none)SDK smoke 48/48,PATH inherit 7/7、fallback 4/4

Linux(Debian 13,root,真实 FUSE)

  • 全仓 clippy -D warnings、layerfs-sandbox 全过
  • layerfs-node:native_discovery 6/6、native_admission 2/2、native_local 7/7
  • 原生 SDK smoke 48/48(N6b 真实 FUSE 挂载、N8 exec 身份 501、Z1 没有挂载、daemon 或目录残留)

真实 Docker(Docker 29.8.2,runtime/sandbox 镜像 layerfs-352-runtime:node24)
在 runtime 镜像里用 node api/agent/test/verify/<场景>.mjs inner 跑(环境变量和 tools/worktree_run.py 一样,--network host):

场景 本 PR 基线 d55acc1
sandbox_real(16 个挂载到 Capacity、MountsActive、close) ok ok
idle_close_real ok ok
calls_real store / workspace PASS 28 / PASS 15 PASS 28 / PASS 15
reap_real ok ok
sandbox_image_real ok ok
fs_mode_real ok ok
exec_mount_lookup_real ok ok
exec_output_real check(每路 64 MiB,SHA-256 一致) ok ok

两边的残留容器数也一致(idle_close 等场景按设计留下一个已停止的 Sandbox)。

需要作者确认

  1. 接口形状:trait 的划分、SandboxRecord 枚举、ProviderLaunchOptions 按后端分变体,这样设计是否认可
  2. Linux native 在本 PR 里重新可以对外选择(feat: macOS 原生适配——daemon 内置 loopback NFSv3 导出,普通用户 mount_nfs 挂载 Workspace(无 Docker / 无 macFUSE / 无 root) #427 本身只开放 macOS),这一点是否符合 B2 的预期
  3. 本 PR 不包含 B8/B9/B10

…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: core/crates/layerfs-sandbox/Cargo.toml (not production (test, manifest or docs)).
No tests changed or deleted.

Production LOC: 108,451 -> 108,451 (delta 0)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/mod.rs (production).
No tests changed or deleted.

Production LOC: 108,451 -> 108,460 (delta +9)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/process.rs (production).
No tests changed or deleted.

Production LOC: 108,460 -> 108,505 (delta +45)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/mounts.rs (production).
No tests changed or deleted.

Production LOC: 108,505 -> 108,576 (delta +71)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/create.rs (production).
No tests changed or deleted.

Production LOC: 108,576 -> 108,834 (delta +258)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/sandbox.rs (production).
No tests changed or deleted.

Production LOC: 108,834 -> 109,057 (delta +223)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/exec.rs (production).
No tests changed or deleted.

Production LOC: 109,057 -> 109,166 (delta +109)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: core/crates/layerfs-sandbox/src/backend/mod.rs (production).
No tests changed or deleted.

Production LOC: 109,166 -> 109,167 (delta +1)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: core/crates/layerfs-sandbox/tests/local_runtime.rs (not production (test, manifest or docs)).
No tests changed or deleted.

Production LOC: 109,167 -> 109,167 (delta 0)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/native/src/runtime.rs (production).
No tests changed or deleted.

Production LOC: 109,167 -> 109,274 (delta +107)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/native/src/local.rs (production).
No tests changed or deleted.

Production LOC: 109,274 -> 109,527 (delta +253)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/native/src/registry.rs (production).
No tests changed or deleted.

Production LOC: 109,527 -> 109,527 (delta 0)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/native/src/discovery.rs (production).
No tests changed or deleted.

Production LOC: 109,527 -> 109,644 (delta +117)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/native/src/exec.rs (production).
No tests changed or deleted.

Production LOC: 109,644 -> 109,633 (delta -11)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/native/src/sandbox.rs (production).
No tests changed or deleted.

Production LOC: 109,633 -> 109,641 (delta +8)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/native/src/workspace.rs (production).
No tests changed or deleted.

Production LOC: 109,641 -> 109,636 (delta -5)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/native/src/lib.rs (production).
No tests changed or deleted.

Production LOC: 109,636 -> 109,638 (delta +2)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/native/tests/unit/runtime.rs (not production (test, manifest or docs)).
No tests changed or deleted.

Production LOC: 109,638 -> 109,638 (delta 0)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/src/client.js (production).
No tests changed or deleted.

Production LOC: 109,638 -> 109,656 (delta +18)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/src/index.js (production).
No tests changed or deleted.

Production LOC: 109,656 -> 109,658 (delta +2)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/test/local_deployment.mjs (not production (test, manifest or docs)).
No tests changed or deleted.

Production LOC: 109,658 -> 109,658 (delta 0)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch).
This commit: sdk/js/README.md (not production (test, manifest or docs)).
No tests changed or deleted.

Production LOC: 109,658 -> 109,658 (delta 0)
Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…l-AI-Lab#402)

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/process.rs (production).
No tests changed or deleted except additions.

Production LOC: 109,658 -> 109,696 (delta +38)
Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402)

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/groups.rs (production).
No tests changed or deleted except additions.

Production LOC: 109,696 -> 109,818 (delta +122)
Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402)

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/exec.rs (production).
No tests changed or deleted except additions.

Production LOC: 109,818 -> 109,835 (delta +17)
Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402)

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/sandbox.rs (production).
No tests changed or deleted except additions.

Production LOC: 109,835 -> 109,868 (delta +33)
Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402)

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/create.rs (production).
No tests changed or deleted except additions.

Production LOC: 109,868 -> 109,882 (delta +14)
Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402)

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/mounts.rs (production).
No tests changed or deleted except additions.

Production LOC: 109,882 -> 109,870 (delta -12)
Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402)

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch).
This commit: core/crates/layerfs-sandbox/src/backend/local/mod.rs (production).
No tests changed or deleted except additions.

Production LOC: 109,870 -> 109,871 (delta +1)
Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402)

Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch).
This commit: core/crates/layerfs-sandbox/src/backend/mod.rs (production).
No tests changed or deleted except additions.

Production LOC: 109,871 -> 109,872 (delta +1)
Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
jiangkoumo and others added 30 commits October 26, 2026 02:35
Replace the unconditional wait after helper KILL with a five-second
try_wait observation; retain unreaped children and report their PID.
Read helper pipes nonblockingly with capped diagnostics, and keep the
export and engine custody when a mount helper times out after spawn.

Detach local Sandbox mounts while the daemon still serves NFS. Continue
stopping after detach failure; keep Delete's checked detach as its fence.
Bound macOS force-unmount helpers too, including stderr capture.
Keep mount-table process imports macOS-only so Linux has no unused imports.

Give discovery ownership acquisition one 70-second deadline, covering a
single 60+5-second helper and return headroom. Preserve lock scope, ordering
and Busy refusal; do not release ownership around an in-flight mutation.
A real SDK lock test refuses at 60s before and succeeds after release at
62s with the new budget. Multiple queued holders can still yield Busy.

Add process-helper regression tests and an opt-in real macOS mount proof.
Mark the actual 60-second deadline test slow. No mount-option changes.
Original kernel U-state hang remains unreproduced on this Mac.

Validation: daemon/NFS/sandbox all-target tests 102 passed, 0 failed,
4 ignored; native smoke 48/48; macOS all-target Clippy including binding
and Rust formatting pass. Prior real Stop fixture failed before and passed
after. Existing product guard reports five binding inline-test violations.
Linux cross-Clippy could not build SQLite/zstd without x86_64-linux-gnu-gcc;
the cfg correction still needs native Linux confirmation. Fresh-context
read-only review of the teardown fix found no P0/P1.

Production LOC: 108058 -> 108240 (delta +182)
Scope: first-party core Rust/runtime SQL plus api Rust/JS; core 101993 ->
102175, api 6065 unchanged. Daemon 7058 -> 7157 (+99), sandbox 4476 ->
4559 (+83). Other packages unchanged; helper relocation included.
Method: tools/production_loc.py at a700af8, same counter over git archive
parent a700af8 and exact staged tree 0edfb78.
Excludes tests/docs/tooling and test-only source. Reference absent (retired
R9c; last counted 65,417 at tree 498dd19).
Register a Workspace-scoped unmount intent and exclusive flock under the
Sandbox guard, then release Sandbox ownership during the daemon exchange.
Publish the known result under the Workspace lease before shared-record
cleanup. Repeated requests read progress or the saved outcome, never resend.

Use existing Busy/Unknown refusals with Unmounting/Retained state and elapsed
milliseconds. Gate token/directory and backend reads for that Workspace while
other Workspaces and discovery reads proceed. Keep the 60+5-second helper
budget and restore the discovery readiness wait to 60 seconds.

Explicit Sandbox close may take over orphaned or retained unmounts but still
refuses unrelated active Workspaces. Preserve Stop's detach-before-termination
and Delete's checked detach. Keep known daemon success on cleanup failure and
report cleanupError separately. Nonblocking shared lease probes cannot block
one another; known-token lookup avoids scanning historical result records.

Add a real native SDK regression with paused helpers, cross-process progress,
Busy/Dirty reversibility, publication during a held discovery lock, caller
crash, saved failure, and explicit close takeover. The old binding fails its
900ms discovery-read assertion; the new binding completes unrelated calls in
milliseconds. Update lifecycle documentation and describe interface details.

Validation: daemon/NFS/sandbox all-target tests 102 passed/0 failed/4 ignored;
binding 9/9; native smoke 48/48; real concurrency fixture PASS; macOS Clippy
(all targets, including binding), fmt, JS syntax and diff checks pass.
Linux cross-Clippy blocked by missing x86_64-linux-gnu-gcc in SQLite/zstd;
existing five binding product-boundary violations remain unchanged in kind.
Fresh-context review identified two recovery issues, both fixed and verified.
No public method or daemon wire change; new detail shapes and close takeover
semantics require maintainer alignment before upstream publication.

Production LOC: 108240 -> 108557 (delta +317)
Scope: first-party core Rust/runtime SQL plus api Rust/JS; core 102175 unchanged,
api 6065 -> 6382, binding 3412 -> 3729; other packages unchanged.
Method: tools/production_loc.py over git archive first parent 4c1648e and
exact staged tree 5394d16, same counter,
excluding tests/docs/tooling and test-only source. Reference absent (retired
R9c; last counted 65,417 at tree 498dd19).
The Workspace unmount lease opened its record with a hand-written
O_NOFOLLOW: 0x100 on macOS and 0x20000 on Linux. 0x20000 is O_NOFOLLOW
only on x86_64 Linux; on aarch64 Linux O_NOFOLLOW is 0x8000 and 0x20000 is
O_LARGEFILE, so the lease would silently follow a symlink there. Take the
constant from libc (already in the lock at =0.2.189 through other crates)
so every target gets its own value.

No behavior change on macOS or x86_64 Linux. The binding's existing
discovery::no_follow() carries the same hand-written values and is left
as it is (it predates this branch).
Sandbox::stop() ended the daemon first and left the Sandbox's mounts in
place until Delete detached them. Detach them first, while the daemon can
still answer, then stop the daemon as before. A failed detach is reported
and does not block the crash teardown; Delete keeps its own checked detach
before removing anything through a mount.

On Linux FUSE this is ordering only: Stop now returns with the mounts
already gone (measured 21.6 ms with a live holder and a dirty writer; before
the change the mounts stayed until Delete), and reclaim is unchanged
(0.09-0.10 s). MNT_DETACH does not flush dirty pages, so it adds no data
guarantee; Stop stays the crash teardown it was.

Same hunk as the local-backend part of the macOS native branch's NFS
unmount-hang fix, so merging this branch there stays clean.
…meral-AI-Lab#402)

Upstream now keeps the Node binding's tests outside the crate source: they
load the ordinary built addon and call its exported methods, never including
private binding source (engine/AGENTS.md, engine/crates/layerfs-node/tests).
The Ephemeral-AI-Lab#402 unit tests that were included by runtime.rs and sandbox.rs
(tests/unit/{runtime,reclaim}.rs before D8) are rewritten as
engine/crates/layerfs-node/tests/native_local.rs, which writes a Sandbox
record as an earlier call would and calls the public
`runtime.sandbox.reclaim` with a local deployment:

- a local record naming any other directory (a data directory, another
  LAYERFS_LOCAL_DIR, another Sandbox or name, a relative or `..` path, `/`)
  is refused InvalidInput, "neither used nor deleted", and nothing is
  removed;
- a local record is used only with an absolute, plain deploy.localDir;
- a stopped local Sandbox where this deployment puts it is deleted and its
  number freed; a record from before boot ids is read without one; a
  damaged boot or port is refused and removes nothing;
- a Docker record is not read as a local Sandbox.

The Docker reclaim cases of the old reclaim.rs are covered by upstream's
native_discovery.rs. Selection of a misspelled deploy.sandbox is refused only
by sandbox.create after the project's Store is installed, so it is not
reachable without a Store and is left to the SDK test (local_deployment.mjs).

Production LOC: 103762 -> 103762 (delta 0)
(tools/production_loc.py --root ., core+engine+api; test-only commit)
…e built addon) into PR Ephemeral-AI-Lab#427 (local port)

Brings Ephemeral-AI-Lab#402's engine/crates/layerfs-node/tests/native_local.rs. One
conflict: api/binding/tests/unit/runtime.rs, deleted on Ephemeral-AI-Lab#402 and kept by
Ephemeral-AI-Lab#427's D8 merge, is deleted here too. api/binding/ no longer exists after
D8, the file was included by nothing, and its cases move to native_local.rs
in the next commit.

Production LOC: first-parent comparison 1931032 -> this merge, see
tools/production_loc.py --root .; tests only, delta 0.
Production LOC: 103762 -> 103762 (delta 0)
(tools/production_loc.py --root ., core+engine+api; formatting of a test file only)
…fter D8 (Ephemeral-AI-Lab#427)

- engine/crates/layerfs-node/tests/native_local.rs gains
  local_and_native_sandboxes_are_never_mixed: through the built addon's
  public runtime.sandbox.reclaim, a native record is refused (InvalidInput,
  "never mixed") by a local deployment and a local record by a native one,
  nothing is removed, and a native deployment reclaims its own stopped
  record. It replaces the native cases of the old included
  api/binding/tests/unit/runtime.rs (removed in the previous merge).
- The real native SDK regression of the unmount lease moves from
  api/binding/tests/unmount-concurrency.mjs (api/binding/ is gone after D8)
  to api/agent/test/verify/native_unmount_real.mjs, next to upstream's other
  *_real.mjs verifications, with its imports updated to D8's layout
  (api/agent/src, api/backend/src). It stays macOS-only and explicit.
- discovery.rs: the "never mixed" refusal lost its line continuation in the
  v1.1 port and carried a run of spaces inside the message; restored.

Production LOC: 108595 -> 108596 (delta +1)
(tools/production_loc.py --root ., core+engine+api; the restored message
continuation splits one source line in two)
…Ephemeral-AI-Lab#427)

D8 (Ephemeral-AI-Lab#465) made tools/check_product_boundary.py fail closed on any LayerFS
crate missing from its core/engine/api layer map. Ephemeral-AI-Lab#427's layerfs-nfs (the
loopback NFSv3 export, beside layerfs-fuse under engine/crates) was not in
it, so the full and the CI (--skip-sdk) runs reported
"crate missing from layer map: layerfs-nfs" for layerfs-nfs and for
layerfs-daemon, which depends on it. It is an engine crate like
layerfs-fuse; its dependencies (content, overlay, workspace, fuse) are at
or below that layer.

Production LOC: 108596 -> 108596 (delta 0)
(tools/production_loc.py --root ., core+engine+api; development tool only)
…-Lab#402)

Production LOC: 103762 -> 103762 (delta 0)
(tools/production_loc.py --root ., core+engine+api; documentation only)
…nto PR Ephemeral-AI-Lab#427 (local port)

Production LOC: 108596 -> 108596 (delta 0), first-parent comparison
(tools/production_loc.py --root ., core+engine+api; documentation only)
…b#427)

Production LOC: 108596 -> 108596 (delta 0)
(tools/production_loc.py --root ., core+engine+api; documentation only)
…-AI-Lab#427)

Brings main up to 582bc78 (Ephemeral-AI-Lab#488, Ephemeral-AI-Lab#481 forced unmount after a stalled
Join, Ephemeral-AI-Lab#479, Ephemeral-AI-Lab#471, Ephemeral-AI-Lab#473, Ephemeral-AI-Lab#445 fs modes, Ephemeral-AI-Lab#475, Ephemeral-AI-Lab#478, Ephemeral-AI-Lab#476 Docker API
negotiation, Ephemeral-AI-Lab#396, Ephemeral-AI-Lab#419, Ephemeral-AI-Lab#418, Ephemeral-AI-Lab#415, Ephemeral-AI-Lab#477, Ephemeral-AI-Lab#468).

Two conflicts, both from Ephemeral-AI-Lab#481 (forced unmount of a normal unmount stalled
at Join):
- engine/crates/layerfs-daemon/src/control/force.rs: guard() keeps this
  branch's ExportKept check (an NFS export kept after a failed teardown
  answers with its stored custody first) and takes main's comment and its
  forceable() exception for a retained FUSE session. The non-Linux
  `let _ = token;` stays removed: ExportKept uses the token on every
  platform.
- engine/crates/layerfs-node/src/workspace.rs: unmount() keeps this
  branch's per-Workspace unmount lease (repeat a published outcome, drop
  the admission guard before the daemon call, publish the outcome before
  cleanup) and takes main's `force` option inside it: forced, one
  api.unmount(token, true) whose None answer is Unmounted::Closed;
  otherwise unmount_checked with `discard` as before.

Production LOC: first-parent 9478190 108596 -> 109158
(tools/production_loc.py --root .; the delta is main's own changes)
…er selectable (Ephemeral-AI-Lab#427)

Ephemeral-AI-Lab#427 is the macOS native adaptation. The local-daemon runtime it builds
on (first proposed in the closed Ephemeral-AI-Lab#402) is now part of this PR, and its
only selectable form is `native`: on macOS a daemon of this user serves
its Workspaces over loopback NFS to the host's own client. Linux keeps
the Docker route exactly as on main.

- deploy.sandbox (LAYERFS_SANDBOX) accepts unset, `docker` or `native`;
  `local` is refused InvalidInput like any other value, in the SDK
  (api/agent/src/client.js) and in the binding (runtime::local).
- local::start refuses `native` off macOS as Unsupported before any
  effect, so no deployment starts a local daemon on Linux. The root
  check and the Local::new (Linux FUSE) branch go: native always takes
  Local::native, LAYERFS_KERNEL=nfs and this user's identity.
  discovery::root() had no other caller and goes too.
- Records: a new record's runtime is always `native`. A native record is
  used only by a native deployment; a record of the former `local`
  runtime (an earlier, unmerged build) is used by no deployment, refused
  InvalidInput ("never mixed", "neither used nor deleted"), and nothing
  it names is deleted.
- Tests: engine/crates/layerfs-node/tests/native_local.rs writes native
  records (record handling is the same code on Linux) and gains
  a_record_of_the_former_local_runtime_is_never_used_nor_deleted;
  api/agent/test/local_deployment.mjs becomes native_deployment.mjs with
  `local` among the refused values.
- api/docs/agent.md drops the "Local Sandboxes" section and keeps its
  layout, record and reclaim facts under "Native Sandboxes (macOS)".

The local backend in layerfs-sandbox (engine/crates/layerfs-sandbox/src/
backend/local) is kept as it is: native uses its process, exec, group,
mount and record code; Local::new and its Linux tests stay as tests of
that shared code.

Production LOC: 109158 -> 109142 (delta -16)
(tools/production_loc.py --root .)
Extract ExecRequest without changing Docker/local execution. Preserve Docker's public export. Launch failures retain phase, original cause and partial ownership. Defer B8 discovery and doctor integration.

Production LOC: 109142 -> 109295 (delta +153). Scope: core 41400, engine 65063 -> 65216, API 2679. Method: unchanged tools/production_loc.py on exact parent and checkout-index staged tree; nonblank noncomment product source excluding tests. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).

Validation: sandbox no-run and tests, product boundary guard, fmt; all exit 0. No tests deleted.
Reuse Local and Darwin process ownership. Retain partial launch resources, enforce native identity and NFS environment, and expose record facts without scanning directories. Add a real-process lifecycle/custody test; no tests deleted.

Production LOC: 109295 -> 109508 (delta +213). Scope: core 41400, engine 65216 -> 65429, API 2679. Method: unchanged tools/production_loc.py on exact parent and checkout-index staged tree; excludes test-only source and comments. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).

Validation: sandbox no-run and tests, fmt, product boundary; exit 0.
Accept Arc<dyn SandboxProvider> through with_provider/create_provider_file. Share TCP, Noise, Hello and Store install with the compatible Docker entrypoints. Node native launch and command execution use provider owners; preserve existing records and explicit failure cleanup. Validate owned local topology before launch.

Tests: add provider ownership refusal coverage. Adapt the existing local-install success test to macOS, whose install is supported by Ephemeral-AI-Lab#427; retain all success/permissions/second-install refusal assertions. No tests deleted; obsolete off-Linux refusal is now limited to unsupported platforms.

Production LOC: 109508 -> 109773 (delta +265). Scope: core 41400, engine 65429 -> 65694, API 2679. Method: unchanged tools/production_loc.py on exact parent and checkout-index staged tree 5a04326; excludes comments, tests, fixtures and tools. Reference absent (retired R9c; last counted 65,417 at tree 498dd19).

Validation: fmt, all-target locked Clippy, all-target no-run, boundary scan and 155 guard self-tests exit 0; sandbox 38 tests passed (1 ignored), API 160 passed (15 ignored) plus new ownership regression; changed API targets 10/10; node Rust command passes with Linux-only targets absent on macOS; CLI Node 75/75 and agent 39 cases pass. Final release strip=none, native smoke 48/48 and PATH 11/11 pass. Independent read-only review finding fixed. Linux runtime/Docker host qualification not run; Docker protocol tests pass on macOS.
Expose NativeLinuxProvider/NativeLinuxInstance alongside the macOS aliases.
Share launch/reopen/inspection/lifecycle implementation, retain Linux root,
traversable runtime root, nonroot command identity and fixed PATH, and select
FUSE explicitly. Add platform refusal coverage and extend lifecycle coverage
to Linux root runs. No provider trait signatures change.

Validation: Mac sandbox tests 40 passed, 0 failed, 1 ignored; warning-denying
all-target Clippy passed. Linux x86_64 all-target sandbox cargo check passed;
real Linux daemon/FUSE qualification remains pending.

Production LOC: 109773 -> 109803 (delta +30)
Scope: core 41400 unchanged; engine 65694 -> 65724; JS API 2679 unchanged.
Method: unchanged tools/production_loc.py, exact first-parent and staged tree
export via git archive / git checkout-index, same source scope and exclusions
including inline test removal. Staged tree 60e2246.
reference absent (retired R9c; last counted 65,417 at tree 498dd19).
…line

Select the native Linux provider with the current Ephemeral-AI-Lab#427 native configuration
name, keep Linux root admission and 501:20 command setup, and select FUSE
without inheriting macOS NFS overrides. Both platforms share the H20 API
connection/Noise/Hello/Store-install path. Update provider and SDK docs.
Docker remains the default; former local selectors/records remain refused.

Validation: warning-denying all-target Clippy, fmt, boundary guard and its 155
self-tests passed. Final macOS release (strip=none): smoke 48/48, PATH inherit
7/7 and fallback 4/4, no daemon/mount residue. SDK calls 39 and CLI 75/75 passed.
Sandbox 40 passed / 1 ignored; Mac Node targets execute 0 Linux-only tests.
API package invocation timed out at 90s during the unrelated 100k-file scale
fixture; prior targets passed and remaining 13 targets passed separately.
Linux sandbox --all-targets cross-check passed; API/Node cross-check blocked
by missing x86_64-linux-gnu-gcc. Real Linux qualification remains pending.
Fresh-context read-only review reported no concrete correctness findings.

Production LOC: 109803 -> 109814 (delta +11)
Scope: core 41400 unchanged; engine 65724 -> 65735; JS API 2679 unchanged.
Method: unchanged tools/production_loc.py (SHA256
7b540142dd835c4aaaaadbd754d456e824ac25fe3033c4328f9ec192c3ceacff),
first-parent tree 60e2246 versus staged
checkout-index export tree 5b32bf0;
same production classification and test/inline-test exclusions.
reference absent (retired R9c; last counted 65,417 at tree 498dd19).
DockerProvider and DockerInstance adapt the existing Docker Engine
client and container owner to SandboxProvider/SandboxInstance. Launch
keeps the one-attempt create, streamed private upload, Start and bounded
listener wait, and returns the acknowledged instance with the original
typed stage receipt on failure. Ordinary run streams output with bounded
windows and reports an exit only after one exact inspection; EOF, an
unknown exit, a failed sink or a failed inspection keeps the original
handle and delivered progress without polling, replay or cleanup.

Inspect now returns physical running/exit/PID facts for the shared port
while endpoint keeps its exact stopped observation, and a stopped
endpoint failure is constructed from that original HTTP response. The
shared launch options and provider trait signatures are unchanged, so
SandboxApi and the Node binding keep compiling; Docker deployment facts
stay in the provider constructor and the existing trait LocalRecord
reopen methods refuse Docker before effects. No record format, API path
or Node route changes in this slice.

Tests add Docker provider launch/version/stream/log/stop/delete,
upload/start/listener and Exec create/start/inspect custody, unknown
exit, failed sink, reopen/inspect, and stopped-response receipt coverage.

Production LOC: 109814 -> 110149 (delta +35); scope and counting method
unchanged with tools/production_loc.py at the committed tree; reference
absent (retired R9c; last counted 65,417 at tree 498dd19).
Upstream 73d974d (Docker API negotiation) makes every Docker client send
GET /version before its first business request. The sandbox crate adapted its
own fake Engine; the binding's fixture still spent a scripted response on the
probe and closed the listener, so native_discovery (3 failures) and
native_admission (2) failed in Linux runs, and two other discovery cases passed
without reaching the checks they name.

Answer /version on its own connection with {"ApiVersion":"1.54"} without
consuming a scripted response, invoking `before`, or recording a business
request. Each new Docker client probes, so the fixture stays listening while
scripted responses remain rather than serving one probe at the front.

Two cases that merely asserted `Failed` now check the request they actually
failed on: the ordinary 404 keeps its container GET and reports status 404 with
"page not found"; the refused delete keeps GET then DELETE and reports status
409. Assertions and request order for the other cases are unchanged. No product
source changes; ConnectionRefused is not reclassified as Busy.

Checks (macOS, both targets are Linux-only and list 0 tests here):
cargo +1.85.1 test --locked -p layerfs-node --no-run; clippy --locked -p
layerfs-node --all-targets -- -D warnings; fmt --all -- --check;
tools/check_product_boundary.py; a throwaway host probe driving this fixture
(two probes, two business requests) confirmed probes consume nothing and the
`before` callback fires once.

Production LOC: 109142 -> 109142 (delta +0). Scope: tools/production_loc.py
combined core+engine+api; only files under engine/crates/layerfs-node/tests/
change, which that counter excludes by construction. reference absent (retired
R9c; last counted 65,417 at tree 498dd19).
Test-only fixture change from the H25 branch: the binding's fake Engine now
answers the Docker version probe on its own connection. No product source
changes in the merged commit.

Production LOC: 109142 -> 109142 (delta +0). Scope: tools/production_loc.py
combined core+engine+api; merge changes only engine/crates/layerfs-node/tests/,
which that counter excludes. reference absent (retired R9c; last counted
65,417 at tree 498dd19).
…its (H24)

ProviderLaunchOptions gains the Docker variant (image, store_volume, port,
connect_host), so DockerProvider::new no longer accepts deployment arguments.
The unified LaunchRequest.labels provides container labels across all
providers.

SandboxProvider::reopen and reopen_stopped now accept &SandboxRecord, which
unifies LocalRecord and DockerRecord. DockerProvider and NativeProvider
validate and handle their corresponding record variants before effects, returning
Unsupported on foreign records. Separate reopen_docker and reopen_stopped_docker
methods are removed as the trait directly serves both runtime implementations.
SandboxInstance::record exposes the persisted deployment facts for reopening.

Minimal follow-up in layerfs-api allows create_provider_file to compile against
the new options variant without performing the H26 switch.

Production LOC: 110149 -> 110285 (delta +136). Scope: tools/production_loc.py
combined core+engine+api; engine layerfs-sandbox +125, layerfs-api +11, core
41400 and api 2679 unchanged. reference absent (retired R9c; last counted
65,417 at tree 498dd19).
Route Docker and native Sandbox creation, command execution, and lifecycle
closing exclusively through runtime providers in layerfs-api.

- SandboxApi: support Arc<dyn SandboxProvider> directly via with_provider;
  Docker callers route through DockerProvider::new(docker).
- Creation: add SandboxApi::create for ProviderCreate; unify startup and
  Store installation via create_provider_file.
- Execution: WorkspaceApi::exec delegates ordinary noninteractive command
  execution through SandboxInstance::run with streaming stdout/stderr buffers.
- Lifecycle: SandboxApi::close accepts ProviderSandbox and stops/deletes
  the instance via SandboxInstance::stop and delete.
- Examples and tests: sandbox_operations, sandbox_lifecycle, and
  sandbox_kernel_mount migrated to DockerProvider and ProviderSandbox.
- Legacy backward compatibility: retain SandboxApi::new(Docker),
  SandboxCreate, ManagedSandbox, and internal::split/exec as deprecated
  forwarders to ensure layerfs-node compiles without modification until H27.

Production LOC: 110285 -> 110441 (delta +156)
- engine: 66206 -> 66362 (delta +156)
  - layerfs-api: 2219 -> 2375 (delta +156)
- core: 41400 -> 41400 (delta 0)
- api: 2679 -> 2679 (delta 0)
- reference absent (retired R9c; last counted 65,417 at tree 498dd19)
Method: python3 tools/production_loc.py --root against parent 7922044 and committed tree.
… forwards (H27)

Unify layerfs-node behind DockerProvider + ProviderCreate/ProviderSandbox,
sharing the create_provider_file startup and installation pipeline with
the native provider, and remove the deprecated H26 compatibility forwarders
from layerfs-api.

- layerfs-node:
  - sandbox.rs: start_claimed routes Docker launches through DockerProvider
    and create_provider_file, using ProviderCreate with ProviderLaunchOptions::Docker;
    abandon/settled adapt to ProviderFailure while keeping identical refusal
    detail, settled criteria, and stop(1) cleanup semantics.
  - runtime.rs: Runtime::Docker wraps Box<dyn SandboxInstance>, aligning with
    Local; Runtime::run unifies on sandbox.run(...) with identical streaming,
    exit code observation, and absence of manual exec creation boilerplate.
  - discovery.rs: Recorded::Docker holds DockerProvider, ContainerId, and
    SandboxRecord; reopen and reopen_stopped route through DockerProvider;
    sandbox.record persistence formats and labels remain byte-for-byte identical.
- layerfs-api:
  - Delete deprecated H26 thin forwarders: SandboxApi::new(Docker),
    SandboxCreate, ManagedSandbox, SandboxFailure, create_file, close_managed,
    WorkspaceApi::from_managed, and internal::control_managed/split/exec.
  - Streamline SandboxApi to wrap Arc<dyn SandboxProvider> directly; prune
    unused SandboxCause::Create/Upload/Listener variants.
- Behavior preservation:
  - Exec streaming/exit codes, 5-second graceful sandbox.close timeout,
    1-second create abandon stop, sandbox.record schema, and typed refusals
    remain strictly preserved.

Production LOC: 110441 -> 110143 (delta -298)
- engine: 66362 -> 66064 (delta -298)
  - layerfs-api: 2375 -> 2074 (delta -301)
  - layerfs-node: 3720 -> 3723 (delta +3)
- core: 41400 -> 41400 (delta 0)
- api: 2679 -> 2679 (delta 0)
- reference absent (retired R9c; last counted 65,417 at tree 498dd19)
Method: python3 tools/production_loc.py --root against parent 70ec59e and committed tree.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant