Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
79190bf
feat(persistence): Windows-native Store writer lane, volume guard and…
NelsonLongxiang Oct 25, 2026
0becc9b
feat(sdk): Windows-native project Init, overlay identity and the Stor…
NelsonLongxiang Oct 25, 2026
94bdb9f
docs: record the Windows-native Store route (owner direction 2026-10-25)
NelsonLongxiang Oct 25, 2026
ee6c34f
test(windows): run the Store suites natively on Windows; lane-covers …
NelsonLongxiang Oct 25, 2026
843d22f
test(sdk): the full Store method matrix on the native route
NelsonLongxiang Oct 25, 2026
ada6f39
test(sdk): cross-process concurrent Store writers on the native route
NelsonLongxiang Oct 25, 2026
5a41b5f
test(windows): abandoned-lane acquisition and divergent merge readback
NelsonLongxiang Oct 25, 2026
3d90143
fix(persistence): lane-hold open without Linux self-deadlock; dead-la…
NelsonLongxiang Oct 25, 2026
27ace14
fix(persistence): the close-time checkpoint runs under the writer lane
NelsonLongxiang Oct 25, 2026
27f6cf8
fix(store-service): the service close coordinates its Store teardown
NelsonLongxiang Oct 25, 2026
636a641
fix(windows): versioned-stable lane and discovery mutex names (K3 F1)
NelsonLongxiang Oct 26, 2026
9ca6b3c
docs(architecture): the lane's open/teardown holds and stable naming …
NelsonLongxiang Oct 26, 2026
3f68b36
docs(readme): align the user-facing Store-route platform wording
NelsonLongxiang Oct 26, 2026
2a4cf66
fix(store-service): Windows overlay last-owner deletion through the a…
NelsonLongxiang Oct 27, 2026
64035fe
docs(direct-store): the deletion-parity line replaces the stale limit…
NelsonLongxiang Oct 27, 2026
3d3272c
feat(overlay): Windows allocation reads the real on-disk footprint
NelsonLongxiang Oct 27, 2026
e39f4ac
feat(project): the alias check walks identities, Linux's own algorithm
NelsonLongxiang Oct 27, 2026
6b63a64
docs(direct-store): the post-merge observation checklist in §6
NelsonLongxiang Oct 27, 2026
f98734c
merge: upstream main (1.1.0 restructure) into windows-native-store
NelsonLongxiang Oct 28, 2026
1e4f240
feat(project): Windows host names travel as lossless WTF-8
NelsonLongxiang Oct 28, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ simplification.
| Cluster one | Core Content/Storage/History/Telemetry algorithms; Engine Persistence and Project Init | [Cluster-one handbook](docs/general/cluster_one_handbook.md), [CAS/CDC/delta handbook](docs/general/cas_cdc_deltaencoding_handbook.md) |
| Cluster two | Engine Workspace/FUSE/daemon/Sandbox; API execution and Commit/runtime integration | [Current design index](core/docs/issues/303/README.md) and its seven primary contracts |
| Legacy reference | v0.1.6, retired from `main` by R9c; read at the recovery tag only | [R9c record](core/docs/issues/307/R9C-COMPLETION-20261010.md) |
| Windows-native Store | Store engine and the SDK's in-process Store face on native Windows (owner, 2026-10-25); FUSE, daemon and Workspace stay Linux-only | [Direct Store contract](core/docs/issues/352/DIRECT_STORE.md) |

**The LayerFS UI is the dsh LayerFS plugin** (owner, 2026-10-19): the LayerFS tab in
DeepSeek Harness. It lives in its own repository,
Expand Down Expand Up @@ -154,6 +155,20 @@ remains a separate MEMORY/OFF/EXCLUSIVE database file.
native handles and project discovery. This is independent of the macOS
adapter and changes neither the product SDK nor the daemon. A lost reply
is never replayed, and an uncertain Init retains its Linux home and volume.
- **Windows-native Store route (owner direction 2026-10-25).** The Store
engine (SQLite, the cross-process writer lane, the kernel guard) and the
SDK's in-process Store face (fs, branch, project, operator) also run
natively on Windows: local NTFS volumes only, a named kernel mutex as the
cross-process writer lane, real volume-serial + file-index identity for
acquisition rows, and bundled SQLite, each in audited FFI modules
(core/AGENTS.md). No WSL2 and no daemon are involved. This supersedes the
Linux-only Store sentence above for this route only; Workspace, FUSE, Exec
and daemon calls still refuse `Unsupported` on Windows, and the dsh macOS
and WSL2 exceptions are unchanged. Recorded limitations: Windows Init has
no hard-link membership and no backing-inside-source alias check, carries
non-UTF-8 names lossily and synthesizes the mode from the read-only
attribute; overlay allocation is the logical length. See
[the Windows-native route](core/docs/issues/352/DIRECT_STORE.md).
- **Plugin-neutral LayerFS (owner, 2026-10-24).** LayerFS is agnostic to dsh
and every other plugin. The macOS and Windows worker routes above are open
to any integration: `@layerfs/runtime` starts the worker on those platforms
Expand Down
6 changes: 6 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 11 additions & 1 deletion api/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,12 @@ Store calls run in the calling process on Linux only. On macOS and Windows they
go through the LayerFS home's worker, which `@layerfs/runtime` starts; see
[where calls run](docs/agent.md#where-calls-run) and
[platforms and the worker](docs/runtime.md#platforms-and-the-worker).
On Windows there is also the native direct route (owner direction
2026-10-25): the binding built from this tree opens a local NTFS Store
itself — fs, branch, project and operator calls in this process, no
worker and no daemon; workspace, exec and daemon calls refuse
`Unsupported`. See
[the Windows-native route](../engine/docs/issues/352/DIRECT_STORE.md).

## The folder

Expand Down Expand Up @@ -75,7 +81,11 @@ addon. It uses Cargo +1.85.1, `--offline --locked --release`, and the repository
ARM64 inputs. `-- --debug` selects dev. The artifact is atomically renamed into
place, avoiding stale macOS executable-page signature state. `CARGO_TARGET_DIR`
must stay inside this worktree; `LAYERFS_NATIVE_OUTPUT` can select a separate
platform artifact. Keep Linux addons separate from the host addon.
platform artifact. Keep Linux addons separate from the host addon. On
Windows the script refuses to run; build the addon directly with
`cargo build --manifest-path engine/Cargo.toml -p layerfs-node` (locked,
`CARGO_TARGET_DIR` inside this worktree) and rename
`target/<profile>/layerfs_node.dll` to the addon path.

At runtime, the Docker Engine server selected by `LAYERFS_DOCKER` must speak
Engine API v1.48 (Docker 28.0) or newer. Before its first request the SDK reads
Expand Down
8 changes: 8 additions & 0 deletions api/agent/test/init_refusals.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,14 @@ import {
import { tmpdir } from "node:os";
import { dirname, join } from "node:path";

if (process.platform === "win32") {
// Every refusal this test drives is a unix mode-bit or filesystem-type
// contract (setgid, setuid, sticky, overlayfs, FIFO); Windows carries
// none of them. The Windows Store-face lifecycle is store_lifecycle.mjs.
console.log("init_refusals: unix-only contracts, not run on Windows");
process.exit(0);
}

const OVERLAYFS = 0x794c7630;
const work = mkdtempSync(join(tmpdir(), "layerfs-init-refusals-"));
const home = join(work, "home");
Expand Down
20 changes: 12 additions & 8 deletions api/agent/test/reinit.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -39,14 +39,18 @@ try {
};
const layerfs = client();

// A set-group-ID source directory has a mode LayerFS cannot store.
chmodSync(source, 0o2755);
await assert.rejects(layerfs.project.init(source), {
code: "InvalidInput",
detail: { reason: "Mode", path: source, mode: "2755" },
});
assert.ok(!existsSync(join(work, "home", "website")), "a refused Init leaves no entry");
chmodSync(source, 0o755);
// A set-group-ID source directory has a mode LayerFS cannot store. The
// bit does not exist on Windows, whose chmod maps only the read-only
// attribute; the refusal contract is checked on unix platforms.
if (process.platform !== "win32") {
chmodSync(source, 0o2755);
await assert.rejects(layerfs.project.init(source), {
code: "InvalidInput",
detail: { reason: "Mode", path: source, mode: "2755" },
});
assert.ok(!existsSync(join(work, "home", "website")), "a refused Init leaves no entry");
chmodSync(source, 0o755);
}

const first = await layerfs.project.init(source);
assert.equal(first.kind, "Initialized");
Expand Down
92 changes: 92 additions & 0 deletions api/agent/test/store_concurrent.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
// Cross-process Store writers on the real native binding (owner direction
// 2026-10-25, route 2): the parent initializes a project and forks one
// branch, then four re-exec'd child processes write distinct files to that
// branch concurrently, each through its own client of the same Store - the
// direct route's cross-process writer lane and the discovery record guard
// serialize them; every write must land. The parent reads all four files
// back and counts the Commits. No Docker, no daemon, no WSL2; on Linux the
// Store sits on the E2E launcher's volume (LAYERFS_STORE_PATH):
// node sdk/js/test/store_concurrent.mjs # parent run
// node sdk/js/test/store_concurrent.mjs --child <home> <store> <name>
import assert from "node:assert/strict";
import { spawn } from "node:child_process";
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { dirname, join } from "node:path";

// One child writer: open the project's Store and write its own file once.
if (process.argv[2] === "--child") {
const [, , , home, store, name] = process.argv;
process.env.LAYERFS_HOME = home;
const { LayerFS } = await import("@layerfs/agent");
process.env.LAYERFS_STORE_PATH = store;
const writer = await LayerFS.open("concurrent");
const body = `written by ${name} in pid ${process.pid}\n`;
const done = await writer.fs.write("load", `/${name}.md`, body, `write ${name}`);
await writer.close();
assert.equal(done.kind, "Committed");
process.exit(0);
}

const WRITERS = 4;
const work = mkdtempSync(join(tmpdir(), "layerfs-concurrent-"));
const volume = process.env.LAYERFS_STORE_PATH
? join(dirname(dirname(process.env.LAYERFS_STORE_PATH)), `concurrent-${process.pid}`)
: join(work, "volume");
const store = join(volume, "global", "store.sqlite");
const home = join(work, "home");
const retryRemove = (path) => {
for (let attempt = 0; attempt < 5; attempt++) {
try { rmSync(path, { recursive: true, force: true }); return; } catch { }
Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, 200);
}
};
try {
const source = join(work, "source");
mkdirSync(source);
writeFileSync(join(source, "README.md"), "concurrent writers\n");
process.env.LAYERFS_HOME = home;
mkdirSync(join(volume, "global"), { recursive: true });
const { LayerFS } = await import("@layerfs/agent");
process.env.LAYERFS_STORE_PATH = store;
const parent = await LayerFS.init("concurrent", source);
await parent.branch.fork("load");
const base = (await parent.branch.list_commits("load")).items.length;

const children = [];
for (let n = 1; n <= WRITERS; n++) {
children.push(spawn(process.execPath, [import.meta.url.pathname ?? process.argv[1], "--child", home, store, `w${n}`], {
stdio: ["ignore", "pipe", "pipe"],
env: { ...process.env, LAYERFS_STORE_PATH: store },
}));
}
const reports = await Promise.all(children.map(async (child, index) => {
const [code, output] = await new Promise((resolve) => {
let captured = "";
child.stdout.on("data", (chunk) => { captured += chunk; });
child.stderr.on("data", (chunk) => { captured += chunk; });
const timer = setTimeout(() => {
child.kill();
resolve([-1, `${captured} (killed: 30s ceiling)`]);
}, 30_000);
child.on("exit", (exitCode) => { clearTimeout(timer); resolve([exitCode, captured]); });
});
return { index, code, output };
}));
for (const { index, code, output } of reports) {
assert.equal(code, 0, `writer w${index + 1} failed (${code}): ${output.slice(0, 400)}`);
}

// Every writer's file is committed and readable; the branch carries them all.
for (let n = 1; n <= WRITERS; n++) {
const read = await parent.fs.read("load", `/w${n}.md`);
assert.ok(read.text.startsWith(`written by w${n} in pid `), read.text);
}
const commits = await parent.branch.list_commits("load", 50);
assert.equal(commits.items.length, base + WRITERS, `${commits.items.length} commits, expected ${base + WRITERS}`);
await parent.close();
console.log(`store_concurrent: ok (${WRITERS} writers, ${commits.items.length} commits)`);
} finally {
retryRemove(work);
retryRemove(volume);
}
48 changes: 48 additions & 0 deletions api/agent/test/store_lifecycle.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
// The Store-face lifecycle on the real native binding (owner direction
// 2026-10-25, route 2): project init, fork, a one-Commit fs.write, reading
// back on the new branch and on main, stat, the Commit listing and close —
// all in this process through the SDK's direct Store route. No Docker, no
// daemon, no WSL2. The Store's volume is LAYERFS_STORE_PATH's parent when
// the E2E launcher provides one (Linux); otherwise this machine's temporary
// volume, which the Windows route guards as NTFS:
// node sdk/js/test/store_lifecycle.mjs
import assert from "node:assert/strict";
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { dirname, join } from "node:path";

const work = mkdtempSync(join(tmpdir(), "layerfs-lifecycle-"));
const volume = process.env.LAYERFS_STORE_PATH
? join(dirname(dirname(process.env.LAYERFS_STORE_PATH)), `lifecycle-${process.pid}`)
: join(work, "volume");
const store = join(volume, "global", "store.sqlite");
try {
const source = join(work, "source");
mkdirSync(source);
writeFileSync(join(source, "a.txt"), "hello windows-native store\n");
process.env.LAYERFS_HOME = join(work, "home");
mkdirSync(join(volume, "global"), { recursive: true });
const { LayerFS } = await import("@layerfs/agent");
process.env.LAYERFS_STORE_PATH = store;
const layerfs = await LayerFS.init("lifecycle", source);

assert.equal(layerfs.created.kind, "Initialized");
await layerfs.branch.fork("dev");

const body = "written by the native Store face\n";
assert.equal((await layerfs.fs.write("dev", "/windows.md", body, "lifecycle write")).kind, "Committed");

assert.equal((await layerfs.fs.read("dev", "/windows.md")).text, body);
assert.equal((await layerfs.fs.read("main", "/a.txt")).text, "hello windows-native store\n");
assert.equal((await layerfs.fs.stat("dev", "/windows.md")).size, Buffer.byteLength(body));

const commits = await layerfs.branch.list_commits("dev");
const records = commits.records ?? commits.items ?? commits;
assert.ok(Array.isArray(records) && records.length >= 1, JSON.stringify(commits).slice(0, 200));

await layerfs.close();
console.log(`store_lifecycle: ok (${records.length} commit listed)`);
} finally {
rmSync(work, { recursive: true, force: true });
rmSync(volume, { recursive: true, force: true });
}
Loading