From 41324ea9f7eef9e156f7f91e0d926a81bdb2b992 Mon Sep 17 00:00:00 2001 From: Lily Shen <115414357+lilyshen0722@users.noreply.github.com> Date: Thu, 15 Oct 2026 05:59:36 -0700 Subject: [PATCH 1/3] docs(adr-023): hosted turns move to the native runtime; tools run in an E2B sandbox Amendment ruled 2026-10-15 by Cofounder under Sam's delegation (Sam may overrule). Measured from a fresh account, the Durable Object hosted agent is fast and honest but has one tool (read_pod_context), no broker and no run records. A1: hosted turns run on nativeRuntimeService behind a per-install flag, the Durable Object kept as fallback (Stage 1, ~10-22). A2: tools run in an E2B microVM stopped every turn, egress allowlisted, credentials from the broker (Stage 2, ~10-29). A3: AgentRun per turn, a $2/user/day alarm, and the 30/200-turn plan line with tools. Scope note: depends on ADR-011's 2026-10-15 lift of the cloud sandbox pause (#2205). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015NDyNbwmCco62PAAviSL1k Co-authored-by: folio Agent-Model: claude/claude-haiku-5-5/high --- docs/adr/ADR-023-agent-runtime-substrate.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/docs/adr/ADR-023-agent-runtime-substrate.md b/docs/adr/ADR-023-agent-runtime-substrate.md index 8c6794e206..7a29f2aa88 100644 --- a/docs/adr/ADR-023-agent-runtime-substrate.md +++ b/docs/adr/ADR-023-agent-runtime-substrate.md @@ -12,6 +12,24 @@ - **Amends, if accepted:** ADR-021 Part A.1 (the `agent-runtime` build spec). Everything else in ADR-021 — pi as the turn engine, staged OpenClaw retirement, the M1–M4 ladder, the fallback discipline — stands unchanged. - **Does NOT touch:** the shell (Express, MongoDB, PostgreSQL, Socket.io, the v2 frontend). See "What this is not" below, which is the most important section in this document. +## Amendment, 2026-10-15: hosted turns run on the native runtime, and tools run in a sandbox + +- **Status:** **Accepted** 2026-10-15, ruled by Cofounder under Sam's delegation; Sam may overrule. Acknowledged unknowns this does not settle: per-user usage was measured on Scout as a proxy, because hosted agents write no run records yet (re-measured two weeks after Stage 2); how hosted turns behave under a spot-node reclaim; and E2B as an outside dependency. +- **Scope-boundary note:** [ADR-011](ADR-011-shell-first-pre-gtm.md) paused the Tier 2 cloud sandbox. Its 2026-10-15 status update lifts that pause for the out-of-the-box goal only, and A2 below depends on that lift. [ADR-026](ADR-026-local-agent-daemon.md) (local execution) is unchanged. + +**Why.** W2 shipped as D1 selected: one Durable Object per hosted agent. Measured from a fresh account on 2026-10-15, it is fast (agent ready 26 s after signup, first useful reply about 10 s later) and honest, but its only tool reads pod context. It has no web, no code, no files, no connectors and no run records. It answered one of four basic asks (summarize the pod) and declined web lookup, running a script and reading a repo. Making it useful needs the connector broker, an `AgentRun` per turn, the per-user daily ceiling ([ADR-022](ADR-022-persona-colleagues.md) D5), the tool gate and the connector eligibility rule. `nativeRuntimeService` already has all five; the Durable Object has none. + +**A1. Hosted turns run on `nativeRuntimeService`** (Stage 1, about 2026-10-22). A `runtimeType: 'hosted'` install answers through the native loop, with DeepSeek through LiteLLM, so the platform fallbacks apply. It sits behind a per-install flag, and the Durable Object stays as the fallback until the native path reaches parity, then retires. This narrows D2: hosted agents move instead of the runtime being strangled in place, because the one-loop rule (CLAUDE.md design rule 6) outweighs keeping a second loop that would need the same five features. + +**A2. Tools run in an E2B sandbox** (Stage 2, about 2026-10-29). Each turn gets one Firecracker microVM, stopped when the turn ends so there is no idle billing. Outbound traffic is limited to listed hosts, and credentials come from the broker, never the prompt. Web lookup goes through the same egress path or a search API, whichever costs less per turn. +- **Rejected:** the Cloudflare Sandbox SDK runs inside Workers, so the backend would need a proxy, and a sandbox left idle bills memory for 10 minutes. gVisor on our own cluster needs warm slots we would run, with no TypeScript SDK. Daytona shares the host kernel by default and has been closed source since June 2026. +- **The exit:** E2B's runtime is Apache-2.0 and can be self-hosted. + +**A3. Cost is recorded and bounded.** Every hosted turn writes an `AgentRun`, and an alarm fires when any one user passes $2 in a day. The plan line ships with tools: 30 turns a day for free or individual, 200 for team or paid. DeepSeek stays the default, and a premium model is a paid add-on. +- **Measured on 2026-10-15,** on DeepSeek V4.1-Flash at $0.30/$1.20 per million tokens, with Scout runs averaging 5,468 tokens: about $1–2.30 per user a month at the 90th percentile of use, and at most $6.70 at the 30-turn cap. +- **The 200-turn cap is the tail,** at $14–44. +- **The model sets the price, not the sandbox:** about $0.0004 per 20-second turn on E2B. + ## Context Four things converged on 2026-08-15. From ab1afc430e581c9fed34a0e684321f310d2cdc38 Mon Sep 17 00:00:00 2001 From: Lily Shen <115414357+lilyshen0722@users.noreply.github.com> Date: Tue, 27 Oct 2026 17:40:21 -0700 Subject: [PATCH 2/3] docs(adr-023): A1 and A2 match main; stage dates are the shipped ones A2 claimed Firecracker microVMs, listed-host egress, broker credentials and a web-lookup path. Main has none of these: run_code is the only tool that runs in E2B, with no internet, no public inbound and no environment variables, at most 5 runs per turn, 30 s and 20,000 characters each, and Python or JavaScript only (backend/services/turnSandboxService.ts, nativeRuntimeService.ts:1042). Stage dates: A1 shipped as TASK-261 (#2224) and A2 as TASK-266 (#2232), both merged 2026-10-15. Co-Authored-By: Claude Haiku 5.5 Co-authored-by: folio Agent-Model: claude/claude-haiku-5-5/high --- docs/adr/ADR-023-agent-runtime-substrate.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/adr/ADR-023-agent-runtime-substrate.md b/docs/adr/ADR-023-agent-runtime-substrate.md index 7a29f2aa88..0881499736 100644 --- a/docs/adr/ADR-023-agent-runtime-substrate.md +++ b/docs/adr/ADR-023-agent-runtime-substrate.md @@ -19,9 +19,9 @@ **Why.** W2 shipped as D1 selected: one Durable Object per hosted agent. Measured from a fresh account on 2026-10-15, it is fast (agent ready 26 s after signup, first useful reply about 10 s later) and honest, but its only tool reads pod context. It has no web, no code, no files, no connectors and no run records. It answered one of four basic asks (summarize the pod) and declined web lookup, running a script and reading a repo. Making it useful needs the connector broker, an `AgentRun` per turn, the per-user daily ceiling ([ADR-022](ADR-022-persona-colleagues.md) D5), the tool gate and the connector eligibility rule. `nativeRuntimeService` already has all five; the Durable Object has none. -**A1. Hosted turns run on `nativeRuntimeService`** (Stage 1, about 2026-10-22). A `runtimeType: 'hosted'` install answers through the native loop, with DeepSeek through LiteLLM, so the platform fallbacks apply. It sits behind a per-install flag, and the Durable Object stays as the fallback until the native path reaches parity, then retires. This narrows D2: hosted agents move instead of the runtime being strangled in place, because the one-loop rule (CLAUDE.md design rule 6) outweighs keeping a second loop that would need the same five features. +**A1. Hosted turns run on `nativeRuntimeService`** (Stage 1, shipped 2026-10-15 as TASK-261, #2224). A `runtimeType: 'hosted'` install answers through the native loop, with DeepSeek through LiteLLM, so the platform fallbacks apply. It sits behind a per-install flag, and the Durable Object stays as the fallback until the native path reaches parity, then retires. This narrows D2: hosted agents move instead of the runtime being strangled in place, because the one-loop rule (CLAUDE.md design rule 6) outweighs keeping a second loop that would need the same five features. -**A2. Tools run in an E2B sandbox** (Stage 2, about 2026-10-29). Each turn gets one Firecracker microVM, stopped when the turn ends so there is no idle billing. Outbound traffic is limited to listed hosts, and credentials come from the broker, never the prompt. Web lookup goes through the same egress path or a search API, whichever costs less per turn. +**A2. Code runs in an E2B sandbox** (Stage 2, shipped 2026-10-15 as TASK-266, #2232). `commonly_run_code` is the only native tool that runs in E2B. Each native turn gets one sandbox, created on its first run and killed when the turn ends, with E2B's three-minute lifetime as the backstop. The sandbox has no internet, no public inbound traffic and no environment variables, so no secret is inside it. A turn gets at most 5 runs, each capped at 30 seconds and 20,000 characters of code, in Python or JavaScript. - **Rejected:** the Cloudflare Sandbox SDK runs inside Workers, so the backend would need a proxy, and a sandbox left idle bills memory for 10 minutes. gVisor on our own cluster needs warm slots we would run, with no TypeScript SDK. Daytona shares the host kernel by default and has been closed source since June 2026. - **The exit:** E2B's runtime is Apache-2.0 and can be self-hosted. From 38d00a4dabd4b4d5eb7bdf61a47ac918f0151eb1 Mon Sep 17 00:00:00 2001 From: Lily Shen <115414357+lilyshen0722@users.noreply.github.com> Date: Tue, 27 Oct 2026 17:45:45 -0700 Subject: [PATCH 3/3] docs(adr-023): amendment heading says code runs in a sandbox, not tools The heading contradicted A2, where only commonly_run_code runs in E2B. Co-Authored-By: Claude Haiku 5.5 Co-authored-by: folio Agent-Model: claude/claude-haiku-5-5/high --- docs/adr/ADR-023-agent-runtime-substrate.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/adr/ADR-023-agent-runtime-substrate.md b/docs/adr/ADR-023-agent-runtime-substrate.md index 0881499736..92d1e69618 100644 --- a/docs/adr/ADR-023-agent-runtime-substrate.md +++ b/docs/adr/ADR-023-agent-runtime-substrate.md @@ -12,7 +12,7 @@ - **Amends, if accepted:** ADR-021 Part A.1 (the `agent-runtime` build spec). Everything else in ADR-021 — pi as the turn engine, staged OpenClaw retirement, the M1–M4 ladder, the fallback discipline — stands unchanged. - **Does NOT touch:** the shell (Express, MongoDB, PostgreSQL, Socket.io, the v2 frontend). See "What this is not" below, which is the most important section in this document. -## Amendment, 2026-10-15: hosted turns run on the native runtime, and tools run in a sandbox +## Amendment, 2026-10-15: hosted turns run on the native runtime, and code runs in a sandbox - **Status:** **Accepted** 2026-10-15, ruled by Cofounder under Sam's delegation; Sam may overrule. Acknowledged unknowns this does not settle: per-user usage was measured on Scout as a proxy, because hosted agents write no run records yet (re-measured two weeks after Stage 2); how hosted turns behave under a spot-node reclaim; and E2B as an outside dependency. - **Scope-boundary note:** [ADR-011](ADR-011-shell-first-pre-gtm.md) paused the Tier 2 cloud sandbox. Its 2026-10-15 status update lifts that pause for the out-of-the-box goal only, and A2 below depends on that lift. [ADR-026](ADR-026-local-agent-daemon.md) (local execution) is unchanged.