From 1447be522e9461add519c17bd1ac72f290e27245 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Mon, 12 Oct 2026 19:00:58 +0800 Subject: [PATCH 01/69] docs(rfc): define consistent research checkpoint capture Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- .../hierarchical-agent-stride-control-v0.md | 39 +++++++++++++++++++ ...rarchical-agent-stride-control-v0.zh-CN.md | 28 ++++++++++++- 2 files changed, 66 insertions(+), 1 deletion(-) diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md index 5b12d0d130..f230066218 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md @@ -655,6 +655,41 @@ Record the actual boundary and any skew: - remaining total budget, accepted submission history and next feedback opportunity under the original protocol. +Capture belongs at the existing **settled work-Turn boundary**, before the next +work admission. A time-based request waits for that boundary and records its +actual elapsed budget; it cannot interrupt an arbitrary tool operation and +label the resulting files consistent. The runner/Host owns this barrier, while +each existing participant owns its snapshot: + +1. Finish owed Turn settlement and reconcile outstanding tool, child-work and + evaluator effects by their original identities. Unresolved effects make the + point unqualified; copying their journals does not resolve them. +2. Capture the native completed-turn cursor, canonical research-state revision, + working-artifact manifest and evaluator opportunity state at that barrier. + Use the existing SQLite snapshot owner where applicable, not a live database + file copy. Keep code and incumbent/result bindings independently verifiable. +3. Recheck each participant's revision/cursor and artifact digests before + publishing one immutable verification unit. Any movement, missing member or + interrupted capture leaves an incomplete unit, never a usable point. This + establishes the declared barrier's consistency, not a cross-owner transaction. +4. Restore into an isolated destination, verify its contents, then use the + existing owners to admit fresh sessions and research work. Restore alone + cannot start a model, claim work, register a timer or submit a candidate. + +The Host adapter must qualify native history reconstruction at its pinned +version, including compaction and paginated storage. For example, Codex's +[native fork protocol](https://learn.chatgpt.com/docs/app-server) +can bound a branch by a completed turn. An exported file still needs the native +store's import/readback path; an arbitrary file path or successful log parse is +insufficient. Compare the reconstructed prefixes and excluded later history, +not just returned session identifiers. Keep provider-specific migration and +storage handling outside the generic consistency decision. + +Historical conversation truncation cannot reconstruct earlier code, research +state or feedback consumption from their terminal versions. Without a qualified +contemporaneous capture of those participants, such a probe establishes Host +feasibility only and must not enter a same-state intervention comparison. + An inert verification unit grants no execution authority. The destination uses fresh experiment/Host identities and the existing Goal/session/state admission paths. Historical leases, quota, timers, pending effects and submission tokens @@ -673,6 +708,10 @@ continuation and evidence-conditioned scope review, with equal continuation budgets and resources. Change one control factor; preserve feedback opportunity counts and cooldown timing through the evaluator owner. Label session restart, summary replacement, source upgrades or policy bundles as additional factors. +The evaluator carries the remaining cooldown and consumed opportunities into +each newly authorized branch; a fresh identity must not create an extra immediate +submission. Wall-clock waiting between capture and admission does not replenish +the checkpoint's opportunity allowance or extend the continuation budget. Keep all branches and invalid/negative results; compare adopted native outcomes, information gained and full continuation cost. The result qualifies the scoped intervention, not all LoopX behavior or an optimal universal stride. diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md index 700d47dae6..c218d8dda3 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md @@ -558,6 +558,30 @@ Host 会话 owner 和 artifact provider；本 RFC 定义研究实验的一致性 - source/model/tool/evaluator pins 与已消费的 solver-visible feedback； - 剩余总预算、已受理提交历史及原协议下的下一次反馈机会。 +捕获接在既有**有效工作 Turn 已结算、下一次工作尚未准入**的边界。按时间请求 +捕获时，等待该边界并记录实际已用预算；不能任意打断工具操作，再把所得文件 +称为一致状态。Runner/Host 持有这道 barrier，各既有参与 owner 分别负责快照： + +1. 完成欠账的 Turn 结算，并按原身份 reconcile 未决工具、子任务和 evaluator + 效果。未决效果使研究点尚未资格化；复制 journal 不等于解决效果。 +2. 在 barrier 记录原生已完成 turn cursor、canonical 研究状态 revision、工作 + 产物 manifest 和 evaluator 机会状态。适用时复用既有 SQLite snapshot owner， + 不复制活数据库文件；代码与 incumbent/result 绑定分别可验证。 +3. 发布一份不可变 verification unit 前，复查各参与者 revision/cursor 和产物 + digest。来源移动、缺失成员或中断捕获都留下 incomplete unit，不能成为可用 + 研究点；这只证明所声明 barrier 的一致性，不宣称跨 owner 事务。 +4. 恢复到隔离目标、验证内容，再由既有 owner 准入新 session 和研究工作。 + Restore 本身不能启动模型、claim 工作、注册 timer 或提交候选。 + +Host adapter 须在固定版本验证原生历史重建，包括 compaction 和分页存储。 +例如 Codex 的[原生 fork 协议](https://learn.chatgpt.com/docs/app-server) +可按已完成 turn 限定分支；导出文件仍须通过原生 store 的 import/readback 路径， +任意文件路径或日志解析成功并不充分。比较重建的历史前缀及被排除的后续历史， +不只检查返回了 session id。Provider 特有迁移与存储处理不进入通用一致性决策。 + +截短历史会话不能从终局版本重建当时的代码、研究状态或反馈消费。没有其他参与者 +同时捕获且已资格化的状态时，这类探针只证明 Host 可行性，不能进入同状态干预比较。 + inert verification unit 不授予执行权。目标侧使用新的实验/Host 身份，通过既有 Goal/session/state admission 进入。历史租约、quota、timer、pending effect 和 submission token 不能变成目标侧权限；未知效果先按原身份 reconcile，再决定 @@ -571,7 +595,9 @@ submission token 不能变成目标侧权限；未知效果先按原身份 recon 研究消融从**同一份**已资格化研究点分成原策略接续与证据驱动范围复盘，使用 等额接续预算与资源，只改变一个控制因素；通过 evaluator owner 保留反馈机会数 与冷却时机。Session 重启、摘要替换、版本升级或策略组合属于额外变量。保留 -所有分支及无效/负结果，比较实际采用的原生成绩、新信息与完整接续成本。结果 +所有分支及无效/负结果。Evaluator 向每个新获授权分支传递剩余冷却和已消费机会， +不能因新身份多得一次立即提交；捕获到准入之间的墙钟等待不会补充 checkpoint +的机会额度，也不会延长接续预算。比较实际采用的原生成绩、新信息与完整接续成本。结果 只资格化这项有范围干预，不证明整个 LoopX 或一个通用最优 stride。 ## 8. 测量模型 From b1fbcc0687ff489e0189de8c8be39bc9eba37a36 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Mon, 12 Oct 2026 20:06:49 +0800 Subject: [PATCH 02/69] feat(runtime): capture inert artifacts after settled worker wakes Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/runtime/checkpoint.py | 100 ++++++++ benchmark/runtime/harbor.py | 9 + benchmark/runtime/sforge.py | 12 +- benchmark/runtime/worker.py | 4 + loopx/control_plane/configuration_backup.ts | 15 +- .../control_plane/effect_runtime_handlers.ts | 1 + loopx/control_plane/effect_runtime_io.ts | 15 +- .../runtime/checkpoint_artifacts.ts | 228 ++++++++++++++++++ 8 files changed, 370 insertions(+), 14 deletions(-) create mode 100644 benchmark/runtime/checkpoint.py create mode 100644 loopx/control_plane/runtime/checkpoint_artifacts.ts diff --git a/benchmark/runtime/checkpoint.py b/benchmark/runtime/checkpoint.py new file mode 100644 index 0000000000..e74a486fb4 --- /dev/null +++ b/benchmark/runtime/checkpoint.py @@ -0,0 +1,100 @@ +"""Opt-in inert artifact capture after the existing benchmark wake has settled. + +The TS backup/artifact owner verifies source stability and materialized bytes. +Python adapts the owned wake, source admission and explicit operator file list; +it never restores claims, starts a model or certifies a complete research point. +""" +from __future__ import annotations + +import hashlib +import json +import math +import time +from pathlib import Path, PurePosixPath +from typing import Any + +from loopx.control_plane.effect_runtime import effect_runtime_result +from loopx.control_plane.quota.settlement import read_heartbeat_settlement + + +def normalize_checkpoint_request_path(value: str) -> str: + """An explicit guest path; no ambient host variable enables capture.""" + if (not isinstance(value, str) or not value or + not PurePosixPath(value).is_absolute() or str(PurePosixPath(value)) != value or + ".." in PurePosixPath(value).parts): + raise ValueError("checkpoint_request_path must be a normalized absolute guest file path") + return value + + +def capture_after_wake(env: dict[str, str], receipt: dict[str, Any]) -> None: + request_path = env.get("LOOPX_CHECKPOINT_REQUEST") + if not request_path: + return # Feature-off path performs no additional read or write. + receipt["checkpoint"] = {"captured": False, "research_point_qualified": False} + if (not receipt.get("ok") or receipt.get("stage") != "execute" or + receipt.get("mode") not in {"heartbeat", "turn"} or receipt.get("budget_exhausted")): + receipt["checkpoint"]["reason"] = "wake_not_successful_work" + return + started = time.monotonic() + try: + phase_deadline = env.get("LOOPX_PHASE_DEADLINE_EPOCH") + deadline = time.time() + 30 + if phase_deadline is not None: + original_deadline = float(phase_deadline) + if not math.isfinite(original_deadline): + raise ValueError("checkpoint requires a finite original phase deadline") + deadline = min(deadline, original_deadline) + if deadline <= time.time(): + receipt["checkpoint"]["reason"] = "capture_window_closed" + return + raw = Path(request_path).read_bytes() + request = json.loads(raw) + if not isinstance(request, dict) or set(request) != {"destination", "components", "point_metadata"}: + raise ValueError("checkpoint request requires destination, components and point_metadata") + # This is a private evidence artifact below the runner's log owner. + destination = Path(request["destination"]) + root = Path(env["LOOPX_WAKE_LOG_DIR"]).parent / "checkpoints" + if not destination.is_absolute() or destination != destination.resolve() or destination.parent != root: + raise ValueError("checkpoint destination must be a normalized direct child of this trial's checkpoints directory") + if destination.exists(): + receipt["checkpoint"]["reason"] = "destination_already_present_no_replay" + return + observed = read_heartbeat_settlement( + Path(env["LOOPX_RUNTIME_ROOT"]), goal_id=env["LOOPX_GOAL_ID"], + agent_id=env["LOOPX_AGENT_ID"], todo_id=None, + turn_instance_id=receipt["turn_id"], resolve_original_binding=True, + registry_path=Path(env["LOOPX_REGISTRY"]), + ) + if observed is None or (observed.progress or {}).get("state") != "settled": + receipt["checkpoint"]["reason"] = "owned_turn_not_settled" + return + identity = observed.identity.value + if identity is None: + raise ValueError("checkpoint settlement identity is unavailable") + # Only this returned typed owner decision admits the capture boundary. + # Point metadata remains evidence, never authority or scientific validity. + metadata = { + "operator_metadata": request["point_metadata"], + "wake_turn_id": receipt["turn_id"], + "settlement_effect_id": identity.effect_id, + "session": receipt.get("session"), + "request_sha256": hashlib.sha256(raw).hexdigest(), + } + probe = effect_runtime_result("runtime.checkpoint_artifacts", { + "action": "probe", "components": request["components"], + "deadline_epoch_ms": deadline * 1000, + }, timeout=max(.1, deadline - time.time()), retry_safe=False) + destination.parent.mkdir(parents=True, exist_ok=True) + captured = effect_runtime_result("runtime.checkpoint_artifacts", { + "action": "capture", "destination": str(destination), + "components": request["components"], "point_metadata": metadata, + "expected_source_sha256": probe["source_sha256"], + "deadline_epoch_ms": deadline * 1000, + }, timeout=max(.1, deadline - time.time()), retry_safe=False) + receipt["checkpoint"] = {"captured": True, "destination": str(destination), **captured} + except Exception as error: + # Capture failure is explicit but cannot rewrite an already settled work + # outcome or replay its model/evaluation. Keep the source trial running. + receipt["checkpoint"].update(reason="capture_failed", error_kind=type(error).__name__, error=str(error)[:500]) + finally: + receipt["checkpoint"]["elapsed_seconds"] = time.monotonic() - started diff --git a/benchmark/runtime/harbor.py b/benchmark/runtime/harbor.py index ee5e8fe435..9b73ee047c 100644 --- a/benchmark/runtime/harbor.py +++ b/benchmark/runtime/harbor.py @@ -59,6 +59,7 @@ def __init__( replan_after_todos=None, replan_after_turns=None, stride_strategy=None, + checkpoint_request_path=None, task_entry=None, planning_timeout_sec=300, turn_envelope=False, @@ -102,6 +103,12 @@ def __init__( raise ValueError("stride_strategy requires a LoopX execution mode") self.stride_strategy = (normalize_stride_strategy(stride_strategy) if stride_strategy is not None else None) + self.checkpoint_request_path = None + if checkpoint_request_path is not None: + if self.execution.mode not in {"heartbeat", "turn"}: + raise ValueError("checkpoint_request_path requires a settled-work worker") + from .checkpoint import normalize_checkpoint_request_path + self.checkpoint_request_path = normalize_checkpoint_request_path(checkpoint_request_path) self._task_path = "/usr/local/bin:/usr/bin:/bin" self._phase_number = 0 self._seeded_todo_id: str | None = None @@ -525,6 +532,8 @@ def _worker_env(self, *, cwd: str) -> dict[str, str]: "LOOPX_SHARED_SKILLS": _SHARED_SKILLS, "LOOPX_EXECUTION_MODE": self.execution.mode, "LOOPX_TASK_ENTRY": self.execution.task_entry, + **({"LOOPX_CHECKPOINT_REQUEST": self.checkpoint_request_path} + if self.checkpoint_request_path is not None else {}), **({"LOOPX_TURN_ENVELOPE": "1"} if self.execution.turn_envelope else {}), "LOOPX_ITERATION_CONTEXT": self.execution.context, "LOOPX_CODEX_SANDBOX": self.execution.sandbox, diff --git a/benchmark/runtime/sforge.py b/benchmark/runtime/sforge.py index b2bfff5e0f..a7a968d94d 100644 --- a/benchmark/runtime/sforge.py +++ b/benchmark/runtime/sforge.py @@ -91,7 +91,8 @@ def __init__(self, config, *, profile: str, cwd: str, turn_envelope: bool = False, replan_after_turns: int | None = None, replan_after_todos: int | None = None, - stride_strategy: str | None = None): + stride_strategy: str | None = None, + checkpoint_request_path: str | None = None): super().__init__(config) if profile not in PROFILES: raise ValueError("Unknown benchmark worker profile") @@ -136,6 +137,12 @@ def __init__(self, config, *, profile: str, cwd: str, from loopx.control_plane.work_items.stride_policy_codec import normalize_stride_strategy stride_strategy = normalize_stride_strategy(stride_strategy) self.stride_strategy = stride_strategy + self.checkpoint_request_path = None + if checkpoint_request_path is not None: + if not profile.startswith("heartbeat-"): + raise ValueError("checkpoint_request_path requires a heartbeat profile") + from .checkpoint import normalize_checkpoint_request_path + self.checkpoint_request_path = normalize_checkpoint_request_path(checkpoint_request_path) self.profile, self.cwd = profile, cwd if feedback not in FEEDBACK_MODES: raise ValueError("Unknown feedback mode") @@ -222,6 +229,7 @@ def _install_worker(self, backend, handle, log_dir, logger): replan_after_turns=self.replan_after_turns, replan_after_todos=self.replan_after_todos, stride_strategy=self.stride_strategy, + checkpoint_request_path=self.checkpoint_request_path, task_objective=self.task_objective, ) asyncio.run(self.runtime.install(self.environment)) @@ -260,6 +268,8 @@ def _install_worker(self, backend, handle, log_dir, logger): "task_entry": self.task_entry, "reasoning_effort": effort, "timeout_seconds": self.timeout_seconds, "loopx_usage_ping_enabled": False, + **({"checkpoint_capture_enabled": True} + if self.checkpoint_request_path is not None else {}), **({"planning_timeout_seconds": None} if self.task_entry == "loopx-planned" else {}), "stop_hook": self.profile == "official", diff --git a/benchmark/runtime/worker.py b/benchmark/runtime/worker.py index a1f2b7b05b..5118ecd879 100644 --- a/benchmark/runtime/worker.py +++ b/benchmark/runtime/worker.py @@ -397,6 +397,10 @@ def run_once(env: dict[str, str]) -> dict: receipt.update(ok=False, error_kind=type(exc).__name__) raise finally: + if env.get("LOOPX_CHECKPOINT_REQUEST"): + from benchmark.runtime.checkpoint import capture_after_wake + + capture_after_wake(env, receipt) if (home / "sessions").is_dir(): # One authoritative copy; resumed prefixes are never double counted. shutil.copytree( diff --git a/loopx/control_plane/configuration_backup.ts b/loopx/control_plane/configuration_backup.ts index 7d83b37e21..f7189928e7 100644 --- a/loopx/control_plane/configuration_backup.ts +++ b/loopx/control_plane/configuration_backup.ts @@ -1,10 +1,10 @@ /** Configuration checkpoint IO. Configuration owners retain validation and activation. */ -import {lstat, mkdir, mkdtemp, readFile, rename, rm, rmdir} from "node:fs/promises"; +import {lstat, mkdir, mkdtemp, readFile, rm} from "node:fs/promises"; import {dirname, isAbsolute, join, resolve} from "node:path"; import type {JsonObject} from "./effect_program.ts"; import {requireJsonObject, requireNonEmptyString} from "./runtime_decode.ts"; import {canonicalAuthorityJson, canonicalAuthoritySha256, hasExactAuthorityKeys} from "./coordination/authority_store_codec.ts"; -import {durableWriteJson} from "./effect_runtime_io.ts"; +import {durableWriteJson, publishNewDirectory} from "./effect_runtime_io.ts"; const SCHEMA = "loopx_configuration_backup_v0"; @@ -87,16 +87,7 @@ export async function restoreConfigurationBackup(value: unknown): Promise name)}; await durableWriteJson(join(staging, "restore-receipt.json"), receipt); - if (process.platform === "win32") { - // Windows refuses a directory rename into an existing destination, - // so the absent-target rename publishes the complete checkpoint. - await rename(staging, destination); - } else { - // POSIX can replace an empty directory; reserve the name exclusively. - await mkdir(destination, {mode: 0o700}); - try { await rename(staging, destination); } - catch (error) { await rmdir(destination); throw error; } - } + await publishNewDirectory(staging, destination); return receipt; } finally { await rm(staging, {recursive: true, force: true}); } } diff --git a/loopx/control_plane/effect_runtime_handlers.ts b/loopx/control_plane/effect_runtime_handlers.ts index 9558769848..f71585866d 100644 --- a/loopx/control_plane/effect_runtime_handlers.ts +++ b/loopx/control_plane/effect_runtime_handlers.ts @@ -407,6 +407,7 @@ export function createEffectRuntimeHandlers( ["coordination.local_authority.promote", lazyHandler(() => import("./coordination/local_authority_runtime.ts"), ({promoteLocalCoordinationAuthority}) => promoteLocalCoordinationAuthority)], ["coordination.authority_archive.manage", lazyHandler(() => import("./coordination/local_authority_archive.ts"), ({manageLocalAuthorityArchive}) => manageLocalAuthorityArchive)], ["configuration.backup", lazyHandler(() => import("./configuration_backup.ts"), ({configurationBackupOperation}) => configurationBackupOperation)], + ["runtime.checkpoint_artifacts", lazyHandler(() => import("./runtime/checkpoint_artifacts.ts"), ({checkpointArtifacts}) => checkpointArtifacts)], ["coordination.sqlite_backup.snapshot", lazyHandler(() => import("./coordination/sqlite_backup.ts"), ({snapshotSqliteBackup}) => snapshotSqliteBackup)], ["coordination.local_authority.new_goal_storage", lazyHandler(() => import("./coordination/local_authority_defaults.ts"), ({manageNewGoalStorage}) => manageNewGoalStorage)], ["coordination.local_authority.promotion_review", lazyHandler(() => Promise.all([import("./coordination/local_authority_runtime.ts"), import("./coordination/source_transfer.ts")]), ([{reviewLocalCoordinationAuthorityPromotion}, {withCoordinationSourceTransfer}]) => withCoordinationSourceTransfer("coordination.local_authority.promotion_review", reviewLocalCoordinationAuthorityPromotion))], diff --git a/loopx/control_plane/effect_runtime_io.ts b/loopx/control_plane/effect_runtime_io.ts index bf44e1ebcd..a8f4b14dc4 100644 --- a/loopx/control_plane/effect_runtime_io.ts +++ b/loopx/control_plane/effect_runtime_io.ts @@ -8,7 +8,7 @@ import { readSync, writeFileSync, } from "node:fs"; -import { mkdir, open, readFile, rename, rm, stat } from "node:fs/promises"; +import { mkdir, open, readFile, rename, rm, rmdir, stat } from "node:fs/promises"; import { dirname } from "node:path"; import type { JsonObject } from "./effect_program.ts"; @@ -484,6 +484,19 @@ export async function atomicWriteText( await atomicWriteTextFile(path, content); } +/** Publish a complete staged directory without replacing an existing target. + * Callers own source validation, staging cleanup and content durability. + */ +export async function publishNewDirectory(staging: string, destination: string): Promise { + if (process.platform === "win32") await rename(staging, destination); + else { + // POSIX rename can replace an empty directory; reserve the name first. + await mkdir(destination, {mode: 0o700}); + try {await rename(staging, destination);} + catch (error) {await rmdir(destination); throw error;} + } +} + export async function atomicWriteJson( path: string, payload: JsonObject, diff --git a/loopx/control_plane/runtime/checkpoint_artifacts.ts b/loopx/control_plane/runtime/checkpoint_artifacts.ts new file mode 100644 index 0000000000..de61de7260 --- /dev/null +++ b/loopx/control_plane/runtime/checkpoint_artifacts.ts @@ -0,0 +1,228 @@ +/** Inert artifact capture for an already-owned source barrier. + * Source stability and byte integrity are not research-point or execution authority. + * Native sessions, canonical state adoption and evaluator opportunities retain owners. + */ +import {createHash} from "node:crypto"; +import {createReadStream} from "node:fs"; +import {copyFile, lstat, mkdir, mkdtemp, open, readFile, readdir, rm} from "node:fs/promises"; +import {dirname, isAbsolute, join, resolve, sep} from "node:path"; +import type {JsonObject} from "../effect_program.ts"; +import {requireJsonObject, requireNonEmptyString} from "../runtime_decode.ts"; +import {canonicalAuthoritySha256, hasExactAuthorityKeys} from "../coordination/authority_store_codec.ts"; +import {snapshotSqliteDatabase} from "../coordination/sqlite_backup.ts"; +import {durableWriteJson, publishNewDirectory} from "../effect_runtime_io.ts"; + +const SCHEMA = "loopx_checkpoint_artifact_set_v0"; +type Component = {id: string; source_path: string}; +type Member = {path: string; kind: "file" | "directory"; mode: number; size?: number; sha256?: string}; + +function absolute(value: unknown, label: string): string { + const path = requireNonEmptyString(value, label); + if (!isAbsolute(path) || resolve(path) !== path) throw new Error(`${label} must be normalized and absolute`); + return path; +} +async function physical(path: string): Promise { + for (let current = path;; current = dirname(current)) { + const entry = await lstat(current); + if (entry.isSymbolicLink()) throw new Error("checkpoint path must not contain symlinks"); + if (current !== path && !entry.isDirectory()) throw new Error("checkpoint ancestor must be a directory"); + if (dirname(current) === current) break; + } +} +function components(value: unknown): Component[] { + if (!Array.isArray(value) || value.length === 0) throw new Error("checkpoint requires explicit components"); + const ids = new Set(); + const result = value.map(raw => { + const entry = requireJsonObject(raw, "checkpoint component"); + if (!hasExactAuthorityKeys(entry, ["id", "source_path"])) throw new Error("invalid checkpoint component fields"); + const id = requireNonEmptyString(entry.id, "checkpoint component id"); + if (!/^[a-z][a-z0-9_-]{0,63}$/.test(id) || ids.has(id)) throw new Error("checkpoint component identity is invalid or duplicated"); + ids.add(id); + return {id, source_path: absolute(entry.source_path, "checkpoint component source")}; + }); + for (const a of result) for (const b of result) { + if (a !== b && (a.source_path === b.source_path || a.source_path.startsWith(b.source_path + sep))) { + throw new Error("checkpoint component sources overlap"); + } + } + return result; +} +async function digest(path: string): Promise { + const hash = createHash("sha256"); + for await (const block of createReadStream(path)) hash.update(block); + return hash.digest("hex"); +} +async function members(root: string, sourceObservation: boolean): Promise { + await physical(root); + const output: Member[] = []; + async function visit(path: string, name: string): Promise { + const stat = await lstat(path); + if (stat.isSymbolicLink()) throw new Error("checkpoint source contains a symlink"); + // SQLite shared-memory is a transient reader index, not committed source + // data. Opening a read-only backup may update its reader slots. + if (sourceObservation && stat.isFile() && await sqliteSidecar(path, ["-shm"])) return; + if (stat.isDirectory()) { + output.push({path: name, kind: "directory", mode: stat.mode & 0o777}); + for (const child of (await readdir(path)).sort()) await visit(join(path, child), name ? `${name}/${child}` : child); + } else if (stat.isFile()) { + output.push({path: name, kind: "file", mode: stat.mode & 0o777, size: stat.size, sha256: await digest(path)}); + } else throw new Error("checkpoint source contains an unsupported file type"); + } + await visit(root, ""); + if ((await lstat(root)).isFile() && await sqliteFile(root)) { + for (const suffix of sourceObservation ? ["-wal", "-journal"] : ["-wal", "-shm", "-journal"]) { + try {await visit(root + suffix, suffix);} + catch (error) {if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw error;} + } + } + return output; +} +async function witness(entries: Component[], sourceObservation = true): Promise { + const result: JsonObject = {}; + for (const entry of entries) result[entry.id] = await members(entry.source_path, sourceObservation); + return result; +} +function unchanged(a: unknown, b: unknown, message = "checkpoint source moved; retry at a new owned barrier"): void { + if (canonicalAuthoritySha256(a) !== canonicalAuthoritySha256(b)) throw new Error(message); +} +async function absentDestination(path: string): Promise { + await physical(dirname(path)); + try {await lstat(path); throw new Error("checkpoint destination already exists");} + catch (error) {if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw error;} +} +async function sqliteFile(path: string): Promise { + const handle = await open(path, "r"); + try { + const bytes = Buffer.alloc(16); + const {bytesRead} = await handle.read(bytes, 0, 16, 0); + return bytesRead === 16 && bytes.equals(Buffer.from("SQLite format 3\0")); + } finally {await handle.close();} +} +async function sqliteSidecar(path: string, suffixes: string[]): Promise { + const suffix = suffixes.find(s => path.endsWith(s)); + if (!suffix) return false; + const database = path.slice(0, -suffix.length); + try {return (await lstat(database)).isFile() && await sqliteFile(database);} + catch (error) {if ((error as NodeJS.ErrnoException).code === "ENOENT") return false; throw error;} +} +async function copyComponent(source: string, destination: string, snapshotDatabases = true): Promise { + const stat = await lstat(source); + if (stat.isSymbolicLink()) throw new Error("checkpoint source became a symlink"); + if (stat.isDirectory()) { + await mkdir(destination, {mode: 0o700}); + const children = (await readdir(source)).sort(); + const databases = new Set(); + for (const child of snapshotDatabases ? children : []) { + const path = join(source, child); + if ((await lstat(path)).isFile()) { + if (await sqliteFile(path)) databases.add(child); + } + } + for (const child of children) { + if (["-wal", "-shm", "-journal"].some(suffix => child.endsWith(suffix) && databases.has(child.slice(0, -suffix.length)))) continue; + await copyComponent(join(source, child), join(destination, child), snapshotDatabases); + } + } else if (stat.isFile()) { + // SQLite's existing physical snapshot owner seals committed WAL contents. + if (snapshotDatabases && await sqliteFile(source)) await snapshotSqliteDatabase(source, destination); + else await copyFile(source, destination); + const handle = await open(destination, "r+"); + try {await handle.sync();} finally {await handle.close();} + } else throw new Error("checkpoint source contains an unsupported file type"); +} +function manifestBody(raw: unknown): JsonObject { + const value = requireJsonObject(raw, "checkpoint manifest"); + if (!hasExactAuthorityKeys(value, ["schema_version", "point_metadata", "components", "source_witness", "artifact_witness", "research_point_qualified", "execution_authority_granted", "privacy_certified", "sha256"]) || + value.schema_version !== SCHEMA || value.research_point_qualified !== false || + value.execution_authority_granted !== false || value.privacy_certified !== false) throw new Error("checkpoint manifest fields are invalid"); + const {sha256, ...body} = value; + if (sha256 !== canonicalAuthoritySha256(body)) throw new Error("checkpoint manifest digest mismatch"); + components(value.components); + requireJsonObject(value.point_metadata, "checkpoint point metadata"); + requireJsonObject(value.source_witness, "checkpoint source witness"); + requireJsonObject(value.artifact_witness, "checkpoint artifact witness"); + return value; +} +async function verify(root: string, expected: unknown): Promise { + await physical(root); + await physical(join(root, "checkpoint.json")); + const value = manifestBody(JSON.parse(await readFile(join(root, "checkpoint.json"), "utf8"))); + if (value.sha256 !== expected) throw new Error("checkpoint reviewed digest mismatch"); + const entries = components(value.components); + const actual = await witness(entries.map(c => ({id: c.id, source_path: join(root, "components", c.id)})), false); + unchanged(value.artifact_witness, actual, "checkpoint artifact members differ from the reviewed capture"); + if ((await readdir(root)).sort().join(",") !== "checkpoint.json,components" || + (await readdir(join(root, "components"))).sort().join(",") !== entries.map(c => c.id).sort().join(",")) throw new Error("checkpoint has unexpected members"); + return {ok: true, sha256: value.sha256, integrity_verified: true, research_point_qualified: false, + execution_authority_granted: false, privacy_certified: false}; +} +export async function checkpointArtifacts(value: unknown): Promise { + const request = requireJsonObject(value, "checkpoint artifacts request"); + const fields = request.action === "probe" ? ["action", "components"] : + request.action === "verify" ? ["action", "root", "expected_sha256"] : + request.action === "capture" ? ["action", "components", "destination", "point_metadata", "expected_source_sha256"] : + request.action === "materialize" ? ["action", "root", "expected_sha256", "destination"] : null; + if (fields === null) throw new Error("unsupported checkpoint artifact operation"); + if (!hasExactAuthorityKeys(request, [...fields, ...(request.deadline_epoch_ms === undefined ? [] : ["deadline_epoch_ms"])])) { + throw new Error("checkpoint request fields are invalid"); + } + const deadline = request.deadline_epoch_ms; + if (deadline !== undefined && (typeof deadline !== "number" || !Number.isFinite(deadline))) throw new Error("checkpoint deadline must be finite"); + function withinDeadline(): void { + if (typeof deadline === "number" && Date.now() >= deadline) throw new Error("checkpoint capture window has closed"); + } + withinDeadline(); + if (request.action === "probe") { + const sources = components(request.components); + const observed = await witness(sources); + return {source_witness: observed, source_sha256: canonicalAuthoritySha256(observed)}; + } + if (request.action === "verify") return verify(absolute(request.root, "checkpoint root"), request.expected_sha256); + if (request.action !== "capture" && request.action !== "materialize") throw new Error("unsupported checkpoint artifact operation"); + const destination = absolute(request.destination, "checkpoint destination"); + await absentDestination(destination); + const staging = await mkdtemp(join(dirname(destination), ".loopx-checkpoint-")); + try { + if (request.action === "materialize") { + const root = absolute(request.root, "checkpoint root"); + const verification = await verify(root, request.expected_sha256); + await copyComponent(root, join(staging, "unit"), false); + await verify(join(staging, "unit"), request.expected_sha256); + withinDeadline(); + await publishNewDirectory(join(staging, "unit"), destination); + return {...verification, materialized: true, activation_performed: false}; + } + const sources = components(request.components); + if (sources.some(c => destination === c.source_path || destination.startsWith(c.source_path + sep) || c.source_path.startsWith(destination + sep))) throw new Error("checkpoint output overlaps its source"); + const before = await witness(sources); + if (request.expected_source_sha256 !== canonicalAuthoritySha256(before)) throw new Error("checkpoint source moved before capture"); + await mkdir(join(staging, "components"), {mode: 0o700}); + for (const c of sources) await copyComponent(c.source_path, join(staging, "components", c.id)); + unchanged(before, await witness(sources)); + const artifacts = await witness(sources.map(c => ({id: c.id, source_path: join(staging, "components", c.id)})), false); + // Byte-copy members must match the reviewed source, including ABA movement + // that could otherwise escape a before/after comparison. SQLite is sealed + // by its existing snapshot owner and has a separate physical witness. + for (const c of sources) { + for (const member of before[c.id] as Member[]) { + if (member.kind !== "file") continue; + const source = (await lstat(c.source_path)).isDirectory() ? join(c.source_path, member.path) : c.source_path + member.path; + if (await sqliteFile(source) || await sqliteSidecar(source, ["-wal", "-shm", "-journal"])) continue; + const copied = (artifacts[c.id] as Member[]).find(m => m.path === member.path && m.kind === "file"); + if (!copied || copied.sha256 !== member.sha256 || copied.size !== member.size || copied.mode !== member.mode) { + throw new Error("checkpoint copied bytes do not match the reviewed source"); + } + } + } + const body: JsonObject = {schema_version: SCHEMA, components: sources, point_metadata: requireJsonObject(request.point_metadata, "checkpoint point metadata"), + source_witness: before, artifact_witness: artifacts, research_point_qualified: false, + execution_authority_granted: false, privacy_certified: false}; + const manifest = {...body, sha256: canonicalAuthoritySha256(body)}; + await durableWriteJson(join(staging, "checkpoint.json"), manifest); + await verify(staging, manifest.sha256); + unchanged(before, await witness(sources)); + withinDeadline(); + await publishNewDirectory(staging, destination); + return {...await verify(destination, manifest.sha256), source_stability_verified: true}; + } finally {await rm(staging, {recursive: true, force: true});} +} From 84fcb075dde4b73a269ae0abfbe80c2fdb9d5b43 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Mon, 12 Oct 2026 20:06:50 +0800 Subject: [PATCH 03/69] test(checkpoint): verify capture integrity and runner isolation Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/tests/test_sforge_runtime.py | 22 +++ benchmark/tests/test_shared_codex_runtime.py | 15 ++ .../test_research_checkpoint_capture.py | 93 ++++++++++++ .../checkpoint_artifacts.test.ts | 140 ++++++++++++++++++ 4 files changed, 270 insertions(+) create mode 100644 tests/control_plane/test_research_checkpoint_capture.py create mode 100644 tests/control_plane_ts/checkpoint_artifacts.test.ts diff --git a/benchmark/tests/test_sforge_runtime.py b/benchmark/tests/test_sforge_runtime.py index 4651474de4..bbf566f5e8 100644 --- a/benchmark/tests/test_sforge_runtime.py +++ b/benchmark/tests/test_sforge_runtime.py @@ -153,6 +153,28 @@ def command(handle, cmd, **kwargs): assert len(staged) == int(profile in {"official", "single"}) +@pytest.mark.parametrize("capture_path", [None, "/control/checkpoint-request.json"]) +def test_checkpoint_request_reaches_sforge_worker_without_changing_default_profile(tmp_path, monkeypatch, capture_path): + pytest.importorskip("sforge") + pytest.importorskip("harbor") + from sforge.harness.config import SForgeConfig + from benchmark.runtime.sforge import SForgeWorker, BenchmarkCodex + monkeypatch.setenv("CODEX_AUTH_JSON_PATH", "/synthetic-credential") + async def installed(self, environment): + pass # Exercise the real adapter handoff without starting a trial. + monkeypatch.setattr(BenchmarkCodex, "install", installed) + config = SForgeConfig(agent_model="fixture", agent_effort="xhigh") + worker = SForgeWorker(config, profile="heartbeat-explore", cwd="/task", checkpoint_request_path=capture_path) + worker.install_stop_hook(None, None, tmp_path, None) + env = worker.runtime._worker_env(cwd="/task") + profile = json.loads((tmp_path / "worker-profile.json").read_text()) + assert env.get("LOOPX_CHECKPOINT_REQUEST") == capture_path + assert ("checkpoint_capture_enabled" in profile) is (capture_path is not None) + assert profile.get("checkpoint_capture_enabled") is (True if capture_path else None) + with pytest.raises(ValueError, match="checkpoint_request_path"): + SForgeWorker(config, profile="official", cwd="/task", checkpoint_request_path="/control/request.json") + + @pytest.mark.parametrize("profile,total,expected", [ ("native-goal", 64800, 64640), ("native-goal", 1800, 1640), ("heartbeat-resume", 64800, 64640), ("heartbeat-explore", 64800, 64640), diff --git a/benchmark/tests/test_shared_codex_runtime.py b/benchmark/tests/test_shared_codex_runtime.py index e9e3e2787d..791d41fd74 100644 --- a/benchmark/tests/test_shared_codex_runtime.py +++ b/benchmark/tests/test_shared_codex_runtime.py @@ -362,6 +362,21 @@ def test_baseline_and_treatment_use_same_harbor_entry(tmp_path): assert agent.replan_after_turns == (6 if agent.execution.uses_loopx else None) +def test_checkpoint_request_is_explicit_and_reaches_the_existing_worker_env(tmp_path, monkeypatch): + pytest.importorskip("harbor") + from benchmark.runtime.harbor import BenchmarkCodex + monkeypatch.setenv("LOOPX_CHECKPOINT_REQUEST", "/ambient-request.json") + off = BenchmarkCodex(logs_dir=tmp_path / "off", model_name="openai/fixture") + assert "LOOPX_CHECKPOINT_REQUEST" not in off._worker_env(cwd="/workspace") + on = BenchmarkCodex(logs_dir=tmp_path / "on", model_name="openai/fixture", + checkpoint_request_path="/control/checkpoint-request.json") + assert on._worker_env(cwd="/workspace")["LOOPX_CHECKPOINT_REQUEST"] == "/control/checkpoint-request.json" + for settings in ({"execution_mode": "plain", "checkpoint_request_path": "/control/request.json"}, + {"checkpoint_request_path": "relative"}, {"checkpoint_request_path": "/control/../request.json"}): + with pytest.raises(ValueError, match="checkpoint_request_path"): + BenchmarkCodex(logs_dir=tmp_path / "invalid", model_name="openai/fixture", **settings) + + @pytest.mark.parametrize("existing", [False, True]) @pytest.mark.parametrize("settings,field,value", [ ({}, "replan_after_effective_turns", 6), diff --git a/tests/control_plane/test_research_checkpoint_capture.py b/tests/control_plane/test_research_checkpoint_capture.py new file mode 100644 index 0000000000..c9105138ea --- /dev/null +++ b/tests/control_plane/test_research_checkpoint_capture.py @@ -0,0 +1,93 @@ +"""Inert checkpoint capture reuses the real settled-Turn source owner.""" +from pathlib import Path +import json +import time + +import test_quota_settlement_cli as cli +from canonical_authority_fixture import isolate_sqlite_runtime + +from benchmark.runtime.checkpoint import capture_after_wake +from loopx.control_plane.effect_runtime import effect_runtime_result + + +def test_feature_off_and_failed_wakes_do_not_read_request_or_settlement(monkeypatch): + def forbidden(*args, **kwargs): + raise AssertionError("feature-off/wake-failure must perform no IO") + monkeypatch.setattr("benchmark.runtime.checkpoint.read_heartbeat_settlement", forbidden) + monkeypatch.setattr(Path, "read_bytes", forbidden) + receipt = {"ok": True, "stage": "execute", "mode": "heartbeat"} + capture_after_wake({}, receipt) + assert receipt == {"ok": True, "stage": "execute", "mode": "heartbeat"} + failed = {"ok": False, "stage": "execute", "mode": "heartbeat"} + capture_after_wake({"LOOPX_CHECKPOINT_REQUEST": "not-readable"}, failed) + assert failed["ok"] is False + assert failed["checkpoint"]["captured"] is False + + +def test_closed_budget_never_reads_the_request_or_replays_work(monkeypatch): + def forbidden(*args, **kwargs): + raise AssertionError("closed budget must not read sources") + monkeypatch.setattr(Path, "read_bytes", forbidden) + receipt = {"ok": True, "stage": "execute", "mode": "heartbeat"} + capture_after_wake({"LOOPX_CHECKPOINT_REQUEST": "not-readable", + "LOOPX_PHASE_DEADLINE_EPOCH": str(time.time() - 1)}, receipt) + assert receipt["ok"] is True + assert receipt["checkpoint"]["reason"] == "capture_window_closed" + capture_after_wake({"LOOPX_CHECKPOINT_REQUEST": "not-readable", + "LOOPX_PHASE_DEADLINE_EPOCH": "nan"}, receipt) + assert receipt["ok"] is True + assert receipt["checkpoint"]["reason"] == "capture_failed" + + +def test_real_cli_settlement_capture_verification_and_inert_materialization(tmp_path, monkeypatch): + isolate_sqlite_runtime(tmp_path, monkeypatch) + project, runtime, registry = cli._write_fixture(tmp_path) + binding = ("--agent-id", cli.AGENT_ID, "--todo-id", cli.TODO_ID, "--turn-instance-id", cli.TURN_ID) + def run(*args): + code, result = cli._run_cli(registry, runtime, *args) + assert code == 0, result + return result + run("quota", "should-run", "--goal-id", cli.GOAL_ID, "--codex-app", *binding) + logs = tmp_path / "logs" + wake = logs / "wakes" + wake.mkdir(parents=True) + (project / "solution.py").write_text("retained_candidate = 7\n") + request = tmp_path / "request.json" + target = logs / "checkpoints" / "point" + request.write_text(json.dumps({"destination": str(target), "components": [ + {"id": "code", "source_path": str(project / "solution.py")}, + {"id": "research", "source_path": str(project / f".codex/goals/{cli.GOAL_ID}/ACTIVE_GOAL_STATE.md")}, + ], "point_metadata": {"meaning": "synthetic capture; no model/session or evaluator qualification"}})) + env = {"LOOPX_CHECKPOINT_REQUEST": str(request), "LOOPX_WAKE_LOG_DIR": str(wake), + "LOOPX_RUNTIME_ROOT": str(runtime), "LOOPX_GOAL_ID": cli.GOAL_ID, + "LOOPX_AGENT_ID": cli.AGENT_ID, "LOOPX_REGISTRY": str(registry)} + receipt = {"ok": True, "stage": "execute", "mode": "heartbeat", "turn_id": cli.TURN_ID} + capture_after_wake(env, receipt) + assert receipt["checkpoint"]["reason"] == "owned_turn_not_settled" + assert not target.exists() + vision = tmp_path / "vision.json" + vision.write_text(json.dumps({"schema_version": "goal_vision_replan_contract_v0", "state": "vision_patch_proposed", + "vision_patch": {"acceptance_summary": "Synthetic source fixture checked; checkpoint work remains."}, + "path_delta": {"outcome": "continue", "prior_assumption": "Fixture source is available.", + "observed_reality": "The isolated source fixture was verified.", "retained": ["Inert checkpoint qualification"], + "evidence_refs": ["evidence:synthetic-capture"]}})) + run("refresh-state", "--goal-id", cli.GOAL_ID, *binding, "--classification", "checkpoint_fixture_verified", + "--delivery-batch-scale", "multi_surface", "--delivery-outcome", "outcome_progress", "--agent-vision-json", str(vision)) + run("quota", "spend-slot", "--goal-id", cli.GOAL_ID, *binding, "--source", "heartbeat", "--execute") + goal_state = project / f".codex/goals/{cli.GOAL_ID}/ACTIVE_GOAL_STATE.md" + retained_goal = goal_state.read_bytes() + capture_after_wake(env, receipt) + assert receipt["ok"] is True + assert receipt["checkpoint"].get("captured") is True, receipt + assert receipt["checkpoint"]["research_point_qualified"] is False + assert goal_state.read_bytes() == retained_goal + digest = receipt["checkpoint"]["sha256"] + restored = effect_runtime_result("runtime.checkpoint_artifacts", {"action": "materialize", "root": str(target), + "expected_sha256": digest, "destination": str(tmp_path / "inert")}) + assert restored["activation_performed"] is False + assert (tmp_path / "inert/components/code").read_text() == "retained_candidate = 7\n" + before = (target / "checkpoint.json").read_bytes() + capture_after_wake(env, receipt) + assert receipt["checkpoint"]["reason"] == "destination_already_present_no_replay" + assert (target / "checkpoint.json").read_bytes() == before + assert goal_state.read_bytes() == retained_goal diff --git a/tests/control_plane_ts/checkpoint_artifacts.test.ts b/tests/control_plane_ts/checkpoint_artifacts.test.ts new file mode 100644 index 0000000000..3365b9bb6f --- /dev/null +++ b/tests/control_plane_ts/checkpoint_artifacts.test.ts @@ -0,0 +1,140 @@ +import assert from "node:assert/strict"; +import test from "node:test"; +import {chmod, mkdir, mkdtemp, readFile, readdir, realpath, rm, symlink, writeFile} from "node:fs/promises"; +import {tmpdir} from "node:os"; +import {join} from "node:path"; +import {checkpointArtifacts} from "../../loopx/control_plane/runtime/checkpoint_artifacts.ts"; +import {sqliteAuthorityRuntime} from "../../loopx/control_plane/coordination/sqlite_runtime.ts"; + +async function fixture(t: test.TestContext) { + const root = await realpath(await mkdtemp(join(tmpdir(), "checkpoint-artifacts-"))); + t.after(() => rm(root, {recursive: true, force: true})); + await mkdir(join(root, "code")); + await mkdir(join(root, "code/empty")); + await writeFile(join(root, "code/solution.sh"), "#!/bin/sh\nexit 0\n"); + await chmod(join(root, "code/solution.sh"), 0o755); + await writeFile(join(root, "conversation"), "native cursor fixture\n"); + const components = [{id: "code", source_path: join(root, "code")}, {id: "conversation", source_path: join(root, "conversation")}]; + const probe = await checkpointArtifacts({action: "probe", components}); + const request = {action: "capture", components, expected_source_sha256: probe.source_sha256, + destination: join(root, "captured"), point_metadata: {cursor: "completed-native-turn", effects: "historical only"}}; + return {root, request}; +} + +test("capture, verify and inert materialization preserve executable code and empty directories", async t => { + const {root, request} = await fixture(t); + const captured = await checkpointArtifacts(request); + assert.equal(captured.integrity_verified, true); + assert.equal(captured.source_stability_verified, true); + assert.equal(captured.research_point_qualified, false); + assert.equal(captured.execution_authority_granted, false); + assert.equal(captured.privacy_certified, false); + const restored = await checkpointArtifacts({action: "materialize", root: request.destination, + expected_sha256: captured.sha256, destination: join(root, "restored")}); + assert.equal(restored.activation_performed, false); + assert.equal(await readFile(join(root, "restored/components/code/solution.sh"), "utf8"), "#!/bin/sh\nexit 0\n"); + assert.deepEqual(await readdir(join(root, "restored/components/code/empty")), []); + assert.deepEqual(await readFile(join(root, "restored/checkpoint.json")), await readFile(join(root, "captured/checkpoint.json"))); + await assert.rejects(checkpointArtifacts({action: "materialize", root: request.destination, + expected_sha256: captured.sha256, destination: join(root, "restored")}), /already exists/); +}); + +test("changed source, duplicate identities and overlapping sources publish no unit", async t => { + const {root, request} = await fixture(t); + await writeFile(join(root, "conversation"), "new native cursor"); + await assert.rejects(checkpointArtifacts(request), /source moved/); + await assert.rejects(checkpointArtifacts({...request, components: [...request.components, request.components[0]]}), /duplicated/); + await assert.rejects(checkpointArtifacts({action: "probe", components: [request.components[0], {id: "nested", source_path: join(root, "code/empty")}]}), /overlap/); + assert.deepEqual((await readdir(root)).sort(), ["code", "conversation"]); +}); + +test("missing, changed and extra artifact members or foreign expected digest reject", async t => { + const {root, request} = await fixture(t), captured = await checkpointArtifacts(request); + const verify = {action: "verify", root: request.destination, expected_sha256: captured.sha256}; + await assert.rejects(checkpointArtifacts({...verify, expected_sha256: "foreign"}), /digest/); + await writeFile(join(root, "captured/components/conversation"), "tampered"); + await assert.rejects(checkpointArtifacts(verify), /artifact members/); + await writeFile(join(root, "captured/components/conversation"), "native cursor fixture\n"); + await writeFile(join(root, "captured/foreign"), "unreviewed member"); + await assert.rejects(checkpointArtifacts(verify), /unexpected members/); + await rm(join(root, "captured/foreign")); + await rm(join(root, "captured/components/conversation")); + await assert.rejects(checkpointArtifacts(verify), /ENOENT/); + await assert.rejects(checkpointArtifacts({action: "materialize", root: request.destination, + expected_sha256: captured.sha256, destination: join(root, "must-not-exist")})); + assert.ok(!(await readdir(root)).includes("must-not-exist")); +}); + +test("expired or invalid capture windows publish no unit and leave the source intact", async t => { + const {root, request} = await fixture(t); + const bytes = await readFile(join(root, "conversation")); + await assert.rejects(checkpointArtifacts({...request, deadline_epoch_ms: Date.now() - 1}), /window has closed/); + await assert.rejects(checkpointArtifacts({...request, deadline_epoch_ms: Number.NaN}), /finite/); + await assert.rejects(checkpointArtifacts({...request, execution_authority_granted: true}), /fields are invalid/); + assert.deepEqual(await readFile(join(root, "conversation")), bytes); + assert.deepEqual((await readdir(root)).sort(), ["code", "conversation"]); +}); + +test("concurrent capture reserves one destination and cannot replace an existing unit", async t => { + const {root, request} = await fixture(t); + const results = await Promise.allSettled([checkpointArtifacts(request), checkpointArtifacts(request)]); + assert.equal(results.filter(r => r.status === "fulfilled").length, 1); + assert.equal(results.filter(r => r.status === "rejected").length, 1); + assert.deepEqual((await readdir(root)).sort(), ["captured", "code", "conversation"]); + const bytes = await readFile(join(root, "captured/checkpoint.json")); + await assert.rejects(checkpointArtifacts(request), /already exists/); + assert.deepEqual(await readFile(join(root, "captured/checkpoint.json")), bytes); +}); + +test("symlink sources, nested links and redirected destinations reject without publication", async t => { + const {root, request} = await fixture(t); + await symlink(join(root, "code"), join(root, "alias"), process.platform === "win32" ? "junction" : "dir"); + await assert.rejects(checkpointArtifacts({action: "probe", components: [{id: "code", source_path: join(root, "alias")}]}), /symlink/); + await symlink(join(root, "conversation"), join(root, "code/link")); + await assert.rejects(checkpointArtifacts(request), /symlink/); + await assert.rejects(checkpointArtifacts({...request, destination: join(root, "alias/capture")}), /symlink/); +}); + +test("SQLite uses the existing committed WAL snapshot owner and materializes verbatim", async t => { + const {root, request} = await fixture(t); + await mkdir(join(root, "state")); + const {driver} = sqliteAuthorityRuntime(); + const writer = new driver.DatabaseSync(join(root, "state/authority.sqlite")); + try { + writer.exec("PRAGMA journal_mode=WAL; PRAGMA wal_autocheckpoint=0; CREATE TABLE evidence(value TEXT); INSERT INTO evidence VALUES('retained negative result')"); + await writeFile(join(root, "state/notes-wal"), "ordinary retained note"); + const components = [...request.components, {id: "state", source_path: join(root, "state")}]; + const probe = await checkpointArtifacts({action: "probe", components}); + const captured = await checkpointArtifacts({...request, components, expected_source_sha256: probe.source_sha256}); + assert.deepEqual((await readdir(join(root, "captured/components/state"))).sort(), ["authority.sqlite", "notes-wal"]); + await writeFile(join(root, "captured/components/state/authority.sqlite-shm"), "foreign index"); + await assert.rejects(checkpointArtifacts({action: "verify", root: request.destination, + expected_sha256: captured.sha256}), /artifact members/); + await rm(join(root, "captured/components/state/authority.sqlite-shm")); + await checkpointArtifacts({action: "materialize", root: request.destination, + expected_sha256: captured.sha256, destination: join(root, "restored")}); + assert.deepEqual(await readFile(join(root, "restored/components/state/authority.sqlite")), await readFile(join(root, "captured/components/state/authority.sqlite"))); + const copy = new driver.DatabaseSync(join(root, "restored/components/state/authority.sqlite"), {readOnly: true}); + try {assert.equal(copy.prepare("SELECT value FROM evidence").get()?.value, "retained negative result");} + finally {copy.close();} + } finally {writer.close();} +}); + +test("a committed WAL change after the probe invalidates capture, including a standalone database component", async t => { + const {root, request} = await fixture(t); + const path = join(root, "authority.sqlite"), {driver} = sqliteAuthorityRuntime(); + const writer = new driver.DatabaseSync(path); + try { + writer.exec("PRAGMA journal_mode=WAL; PRAGMA wal_autocheckpoint=0; CREATE TABLE evidence(value TEXT)"); + const components = [{id: "state", source_path: path}]; + const probe = await checkpointArtifacts({action: "probe", components}); + writer.exec("INSERT INTO evidence VALUES('new committed evidence')"); + await assert.rejects(checkpointArtifacts({...request, components, expected_source_sha256: probe.source_sha256}), /source moved/); + assert.ok(!(await readdir(root)).includes("captured")); + const stable = await checkpointArtifacts({action: "probe", components}); + await checkpointArtifacts({...request, components, expected_source_sha256: stable.source_sha256}); + const copy = new driver.DatabaseSync(join(root, "captured/components/state"), {readOnly: true}); + try {assert.equal(copy.prepare("SELECT value FROM evidence").get()?.value, "new committed evidence");} + finally {copy.close();} + } finally {writer.close();} +}); From f1274702fbfd3a9b4fceb19e2357ca68a16a3569 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Mon, 12 Oct 2026 20:06:50 +0800 Subject: [PATCH 04/69] docs(checkpoint): document capture scope and remaining acceptance Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/README.md | 50 +++++++++++++++++++ .../hierarchical-agent-stride-control-v0.md | 8 +++ ...rarchical-agent-stride-control-v0.zh-CN.md | 6 +++ 3 files changed, 64 insertions(+) diff --git a/benchmark/README.md b/benchmark/README.md index 7cf8ce709f..c43ba24805 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -113,3 +113,53 @@ The [DeepSWE GPT xhigh v1 archive note](deepswe-gptxhigh-versions.md) registers preserved for inspection; the companion notice governs their current evidence status. Neither archive category is the architecture for new work or grants runner, verifier, submission, or publication authority. + +## Inert checkpoint capture + +This is a developer prerequisite for the [research checkpoint contract](../docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md#77-research-state-branching-and-matched-continuation), +not a qualified research continuation. The programmatic `BenchmarkCodex` and +heartbeat `SForgeWorker` adapters accept an explicit `checkpoint_request_path` +to a normalized absolute **guest** JSON file. Stage that file through the existing +environment transport. Ambient host variables do not enable capture; omitting +the option disables the hook. The EdgeBench CLI does not expose this option yet. + +The request contains exactly these fields; select only authorized private +artifacts and exclude credentials and hidden evaluation material: + +```json +{ + "destination": "/logs/agent/checkpoints/research-point", + "components": [ + {"id": "code", "source_path": "/workspace/solution.py"}, + {"id": "evidence", "source_path": "/control/source-evidence.json"} + ], + "point_metadata": {"purpose": "inert source capture"} +} +``` + +The destination must be a new direct child of the trial log owner's +`checkpoints` directory. Capture runs after a successful execute wake's original +settlement, before the next wake. It uses the remaining phase budget with a +30-second capture ceiling, preserves SQLite's committed WAL through the existing +backup owner, rejects observed source changes/symlinks and never overwrites a +destination. Capture failures are recorded separately in the wake receipt; +they do not replay work or change its settled outcome. The ceiling bounds +publication eligibility and RPC waits, not an operating-system I/O cancellation +guarantee. Capture cost remains part of the trial's elapsed time. + +For readback, call the existing `effect_runtime_result` adapter with method +`runtime.checkpoint_artifacts`, action `verify`, the captured `root` and the +receipt's `expected_sha256`. Action `materialize` with those fields and a new +`destination` verifies and copies the unit into isolation. Neither action adopts +state or starts execution. The new `loopx_checkpoint_artifact_set_v0` manifest +is an internal artifact contract; `point_metadata` is unverified evidence. +`research_point_qualified`, `execution_authority_granted` and `privacy_certified` +remain false. Integrity checks cover the declared members and observed stable +source boundary, not adversarial filesystem mutation or a cross-owner transaction. +Native session reconstruction, effect reconciliation, fresh authority admission +and evaluator cooldown/opportunity inheritance must pass before matched continuation. + +这是研究 checkpoint 的开发前置切片。仅 programmatic adapter 的显式 guest +请求路径会开启捕获，省略该参数即关闭；当前 EdgeBench CLI 尚无此选项。产物 +验证和隔离物化不授予执行权，不代表研究点已资格化。请求中的 metadata 只是 +证据，不能代替会话、效果、状态采用及评测机会的联合验收。 diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md index f230066218..7db1565ff5 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md @@ -690,6 +690,14 @@ state or feedback consumption from their terminal versions. Without a qualified contemporaneous capture of those participants, such a probe establishes Host feasibility only and must not enter a same-state intervention comparison. +The initial implementation slice is [inert artifact capture](../../../benchmark/README.md#inert-checkpoint-capture). +The benchmark adapter requests it after the existing Turn settlement; the typed +artifact owner seals explicit files/directories and committed SQLite snapshots, +rechecks source witnesses and verifies an isolated materialization. Its manifest +always declares the research point unqualified. Participant/effect reconciliation, +native session reconstruction, fresh state adoption and preserved evaluator +opportunities remain separate acceptance work; operator metadata cannot certify them. + An inert verification unit grants no execution authority. The destination uses fresh experiment/Host identities and the existing Goal/session/state admission paths. Historical leases, quota, timers, pending effects and submission tokens diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md index c218d8dda3..f743cb8cf8 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md @@ -582,6 +582,12 @@ Host adapter 须在固定版本验证原生历史重建，包括 compaction 和 截短历史会话不能从终局版本重建当时的代码、研究状态或反馈消费。没有其他参与者 同时捕获且已资格化的状态时，这类探针只证明 Host 可行性，不能进入同状态干预比较。 +第一项实现切片是 [inert 产物捕获](../../../benchmark/README.md#inert-checkpoint-capture)。 +Benchmark adapter 在既有 Turn 结算后请求捕获；typed 产物 owner 封存显式文件/ +目录及已提交 SQLite 快照，复查来源 witness 并验证隔离物化。Manifest 始终声明 +研究点尚未资格化。参与者/效果 reconciliation、原生会话重建、新状态采用及评测 +机会继承仍各自待验收；operator metadata 不能认证这些条件。 + inert verification unit 不授予执行权。目标侧使用新的实验/Host 身份，通过既有 Goal/session/state admission 进入。历史租约、quota、timer、pending effect 和 submission token 不能变成目标侧权限；未知效果先按原身份 reconcile，再决定 From 6a75518211378f45a7c80c4714b341ca0eb15352 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Mon, 12 Oct 2026 21:58:14 +0800 Subject: [PATCH 05/69] fix(runtime): preserve closed WAL capture stability Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- .../runtime/checkpoint_artifacts.ts | 12 ++++-- .../checkpoint_artifacts.test.ts | 39 +++++++++++++++++++ 2 files changed, 47 insertions(+), 4 deletions(-) diff --git a/loopx/control_plane/runtime/checkpoint_artifacts.ts b/loopx/control_plane/runtime/checkpoint_artifacts.ts index de61de7260..cf7c59b610 100644 --- a/loopx/control_plane/runtime/checkpoint_artifacts.ts +++ b/loopx/control_plane/runtime/checkpoint_artifacts.ts @@ -58,9 +58,13 @@ async function members(root: string, sourceObservation: boolean): Promise { const stat = await lstat(path); if (stat.isSymbolicLink()) throw new Error("checkpoint source contains a symlink"); - // SQLite shared-memory is a transient reader index, not committed source - // data. Opening a read-only backup may update its reader slots. - if (sourceObservation && stat.isFile() && await sqliteSidecar(path, ["-shm"])) return; + // SQLite's reader index and an empty WAL contain no committed data. A + // read-only backup can create either on a closed WAL database. Exclude + // only these recognized sidecars, not a nonempty WAL or an explicitly + // requested standalone file. Artifact verification excludes nothing. + if (sourceObservation && name !== "" && stat.isFile() && + (await sqliteSidecar(path, ["-shm"]) || + (stat.size === 0 && await sqliteSidecar(path, ["-wal"])))) return; if (stat.isDirectory()) { output.push({path: name, kind: "directory", mode: stat.mode & 0o777}); for (const child of (await readdir(path)).sort()) await visit(join(path, child), name ? `${name}/${child}` : child); @@ -207,7 +211,7 @@ export async function checkpointArtifacts(value: unknown): Promise { for (const member of before[c.id] as Member[]) { if (member.kind !== "file") continue; const source = (await lstat(c.source_path)).isDirectory() ? join(c.source_path, member.path) : c.source_path + member.path; - if (await sqliteFile(source) || await sqliteSidecar(source, ["-wal", "-shm", "-journal"])) continue; + if (await sqliteFile(source) || (member.path !== "" && await sqliteSidecar(source, ["-wal", "-shm", "-journal"]))) continue; const copied = (artifacts[c.id] as Member[]).find(m => m.path === member.path && m.kind === "file"); if (!copied || copied.sha256 !== member.sha256 || copied.size !== member.size || copied.mode !== member.mode) { throw new Error("checkpoint copied bytes do not match the reviewed source"); diff --git a/tests/control_plane_ts/checkpoint_artifacts.test.ts b/tests/control_plane_ts/checkpoint_artifacts.test.ts index 3365b9bb6f..684084a66d 100644 --- a/tests/control_plane_ts/checkpoint_artifacts.test.ts +++ b/tests/control_plane_ts/checkpoint_artifacts.test.ts @@ -138,3 +138,42 @@ test("a committed WAL change after the probe invalidates capture, including a st finally {copy.close();} } finally {writer.close();} }); + +test("read-only snapshot housekeeping on a closed WAL database is not source movement", async t => { + const {root, request} = await fixture(t), {driver} = sqliteAuthorityRuntime(); + await mkdir(join(root, "state")); + const path = join(root, "state/authority.sqlite"), writer = new driver.DatabaseSync(path); + writer.exec("PRAGMA journal_mode=WAL; CREATE TABLE evidence(value TEXT); INSERT INTO evidence VALUES('retained')"); + writer.close(); + assert.deepEqual(await readdir(join(root, "state")), ["authority.sqlite"]); + const components = [{id: "state", source_path: join(root, "state")}]; + const probe = await checkpointArtifacts({action: "probe", components}); + const captured = await checkpointArtifacts({...request, components, expected_source_sha256: probe.source_sha256}); + assert.equal(captured.source_stability_verified, true); + const after = await checkpointArtifacts({action: "probe", components}); + assert.equal(after.source_sha256, probe.source_sha256); + assert.deepEqual(await readdir(join(root, "captured/components/state")), ["authority.sqlite"]); + const changed = new driver.DatabaseSync(path); + try { + changed.exec("INSERT INTO evidence VALUES('new committed data')"); + await assert.rejects(checkpointArtifacts({...request, components, destination: join(root, "must-not-exist"), + expected_source_sha256: probe.source_sha256}), /source moved/); + } finally {changed.close();} + const copy = new driver.DatabaseSync(join(root, "captured/components/state/authority.sqlite"), {readOnly: true}); + try {assert.equal(copy.prepare("SELECT COUNT(*) AS n FROM evidence").get()?.n, 1);} + finally {copy.close();} +}); + +test("an explicitly selected sidecar remains a byte artifact rather than disappearing from its witness", async t => { + const {root, request} = await fixture(t), {driver} = sqliteAuthorityRuntime(); + const path = join(root, "authority.sqlite"), writer = new driver.DatabaseSync(path); + try { + writer.exec("PRAGMA journal_mode=WAL; CREATE TABLE evidence(value TEXT)"); + const components = [{id: "explicit_index", source_path: path + "-shm"}]; + const probe = await checkpointArtifacts({action: "probe", components}); + assert.equal((probe.source_witness as Record).explicit_index.length, 1); + const captured = await checkpointArtifacts({...request, components, expected_source_sha256: probe.source_sha256}); + assert.equal(captured.integrity_verified, true); + assert.deepEqual(await readFile(join(root, "captured/components/explicit_index")), await readFile(path + "-shm")); + } finally {writer.close();} +}); From 32911433efb1cca453cfcf56342a3f5ecf60f804 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Mon, 12 Oct 2026 21:58:14 +0800 Subject: [PATCH 06/69] docs(rfc): propose capture before each fresh replan Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- .../hierarchical-agent-stride-control-v0.md | 23 ++++++++++++++++++- ...rarchical-agent-stride-control-v0.zh-CN.md | 20 ++++++++++++++-- 2 files changed, 40 insertions(+), 3 deletions(-) diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md index 7db1565ff5..b91aa2ea7c 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md @@ -643,6 +643,26 @@ consistency and comparison requirements, not a second restore authority. Before capture, declare the point by elapsed research budget or a semantic event; do not choose it retrospectively for a favorable score. A later point such as ten hours is an experiment setting, not a product default or automatic pivot. + +The proposed automatic capture policy is **off by default; when explicitly +enabled, capture at each fresh replan node before its strategy decision**. Reuse +the canonical replan obligation identity: repeated context reads or retries of +that node do not create another point. Ordinary work Turns do not trigger a full +capture. A successful point records the replan identity, the completed Host +cursor, source revisions and current strategy version; these are evidence +bindings, not authority. Branches can then compare different replan decisions +from the same retained research state, including a decision to keep the route. + +The trigger still waits for the settled boundary below and must precede the +replan decision and its new work. A replan can occur inside one Host wake, so an +after-wake hook alone cannot implement this ordering. The replan and Host owners +must provide that handoff without an extra model call or work settlement. If the +boundary cannot be qualified, record the missed point and continue under the +existing execution contract; do not substitute a later state for that node. +Measure capture time and storage per node before choosing compression, shared +immutable members or retention limits. Retention must preserve declared study +points and their branches rather than silently selecting favorable outcomes. + Record the actual boundary and any skew: - native conversation cursor and a Host-qualified export/fork path; copied log @@ -694,7 +714,8 @@ The initial implementation slice is [inert artifact capture](../../../benchmark/ The benchmark adapter requests it after the existing Turn settlement; the typed artifact owner seals explicit files/directories and committed SQLite snapshots, rechecks source witnesses and verifies an isolated materialization. Its manifest -always declares the research point unqualified. Participant/effect reconciliation, +always declares the research point unqualified. This explicit one-point hook +does not implement the proposed automatic replan trigger. Participant/effect reconciliation, native session reconstruction, fresh state adoption and preserved evaluator opportunities remain separate acceptance work; operator metadata cannot certify them. diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md index f743cb8cf8..849a458f6f 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md @@ -548,7 +548,22 @@ Host 会话 owner 和 artifact provider；本 RFC 定义研究实验的一致性 不新建 restore 权威。 捕获前按已分配研究预算或语义事件声明位置，不事后挑分数有利的点。十小时等 -中后期位置属于实验设置，不是产品默认值或自动转向条件。记录实际边界与 skew： +中后期位置属于实验设置，不是产品默认值或自动转向条件。 + +自动捕获策略提案是**默认关闭；显式开启后，在每个新的 replan 节点、策略决策 +之前捕获**。复用 canonical replan obligation 身份：重复读取 context 或重试 +同一节点不会生成另一研究点。普通工作 Turn 不触发完整捕获。成功研究点绑定 +replan 身份、已完成 Host cursor、来源 revision 及当前策略版本；这些是证据 +绑定，不是权限。不同分支可从同一份积累比较不同 replan 决策，包括保留原路线。 + +触发仍须等待下述已结算边界，并先于 replan 决策及其新工作。一次 Host wake +内部也可能发生 replan，因此单独的 after-wake hook 不能实现这个顺序。Replan +与 Host owner 须提供该衔接，不增加模型调用或工作结算。边界无法资格化时记录 +缺失研究点，按既有执行契约继续；不能用更晚状态代替该节点。先记录每个节点 +的捕获时间和存储量，再决定压缩、共享不可变成员或保留上限。保留策略须保护 +已声明研究点及其分支，不能静默挑选有利结果。 + +记录实际边界与 skew： - 原生会话 cursor 和已资格化的 Host export/fork 路径；复制日志不证明能恢复 session； @@ -585,7 +600,8 @@ Host adapter 须在固定版本验证原生历史重建，包括 compaction 和 第一项实现切片是 [inert 产物捕获](../../../benchmark/README.md#inert-checkpoint-capture)。 Benchmark adapter 在既有 Turn 结算后请求捕获；typed 产物 owner 封存显式文件/ 目录及已提交 SQLite 快照，复查来源 witness 并验证隔离物化。Manifest 始终声明 -研究点尚未资格化。参与者/效果 reconciliation、原生会话重建、新状态采用及评测 +研究点尚未资格化。这项显式单点 hook 尚未实现上述自动 replan 触发。参与者/ +效果 reconciliation、原生会话重建、新状态采用及评测 机会继承仍各自待验收；operator metadata 不能认证这些条件。 inert verification unit 不授予执行权。目标侧使用新的实验/Host 身份，通过既有 From cbddd56517c4683a1506cea90902fe173ff5ddf3 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Mon, 12 Oct 2026 22:12:33 +0800 Subject: [PATCH 07/69] docs(rfc): qualify complete native history and path profile Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- .../rfcs/hierarchical-agent-stride-control-v0.md | 14 ++++++++++++-- .../hierarchical-agent-stride-control-v0.zh-CN.md | 10 ++++++++-- 2 files changed, 20 insertions(+), 4 deletions(-) diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md index b91aa2ea7c..ecf47b3b4c 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md @@ -665,8 +665,9 @@ points and their branches rather than silently selecting favorable outcomes. Record the actual boundary and any skew: -- native conversation cursor and a Host-qualified export/fork path; copied log - text does not establish a resumable session; +- native conversation cursor and a Host-qualified export/fork path, including + compaction records, paginated history and the session index needed for native + readback; copied log text does not establish a resumable session; - settled LoopX research evidence, original goal/priorities, prior decisions, scoped negative results and unresolved questions; - exact working code, necessary authorized untracked files, incumbent artifact @@ -705,6 +706,15 @@ insufficient. Compare the reconstructed prefixes and excluded later history, not just returned session identifiers. Keep provider-specific migration and storage handling outside the generic consistency decision. +The Host profile must also declare path portability. A native index can retain +absolute guest paths: qualification must either preserve those paths inside a +fresh isolated environment or use the Host's supported import/rebinding API. +Editing native database rows by hand does not qualify import. Bind the effective +Host configuration and installed tool/skill versions separately from credentials; +reauthentication remains with the existing owner. Full research state means the +declared participants needed to resume useful work, not a copy of every file or +the model's unobservable internal state. + Historical conversation truncation cannot reconstruct earlier code, research state or feedback consumption from their terminal versions. Without a qualified contemporaneous capture of those participants, such a probe establishes Host diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md index 849a458f6f..93fd2c06e9 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md @@ -565,8 +565,8 @@ replan 身份、已完成 Host cursor、来源 revision 及当前策略版本； 记录实际边界与 skew： -- 原生会话 cursor 和已资格化的 Host export/fork 路径；复制日志不证明能恢复 - session； +- 原生会话 cursor 和已资格化的 Host export/fork 路径，包括 compaction 记录、 + 分页历史及原生读回所需的 session 索引；复制日志不证明能恢复 session； - 已结算的 LoopX 研究证据、原目标/优先级、既有决定、有范围负结果及未解问题； - 精确工作源码、必要且获授权的未跟踪文件、incumbent 与其 qualifying result； 尚未测量的候选与 incumbent 分开保留； @@ -594,6 +594,12 @@ Host adapter 须在固定版本验证原生历史重建，包括 compaction 和 任意文件路径或日志解析成功并不充分。比较重建的历史前缀及被排除的后续历史， 不只检查返回了 session id。Provider 特有迁移与存储处理不进入通用一致性决策。 +Host profile 还须声明路径可移植性。原生索引可能保留绝对 guest 路径：资格化 +须在新隔离环境保留这些路径，或使用 Host 支持的 import/rebinding API；手改 +原生数据库行不能资格化导入。有效 Host 配置及已安装工具/skill 版本须与凭证 +分开绑定，重新认证仍由既有 owner 负责。完整研究状态是继续有效工作所需的 +已声明参与者，不是复制所有文件或模型不可观测的内部状态。 + 截短历史会话不能从终局版本重建当时的代码、研究状态或反馈消费。没有其他参与者 同时捕获且已资格化的状态时，这类探针只证明 Host 可行性，不能进入同状态干预比较。 From da4e182aeb51dc5175b6d1afea76f17414540721 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Mon, 12 Oct 2026 23:31:01 +0800 Subject: [PATCH 08/69] feat(benchmark): capture pre-replan native and evaluation participants Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/edgebench/checkpoint_judge.py | 119 ++++++++++++ benchmark/edgebench/run.py | 14 +- benchmark/runtime/checkpoint.py | 169 +++++++++++++++++- benchmark/runtime/native_checkpoint.py | 167 +++++++++++++++++ benchmark/runtime/sforge.py | 22 ++- benchmark/runtime/sforge_checkpoint.py | 30 ++++ benchmark/runtime/worker.py | 4 + benchmark/tests/test_checkpoint_judge.py | 86 +++++++++ .../tests/test_edgebench_feedback_docker.py | 122 +++++++++++++ benchmark/tests/test_native_checkpoint.py | 79 ++++++++ benchmark/tests/test_sforge_runtime.py | 56 ++++++ .../runtime/checkpoint_artifacts.ts | 33 ++++ .../test_research_checkpoint_capture.py | 107 ++++++++++- .../checkpoint_artifacts.test.ts | 30 ++++ 14 files changed, 1025 insertions(+), 13 deletions(-) create mode 100644 benchmark/edgebench/checkpoint_judge.py create mode 100644 benchmark/runtime/native_checkpoint.py create mode 100644 benchmark/runtime/sforge_checkpoint.py create mode 100644 benchmark/tests/test_checkpoint_judge.py create mode 100644 benchmark/tests/test_native_checkpoint.py diff --git a/benchmark/edgebench/checkpoint_judge.py b/benchmark/edgebench/checkpoint_judge.py new file mode 100644 index 0000000000..d1ed8dc1ec --- /dev/null +++ b/benchmark/edgebench/checkpoint_judge.py @@ -0,0 +1,119 @@ +"""Opt-in read-only checkpoint participant over the native SForge judge. + +Native admission, grading, history and ranking retain their owners. This +adapter exports a quiescent registration's public history and opportunity +clock; it cannot register, submit, restore or extend a trial. +""" +from __future__ import annotations + +import argparse +import hashlib +import json +import time +import uuid +from dataclasses import replace +from pathlib import Path +from benchmark.runtime.sforge_checkpoint import ROUTE, SCHEMA + + +def create_app(config, *, checkpoint_capture=False): + if type(checkpoint_capture) is not bool: + raise ValueError("checkpoint_capture must be an explicit boolean") + from sforge.harness.judge_server import SubmissionStatus, create_app as native_app + app = native_app(config) + if not checkpoint_capture: + return app # Preserve the native route/admission surface when disabled. + from fastapi import HTTPException + from pydantic import BaseModel, ConfigDict + import sforge + + state = getattr(app.state, "judge", None) + if state is None or not all(hasattr(state, key) for key in ( + "_admission_lock", "_outstanding_by_run", "_tokens_lock", "_history_lock")): + raise ValueError("Checkpoint capture requires a pinned SForge with its native admission owner exposed") + epoch = uuid.uuid4().hex + source = hashlib.sha256() + source_root = Path(sforge.__file__).parent + for path in sorted(source_root.rglob("*.py")): + source.update(path.relative_to(source_root).as_posix().encode() + b"\0") + source.update(hashlib.sha256(path.read_bytes()).digest()) + tasks = {key: { + "native_source_sha256": source.hexdigest(), + "task_spec_sha256": hashlib.sha256((config.tasks_dir / f"{key}.json").read_bytes()).hexdigest(), + "judge_image_key": task.judge_image_key, + "selection": task.judge.selection, "score_direction": task.judge.score_direction, + } for key, task in state.tasks.items()} + + class ReadRequest(BaseModel): + model_config = ConfigDict(extra="forbid") + token: str + + # FastAPI resolves annotations in module globals; keep this factory's + # private request type concrete rather than a forward reference. + def capture(request): + # Reuse the native lock order and outstanding admission counter. A + # pending grade or unexplained consumed round is never a complete point. + with state._admission_lock, state._tokens_lock, state._history_lock: + info = state.tokens.get(request.token) + if info is None: + raise HTTPException(403, "Unknown registration") + run, task = info["run_id"], info["task_id"] + if state._closed or state.tasks[task].game_mode: + raise HTTPException(409, "Checkpoint requires an open non-game registration") + if state._outstanding_by_run.get((run, task), 0): + raise HTTPException(409, "Accepted evaluation is still outstanding") + if info["next_auto"] != 1: + raise HTTPException(409, "This checkpoint profile requires no automatic evaluations") + history = state.run_history.get(state._run_key(run, task), []) + rounds = [entry.get("round") for entry in history] + expected = {f"agent-{index}" for index in range(1, info["next_agent"])} + if (len(rounds) != len(expected) or set(rounds) != expected or + any(entry.get("status") not in {SubmissionStatus.COMPLETED, SubmissionStatus.ERROR} for entry in history)): + raise HTTPException(409, "Consumed rounds and released history are not reconciled") + results = {} + for entry in history: + identifier = entry.get("submission_id") + native = state.submissions.get(identifier) + if (native is None or native.get("run_id") != run or native.get("task_id") != task or + native.get("round") != entry["round"] or native.get("status") != entry["status"]): + raise HTTPException(409, "Released result details are unavailable or differ") + results[identifier] = {key: native.get(key) for key in ("status", "report", "error")} + observed = time.time() + last = info["last_agent_submit_at"] + cooldown = info["submission_cooldown"] + remaining = max(0, last + (cooldown or 0) - observed) if last else 0 + packet = { + "schema_version": SCHEMA, "judge_epoch": epoch, + "run_id": run, "task_id": task, "evaluator": tasks[task], + "observed_at_epoch": observed, "last_agent_submit_at_epoch": last, + "submission_cooldown_seconds": cooldown, + "remaining_cooldown_seconds": remaining, + "max_agent_submissions": info["max_agent_submissions"], + "next_agent_round": info["next_agent"], "next_auto_round": info["next_auto"], + "released_history": history, "released_results": results, "outstanding_evaluations": 0, + "execution_authority_granted": False, + } + return json.loads(json.dumps(packet)) # No live references or tokens leave the lock. + capture.__annotations__["request"] = ReadRequest + app.post(ROUTE)(capture) + return app + + +def main(argv=None): + import uvicorn + from sforge.harness.config import load_config + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--checkpoint-capture", action="store_true") + parser.add_argument("--host", default="127.0.0.1") + parser.add_argument("--port", type=int, default=8080) + parser.add_argument("--tasks-dir", type=Path) + parser.add_argument("--log-dir", type=Path) + args = parser.parse_args(argv) + config = load_config() + overrides = {key: getattr(args, key) for key in ("tasks_dir", "log_dir") if getattr(args, key) is not None} + uvicorn.run(create_app(replace(config, **overrides), checkpoint_capture=args.checkpoint_capture), + host=args.host, port=args.port, access_log=False) + + +if __name__ == "__main__": + main() diff --git a/benchmark/edgebench/run.py b/benchmark/edgebench/run.py index 05b2bf6c3b..73208368f0 100644 --- a/benchmark/edgebench/run.py +++ b/benchmark/edgebench/run.py @@ -119,6 +119,8 @@ def main(argv=None): help="Explicit completed-Todo cadence ablation for heartbeat profiles") parser.add_argument("--stride-strategy", choices=stride_strategy_options(), help="Explicit heartbeat Goal-review strategy; omitted keeps the ordinary off treatment") + parser.add_argument("--checkpoint-request", type=Path, + help="Explicit host JSON declaration staged into a heartbeat worker; omitted disables capture") parser.add_argument("--eval-interval", type=int, help="Native default: 0 (no periodic evaluation); other modes use task defaults; explicit 0 disables") parser.add_argument("--submission-cooldown", type=int, @@ -142,6 +144,12 @@ def main(argv=None): parser.error("Replan cadence requires a heartbeat worker") if args.stride_strategy is not None and not args.worker.startswith("heartbeat-"): parser.error("--stride-strategy requires a heartbeat worker") + checkpoint_request = None + if args.checkpoint_request is not None: + if not args.worker.startswith("heartbeat-"): + parser.error("--checkpoint-request requires a heartbeat worker") + from benchmark.runtime.checkpoint import read_checkpoint_request + _, checkpoint_request = read_checkpoint_request(str(args.checkpoint_request)) # One directory is one attempt: never reuse native registration or overwrite # source/profile evidence after an ambiguous launch. trial = args.log_dir / "runs" / args.run_id / args.task @@ -196,7 +204,9 @@ def main(argv=None): replan_after_turns=args.replan_after_turns, replan_after_todos=args.replan_after_todos, **({"stride_strategy": args.stride_strategy} - if args.stride_strategy is not None else {})) + if args.stride_strategy is not None else {}), + **({"checkpoint_request": checkpoint_request} + if checkpoint_request is not None else {})) if args.api_proxy_url: agent.default_api_base_url = args.api_proxy_url logger = logging.getLogger("edgebench-runtime") @@ -242,6 +252,8 @@ def main(argv=None): if agent.replan_after_todos is not None else {}), **({"stride_strategy": args.stride_strategy} if args.stride_strategy is not None else {}), + **({"checkpoint_request_sha256": hashlib.sha256(checkpoint_request).hexdigest()} + if checkpoint_request is not None else {}), } receipt_path = trial / "runtime-receipt.json" receipt_path.write_text(json.dumps(receipt, indent=2)) diff --git a/benchmark/runtime/checkpoint.py b/benchmark/runtime/checkpoint.py index e74a486fb4..bdc4d1392c 100644 --- a/benchmark/runtime/checkpoint.py +++ b/benchmark/runtime/checkpoint.py @@ -9,6 +9,7 @@ import hashlib import json import math +import subprocess import time from pathlib import Path, PurePosixPath from typing import Any @@ -26,10 +27,144 @@ def normalize_checkpoint_request_path(value: str) -> str: return value -def capture_after_wake(env: dict[str, str], receipt: dict[str, Any]) -> None: +def read_checkpoint_request(path: str) -> tuple[dict[str, Any], bytes]: + raw = Path(path).read_bytes() + return parse_checkpoint_request(raw), raw + + +def parse_checkpoint_request(raw: bytes) -> dict[str, Any]: + """Validate the same explicit declaration before staging and before capture.""" + request = json.loads(raw) + if (not isinstance(request, dict) or + set(request) - {"destination", "components", "point_metadata", "native_session", "evaluation_opportunity", "trigger"} or + not {"destination", "components", "point_metadata"}.issubset(request)): + raise ValueError("checkpoint request requires destination, components and point_metadata") + if "native_session" in request and type(request["native_session"]) is not bool: + raise ValueError("checkpoint native_session must be an explicit boolean") + if "evaluation_opportunity" in request and type(request["evaluation_opportunity"]) is not bool: + raise ValueError("checkpoint evaluation_opportunity must be an explicit boolean") + if "trigger" in request and request["trigger"] != "fresh_replan": + raise ValueError("checkpoint trigger must be fresh_replan or omitted") + if not isinstance(request["components"], list) or not request["components"]: + raise ValueError("checkpoint requires explicit components") + if not isinstance(request["point_metadata"], dict): + raise ValueError("checkpoint point_metadata must be an object") + return request + + +def capture_before_replan(env: dict[str, str], receipt: dict[str, Any], body: str) -> str: + """Preload the original guard once, then capture before invoking the model. + + The typed owner selects only an admitted replan identity. The prior wake's + existing settlement owner still qualifies the source boundary. A missed + point is explicit and cannot be replaced by an after-decision capture. + """ + path = env.get("LOOPX_CHECKPOINT_REQUEST") + if not path: + return body + request, raw = read_checkpoint_request(path) + if request.get("trigger") != "fresh_replan": + return body + if receipt.get("mode") != "heartbeat" or receipt.get("stage") != "execute": + raise ValueError("fresh_replan checkpoint trigger requires a heartbeat execute wake") + started = time.monotonic() + guard_path = Path(env["LOOPX_WAKE_LOG_DIR"]) / receipt["turn_id"] / "checkpoint-start-guard.json" + command = [env["LOOPX_CLI"], "--format", "json", "--registry", env["LOOPX_REGISTRY"], + "--runtime-root", env["LOOPX_RUNTIME_ROOT"], "quota", "should-run", + "--goal-id", env["LOOPX_GOAL_ID"], "--agent-id", env["LOOPX_AGENT_ID"], + "--runtime-profile", "generic_cli", "--turn-instance-id", receipt["turn_id"], + "--available-capability", "shell", "--available-capability", "filesystem_write"] + guard_run = subprocess.run(command, cwd=env["LOOPX_PROJECT"], env=env, capture_output=True, + text=True, timeout=120, check=True) + guard = json.loads(guard_run.stdout) + if not isinstance(guard, dict) or guard.get("ok") is not True: + raise ValueError("checkpoint requires a successful original quota guard") + guard_path.write_text(guard_run.stdout, encoding="utf-8") + guard_path.chmod(0o600) + preloaded_body = (f"Host checkpoint preflight has already executed this wake's original quota guard once. " + f"Read the complete packet at {guard_path} before work, then follow its interaction_contract. " + "Do not repeat the initial guard below; use its selection or recovery command when required.\n\n" + body) + receipt["checkpoint"] = {"captured": False, "research_point_qualified": False, "phase": "before_replan"} + try: + intent = effect_runtime_result("runtime.checkpoint_artifacts", { + "action": "select_replan", "quota": guard, "goal_id": env["LOOPX_GOAL_ID"], + "agent_id": env["LOOPX_AGENT_ID"], "turn_instance_id": receipt["turn_id"], + }) + if not intent["capture"]: + receipt["checkpoint"]["reason"] = intent["reason"] + else: + root = Path(env["LOOPX_WAKE_LOG_DIR"]).parent / "checkpoints" + requested_root = Path(request["destination"]) + if not requested_root.is_absolute() or requested_root != requested_root.resolve() or requested_root != root: + raise ValueError("fresh_replan destination must be this trial's checkpoints directory") + destination = root / intent["obligation_id"] + if destination.parent != root or destination != destination.resolve(): + raise ValueError("replan checkpoint identity must select a direct child") + if destination.exists(): + # Readback of the first point, never overwrite it with later work. + manifest = json.loads((destination / "checkpoint.json").read_text()) + metadata = manifest["point_metadata"] + if (metadata["request_sha256"] != hashlib.sha256(raw).hexdigest() or + metadata["operator_metadata"]["replan"]["obligation_id"] != intent["obligation_id"]): + raise ValueError("existing replan point has a different declaration") + verified = effect_runtime_result("runtime.checkpoint_artifacts", { + "action": "verify", "root": str(destination), "expected_sha256": manifest["sha256"], + }) + receipt["checkpoint"].update(reason="replan_already_captured", destination=str(destination), **verified) + else: + # A failed first capture is a missed point. Once its model is + # allowed to decide/work, a retry must not relabel later state + # as the original pre-decision checkpoint. + attempts = root / ".replan-attempts" + attempts.mkdir(parents=True, exist_ok=True, mode=0o700) + if attempts != attempts.resolve(): + raise ValueError("replan attempt directory must not contain symlinks") + attempt_path = attempts / (intent["obligation_id"] + ".json") + attempt = {"obligation_id": intent["obligation_id"], "turn_id": receipt["turn_id"], + "request_sha256": hashlib.sha256(raw).hexdigest()} + try: + with attempt_path.open("x", encoding="utf-8") as marker: + marker.write(json.dumps(attempt)) + attempt_path.chmod(0o600) + except FileExistsError: + if attempt_path.is_symlink(): + raise ValueError("replan attempt must not be a symlink") + prior_attempt = json.loads(attempt_path.read_text()) + if (prior_attempt["obligation_id"] != attempt["obligation_id"] or + prior_attempt["request_sha256"] != attempt["request_sha256"]): + raise ValueError("existing replan attempt has a different declaration") + receipt["checkpoint"]["reason"] = "pre_decision_capture_already_attempted" + return preloaded_body + prior_paths = sorted(Path(env["LOOPX_WAKE_LOG_DIR"]).glob("wake-*/receipt.json")) + prior_paths = [p for p in prior_paths if p.parent.name != receipt["turn_id"]] + if not prior_paths: + raise ValueError("replan capture has no completed prior Host wake") + prior = json.loads(prior_paths[-1].read_text()) + if prior.get("stage") != "execute" or prior.get("mode") != "heartbeat" or not prior.get("ok"): + raise ValueError("replan capture requires the latest successful work wake") + effective_request = {**request, "destination": str(destination), + "point_metadata": {**request["point_metadata"], "replan": intent, + "capture_order": "before_model_replan_decision"}} + effective_request.pop("trigger") + capture_after_wake(env, prior, supplied_request=(effective_request, raw)) + receipt["checkpoint"].update(prior["checkpoint"], phase="before_replan") + receipt["checkpoint"]["source_turn_id"] = prior["turn_id"] + except Exception as error: + receipt["checkpoint"].update(reason="capture_failed", error_kind=type(error).__name__, error=str(error)[:500]) + finally: + receipt["checkpoint"]["elapsed_seconds"] = time.monotonic() - started + # The model consumes exactly the owner's original decision and keeps its + # selection, settlement, required-read and recovery instructions intact. + return preloaded_body + + +def capture_after_wake(env: dict[str, str], receipt: dict[str, Any], *, + supplied_request: tuple[dict[str, Any], bytes] | None = None) -> None: request_path = env.get("LOOPX_CHECKPOINT_REQUEST") if not request_path: return # Feature-off path performs no additional read or write. + if supplied_request is None and (receipt.get("checkpoint") or {}).get("phase") == "before_replan": + return # The after-wake hook cannot replace the pre-decision point. receipt["checkpoint"] = {"captured": False, "research_point_qualified": False} if (not receipt.get("ok") or receipt.get("stage") != "execute" or receipt.get("mode") not in {"heartbeat", "turn"} or receipt.get("budget_exhausted")): @@ -47,10 +182,13 @@ def capture_after_wake(env: dict[str, str], receipt: dict[str, Any]) -> None: if deadline <= time.time(): receipt["checkpoint"]["reason"] = "capture_window_closed" return - raw = Path(request_path).read_bytes() - request = json.loads(raw) - if not isinstance(request, dict) or set(request) != {"destination", "components", "point_metadata"}: - raise ValueError("checkpoint request requires destination, components and point_metadata") + if supplied_request is None: + request, raw = read_checkpoint_request(request_path) + else: + request, raw = supplied_request + if request.get("trigger") == "fresh_replan": + receipt["checkpoint"]["reason"] = "pre_replan_hook_required" + return # This is a private evidence artifact below the runner's log owner. destination = Path(request["destination"]) root = Path(env["LOOPX_WAKE_LOG_DIR"]).parent / "checkpoints" @@ -80,14 +218,31 @@ def capture_after_wake(env: dict[str, str], receipt: dict[str, Any]) -> None: "session": receipt.get("session"), "request_sha256": hashlib.sha256(raw).hexdigest(), } + components = request["components"] + if request.get("evaluation_opportunity"): + from .sforge_checkpoint import capture_opportunity + + opportunity = capture_opportunity(env, timeout=max(.1, deadline - time.time())) + evidence = Path(env["LOOPX_WAKE_LOG_DIR"]) / receipt["turn_id"] / "checkpoint-opportunity.json" + evidence.write_text(json.dumps(opportunity), encoding="utf-8") + evidence.chmod(0o600) + components = [*components, {"id": "evaluation_opportunity", "source_path": str(evidence)}] + if request.get("native_session"): + from .native_checkpoint import native_session_checkpoint + + session = receipt.get("session") or {} + native = native_session_checkpoint(env, session.get("session_id", ""), + timeout=max(.1, deadline - time.time())) + components = [*components, *native["components"]] + metadata["native_session"] = native["metadata"] probe = effect_runtime_result("runtime.checkpoint_artifacts", { - "action": "probe", "components": request["components"], + "action": "probe", "components": components, "deadline_epoch_ms": deadline * 1000, }, timeout=max(.1, deadline - time.time()), retry_safe=False) destination.parent.mkdir(parents=True, exist_ok=True) captured = effect_runtime_result("runtime.checkpoint_artifacts", { "action": "capture", "destination": str(destination), - "components": request["components"], "point_metadata": metadata, + "components": components, "point_metadata": metadata, "expected_source_sha256": probe["source_sha256"], "deadline_epoch_ms": deadline * 1000, }, timeout=max(.1, deadline - time.time()), retry_safe=False) diff --git a/benchmark/runtime/native_checkpoint.py b/benchmark/runtime/native_checkpoint.py new file mode 100644 index 0000000000..69a57a81f9 --- /dev/null +++ b/benchmark/runtime/native_checkpoint.py @@ -0,0 +1,167 @@ +"""Codex-specific checkpoint inventory and native history readback. + +This transport adapter does not decide research validity or adopt authority. +The artifact owner seals its declared native-store members after the Host exits. +""" +from __future__ import annotations + +import hashlib +import json +import re +import shutil +import subprocess +import time +from pathlib import Path +from typing import Any + +from loopx.capabilities.benchmark_toolkit.native_codex_goal import StdioNativeGoalTransport + + +def _complete_history(request, session_id: str) -> list[dict[str, Any]]: + turns = [] + cursor = None + seen = set() + while True: + page = request("thread/turns/list", { + "threadId": session_id, "sortDirection": "asc", "itemsView": "full", "limit": 100, + **({"cursor": cursor} if cursor else {}), + }) + if not isinstance(page.get("data"), list): + raise ValueError("native checkpoint history page is unavailable") + turns.extend(page["data"]) + cursor = page.get("nextCursor") + if not cursor: + break + if cursor in seen: + raise ValueError("native checkpoint repeated history cursor") + seen.add(cursor) + if not turns or any(t.get("status") not in {"completed", "failed", "interrupted"} or not t.get("id") for t in turns): + raise ValueError("native checkpoint requires complete retained turns") + if turns[-1].get("status") != "completed": + raise ValueError("native checkpoint requires a completed final cursor") + ids = [t["id"] for t in turns] + if len(set(ids)) != len(ids): + raise ValueError("native checkpoint history contains duplicate turns") + return turns + + +def native_session_checkpoint(env: dict[str, str], session_id: str, *, timeout: float) -> dict[str, Any]: + home = Path(env["CODEX_HOME"]) + if not session_id or timeout <= 0: + raise ValueError("native checkpoint requires a session and an open capture window") + deadline = time.monotonic() + timeout + version = subprocess.check_output([env["CODEX_BIN"], "--version"], env=env, text=True, + timeout=timeout).strip() + # Native storage is opaque. Retain both rollout and paginated/index stores; + # auth, logs, memories and the symlinked skill installation are not members. + with StdioNativeGoalTransport.spawn( + [env["CODEX_BIN"], "-c", "features.goals=false", "app-server", "--listen", "stdio://"], + cwd=env["LOOPX_PROJECT"], env=env, stderr=subprocess.DEVNULL, + response_timeout_sec=timeout, + ) as transport: + def request(method, params): + remaining = deadline - time.monotonic() + if remaining <= 0: + raise TimeoutError("native checkpoint window closed") + transport.response_timeout_sec = remaining + return transport.request(method, params) + request("initialize", {"clientInfo": {"name": "loopx_checkpoint", "version": "0.1.0"}, + "capabilities": {"experimentalApi": True}}) + transport.notify("initialized", {}) + readback = request("thread/read", {"threadId": session_id, "includeTurns": False}) + if readback.get("thread", {}).get("id") != session_id: + raise ValueError("native checkpoint session identity mismatch") + turns = _complete_history(request, session_id) + ids = [t["id"] for t in turns] + members = [] + for entry in sorted(home.iterdir()): + if entry.name in {"sessions", "archived_sessions", "session_index.jsonl"} or re.fullmatch( + r"(?:state|thread_history)_[0-9]+\.sqlite", entry.name + ): + members.append({"id": "native_" + entry.name.replace(".", "_"), "source_path": str(entry)}) + if not any(m["id"].startswith("native_state_") for m in members): + raise ValueError("native checkpoint session index is unavailable") + encoded = json.dumps(turns, sort_keys=True, separators=(",", ":")).encode() + return {"components": members, "metadata": { + "session_id": session_id, "completed_turn_id": ids[-1], "turn_ids": ids, + "visible_history_sha256": hashlib.sha256(encoded).hexdigest(), "host_version": version, + "guest_home": str(home), "guest_workspace": env["LOOPX_PROJECT"], + "path_profile": "preserve_guest_paths_in_isolated_environment", + "config_sha256": hashlib.sha256((home / "config.toml").read_bytes()).hexdigest(), + "store_members": [{"id": c["id"], "name": Path(c["source_path"]).name} for c in members], + "model_calls": 0, "execution_authority_granted": False, + }} + + +def fork_native_checkpoint(env: dict[str, str], unit: Path, expected_sha256: str, + config_path: Path, *, timeout: float) -> dict[str, Any]: + """Reconstruct one native branch in a new, fixed-path isolated environment. + + No model starts and no LoopX binding is registered. Existing session/Goal + owners must separately admit this new session before useful continuation. + """ + from loopx.control_plane.effect_runtime import effect_runtime_result + + deadline = time.monotonic() + timeout + effect_runtime_result("runtime.checkpoint_artifacts", { + "action": "verify", "root": str(unit), "expected_sha256": expected_sha256, + }, timeout=timeout) + manifest = json.loads((unit / "checkpoint.json").read_text()) + native = manifest["point_metadata"]["native_session"] + home = Path(env["CODEX_HOME"]) + if str(home) != native["guest_home"] or env["LOOPX_PROJECT"] != native["guest_workspace"]: + raise ValueError("native checkpoint requires its qualified fixed guest paths") + if home.exists() or home.is_symlink(): + raise ValueError("native checkpoint destination home must be new") + config = config_path.read_bytes() + if hashlib.sha256(config).hexdigest() != native["config_sha256"]: + raise ValueError("native checkpoint effective configuration differs") + version = subprocess.check_output([env["CODEX_BIN"], "--version"], env=env, text=True, + timeout=max(.1, deadline - time.monotonic())).strip() + if version != native["host_version"]: + raise ValueError("native checkpoint Host version differs") + names = set() + for member in native["store_members"]: + name = member["name"] + if (name in names or member["id"] != "native_" + name.replace(".", "_") or + not (name in {"sessions", "archived_sessions", "session_index.jsonl"} or + re.fullmatch(r"(?:state|thread_history)_[0-9]+\.sqlite", name))): + raise ValueError("native checkpoint member inventory is invalid") + names.add(name) + home.mkdir(parents=True, mode=0o700) + for member in native["store_members"]: + source, destination = unit / "components" / member["id"], home / member["name"] + if source.is_dir(): + shutil.copytree(source, destination) + else: + shutil.copy2(source, destination) + (home / "config.toml").write_bytes(config) + with StdioNativeGoalTransport.spawn( + [env["CODEX_BIN"], "-c", "features.goals=false", "app-server", "--listen", "stdio://"], + cwd=env["LOOPX_PROJECT"], env=env, stderr=subprocess.DEVNULL, + response_timeout_sec=max(.1, deadline - time.monotonic()), + ) as transport: + def request(method, params): + remaining = deadline - time.monotonic() + if remaining <= 0: + raise TimeoutError("native checkpoint reconstruction window closed") + transport.response_timeout_sec = remaining + return transport.request(method, params) + request("initialize", {"clientInfo": {"name": "loopx_checkpoint", "version": "0.1.0"}, + "capabilities": {"experimentalApi": True}}) + transport.notify("initialized", {}) + fork = request("thread/fork", {"threadId": native["session_id"], + "lastTurnId": native["completed_turn_id"], "cwd": env["LOOPX_PROJECT"], + "sandbox": "read-only", "approvalPolicy": "never", "deferGoalContinuation": True, + "excludeTurns": True}) + session = fork.get("thread", {}).get("id") + if not session or session == native["session_id"]: + raise ValueError("native checkpoint did not create a distinct session") + turns = _complete_history(request, session) + encoded = json.dumps(turns, sort_keys=True, separators=(",", ":")).encode() + if ([t["id"] for t in turns] != native["turn_ids"] or + hashlib.sha256(encoded).hexdigest() != native["visible_history_sha256"]): + raise ValueError("native checkpoint reconstructed history differs") + return {"session_id": session, "completed_turn_id": native["completed_turn_id"], + "visible_history_sha256": native["visible_history_sha256"], "history_verified": True, + "model_calls": 0, "execution_authority_granted": False, "continuation_qualified": False} diff --git a/benchmark/runtime/sforge.py b/benchmark/runtime/sforge.py index a7a968d94d..97df4341ef 100644 --- a/benchmark/runtime/sforge.py +++ b/benchmark/runtime/sforge.py @@ -21,7 +21,7 @@ from .codex import DEFAULT_REPLAN_AFTER_TURNS, Execution, prepare_codex_home from .codex_offline import CodexOffline, stage_portable_python from .harbor import ( - BenchmarkCodex, _GOAL_ID, _PYTHON, _SCHEDULER_STATE, _SRC, + BenchmarkCodex, _CONTROL, _GOAL_ID, _PYTHON, _SCHEDULER_STATE, _SRC, ) from .scheduler import worker_command @@ -92,7 +92,8 @@ def __init__(self, config, *, profile: str, cwd: str, replan_after_turns: int | None = None, replan_after_todos: int | None = None, stride_strategy: str | None = None, - checkpoint_request_path: str | None = None): + checkpoint_request_path: str | None = None, + checkpoint_request: bytes | None = None): super().__init__(config) if profile not in PROFILES: raise ValueError("Unknown benchmark worker profile") @@ -138,6 +139,13 @@ def __init__(self, config, *, profile: str, cwd: str, stride_strategy = normalize_stride_strategy(stride_strategy) self.stride_strategy = stride_strategy self.checkpoint_request_path = None + self.checkpoint_request = checkpoint_request + if checkpoint_request is not None: + if checkpoint_request_path is not None or not profile.startswith("heartbeat-"): + raise ValueError("checkpoint_request requires a heartbeat profile and no guest-path override") + from .checkpoint import parse_checkpoint_request + parse_checkpoint_request(checkpoint_request) + checkpoint_request_path = f"{_CONTROL}/checkpoint-request.json" if checkpoint_request_path is not None: if not profile.startswith("heartbeat-"): raise ValueError("checkpoint_request_path requires a heartbeat profile") @@ -233,6 +241,16 @@ def _install_worker(self, backend, handle, log_dir, logger): task_objective=self.task_objective, ) asyncio.run(self.runtime.install(self.environment)) + if self.checkpoint_request is not None: + with tempfile.TemporaryDirectory(prefix="benchmark-checkpoint-") as directory: + declaration = Path(directory) / "request.json" + declaration.write_bytes(self.checkpoint_request) + asyncio.run(self.environment.upload_file(declaration, self.checkpoint_request_path)) + result = backend.exec_run(handle, ["/bin/bash", "-c", + f"chown agent:agent {shlex.quote(self.checkpoint_request_path)} && " + f"chmod 0600 {shlex.quote(self.checkpoint_request_path)}"], user="root") + if result.exit_code: + raise RuntimeError("Could not protect the explicit checkpoint declaration") # Prompt replacement describes the policy; the feedback mode owns access. if self.feedback != "native": result = backend.exec_run(handle, ["rm", "-f", "/usr/local/bin/sforge-submit"], user="root") diff --git a/benchmark/runtime/sforge_checkpoint.py b/benchmark/runtime/sforge_checkpoint.py new file mode 100644 index 0000000000..ce1d7e36eb --- /dev/null +++ b/benchmark/runtime/sforge_checkpoint.py @@ -0,0 +1,30 @@ +"""SForge checkpoint read transport, included in the existing worker payload.""" +from __future__ import annotations + +import json +import math +import time +from urllib.request import Request, urlopen + +SCHEMA = "sforge_checkpoint_opportunity_v0" # Local provider artifact, not a LoopX authority contract. +ROUTE = "/api/v1/checkpoint/opportunity" + + +def capture_opportunity(env, *, timeout): + """Read only this worker's registration; retain its original deadline.""" + deadline = float(env["LOOPX_PHASE_DEADLINE_EPOCH"]) + if not math.isfinite(deadline) or deadline <= time.time() or timeout <= 0: + raise ValueError("checkpoint requires an open original trial deadline") + request = Request(env["SFORGE_JUDGE_URL"].rstrip("/") + ROUTE, + data=json.dumps({"token": env["SFORGE_TOKEN"]}).encode(), + headers={"Content-Type": "application/json"}, method="POST") + with urlopen(request, timeout=timeout) as response: + packet = json.load(response) + if (packet.get("schema_version") != SCHEMA or packet.get("execution_authority_granted") is not False or + packet.get("outstanding_evaluations") != 0): + raise ValueError("native opportunity checkpoint readback is unavailable") + observed = time.time() + return {**packet, "source_trial_deadline_epoch": deadline, + "client_observed_at_epoch": observed, + "remaining_trial_seconds": max(0, deadline - observed), + "continuation_qualified": False} diff --git a/benchmark/runtime/worker.py b/benchmark/runtime/worker.py index 5118ecd879..598ea35577 100644 --- a/benchmark/runtime/worker.py +++ b/benchmark/runtime/worker.py @@ -317,6 +317,10 @@ def run_once(env: dict[str, str]) -> dict: from benchmark.runtime.sessions import BenchmarkSessionWake session_wake = BenchmarkSessionWake(env, execution, receipt) + if stage == "execute" and execution.mode == "heartbeat" and env.get("LOOPX_CHECKPOINT_REQUEST"): + from benchmark.runtime.checkpoint import capture_before_replan + + body = capture_before_replan(env, receipt, body) with (wake / "stderr.log").open("w") as stderr: if execution.native_goal and stage == "execute": run_native_goal(env, execution, body, receipt, stderr) diff --git a/benchmark/tests/test_checkpoint_judge.py b/benchmark/tests/test_checkpoint_judge.py new file mode 100644 index 0000000000..fe93fd40f5 --- /dev/null +++ b/benchmark/tests/test_checkpoint_judge.py @@ -0,0 +1,86 @@ +"""Native owner semantics; real grading/transport is qualified by the Docker smoke.""" +import json +from types import SimpleNamespace + +import pytest + + +@pytest.fixture +def native_config(tmp_path, monkeypatch): + pytest.importorskip("sforge") + from sforge.harness import judge_server + from sforge.harness.config import SForgeConfig + tasks = tmp_path / "tasks" + tasks.mkdir() + (tasks / "BENCHMARK.yaml").write_text("name: fixture\nbase_images: {python: {}}\n") + (tasks / "fixture.json").write_text(json.dumps({ + "task_id": "fixture", "name": "Synthetic checkpoint", "base_image": "python", + "platform": "linux/arm64", "cwd": "/tmp", "internet": False, + "submit_paths": ["solver.py"], "submit_exclude": [], + "work": {"image_tag": "fixture", "agent_query": "Synthetic fixture"}, + "judge": {"image_tag": "fixture", "eval_timeout": 15, "parser": "structured_json", + "selection": "score_first", "score_direction": "maximize", "eval_cmd": "true"}, + })) + monkeypatch.setattr(judge_server, "create_backend_from_config", lambda config: SimpleNamespace()) + return SForgeConfig(tasks_dir=tasks, log_dir=tmp_path) + + +def test_disabled_extension_preserves_the_native_route_surface(native_config): + from benchmark.edgebench.checkpoint_judge import create_app + from sforge.harness.judge_server import create_app as native_app + first, second = native_app(native_config), create_app(native_config) + try: + assert [(r.path, r.methods) for r in first.routes] == [(r.path, r.methods) for r in second.routes] + finally: + first.state.judge.close() + second.state.judge.close() + + +def test_capture_retains_consumed_round_history_and_cooldown_without_mutation(native_config): + from fastapi.testclient import TestClient + from benchmark.edgebench.checkpoint_judge import create_app, ROUTE + app = create_app(native_config, checkpoint_capture=True) + state = app.state.judge + with TestClient(app) as client: + token = state.register_session("fixture", "source", max_agent_submissions=12, submission_cooldown=3600) + state.consume_round(token) + state._record_submission("source", "accepted-error", "fixture", "agent-1", None, + "synthetic evaluation failure", status="error") + state.submissions["accepted-error"] = {"run_id": "source", "task_id": "fixture", "round": "agent-1", + "status": "error", "report": None, "error": "synthetic evaluation failure"} + before = state.resolve_token(token) + result = client.post(ROUTE, json={"token": token}) + assert result.status_code == 200 + point = result.json() + assert point["next_agent_round"] == 2 and point["next_auto_round"] == 1 + assert point["max_agent_submissions"] == 12 + assert 0 < point["remaining_cooldown_seconds"] <= 3600 + assert point["last_agent_submit_at_epoch"] == before["last_agent_submit_at"] + assert point["released_history"] == state.get_run_history("source", "fixture")["entries"] + assert point["released_history"][0]["status"] == "error" + assert point["released_results"]["accepted-error"]["error"] == "synthetic evaluation failure" + assert point["execution_authority_granted"] is False + assert token not in json.dumps(point) and "admin_secret" not in json.dumps(point) + assert state.resolve_token(token) == before + assert client.post(ROUTE, json={"token": token}).json()["judge_epoch"] == point["judge_epoch"] + + +@pytest.mark.parametrize("gap", ["pending", "unreconciled", "automatic", "closed"]) +def test_incomplete_opportunity_state_cannot_be_a_checkpoint(native_config, gap): + from fastapi.testclient import TestClient + from benchmark.edgebench.checkpoint_judge import create_app, ROUTE + app = create_app(native_config, checkpoint_capture=True) + state = app.state.judge + with TestClient(app) as client: + token = state.register_session("fixture", "source", submission_cooldown=3600) + if gap == "pending": + state._outstanding_by_run[("source", "fixture")] = 1 + elif gap == "unreconciled": + state.consume_round(token) + elif gap == "automatic": + state.consume_round(token, "auto") + else: + state._closed = True + assert client.post(ROUTE, json={"token": token}).status_code == 409 + assert client.post(ROUTE, json={"token": "unregistered"}).status_code == 403 + assert client.post(ROUTE, json={"token": token, "restore": True}).status_code == 422 diff --git a/benchmark/tests/test_edgebench_feedback_docker.py b/benchmark/tests/test_edgebench_feedback_docker.py index 921c067cc2..58a6f2af95 100644 --- a/benchmark/tests/test_edgebench_feedback_docker.py +++ b/benchmark/tests/test_edgebench_feedback_docker.py @@ -93,6 +93,128 @@ def test_native_execution_disposition_uses_real_docker_exit(tmp_path, agent_imag backend.client.close() +def test_sforge_checkpoint_declaration_is_a_private_guest_artifact(tmp_path, agent_image, monkeypatch): + from sforge.harness.config import SForgeConfig + from benchmark.runtime.sforge import SForgeWorker, BenchmarkCodex + from benchmark.runtime.sforge_backend import RecordingDockerBackend + + # Existing installation prepares this directory. Substitute installation, + # while retaining the production upload, permissions and worker environment. + async def installed(self, environment): + result = await environment.exec( + "mkdir -p /opt/loopx-benchmark/control", user="root") + assert result.return_code == 0 + monkeypatch.setattr(BenchmarkCodex, "install", installed) + monkeypatch.setenv("CODEX_AUTH_JSON_PATH", "/synthetic-unused-credential") + declaration = b'{"destination":"/logs/agent/checkpoints","trigger":"fresh_replan","native_session":true,"components":[{"id":"code","source_path":"/tmp/solver.py"}],"point_metadata":{}}' + worker = SForgeWorker(SForgeConfig(agent_model="fixture", agent_effort="xhigh"), + profile="heartbeat-resume", cwd="/tmp", checkpoint_request=declaration) + backend = RecordingDockerBackend(log_dir=tmp_path / "collected", + logger=logging.getLogger("checkpoint-staging-smoke"), oauth_proxy=True) + handle = backend.create_container(agent_image.id, "checkpoint-stage-" + uuid.uuid4().hex[:10], + cpu_limit=1, mem_limit="256m") + backend.start_container(handle) + try: + worker.install_stop_hook(backend, handle, tmp_path, None) + path = worker.runtime._worker_env(cwd="/tmp")["LOOPX_CHECKPOINT_REQUEST"] + readback = backend.exec_run(handle, ["cat", path], user="agent") + assert readback.exit_code == 0 and readback.output.encode() == declaration + mode = backend.exec_run(handle, ["stat", "-c", "%a:%U", path], user="root") + assert mode.exit_code == 0 and mode.output.strip() == "600:agent" + finally: + backend.cleanup_container(handle) + backend.client.close() + + +def test_checkpoint_opportunity_uses_native_admission_and_real_grade(tmp_path, monkeypatch): + import docker + import requests + import uvicorn + from sforge.harness import judge_server + from sforge.harness.config import SForgeConfig + from benchmark.edgebench.checkpoint_judge import create_app, ROUTE + from benchmark.runtime.sforge_checkpoint import capture_opportunity + + name = "checkpoint-judge-" + uuid.uuid4().hex[:10] + tag = f"{name}.judge.fixture:fixture" + client = docker.from_env() + client.images.get("python:3.12-slim").tag(f"{name}.judge.fixture", tag="fixture") + tasks = tmp_path / "tasks" + tasks.mkdir() + (tasks / "BENCHMARK.yaml").write_text(f"name: {name}\nbase_images: {{python: {{}}}}\n") + (tasks / "fixture.json").write_text(json.dumps({ + "task_id": "fixture", "name": "Synthetic checkpoint", "base_image": "python", + "platform": "linux/arm64", "cwd": "/tmp", "internet": False, + "submit_paths": ["candidate.json"], "submit_exclude": [], + "work": {"image_tag": "fixture", "agent_query": "Synthetic fixture"}, + "judge": {"image_tag": "fixture", "eval_timeout": 15, "parser": "structured_json", + "selection": "score_first", "score_direction": "maximize", + "eval_cmd": "python3 -c 'import json; print(json.dumps(dict(score=7, valid=True)))'"}, + })) + config = SForgeConfig(tasks_dir=tasks, log_dir=tmp_path, judge_cpu_limit=1, judge_mem_limit="256m", + judge_max_outstanding_per_run=1) + finish_grade = threading.Event() + original_grade = judge_server.judge_submission + def held_grade(**kwargs): + assert finish_grade.wait(60), "Synthetic native grade was not released" + return original_grade(**kwargs) + monkeypatch.setattr(judge_server, "judge_submission", held_grade) + app = create_app(config, checkpoint_capture=True) + with socket.socket() as port_probe: + port_probe.bind(("127.0.0.1", 0)) + port = port_probe.getsockname()[1] + url = f"http://127.0.0.1:{port}" + server = uvicorn.Server(uvicorn.Config(app, host="127.0.0.1", port=port, + log_level="error", access_log=False)) + thread = threading.Thread(target=server.run) + thread.start() + try: + wait_for(lambda: server.started, timeout=20) + registered = requests.post(url + "/api/v1/register", json={"task_id": "fixture", + "run_id": "synthetic-source", "admin_secret": app.state.judge.admin_secret, + "submission_cooldown": 3600, "max_agent_submissions": 12}, timeout=10) + registered.raise_for_status() + token = registered.json()["token"] + payload = io.BytesIO() + with tarfile.open(fileobj=payload, mode="w:gz") as archive: + member = tarfile.TarInfo("candidate.json") + data = b"{}" + member.size = len(data) + archive.addfile(member, io.BytesIO(data)) + accepted = requests.post(url + "/api/v1/submit", data={"token": token}, + files={"archive": ("candidate.tar.gz", payload.getvalue())}, timeout=15) + accepted.raise_for_status() + assert accepted.json()["round_id"] == "agent-1" + assert requests.post(url + ROUTE, json={"token": token}, timeout=10).status_code == 409 + finish_grade.set() + wait_for(lambda: not app.state.judge._outstanding_by_run, timeout=60) + released = requests.post(url + ROUTE, json={"token": token}, timeout=10) + assert released.status_code == 200, released.json() + env = {"SFORGE_JUDGE_URL": url, "SFORGE_TOKEN": token, + "LOOPX_PHASE_DEADLINE_EPOCH": str(time.time() + 100)} + point = capture_opportunity(env, timeout=10) + assert point["released_history"][0]["score"] == 7 + assert point["next_agent_round"] == 2 and 0 < point["remaining_cooldown_seconds"] <= 3600 + assert 0 < point["remaining_trial_seconds"] <= 100 + assert point["continuation_qualified"] is False + before = app.state.judge.resolve_token(token) + assert capture_opportunity(env, timeout=10)["judge_epoch"] == point["judge_epoch"] + assert app.state.judge.resolve_token(token) == before + cooled = requests.post(url + "/api/v1/submit", data={"token": token}, + files={"archive": ("candidate.tar.gz", payload.getvalue())}, timeout=15) + assert cooled.status_code == 429 # Reading the checkpoint cannot refill admission. + assert app.state.judge.resolve_token(token) == before + with pytest.raises(ValueError, match="original trial deadline"): + capture_opportunity({**env, "LOOPX_PHASE_DEADLINE_EPOCH": "0"}, timeout=10) + finally: + finish_grade.set() + server.should_exit = True + thread.join(timeout=60) + assert not thread.is_alive(), "Synthetic checkpoint service did not drain" + client.images.remove(tag) + client.close() + + def test_sforge_trial_deadline_requires_execution_entry(tmp_path, agent_image, monkeypatch): """Exercise the actual formatted shell, including its inner GNU timeout.""" from sforge.harness.config import SForgeConfig diff --git a/benchmark/tests/test_native_checkpoint.py b/benchmark/tests/test_native_checkpoint.py new file mode 100644 index 0000000000..c6e7108da3 --- /dev/null +++ b/benchmark/tests/test_native_checkpoint.py @@ -0,0 +1,79 @@ +"""Native checkpoint history must be complete and cannot become execution authority.""" +import hashlib +import json + +import pytest + +from benchmark.runtime.native_checkpoint import native_session_checkpoint + + +def fixture(tmp_path, monkeypatch, pages): + home = tmp_path / "native" + home.mkdir() + for name in ("state_5.sqlite", "thread_history_1.sqlite", "config.toml", "auth.json", "logs_1.sqlite"): + (home / name).write_text("synthetic private fixture") + (home / "sessions").mkdir() + (home / "skills").symlink_to(home / "sessions", target_is_directory=True) + calls = [] + + class Native: + response_timeout_sec = 1 + + def __enter__(self): + return self + + def __exit__(self, *args): + pass + + def notify(self, method, params): + calls.append(method) + + def request(self, method, params): + calls.append(method) + if method == "initialize": + return {} + if method == "thread/read": + return {"thread": {"id": "fixture-session"}} + assert method == "thread/turns/list" # No model or fork is requested. + return pages.pop(0) + + monkeypatch.setattr("benchmark.runtime.native_checkpoint.StdioNativeGoalTransport.spawn", lambda *a, **k: Native()) + monkeypatch.setattr("benchmark.runtime.native_checkpoint.subprocess.check_output", lambda *a, **k: "codex-cli fixture") + return {"CODEX_HOME": str(home), "CODEX_BIN": "/fixture/codex", "LOOPX_PROJECT": str(tmp_path)}, calls + + +def test_all_native_pages_and_stores_are_retained_without_credentials(tmp_path, monkeypatch): + turns = [{"id": "before-compaction", "status": "completed", "items": [{"type": "compaction"}]}, + {"id": "after-compaction", "status": "completed", "items": []}] + env, calls = fixture(tmp_path, monkeypatch, [{"data": turns[:1], "nextCursor": "next"}, + {"data": turns[1:], "nextCursor": None}]) + result = native_session_checkpoint(env, "fixture-session", timeout=10) + metadata = result["metadata"] + assert metadata["turn_ids"] == [t["id"] for t in turns] + assert metadata["completed_turn_id"] == "after-compaction" + assert metadata["visible_history_sha256"] == hashlib.sha256( + json.dumps(turns, sort_keys=True, separators=(",", ":")).encode()).hexdigest() + assert metadata["execution_authority_granted"] is False + assert {c["id"] for c in result["components"]} == { + "native_sessions", "native_state_5_sqlite", "native_thread_history_1_sqlite"} + assert calls == ["initialize", "initialized", "thread/read", "thread/turns/list", "thread/turns/list"] + + +@pytest.mark.parametrize("pages,reason", [ + ([{"data": [], "nextCursor": None}], "complete retained"), + ([{"data": [{"id": "active", "status": "inProgress"}], "nextCursor": None}], "complete retained"), + ([{"data": [{"id": "failed", "status": "failed"}], "nextCursor": None}], "completed final"), + ([{"data": [], "nextCursor": "same"}, {"data": [], "nextCursor": "same"}], "repeated history"), + ([{"data": [{"id": "same", "status": "completed"}] * 2, "nextCursor": None}], "duplicate turns"), +]) +def test_incomplete_or_ambiguous_native_history_is_not_a_checkpoint(tmp_path, monkeypatch, pages, reason): + env, _ = fixture(tmp_path, monkeypatch, pages) + with pytest.raises(ValueError, match=reason): + native_session_checkpoint(env, "fixture-session", timeout=10) + + +def test_foreign_native_session_is_rejected_before_history(tmp_path, monkeypatch): + env, calls = fixture(tmp_path, monkeypatch, []) + with pytest.raises(ValueError, match="identity mismatch"): + native_session_checkpoint(env, "foreign-session", timeout=10) + assert "thread/turns/list" not in calls diff --git a/benchmark/tests/test_sforge_runtime.py b/benchmark/tests/test_sforge_runtime.py index bbf566f5e8..10eebe86f3 100644 --- a/benchmark/tests/test_sforge_runtime.py +++ b/benchmark/tests/test_sforge_runtime.py @@ -479,6 +479,8 @@ def test_envelope_rejects_incompatible_sforge_worker(profile): ("official", ["--stride-strategy", "off"]), ("native-goal", ["--stride-strategy", "evidence_conditioned_v1"]), ("heartbeat-resume", ["--stride-strategy", "future_policy"]), + ("official", ["--checkpoint-request", "missing.json"]), + ("native-goal", ["--checkpoint-request", "missing.json"]), ]) def test_edgebench_rejects_invalid_profile_settings_before_creating_trial(tmp_path, profile, options): pytest.importorskip('sforge') @@ -492,6 +494,60 @@ def test_edgebench_rejects_invalid_profile_settings_before_creating_trial(tmp_pa assert not (tmp_path / 'runs').exists() +@pytest.mark.parametrize("protect_fails", [False, True]) +def test_host_checkpoint_declaration_is_staged_verbatim_and_protected(tmp_path, monkeypatch, protect_fails): + pytest.importorskip("sforge") + pytest.importorskip("harbor") + from types import SimpleNamespace + from sforge.harness.config import SForgeConfig + from benchmark.runtime.sforge import SForgeWorker, BenchmarkCodex + monkeypatch.setenv("CODEX_AUTH_JSON_PATH", "/synthetic-credential") + async def installed(self, environment): + pass # Exercise the installation transport; no model or evaluator. + monkeypatch.setattr(BenchmarkCodex, "install", installed) + staged, commands = [], [] + def copy(handle, source, target): + staged.append((str(target), source.read_bytes())) + def command(handle, cmd, **kwargs): + commands.append((cmd, kwargs)) + return SimpleNamespace(exit_code=int(protect_fails)) + backend = SimpleNamespace(copy_to_container=copy, exec_run=command) + declaration = b'{"destination":"/logs/agent/checkpoints","trigger":"fresh_replan", "native_session":true,"components":[{"id":"code","source_path":"/task"}],"point_metadata":{}}' + config = SForgeConfig(agent_model="fixture", agent_effort="xhigh") + worker = SForgeWorker(config, profile="heartbeat-resume", cwd="/task", checkpoint_request=declaration) + if protect_fails: + with pytest.raises(RuntimeError, match="protect the explicit checkpoint declaration"): + worker.install_stop_hook(backend, None, tmp_path, None) + assert not (tmp_path / "worker-profile.json").exists() + else: + worker.install_stop_hook(backend, None, tmp_path, None) + assert json.loads((tmp_path / "worker-profile.json").read_text())["checkpoint_capture_enabled"] + guest_path = "/opt/loopx-benchmark/control/checkpoint-request.json" + assert staged == [(guest_path, declaration)] + assert worker.runtime._worker_env(cwd="/task")["LOOPX_CHECKPOINT_REQUEST"] == guest_path + assert len(commands) == 1 and commands[0][1]["user"] == "root" + assert "chmod 0600" in commands[0][0][-1] + with pytest.raises(ValueError, match="no guest-path override"): + SForgeWorker(config, profile="heartbeat-resume", cwd="/task", checkpoint_request=declaration, + checkpoint_request_path="/other") + with pytest.raises(ValueError, match="requires destination"): + SForgeWorker(config, profile="heartbeat-resume", cwd="/task", checkpoint_request=b'{}') + + +def test_invalid_checkpoint_declaration_rejects_before_trial_registration(tmp_path): + pytest.importorskip("sforge") + pytest.importorskip("harbor") + from benchmark.edgebench.run import main + declaration = tmp_path / "request.json" + declaration.write_text('{"components":[]}') + with pytest.raises(ValueError, match="requires destination"): + main(["--task", "fixture", "--tasks-dir", str(tmp_path), "--log-dir", str(tmp_path), + "--run-id", "invalid", "--worker", "heartbeat-resume", "--model", "fixture", + "--effort", "xhigh", "--judge-url", "http://127.0.0.1:9999", + "--checkpoint-request", str(declaration)]) + assert not (tmp_path / "runs").exists() + + @pytest.mark.parametrize("cadence_args,field,count", [ ([], "replan_after_effective_turns", 6), (["--replan-after-turns", "2"], "replan_after_effective_turns", 2), diff --git a/loopx/control_plane/runtime/checkpoint_artifacts.ts b/loopx/control_plane/runtime/checkpoint_artifacts.ts index cf7c59b610..d881207a64 100644 --- a/loopx/control_plane/runtime/checkpoint_artifacts.ts +++ b/loopx/control_plane/runtime/checkpoint_artifacts.ts @@ -11,6 +11,9 @@ import {requireJsonObject, requireNonEmptyString} from "../runtime_decode.ts"; import {canonicalAuthoritySha256, hasExactAuthorityKeys} from "../coordination/authority_store_codec.ts"; import {snapshotSqliteDatabase} from "../coordination/sqlite_backup.ts"; import {durableWriteJson, publishNewDirectory} from "../effect_runtime_io.ts"; +import {settlementIdentity} from "../effect_program.ts"; +import {normalizeHeartbeatReplanObligationId, normalizeHeartbeatTodoId} from "../quota/heartbeat_receipt_identity.ts"; +import {EffectiveAction} from "../quota/effective_action.generated.ts"; const SCHEMA = "loopx_checkpoint_artifact_set_v0"; type Component = {id: string; source_path: string}; @@ -162,6 +165,36 @@ async function verify(root: string, expected: unknown): Promise { } export async function checkpointArtifacts(value: unknown): Promise { const request = requireJsonObject(value, "checkpoint artifacts request"); + if (request.action === "select_replan") { + if (!hasExactAuthorityKeys(request, ["action", "quota", "goal_id", "agent_id", "turn_instance_id"])) { + throw new Error("checkpoint replan selection fields are invalid"); + } + const quota = requireJsonObject(request.quota, "checkpoint quota readback"); + const goal = requireNonEmptyString(request.goal_id, "checkpoint goal"); + const agent = requireNonEmptyString(request.agent_id, "checkpoint agent"); + const turn = requireNonEmptyString(request.turn_instance_id, "checkpoint Turn"); + if (quota.ok !== true || quota.goal_id !== goal) throw new Error("checkpoint requires a matching current quota readback"); + if (quota.effective_action !== EffectiveAction.AUTONOMOUS_REPLAN_REQUIRED) { + return {capture: false, reason: "no_fresh_replan", execution_authority_granted: false}; + } + const packet = requireJsonObject(quota.replan_action_packet, "checkpoint replan packet"); + const id = normalizeHeartbeatReplanObligationId(packet.obligation_id); + const receipt = requireJsonObject(quota.heartbeat_receipt, "checkpoint heartbeat receipt"); + const identity = requireJsonObject(receipt.settlement_identity, "checkpoint settlement identity"); + const todo = normalizeHeartbeatTodoId(identity.todo_id); + const boundReplan = normalizeHeartbeatReplanObligationId(identity.replan_obligation_id); + const semanticReplan = normalizeHeartbeatReplanObligationId(receipt.semantic_replan_obligation_id); + if (packet.schema_version !== "replan_action_packet_v0" || packet.decision !== "replan_required" || + !id || (receipt.status !== "committed" && receipt.status !== "replayed") || receipt.turn_instance_id !== turn || + identity.goal_id !== goal || identity.agent_id !== agent || identity.turn_instance_id !== turn || + (todo ? boundReplan !== null || semanticReplan !== id : boundReplan !== id) || + identity.effect_id !== settlementIdentity({goal_id: goal, agent_id: agent, todo_id: todo, + replan_obligation_id: boundReplan, turn_instance_id: turn}).effect_id) { + throw new Error("checkpoint replan does not match the current admitted identity"); + } + return {capture: true, obligation_id: id, settlement_effect_id: identity.effect_id, + execution_authority_granted: false}; + } const fields = request.action === "probe" ? ["action", "components"] : request.action === "verify" ? ["action", "root", "expected_sha256"] : request.action === "capture" ? ["action", "components", "destination", "point_metadata", "expected_source_sha256"] : diff --git a/tests/control_plane/test_research_checkpoint_capture.py b/tests/control_plane/test_research_checkpoint_capture.py index c9105138ea..055b653d6d 100644 --- a/tests/control_plane/test_research_checkpoint_capture.py +++ b/tests/control_plane/test_research_checkpoint_capture.py @@ -1,12 +1,13 @@ """Inert checkpoint capture reuses the real settled-Turn source owner.""" from pathlib import Path import json +import pytest import time import test_quota_settlement_cli as cli from canonical_authority_fixture import isolate_sqlite_runtime -from benchmark.runtime.checkpoint import capture_after_wake +from benchmark.runtime.checkpoint import capture_after_wake, capture_before_replan from loopx.control_plane.effect_runtime import effect_runtime_result @@ -18,6 +19,7 @@ def forbidden(*args, **kwargs): receipt = {"ok": True, "stage": "execute", "mode": "heartbeat"} capture_after_wake({}, receipt) assert receipt == {"ok": True, "stage": "execute", "mode": "heartbeat"} + assert capture_before_replan({}, receipt, "original body") == "original body" failed = {"ok": False, "stage": "execute", "mode": "heartbeat"} capture_after_wake({"LOOPX_CHECKPOINT_REQUEST": "not-readable"}, failed) assert failed["ok"] is False @@ -39,7 +41,8 @@ def forbidden(*args, **kwargs): assert receipt["checkpoint"]["reason"] == "capture_failed" -def test_real_cli_settlement_capture_verification_and_inert_materialization(tmp_path, monkeypatch): +@pytest.mark.parametrize("opportunity", [False, True]) +def test_real_cli_settlement_capture_verification_and_inert_materialization(tmp_path, monkeypatch, opportunity): isolate_sqlite_runtime(tmp_path, monkeypatch) project, runtime, registry = cli._write_fixture(tmp_path) binding = ("--agent-id", cli.AGENT_ID, "--todo-id", cli.TODO_ID, "--turn-instance-id", cli.TURN_ID) @@ -51,13 +54,22 @@ def run(*args): logs = tmp_path / "logs" wake = logs / "wakes" wake.mkdir(parents=True) + (wake / cli.TURN_ID).mkdir() + observed = [] + opportunity_packet = {"meaning": "synthetic opportunity participant; no admission"} + def export(env, *, timeout): + assert 0 < timeout <= 30 + observed.append(True) + return opportunity_packet + monkeypatch.setattr("benchmark.runtime.sforge_checkpoint.capture_opportunity", export) (project / "solution.py").write_text("retained_candidate = 7\n") request = tmp_path / "request.json" target = logs / "checkpoints" / "point" request.write_text(json.dumps({"destination": str(target), "components": [ {"id": "code", "source_path": str(project / "solution.py")}, {"id": "research", "source_path": str(project / f".codex/goals/{cli.GOAL_ID}/ACTIVE_GOAL_STATE.md")}, - ], "point_metadata": {"meaning": "synthetic capture; no model/session or evaluator qualification"}})) + ], "point_metadata": {"meaning": "synthetic capture; no model/session or evaluator qualification"}, + **({"evaluation_opportunity": True} if opportunity else {})})) env = {"LOOPX_CHECKPOINT_REQUEST": str(request), "LOOPX_WAKE_LOG_DIR": str(wake), "LOOPX_RUNTIME_ROOT": str(runtime), "LOOPX_GOAL_ID": cli.GOAL_ID, "LOOPX_AGENT_ID": cli.AGENT_ID, "LOOPX_REGISTRY": str(registry)} @@ -65,6 +77,7 @@ def run(*args): capture_after_wake(env, receipt) assert receipt["checkpoint"]["reason"] == "owned_turn_not_settled" assert not target.exists() + assert observed == [] vision = tmp_path / "vision.json" vision.write_text(json.dumps({"schema_version": "goal_vision_replan_contract_v0", "state": "vision_patch_proposed", "vision_patch": {"acceptance_summary": "Synthetic source fixture checked; checkpoint work remains."}, @@ -81,6 +94,9 @@ def run(*args): assert receipt["checkpoint"].get("captured") is True, receipt assert receipt["checkpoint"]["research_point_qualified"] is False assert goal_state.read_bytes() == retained_goal + if opportunity: + assert json.loads((target / "components/evaluation_opportunity").read_text()) == opportunity_packet + digest = receipt["checkpoint"]["sha256"] restored = effect_runtime_result("runtime.checkpoint_artifacts", {"action": "materialize", "root": str(target), "expected_sha256": digest, "destination": str(tmp_path / "inert")}) @@ -91,3 +107,88 @@ def run(*args): assert receipt["checkpoint"]["reason"] == "destination_already_present_no_replay" assert (target / "checkpoint.json").read_bytes() == before assert goal_state.read_bytes() == retained_goal + assert len(observed) == int(opportunity) + + +@pytest.mark.parametrize("capture_fails", [False, True]) +def test_real_pre_replan_capture_is_before_decision_and_same_node_is_not_recaptured(tmp_path, monkeypatch, capture_fails): + import os + import sys + + isolate_sqlite_runtime(tmp_path, monkeypatch) + project, runtime, registry = cli._write_fixture(tmp_path) + binding = ("--agent-id", cli.AGENT_ID, "--todo-id", cli.TODO_ID, "--turn-instance-id", cli.TURN_ID) + + def run(*args): + code, result = cli._run_cli(registry, runtime, *args) + assert code == 0, result + return result + + run("quota", "should-run", "--goal-id", cli.GOAL_ID, "--codex-app", *binding) + vision = tmp_path / "vision.json" + vision.write_text(json.dumps({"schema_version": "goal_vision_replan_contract_v0", "state": "vision_patch_proposed", + "vision_patch": {"acceptance_summary": "Synthetic source fixture checked; capture work remains."}, + "path_delta": {"outcome": "continue", "prior_assumption": "Source fixture is available.", + "observed_reality": "Isolated source fixture verified.", "retained": ["Capture qualification"], + "evidence_refs": ["evidence:synthetic-pre-replan"]}})) + run("refresh-state", "--goal-id", cli.GOAL_ID, *binding, "--classification", "pre_replan_fixture_verified", + "--delivery-batch-scale", "multi_surface", "--delivery-outcome", "outcome_progress", "--agent-vision-json", str(vision)) + run("quota", "spend-slot", "--goal-id", cli.GOAL_ID, *binding, "--source", "heartbeat", "--execute") + index = runtime / "goals" / cli.GOAL_ID / "runs/index.jsonl" + original_settlement = index.read_bytes() + cli._configure_autonomous_replan_fixture(project, runtime, registry) + # This helper supplies synthetic repeated observations; retain the real + # source settlement history instead of replacing its authoritative readback. + index.write_bytes(original_settlement + index.read_bytes()) + logs = tmp_path / "logs" + wakes = logs / "wakes" + prior = wakes / "wake-100-prior" + prior.mkdir(parents=True) + prior_receipt = {"turn_id": cli.TURN_ID, "mode": "heartbeat", "stage": "execute", "ok": True} + (prior / "receipt.json").write_text(json.dumps(prior_receipt)) + solution = project / "solution.py" + solution.write_text("retained_candidate = 7\n") + request = tmp_path / "request.json" + request.write_text(json.dumps({"trigger": "fresh_replan", "destination": str(logs / "checkpoints"), + "components": [{"id": "code", "source_path": str(solution)}], + "point_metadata": {"purpose": "synthetic original settlement and replan ordering"}})) + env = {**os.environ, "LOOPX_CLI": str(Path(sys.executable).parent / "loopx"), + "LOOPX_PROJECT": str(project), "LOOPX_CHECKPOINT_REQUEST": str(request), + "LOOPX_WAKE_LOG_DIR": str(wakes), "LOOPX_RUNTIME_ROOT": str(runtime), + "LOOPX_GOAL_ID": cli.GOAL_ID, "LOOPX_AGENT_ID": cli.AGENT_ID, "LOOPX_REGISTRY": str(registry)} + receipt = {"turn_id": "synthetic-replan-point", "mode": "heartbeat", "stage": "execute", "ok": False} + (wakes / receipt["turn_id"]).mkdir() + attempted = [] + if capture_fails: + def fail_probe(method, params, **kwargs): + if params["action"] == "probe": + attempted.append(True) + raise RuntimeError("synthetic capture failure") + return effect_runtime_result(method, params, **kwargs) + monkeypatch.setattr("benchmark.runtime.checkpoint.effect_runtime_result", fail_probe) + body = capture_before_replan(env, receipt, "original instructions") + if capture_fails: + assert receipt["checkpoint"]["reason"] == "capture_failed", receipt + solution.write_text("later_decision = 11\n") + again = {"turn_id": receipt["turn_id"], "mode": "heartbeat", "stage": "execute", "ok": False} + capture_before_replan(env, again, "original instructions") + assert again["checkpoint"]["reason"] == "pre_decision_capture_already_attempted", again + assert attempted == [True] and not list((logs / "checkpoints").glob("replan-*/checkpoint.json")) + assert body.endswith("original instructions") and receipt["ok"] is False + return + assert receipt["checkpoint"]["captured"] is True, json.dumps(receipt) + target = Path(receipt["checkpoint"]["destination"]) + captured = (target / "checkpoint.json").read_bytes() + assert (target / "components/code").read_text() == "retained_candidate = 7\n" + assert body.endswith("original instructions") and "original quota guard once" in body + assert receipt["ok"] is False # Capturing cannot declare the new work completed. + assert (prior / "receipt.json").read_text() == json.dumps(prior_receipt) + # The model's subsequent decision/work must not replace the pre-decision point. + solution.write_text("new_route = 11\n") + capture_after_wake(env, receipt) + assert (target / "checkpoint.json").read_bytes() == captured + # Reentry on the exact original guard reads back one point, even if code moved. + again = {"turn_id": receipt["turn_id"], "mode": "heartbeat", "stage": "execute", "ok": False} + capture_before_replan(env, again, "original instructions") + assert again["checkpoint"]["reason"] == "replan_already_captured", json.dumps(again) + assert (target / "checkpoint.json").read_bytes() == captured diff --git a/tests/control_plane_ts/checkpoint_artifacts.test.ts b/tests/control_plane_ts/checkpoint_artifacts.test.ts index 684084a66d..129440d5ab 100644 --- a/tests/control_plane_ts/checkpoint_artifacts.test.ts +++ b/tests/control_plane_ts/checkpoint_artifacts.test.ts @@ -177,3 +177,33 @@ test("an explicitly selected sidecar remains a byte artifact rather than disappe assert.deepEqual(await readFile(join(root, "captured/components/explicit_index")), await readFile(path + "-shm")); } finally {writer.close();} }); + +test("replan capture selection requires the original admitted identity and grants no execution", async () => { + const id = "replan-0123456789abcdef", turn = "fixture-turn"; + const identity = {goal_id: "fixture-goal", agent_id: "fixture-agent", turn_instance_id: turn, + replan_obligation_id: id, binding_kind: "autonomous_replan", binding_id: id, + effect_id: `fixture-goal:fixture-agent:autonomous_replan:${id}:${turn}`}; + const quota = {ok: true, goal_id: "fixture-goal", effective_action: "autonomous_replan_required", + replan_action_packet: {schema_version: "replan_action_packet_v0", decision: "replan_required", obligation_id: id}, + heartbeat_receipt: {status: "committed", turn_instance_id: turn, settlement_identity: identity}}; + const request = {action: "select_replan", quota, goal_id: "fixture-goal", agent_id: "fixture-agent", turn_instance_id: turn}; + const selected = await checkpointArtifacts(request); + assert.equal(selected.capture, true); + assert.equal(selected.obligation_id, id); + assert.equal(selected.execution_authority_granted, false); + const todoBound = await checkpointArtifacts({...request, quota: {...quota, heartbeat_receipt: { + ...quota.heartbeat_receipt, semantic_replan_obligation_id: id, settlement_identity: { + goal_id: "fixture-goal", agent_id: "fixture-agent", turn_instance_id: turn, todo_id: "todo_existing", + effect_id: `fixture-goal:fixture-agent:todo_existing:${turn}`}}}}); + assert.equal(todoBound.capture, true); + assert.equal(todoBound.settlement_effect_id, `fixture-goal:fixture-agent:todo_existing:${turn}`); + const ordinary = await checkpointArtifacts({...request, quota: {...quota, effective_action: "normal_run"}}); + assert.equal(ordinary.capture, false); + await assert.rejects(checkpointArtifacts({...request, turn_instance_id: "another-turn"}), /admitted identity/); + await assert.rejects(checkpointArtifacts({...request, agent_id: "another-agent"}), /admitted identity/); + await assert.rejects(checkpointArtifacts({...request, quota: {...quota, replan_action_packet: + {...quota.replan_action_packet, obligation_id: "replan-fedcba9876543210"}}}), /admitted identity/); + await assert.rejects(checkpointArtifacts({...request, quota: {...quota, heartbeat_receipt: + {...quota.heartbeat_receipt, settlement_identity: {...identity, todo_id: "todo_conflict"}}}}), /admitted identity/); + await assert.rejects(checkpointArtifacts({...request, quota: {...quota, ok: false}}), /matching current/); +}); From 93f61ff7d506473872d851ff0c1c1200dd3e3aba Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Mon, 12 Oct 2026 23:31:35 +0800 Subject: [PATCH 09/69] docs(benchmark): define capture activation and continuation limits Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/README.md | 79 +++++++++++++++++-- .../hierarchical-agent-stride-control-v0.md | 16 +++- ...rarchical-agent-stride-control-v0.zh-CN.md | 11 ++- 3 files changed, 92 insertions(+), 14 deletions(-) diff --git a/benchmark/README.md b/benchmark/README.md index c43ba24805..ee9979d49d 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -117,11 +117,14 @@ runner, verifier, submission, or publication authority. ## Inert checkpoint capture This is a developer prerequisite for the [research checkpoint contract](../docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md#77-research-state-branching-and-matched-continuation), -not a qualified research continuation. The programmatic `BenchmarkCodex` and -heartbeat `SForgeWorker` adapters accept an explicit `checkpoint_request_path` -to a normalized absolute **guest** JSON file. Stage that file through the existing -environment transport. Ambient host variables do not enable capture; omitting -the option disables the hook. The EdgeBench CLI does not expose this option yet. +not a qualified research continuation. The EdgeBench CLI accepts an explicit +`--checkpoint-request /path/to/request.json` **host** file and stages its exact +bytes into the heartbeat worker's protected control directory. The declaration +is validated before trial creation/registration and its digest appears in the +runtime receipt. The programmatic `BenchmarkCodex` and heartbeat `SForgeWorker` +adapters also accept `checkpoint_request_path` to an already staged normalized +absolute **guest** file. Ambient host variables do not enable capture; omitting +these options disables the hook. Official and native-goal profiles reject them. The request contains exactly these fields; select only authorized private artifacts and exclude credentials and hidden evaluation material: @@ -159,7 +162,69 @@ source boundary, not adversarial filesystem mutation or a cross-owner transactio Native session reconstruction, effect reconciliation, fresh authority admission and evaluator cooldown/opportunity inheritance must pass before matched continuation. -这是研究 checkpoint 的开发前置切片。仅 programmatic adapter 的显式 guest -请求路径会开启捕获，省略该参数即关闭；当前 EdgeBench CLI 尚无此选项。产物 +For pre-decision capture at a **fresh replan projected at the start of a heartbeat +wake**, add `"trigger": "fresh_replan"` and set `destination` to the trial's +`checkpoints` directory itself. The Host preloads that wake's original quota guard +once, the typed owner validates its replan/settlement binding, and the adapter +captures from the latest successful, settled work wake before invoking the model. +The model reads the complete preloaded guard and follows its existing selection, +required-read, recovery and settlement contract. Each obligation has one immutable +destination; a repeated read verifies that point instead of replacing it with later +work. The after-wake hook cannot substitute a post-decision state. Capture failures +remain explicit and do not settle or complete the new work. +The first attempt is also retained: if it fails and the model proceeds, retries +cannot substitute a later state for that missed pre-decision point. + +Add `"native_session": true` to include Codex's completed native cursor, all +retained history pages, rollout/compaction records and native index/history SQLite +stores. The adapter uses native readback before sealing these members and excludes +credential files, logs, memories and the symlinked skill installation. Record the +effective configuration digest and Host version; tools/skills and credentials need +their existing installation/authentication owners. Native state is opaque and its +storage layout remains a pinned provider qualification. + +`benchmark.runtime.native_checkpoint.fork_native_checkpoint` verifies a unit and +reconstructs a distinct native session in a **new isolated environment preserving +the captured guest paths**, with matching configuration and Host version. It checks +the full history digest and cursor, starts no model and registers no LoopX binding. +A failed or partial reconstruction is not an admitted session. This is a session +participant prerequisite, not full research continuation or arbitrary path import. + +For the native feedback protocol with **no automatic evaluations**, start an +explicit checkpoint-aware native judge: + +```sh +python -m benchmark.edgebench.checkpoint_judge --checkpoint-capture \ + --tasks-dir /path/to/tasks --log-dir /path/to/private-run-logs +``` + +Its default-off factory otherwise preserves the native routes. With +`"evaluation_opportunity": true` in the worker declaration, the capture hook +reads that registration's released history and complete public result details, +next round counters, accepted submission clock, remaining cooldown, task/native +source identity and original trial deadline. The provider uses the existing +native admission/history locks and rejects an outstanding grade, unreconciled +consumed round, missing result details, automatic-evaluation history or closed +registration. It never returns tokens or hidden task material and never grades, +registers or refills an opportunity. The local `sforge_checkpoint_opportunity_v0` +artifact records these facts; it grants no restored admission. Disabling capture +means omitting both the judge flag and worker declaration option on a new run. + +Declare the workspace's authorized code/research files, LoopX runtime and +registry, effective configuration and installed source/profile pins as separate +non-overlapping components. The native and opportunity participants do not imply +that omitted workspace state was saved. Every source remains private; inspect +the verified member inventory before qualifying a complete research point. + +The current Host hook cannot capture a new replan created **inside an already +running model wake** before that decision. Those boundaries still need a qualified +Host handoff; a later snapshot must not be labeled as the missed point. Complete +LoopX research-state adoption, restored evaluator admission and useful continued work +remain open. No automatic retention or extra evaluation opportunity is introduced. + +这是研究 checkpoint 的开发前置切片。EdgeBench CLI 的显式 +`--checkpoint-request` 会把 host JSON 原文安全放入 heartbeat worker，并在 +runtime receipt 绑定摘要；programmatic adapter 也可指定已放入 guest 的请求 +路径。省略选项即关闭，Official/native-goal 不接受。产物 验证和隔离物化不授予执行权，不代表研究点已资格化。请求中的 metadata 只是 证据，不能代替会话、效果、状态采用及评测机会的联合验收。 diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md index ecf47b3b4c..28bb20b3f1 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md @@ -724,10 +724,18 @@ The initial implementation slice is [inert artifact capture](../../../benchmark/ The benchmark adapter requests it after the existing Turn settlement; the typed artifact owner seals explicit files/directories and committed SQLite snapshots, rechecks source witnesses and verifies an isolated materialization. Its manifest -always declares the research point unqualified. This explicit one-point hook -does not implement the proposed automatic replan trigger. Participant/effect reconciliation, -native session reconstruction, fresh state adoption and preserved evaluator -opportunities remain separate acceptance work; operator metadata cannot certify them. +always declares the research point unqualified. The heartbeat adapter also accepts +an explicit fresh-replan trigger: it preloads the original quota guard and captures +an admitted replan projected at wake start before invoking the model, deduplicated +by the existing obligation. It does not cover replans first created inside a running +Host wake. The optional native participant reads complete history through Codex, +seals its rollout/index/paged stores and verifies native forks at fixed guest paths. +An explicit checkpoint-aware native judge also exports released public result +details, accepted counters/clock, remaining cooldown and original trial deadline +without grading or refilling admission; pending or unreconciled grades reject. +Participant/effect reconciliation, fresh research-state adoption, preserved evaluator +admission and useful continued work remain separate acceptance work; operator +metadata and native history equality cannot certify them. An inert verification unit grants no execution authority. The destination uses fresh experiment/Host identities and the existing Goal/session/state admission diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md index 93fd2c06e9..90965ab698 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md @@ -606,9 +606,14 @@ Host profile 还须声明路径可移植性。原生索引可能保留绝对 gue 第一项实现切片是 [inert 产物捕获](../../../benchmark/README.md#inert-checkpoint-capture)。 Benchmark adapter 在既有 Turn 结算后请求捕获；typed 产物 owner 封存显式文件/ 目录及已提交 SQLite 快照，复查来源 witness 并验证隔离物化。Manifest 始终声明 -研究点尚未资格化。这项显式单点 hook 尚未实现上述自动 replan 触发。参与者/ -效果 reconciliation、原生会话重建、新状态采用及评测 -机会继承仍各自待验收；operator metadata 不能认证这些条件。 +研究点尚未资格化。Heartbeat adapter 还支持显式 fresh-replan 触发：预加载原 +quota guard，在模型调用前捕获 wake 开始已投影的准入 replan，并按既有义务 +去重；尚不覆盖运行中 Host wake 首次产生的 replan。可选 native 参与者通过 +Codex 读取完整历史，封存 rollout/索引/分页库并验证固定 guest 路径下的原生 +分叉。显式 checkpoint-aware native judge 还可导出已公开结果明细、受理计数/ +时钟、剩余冷却及原试验期限；不评测、不补发机会，未完成或未对账评测会拒绝。 +参与者/效果 reconciliation、新研究状态采用、目标侧评测准入及实际有效 +接续仍各自待验收；operator metadata 或历史一致不能认证这些条件。 inert verification unit 不授予执行权。目标侧使用新的实验/Host 身份，通过既有 Goal/session/state admission 进入。历史租约、quota、timer、pending effect 和 From b6ae572a100a3a84dde48638b0b77dc1a45d17ee Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 01:19:37 +0800 Subject: [PATCH 10/69] fix(checkpoint): seal the native-selected incumbent with its result Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/edgebench/checkpoint_judge.py | 148 +++++++++++++----- benchmark/runtime/checkpoint.py | 13 +- benchmark/runtime/sforge_checkpoint.py | 50 +++++- benchmark/tests/test_checkpoint_judge.py | 84 ++++++++++ .../tests/test_edgebench_feedback_docker.py | 45 ++++-- .../test_research_checkpoint_capture.py | 26 ++- 6 files changed, 304 insertions(+), 62 deletions(-) diff --git a/benchmark/edgebench/checkpoint_judge.py b/benchmark/edgebench/checkpoint_judge.py index d1ed8dc1ec..77117b7e80 100644 --- a/benchmark/edgebench/checkpoint_judge.py +++ b/benchmark/edgebench/checkpoint_judge.py @@ -9,11 +9,34 @@ import argparse import hashlib import json +import os +import stat import time import uuid from dataclasses import replace from pathlib import Path -from benchmark.runtime.sforge_checkpoint import ROUTE, SCHEMA +from benchmark.runtime.sforge_checkpoint import ARCHIVE_ROUTE, ROUTE, SCHEMA + + +def _read_archive(root, run, task, round_id): + """Read an opaque native archive, never extract or inspect its members.""" + root = root.resolve(strict=True) + path = root / "runs" / run / task / "submissions" / round_id / "submission.tar.gz" + if not path.is_relative_to(root) or path.resolve(strict=True) != path: + raise ValueError("Native incumbent archive must be a physical file below the log owner") + with os.fdopen(os.open(path, os.O_RDONLY | os.O_NOFOLLOW), "rb") as source: + before = os.fstat(source.fileno()) + if not stat.S_ISREG(before.st_mode): + raise ValueError("Native incumbent archive must be a regular file") + raw = source.read() + after = os.fstat(source.fileno()) + observed = path.stat(follow_symlinks=False) + def signature(value): + return value.st_dev, value.st_ino, value.st_size, value.st_mtime_ns, value.st_ctime_ns + if (signature(before) != signature(after) or signature(after) != signature(observed) or + path.resolve(strict=True) != path or len(raw) != before.st_size): + raise ValueError("Native incumbent archive moved during capture") + return raw def create_app(config, *, checkpoint_capture=False): @@ -24,6 +47,7 @@ def create_app(config, *, checkpoint_capture=False): if not checkpoint_capture: return app # Preserve the native route/admission surface when disabled. from fastapi import HTTPException + from fastapi.responses import Response from pydantic import BaseModel, ConfigDict import sforge @@ -48,54 +72,94 @@ class ReadRequest(BaseModel): model_config = ConfigDict(extra="forbid") token: str + class ArchiveRequest(ReadRequest): + capture_binding_sha256: str + # FastAPI resolves annotations in module globals; keep this factory's # private request type concrete rather than a forward reference. - def capture(request): + def snapshot_locked(token): + from sforge.harness.selection import select_best + # Reuse the native lock order and outstanding admission counter. A # pending grade or unexplained consumed round is never a complete point. + # Called only under the native admission -> token -> history locks. + info = state.tokens.get(token) + if info is None: + raise HTTPException(403, "Unknown registration") + run, task = info["run_id"], info["task_id"] + if state._closed or state.tasks[task].game_mode: + raise HTTPException(409, "Checkpoint requires an open non-game registration") + if state._outstanding_by_run.get((run, task), 0): + raise HTTPException(409, "Accepted evaluation is still outstanding") + if info["next_auto"] != 1: + raise HTTPException(409, "This checkpoint profile requires no automatic evaluations") + history = state.run_history.get(state._run_key(run, task), []) + rounds = [entry.get("round") for entry in history] + expected = {f"agent-{index}" for index in range(1, info["next_agent"])} + if (len(rounds) != len(expected) or set(rounds) != expected or + any(entry.get("status") not in {SubmissionStatus.COMPLETED, SubmissionStatus.ERROR} for entry in history)): + raise HTTPException(409, "Consumed rounds and released history are not reconciled") + results = {} + for entry in history: + identifier = entry.get("submission_id") + native = state.submissions.get(identifier) + if (native is None or native.get("run_id") != run or native.get("task_id") != task or + native.get("round") != entry["round"] or native.get("status") != entry["status"]): + raise HTTPException(409, "Released result details are unavailable or differ") + results[identifier] = {key: native.get(key) for key in ("status", "report", "error")} + # Reuse native ranking, including validity, direction and tie rules. + # The working candidate can differ from this older selected archive. + best = select_best(history, state.tasks[task].judge.score_direction, + state.tasks[task].judge.selection) + incumbent = None + raw = None + if best["best_round"]: + selected = next(entry for entry in history if entry["round"] == best["best_round"]) + try: + raw = _read_archive(config.log_dir, run, task, selected["round"]) + except (OSError, ValueError) as error: + raise HTTPException(409, "Native selected incumbent archive is unavailable or moved") from error + incumbent = {"round": selected["round"], "submission_id": selected["submission_id"], + "archive_sha256": hashlib.sha256(raw).hexdigest(), "archive_bytes": len(raw)} + observed = time.time() + last = info["last_agent_submit_at"] + cooldown = info["submission_cooldown"] + remaining = max(0, last + (cooldown or 0) - observed) if last else 0 + packet = { + "schema_version": SCHEMA, "judge_epoch": epoch, + "run_id": run, "task_id": task, "evaluator": tasks[task], + "observed_at_epoch": observed, "last_agent_submit_at_epoch": last, + "submission_cooldown_seconds": cooldown, + "remaining_cooldown_seconds": remaining, + "max_agent_submissions": info["max_agent_submissions"], + "next_agent_round": info["next_agent"], "next_auto_round": info["next_auto"], + "released_history": history, "released_results": results, "outstanding_evaluations": 0, + "native_incumbent": incumbent, + "execution_authority_granted": False, + } + # Bind both reads to the same released state, not a wall clock. A new + # accepted round, source epoch or changed archive invalidates download. + binding = {key: value for key, value in packet.items() + if key not in {"observed_at_epoch", "remaining_cooldown_seconds"}} + packet["capture_binding_sha256"] = hashlib.sha256( + json.dumps(binding, sort_keys=True, separators=(",", ":")).encode()).hexdigest() + return json.loads(json.dumps(packet)), raw # No live references or tokens leave the lock. + + def capture(request): + with state._admission_lock, state._tokens_lock, state._history_lock: + return snapshot_locked(request.token)[0] + + def archive(request): with state._admission_lock, state._tokens_lock, state._history_lock: - info = state.tokens.get(request.token) - if info is None: - raise HTTPException(403, "Unknown registration") - run, task = info["run_id"], info["task_id"] - if state._closed or state.tasks[task].game_mode: - raise HTTPException(409, "Checkpoint requires an open non-game registration") - if state._outstanding_by_run.get((run, task), 0): - raise HTTPException(409, "Accepted evaluation is still outstanding") - if info["next_auto"] != 1: - raise HTTPException(409, "This checkpoint profile requires no automatic evaluations") - history = state.run_history.get(state._run_key(run, task), []) - rounds = [entry.get("round") for entry in history] - expected = {f"agent-{index}" for index in range(1, info["next_agent"])} - if (len(rounds) != len(expected) or set(rounds) != expected or - any(entry.get("status") not in {SubmissionStatus.COMPLETED, SubmissionStatus.ERROR} for entry in history)): - raise HTTPException(409, "Consumed rounds and released history are not reconciled") - results = {} - for entry in history: - identifier = entry.get("submission_id") - native = state.submissions.get(identifier) - if (native is None or native.get("run_id") != run or native.get("task_id") != task or - native.get("round") != entry["round"] or native.get("status") != entry["status"]): - raise HTTPException(409, "Released result details are unavailable or differ") - results[identifier] = {key: native.get(key) for key in ("status", "report", "error")} - observed = time.time() - last = info["last_agent_submit_at"] - cooldown = info["submission_cooldown"] - remaining = max(0, last + (cooldown or 0) - observed) if last else 0 - packet = { - "schema_version": SCHEMA, "judge_epoch": epoch, - "run_id": run, "task_id": task, "evaluator": tasks[task], - "observed_at_epoch": observed, "last_agent_submit_at_epoch": last, - "submission_cooldown_seconds": cooldown, - "remaining_cooldown_seconds": remaining, - "max_agent_submissions": info["max_agent_submissions"], - "next_agent_round": info["next_agent"], "next_auto_round": info["next_auto"], - "released_history": history, "released_results": results, "outstanding_evaluations": 0, - "execution_authority_granted": False, - } - return json.loads(json.dumps(packet)) # No live references or tokens leave the lock. + packet, raw = snapshot_locked(request.token) + incumbent = packet["native_incumbent"] + if packet["capture_binding_sha256"] != request.capture_binding_sha256 or incumbent is None: + raise HTTPException(409, "Native checkpoint basis moved or has no selected incumbent") + return Response(raw, media_type="application/octet-stream") capture.__annotations__["request"] = ReadRequest + archive.__annotations__["request"] = ArchiveRequest app.post(ROUTE)(capture) + app.post(ARCHIVE_ROUTE)(archive) return app diff --git a/benchmark/runtime/checkpoint.py b/benchmark/runtime/checkpoint.py index bdc4d1392c..ddd8e221d8 100644 --- a/benchmark/runtime/checkpoint.py +++ b/benchmark/runtime/checkpoint.py @@ -219,14 +219,18 @@ def capture_after_wake(env: dict[str, str], receipt: dict[str, Any], *, "request_sha256": hashlib.sha256(raw).hexdigest(), } components = request["components"] + opportunity = None if request.get("evaluation_opportunity"): from .sforge_checkpoint import capture_opportunity - opportunity = capture_opportunity(env, timeout=max(.1, deadline - time.time())) evidence = Path(env["LOOPX_WAKE_LOG_DIR"]) / receipt["turn_id"] / "checkpoint-opportunity.json" + incumbent = evidence.with_name("checkpoint-incumbent.tar.gz") + opportunity = capture_opportunity(env, timeout=max(.1, deadline - time.time()), incumbent_path=incumbent) evidence.write_text(json.dumps(opportunity), encoding="utf-8") evidence.chmod(0o600) components = [*components, {"id": "evaluation_opportunity", "source_path": str(evidence)}] + if opportunity["native_incumbent"] is not None: + components = [*components, {"id": "evaluation_incumbent", "source_path": str(incumbent)}] if request.get("native_session"): from .native_checkpoint import native_session_checkpoint @@ -239,6 +243,13 @@ def capture_after_wake(env: dict[str, str], receipt: dict[str, Any], *, "action": "probe", "components": components, "deadline_epoch_ms": deadline * 1000, }, timeout=max(.1, deadline - time.time()), retry_safe=False) + if opportunity is not None and opportunity["native_incumbent"] is not None: + selected = opportunity["native_incumbent"] + witness = probe["source_witness"].get("evaluation_incumbent", []) + if (len(witness) != 1 or witness[0].get("path") != "" or witness[0].get("kind") != "file" or + witness[0].get("sha256") != selected["archive_sha256"] or + witness[0].get("size") != selected["archive_bytes"]): + raise ValueError("native incumbent participant differs from its captured result") destination.parent.mkdir(parents=True, exist_ok=True) captured = effect_runtime_result("runtime.checkpoint_artifacts", { "action": "capture", "destination": str(destination), diff --git a/benchmark/runtime/sforge_checkpoint.py b/benchmark/runtime/sforge_checkpoint.py index ce1d7e36eb..2f19c7da96 100644 --- a/benchmark/runtime/sforge_checkpoint.py +++ b/benchmark/runtime/sforge_checkpoint.py @@ -2,28 +2,72 @@ from __future__ import annotations import json +import hashlib import math import time from urllib.request import Request, urlopen +from pathlib import Path SCHEMA = "sforge_checkpoint_opportunity_v0" # Local provider artifact, not a LoopX authority contract. ROUTE = "/api/v1/checkpoint/opportunity" +ARCHIVE_ROUTE = "/api/v1/checkpoint/incumbent" -def capture_opportunity(env, *, timeout): +def capture_opportunity(env, *, timeout, incumbent_path: Path): """Read only this worker's registration; retain its original deadline.""" deadline = float(env["LOOPX_PHASE_DEADLINE_EPOCH"]) - if not math.isfinite(deadline) or deadline <= time.time() or timeout <= 0: + if not math.isfinite(deadline) or deadline <= time.time() or not math.isfinite(timeout) or timeout <= 0: raise ValueError("checkpoint requires an open original trial deadline") + stop = min(deadline, time.time() + timeout) request = Request(env["SFORGE_JUDGE_URL"].rstrip("/") + ROUTE, data=json.dumps({"token": env["SFORGE_TOKEN"]}).encode(), headers={"Content-Type": "application/json"}, method="POST") - with urlopen(request, timeout=timeout) as response: + with urlopen(request, timeout=max(.1, stop - time.time())) as response: packet = json.load(response) if (packet.get("schema_version") != SCHEMA or packet.get("execution_authority_granted") is not False or packet.get("outstanding_evaluations") != 0): raise ValueError("native opportunity checkpoint readback is unavailable") + if "native_incumbent" not in packet or not packet.get("capture_binding_sha256"): + raise ValueError("native opportunity checkpoint omits incumbent coverage") + incumbent = packet["native_incumbent"] + if incumbent is not None: + if (not isinstance(incumbent, dict) or type(incumbent.get("archive_bytes")) is not int or + incumbent["archive_bytes"] < 0 or not isinstance(incumbent.get("archive_sha256"), str)): + raise ValueError("native incumbent descriptor is invalid") + incumbent_path = Path(incumbent_path) + if not incumbent_path.is_absolute() or incumbent_path != incumbent_path.resolve(): + raise ValueError("incumbent destination must be a normalized physical private file") + if time.time() >= stop: + raise ValueError("checkpoint capture window closed before incumbent transport") + request = Request(env["SFORGE_JUDGE_URL"].rstrip("/") + ARCHIVE_ROUTE, + data=json.dumps({"token": env["SFORGE_TOKEN"], + "capture_binding_sha256": packet["capture_binding_sha256"]}).encode(), + headers={"Content-Type": "application/json"}, method="POST") + created = False + try: + with urlopen(request, timeout=max(.1, stop - time.time())) as response: + # Never overwrite a prior participant or follow its symlink. + with incumbent_path.open("xb") as target: + created = True + incumbent_path.chmod(0o600) + digest, size = hashlib.sha256(), 0 + while chunk := response.read(1024 * 1024): + size += len(chunk) + if size > incumbent["archive_bytes"] or time.time() >= stop: + raise ValueError("native incumbent transport exceeded its bound") + digest.update(chunk) + target.write(chunk) + if size != incumbent["archive_bytes"] or digest.hexdigest() != incumbent["archive_sha256"]: + raise ValueError("native incumbent bytes differ from the captured result") + except Exception: + if created: + incumbent_path.unlink() + raise observed = time.time() + if observed >= stop: + if incumbent is not None: + incumbent_path.unlink() + raise ValueError("checkpoint capture window closed during opportunity transport") return {**packet, "source_trial_deadline_epoch": deadline, "client_observed_at_epoch": observed, "remaining_trial_seconds": max(0, deadline - observed), diff --git a/benchmark/tests/test_checkpoint_judge.py b/benchmark/tests/test_checkpoint_judge.py index fe93fd40f5..0c6708ccae 100644 --- a/benchmark/tests/test_checkpoint_judge.py +++ b/benchmark/tests/test_checkpoint_judge.py @@ -1,5 +1,6 @@ """Native owner semantics; real grading/transport is qualified by the Docker smoke.""" import json +import hashlib from types import SimpleNamespace import pytest @@ -60,6 +61,7 @@ def test_capture_retains_consumed_round_history_and_cooldown_without_mutation(na assert point["released_history"][0]["status"] == "error" assert point["released_results"]["accepted-error"]["error"] == "synthetic evaluation failure" assert point["execution_authority_granted"] is False + assert point["native_incumbent"] is None # Failure does not fabricate a candidate. assert token not in json.dumps(point) and "admin_secret" not in json.dumps(point) assert state.resolve_token(token) == before assert client.post(ROUTE, json={"token": token}).json()["judge_epoch"] == point["judge_epoch"] @@ -84,3 +86,85 @@ def test_incomplete_opportunity_state_cannot_be_a_checkpoint(native_config, gap) assert client.post(ROUTE, json={"token": token}).status_code == 409 assert client.post(ROUTE, json={"token": "unregistered"}).status_code == 403 assert client.post(ROUTE, json={"token": token, "restore": True}).status_code == 422 + + +@pytest.mark.parametrize("policy,direction,expected", [ + ("score_first", "maximize", "agent-1"), ("score_first", "minimize", "agent-2"), + ("pass_rate_first", "maximize", "agent-2"), ("valid_then_score", "maximize", "agent-2"), +]) +def test_native_selected_incumbent_archive_is_bound_to_its_result(native_config, policy, direction, expected): + from fastapi.testclient import TestClient + from benchmark.edgebench.checkpoint_judge import create_app, ROUTE, ARCHIVE_ROUTE + + task = json.loads((native_config.tasks_dir / "fixture.json").read_text()) + task["judge"].update(selection=policy, score_direction=direction) + (native_config.tasks_dir / "fixture.json").write_text(json.dumps(task)) + app = create_app(native_config, checkpoint_capture=True) + state = app.state.judge + with TestClient(app) as client: + token = state.register_session("fixture", "source") + candidates = {"agent-1": b"earlier higher score", "agent-2": b"later candidate"} + for index, (round_id, raw) in enumerate(candidates.items(), 1): + state.consume_round(token) + report = {"score": 9 if index == 1 else 3, "valid": index == 2, + "pass_rate": .5 if index == 1 else 1.0} + identifier = "submission-" + str(index) + state._record_submission("source", identifier, "fixture", round_id, report, None) + state.submissions[identifier] = {"run_id": "source", "task_id": "fixture", "round": round_id, + "status": "completed", "report": report, "error": None} + path = native_config.log_dir / "runs/source/fixture/submissions" / round_id / "submission.tar.gz" + path.parent.mkdir(parents=True) + path.write_bytes(raw) + before = state.resolve_token(token) + point = client.post(ROUTE, json={"token": token}).json() + native = point["native_incumbent"] + assert native["round"] == expected + assert native["archive_sha256"] == hashlib.sha256(candidates[expected]).hexdigest() + assert native["archive_bytes"] == len(candidates[expected]) + request = {"token": token, "capture_binding_sha256": point["capture_binding_sha256"]} + downloaded = client.post(ARCHIVE_ROUTE, json=request) + assert downloaded.status_code == 200 and downloaded.content == candidates[expected] + assert state.resolve_token(token) == before + assert client.post(ARCHIVE_ROUTE, json={**request, "token": "unregistered"}).status_code == 403 + assert client.post(ARCHIVE_ROUTE, json={**request, "capture_binding_sha256": "wrong-epoch"}).status_code == 409 + selected = native_config.log_dir / "runs/source/fixture/submissions" / expected / "submission.tar.gz" + selected.write_bytes(b"changed selected bytes") + assert client.post(ARCHIVE_ROUTE, json=request).status_code == 409 + selected.unlink() + assert client.post(ROUTE, json={"token": token}).status_code == 409 + selected.symlink_to(native_config.tasks_dir / "fixture.json") + assert client.post(ROUTE, json={"token": token}).status_code == 409 + selected.unlink() + selected.write_bytes(candidates[expected]) + state.consume_round(token) # A new accepted, unreconciled round invalidates the old point. + assert client.post(ARCHIVE_ROUTE, json=request).status_code == 409 + + +@pytest.mark.parametrize('payload', [b'wrong bytes', b'truncated']) +def test_incumbent_transport_mismatch_never_publishes_a_participant(tmp_path, monkeypatch, payload): + import io + import time + from benchmark.runtime import sforge_checkpoint + + expected = b'original accepted bytes' + packet = {'schema_version': sforge_checkpoint.SCHEMA, 'execution_authority_granted': False, + 'outstanding_evaluations': 0, 'capture_binding_sha256': 'bound-state', + 'native_incumbent': {'archive_bytes': len(expected), + 'archive_sha256': hashlib.sha256(expected).hexdigest()}} + calls = [] + def read(request, *, timeout): + calls.append(request) + return io.BytesIO(json.dumps(packet).encode() if len(calls) == 1 else payload) + monkeypatch.setattr(sforge_checkpoint, 'urlopen', read) + env = {'SFORGE_JUDGE_URL': 'http://synthetic.invalid', 'SFORGE_TOKEN': 'synthetic', + 'LOOPX_PHASE_DEADLINE_EPOCH': str(time.time() + 30)} + target = tmp_path / 'incumbent.tar.gz' + with pytest.raises(ValueError, match='differ'): + sforge_checkpoint.capture_opportunity(env, timeout=10, incumbent_path=target) + assert len(calls) == 2 and not target.exists() + assert json.loads(calls[1].data)['capture_binding_sha256'] == 'bound-state' + target.write_bytes(b'prior participant') + calls.clear() + with pytest.raises(FileExistsError): + sforge_checkpoint.capture_opportunity(env, timeout=10, incumbent_path=target) + assert target.read_bytes() == b'prior participant' diff --git a/benchmark/tests/test_edgebench_feedback_docker.py b/benchmark/tests/test_edgebench_feedback_docker.py index 58a6f2af95..f2a743e55d 100644 --- a/benchmark/tests/test_edgebench_feedback_docker.py +++ b/benchmark/tests/test_edgebench_feedback_docker.py @@ -149,7 +149,7 @@ def test_checkpoint_opportunity_uses_native_admission_and_real_grade(tmp_path, m "work": {"image_tag": "fixture", "agent_query": "Synthetic fixture"}, "judge": {"image_tag": "fixture", "eval_timeout": 15, "parser": "structured_json", "selection": "score_first", "score_direction": "maximize", - "eval_cmd": "python3 -c 'import json; print(json.dumps(dict(score=7, valid=True)))'"}, + "eval_cmd": "python3 -c 'import json; print(json.dumps(dict(score=json.load(open(\"candidate.json\"))[\"score\"], valid=True)))'"}, })) config = SForgeConfig(tasks_dir=tasks, log_dir=tmp_path, judge_cpu_limit=1, judge_mem_limit="256m", judge_max_outstanding_per_run=1) @@ -175,14 +175,17 @@ def held_grade(**kwargs): "submission_cooldown": 3600, "max_agent_submissions": 12}, timeout=10) registered.raise_for_status() token = registered.json()["token"] - payload = io.BytesIO() - with tarfile.open(fileobj=payload, mode="w:gz") as archive: - member = tarfile.TarInfo("candidate.json") - data = b"{}" - member.size = len(data) - archive.addfile(member, io.BytesIO(data)) + def candidate(score): + payload = io.BytesIO() + with tarfile.open(fileobj=payload, mode="w:gz") as archive: + member = tarfile.TarInfo("candidate.json") + data = json.dumps({"score": score}).encode() + member.size = len(data) + archive.addfile(member, io.BytesIO(data)) + return payload.getvalue() + first_archive = candidate(7) accepted = requests.post(url + "/api/v1/submit", data={"token": token}, - files={"archive": ("candidate.tar.gz", payload.getvalue())}, timeout=15) + files={"archive": ("candidate.tar.gz", first_archive)}, timeout=15) accepted.raise_for_status() assert accepted.json()["round_id"] == "agent-1" assert requests.post(url + ROUTE, json={"token": token}, timeout=10).status_code == 409 @@ -192,20 +195,38 @@ def held_grade(**kwargs): assert released.status_code == 200, released.json() env = {"SFORGE_JUDGE_URL": url, "SFORGE_TOKEN": token, "LOOPX_PHASE_DEADLINE_EPOCH": str(time.time() + 100)} - point = capture_opportunity(env, timeout=10) + point = capture_opportunity(env, timeout=10, incumbent_path=tmp_path / "incumbent.tar.gz") + assert (tmp_path / "incumbent.tar.gz").read_bytes() == first_archive assert point["released_history"][0]["score"] == 7 assert point["next_agent_round"] == 2 and 0 < point["remaining_cooldown_seconds"] <= 3600 assert 0 < point["remaining_trial_seconds"] <= 100 assert point["continuation_qualified"] is False before = app.state.judge.resolve_token(token) - assert capture_opportunity(env, timeout=10)["judge_epoch"] == point["judge_epoch"] + assert capture_opportunity(env, timeout=10, incumbent_path=tmp_path / "second-read.tar.gz")["judge_epoch"] == point["judge_epoch"] assert app.state.judge.resolve_token(token) == before cooled = requests.post(url + "/api/v1/submit", data={"token": token}, - files={"archive": ("candidate.tar.gz", payload.getvalue())}, timeout=15) + files={"archive": ("candidate.tar.gz", first_archive)}, timeout=15) assert cooled.status_code == 429 # Reading the checkpoint cannot refill admission. assert app.state.judge.resolve_token(token) == before with pytest.raises(ValueError, match="original trial deadline"): - capture_opportunity({**env, "LOOPX_PHASE_DEADLINE_EPOCH": "0"}, timeout=10) + capture_opportunity({**env, "LOOPX_PHASE_DEADLINE_EPOCH": "0"}, timeout=10, + incumbent_path=tmp_path / "closed.tar.gz") + # Separate native registration with no cooldown: two actual grades must + # retain the older winner, not substitute the later working candidate. + registered = requests.post(url + "/api/v1/register", json={"task_id": "fixture", + "run_id": "synthetic-selection", "admin_secret": app.state.judge.admin_secret}, timeout=10) + registered.raise_for_status() + selection_token = registered.json()["token"] + for raw in (first_archive, candidate(3)): + response = requests.post(url + "/api/v1/submit", data={"token": selection_token}, + files={"archive": ("candidate.tar.gz", raw)}, timeout=15) + response.raise_for_status() + wait_for(lambda: not app.state.judge._outstanding_by_run, timeout=60) + selected = capture_opportunity({**env, "SFORGE_TOKEN": selection_token}, timeout=10, + incumbent_path=tmp_path / "selected.tar.gz") + assert selected["next_agent_round"] == 3 and selected["native_incumbent"]["round"] == "agent-1" + assert [entry["score"] for entry in selected["released_history"]] == [7, 3] + assert (tmp_path / "selected.tar.gz").read_bytes() == first_archive finally: finish_grade.set() server.should_exit = True diff --git a/tests/control_plane/test_research_checkpoint_capture.py b/tests/control_plane/test_research_checkpoint_capture.py index 055b653d6d..14f6d488a7 100644 --- a/tests/control_plane/test_research_checkpoint_capture.py +++ b/tests/control_plane/test_research_checkpoint_capture.py @@ -1,6 +1,7 @@ """Inert checkpoint capture reuses the real settled-Turn source owner.""" from pathlib import Path import json +import hashlib import pytest import time @@ -41,8 +42,8 @@ def forbidden(*args, **kwargs): assert receipt["checkpoint"]["reason"] == "capture_failed" -@pytest.mark.parametrize("opportunity", [False, True]) -def test_real_cli_settlement_capture_verification_and_inert_materialization(tmp_path, monkeypatch, opportunity): +@pytest.mark.parametrize("opportunity,corrupt", [(False, False), (True, False), (True, True)]) +def test_real_cli_settlement_capture_verification_and_inert_materialization(tmp_path, monkeypatch, opportunity, corrupt): isolate_sqlite_runtime(tmp_path, monkeypatch) project, runtime, registry = cli._write_fixture(tmp_path) binding = ("--agent-id", cli.AGENT_ID, "--todo-id", cli.TODO_ID, "--turn-instance-id", cli.TURN_ID) @@ -56,12 +57,21 @@ def run(*args): wake.mkdir(parents=True) (wake / cli.TURN_ID).mkdir() observed = [] - opportunity_packet = {"meaning": "synthetic opportunity participant; no admission"} - def export(env, *, timeout): + incumbent_bytes = b"opaque older adopted candidate" + opportunity_packet = {"meaning": "synthetic opportunity participant; no admission", "native_incumbent": { + "round": "agent-1", "archive_sha256": hashlib.sha256(incumbent_bytes).hexdigest(), "archive_bytes": len(incumbent_bytes)}} + def export(env, *, timeout, incumbent_path): assert 0 < timeout <= 30 observed.append(True) + incumbent_path.write_bytes(incumbent_bytes) return opportunity_packet monkeypatch.setattr("benchmark.runtime.sforge_checkpoint.capture_opportunity", export) + if corrupt: + def altered_probe(method, params, **kwargs): + if params["action"] == "probe": + next(wake.glob("*/checkpoint-incumbent.tar.gz")).write_bytes(b"altered after transport") + return effect_runtime_result(method, params, **kwargs) + monkeypatch.setattr("benchmark.runtime.checkpoint.effect_runtime_result", altered_probe) (project / "solution.py").write_text("retained_candidate = 7\n") request = tmp_path / "request.json" target = logs / "checkpoints" / "point" @@ -91,17 +101,25 @@ def export(env, *, timeout): retained_goal = goal_state.read_bytes() capture_after_wake(env, receipt) assert receipt["ok"] is True + if corrupt: + assert receipt["checkpoint"]["captured"] is False and not target.exists() + assert "differs" in receipt["checkpoint"]["error"] + assert goal_state.read_bytes() == retained_goal + return assert receipt["checkpoint"].get("captured") is True, receipt assert receipt["checkpoint"]["research_point_qualified"] is False assert goal_state.read_bytes() == retained_goal if opportunity: assert json.loads((target / "components/evaluation_opportunity").read_text()) == opportunity_packet + assert (target / "components/evaluation_incumbent").read_bytes() == incumbent_bytes digest = receipt["checkpoint"]["sha256"] restored = effect_runtime_result("runtime.checkpoint_artifacts", {"action": "materialize", "root": str(target), "expected_sha256": digest, "destination": str(tmp_path / "inert")}) assert restored["activation_performed"] is False assert (tmp_path / "inert/components/code").read_text() == "retained_candidate = 7\n" + if opportunity: + assert (tmp_path / "inert/components/evaluation_incumbent").read_bytes() == incumbent_bytes before = (target / "checkpoint.json").read_bytes() capture_after_wake(env, receipt) assert receipt["checkpoint"]["reason"] == "destination_already_present_no_replay" From a016107d552bf220e156c73bb9aa8ade2e567eeb Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 01:20:10 +0800 Subject: [PATCH 11/69] docs(checkpoint): distinguish native incumbent capture from continuation Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/README.md | 14 ++++++++++++-- .../rfcs/hierarchical-agent-stride-control-v0.md | 7 +++++-- .../hierarchical-agent-stride-control-v0.zh-CN.md | 5 ++++- 3 files changed, 21 insertions(+), 5 deletions(-) diff --git a/benchmark/README.md b/benchmark/README.md index ee9979d49d..54c5c8eb23 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -202,11 +202,21 @@ Its default-off factory otherwise preserves the native routes. With `"evaluation_opportunity": true` in the worker declaration, the capture hook reads that registration's released history and complete public result details, next round counters, accepted submission clock, remaining cooldown, task/native -source identity and original trial deadline. The provider uses the existing +source identity and original trial deadline. It also preserves the native-selected +incumbent's opaque submission archive as `evaluation_incumbent`, separate from the +current working candidate. Native ranking chooses its original released round; +the result identity, archive digest and byte count bind its download to the same +judge epoch and consumed-round state. Missing or moved selected bytes reject +capture; no selected result is explicitly recorded as absent. The transport never +extracts archive members, overwrites a prior participant or silently accepts a +truncated response. Its checked file witness must match the descriptor before +the artifact owner publishes the unit. The provider uses the existing native admission/history locks and rejects an outstanding grade, unreconciled consumed round, missing result details, automatic-evaluation history or closed registration. It never returns tokens or hidden task material and never grades, -registers or refills an opportunity. The local `sforge_checkpoint_opportunity_v0` +registers or refills an opportunity. Download is limited to the same registration +and a matching capture digest; source movement requires a new owned boundary. +The local `sforge_checkpoint_opportunity_v0` artifact records these facts; it grants no restored admission. Disabling capture means omitting both the judge flag and worker declaration option on a new run. diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md index 28bb20b3f1..475ad06320 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md @@ -731,8 +731,11 @@ by the existing obligation. It does not cover replans first created inside a run Host wake. The optional native participant reads complete history through Codex, seals its rollout/index/paged stores and verifies native forks at fixed guest paths. An explicit checkpoint-aware native judge also exports released public result -details, accepted counters/clock, remaining cooldown and original trial deadline -without grading or refilling admission; pending or unreconciled grades reject. +details, accepted counters/clock, remaining cooldown and original trial deadline, +plus the opaque native-selected incumbent archive bound to its released result. +The incumbent can precede the working candidate; native ranking remains the owner. +Missing bytes, changed capture bindings or mismatched file witnesses reject before +publication, without grading or refilling admission; pending or unreconciled grades reject. Participant/effect reconciliation, fresh research-state adoption, preserved evaluator admission and useful continued work remain separate acceptance work; operator metadata and native history equality cannot certify them. diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md index 90965ab698..e52a8bd869 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md @@ -611,7 +611,10 @@ quota guard，在模型调用前捕获 wake 开始已投影的准入 replan， 去重；尚不覆盖运行中 Host wake 首次产生的 replan。可选 native 参与者通过 Codex 读取完整历史，封存 rollout/索引/分页库并验证固定 guest 路径下的原生 分叉。显式 checkpoint-aware native judge 还可导出已公开结果明细、受理计数/ -时钟、剩余冷却及原试验期限；不评测、不补发机会，未完成或未对账评测会拒绝。 +时钟、剩余冷却及原试验期限，并封存与已公开结果绑定的原生选优 incumbent +原始 archive。它可能早于工作区候选；排序仍由原生规则决定。缺失源码、捕获 +绑定变化或文件 witness 不符均在发布前拒绝；不评测、不补发机会，未完成或 +未对账评测会拒绝。 参与者/效果 reconciliation、新研究状态采用、目标侧评测准入及实际有效 接续仍各自待验收；operator metadata 或历史一致不能认证这些条件。 From 6050541e5b9d6c3d37b766eecc6b3f7a276c7aab Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 02:17:20 +0800 Subject: [PATCH 12/69] feat(checkpoint): capture owned LoopX state as inert participants Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/runtime/checkpoint.py | 76 ++++++++- .../test_research_checkpoint_capture.py | 145 +++++++++++++++++- 2 files changed, 219 insertions(+), 2 deletions(-) diff --git a/benchmark/runtime/checkpoint.py b/benchmark/runtime/checkpoint.py index ddd8e221d8..bdf06181fc 100644 --- a/benchmark/runtime/checkpoint.py +++ b/benchmark/runtime/checkpoint.py @@ -36,13 +36,15 @@ def parse_checkpoint_request(raw: bytes) -> dict[str, Any]: """Validate the same explicit declaration before staging and before capture.""" request = json.loads(raw) if (not isinstance(request, dict) or - set(request) - {"destination", "components", "point_metadata", "native_session", "evaluation_opportunity", "trigger"} or + set(request) - {"destination", "components", "point_metadata", "native_session", "evaluation_opportunity", "loopx_state", "trigger"} or not {"destination", "components", "point_metadata"}.issubset(request)): raise ValueError("checkpoint request requires destination, components and point_metadata") if "native_session" in request and type(request["native_session"]) is not bool: raise ValueError("checkpoint native_session must be an explicit boolean") if "evaluation_opportunity" in request and type(request["evaluation_opportunity"]) is not bool: raise ValueError("checkpoint evaluation_opportunity must be an explicit boolean") + if "loopx_state" in request and type(request["loopx_state"]) is not bool: + raise ValueError("checkpoint loopx_state must be an explicit boolean") if "trigger" in request and request["trigger"] != "fresh_replan": raise ValueError("checkpoint trigger must be fresh_replan or omitted") if not isinstance(request["components"], list) or not request["components"]: @@ -52,6 +54,70 @@ def parse_checkpoint_request(raw: bytes) -> dict[str, Any]: return request +def _loopx_state_components(env, components, output_dir): + """Adapt the existing backup owner's inventory for one isolated trial. + + No second state-discovery rule, archive execution or authority adoption. + The runtime owns its original evidence, quota and effect journals; this + adapter declares those private files to the inert artifact owner only. + """ + from loopx.history import load_registry + from loopx.state_backup import build_state_backup_plan + + project, runtime, registry = (Path(env[key]) for key in ( + "LOOPX_PROJECT", "LOOPX_RUNTIME_ROOT", "LOOPX_REGISTRY")) + for path in (project, runtime, registry): + if not path.is_absolute() or path.resolve(strict=True) != path: + raise ValueError("LoopX checkpoint routes must be normalized physical paths") + raw_registry = registry.read_bytes() + source = load_registry(registry) + goals = source.get("goals", []) + if (len(goals) != 1 or goals[0].get("id") != env["LOOPX_GOAL_ID"] or + env["LOOPX_AGENT_ID"] not in goals[0].get("coordination", {}).get("registered_agents", [])): + raise ValueError("LoopX state capture requires this worker's single-Goal trial registry") + # Reject lexical route aliases before backup discovery resolves them. + goal = goals[0] + if Path(goal["repo"]) != project or Path(source.get("common_runtime_root", "")) != runtime: + raise ValueError("LoopX checkpoint routes differ from the registered trial") + state = Path(goal["state_file"]) + state = state if state.is_absolute() else Path(goal["repo"]) / state + if not state.is_file(): + raise ValueError("LoopX checkpoint requires its registered active state source") + routes = [state, *(project / path for path in ( + ".loopx", ".codex/goals", ".claude/goals", ".local/goals"))] + if any(path.exists() and path.resolve(strict=True) != path for path in routes): + raise ValueError("LoopX checkpoint state routes must not contain symlinks") + plan = build_state_backup_plan(project=project, runtime_root=runtime, + registry_path=registry, output_dir=output_dir, + include_registry_projects=False, include_automations=False, include_skills=False) + if plan["warnings"]: + raise ValueError("LoopX backup inventory is incomplete; reconcile its source routes before capture") + # Select physical parent roots once. A caller-declared parent can already + # cover a discovered file; do not duplicate the same runtime/registry/state. + entries = list(components) + targets = sorted(plan["included"], key=lambda entry: len(Path(entry["source_path"]).parts)) + for target in targets: + path = Path(target["source_path"]) + if any(path.is_relative_to(Path(entry["source_path"])) for entry in entries): + continue + if any(Path(entry["source_path"]).is_relative_to(path) for entry in entries): + raise ValueError("LoopX state inventory overlaps a narrower declared component; declare its parent once") + entries.append({"id": f"loopx_state_{len(entries)}", "source_path": str(path)}) + coverage = [{"key": target["key"], "source_path": target["source_path"], + "component_id": next(entry["id"] for entry in entries + if Path(target["source_path"]).is_relative_to(Path(entry["source_path"]))) } + for target in plan["included"]] + if registry.read_bytes() != raw_registry: + raise ValueError("LoopX registry moved during participant inventory") + registry_component = next(entry for entry in entries if registry.is_relative_to(Path(entry["source_path"]))) + return entries, {"goal_id": env["LOOPX_GOAL_ID"], "agent_id": env["LOOPX_AGENT_ID"], + "source_inventory": coverage, "registry_sha256": hashlib.sha256(raw_registry).hexdigest(), + "registry_component_id": registry_component["id"], + "registry_member_path": (registry.relative_to(Path(registry_component["source_path"])).as_posix() + if registry != Path(registry_component["source_path"]) else ""), + "execution_authority_granted": False} + + def capture_before_replan(env: dict[str, str], receipt: dict[str, Any], body: str) -> str: """Preload the original guard once, then capture before invoking the model. @@ -219,6 +285,8 @@ def capture_after_wake(env: dict[str, str], receipt: dict[str, Any], *, "request_sha256": hashlib.sha256(raw).hexdigest(), } components = request["components"] + if request.get("loopx_state"): + components, metadata["loopx_state"] = _loopx_state_components(env, components, destination) opportunity = None if request.get("evaluation_opportunity"): from .sforge_checkpoint import capture_opportunity @@ -243,6 +311,12 @@ def capture_after_wake(env: dict[str, str], receipt: dict[str, Any], *, "action": "probe", "components": components, "deadline_epoch_ms": deadline * 1000, }, timeout=max(.1, deadline - time.time()), retry_safe=False) + if request.get("loopx_state"): + declared = metadata["loopx_state"] + witness = probe["source_witness"][declared["registry_component_id"]] + member = next((entry for entry in witness if entry["path"] == declared["registry_member_path"]), None) + if member is None or member.get("sha256") != declared["registry_sha256"]: + raise ValueError("LoopX registry differs from the scoped participant inventory") if opportunity is not None and opportunity["native_incumbent"] is not None: selected = opportunity["native_incumbent"] witness = probe["source_witness"].get("evaluation_incumbent", []) diff --git a/tests/control_plane/test_research_checkpoint_capture.py b/tests/control_plane/test_research_checkpoint_capture.py index 14f6d488a7..99d96db01f 100644 --- a/tests/control_plane/test_research_checkpoint_capture.py +++ b/tests/control_plane/test_research_checkpoint_capture.py @@ -167,7 +167,7 @@ def run(*args): solution = project / "solution.py" solution.write_text("retained_candidate = 7\n") request = tmp_path / "request.json" - request.write_text(json.dumps({"trigger": "fresh_replan", "destination": str(logs / "checkpoints"), + request.write_text(json.dumps({"trigger": "fresh_replan", "loopx_state": True, "destination": str(logs / "checkpoints"), "components": [{"id": "code", "source_path": str(solution)}], "point_metadata": {"purpose": "synthetic original settlement and replan ordering"}})) env = {**os.environ, "LOOPX_CLI": str(Path(sys.executable).parent / "loopx"), @@ -210,3 +210,146 @@ def fail_probe(method, params, **kwargs): capture_before_replan(env, again, "original instructions") assert again["checkpoint"]["reason"] == "replan_already_captured", json.dumps(again) assert (target / "checkpoint.json").read_bytes() == captured + + +@pytest.mark.parametrize('provider', ['file', 'sqlite']) +def test_controller_participant_keeps_canonical_state_and_journals_inert(tmp_path, monkeypatch, provider): + from canonical_authority_fixture import initialize_canonical_authority + from loopx.control_plane.coordination.runtime_shadow import build_todo_runtime_shadow_projection + from loopx.control_plane.goals.first_party_host_admission import capture_first_party_host_goal_ref + from loopx.control_plane.quota.settlement import read_heartbeat_settlement + import subprocess + + def authority_head(root): + module = Path(__file__).resolve().parents[2]/'loopx/control_plane/coordination/local_authority_provider.ts' + script = ('import {openLocalAuthorityStore} from '+json.dumps(module.as_uri())+';' + "let input='';for await(const chunk of process.stdin)input+=chunk;" + 'const x=JSON.parse(input);const store=await openLocalAuthorityStore(x.root,x.goal,{}, {existingOnly:true});' + 'process.stdout.write(JSON.stringify(await store.loadAuthority()));') + result = subprocess.run(['node','--no-warnings','--experimental-strip-types','--input-type=module','-e',script], + input=json.dumps({'root':str(root),'goal':cli.GOAL_ID}),capture_output=True,text=True,timeout=45) + assert result.returncode == 0, result.stderr + return json.loads(result.stdout) + + isolate_sqlite_runtime(tmp_path, monkeypatch) + project, runtime, registry = cli._write_fixture(tmp_path) + source_ref = capture_first_party_host_goal_ref(registry_path=registry, goal_id=cli.GOAL_ID) + state = project / f'.codex/goals/{cli.GOAL_ID}/ACTIVE_GOAL_STATE.md' + code, listed = cli._run_cli(registry, runtime, 'todo', 'list', '--goal-id', cli.GOAL_ID) + assert code == 0, listed + initialize_canonical_authority(runtime, cli.GOAL_ID, + build_todo_runtime_shadow_projection(goal_id=cli.GOAL_ID, todos=listed['todos'], + handoff_mode='hard_lease', leases=[]), + state_path=state, provider=provider) + def run(*args): + code, result = cli._run_cli(registry, runtime, *args) + assert code == 0 and result['ok'] is True, result + return result + lease = run('task-lease', 'acquire', '--goal-id', cli.GOAL_ID, '--todo-id', cli.TODO_ID, + '--owner', cli.AGENT_ID, '--idempotency-key', 'source-lease') + binding = ('--agent-id', cli.AGENT_ID, '--todo-id', cli.TODO_ID, '--turn-instance-id', cli.TURN_ID) + run('quota', 'should-run', '--goal-id', cli.GOAL_ID, '--codex-app', *binding) + vision = tmp_path / 'vision.json' + vision.write_text(json.dumps({'schema_version':'goal_vision_replan_contract_v0', 'state':'vision_patch_proposed', + 'vision_patch':{'acceptance_summary':'Controlled participant probe passed; research continuation remains.'}, + 'path_delta':{'outcome':'continue','prior_assumption':'Capture requires settled original work.', + 'observed_reality':'Isolated source work was verified.', 'retained':['Participant capture'], + 'evidence_refs':['evidence:controller-fixture']}})) + run('refresh-state', '--goal-id', cli.GOAL_ID, *binding, '--classification','controller_fixture_verified', + '--delivery-batch-scale','multi_surface','--delivery-outcome','outcome_progress','--agent-vision-json',str(vision)) + run('quota','spend-slot','--goal-id',cli.GOAL_ID,*binding,'--source','heartbeat','--execute') + before = run('todo','list','--goal-id',cli.GOAL_ID,'--todo-id',cli.TODO_ID) + original_head = authority_head(runtime) + assert original_head['status'] == 'loaded' + source_settlement = read_heartbeat_settlement(runtime, goal_id=cli.GOAL_ID,agent_id=cli.AGENT_ID, + todo_id=cli.TODO_ID,turn_instance_id=cli.TURN_ID,registry_path=registry) + assert source_settlement.progress['state'] == 'settled' + logs = tmp_path / 'logs' + wakes = logs / 'wakes' + (wakes / cli.TURN_ID).mkdir(parents=True) + (project / 'solver.py').write_text('candidate = 7\n') + target = logs / 'checkpoints/controller' + request = tmp_path / 'request.json' + request.write_text(json.dumps({'destination':str(target), 'loopx_state':True, + 'components':[{'id':'code','source_path':str(project/'solver.py')}], 'point_metadata':{}})) + env = {'LOOPX_CHECKPOINT_REQUEST':str(request),'LOOPX_WAKE_LOG_DIR':str(wakes), + 'LOOPX_PROJECT':str(project),'LOOPX_RUNTIME_ROOT':str(runtime),'LOOPX_REGISTRY':str(registry), + 'LOOPX_GOAL_ID':cli.GOAL_ID,'LOOPX_AGENT_ID':cli.AGENT_ID} + receipt = {'ok':True,'stage':'execute','mode':'heartbeat','turn_id':cli.TURN_ID} + capture_after_wake(env, receipt) + assert receipt['checkpoint'].get('captured') is True, json.dumps(receipt['checkpoint']) + manifest = json.loads((target/'checkpoint.json').read_text()) + coverage = manifest['point_metadata']['loopx_state']['source_inventory'] + covered = {entry['source_path']:entry['component_id'] for entry in coverage} + assert str(runtime) in covered and str(registry) in covered and str(state) in covered + materialized = tmp_path / 'materialized' + effect_runtime_result('runtime.checkpoint_artifacts', {'action':'materialize','root':str(target), + 'expected_sha256':receipt['checkpoint']['sha256'],'destination':str(materialized)}) + copied_runtime = materialized/'components'/covered[str(runtime)] + assert authority_head(copied_runtime) == original_head + copied_code, copied_todos = cli._run_cli(registry, copied_runtime, 'todo', 'list', + '--goal-id', cli.GOAL_ID, '--todo-id', cli.TODO_ID) + assert copied_code == 0, copied_todos + assert copied_todos['todo'] == before['todo'] + assert copied_todos['authority_read']['source_authority'] == provider + '_v0' + copied = read_heartbeat_settlement(copied_runtime,goal_id=cli.GOAL_ID,agent_id=cli.AGENT_ID, + todo_id=cli.TODO_ID,turn_instance_id=cli.TURN_ID,registry_path=registry) + assert copied.progress['state'] == 'settled' and copied.identity.value == source_settlement.identity.value + registry_copy = materialized/'components'/covered[str(registry)]/manifest['point_metadata']['loopx_state']['registry_member_path'] + assert registry_copy.read_bytes() == registry.read_bytes() + assert manifest['research_point_qualified'] is False and manifest['execution_authority_granted'] is False + assert source_ref is None # Preserve this legacy fixture; capture does not create an exact GoalRef. + assert lease['lease']['status'] == 'active' + after = run('todo','list','--goal-id',cli.GOAL_ID,'--todo-id',cli.TODO_ID) + assert after['todo'] == before['todo'] + assert after['authority_read']['provider_revision'] == before['authority_read']['provider_revision'] + + +@pytest.mark.parametrize('gap', ['shared_registry','wrong_actor','wrong_runtime','missing_state','symlink_state','overlap']) +def test_loopx_participant_rejects_wrong_or_ambiguous_routes(tmp_path, gap): + from benchmark.runtime.checkpoint import _loopx_state_components + project, runtime, registry = cli._write_fixture(tmp_path) + runtime.mkdir() + (project/'solver.py').write_text('candidate = 7\n') + env = {'LOOPX_PROJECT':str(project),'LOOPX_RUNTIME_ROOT':str(runtime),'LOOPX_REGISTRY':str(registry), + 'LOOPX_GOAL_ID':cli.GOAL_ID,'LOOPX_AGENT_ID':cli.AGENT_ID} + state = project/f'.codex/goals/{cli.GOAL_ID}/ACTIVE_GOAL_STATE.md' + components = [{'id':'code','source_path':str(project/'solver.py')}] + source = json.loads(registry.read_text()) + if gap == 'shared_registry': + source['goals'].append({**source['goals'][0],'id':'other-goal'}) + registry.write_text(json.dumps(source)) + elif gap == 'wrong_actor': + env['LOOPX_AGENT_ID'] = 'other-agent' + elif gap == 'wrong_runtime': + other = tmp_path/'other' + other.mkdir() + env['LOOPX_RUNTIME_ROOT'] = str(other) + elif gap == 'missing_state': + state.unlink() + elif gap == 'symlink_state': + state.unlink() + state.symlink_to(project/'solver.py') + else: + components.append({'id':'narrow','source_path':str(registry)}) + with pytest.raises(ValueError): + _loopx_state_components(env, components, tmp_path/'logs/checkpoints/point') + assert not (tmp_path/'logs').exists() + + +def test_loopx_participant_preserves_legacy_and_current_owned_goal_directories(tmp_path): + from benchmark.runtime.checkpoint import _loopx_state_components, parse_checkpoint_request + project, runtime, registry = cli._write_fixture(tmp_path) + # These are legitimate controller partitions, not two Goal registrations. + # Inventory follows the backup owner, not a basename-based Goal classifier. + for name in (cli.GOAL_ID, cli.GOAL_ID+'-partition'): + (runtime/'goals'/name).mkdir(parents=True) + components, metadata = _loopx_state_components({'LOOPX_PROJECT':str(project), + 'LOOPX_RUNTIME_ROOT':str(runtime),'LOOPX_REGISTRY':str(registry), + 'LOOPX_GOAL_ID':cli.GOAL_ID,'LOOPX_AGENT_ID':cli.AGENT_ID}, [], tmp_path/'logs/checkpoints/point') + assert sum(entry['source_path'] == str(runtime) for entry in components) == 1 + assert metadata['execution_authority_granted'] is False + with pytest.raises(ValueError, match='explicit boolean'): + parse_checkpoint_request(json.dumps({'destination':'/logs/agent/checkpoints/point', + 'components':[{'id':'code','source_path':'/tmp/solver.py'}], 'point_metadata':{}, + 'loopx_state':'true'}).encode()) From 26bdd1089b4e964f61360b2d00b1bbc3ce0add8e Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 02:17:20 +0800 Subject: [PATCH 13/69] docs(checkpoint): explain isolated controller state capture Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/README.md | 22 +++++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-) diff --git a/benchmark/README.md b/benchmark/README.md index 54c5c8eb23..1740ec3608 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -220,11 +220,18 @@ The local `sforge_checkpoint_opportunity_v0` artifact records these facts; it grants no restored admission. Disabling capture means omitting both the judge flag and worker declaration option on a new run. -Declare the workspace's authorized code/research files, LoopX runtime and -registry, effective configuration and installed source/profile pins as separate -non-overlapping components. The native and opportunity participants do not imply -that omitted workspace state was saved. Every source remains private; inspect -the verified member inventory before qualifying a complete research point. +For an isolated single-Goal trial, `"loopx_state": true` adds the existing +state-backup owner's inventory: registered active state, project research state, +runtime authority/effect/quota journals and registry. The declared worker and +physical routes must match that registry; missing registered state, incomplete +inventory, symlink routes and narrower overlapping components reject capture. +File and SQLite authority providers remain inert artifacts after materialization; +copying their old leases or settlement records grants no new execution authority. + +Declare authorized workspace code/research files, effective configuration and +installed source/profile pins as non-overlapping components. The native and +opportunity participants do not imply that omitted state was saved. Every source +remains private; inspect the verified inventory before qualifying a complete point. The current Host hook cannot capture a new replan created **inside an already running model wake** before that decision. Those boundaries still need a qualified @@ -238,3 +245,8 @@ runtime receipt 绑定摘要；programmatic adapter 也可指定已放入 guest 路径。省略选项即关闭，Official/native-goal 不接受。产物 验证和隔离物化不授予执行权，不代表研究点已资格化。请求中的 metadata 只是 证据，不能代替会话、效果、状态采用及评测机会的联合验收。 + +显式 `"loopx_state": true` 复用现有 state-backup inventory，捕获单 Goal 试验的 +注册状态、项目研究状态、runtime journal 和 registry；File/SQLite authority +物化后仍是惰性历史。缺少来源、共享注册表、路径别名或重叠声明会拒绝，捕获 +不授予新的租约、配额、效果或执行权，不能替代完整接续验收。 From 8168c31e6c77fc3dbbbdb29ad8f283adb9e1cff9 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 03:00:10 +0800 Subject: [PATCH 14/69] test(checkpoint): distinguish fresh registration from restored admission Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- .../tests/test_edgebench_feedback_docker.py | 26 +++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/benchmark/tests/test_edgebench_feedback_docker.py b/benchmark/tests/test_edgebench_feedback_docker.py index f2a743e55d..6148984671 100644 --- a/benchmark/tests/test_edgebench_feedback_docker.py +++ b/benchmark/tests/test_edgebench_feedback_docker.py @@ -208,6 +208,32 @@ def candidate(score): files={"archive": ("candidate.tar.gz", first_archive)}, timeout=15) assert cooled.status_code == 429 # Reading the checkpoint cannot refill admission. assert app.state.judge.resolve_token(token) == before + # A fresh identity is a valid new trial, but not a same-point branch. + # Even with the same remaining deadline and declared limits, native + # registration resets the counter and makes its first submission eligible. + cold = requests.post(url + "/api/v1/register", json={"task_id": "fixture", + "run_id": "synthetic-new-registration", "admin_secret": app.state.judge.admin_secret, + "submission_cooldown": 3600, "max_agent_submissions": 12}, timeout=10) + cold.raise_for_status() + cold_token = cold.json()["token"] + cold_point = capture_opportunity({**env, "SFORGE_TOKEN": cold_token}, timeout=10, + incumbent_path=tmp_path / "cold-incumbent.tar.gz") + assert cold_point["next_agent_round"] == 1 and cold_point["remaining_cooldown_seconds"] == 0 + assert cold_point["released_history"] == [] and cold_point["native_incumbent"] is None + assert cold_point["max_agent_submissions"] - (cold_point["next_agent_round"] - 1) == 12 + assert point["max_agent_submissions"] - (point["next_agent_round"] - 1) == 11 + cold_accepted = requests.post(url + "/api/v1/submit", data={"token": cold_token}, + files={"archive": ("candidate.tar.gz", first_archive)}, timeout=15) + cold_accepted.raise_for_status() + assert cold_accepted.json()["round_id"] == "agent-1" + wait_for(lambda: not app.state.judge._outstanding_by_run, timeout=60) + # Neither copying the incumbent bytes nor replaying its grade restored + # consumed source rounds/history or its currently active cooldown. + cold_after = capture_opportunity({**env, "SFORGE_TOKEN": cold_token}, timeout=10, + incumbent_path=tmp_path / "cold-after-incumbent.tar.gz") + assert cold_after["next_agent_round"] == 2 and len(cold_after["released_history"]) == 1 + assert cold_after["released_history"][0]["submission_id"] != point["released_history"][0]["submission_id"] + assert app.state.judge.resolve_token(token) == before with pytest.raises(ValueError, match="original trial deadline"): capture_opportunity({**env, "LOOPX_PHASE_DEADLINE_EPOCH": "0"}, timeout=10, incumbent_path=tmp_path / "closed.tar.gz") From ab28d6b839d6208ef03ea6412e63b2b5e46d94ff Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 03:00:11 +0800 Subject: [PATCH 15/69] docs(checkpoint): qualify the native continuation admission boundary Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- .../rfcs/hierarchical-agent-stride-control-v0.md | 10 ++++++++++ .../rfcs/hierarchical-agent-stride-control-v0.zh-CN.md | 7 +++++++ 2 files changed, 17 insertions(+) diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md index 475ad06320..d1fe07fb5d 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md @@ -740,6 +740,16 @@ Participant/effect reconciliation, fresh research-state adoption, preserved eval admission and useful continued work remain separate acceptance work; operator metadata and native history equality cannot certify them. +A native fresh registration is not restored admission. In the controlled native +Docker qualification, one of twelve source submissions was consumed and its +cooldown remained active. A same-limit fresh registration admitted `agent-1` +immediately with twelve unused rounds. Regrading the retained archive created a +new result, without restoring the source history or opportunity state. This is +valid new-trial behavior; it cannot substitute for a same-point continuation. +The evaluator owner must separately qualify destination initialization from the +bound history/counters and remaining cooldown duration, together with the Host's +remaining total budget. Copying captured token rows is not admission proof. + An inert verification unit grants no execution authority. The destination uses fresh experiment/Host identities and the existing Goal/session/state admission paths. Historical leases, quota, timers, pending effects and submission tokens diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md index e52a8bd869..e696477293 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md @@ -618,6 +618,13 @@ Codex 读取完整历史，封存 rollout/索引/分页库并验证固定 guest 参与者/效果 reconciliation、新研究状态采用、目标侧评测准入及实际有效 接续仍各自待验收；operator metadata 或历史一致不能认证这些条件。 +原生新注册不等于恢复准入。真实原生 Docker 资格测试中，source 已消耗十二次 +中的一次，且冷却仍有效；同额度的新注册却立即受理 `agent-1`，保留十二次未用 +机会。重评已保存 archive 产生新的结果，没有恢复 source 的历史或机会状态。 +这是合法的新试验行为，不能代替同一研究点接续。Evaluator owner 须另行资格化 +从已绑定历史/计数及剩余冷却时长初始化目标状态，同时保留 Host 剩余总预算； +复制旧 token 行不是准入证明。 + inert verification unit 不授予执行权。目标侧使用新的实验/Host 身份，通过既有 Goal/session/state admission 进入。历史租约、quota、timer、pending effect 和 submission token 不能变成目标侧权限；未知效果先按原身份 reconcile，再决定 From 32bd782ac7f1f6edc8a36f9146c8ddf046cb5a9b Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 03:44:55 +0800 Subject: [PATCH 16/69] fix(benchmark): isolate optional checkpoint declaration failure Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/runtime/checkpoint.py | 74 ++++++++++------ .../test_research_checkpoint_capture.py | 84 +++++++++++++++++++ 2 files changed, 132 insertions(+), 26 deletions(-) diff --git a/benchmark/runtime/checkpoint.py b/benchmark/runtime/checkpoint.py index bdf06181fc..9984a53664 100644 --- a/benchmark/runtime/checkpoint.py +++ b/benchmark/runtime/checkpoint.py @@ -118,6 +118,34 @@ def _loopx_state_components(env, components, output_dir): "execution_authority_granted": False} +def _mark_replan_attempt(root: Path, obligation_id: str, turn_id: str, request_sha256: str | None) -> bool: + """The first boundary wins even when its optional declaration was lost.""" + if not root.is_absolute() or root != root.resolve(): + raise ValueError("replan checkpoint root must be a normalized physical path") + attempts = root / ".replan-attempts" + attempts.mkdir(parents=True, exist_ok=True, mode=0o700) + if attempts != attempts.resolve(): + raise ValueError("replan attempt directory must not contain symlinks") + attempt_path = attempts / (obligation_id + ".json") + attempt = {"obligation_id": obligation_id, "turn_id": turn_id, + "request_sha256": request_sha256} + try: + with attempt_path.open("x", encoding="utf-8") as marker: + marker.write(json.dumps(attempt)) + attempt_path.chmod(0o600) + return True + except FileExistsError: + if attempt_path.is_symlink(): + raise ValueError("replan attempt must not be a symlink") + prior_attempt = json.loads(attempt_path.read_text()) + # None means the first declaration was unavailable, not permission to + # replace that lost point with a repaired file and later model state. + if (prior_attempt["obligation_id"] != obligation_id or + prior_attempt["request_sha256"] not in (None, request_sha256)): + raise ValueError("existing replan attempt has a different declaration") + return False + + def capture_before_replan(env: dict[str, str], receipt: dict[str, Any], body: str) -> str: """Preload the original guard once, then capture before invoking the model. @@ -128,8 +156,13 @@ def capture_before_replan(env: dict[str, str], receipt: dict[str, Any], body: st path = env.get("LOOPX_CHECKPOINT_REQUEST") if not path: return body - request, raw = read_checkpoint_request(path) - if request.get("trigger") != "fresh_replan": + declaration_error = None + try: + request, raw = read_checkpoint_request(path) + except (OSError, ValueError) as error: + declaration_error = error + request, raw = {}, None + if declaration_error is None and request.get("trigger") != "fresh_replan": return body if receipt.get("mode") != "heartbeat" or receipt.get("stage") != "execute": raise ValueError("fresh_replan checkpoint trigger requires a heartbeat execute wake") @@ -156,16 +189,25 @@ def capture_before_replan(env: dict[str, str], receipt: dict[str, Any], body: st "action": "select_replan", "quota": guard, "goal_id": env["LOOPX_GOAL_ID"], "agent_id": env["LOOPX_AGENT_ID"], "turn_instance_id": receipt["turn_id"], }) + root = Path(env["LOOPX_WAKE_LOG_DIR"]).parent / "checkpoints" + if declaration_error is not None: + if intent["capture"]: + _mark_replan_attempt(root, intent["obligation_id"], receipt["turn_id"], None) + raise declaration_error if not intent["capture"]: receipt["checkpoint"]["reason"] = intent["reason"] else: - root = Path(env["LOOPX_WAKE_LOG_DIR"]).parent / "checkpoints" - requested_root = Path(request["destination"]) - if not requested_root.is_absolute() or requested_root != requested_root.resolve() or requested_root != root: - raise ValueError("fresh_replan destination must be this trial's checkpoints directory") destination = root / intent["obligation_id"] if destination.parent != root or destination != destination.resolve(): raise ValueError("replan checkpoint identity must select a direct child") + if not destination.exists() and not _mark_replan_attempt( + root, intent["obligation_id"], receipt["turn_id"], hashlib.sha256(raw).hexdigest(), + ): + receipt["checkpoint"]["reason"] = "pre_decision_capture_already_attempted" + return preloaded_body + requested_root = Path(request["destination"]) + if not requested_root.is_absolute() or requested_root != requested_root.resolve() or requested_root != root: + raise ValueError("fresh_replan destination must be this trial's checkpoints directory") if destination.exists(): # Readback of the first point, never overwrite it with later work. manifest = json.loads((destination / "checkpoint.json").read_text()) @@ -181,26 +223,6 @@ def capture_before_replan(env: dict[str, str], receipt: dict[str, Any], body: st # A failed first capture is a missed point. Once its model is # allowed to decide/work, a retry must not relabel later state # as the original pre-decision checkpoint. - attempts = root / ".replan-attempts" - attempts.mkdir(parents=True, exist_ok=True, mode=0o700) - if attempts != attempts.resolve(): - raise ValueError("replan attempt directory must not contain symlinks") - attempt_path = attempts / (intent["obligation_id"] + ".json") - attempt = {"obligation_id": intent["obligation_id"], "turn_id": receipt["turn_id"], - "request_sha256": hashlib.sha256(raw).hexdigest()} - try: - with attempt_path.open("x", encoding="utf-8") as marker: - marker.write(json.dumps(attempt)) - attempt_path.chmod(0o600) - except FileExistsError: - if attempt_path.is_symlink(): - raise ValueError("replan attempt must not be a symlink") - prior_attempt = json.loads(attempt_path.read_text()) - if (prior_attempt["obligation_id"] != attempt["obligation_id"] or - prior_attempt["request_sha256"] != attempt["request_sha256"]): - raise ValueError("existing replan attempt has a different declaration") - receipt["checkpoint"]["reason"] = "pre_decision_capture_already_attempted" - return preloaded_body prior_paths = sorted(Path(env["LOOPX_WAKE_LOG_DIR"]).glob("wake-*/receipt.json")) prior_paths = [p for p in prior_paths if p.parent.name != receipt["turn_id"]] if not prior_paths: diff --git a/tests/control_plane/test_research_checkpoint_capture.py b/tests/control_plane/test_research_checkpoint_capture.py index 99d96db01f..d7c9f27ac0 100644 --- a/tests/control_plane/test_research_checkpoint_capture.py +++ b/tests/control_plane/test_research_checkpoint_capture.py @@ -42,6 +42,90 @@ def forbidden(*args, **kwargs): assert receipt["checkpoint"]["reason"] == "capture_failed" +@pytest.mark.parametrize("trigger", [None, "fresh_replan"]) +@pytest.mark.parametrize("failure", ["missing", "denied", "malformed"]) +def test_worker_keeps_work_running_when_staged_capture_declaration_fails( + tmp_path, monkeypatch, trigger, failure, +): + """Real worker/session/guard; synthetic executable replaces only the model.""" + import os + import sys + from benchmark.runtime.worker import run_once + + isolate_sqlite_runtime(tmp_path, monkeypatch) + project, runtime, registry = cli._write_fixture(tmp_path) + if trigger: + cli._configure_autonomous_replan_fixture(project, runtime, registry) + skills = tmp_path / "skills" + skills.mkdir() + binary = tmp_path / "codex" + binary.write_text(f"#!{sys.executable}\n" + + "import json,pathlib,sys\n" + "body=sys.stdin.read()\n" + "pathlib.Path('host-input.txt').write_text(body)\n" + "pathlib.Path('solution.py').write_text('after_decision = 11\\n')\n" + "print(json.dumps({'type':'thread.started','thread_id':'synthetic-capture-session'}))\n") + binary.chmod(0o755) + logs = tmp_path / "logs" + request = tmp_path / "request.json" + declaration = json.dumps({"destination": str(logs / "checkpoints" / "point"), + "components": [{"id": "code", "source_path": str(project / "solution.py")}], + "point_metadata": {"purpose": "synthetic worker observer failure"}, + **({"trigger": trigger, "destination": str(logs / "checkpoints")} if trigger else {})}) + request.write_text(declaration) + env = {**os.environ, "LOOPX_PROJECT": str(project), "LOOPX_CLI": str(Path(sys.executable).parent / "loopx"), + "LOOPX_CODEX_HOME": str(tmp_path / "home"), "LOOPX_WAKE_LOG_DIR": str(logs / "wakes"), + "LOOPX_SHARED_SKILLS": str(skills), "CODEX_BIN": str(binary), + "MODEL_NAME": "fixture", "REASONING_EFFORT": "high", "OPENAI_BASE_URL": "http://localhost:8123/v1", + "OPENAI_API_KEY": "fixture-key", "LOOPX_RUNTIME_ROOT": str(runtime), + "LOOPX_REGISTRY": str(registry), "LOOPX_GOAL_ID": cli.GOAL_ID, "LOOPX_AGENT_ID": cli.AGENT_ID} + monkeypatch.setattr("benchmark.runtime.worker.heartbeat_body", lambda *a, **kw: "original work contract") + off = run_once(env) + assert off["ok"] and "checkpoint" not in off + (project / "host-input.txt").unlink() + if failure == "missing": + request.unlink() + elif failure == "malformed": + request.write_text("{") + read_bytes = Path.read_bytes + with monkeypatch.context() as faults: + if failure == "denied": + def denied_read(path): + if path == request: + raise PermissionError("synthetic staged-file permission loss") + return read_bytes(path) + faults.setattr(Path, "read_bytes", denied_read) + failed = run_once({**env, "LOOPX_CHECKPOINT_REQUEST": str(request)}) + assert failed["ok"] and failed["session"]["session_id"] == "synthetic-capture-session" + assert (project / "host-input.txt").read_text().endswith("original work contract") + assert failed["checkpoint"]["reason"] == "capture_failed" + assert not failed["checkpoint"]["captured"] + saved = json.loads((logs / "wakes" / failed["turn_id"] / "receipt.json").read_text()) + assert saved == failed + request.write_text(declaration) + if trigger: + # A repaired declaration cannot label the model's later code as the + # original missed pre-decision point, even on exact guard replay. + replay = {"turn_id": failed["turn_id"], "mode": "heartbeat", "stage": "execute", "ok": False} + body = capture_before_replan({**env, "LOOPX_CHECKPOINT_REQUEST": str(request)}, replay, + "original work contract") + assert replay["checkpoint"]["reason"] == "pre_decision_capture_already_attempted", replay + assert body.endswith("original work contract") + assert not list((logs / "checkpoints").glob("replan-*/checkpoint.json")) + recovered = run_once({**env, "LOOPX_CHECKPOINT_REQUEST": str(request)}) + assert recovered["ok"] + if trigger: + # An unavailable declaration is optional; an unavailable original + # admission owner is not. This boundary must still stop before work. + import subprocess + (project / "host-input.txt").unlink() + request.unlink() + registry.rename(registry.with_suffix(".unavailable")) + with pytest.raises(subprocess.CalledProcessError): + run_once({**env, "LOOPX_CHECKPOINT_REQUEST": str(request)}) + assert not (project / "host-input.txt").exists() + + @pytest.mark.parametrize("opportunity,corrupt", [(False, False), (True, False), (True, True)]) def test_real_cli_settlement_capture_verification_and_inert_materialization(tmp_path, monkeypatch, opportunity, corrupt): isolate_sqlite_runtime(tmp_path, monkeypatch) From de9ee64c0131fb1b9b2b502a2618a8377573d93c Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 03:44:55 +0800 Subject: [PATCH 17/69] docs(benchmark): pin checkpoint provider qualification path Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/README.md | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/benchmark/README.md b/benchmark/README.md index 1740ec3608..2dca10d655 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -174,6 +174,12 @@ work. The after-wake hook cannot substitute a post-decision state. Capture failu remain explicit and do not settle or complete the new work. The first attempt is also retained: if it fails and the model proceeds, retries cannot substitute a later state for that missed pre-decision point. +Losing, denying access to or corrupting the staged declaration records a capture +failure while the ordinary wake continues. The Host still obtains and delivers +the original quota guard; an unavailable admission owner stops work as before. +If that guard projects a fresh replan, the missed first attempt is retained even +without readable declaration bytes, so repairing the file cannot capture later +state under the same obligation. Subsequent distinct replans remain eligible. Add `"native_session": true` to include Codex's completed native cursor, all retained history pages, rollout/compaction records and native index/history SQLite @@ -198,6 +204,37 @@ python -m benchmark.edgebench.checkpoint_judge --checkpoint-capture \ --tasks-dir /path/to/tasks --log-dir /path/to/private-run-logs ``` +This adapter's supported SForge source is the public compatibility fork at +[`0e20fb499ee248f012d6467c00b3dec00a29fe85`](https://github.com/huangruiteng/EdgeBench/commit/0e20fb499ee248f012d6467c00b3dec00a29fe85). +It exposes the existing native judge/admission owner; the upstream `sforge==1.1.0` +distribution does not provide that seam. Enabling capture against an unsupported +provider fails before adding routes. This pin qualifies the capture prerequisite, +not an upstream release or restored admission. From this LoopX checkout, prepare +an isolated Python 3.13 environment and verify the actual imported source: + +```sh +CKPT_PROVIDER_ROOT=$(mktemp -d) +git clone https://github.com/huangruiteng/EdgeBench.git "$CKPT_PROVIDER_ROOT/provider" +git -C "$CKPT_PROVIDER_ROOT/provider" checkout --detach 0e20fb499ee248f012d6467c00b3dec00a29fe85 +uv venv --python 3.13 "$CKPT_PROVIDER_ROOT/venv" +uv pip install --python "$CKPT_PROVIDER_ROOT/venv/bin/python" \ + -e '.[test]' -e "$CKPT_PROVIDER_ROOT/provider" 'httpx>=0.28,<1' +"$CKPT_PROVIDER_ROOT/venv/bin/python" -c 'import sforge,loopx; print(sforge.__file__); print(loopx.__file__)' +"$CKPT_PROVIDER_ROOT/venv/bin/python" -m pytest -q benchmark/tests/test_checkpoint_judge.py +# Linux with an owned disposable Docker daemon; grades only synthetic fixtures. +LOOPX_EDGEBENCH_DOCKER_SMOKE=1 "$CKPT_PROVIDER_ROOT/venv/bin/python" -m pytest -q \ + benchmark/tests/test_edgebench_feedback_docker.py::test_checkpoint_opportunity_uses_native_admission_and_real_grade +``` + +The Docker journey checks a real grade, remaining rounds/cooldown, incumbent bytes, +and the cold-registration counterexample; it does not restore evaluator authority. +Native session qualification is separate: the tested Host is `codex-cli 0.160.0`. +Retain the complete paginated history/compaction and actual native stores from an +authorized completed source, then run capture and fork in isolation with that +same binary and fixed guest paths. Compare every retained Turn ID and the full +history digest, including exclusion of later history; mocked pages alone do not +qualify this participant. No model call or LoopX registration is implied by fork. + Its default-off factory otherwise preserves the native routes. With `"evaluation_opportunity": true` in the worker declaration, the capture hook reads that registration's released history and complete public result details, @@ -245,6 +282,11 @@ runtime receipt 绑定摘要；programmatic adapter 也可指定已放入 guest 路径。省略选项即关闭，Official/native-goal 不接受。产物 验证和隔离物化不授予执行权，不代表研究点已资格化。请求中的 metadata 只是 证据，不能代替会话、效果、状态采用及评测机会的联合验收。 +已放入 guest 的声明文件丢失、权限失败或损坏只记录捕获失败，原工作仍按 +完整的原始 quota guard 继续；准入 owner 不可用仍停止。首次 replan 捕获 +失败后，修复声明不得把后来的状态补记成同一原点。评测参与者使用上文明确的 +公开 SForge fork/pin 和 Python 3.13；普通 upstream 安装缺少该 seam 时明确 +拒绝。Codex 原生会话的真实分页/compaction 与隔离 fork 须单独验证。 显式 `"loopx_state": true` 复用现有 state-backup inventory，捕获单 Goal 试验的 注册状态、项目研究状态、runtime journal 和 registry；File/SQLite authority From 2edcfa24c01c8be3209d22d32e1ffc9de5285287 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 06:35:32 +0800 Subject: [PATCH 18/69] fix(benchmark): retain missed replan points after index failure Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/runtime/checkpoint.py | 33 ++++- .../test_research_checkpoint_capture.py | 123 ++++++++++++++++++ 2 files changed, 153 insertions(+), 3 deletions(-) diff --git a/benchmark/runtime/checkpoint.py b/benchmark/runtime/checkpoint.py index 9984a53664..f92e365e1c 100644 --- a/benchmark/runtime/checkpoint.py +++ b/benchmark/runtime/checkpoint.py @@ -146,6 +146,28 @@ def _mark_replan_attempt(root: Path, obligation_id: str, turn_id: str, request_s return False +def _prior_replan_guard_seen(env: dict[str, str], current: Path, obligation_id: str) -> bool: + """An absent optional index does not erase the original Host preflight. + + Reuse saved guard packets and the typed replan selector. They are already + written before optional capture and before the model can decide or work. + No separate journal, model-text classifier or new replan owner is needed. + """ + for path in sorted(Path(env["LOOPX_WAKE_LOG_DIR"]).glob("*/checkpoint-start-guard.json")): + if path == current: + continue + if path.resolve(strict=True) != path: + raise ValueError("prior checkpoint guard must be a physical Host file") + guard = json.loads(path.read_text()) + prior = effect_runtime_result("runtime.checkpoint_artifacts", { + "action": "select_replan", "quota": guard, "goal_id": env["LOOPX_GOAL_ID"], + "agent_id": env["LOOPX_AGENT_ID"], "turn_instance_id": path.parent.name, + }) + if prior["capture"] and prior["obligation_id"] == obligation_id: + return True + return False + + def capture_before_replan(env: dict[str, str], receipt: dict[str, Any], body: str) -> str: """Preload the original guard once, then capture before invoking the model. @@ -178,6 +200,10 @@ def capture_before_replan(env: dict[str, str], receipt: dict[str, Any], body: st guard = json.loads(guard_run.stdout) if not isinstance(guard, dict) or guard.get("ok") is not True: raise ValueError("checkpoint requires a successful original quota guard") + # A previous preflight on this wake may already have allowed model work, + # even if its optional attempt marker (or final receipt) could not be saved. + # Such reentry can verify an existing point, but cannot capture a later one. + preflight_seen = guard_path.exists() guard_path.write_text(guard_run.stdout, encoding="utf-8") guard_path.chmod(0o600) preloaded_body = (f"Host checkpoint preflight has already executed this wake's original quota guard once. " @@ -200,9 +226,10 @@ def capture_before_replan(env: dict[str, str], receipt: dict[str, Any], body: st destination = root / intent["obligation_id"] if destination.parent != root or destination != destination.resolve(): raise ValueError("replan checkpoint identity must select a direct child") - if not destination.exists() and not _mark_replan_attempt( - root, intent["obligation_id"], receipt["turn_id"], hashlib.sha256(raw).hexdigest(), - ): + if not destination.exists() and (preflight_seen or + _prior_replan_guard_seen(env, guard_path, intent["obligation_id"]) or + not _mark_replan_attempt(root, intent["obligation_id"], receipt["turn_id"], + hashlib.sha256(raw).hexdigest())): receipt["checkpoint"]["reason"] = "pre_decision_capture_already_attempted" return preloaded_body requested_root = Path(request["destination"]) diff --git a/tests/control_plane/test_research_checkpoint_capture.py b/tests/control_plane/test_research_checkpoint_capture.py index d7c9f27ac0..af80543f38 100644 --- a/tests/control_plane/test_research_checkpoint_capture.py +++ b/tests/control_plane/test_research_checkpoint_capture.py @@ -437,3 +437,126 @@ def test_loopx_participant_preserves_legacy_and_current_owned_goal_directories(t parse_checkpoint_request(json.dumps({'destination':'/logs/agent/checkpoints/point', 'components':[{'id':'code','source_path':'/tmp/solver.py'}], 'point_metadata':{}, 'loopx_state':'true'}).encode()) + + +def test_failed_replan_index_cannot_relabel_later_work_after_permissions_recover(tmp_path, monkeypatch): + """Real CLI/worker/permissions; only the model executable is synthetic.""" + import os + import sys + from benchmark.runtime.worker import run_once + + if os.geteuid() == 0: + pytest.skip("Real POSIX permission denial requires an unprivileged test user") + isolate_sqlite_runtime(tmp_path, monkeypatch) + project, runtime, registry = cli._write_fixture(tmp_path) + skills = tmp_path / "skills" + skills.mkdir() + binary = tmp_path / "codex" + binary.write_text(f"#!{sys.executable}\n" + + "import json,pathlib,sys\n" + "pathlib.Path('host-input.txt').write_text(sys.stdin.read())\n" + "pathlib.Path('solution.py').write_text('after_strategy_decision = 999\\n')\n" + "print(json.dumps({'type':'thread.started','thread_id':'synthetic-index-session'}))\n") + binary.chmod(0o755) + logs = tmp_path / "logs" + env = {**os.environ, "LOOPX_PROJECT": str(project), "LOOPX_CLI": str(Path(sys.executable).parent / "loopx"), + "LOOPX_CODEX_HOME": str(tmp_path / "home"), "LOOPX_WAKE_LOG_DIR": str(logs / "wakes"), + "LOOPX_SHARED_SKILLS": str(skills), "CODEX_BIN": str(binary), "MODEL_NAME": "fixture", + "REASONING_EFFORT": "high", "OPENAI_BASE_URL": "http://localhost:8123/v1", + "OPENAI_API_KEY": "fixture-key", "LOOPX_RUNTIME_ROOT": str(runtime), + "LOOPX_REGISTRY": str(registry), "LOOPX_GOAL_ID": cli.GOAL_ID, "LOOPX_AGENT_ID": cli.AGENT_ID} + monkeypatch.setattr("benchmark.runtime.worker.heartbeat_body", lambda *a, **kw: "original work contract") + prior = run_once(env) + assert prior["ok"] + + def run(*args): + code, packet = cli._run_cli(registry, runtime, *args) + assert code == 0, packet + return packet + + vision = tmp_path / "vision.json" + vision.write_text(json.dumps({"schema_version": "goal_vision_replan_contract_v0", "state": "vision_patch_proposed", + "vision_patch": {"acceptance_summary": "Synthetic work completed; original checkpoint qualification remains."}, + "path_delta": {"outcome": "continue", "prior_assumption": "The original worker remains available.", + "observed_reality": "Real worker ran the explicitly synthetic host successfully.", + "retained": ["Checkpoint source boundary"], "evidence_refs": ["evidence:index-permission-fixture"]}})) + binding = ("--agent-id", cli.AGENT_ID, "--todo-id", cli.TODO_ID, "--turn-instance-id", prior["turn_id"]) + run("quota", "should-run", "--goal-id", cli.GOAL_ID, "--codex-app", *binding) + run("refresh-state", "--goal-id", cli.GOAL_ID, *binding, "--classification", "worker_fixture_completed", + "--delivery-batch-scale", "multi_surface", "--delivery-outcome", "outcome_progress", "--agent-vision-json", str(vision)) + run("quota", "spend-slot", "--goal-id", cli.GOAL_ID, *binding, "--source", "heartbeat", "--execute") + index = runtime / "goals" / cli.GOAL_ID / "runs/index.jsonl" + original = index.read_bytes() + cli._configure_autonomous_replan_fixture(project, runtime, registry) + index.write_bytes(original + index.read_bytes()) + solution = project / "solution.py" + solution.write_text("retained_candidate = 7\n") + request = tmp_path / "request.json" + request.write_text(json.dumps({"trigger": "fresh_replan", "destination": str(logs / "checkpoints"), + "components": [{"id": "code", "source_path": str(solution)}], + "point_metadata": {"purpose": "Original research boundary, not post-decision code"}})) + env["LOOPX_CHECKPOINT_REQUEST"] = str(request) + attempts = logs / "checkpoints/.replan-attempts" + attempts.mkdir(parents=True) + attempts.chmod(0o500) + try: + failed = run_once(env) + finally: + attempts.chmod(0o700) + assert failed["ok"] + assert failed["checkpoint"]["reason"] == "capture_failed" + assert failed["checkpoint"]["error_kind"] == "PermissionError" + assert not list(attempts.iterdir()) + saved_path = logs / "wakes" / failed["turn_id"] / "receipt.json" + saved = json.loads(saved_path.read_text()) + assert saved == failed + assert solution.read_text() == "after_strategy_decision = 999\n" + replay = {"turn_id": failed["turn_id"], "mode": "heartbeat", "stage": "execute", "ok": False} + body = capture_before_replan(env, replay, "original work contract") + assert body.endswith("original work contract") + assert not replay["checkpoint"]["captured"], replay + assert replay["checkpoint"]["reason"] == "pre_decision_capture_already_attempted", replay + assert not list((logs / "checkpoints").glob("replan-*/checkpoint.json")) + assert json.loads(saved_path.read_text()) == saved + # A new wake on the same outstanding obligation is also a retry, not a + # new original point. The durable guard remains evidence without its index. + again = run_once(env) + assert again["ok"] and again["turn_id"] != failed["turn_id"] + assert not again["checkpoint"]["captured"] + assert again["checkpoint"]["reason"] == "pre_decision_capture_already_attempted" + assert not list((logs / "checkpoints").glob("replan-*/checkpoint.json")) + # Settle that actual admitted review, then supply a separately declared + # later pair of synthetic observations. The real owner must issue a new + # obligation; the lost point must not suppress this distinct boundary. + guard = json.loads((logs / "wakes" / again["turn_id"] / "checkpoint-start-guard.json").read_text()) + old_obligation = guard["replan_action_packet"]["obligation_id"] + replan_binding = ("--agent-id", cli.AGENT_ID, "--replan-obligation-id", old_obligation, + "--turn-instance-id", again["turn_id"]) + data = json.loads(vision.read_text()) + data["vision_patch"]["acceptance_summary"] = "The missed original point stays missed; later research boundaries still qualify separately." + data["path_delta"]["observed_reality"] = "Real permission recovery did not recapture later code as the original research point." + data["path_delta"].update(schema_version="goal_path_delta_v0", outcome="replan") + vision.write_text(json.dumps(data)) + run("refresh-state", "--goal-id", cli.GOAL_ID, *replan_binding, + "--classification", "missed_original_point_preserved", "--delivery-batch-scale", "multi_surface", + "--delivery-outcome", "outcome_progress", "--agent-vision-json", str(vision), + "--autonomous-replan-recorded", "--repair-delta-kind", "goal_vision_patch", + "--progress-result-class", "advanced", "--progress-surface-id", "surface:checkpoint-boundary", + "--progress-hypothesis-id", "hypothesis:missed-index", "--progress-probe-kind", "probe:real-permission", + "--progress-evidence-id", "evidence:index-recovery") + run("quota", "spend-slot", "--goal-id", cli.GOAL_ID, *replan_binding, "--source", "heartbeat", "--execute") + from datetime import datetime, timedelta, timezone + source_history = index.read_bytes() + cli._configure_autonomous_replan_fixture(project, runtime, registry) + observations = [json.loads(line) for line in index.read_text().splitlines()] + observed_at = datetime.now(timezone.utc) + for number, row in enumerate(observations): + row["generated_at"] = (observed_at + timedelta(microseconds=number)).isoformat() + row["progress_observation"].update(surface_id="surface-later", hypothesis_id="hypothesis-later") + Path(row["json_path"]).write_text(json.dumps(row)) + index.write_bytes(source_history + b"".join((json.dumps(row) + "\n").encode() for row in observations)) + fresh = run_once(env) + assert fresh["ok"] and fresh["checkpoint"]["captured"], fresh + point = Path(fresh["checkpoint"]["destination"]) + assert point.name != old_obligation + assert (point / "components/code").read_text() == "after_strategy_decision = 999\n" From a7576dd0338bbff93633f61f92f5d397150863cc Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 07:12:13 +0800 Subject: [PATCH 19/69] feat(benchmark): transport native judge capsules through controller authority Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/edgebench/checkpoint_controller.py | 220 +++++++++++++++++++ 1 file changed, 220 insertions(+) create mode 100644 benchmark/edgebench/checkpoint_controller.py diff --git a/benchmark/edgebench/checkpoint_controller.py b/benchmark/edgebench/checkpoint_controller.py new file mode 100644 index 0000000000..a979e87493 --- /dev/null +++ b/benchmark/edgebench/checkpoint_controller.py @@ -0,0 +1,220 @@ +"""Controller-only transport for the native SForge checkpoint API. + +Native admission/ranking retain their native owner. The existing typed artifact +owner seals and verifies the bank bytes. Nothing here belongs in a solver payload. +""" +from __future__ import annotations + +import argparse +import hashlib +import json +import math +import os +import re +import shutil +import tempfile +from pathlib import Path +from urllib.error import HTTPError +from urllib.parse import urlsplit +from urllib.request import HTTPRedirectHandler, Request, build_opener + +from loopx.control_plane.effect_runtime import effect_runtime_result + + +class _NoRedirect(HTTPRedirectHandler): + def redirect_request(self, req, fp, code, msg, headers, newurl): + return None # Never forward controller credentials to another origin. + + +def _physical(path: Path) -> Path: + path = Path(path) + if not path.is_absolute() or path.resolve(strict=True) != path: + raise ValueError("Controller paths must be normalized, physical and absolute") + return path + + +def _controller(path: Path) -> dict: + path = _physical(path) + if path.stat().st_mode & 0o077: + raise ValueError("Controller configuration must be owner-only") + value = json.loads(path.read_bytes()) + if not isinstance(value, dict) or set(value) != {"url", "admin_secret", "bank_root"}: + raise ValueError("Controller configuration requires url, admin_secret and bank_root") + if not isinstance(value["admin_secret"], str) or not value["admin_secret"]: + raise ValueError("Fresh controller authority is unavailable") + url = urlsplit(value["url"]) + if (url.scheme not in {"http", "https"} or not url.hostname or url.username or + url.password or url.query or url.fragment or url.path not in {"", "/"}): + raise ValueError("Controller URL must identify one explicit HTTP origin") + value["bank_root"] = _physical(Path(value["bank_root"])) + return value + + +def _post(controller: dict, route: str, body: dict, timeout: float) -> dict: + if not math.isfinite(timeout) or timeout <= 0: + raise ValueError("Controller timeout must be finite and positive") + request = Request(controller["url"].rstrip("/") + route, + data=json.dumps({**body, "admin_secret": controller["admin_secret"]}).encode(), + headers={"Content-Type": "application/json"}, method="POST") + try: + with build_opener(_NoRedirect).open(request, timeout=timeout) as response: + result = json.load(response) + except HTTPError as error: + # A remote body can echo submitted credentials. Keep only the status. + raise ValueError(f"Native checkpoint operation rejected: HTTP {error.code}") from None + if not isinstance(result, dict): + raise ValueError("Native checkpoint operation returned an invalid object") + return result + + +def _reference(value: dict) -> dict: + if (set(value) != {"checkpoint_id", "manifest_sha256", "task_id", "source_run_id", "opportunity", "incumbent"} or + not re.fullmatch(r"[0-9a-f]{32}", value.get("checkpoint_id", "")) or + not re.fullmatch(r"[0-9a-f]{64}", value.get("manifest_sha256", ""))): + raise ValueError("Native checkpoint reference is malformed") + return value + + +def _bound_bank(folder: Path, reference: dict) -> None: + """Byte/source binding only; native restore owns eligibility and validity.""" + raw = _physical(folder / "manifest.json").read_bytes() + digest = hashlib.sha256(raw).hexdigest() + witnessed = _physical(folder / "manifest.sha256").read_text() + packet = json.loads(raw) + if (digest != reference["manifest_sha256"] or witnessed != digest or + packet.get("schema_version") != "sforge_judge_checkpoint_v1" or + any(packet.get(key) != reference[key] for key in ( + "task_id", "source_run_id", "opportunity", "incumbent"))): + raise ValueError("Native bank bytes differ from the captured reference") + expected = {"manifest.json", "manifest.sha256"} + incumbent = reference["incumbent"] + if incumbent is not None: + expected.add("incumbent.tar.gz") + archive = _physical(folder / "incumbent.tar.gz").read_bytes() + if (len(archive) != incumbent["archive_bytes"] or + hashlib.sha256(archive).hexdigest() != incumbent["archive_sha256"]): + raise ValueError("Native selected archive differs from the captured reference") + if {p.name for p in folder.iterdir()} != expected: + raise ValueError("Native bank contains undeclared members") + + +def capture(controller_file: Path, *, token: str, capture_id: str, destination: Path, + timeout: float = 30) -> dict: + """Capture one native-owned point without exporting controller configuration.""" + if not re.fullmatch(r"[0-9a-f]{32}", capture_id): + raise ValueError("Capture identity must be the declared 32-hex native identity") + destination = Path(destination) + _physical(destination.parent) + if not destination.is_absolute() or destination != destination.resolve() or destination.exists(): + raise ValueError("Capture export requires a fresh physical destination") + controller = _controller(controller_file) + reference = _reference(_post(controller, "/api/v1/checkpoint/capture", + {"token": token, "capture_id": capture_id}, timeout)) + if reference["checkpoint_id"] != capture_id: + raise ValueError("Native capture returned a different identity") + folder = _physical(controller["bank_root"] / capture_id) + _bound_bank(folder, reference) + components = [{"id": "native_judge", "source_path": str(folder)}] + probe = effect_runtime_result("runtime.checkpoint_artifacts", {"action": "probe", "components": components}) + result = effect_runtime_result("runtime.checkpoint_artifacts", { + "action": "capture", "components": components, "destination": str(destination), + "expected_source_sha256": probe["source_sha256"], + "point_metadata": {"native_reference": reference}, + }, retry_safe=False) + # Bind the actually copied bytes, not merely a prior source observation. + _bound_bank(destination / "components/native_judge", reference) + return {**result, "native_reference": reference} + + +def restore(controller_file: Path, *, root: Path, expected_sha256: str, run_id: str, + timeout: float = 30) -> dict: + """Import verified inert bytes, then ask fresh native authority to admit them. + + Does not restore a model, LoopX Goal, effect authority or whole-trial budget. + An uncertain response is not replayed under a different run or capture id. + """ + if not re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9_.-]{0,127}", run_id): + raise ValueError("Continuation requires one explicit native run component") + controller = _controller(controller_file) + root = _physical(root) + effect_runtime_result("runtime.checkpoint_artifacts", { + "action": "verify", "root": str(root), "expected_sha256": expected_sha256}) + manifest = json.loads((root / "checkpoint.json").read_bytes()) + if [c["id"] for c in manifest["components"]] != ["native_judge"]: + raise ValueError("Native import requires its single declared capsule component") + reference = _reference(manifest["point_metadata"]["native_reference"]) + if run_id == reference["source_run_id"]: + raise ValueError("Continuation requires a fresh run identity") + folder = root / "components/native_judge" + _bound_bank(folder, reference) + bank = controller["bank_root"] + target = bank / reference["checkpoint_id"] + stage = Path(tempfile.mkdtemp(prefix=".loopx-native-import-", dir=bank)) + try: + effect_runtime_result("runtime.checkpoint_artifacts", { + "action": "materialize", "root": str(root), "expected_sha256": expected_sha256, + "destination": str(stage / "unit"), + }, retry_safe=False) + imported = stage / "unit/components/native_judge" + _bound_bank(imported, reference) + if target.exists(): + _bound_bank(_physical(target), reference) # No overwrite, even on retry. + else: + # POSIX rename can replace an empty destination. Reserve its name + # exclusively; interruption leaves an unusable point, never grants. + target.mkdir(mode=0o700) + try: + imported.rename(target) + except BaseException: + target.rmdir() + raise + descriptor = os.open(bank, os.O_RDONLY | os.O_DIRECTORY) + try: + os.fsync(descriptor) + finally: + os.close(descriptor) + return _post(controller, "/api/v1/checkpoint/restore", { + "checkpoint_id": reference["checkpoint_id"], "manifest_sha256": reference["manifest_sha256"], + "run_id": run_id, + }, timeout) + finally: + shutil.rmtree(stage) + + +def main(argv=None) -> None: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--controller-file", required=True, type=Path) + commands = parser.add_subparsers(dest="command", required=True) + capture_parser = commands.add_parser("capture") + capture_parser.add_argument("--token-file", required=True, type=Path) + capture_parser.add_argument("--capture-id", required=True) + capture_parser.add_argument("--destination", required=True, type=Path) + restore_parser = commands.add_parser("restore") + restore_parser.add_argument("--root", required=True, type=Path) + restore_parser.add_argument("--expected-sha256", required=True) + restore_parser.add_argument("--run-id", required=True) + restore_parser.add_argument("--result-file", required=True, type=Path) + args = parser.parse_args(argv) + if args.command == "capture": + source_token = _physical(args.token_file).read_text().strip() + result = capture(args.controller_file, token=source_token, + capture_id=args.capture_id, destination=args.destination) + print(json.dumps(result)) + else: + # Reserve before admission. A lost output is explicit recovery, never a + # fresh registration that silently refills the captured opportunity. + _physical(args.result_file.parent) + if args.result_file != args.result_file.resolve(): + raise ValueError("Continuation result must be a normalized physical path") + with args.result_file.open("x") as result_file: + args.result_file.chmod(0o600) + result = restore(args.controller_file, root=args.root, + expected_sha256=args.expected_sha256, run_id=args.run_id) + json.dump(result, result_file) + result_file.flush() + os.fsync(result_file.fileno()) + print(json.dumps({"result_file": str(args.result_file), "run_id": args.run_id})) + + +if __name__ == "__main__": + main() From 8b300663bf5c9354f9d486bf18a2d9817922a17c Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 07:12:25 +0800 Subject: [PATCH 20/69] test(benchmark): qualify native controller transport and retained opportunities Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/tests/test_checkpoint_controller.py | 300 ++++++++++++++++++ 1 file changed, 300 insertions(+) create mode 100644 benchmark/tests/test_checkpoint_controller.py diff --git a/benchmark/tests/test_checkpoint_controller.py b/benchmark/tests/test_checkpoint_controller.py new file mode 100644 index 0000000000..9e739d9db0 --- /dev/null +++ b/benchmark/tests/test_checkpoint_controller.py @@ -0,0 +1,300 @@ +"""Actual native HTTP and typed artifact transport; grading is synthetic here.""" +from __future__ import annotations + +import copy +import io +import json +import os +import socket +import tarfile +import threading +import time +from contextlib import contextmanager +from dataclasses import replace +from types import SimpleNamespace +from unittest.mock import Mock + +import pytest + +from benchmark.edgebench import checkpoint_controller as transport + + +@contextmanager +def server(app): + import uvicorn + with socket.socket() as probe: + probe.bind(("127.0.0.1", 0)) + port = probe.getsockname()[1] + host = uvicorn.Server(uvicorn.Config(app, host="127.0.0.1", port=port, + log_level="error", access_log=False)) + thread = threading.Thread(target=host.run) + thread.start() + try: + wait(lambda: host.started) + yield f"http://127.0.0.1:{port}" + finally: + host.should_exit = True + thread.join(timeout=10) + assert not thread.is_alive() + + +def wait(predicate): + until = time.monotonic() + 30 + while time.monotonic() < until: + value = predicate() + if value: + return value + time.sleep(.01) + raise AssertionError("Native synthetic HTTP journey did not settle") + + +@pytest.fixture +def controllers(tmp_path, monkeypatch): + pytest.importorskip("sforge.harness.checkpoint", reason="Requires native checkpoint API source pin") + from sforge.harness import judge_server as native + from sforge.harness.config import SForgeConfig + tasks = tmp_path / "tasks" + tasks.mkdir() + (tasks / "BENCHMARK.yaml").write_text("name: transport-test\nbase_images: {python: {}}\n") + (tasks / "fixture.json").write_text(json.dumps({ + "task_id": "fixture", "name": "Synthetic transport", "base_image": "python", + "platform": "linux/arm64", "cwd": "/tmp", "internet": False, + "submit_paths": ["candidate.json"], "submit_exclude": [], + "work": {"image_tag": "fixture", "agent_query": "Synthetic fixture"}, + "judge": {"image_tag": "fixture", "eval_timeout": 15, "parser": "structured_json", + "selection": "score_first", "score_direction": "maximize", "eval_cmd": "true"}, + })) + backend = Mock(backend_name="docker") + backend.client.images.get.return_value.id = "sha256:synthetic-transport-image" + monkeypatch.setattr(native, "create_backend_from_config", lambda config: backend) + grades = [] + def grade(**kwargs): + grades.append(kwargs["archive"]) + kwargs["log_dir"].mkdir(parents=True, exist_ok=True) + (kwargs["log_dir"] / "submission.tar.gz").write_bytes(kwargs["archive"]) + return SimpleNamespace(to_dict=lambda: {"score": float(kwargs["archive"]), + "valid": True, "pass_rate": 1.0, "passed": 1, "failed": 0, "total_tests": 1}) + monkeypatch.setattr(native, "judge_submission", grade) + config = SForgeConfig(tasks_dir=tasks, log_dir=tmp_path / "source") + source = native.create_app(config, checkpoints=True) + destination = native.create_app(replace(config, log_dir=tmp_path / "branch"), checkpoints=True) + try: + with server(source) as first, server(destination) as second: + files = [] + for name, app, url in (("source", source, first), ("branch", destination, second)): + bank = app.state.judge.config.log_dir / "checkpoints" + bank.mkdir(parents=True, mode=0o700) + path = tmp_path / (name + "-controller.json") + path.write_text(json.dumps({"url": url, "admin_secret": app.state.judge.admin_secret, + "bank_root": str(bank)})) + path.chmod(0o600) + files.append(path) + yield source, destination, first, second, files, grades + finally: + for app in (source, destination): + app.state.judge._reaper_stop.set() + app.state.judge._reaper_thread.join(timeout=1) + + +def register(app, url, **options): + import requests + response = requests.post(url + "/api/v1/register", json={"task_id": "fixture", "run_id": "source", + "admin_secret": app.state.judge.admin_secret, **options}, timeout=5) + response.raise_for_status() + return response.json()["token"] + + +def submit(url, token, score): + import requests + response = requests.post(url + "/api/v1/submit", data={"token": token}, + files={"archive": ("candidate.tar.gz", str(score).encode())}, timeout=5) + if response.status_code == 200: + identifier = response.json()["submission_id"] + wait(lambda: requests.get(url + "/api/v1/result/" + identifier, timeout=5).json()["status"] == "completed") + return response + + +def test_native_controller_transport_preserves_original_point_and_fresh_authority(controllers, tmp_path, capsys): + import requests + source, destination, first, second, files, grades = controllers + token = register(source, first, max_agent_submissions=3, submission_cooldown=0) + assert submit(first, token, 7).status_code == 200 + assert submit(first, token, 3).status_code == 200 + # Retain a future cooldown without changing the original accepted clock. + source.state.judge.tokens[token]["submission_cooldown"] = 3600 + before = copy.deepcopy((source.state.judge.tokens, source.state.judge.submissions, source.state.judge.run_history)) + token_file = tmp_path / "token" + token_file.write_text(token) + exported = tmp_path / "exported" + transport.main(["--controller-file", str(files[0]), "capture", "--token-file", str(token_file), + "--capture-id", "a" * 32, "--destination", str(exported)]) + receipt = json.loads(capsys.readouterr().out) + assert receipt["research_point_qualified"] is False + assert receipt["native_reference"]["incumbent"]["round"] == "agent-1" + assert (exported / "components/native_judge/incumbent.tar.gz").read_bytes() == b"7" + manifest = json.loads((exported / "components/native_judge/manifest.json").read_text()) + assert len(manifest["history"]) == 2 and manifest["opportunity"]["next_agent"] == 3 + assert 0 < manifest["opportunity"]["remaining_cooldown_seconds"] <= 3600 + for secret in (token, source.state.judge.admin_secret, destination.state.judge.admin_secret): + assert secret not in json.dumps(receipt) + assert all(secret.encode() not in p.read_bytes() for p in exported.rglob("*") if p.is_file()) + assert (source.state.judge.tokens, source.state.judge.submissions, source.state.judge.run_history) == before + # Later source state cannot replace this first native capture identity. + source.state.judge.tokens[token]["submission_cooldown"] = 0 + assert submit(first, token, 99).status_code == 200 + repeated = transport.capture(files[0], token=token, capture_id="a" * 32, destination=tmp_path / "same-point") + assert repeated["native_reference"] == receipt["native_reference"] + private_result = tmp_path / "fresh-result.json" + transport.main(["--controller-file", str(files[1]), "restore", "--root", str(exported), + "--expected-sha256", receipt["sha256"], "--run-id", "branch", "--result-file", str(private_result)]) + output = capsys.readouterr().out + fresh = json.loads(private_result.read_text())["token"] + assert fresh != token and fresh not in output + assert private_result.stat().st_mode & 0o077 == 0 and len(grades) == 3 + history = requests.get(second + "/api/v1/history", params={"token": fresh}, timeout=5).json() + assert history["best_score"] == 7 and history["best_round"] == "agent-1" and history["agent_submissions"] == 2 + assert submit(second, fresh, 1).status_code == 429 + assert source.state.judge.get_run_history("source", "fixture")["best_score"] == 99 + with pytest.raises(ValueError, match="HTTP 409"): + transport.restore(files[1], root=exported, expected_sha256=receipt["sha256"], run_id="branch") + assert len(grades) == 3 + + +@pytest.mark.parametrize("fault", ["wrong_admin", "corrupt", "wrong_digest", "aliased", "existing_bank"]) +def test_invalid_controller_import_never_registers_or_regrades(controllers, tmp_path, fault): + source, destination, first, second, files, grades = controllers + token = register(source, first) + assert submit(first, token, 7).status_code == 200 + exported = tmp_path / "exported" + point = transport.capture(files[0], token=token, capture_id="b" * 32, destination=exported) + root, digest = exported, point["sha256"] + if fault == "wrong_admin": + config = json.loads(files[1].read_text()) + config["admin_secret"] = "not-current-admin" + files[1].write_text(json.dumps(config)) + elif fault == "corrupt": + (exported / "components/native_judge/incumbent.tar.gz").write_bytes(b"99") + elif fault == "wrong_digest": + digest = "0" * 64 + elif fault == "aliased": + root = tmp_path / "alias" + root.symlink_to(exported, target_is_directory=True) + else: + bank = destination.state.judge.config.log_dir / "checkpoints" / ("b" * 32) + bank.mkdir() + (bank / "foreign").write_text("keep") + with pytest.raises((ValueError, RuntimeError, FileNotFoundError)): + transport.restore(files[1], root=root, expected_sha256=digest, run_id="branch") + assert destination.state.judge.tokens == {} and len(grades) == 1 + if fault == "existing_bank": + assert (bank / "foreign").read_text() == "keep" + + +def test_redirect_never_forwards_controller_authority(tmp_path): + pytest.importorskip("fastapi") + from fastapi import FastAPI + from fastapi.responses import RedirectResponse + first, second = FastAPI(), FastAPI() + received = [] + @second.post("/steal") + def steal(): + received.append(True) + return {} + with server(second) as target: + @first.post("/api/v1/checkpoint/capture") + def redirect(): + return RedirectResponse(target + "/steal", status_code=307) + with server(first) as origin: + config = {"url": origin, "admin_secret": "synthetic-credential"} + with pytest.raises(ValueError, match="HTTP 307"): + transport._post(config, "/api/v1/checkpoint/capture", {"token": "source"}, 5) + assert received == [] + + +@pytest.mark.skipif(os.environ.get("LOOPX_EDGEBENCH_DOCKER_SMOKE") != "1", + reason="Requires explicit isolated native Docker qualification") +def test_real_docker_controller_export_import_and_next_grade(tmp_path, monkeypatch): + import uuid + import docker + import requests + from sforge.harness import checkpoint, judge_server as native + from sforge.harness.config import SForgeConfig + + name = "loopx-controller-" + uuid.uuid4().hex[:10] + client = docker.from_env() + image = client.images.get("python:3.12-slim") + image.tag(name + ".judge.fixture", tag="fixture") + tasks = tmp_path / "tasks" + tasks.mkdir() + (tasks / "BENCHMARK.yaml").write_text(f"name: {name}\nbase_images: {{python: {{}}}}\n") + (tasks / "fixture.json").write_text(json.dumps({ + "task_id": "fixture", "name": "Synthetic real grade", "base_image": "python", + "platform": "linux/arm64", "cwd": "/tmp", "internet": False, + "submit_paths": ["candidate.json"], "submit_exclude": [], + "work": {"image_tag": "fixture", "agent_query": "Synthetic fixture"}, + "judge": {"image_tag": "fixture", "eval_timeout": 20, "parser": "structured_json", + "selection": "score_first", "score_direction": "maximize", + "eval_cmd": "python3 -c 'import json; print(json.dumps(dict(score=json.load(open(\"candidate.json\"))[\"score\"], valid=True)))'"}, + })) + config = SForgeConfig(tasks_dir=tasks, log_dir=tmp_path / "source", judge_cpu_limit=1, judge_mem_limit="256m") + source = native.create_app(config, checkpoints=True) + destination = native.create_app(replace(config, log_dir=tmp_path / "branch"), checkpoints=True) + grades = [] + original = native.judge_submission + def actual_grade(**kwargs): + grades.append(kwargs["archive"]) + return original(**kwargs) + monkeypatch.setattr(native, "judge_submission", actual_grade) + def grade(url, token, score): + raw = io.BytesIO() + with tarfile.open(fileobj=raw, mode="w:gz") as archive: + body = json.dumps({"score": score}).encode() + member = tarfile.TarInfo("candidate.json") + member.size = len(body) + archive.addfile(member, io.BytesIO(body)) + response = requests.post(url + "/api/v1/submit", data={"token": token}, + files={"archive": ("candidate.tar.gz", raw.getvalue())}, timeout=10) + if response.status_code == 200: + sid = response.json()["submission_id"] + report = wait(lambda: (packet if (packet := requests.get(url + "/api/v1/result/" + sid, + timeout=10).json())["status"] in {"completed", "error"} else None)) + assert report["status"] == "completed" and report["report"]["score"] == score + return response + try: + with server(source) as first, server(destination) as second: + files = [] + for label, app, url in (("source", source, first), ("branch", destination, second)): + bank = app.state.judge.config.log_dir / "checkpoints" + bank.mkdir(parents=True, mode=0o700) + path = tmp_path / (label + "-admin.json") + path.write_text(json.dumps({"url": url, "admin_secret": app.state.judge.admin_secret, + "bank_root": str(bank)})) + path.chmod(0o600) + files.append(path) + token = register(source, first, max_agent_submissions=2, submission_cooldown=3600) + assert grade(first, token, 7).status_code == 200 + before = copy.deepcopy((source.state.judge.tokens, source.state.judge.submissions, source.state.judge.run_history)) + exported = tmp_path / "capsule" + point = transport.capture(files[0], token=token, capture_id="c" * 32, destination=exported) + # Admission clock only: HTTP, transport/fsync, Docker and native + # grading remain real. Offline/import IO must not spend cooldown. + now = [time.time() + 7200] + clock = SimpleNamespace(time=lambda: now[0]) + monkeypatch.setattr(native, "time", clock) + monkeypatch.setattr(checkpoint, "time", clock) + fresh = transport.restore(files[1], root=exported, expected_sha256=point["sha256"], run_id="branch")["token"] + assert fresh != token and len(grades) == 1 + assert grade(second, fresh, 3).status_code == 429 + now[0] += 3601 + assert grade(second, fresh, 3).json()["round_id"] == "agent-2" + history = requests.get(second + "/api/v1/history", params={"token": fresh}, timeout=5).json() + assert history["best_score"] == 7 and history["best_round"] == "agent-1" + assert grade(second, fresh, 99).status_code == 429 and len(grades) == 2 + assert (source.state.judge.tokens, source.state.judge.submissions, source.state.judge.run_history) == before + with pytest.raises(ValueError, match="HTTP 409"): + transport.restore(files[1], root=exported, expected_sha256=point["sha256"], run_id="branch") + finally: + for app in (source, destination): + app.state.judge._reaper_stop.set() + app.state.judge._reaper_thread.join(timeout=1) + client.images.remove(name + ".judge.fixture:fixture") From b25cb26d68d122b05204115227f177329ea2d0b9 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 07:12:36 +0800 Subject: [PATCH 21/69] docs(benchmark): qualify controller transport without claiming joint continuation Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/README.md | 60 ++++++++++++++++++- .../hierarchical-agent-stride-control-v0.md | 10 +++- ...rarchical-agent-stride-control-v0.zh-CN.md | 8 ++- 3 files changed, 75 insertions(+), 3 deletions(-) diff --git a/benchmark/README.md b/benchmark/README.md index 2dca10d655..cf0b5a663a 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -273,9 +273,67 @@ remains private; inspect the verified inventory before qualifying a complete poi The current Host hook cannot capture a new replan created **inside an already running model wake** before that decision. Those boundaries still need a qualified Host handoff; a later snapshot must not be labeled as the missed point. Complete -LoopX research-state adoption, restored evaluator admission and useful continued work +LoopX research-state adoption, joint evaluator/Host admission and useful continued work remain open. No automatic retention or extra evaluation opportunity is introduced. +### Controller-only native capsule transport + +The read-only participant above does **not** implement the newer native capsule +API. The separate controller adapter targets +[EdgeBench PR #23](https://github.com/ByteDance-Seed/EdgeBench/pull/23), pinned at +`4db79f4ea332ffd30c265ec7cb076f93471bd201`; this is proposed source, not a released +upstream version. Start that pinned native judge with `sforge serve --checkpoints`. +Its private controller configuration file has exactly `url`, `admin_secret` and +`bank_root` (the native log directory's `checkpoints` directory). Keep this file, +the bank and controller process outside every solver's filesystem/network authority. +An owner-only mode is a storage precaution, not proof of solver exclusion. + +```sh +python -m benchmark.edgebench.checkpoint_controller \ + --controller-file /private/source-controller.json capture \ + --token-file /private/source-token --capture-id 0123456789abcdef0123456789abcdef \ + --destination /private/new-native-export +python -m benchmark.edgebench.checkpoint_controller \ + --controller-file /private/fresh-controller.json restore \ + --root /private/new-native-export --expected-sha256 EXPORT_DIGEST \ + --run-id isolated-branch --result-file /private/fresh-registration.json +``` + +Use a stable declared capture identity for one point; retries read its original +native state, including after the source improves. The existing typed artifact +owner seals and verifies the native capsule. The native owner +alone validates evaluator compatibility, ranking, released feedback, consumed +rounds, resources and retained cooldown, then creates a fresh registration. +The adapter never grades, extracts incumbent archives or adds controller +configuration/live registration token records to the export. Opaque incumbent +bytes and released reports still require private handling; privacy is not certified. +Restore writes its new token only to the explicit +private result file; stdout contains its path. Wrong authority, corrupt bytes, +aliases, foreign bank entries, source-run reuse and duplicate native admission +fail without a fresh evaluation. Redirects cannot forward the admin secret. +After an uncertain restore response, inspect native readback with the controller; +do not switch run identities or perform ordinary fresh registration as a retry. + +This is an independently usable **native participant transport**, still partial +relative to full research continuation. It does not join the worker's replan +barrier, restore Codex/LoopX authority or carry the remaining whole-trial budget. +Never put its controller configuration in `--checkpoint-request` or a worker +payload. Omit both these controller operations and the native `--checkpoints` +flag to disable this transport on a fresh judge; preserved capsules stay inert. +Synthetic native HTTP tests and an explicitly enabled Docker qualification are: + +```sh +python -m pytest -q benchmark/tests/test_checkpoint_controller.py +LOOPX_EDGEBENCH_DOCKER_SMOKE=1 python -m pytest -q \ + benchmark/tests/test_checkpoint_controller.py::test_real_docker_controller_export_import_and_next_grade +``` + +新的 controller transport 使用上述 PR #23 固定源及 native `--checkpoints`， +不复用旧只读接口来伪装接续。admin 配置和 bank 必须由 solver 权限范围外的 +controller 保管，不能放进 worker；0600 不能证明隔离。原生 owner 保留历史、 +已用轮次和剩余冷却，生成新注册；完整研究接续仍需联合会话、LoopX 新权限、 +原点剩余总预算及 replan 前决策边界。此切片不启动研究、不证明收益，也不改历史源。 + 这是研究 checkpoint 的开发前置切片。EdgeBench CLI 的显式 `--checkpoint-request` 会把 host JSON 原文安全放入 heartbeat worker，并在 runtime receipt 绑定摘要；programmatic adapter 也可指定已放入 guest 的请求 diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md index d1fe07fb5d..c9065917a6 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md @@ -736,7 +736,7 @@ plus the opaque native-selected incumbent archive bound to its released result. The incumbent can precede the working candidate; native ranking remains the owner. Missing bytes, changed capture bindings or mismatched file witnesses reject before publication, without grading or refilling admission; pending or unreconciled grades reject. -Participant/effect reconciliation, fresh research-state adoption, preserved evaluator +Participant/effect reconciliation, fresh research-state adoption, joint evaluator/Host admission and useful continued work remain separate acceptance work; operator metadata and native history equality cannot certify them. @@ -750,6 +750,14 @@ The evaluator owner must separately qualify destination initialization from the bound history/counters and remaining cooldown duration, together with the Host's remaining total budget. Copying captured token rows is not admission proof. +The [controller-only native capsule transport](../../../benchmark/README.md#controller-only-native-capsule-transport) +now connects the proposed native initializer to the existing typed artifact owner: +explicit controller authority captures/imports a sealed native bank and requests +fresh native registration. Controller configuration stays out of solver payloads +and exported participants; opaque data remains private and uncertified. This slice preserves native history, selected +archive, counters and cooldown; it does not yet qualify the joint replan barrier, +model/LoopX state adoption, remaining whole-trial budget or useful research. + An inert verification unit grants no execution authority. The destination uses fresh experiment/Host identities and the existing Goal/session/state admission paths. Historical leases, quota, timers, pending effects and submission tokens diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md index e696477293..645cb18b8d 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md @@ -615,7 +615,7 @@ Codex 读取完整历史，封存 rollout/索引/分页库并验证固定 guest 原始 archive。它可能早于工作区候选；排序仍由原生规则决定。缺失源码、捕获 绑定变化或文件 witness 不符均在发布前拒绝；不评测、不补发机会，未完成或 未对账评测会拒绝。 -参与者/效果 reconciliation、新研究状态采用、目标侧评测准入及实际有效 +参与者/效果 reconciliation、新研究状态采用、目标侧评测与 Host 联合准入及实际有效 接续仍各自待验收；operator metadata 或历史一致不能认证这些条件。 原生新注册不等于恢复准入。真实原生 Docker 资格测试中，source 已消耗十二次 @@ -625,6 +625,12 @@ Codex 读取完整历史，封存 rollout/索引/分页库并验证固定 guest 从已绑定历史/计数及剩余冷却时长初始化目标状态，同时保留 Host 剩余总预算； 复制旧 token 行不是准入证明。 +[Controller-only native capsule transport](../../../benchmark/README.md#controller-only-native-capsule-transport) +已将提议中的 native initializer 接入既有 typed 产物 owner：显式 controller +权限捕获/导入封存 native bank，再请求新的原生注册。controller 配置不进入 solver +payload 或导出参与者；opaque 内容仍私有且未获隐私认证。此切片保留原生历史、选优 archive、计数及冷却，仍不 +资格化联合 replan barrier、模型/LoopX 状态采用、剩余总预算或实际有效研究。 + inert verification unit 不授予执行权。目标侧使用新的实验/Host 身份，通过既有 Goal/session/state admission 进入。历史租约、quota、timer、pending effect 和 submission token 不能变成目标侧权限；未知效果先按原身份 reconcile，再决定 From a8a09080670f02d3d0bcda62585056986014de1b Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 07:56:58 +0800 Subject: [PATCH 22/69] fix(benchmark): bind native cursor and workspace to sealed source basis Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/runtime/checkpoint.py | 9 +++++++++ benchmark/runtime/native_checkpoint.py | 4 ++++ 2 files changed, 13 insertions(+) diff --git a/benchmark/runtime/checkpoint.py b/benchmark/runtime/checkpoint.py index f92e365e1c..289301b208 100644 --- a/benchmark/runtime/checkpoint.py +++ b/benchmark/runtime/checkpoint.py @@ -360,6 +360,15 @@ def capture_after_wake(env: dict[str, str], receipt: dict[str, Any], *, "action": "probe", "components": components, "deadline_epoch_ms": deadline * 1000, }, timeout=max(.1, deadline - time.time()), retry_safe=False) + if request.get("native_session"): + # Native readback precedes the byte witness. A completed later Turn + # in that gap must not be sealed under the earlier cursor. Re-read + # after the probe; capture's existing witness comparison then guards + # source writes during this recheck and through atomic publication. + confirmed = native_session_checkpoint(env, session.get("session_id", ""), + timeout=max(.1, deadline - time.time())) + if confirmed != native: + raise ValueError("native checkpoint cursor or participant basis moved before sealing") if request.get("loopx_state"): declared = metadata["loopx_state"] witness = probe["source_witness"][declared["registry_component_id"]] diff --git a/benchmark/runtime/native_checkpoint.py b/benchmark/runtime/native_checkpoint.py index 69a57a81f9..3d595193cd 100644 --- a/benchmark/runtime/native_checkpoint.py +++ b/benchmark/runtime/native_checkpoint.py @@ -71,6 +71,10 @@ def request(method, params): readback = request("thread/read", {"threadId": session_id, "includeTurns": False}) if readback.get("thread", {}).get("id") != session_id: raise ValueError("native checkpoint session identity mismatch") + workspace = Path(env["LOOPX_PROJECT"]) + if (not workspace.is_absolute() or workspace.resolve(strict=True) != workspace or + readback["thread"].get("cwd") != str(workspace)): + raise ValueError("native checkpoint session workspace differs from the declared code basis") turns = _complete_history(request, session_id) ids = [t["id"] for t in turns] members = [] From 5b3d2519cb2810325cb3d1bdea03a6d597adde6f Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 07:57:09 +0800 Subject: [PATCH 23/69] test(benchmark): exercise native store skew through real Host and CLI owners Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/tests/test_native_checkpoint.py | 106 +++++++++++++++++- .../test_research_checkpoint_capture.py | 60 ++++++++++ 2 files changed, 165 insertions(+), 1 deletion(-) diff --git a/benchmark/tests/test_native_checkpoint.py b/benchmark/tests/test_native_checkpoint.py index c6e7108da3..5d5d1d2254 100644 --- a/benchmark/tests/test_native_checkpoint.py +++ b/benchmark/tests/test_native_checkpoint.py @@ -1,6 +1,12 @@ """Native checkpoint history must be complete and cannot become execution authority.""" import hashlib import json +import os +import threading +import time +import uuid +from contextlib import contextmanager +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer import pytest @@ -33,7 +39,7 @@ def request(self, method, params): if method == "initialize": return {} if method == "thread/read": - return {"thread": {"id": "fixture-session"}} + return {"thread": {"id": "fixture-session", "cwd": str(tmp_path)}} assert method == "thread/turns/list" # No model or fork is requested. return pages.pop(0) @@ -77,3 +83,101 @@ def test_foreign_native_session_is_rejected_before_history(tmp_path, monkeypatch with pytest.raises(ValueError, match="identity mismatch"): native_session_checkpoint(env, "foreign-session", timeout=10) assert "thread/turns/list" not in calls + + +@contextmanager +def real_native_source(root, binary, workspace): + """Real native writer/readback; a local static stream replaces only the model. + + No paid model, tool invocation, user credential or benchmark cell. Fixtures + become completed Turns through the native protocol, not copied fake receipts. + """ + from loopx.capabilities.benchmark_toolkit.native_codex_goal import ( + NativeGoalProtocolError, StdioNativeGoalTransport, + ) + home = root / "home" + home.mkdir(parents=True) + calls = [] + class Provider(BaseHTTPRequestHandler): + def log_message(self, *args): + pass + + def do_POST(self): + self.rfile.read(int(self.headers.get("Content-Length", 0))) + calls.append(self.path) + message = {"type": "message", "id": "msg_" + uuid.uuid4().hex, "role": "assistant", + "content": [{"type": "output_text", "text": "Synthetic source result."}]} + result = {"id": "resp_" + uuid.uuid4().hex, "status": "completed", "output": [message], + "usage": {"input_tokens": 1, "output_tokens": 1, "total_tokens": 2}} + events = [{"type": "response.created", "response": {**result, "status": "in_progress", "output": []}}, + {"type": "response.output_item.done", "output_index": 0, "item": message}, + {"type": "response.completed", "response": result}] + raw = "".join("event: " + event["type"] + "\ndata: " + json.dumps(event) + "\n\n" + for event in events).encode() + self.send_response(200) + self.send_header("Content-Type", "text/event-stream") + self.send_header("Content-Length", str(len(raw))) + self.end_headers() + self.wfile.write(raw) + server = ThreadingHTTPServer(("127.0.0.1", 0), Provider) + thread = threading.Thread(target=server.serve_forever) + thread.start() + (home / "config.toml").write_text( + 'model="fixture"\nmodel_provider="fixture"\napproval_policy="never"\nsandbox_mode="read-only"\n' + '[features]\ngoals=false\nresponses_websockets_v2=false\n' + '[model_providers.fixture]\nname="Synthetic local"\n' + f'base_url="http://127.0.0.1:{server.server_port}/v1"\nwire_api="responses"\n' + 'env_key="SYNTHETIC_PROVIDER_KEY"\n') + env = {key: value for key, value in os.environ.items() + if key not in {"OPENAI_API_KEY", "CODEX_API_KEY", "OPENAI_BASE_URL"}} + env.update(CODEX_HOME=str(home), CODEX_BIN=binary, LOOPX_PROJECT=str(workspace), + SYNTHETIC_PROVIDER_KEY="synthetic-local-only", LOOPX_USAGE_PING="0") + try: + with (root / "stderr.txt").open("w") as stderr, StdioNativeGoalTransport.spawn( + [binary, "-c", "features.goals=false", "app-server", "--listen", "stdio://"], + cwd=str(workspace), env=env, stderr=stderr, response_timeout_sec=20, + ) as native: + native.request("initialize", {"clientInfo": {"name": "checkpoint_qualification", "version": "0.1.0"}, + "capabilities": {"experimentalApi": True}}) + native.notify("initialized", {}) + source = native.request("thread/start", {"cwd": str(workspace), "sandbox": "read-only", + "approvalPolicy": "never", "persistExtendedHistory": True})["thread"]["id"] + def complete_turn(): + expected_turn = native.request("turn/start", {"threadId": source, + "input": [{"type": "text", "text": "Synthetic storage fixture. Return its source result."}]})["turn"]["id"] + stop = time.monotonic() + 30 + while time.monotonic() < stop: + try: + page = native.request("thread/turns/list", {"threadId": source, + "sortDirection": "asc", "itemsView": "full", "limit": 100}) + except NativeGoalProtocolError: + # A first live Turn may not have a persisted page yet. + time.sleep(.05) + continue + if (page.get("data") and page["data"][-1].get("id") == expected_turn and + page["data"][-1].get("status") == "completed"): + return page["data"][-1]["id"] + time.sleep(.05) + raise AssertionError("Real native fixture did not complete its source Turn") + complete_turn() + yield env, source, complete_turn, calls + finally: + server.shutdown() + server.server_close() + thread.join() + + +@pytest.mark.skipif(not os.environ.get("LOOPX_NATIVE_CHECKPOINT_BIN"), + reason="Requires an explicit real native Host binary; no model service used") +def test_real_native_workspace_readback_rejects_foreign_code_basis(tmp_path): + from pathlib import Path + workspace = tmp_path / "source-workspace" + workspace.mkdir() + with real_native_source(tmp_path / "native", os.environ["LOOPX_NATIVE_CHECKPOINT_BIN"], workspace) as (env, source, _, calls): + valid = native_session_checkpoint(env, source, timeout=20) + assert valid["metadata"]["guest_workspace"] == str(workspace) + assert len(valid["metadata"]["turn_ids"]) == 1 and calls == ["/v1/responses"] + foreign = Path(env["LOOPX_PROJECT"]).with_name("foreign-workspace") + foreign.mkdir() + with pytest.raises(ValueError, match="workspace"): + native_session_checkpoint({**env, "LOOPX_PROJECT": str(foreign)}, source, timeout=20) diff --git a/tests/control_plane/test_research_checkpoint_capture.py b/tests/control_plane/test_research_checkpoint_capture.py index af80543f38..65d12a3125 100644 --- a/tests/control_plane/test_research_checkpoint_capture.py +++ b/tests/control_plane/test_research_checkpoint_capture.py @@ -2,6 +2,7 @@ from pathlib import Path import json import hashlib +import os import pytest import time @@ -212,6 +213,65 @@ def altered_probe(method, params, **kwargs): assert len(observed) == int(opportunity) +@pytest.mark.skipif(not os.environ.get("LOOPX_NATIVE_CHECKPOINT_BIN"), + reason="Requires an explicit real native Host binary; static local provider only") +@pytest.mark.parametrize("source_moves", [False, True]) +def test_real_native_cursor_must_match_sealed_store_basis(tmp_path, monkeypatch, source_moves): + """Actual CLI settlement, native completed Turns and TS SQLite artifacts.""" + from benchmark.tests.test_native_checkpoint import real_native_source + + isolate_sqlite_runtime(tmp_path, monkeypatch) + project, runtime, registry = cli._write_fixture(tmp_path) + binding = ("--agent-id", cli.AGENT_ID, "--todo-id", cli.TODO_ID, "--turn-instance-id", cli.TURN_ID) + def run(*args): + code, result = cli._run_cli(registry, runtime, *args) + assert code == 0, result + run("quota", "should-run", "--goal-id", cli.GOAL_ID, "--codex-app", *binding) + vision = tmp_path / "vision.json" + vision.write_text(json.dumps({"schema_version": "goal_vision_replan_contract_v0", "state": "vision_patch_proposed", + "vision_patch": {"acceptance_summary": "Synthetic source boundary; full continuation remains open."}, + "path_delta": {"outcome": "continue", "prior_assumption": "Declared source is available.", + "observed_reality": "The isolated fixture source is checked.", "retained": ["Original capture acceptance"], + "evidence_refs": ["evidence:native-source-fixture"]}})) + run("refresh-state", "--goal-id", cli.GOAL_ID, *binding, "--classification", "native_fixture_verified", + "--delivery-batch-scale", "multi_surface", "--delivery-outcome", "outcome_progress", "--agent-vision-json", str(vision)) + run("quota", "spend-slot", "--goal-id", cli.GOAL_ID, *binding, "--source", "heartbeat", "--execute") + logs = tmp_path / "logs" + wake = logs / "wakes" / cli.TURN_ID + wake.mkdir(parents=True) + destination = logs / "checkpoints/point" + destination.parent.mkdir() + candidate = project / "solution.py" + candidate.write_text("source_candidate = 7\n") + request = tmp_path / "request.json" + request.write_text(json.dumps({"destination": str(destination), "components": [ + {"id": "code", "source_path": str(candidate)}], "native_session": True, + "point_metadata": {"meaning": "Native/CLI/TS fixture, not research or model utility"}})) + with real_native_source(tmp_path / "native", os.environ["LOOPX_NATIVE_CHECKPOINT_BIN"], project) as (env, source, complete, calls): + env.update(LOOPX_CHECKPOINT_REQUEST=str(request), LOOPX_WAKE_LOG_DIR=str(logs / "wakes"), + LOOPX_RUNTIME_ROOT=str(runtime), LOOPX_REGISTRY=str(registry), LOOPX_GOAL_ID=cli.GOAL_ID, + LOOPX_AGENT_ID=cli.AGENT_ID) + receipt = {"ok": True, "stage": "execute", "mode": "heartbeat", "turn_id": cli.TURN_ID, + "session": {"session_id": source}} + injected = [] + def between_read_and_probe(method, params, **kwargs): + if source_moves and params["action"] == "probe" and not injected: + injected.append(complete()) # Native writer commits later history before the byte witness. + return effect_runtime_result(method, params, **kwargs) + monkeypatch.setattr("benchmark.runtime.checkpoint.effect_runtime_result", between_read_and_probe) + capture_after_wake(env, receipt) + assert receipt["ok"] and candidate.read_text() == "source_candidate = 7\n" + assert len(calls) == (2 if source_moves else 1) # Static local streams only; no implicit new work. + if source_moves: + assert not receipt["checkpoint"]["captured"], receipt + assert "native" in receipt["checkpoint"]["error"] and not destination.exists() + else: + assert receipt["checkpoint"]["captured"], receipt + manifest = json.loads((destination / "checkpoint.json").read_text()) + assert len(manifest["point_metadata"]["native_session"]["turn_ids"]) == 1 + assert receipt["checkpoint"]["research_point_qualified"] is False + + @pytest.mark.parametrize("capture_fails", [False, True]) def test_real_pre_replan_capture_is_before_decision_and_same_node_is_not_recaptured(tmp_path, monkeypatch, capture_fails): import os From 087b6aac736b54999cfd261498fab7d7a7c82d39 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 07:57:20 +0800 Subject: [PATCH 24/69] docs(benchmark): state native source-basis qualification and remaining adoption gap Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/README.md | 18 +++++++++++++++++- .../hierarchical-agent-stride-control-v0.md | 6 +++++- ...erarchical-agent-stride-control-v0.zh-CN.md | 6 ++++-- 3 files changed, 26 insertions(+), 4 deletions(-) diff --git a/benchmark/README.md b/benchmark/README.md index cf0b5a663a..236eca62d2 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -183,7 +183,11 @@ state under the same obligation. Subsequent distinct replans remain eligible. Add `"native_session": true` to include Codex's completed native cursor, all retained history pages, rollout/compaction records and native index/history SQLite -stores. The adapter uses native readback before sealing these members and excludes +stores. The adapter verifies the native session's actual workspace, reads its full +cursor before the byte probe and reads it again after that probe. A changed cursor, +configuration or participant inventory rejects before publication; the artifact +owner then verifies the same physical witness through capture. This prevents later +Turns on the selected source session from being sealed under an earlier cursor. The adapter excludes credential files, logs, memories and the symlinked skill installation. Record the effective configuration digest and Host version; tools/skills and credentials need their existing installation/authentication owners. Native state is opaque and its @@ -235,6 +239,18 @@ same binary and fixed guest paths. Compare every retained Turn ID and the full history digest, including exclusion of later history; mocked pages alone do not qualify this participant. No model call or LoopX registration is implied by fork. +The current cursor/workspace regressions also exercise `codex-cli 0.162.0-alpha.2` +through real native writer/readback and the actual CLI/typed artifact owners. A +static localhost response stream supplies synthetic source Turns; this tests the +storage protocol and capture boundary, not model behavior or research utility. +Run with an explicit native binary; ordinary runs keep the qualification skip: + +```sh +LOOPX_NATIVE_CHECKPOINT_BIN=/path/to/pinned/codex python -m pytest -q \ + tests/control_plane/test_research_checkpoint_capture.py::test_real_native_cursor_must_match_sealed_store_basis \ + benchmark/tests/test_native_checkpoint.py::test_real_native_workspace_readback_rejects_foreign_code_basis +``` + Its default-off factory otherwise preserves the native routes. With `"evaluation_opportunity": true` in the worker declaration, the capture hook reads that registration's released history and complete public result details, diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md index c9065917a6..c616c90658 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md @@ -729,7 +729,11 @@ an explicit fresh-replan trigger: it preloads the original quota guard and captu an admitted replan projected at wake start before invoking the model, deduplicated by the existing obligation. It does not cover replans first created inside a running Host wake. The optional native participant reads complete history through Codex, -seals its rollout/index/paged stores and verifies native forks at fixed guest paths. +checks the session's actual workspace, and rechecks that history after the byte +probe. A later source Turn in the read-to-probe gap is rejected; the artifact +owner's witness checks cover subsequent writes. It seals rollout/index/paged +stores and verifies native forks at fixed guest paths. These source-basis checks +are qualified separately from session adoption or research utility. An explicit checkpoint-aware native judge also exports released public result details, accepted counters/clock, remaining cooldown and original trial deadline, plus the opaque native-selected incumbent archive bound to its released result. diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md index 645cb18b8d..f7170a9faf 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md @@ -609,8 +609,10 @@ Benchmark adapter 在既有 Turn 结算后请求捕获；typed 产物 owner 封 研究点尚未资格化。Heartbeat adapter 还支持显式 fresh-replan 触发：预加载原 quota guard，在模型调用前捕获 wake 开始已投影的准入 replan，并按既有义务 去重；尚不覆盖运行中 Host wake 首次产生的 replan。可选 native 参与者通过 -Codex 读取完整历史，封存 rollout/索引/分页库并验证固定 guest 路径下的原生 -分叉。显式 checkpoint-aware native judge 还可导出已公开结果明细、受理计数/ +Codex 读取完整历史、核对会话真实工作区，并在字节 probe 后复查该历史。读取 +到 probe 之间出现的后续 source Turn 会拒绝；之后的写入由产物 witness 校验 +拦截。它封存 rollout/索引/分页库并验证固定 guest 路径下的原生分叉；来源基准 +校验不等于会话采用或研究收益。显式 checkpoint-aware native judge 还可导出已公开结果明细、受理计数/ 时钟、剩余冷却及原试验期限，并封存与已公开结果绑定的原生选优 incumbent 原始 archive。它可能早于工作区候选；排序仍由原生规则决定。缺失源码、捕获 绑定变化或文件 witness 不符均在发布前拒绝；不评测、不补发机会，未完成或 From 01b02541e67e61271be12229aa90aa82a856ee7a Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 09:01:04 +0800 Subject: [PATCH 25/69] feat(benchmark): prepare checkpointed native sessions for governed continuation Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/runtime/checkpoint.py | 12 +++ benchmark/runtime/native_checkpoint.py | 89 +++++++++++++++++++ benchmark/runtime/sessions.py | 24 +++++ .../turn_driver/codex_sessions.py | 6 +- .../project_registry_io_manifest_v1.json | 2 +- 5 files changed, 131 insertions(+), 2 deletions(-) diff --git a/benchmark/runtime/checkpoint.py b/benchmark/runtime/checkpoint.py index 289301b208..f5dd0fa903 100644 --- a/benchmark/runtime/checkpoint.py +++ b/benchmark/runtime/checkpoint.py @@ -350,8 +350,15 @@ def capture_after_wake(env: dict[str, str], receipt: dict[str, Any], *, components = [*components, {"id": "evaluation_incumbent", "source_path": str(incumbent)}] if request.get("native_session"): from .native_checkpoint import native_session_checkpoint + from .codex import Execution + from .sessions import BenchmarkSessionWake session = receipt.get("session") or {} + source_wake = BenchmarkSessionWake(env, Execution(context="resume", + sandbox=env.get("LOOPX_CODEX_SANDBOX", "danger-full-access")), {}) + if source_wake.session_id != session.get("session_id"): + raise ValueError("native checkpoint differs from the current source session binding") + metadata["session"] = {**session, "session_profile_digest": source_wake.digest} native = native_session_checkpoint(env, session.get("session_id", ""), timeout=max(.1, deadline - time.time())) components = [*components, *native["components"]] @@ -369,6 +376,11 @@ def capture_after_wake(env: dict[str, str], receipt: dict[str, Any], *, timeout=max(.1, deadline - time.time())) if confirmed != native: raise ValueError("native checkpoint cursor or participant basis moved before sealing") + confirmed_wake = BenchmarkSessionWake(env, Execution(context="resume", + sandbox=env.get("LOOPX_CODEX_SANDBOX", "danger-full-access")), {}) + if ((confirmed_wake.session_id, confirmed_wake.goal_ref, confirmed_wake.digest) != + (source_wake.session_id, source_wake.goal_ref, source_wake.digest)): + raise ValueError("native checkpoint source session binding moved before sealing") if request.get("loopx_state"): declared = metadata["loopx_state"] witness = probe["source_witness"][declared["registry_component_id"]] diff --git a/benchmark/runtime/native_checkpoint.py b/benchmark/runtime/native_checkpoint.py index 3d595193cd..c3f880aab7 100644 --- a/benchmark/runtime/native_checkpoint.py +++ b/benchmark/runtime/native_checkpoint.py @@ -5,16 +5,20 @@ """ from __future__ import annotations +import argparse import hashlib import json +import math import re import shutil import subprocess import time +import tomllib from pathlib import Path from typing import Any from loopx.capabilities.benchmark_toolkit.native_codex_goal import StdioNativeGoalTransport +from .codex import Execution def _complete_history(request, session_id: str) -> list[dict[str, Any]]: @@ -169,3 +173,88 @@ def request(method, params): return {"session_id": session, "completed_turn_id": native["completed_turn_id"], "visible_history_sha256": native["visible_history_sha256"], "history_verified": True, "model_calls": 0, "execution_authority_granted": False, "continuation_qualified": False} + + +def admit_native_checkpoint_session(env: dict[str, str], unit: Path, expected_sha256: str, + config_path: Path, *, execution: Execution, timeout: float) -> dict[str, Any]: + """Explicit controller admission of a verified fork into a destination Goal. + + Existing native fork, Goal lifetime and session owners retain their rules. + Historical research/effects and evaluator/whole-trial budget are not adopted + by this participant. Admission alone never launches work or a model. + """ + from loopx.control_plane.effect_runtime import effect_runtime_result + from loopx.agent_registry import load_goal_from_registry, require_registered_agent_id + from loopx.control_plane.projects.registry_codec import ( + project_registry_transaction, + ) + from loopx.control_plane.turn_driver.codex_sessions import select_codex_cli_session + from .sessions import BenchmarkSessionWake + + if execution.context != "resume" or execution.mode != "heartbeat": + raise ValueError("checkpoint session admission requires explicit heartbeat resume") + if not math.isfinite(timeout) or timeout <= 0: + raise ValueError("checkpoint session admission timeout must be finite and positive") + deadline = time.monotonic() + timeout + effect_runtime_result("runtime.checkpoint_artifacts", { + "action": "verify", "root": str(unit), "expected_sha256": expected_sha256, + }, timeout=timeout) + metadata = json.loads((unit / "checkpoint.json").read_bytes())["point_metadata"] + source_profile = (metadata.get("session") or {}).get("session_profile_digest") + if not isinstance(source_profile, str) or not re.fullmatch(r"[0-9a-f]{64}", source_profile): + raise ValueError("checkpoint session admission requires its captured source session profile") + if tomllib.loads(config_path.read_text()).get("features", {}).get("goals") is not False: + raise ValueError("checkpoint session admission requires native automatic Goals explicitly disabled") + source = metadata.get("loopx_state", {}) + lineage = {"goal_id": env["LOOPX_GOAL_ID"], "agent_id": env["LOOPX_AGENT_ID"]} + if (not source.get("goal_id") or not source.get("agent_id") or + any(source[key] == lineage[key] for key in lineage)): + raise ValueError("checkpoint session admission requires distinct destination Goal and Agent identities") + registry = Path(env["LOOPX_REGISTRY"]) + # Use the current governed benchmark path. The lifecycle-only source-session + # profile cannot run governed Turns and is rejected by the registry owner. + goal = load_goal_from_registry(registry, lineage["goal_id"]) + require_registered_agent_id(registry_path=registry, goal_id=lineage["goal_id"], agent_id=lineage["agent_id"]) + if select_codex_cli_session(Path(env["LOOPX_RUNTIME_ROOT"]), lineage=lineage, + session_scope="agent") is not None: + raise ValueError("destination session binding already exists; read back its original admission") + result = fork_native_checkpoint(env, unit, expected_sha256, config_path, + timeout=max(.1, deadline - time.monotonic())) + # Hold the existing registry owner's fence through the session write. This + # does not mint a lease, task or work admission from historical state. + with project_registry_transaction(registry, operation="checkpoint_session_binding", agent_id=lineage["agent_id"]): + current = load_goal_from_registry(registry, lineage["goal_id"]) + require_registered_agent_id(registry_path=registry, goal_id=lineage["goal_id"], agent_id=lineage["agent_id"]) + if current != goal: + raise ValueError("destination Goal configuration changed during native checkpoint reconstruction") + receipt: dict[str, Any] = {} + wake = BenchmarkSessionWake(env, execution, receipt) + wake._bind_verified_fork(result["session_id"], source_profile) + return {**result, "destination_goal_ref": wake.goal_ref, "destination_agent_id": lineage["agent_id"], + "session_binding_admitted": True, "session_profile_digest": wake.digest} + + +def main(argv=None) -> None: + """Controller entrypoint inside an explicitly prepared isolated guest.""" + import os + from .codex import SANDBOXES + + parser = argparse.ArgumentParser(description="Admit a checkpoint's native session participant; no model starts.") + parser.add_argument("--root", type=Path, required=True) + parser.add_argument("--expected-sha256", required=True) + parser.add_argument("--config-path", type=Path, required=True) + parser.add_argument("--sandbox", choices=SANDBOXES, required=True) + parser.add_argument("--timeout-seconds", type=float, default=30) + args = parser.parse_args(argv) + required = ("CODEX_BIN", "CODEX_HOME", "LOOPX_PROJECT", "LOOPX_REGISTRY", "LOOPX_RUNTIME_ROOT", + "LOOPX_GOAL_ID", "LOOPX_AGENT_ID", "MODEL_NAME", "REASONING_EFFORT") + missing = [key for key in required if not os.environ.get(key)] + if missing: + parser.error("Explicit destination controller environment is missing: " + ", ".join(missing)) + result = admit_native_checkpoint_session(dict(os.environ), args.root, args.expected_sha256, + args.config_path, execution=Execution(context="resume", sandbox=args.sandbox), timeout=args.timeout_seconds) + print(json.dumps(result)) + + +if __name__ == "__main__": + main() diff --git a/benchmark/runtime/sessions.py b/benchmark/runtime/sessions.py index 87eb73a774..1fbdc8fa0b 100644 --- a/benchmark/runtime/sessions.py +++ b/benchmark/runtime/sessions.py @@ -115,3 +115,27 @@ def observe(self, path: Path) -> None: ) ) self.receipt["session"]["session_id"] = observed + + def _bind_verified_fork(self, session_id: str, source_profile_digest: str) -> None: + """Bind a controller-verified native fork through the existing session owner. + + This admits the conversation binding only. It does not claim work, + restore research authority or start a model. Unknown result delivery + requires readback; it cannot replace an existing binding. + """ + if self.session_id is not None: + raise ValueError("checkpoint session adoption requires an absent destination binding") + require_codex_session_profile({"session_profile_digest": source_profile_digest}, self.digest) + self.admission.accept_result( + lambda: _store_codex_cli_session( + self.root, + lineage=self.lineage, + session_scope="agent", + session_id=session_id, + goal_ref=self.goal_ref, + session_profile_digest=self.digest, + require_absent=True, + ) + ) + self.session_id = session_id + self.receipt["session"].update(action="resume", session_id=session_id) diff --git a/loopx/control_plane/turn_driver/codex_sessions.py b/loopx/control_plane/turn_driver/codex_sessions.py index 7b486ac48d..f13e864ef5 100644 --- a/loopx/control_plane/turn_driver/codex_sessions.py +++ b/loopx/control_plane/turn_driver/codex_sessions.py @@ -174,8 +174,12 @@ def _store_codex_cli_session( operation_profile_digest: str | None = None, operation_model: str | None = None, operation_reasoning_effort: str | None = None, + require_absent: bool = False, ) -> None: - with exclusive_file_lock(_session_path(runtime_root, lineage, session_scope)): + path = _session_path(runtime_root, lineage, session_scope) + with exclusive_file_lock(path): + if require_absent and (path.exists() or path.is_symlink()): + raise ValueError("session binding already exists; read back the original admission") _write_codex_cli_session( runtime_root, lineage=lineage, diff --git a/loopx/semantics/project_registry_io_manifest_v1.json b/loopx/semantics/project_registry_io_manifest_v1.json index f909e8d80e..16519d0557 100644 --- a/loopx/semantics/project_registry_io_manifest_v1.json +++ b/loopx/semantics/project_registry_io_manifest_v1.json @@ -1783,7 +1783,7 @@ }, { "site": "loopx/control_plane/turn_driver/codex_sessions.py::.approved_codex_workspace_write_resume::codec_read:load_project_registry#1", - "line": 354, + "line": 358, "column": 20, "kind": "codec_read", "api": "load_project_registry", From 3585b06e6cf028b274bf2489266e8c27f6916367 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 09:01:04 +0800 Subject: [PATCH 26/69] test(benchmark): qualify native session continuation on the governed path Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/tests/test_native_checkpoint.py | 244 +++++++++++++++++- benchmark/tests/test_resume_sessions.py | 25 ++ .../test_research_checkpoint_capture.py | 25 +- 3 files changed, 284 insertions(+), 10 deletions(-) diff --git a/benchmark/tests/test_native_checkpoint.py b/benchmark/tests/test_native_checkpoint.py index 5d5d1d2254..dc08e509bf 100644 --- a/benchmark/tests/test_native_checkpoint.py +++ b/benchmark/tests/test_native_checkpoint.py @@ -86,7 +86,7 @@ def test_foreign_native_session_is_rejected_before_history(tmp_path, monkeypatch @contextmanager -def real_native_source(root, binary, workspace): +def real_native_source(root, binary, workspace, *, close_source=False, response_builder=None): """Real native writer/readback; a local static stream replaces only the model. No paid model, tool invocation, user credential or benchmark cell. Fixtures @@ -103,10 +103,12 @@ def log_message(self, *args): pass def do_POST(self): - self.rfile.read(int(self.headers.get("Content-Length", 0))) + request = json.loads(self.rfile.read(int(self.headers.get("Content-Length", 0)))) calls.append(self.path) message = {"type": "message", "id": "msg_" + uuid.uuid4().hex, "role": "assistant", "content": [{"type": "output_text", "text": "Synthetic source result."}]} + if response_builder is not None: + message = response_builder(request, len(calls), message) result = {"id": "resp_" + uuid.uuid4().hex, "status": "completed", "output": [message], "usage": {"input_tokens": 1, "output_tokens": 1, "total_tokens": 2}} events = [{"type": "response.created", "response": {**result, "status": "in_progress", "output": []}}, @@ -123,13 +125,15 @@ def do_POST(self): thread = threading.Thread(target=server.serve_forever) thread.start() (home / "config.toml").write_text( - 'model="fixture"\nmodel_provider="fixture"\napproval_policy="never"\nsandbox_mode="read-only"\n' - '[features]\ngoals=false\nresponses_websockets_v2=false\n' + 'model="fixture"\nmodel_reasoning_effort="high"\nmodel_provider="fixture"\napproval_policy="never"\nsandbox_mode="read-only"\n' + '[features]\ngoals=false\nresponses_websockets_v2=false\nplugins=false\nremote_plugin=false\nplugin_sharing=false\n' '[model_providers.fixture]\nname="Synthetic local"\n' f'base_url="http://127.0.0.1:{server.server_port}/v1"\nwire_api="responses"\n' 'env_key="SYNTHETIC_PROVIDER_KEY"\n') + # No ambient provider credentials or app integrations enter this fixture. env = {key: value for key, value in os.environ.items() - if key not in {"OPENAI_API_KEY", "CODEX_API_KEY", "OPENAI_BASE_URL"}} + if key in {"PATH", "HOME", "USER", "LOGNAME", "SHELL", "TMPDIR", "LANG", "LC_ALL", + "SYSTEMROOT", "PYTHONPATH", "VIRTUAL_ENV"}} env.update(CODEX_HOME=str(home), CODEX_BIN=binary, LOOPX_PROJECT=str(workspace), SYNTHETIC_PROVIDER_KEY="synthetic-local-only", LOOPX_USAGE_PING="0") try: @@ -160,6 +164,8 @@ def complete_turn(): time.sleep(.05) raise AssertionError("Real native fixture did not complete its source Turn") complete_turn() + if close_source: + native.close() yield env, source, complete_turn, calls finally: server.shutdown() @@ -181,3 +187,231 @@ def test_real_native_workspace_readback_rejects_foreign_code_basis(tmp_path): foreign.mkdir() with pytest.raises(ValueError, match="workspace"): native_session_checkpoint({**env, "LOOPX_PROJECT": str(foreign)}, source, timeout=20) + + +@pytest.mark.skipif(not os.environ.get("LOOPX_NATIVE_CHECKPOINT_BIN"), + reason="Requires an explicit real native Host binary; only a local static provider is used") +@pytest.mark.parametrize("case", ["resume", "scope_change", "profile_change"]) +def test_real_fork_admission_and_native_resume_use_retained_state(tmp_path, capsys, monkeypatch, case): + """Real Host/CLI/Goal/session/artifact owners; deterministic synthetic work. + + This qualifies the conversation participant, not full research/evaluator + adoption or benchmark utility. No source trial or paid model is launched. + """ + import shutil + import subprocess + import sys + from pathlib import Path + from benchmark.runtime.codex import Execution + from benchmark.runtime.native_checkpoint import admit_native_checkpoint_session + from benchmark.runtime.sessions import BenchmarkSessionWake + from benchmark.runtime.worker import native_command + from loopx.cli import main + from loopx.control_plane.effect_runtime import effect_runtime_result + from loopx.control_plane.turn_driver.codex_sessions import load_codex_cli_session, _store_codex_cli_session + from loopx.control_plane.quota.settlement import read_heartbeat_settlement + from loopx.control_plane.projects.registry_codec import ProjectRegistryProtocolError + + workspace = tmp_path / "workspace" + workspace.mkdir() + candidate = workspace / "candidate.txt" + candidate.write_text("retained candidate: 7\n") + research = workspace / "research.txt" + research.write_text("candidate 3 was rejected by the original synthetic check\n") + source_goal, source_agent = "source-research", "source-researcher" + destination_goal, destination_agent = "branch-research", "branch-researcher" + def register(name, agent): + registry = tmp_path / name / "registry.json" + runtime = tmp_path / name / "runtime" + prefix = ["--format", "json", "--registry", str(registry), "--runtime-root", str(runtime)] + assert main([*prefix, "bootstrap", "--project", str(workspace), "--goal-id", name, + "--objective", "Continue the synthetic candidate investigation.", "--adapter-kind", "read_only_project_map_v0", + "--adapter-status", "connected-read-only", "--write-scope", "**", "--no-global-sync"]) == 0 + capsys.readouterr() + assert main([*prefix, "configure-goal", "--goal-id", name, "--registered-agent", agent, + "--agent-work-mode", agent + "=active", "--execute"]) == 0 + capsys.readouterr() + assert main([*prefix, "todo", "add", "--goal-id", name, "--role", "agent", "--claimed-by", agent, + "--task-class", "advancement_task", "--required-capability", "shell", + "--text", "Verify the retained candidate and preserve applicable negative evidence."]) == 0 + added = json.loads(capsys.readouterr().out) + return registry, runtime, added["todo_id"] + source_registry, source_runtime, _ = register(source_goal, source_agent) + registry, runtime, destination_todo = register(destination_goal, destination_agent) + prefix = ["--format", "json", "--registry", str(registry), "--runtime-root", str(runtime)] + turn_id = "checkpoint-continuation-fixture" + binding = ["--goal-id", destination_goal, "--agent-id", destination_agent, + "--todo-id", destination_todo, "--turn-instance-id", turn_id] + assert main([*prefix, "quota", "should-run", "--runtime-profile", "generic_cli", *binding]) == 0 + guard = json.loads(capsys.readouterr().out) + assert guard["interaction_contract"]["agent_channel"]["delivery_allowed"] is True + requests = [] + def response(request, index, message): + requests.append(request) + if index == 2: + return {"type": "function_call", "id": "fc_retained", "call_id": "call_retained", + "name": "exec_command", "arguments": json.dumps({ + "cmd": "cat candidate.txt research.txt", "max_output_tokens": 1000})} + if index == 3: + message["content"][0]["text"] = "Retained candidate 7 verified; candidate 3 remains rejected." + return message + + binary = os.environ["LOOPX_NATIVE_CHECKPOINT_BIN"] + execution = Execution(context="resume", sandbox="read-only") + with real_native_source(tmp_path / "native", binary, workspace, close_source=True, + response_builder=response) as (env, source_session, _, calls): + env.update(LOOPX_REGISTRY=str(source_registry), LOOPX_RUNTIME_ROOT=str(source_runtime), + LOOPX_GOAL_ID=source_goal, LOOPX_AGENT_ID=source_agent, + MODEL_NAME="fixture", REASONING_EFFORT="high", LOOPX_CODEX_SANDBOX="read-only") + source_wake = BenchmarkSessionWake(env, execution, {}) + source_wake.admission.accept_result(lambda: _store_codex_cli_session(source_runtime, + lineage={"goal_id": source_goal, "agent_id": source_agent}, session_scope="agent", + session_id=source_session, goal_ref=source_wake.goal_ref, session_profile_digest=source_wake.digest)) + native = native_session_checkpoint(env, source_session, timeout=20) + config = tmp_path / "config.toml" + config.write_bytes((Path(env["CODEX_HOME"]) / "config.toml").read_bytes()) + components = [*native["components"], {"id": "code", "source_path": str(candidate)}, + {"id": "research", "source_path": str(research)}] + probe = effect_runtime_result("runtime.checkpoint_artifacts", {"action": "probe", "components": components}) + unit = tmp_path / "point" + captured = effect_runtime_result("runtime.checkpoint_artifacts", {"action": "capture", + "destination": str(unit), "components": components, "expected_source_sha256": probe["source_sha256"], + "point_metadata": {"native_session": native["metadata"], + "session": {"session_id": source_session, "session_profile_digest": source_wake.digest}, + "loopx_state": {"goal_id": source_goal, "agent_id": source_agent}}}, retry_safe=False) + digest = captured["sha256"] + # Qualify use of captured inputs, not the source's later working files. + # A quiesced same-machine fixture swaps fixed paths; this is not evidence + # of complete guest/environment isolation or a scientific matched trial. + candidate.write_text("later candidate: 999\n") + research.write_text("later evidence unavailable at the captured point\n") + late_source = tmp_path / "retained-later-source" + late_source.mkdir() + shutil.move(candidate, late_source / candidate.name) + shutil.move(research, late_source / research.name) + materialized = tmp_path / "materialized" + effect_runtime_result("runtime.checkpoint_artifacts", {"action": "materialize", "root": str(unit), + "expected_sha256": digest, "destination": str(materialized)}, retry_safe=False) + shutil.copy2(materialized / "components/code", candidate) + shutil.copy2(materialized / "components/research", research) + original_home = tmp_path / "original-home" + shutil.move(env["CODEX_HOME"], original_home) + retained_names = {m["name"] for m in native["metadata"]["store_members"]} | {"config.toml"} + def retained_source(): + return {p.relative_to(original_home).as_posix(): p.read_bytes() + for p in original_home.rglob("*") if p.is_file() and p.relative_to(original_home).parts[0] in retained_names} + before = retained_source() + env.update(LOOPX_REGISTRY=str(registry), LOOPX_RUNTIME_ROOT=str(runtime), + LOOPX_GOAL_ID=destination_goal, LOOPX_AGENT_ID=destination_agent, + MODEL_NAME="fixture", REASONING_EFFORT="high") + lifecycle_registry = tmp_path / "lifecycle-only" / "registry.json" + assert main(["--format", "json", "--registry", str(lifecycle_registry), + "--runtime-root", str(tmp_path / "lifecycle-only/runtime"), "project", "register", + "--goal-instance-profile", "source_session_v1", "--operation-id", "lifecycle-only", + "--project-id", "lifecycle-only", "--project-kind", "work", "--knowledge-root", str(workspace), + "--goal-id", "lifecycle-only", "--objective", "Synthetic lifecycle fixture.", + "--acceptance", "Lifecycle only.", "--next-effect", "Read.", "--stop-condition", "Stop."]) == 0 + capsys.readouterr() + with pytest.raises(ProjectRegistryProtocolError, match="lifecycle-only"): + admit_native_checkpoint_session({**env, "LOOPX_REGISTRY": str(lifecycle_registry), + "LOOPX_GOAL_ID": "lifecycle-only"}, unit, digest, config, execution=execution, timeout=30) + assert not Path(env["CODEX_HOME"]).exists() and len(calls) == 1 + unsupported_config = tmp_path / "automatic-goal-config.toml" + unsupported_config.write_text(config.read_text().replace("goals=false", "goals=true")) + with monkeypatch.context() as automatic: + def forbidden_automatic_fork(*args, **kwargs): + raise AssertionError("an unsupported automatic Goal profile must fail before native reconstruction") + automatic.setattr("benchmark.runtime.native_checkpoint.fork_native_checkpoint", forbidden_automatic_fork) + with pytest.raises(ValueError, match="automatic Goals explicitly disabled"): + admit_native_checkpoint_session(env, unit, digest, unsupported_config, execution=execution, timeout=30) + with pytest.raises(ValueError, match="distinct destination"): + admit_native_checkpoint_session({**env, "LOOPX_GOAL_ID": source_goal}, unit, digest, + config, execution=execution, timeout=30) + assert not Path(env["CODEX_HOME"]).exists() and len(calls) == 1 + if case == "scope_change": + from benchmark.runtime import native_checkpoint + original_fork = native_checkpoint.fork_native_checkpoint + def change_scope_after_real_fork(*args, **kwargs): + result = original_fork(*args, **kwargs) + assert main([*prefix, "configure-goal", "--goal-id", destination_goal, + "--agent-work-mode", destination_agent + "=monitor_only", "--execute"]) == 0 + capsys.readouterr() + return result + monkeypatch.setattr(native_checkpoint, "fork_native_checkpoint", change_scope_after_real_fork) + with pytest.raises(ValueError, match="Goal configuration changed"): + admit_native_checkpoint_session(env, unit, digest, config, execution=execution, timeout=30) + assert load_codex_cli_session(runtime, lineage={"goal_id": destination_goal, + "agent_id": destination_agent}, session_scope="agent") is None + assert len(calls) == 1 # A native fork occurred; no binding or model did. + assert retained_source() == before + return + if case == "profile_change": + with pytest.raises(ValueError, match="profile changed"): + admit_native_checkpoint_session({**env, "REASONING_EFFORT": "low"}, unit, digest, + config, execution=execution, timeout=30) + assert load_codex_cli_session(runtime, lineage={"goal_id": destination_goal, + "agent_id": destination_agent}, session_scope="agent") is None + assert len(calls) == 1 + assert retained_source() == before + return + admitted_cli = subprocess.run([sys.executable, "-m", "benchmark.runtime.native_checkpoint", + "--root", str(unit), "--expected-sha256", digest, "--config-path", str(config), "--sandbox", "read-only"], + env=env, text=True, capture_output=True, timeout=40) + assert admitted_cli.returncode == 0, admitted_cli.stderr + admitted = json.loads(admitted_cli.stdout) + assert admitted["session_binding_admitted"] and admitted["history_verified"] + assert admitted["model_calls"] == 0 and not admitted["continuation_qualified"] + assert admitted["session_id"] != source_session and len(calls) == 1 + bound = load_codex_cli_session(runtime, lineage={"goal_id": destination_goal, "agent_id": destination_agent}, + session_scope="agent") + assert bound["session_id"] == admitted["session_id"] + assert bound.get("goal_ref") == admitted["destination_goal_ref"] + with monkeypatch.context() as replay: + def forbidden_fork(*args, **kwargs): + raise AssertionError("a repeated admission must not create a second native fork") + replay.setattr("benchmark.runtime.native_checkpoint.fork_native_checkpoint", forbidden_fork) + with pytest.raises(ValueError, match="already exists"): + admit_native_checkpoint_session(env, unit, digest, config, execution=execution, timeout=30) + assert len(calls) == 1 and load_codex_cli_session(runtime, lineage={ + "goal_id": destination_goal, "agent_id": destination_agent}, session_scope="agent") == bound + + receipt = {} + wake = BenchmarkSessionWake(env, execution, receipt) + assert wake.session_id == admitted["session_id"] and receipt["session"]["action"] == "resume" + logs = tmp_path / "continuation" + logs.mkdir() + command = native_command(env, execution, "execute", logs, wake) + current_body = json.dumps(guard["interaction_contract"], ensure_ascii=False) + result = subprocess.run(command, env=env, cwd=workspace, input=current_body + "\nContinue: verify the retained files.", + text=True, capture_output=True, timeout=40) + assert result.returncode == 0, result.stderr + (logs / "stdout.jsonl").write_text(result.stdout) + wake.observe(logs / "stdout.jsonl") + assert wake.session_id == admitted["session_id"] and len(calls) == 3 + assert "Synthetic source result." in json.dumps(requests[1]) + assert destination_goal in json.dumps(requests[1]) and destination_todo in json.dumps(requests[1]) + assert "retained candidate: 7" in json.dumps(requests[2]) + assert "candidate 3 was rejected" in json.dumps(requests[2]) + assert "Retained candidate 7 verified" in result.stdout + retained = native_session_checkpoint(env, admitted["session_id"], timeout=20) + assert retained["metadata"]["turn_ids"][:-1] == native["metadata"]["turn_ids"] + assert retained["metadata"]["completed_turn_id"] != native["metadata"]["completed_turn_id"] + assert candidate.read_text() == "retained candidate: 7\n" + assert (late_source / candidate.name).read_text() == "later candidate: 999\n" + assert (late_source / research.name).read_text() == "later evidence unavailable at the captured point\n" + assert retained_source() == before + vision = tmp_path / "continuation-vision.json" + vision.write_text(json.dumps({"schema_version": "goal_vision_replan_contract_v0", "state": "vision_patch_proposed", + "vision_patch": {"acceptance_summary": "Synthetic conversation continuation verified; full research remains open."}, + "path_delta": {"outcome": "continue", "prior_assumption": "Retained candidate and evidence are available.", + "observed_reality": "The resumed native tool read verifies candidate 7 and the rejection of candidate 3.", + "retained": ["Complete checkpoint qualification"], "evidence_refs": ["evidence:native-continuation-fixture"]}})) + assert main([*prefix, "refresh-state", *binding, "--classification", "checkpoint_continuation_verified", + "--delivery-batch-scale", "multi_surface", "--delivery-outcome", "outcome_progress", + "--agent-vision-json", str(vision)]) == 0 + assert json.loads(capsys.readouterr().out)["ok"] is True + assert main([*prefix, "quota", "spend-slot", *binding, "--source", "heartbeat", "--execute"]) == 0 + assert json.loads(capsys.readouterr().out)["ok"] is True + settled = read_heartbeat_settlement(runtime, goal_id=destination_goal, agent_id=destination_agent, + todo_id=destination_todo, turn_instance_id=turn_id, registry_path=registry) + assert settled.progress["state"] == "settled" diff --git a/benchmark/tests/test_resume_sessions.py b/benchmark/tests/test_resume_sessions.py index 5d98f734a0..417154bf41 100644 --- a/benchmark/tests/test_resume_sessions.py +++ b/benchmark/tests/test_resume_sessions.py @@ -15,6 +15,31 @@ from loopx.control_plane.turn_driver.codex_sessions import load_codex_cli_session +@pytest.mark.parametrize("existing", ["valid", "malformed", "dangling"]) +def test_absent_only_session_admission_preserves_every_existing_binding(tmp_path, existing): + """The no-replacement condition is checked under the session owner's lock.""" + from loopx.control_plane.turn_driver.codex_sessions import _session_path, _store_codex_cli_session + + lineage = {"goal_id": "continuation-fixture", "agent_id": "continuation-agent"} + path = _session_path(tmp_path, lineage, "agent") + path.parent.mkdir(parents=True) + if existing == "valid": + _store_codex_cli_session(tmp_path, lineage=lineage, session_scope="agent", session_id="original-session") + elif existing == "malformed": + path.write_bytes(b"unconfirmed original binding") + else: + path.symlink_to(path.with_name("unavailable-original")) + before = path.readlink() if path.is_symlink() else path.read_bytes() + with pytest.raises(ValueError, match="already exists"): + _store_codex_cli_session(tmp_path, lineage=lineage, session_scope="agent", session_id="replacement-session", + require_absent=True) + assert (path.readlink() if path.is_symlink() else path.read_bytes()) == before + if existing == "valid": + # Ordinary native observation retains its existing default behavior. + _store_codex_cli_session(tmp_path, lineage=lineage, session_scope="agent", session_id="observed-session") + assert load_codex_cli_session(tmp_path, lineage=lineage, session_scope="agent")["session_id"] == "observed-session" + + def resume_env(tmp_path, monkeypatch): env = worker_env(tmp_path) skills = tmp_path / "skills" diff --git a/tests/control_plane/test_research_checkpoint_capture.py b/tests/control_plane/test_research_checkpoint_capture.py index 65d12a3125..ee5f48eb07 100644 --- a/tests/control_plane/test_research_checkpoint_capture.py +++ b/tests/control_plane/test_research_checkpoint_capture.py @@ -215,10 +215,13 @@ def altered_probe(method, params, **kwargs): @pytest.mark.skipif(not os.environ.get("LOOPX_NATIVE_CHECKPOINT_BIN"), reason="Requires an explicit real native Host binary; static local provider only") -@pytest.mark.parametrize("source_moves", [False, True]) +@pytest.mark.parametrize("source_moves", [False, True, "binding"]) def test_real_native_cursor_must_match_sealed_store_basis(tmp_path, monkeypatch, source_moves): """Actual CLI settlement, native completed Turns and TS SQLite artifacts.""" from benchmark.tests.test_native_checkpoint import real_native_source + from benchmark.runtime.codex import Execution + from benchmark.runtime.sessions import BenchmarkSessionWake + from loopx.control_plane.turn_driver.codex_sessions import _store_codex_cli_session isolate_sqlite_runtime(tmp_path, monkeypatch) project, runtime, registry = cli._write_fixture(tmp_path) @@ -250,21 +253,33 @@ def run(*args): with real_native_source(tmp_path / "native", os.environ["LOOPX_NATIVE_CHECKPOINT_BIN"], project) as (env, source, complete, calls): env.update(LOOPX_CHECKPOINT_REQUEST=str(request), LOOPX_WAKE_LOG_DIR=str(logs / "wakes"), LOOPX_RUNTIME_ROOT=str(runtime), LOOPX_REGISTRY=str(registry), LOOPX_GOAL_ID=cli.GOAL_ID, - LOOPX_AGENT_ID=cli.AGENT_ID) + LOOPX_AGENT_ID=cli.AGENT_ID, MODEL_NAME="fixture", REASONING_EFFORT="high", + LOOPX_CODEX_SANDBOX="read-only") + source_wake = BenchmarkSessionWake(env, Execution(context="resume", sandbox="read-only"), {}) + source_wake.admission.accept_result(lambda: _store_codex_cli_session(runtime, + lineage={"goal_id": cli.GOAL_ID, "agent_id": cli.AGENT_ID}, session_scope="agent", session_id=source, + goal_ref=source_wake.goal_ref, session_profile_digest=source_wake.digest)) receipt = {"ok": True, "stage": "execute", "mode": "heartbeat", "turn_id": cli.TURN_ID, "session": {"session_id": source}} injected = [] def between_read_and_probe(method, params, **kwargs): if source_moves and params["action"] == "probe" and not injected: - injected.append(complete()) # Native writer commits later history before the byte witness. + if source_moves == "binding": + injected.append(True) + _store_codex_cli_session(runtime, lineage={"goal_id": cli.GOAL_ID, "agent_id": cli.AGENT_ID}, + session_scope="agent", session_id=source, goal_ref=source_wake.goal_ref, + session_profile_digest="b" * 64) + else: + injected.append(complete()) # Native writer commits later history before the byte witness. return effect_runtime_result(method, params, **kwargs) monkeypatch.setattr("benchmark.runtime.checkpoint.effect_runtime_result", between_read_and_probe) capture_after_wake(env, receipt) assert receipt["ok"] and candidate.read_text() == "source_candidate = 7\n" - assert len(calls) == (2 if source_moves else 1) # Static local streams only; no implicit new work. + assert len(calls) == (2 if source_moves is True else 1) # Static local streams only; no implicit new work. if source_moves: assert not receipt["checkpoint"]["captured"], receipt - assert "native" in receipt["checkpoint"]["error"] and not destination.exists() + assert ("profile changed" if source_moves == "binding" else "native") in receipt["checkpoint"]["error"] + assert not destination.exists() else: assert receipt["checkpoint"]["captured"], receipt manifest = json.loads((destination / "checkpoint.json").read_text()) From e2eeeaf83838dd96635ef7d740e95b64304f7f93 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 09:01:04 +0800 Subject: [PATCH 27/69] docs(benchmark): define native session adoption and remaining recovery gaps Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/README.md | 42 ++++++++++++++++++- .../hierarchical-agent-stride-control-v0.md | 16 ++++++- ...rarchical-agent-stride-control-v0.zh-CN.md | 13 +++++- 3 files changed, 66 insertions(+), 5 deletions(-) diff --git a/benchmark/README.md b/benchmark/README.md index 236eca62d2..c31737e92e 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -185,7 +185,7 @@ Add `"native_session": true` to include Codex's completed native cursor, all retained history pages, rollout/compaction records and native index/history SQLite stores. The adapter verifies the native session's actual workspace, reads its full cursor before the byte probe and reads it again after that probe. A changed cursor, -configuration or participant inventory rejects before publication; the artifact +configuration, source session binding or participant inventory rejects before publication; the artifact owner then verifies the same physical witness through capture. This prevents later Turns on the selected source session from being sealed under an earlier cursor. The adapter excludes credential files, logs, memories and the symlinked skill installation. Record the @@ -200,6 +200,46 @@ the full history digest and cursor, starts no model and registers no LoopX bindi A failed or partial reconstruction is not an admitted session. This is a session participant prerequisite, not full research continuation or arbitrary path import. +An explicit controller can then prepare that participant through the existing +registry and session owners. Prepare a fresh governed benchmark Goal and +distinct Agent identity in the isolated guest; set its destination `LOOPX_REGISTRY`, +`LOOPX_RUNTIME_ROOT`, `LOOPX_GOAL_ID` and `LOOPX_AGENT_ID`, plus the qualified +`CODEX_BIN`, `CODEX_HOME`, `LOOPX_PROJECT`, `MODEL_NAME` and `REASONING_EFFORT`. +The captured session profile must match the destination, and native automatic +Goals must remain explicitly disabled. Fork deferral preserves native Goal data; +it does not remove historical authority. The controller environment and explicit +configuration are not exported as authority or credentials. + +```sh +python -m benchmark.runtime.native_checkpoint \ + --root /private/captured-point --expected-sha256 CAPTURE_DIGEST \ + --config-path /private/qualified-config.toml --sandbox read-only +``` + +This command starts no model and claims no work. It stores the verified fork for +the registered destination Goal/Agent, without replacing an existing binding. +The registry owner's fence guards the configuration check and binding write; +this preserves the existing profile's lifetime semantics. The lifecycle-only +`source_session_v1` profile is rejected by its owner because it cannot run +governed benchmark work. Changed configuration/profiles or ambiguous prior admission reject. After a +lost result, read the original agent-scoped session binding through +`BenchmarkSessionWake`; a repeated command cannot silently fork again. Omit this +command to leave the unit inert. Code/research authority, unresolved effects, +native evaluator admission and remaining whole-trial budget still require their +owners before executing a research continuation. This source-checkout controller +entrypoint is a partial participant journey; it does not ship a frontend restore +operation or qualify a complete research point. + +The explicit real-Host tests below also qualify this CLI and the worker's native +resume transport: a local static provider receives the retained conversation, +runs a read-only verification of materialized code/negative evidence and completes +a new native Turn plus its original LoopX guard/writeback/spend. Later source +candidate/evidence and original native participants remain unchanged. These tests +swap fixed paths only after their synthetic source is quiescent; they do not prove +complete environment isolation. Goal configuration change, unsupported lifecycle +profile, profile change and duplicate binding are rejected. This proves protocol adoption +and synthetic continued work, not model research utility or native score gains. + For the native feedback protocol with **no automatic evaluations**, start an explicit checkpoint-aware native judge: diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md index c616c90658..764a5ae2f8 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md @@ -729,11 +729,23 @@ an explicit fresh-replan trigger: it preloads the original quota guard and captu an admitted replan projected at wake start before invoking the model, deduplicated by the existing obligation. It does not cover replans first created inside a running Host wake. The optional native participant reads complete history through Codex, -checks the session's actual workspace, and rechecks that history after the byte +checks the session's actual workspace and current source binding/profile, and rechecks them after the byte probe. A later source Turn in the read-to-probe gap is rejected; the artifact owner's witness checks cover subsequent writes. It seals rollout/index/paged stores and verifies native forks at fixed guest paths. These source-basis checks -are qualified separately from session adoption or research utility. +are qualified separately from research utility. An explicit source-checkout +controller now prepares a verified fork through the existing registry/session +owners, with distinct registered Goal/Agent identities and a matching captured +session profile. The registry fence guards configuration through the binding write; +existing lifetime semantics remain. The lifecycle-only source-session profile +cannot run governed work and is rejected rather than used as a benchmark surrogate. +Replacement, changed configuration/profile and native automatic Goal activation +are refused. Real native CLI/resume qualification uses a local static provider +to carry retained conversation into a new Turn, verify materialized code/negative +evidence and settle the original LoopX guard/writeback/spend. The quiesced fixture +swaps fixed guest paths; full environment isolation remains unqualified. This proves +the conversation participant's adoption and synthetic continuation, not full research/effect adoption, remaining +whole-trial budget, installed frontend recovery or scientific benefit. An explicit checkpoint-aware native judge also exports released public result details, accepted counters/clock, remaining cooldown and original trial deadline, plus the opaque native-selected incumbent archive bound to its released result. diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md index f7170a9faf..0240453868 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md @@ -609,10 +609,19 @@ Benchmark adapter 在既有 Turn 结算后请求捕获；typed 产物 owner 封 研究点尚未资格化。Heartbeat adapter 还支持显式 fresh-replan 触发：预加载原 quota guard，在模型调用前捕获 wake 开始已投影的准入 replan，并按既有义务 去重；尚不覆盖运行中 Host wake 首次产生的 replan。可选 native 参与者通过 -Codex 读取完整历史、核对会话真实工作区，并在字节 probe 后复查该历史。读取 +Codex 读取完整历史、核对会话真实工作区及当前 source binding/profile，并在字节 probe 后复查。读取 到 probe 之间出现的后续 source Turn 会拒绝；之后的写入由产物 witness 校验 拦截。它封存 rollout/索引/分页库并验证固定 guest 路径下的原生分叉；来源基准 -校验不等于会话采用或研究收益。显式 checkpoint-aware native judge 还可导出已公开结果明细、受理计数/ +校验不等于研究收益。显式 source-checkout controller 现可通过既有 registry/ +session owner 准备已验证 fork，要求不同且已注册的 Goal/Agent 身份与匹配的 +捕获 session profile。Registry fence 保护配置复查到绑定写入，保持原有 profile +的 lifetime 语义；lifecycle-only source-session profile 不能运行 governed 工作， +会被拒绝而非当作 benchmark 替代路径。已有绑定、配置/profile 变化和原生自动 +Goal 激活均会拒绝。真实 native CLI/resume 资格化通过本地静态 provider，将保留 +会话传入新 Turn，实际验证物化代码与负证据，并按原身份结算 LoopX guard/ +writeback/spend。合成源静止后交换固定 guest 路径；完整环境隔离仍未资格化。 +这证明会话参与者采用与合成接续，不等于完整研究/ +effect 采用、剩余总预算、已安装前端恢复或科学收益。显式 checkpoint-aware native judge 还可导出已公开结果明细、受理计数/ 时钟、剩余冷却及原试验期限，并封存与已公开结果绑定的原生选优 incumbent 原始 archive。它可能早于工作区候选；排序仍由原生规则决定。缺失源码、捕获 绑定变化或文件 witness 不符均在发布前拒绝；不评测、不补发机会，未完成或 From 67a2263bb855273aec4fd39b589f6d117c51373d Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 10:18:24 +0800 Subject: [PATCH 28/69] feat(benchmark): conserve declared Host phase time with native checkpoint admission Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/edgebench/checkpoint_controller.py | 132 ++++++++- benchmark/runtime/sforge.py | 5 +- benchmark/runtime/sforge_entry.py | 6 +- benchmark/runtime/worker.py | 8 +- benchmark/tests/test_checkpoint_controller.py | 266 +++++++++++++++++- 5 files changed, 392 insertions(+), 25 deletions(-) diff --git a/benchmark/edgebench/checkpoint_controller.py b/benchmark/edgebench/checkpoint_controller.py index a979e87493..205b6252f0 100644 --- a/benchmark/edgebench/checkpoint_controller.py +++ b/benchmark/edgebench/checkpoint_controller.py @@ -13,12 +13,14 @@ import re import shutil import tempfile +import time from pathlib import Path from urllib.error import HTTPError from urllib.parse import urlsplit from urllib.request import HTTPRedirectHandler, Request, build_opener from loopx.control_plane.effect_runtime import effect_runtime_result +from benchmark.runtime.worker import PHASE_WORK_RESERVE_SECONDS class _NoRedirect(HTTPRedirectHandler): @@ -75,7 +77,7 @@ def _reference(value: dict) -> dict: return value -def _bound_bank(folder: Path, reference: dict) -> None: +def _bound_bank(folder: Path, reference: dict) -> dict: """Byte/source binding only; native restore owns eligibility and validity.""" raw = _physical(folder / "manifest.json").read_bytes() digest = hashlib.sha256(raw).hexdigest() @@ -96,10 +98,37 @@ def _bound_bank(folder: Path, reference: dict) -> None: raise ValueError("Native selected archive differs from the captured reference") if {p.name for p in folder.iterdir()} != expected: raise ValueError("Native bank contains undeclared members") + return packet + + +def _phase_budget(raw: bytes, native: dict) -> dict: + """Adapt the existing SForge integer phase clock, not quota authority. + + Native capture owns this time. The declared physical Host file owns its + deadline. Neither fact qualifies complete Host/research-state capture. + """ + if len(raw) > 64 or not re.fullmatch(rb"[0-9]+\n?", raw): + raise ValueError("Host phase deadline must be an integer epoch file") + epoch = int(raw) + captured = native.get("captured_at_epoch") + if (not 0 < epoch < 2 ** 53 or type(captured) not in (int, float) or + not math.isfinite(captured) or captured <= 0): + raise ValueError("Host/native phase clocks cannot be represented exactly") + return {"component_id": "host_phase_deadline", "source_deadline_epoch": epoch, + "captured_at_epoch": captured, "remaining_seconds": max(0, epoch - captured)} + + +def _phase_destination(path: Path, *, root: Path, bank: Path) -> Path: + path = Path(path) + _physical(path.parent) + if (not path.is_absolute() or path.resolve() != path or path.exists() or path.is_symlink() + or path.is_relative_to(root) or path.is_relative_to(bank)): + raise ValueError("Host phase restoration requires a fresh physical file outside artifact/bank roots") + return path def capture(controller_file: Path, *, token: str, capture_id: str, destination: Path, - timeout: float = 30) -> dict: + timeout: float = 30, phase_deadline_file: Path | None = None) -> dict: """Capture one native-owned point without exporting controller configuration.""" if not re.fullmatch(r"[0-9a-f]{32}", capture_id): raise ValueError("Capture identity must be the declared 32-hex native identity") @@ -108,29 +137,52 @@ def capture(controller_file: Path, *, token: str, capture_id: str, destination: if not destination.is_absolute() or destination != destination.resolve() or destination.exists(): raise ValueError("Capture export requires a fresh physical destination") controller = _controller(controller_file) + phase_raw = None + if phase_deadline_file is not None: + phase_deadline_file = _physical(phase_deadline_file) + phase_raw = phase_deadline_file.read_bytes() + _phase_budget(phase_raw, {"captured_at_epoch": time.time()}) + # An earlier native-only point cannot acquire a later Host clock. + if (controller["bank_root"] / capture_id).exists(): + raise ValueError("Existing native point has no new Host phase capture boundary") + attempts = controller["bank_root"] / ".loopx-phase-attempts" + attempts.mkdir(mode=0o700, exist_ok=True) + _physical(attempts) + # First attempt wins, including lost responses or failed exports. + # A later source cannot replace the Host clock at that native point. + (attempts / capture_id).mkdir(mode=0o700) reference = _reference(_post(controller, "/api/v1/checkpoint/capture", {"token": token, "capture_id": capture_id}, timeout)) if reference["checkpoint_id"] != capture_id: raise ValueError("Native capture returned a different identity") folder = _physical(controller["bank_root"] / capture_id) - _bound_bank(folder, reference) + native = _bound_bank(folder, reference) components = [{"id": "native_judge", "source_path": str(folder)}] + metadata = {"native_reference": reference} + if phase_raw is not None: + if phase_deadline_file.read_bytes() != phase_raw: + raise ValueError("Host phase deadline moved across native capture") + components.append({"id": "host_phase_deadline", "source_path": str(phase_deadline_file)}) + metadata["host_phase_budget"] = _phase_budget(phase_raw, native) probe = effect_runtime_result("runtime.checkpoint_artifacts", {"action": "probe", "components": components}) result = effect_runtime_result("runtime.checkpoint_artifacts", { "action": "capture", "components": components, "destination": str(destination), "expected_source_sha256": probe["source_sha256"], - "point_metadata": {"native_reference": reference}, + "point_metadata": metadata, }, retry_safe=False) # Bind the actually copied bytes, not merely a prior source observation. _bound_bank(destination / "components/native_judge", reference) + if phase_raw is not None and (destination / "components/host_phase_deadline").read_bytes() != phase_raw: + raise ValueError("Captured Host phase bytes differ from the native boundary") return {**result, "native_reference": reference} def restore(controller_file: Path, *, root: Path, expected_sha256: str, run_id: str, - timeout: float = 30) -> dict: + timeout: float = 30, phase_deadline_file: Path | None = None) -> dict: """Import verified inert bytes, then ask fresh native authority to admit them. - Does not restore a model, LoopX Goal, effect authority or whole-trial budget. + An explicitly captured Host clock preserves its remaining phase duration. + Does not restore a model, LoopX Goal or effect authority. An uncertain response is not replayed under a different run or capture id. """ if not re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9_.-]{0,127}", run_id): @@ -140,17 +192,36 @@ def restore(controller_file: Path, *, root: Path, expected_sha256: str, run_id: effect_runtime_result("runtime.checkpoint_artifacts", { "action": "verify", "root": str(root), "expected_sha256": expected_sha256}) manifest = json.loads((root / "checkpoint.json").read_bytes()) - if [c["id"] for c in manifest["components"]] != ["native_judge"]: - raise ValueError("Native import requires its single declared capsule component") + expected_components = ["native_judge"] + if phase_deadline_file is not None: + expected_components.append("host_phase_deadline") + if [c["id"] for c in manifest["components"]] != expected_components: + raise ValueError("Native import requires its declared capsule and explicit matching Host clock") reference = _reference(manifest["point_metadata"]["native_reference"]) if run_id == reference["source_run_id"]: raise ValueError("Continuation requires a fresh run identity") folder = root / "components/native_judge" - _bound_bank(folder, reference) + native = _bound_bank(folder, reference) bank = controller["bank_root"] + phase_path = None + phase = None + if phase_deadline_file is not None: + phase = _phase_budget((root / "components/host_phase_deadline").read_bytes(), native) + if manifest["point_metadata"].get("host_phase_budget") != phase: + raise ValueError("Host phase metadata differs from sealed deadline/native clock") + if phase["remaining_seconds"] <= PHASE_WORK_RESERVE_SECONDS: + raise ValueError("Captured Host phase has no work allowance beyond its existing reserve") + phase_path = _phase_destination(phase_deadline_file, root=root, bank=bank) target = bank / reference["checkpoint_id"] stage = Path(tempfile.mkdtemp(prefix=".loopx-native-import-", dir=bank)) + phase_fd = None + phase_published = False + native_attempted = False try: + if phase_path is not None: + # Empty is unusable by the existing SForge phase-entry shell. + # An uncertain native response must not fall back to a new budget. + phase_fd = os.open(phase_path, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600) effect_runtime_result("runtime.checkpoint_artifacts", { "action": "materialize", "root": str(root), "expected_sha256": expected_sha256, "destination": str(stage / "unit"), @@ -173,11 +244,42 @@ def restore(controller_file: Path, *, root: Path, expected_sha256: str, run_id: os.fsync(descriptor) finally: os.close(descriptor) - return _post(controller, "/api/v1/checkpoint/restore", { + phase_epoch = None if phase is None else math.floor(time.time() + phase["remaining_seconds"]) + if phase_epoch is not None and not 0 < phase_epoch < 2 ** 53: + raise ValueError("Destination Host phase clock is incompatible") + native_attempted = True + result = _post(controller, "/api/v1/checkpoint/restore", { "checkpoint_id": reference["checkpoint_id"], "manifest_sha256": reference["manifest_sha256"], "run_id": run_id, }, timeout) + if phase_fd is not None: + if not isinstance(result.get("token"), str) or not result["token"]: + raise ValueError("Native phase activation returned no fresh authority result") + # Charge activation/transport, not offline time or a fresh trial. + # Native alone owns accepted counters and remaining cooldown. + encoded = f"{phase_epoch}\n".encode() + if os.write(phase_fd, encoded) != len(encoded): + raise OSError("Destination Host phase write was incomplete") + os.fsync(phase_fd) + directory = os.open(phase_path.parent, os.O_RDONLY | os.O_DIRECTORY) + try: + os.fsync(directory) + finally: + os.close(directory) + if phase_path.read_bytes() != f"{phase_epoch}\n".encode(): + raise ValueError("Destination Host phase readback differs from activation") + phase_published = True + return result finally: + if phase_fd is not None: + try: + if native_attempted and not phase_published: + os.ftruncate(phase_fd, 0) + os.fsync(phase_fd) + finally: + os.close(phase_fd) + if not native_attempted: + phase_path.unlink() shutil.rmtree(stage) @@ -189,16 +291,21 @@ def main(argv=None) -> None: capture_parser.add_argument("--token-file", required=True, type=Path) capture_parser.add_argument("--capture-id", required=True) capture_parser.add_argument("--destination", required=True, type=Path) + capture_parser.add_argument("--phase-deadline-file", type=Path, + help="Explicit physical SForge Host phase file; omitted keeps native-only capture") restore_parser = commands.add_parser("restore") restore_parser.add_argument("--root", required=True, type=Path) restore_parser.add_argument("--expected-sha256", required=True) restore_parser.add_argument("--run-id", required=True) restore_parser.add_argument("--result-file", required=True, type=Path) + restore_parser.add_argument("--phase-deadline-file", type=Path, + help="Fresh physical Host phase file, required for a capsule containing that participant") args = parser.parse_args(argv) if args.command == "capture": source_token = _physical(args.token_file).read_text().strip() result = capture(args.controller_file, token=source_token, - capture_id=args.capture_id, destination=args.destination) + capture_id=args.capture_id, destination=args.destination, + phase_deadline_file=args.phase_deadline_file) print(json.dumps(result)) else: # Reserve before admission. A lost output is explicit recovery, never a @@ -209,7 +316,8 @@ def main(argv=None) -> None: with args.result_file.open("x") as result_file: args.result_file.chmod(0o600) result = restore(args.controller_file, root=args.root, - expected_sha256=args.expected_sha256, run_id=args.run_id) + expected_sha256=args.expected_sha256, run_id=args.run_id, + phase_deadline_file=args.phase_deadline_file) json.dump(result, result_file) result_file.flush() os.fsync(result_file.fileno()) diff --git a/benchmark/runtime/sforge.py b/benchmark/runtime/sforge.py index 97df4341ef..08df45236d 100644 --- a/benchmark/runtime/sforge.py +++ b/benchmark/runtime/sforge.py @@ -24,6 +24,7 @@ BenchmarkCodex, _CONTROL, _GOAL_ID, _PYTHON, _SCHEDULER_STATE, _SRC, ) from .scheduler import worker_command +from .worker import PHASE_WORK_RESERVE_SECONDS # Experiment profiles, local to the research runner; not new LoopX modes. @@ -102,7 +103,7 @@ def __init__(self, config, *, profile: str, cwd: str, ): raise ValueError("turn_envelope requires a heartbeat worker") self.turn_envelope = turn_envelope - if timeout_seconds <= 160: + if timeout_seconds <= PHASE_WORK_RESERVE_SECONDS: raise ValueError("Worker budget must exceed the 160s startup/settlement reserve") if not config.agent_model or not config.agent_effort: raise ValueError("Explicit model and reasoning effort are required") @@ -175,7 +176,7 @@ def __init__(self, config, *, profile: str, cwd: str, self.timeout_seconds = timeout_seconds # Let one call use the trial budget. The shared worker recomputes the # remaining deadline on every continuation, retaining cleanup time. - self.turn_timeout = timeout_seconds - 160 + self.turn_timeout = timeout_seconds - PHASE_WORK_RESERVE_SECONDS self.runtime = None self.prepared = False # Only the official profile delegates continuation to SForge. Heartbeat diff --git a/benchmark/runtime/sforge_entry.py b/benchmark/runtime/sforge_entry.py index 01a0f76da9..e3cf9886bb 100644 --- a/benchmark/runtime/sforge_entry.py +++ b/benchmark/runtime/sforge_entry.py @@ -11,7 +11,7 @@ import time from .planning import task_plan_packet, validate_plan_readback -from .worker import loopx_command, run_once +from .worker import PHASE_WORK_RESERVE_SECONDS, loopx_command, run_once def prepare_entry(env: dict[str, str]) -> bool: @@ -21,7 +21,7 @@ def prepare_entry(env: dict[str, str]) -> bool: remaining = float(env["LOOPX_PHASE_DEADLINE_EPOCH"]) - time.time() if not math.isfinite(remaining): raise ValueError("Invalid planning budget") - if remaining <= 160: + if remaining <= PHASE_WORK_RESERVE_SECONDS: return False path = Path(env["LOOPX_PLANNING_RESULT"]) if path.exists(): @@ -70,7 +70,7 @@ def cancelled(signum, frame): # Native process completion must not qualify it as a finished trial. return 1 # A successful plan may itself consume the remaining execution allowance. - if float(env["LOOPX_PHASE_DEADLINE_EPOCH"]) - time.time() <= 160: + if float(env["LOOPX_PHASE_DEADLINE_EPOCH"]) - time.time() <= PHASE_WORK_RESERVE_SECONDS: return 1 # Native SForge owns environment filtering and the outer phase deadline; # retain its proxy/isolation inputs rather than launching from setup hooks. diff --git a/benchmark/runtime/worker.py b/benchmark/runtime/worker.py index 598ea35577..9a950fd1e4 100644 --- a/benchmark/runtime/worker.py +++ b/benchmark/runtime/worker.py @@ -16,6 +16,8 @@ from benchmark.runtime.codex import Execution, prepare_codex_home, process_environment +PHASE_WORK_RESERVE_SECONDS = 160 + def read_json(text: str) -> dict: payload = json.loads(text) @@ -282,12 +284,12 @@ def run_once(env: dict[str, str]) -> dict: remaining = float(env["LOOPX_PHASE_DEADLINE_EPOCH"]) - time.time() # Reserve startup and settlement on every wake, then allow the # remaining time for work instead of reusing the initial timeout. - if remaining <= 160: + if remaining <= PHASE_WORK_RESERVE_SECONDS: receipt.update(ok=True, budget_exhausted=True, host_invoked=False) return receipt execution = replace( - execution, timeout_seconds=(remaining - 160 if execution.timeout_seconds is None - else min(execution.timeout_seconds, remaining - 160)) + execution, timeout_seconds=(remaining - PHASE_WORK_RESERVE_SECONDS if execution.timeout_seconds is None + else min(execution.timeout_seconds, remaining - PHASE_WORK_RESERVE_SECONDS)) ) prepare_codex_home( home, diff --git a/benchmark/tests/test_checkpoint_controller.py b/benchmark/tests/test_checkpoint_controller.py index 9e739d9db0..7f683b3ae1 100644 --- a/benchmark/tests/test_checkpoint_controller.py +++ b/benchmark/tests/test_checkpoint_controller.py @@ -5,6 +5,7 @@ import io import json import os +import subprocess import socket import tarfile import threading @@ -133,6 +134,8 @@ def test_native_controller_transport_preserves_original_point_and_fresh_authorit assert receipt["native_reference"]["incumbent"]["round"] == "agent-1" assert (exported / "components/native_judge/incumbent.tar.gz").read_bytes() == b"7" manifest = json.loads((exported / "components/native_judge/manifest.json").read_text()) + assert set(json.loads((exported / "checkpoint.json").read_bytes())["point_metadata"]) == {"native_reference"} + assert not (source.state.judge.config.log_dir / "checkpoints/.loopx-phase-attempts").exists() assert len(manifest["history"]) == 2 and manifest["opportunity"]["next_agent"] == 3 assert 0 < manifest["opportunity"]["remaining_cooldown_seconds"] <= 3600 for secret in (token, source.state.judge.admin_secret, destination.state.judge.admin_secret): @@ -160,6 +163,210 @@ def test_native_controller_transport_preserves_original_point_and_fresh_authorit assert len(grades) == 3 +def host_phase_command(tmp_path, monkeypatch): + """Real command builder; only installation/inference are outside this oracle.""" + pytest.importorskip("harbor") + from sforge.harness.config import SForgeConfig + from benchmark.runtime.sforge import BenchmarkCodex, SForgeWorker + + async def installed(self, environment): + pass + monkeypatch.setattr(BenchmarkCodex, "install", installed) + monkeypatch.setenv("CODEX_AUTH_JSON_PATH", "/synthetic-not-read") + logs = tmp_path / "phase-worker" + logs.mkdir() + worker = SForgeWorker(SForgeConfig(agent_model="fixture", agent_effort="xhigh"), + profile="heartbeat-resume", cwd="/task", timeout_seconds=43200, task_entry="seeded-todo") + worker.install_stop_hook(None, None, logs, None) + worker.prepared = True + # The provider's own phase-file/admission shell executes. Its following + # model command is excluded explicitly; candidate writing below is synthetic. + return worker.format_run_cmd("/task.md").split( + "set +e; timeout --signal=TERM --kill-after=30 ${remaining}s env", 1)[0] + + +def test_host_phase_and_native_opportunities_continue_without_offline_refill( + controllers, tmp_path, monkeypatch, capsys): + import requests + source, destination, first, second, files, grades = controllers + token = register(source, first, max_agent_submissions=3, submission_cooldown=2) + assert submit(first, token, 7).status_code == 200 + before = copy.deepcopy((source.state.judge.tokens, source.state.judge.submissions, source.state.judge.run_history)) + # An already admitted Host has 2h left from its original 12h configured + # trial. Fixture seeding represents that prior elapsed research, not a new run. + source_phase = tmp_path / "source-phase" + source_phase.write_text(f"{int(time.time()) + 7200}\n") + source_bytes = source_phase.read_bytes() + unit = tmp_path / "unit" + point = transport.capture(files[0], token=token, capture_id="c" * 32, + destination=unit, phase_deadline_file=source_phase) + body = json.loads((unit / "checkpoint.json").read_bytes()) + retained = body["point_metadata"]["host_phase_budget"]["remaining_seconds"] + assert 7190 < retained <= 7200 + prefix = host_phase_command(tmp_path, monkeypatch) + control = tmp_path / "host-control" + control.mkdir() + phase = control / "phase-deadline" + # A destination Host clock 8h later must not expire a copied source epoch + # or mint the configured 12h allowance. Native uses its real server clock. + clock = time.time() + 28800 + monkeypatch.setattr(transport, "time", SimpleNamespace(time=lambda: clock)) + private_result = tmp_path / "branch-result" + transport.main(["--controller-file", str(files[1]), "restore", "--root", str(unit), + "--expected-sha256", point["sha256"], "--run-id", "phase-branch", "--result-file", str(private_result), + "--phase-deadline-file", str(phase)]) + fresh = json.loads(private_result.read_bytes())["token"] + assert fresh not in capsys.readouterr().out and phase.stat().st_mode & 0o077 == 0 + assert 0 <= clock + retained - int(phase.read_bytes()) < 1 + bin = tmp_path / "bin" + bin.mkdir() + date = bin / "date" + date.write_text(f"#!/bin/sh\nprintf '%s\\n' {int(clock)}\n") + date.chmod(0o700) + candidate = tmp_path / "continued-candidate" + shell = prefix.replace("/opt/loopx-benchmark/control", str(control)) + result = subprocess.run(["/bin/bash", "-c", shell + + f"printf '8' > '{candidate}'; printf '%s' \"$remaining\""], + env={"PATH": str(bin) + ":/usr/bin:/bin"}, capture_output=True, text=True, check=True) + assert 7180 < int(result.stdout) <= 7200 and candidate.read_bytes() == b"8" + history = requests.get(second + "/api/v1/history", params={"token": fresh}, timeout=5).json() + assert history["agent_submissions"] == 1 and history["best_score"] == 7 + assert submit(second, fresh, 8).status_code == 429 + time.sleep(body["point_metadata"]["native_reference"]["opportunity"]["remaining_cooldown_seconds"] + .05) + assert submit(second, fresh, float(candidate.read_bytes())).status_code == 200 + history = requests.get(second + "/api/v1/history", params={"token": fresh}, timeout=5).json() + assert history["agent_submissions"] == 2 and history["best_round"] == "agent-2" and history["best_score"] == 8 + assert source_phase.read_bytes() == source_bytes + assert (source.state.judge.tokens, source.state.judge.submissions, source.state.judge.run_history) == before + assert len(grades) == 2 # No implicit grading of the retained incumbent. + with pytest.raises(ValueError, match="fresh physical"): + transport.restore(files[1], root=unit, expected_sha256=point["sha256"], + run_id="second-branch", phase_deadline_file=phase) + assert len(destination.state.judge.tokens) == 1 and len(grades) == 2 + + +@pytest.mark.parametrize("fault", ["missing_clock", "exhausted", "existing", "alias", "modified"]) +def test_host_phase_faults_refuse_before_native_admission(controllers, tmp_path, fault): + source, destination, first, second, files, grades = controllers + token = register(source, first) + assert submit(first, token, 7).status_code == 200 + phase_source = tmp_path / "source-phase" + phase_source.write_text(f"{int(time.time()) + (10 if fault == 'exhausted' else 7200)}\n") + unit = tmp_path / "unit" + point = transport.capture(files[0], token=token, capture_id="d" * 32, + destination=unit, phase_deadline_file=phase_source) + target = tmp_path / "phase" + if fault == "existing": + target.write_text("keep") + elif fault == "alias": + target.symlink_to(phase_source) + elif fault == "modified": + (unit / "components/host_phase_deadline").write_text(f"{int(time.time()) + 43200}\n") + with pytest.raises((ValueError, RuntimeError)): + transport.restore(files[1], root=unit, expected_sha256=point["sha256"], run_id="phase-branch", + phase_deadline_file=None if fault == "missing_clock" else target) + assert destination.state.judge.tokens == {} and len(grades) == 1 + if fault == "existing": + assert target.read_text() == "keep" + + +@pytest.mark.parametrize("fault", ["source_moves", "response_lost", "empty_response"]) +def test_phase_ambiguity_never_replaces_point_or_refills_budget(controllers, tmp_path, monkeypatch, fault): + source, destination, first, second, files, grades = controllers + token = register(source, first) + phase_source = tmp_path / "source-phase" + phase_source.write_text(f"{int(time.time()) + 7200}\n") + unit = tmp_path / "unit" + native_post = transport._post + def post(controller, route, body, timeout): + result = native_post(controller, route, body, timeout) + if route.endswith("/capture") and fault == "source_moves": + phase_source.write_text(f"{int(time.time()) + 43200}\n") + if route.endswith("/restore"): + if fault == "response_lost": + raise TimeoutError("Synthetic transport lost a committed native result") + if fault == "empty_response": + return {} + return result + monkeypatch.setattr(transport, "_post", post) + if fault == "source_moves": + with pytest.raises(ValueError, match="moved"): + transport.capture(files[0], token=token, capture_id="e" * 32, + destination=unit, phase_deadline_file=phase_source) + assert not unit.exists() + with pytest.raises(ValueError, match="Existing native point"): + transport.capture(files[0], token=token, capture_id="e" * 32, + destination=unit, phase_deadline_file=phase_source) + assert destination.state.judge.tokens == {} + else: + point = transport.capture(files[0], token=token, capture_id="e" * 32, + destination=unit, phase_deadline_file=phase_source) + control = tmp_path / "host-control" + control.mkdir() + target = control / "phase-deadline" + with pytest.raises((TimeoutError, ValueError)): + transport.restore(files[1], root=unit, expected_sha256=point["sha256"], + run_id="phase-branch", phase_deadline_file=target) + assert target.read_bytes() == b"" and len(destination.state.judge.tokens) == 1 + # Existing SForge entry treats empty as expired, never initializes a + # fresh configured allowance. Same identity requires owner readback. + shell = host_phase_command(tmp_path, monkeypatch).replace("/opt/loopx-benchmark/control", str(control)) + observed = subprocess.run(["/bin/bash", "-c", shell + "printf unexpected-work"], + capture_output=True, text=True) + assert observed.returncode != 0 and observed.stdout == "" + with pytest.raises(ValueError, match="fresh physical"): + transport.restore(files[1], root=unit, expected_sha256=point["sha256"], + run_id="different-branch", phase_deadline_file=target) + assert grades == [] + + +@pytest.mark.parametrize("raw", [b"", b"nan\n", b"1.5\n", b"9007199254740992\n"]) +def test_invalid_host_epoch_cannot_create_native_point(controllers, tmp_path, raw): + source, destination, first, second, files, grades = controllers + token = register(source, first) + phase = tmp_path / "phase" + phase.write_bytes(raw) + with pytest.raises(ValueError, match="phase|clocks"): + transport.capture(files[0], token=token, capture_id="f" * 32, + destination=tmp_path / "unit", phase_deadline_file=phase) + bank = source.state.judge.config.log_dir / "checkpoints" + assert not (bank / ("f" * 32)).exists() and not (bank / ".loopx-phase-attempts").exists() + assert destination.state.judge.tokens == {} and grades == [] + + +def test_native_only_point_cannot_acquire_a_later_host_budget(controllers, tmp_path): + source, destination, first, second, files, grades = controllers + token = register(source, first) + transport.capture(files[0], token=token, capture_id="f" * 32, destination=tmp_path / "first") + phase = tmp_path / "phase" + phase.write_text(f"{int(time.time()) + 43200}\n") + with pytest.raises(ValueError, match="Existing native point"): + transport.capture(files[0], token=token, capture_id="f" * 32, + destination=tmp_path / "later", phase_deadline_file=phase) + assert not (tmp_path / "later").exists() and destination.state.judge.tokens == {} and grades == [] + + +def test_partial_phase_write_keeps_committed_native_result_inert(controllers, tmp_path, monkeypatch): + source, destination, first, second, files, grades = controllers + token = register(source, first) + phase_source = tmp_path / "source-phase" + phase_source.write_text(f"{int(time.time()) + 7200}\n") + unit = tmp_path / "unit" + point = transport.capture(files[0], token=token, capture_id="f" * 32, + destination=unit, phase_deadline_file=phase_source) + class PartialWrite: + def __getattr__(self, name): + return getattr(os, name) + def write(self, descriptor, data): + return os.write(descriptor, data[:1]) + monkeypatch.setattr(transport, "os", PartialWrite()) + target = tmp_path / "phase" + with pytest.raises(OSError, match="incomplete"): + transport.restore(files[1], root=unit, expected_sha256=point["sha256"], + run_id="phase-branch", phase_deadline_file=target) + assert target.read_bytes() == b"" and len(destination.state.judge.tokens) == 1 and grades == [] + + @pytest.mark.parametrize("fault", ["wrong_admin", "corrupt", "wrong_digest", "aliased", "existing_bank"]) def test_invalid_controller_import_never_registers_or_regrades(controllers, tmp_path, fault): source, destination, first, second, files, grades = controllers @@ -213,7 +420,8 @@ def redirect(): @pytest.mark.skipif(os.environ.get("LOOPX_EDGEBENCH_DOCKER_SMOKE") != "1", reason="Requires explicit isolated native Docker qualification") -def test_real_docker_controller_export_import_and_next_grade(tmp_path, monkeypatch): +@pytest.mark.parametrize("phase_budget", [False, True]) +def test_real_docker_controller_export_import_and_next_grade(tmp_path, monkeypatch, phase_budget): import uuid import docker import requests @@ -275,26 +483,74 @@ def grade(url, token, score): assert grade(first, token, 7).status_code == 200 before = copy.deepcopy((source.state.judge.tokens, source.state.judge.submissions, source.state.judge.run_history)) exported = tmp_path / "capsule" - point = transport.capture(files[0], token=token, capture_id="c" * 32, destination=exported) + phase_source = tmp_path / "source-phase" + if phase_budget: + phase_source.write_text(f"{int(time.time()) + 7200}\n") + point = transport.capture(files[0], token=token, capture_id="c" * 32, destination=exported, + phase_deadline_file=phase_source if phase_budget else None) # Admission clock only: HTTP, transport/fsync, Docker and native # grading remain real. Offline/import IO must not spend cooldown. now = [time.time() + 7200] clock = SimpleNamespace(time=lambda: now[0]) monkeypatch.setattr(native, "time", clock) monkeypatch.setattr(checkpoint, "time", clock) - fresh = transport.restore(files[1], root=exported, expected_sha256=point["sha256"], run_id="branch")["token"] + monkeypatch.setattr(transport, "time", clock) + control = tmp_path / "host-control" + control.mkdir() + phase = control / "phase-deadline" + fresh = transport.restore(files[1], root=exported, expected_sha256=point["sha256"], run_id="branch", + phase_deadline_file=phase if phase_budget else None)["token"] assert fresh != token and len(grades) == 1 assert grade(second, fresh, 3).status_code == 429 now[0] += 3601 + if phase_budget: + # The actual SForge phase shell consumes the restored duration + # inside a disposable Linux container, then writes a new candidate. + prefix = host_phase_command(tmp_path, monkeypatch) + guest_control = "/opt/loopx-benchmark/control" + command = prefix + f"printf '%s' \"$remaining\"; printf '{{\"score\":3}}' > '{guest_control}/candidate.json'" + guest = client.containers.create(image.id, ["/bin/bash", "-c", command], + environment={"PATH": guest_control + "/bin:/usr/bin:/bin"}, + network_disabled=True, mem_limit="128m", nano_cpus=1_000_000_000) + try: + # Upload explicitly: a remote Docker daemon cannot read + # the caller's temporary directory through a host mount. + upload = io.BytesIO() + with tarfile.open(fileobj=upload, mode="w") as archive: + for relative in ("opt", "opt/loopx-benchmark", "opt/loopx-benchmark/control", + "opt/loopx-benchmark/control/bin"): + member = tarfile.TarInfo(relative) + member.type, member.mode = tarfile.DIRTYPE, 0o700 + archive.addfile(member) + for relative, data, mode in ( + ("phase-deadline", phase.read_bytes(), 0o600), + ("bin/date", f"#!/bin/sh\nprintf '%s\\n' {int(now[0])}\n".encode(), 0o700)): + member = tarfile.TarInfo("opt/loopx-benchmark/control/" + relative) + member.size, member.mode = len(data), mode + archive.addfile(member, io.BytesIO(data)) + assert guest.put_archive("/", upload.getvalue()) + guest.start() + assert guest.wait(timeout=30)["StatusCode"] == 0 + assert 3580 < int(guest.logs()) <= 3600 + stream, _ = guest.get_archive(guest_control + "/candidate.json") + try: + with tarfile.open(fileobj=io.BytesIO(b"".join(stream))) as archive: + assert json.loads(archive.extractfile("candidate.json").read()) == {"score": 3} + finally: + stream.close() + finally: + guest.remove(force=True) assert grade(second, fresh, 3).json()["round_id"] == "agent-2" history = requests.get(second + "/api/v1/history", params={"token": fresh}, timeout=5).json() assert history["best_score"] == 7 and history["best_round"] == "agent-1" assert grade(second, fresh, 99).status_code == 429 and len(grades) == 2 assert (source.state.judge.tokens, source.state.judge.submissions, source.state.judge.run_history) == before - with pytest.raises(ValueError, match="HTTP 409"): - transport.restore(files[1], root=exported, expected_sha256=point["sha256"], run_id="branch") + with pytest.raises(ValueError, match="fresh physical" if phase_budget else "HTTP 409"): + transport.restore(files[1], root=exported, expected_sha256=point["sha256"], run_id="branch", + phase_deadline_file=phase if phase_budget else None) finally: for app in (source, destination): app.state.judge._reaper_stop.set() app.state.judge._reaper_thread.join(timeout=1) client.images.remove(name + ".judge.fixture:fixture") + client.close() From 79ac163f949a84723b05c1491b2925bbb8203f76 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 10:18:24 +0800 Subject: [PATCH 29/69] docs(benchmark): define conserved phase activation and recovery boundaries Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/README.md | 32 ++++++++++++++++++- .../hierarchical-agent-stride-control-v0.md | 10 +++++- ...rarchical-agent-stride-control-v0.zh-CN.md | 8 ++++- 3 files changed, 47 insertions(+), 3 deletions(-) diff --git a/benchmark/README.md b/benchmark/README.md index c31737e92e..85db384262 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -370,9 +370,39 @@ fail without a fresh evaluation. Redirects cannot forward the admin secret. After an uncertain restore response, inspect native readback with the controller; do not switch run identities or perform ordinary fresh registration as a retry. +An explicit **SForge phase-clock participant** preserves a declared Host file's +remaining duration alongside native opportunities. Add +`--phase-deadline-file /private/source-phase-deadline` to capture, and +`--phase-deadline-file /private/new-host-control/phase-deadline` to restore. +Supply the original physical integer deadline from the admitted source Host, +through an authorized controller read route. The destination file must be new, +outside the artifact and native bank, and installed at the Host's existing +phase-file route before starting its ordinary entry command. + +The sealed deadline and native capture timestamp determine the remainder; the +receiver recomputes it. A point with two hours remaining keeps that duration +after an eight-hour offline gap. It does not reuse the old absolute epoch or +initialize another configured twelve-hour trial. Native admission still owns +consumed rounds and remaining cooldown. Host time begins immediately before +native activation, including its transport time; the integer phase format can +round the retained allowance down by less than one second. The existing worker's +startup/settlement reserve remains in force. + +A native-only point or visited native/Host capture attempt cannot acquire a later Host clock. A capsule +with this participant requires the matching explicit restore option; omitting it +cannot discard its allowance. Existing/aliased files, source movement, changed +sealed bytes and exhausted work allowance refuse activation. An uncertain native +response leaves the reserved phase file empty and unusable by the ordinary Host +entry, preventing a fresh-budget fallback. Reconcile that original native +operation before any owner-authorized recovery; no automatic replay, new run or +budget reset is supplied. Omit the option on **new captures** to retain native-only +transport. Full Host source attribution, source/native clock alignment and the +joint session/research/effect barrier remain qualification requirements. + This is an independently usable **native participant transport**, still partial relative to full research continuation. It does not join the worker's replan -barrier, restore Codex/LoopX authority or carry the remaining whole-trial budget. +barrier or restore Codex/LoopX authority. Its optional declared phase clock does +not qualify full research-state adoption or a same-point scientific comparison. Never put its controller configuration in `--checkpoint-request` or a worker payload. Omit both these controller operations and the native `--checkpoints` flag to disable this transport on a fresh judge; preserved capsules stay inert. diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md index 764a5ae2f8..445ab442fb 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md @@ -772,7 +772,15 @@ explicit controller authority captures/imports a sealed native bank and requests fresh native registration. Controller configuration stays out of solver payloads and exported participants; opaque data remains private and uncertified. This slice preserves native history, selected archive, counters and cooldown; it does not yet qualify the joint replan barrier, -model/LoopX state adoption, remaining whole-trial budget or useful research. +model/LoopX state adoption or useful research. Its explicit SForge phase-file +participant also seals the declared Host deadline against the native capture +timestamp. Restore recomputes the remaining duration and anchors a new Host phase +file at activation, charging transport rather than offline time or a fresh trial. +The existing work reserve and native opportunity owner remain unchanged. An +uncertain response leaves an unusable phase reservation, not a fresh-budget +fallback. Full Host provenance/clock alignment, joint research/effect adoption +and original-operation recovery still need their profile's qualification; this +participant transport cannot certify them. An inert verification unit grants no execution authority. The destination uses fresh experiment/Host identities and the existing Goal/session/state admission diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md index 0240453868..a3ba91ecbc 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md @@ -640,7 +640,13 @@ effect 采用、剩余总预算、已安装前端恢复或科学收益。显式 已将提议中的 native initializer 接入既有 typed 产物 owner：显式 controller 权限捕获/导入封存 native bank，再请求新的原生注册。controller 配置不进入 solver payload 或导出参与者；opaque 内容仍私有且未获隐私认证。此切片保留原生历史、选优 archive、计数及冷却，仍不 -资格化联合 replan barrier、模型/LoopX 状态采用、剩余总预算或实际有效研究。 +资格化联合 replan barrier、模型/LoopX 状态采用或实际有效研究。其显式 SForge +phase 文件参与者还将声明的 Host deadline 与 native 捕获时间绑定；恢复时重算 +剩余时长，在激活处锚定新的 Host phase 文件，计入传输时间，不计离线等待， +也不重置为新试验的完整预算。既有工作预留和 native 评测机会 owner 不变。 +不确定响应留下不可执行的 phase 预留，不能回退到新预算。完整 Host 来源/ +时钟对齐、联合研究/effect 采用与原操作恢复仍须按 profile 资格化；参与者 +传输本身不能证明这些结果。 inert verification unit 不授予执行权。目标侧使用新的实验/Host 身份，通过既有 Goal/session/state admission 进入。历史租约、quota、timer、pending effect 和 From 510ede10630673a794dd92ff6b68e06d454058f7 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 11:53:54 +0800 Subject: [PATCH 30/69] fix(checkpoint): witness source session binding through capture Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/runtime/checkpoint.py | 7 +++++++ loopx/control_plane/runtime/checkpoint_artifacts.ts | 5 +++-- tests/control_plane/test_research_checkpoint_capture.py | 9 ++++++--- 3 files changed, 16 insertions(+), 5 deletions(-) diff --git a/benchmark/runtime/checkpoint.py b/benchmark/runtime/checkpoint.py index f5dd0fa903..26b69a195d 100644 --- a/benchmark/runtime/checkpoint.py +++ b/benchmark/runtime/checkpoint.py @@ -358,6 +358,13 @@ def capture_after_wake(env: dict[str, str], receipt: dict[str, Any], *, sandbox=env.get("LOOPX_CODEX_SANDBOX", "danger-full-access")), {}) if source_wake.session_id != session.get("session_id"): raise ValueError("native checkpoint differs from the current source session binding") + # This sideband record belongs to the existing session owner. It + # must participate even when loopx_state was not requested: native + # store bytes alone cannot witness a binding/profile replacement. + from loopx.control_plane.turn_driver.codex_sessions import _session_path + binding_path = _session_path(source_wake.root, source_wake.lineage, "agent") + if not any(binding_path.is_relative_to(Path(entry["source_path"])) for entry in components): + components = [*components, {"id": "source_session_binding", "source_path": str(binding_path)}] metadata["session"] = {**session, "session_profile_digest": source_wake.digest} native = native_session_checkpoint(env, session.get("session_id", ""), timeout=max(.1, deadline - time.time())) diff --git a/loopx/control_plane/runtime/checkpoint_artifacts.ts b/loopx/control_plane/runtime/checkpoint_artifacts.ts index d881207a64..f7d0ec18fc 100644 --- a/loopx/control_plane/runtime/checkpoint_artifacts.ts +++ b/loopx/control_plane/runtime/checkpoint_artifacts.ts @@ -13,6 +13,7 @@ import {snapshotSqliteDatabase} from "../coordination/sqlite_backup.ts"; import {durableWriteJson, publishNewDirectory} from "../effect_runtime_io.ts"; import {settlementIdentity} from "../effect_program.ts"; import {normalizeHeartbeatReplanObligationId, normalizeHeartbeatTodoId} from "../quota/heartbeat_receipt_identity.ts"; +import {EffectRuntimeConflictError} from "../effect_runtime_errors.ts"; import {EffectiveAction} from "../quota/effective_action.generated.ts"; const SCHEMA = "loopx_checkpoint_artifact_set_v0"; @@ -90,7 +91,7 @@ async function witness(entries: Component[], sourceObservation = true): Promise< return result; } function unchanged(a: unknown, b: unknown, message = "checkpoint source moved; retry at a new owned barrier"): void { - if (canonicalAuthoritySha256(a) !== canonicalAuthoritySha256(b)) throw new Error(message); + if (canonicalAuthoritySha256(a) !== canonicalAuthoritySha256(b)) throw new EffectRuntimeConflictError(message); } async function absentDestination(path: string): Promise { await physical(dirname(path)); @@ -232,7 +233,7 @@ export async function checkpointArtifacts(value: unknown): Promise { const sources = components(request.components); if (sources.some(c => destination === c.source_path || destination.startsWith(c.source_path + sep) || c.source_path.startsWith(destination + sep))) throw new Error("checkpoint output overlaps its source"); const before = await witness(sources); - if (request.expected_source_sha256 !== canonicalAuthoritySha256(before)) throw new Error("checkpoint source moved before capture"); + if (request.expected_source_sha256 !== canonicalAuthoritySha256(before)) throw new EffectRuntimeConflictError("checkpoint source moved before capture"); await mkdir(join(staging, "components"), {mode: 0o700}); for (const c of sources) await copyComponent(c.source_path, join(staging, "components", c.id)); unchanged(before, await witness(sources)); diff --git a/tests/control_plane/test_research_checkpoint_capture.py b/tests/control_plane/test_research_checkpoint_capture.py index ee5f48eb07..fe68dd21e1 100644 --- a/tests/control_plane/test_research_checkpoint_capture.py +++ b/tests/control_plane/test_research_checkpoint_capture.py @@ -216,7 +216,8 @@ def altered_probe(method, params, **kwargs): @pytest.mark.skipif(not os.environ.get("LOOPX_NATIVE_CHECKPOINT_BIN"), reason="Requires an explicit real native Host binary; static local provider only") @pytest.mark.parametrize("source_moves", [False, True, "binding"]) -def test_real_native_cursor_must_match_sealed_store_basis(tmp_path, monkeypatch, source_moves): +@pytest.mark.parametrize("mutation_stage", ["probe", "capture"]) +def test_real_native_cursor_must_match_sealed_store_basis(tmp_path, monkeypatch, source_moves, mutation_stage): """Actual CLI settlement, native completed Turns and TS SQLite artifacts.""" from benchmark.tests.test_native_checkpoint import real_native_source from benchmark.runtime.codex import Execution @@ -263,7 +264,7 @@ def run(*args): "session": {"session_id": source}} injected = [] def between_read_and_probe(method, params, **kwargs): - if source_moves and params["action"] == "probe" and not injected: + if source_moves and params["action"] == mutation_stage and not injected: if source_moves == "binding": injected.append(True) _store_codex_cli_session(runtime, lineage={"goal_id": cli.GOAL_ID, "agent_id": cli.AGENT_ID}, @@ -278,8 +279,10 @@ def between_read_and_probe(method, params, **kwargs): assert len(calls) == (2 if source_moves is True else 1) # Static local streams only; no implicit new work. if source_moves: assert not receipt["checkpoint"]["captured"], receipt - assert ("profile changed" if source_moves == "binding" else "native") in receipt["checkpoint"]["error"] assert not destination.exists() + expected_error = ("checkpoint source moved" if mutation_stage == "capture" else + "profile changed" if source_moves == "binding" else "native") + assert expected_error in receipt["checkpoint"]["error"] else: assert receipt["checkpoint"]["captured"], receipt manifest = json.loads((destination / "checkpoint.json").read_text()) From 2decc8761aa1c0c4361ea382557b77198e0391f1 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 12:09:38 +0800 Subject: [PATCH 31/69] feat(benchmark): adopt captured Todo evidence into native continuation Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/runtime/checkpoint.py | 28 +++ benchmark/runtime/research_checkpoint.py | 203 ++++++++++++++++++ benchmark/tests/test_native_checkpoint.py | 75 ++++++- .../test_research_checkpoint_capture.py | 35 ++- .../test_research_context_adoption.py | 124 +++++++++++ 5 files changed, 456 insertions(+), 9 deletions(-) create mode 100644 benchmark/runtime/research_checkpoint.py create mode 100644 tests/control_plane/test_research_context_adoption.py diff --git a/benchmark/runtime/checkpoint.py b/benchmark/runtime/checkpoint.py index 26b69a195d..cef51f61de 100644 --- a/benchmark/runtime/checkpoint.py +++ b/benchmark/runtime/checkpoint.py @@ -9,6 +9,7 @@ import hashlib import json import math +import os import subprocess import time from pathlib import Path, PurePosixPath @@ -336,6 +337,25 @@ def capture_after_wake(env: dict[str, str], receipt: dict[str, Any], *, components = request["components"] if request.get("loopx_state"): components, metadata["loopx_state"] = _loopx_state_components(env, components, destination) + from loopx.control_plane.coordination.local_authority import local_authority_is_promoted + research = None + if local_authority_is_promoted(runtime_root=Path(env["LOOPX_RUNTIME_ROOT"]), goal_id=env["LOOPX_GOAL_ID"]): + from .research_checkpoint import canonical_context, _encoded + research = canonical_context(env, deadline=deadline) + evidence = Path(env["LOOPX_WAKE_LOG_DIR"]) / receipt["turn_id"] / "checkpoint-research-context.json" + research_raw = _encoded(research) + with os.fdopen(os.open(evidence, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600), "wb") as file: + file.write(research_raw) + parent = next((entry for entry in components if evidence.is_relative_to(Path(entry["source_path"]))), None) + if parent is None: + parent = {"id": "research_context", "source_path": str(evidence)} + components = [*components, parent] + metadata["research_context"] = {"component_id": parent["id"], + "member_path": evidence.relative_to(Path(parent["source_path"])).as_posix() if evidence != Path(parent["source_path"]) else "", + "sha256": hashlib.sha256(research_raw).hexdigest(), + "provider_revision": research["snapshot"]["provider_revision"], "store_identity": research["snapshot"]["store_identity"]} + else: + metadata["canonical_context_unavailable"] = "Source coordination is unpromoted; its retained inventory remains inert." opportunity = None if request.get("evaluation_opportunity"): from .sforge_checkpoint import capture_opportunity @@ -389,6 +409,14 @@ def capture_after_wake(env: dict[str, str], receipt: dict[str, Any], *, (source_wake.session_id, source_wake.goal_ref, source_wake.digest)): raise ValueError("native checkpoint source session binding moved before sealing") if request.get("loopx_state"): + if research is not None: + if canonical_context(env, deadline=deadline) != research: + raise ValueError("canonical research context moved before sealing") + retained = metadata["research_context"] + member = next((entry for entry in probe["source_witness"][retained["component_id"]] + if entry["path"] == retained["member_path"]), None) + if member is None or member.get("sha256") != retained["sha256"]: + raise ValueError("canonical research context bytes moved before sealing") declared = metadata["loopx_state"] witness = probe["source_witness"][declared["registry_component_id"]] member = next((entry for entry in witness if entry["path"] == declared["registry_member_path"]), None) diff --git a/benchmark/runtime/research_checkpoint.py b/benchmark/runtime/research_checkpoint.py new file mode 100644 index 0000000000..14a66f340f --- /dev/null +++ b/benchmark/runtime/research_checkpoint.py @@ -0,0 +1,203 @@ +"""Private checkpoint context transport through existing canonical Todo owners. + +Captured records are historical input, never a replacement authority store. +Only the existing explicit continuation owner can adopt a note on an already +claimed destination Todo. Graph/effect/vision reactivation is outside this seam. +""" +from __future__ import annotations + +import argparse +import hashlib +import json +import os +import time +from pathlib import Path, PurePosixPath + +from loopx.control_plane.effect_runtime import effect_runtime_result +from loopx.control_plane.coordination.canonical_snapshot import read_canonical_snapshot + + +def canonical_context(env: dict[str, str], *, deadline: float) -> dict: + """Retain the complete revision-bound read, without the execution leases.""" + def rpc(method, payload, **kwargs): + remaining = deadline - time.time() + if remaining <= 0: + raise TimeoutError("research checkpoint capture window closed") + return effect_runtime_result(method, payload, timeout=remaining) + result = read_canonical_snapshot(rpc=rpc, runtime_root=env["LOOPX_RUNTIME_ROOT"], + goal_id=env["LOOPX_GOAL_ID"], include_leases=False, projection_readback=None, + timeout=max(.1, deadline - time.time())) + if result.get("status") != "loaded": + raise ValueError("research checkpoint requires complete canonical Todo context") + return {"goal_id": env["LOOPX_GOAL_ID"], "snapshot": result} + + +def _encoded(value): + return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode() + + +def _handoff(env, payload): + from loopx.agent_registry import registered_agent_ids_from_registry + request = {"runtime_root": env["LOOPX_RUNTIME_ROOT"], "goal_id": env["LOOPX_GOAL_ID"], + "agent_id": env["LOOPX_AGENT_ID"], "registered_agents": registered_agent_ids_from_registry( + Path(env["LOOPX_REGISTRY"]), env["LOOPX_GOAL_ID"]), **payload} + result = effect_runtime_result("coordination.local_authority.todo_continuation", request) + if result.get("ok") is not True: + raise ValueError("checkpoint context continuation rejected: " + str(result.get("reason_code", result.get("status")))) + return result + + + +def _context_session(env, metadata): + """Read the admitted native branch before publishing captured Todo context.""" + from .codex import Execution + from .sessions import BenchmarkSessionWake + from .native_checkpoint import native_session_checkpoint + + wake = BenchmarkSessionWake(env, Execution(context="resume", sandbox=env["LOOPX_CODEX_SANDBOX"]), {}) + native = metadata["native_session"] + if (not wake.session_id or wake.session_id == native["session_id"] or + wake.digest != metadata.get("session", {}).get("session_profile_digest")): + raise ValueError("context adoption requires the admitted destination session profile") + observed = native_session_checkpoint(env, wake.session_id, timeout=30)["metadata"] + for key in ("visible_history_sha256", "turn_ids", "completed_turn_id", "host_version", "config_sha256"): + if observed[key] != native[key]: + raise ValueError("destination native history or configuration moved beyond the captured context") + return wake.session_id + + +def adopt_todo_context(env: dict[str, str], unit: Path, digest: str, *, source_todo_id: str, + todo_id: str, operation_id: str, expected_revision: str, + lease_key: str, lease_version: int) -> dict: + """Adopt retained context using the destination's current claim and lease. + + No Todo is created, completed or transferred, and no historical authority, + effect, strategy or graph is installed. Prepare/adopt each use stable original + operation identities. A failed/uncertain call must be read back through that + same owner; this adapter never guesses a new identity or starts model work. + """ + from loopx.control_plane.turn_driver.codex_sessions import select_codex_cli_session + from loopx.control_plane.todos.provider_projection import settle_canonical_todo_projection + + effect_runtime_result("runtime.checkpoint_artifacts", { + "action": "verify", "root": str(unit), "expected_sha256": digest}) + metadata = json.loads((unit / "checkpoint.json").read_bytes())["point_metadata"] + if not all(isinstance(metadata.get(key), dict) for key in ("loopx_state", "native_session", "research_context")): + raise ValueError("checkpoint lacks its canonical Todo or native session context participant") + source = metadata["loopx_state"] + if (source["goal_id"] == env["LOOPX_GOAL_ID"] or source["agent_id"] == env["LOOPX_AGENT_ID"]): + raise ValueError("context adoption requires distinct destination Goal and Agent") + declared = metadata["research_context"] + component, member = declared["component_id"], declared["member_path"] + if (not isinstance(component, str) or Path(component).name != component or component in {".", ".."} or + not isinstance(member, str) or PurePosixPath(member).is_absolute() or + ".." in PurePosixPath(member).parts or (member and str(PurePosixPath(member)) != member)): + raise ValueError("retained context member must stay within its artifact component") + context_file = unit / "components" / component + if member: + context_file /= member + raw = context_file.read_bytes() + if hashlib.sha256(raw).hexdigest() != declared["sha256"]: + raise ValueError("retained canonical context digest differs") + carrier = json.loads(raw) + context = carrier["snapshot"] + if (carrier.get("goal_id") != source["goal_id"] or + context.get("provider_revision") != declared["provider_revision"] or + context.get("store_identity") != declared["store_identity"]): + raise ValueError("retained canonical context revision differs") + retained = next((row for row in context["todos"] if row["todo_id"] == source_todo_id), None) + if retained is None: + raise ValueError("source Todo is absent from the captured context") + session_id = _context_session(env, metadata) + bound = select_codex_cli_session(Path(env["LOOPX_RUNTIME_ROOT"]), lineage={ + "goal_id": env["LOOPX_GOAL_ID"], "agent_id": env["LOOPX_AGENT_ID"]}, session_scope="agent") + if bound is None or bound["session_id"] != session_id: + raise ValueError("context adoption requires the current distinct destination session binding") + workspace = Path(env["LOOPX_PROJECT"]) + if not workspace.is_absolute() or workspace.resolve(strict=True) != workspace: + raise ValueError("destination context workspace must be a physical absolute path") + proof = {"idempotency_key": lease_key, "expected_version": lease_version} + common = {"todo_id": todo_id, "session_id": bound["session_id"], "workspace": str(workspace), + "artifacts": [], "lease_proof": proof} + # The owner's inspect performs the real current claim/lease/dependency and + # acceptance checks. Source records only contribute context; they are never + # passed as operational fields to prepare or adopt. + inspect = _handoff(env, {**common, "action": "inspect"}) + if (inspect["provider_revision"] != expected_revision or + inspect["claimed_by"] != env["LOOPX_AGENT_ID"] or + inspect.get("execution_authority", {}).get("allowed") is not True): + raise ValueError("destination context revision, owner or execution authority changed") + destination = canonical_context(env, deadline=time.time() + 30)["snapshot"] + current = next((row for row in destination["todos"] if row["todo_id"] == todo_id), None) + if destination["provider_revision"] != expected_revision or current is None: + raise ValueError("destination canonical context moved before preparation") + if current.get("note"): + raise ValueError("destination Todo already has context; inspect its original owner instead of replacing it") + # Keep the complete source read in a workspace-relative private artifact. + # Never overwrite a moved artifact on an operation retry. + relative = Path(".local/checkpoint-context") / (digest + ".json") + target = workspace / relative + for directory in (workspace / ".local", target.parent): + if directory.is_symlink() or (directory.exists() and directory.resolve(strict=True) != directory): + raise ValueError("destination context artifact must not use a symlink") + directory.mkdir(exist_ok=True, mode=0o700) + if target.is_symlink(): + raise ValueError("destination context artifact must not use a symlink") + try: + with os.fdopen(os.open(target, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600), "wb") as file: + if file.write(raw) != len(raw): + raise OSError("destination context artifact write is incomplete") + file.flush() + os.fsync(file.fileno()) + except FileExistsError: + if target.read_bytes() != raw: + raise ValueError("destination context artifact already differs") + if target.read_bytes() != raw: + raise ValueError("destination context artifact readback differs") + effect_runtime_result("runtime.checkpoint_artifacts", { + "action": "verify", "root": str(unit), "expected_sha256": digest}) + note = {"work_summary": "Retained canonical Todo context from the verified checkpoint is available in " + relative.as_posix() + + ". Historical owners, statuses and operational references are evidence only; current Goal/Todo/lease rules govern work.", + "source_refs": ["checkpoint:" + digest, "source-todo:" + source_todo_id, + "source-revision:" + declared["provider_revision"]], + "files_touched": [{"path": relative.as_posix(), "action": "read", + "summary": "Complete captured canonical Todo population, including retained notes and rejected approaches."}]} + prepared = _handoff(env, {**common, "action": "prepare", "session_id": metadata["native_session"]["session_id"], + "operation_id": operation_id + "-prepare", "expected_provider_revision": expected_revision, "context": note}) + if select_codex_cli_session(Path(env["LOOPX_RUNTIME_ROOT"]), lineage={ + "goal_id": env["LOOPX_GOAL_ID"], "agent_id": env["LOOPX_AGENT_ID"]}, session_scope="agent") != bound: + raise ValueError("destination session changed; inspect the original prepared context") + if _context_session(env, metadata) != session_id: + raise ValueError("destination native branch changed; inspect the original prepared context") + adopted = _handoff(env, {**common, "action": "adopt", "artifacts": [relative.as_posix()], + "operation_id": operation_id + "-adopt", "expected_provider_revision": prepared["provider_revision"]}) + effect_runtime_result("runtime.checkpoint_artifacts", { + "action": "verify", "root": str(unit), "expected_sha256": digest}) + if target.read_bytes() != raw: + raise ValueError("adopted context artifact changed; inspect the original continuation receipt") + result = settle_canonical_todo_projection(adopted, registry_path=Path(env["LOOPX_REGISTRY"]), + runtime_root=Path(env["LOOPX_RUNTIME_ROOT"]), goal_id=env["LOOPX_GOAL_ID"]) + if result.get("ok") is not True: + raise ValueError("continuation committed; reconcile its original projection receipt") + return {"context_adopted": True, "session_id": bound["session_id"], "todo_id": todo_id, + "artifact": str(target), "context_sha256": declared["sha256"], + "continuation": result, "model_calls": 0, "research_point_qualified": False} + + +def main(argv=None): + parser = argparse.ArgumentParser(description="Adopt checkpoint Todo context on an already owned destination Todo.") + for name in ("root", "expected-sha256", "source-todo-id", "todo-id", "operation-id", "expected-revision", "lease-key"): + parser.add_argument("--" + name, required=True) + parser.add_argument("--lease-version", type=int, required=True) + args = parser.parse_args(argv) + required = ("LOOPX_PROJECT", "LOOPX_REGISTRY", "LOOPX_RUNTIME_ROOT", "LOOPX_GOAL_ID", "LOOPX_AGENT_ID", + "CODEX_BIN", "CODEX_HOME", "MODEL_NAME", "REASONING_EFFORT", "LOOPX_CODEX_SANDBOX") + if any(not os.environ.get(key) for key in required): + parser.error("Explicit destination registry, runtime, workspace, Goal and Agent are required") + print(json.dumps(adopt_todo_context(dict(os.environ), Path(args.root), args.expected_sha256, + source_todo_id=args.source_todo_id, todo_id=args.todo_id, operation_id=args.operation_id, + expected_revision=args.expected_revision, lease_key=args.lease_key, lease_version=args.lease_version))) + + +if __name__ == "__main__": + main() diff --git a/benchmark/tests/test_native_checkpoint.py b/benchmark/tests/test_native_checkpoint.py index dc08e509bf..8c4657c532 100644 --- a/benchmark/tests/test_native_checkpoint.py +++ b/benchmark/tests/test_native_checkpoint.py @@ -195,8 +195,8 @@ def test_real_native_workspace_readback_rejects_foreign_code_basis(tmp_path): def test_real_fork_admission_and_native_resume_use_retained_state(tmp_path, capsys, monkeypatch, case): """Real Host/CLI/Goal/session/artifact owners; deterministic synthetic work. - This qualifies the conversation participant, not full research/evaluator - adoption or benchmark utility. No source trial or paid model is launched. + This jointly qualifies native conversation and canonical Todo evidence + adoption, not full research/evaluator state or benchmark utility. No source trial or paid model is launched. """ import shutil import subprocess @@ -236,7 +236,7 @@ def register(name, agent): "--text", "Verify the retained candidate and preserve applicable negative evidence."]) == 0 added = json.loads(capsys.readouterr().out) return registry, runtime, added["todo_id"] - source_registry, source_runtime, _ = register(source_goal, source_agent) + source_registry, source_runtime, source_todo = register(source_goal, source_agent) registry, runtime, destination_todo = register(destination_goal, destination_agent) prefix = ["--format", "json", "--registry", str(registry), "--runtime-root", str(runtime)] turn_id = "checkpoint-continuation-fixture" @@ -245,13 +245,29 @@ def register(name, agent): assert main([*prefix, "quota", "should-run", "--runtime-profile", "generic_cli", *binding]) == 0 guard = json.loads(capsys.readouterr().out) assert guard["interaction_contract"]["agent_channel"]["delivery_allowed"] is True + from benchmark.runtime.research_checkpoint import canonical_context, adopt_todo_context, _encoded + source_prefix = ["--format", "json", "--registry", str(source_registry), "--runtime-root", str(source_runtime)] + def invoke(arguments): + code = main(arguments) + output = capsys.readouterr().out + assert code == 0, output + return json.loads(output) + source_binding = ["--goal-id", source_goal, "--todo-id", source_todo] + invoke([*source_prefix, "task-lease", "acquire", *source_binding, + "--owner", source_agent, "--idempotency-key", "source-context"]) + source_revision = invoke([*source_prefix, "todo", "list", *source_binding])["authority_read"]["provider_revision"] + invoke([*source_prefix, "todo", "update", *source_binding, "--agent-id", source_agent, + "--note", "Candidate 3 failed the synthetic check; preserve candidate 7.", + "--update-operation-id", "source-negative-evidence", "--update-expected-provider-revision", source_revision, + "--task-lease-idempotency-key", "source-context", "--task-lease-expected-version", "1"]) requests = [] + retained_context_path = None def response(request, index, message): requests.append(request) if index == 2: return {"type": "function_call", "id": "fc_retained", "call_id": "call_retained", "name": "exec_command", "arguments": json.dumps({ - "cmd": "cat candidate.txt research.txt", "max_output_tokens": 1000})} + "cmd": "cat candidate.txt research.txt " + retained_context_path, "max_output_tokens": 6000})} if index == 3: message["content"][0]["text"] = "Retained candidate 7 verified; candidate 3 remains rejected." return message @@ -270,16 +286,31 @@ def response(request, index, message): native = native_session_checkpoint(env, source_session, timeout=20) config = tmp_path / "config.toml" config.write_bytes((Path(env["CODEX_HOME"]) / "config.toml").read_bytes()) + source_context = canonical_context(env, deadline=time.time() + 30) + context_raw = _encoded(source_context) + context_export = tmp_path / "source-context.json" + context_export.write_bytes(context_raw) components = [*native["components"], {"id": "code", "source_path": str(candidate)}, - {"id": "research", "source_path": str(research)}] + {"id": "research", "source_path": str(research)}, + {"id": "research_context", "source_path": str(context_export)}] probe = effect_runtime_result("runtime.checkpoint_artifacts", {"action": "probe", "components": components}) unit = tmp_path / "point" captured = effect_runtime_result("runtime.checkpoint_artifacts", {"action": "capture", "destination": str(unit), "components": components, "expected_source_sha256": probe["source_sha256"], "point_metadata": {"native_session": native["metadata"], "session": {"session_id": source_session, "session_profile_digest": source_wake.digest}, - "loopx_state": {"goal_id": source_goal, "agent_id": source_agent}}}, retry_safe=False) + "loopx_state": {"goal_id": source_goal, "agent_id": source_agent}, + "research_context": {"component_id": "research_context", "member_path": "", + "sha256": hashlib.sha256(context_raw).hexdigest(), + "provider_revision": source_context["snapshot"]["provider_revision"], + "store_identity": source_context["snapshot"]["store_identity"]}}}, retry_safe=False) digest = captured["sha256"] + invoke([*source_prefix, "todo", "update", *source_binding, "--agent-id", source_agent, + "--note", "Candidate 999 is later evidence outside the captured point.", + "--update-operation-id", "later-negative-evidence", "--update-expected-provider-revision", + source_context["snapshot"]["provider_revision"], "--task-lease-idempotency-key", "source-context", + "--task-lease-expected-version", "1"]) + later_context = canonical_context(env, deadline=time.time() + 30) # Qualify use of captured inputs, not the source's later working files. # A quiesced same-machine fixture swaps fixed paths; this is not evidence # of complete guest/environment isolation or a scientific matched trial. @@ -375,6 +406,26 @@ def forbidden_fork(*args, **kwargs): assert len(calls) == 1 and load_codex_cli_session(runtime, lineage={ "goal_id": destination_goal, "agent_id": destination_agent}, session_scope="agent") == bound + # Both participants meet at the real destination continuation owner. + # No source claim/lease is replayed; adoption alone still starts no work. + lease = invoke([*prefix, "task-lease", "acquire", "--goal-id", destination_goal, + "--todo-id", destination_todo, "--owner", destination_agent, "--idempotency-key", "branch-context"]) + current = invoke([*prefix, "todo", "list", "--goal-id", destination_goal, "--todo-id", destination_todo]) + adopted_cli = subprocess.run([sys.executable, "-m", "benchmark.runtime.research_checkpoint", + "--root", str(unit), "--expected-sha256", digest, "--source-todo-id", source_todo, + "--todo-id", destination_todo, "--operation-id", "branch-context", + "--expected-revision", current["authority_read"]["provider_revision"], + "--lease-key", "branch-context", "--lease-version", str(lease["lease"]["version"])], + env=env, text=True, capture_output=True, timeout=60) + assert adopted_cli.returncode == 0, adopted_cli.stderr + context_result = json.loads(adopted_cli.stdout) + retained_context_path = str(Path(context_result["artifact"]).relative_to(workspace)) + assert Path(context_result["artifact"]).read_bytes() == context_raw and len(calls) == 1 + adopted_todo = invoke([*prefix, "todo", "list", "--goal-id", destination_goal, "--todo-id", destination_todo]) + assert adopted_todo["todo"]["claimed_by"] == destination_agent + assert json.loads(adopted_todo["todo"]["note"])["source_session"] == source_session + guard = invoke([*prefix, "quota", "should-run", "--runtime-profile", "generic_cli", *binding]) + assert guard["interaction_contract"]["agent_channel"]["delivery_allowed"] is True receipt = {} wake = BenchmarkSessionWake(env, execution, receipt) assert wake.session_id == admitted["session_id"] and receipt["session"]["action"] == "resume" @@ -392,10 +443,22 @@ def forbidden_fork(*args, **kwargs): assert destination_goal in json.dumps(requests[1]) and destination_todo in json.dumps(requests[1]) assert "retained candidate: 7" in json.dumps(requests[2]) assert "candidate 3 was rejected" in json.dumps(requests[2]) + assert "Candidate 3 failed the synthetic check" in json.dumps(requests[2]) + assert "Candidate 999" not in json.dumps(requests[2]) + assert canonical_context({**env, "LOOPX_GOAL_ID": source_goal, + "LOOPX_RUNTIME_ROOT": str(source_runtime)}, deadline=time.time() + 30) == later_context assert "Retained candidate 7 verified" in result.stdout retained = native_session_checkpoint(env, admitted["session_id"], timeout=20) assert retained["metadata"]["turn_ids"][:-1] == native["metadata"]["turn_ids"] assert retained["metadata"]["completed_turn_id"] != native["metadata"]["completed_turn_id"] + with pytest.raises(ValueError, match="native history or configuration moved"): + adopt_todo_context(env, unit, digest, source_todo_id=source_todo, + todo_id=destination_todo, operation_id="must-not-adopt-later-session", + expected_revision=adopted_todo["authority_read"]["provider_revision"], + lease_key="branch-context", lease_version=lease["lease"]["version"]) + assert len(calls) == 3 + assert invoke([*prefix, "todo", "list", "--goal-id", destination_goal, + "--todo-id", destination_todo])["todo"] == adopted_todo["todo"] assert candidate.read_text() == "retained candidate: 7\n" assert (late_source / candidate.name).read_text() == "later candidate: 999\n" assert (late_source / research.name).read_text() == "later evidence unavailable at the captured point\n" diff --git a/tests/control_plane/test_research_checkpoint_capture.py b/tests/control_plane/test_research_checkpoint_capture.py index fe68dd21e1..157498eaaf 100644 --- a/tests/control_plane/test_research_checkpoint_capture.py +++ b/tests/control_plane/test_research_checkpoint_capture.py @@ -375,21 +375,27 @@ def fail_probe(method, params, **kwargs): @pytest.mark.parametrize('provider', ['file', 'sqlite']) -def test_controller_participant_keeps_canonical_state_and_journals_inert(tmp_path, monkeypatch, provider): +@pytest.mark.parametrize(("drift", "mutation_stage"), [(None, "probe"), + ("canonical_head", "probe"), ("exported_bytes", "probe"), + ("canonical_head", "capture"), ("exported_bytes", "capture")]) +def test_controller_participant_keeps_canonical_state_and_journals_inert(tmp_path, monkeypatch, provider, drift, mutation_stage): from canonical_authority_fixture import initialize_canonical_authority from loopx.control_plane.coordination.runtime_shadow import build_todo_runtime_shadow_projection from loopx.control_plane.goals.first_party_host_admission import capture_first_party_host_goal_ref from loopx.control_plane.quota.settlement import read_heartbeat_settlement import subprocess - def authority_head(root): + def authority_head(root, advance=False): module = Path(__file__).resolve().parents[2]/'loopx/control_plane/coordination/local_authority_provider.ts' script = ('import {openLocalAuthorityStore} from '+json.dumps(module.as_uri())+';' "let input='';for await(const chunk of process.stdin)input+=chunk;" 'const x=JSON.parse(input);const store=await openLocalAuthorityStore(x.root,x.goal,{}, {existingOnly:true});' + "if(x.advance){const head=await store.loadAuthority();const result=await store.commitAuthority({" + "expected_provider_revision:head.provider_revision,operation_id:'synthetic-source-observation'," + "events:[],receipts:[],next_projection:head.head});if(result.status!=='applied')throw new Error(JSON.stringify(result));}" 'process.stdout.write(JSON.stringify(await store.loadAuthority()));') result = subprocess.run(['node','--no-warnings','--experimental-strip-types','--input-type=module','-e',script], - input=json.dumps({'root':str(root),'goal':cli.GOAL_ID}),capture_output=True,text=True,timeout=45) + input=json.dumps({'root':str(root),'goal':cli.GOAL_ID,'advance':advance}),capture_output=True,text=True,timeout=45) assert result.returncode == 0, result.stderr return json.loads(result.stdout) @@ -438,9 +444,32 @@ def run(*args): 'LOOPX_PROJECT':str(project),'LOOPX_RUNTIME_ROOT':str(runtime),'LOOPX_REGISTRY':str(registry), 'LOOPX_GOAL_ID':cli.GOAL_ID,'LOOPX_AGENT_ID':cli.AGENT_ID} receipt = {'ok':True,'stage':'execute','mode':'heartbeat','turn_id':cli.TURN_ID} + if drift: + def changed_context(method, params, **kwargs): + if method == "runtime.checkpoint_artifacts" and params.get("action") == mutation_stage: + if drift == "canonical_head": + # Receipt-only provider commits also move the consistency + # cursor; no new knowledge is needed to invalidate the point. + authority_head(runtime, advance=True) + else: + (wakes/cli.TURN_ID/'checkpoint-research-context.json').write_text('{"later":true}') + return effect_runtime_result(method, params, **kwargs) + monkeypatch.setattr('benchmark.runtime.checkpoint.effect_runtime_result', changed_context) capture_after_wake(env, receipt) + if drift: + assert not receipt['checkpoint']['captured'] and not target.exists(), receipt + assert ('context' if mutation_stage == 'probe' else 'source') in receipt['checkpoint']['error'] + assert receipt['ok'] is True + return assert receipt['checkpoint'].get('captured') is True, json.dumps(receipt['checkpoint']) manifest = json.loads((target/'checkpoint.json').read_text()) + declared_context = manifest['point_metadata']['research_context'] + context_root = target/'components'/declared_context['component_id'] + exported_context = context_root/declared_context['member_path'] if declared_context['member_path'] else context_root + assert hashlib.sha256(exported_context.read_bytes()).hexdigest() == declared_context['sha256'] + context = json.loads(exported_context.read_bytes()) + assert context['snapshot']['provider_revision'] == before['authority_read']['provider_revision'] + assert context['goal_id'] == cli.GOAL_ID and 'leases' not in context['snapshot'] coverage = manifest['point_metadata']['loopx_state']['source_inventory'] covered = {entry['source_path']:entry['component_id'] for entry in coverage} assert str(runtime) in covered and str(registry) in covered and str(state) in covered diff --git a/tests/control_plane/test_research_context_adoption.py b/tests/control_plane/test_research_context_adoption.py new file mode 100644 index 0000000000..d6feb36875 --- /dev/null +++ b/tests/control_plane/test_research_context_adoption.py @@ -0,0 +1,124 @@ +"""Actual bootstrap/SQLite, artifact, session and leased-continuation owners. + +This stage adopts Todo context, not a native/whole research-state continuation. +No model call or historical lease/effect is activated by these fixtures. +""" +import hashlib +import json +import time +from pathlib import Path + +import pytest + +from loopx.cli import main +from loopx.control_plane.effect_runtime import effect_runtime_result +from benchmark.runtime.research_checkpoint import canonical_context, adopt_todo_context, _encoded +from canonical_authority_fixture import isolate_sqlite_runtime +from loopx.control_plane.turn_driver.codex_sessions import _store_codex_cli_session + + +def test_retained_context_is_adopted_only_with_fresh_current_ownership(tmp_path, monkeypatch, capsys): + isolate_sqlite_runtime(tmp_path, monkeypatch) + workspace = tmp_path / 'branch' + workspace.mkdir() + def setup(name, agent): + registry, root = tmp_path/name/'registry.json', tmp_path/name/'runtime' + prefix = ['--format','json','--registry',str(registry),'--runtime-root',str(root)] + def run(*args): + code = main([*prefix,*args]) + output = capsys.readouterr().out + assert code == 0, output + return json.loads(output) + run('bootstrap','--project',str(workspace),'--goal-id',name,'--objective','Investigate synthetic candidates.', + '--adapter-kind','read_only_project_map_v0','--adapter-status','connected-read-only','--write-scope','**','--no-global-sync') + run('configure-goal','--goal-id',name,'--registered-agent',agent,'--agent-work-mode',agent+'=active','--execute') + todo = run('todo','add','--goal-id',name,'--role','agent','--claimed-by',agent, + '--task-class','advancement_task','--required-capability','shell','--text','Investigate the retained synthetic candidates.')['todo_id'] + return registry,root,todo,run + source_registry,source_root,source_todo,source_run = setup('source','source-actor') + destination_registry,destination_root,destination_todo,run = setup('branch','branch-actor') + source_env={'LOOPX_RUNTIME_ROOT':str(source_root),'LOOPX_GOAL_ID':'source'} + source_revision=source_run('todo','list','--goal-id','source','--todo-id',source_todo)['authority_read']['provider_revision'] + source_run('task-lease','acquire','--goal-id','source','--todo-id',source_todo,'--owner','source-actor','--idempotency-key','source-lease') + # The evidence includes a rejected approach, with its original status/owner. + source_current=source_run('todo','list','--goal-id','source','--todo-id',source_todo)['authority_read']['provider_revision'] + source_run('todo','update','--goal-id','source','--todo-id',source_todo,'--agent-id','source-actor', + '--note','Candidate 3 failed the synthetic check; preserve candidate 7.', + '--update-operation-id','source-evidence','--update-expected-provider-revision',source_current, + '--task-lease-idempotency-key','source-lease','--task-lease-expected-version','1') + source_context=canonical_context(source_env,deadline=time.time()+30) + assert source_context['snapshot']['provider_revision'] != source_revision + assert 'leases' not in source_context['snapshot'] + raw=_encoded(source_context) + export=tmp_path/'context.json' + export.write_bytes(raw) + unit=tmp_path/'point' + components=[{'id':'research_context','source_path':str(export)}] + probe=effect_runtime_result('runtime.checkpoint_artifacts',{'action':'probe','components':components}) + captured=effect_runtime_result('runtime.checkpoint_artifacts',{'action':'capture','destination':str(unit), + 'components':components,'expected_source_sha256':probe['source_sha256'], + 'point_metadata':{'native_session':{'session_id':'source-session'}, + 'loopx_state':{'goal_id':'source','agent_id':'source-actor'}, + 'research_context':{'component_id':'research_context','member_path':'','sha256':hashlib.sha256(raw).hexdigest(), + 'provider_revision':source_context['snapshot']['provider_revision'],'store_identity':source_context['snapshot']['store_identity']}}}) + digest=captured['sha256'] + env={'LOOPX_REGISTRY':str(destination_registry),'LOOPX_RUNTIME_ROOT':str(destination_root), + 'LOOPX_PROJECT':str(workspace),'LOOPX_GOAL_ID':'branch','LOOPX_AGENT_ID':'branch-actor'} + _store_codex_cli_session(destination_root,lineage={'goal_id':'branch','agent_id':'branch-actor'}, + session_scope='agent',session_id='branch-session',session_profile_digest='synthetic-profile') + # This owner-focused fixture substitutes only native history verification. + # The real Host + CLI + canonical context journey lives in test_native_checkpoint. + monkeypatch.setattr('benchmark.runtime.research_checkpoint._context_session', lambda *_: 'branch-session') + before=run('todo','list','--goal-id','branch','--todo-id',destination_todo) + args=dict(source_todo_id=source_todo,todo_id=destination_todo,operation_id='context-adoption', + expected_revision=before['authority_read']['provider_revision'],lease_key='branch-lease',lease_version=1) + with pytest.raises(ValueError,match='execution authority changed'): + adopt_todo_context(env,unit,digest,**args) + assert not (workspace/'.local/checkpoint-context').exists() + lease=run('task-lease','acquire','--goal-id','branch','--todo-id',destination_todo, + '--owner','branch-actor','--idempotency-key','branch-lease') + assert lease['lease']['owner']=='branch-actor' and lease['lease']['version']==1 + current=run('todo','list','--goal-id','branch','--todo-id',destination_todo) + args['expected_revision']=current['authority_read']['provider_revision'] + with pytest.raises(ValueError,match='absent'): + adopt_todo_context(env,unit,digest,**{**args,'source_todo_id':'missing'}) + with pytest.raises(ValueError,match='execution authority changed'): + adopt_todo_context(env,unit,digest,**{**args,'expected_revision':'stale-revision'}) + outside=tmp_path/'outside' + outside.mkdir() + (workspace/'.local').symlink_to(outside,target_is_directory=True) + with pytest.raises(ValueError,match='symlink'): + adopt_todo_context(env,unit,digest,**args) + assert list(outside.iterdir())==[] + (workspace/'.local').unlink() + source_run('todo','update','--goal-id','source','--todo-id',source_todo,'--agent-id','source-actor', + '--note','Candidate 999 is later evidence excluded from the captured point.', + '--update-operation-id','later-source-evidence','--update-expected-provider-revision',source_context['snapshot']['provider_revision'], + '--task-lease-idempotency-key','source-lease','--task-lease-expected-version','1') + source_before=source_run('todo','list','--goal-id','source','--todo-id',source_todo) + result=adopt_todo_context(env,unit,digest,**args) + assert result['context_adopted'] and result['model_calls']==0 and not result['research_point_qualified'] + assert Path(result['artifact']).read_bytes()==raw + after=run('todo','list','--goal-id','branch','--todo-id',destination_todo) + assert after['todo']['claimed_by']=='branch-actor' and after['todo']['status']=='open' + for key,value in current['todo'].items(): + if key not in {'note','updated_at','last_actor_agent_id'}: + assert after['todo'].get(key)==value + note=json.loads(after['todo']['note']) + assert note['source_session']=='source-session' and 'checkpoint:'+digest in note['source_refs'] + inspected=run('handoff','inspect','--goal-id','branch','--todo-id',destination_todo,'--agent-id','branch-actor', + '--session-id','branch-session','--workspace',str(workspace),'--artifact',str(Path(result['artifact']).relative_to(workspace)), + '--task-lease-idempotency-key','branch-lease','--task-lease-expected-version','1') + assert inspected['can_adopt'] and inspected['availability']['ready'] + assert 'source-revision:'+source_context['snapshot']['provider_revision'] in inspected['evidence_refs'] + assert 'Candidate 3 failed' in Path(result['artifact']).read_text() + assert 'Candidate 999' not in Path(result['artifact']).read_text() + assert source_run('todo','list','--goal-id','source','--todo-id',source_todo)==source_before + assert run('task-lease','inspect','--goal-id','branch','--todo-id',destination_todo)['lease']==lease['lease'] + # The adapter does not mint another receipt/session when retried after commit. + with pytest.raises(ValueError,match='execution authority changed'): + adopt_todo_context(env,unit,digest,**args) + with pytest.raises(ValueError,match='already has context'): + adopt_todo_context(env,unit,digest,**{**args,'operation_id':'different-operation', + 'expected_revision':after['authority_read']['provider_revision']}) + assert run('todo','list','--goal-id','branch','--todo-id',destination_todo)['todo']==after['todo'] From 1ab1d2b7f823433618b32315c88c2da5248400c5 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 12:09:45 +0800 Subject: [PATCH 32/69] docs(checkpoint): explain governed context adoption and remaining qualification Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/README.md | 50 +++++++++++++++++++ .../hierarchical-agent-stride-control-v0.md | 20 ++++++-- 2 files changed, 67 insertions(+), 3 deletions(-) diff --git a/benchmark/README.md b/benchmark/README.md index 85db384262..9b35383ab2 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -321,6 +321,56 @@ inventory, symlink routes and narrower overlapping components reject capture. File and SQLite authority providers remain inert artifacts after materialization; copying their old leases or settlement records grants no new execution authority. +With this option, capture also retains the **complete paged canonical Todo read** +at one provider revision when its source coordination is promoted, as a private +artifact. It rechecks the canonical head and exported bytes before sealing; +missing promoted authority or a moved revision fails this optional capture while +ordinary work continues. An unpromoted source keeps the previous inert inventory +capture and explicitly records canonical context as unavailable; it cannot use +this context-adoption path. Leases are excluded from the read, while historical +claim/status fields remain evidence. Declarations without `loopx_state` keep +their previous path. + +After separately admitting the checkpoint's native session, an operator can +adopt this retained context on an **already claimed and leased destination Todo** +whose note is empty: + +```sh +python -m benchmark.runtime.research_checkpoint \ + --root /private/checkpoints/point --expected-sha256 CHECKPOINT_SHA256 \ + --source-todo-id SOURCE_TODO --todo-id DESTINATION_TODO \ + --operation-id CONTEXT_OPERATION --expected-revision CURRENT_REVISION \ + --lease-key CURRENT_LEASE_KEY --lease-version CURRENT_LEASE_VERSION +``` + +Supply the destination `LOOPX_PROJECT`, `LOOPX_REGISTRY`, `LOOPX_RUNTIME_ROOT`, +`LOOPX_GOAL_ID`, `LOOPX_AGENT_ID`, `CODEX_BIN`, `CODEX_HOME`, `MODEL_NAME`, +`REASONING_EFFORT` and `LOOPX_CODEX_SANDBOX` explicitly. The Goal and Agent must +differ from the captured source. The session owner must admit the distinct +current native binding and matching execution profile. Before prepare and adopt, +readback must match the captured complete native history, Host and configuration; +a branch that has already continued cannot adopt context from this older point. The adapter uses the existing continuation prepare/adopt owners, +then reads back the canonical note and Markdown projection. It retains the full +source read in a private workspace-relative file, with its checkpoint and source +revision references; current requirements, status, claim and lease stay governed +by their existing owners. The operation starts no model and submits no candidate. +The option is disabled by omitting the command; an adopted note is ordinary +current Todo context and can be changed through its owner. + +This is **Todo context adoption**, not canonical graph/vision/effect reactivation +or a complete research continuation. A deterministic local-provider qualification +uses the real native Host fork and resume, CLI and canonical continuation owner: +a tool reads the retained candidate and captured negative Todo evidence, excludes +later source evidence, then settles current governed work. This proves mechanical +adoption and use, not model judgment or research benefit. It does not replace +current Todo requirements with source requirements, reactivate source claims, or +certify Host budget and evaluator participants as part of the same point. If prepare/adopt commits but +its response/projection is uncertain, inspect the existing `handoff` and canonical +Todo owner using the original operation identities; do not create a new session, +claim or context operation to conceal that uncertainty. Automatic original-operation +recovery and the packaged frontend journey remain qualification work. + + Declare authorized workspace code/research files, effective configuration and installed source/profile pins as non-overlapping components. The native and opportunity participants do not imply that omitted state was saved. Every source diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md index 445ab442fb..d5296a5cbc 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md @@ -752,9 +752,23 @@ plus the opaque native-selected incumbent archive bound to its released result. The incumbent can precede the working candidate; native ranking remains the owner. Missing bytes, changed capture bindings or mismatched file witnesses reject before publication, without grading or refilling admission; pending or unreconciled grades reject. -Participant/effect reconciliation, fresh research-state adoption, joint evaluator/Host -admission and useful continued work remain separate acceptance work; operator -metadata and native history equality cannot certify them. +The opt-in LoopX-state participant additionally seals a complete paged canonical +Todo read at one promoted provider revision, with byte/revision rechecks. +Unpromoted coordination retains its previous inert inventory only, explicitly +without canonical context adoption. A controller can +attach this retained context to an already owned destination Todo through the +existing leased continuation prepare/adopt owner and canonical projection +readback. The source population, notes and original negative evidence are +available as private context; current requirements, status, claims and leases +remain current authority. Native profile and complete-history readback precede +both preparation and adoption. A real native fork/resume plus CLI and canonical +owner fixture consumes the captured candidate and negative Todo evidence, excludes +later source evidence, and settles governed work. Its local deterministic provider +qualifies transport/adoption and mechanical use, not research judgment or benefit; +canonical graph/vision/effect activation, full native/Host/evaluator consistency, +original-operation recovery, packaged delivery and useful continued research +remain open. Participant/effect reconciliation and complete fresh research-state +adoption cannot be certified by operator metadata or native history equality. A native fresh registration is not restored admission. In the controlled native Docker qualification, one of twelve source submissions was consumed and its From 4e9d5f8e497afb9e7a1b21c283f4cd9e27140bfc Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 12:31:06 +0800 Subject: [PATCH 33/69] fix(benchmark): recover original checkpoint context adoption Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/runtime/research_checkpoint.py | 25 ++++++---- benchmark/tests/test_native_checkpoint.py | 14 ++++-- .../test_research_context_adoption.py | 48 ++++++++++++++++++- 3 files changed, 74 insertions(+), 13 deletions(-) diff --git a/benchmark/runtime/research_checkpoint.py b/benchmark/runtime/research_checkpoint.py index 14a66f340f..18d27da475 100644 --- a/benchmark/runtime/research_checkpoint.py +++ b/benchmark/runtime/research_checkpoint.py @@ -77,7 +77,9 @@ def adopt_todo_context(env: dict[str, str], unit: Path, digest: str, *, source_t same owner; this adapter never guesses a new identity or starts model work. """ from loopx.control_plane.turn_driver.codex_sessions import select_codex_cli_session - from loopx.control_plane.todos.provider_projection import settle_canonical_todo_projection + from loopx.control_plane.todos.provider_projection import ( + settle_canonical_todo_projection, projection_delivery_requires_ack, + ) effect_runtime_result("runtime.checkpoint_artifacts", { "action": "verify", "root": str(unit), "expected_sha256": digest}) @@ -123,16 +125,20 @@ def adopt_todo_context(env: dict[str, str], unit: Path, digest: str, *, source_t # acceptance checks. Source records only contribute context; they are never # passed as operational fields to prepare or adopt. inspect = _handoff(env, {**common, "action": "inspect"}) - if (inspect["provider_revision"] != expected_revision or - inspect["claimed_by"] != env["LOOPX_AGENT_ID"] or + if (inspect["claimed_by"] != env["LOOPX_AGENT_ID"] or inspect.get("execution_authority", {}).get("allowed") is not True): raise ValueError("destination context revision, owner or execution authority changed") destination = canonical_context(env, deadline=time.time() + 30)["snapshot"] current = next((row for row in destination["todos"] if row["todo_id"] == todo_id), None) - if destination["provider_revision"] != expected_revision or current is None: + if destination["provider_revision"] != inspect["provider_revision"] or current is None: raise ValueError("destination canonical context moved before preparation") - if current.get("note"): - raise ValueError("destination Todo already has context; inspect its original owner instead of replacing it") + if current.get("note") and destination["provider_revision"] == expected_revision: + raise ValueError("destination Todo already has context; replay its original operation and revision") + if not current.get("note") and destination["provider_revision"] != expected_revision: + raise ValueError("destination context revision, owner or execution authority changed") + # With an existing note, the caller's stale ORIGINAL revision can only + # replay prepare through its typed receipt owner; it cannot overwrite the + # current note. Changed operation/context/proof must still fail there. # Keep the complete source read in a workspace-relative private artifact. # Never overwrite a moved artifact on an operation retry. relative = Path(".local/checkpoint-context") / (digest + ".json") @@ -169,15 +175,18 @@ def adopt_todo_context(env: dict[str, str], unit: Path, digest: str, *, source_t raise ValueError("destination session changed; inspect the original prepared context") if _context_session(env, metadata) != session_id: raise ValueError("destination native branch changed; inspect the original prepared context") + # Preserve the original prepare commit revision. The outer readback may + # already include adoption if its response was lost; using that newer head + # would change the adopt request hash and defeat original-receipt recovery. adopted = _handoff(env, {**common, "action": "adopt", "artifacts": [relative.as_posix()], - "operation_id": operation_id + "-adopt", "expected_provider_revision": prepared["provider_revision"]}) + "operation_id": operation_id + "-adopt", "expected_provider_revision": prepared["result"]["provider_revision"]}) effect_runtime_result("runtime.checkpoint_artifacts", { "action": "verify", "root": str(unit), "expected_sha256": digest}) if target.read_bytes() != raw: raise ValueError("adopted context artifact changed; inspect the original continuation receipt") result = settle_canonical_todo_projection(adopted, registry_path=Path(env["LOOPX_REGISTRY"]), runtime_root=Path(env["LOOPX_RUNTIME_ROOT"]), goal_id=env["LOOPX_GOAL_ID"]) - if result.get("ok") is not True: + if result.get("ok") is not True or not projection_delivery_requires_ack(result["projection_delivery"]): raise ValueError("continuation committed; reconcile its original projection receipt") return {"context_adopted": True, "session_id": bound["session_id"], "todo_id": todo_id, "artifact": str(target), "context_sha256": declared["sha256"], diff --git a/benchmark/tests/test_native_checkpoint.py b/benchmark/tests/test_native_checkpoint.py index 8c4657c532..753086d2e3 100644 --- a/benchmark/tests/test_native_checkpoint.py +++ b/benchmark/tests/test_native_checkpoint.py @@ -411,14 +411,22 @@ def forbidden_fork(*args, **kwargs): lease = invoke([*prefix, "task-lease", "acquire", "--goal-id", destination_goal, "--todo-id", destination_todo, "--owner", destination_agent, "--idempotency-key", "branch-context"]) current = invoke([*prefix, "todo", "list", "--goal-id", destination_goal, "--todo-id", destination_todo]) - adopted_cli = subprocess.run([sys.executable, "-m", "benchmark.runtime.research_checkpoint", + context_command = [sys.executable, "-m", "benchmark.runtime.research_checkpoint", "--root", str(unit), "--expected-sha256", digest, "--source-todo-id", source_todo, "--todo-id", destination_todo, "--operation-id", "branch-context", "--expected-revision", current["authority_read"]["provider_revision"], - "--lease-key", "branch-context", "--lease-version", str(lease["lease"]["version"])], - env=env, text=True, capture_output=True, timeout=60) + "--lease-key", "branch-context", "--lease-version", str(lease["lease"]["version"])] + adopted_cli = subprocess.run(context_command, env=env, text=True, capture_output=True, timeout=60) assert adopted_cli.returncode == 0, adopted_cli.stderr context_result = json.loads(adopted_cli.stdout) + # Retrying the exact CLI after a lost result reuses both original + # receipts; it must not fork, charge, edit the Todo or run a model. + replay_cli = subprocess.run(context_command, env=env, text=True, capture_output=True, timeout=60) + assert replay_cli.returncode == 0, replay_cli.stderr + replay_result = json.loads(replay_cli.stdout) + assert replay_result["continuation"]["adoption"]["status"] == "replayed" + assert replay_result["continuation"]["provider_revision"] == context_result["continuation"]["provider_revision"] + assert len(calls) == 1 retained_context_path = str(Path(context_result["artifact"]).relative_to(workspace)) assert Path(context_result["artifact"]).read_bytes() == context_raw and len(calls) == 1 adopted_todo = invoke([*prefix, "todo", "list", "--goal-id", destination_goal, "--todo-id", destination_todo]) diff --git a/tests/control_plane/test_research_context_adoption.py b/tests/control_plane/test_research_context_adoption.py index d6feb36875..1b86b5df48 100644 --- a/tests/control_plane/test_research_context_adoption.py +++ b/tests/control_plane/test_research_context_adoption.py @@ -17,7 +17,8 @@ from loopx.control_plane.turn_driver.codex_sessions import _store_codex_cli_session -def test_retained_context_is_adopted_only_with_fresh_current_ownership(tmp_path, monkeypatch, capsys): +@pytest.mark.parametrize("lost_response", [None, "prepare", "adopt", "projection"]) +def test_retained_context_is_adopted_only_with_fresh_current_ownership(tmp_path, monkeypatch, capsys, lost_response): isolate_sqlite_runtime(tmp_path, monkeypatch) workspace = tmp_path / 'branch' workspace.mkdir() @@ -96,6 +97,31 @@ def run(*args): '--update-operation-id','later-source-evidence','--update-expected-provider-revision',source_context['snapshot']['provider_revision'], '--task-lease-idempotency-key','source-lease','--task-lease-expected-version','1') source_before=source_run('todo','list','--goal-id','source','--todo-id',source_todo) + if lost_response: + with monkeypatch.context() as interrupted: + if lost_response == "projection": + def lost_projection(*args, **kwargs): + raise OSError("synthetic projection response lost") + interrupted.setattr("loopx.control_plane.todos.provider_projection.project_current_canonical_todos", lost_projection) + else: + from benchmark.runtime import research_checkpoint + original_handoff = research_checkpoint._handoff + def lost_handoff(env, payload): + result = original_handoff(env, payload) + if payload["action"] == lost_response: + raise OSError("synthetic committed response lost") + return result + interrupted.setattr(research_checkpoint, "_handoff", lost_handoff) + expected_error = ValueError if lost_response == "projection" else OSError + expected_message = "original projection receipt" if lost_response == "projection" else "response lost" + with pytest.raises(expected_error, match=expected_message): + adopt_todo_context(env,unit,digest,**args) + interrupted_todo=run('todo','list','--goal-id','branch','--todo-id',destination_todo) + assert json.loads(interrupted_todo['todo']['note'])['source_session']=='source-session' + # A replacement identity cannot recover the original accepted operation. + with pytest.raises(ValueError, match='continuation rejected'): + adopt_todo_context(env,unit,digest,**{**args,'operation_id':'replacement-operation'}) + assert run('todo','list','--goal-id','branch','--todo-id',destination_todo)['todo']==interrupted_todo['todo'] result=adopt_todo_context(env,unit,digest,**args) assert result['context_adopted'] and result['model_calls']==0 and not result['research_point_qualified'] assert Path(result['artifact']).read_bytes()==raw @@ -116,9 +142,27 @@ def run(*args): assert source_run('todo','list','--goal-id','source','--todo-id',source_todo)==source_before assert run('task-lease','inspect','--goal-id','branch','--todo-id',destination_todo)['lease']==lease['lease'] # The adapter does not mint another receipt/session when retried after commit. + replay=adopt_todo_context(env,unit,digest,**args) + assert replay['continuation']['adoption']['status']=='replayed' + assert replay['continuation']['provider_revision']==result['continuation']['provider_revision'] + assert run('todo','list','--goal-id','branch','--todo-id',destination_todo)==after with pytest.raises(ValueError,match='execution authority changed'): - adopt_todo_context(env,unit,digest,**args) + adopt_todo_context(env,unit,digest,**{**args,'lease_version':999}) with pytest.raises(ValueError,match='already has context'): adopt_todo_context(env,unit,digest,**{**args,'operation_id':'different-operation', 'expected_revision':after['authority_read']['provider_revision']}) assert run('todo','list','--goal-id','branch','--todo-id',destination_todo)['todo']==after['todo'] + + target=Path(result['artifact']) + target.write_bytes(b'changed retained context') + with pytest.raises(ValueError,match='artifact already differs'): + adopt_todo_context(env,unit,digest,**args) + target.write_bytes(raw) + run('todo','update','--goal-id','branch','--todo-id',destination_todo,'--agent-id','branch-actor', + '--note','New destination evidence must not be replaced by a historical retry.', + '--update-operation-id','new-destination-evidence','--update-expected-provider-revision',after['authority_read']['provider_revision'], + '--task-lease-idempotency-key','branch-lease','--task-lease-expected-version','1') + moved=run('todo','list','--goal-id','branch','--todo-id',destination_todo) + with pytest.raises(ValueError,match='continuation rejected'): + adopt_todo_context(env,unit,digest,**args) + assert run('todo','list','--goal-id','branch','--todo-id',destination_todo)==moved From 2769fe105dcfedbb16ca344bf52a228018ce9048 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 12:31:06 +0800 Subject: [PATCH 34/69] docs(benchmark): explain context retry and remaining checkpoint limits Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/README.md | 12 ++++++++---- .../rfcs/hierarchical-agent-stride-control-v0.md | 8 +++++--- ...hierarchical-agent-stride-control-v0.zh-CN.md | 16 ++++++++++++++-- 3 files changed, 27 insertions(+), 9 deletions(-) diff --git a/benchmark/README.md b/benchmark/README.md index 9b35383ab2..a0c82a4992 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -365,10 +365,14 @@ later source evidence, then settles current governed work. This proves mechanica adoption and use, not model judgment or research benefit. It does not replace current Todo requirements with source requirements, reactivate source claims, or certify Host budget and evaluator participants as part of the same point. If prepare/adopt commits but -its response/projection is uncertain, inspect the existing `handoff` and canonical -Todo owner using the original operation identities; do not create a new session, -claim or context operation to conceal that uncertainty. Automatic original-operation -recovery and the packaged frontend journey remain qualification work. +its response/projection is uncertain, repeat this command with the **same original +operation ID, expected revision, checkpoint and lease proof**. It replays the +existing prepare/adopt receipts and reconciles projection; it does not create a +new note or reset execution authority. Changed notes, artifacts, native history, +leases or request identity still refuse recovery. Do not substitute a new session, +claim, revision or context operation to conceal uncertainty. This qualifies only +context-command recovery; joint participant recovery and the packaged frontend +journey remain open. Declare authorized workspace code/research files, effective configuration and diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md index d5296a5cbc..0f9e2e597a 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md @@ -765,9 +765,11 @@ both preparation and adoption. A real native fork/resume plus CLI and canonical owner fixture consumes the captured candidate and negative Todo evidence, excludes later source evidence, and settles governed work. Its local deterministic provider qualifies transport/adoption and mechanical use, not research judgment or benefit; -canonical graph/vision/effect activation, full native/Host/evaluator consistency, -original-operation recovery, packaged delivery and useful continued research -remain open. Participant/effect reconciliation and complete fresh research-state +context-command retries now reuse the original prepare/adopt identities and +revision after lost responses or projection failure, while changed authority, +notes, artifacts and advanced native history still refuse recovery. Canonical +graph/vision/effect activation, full native/Host/evaluator consistency, joint +participant recovery, packaged delivery and useful continued research remain open. Participant/effect reconciliation and complete fresh research-state adoption cannot be certified by operator metadata or native history equality. A native fresh registration is not restored admission. In the controlled native diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md index a3ba91ecbc..6562f00f9a 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md @@ -626,8 +626,20 @@ effect 采用、剩余总预算、已安装前端恢复或科学收益。显式 原始 archive。它可能早于工作区候选；排序仍由原生规则决定。缺失源码、捕获 绑定变化或文件 witness 不符均在发布前拒绝；不评测、不补发机会，未完成或 未对账评测会拒绝。 -参与者/效果 reconciliation、新研究状态采用、目标侧评测与 Host 联合准入及实际有效 -接续仍各自待验收；operator metadata 或历史一致不能认证这些条件。 +可选 LoopX-state 参与者还封存 promoted provider 同一 revision 的完整分页 +canonical Todo 读取，并复查字节和 revision。未 promoted 的来源仍只保存原有 +惰性 inventory，明确不支持 canonical context 采用。控制器通过既有 leased +continuation prepare/adopt owner，将保留上下文挂到已被目标 Agent 持有的 Todo， +读回 canonical note 与 Markdown 投影。原始 Todo 集合、note 和负证据仅作为私有 +上下文；目标 requirements、status、claim 和 lease 仍由当前 authority 决定。 +prepare/adopt 前均复查 native profile 与完整历史。真实 native fork/resume、CLI +和 canonical owner 的合成资格测试实际读取保留候选与负证据，排除来源后续证据， +并完成当前 governed 工作结算。本地确定性 provider 只证明传输、采用及机械使用， +不证明研究判断或收益。context 命令在响应丢失或投影失败后，复用原 prepare/adopt +身份与 revision 恢复；authority、note、产物变化或 native 历史已推进仍拒绝。 +canonical graph/vision/effect 激活、完整 native/Host/evaluator 一致性、参与者联合 +恢复、packaged 交付和实际有效研究仍待验收。operator metadata 或历史一致不能 +认证参与者/effect 对账与完整新研究状态采用。 原生新注册不等于恢复准入。真实原生 Docker 资格测试中，source 已消耗十二次 中的一次，且冷却仍有效；同额度的新注册却立即受理 `agent-1`，保留十二次未用 From 67b77a4ff4a38f9586429dfe6add4fc6b0666a77 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 12:46:14 +0800 Subject: [PATCH 35/69] feat(benchmark): retain source-bound research evidence in checkpoint context Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/runtime/checkpoint.py | 7 +- benchmark/runtime/research_checkpoint.py | 91 ++++++++++++++++++- benchmark/tests/test_native_checkpoint.py | 29 +++++- .../test_research_checkpoint_capture.py | 22 ++++- .../test_research_context_adoption.py | 77 +++++++++++++++- 5 files changed, 218 insertions(+), 8 deletions(-) diff --git a/benchmark/runtime/checkpoint.py b/benchmark/runtime/checkpoint.py index cef51f61de..7932523dab 100644 --- a/benchmark/runtime/checkpoint.py +++ b/benchmark/runtime/checkpoint.py @@ -340,8 +340,8 @@ def capture_after_wake(env: dict[str, str], receipt: dict[str, Any], *, from loopx.control_plane.coordination.local_authority import local_authority_is_promoted research = None if local_authority_is_promoted(runtime_root=Path(env["LOOPX_RUNTIME_ROOT"]), goal_id=env["LOOPX_GOAL_ID"]): - from .research_checkpoint import canonical_context, _encoded - research = canonical_context(env, deadline=deadline) + from .research_checkpoint import research_context, verify_context_sources, _encoded + research = research_context(env, deadline=deadline) evidence = Path(env["LOOPX_WAKE_LOG_DIR"]) / receipt["turn_id"] / "checkpoint-research-context.json" research_raw = _encoded(research) with os.fdopen(os.open(evidence, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600), "wb") as file: @@ -410,8 +410,9 @@ def capture_after_wake(env: dict[str, str], receipt: dict[str, Any], *, raise ValueError("native checkpoint source session binding moved before sealing") if request.get("loopx_state"): if research is not None: - if canonical_context(env, deadline=deadline) != research: + if research_context(env, deadline=deadline) != research: raise ValueError("canonical research context moved before sealing") + verify_context_sources(research, components, probe["source_witness"]) retained = metadata["research_context"] member = next((entry for entry in probe["source_witness"][retained["component_id"]] if entry["path"] == retained["member_path"]), None) diff --git a/benchmark/runtime/research_checkpoint.py b/benchmark/runtime/research_checkpoint.py index 18d27da475..730efe3cb9 100644 --- a/benchmark/runtime/research_checkpoint.py +++ b/benchmark/runtime/research_checkpoint.py @@ -32,6 +32,91 @@ def rpc(method, payload, **kwargs): return {"goal_id": env["LOOPX_GOAL_ID"], "snapshot": result} +def research_context(env: dict[str, str], *, deadline: float) -> dict: + """Retain decision evidence as history, without importing operational state. + + The existing Explore codec owns event validity and topology; run-index + records own vision/stride history. This transport preserves their values + instead of inferring a new frontier or reviving a historical decision. + """ + from loopx.capabilities.explore.result_log import ( + explore_result_log_path, validate_explore_result_event, + ) + from loopx.history import load_index_snapshot + from loopx.runtime import validate_goal_id_path_segment + + root = Path(env["LOOPX_RUNTIME_ROOT"]) + goal = validate_goal_id_path_segment(env["LOOPX_GOAL_ID"]) + paths = [explore_result_log_path(root, goal), root / "goals" / goal / "runs/index.jsonl"] + before = {} + for path in paths: + if not path.is_absolute() or path.resolve() != path: + raise ValueError("research evidence source must be a physical absolute path") + raw = path.read_bytes() if path.exists() else None + # An incomplete append is not a complete research boundary. The source + # owners validate event fields; the transport checks complete framing. + if raw and not raw.endswith(b"\n"): + raise ValueError("research evidence source has an incomplete tail") + before[path] = raw + events = [] + # Decode exactly the witnessed bytes through the existing event validator; + # rereading the file for decoding would leave an avoidable ABA window. + for line in (before[paths[0]] or b"").splitlines(): + if line.strip(): + try: + row = json.loads(line) + if not isinstance(row, dict): + raise ValueError("event must be an object") + events.append(validate_explore_result_event(row, expected_goal_id=goal)) + except ValueError as error: + raise ValueError("invalid Explore result in research context") from error + raw_index = before[paths[1]] + if raw_index: + for line in raw_index.splitlines(): + if line.strip(): + row = json.loads(line) + if not isinstance(row, dict) or row.get("goal_id", goal) != goal: + raise ValueError("research history contains a malformed or foreign record") + index = load_index_snapshot(paths[1], include_artifact_status=False) + if index.digest != ("sha256:" + hashlib.sha256(raw_index).hexdigest() if raw_index is not None else None): + raise ValueError("research history source moved during decoding") + runs = index.records + # Preserve every indexed vision, including retired decisions and their + # explicit provenance; a latest-only display could erase negative evidence. + visions = [{key: row[key] for key in ("generated_at", "goal_id", "agent_id", "goal_ref", + "turn_instance_id", "agent_vision", "vision_checkpoint") if key in row} + for row in runs if row.get("agent_vision") or row.get("vision_checkpoint")] + result = canonical_context(env, deadline=deadline) + for path, raw in before.items(): + if (path.read_bytes() if path.exists() else None) != raw: + raise ValueError("research evidence source moved during the context read") + return {**result, "research_evidence": {"explore_events": events, "vision_history": visions}, + "source_files": [{"path": str(path), "sha256": hashlib.sha256(raw).hexdigest() if raw is not None else None} + for path, raw in before.items()]} + + +def verify_context_sources(context: dict, components: list[dict], witness: dict) -> None: + """Bind exported history to the artifact owner's actual source witness. + + A second read alone ends too early: a source append immediately before the + probe must not be captured next to an older exported context. Absence is + witnessed by the containing component too, so a first append also rejects. + """ + for source in context.get("source_files", []): + path = Path(source["path"]) + parent = next((entry for entry in components if path.is_relative_to(Path(entry["source_path"]))), None) + if parent is None: + raise ValueError("research evidence source is outside the captured inventory") + relative = path.relative_to(Path(parent["source_path"])) + member = "" if str(relative) == "." else relative.as_posix() + observed = next((item for item in witness[parent["id"]] if item["path"] == member), None) + if source["sha256"] is None: + if observed is not None: + raise ValueError("research evidence source appeared before sealing") + elif observed is None or observed.get("kind") != "file" or observed.get("sha256") != source["sha256"]: + raise ValueError("research evidence source differs from the captured context") + + def _encoded(value): return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode() @@ -162,8 +247,12 @@ def adopt_todo_context(env: dict[str, str], unit: Path, digest: str, *, source_t raise ValueError("destination context artifact readback differs") effect_runtime_result("runtime.checkpoint_artifacts", { "action": "verify", "root": str(unit), "expected_sha256": digest}) + # Keep old context-only request bytes stable for original receipt recovery + # across an adapter upgrade. New history is disclosed only when captured. + history_guidance = (" Read its research_evidence for prior branches, negative results and vision/stride decisions." + if "research_evidence" in carrier else "") note = {"work_summary": "Retained canonical Todo context from the verified checkpoint is available in " + relative.as_posix() + - ". Historical owners, statuses and operational references are evidence only; current Goal/Todo/lease rules govern work.", + ". Historical owners, statuses and operational references are evidence only; current Goal/Todo/lease rules govern work." + history_guidance, "source_refs": ["checkpoint:" + digest, "source-todo:" + source_todo_id, "source-revision:" + declared["provider_revision"]], "files_touched": [{"path": relative.as_posix(), "action": "read", diff --git a/benchmark/tests/test_native_checkpoint.py b/benchmark/tests/test_native_checkpoint.py index 753086d2e3..30ab78f08d 100644 --- a/benchmark/tests/test_native_checkpoint.py +++ b/benchmark/tests/test_native_checkpoint.py @@ -245,7 +245,7 @@ def register(name, agent): assert main([*prefix, "quota", "should-run", "--runtime-profile", "generic_cli", *binding]) == 0 guard = json.loads(capsys.readouterr().out) assert guard["interaction_contract"]["agent_channel"]["delivery_allowed"] is True - from benchmark.runtime.research_checkpoint import canonical_context, adopt_todo_context, _encoded + from benchmark.runtime.research_checkpoint import canonical_context, research_context, adopt_todo_context, _encoded source_prefix = ["--format", "json", "--registry", str(source_registry), "--runtime-root", str(source_runtime)] def invoke(arguments): code = main(arguments) @@ -286,7 +286,24 @@ def response(request, index, message): native = native_session_checkpoint(env, source_session, timeout=20) config = tmp_path / "config.toml" config.write_bytes((Path(env["CODEX_HOME"]) / "config.toml").read_bytes()) - source_context = canonical_context(env, deadline=time.time() + 30) + from loopx.capabilities.explore.result_log import ( + append_explore_result_event, build_explore_node_event, explore_result_log_path, + ) + graph_path = explore_result_log_path(source_runtime, source_goal) + append_explore_result_event(graph_path, build_explore_node_event( + goal_id=source_goal, node_id="rejected-route", title="Rejected route", status="dead_end", + summary="Route rejected because fresh samples reversed its pilot gain.")) + source_vision = tmp_path / "source-vision.json" + source_vision.write_text(json.dumps({"schema_version": "goal_vision_replan_contract_v0", + "state": "vision_patch_proposed", "vision_patch": {"acceptance_summary": + "Complete a fresh-sample experiment before changing the retained route."}})) + invoke([*source_prefix, "quota", "should-run", *source_binding, "--agent-id", source_agent, + "--turn-instance-id", "source-research-evidence", "--runtime-profile", "generic_cli"]) + invoke([*source_prefix, "refresh-state", *source_binding, "--agent-id", source_agent, + "--turn-instance-id", "source-research-evidence", "--classification", "synthetic_research_evidence", + "--delivery-batch-scale", "multi_surface", "--delivery-outcome", "outcome_progress", + "--agent-vision-json", str(source_vision)]) + source_context = research_context(env, deadline=time.time() + 30) context_raw = _encoded(source_context) context_export = tmp_path / "source-context.json" context_export.write_bytes(context_raw) @@ -310,6 +327,9 @@ def response(request, index, message): "--update-operation-id", "later-negative-evidence", "--update-expected-provider-revision", source_context["snapshot"]["provider_revision"], "--task-lease-idempotency-key", "source-context", "--task-lease-expected-version", "1"]) + append_explore_result_event(graph_path, build_explore_node_event( + goal_id=source_goal, node_id="future-route", title="Future route outside this checkpoint", + status="exploring")) later_context = canonical_context(env, deadline=time.time() + 30) # Qualify use of captured inputs, not the source's later working files. # A quiesced same-machine fixture swaps fixed paths; this is not evidence @@ -452,7 +472,12 @@ def forbidden_fork(*args, **kwargs): assert "retained candidate: 7" in json.dumps(requests[2]) assert "candidate 3 was rejected" in json.dumps(requests[2]) assert "Candidate 3 failed the synthetic check" in json.dumps(requests[2]) + assert "fresh samples reversed its pilot gain" in json.dumps(requests[2]) + assert "Complete a fresh-sample experiment" in json.dumps(requests[2]) + assert source_context["research_evidence"]["explore_events"][0]["status"] == "dead_end" + assert not explore_result_log_path(runtime, destination_goal).exists() assert "Candidate 999" not in json.dumps(requests[2]) + assert "Future route outside this checkpoint" not in json.dumps(requests[2]) assert canonical_context({**env, "LOOPX_GOAL_ID": source_goal, "LOOPX_RUNTIME_ROOT": str(source_runtime)}, deadline=time.time() + 30) == later_context assert "Retained candidate 7 verified" in result.stdout diff --git a/tests/control_plane/test_research_checkpoint_capture.py b/tests/control_plane/test_research_checkpoint_capture.py index 157498eaaf..ba1b8a8a04 100644 --- a/tests/control_plane/test_research_checkpoint_capture.py +++ b/tests/control_plane/test_research_checkpoint_capture.py @@ -377,7 +377,9 @@ def fail_probe(method, params, **kwargs): @pytest.mark.parametrize('provider', ['file', 'sqlite']) @pytest.mark.parametrize(("drift", "mutation_stage"), [(None, "probe"), ("canonical_head", "probe"), ("exported_bytes", "probe"), - ("canonical_head", "capture"), ("exported_bytes", "capture")]) + ("canonical_head", "capture"), ("exported_bytes", "capture"), + ("research_log", "probe"), ("research_log", "capture"), + ("vision_index", "probe"), ("vision_index", "capture")]) def test_controller_participant_keeps_canonical_state_and_journals_inert(tmp_path, monkeypatch, provider, drift, mutation_stage): from canonical_authority_fixture import initialize_canonical_authority from loopx.control_plane.coordination.runtime_shadow import build_todo_runtime_shadow_projection @@ -427,6 +429,14 @@ def run(*args): '--delivery-batch-scale','multi_surface','--delivery-outcome','outcome_progress','--agent-vision-json',str(vision)) run('quota','spend-slot','--goal-id',cli.GOAL_ID,*binding,'--source','heartbeat','--execute') before = run('todo','list','--goal-id',cli.GOAL_ID,'--todo-id',cli.TODO_ID) + from loopx.capabilities.explore.result_log import ( + explore_result_log_path, append_explore_result_event, build_explore_node_event, + ) + graph_path = explore_result_log_path(runtime, cli.GOAL_ID) + def append_source_evidence(title): + append_explore_result_event(graph_path, build_explore_node_event(goal_id=cli.GOAL_ID, + title=title, status="dead_end", summary="Fresh validation rejected this route.")) + append_source_evidence("Original rejected route") original_head = authority_head(runtime) assert original_head['status'] == 'loaded' source_settlement = read_heartbeat_settlement(runtime, goal_id=cli.GOAL_ID,agent_id=cli.AGENT_ID, @@ -451,6 +461,13 @@ def changed_context(method, params, **kwargs): # Receipt-only provider commits also move the consistency # cursor; no new knowledge is needed to invalidate the point. authority_head(runtime, advance=True) + elif drift == "research_log": + append_source_evidence("Later evidence outside the point") + elif drift == "vision_index": + with (runtime / "goals" / cli.GOAL_ID / "runs/index.jsonl").open("a") as stream: + stream.write(json.dumps({"goal_id": cli.GOAL_ID, "agent_id": cli.AGENT_ID, + "generated_at": "2099-01-01T00:00:00Z", "agent_vision": { + "vision_patch": {"acceptance_summary": "Later source vision"}}}) + "\n") else: (wakes/cli.TURN_ID/'checkpoint-research-context.json').write_text('{"later":true}') return effect_runtime_result(method, params, **kwargs) @@ -470,6 +487,9 @@ def changed_context(method, params, **kwargs): context = json.loads(exported_context.read_bytes()) assert context['snapshot']['provider_revision'] == before['authority_read']['provider_revision'] assert context['goal_id'] == cli.GOAL_ID and 'leases' not in context['snapshot'] + assert context['research_evidence']['explore_events'][0]['title'] == 'Original rejected route' + assert any('Controlled participant probe passed' in json.dumps(row) + for row in context['research_evidence']['vision_history']) coverage = manifest['point_metadata']['loopx_state']['source_inventory'] covered = {entry['source_path']:entry['component_id'] for entry in coverage} assert str(runtime) in covered and str(registry) in covered and str(state) in covered diff --git a/tests/control_plane/test_research_context_adoption.py b/tests/control_plane/test_research_context_adoption.py index 1b86b5df48..da9ef62e3f 100644 --- a/tests/control_plane/test_research_context_adoption.py +++ b/tests/control_plane/test_research_context_adoption.py @@ -12,7 +12,7 @@ from loopx.cli import main from loopx.control_plane.effect_runtime import effect_runtime_result -from benchmark.runtime.research_checkpoint import canonical_context, adopt_todo_context, _encoded +from benchmark.runtime.research_checkpoint import canonical_context, research_context, verify_context_sources, adopt_todo_context, _encoded from canonical_authority_fixture import isolate_sqlite_runtime from loopx.control_plane.turn_driver.codex_sessions import _store_codex_cli_session @@ -166,3 +166,78 @@ def lost_handoff(env, payload): with pytest.raises(ValueError,match='continuation rejected'): adopt_todo_context(env,unit,digest,**args) assert run('todo','list','--goal-id','branch','--todo-id',destination_todo)==moved + + +def test_research_history_is_complete_and_bound_to_original_source(tmp_path, monkeypatch, capsys): + from loopx.capabilities.explore.result_log import ( + append_explore_result_events, build_explore_node_event, build_explore_edge_event, + build_explore_finding_event, explore_result_log_path, + ) + isolate_sqlite_runtime(tmp_path, monkeypatch) + registry, root = tmp_path / "registry.json", tmp_path / "runtime" + prefix = ["--format", "json", "--registry", str(registry), "--runtime-root", str(root)] + assert main([*prefix, "bootstrap", "--project", str(tmp_path), "--goal-id", "source", + "--objective", "Investigate synthetic candidates.", "--adapter-kind", "read_only_project_map_v0", + "--adapter-status", "connected-read-only", "--write-scope", "**", "--no-global-sync"]) == 0 + capsys.readouterr() + env = {"LOOPX_RUNTIME_ROOT": str(root), "LOOPX_GOAL_ID": "source"} + log = explore_result_log_path(root, "source") + events = [build_explore_node_event(goal_id="source", node_id="candidate-3", title="Rejected candidate", + status="dead_end", summary="Fresh samples refuted candidate 3."), + build_explore_node_event(goal_id="source", node_id="candidate-7", title="Retained candidate", + status="exploring"), + build_explore_edge_event(goal_id="source", from_node="candidate-7", to_node="candidate-3", + edge_type="refutes")] + # More than the display finding limit: a checkpoint must retain the earliest + # counterexample as well as recent observations, with original topology. + events += [build_explore_finding_event(goal_id="source", node_id="candidate-3", finding_id=f"finding-{i}", + title=f"Negative observation {i}", status="confirmed") for i in range(45)] + append_explore_result_events(log, events, expected_goal_id="source") + index = root / "goals/source/runs/index.jsonl" + # Synthetic persisted source records test transport, not vision judgment. + rows = [{"goal_id": "source", "agent_id": "source-actor", "generated_at": f"2026-01-01T00:00:{i:02d}Z", + "agent_vision": {"vision_patch": {"current_path": f"Complete experiment {i}"}, + "stride_decision": {"original": i}}} for i in range(2)] + index.parent.mkdir(parents=True, exist_ok=True) + index.write_text("".join(json.dumps(row) + "\n" for row in rows)) + captured = research_context(env, deadline=time.time() + 30) + assert captured["research_evidence"]["explore_events"] == events + assert captured["research_evidence"]["vision_history"] == rows + assert "leases" not in captured["snapshot"] + components = [{"id": "runtime", "source_path": str(root)}] + probe = effect_runtime_result("runtime.checkpoint_artifacts", {"action": "probe", "components": components}) + verify_context_sources(captured, components, probe["source_witness"]) + raw = index.read_bytes() + index.write_bytes(raw + json.dumps({**rows[-1], "generated_at": "2026-01-01T01:00:00Z"}).encode() + b"\n") + later = effect_runtime_result("runtime.checkpoint_artifacts", {"action": "probe", "components": components}) + with pytest.raises(ValueError, match="differs from the captured context"): + verify_context_sources(captured, components, later["source_witness"]) + with pytest.raises(ValueError, match="outside the captured inventory"): + verify_context_sources(captured, [], {}) + for bad in (b'{', b'not-json\n', b'[]\n', b'{"goal_id":"foreign"}\n'): + index.write_bytes(raw + bad) + with pytest.raises(ValueError): + research_context(env, deadline=time.time() + 30) + index.write_bytes(raw) + original = log.read_bytes() + log.write_bytes(original + b'not-json\n') + with pytest.raises(ValueError, match="invalid Explore result"): + research_context(env, deadline=time.time() + 30) + log.write_bytes(original) + log.unlink() + absent = research_context(env, deadline=time.time() + 30) + assert absent["research_evidence"]["explore_events"] == [] + log.write_bytes(original) + appeared = effect_runtime_result("runtime.checkpoint_artifacts", {"action": "probe", "components": components}) + with pytest.raises(ValueError, match="appeared before sealing"): + verify_context_sources(absent, components, appeared["source_witness"]) + # An append during the canonical read is also rejected, not silently mixed. + from benchmark.runtime import research_checkpoint + original_read = research_checkpoint.canonical_context + def racing_read(*args, **kwargs): + result = original_read(*args, **kwargs) + index.write_bytes(raw + b'{}\n') + return result + monkeypatch.setattr(research_checkpoint, "canonical_context", racing_read) + with pytest.raises(ValueError, match="moved during the context read"): + research_context(env, deadline=time.time() + 30) From 753008e1b35d995132332ea292c3cdd1e045f5db Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 12:46:14 +0800 Subject: [PATCH 36/69] docs(benchmark): describe historical research context and adoption limits Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/README.md | 10 ++++++++-- .../rfcs/hierarchical-agent-stride-control-v0.md | 9 +++++++-- .../rfcs/hierarchical-agent-stride-control-v0.zh-CN.md | 8 ++++++-- 3 files changed, 21 insertions(+), 6 deletions(-) diff --git a/benchmark/README.md b/benchmark/README.md index a0c82a4992..f3628d3b52 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -323,7 +323,13 @@ copying their old leases or settlement records grants no new execution authority With this option, capture also retains the **complete paged canonical Todo read** at one provider revision when its source coordination is promoted, as a private -artifact. It rechecks the canonical head and exported bytes before sealing; +artifact. That context also preserves the complete validated Explore event log +(nodes, edges and findings, including dead ends) and all indexed vision/checkpoint +records, including their original stride decisions. These are historical inputs, +not imported destination graph, vision or effect authority. Empty histories are +explicit; malformed rows and incomplete appends reject capture. The original +source-file digests/absence must match the artifact owner's source witness, so an +append between the context read and sealing cannot mix research points. It rechecks the canonical head and exported bytes before sealing; missing promoted authority or a moved revision fails this optional capture while ordinary work continues. An unpromoted source keeps the previous inert inventory capture and explicitly records canonical context as unavailable; it cannot use @@ -360,7 +366,7 @@ current Todo context and can be changed through its owner. This is **Todo context adoption**, not canonical graph/vision/effect reactivation or a complete research continuation. A deterministic local-provider qualification uses the real native Host fork and resume, CLI and canonical continuation owner: -a tool reads the retained candidate and captured negative Todo evidence, excludes +a tool reads the retained candidate, negative Todo/Explore evidence and vision, excludes later source evidence, then settles current governed work. This proves mechanical adoption and use, not model judgment or research benefit. It does not replace current Todo requirements with source requirements, reactivate source claims, or diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md index 0f9e2e597a..8dec865034 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md @@ -753,7 +753,12 @@ The incumbent can precede the working candidate; native ranking remains the owne Missing bytes, changed capture bindings or mismatched file witnesses reject before publication, without grading or refilling admission; pending or unreconciled grades reject. The opt-in LoopX-state participant additionally seals a complete paged canonical -Todo read at one promoted provider revision, with byte/revision rechecks. +Todo read at one promoted provider revision, with byte/revision rechecks. It +retains complete Explore events and indexed vision/checkpoint history, including +stride decisions, as historical context. The source owner validates Explore rows; +framing, source-file digests and absence are checked against the sealed inventory. +Malformed or moved history rejects the optional capture. These retained records +do not activate a destination graph, old vision or historical effects. Unpromoted coordination retains its previous inert inventory only, explicitly without canonical context adoption. A controller can attach this retained context to an already owned destination Todo through the @@ -762,7 +767,7 @@ readback. The source population, notes and original negative evidence are available as private context; current requirements, status, claims and leases remain current authority. Native profile and complete-history readback precede both preparation and adoption. A real native fork/resume plus CLI and canonical -owner fixture consumes the captured candidate and negative Todo evidence, excludes +owner fixture consumes the captured candidate, negative Todo/Explore evidence and vision, excludes later source evidence, and settles governed work. Its local deterministic provider qualifies transport/adoption and mechanical use, not research judgment or benefit; context-command retries now reuse the original prepare/adopt identities and diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md index 6562f00f9a..be995f3dc6 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md @@ -627,13 +627,17 @@ effect 采用、剩余总预算、已安装前端恢复或科学收益。显式 绑定变化或文件 witness 不符均在发布前拒绝；不评测、不补发机会，未完成或 未对账评测会拒绝。 可选 LoopX-state 参与者还封存 promoted provider 同一 revision 的完整分页 -canonical Todo 读取，并复查字节和 revision。未 promoted 的来源仍只保存原有 +canonical Todo 读取，并复查字节和 revision。同时保留完整 Explore 事件、索引中的 +vision/checkpoint 历史及原 stride 决策，供后续读取原分支、负证据与路线判断。 +Explore 行仍由原 owner 校验；不完整追加、坏行、源文件 digest/缺失状态与封存 +inventory 不一致均拒绝可选捕获。历史记录不激活目标 graph、旧 vision 或 effect。 +未 promoted 的来源仍只保存原有 惰性 inventory，明确不支持 canonical context 采用。控制器通过既有 leased continuation prepare/adopt owner，将保留上下文挂到已被目标 Agent 持有的 Todo， 读回 canonical note 与 Markdown 投影。原始 Todo 集合、note 和负证据仅作为私有 上下文；目标 requirements、status、claim 和 lease 仍由当前 authority 决定。 prepare/adopt 前均复查 native profile 与完整历史。真实 native fork/resume、CLI -和 canonical owner 的合成资格测试实际读取保留候选与负证据，排除来源后续证据， +和 canonical owner 的合成资格测试实际读取保留候选、Todo/Explore 负证据与 vision，排除来源后续证据， 并完成当前 governed 工作结算。本地确定性 provider 只证明传输、采用及机械使用， 不证明研究判断或收益。context 命令在响应丢失或投影失败后，复用原 prepare/adopt 身份与 revision 恢复；authority、note、产物变化或 native 历史已推进仍拒绝。 From f935d73b6f42c0d98949c7078b6f409c26027bc5 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 13:00:50 +0800 Subject: [PATCH 37/69] test(benchmark): qualify composed native checkpoint continuation Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/README.md | 13 ++ benchmark/tests/test_native_checkpoint.py | 116 ++++++++++++++++-- .../hierarchical-agent-stride-control-v0.md | 12 ++ ...rarchical-agent-stride-control-v0.zh-CN.md | 8 ++ 4 files changed, 138 insertions(+), 11 deletions(-) diff --git a/benchmark/README.md b/benchmark/README.md index f3628d3b52..146d96d9d9 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -381,6 +381,19 @@ context-command recovery; joint participant recovery and the packaged frontend journey remain open. +The composed local qualification also seals the native evaluator capsule with the +conversation/code/context artifact. An actual resumed Codex tool reads captured +research and public feedback, computes a synthetic candidate from captured code, +and submits through the native HTTP owner. It retains consumed rounds and the +active cooldown, rejects a further submission at the inherited limit, and enters +through the existing SForge Host phase check with the captured remaining duration. +Later source evidence stays excluded and the source service remains unchanged. +The fixture quiesces participants itself, uses a local deterministic model and +synthetic grading, and does not activate a destination Explore graph or old +vision/effects. It qualifies this composed mechanical path, not an automatic +cross-owner barrier, ambiguous activation recovery, isolated Linux solver restore, +or research utility. Those remain requirements before scientific continuation. + Declare authorized workspace code/research files, effective configuration and installed source/profile pins as non-overlapping components. The native and opportunity participants do not imply that omitted state was saved. Every source diff --git a/benchmark/tests/test_native_checkpoint.py b/benchmark/tests/test_native_checkpoint.py index 30ab78f08d..631d9abbe9 100644 --- a/benchmark/tests/test_native_checkpoint.py +++ b/benchmark/tests/test_native_checkpoint.py @@ -11,6 +11,7 @@ import pytest from benchmark.runtime.native_checkpoint import native_session_checkpoint +from benchmark.tests.test_checkpoint_controller import controllers # noqa: F401 -- shared real HTTP fixture def fixture(tmp_path, monkeypatch, pages): @@ -86,7 +87,7 @@ def test_foreign_native_session_is_rejected_before_history(tmp_path, monkeypatch @contextmanager -def real_native_source(root, binary, workspace, *, close_source=False, response_builder=None): +def real_native_source(root, binary, workspace, *, close_source=False, response_builder=None, sandbox="read-only"): """Real native writer/readback; a local static stream replaces only the model. No paid model, tool invocation, user credential or benchmark cell. Fixtures @@ -125,7 +126,9 @@ def do_POST(self): thread = threading.Thread(target=server.serve_forever) thread.start() (home / "config.toml").write_text( - 'model="fixture"\nmodel_reasoning_effort="high"\nmodel_provider="fixture"\napproval_policy="never"\nsandbox_mode="read-only"\n' + 'model="fixture"\nmodel_reasoning_effort="high"\nmodel_provider="fixture"\napproval_policy="never"\n' + f'sandbox_mode={json.dumps(sandbox)}\n' + + ('[sandbox_workspace_write]\nnetwork_access=true\n' if sandbox == "workspace-write" else "") + '[features]\ngoals=false\nresponses_websockets_v2=false\nplugins=false\nremote_plugin=false\nplugin_sharing=false\n' '[model_providers.fixture]\nname="Synthetic local"\n' f'base_url="http://127.0.0.1:{server.server_port}/v1"\nwire_api="responses"\n' @@ -144,7 +147,7 @@ def do_POST(self): native.request("initialize", {"clientInfo": {"name": "checkpoint_qualification", "version": "0.1.0"}, "capabilities": {"experimentalApi": True}}) native.notify("initialized", {}) - source = native.request("thread/start", {"cwd": str(workspace), "sandbox": "read-only", + source = native.request("thread/start", {"cwd": str(workspace), "sandbox": sandbox, "approvalPolicy": "never", "persistExtendedHistory": True})["thread"]["id"] def complete_turn(): expected_turn = native.request("turn/start", {"threadId": source, @@ -191,12 +194,14 @@ def test_real_native_workspace_readback_rejects_foreign_code_basis(tmp_path): @pytest.mark.skipif(not os.environ.get("LOOPX_NATIVE_CHECKPOINT_BIN"), reason="Requires an explicit real native Host binary; only a local static provider is used") -@pytest.mark.parametrize("case", ["resume", "scope_change", "profile_change"]) -def test_real_fork_admission_and_native_resume_use_retained_state(tmp_path, capsys, monkeypatch, case): +@pytest.mark.parametrize("case", ["resume", "scope_change", "profile_change", "joint"]) +def test_real_fork_admission_and_native_resume_use_retained_state(tmp_path, capsys, monkeypatch, case, request): """Real Host/CLI/Goal/session/artifact owners; deterministic synthetic work. This jointly qualifies native conversation and canonical Todo evidence - adoption, not full research/evaluator state or benchmark utility. No source trial or paid model is launched. + adoption. The joint case adds actual native HTTP and Host phase entry, with + synthetic grading. Full cross-owner barriers and research utility remain + outside this fixture. No benchmark cell or paid model is launched. """ import shutil import subprocess @@ -262,23 +267,44 @@ def invoke(arguments): "--task-lease-idempotency-key", "source-context", "--task-lease-expected-version", "1"]) requests = [] retained_context_path = None + joint_command = "" + if case == "joint": + import copy + import shlex + import requests as http + from benchmark.edgebench import checkpoint_controller as transport + from benchmark.tests.test_checkpoint_controller import register as register_judge, submit, host_phase_command + source_judge, branch_judge, source_url, branch_url, controllers_files, grades = request.getfixturevalue("controllers") + source_token = register_judge(source_judge, source_url, max_agent_submissions=3, submission_cooldown=0) + assert submit(source_url, source_token, 7).status_code == 200 + assert submit(source_url, source_token, 3).status_code == 200 + source_judge.state.judge.tokens[source_token]["submission_cooldown"] = 3 + source_phase = tmp_path / "source-phase" + source_phase.write_text(f"{int(time.time()) + 7200}\n") + evaluator_unit = tmp_path / "evaluator-point" + evaluator_point = transport.capture(controllers_files[0], token=source_token, + capture_id="e" * 32, destination=evaluator_unit, phase_deadline_file=source_phase) + evaluator_body = json.loads((evaluator_unit / "checkpoint.json").read_bytes()) + # Real participants are quiesced by this fixture. It is not a production + # cross-owner capture barrier or a proof of autonomous research utility. + def response(request, index, message): requests.append(request) if index == 2: return {"type": "function_call", "id": "fc_retained", "call_id": "call_retained", "name": "exec_command", "arguments": json.dumps({ - "cmd": "cat candidate.txt research.txt " + retained_context_path, "max_output_tokens": 6000})} + "cmd": "cat candidate.txt research.txt " + retained_context_path + joint_command, "max_output_tokens": 6000})} if index == 3: message["content"][0]["text"] = "Retained candidate 7 verified; candidate 3 remains rejected." return message binary = os.environ["LOOPX_NATIVE_CHECKPOINT_BIN"] - execution = Execution(context="resume", sandbox="read-only") + execution = Execution(context="resume", sandbox="workspace-write" if case == "joint" else "read-only") with real_native_source(tmp_path / "native", binary, workspace, close_source=True, - response_builder=response) as (env, source_session, _, calls): + response_builder=response, sandbox=execution.sandbox) as (env, source_session, _, calls): env.update(LOOPX_REGISTRY=str(source_registry), LOOPX_RUNTIME_ROOT=str(source_runtime), LOOPX_GOAL_ID=source_goal, LOOPX_AGENT_ID=source_agent, - MODEL_NAME="fixture", REASONING_EFFORT="high", LOOPX_CODEX_SANDBOX="read-only") + MODEL_NAME="fixture", REASONING_EFFORT="high", LOOPX_CODEX_SANDBOX=execution.sandbox) source_wake = BenchmarkSessionWake(env, execution, {}) source_wake.admission.accept_result(lambda: _store_codex_cli_session(source_runtime, lineage={"goal_id": source_goal, "agent_id": source_agent}, session_scope="agent", @@ -310,6 +336,8 @@ def response(request, index, message): components = [*native["components"], {"id": "code", "source_path": str(candidate)}, {"id": "research", "source_path": str(research)}, {"id": "research_context", "source_path": str(context_export)}] + if case == "joint": + components.append({"id": "evaluation", "source_path": str(evaluator_unit)}) probe = effect_runtime_result("runtime.checkpoint_artifacts", {"action": "probe", "components": components}) unit = tmp_path / "point" captured = effect_runtime_result("runtime.checkpoint_artifacts", {"action": "capture", @@ -322,6 +350,11 @@ def response(request, index, message): "provider_revision": source_context["snapshot"]["provider_revision"], "store_identity": source_context["snapshot"]["store_identity"]}}}, retry_safe=False) digest = captured["sha256"] + if case == "joint": + source_judge.state.judge.tokens[source_token]["submission_cooldown"] = 0 + assert submit(source_url, source_token, 99).status_code == 200 + judge_before = copy.deepcopy((source_judge.state.judge.tokens, + source_judge.state.judge.submissions, source_judge.state.judge.run_history)) invoke([*source_prefix, "todo", "update", *source_binding, "--agent-id", source_agent, "--note", "Candidate 999 is later evidence outside the captured point.", "--update-operation-id", "later-negative-evidence", "--update-expected-provider-revision", @@ -406,7 +439,7 @@ def change_scope_after_real_fork(*args, **kwargs): assert retained_source() == before return admitted_cli = subprocess.run([sys.executable, "-m", "benchmark.runtime.native_checkpoint", - "--root", str(unit), "--expected-sha256", digest, "--config-path", str(config), "--sandbox", "read-only"], + "--root", str(unit), "--expected-sha256", digest, "--config-path", str(config), "--sandbox", execution.sandbox], env=env, text=True, capture_output=True, timeout=40) assert admitted_cli.returncode == 0, admitted_cli.stderr admitted = json.loads(admitted_cli.stdout) @@ -454,12 +487,63 @@ def forbidden_fork(*args, **kwargs): assert json.loads(adopted_todo["todo"]["note"])["source_session"] == source_session guard = invoke([*prefix, "quota", "should-run", "--runtime-profile", "generic_cli", *binding]) assert guard["interaction_contract"]["agent_channel"]["delivery_allowed"] is True + if case == "joint": + control = tmp_path / "host-control" + control.mkdir() + branch_phase = control / "phase-deadline" + # Restore only the nested capsule from the same verified artifact, + # never the later source service or an unrelated caller file. + fresh = transport.restore(controllers_files[1], root=materialized / "components/evaluation", + expected_sha256=evaluator_point["sha256"], run_id="joint-branch", phase_deadline_file=branch_phase) + assert submit(branch_url, fresh["token"], 8).status_code == 429 + assert len(grades) == 3 # No implicit regrade on restore or cooldown rejection. + shell = host_phase_command(tmp_path, monkeypatch).replace( + "/opt/loopx-benchmark/control", str(control)) + budget = subprocess.run(["/bin/bash", "-c", shell + 'printf "%s" "$remaining"'], + text=True, capture_output=True, check=True) + assert 7100 < int(budget.stdout) <= 7200 + assert int(branch_phase.read_bytes()) <= time.time() + evaluator_body["point_metadata"]["host_phase_budget"]["remaining_seconds"] + # Explicit fresh token only, never controller/admin credentials. + # The native resumed tool reads public history, uses captured code, + # submits one new candidate, and observes the preserved round cap. + solver_input = workspace / "branch-judge.json" + solver_input.write_text(json.dumps({"url": branch_url, "token": fresh["token"]})) + solver_input.chmod(0o600) + solver = workspace / "continue_synthetic.py" + solver.write_text("""import json, time +from pathlib import Path +import requests +binding = json.loads(Path('branch-judge.json').read_text()) +url, token = binding['url'], binding['token'] +history = requests.get(url + '/api/v1/history', params={'token': token}, timeout=5).json() +assert history['best_score'] == 7 and history['agent_submissions'] == 2 +value = float(Path('candidate.txt').read_text().split(':')[1]) + 1 +response = requests.post(url + '/api/v1/submit', data={'token': token}, + files={'archive': ('candidate.tar.gz', str(value).encode())}, timeout=5) +response.raise_for_status() +identifier = response.json()['submission_id'] +for _ in range(100): + result = requests.get(url + '/api/v1/result/' + identifier, timeout=5).json() + if result['status'] == 'completed': break + time.sleep(.01) +assert result['status'] == 'completed' +exhausted = requests.post(url + '/api/v1/submit', data={'token': token}, + files={'archive': ('candidate.tar.gz', b'999')}, timeout=5) +assert exhausted.status_code == 429 +print('Captured public best 7; continued candidate 8; consumed round agent-3; no extra opportunity.') +""") + # Wait only the inherited short synthetic cooldown, never create an + # opportunity by changing source/destination counters or timestamps. + time.sleep(evaluator_body["point_metadata"]["native_reference"]["opportunity"]["remaining_cooldown_seconds"] + .05) + joint_command = " && " + shlex.quote(sys.executable) + " continue_synthetic.py" receipt = {} wake = BenchmarkSessionWake(env, execution, receipt) assert wake.session_id == admitted["session_id"] and receipt["session"]["action"] == "resume" logs = tmp_path / "continuation" logs.mkdir() command = native_command(env, execution, "execute", logs, wake) + if case == "joint": + command = ["/bin/bash", "-c", shell + "exec " + shlex.join(command)] current_body = json.dumps(guard["interaction_contract"], ensure_ascii=False) result = subprocess.run(command, env=env, cwd=workspace, input=current_body + "\nContinue: verify the retained files.", text=True, capture_output=True, timeout=40) @@ -480,6 +564,16 @@ def forbidden_fork(*args, **kwargs): assert "Future route outside this checkpoint" not in json.dumps(requests[2]) assert canonical_context({**env, "LOOPX_GOAL_ID": source_goal, "LOOPX_RUNTIME_ROOT": str(source_runtime)}, deadline=time.time() + 30) == later_context + if case == "joint": + assert "Captured public best 7; continued candidate 8" in json.dumps(requests[2]) + history = http.get(branch_url + "/api/v1/history", params={"token": fresh["token"]}, timeout=5).json() + assert history["best_score"] == 8 and history["best_round"] == "agent-3" + assert history["agent_submissions"] == 3 and len(grades) == 4 + assert (source_judge.state.judge.tokens, source_judge.state.judge.submissions, + source_judge.state.judge.run_history) == judge_before + for secret in (source_token, fresh["token"], source_judge.state.judge.admin_secret, + branch_judge.state.judge.admin_secret): + assert secret not in json.dumps(requests) assert "Retained candidate 7 verified" in result.stdout retained = native_session_checkpoint(env, admitted["session_id"], timeout=20) assert retained["metadata"]["turn_ids"][:-1] == native["metadata"]["turn_ids"] diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md index 8dec865034..4946318d00 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md @@ -803,6 +803,18 @@ fallback. Full Host provenance/clock alignment, joint research/effect adoption and original-operation recovery still need their profile's qualification; this participant transport cannot certify them. +The composed local continuation qualification now carries the evaluator capsule +inside the verified conversation/code/context artifact. The actual resumed Codex +tool reads retained research and public feedback, computes a synthetic candidate +from captured code, submits it through the native HTTP owner, and observes the +inherited round limit. The original cooldown rejects an immediate submission; +SForge's phase entry uses the remaining duration instead of a new trial budget. +Source future evidence is excluded and its later state is unchanged. Participants +are explicitly quiesced by the fixture; inference and grading are synthetic. +This narrows the integration gap but does not prove an automatic joint barrier, +canonical graph/vision/effect activation, original activation recovery, isolated +Linux solver restoration, or useful research. + An inert verification unit grants no execution authority. The destination uses fresh experiment/Host identities and the existing Goal/session/state admission paths. Historical leases, quota, timers, pending effects and submission tokens diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md index be995f3dc6..adfd06c967 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md @@ -664,6 +664,14 @@ phase 文件参与者还将声明的 Host deadline 与 native 捕获时间绑定 时钟对齐、联合研究/effect 采用与原操作恢复仍须按 profile 资格化；参与者 传输本身不能证明这些结果。 +组合式本地接续验证现将 evaluator capsule 放入同一已验证的会话/代码/上下文产物。 +实际恢复的 Codex 工具读取保留研究与公开反馈，从捕获代码计算合成候选，经原生 +HTTP owner 提交并观察继承的轮次上限。恢复后的首次立即提交仍被原冷却拒绝； +SForge phase 入口使用剩余时长，不重置整场预算。来源的后续证据被排除，其后续 +状态保持不变。参与者由 fixture 显式静置；模型响应与评分计算均为合成。 +这缩小了集成缺口，但不证明自动联合 barrier、canonical graph/vision/effect +激活、原激活操作恢复、隔离 Linux solver 恢复或有效研究。 + inert verification unit 不授予执行权。目标侧使用新的实验/Host 身份，通过既有 Goal/session/state admission 进入。历史租约、quota、timer、pending effect 和 submission token 不能变成目标侧权限；未知效果先按原身份 reconcile，再决定 From 4d1e7f277beeba502b74fae3952324c9e051daca Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 13:13:54 +0800 Subject: [PATCH 38/69] fix(benchmark): recover committed native checkpoint admission Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/runtime/native_checkpoint.py | 32 ++++++++++- benchmark/runtime/sessions.py | 4 +- benchmark/tests/test_native_checkpoint.py | 55 ++++++++++++++++++- benchmark/tests/test_resume_sessions.py | 33 +++++++++++ .../turn_driver/codex_sessions.py | 24 ++++++++ .../project_registry_io_manifest_v1.json | 2 +- 6 files changed, 143 insertions(+), 7 deletions(-) diff --git a/benchmark/runtime/native_checkpoint.py b/benchmark/runtime/native_checkpoint.py index c3f880aab7..960cf5259a 100644 --- a/benchmark/runtime/native_checkpoint.py +++ b/benchmark/runtime/native_checkpoint.py @@ -205,6 +205,8 @@ def admit_native_checkpoint_session(env: dict[str, str], unit: Path, expected_sh raise ValueError("checkpoint session admission requires its captured source session profile") if tomllib.loads(config_path.read_text()).get("features", {}).get("goals") is not False: raise ValueError("checkpoint session admission requires native automatic Goals explicitly disabled") + if hashlib.sha256(config_path.read_bytes()).hexdigest() != metadata["native_session"]["config_sha256"]: + raise ValueError("native checkpoint effective configuration differs") source = metadata.get("loopx_state", {}) lineage = {"goal_id": env["LOOPX_GOAL_ID"], "agent_id": env["LOOPX_AGENT_ID"]} if (not source.get("goal_id") or not source.get("agent_id") or @@ -217,7 +219,33 @@ def admit_native_checkpoint_session(env: dict[str, str], unit: Path, expected_sh require_registered_agent_id(registry_path=registry, goal_id=lineage["goal_id"], agent_id=lineage["agent_id"]) if select_codex_cli_session(Path(env["LOOPX_RUNTIME_ROOT"]), lineage=lineage, session_scope="agent") is not None: - raise ValueError("destination session binding already exists; read back its original admission") + # The original session binding is also the admission receipt. Only a + # source-bound committed admission can recover a lost controller reply; + # an unrelated binding or a partial native fork is never replaced. + with project_registry_transaction(registry, operation="checkpoint_session_binding", agent_id=lineage["agent_id"]): + current = load_goal_from_registry(registry, lineage["goal_id"]) + require_registered_agent_id(registry_path=registry, goal_id=lineage["goal_id"], agent_id=lineage["agent_id"]) + if current != goal: + raise ValueError("destination Goal configuration changed during native checkpoint readback") + wake = BenchmarkSessionWake(env, execution, {}) + binding = select_codex_cli_session(wake.root, lineage=lineage, session_scope="agent", + goal_admission=wake.admission) + if not binding or binding.get("checkpoint_artifact_sha256") != expected_sha256: + raise ValueError("destination session binding already exists without this checkpoint admission") + if wake.digest != source_profile: + raise ValueError("checkpoint source session profile changed") + native = metadata["native_session"] + observed = native_session_checkpoint(env, binding["session_id"], + timeout=max(.1, deadline - time.monotonic()))["metadata"] + keys = ("completed_turn_id", "turn_ids", "visible_history_sha256", "host_version", + "guest_home", "guest_workspace", "config_sha256") + if binding["session_id"] == native["session_id"] or any(observed[key] != native[key] for key in keys): + raise ValueError("checkpoint admitted native session advanced or differs; use its existing continuation") + return {"session_id": binding["session_id"], "completed_turn_id": observed["completed_turn_id"], + "visible_history_sha256": observed["visible_history_sha256"], "history_verified": True, + "model_calls": 0, "execution_authority_granted": False, "continuation_qualified": False, + "destination_goal_ref": wake.goal_ref, "destination_agent_id": lineage["agent_id"], + "session_binding_admitted": True, "session_profile_digest": wake.digest} result = fork_native_checkpoint(env, unit, expected_sha256, config_path, timeout=max(.1, deadline - time.monotonic())) # Hold the existing registry owner's fence through the session write. This @@ -229,7 +257,7 @@ def admit_native_checkpoint_session(env: dict[str, str], unit: Path, expected_sh raise ValueError("destination Goal configuration changed during native checkpoint reconstruction") receipt: dict[str, Any] = {} wake = BenchmarkSessionWake(env, execution, receipt) - wake._bind_verified_fork(result["session_id"], source_profile) + wake._bind_verified_fork(result["session_id"], source_profile, expected_sha256) return {**result, "destination_goal_ref": wake.goal_ref, "destination_agent_id": lineage["agent_id"], "session_binding_admitted": True, "session_profile_digest": wake.digest} diff --git a/benchmark/runtime/sessions.py b/benchmark/runtime/sessions.py index 1fbdc8fa0b..131c60f149 100644 --- a/benchmark/runtime/sessions.py +++ b/benchmark/runtime/sessions.py @@ -116,7 +116,8 @@ def observe(self, path: Path) -> None: ) self.receipt["session"]["session_id"] = observed - def _bind_verified_fork(self, session_id: str, source_profile_digest: str) -> None: + def _bind_verified_fork(self, session_id: str, source_profile_digest: str, + checkpoint_artifact_sha256: str) -> None: """Bind a controller-verified native fork through the existing session owner. This admits the conversation binding only. It does not claim work, @@ -135,6 +136,7 @@ def _bind_verified_fork(self, session_id: str, source_profile_digest: str) -> No goal_ref=self.goal_ref, session_profile_digest=self.digest, require_absent=True, + checkpoint_artifact_sha256=checkpoint_artifact_sha256, ) ) self.session_id = session_id diff --git a/benchmark/tests/test_native_checkpoint.py b/benchmark/tests/test_native_checkpoint.py index 631d9abbe9..922ee8ceb7 100644 --- a/benchmark/tests/test_native_checkpoint.py +++ b/benchmark/tests/test_native_checkpoint.py @@ -194,7 +194,7 @@ def test_real_native_workspace_readback_rejects_foreign_code_basis(tmp_path): @pytest.mark.skipif(not os.environ.get("LOOPX_NATIVE_CHECKPOINT_BIN"), reason="Requires an explicit real native Host binary; only a local static provider is used") -@pytest.mark.parametrize("case", ["resume", "scope_change", "profile_change", "joint"]) +@pytest.mark.parametrize("case", ["resume", "scope_change", "profile_change", "joint", "binding_reply_loss"]) def test_real_fork_admission_and_native_resume_use_retained_state(tmp_path, capsys, monkeypatch, case, request): """Real Host/CLI/Goal/session/artifact owners; deterministic synthetic work. @@ -438,6 +438,18 @@ def change_scope_after_real_fork(*args, **kwargs): assert len(calls) == 1 assert retained_source() == before return + if case == "binding_reply_loss": + original_bind = BenchmarkSessionWake._bind_verified_fork + def lose_binding_reply(self, *args): + original_bind(self, *args) + raise TimeoutError("synthetic loss after atomic binding") + with monkeypatch.context() as lost: + lost.setattr(BenchmarkSessionWake, "_bind_verified_fork", lose_binding_reply) + with pytest.raises(TimeoutError, match="loss after atomic binding"): + admit_native_checkpoint_session(env, unit, digest, config, execution=execution, timeout=30) + assert load_codex_cli_session(runtime, lineage={"goal_id": destination_goal, + "agent_id": destination_agent}, session_scope="agent")["checkpoint_artifact_sha256"] == digest + assert len(calls) == 1 admitted_cli = subprocess.run([sys.executable, "-m", "benchmark.runtime.native_checkpoint", "--root", str(unit), "--expected-sha256", digest, "--config-path", str(config), "--sandbox", execution.sandbox], env=env, text=True, capture_output=True, timeout=40) @@ -454,10 +466,40 @@ def change_scope_after_real_fork(*args, **kwargs): def forbidden_fork(*args, **kwargs): raise AssertionError("a repeated admission must not create a second native fork") replay.setattr("benchmark.runtime.native_checkpoint.fork_native_checkpoint", forbidden_fork) - with pytest.raises(ValueError, match="already exists"): - admit_native_checkpoint_session(env, unit, digest, config, execution=execution, timeout=30) + recovered = admit_native_checkpoint_session(env, unit, digest, config, execution=execution, timeout=30) + assert recovered == admitted + assert bound["checkpoint_artifact_sha256"] == digest + # Transport reply loss leaves the atomic source-bound binding as the + # original receipt. An exact CLI retry must also recover it. + retry_cli = subprocess.run(admitted_cli.args, env=env, text=True, capture_output=True, timeout=40) + assert retry_cli.returncode == 0, retry_cli.stderr + assert json.loads(retry_cli.stdout) == admitted + with pytest.raises(ValueError, match="profile changed"): + admit_native_checkpoint_session({**env, "REASONING_EFFORT": "low"}, unit, digest, + config, execution=execution, timeout=30) assert len(calls) == 1 and load_codex_cli_session(runtime, lineage={ "goal_id": destination_goal, "agent_id": destination_agent}, session_scope="agent") == bound + if case == "resume": + wrong_config = tmp_path / "changed-source-config.toml" + wrong_config.write_text(config.read_text() + "\n# different source configuration\n") + with pytest.raises(ValueError, match="effective configuration differs"): + admit_native_checkpoint_session(env, unit, digest, wrong_config, execution=execution, timeout=30) + from loopx.control_plane.turn_driver.codex_sessions import _session_path + binding_path = _session_path(runtime, {"goal_id": destination_goal, "agent_id": destination_agent}, "agent") + original_binding = binding_path.read_bytes() + # Isolated fixture corruption: neither an unrelated binding nor a + # binding for another checkpoint may be relabeled as this admission. + for wrong_source in (None, "f" * 64): + changed = dict(bound) + if wrong_source is None: + changed.pop("checkpoint_artifact_sha256") + else: + changed["checkpoint_artifact_sha256"] = wrong_source + binding_path.write_text(json.dumps(changed)) + with pytest.raises(ValueError, match="without this checkpoint admission"): + admit_native_checkpoint_session(env, unit, digest, config, execution=execution, timeout=30) + assert json.loads(binding_path.read_text()) == changed and len(calls) == 1 + binding_path.write_bytes(original_binding) # Both participants meet at the real destination continuation owner. # No source claim/lease is replayed; adoption alone still starts no work. @@ -578,6 +620,13 @@ def forbidden_fork(*args, **kwargs): retained = native_session_checkpoint(env, admitted["session_id"], timeout=20) assert retained["metadata"]["turn_ids"][:-1] == native["metadata"]["turn_ids"] assert retained["metadata"]["completed_turn_id"] != native["metadata"]["completed_turn_id"] + after_work = load_codex_cli_session(runtime, lineage={"goal_id": destination_goal, + "agent_id": destination_agent}, session_scope="agent") + assert after_work["checkpoint_artifact_sha256"] == digest + with pytest.raises(ValueError, match="advanced or differs"): + admit_native_checkpoint_session(env, unit, digest, config, execution=execution, timeout=30) + assert load_codex_cli_session(runtime, lineage={"goal_id": destination_goal, + "agent_id": destination_agent}, session_scope="agent") == after_work with pytest.raises(ValueError, match="native history or configuration moved"): adopt_todo_context(env, unit, digest, source_todo_id=source_todo, todo_id=destination_todo, operation_id="must-not-adopt-later-session", diff --git a/benchmark/tests/test_resume_sessions.py b/benchmark/tests/test_resume_sessions.py index 417154bf41..5c36ff37ea 100644 --- a/benchmark/tests/test_resume_sessions.py +++ b/benchmark/tests/test_resume_sessions.py @@ -40,6 +40,39 @@ def test_absent_only_session_admission_preserves_every_existing_binding(tmp_path assert load_codex_cli_session(tmp_path, lineage=lineage, session_scope="agent")["session_id"] == "observed-session" +@pytest.mark.parametrize("replacement", [None, "session", "profile", "lifetime"]) +def test_checkpoint_provenance_belongs_only_to_original_binding(tmp_path, replacement): + from loopx.control_plane.turn_driver.codex_sessions import _session_path, _store_codex_cli_session + + args = dict(lineage={"goal_id": "branch", "agent_id": "actor"}, session_scope="agent", + session_id="original", session_profile_digest="profile", goal_ref={"goal_id": "branch"}) + _store_codex_cli_session(tmp_path, **args, checkpoint_artifact_sha256="a" * 64, require_absent=True) + path = _session_path(tmp_path, args["lineage"], "agent") + original = path.read_bytes() + with pytest.raises(ValueError, match="cannot change"): + _store_codex_cli_session(tmp_path, **args, checkpoint_artifact_sha256="b" * 64) + assert path.read_bytes() == original + if replacement == "session": + args["session_id"] = "replacement" + elif replacement == "profile": + args["session_profile_digest"] = "changed" + elif replacement == "lifetime": + args["goal_ref"] = {"goal_id": "branch", "goal_instance_id": "replacement"} + _store_codex_cli_session(tmp_path, **args) + value = load_codex_cli_session(tmp_path, lineage=args["lineage"], session_scope="agent") + if replacement is None: + assert path.read_bytes() == original + assert value["checkpoint_artifact_sha256"] == "a" * 64 + else: + assert "checkpoint_artifact_sha256" not in value + # A never-enabled ordinary record retains exactly the original field shape. + plain = dict(args, session_id="ordinary") + _store_codex_cli_session(tmp_path, **plain) + assert set(load_codex_cli_session(tmp_path, lineage=args["lineage"], session_scope="agent")) == { + "schema_version", "host", "goal_id", "agent_id", "session_scope", "session_id", + "session_profile_digest", "goal_ref"} + + def resume_env(tmp_path, monkeypatch): env = worker_env(tmp_path) skills = tmp_path / "skills" diff --git a/loopx/control_plane/turn_driver/codex_sessions.py b/loopx/control_plane/turn_driver/codex_sessions.py index f13e864ef5..7381765d09 100644 --- a/loopx/control_plane/turn_driver/codex_sessions.py +++ b/loopx/control_plane/turn_driver/codex_sessions.py @@ -5,6 +5,7 @@ import hashlib import json import os +import re import tempfile from collections.abc import Mapping from pathlib import Path @@ -175,11 +176,27 @@ def _store_codex_cli_session( operation_model: str | None = None, operation_reasoning_effort: str | None = None, require_absent: bool = False, + checkpoint_artifact_sha256: str | None = None, ) -> None: + if checkpoint_artifact_sha256 is not None and not re.fullmatch(r"[0-9a-f]{64}", checkpoint_artifact_sha256): + raise ValueError("checkpoint session source digest is invalid") path = _session_path(runtime_root, lineage, session_scope) with exclusive_file_lock(path): if require_absent and (path.exists() or path.is_symlink()): raise ValueError("session binding already exists; read back the original admission") + # Provenance survives ordinary observations of this exact conversation, + # but cannot migrate to a replacement session, profile or Goal lifetime. + prior = load_codex_cli_session(runtime_root, lineage=lineage, session_scope=session_scope) + retained_source = None + if prior and (prior.get("session_id"), prior.get("goal_ref"), prior.get("session_profile_digest")) == ( + session_id, dict(goal_ref) if goal_ref is not None else None, session_profile_digest + ): + retained_source = prior.get("checkpoint_artifact_sha256") + if retained_source is not None: + if not isinstance(retained_source, str) or not re.fullmatch(r"[0-9a-f]{64}", retained_source): + raise ValueError("stored checkpoint session source digest is invalid") + if checkpoint_artifact_sha256 not in (None, retained_source): + raise ValueError("checkpoint session source digest cannot change") _write_codex_cli_session( runtime_root, lineage=lineage, @@ -190,6 +207,7 @@ def _store_codex_cli_session( operation_profile_digest=operation_profile_digest, operation_model=operation_model, operation_reasoning_effort=operation_reasoning_effort, + checkpoint_artifact_sha256=checkpoint_artifact_sha256 or retained_source, ) @@ -204,6 +222,7 @@ def _write_codex_cli_session( operation_profile_digest: str | None = None, operation_model: str | None = None, operation_reasoning_effort: str | None = None, + checkpoint_artifact_sha256: str | None = None, ) -> None: normalized_session_id = _valid_session_id(session_id) if not normalized_session_id: @@ -232,6 +251,8 @@ def _write_codex_cli_session( payload["session_profile_digest"] = session_profile_digest if goal_ref is not None: payload["goal_ref"] = dict(goal_ref) + if checkpoint_artifact_sha256 is not None: + payload["checkpoint_artifact_sha256"] = checkpoint_artifact_sha256 if operation_profile_digest is not None: payload["operation_transport"] = "app-server-operation-tools-v0" payload["operation_profile_digest"] = operation_profile_digest @@ -245,6 +266,9 @@ def _write_codex_cli_session( sort_keys=True, ) handle.write("\n") + if checkpoint_artifact_sha256 is not None: + handle.flush() + os.fsync(handle.fileno()) os.replace(temporary, path) path.chmod(0o600) finally: diff --git a/loopx/semantics/project_registry_io_manifest_v1.json b/loopx/semantics/project_registry_io_manifest_v1.json index 16519d0557..7275e8d844 100644 --- a/loopx/semantics/project_registry_io_manifest_v1.json +++ b/loopx/semantics/project_registry_io_manifest_v1.json @@ -1783,7 +1783,7 @@ }, { "site": "loopx/control_plane/turn_driver/codex_sessions.py::.approved_codex_workspace_write_resume::codec_read:load_project_registry#1", - "line": 358, + "line": 382, "column": 20, "kind": "codec_read", "api": "load_project_registry", From fa8a67fe1857b9bf6776bf5e6d455be229ec8ca2 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 13:13:55 +0800 Subject: [PATCH 39/69] docs(benchmark): explain native admission reply recovery Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/README.md | 15 +++++++++++---- .../rfcs/hierarchical-agent-stride-control-v0.md | 11 +++++++++++ .../hierarchical-agent-stride-control-v0.zh-CN.md | 8 ++++++++ 3 files changed, 30 insertions(+), 4 deletions(-) diff --git a/benchmark/README.md b/benchmark/README.md index 146d96d9d9..42ffd26aea 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -221,10 +221,17 @@ the registered destination Goal/Agent, without replacing an existing binding. The registry owner's fence guards the configuration check and binding write; this preserves the existing profile's lifetime semantics. The lifecycle-only `source_session_v1` profile is rejected by its owner because it cannot run -governed benchmark work. Changed configuration/profiles or ambiguous prior admission reject. After a -lost result, read the original agent-scoped session binding through -`BenchmarkSessionWake`; a repeated command cannot silently fork again. Omit this -command to leave the unit inert. Code/research authority, unresolved effects, +governed benchmark work. The binding atomically retains the captured artifact digest. +If the binding committed but its controller reply was lost, repeat this exact +command: it reads the original binding and verifies current Goal/Agent/profile, +Host/configuration and complete native history, returning the same admission +without another fork or model call. Changed or unmarked bindings reject; this +cannot retrofit recovery onto older bindings. Once native work has advanced, use +the existing continuation rather than retrying admission at the old cursor. +Provenance survives ordinary observations of the same session, but does not follow +a replacement session, profile or Goal lifetime. A fork whose reply was lost +**before binding** remains unresolved and cannot be automatically repeated. Omit +this command to leave the unit inert. Code/research authority, unresolved effects, native evaluator admission and remaining whole-trial budget still require their owners before executing a research continuation. This source-checkout controller entrypoint is a partial participant journey; it does not ship a frontend restore diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md index 4946318d00..daf166102d 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md @@ -815,6 +815,17 @@ This narrows the integration gap but does not prove an automatic joint barrier, canonical graph/vision/effect activation, original activation recovery, isolated Linux solver restoration, or useful research. +Committed native session admission now retains the captured artifact digest in +the original atomic session binding. Exact controller retries after a lost binding +reply verify that receipt, current Goal/Agent/profile and unchanged complete +native history, then return the same session without a fork or model call. +Ordinary subsequent observations preserve provenance only for that conversation; +replacement identity drops it. Unmarked/foreign bindings and advanced native +history reject old-point admission. This closes post-binding reply loss, not an +unknown fork before binding or joint evaluator/Host activation recovery. Tests +inject reply loss after the actual binding, then recover through the CLI and +continue real native governed work with a deterministic local provider. + An inert verification unit grants no execution authority. The destination uses fresh experiment/Host identities and the existing Goal/session/state admission paths. Historical leases, quota, timers, pending effects and submission tokens diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md index adfd06c967..57e88aac8a 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md @@ -645,6 +645,14 @@ canonical graph/vision/effect 激活、完整 native/Host/evaluator 一致性、 恢复、packaged 交付和实际有效研究仍待验收。operator metadata 或历史一致不能 认证参与者/effect 对账与完整新研究状态采用。 +已提交的 native session 准入现在将捕获产物 digest 与原 session binding 原子保存。 +binding 已提交但控制器响应丢失时，重复原命令会核对原凭据、当前 Goal/Agent/profile +及未推进的完整 native 历史，返回同一 session，不另起 fork 或模型调用。同一会话的 +后续正常观察保留来源，替换 session、profile 或 Goal lifetime 不继承来源。无来源 +标记、外来 binding 或已推进历史均拒绝旧点准入。真实 binding 后注入响应丢失， +再经 CLI 恢复并继续实际 native governed 工作的确定性测试覆盖这一边界；fork 在 +binding 前的未知结果与 evaluator/Host 联合激活恢复仍未资格化。 + 原生新注册不等于恢复准入。真实原生 Docker 资格测试中，source 已消耗十二次 中的一次，且冷却仍有效；同额度的新注册却立即受理 `agent-1`，保留十二次未用 机会。重评已保存 archive 产生新的结果，没有恢复 source 的历史或机会状态。 From d2d7ec928262fc6fbea625dbe9f1eabda09a5de5 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 14:43:18 +0800 Subject: [PATCH 40/69] fix(benchmark): reconcile native checkpoint fork reply loss Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/runtime/native_checkpoint.py | 114 +++++++++++++++++++--- benchmark/tests/test_native_checkpoint.py | 44 ++++++++- 2 files changed, 142 insertions(+), 16 deletions(-) diff --git a/benchmark/runtime/native_checkpoint.py b/benchmark/runtime/native_checkpoint.py index 960cf5259a..9a1fafd0e0 100644 --- a/benchmark/runtime/native_checkpoint.py +++ b/benchmark/runtime/native_checkpoint.py @@ -9,8 +9,10 @@ import hashlib import json import math +import os import re import shutil +import sqlite3 import subprocess import time import tomllib @@ -101,8 +103,47 @@ def request(method, params): }} +def _native_thread_ids(home: Path) -> set[str]: + """Read opaque identities omitted by the native interactive thread list. + + At the qualified Host version, thread/list hides a fresh fork until its + first user event. The native index locates candidates only: thread/read + and complete native history still prove provenance and the captured cursor. + Never edit this provider-owned store or infer authority from an index row. + """ + indexes = [path for path in home.iterdir() if re.fullmatch(r"state_[0-9]+\.sqlite", path.name)] + if len(indexes) != 1 or indexes[0].resolve(strict=True) != indexes[0] or not indexes[0].is_file(): + raise ValueError("native fork recovery requires one qualified physical session index") + with sqlite3.connect(indexes[0].as_uri() + "?mode=ro", uri=True) as connection: + rows = connection.execute("SELECT id FROM threads").fetchall() + ids = [row[0] for row in rows] + if any(not isinstance(value, str) or not value for value in ids) or len(ids) != len(set(ids)): + raise ValueError("native fork recovery index has invalid or duplicate identities") + return set(ids) + + def fork_native_checkpoint(env: dict[str, str], unit: Path, expected_sha256: str, config_path: Path, *, timeout: float) -> dict[str, Any]: + # The existing process lock serializes reconstruction in this isolated home. + # The private attempt below describes one opaque native RPC, not authority + # to admit a Goal, a work Turn, or a second fork after an uncertain response. + from loopx.file_lock import exclusive_file_lock + + if not math.isfinite(timeout) or timeout <= 0: + raise ValueError("native checkpoint reconstruction timeout must be finite and positive") + deadline = time.monotonic() + timeout + home = Path(env["CODEX_HOME"]) + if not home.is_absolute() or home != home.resolve(): + raise ValueError("native checkpoint destination must be a physical absolute home") + with exclusive_file_lock(home, timeout_seconds=timeout, operation="checkpoint_native_fork"): + remaining = deadline - time.monotonic() + if remaining <= 0: + raise TimeoutError("native checkpoint reconstruction window closed") + return _fork_native_checkpoint(env, unit, expected_sha256, config_path, timeout=remaining) + + +def _fork_native_checkpoint(env: dict[str, str], unit: Path, expected_sha256: str, + config_path: Path, *, timeout: float) -> dict[str, Any]: """Reconstruct one native branch in a new, fixed-path isolated environment. No model starts and no LoopX binding is registered. Existing session/Goal @@ -119,8 +160,10 @@ def fork_native_checkpoint(env: dict[str, str], unit: Path, expected_sha256: str home = Path(env["CODEX_HOME"]) if str(home) != native["guest_home"] or env["LOOPX_PROJECT"] != native["guest_workspace"]: raise ValueError("native checkpoint requires its qualified fixed guest paths") - if home.exists() or home.is_symlink(): - raise ValueError("native checkpoint destination home must be new") + attempt_path = home / ".loopx-checkpoint-fork.json" + recovering = home.exists() + if recovering and (not attempt_path.is_file() or attempt_path.resolve() != attempt_path): + raise ValueError("native checkpoint destination home must be new or retain its original fork attempt") config = config_path.read_bytes() if hashlib.sha256(config).hexdigest() != native["config_sha256"]: raise ValueError("native checkpoint effective configuration differs") @@ -136,14 +179,30 @@ def fork_native_checkpoint(env: dict[str, str], unit: Path, expected_sha256: str re.fullmatch(r"(?:state|thread_history)_[0-9]+\.sqlite", name))): raise ValueError("native checkpoint member inventory is invalid") names.add(name) - home.mkdir(parents=True, mode=0o700) - for member in native["store_members"]: - source, destination = unit / "components" / member["id"], home / member["name"] - if source.is_dir(): - shutil.copytree(source, destination) - else: - shutil.copy2(source, destination) - (home / "config.toml").write_bytes(config) + basis = {"checkpoint_sha256": expected_sha256, "source_session_id": native["session_id"], + "completed_turn_id": native["completed_turn_id"], "config_sha256": native["config_sha256"], + "destination_goal_id": env.get("LOOPX_GOAL_ID"), "destination_agent_id": env.get("LOOPX_AGENT_ID"), + "guest_workspace": env["LOOPX_PROJECT"], "guest_home": str(home)} + attempt = None + if recovering: + attempt = json.loads(attempt_path.read_bytes()) + if (not isinstance(attempt, dict) or set(attempt) != {"basis", "source_thread_ids"} or + attempt["basis"] != basis or not isinstance(attempt["source_thread_ids"], list) or + any(not isinstance(v, str) or not v for v in attempt["source_thread_ids"]) or + len(set(attempt["source_thread_ids"])) != len(attempt["source_thread_ids"]) or + native["session_id"] not in attempt["source_thread_ids"]): + raise ValueError("native fork attempt differs from the requested checkpoint or destination") + if (home / "config.toml").read_bytes() != config: + raise ValueError("native fork recovery effective configuration differs") + else: + home.mkdir(parents=True, mode=0o700) + for member in native["store_members"]: + source, destination = unit / "components" / member["id"], home / member["name"] + if source.is_dir(): + shutil.copytree(source, destination) + else: + shutil.copy2(source, destination) + (home / "config.toml").write_bytes(config) with StdioNativeGoalTransport.spawn( [env["CODEX_BIN"], "-c", "features.goals=false", "app-server", "--listen", "stdio://"], cwd=env["LOOPX_PROJECT"], env=env, stderr=subprocess.DEVNULL, @@ -158,13 +217,38 @@ def request(method, params): request("initialize", {"clientInfo": {"name": "loopx_checkpoint", "version": "0.1.0"}, "capabilities": {"experimentalApi": True}}) transport.notify("initialized", {}) - fork = request("thread/fork", {"threadId": native["session_id"], - "lastTurnId": native["completed_turn_id"], "cwd": env["LOOPX_PROJECT"], - "sandbox": "read-only", "approvalPolicy": "never", "deferGoalContinuation": True, - "excludeTurns": True}) - session = fork.get("thread", {}).get("id") + inventory = _native_thread_ids(home) + if recovering: + added = set(inventory) - set(attempt["source_thread_ids"]) + if len(added) != 1 or not set(attempt["source_thread_ids"]).issubset(inventory): + raise ValueError("native fork outcome remains absent or ambiguous; a second fork is prohibited") + session = added.pop() + fork = request("thread/read", {"threadId": session, "includeTurns": False}) + else: + if native["session_id"] not in inventory: + raise ValueError("native checkpoint source is missing from the reconstructed store") + attempt = {"basis": basis, "source_thread_ids": sorted(inventory)} + # Persist before dispatch. Partial preparation is not a license to + # fork again; a missing outcome stays explicitly unresolved. + with os.fdopen(os.open(attempt_path, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600), "w") as file: + json.dump(attempt, file, sort_keys=True) + file.flush() + os.fsync(file.fileno()) + directory = os.open(home, os.O_RDONLY) + try: + os.fsync(directory) + finally: + os.close(directory) + fork = request("thread/fork", {"threadId": native["session_id"], + "lastTurnId": native["completed_turn_id"], "cwd": env["LOOPX_PROJECT"], + "sandbox": "read-only", "approvalPolicy": "never", "deferGoalContinuation": True, + "excludeTurns": True}) + session = fork.get("thread", {}).get("id") if not session or session == native["session_id"]: raise ValueError("native checkpoint did not create a distinct session") + if (fork.get("thread", {}).get("forkedFromId") != native["session_id"] or + fork["thread"].get("cwd") != env["LOOPX_PROJECT"]): + raise ValueError("native fork provenance differs from the original attempt") turns = _complete_history(request, session) encoded = json.dumps(turns, sort_keys=True, separators=(",", ":")).encode() if ([t["id"] for t in turns] != native["turn_ids"] or diff --git a/benchmark/tests/test_native_checkpoint.py b/benchmark/tests/test_native_checkpoint.py index 922ee8ceb7..62ad30e740 100644 --- a/benchmark/tests/test_native_checkpoint.py +++ b/benchmark/tests/test_native_checkpoint.py @@ -194,7 +194,8 @@ def test_real_native_workspace_readback_rejects_foreign_code_basis(tmp_path): @pytest.mark.skipif(not os.environ.get("LOOPX_NATIVE_CHECKPOINT_BIN"), reason="Requires an explicit real native Host binary; only a local static provider is used") -@pytest.mark.parametrize("case", ["resume", "scope_change", "profile_change", "joint", "binding_reply_loss"]) +@pytest.mark.parametrize("case", ["resume", "scope_change", "profile_change", "joint", "binding_reply_loss", + "fork_reply_loss", "fork_missing", "fork_ambiguous"]) def test_real_fork_admission_and_native_resume_use_retained_state(tmp_path, capsys, monkeypatch, case, request): """Real Host/CLI/Goal/session/artifact owners; deterministic synthetic work. @@ -438,6 +439,45 @@ def change_scope_after_real_fork(*args, **kwargs): assert len(calls) == 1 assert retained_source() == before return + recovered_fork = None + if case in {"fork_reply_loss", "fork_missing", "fork_ambiguous"}: + from benchmark.runtime.native_checkpoint import StdioNativeGoalTransport + original_request = StdioNativeGoalTransport.request + forks = [] + def lose_fork_reply(self, method, params): + if method != "thread/fork": + return original_request(self, method, params) + if case != "fork_missing": + forks.append(original_request(self, method, params)["thread"]["id"]) + if case == "fork_ambiguous": + forks.append(original_request(self, method, params)["thread"]["id"]) + raise TimeoutError("synthetic loss before fork reply") + with monkeypatch.context() as lost: + lost.setattr(StdioNativeGoalTransport, "request", lose_fork_reply) + with pytest.raises(TimeoutError, match="loss before fork reply"): + admit_native_checkpoint_session(env, unit, digest, config, execution=execution, timeout=30) + assert load_codex_cli_session(runtime, lineage={"goal_id": destination_goal, + "agent_id": destination_agent}, session_scope="agent") is None + assert len(calls) == 1 and retained_source() == before + if case != "fork_reply_loss": + def no_second_fork(self, method, params): + assert method != "thread/fork", "uncertainty must not dispatch a second fork" + return original_request(self, method, params) + with monkeypatch.context() as retry: + retry.setattr(StdioNativeGoalTransport, "request", no_second_fork) + with pytest.raises(ValueError, match="absent or ambiguous"): + admit_native_checkpoint_session(env, unit, digest, config, execution=execution, timeout=30) + assert load_codex_cli_session(runtime, lineage={"goal_id": destination_goal, + "agent_id": destination_agent}, session_scope="agent") is None + assert len(calls) == 1 and retained_source() == before + return + recovered_fork = forks[0] + # The isolated attempt is bound to this destination, not merely an + # equivalent visible history or another Goal's transport directory. + from benchmark.runtime.native_checkpoint import fork_native_checkpoint + with pytest.raises(ValueError, match="attempt differs"): + fork_native_checkpoint({**env, "LOOPX_AGENT_ID": "unrelated-destination"}, + unit, digest, config, timeout=30) if case == "binding_reply_loss": original_bind = BenchmarkSessionWake._bind_verified_fork def lose_binding_reply(self, *args): @@ -458,6 +498,8 @@ def lose_binding_reply(self, *args): assert admitted["session_binding_admitted"] and admitted["history_verified"] assert admitted["model_calls"] == 0 and not admitted["continuation_qualified"] assert admitted["session_id"] != source_session and len(calls) == 1 + if recovered_fork: + assert admitted["session_id"] == recovered_fork bound = load_codex_cli_session(runtime, lineage={"goal_id": destination_goal, "agent_id": destination_agent}, session_scope="agent") assert bound["session_id"] == admitted["session_id"] From 9021c9536f4b5ab5cdcc21499fbf074476d82eb1 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 14:43:18 +0800 Subject: [PATCH 41/69] fix(benchmark): preserve caller checkpoint component identities Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/runtime/checkpoint.py | 17 +++++++++-- .../test_research_checkpoint_capture.py | 29 ++++++++++++++++--- 2 files changed, 39 insertions(+), 7 deletions(-) diff --git a/benchmark/runtime/checkpoint.py b/benchmark/runtime/checkpoint.py index 7932523dab..4714e10113 100644 --- a/benchmark/runtime/checkpoint.py +++ b/benchmark/runtime/checkpoint.py @@ -55,6 +55,16 @@ def parse_checkpoint_request(raw: bytes) -> dict[str, Any]: return request +def _participant_id(components, preferred): + """Automatic participants cannot reserve an otherwise legal caller ID.""" + occupied = {entry["id"] for entry in components} + candidate, suffix = preferred, 0 + while candidate in occupied: + suffix += 1 + candidate = f"{preferred}_{suffix}" + return candidate + + def _loopx_state_components(env, components, output_dir): """Adapt the existing backup owner's inventory for one isolated trial. @@ -103,7 +113,7 @@ def _loopx_state_components(env, components, output_dir): continue if any(Path(entry["source_path"]).is_relative_to(path) for entry in entries): raise ValueError("LoopX state inventory overlaps a narrower declared component; declare its parent once") - entries.append({"id": f"loopx_state_{len(entries)}", "source_path": str(path)}) + entries.append({"id": _participant_id(entries, f"loopx_state_{len(entries)}"), "source_path": str(path)}) coverage = [{"key": target["key"], "source_path": target["source_path"], "component_id": next(entry["id"] for entry in entries if Path(target["source_path"]).is_relative_to(Path(entry["source_path"]))) } @@ -348,7 +358,7 @@ def capture_after_wake(env: dict[str, str], receipt: dict[str, Any], *, file.write(research_raw) parent = next((entry for entry in components if evidence.is_relative_to(Path(entry["source_path"]))), None) if parent is None: - parent = {"id": "research_context", "source_path": str(evidence)} + parent = {"id": _participant_id(components, "research_context"), "source_path": str(evidence)} components = [*components, parent] metadata["research_context"] = {"component_id": parent["id"], "member_path": evidence.relative_to(Path(parent["source_path"])).as_posix() if evidence != Path(parent["source_path"]) else "", @@ -384,7 +394,8 @@ def capture_after_wake(env: dict[str, str], receipt: dict[str, Any], *, from loopx.control_plane.turn_driver.codex_sessions import _session_path binding_path = _session_path(source_wake.root, source_wake.lineage, "agent") if not any(binding_path.is_relative_to(Path(entry["source_path"])) for entry in components): - components = [*components, {"id": "source_session_binding", "source_path": str(binding_path)}] + components = [*components, {"id": _participant_id(components, "source_session_binding"), + "source_path": str(binding_path)}] metadata["session"] = {**session, "session_profile_digest": source_wake.digest} native = native_session_checkpoint(env, session.get("session_id", ""), timeout=max(.1, deadline - time.time())) diff --git a/tests/control_plane/test_research_checkpoint_capture.py b/tests/control_plane/test_research_checkpoint_capture.py index ba1b8a8a04..ecceaef437 100644 --- a/tests/control_plane/test_research_checkpoint_capture.py +++ b/tests/control_plane/test_research_checkpoint_capture.py @@ -217,7 +217,8 @@ def altered_probe(method, params, **kwargs): reason="Requires an explicit real native Host binary; static local provider only") @pytest.mark.parametrize("source_moves", [False, True, "binding"]) @pytest.mark.parametrize("mutation_stage", ["probe", "capture"]) -def test_real_native_cursor_must_match_sealed_store_basis(tmp_path, monkeypatch, source_moves, mutation_stage): +def test_real_native_cursor_must_match_sealed_store_basis(tmp_path, monkeypatch, source_moves, mutation_stage, + caller_id="code"): """Actual CLI settlement, native completed Turns and TS SQLite artifacts.""" from benchmark.tests.test_native_checkpoint import real_native_source from benchmark.runtime.codex import Execution @@ -249,7 +250,7 @@ def run(*args): candidate.write_text("source_candidate = 7\n") request = tmp_path / "request.json" request.write_text(json.dumps({"destination": str(destination), "components": [ - {"id": "code", "source_path": str(candidate)}], "native_session": True, + {"id": caller_id, "source_path": str(candidate)}], "native_session": True, "point_metadata": {"meaning": "Native/CLI/TS fixture, not research or model utility"}})) with real_native_source(tmp_path / "native", os.environ["LOOPX_NATIVE_CHECKPOINT_BIN"], project) as (env, source, complete, calls): env.update(LOOPX_CHECKPOINT_REQUEST=str(request), LOOPX_WAKE_LOG_DIR=str(logs / "wakes"), @@ -287,9 +288,16 @@ def between_read_and_probe(method, params, **kwargs): assert receipt["checkpoint"]["captured"], receipt manifest = json.loads((destination / "checkpoint.json").read_text()) assert len(manifest["point_metadata"]["native_session"]["turn_ids"]) == 1 + assert (destination / "components" / caller_id).read_bytes() == candidate.read_bytes() assert receipt["checkpoint"]["research_point_qualified"] is False +@pytest.mark.skipif(not os.environ.get("LOOPX_NATIVE_CHECKPOINT_BIN"), reason="Requires a real native Host") +def test_native_binding_participant_preserves_legal_caller_id(tmp_path, monkeypatch): + test_real_native_cursor_must_match_sealed_store_basis(tmp_path, monkeypatch, False, "probe", + caller_id="source_session_binding") + + @pytest.mark.parametrize("capture_fails", [False, True]) def test_real_pre_replan_capture_is_before_decision_and_same_node_is_not_recaptured(tmp_path, monkeypatch, capture_fails): import os @@ -380,7 +388,8 @@ def fail_probe(method, params, **kwargs): ("canonical_head", "capture"), ("exported_bytes", "capture"), ("research_log", "probe"), ("research_log", "capture"), ("vision_index", "probe"), ("vision_index", "capture")]) -def test_controller_participant_keeps_canonical_state_and_journals_inert(tmp_path, monkeypatch, provider, drift, mutation_stage): +def test_controller_participant_keeps_canonical_state_and_journals_inert(tmp_path, monkeypatch, provider, drift, + mutation_stage, caller_ids=False): from canonical_authority_fixture import initialize_canonical_authority from loopx.control_plane.coordination.runtime_shadow import build_todo_runtime_shadow_projection from loopx.control_plane.goals.first_party_host_admission import capture_first_party_host_goal_ref @@ -448,8 +457,12 @@ def append_source_evidence(title): (project / 'solver.py').write_text('candidate = 7\n') target = logs / 'checkpoints/controller' request = tmp_path / 'request.json' + components = [{'id':'research_context' if caller_ids else 'code','source_path':str(project/'solver.py')}] + if caller_ids: + (project/'notes.txt').write_text('Retained caller-owned context.') + components.append({'id':'loopx_state_2','source_path':str(project/'notes.txt')}) request.write_text(json.dumps({'destination':str(target), 'loopx_state':True, - 'components':[{'id':'code','source_path':str(project/'solver.py')}], 'point_metadata':{}})) + 'components':components, 'point_metadata':{}})) env = {'LOOPX_CHECKPOINT_REQUEST':str(request),'LOOPX_WAKE_LOG_DIR':str(wakes), 'LOOPX_PROJECT':str(project),'LOOPX_RUNTIME_ROOT':str(runtime),'LOOPX_REGISTRY':str(registry), 'LOOPX_GOAL_ID':cli.GOAL_ID,'LOOPX_AGENT_ID':cli.AGENT_ID} @@ -480,6 +493,8 @@ def changed_context(method, params, **kwargs): return assert receipt['checkpoint'].get('captured') is True, json.dumps(receipt['checkpoint']) manifest = json.loads((target/'checkpoint.json').read_text()) + for component in components: + assert (target/'components'/component['id']).read_bytes() == Path(component['source_path']).read_bytes() declared_context = manifest['point_metadata']['research_context'] context_root = target/'components'/declared_context['component_id'] exported_context = context_root/declared_context['member_path'] if declared_context['member_path'] else context_root @@ -516,6 +531,12 @@ def changed_context(method, params, **kwargs): assert after['authority_read']['provider_revision'] == before['authority_read']['provider_revision'] +@pytest.mark.parametrize('provider', ['file', 'sqlite']) +def test_research_participants_preserve_legal_caller_ids(tmp_path, monkeypatch, provider): + test_controller_participant_keeps_canonical_state_and_journals_inert( + tmp_path, monkeypatch, provider, None, 'probe', caller_ids=True) + + @pytest.mark.parametrize('gap', ['shared_registry','wrong_actor','wrong_runtime','missing_state','symlink_state','overlap']) def test_loopx_participant_rejects_wrong_or_ambiguous_routes(tmp_path, gap): from benchmark.runtime.checkpoint import _loopx_state_components From e2dc7eea867164b3a94c0fdf2aafda5d52819045 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 14:43:18 +0800 Subject: [PATCH 42/69] docs(benchmark): explain native fork recovery boundaries Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/README.md | 19 +++++++++++++++++-- .../hierarchical-agent-stride-control-v0.md | 15 +++++++++++---- ...rarchical-agent-stride-control-v0.zh-CN.md | 12 +++++++++--- 3 files changed, 37 insertions(+), 9 deletions(-) diff --git a/benchmark/README.md b/benchmark/README.md index 42ffd26aea..14f6f595db 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -140,6 +140,10 @@ artifacts and exclude credentials and hidden evaluation material: } ``` +Caller component IDs are preserved. Automatic LoopX-state, research-context and +session-binding participants choose an unused local ID when their preferred +name is already declared; source-path deduplication and byte witnesses still apply. + The destination must be a new direct child of the trial log owner's `checkpoints` directory. Capture runs after a successful execute wake's original settlement, before the next wake. It uses the remaining phase budget with a @@ -229,8 +233,19 @@ without another fork or model call. Changed or unmarked bindings reject; this cannot retrofit recovery onto older bindings. Once native work has advanced, use the existing continuation rather than retrying admission at the old cursor. Provenance survives ordinary observations of the same session, but does not follow -a replacement session, profile or Goal lifetime. A fork whose reply was lost -**before binding** remains unresolved and cannot be automatically repeated. Omit +a replacement session, profile or Goal lifetime. Before dispatching the native +fork, the controller durably records the artifact/destination basis and original +native thread identities under the existing process lock. If the fork reply is +lost **before binding**, retry the same command in the same isolated environment. +It admits only one new native identity whose native parent, workspace and complete +history match the captured source. It never dispatches another fork on recovery. +Missing or multiple outcomes, changed destinations/configuration and advanced +history remain unresolved and require reconciliation; do not erase the attempt. +At the qualified Host version, the interactive `thread/list` omits a fresh fork +until its first user event. Recovery therefore locates opaque IDs through a +read-only native SQLite index, then verifies them through `thread/read` and +`thread/turns/list`; it never edits native rows or treats the index as authority. +Omit this command to leave the unit inert. Code/research authority, unresolved effects, native evaluator admission and remaining whole-trial budget still require their owners before executing a research continuation. This source-checkout controller diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md index daf166102d..7f8ea89448 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md @@ -821,10 +821,17 @@ reply verify that receipt, current Goal/Agent/profile and unchanged complete native history, then return the same session without a fork or model call. Ordinary subsequent observations preserve provenance only for that conversation; replacement identity drops it. Unmarked/foreign bindings and advanced native -history reject old-point admission. This closes post-binding reply loss, not an -unknown fork before binding or joint evaluator/Host activation recovery. Tests -inject reply loss after the actual binding, then recover through the CLI and -continue real native governed work with a deterministic local provider. +history reject old-point admission. Before native fork dispatch, a private +transport attempt retains the source artifact/destination basis and native +identity inventory under the existing process lock. A lost pre-binding reply can +recover the unique new native child, after native parent/workspace and complete +history verification, without dispatching another fork. Missing or ambiguous +outcomes stay unresolved. The pinned native index supplies only opaque lookup +IDs because the interactive listing hides unused forks; native APIs verify their +meaning, and no native rows are rewritten. Tests inject pre- and post-binding +reply loss, recover through the CLI and continue real native governed work with +a deterministic local provider; absent/multiple outcomes and foreign destination +attempts reject. Joint evaluator/Host activation recovery remains unqualified. An inert verification unit grants no execution authority. The destination uses fresh experiment/Host identities and the existing Goal/session/state admission diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md index 57e88aac8a..6c8e4c5f92 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md @@ -649,9 +649,15 @@ canonical graph/vision/effect 激活、完整 native/Host/evaluator 一致性、 binding 已提交但控制器响应丢失时，重复原命令会核对原凭据、当前 Goal/Agent/profile 及未推进的完整 native 历史，返回同一 session，不另起 fork 或模型调用。同一会话的 后续正常观察保留来源，替换 session、profile 或 Goal lifetime 不继承来源。无来源 -标记、外来 binding 或已推进历史均拒绝旧点准入。真实 binding 后注入响应丢失， -再经 CLI 恢复并继续实际 native governed 工作的确定性测试覆盖这一边界；fork 在 -binding 前的未知结果与 evaluator/Host 联合激活恢复仍未资格化。 +标记、外来 binding 或已推进历史均拒绝旧点准入。发送 native fork 前，控制器在 +既有进程锁内持久保存来源产物、目标身份与原 native identity 清单。binding 前 +响应丢失时，只恢复唯一新增且经原生 parent/workspace/完整历史核验的子会话， +不再次发送 fork。结果缺失、多个结果或目标变化继续保持未决。固定版本的原生 +交互列表会隐藏尚无新用户事件的 fork，因此只读 native SQLite index 仅用于定位 +不透明 ID，再由 native API 验证语义；不改写数据库、不从 index 授予权限。 +真实 binding 前后注入响应丢失，再经 CLI 恢复并继续 native governed 工作的 +确定性测试覆盖这一边界，并验证缺失、多结果和外来目标拒绝；evaluator/Host +联合激活恢复仍未资格化。 原生新注册不等于恢复准入。真实原生 Docker 资格测试中，source 已消耗十二次 中的一次，且冷却仍有效；同额度的新注册却立即受理 `agent-1`，保留十二次未用 From 439feea2eeb7020bff69f3378d2b520dd6fdfdce Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 15:21:40 +0800 Subject: [PATCH 43/69] test(benchmark): characterize native interruption before checkpoint handoff Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/README.md | 14 +++- benchmark/tests/test_native_checkpoint.py | 88 +++++++++++++++++++++++ 2 files changed, 101 insertions(+), 1 deletion(-) diff --git a/benchmark/README.md b/benchmark/README.md index 14f6f595db..75b51584f5 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -423,7 +423,13 @@ remains private; inspect the verified inventory before qualifying a complete poi The current Host hook cannot capture a new replan created **inside an already running model wake** before that decision. Those boundaries still need a qualified -Host handoff; a later snapshot must not be labeled as the missed point. Complete +Host handoff; a later snapshot must not be labeled as the missed point. A real +native qualification now demonstrates interrupting at a synthetic tool boundary, +forking the exact retained history and producing a new artifact while preserving +the source history. The existing checkpoint correctly rejects that interrupted +cursor: native resumability does not prove settled LoopX work, a pre-decision +replan boundary or joint evaluator consistency. The probe does not yet wire the +worker handoff or reconstruct a separate solver environment. Complete LoopX research-state adoption, joint evaluator/Host admission and useful continued work remain open. No automatic retention or extra evaluation opportunity is introduced. @@ -531,3 +537,9 @@ runtime receipt 绑定摘要；programmatic adapter 也可指定已放入 guest 注册状态、项目研究状态、runtime journal 和 registry；File/SQLite authority 物化后仍是惰性历史。缺少来源、共享注册表、路径别名或重叠声明会拒绝，捕获 不授予新的租约、配额、效果或执行权，不能替代完整接续验收。 + +运行中才出现的 fresh replan 仍缺 Host 交接。真实 native 探针已证明：在合成工具 +边界 interrupt 后，可 fork 完整原生历史、继续读原候选并产出新文件，原历史不变。 +现有 checkpoint 仍正确拒绝该 interrupted cursor；可继续原生会话不等于已结算 +LoopX 工作、replan 决策前边界或 evaluator 联合一致性。该探针没有接入 worker +自动捕获，也没有证明独立 solver 环境恢复，不能据此宣称每个 replan 已覆盖。 diff --git a/benchmark/tests/test_native_checkpoint.py b/benchmark/tests/test_native_checkpoint.py index 62ad30e740..7b6cade42e 100644 --- a/benchmark/tests/test_native_checkpoint.py +++ b/benchmark/tests/test_native_checkpoint.py @@ -176,6 +176,94 @@ def complete_turn(): thread.join() +@pytest.mark.skipif(not os.environ.get("LOOPX_NATIVE_CHECKPOINT_BIN"), + reason="Requires an explicit real native Host binary; no model service used") +def test_real_native_interrupt_retains_branch_but_does_not_qualify_checkpoint(tmp_path): + """A resumable native interruption is not a completed research boundary. + + This characterizes the provider seam needed by midwake replan handoff. The + existing checkpoint must keep rejecting it until the Host proves the joint + boundary; a successful native fork alone must not loosen that contract. + """ + import subprocess + from loopx.capabilities.benchmark_toolkit.native_codex_goal import StdioNativeGoalTransport + + workspace = tmp_path / "work" + workspace.mkdir() + (workspace / "candidate.txt").write_text("retained-candidate\n") + requests = [] + + def response(request, index, message): + requests.append(request) + if index == 1: + message["content"][0]["text"] = "Retained research marker: retained-candidate." + elif index in {2, 3}: + command = ( + "python3 -c \"from pathlib import Path; import time; " + "Path('boundary-ready').write_text('ready'); time.sleep(30)\"" + if index == 2 else + "cat candidate.txt && printf 'continued-candidate\\n' > continued.txt" + ) + return {"type": "function_call", "id": f"fc_{index}", "call_id": f"call_{index}", + "name": "exec_command", "arguments": json.dumps({"cmd": command, + "yield_time_ms": 1000, "max_output_tokens": 200})} + else: + message["content"][0]["text"] = "Verified retained inputs and produced the next candidate." + return message + + @contextmanager + def native_host(env): + with StdioNativeGoalTransport.spawn( + [env["CODEX_BIN"], "-c", "features.goals=false", "app-server", "--listen", "stdio://"], + cwd=str(workspace), env=env, stderr=subprocess.DEVNULL, response_timeout_sec=15, + ) as native: + native.request("initialize", {"clientInfo": {"name": "handoff_fixture", "version": "0.1.0"}, + "capabilities": {"experimentalApi": True}}) + native.notify("initialized", {}) + yield native + + def history(native, session): + return native.request("thread/turns/list", {"threadId": session, + "sortDirection": "asc", "itemsView": "full", "limit": 100})["data"] + + def wait_terminal(native, session, turn, status): + deadline = time.monotonic() + 15 + while time.monotonic() < deadline: + rows = history(native, session) + if rows and rows[-1]["id"] == turn and rows[-1]["status"] == status: + return rows + time.sleep(.02) + raise AssertionError("Native handoff did not reach its expected terminal state") + + with real_native_source(tmp_path / "native", os.environ["LOOPX_NATIVE_CHECKPOINT_BIN"], + workspace, close_source=True, response_builder=response, + sandbox="workspace-write") as (env, source, _, _calls): + with native_host(env) as native: + native.request("thread/resume", {"threadId": source, "cwd": str(workspace), + "sandbox": "workspace-write", "approvalPolicy": "never", "persistExtendedHistory": True}) + turn = native.request("turn/start", {"threadId": source, "input": [ + {"type": "text", "text": "Reach the synthetic Host handoff boundary."}]})["turn"]["id"] + deadline = time.monotonic() + 10 + while not (workspace / "boundary-ready").exists() and time.monotonic() < deadline: + time.sleep(.01) + assert (workspace / "boundary-ready").exists() + native.request("turn/interrupt", {"threadId": source, "turnId": turn}) + retained = wait_terminal(native, source, turn, "interrupted") + assert len(requests) == 2 # No later model decision consumed the boundary. + with pytest.raises(ValueError, match="completed final cursor"): + native_session_checkpoint(env, source, timeout=15) + with native_host(env) as native: + fork = native.request("thread/fork", {"threadId": source, "cwd": str(workspace), + "lastTurnId": turn, "persistExtendedHistory": True})["thread"]["id"] + assert fork != source and history(native, fork) == retained + next_turn = native.request("turn/start", {"threadId": fork, "input": [ + {"type": "text", "text": "Verify the retained inputs and produce continued.txt."}]})["turn"]["id"] + wait_terminal(native, fork, next_turn, "completed") + assert (workspace / "continued.txt").read_text() == "continued-candidate\n" + assert "retained-candidate" in json.dumps(requests[2]) + assert history(native, source) == retained + + @pytest.mark.skipif(not os.environ.get("LOOPX_NATIVE_CHECKPOINT_BIN"), reason="Requires an explicit real native Host binary; no model service used") def test_real_native_workspace_readback_rejects_foreign_code_basis(tmp_path): From 5706ac51fd65d36e3402b8675a366a5b2311c4ab Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 15:43:11 +0800 Subject: [PATCH 44/69] fix(benchmark): preserve caller IDs across native checkpoint restore Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/README.md | 7 +++ benchmark/runtime/checkpoint.py | 13 ++++- benchmark/runtime/native_checkpoint.py | 14 ++++- .../test_research_checkpoint_capture.py | 58 +++++++++++++++++++ 4 files changed, 87 insertions(+), 5 deletions(-) diff --git a/benchmark/README.md b/benchmark/README.md index 75b51584f5..6ef3250748 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -420,6 +420,10 @@ Declare authorized workspace code/research files, effective configuration and installed source/profile pins as non-overlapping components. The native and opportunity participants do not imply that omitted state was saved. Every source remains private; inspect the verified inventory before qualifying a complete point. +Automatic native-store participants allocate unused artifact IDs without changing +caller IDs. The sealed `native_session.store_members` mapping binds those IDs to +the existing allowlisted native filenames. Reconstruction accepts earlier fixed-ID +artifacts, rejects duplicate/undeclared mappings and keeps source-history checks. The current Host hook cannot capture a new replan created **inside an already running model wake** before that decision. Those boundaries still need a qualified @@ -532,6 +536,9 @@ runtime receipt 绑定摘要；programmatic adapter 也可指定已放入 guest 失败后，修复声明不得把后来的状态补记成同一原点。评测参与者使用上文明确的 公开 SForge fork/pin 和 Python 3.13；普通 upstream 安装缺少该 seam 时明确 拒绝。Codex 原生会话的真实分页/compaction 与隔离 fork 须单独验证。 +自动原生存储参与者分配未占用的 artifact ID，保留调用者原 ID；封存的 +`native_session.store_members` 绑定这些 ID 与既有允许的原生文件名。恢复仍 +支持旧固定 ID，拒绝重复或未声明的映射，并保留源历史一致性校验。 显式 `"loopx_state": true` 复用现有 state-backup inventory，捕获单 Goal 试验的 注册状态、项目研究状态、runtime journal 和 registry；File/SQLite authority diff --git a/benchmark/runtime/checkpoint.py b/benchmark/runtime/checkpoint.py index 4714e10113..cf302ccfaf 100644 --- a/benchmark/runtime/checkpoint.py +++ b/benchmark/runtime/checkpoint.py @@ -399,8 +399,17 @@ def capture_after_wake(env: dict[str, str], receipt: dict[str, Any], *, metadata["session"] = {**session, "session_profile_digest": source_wake.digest} native = native_session_checkpoint(env, session.get("session_id", ""), timeout=max(.1, deadline - time.time())) - components = [*components, *native["components"]] - metadata["native_session"] = native["metadata"] + # Native names identify provider files, not reserved artifact IDs. + # Allocate in the same namespace as caller and other participants; + # retain the provider observation unchanged for the stability reread. + native_ids = {} + for entry in native["components"]: + component_id = _participant_id(components, entry["id"]) + native_ids[entry["id"]] = component_id + components = [*components, {**entry, "id": component_id}] + metadata["native_session"] = {**native["metadata"], "store_members": [ + {**member, "id": native_ids[member["id"]]} + for member in native["metadata"]["store_members"]]} probe = effect_runtime_result("runtime.checkpoint_artifacts", { "action": "probe", "components": components, "deadline_epoch_ms": deadline * 1000, diff --git a/benchmark/runtime/native_checkpoint.py b/benchmark/runtime/native_checkpoint.py index 9a1fafd0e0..77c15bad60 100644 --- a/benchmark/runtime/native_checkpoint.py +++ b/benchmark/runtime/native_checkpoint.py @@ -171,14 +171,22 @@ def _fork_native_checkpoint(env: dict[str, str], unit: Path, expected_sha256: st timeout=max(.1, deadline - time.monotonic())).strip() if version != native["host_version"]: raise ValueError("native checkpoint Host version differs") - names = set() + names, component_ids = set(), set() + declared_ids = {entry["id"] for entry in manifest["components"]} for member in native["store_members"]: - name = member["name"] - if (name in names or member["id"] != "native_" + name.replace(".", "_") or + if not isinstance(member, dict) or set(member) != {"id", "name"}: + raise ValueError("native checkpoint member inventory is invalid") + name, component_id = member["name"], member["id"] + # The sealed manifest binds allocated IDs to bytes; provider names + # still select only the existing allowlisted native-store destinations. + # Historical fixed IDs remain valid without reserving caller names. + if (not isinstance(name, str) or not isinstance(component_id, str) or + name in names or component_id in component_ids or component_id not in declared_ids or not (name in {"sessions", "archived_sessions", "session_index.jsonl"} or re.fullmatch(r"(?:state|thread_history)_[0-9]+\.sqlite", name))): raise ValueError("native checkpoint member inventory is invalid") names.add(name) + component_ids.add(component_id) basis = {"checkpoint_sha256": expected_sha256, "source_session_id": native["session_id"], "completed_turn_id": native["completed_turn_id"], "config_sha256": native["config_sha256"], "destination_goal_id": env.get("LOOPX_GOAL_ID"), "destination_agent_id": env.get("LOOPX_AGENT_ID"), diff --git a/tests/control_plane/test_research_checkpoint_capture.py b/tests/control_plane/test_research_checkpoint_capture.py index ecceaef437..13bc54856d 100644 --- a/tests/control_plane/test_research_checkpoint_capture.py +++ b/tests/control_plane/test_research_checkpoint_capture.py @@ -291,6 +291,64 @@ def between_read_and_probe(method, params, **kwargs): assert (destination / "components" / caller_id).read_bytes() == candidate.read_bytes() assert receipt["checkpoint"]["research_point_qualified"] is False + if not source_moves and caller_id.startswith("native_"): + # The source app-server is now closed. Reconstruct through the actual + # provider fork entry, retaining the original home for byte comparison. + # Fixed paths are swapped in this fixture; this is not guest isolation. + import shutil + from benchmark.runtime.native_checkpoint import fork_native_checkpoint + + native = manifest["point_metadata"]["native_session"] + assert all(member["id"] != caller_id for member in native["store_members"]) + config = tmp_path / "native-config.toml" + home = Path(env["CODEX_HOME"]) + config.write_bytes((home / "config.toml").read_bytes()) + retained_home = tmp_path / "retained-native-home" + shutil.move(home, retained_home) + def source_bytes(): + return {p.relative_to(retained_home).as_posix(): p.read_bytes() + for p in retained_home.rglob("*") if p.is_file()} + before = source_bytes() + if caller_id == "native_sessions": + import copy + # Even a correctly sealed artifact cannot select undeclared bytes, + # alias a participant, duplicate a destination or import credentials. + for fault in ("undeclared", "duplicate_id", "duplicate_name", "credential"): + bad_metadata = copy.deepcopy(manifest["point_metadata"]) + members = bad_metadata["native_session"]["store_members"] + if fault == "undeclared": + members[0]["id"] = "not_declared" + elif fault == "duplicate_id": + members[1]["id"] = members[0]["id"] + elif fault == "duplicate_name": + members[1]["name"] = members[0]["name"] + else: + members[0]["name"] = "auth.json" + bad_unit = tmp_path / fault + entries = [{"id": item["id"], "source_path": str(destination / "components" / item["id"])} + for item in manifest["components"]] + witness = effect_runtime_result("runtime.checkpoint_artifacts", {"action": "probe", "components": entries}) + bad = effect_runtime_result("runtime.checkpoint_artifacts", { + "action": "capture", "destination": str(bad_unit), "components": entries, + "point_metadata": bad_metadata, "expected_source_sha256": witness["source_sha256"]}, retry_safe=False) + with pytest.raises(ValueError, match="member inventory is invalid"): + fork_native_checkpoint(env, bad_unit, bad["sha256"], config, timeout=30) + assert not home.exists() and len(calls) == 1 + fork = fork_native_checkpoint(env, destination, receipt["checkpoint"]["sha256"], config, timeout=30) + assert fork["history_verified"] and fork["session_id"] != source + assert fork["visible_history_sha256"] == native["visible_history_sha256"] + assert source_bytes() == before and len(calls) == 1 + assert (destination / "components" / caller_id).read_bytes() == candidate.read_bytes() + + +@pytest.mark.skipif(not os.environ.get("LOOPX_NATIVE_CHECKPOINT_BIN"), reason="Requires a real native Host") +@pytest.mark.parametrize("caller_id", ["native_state_5_sqlite", "native_sessions"]) +@pytest.mark.parametrize("source_moves", [False, True]) +def test_native_store_participants_preserve_caller_ids_and_source_witness( + tmp_path, monkeypatch, caller_id, source_moves): + test_real_native_cursor_must_match_sealed_store_basis( + tmp_path, monkeypatch, source_moves, "probe", caller_id=caller_id) + @pytest.mark.skipif(not os.environ.get("LOOPX_NATIVE_CHECKPOINT_BIN"), reason="Requires a real native Host") def test_native_binding_participant_preserves_legal_caller_id(tmp_path, monkeypatch): From c2b77152a1b4d4df8d37bdf13f64599c7bd94b81 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 16:03:00 +0800 Subject: [PATCH 45/69] fix(benchmark): preserve evaluator checkpoint participant identities Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/README.md | 12 +++++- benchmark/runtime/checkpoint.py | 13 +++++-- .../test_research_checkpoint_capture.py | 38 +++++++++++++++---- 3 files changed, 51 insertions(+), 12 deletions(-) diff --git a/benchmark/README.md b/benchmark/README.md index 6ef3250748..f7fa31f218 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -318,8 +318,13 @@ Its default-off factory otherwise preserves the native routes. With reads that registration's released history and complete public result details, next round counters, accepted submission clock, remaining cooldown, task/native source identity and original trial deadline. It also preserves the native-selected -incumbent's opaque submission archive as `evaluation_incumbent`, separate from the -current working candidate. Native ranking chooses its original released round; +incumbent's opaque submission archive, separate from the current working candidate. +Automatic participants prefer `evaluation_opportunity` and `evaluation_incumbent` +and allocate unused IDs when a caller already owns either name. The sealed +`point_metadata.evaluation_opportunity` maps `component_id` and +`incumbent_component_id` to the retained evidence and archive; the latter is null +when no native incumbent exists. The source witness follows this same mapping. +Native ranking chooses its original released round; the result identity, archive digest and byte count bind its download to the same judge epoch and consumed-round state. Missing or moved selected bytes reject capture; no selected result is explicitly recorded as absent. The transport never @@ -536,6 +541,9 @@ runtime receipt 绑定摘要；programmatic adapter 也可指定已放入 guest 失败后，修复声明不得把后来的状态补记成同一原点。评测参与者使用上文明确的 公开 SForge fork/pin 和 Python 3.13；普通 upstream 安装缺少该 seam 时明确 拒绝。Codex 原生会话的真实分页/compaction 与隔离 fork 须单独验证。 +评测机会及 incumbent 参与者也分配未占用的 ID，并在封存的 +`point_metadata.evaluation_opportunity` 中记录实际映射；无 incumbent 时对应值为 null。 +文件 witness 校验按该映射定位，不会把同名调用者文件误当原生候选。 自动原生存储参与者分配未占用的 artifact ID，保留调用者原 ID；封存的 `native_session.store_members` 绑定这些 ID 与既有允许的原生文件名。恢复仍 支持旧固定 ID，拒绝重复或未声明的映射，并保留源历史一致性校验。 diff --git a/benchmark/runtime/checkpoint.py b/benchmark/runtime/checkpoint.py index cf302ccfaf..a44be553cc 100644 --- a/benchmark/runtime/checkpoint.py +++ b/benchmark/runtime/checkpoint.py @@ -375,9 +375,16 @@ def capture_after_wake(env: dict[str, str], receipt: dict[str, Any], *, opportunity = capture_opportunity(env, timeout=max(.1, deadline - time.time()), incumbent_path=incumbent) evidence.write_text(json.dumps(opportunity), encoding="utf-8") evidence.chmod(0o600) - components = [*components, {"id": "evaluation_opportunity", "source_path": str(evidence)}] + opportunity_id = _participant_id(components, "evaluation_opportunity") + components = [*components, {"id": opportunity_id, "source_path": str(evidence)}] + incumbent_id = None if opportunity["native_incumbent"] is not None: - components = [*components, {"id": "evaluation_incumbent", "source_path": str(incumbent)}] + incumbent_id = _participant_id(components, "evaluation_incumbent") + components = [*components, {"id": incumbent_id, "source_path": str(incumbent)}] + # Local provider evidence mapping, not evaluation admission. Keep + # caller IDs intact and bind the witness to the allocated archive. + metadata["evaluation_opportunity"] = {"component_id": opportunity_id, + "incumbent_component_id": incumbent_id} if request.get("native_session"): from .native_checkpoint import native_session_checkpoint from .codex import Execution @@ -445,7 +452,7 @@ def capture_after_wake(env: dict[str, str], receipt: dict[str, Any], *, raise ValueError("LoopX registry differs from the scoped participant inventory") if opportunity is not None and opportunity["native_incumbent"] is not None: selected = opportunity["native_incumbent"] - witness = probe["source_witness"].get("evaluation_incumbent", []) + witness = probe["source_witness"].get(metadata["evaluation_opportunity"]["incumbent_component_id"], []) if (len(witness) != 1 or witness[0].get("path") != "" or witness[0].get("kind") != "file" or witness[0].get("sha256") != selected["archive_sha256"] or witness[0].get("size") != selected["archive_bytes"]): diff --git a/tests/control_plane/test_research_checkpoint_capture.py b/tests/control_plane/test_research_checkpoint_capture.py index 13bc54856d..11080c2de1 100644 --- a/tests/control_plane/test_research_checkpoint_capture.py +++ b/tests/control_plane/test_research_checkpoint_capture.py @@ -127,8 +127,12 @@ def denied_read(path): assert not (project / "host-input.txt").exists() -@pytest.mark.parametrize("opportunity,corrupt", [(False, False), (True, False), (True, True)]) -def test_real_cli_settlement_capture_verification_and_inert_materialization(tmp_path, monkeypatch, opportunity, corrupt): +@pytest.mark.parametrize("opportunity,corrupt,collisions", [ + (False, False, False), (True, False, False), (True, True, False), + (True, False, True), (True, True, True), ("empty", False, True), +]) +def test_real_cli_settlement_capture_verification_and_inert_materialization( + tmp_path, monkeypatch, opportunity, corrupt, collisions): isolate_sqlite_runtime(tmp_path, monkeypatch) project, runtime, registry = cli._write_fixture(tmp_path) binding = ("--agent-id", cli.AGENT_ID, "--todo-id", cli.TODO_ID, "--turn-instance-id", cli.TURN_ID) @@ -145,10 +149,13 @@ def run(*args): incumbent_bytes = b"opaque older adopted candidate" opportunity_packet = {"meaning": "synthetic opportunity participant; no admission", "native_incumbent": { "round": "agent-1", "archive_sha256": hashlib.sha256(incumbent_bytes).hexdigest(), "archive_bytes": len(incumbent_bytes)}} + if opportunity == "empty": + opportunity_packet["native_incumbent"] = None def export(env, *, timeout, incumbent_path): assert 0 < timeout <= 30 observed.append(True) - incumbent_path.write_bytes(incumbent_bytes) + if opportunity_packet["native_incumbent"] is not None: + incumbent_path.write_bytes(incumbent_bytes) return opportunity_packet monkeypatch.setattr("benchmark.runtime.sforge_checkpoint.capture_opportunity", export) if corrupt: @@ -160,9 +167,16 @@ def altered_probe(method, params, **kwargs): (project / "solution.py").write_text("retained_candidate = 7\n") request = tmp_path / "request.json" target = logs / "checkpoints" / "point" + caller_components = [] + if collisions: + for name in ("evaluation_opportunity", "evaluation_opportunity_1", "evaluation_incumbent", "evaluation_incumbent_1"): + member = project / name + member.write_text("caller-owned " + name) + caller_components.append({"id": name, "source_path": str(member)}) request.write_text(json.dumps({"destination": str(target), "components": [ {"id": "code", "source_path": str(project / "solution.py")}, {"id": "research", "source_path": str(project / f".codex/goals/{cli.GOAL_ID}/ACTIVE_GOAL_STATE.md")}, + *caller_components, ], "point_metadata": {"meaning": "synthetic capture; no model/session or evaluator qualification"}, **({"evaluation_opportunity": True} if opportunity else {})})) env = {"LOOPX_CHECKPOINT_REQUEST": str(request), "LOOPX_WAKE_LOG_DIR": str(wake), @@ -195,8 +209,15 @@ def altered_probe(method, params, **kwargs): assert receipt["checkpoint"]["research_point_qualified"] is False assert goal_state.read_bytes() == retained_goal if opportunity: - assert json.loads((target / "components/evaluation_opportunity").read_text()) == opportunity_packet - assert (target / "components/evaluation_incumbent").read_bytes() == incumbent_bytes + mapping = json.loads((target / "checkpoint.json").read_text())["point_metadata"]["evaluation_opportunity"] + assert json.loads((target / "components" / mapping["component_id"]).read_text()) == opportunity_packet + if opportunity == "empty": + assert mapping["incumbent_component_id"] is None + assert not (wake / cli.TURN_ID / "checkpoint-incumbent.tar.gz").exists() + else: + assert (target / "components" / mapping["incumbent_component_id"]).read_bytes() == incumbent_bytes + for member in caller_components: + assert (target / "components" / member["id"]).read_text() == "caller-owned " + member["id"] digest = receipt["checkpoint"]["sha256"] restored = effect_runtime_result("runtime.checkpoint_artifacts", {"action": "materialize", "root": str(target), @@ -204,13 +225,16 @@ def altered_probe(method, params, **kwargs): assert restored["activation_performed"] is False assert (tmp_path / "inert/components/code").read_text() == "retained_candidate = 7\n" if opportunity: - assert (tmp_path / "inert/components/evaluation_incumbent").read_bytes() == incumbent_bytes + if opportunity != "empty": + assert (tmp_path / "inert/components" / mapping["incumbent_component_id"]).read_bytes() == incumbent_bytes + for member in caller_components: + assert (tmp_path / "inert/components" / member["id"]).read_text() == "caller-owned " + member["id"] before = (target / "checkpoint.json").read_bytes() capture_after_wake(env, receipt) assert receipt["checkpoint"]["reason"] == "destination_already_present_no_replay" assert (target / "checkpoint.json").read_bytes() == before assert goal_state.read_bytes() == retained_goal - assert len(observed) == int(opportunity) + assert len(observed) == int(bool(opportunity)) @pytest.mark.skipif(not os.environ.get("LOOPX_NATIVE_CHECKPOINT_BIN"), From 106ffe6a68840450c2610fb517d05f81624dd0d8 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 16:43:20 +0800 Subject: [PATCH 46/69] feat(benchmark): freeze owned Docker writers for inert checkpoint export Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/edgebench/checkpoint_controller.py | 21 +- benchmark/edgebench/checkpoint_worker.py | 160 +++++++++++++++ benchmark/tests/test_checkpoint_worker.py | 202 +++++++++++++++++++ 3 files changed, 379 insertions(+), 4 deletions(-) create mode 100644 benchmark/edgebench/checkpoint_worker.py create mode 100644 benchmark/tests/test_checkpoint_worker.py diff --git a/benchmark/edgebench/checkpoint_controller.py b/benchmark/edgebench/checkpoint_controller.py index 205b6252f0..3366c2efad 100644 --- a/benchmark/edgebench/checkpoint_controller.py +++ b/benchmark/edgebench/checkpoint_controller.py @@ -293,6 +293,12 @@ def main(argv=None) -> None: capture_parser.add_argument("--destination", required=True, type=Path) capture_parser.add_argument("--phase-deadline-file", type=Path, help="Explicit physical SForge Host phase file; omitted keeps native-only capture") + worker_parser = commands.add_parser("capture-worker", + help="Controller-owned Docker pause/archive plus native capsule; inert files, not process restoration") + worker_parser.add_argument("--token-file", required=True, type=Path) + worker_parser.add_argument("--capture-id", required=True) + worker_parser.add_argument("--destination", required=True, type=Path) + worker_parser.add_argument("--worker-request", required=True, type=Path) restore_parser = commands.add_parser("restore") restore_parser.add_argument("--root", required=True, type=Path) restore_parser.add_argument("--expected-sha256", required=True) @@ -301,11 +307,18 @@ def main(argv=None) -> None: restore_parser.add_argument("--phase-deadline-file", type=Path, help="Fresh physical Host phase file, required for a capsule containing that participant") args = parser.parse_args(argv) - if args.command == "capture": + if args.command in {"capture", "capture-worker"}: source_token = _physical(args.token_file).read_text().strip() - result = capture(args.controller_file, token=source_token, - capture_id=args.capture_id, destination=args.destination, - phase_deadline_file=args.phase_deadline_file) + if args.command == "capture-worker": + from benchmark.edgebench.checkpoint_worker import capture_worker + + result = capture_worker(args.controller_file, token=source_token, + capture_id=args.capture_id, destination=args.destination, + request_file=args.worker_request) + else: + result = capture(args.controller_file, token=source_token, + capture_id=args.capture_id, destination=args.destination, + phase_deadline_file=args.phase_deadline_file) print(json.dumps(result)) else: # Reserve before admission. A lost output is explicit recovery, never a diff --git a/benchmark/edgebench/checkpoint_worker.py b/benchmark/edgebench/checkpoint_worker.py new file mode 100644 index 0000000000..b23d4e78b8 --- /dev/null +++ b/benchmark/edgebench/checkpoint_worker.py @@ -0,0 +1,160 @@ +"""Explicit controller-owned Docker writer barrier for inert checkpoint export. + +This provider adapts Docker pause/archive; the existing native and typed artifact +owners still own evaluator state and sealing. No code runs inside a frozen worker. +It does not capture process memory or qualify a native conversation cursor. +""" +from __future__ import annotations + +import json +import math +import re +import tarfile +import tempfile +from pathlib import Path, PurePosixPath + +from loopx.control_plane.effect_runtime import effect_runtime_result +from loopx.file_lock import exclusive_file_lock +from benchmark.runtime.checkpoint import _participant_id + +from .checkpoint_controller import _controller, _physical, capture + + +def _request(path: Path) -> dict: + value = json.loads(_physical(path).read_bytes()) + if (not isinstance(value, dict) or + set(value) != {"container_id", "components", "phase_deadline_path"} or + not re.fullmatch(r"[0-9a-f]{64}", value.get("container_id", "")) or + not isinstance(value["components"], list) or not value["components"]): + raise ValueError("Worker request requires a full container ID, explicit components and phase_deadline_path") + ids = set() + paths = [] + for entry in value["components"]: + if (not isinstance(entry, dict) or set(entry) != {"id", "source_path"} or + not isinstance(entry["id"], str) or + not re.fullmatch(r"[A-Za-z][A-Za-z0-9_.-]{0,63}", entry["id"]) or entry["id"] in ids): + raise ValueError("Worker components require unique IDs") + ids.add(entry["id"]) + paths.append(_guest_path(entry["source_path"])) + paths.append(_guest_path(value["phase_deadline_path"])) + if any(a == b or a in b.parents or b in a.parents + for index, a in enumerate(paths) for b in paths[index + 1:]): + raise ValueError("Worker components and phase clock must not overlap") + return value + + +def _guest_path(value: str) -> PurePosixPath: + if not isinstance(value, str): + raise ValueError("Worker paths must be normalized absolute paths below root") + path = PurePosixPath(value) + if not path.is_absolute() or str(path) != value or value == "/" or ".." in path.parts: + raise ValueError("Worker paths must be normalized absolute paths below root") + return path + + +def _archive(container, remote: str, destination: Path) -> Path: + """Read through the daemon, rejecting links/devices and archive traversal. + + Docker's archive reader works while paused, unlike exec-based file readers. + Keep only the declared object, not parent filesystem or process credentials. + """ + stream, stat = container.get_archive(remote) + if stat.get("linkTarget"): + raise ValueError("Worker component must not be a symlink") + destination.mkdir(mode=0o700) + with tempfile.TemporaryFile() as raw: + for chunk in stream: + raw.write(chunk) + raw.seek(0) + with tarfile.open(fileobj=raw) as archive: + members = archive.getmembers() + root = PurePosixPath(remote).name + names = set() + for member in members: + path = PurePosixPath(member.name) + if (path.is_absolute() or ".." in path.parts or not path.parts or + path.parts[0] != root or str(path) in names or + not (member.isfile() or member.isdir())): + raise ValueError("Worker archive contains undeclared, linked or special members") + names.add(str(path)) + archive.extractall(destination, members=members, filter="data") + return _physical(destination / root) + + +def capture_worker(controller_file: Path, *, token: str, capture_id: str, + destination: Path, request_file: Path, timeout: float = 30) -> dict: + """Freeze one explicitly owned worker and seal its files with native state. + + A fresh identity is required. Failed or uncertain attempts never recapture a + later worker into an earlier evaluator point. The original worker is resumed + before publishing the sealed result; an unsuccessful unpause is an error. + """ + import docker + + if not math.isfinite(timeout) or timeout <= 0: + raise ValueError("Worker capture timeout must be finite and positive") + request = _request(request_file) + controller = _controller(controller_file) + destination = Path(destination) + _physical(destination.parent) + if (destination.resolve() != destination or destination.exists() or destination.is_symlink() or + not re.fullmatch(r"[0-9a-f]{32}", capture_id)): + raise ValueError("Worker capture requires a fresh physical destination and 32-hex identity") + if (controller["bank_root"] / capture_id).exists(): + raise ValueError("Existing native point cannot acquire later worker files") + attempts = controller["bank_root"] / ".loopx-worker-attempts" + attempts.mkdir(exist_ok=True, mode=0o700) + _physical(attempts) + (attempts / capture_id).mkdir(mode=0o700) # First attempt, including failures, wins. + client = docker.from_env(timeout=timeout) + try: + with exclusive_file_lock(attempts / request["container_id"], timeout_seconds=timeout, + operation="checkpoint_worker_capture"): + container = client.containers.get(request["container_id"]) + container.reload() + state = container.attrs["State"] + if container.id != request["container_id"] or not state["Running"] or state["Paused"]: + raise ValueError("Worker must be the exact running, unpaused container owned by this controller") + with tempfile.TemporaryDirectory(prefix=".worker-capture-", dir=destination.parent) as directory: + stage = Path(directory) + (stage / "workers").mkdir(mode=0o700) + components = [] + try: + # The cleanup also covers a lost reply after Docker committed pause. + container.pause() + container.reload() + if not container.attrs["State"]["Paused"]: + raise RuntimeError("Worker writer barrier was not established") + for entry in request["components"]: + copied = _archive(container, entry["source_path"], stage / "workers" / entry["id"]) + components.append({"id": entry["id"], "source_path": str(copied)}) + phase = _archive(container, request["phase_deadline_path"], stage / "phase") + native = capture(controller_file, token=token, capture_id=capture_id, + destination=stage / "native", timeout=timeout, phase_deadline_file=phase) + container.reload() + if not container.attrs["State"]["Running"] or not container.attrs["State"]["Paused"]: + raise RuntimeError("Worker writer barrier changed during capture") + finally: + container.reload() + if container.attrs["State"]["Paused"]: + container.unpause() + container.reload() + if container.attrs["State"]["Paused"] or not container.attrs["State"]["Running"]: + raise RuntimeError("Worker was not restored to running; inspect the original container before retry") + native_component = _participant_id(components, "native_capsule") + components.append({"id": native_component, "source_path": str(stage / "native")}) + metadata = {"worker_file_snapshot": { + "container_id": container.id, "image_id": container.attrs["Image"], + "components": request["components"], "phase_deadline_path": request["phase_deadline_path"], + "native_capsule_sha256": native["sha256"], + "native_capsule_component_id": native_component, + "writer_barrier": "docker_pause", "process_memory_captured": False, + "native_cursor_qualified": False, + }} + probe = effect_runtime_result("runtime.checkpoint_artifacts", {"action": "probe", "components": components}) + return effect_runtime_result("runtime.checkpoint_artifacts", { + "action": "capture", "components": components, "destination": str(destination), + "expected_source_sha256": probe["source_sha256"], "point_metadata": metadata, + }, retry_safe=False) + finally: + client.close() diff --git a/benchmark/tests/test_checkpoint_worker.py b/benchmark/tests/test_checkpoint_worker.py new file mode 100644 index 0000000000..e8cc09ec09 --- /dev/null +++ b/benchmark/tests/test_checkpoint_worker.py @@ -0,0 +1,202 @@ +"""Real Docker writer barrier plus native HTTP capture; no model inference.""" +from __future__ import annotations + +import io +import json +import os +import tarfile +import threading +import time +import uuid + +import pytest + +from benchmark.edgebench import checkpoint_controller as transport +from benchmark.edgebench.checkpoint_worker import capture_worker +from benchmark.tests import test_checkpoint_controller as native_fixtures +from benchmark.tests.test_checkpoint_controller import register, submit, wait + + +controllers = native_fixtures.controllers + + +pytestmark = pytest.mark.skipif(os.environ.get("LOOPX_EDGEBENCH_DOCKER_SMOKE") != "1", + reason="Requires explicit isolated Docker writer-barrier qualification") + + +@pytest.fixture +def writer(tmp_path): + import docker + + client = docker.from_env(timeout=15) + image = os.environ.get("LOOPX_CHECKPOINT_WRITER_IMAGE", "python:3.12-slim") + client.images.get(image) # Never pull implicitly for this test. + script = """from pathlib import Path +import time +Path('/data').mkdir() +Path('/phase').write_text(str(int(time.time())+7200)) +Path('/link').symlink_to('/data', target_is_directory=True) +count=0 +while True: + count+=1 + Path('/data/next').write_text(str(count)) + Path('/data/next').replace('/data/counter') + time.sleep(.03) +""" + container = client.containers.run(image, ["python", "-c", script], detach=True, + name="loopx-checkpoint-writer-" + uuid.uuid4().hex[:12], + network_disabled=True, mem_limit="64m", nano_cpus=250_000_000) + request = tmp_path / "worker.json" + request.write_text(json.dumps({"container_id": container.id, + "components": [{"id": "native_capsule", "source_path": "/data"}], + "phase_deadline_path": "/phase"})) + try: + wait(lambda: container.exec_run(["test", "-f", "/data/counter"]).exit_code == 0) + yield container, request + finally: + container.remove(force=True) + assert not client.containers.list(all=True, filters={"id": container.id}) + client.close() + + +def counter(container): + stream, _ = container.get_archive("/data/counter") + with tarfile.open(fileobj=io.BytesIO(b"".join(stream))) as archive: + return int(archive.extractfile("counter").read()) + + +def test_controller_cli_captures_frozen_worker_with_original_native_point( + controllers, writer, tmp_path, monkeypatch, capsys): + source, destination, first, second, files, grades = controllers + container, request = writer + token = register(source, first, max_agent_submissions=3, submission_cooldown=3600) + assert submit(first, token, 7).status_code == 200 + observed = [] + original = transport._post + def while_frozen(*args, **kwargs): + container.reload() + assert container.attrs["State"]["Paused"] + before = counter(container) + result = original(*args, **kwargs) + time.sleep(.15) + assert counter(container) == before + observed.append(before) + return result + monkeypatch.setattr(transport, "_post", while_frozen) + token_file = tmp_path / "token" + token_file.write_text(token) + unit = tmp_path / "unit" + transport.main(["--controller-file", str(files[0]), "capture-worker", + "--token-file", str(token_file), "--capture-id", "7" * 32, + "--destination", str(unit), "--worker-request", str(request)]) + receipt = json.loads(capsys.readouterr().out) + assert receipt["research_point_qualified"] is False + manifest = json.loads((unit / "checkpoint.json").read_bytes()) + metadata = manifest["point_metadata"]["worker_file_snapshot"] + assert metadata["native_cursor_qualified"] is False and metadata["process_memory_captured"] is False + assert metadata["native_capsule_component_id"] == "native_capsule_1" + assert int((unit / "components/native_capsule/counter").read_text()) == observed[0] + nested = unit / "components/native_capsule_1" + capsule = json.loads((nested / "checkpoint.json").read_bytes()) + native = capsule["point_metadata"]["native_reference"] + assert native["opportunity"]["next_agent"] == 2 and native["incumbent"]["round"] == "agent-1" + assert 3590 < native["opportunity"]["remaining_cooldown_seconds"] <= 3600 + assert 7190 < capsule["point_metadata"]["host_phase_budget"]["remaining_seconds"] <= 7200 + container.reload() + assert not container.attrs["State"]["Paused"] + wait(lambda: counter(container) > observed[0]) + assert int((unit / "components/native_capsule/counter").read_text()) == observed[0] + for secret in (token, source.state.judge.admin_secret): + assert all(secret.encode() not in p.read_bytes() for p in unit.rglob("*") if p.is_file()) + # This is an inert joint file export, not a shortcut to activation. + with pytest.raises(ValueError, match="declared capsule"): + transport.restore(files[1], root=unit, expected_sha256=receipt["sha256"], run_id="branch") + with pytest.raises(ValueError, match="later worker"): + capture_worker(files[0], token=token, capture_id="7" * 32, + destination=tmp_path / "later", request_file=request) + assert not (tmp_path / "later").exists() and destination.state.judge.tokens == {} and len(grades) == 1 + + +@pytest.mark.parametrize("fault", ["missing", "symlink", "judge", "pause_reply", "prepaused", "cancel", "unpause"]) +def test_worker_capture_failure_releases_only_its_barrier( + controllers, writer, tmp_path, monkeypatch, fault): + source, _, first, _, files, _ = controllers + container, request = writer + token = register(source, first) + from docker.errors import NotFound + + expected_error = {"missing": NotFound, "symlink": ValueError, "judge": ValueError, + "pause_reply": OSError, "prepaused": ValueError, "cancel": KeyboardInterrupt, + "unpause": OSError}[fault] + if fault in {"missing", "symlink"}: + body = json.loads(request.read_bytes()) + body["components"][0]["source_path"] = "/missing" if fault == "missing" else "/link" + request.write_text(json.dumps(body)) + elif fault == "judge": + body = json.loads(files[0].read_bytes()) + body["admin_secret"] = "wrong-synthetic-controller" + files[0].write_text(json.dumps(body)) + elif fault == "pause_reply": + original = type(container).pause + def lost_reply(self): + original(self) + raise OSError("injected lost pause reply") + monkeypatch.setattr(type(container), "pause", lost_reply) + elif fault == "prepaused": + container.pause() + elif fault == "cancel": + def cancel(*args, **kwargs): + raise KeyboardInterrupt("injected controller cancellation") + monkeypatch.setattr(transport, "_post", cancel) + else: + def unavailable(self): + raise OSError("injected daemon unpause failure") + monkeypatch.setattr(type(container), "unpause", unavailable) + before = counter(container) + with pytest.raises(expected_error): + capture_worker(files[0], token=token, capture_id="8" * 32, + destination=tmp_path / "unit", request_file=request) + assert not (tmp_path / "unit").exists() + container.reload() + assert container.attrs["State"]["Paused"] == (fault in {"prepaused", "unpause"}) + if fault not in {"prepaused", "unpause"}: + wait(lambda: counter(container) > before) + # No failed attempt is silently replaced by a later research state. + with pytest.raises((FileExistsError, ValueError)): + capture_worker(files[0], token=token, capture_id="8" * 32, + destination=tmp_path / "later", request_file=request) + assert not (tmp_path / "later").exists() + + +def test_concurrent_capture_does_not_release_another_operation_barrier( + controllers, writer, tmp_path, monkeypatch): + from concurrent.futures import ThreadPoolExecutor + from loopx.file_lock import LockAcquireTimeoutError + + source, _, first, _, files, _ = controllers + container, request = writer + token = register(source, first) + entered, release = threading.Event(), threading.Event() + original = transport._post + def held(*args, **kwargs): + entered.set() + assert release.wait(15) + return original(*args, **kwargs) + monkeypatch.setattr(transport, "_post", held) + with ThreadPoolExecutor(max_workers=1) as pool: + pending = pool.submit(capture_worker, files[0], token=token, capture_id="1" * 32, + destination=tmp_path / "first", request_file=request) + try: + assert entered.wait(15) + frozen = counter(container) + with pytest.raises(LockAcquireTimeoutError): + capture_worker(files[0], token=token, capture_id="2" * 32, + destination=tmp_path / "second", request_file=request, timeout=.5) + container.reload() + assert container.attrs["State"]["Paused"] and counter(container) == frozen + assert not (tmp_path / "second").exists() + finally: + release.set() + assert pending.result(timeout=20)["research_point_qualified"] is False + container.reload() + assert not container.attrs["State"]["Paused"] From 48af2b53a0fd247dd553c68d47feab2e88158a6c Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 16:43:28 +0800 Subject: [PATCH 47/69] docs(benchmark): delimit Docker capture and remaining continuation work Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/README.md | 32 +++++++++++++++++++ .../hierarchical-agent-stride-control-v0.md | 10 ++++++ ...rarchical-agent-stride-control-v0.zh-CN.md | 7 ++++ 3 files changed, 49 insertions(+) diff --git a/benchmark/README.md b/benchmark/README.md index f7fa31f218..50f874e4b6 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -480,6 +480,38 @@ fail without a fresh evaluation. Redirects cannot forward the admin secret. After an uncertain restore response, inspect native readback with the controller; do not switch run identities or perform ordinary fresh registration as a retry. +For an explicitly owned Docker worker, `capture-worker` can additionally freeze +its writers while copying declared files and capturing the native capsule. Supply +`--worker-request` with a physical JSON file containing the full `container_id`, +`components: [{"id": "code", "source_path": "/task/code"}]`, and the separate +`phase_deadline_path` inside that container. Use the same controller, token-file, +capture-id and destination arguments as `capture`; no phase file is read through +an executing container process. The controller uses the Docker daemon's +pause/archive/unpause path, then seals the copied files and nested native capsule +with the existing typed artifact owner. Component IDs are preserved when they +collide with the automatic native capsule name. Neither Docker access nor judge +administration is passed into the solver. Omit this command to retain ordinary +capture behavior; there is no automatic activation or retention policy. + +The container must initially be running and unpaused. Cancellation, failed file +reads, rejected native capture and a lost pause response attempt to release only +this operation's barrier. Failure to resume the worker is reported and publishes +no outer artifact. An existing pause is never released. The first attempt owns +the capture identity; a failure cannot silently pair later files with an older +native point. Missing paths, archive links, traversal and special members reject. +Declare only the intended private code/state stores: a whole home can contain +credentials, and neither archive filtering nor a freeze is privacy certification. + +This is an **inert joint file snapshot**, still `research_point_qualified=false`. +It does not snapshot process memory, qualify a pending/interrupted Codex cursor, +verify externally hosted writers or activate canonical research state. Ordinary +native restore rejects this outer unit. Worker/evaluator pairing is explicitly +caller-declared here, not a verified automatic SForge run binding. A real Docker writer plus native HTTP +test verifies unchanged file bytes throughout evaluator capture, retained used +rounds/cooldown/phase, source resumption and failure cleanup. Automatic quota +handoff before the replan decision and isolated useful continuation remain the +next integration boundary; this command alone does not qualify experiment restart. + An explicit **SForge phase-clock participant** preserves a declared Host file's remaining duration alongside native opportunities. Add `--phase-deadline-file /private/source-phase-deadline` to capture, and diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md index 7f8ea89448..88b5d74e7a 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md @@ -777,6 +777,16 @@ graph/vision/effect activation, full native/Host/evaluator consistency, joint participant recovery, packaged delivery and useful continued research remain open. Participant/effect reconciliation and complete fresh research-state adoption cannot be certified by operator metadata or native history equality. +The explicit Docker controller now offers a bounded writer barrier: pause the +owned running worker, archive declared files and its phase clock through the +daemon, capture the native evaluator capsule, then unpause before publishing an +inert joint unit. A live-writer/native-HTTP qualification proves file stability +and original opportunity retention; cancellation, lost pause responses and failed +capture exercise cleanup. An existing pause is not owned or released, and failed +resume prevents publication. This closes the manual provider's file-writer +barrier gap only. No process memory, native cursor eligibility, external writers, +automatic pre-decision handoff or isolated continuation are qualified by it. + A native fresh registration is not restored admission. In the controlled native Docker qualification, one of twelve source submissions was consumed and its cooldown remained active. A same-limit fresh registration admitted `agent-1` diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md index 6c8e4c5f92..331804c97a 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md @@ -659,6 +659,13 @@ binding 已提交但控制器响应丢失时，重复原命令会核对原凭据 确定性测试覆盖这一边界，并验证缺失、多结果和外来目标拒绝；evaluator/Host 联合激活恢复仍未资格化。 +显式 Docker controller 现提供有界的文件写入屏障：暂停自己拥有的运行中 worker， +经 daemon archive 复制声明文件及 phase 时钟，捕获原生 evaluator capsule，恢复 +worker 后再发布惰性联合产物。真实持续写入容器与 native HTTP 验证了文件稳定、 +原机会保留，以及取消、暂停响应丢失和捕获失败后的清理。已有暂停不会被接管或 +释放；恢复失败不发布产物。此项仅关闭手动 provider 的文件写入屏障缺口，未 +资格化进程内存、native cursor、外部 writer、自动策略决策前交接或隔离接续。 + 原生新注册不等于恢复准入。真实原生 Docker 资格测试中，source 已消耗十二次 中的一次，且冷却仍有效；同额度的新注册却立即受理 `agent-1`，保留十二次未用 机会。重评已保存 archive 产生新的结果，没有恢复 source 的历史或机会状态。 From ec2e815e6f61d6fb4c7977acfad460adbed9ad40 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 17:06:40 +0800 Subject: [PATCH 48/69] test(checkpoint): characterize live native fork handoff Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- .../tests/test_native_checkpoint_handoff.py | 114 ++++++++++++++++++ .../hierarchical-agent-stride-control-v0.md | 14 +++ ...rarchical-agent-stride-control-v0.zh-CN.md | 10 ++ 3 files changed, 138 insertions(+) create mode 100644 benchmark/tests/test_native_checkpoint_handoff.py diff --git a/benchmark/tests/test_native_checkpoint_handoff.py b/benchmark/tests/test_native_checkpoint_handoff.py new file mode 100644 index 0000000000..b84cefd214 --- /dev/null +++ b/benchmark/tests/test_native_checkpoint_handoff.py @@ -0,0 +1,114 @@ +"""Characterize live native forks before choosing a midwake capture transport. + +A local deterministic provider replaces inference; the actual Host owns fork, +history and tool execution. This does not qualify a joint research checkpoint. +""" +import json +import os +import shlex +import subprocess +import time + +import pytest + +from benchmark.runtime.native_checkpoint import native_session_checkpoint +from benchmark.tests.test_native_checkpoint import real_native_source +from loopx.capabilities.benchmark_toolkit.native_codex_goal import ( + NativeGoalProtocolError, StdioNativeGoalTransport, +) + + +@pytest.mark.skipif(not os.environ.get("LOOPX_NATIVE_CHECKPOINT_BIN"), + reason="Requires an explicit real native Host; no model service used") +def test_live_fork_preserves_context_without_quiescing_source(tmp_path): + source_work, branch_work = tmp_path / "source", tmp_path / "branch" + source_work.mkdir() + branch_work.mkdir() + requests = [] + # A bounded tool withholds its output until the test Host releases it. + # This models transport ordering only, not a LoopX guard or writer barrier. + code = """from pathlib import Path +import time +Path('waiting').write_text('ready') +deadline = time.monotonic() + 20 +while not Path('release').exists() and time.monotonic() < deadline: + time.sleep(.01) +assert Path('release').exists(), 'fixture release missing' +Path('source-result').write_text('one original execution') +print('WITHHELD_RESULT') +""" + + def response(request, index, message): + requests.append(request) + if index == 2: + return {"type": "function_call", "id": "fc_boundary", "call_id": "call_boundary", + "name": "exec_command", "arguments": json.dumps({ + "cmd": "python3 -c " + shlex.quote(code), + "yield_time_ms": 10000, "max_output_tokens": 100})} + message["content"][0]["text"] = "RETAINED_RESEARCH" if index == 1 else "Continued." + return message + + def history(host, session): + return host.request("thread/turns/list", {"threadId": session, + "sortDirection": "asc", "itemsView": "full", "limit": 100})["data"] + + def terminal(host, session, turn): + deadline = time.monotonic() + 10 + while time.monotonic() < deadline: + rows = history(host, session) + if rows and rows[-1]["id"] == turn and rows[-1]["status"] == "completed": + return rows + time.sleep(.02) + raise AssertionError("Native fixture did not complete") + + binary = os.environ["LOOPX_NATIVE_CHECKPOINT_BIN"] + with real_native_source(tmp_path / "native", binary, source_work, close_source=True, + response_builder=response, sandbox="workspace-write") as (env, source, _, calls): + with StdioNativeGoalTransport.spawn([binary, "app-server", "--listen", "stdio://"], + cwd=str(source_work), env=env, stderr=subprocess.DEVNULL, response_timeout_sec=15) as host: + host.request("initialize", {"clientInfo": {"name": "handoff_fixture", "version": "0.1"}, + "capabilities": {"experimentalApi": True}}) + host.notify("initialized", {}) + host.request("thread/resume", {"threadId": source, "cwd": str(source_work), + "sandbox": "workspace-write", "approvalPolicy": "never", "persistExtendedHistory": True}) + turn = host.request("turn/start", {"threadId": source, "input": [ + {"type": "text", "text": "Reach the synthetic withheld-output boundary."}]})["turn"]["id"] + try: + deadline = time.monotonic() + 10 + while not (source_work / "waiting").exists() and time.monotonic() < deadline: + time.sleep(.01) + assert (source_work / "waiting").exists() and len(calls) == 2 + source_history = history(host, source) + assert source_history[-1]["status"] == "inProgress" + with pytest.raises(NativeGoalProtocolError, match="thread/fork"): + host.request("thread/fork", {"threadId": source, "cwd": str(branch_work), + "lastTurnId": turn, "persistExtendedHistory": True}) + fork = host.request("thread/fork", {"threadId": source, "cwd": str(branch_work), + "persistExtendedHistory": True})["thread"]["id"] + fork_history = history(host, fork) + assert fork != source and len(calls) == 2 + assert fork_history[:-1] == source_history[:-1] + assert fork_history[-1]["id"] == turn and fork_history[-1]["status"] == "interrupted" + assert history(host, source)[-1]["status"] == "inProgress" + # Native fork success grants neither a complete cursor nor a + # writer barrier. Keep the production capture rejection intact. + with pytest.raises(ValueError, match="completed final cursor"): + native_session_checkpoint({**env, "LOOPX_PROJECT": str(branch_work)}, fork, timeout=15) + branch_turn = host.request("turn/start", {"threadId": fork, "input": [ + {"type": "text", "text": "Fresh fixture admission: retain context; do not replay the source tool."}]})["turn"]["id"] + terminal(host, fork, branch_turn) + assert len(calls) == 3 + items = requests[2]["input"] + assert "RETAINED_RESEARCH" in json.dumps(items) + assert any(item.get("type") == "function_call_output" and + item.get("call_id") == "call_boundary" and item.get("output") == "aborted" + for item in items) + assert history(host, source)[-1]["status"] == "inProgress" + assert not (source_work / "source-result").exists() + assert not (branch_work / "waiting").exists() # No automatic tool replay. + finally: + (source_work / "release").write_text("release original tool") + terminal(host, source, turn) + assert (source_work / "source-result").read_text() == "one original execution" + assert not (branch_work / "source-result").exists() + assert len(calls) == 4 and "WITHHELD_RESULT" in json.dumps(requests[3]) diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md index 88b5d74e7a..8286470a97 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md @@ -706,6 +706,20 @@ insufficient. Compare the reconstructed prefixes and excluded later history, not just returned session identifiers. Keep provider-specific migration and storage handling outside the generic consistency decision. +Live native forks need a different qualification from completed-cursor forks. +The real Codex `0.162.0-alpha.2` characterization in +`benchmark/tests/test_native_checkpoint_handoff.py` rejects an explicit +`lastTurnId` while that turn is active. Omitting it can retain the current prefix +as an interrupted branch while the source tool keeps running. Resuming that +branch exposes an aborted tool result; it does not replay the tool automatically. +This avoids requiring a source interruption merely to investigate a fork, but +neither the returned branch nor its history proves source quiescence. The next +joint handoff must bind the withheld original guard and pending tool identity, +reconcile any admitted effects, and verify frozen participant bytes before +qualifying that cursor. Keep the completed-final-cursor production check until +this separate path is implemented and qualified; do not silently drop the active +turn or classify every interrupted session as resumable research state. + The Host profile must also declare path portability. A native index can retain absolute guest paths: qualification must either preserve those paths inside a fresh isolated environment or use the Host's supported import/rebinding API. diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md index 331804c97a..854566f4bb 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md @@ -594,6 +594,16 @@ Host adapter 须在固定版本验证原生历史重建，包括 compaction 和 任意文件路径或日志解析成功并不充分。比较重建的历史前缀及被排除的后续历史， 不只检查返回了 session id。Provider 特有迁移与存储处理不进入通用一致性决策。 +运行中会话的原生 fork 须与已完成 cursor 分别资格化。 +`benchmark/tests/test_native_checkpoint_handoff.py` 在真实 Codex +`0.162.0-alpha.2` 上确认：显式 `lastTurnId` 不能指向 active turn；省略后可保留 +当前前缀为 interrupted 分支，而来源工具仍运行。接续分支会看到 aborted 工具 +结果，不会自动重放工具。因此研究 fork 路径不必先中断来源，但返回分支或历史 +都不能证明来源已静止。下一步联合 handoff 须绑定尚未返回的原 guard 与 pending +工具身份、核回已受理效果，并验证冻结的参与者字节，再资格化该 cursor。 +在该独立路径实现并验证前保留生产代码的 completed-final-cursor 检查；不能 +静默丢弃 active turn，也不能把所有 interrupted 会话视为可接续研究状态。 + Host profile 还须声明路径可移植性。原生索引可能保留绝对 guest 路径：资格化 须在新隔离环境保留这些路径，或使用 Host 支持的 import/rebinding API；手改 原生数据库行不能资格化导入。有效 Host 配置及已安装工具/skill 版本须与凭证 From 490261387cf09d91bc691096dc21de44d9d5194f Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 17:31:03 +0800 Subject: [PATCH 49/69] feat(checkpoint): hold original quota output for Host capture Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/README.md | 30 ++++++ benchmark/runtime/checkpoint.py | 41 +++++++ .../tests/test_native_checkpoint_handoff.py | 92 ++++++++++++++++ .../hierarchical-agent-stride-control-v0.md | 10 ++ ...rarchical-agent-stride-control-v0.zh-CN.md | 7 ++ loopx/cli_commands/quota.py | 5 +- loopx/cli_commands/quota_capture.py | 58 ++++++++++ loopx/cli_commands/quota_registration.py | 6 ++ .../test_quota_decision_capture.py | 102 ++++++++++++++++++ 9 files changed, 350 insertions(+), 1 deletion(-) diff --git a/benchmark/README.md b/benchmark/README.md index 50f874e4b6..eb13145ed1 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -185,6 +185,36 @@ If that guard projects a fresh replan, the missed first attempt is retained even without readable declaration bytes, so repairing the file cannot capture later state under the same obligation. Subsequent distinct replans remain eligible. +A Host can now explicitly hold a **mid-wake quota response** using +`quota should-run ... --decision-output-root CAPTURE_ROOT +--decision-handoff-socket SOCKET`. The socket must already be listening in a +private physical directory owned by the same user (Unix only). After writing the +complete original decision, the CLI sends one newline-delimited JSON object +`{"path": ".../decision.json", "sha256": "..."}` and withholds successful output +until it receives exactly `{"released": true, "sha256": "..."}` for those bytes. +The existing selection command preserves both transport options. Ordinary CLI +output and admission remain unchanged when the option is absent; this option +does not enable TurnEnvelope. + +`benchmark.runtime.checkpoint.held_quota_decision` is the matching local Host +context: it validates the declared capture root/digest and asks the existing +TypeScript replan selector for capture intent, then releases the original +response on normal context exit. A Host exception, disconnected/lost reply, +changed capture or the 120-second transport deadline fails the call while +retaining the original admission and captured decision. Recover that original +identity; do not obtain a replacement guard to reconstruct a missed point. + +This transport is a prerequisite, **not automatic mid-wake checkpoint capture**. +The actual native Host/CLI test binds the pending command event, forks while the +real guard output is withheld, and verifies no duplicate guard receipt. It does +not pause writers, reconcile branch effects or certify an interrupted cursor. +Native tools can yield control before their subprocess exits; withholding stdout +alone is not a model or writer barrier. The production worker still needs the +Host rendezvous connected to its controller-owned writer barrier, canonical state +adoption and isolated useful continuation before this profile can launch research +runs. Omit the option to disable the transport; no socket, hold or extra read is +then performed. + Add `"native_session": true` to include Codex's completed native cursor, all retained history pages, rollout/compaction records and native index/history SQLite stores. The adapter verifies the native session's actual workspace, reads its full diff --git a/benchmark/runtime/checkpoint.py b/benchmark/runtime/checkpoint.py index a44be553cc..80122f218a 100644 --- a/benchmark/runtime/checkpoint.py +++ b/benchmark/runtime/checkpoint.py @@ -12,6 +12,7 @@ import os import subprocess import time +from contextlib import contextmanager from pathlib import Path, PurePosixPath from typing import Any @@ -471,3 +472,43 @@ def capture_after_wake(env: dict[str, str], receipt: dict[str, Any], *, receipt["checkpoint"].update(reason="capture_failed", error_kind=type(error).__name__, error=str(error)[:500]) finally: receipt["checkpoint"]["elapsed_seconds"] = time.monotonic() - started + + +@contextmanager +def held_quota_decision(listener, capture_root: Path, *, goal_id: str, agent_id: str, + turn_instance_id: str): + """Host-side counterpart of the explicit quota pre-output transport. + + The existing typed owner decides whether this exact packet admits a fresh + replan. The caller must capture while inside the context. Normal exit only + releases the original tool response; it certifies no research checkpoint. + Failure/disconnection never retries the guard or substitutes a new point. + The listener must already be bound by the Host in a private directory. + """ + if not capture_root.is_absolute() or capture_root.resolve(strict=True) != capture_root: + raise ValueError("decision capture root must be a physical absolute directory") + channel, _ = listener.accept() + with channel: + channel.settimeout(120) + with channel.makefile("rb") as stream: + line = stream.readline(4097) + message = json.loads(line) + if (not isinstance(message, dict) or set(message) != {"path", "sha256"} or + not isinstance(message["path"], str) or not isinstance(message["sha256"], str)): + raise ValueError("invalid decision handoff request") + path = Path(message["path"]) + if (path.name != "decision.json" or path.parent.parent != capture_root or + path.resolve(strict=True) != path): + raise ValueError("decision handoff must reference this Host's capture root") + raw = path.read_bytes() + if hashlib.sha256(raw).hexdigest() != message["sha256"]: + raise ValueError("decision handoff digest mismatch") + quota = json.loads(raw) + intent = effect_runtime_result("runtime.checkpoint_artifacts", { + "action": "select_replan", "quota": quota, "goal_id": goal_id, + "agent_id": agent_id, "turn_instance_id": turn_instance_id, + }) + yield {"path": path, "sha256": message["sha256"], "quota": quota, "intent": intent} + if path.read_bytes() != raw: + raise ValueError("original decision changed while held") + channel.sendall(json.dumps({"released": True, "sha256": message["sha256"]}).encode() + b"\n") diff --git a/benchmark/tests/test_native_checkpoint_handoff.py b/benchmark/tests/test_native_checkpoint_handoff.py index b84cefd214..b962ed19f1 100644 --- a/benchmark/tests/test_native_checkpoint_handoff.py +++ b/benchmark/tests/test_native_checkpoint_handoff.py @@ -112,3 +112,95 @@ def terminal(host, session, turn): assert (source_work / "source-result").read_text() == "one original execution" assert not (branch_work / "source-result").exists() assert len(calls) == 4 and "WITHHELD_RESULT" in json.dumps(requests[3]) + + +@pytest.mark.skipif(not os.environ.get("LOOPX_NATIVE_CHECKPOINT_BIN") or os.name != "posix", + reason="Requires a real native Host and local Unix transport") +def test_real_quota_output_is_held_across_native_fork(tmp_path): + """The real guard executes once; neither branch observes it before release.""" + import hashlib + import socket + import sys + import tempfile + from pathlib import Path + + from benchmark.runtime.checkpoint import held_quota_decision + from tests.control_plane.test_quota_settlement_cli import ( + AGENT_ID, GOAL_ID, TURN_ID, _write_fixture, _configure_autonomous_replan_fixture, + _heartbeat_receipt_count, + ) + + project, runtime, registry = _write_fixture(tmp_path / "authority") + _configure_autonomous_replan_fixture(project, runtime, registry) + capture_root = project / "captures" + capture_root.mkdir(mode=0o700) + requests = [] + with tempfile.TemporaryDirectory(prefix="handoff-", dir="/tmp") as short: + socket_path = Path(short).resolve() / "host.sock" + with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as listener: + listener.bind(str(socket_path)) + listener.listen(1) + listener.settimeout(30) + argv = [sys.executable, "-m", "loopx.cli", "--format", "json", "--registry", str(registry), + "--runtime-root", str(runtime), "quota", "should-run", "--goal-id", GOAL_ID, + "--agent-id", AGENT_ID, "--turn-instance-id", TURN_ID, + "--runtime-profile", "generic_cli", "--available-capability", "shell", + "--available-capability", "filesystem_read", "--decision-output-root", str(capture_root), + "--decision-handoff-socket", str(socket_path)] + command = "PYTHONPATH=" + shlex.quote(str(Path(__file__).resolve().parents[2])) + " " + shlex.join(argv) + + def response(request, index, message): + requests.append(request) + if index == 2: + return {"type": "function_call", "id": "fc_guard", "call_id": "call_guard", + "name": "exec_command", "arguments": json.dumps({ + "cmd": command, "yield_time_ms": 10000, "max_output_tokens": 200})} + message["content"][0]["text"] = "RESEARCH_BEFORE_GUARD" if index == 1 else "Continued." + return message + + binary = os.environ["LOOPX_NATIVE_CHECKPOINT_BIN"] + with real_native_source(tmp_path / "native", binary, project, close_source=True, + response_builder=response, sandbox="danger-full-access") as (env, source, _, calls): + with StdioNativeGoalTransport.spawn([binary, "app-server", "--listen", "stdio://"], + cwd=str(project), env=env, stderr=subprocess.DEVNULL, response_timeout_sec=30) as host: + host.request("initialize", {"clientInfo": {"name": "quota_handoff_fixture", "version": "0.1"}, + "capabilities": {"experimentalApi": True}}) + host.notify("initialized", {}) + host.request("thread/resume", {"threadId": source, "cwd": str(project), + "sandbox": "danger-full-access", "approvalPolicy": "never", "persistExtendedHistory": True}) + turn = host.request("turn/start", {"threadId": source, "input": [ + {"type": "text", "text": "Execute the original synthetic guard."}]})["turn"]["id"] + with held_quota_decision(listener, capture_root, goal_id=GOAL_ID, + agent_id=AGENT_ID, turn_instance_id=TURN_ID) as held: + assert held["quota"]["ok"] is True and len(calls) == 2 + assert held["intent"]["capture"] is True + assert held["sha256"] == hashlib.sha256(held["path"].read_bytes()).hexdigest() + receipts = _heartbeat_receipt_count(runtime, TURN_ID) + assert receipts >= 1 + # Native events, rather than the incomplete history projection, + # witness the actual in-flight tool and its exact command. + pending = None + deadline = time.monotonic() + 5 + while time.monotonic() < deadline: + event = host.next_event(timeout_sec=.1) + if event and event.get("method") == "item/started": + params = event["params"] + item = params.get("item", {}) + if item.get("type") == "commandExecution": + pending = params + break + assert pending and pending["threadId"] == source and pending["turnId"] == turn + assert shlex.split(pending["item"]["command"])[-1] == command + fork = host.request("thread/fork", {"threadId": source, + "cwd": str(project), "persistExtendedHistory": True})["thread"]["id"] + rows = host.request("thread/turns/list", {"threadId": fork, + "sortDirection": "asc", "itemsView": "full", "limit": 100})["data"] + assert rows[-1]["status"] == "interrupted" and rows[-1]["id"] == turn + assert len(calls) == 2 # No post-guard inference while held. + assert _heartbeat_receipt_count(runtime, TURN_ID) == receipts + deadline = time.monotonic() + 10 + while len(calls) < 3 and time.monotonic() < deadline: + time.sleep(.02) + assert len(calls) == 3 and "heartbeat_receipt" in json.dumps(requests[2]) + assert _heartbeat_receipt_count(runtime, TURN_ID) == receipts + assert held["path"].read_bytes() # Original receipt stays recoverable. diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md index 8286470a97..80da0b4899 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md @@ -720,6 +720,16 @@ qualifying that cursor. Keep the completed-final-cursor production check until this separate path is implemented and qualified; do not silently drop the active turn or classify every interrupted session as resumable research state. +The quota transport now has an explicit local pre-output rendezvous. Its real +native/CLI qualification binds a pending command event and the saved original +guard digest, forks without another model call, and releases that same response +without another guard receipt. Lost acknowledgement and altered bytes fail +without manufacturing another point. This closes the response-ordering gap only: +work-subprocess yielding can still return control to the model, so source writer +quiescence, effect reconciliation, automatic worker/controller connection and +isolated useful continuation remain required. The completed-cursor acceptance +check remains unchanged. + The Host profile must also declare path portability. A native index can retain absolute guest paths: qualification must either preserve those paths inside a fresh isolated environment or use the Host's supported import/rebinding API. diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md index 854566f4bb..5f0b52df76 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md @@ -604,6 +604,13 @@ Host adapter 须在固定版本验证原生历史重建，包括 compaction 和 在该独立路径实现并验证前保留生产代码的 completed-final-cursor 检查；不能 静默丢弃 active turn，也不能把所有 interrupted 会话视为可接续研究状态。 +quota transport 已增加显式本地 pre-output rendezvous。真实原生 Host/CLI 验证 +绑定 pending command event 与保存的原 guard digest，在没有额外模型调用时 +fork，并释放同一响应，不新增 guard receipt。丢失确认或字节变化会失败，不另造 +捕获点。此处只补齐响应顺序：工具子进程 yield 仍可能让模型继续；来源 writer +静止、effect 核回、worker/controller 自动衔接和隔离有效接续仍须完成。 +completed cursor 的验收检查保持不变。 + Host profile 还须声明路径可移植性。原生索引可能保留绝对 guest 路径：资格化 须在新隔离环境保留这些路径，或使用 Host 支持的 import/rebinding API；手改 原生数据库行不能资格化导入。有效 Host 配置及已安装工具/skill 版本须与凭证 diff --git a/loopx/cli_commands/quota.py b/loopx/cli_commands/quota.py index 94a2efaab9..4542035aef 100644 --- a/loopx/cli_commands/quota.py +++ b/loopx/cli_commands/quota.py @@ -5,7 +5,7 @@ from collections.abc import Callable, Mapping from pathlib import Path -from .quota_capture import bind_capture_selection_transport, capture_decision, prepare_decision_capture +from .quota_capture import bind_capture_selection_transport, capture_decision, handoff_decision, prepare_decision_capture from ..usage_goal import observe_quota_result from ..capabilities.explore.composition_frontier import ( @@ -458,6 +458,8 @@ def _emit_quota_result( ) bind_capture_selection_transport(payload, args) capture_decision(capture_directory, payload) + if payload.get("ok") is True: + handoff_decision(capture_directory, getattr(args, "decision_handoff_socket", None)) payload = _project_quota_cli_payload( payload, args, detail_sections, context.scheduler_context if context is not None else None, @@ -502,6 +504,7 @@ def handle_quota_command( if capture_directory is None and ( getattr(args, "decision_output_dir", None) is not None or getattr(args, "decision_output_root", None) is not None + or getattr(args, "decision_handoff_socket", None) is not None ): validate_quota_command_context_request(args) capture_directory = prepare_decision_capture(args) diff --git a/loopx/cli_commands/quota_capture.py b/loopx/cli_commands/quota_capture.py index 90ba7dc16a..def5f08c9d 100644 --- a/loopx/cli_commands/quota_capture.py +++ b/loopx/cli_commands/quota_capture.py @@ -2,9 +2,12 @@ from __future__ import annotations import argparse +import hashlib import json import os import shlex +import socket +import stat import tempfile from pathlib import Path @@ -14,6 +17,11 @@ def prepare_decision_capture(args: argparse.Namespace) -> Path | None: requested = getattr(args, "decision_output_dir", None) root = getattr(args, "decision_output_root", None) + handoff = getattr(args, "decision_handoff_socket", None) + if handoff is not None: + validate_handoff_socket(handoff) + if requested is None and root is None: + raise QuotaCommandValidationError("decision handoff requires an explicit decision capture destination") if requested is None and root is None: return None if requested is not None and root is not None: @@ -80,6 +88,56 @@ def bind_capture_selection_transport(payload: dict, args: argparse.Namespace) -> if not isinstance(template, str): return suffix = " --decision-output-root " + shlex.quote(str(Path(root).expanduser().absolute())) + if getattr(args, "decision_handoff_socket", None): + suffix += " --decision-handoff-socket " + shlex.quote(args.decision_handoff_socket) if getattr(args, "turn_envelope", False): suffix += " --turn-envelope" selection["command_args_template"] = template + suffix + + +def validate_handoff_socket(value: str) -> Path: + """A caller-owned local transport, not an authority or admission endpoint.""" + path = Path(value) + try: + if (os.name != "posix" or not hasattr(socket, "AF_UNIX") or not path.is_absolute() or + path.resolve(strict=True) != path or + not stat.S_ISSOCK(path.stat().st_mode) or + path.parent.stat().st_mode & 0o077 or + path.stat().st_uid != os.getuid()): + raise OSError("expected a same-user socket in a private physical directory") + except OSError as exc: + raise QuotaCommandValidationError(f"invalid decision handoff socket: {exc}") from exc + return path + + +def handoff_decision(directory: Path | None, socket_path: str | None) -> None: + """Withhold the original output until the Host releases these exact bytes. + + No second guard, admission, settlement or retry is performed here. A lost + acknowledgement leaves the saved decision available for original-identity + recovery; it must never be silently replaced by a later observation. + """ + if socket_path is None: + return + if directory is None: + raise ValueError("decision handoff requires a captured decision") + path = directory / "decision.json" + raw = path.read_bytes() + digest = hashlib.sha256(raw).hexdigest() + try: + with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as channel: + channel.settimeout(120) + channel.connect(str(validate_handoff_socket(socket_path))) + channel.sendall(json.dumps({"path": str(path), "sha256": digest}).encode() + b"\n") + with channel.makefile("rb") as stream: + line = stream.readline(1025) + response = json.loads(line) + if response != {"released": True, "sha256": digest}: + raise ValueError("Host did not release the original decision digest") + if path.read_bytes() != raw: + raise ValueError("captured decision changed during handoff") + except (OSError, ValueError) as exc: + raise RuntimeError( + "Decision handoff failed after guard evaluation; inspect the original " + f"Turn receipt and {path}. Do not blindly rerun the guard." + ) from exc diff --git a/loopx/cli_commands/quota_registration.py b/loopx/cli_commands/quota_registration.py index 5a8027bcb5..ac456517c0 100644 --- a/loopx/cli_commands/quota_registration.py +++ b/loopx/cli_commands/quota_registration.py @@ -191,6 +191,12 @@ def register_quota_command( "--decision-output-root", help="Create a unique private decision directory below this existing root on every guard invocation.", ) + quota_parser.add_argument( + "--decision-handoff-socket", + help=("Opt-in local Host rendezvous before printing a captured should-run decision. " + "Requires a capture destination and a same-user Unix socket in a private directory. " + "Waits at most 120s for a digest-bound release; failure retains the original receipt."), + ) quota_parser.add_argument( "--turn-instance-id", help=( diff --git a/tests/control_plane/test_quota_decision_capture.py b/tests/control_plane/test_quota_decision_capture.py index ac71361115..68578d775d 100644 --- a/tests/control_plane/test_quota_decision_capture.py +++ b/tests/control_plane/test_quota_decision_capture.py @@ -197,3 +197,105 @@ def test_capture_root_allocates_private_distinct_children(tmp_path): assert first.stat().st_mode & 0o777 == 0o700 with pytest.raises(QuotaCommandValidationError): prepare_decision_capture(request(None, decision_output_root=tmp_path, turn_instance_id=None)) + + +@pytest.mark.skipif(os.name != 'posix', reason='Unix local Host transport') +@pytest.mark.parametrize('release', ['exact', 'wrong_digest', 'disconnect', 'changed_capture']) +def test_real_cli_handoff_preserves_original_receipt(tmp_path, release): + import hashlib + import socket + import sys + import tempfile + from pathlib import Path + + project, runtime, registry = _write_fixture(tmp_path) + _configure_read_only_todo(project) + capture = tmp_path / 'handoff-capture' + with tempfile.TemporaryDirectory(prefix='quota-host-', dir='/tmp') as short: + endpoint = Path(short).resolve() / 'host.sock' + with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as listener: + listener.bind(str(endpoint)) + listener.listen(1) + listener.settimeout(20) + command = [sys.executable, '-m', 'loopx.cli', '--format', 'json', + '--registry', str(registry), '--runtime-root', str(runtime), + 'quota', 'should-run', '--goal-id', GOAL_ID, '--agent-id', AGENT_ID, + '--todo-id', TODO_ID, '--turn-instance-id', TURN_ID, + '--runtime-profile', 'generic_cli', '--available-capability', 'shell', + '--available-capability', 'filesystem_read', + '--decision-output-dir', str(capture), '--decision-handoff-socket', str(endpoint)] + with subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, + text=True, env={**os.environ, 'PYTHONPATH': str(Path(__file__).resolve().parents[2])}) as process: + try: + channel, _ = listener.accept() + with channel: + channel.settimeout(5) + with channel.makefile('rb') as stream: + message = json.loads(stream.readline()) + original = (capture / 'decision.json').read_bytes() + digest = hashlib.sha256(original).hexdigest() + assert message == {'path': str(capture / 'decision.json'), 'sha256': digest} + receipts = _heartbeat_receipt_count(runtime, TURN_ID) + assert receipts >= 1 and process.poll() is None + # The actual CLI has persisted admission but cannot expose + # successful output while the Host is taking its snapshot. + import select + assert not select.select([process.stdout], [], [], 0)[0] + if release == 'changed_capture': + (capture / 'decision.json').write_bytes(original + b'\n') + if release != 'disconnect': + channel.sendall(json.dumps({'released': True, 'sha256': ( + '0' * 64 if release == 'wrong_digest' else digest)}).encode() + b'\n') + stdout, stderr = process.communicate(timeout=15) + if release == 'exact': + assert process.returncode == 0 and json.loads(stdout)['ok'] is True + assert not stderr + else: + assert process.returncode != 0 + assert 'Do not blindly rerun' in stderr + stdout + assert _heartbeat_receipt_count(runtime, TURN_ID) == receipts + assert json.loads(original)['heartbeat_receipt']['turn_instance_id'] == TURN_ID + finally: + if process.poll() is None: + process.kill() + process.wait() + + +def test_handoff_requires_explicit_capture_before_guard(tmp_path): + with pytest.raises(QuotaCommandValidationError, match='invalid decision handoff'): + prepare_decision_capture(request(None, decision_handoff_socket=str(tmp_path / 'missing'))) + assert list(tmp_path.iterdir()) == [] + + +@pytest.mark.skipif(os.name != 'posix', reason='Unix local Host transport') +def test_handoff_rejects_shared_or_symlink_endpoint_and_preserves_selection(tmp_path): + import socket + import tempfile + from pathlib import Path + from loopx.cli_commands.quota_capture import validate_handoff_socket, bind_capture_selection_transport + + with tempfile.TemporaryDirectory(prefix='quota-host-', dir='/tmp') as short: + private = Path(short).resolve() + endpoint = private / 'host.sock' + with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as listener: + listener.bind(str(endpoint)) + assert validate_handoff_socket(str(endpoint)) == endpoint + alias = private / 'alias.sock' + alias.symlink_to(endpoint) + with pytest.raises(QuotaCommandValidationError): + validate_handoff_socket(str(alias)) + private.chmod(0o755) + try: + with pytest.raises(QuotaCommandValidationError): + validate_handoff_socket(str(endpoint)) + finally: + private.chmod(0o700) + with pytest.raises(QuotaCommandValidationError, match='explicit decision capture'): + prepare_decision_capture(request(None, decision_handoff_socket=str(endpoint))) + payload = {'interaction_contract': {'cli_channel': {'selection_command': { + 'command_args_template': 'quota should-run --todo-id TODO'}}}} + bind_capture_selection_transport(payload, request(None, decision_output_root=tmp_path, + decision_handoff_socket=str(endpoint))) + words = shlex.split(payload['interaction_contract']['cli_channel']['selection_command']['command_args_template']) + assert words[-4:] == ['--decision-output-root', str(tmp_path), '--decision-handoff-socket', str(endpoint)] + assert '--turn-envelope' not in words # Independent from display mode. From c134dace4f694dcd2975bf33254271cfa9b0f7fb Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 21:17:32 +0800 Subject: [PATCH 50/69] fix(benchmark): preserve rejected worker pause barriers Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/README.md | 11 +++-- benchmark/edgebench/checkpoint_worker.py | 44 +++++++++++++++---- benchmark/tests/test_checkpoint_worker.py | 52 ++++++++++++++++++++++- 3 files changed, 94 insertions(+), 13 deletions(-) diff --git a/benchmark/README.md b/benchmark/README.md index eb13145ed1..54d3b73dfd 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -526,9 +526,14 @@ capture behavior; there is no automatic activation or retention policy. The container must initially be running and unpaused. Cancellation, failed file reads, rejected native capture and a lost pause response attempt to release only this operation's barrier. Failure to resume the worker is reported and publishes -no outer artifact. An existing pause is never released. The first attempt owns -the capture identity; a failure cannot silently pair later files with an older -native point. Missing paths, archive links, traversal and special members reject. +no outer artifact. An initially observed pause or a definite Docker conflict +rejects without unpausing. Controlled captures on the same host/user share a +daemon-ID/container-ID lock under `~/.cache/loopx/checkpoint-workers`, across +controller banks and Docker endpoint aliases. External actors do not participate +in this lock: an uncertain transport outcome retains the original attempt and +cleanup obligation, not proof of exclusive ownership against arbitrary actors. +The first attempt owns the capture identity; a failure cannot silently pair +later files with an older native point. Missing paths, archive links, traversal and special members reject. Declare only the intended private code/state stores: a whole home can contain credentials, and neither archive filtering nor a freeze is privacy certification. diff --git a/benchmark/edgebench/checkpoint_worker.py b/benchmark/edgebench/checkpoint_worker.py index b23d4e78b8..4b9795560d 100644 --- a/benchmark/edgebench/checkpoint_worker.py +++ b/benchmark/edgebench/checkpoint_worker.py @@ -6,8 +6,10 @@ """ from __future__ import annotations +import hashlib import json import math +import os import re import tarfile import tempfile @@ -81,6 +83,22 @@ def _archive(container, remote: str, destination: Path) -> Path: return _physical(destination / root) +def _worker_lock(client, container_id: str) -> Path: + # Controller banks and endpoint aliases may differ for the same daemon. + # Coordinate captures by this user on this host, not by artifact location. + daemon_id = client.info().get("ID") + if not isinstance(daemon_id, str) or not daemon_id: + raise ValueError("Docker daemon identity is required for worker capture") + root = Path.home() / ".cache" / "loopx" / "checkpoint-workers" + root.mkdir(parents=True, exist_ok=True, mode=0o700) + _physical(root) + info = root.stat() + if info.st_uid != os.getuid() or info.st_mode & 0o077: + raise ValueError("Worker capture lock directory must be private and owned by the current user") + daemon_key = hashlib.sha256(daemon_id.encode()).hexdigest() + return root / f"{daemon_key}-{container_id}" + + def capture_worker(controller_file: Path, *, token: str, capture_id: str, destination: Path, request_file: Path, timeout: float = 30) -> dict: """Freeze one explicitly owned worker and seal its files with native state. @@ -108,7 +126,7 @@ def capture_worker(controller_file: Path, *, token: str, capture_id: str, (attempts / capture_id).mkdir(mode=0o700) # First attempt, including failures, wins. client = docker.from_env(timeout=timeout) try: - with exclusive_file_lock(attempts / request["container_id"], timeout_seconds=timeout, + with exclusive_file_lock(_worker_lock(client, request["container_id"]), timeout_seconds=timeout, operation="checkpoint_worker_capture"): container = client.containers.get(request["container_id"]) container.reload() @@ -119,9 +137,16 @@ def capture_worker(controller_file: Path, *, token: str, capture_id: str, stage = Path(directory) (stage / "workers").mkdir(mode=0o700) components = [] + pause_may_have_committed = True try: - # The cleanup also covers a lost reply after Docker committed pause. - container.pause() + try: + container.pause() + except docker.errors.APIError as exc: + # Docker's conflict response rejects this pause. The + # observed pause belongs to another actor; leave it alone. + if exc.response is not None and exc.response.status_code == 409: + pause_may_have_committed = False + raise container.reload() if not container.attrs["State"]["Paused"]: raise RuntimeError("Worker writer barrier was not established") @@ -135,12 +160,15 @@ def capture_worker(controller_file: Path, *, token: str, capture_id: str, if not container.attrs["State"]["Running"] or not container.attrs["State"]["Paused"]: raise RuntimeError("Worker writer barrier changed during capture") finally: - container.reload() - if container.attrs["State"]["Paused"]: - container.unpause() + # An uncertain/lost reply keeps this original attempt's + # cleanup obligation. Its identity cannot be retried later. + if pause_may_have_committed: container.reload() - if container.attrs["State"]["Paused"] or not container.attrs["State"]["Running"]: - raise RuntimeError("Worker was not restored to running; inspect the original container before retry") + if container.attrs["State"]["Paused"]: + container.unpause() + container.reload() + if container.attrs["State"]["Paused"] or not container.attrs["State"]["Running"]: + raise RuntimeError("Worker was not restored to running; inspect the original container before retry") native_component = _participant_id(components, "native_capsule") components.append({"id": native_component, "source_path": str(stage / "native")}) metadata = {"worker_file_snapshot": { diff --git a/benchmark/tests/test_checkpoint_worker.py b/benchmark/tests/test_checkpoint_worker.py index e8cc09ec09..2736c59b25 100644 --- a/benchmark/tests/test_checkpoint_worker.py +++ b/benchmark/tests/test_checkpoint_worker.py @@ -168,14 +168,24 @@ def unavailable(self): assert not (tmp_path / "later").exists() +@pytest.mark.parametrize("separate_bank", [False, True]) def test_concurrent_capture_does_not_release_another_operation_barrier( - controllers, writer, tmp_path, monkeypatch): + controllers, writer, tmp_path, monkeypatch, separate_bank): from concurrent.futures import ThreadPoolExecutor from loopx.file_lock import LockAcquireTimeoutError source, _, first, _, files, _ = controllers container, request = writer token = register(source, first) + other_controller = files[0] + if separate_bank: + bank = tmp_path / "other-bank" + bank.mkdir() + config = json.loads(files[0].read_bytes()) + config["bank_root"] = str(bank) + other_controller = tmp_path / "other-controller.json" + other_controller.write_text(json.dumps(config)) + other_controller.chmod(0o600) entered, release = threading.Event(), threading.Event() original = transport._post def held(*args, **kwargs): @@ -190,7 +200,7 @@ def held(*args, **kwargs): assert entered.wait(15) frozen = counter(container) with pytest.raises(LockAcquireTimeoutError): - capture_worker(files[0], token=token, capture_id="2" * 32, + capture_worker(other_controller, token=token, capture_id="2" * 32, destination=tmp_path / "second", request_file=request, timeout=.5) container.reload() assert container.attrs["State"]["Paused"] and counter(container) == frozen @@ -200,3 +210,41 @@ def held(*args, **kwargs): assert pending.result(timeout=20)["research_point_qualified"] is False container.reload() assert not container.attrs["State"]["Paused"] + + +@pytest.mark.parametrize("entrypoint", ["provider", "cli"]) +def test_rejected_pause_preserves_foreign_barrier( + controllers, writer, tmp_path, monkeypatch, entrypoint): + from docker.errors import APIError + + source, _, first, _, files, _ = controllers + container, request = writer + token = register(source, first) + original = type(container).pause + def raced_pause(self): + # Another actor pauses after our initial state read; the second RPC + # reaches the real daemon and is determinately rejected. + original(container) + original(self) + monkeypatch.setattr(type(container), "pause", raced_pause) + def unexpected_native_call(*args, **kwargs): + pytest.fail("Rejected writer barrier must not capture evaluator state") + monkeypatch.setattr(transport, "_post", unexpected_native_call) + unit = tmp_path / "rejected" + with pytest.raises(APIError) as error: + if entrypoint == "provider": + capture_worker(files[0], token=token, capture_id="3" * 32, + destination=unit, request_file=request) + else: + token_file = tmp_path / "token" + token_file.write_text(token) + transport.main(["--controller-file", str(files[0]), "capture-worker", + "--token-file", str(token_file), "--capture-id", "3" * 32, + "--destination", str(unit), "--worker-request", str(request)]) + assert error.value.response.status_code == 409 + container.reload() + assert container.attrs["State"]["Paused"] + assert not unit.exists() + with pytest.raises(FileExistsError): + capture_worker(files[0], token=token, capture_id="3" * 32, + destination=tmp_path / "retry", request_file=request) From 3fa48b657fa33d1d1b26e580b2b02e97a4d7a4e5 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 22:06:14 +0800 Subject: [PATCH 51/69] fix(benchmark): recover original checkpoint activation without budget refill Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/README.md | 32 +++- benchmark/edgebench/checkpoint_controller.py | 161 ++++++++++++----- benchmark/tests/test_checkpoint_controller.py | 170 ++++++++++++++---- 3 files changed, 274 insertions(+), 89 deletions(-) diff --git a/benchmark/README.md b/benchmark/README.md index 54d3b73dfd..b5e47369d9 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -477,7 +477,7 @@ remain open. No automatic retention or extra evaluation opportunity is introduce The read-only participant above does **not** implement the newer native capsule API. The separate controller adapter targets [EdgeBench PR #23](https://github.com/ByteDance-Seed/EdgeBench/pull/23), pinned at -`4db79f4ea332ffd30c265ec7cb076f93471bd201`; this is proposed source, not a released +`57dce5dd1fded9d5d5cbd44fa59c59d27a068d62`; this is proposed source, not a released upstream version. Start that pinned native judge with `sforge serve --checkpoints`. Its private controller configuration file has exactly `url`, `admin_secret` and `bank_root` (the native log directory's `checkpoints` directory). Keep this file, @@ -507,8 +507,13 @@ Restore writes its new token only to the explicit private result file; stdout contains its path. Wrong authority, corrupt bytes, aliases, foreign bank entries, source-run reuse and duplicate native admission fail without a fresh evaluation. Redirects cannot forward the admin secret. -After an uncertain restore response, inspect native readback with the controller; -do not switch run identities or perform ordinary fresh registration as a retry. +After an uncertain restore response, repeat the same controller `restore` command +with `--recover`, preserving the same capsule digest, run identity, result-file +and optional phase-deadline-file. Recovery reads the original native activation +through `/api/v1/checkpoint/restore-result`; it does not allocate a registration, +replay grading or reset cooldown. A successful result may be read back again. +An absent original activation or lost live authority refuses recovery. Do not +switch identities or perform ordinary fresh registration as a retry. For an explicitly owned Docker worker, `capture-worker` can additionally freeze its writers while copying declared files and capturing the native capsule. Supply @@ -570,10 +575,16 @@ with this participant requires the matching explicit restore option; omitting it cannot discard its allowance. Existing/aliased files, source movement, changed sealed bytes and exhausted work allowance refuse activation. An uncertain native response leaves the reserved phase file empty and unusable by the ordinary Host -entry, preventing a fresh-budget fallback. Reconcile that original native -operation before any owner-authorized recovery; no automatic replay, new run or -budget reset is supplied. Omit the option on **new captures** to retain native-only -transport. Full Host source attribution, source/native clock alignment and the +entry, preventing a fresh-budget fallback. Before native activation, the adapter +syncs a private sibling `.restore.json` containing the exact attempt +binding and destination deadline, without tokens. Explicit `--recover` uses that +original deadline: recovery delay consumes the remaining allowance, and expiry +refuses continuation. Result/phase locks prevent concurrent controller recovery; +partial records, aliases, changed identities and missing records fail closed. +Do not edit, copy independently or replace the attempt files to retry. The record +is local provider state, not part of the portable capsule or a new authorization +source. This recovery is explicit, with no automatic replay or new run. Omit the +option on **new captures** to retain native-only transport. Full Host source attribution, source/native clock alignment and the joint session/research/effect barrier remain qualification requirements. This is an independently usable **native participant transport**, still partial @@ -583,7 +594,12 @@ not qualify full research-state adoption or a same-point scientific comparison. Never put its controller configuration in `--checkpoint-request` or a worker payload. Omit both these controller operations and the native `--checkpoints` flag to disable this transport on a fresh judge; preserved capsules stay inert. -Synthetic native HTTP tests and an explicitly enabled Docker qualification are: +Native HTTP tests cover lost replies, exact readback, unavailable authority and +invalid local recovery state. The explicitly enabled Docker qualification also +loses a reply after real native commit, recovers the original deadline, runs the +ordinary SForge phase-entry shell in Linux, and produces and grades a synthetic +candidate without refilling rounds/cooldown. It excludes model inference and +does not establish full research continuation. Run these checks with: ```sh python -m pytest -q benchmark/tests/test_checkpoint_controller.py diff --git a/benchmark/edgebench/checkpoint_controller.py b/benchmark/edgebench/checkpoint_controller.py index 3366c2efad..35bbce0195 100644 --- a/benchmark/edgebench/checkpoint_controller.py +++ b/benchmark/edgebench/checkpoint_controller.py @@ -6,12 +6,14 @@ from __future__ import annotations import argparse +import fcntl import hashlib import json import math import os import re import shutil +import stat import tempfile import time from pathlib import Path @@ -118,15 +120,61 @@ def _phase_budget(raw: bytes, native: dict) -> dict: "captured_at_epoch": captured, "remaining_seconds": max(0, epoch - captured)} -def _phase_destination(path: Path, *, root: Path, bank: Path) -> Path: +def _phase_destination(path: Path, *, root: Path, bank: Path, recover: bool = False) -> Path: path = Path(path) _physical(path.parent) - if (not path.is_absolute() or path.resolve() != path or path.exists() or path.is_symlink() + if (not path.is_absolute() or path.resolve() != path or (path.exists() and not recover) or path.is_symlink() or path.is_relative_to(root) or path.is_relative_to(bank)): raise ValueError("Host phase restoration requires a fresh physical file outside artifact/bank roots") return path +def _private_file(path: Path) -> None: + info = _physical(path).stat() + if not stat.S_ISREG(info.st_mode) or info.st_mode & 0o077 or info.st_nlink != 1: + raise ValueError("Recovery requires an owner-only physical regular file") + + +def _sync_parent(path: Path) -> None: + descriptor = os.open(path.parent, os.O_RDONLY | os.O_DIRECTORY) + try: + os.fsync(descriptor) + finally: + os.close(descriptor) + + +def _activation_epoch(path: Path, binding: dict, remaining: float, recover: bool) -> int: + """Provider-local attempt binding; no token or historical authority is stored.""" + record = path.with_name(path.name + ".restore.json") + if recover: + _private_file(record) + value = json.loads(record.read_bytes()) + if (not isinstance(value, dict) or set(value) != {"binding", "deadline_epoch"} + or value["binding"] != binding): + raise ValueError("Host recovery does not match the original activation") + epoch = value["deadline_epoch"] + else: + epoch = math.floor(time.time() + remaining) + # Commit this before contacting native admission. A partial record + # cannot be recovered, and retry never computes another deadline. + with os.fdopen(os.open(record, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600), "w") as target: + json.dump({"binding": binding, "deadline_epoch": epoch}, target) + target.flush() + os.fsync(target.fileno()) + _sync_parent(record) + if type(epoch) is not int or not 0 < epoch < 2 ** 53: + raise ValueError("Destination Host phase clock is incompatible") + if epoch - time.time() <= PHASE_WORK_RESERVE_SECONDS: + raise ValueError("Original Host activation budget is exhausted") + return epoch + + +def _activation_result(result: dict, body: dict) -> None: + if (set(result) != {*body, "token"} or any(result.get(k) != v for k, v in body.items()) + or not isinstance(result.get("token"), str) or not result["token"]): + raise ValueError("Native activation result differs from the original request") + + def capture(controller_file: Path, *, token: str, capture_id: str, destination: Path, timeout: float = 30, phase_deadline_file: Path | None = None) -> dict: """Capture one native-owned point without exporting controller configuration.""" @@ -178,7 +226,7 @@ def capture(controller_file: Path, *, token: str, capture_id: str, destination: def restore(controller_file: Path, *, root: Path, expected_sha256: str, run_id: str, - timeout: float = 30, phase_deadline_file: Path | None = None) -> dict: + timeout: float = 30, phase_deadline_file: Path | None = None, recover: bool = False) -> dict: """Import verified inert bytes, then ask fresh native authority to admit them. An explicitly captured Host clock preserves its remaining phase duration. @@ -211,61 +259,74 @@ def restore(controller_file: Path, *, root: Path, expected_sha256: str, run_id: raise ValueError("Host phase metadata differs from sealed deadline/native clock") if phase["remaining_seconds"] <= PHASE_WORK_RESERVE_SECONDS: raise ValueError("Captured Host phase has no work allowance beyond its existing reserve") - phase_path = _phase_destination(phase_deadline_file, root=root, bank=bank) + phase_path = _phase_destination(phase_deadline_file, root=root, bank=bank, recover=recover) target = bank / reference["checkpoint_id"] stage = Path(tempfile.mkdtemp(prefix=".loopx-native-import-", dir=bank)) phase_fd = None phase_published = False + phase_locked = False native_attempted = False try: if phase_path is not None: # Empty is unusable by the existing SForge phase-entry shell. # An uncertain native response must not fall back to a new budget. - phase_fd = os.open(phase_path, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600) - effect_runtime_result("runtime.checkpoint_artifacts", { - "action": "materialize", "root": str(root), "expected_sha256": expected_sha256, - "destination": str(stage / "unit"), - }, retry_safe=False) - imported = stage / "unit/components/native_judge" - _bound_bank(imported, reference) - if target.exists(): - _bound_bank(_physical(target), reference) # No overwrite, even on retry. + if recover: + _private_file(phase_path) + phase_fd = os.open(phase_path, os.O_RDWR | os.O_NOFOLLOW | (0 if recover else os.O_CREAT | os.O_EXCL), 0o600) + fcntl.flock(phase_fd, fcntl.LOCK_EX | fcntl.LOCK_NB) + phase_locked = True + original_phase = phase_path.read_bytes() + phase_published = bool(original_phase) + if recover: + _bound_bank(_physical(target), reference) else: - # POSIX rename can replace an empty destination. Reserve its name - # exclusively; interruption leaves an unusable point, never grants. - target.mkdir(mode=0o700) - try: - imported.rename(target) - except BaseException: - target.rmdir() - raise - descriptor = os.open(bank, os.O_RDONLY | os.O_DIRECTORY) - try: - os.fsync(descriptor) - finally: - os.close(descriptor) - phase_epoch = None if phase is None else math.floor(time.time() + phase["remaining_seconds"]) - if phase_epoch is not None and not 0 < phase_epoch < 2 ** 53: - raise ValueError("Destination Host phase clock is incompatible") + effect_runtime_result("runtime.checkpoint_artifacts", { + "action": "materialize", "root": str(root), "expected_sha256": expected_sha256, + "destination": str(stage / "unit"), + }, retry_safe=False) + imported = stage / "unit/components/native_judge" + _bound_bank(imported, reference) + if target.exists(): + _bound_bank(_physical(target), reference) # No overwrite, even on retry. + else: + # POSIX rename can replace an empty destination. Reserve its name + # exclusively; interruption leaves an unusable point, never grants. + target.mkdir(mode=0o700) + try: + imported.rename(target) + except BaseException: + target.rmdir() + raise + descriptor = os.open(bank, os.O_RDONLY | os.O_DIRECTORY) + try: + os.fsync(descriptor) + finally: + os.close(descriptor) + body = {"checkpoint_id": reference["checkpoint_id"], + "manifest_sha256": reference["manifest_sha256"], "run_id": run_id} + phase_epoch = None + if phase_path is not None: + phase_epoch = _activation_epoch(phase_path, { + **body, "artifact_sha256": expected_sha256, "controller_url": controller["url"], + "bank_root": str(bank), "phase_path": str(phase_path), + }, phase["remaining_seconds"], recover) + if original_phase not in (b"", f"{phase_epoch}\n".encode()): + raise ValueError("Host phase bytes differ from original activation") native_attempted = True - result = _post(controller, "/api/v1/checkpoint/restore", { - "checkpoint_id": reference["checkpoint_id"], "manifest_sha256": reference["manifest_sha256"], - "run_id": run_id, - }, timeout) + route = "/api/v1/checkpoint/restore-result" if recover else "/api/v1/checkpoint/restore" + result = _post(controller, route, body, timeout) + _activation_result(result, body) if phase_fd is not None: - if not isinstance(result.get("token"), str) or not result["token"]: - raise ValueError("Native phase activation returned no fresh authority result") + if phase_epoch - time.time() <= PHASE_WORK_RESERVE_SECONDS: + raise ValueError("Original Host activation budget is exhausted") # Charge activation/transport, not offline time or a fresh trial. # Native alone owns accepted counters and remaining cooldown. encoded = f"{phase_epoch}\n".encode() + os.lseek(phase_fd, 0, os.SEEK_SET) if os.write(phase_fd, encoded) != len(encoded): raise OSError("Destination Host phase write was incomplete") os.fsync(phase_fd) - directory = os.open(phase_path.parent, os.O_RDONLY | os.O_DIRECTORY) - try: - os.fsync(directory) - finally: - os.close(directory) + _sync_parent(phase_path) if phase_path.read_bytes() != f"{phase_epoch}\n".encode(): raise ValueError("Destination Host phase readback differs from activation") phase_published = True @@ -273,12 +334,12 @@ def restore(controller_file: Path, *, root: Path, expected_sha256: str, run_id: finally: if phase_fd is not None: try: - if native_attempted and not phase_published: + if phase_locked and native_attempted and not phase_published: os.ftruncate(phase_fd, 0) os.fsync(phase_fd) finally: os.close(phase_fd) - if not native_attempted: + if phase_locked and not native_attempted and not recover: phase_path.unlink() shutil.rmtree(stage) @@ -306,6 +367,8 @@ def main(argv=None) -> None: restore_parser.add_argument("--result-file", required=True, type=Path) restore_parser.add_argument("--phase-deadline-file", type=Path, help="Fresh physical Host phase file, required for a capsule containing that participant") + restore_parser.add_argument("--recover", action="store_true", + help="Read the exact original native result and retain its original Host deadline") args = parser.parse_args(argv) if args.command in {"capture", "capture-worker"}: source_token = _physical(args.token_file).read_text().strip() @@ -326,14 +389,24 @@ def main(argv=None) -> None: _physical(args.result_file.parent) if args.result_file != args.result_file.resolve(): raise ValueError("Continuation result must be a normalized physical path") - with args.result_file.open("x") as result_file: + if args.recover: + _private_file(args.result_file) + with args.result_file.open("r+" if args.recover else "x") as result_file: args.result_file.chmod(0o600) + fcntl.flock(result_file.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB) + prior_raw = result_file.read() if args.recover else "" + prior_result = json.loads(prior_raw) if prior_raw else None result = restore(args.controller_file, root=args.root, expected_sha256=args.expected_sha256, run_id=args.run_id, - phase_deadline_file=args.phase_deadline_file) + phase_deadline_file=args.phase_deadline_file, recover=args.recover) + if prior_result is not None and prior_result != result: + raise ValueError("Existing private result differs from original native activation") + result_file.seek(0) json.dump(result, result_file) + result_file.truncate() result_file.flush() os.fsync(result_file.fileno()) + _sync_parent(args.result_file) print(json.dumps({"result_file": str(args.result_file), "run_id": args.run_id})) diff --git a/benchmark/tests/test_checkpoint_controller.py b/benchmark/tests/test_checkpoint_controller.py index 7f683b3ae1..6a2a149692 100644 --- a/benchmark/tests/test_checkpoint_controller.py +++ b/benchmark/tests/test_checkpoint_controller.py @@ -320,6 +320,65 @@ def post(controller, route, body, timeout): assert grades == [] +@pytest.mark.parametrize("fault", ["wrong_run", "expired", "partial_record", "phase_alias", "result_corrupt", "lost_authority", "busy"]) +def test_recovery_refuses_mismatched_or_unavailable_original_activation(controllers, tmp_path, monkeypatch, fault): + import fcntl + source, destination, first, second, files, grades = controllers + token = register(source, first) + source_phase = tmp_path / "source-phase" + source_phase.write_text(f"{int(time.time()) + 7200}\n") + unit = tmp_path / "unit" + point = transport.capture(files[0], token=token, capture_id="9" * 32, + destination=unit, phase_deadline_file=source_phase) + phase, result = tmp_path / "phase", tmp_path / "result" + args = ["--controller-file", str(files[1]), "restore", "--root", str(unit), + "--expected-sha256", point["sha256"], "--run-id", "branch", + "--result-file", str(result), "--phase-deadline-file", str(phase)] + post = transport._post + calls = [] + def lost(controller, route, body, timeout): + calls.append(route) + response = post(controller, route, body, timeout) + if route.endswith("/restore"): + raise TimeoutError("Lost committed result") + return response + monkeypatch.setattr(transport, "_post", lost) + with pytest.raises(TimeoutError): + transport.main(args) + record = phase.with_name("phase.restore.json") + original_epoch = json.loads(record.read_bytes())["deadline_epoch"] + lock = None + if fault == "wrong_run": + args[args.index("--run-id") + 1] = "another-branch" + elif fault == "expired": + monkeypatch.setattr(transport, "time", SimpleNamespace(time=lambda: original_epoch + 1)) + elif fault == "partial_record": + record.write_bytes(b"{") + elif fault == "phase_alias": + phase.unlink() + phase.symlink_to(source_phase) + elif fault == "result_corrupt": + result.write_bytes(b"{") + elif fault == "lost_authority": + destination.state.judge.tokens.clear() + elif fault == "busy": + lock = phase.open("r+") + fcntl.flock(lock.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB) + before = copy.deepcopy((destination.state.judge.tokens, destination.state.judge.run_history)) + old_bytes = (phase.read_bytes(), result.read_bytes(), record.read_bytes()) + try: + with pytest.raises((ValueError, OSError)): + transport.main(args + ["--recover"]) + finally: + if lock: + lock.close() + assert (phase.read_bytes(), result.read_bytes(), record.read_bytes()) == old_bytes + assert (destination.state.judge.tokens, destination.state.judge.run_history) == before + assert calls.count("/api/v1/checkpoint/restore") == 1 and grades == [] + if fault != "lost_authority": + assert calls == ["/api/v1/checkpoint/restore"] + + @pytest.mark.parametrize("raw", [b"", b"nan\n", b"1.5\n", b"9007199254740992\n"]) def test_invalid_host_epoch_cannot_create_native_point(controllers, tmp_path, raw): source, destination, first, second, files, grades = controllers @@ -365,6 +424,13 @@ def write(self, descriptor, data): transport.restore(files[1], root=unit, expected_sha256=point["sha256"], run_id="phase-branch", phase_deadline_file=target) assert target.read_bytes() == b"" and len(destination.state.judge.tokens) == 1 and grades == [] + original = copy.deepcopy(destination.state.judge.tokens) + epoch = json.loads(target.with_name("phase.restore.json").read_bytes())["deadline_epoch"] + monkeypatch.setattr(transport, "os", os) + fresh = transport.restore(files[1], root=unit, expected_sha256=point["sha256"], + run_id="phase-branch", phase_deadline_file=target, recover=True) + assert fresh["token"] in original and destination.state.judge.tokens == original + assert int(target.read_bytes()) == epoch and grades == [] @pytest.mark.parametrize("fault", ["wrong_admin", "corrupt", "wrong_digest", "aliased", "existing_bank"]) @@ -420,8 +486,8 @@ def redirect(): @pytest.mark.skipif(os.environ.get("LOOPX_EDGEBENCH_DOCKER_SMOKE") != "1", reason="Requires explicit isolated native Docker qualification") -@pytest.mark.parametrize("phase_budget", [False, True]) -def test_real_docker_controller_export_import_and_next_grade(tmp_path, monkeypatch, phase_budget): +@pytest.mark.parametrize("phase_budget,lose_response", [(False, False), (True, False), (True, True)]) +def test_real_docker_controller_export_import_and_next_grade(tmp_path, monkeypatch, phase_budget, lose_response): import uuid import docker import requests @@ -498,49 +564,79 @@ def grade(url, token, score): control = tmp_path / "host-control" control.mkdir() phase = control / "phase-deadline" - fresh = transport.restore(files[1], root=exported, expected_sha256=point["sha256"], run_id="branch", - phase_deadline_file=phase if phase_budget else None)["token"] + args = ["--controller-file", str(files[1]), "restore", "--root", str(exported), + "--expected-sha256", point["sha256"], "--run-id", "branch", + "--result-file", str(control / "result.json")] + if phase_budget: + args += ["--phase-deadline-file", str(phase)] + post = transport._post + routes = [] + def observed_post(controller, route, body, timeout): + routes.append(route) + result = post(controller, route, body, timeout) + if lose_response and route.endswith("/restore"): + raise TimeoutError("Lost reply after real native commit") + return result + monkeypatch.setattr(transport, "_post", observed_post) + if lose_response: + with pytest.raises(TimeoutError): + transport.main(args) + assert phase.read_bytes() == (control / "result.json").read_bytes() == b"" + retained_epoch = json.loads((control / "phase-deadline.restore.json").read_bytes())["deadline_epoch"] + native_state = copy.deepcopy((destination.state.judge.tokens, destination.state.judge.run_history)) + now[0] += 60 + transport.main(args + ["--recover"]) + transport.main(args + ["--recover"]) + assert int(phase.read_bytes()) == retained_epoch + assert (destination.state.judge.tokens, destination.state.judge.run_history) == native_state + assert routes == ["/api/v1/checkpoint/restore", "/api/v1/checkpoint/restore-result", + "/api/v1/checkpoint/restore-result"] + else: + transport.main(args) + fresh = json.loads((control / "result.json").read_bytes())["token"] assert fresh != token and len(grades) == 1 assert grade(second, fresh, 3).status_code == 429 now[0] += 3601 + continued_score = 3 if phase_budget: # The actual SForge phase shell consumes the restored duration # inside a disposable Linux container, then writes a new candidate. - prefix = host_phase_command(tmp_path, monkeypatch) - guest_control = "/opt/loopx-benchmark/control" - command = prefix + f"printf '%s' \"$remaining\"; printf '{{\"score\":3}}' > '{guest_control}/candidate.json'" - guest = client.containers.create(image.id, ["/bin/bash", "-c", command], - environment={"PATH": guest_control + "/bin:/usr/bin:/bin"}, - network_disabled=True, mem_limit="128m", nano_cpus=1_000_000_000) + prefix = host_phase_command(tmp_path, monkeypatch) + guest_control = "/opt/loopx-benchmark/control" + command = prefix + f"printf '%s' \"$remaining\"; printf '{{\"score\":3}}' > '{guest_control}/candidate.json'" + guest = client.containers.create(image.id, ["/bin/bash", "-c", command], + environment={"PATH": guest_control + "/bin:/usr/bin:/bin"}, + network_disabled=True, mem_limit="128m", nano_cpus=1_000_000_000) + try: + # Upload explicitly: a remote Docker daemon cannot read + # the caller's temporary directory through a host mount. + upload = io.BytesIO() + with tarfile.open(fileobj=upload, mode="w") as archive: + for relative in ("opt", "opt/loopx-benchmark", "opt/loopx-benchmark/control", + "opt/loopx-benchmark/control/bin"): + member = tarfile.TarInfo(relative) + member.type, member.mode = tarfile.DIRTYPE, 0o700 + archive.addfile(member) + for relative, data, mode in ( + ("phase-deadline", phase.read_bytes(), 0o600), + ("bin/date", f"#!/bin/sh\nprintf '%s\\n' {int(now[0])}\n".encode(), 0o700)): + member = tarfile.TarInfo("opt/loopx-benchmark/control/" + relative) + member.size, member.mode = len(data), mode + archive.addfile(member, io.BytesIO(data)) + assert guest.put_archive("/", upload.getvalue()) + guest.start() + assert guest.wait(timeout=30)["StatusCode"] == 0 + assert 3580 - (60 if lose_response else 0) < int(guest.logs()) <= 3600 - (60 if lose_response else 0) + stream, _ = guest.get_archive(guest_control + "/candidate.json") try: - # Upload explicitly: a remote Docker daemon cannot read - # the caller's temporary directory through a host mount. - upload = io.BytesIO() - with tarfile.open(fileobj=upload, mode="w") as archive: - for relative in ("opt", "opt/loopx-benchmark", "opt/loopx-benchmark/control", - "opt/loopx-benchmark/control/bin"): - member = tarfile.TarInfo(relative) - member.type, member.mode = tarfile.DIRTYPE, 0o700 - archive.addfile(member) - for relative, data, mode in ( - ("phase-deadline", phase.read_bytes(), 0o600), - ("bin/date", f"#!/bin/sh\nprintf '%s\\n' {int(now[0])}\n".encode(), 0o700)): - member = tarfile.TarInfo("opt/loopx-benchmark/control/" + relative) - member.size, member.mode = len(data), mode - archive.addfile(member, io.BytesIO(data)) - assert guest.put_archive("/", upload.getvalue()) - guest.start() - assert guest.wait(timeout=30)["StatusCode"] == 0 - assert 3580 < int(guest.logs()) <= 3600 - stream, _ = guest.get_archive(guest_control + "/candidate.json") - try: - with tarfile.open(fileobj=io.BytesIO(b"".join(stream))) as archive: - assert json.loads(archive.extractfile("candidate.json").read()) == {"score": 3} - finally: - stream.close() + with tarfile.open(fileobj=io.BytesIO(b"".join(stream))) as archive: + continued_score = json.loads(archive.extractfile("candidate.json").read())["score"] + assert continued_score == 3 finally: - guest.remove(force=True) - assert grade(second, fresh, 3).json()["round_id"] == "agent-2" + stream.close() + finally: + guest.remove(force=True) + assert grade(second, fresh, continued_score).json()["round_id"] == "agent-2" history = requests.get(second + "/api/v1/history", params={"token": fresh}, timeout=5).json() assert history["best_score"] == 7 and history["best_round"] == "agent-1" assert grade(second, fresh, 99).status_code == 429 and len(grades) == 2 From 295b3571d9db3a0a67c45d4685f72002d8ad3a19 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 22:09:37 +0800 Subject: [PATCH 52/69] docs(checkpoint): reconcile native phase recovery qualification Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- .../rfcs/hierarchical-agent-stride-control-v0.md | 15 +++++++++++---- .../hierarchical-agent-stride-control-v0.zh-CN.md | 10 +++++++--- 2 files changed, 18 insertions(+), 7 deletions(-) diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md index f9dc176a5f..eb9870a8c8 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md @@ -860,9 +860,15 @@ timestamp. Restore recomputes the remaining duration and anchors a new Host phas file at activation, charging transport rather than offline time or a fresh trial. The existing work reserve and native opportunity owner remain unchanged. An uncertain response leaves an unusable phase reservation, not a fresh-budget -fallback. Full Host provenance/clock alignment, joint research/effect adoption -and original-operation recovery still need their profile's qualification; this -participant transport cannot certify them. +fallback. Explicit controller recovery now reads the original native activation +and retains the destination deadline durably bound before the first admission. +Lost replies and partial phase writes recover without a new registration, +round, cooldown or budget reset; exhausted or mismatched attempts refuse. +An isolated real native HTTP/Docker journey verifies recovery, the ordinary +Linux phase-entry shell, candidate production and the next accepted grade. +Model inference is excluded. Full Host provenance/clock alignment and joint +session/research/effect recovery still need their profile's qualification; +this participant transport cannot certify them. The composed local continuation qualification now carries the evaluator capsule inside the verified conversation/code/context artifact. The actual resumed Codex @@ -892,7 +898,8 @@ IDs because the interactive listing hides unused forks; native APIs verify their meaning, and no native rows are rewritten. Tests inject pre- and post-binding reply loss, recover through the CLI and continue real native governed work with a deterministic local provider; absent/multiple outcomes and foreign destination -attempts reject. Joint evaluator/Host activation recovery remains unqualified. +attempts reject. The controller participant recovery above does not yet qualify +automatic joint session/research/evaluator/Host activation recovery. An inert verification unit grants no execution authority. The destination uses fresh experiment/Host identities and the existing Goal/session/state admission diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md index 62820dc7d9..557bd08d00 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md @@ -718,9 +718,13 @@ payload 或导出参与者；opaque 内容仍私有且未获隐私认证。此 phase 文件参与者还将声明的 Host deadline 与 native 捕获时间绑定；恢复时重算 剩余时长，在激活处锚定新的 Host phase 文件，计入传输时间，不计离线等待， 也不重置为新试验的完整预算。既有工作预留和 native 评测机会 owner 不变。 -不确定响应留下不可执行的 phase 预留，不能回退到新预算。完整 Host 来源/ -时钟对齐、联合研究/effect 采用与原操作恢复仍须按 profile 资格化；参与者 -传输本身不能证明这些结果。 +不确定响应留下不可执行的 phase 预留，不能回退到新预算。显式 controller +恢复现可读回原 native 激活，并沿用首次准入前持久绑定的目标 deadline。 +响应丢失或 phase 部分写入可恢复，不新建注册、不补轮次、不重置冷却或预算； +耗尽与不匹配的尝试拒绝。隔离的真实 native HTTP/Docker 链路已验证恢复、 +普通 Linux phase 入口、合成候选产出及下一次受理评分；不包含模型推理。 +完整 Host 来源/时钟对齐及联合会话/研究/effect 恢复仍须按 profile 资格化， +参与者传输本身不能证明这些结果。 组合式本地接续验证现将 evaluator capsule 放入同一已验证的会话/代码/上下文产物。 实际恢复的 Codex 工具读取保留研究与公开反馈，从捕获代码计算合成候选，经原生 From 3139ace480d892bea0c7b88f3a52d787fd85fff0 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Tue, 13 Oct 2026 22:49:59 +0800 Subject: [PATCH 53/69] fix(benchmark): retain SQLite journals across worker capture Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/edgebench/checkpoint_worker.py | 34 +++++++- benchmark/tests/test_checkpoint_worker.py | 86 ++++++++++++++++++- .../hierarchical-agent-stride-control-v0.md | 9 +- 3 files changed, 125 insertions(+), 4 deletions(-) diff --git a/benchmark/edgebench/checkpoint_worker.py b/benchmark/edgebench/checkpoint_worker.py index 4b9795560d..2f66fd58fb 100644 --- a/benchmark/edgebench/checkpoint_worker.py +++ b/benchmark/edgebench/checkpoint_worker.py @@ -83,6 +83,38 @@ def _archive(container, remote: str, destination: Path) -> Path: return _physical(destination / root) +def _archive_component(container, remote: str, destination: Path) -> Path: + """Transport a standalone SQLite participant's committed storage too. + + A paused process can still have committed data only in its WAL. Reading + the main file alone silently exports an older, internally valid database. + The typed artifact owner's SQLite backup still decides the committed + snapshot; this Docker adapter only transports its physical input while + the same writer barrier is held. A directory archive already includes it. + """ + from docker.errors import NotFound + + copied = _archive(container, remote, destination) + if not copied.is_file(): + return copied + with copied.open("rb") as source: + if source.read(16) != b"SQLite format 3\0": + return copied + # SHM is a reader index, rebuilt locally. WAL and rollback journals can + # affect committed state and must not be dropped or treated as optional + # when transport, permissions or link validation fails. + for suffix in ("-wal", "-journal"): + with tempfile.TemporaryDirectory(prefix=".sqlite-transport-", dir=destination) as stage: + try: + companion = _archive(container, remote + suffix, Path(stage) / "member") + except NotFound: + continue + if not companion.is_file(): + raise ValueError("SQLite companion must be a physical regular file") + companion.rename(copied.with_name(copied.name + suffix)) + return copied + + def _worker_lock(client, container_id: str) -> Path: # Controller banks and endpoint aliases may differ for the same daemon. # Coordinate captures by this user on this host, not by artifact location. @@ -151,7 +183,7 @@ def capture_worker(controller_file: Path, *, token: str, capture_id: str, if not container.attrs["State"]["Paused"]: raise RuntimeError("Worker writer barrier was not established") for entry in request["components"]: - copied = _archive(container, entry["source_path"], stage / "workers" / entry["id"]) + copied = _archive_component(container, entry["source_path"], stage / "workers" / entry["id"]) components.append({"id": entry["id"], "source_path": str(copied)}) phase = _archive(container, request["phase_deadline_path"], stage / "phase") native = capture(controller_file, token=token, capture_id=capture_id, diff --git a/benchmark/tests/test_checkpoint_worker.py b/benchmark/tests/test_checkpoint_worker.py index 2736c59b25..7a64a4cf8e 100644 --- a/benchmark/tests/test_checkpoint_worker.py +++ b/benchmark/tests/test_checkpoint_worker.py @@ -4,10 +4,12 @@ import io import json import os +import sqlite3 import tarfile import threading import time import uuid +from pathlib import Path import pytest @@ -32,10 +34,19 @@ def writer(tmp_path): image = os.environ.get("LOOPX_CHECKPOINT_WRITER_IMAGE", "python:3.12-slim") client.images.get(image) # Never pull implicitly for this test. script = """from pathlib import Path -import time +import time, sqlite3 Path('/data').mkdir() Path('/phase').write_text(str(int(time.time())+7200)) Path('/link').symlink_to('/data', target_is_directory=True) +db=sqlite3.connect('/data/research.sqlite') +db.execute('PRAGMA journal_mode=WAL') +db.execute('PRAGMA wal_autocheckpoint=0') +db.execute('CREATE TABLE evidence (value INTEGER)') +db.commit() +db.execute('PRAGMA wal_checkpoint(TRUNCATE)') +db.execute('INSERT INTO evidence VALUES (42)') +db.commit() +db.execute('INSERT INTO evidence VALUES (99)') count=0 while True: count+=1 @@ -65,6 +76,79 @@ def counter(container): return int(archive.extractfile("counter").read()) +@pytest.mark.parametrize("directory", [False, True]) +def test_paused_sqlite_capture_retains_committed_wal_not_pending_transaction( + controllers, writer, tmp_path, directory): + """A valid main DB can omit all recent research while integrity_check passes.""" + source, _, first, _, files, _ = controllers + container, request = writer + token = register(source, first) + payload = json.loads(request.read_text()) + payload["components"] = [{"id": "research", "source_path": + "/data" if directory else "/data/research.sqlite"}] + request.write_text(json.dumps(payload)) + unit = tmp_path / "wal-point" + capture_worker(files[0], token=token, capture_id="b" * 32, + destination=unit, request_file=request) + captured = unit / "components/research" + database = captured / "research.sqlite" if directory else captured + with sqlite3.connect(database) as readback: + assert readback.execute("PRAGMA integrity_check").fetchone() == ("ok",) + assert readback.execute("SELECT value FROM evidence").fetchall() == [(42,)] + assert not database.with_name(database.name + "-wal").exists() + container.reload() + assert container.attrs["State"]["Running"] and not container.attrs["State"]["Paused"] + + +@pytest.mark.parametrize("fault", ["symlink", "directory", "transport"]) +def test_sqlite_companion_failure_cannot_publish_an_older_valid_database( + controllers, writer, tmp_path, monkeypatch, fault): + from docker.errors import APIError + from benchmark.edgebench import checkpoint_worker + + source, _, first, _, files, grades = controllers + container, request = writer + token = register(source, first) + payload = json.loads(request.read_text()) + payload["components"] = [{"id": "research", "source_path": "/data/research.sqlite"}] + request.write_text(json.dumps(payload)) + if fault == "transport": + original = checkpoint_worker._archive + def unavailable(worker, remote, destination): + if remote.endswith("-wal"): + raise APIError("Synthetic Docker archive transport failure") + return original(worker, remote, destination) + monkeypatch.setattr(checkpoint_worker, "_archive", unavailable) + else: + replacement = "p.symlink_to('/phase')" if fault == "symlink" else "p.mkdir()" + assert container.exec_run(["python", "-c", "from pathlib import Path; " + "p=Path('/data/research.sqlite-wal'); p.unlink(); " + replacement]).exit_code == 0 + unit = tmp_path / "invalid-point" + with pytest.raises((ValueError, APIError)): + capture_worker(files[0], token=token, capture_id="c" * 32, + destination=unit, request_file=request) + assert not unit.exists() and not grades + assert not (Path(json.loads(files[0].read_text())["bank_root"]) / ("c" * 32)).exists() + container.reload() + assert container.attrs["State"]["Running"] and not container.attrs["State"]["Paused"] + + +def test_non_sqlite_file_does_not_capture_similarly_named_neighbors(controllers, writer, tmp_path): + source, _, first, _, files, _ = controllers + container, request = writer + token = register(source, first) + assert container.exec_run(["python", "-c", + "from pathlib import Path; Path('/data/counter-wal').write_text('PRIVATE_NEIGHBOR')"]).exit_code == 0 + payload = json.loads(request.read_text()) + payload["components"] = [{"id": "counter", "source_path": "/data/counter"}] + request.write_text(json.dumps(payload)) + unit = tmp_path / "ordinary-point" + capture_worker(files[0], token=token, capture_id="d" * 32, + destination=unit, request_file=request) + assert all(b"PRIVATE_NEIGHBOR" not in member.read_bytes() + for member in unit.rglob("*") if member.is_file()) + + def test_controller_cli_captures_frozen_worker_with_original_native_point( controllers, writer, tmp_path, monkeypatch, capsys): source, destination, first, second, files, grades = controllers diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md index eb9870a8c8..fadbb8e242 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md @@ -832,8 +832,13 @@ The explicit Docker controller now offers a bounded writer barrier: pause the owned running worker, archive declared files and its phase clock through the daemon, capture the native evaluator capsule, then unpause before publishing an inert joint unit. A live-writer/native-HTTP qualification proves file stability -and original opportunity retention; cancellation, lost pause responses and failed -capture exercise cleanup. An existing pause is not owned or released, and failed +and original opportunity retention. Standalone SQLite participants also transport +their WAL and rollback journal under that same pause, then use the existing +SQLite snapshot owner: a valid main database alone can omit committed research. +Committed WAL rows survive, uncommitted rows stay excluded, and unavailable or +linked companions reject capture before native publication. Cancellation, lost +pause responses and failed capture exercise cleanup. An existing pause is not +owned or released, and failed resume prevents publication. This closes the manual provider's file-writer barrier gap only. No process memory, native cursor eligibility, external writers, automatic pre-decision handoff or isolated continuation are qualified by it. From 4b2ac8ca3a1c079064df80708ddf64812467a936 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Wed, 14 Oct 2026 00:43:19 +0800 Subject: [PATCH 54/69] fix(benchmark): interrupt owned native CLI at held guard boundary Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/README.md | 17 +++ benchmark/runtime/checkpoint.py | 36 ++++- benchmark/runtime/worker.py | 57 ++++++-- .../tests/test_native_checkpoint_handoff.py | 137 ++++++++++++++++++ .../hierarchical-agent-stride-control-v0.md | 11 ++ ...rarchical-agent-stride-control-v0.zh-CN.md | 8 + 6 files changed, 247 insertions(+), 19 deletions(-) diff --git a/benchmark/README.md b/benchmark/README.md index b5e47369d9..5204a389ca 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -204,6 +204,23 @@ changed capture or the 120-second transport deadline fails the call while retaining the original admission and captured decision. Recover that original identity; do not obtain a replacement guard to reconstruct a missed point. +An explicit native CLI caller can pass its live `child_process` handle as +`native_process` to `held_quota_decision`. The Host interrupts its owned process +group, requires graceful leader exit, and reaps descendants before calling the +selector or entering the capture body. Failed interruption stops that capture; +it does not turn an unfinished native history into a completed cursor. After a +successful context exit, `continuation_body` contains the original guard for an +explicit resume of **that source session**. The killed tool remains aborted in +native history; the packet is supplied on a new native turn and is never rerun. +A dead source socket needs no acknowledgement in this path. Omit the process +argument to retain the original output-only transport. + +A real native CLI regression holds this interrupted boundary beyond the tool's +yield interval and verifies retained context, explicit original-packet delivery +and one guard receipt. This qualifies interruption/recovery after the Host +receives the packet, not absence of earlier inference, independent writer +quiescence, or an automatically captured research checkpoint. + This transport is a prerequisite, **not automatic mid-wake checkpoint capture**. The actual native Host/CLI test binds the pending command event, forks while the real guard output is withheld, and verifies no duplicate guard receipt. It does diff --git a/benchmark/runtime/checkpoint.py b/benchmark/runtime/checkpoint.py index 80122f218a..7f9ad2893c 100644 --- a/benchmark/runtime/checkpoint.py +++ b/benchmark/runtime/checkpoint.py @@ -476,7 +476,7 @@ def capture_after_wake(env: dict[str, str], receipt: dict[str, Any], *, @contextmanager def held_quota_decision(listener, capture_root: Path, *, goal_id: str, agent_id: str, - turn_instance_id: str): + turn_instance_id: str, native_process=None, interrupt_timeout: float = 10): """Host-side counterpart of the explicit quota pre-output transport. The existing typed owner decides whether this exact packet admits a fresh @@ -484,6 +484,14 @@ def held_quota_decision(listener, capture_root: Path, *, goal_id: str, agent_id: releases the original tool response; it certifies no research checkpoint. Failure/disconnection never retries the guard or substitutes a new point. The listener must already be bound by the Host in a private directory. + + Passing the live child_process-owned native CLI explicitly interrupts and + reaps it before selection/capture. Its pending tool is aborted; normal exit + returns a source-only continuation body containing the original packet, + rather than acknowledging a dead socket. This is not a complete workspace + barrier, destination admission, or a qualified interrupted native cursor. + Without native_process this is output ordering only; a slow capture can + outlive the native tool's yield interval and does not stop inference. """ if not capture_root.is_absolute() or capture_root.resolve(strict=True) != capture_root: raise ValueError("decision capture root must be a physical absolute directory") @@ -504,11 +512,33 @@ def held_quota_decision(listener, capture_root: Path, *, goal_id: str, agent_id: if hashlib.sha256(raw).hexdigest() != message["sha256"]: raise ValueError("decision handoff digest mismatch") quota = json.loads(raw) + interrupted = None + if native_process is not None: + from .worker import interrupt_checkpoint_process + + # Interrupt before the typed selector: even this read can outlive a + # native tool yield. Keep original bytes for same-identity recovery. + interrupted = interrupt_checkpoint_process(native_process, timeout=interrupt_timeout) intent = effect_runtime_result("runtime.checkpoint_artifacts", { "action": "select_replan", "quota": quota, "goal_id": goal_id, "agent_id": agent_id, "turn_instance_id": turn_instance_id, }) - yield {"path": path, "sha256": message["sha256"], "quota": quota, "intent": intent} + held = {"path": path, "sha256": message["sha256"], "quota": quota, "intent": intent} + if native_process is not None: + held["native_process_return_code"] = interrupted + yield held if path.read_bytes() != raw: raise ValueError("original decision changed while held") - channel.sendall(json.dumps({"released": True, "sha256": message["sha256"]}).encode() + b"\n") + if native_process is None: + channel.sendall(json.dumps({"released": True, "sha256": message["sha256"]}).encode() + b"\n") + else: + # Expose only after capture/readback succeeds. A failing consumer + # cannot mistake the interruption for ordinary continuation. + held["continuation_body"] = ( + "The Host interrupted this source session while its quota tool response was held. " + "The pending tool was aborted; its guard already committed. Consume the original " + "packet below in this same source session and preserve its Turn identity, required " + "reads, selection and settlement instructions. Do not rerun that guard or infer " + "a successful checkpoint from this message. This grants no destination authority.\n\n" + + raw.decode("utf-8") + ) diff --git a/benchmark/runtime/worker.py b/benchmark/runtime/worker.py index 9a950fd1e4..6678db2d85 100644 --- a/benchmark/runtime/worker.py +++ b/benchmark/runtime/worker.py @@ -100,22 +100,47 @@ def child_process(command: list[str], *, env: dict[str, str], stdout, stderr): try: yield process finally: - # Reap descendants even when their leader exited or Harbor cancelled us. - try: - # Python's standard SIGINT handler lets the Turn host's finally - # path reap its separately grouped Codex child before forced kill. - os.killpg(process.pid, signal.SIGINT) - except ProcessLookupError: - pass - try: - process.wait(timeout=5) - except subprocess.TimeoutExpired: - pass - try: - os.killpg(process.pid, signal.SIGKILL) - except ProcessLookupError: - pass - process.wait(timeout=10) + _reap_child_process(process) + + +def _reap_child_process(process): + """Reap the process group created by child_process, including descendants.""" + try: + os.killpg(process.pid, signal.SIGINT) + except ProcessLookupError: + pass + try: + process.wait(timeout=5) + except subprocess.TimeoutExpired: + pass + try: + os.killpg(process.pid, signal.SIGKILL) + except ProcessLookupError: + pass + process.wait(timeout=10) + + +def interrupt_checkpoint_process(process, *, timeout: float = 10) -> int: + """Close an owned native CLI before inspecting a mid-wake checkpoint. + + A held tool response is not a native execution barrier: the native tool can + yield and schedule more inference. Require graceful leader exit so native + history can flush, then reap its group. An unacknowledged interruption is a + missed capture, not a completed cursor or permission to continue elsewhere. + This does not freeze independent workspace writers. + """ + import math + + if not math.isfinite(timeout) or timeout <= 0: + raise ValueError("checkpoint interruption requires a finite positive timeout") + if (not isinstance(process, subprocess.Popen) or process.poll() is not None or + os.getpgid(process.pid) != process.pid or os.getsid(process.pid) != process.pid): + raise ValueError("checkpoint interruption requires a live owned child process group") + try: + os.killpg(process.pid, signal.SIGINT) + return process.wait(timeout=timeout) + finally: + _reap_child_process(process) def turn_command( diff --git a/benchmark/tests/test_native_checkpoint_handoff.py b/benchmark/tests/test_native_checkpoint_handoff.py index b962ed19f1..189ab52460 100644 --- a/benchmark/tests/test_native_checkpoint_handoff.py +++ b/benchmark/tests/test_native_checkpoint_handoff.py @@ -204,3 +204,140 @@ def response(request, index, message): assert len(calls) == 3 and "heartbeat_receipt" in json.dumps(requests[2]) assert _heartbeat_receipt_count(runtime, TURN_ID) == receipts assert held["path"].read_bytes() # Original receipt stays recoverable. + + +@pytest.mark.skipif(not os.environ.get("LOOPX_NATIVE_CHECKPOINT_BIN") or os.name != "posix", + reason="Requires a real native Host and local Unix transport") +@pytest.mark.parametrize("outcome", ["resume", "capture_failure", "changed_packet"]) +def test_native_cli_interruption_retains_guard_beyond_tool_yield(tmp_path, outcome): + """A real native CLI stops, then explicitly resumes the one committed guard.""" + import socket + import sys + import tempfile + from pathlib import Path + + from benchmark.runtime.checkpoint import held_quota_decision + from tests.control_plane.test_quota_settlement_cli import ( + AGENT_ID, GOAL_ID, TURN_ID, _write_fixture, _configure_autonomous_replan_fixture, + _heartbeat_receipt_count, + ) + + project, runtime, registry = _write_fixture(tmp_path / "authority") + _configure_autonomous_replan_fixture(project, runtime, registry) + capture_root = project / "captures" + capture_root.mkdir(mode=0o700) + requests = [] + with tempfile.TemporaryDirectory(prefix="handoff-", dir="/tmp") as short: + socket_path = Path(short).resolve() / "host.sock" + with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as listener: + listener.bind(str(socket_path)) + listener.listen(1) + listener.settimeout(30) + argv = [sys.executable, "-m", "loopx.cli", "--format", "json", "--registry", str(registry), + "--runtime-root", str(runtime), "quota", "should-run", "--goal-id", GOAL_ID, + "--agent-id", AGENT_ID, "--turn-instance-id", TURN_ID, + "--runtime-profile", "generic_cli", "--available-capability", "shell", + "--available-capability", "filesystem_read", "--decision-output-root", str(capture_root), + "--decision-handoff-socket", str(socket_path)] + command = "PYTHONPATH=" + shlex.quote(str(Path(__file__).resolve().parents[2])) + " " + shlex.join(argv) + + def response(request, index, message): + requests.append(request) + if index == 2: + return {"type": "function_call", "id": "fc_guard", "call_id": "call_guard", + "name": "exec_command", "arguments": json.dumps({ + "cmd": command, "yield_time_ms": 10000, "max_output_tokens": 200})} + message["content"][0]["text"] = "RESEARCH_BEFORE_GUARD" if index == 1 else "Continued." + return message + + binary = os.environ["LOOPX_NATIVE_CHECKPOINT_BIN"] + with real_native_source(tmp_path / "native", binary, project, close_source=True, + response_builder=response, sandbox="danger-full-access") as (env, source, _, calls): + from types import SimpleNamespace + from benchmark.runtime.worker import child_process, native_command + from benchmark.runtime.codex import Execution + env.update(MODEL_NAME="fixture", REASONING_EFFORT="high") + wake = tmp_path / "cli-wake" + wake.mkdir() + cli_command = native_command(env, Execution(context="resume", sandbox="danger-full-access"), + "execute", wake, SimpleNamespace(session_id=source)) + with (wake / "stdout.jsonl").open("w") as output, (wake / "stderr.log").open("w") as error: + with child_process(cli_command, env=env, stdout=output, stderr=error) as process: + process.stdin.write("Execute the original synthetic guard.\n") + process.stdin.flush() + process.stdin.close() + from contextlib import nullcontext + expected = (nullcontext() if outcome == "resume" else + pytest.raises(ValueError, match="capture failed|decision changed")) + with expected: + with held_quota_decision(listener, capture_root, goal_id=GOAL_ID, + agent_id=AGENT_ID, turn_instance_id=TURN_ID, + native_process=process) as held: + assert held["intent"]["capture"] and len(calls) == 2 + assert process.poll() is not None + assert "continuation_body" not in held + receipts = _heartbeat_receipt_count(runtime, TURN_ID) + if outcome == "capture_failure": + raise ValueError("capture failed") + if outcome == "changed_packet": + held["path"].write_text("{}") + else: + time.sleep(12) # Exceeds the real tool's 10-second yield. + assert len(calls) == 2 + assert _heartbeat_receipt_count(runtime, TURN_ID) == receipts + if outcome != "resume": + assert "continuation_body" not in held + assert process.poll() is not None and len(calls) == 2 + assert _heartbeat_receipt_count(runtime, TURN_ID) == receipts + return + original = held["path"].read_text() + assert held["continuation_body"].endswith(original) + result = subprocess.run(cli_command, env=env, cwd=project, + input=held["continuation_body"], + text=True,capture_output=True, timeout=30) + assert result.returncode == 0, result.stderr + assert len(calls) == 3 and "heartbeat_receipt" in json.dumps(requests[2]) + assert _heartbeat_receipt_count(runtime, TURN_ID) == receipts + assert "RESEARCH_BEFORE_GUARD" in json.dumps(requests[2]) + assert any(item.get("type") == "function_call_output" and + item.get("call_id") == "call_guard" and + "aborted" in item.get("output", "") + for item in requests[2]["input"]) + + +@pytest.mark.skipif(os.name != "posix", reason="Requires owned Unix process groups") +def test_checkpoint_interrupt_rejects_shared_group_and_invalid_timeout(tmp_path): + import sys + from benchmark.runtime.worker import interrupt_checkpoint_process + + with subprocess.Popen([sys.executable, "-c", "import time; time.sleep(20)"]) as process: + try: + with pytest.raises(ValueError, match="positive timeout"): + interrupt_checkpoint_process(process, timeout=float("nan")) + with pytest.raises(ValueError, match="owned child process group"): + interrupt_checkpoint_process(process) + assert process.poll() is None # No signal to the caller's own group. + finally: + process.terminate() + process.wait(timeout=5) + + +@pytest.mark.skipif(os.name != "posix", reason="Requires owned Unix process groups") +def test_checkpoint_interrupt_timeout_reaps_without_qualifying_capture(tmp_path): + import sys + from benchmark.runtime.worker import child_process, interrupt_checkpoint_process + + ready = tmp_path / "ready" + code = ("import signal,time; from pathlib import Path; " + "signal.signal(signal.SIGINT, signal.SIG_IGN); " + f"Path({str(ready)!r}).touch(); time.sleep(30)") + with child_process([sys.executable, "-c", code], + env={**os.environ, "LOOPX_PROJECT": str(tmp_path)}, + stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) as process: + deadline = time.monotonic() + 5 + while not ready.exists() and time.monotonic() < deadline: + time.sleep(.01) + assert ready.exists() + with pytest.raises(subprocess.TimeoutExpired): + interrupt_checkpoint_process(process, timeout=.1) + assert process.poll() is not None diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md index fadbb8e242..744e63c99a 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md @@ -757,6 +757,17 @@ quiescence, effect reconciliation, automatic worker/controller connection and isolated useful continuation remain required. The completed-cursor acceptance check remains unchanged. +The existing CLI execution path now has an explicit interruption/recovery +adapter at the rendezvous: it closes the owned native process group before +selection and capture, requires graceful leader exit, reaps descendants, and +returns the retained original guard only after successful context exit. A real +CLI test exceeds the tool-yield interval and resumes the same source session +without another quota receipt. The aborted tool remains visible; this is a new +native turn, not transparent live-stack recovery. The adapter does not yet prove +that no model action preceded its interruption. Automatic worker wiring, +interrupted-cursor validation and the controller's all-writer barrier remain the +next joint acceptance; completed-cursor checks remain strict. + The Host profile must also declare path portability. A native index can retain absolute guest paths: qualification must either preserve those paths inside a fresh isolated environment or use the Host's supported import/rebinding API. diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md index 557bd08d00..c40f13ce14 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md @@ -631,6 +631,14 @@ fork，并释放同一响应，不新增 guard receipt。丢失确认或字节 静止、effect 核回、worker/controller 自动衔接和隔离有效接续仍须完成。 completed cursor 的验收检查保持不变。 +既有 CLI 路径现在提供显式中断/恢复适配：在 rendezvous 收到原包后，先关闭 +Host 拥有的原生进程组，要求主进程正常响应中断退出并清理后代，再进行选择和 +捕获；只有捕获上下文成功退出才返回原 guard 的接续正文。真实 CLI 测试等待 +超过工具 yield 周期后，仍能在同一来源 session 显式接续且不新增 quota receipt。 +被中断的工具保留 aborted 记录；接续是新的原生 turn，不是透明恢复执行栈。 +它尚未证明中断之前没有模型动作；自动 worker 接线、interrupted cursor 验证及 +controller 的全部 writer 屏障仍属下一步联合验收，completed cursor 检查不放宽。 + Host profile 还须声明路径可移植性。原生索引可能保留绝对 guest 路径：资格化 须在新隔离环境保留这些路径，或使用 Host 支持的 import/rebinding API；手改 原生数据库行不能资格化导入。有效 Host 配置及已安装工具/skill 版本须与凭证 From 4253bcec4b79dc1d553a556e33a818513228d622 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Wed, 14 Oct 2026 01:10:08 +0800 Subject: [PATCH 55/69] Connect heartbeat worker to checkpoint handoff and source continuation Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/runtime/checkpoint.py | 32 +++- benchmark/runtime/checkpoint_wake.py | 146 ++++++++++++++++++ benchmark/runtime/worker.py | 54 ++++--- .../tests/test_native_checkpoint_handoff.py | 125 +++++++++++++++ loopx/cli_commands/quota_capture.py | 17 +- loopx/cli_commands/support_control.py | 1 + .../support_control_heartbeat_registration.py | 5 + loopx/control_plane/heartbeat/builder.py | 11 ++ loopx/control_plane/heartbeat/task_body.py | 5 +- .../turn_driver/decision_capture.py | 18 +++ .../test_heartbeat_envelope_transport.py | 23 +++ 11 files changed, 389 insertions(+), 48 deletions(-) create mode 100644 benchmark/runtime/checkpoint_wake.py diff --git a/benchmark/runtime/checkpoint.py b/benchmark/runtime/checkpoint.py index 7f9ad2893c..75cbd6e7cc 100644 --- a/benchmark/runtime/checkpoint.py +++ b/benchmark/runtime/checkpoint.py @@ -476,7 +476,8 @@ def capture_after_wake(env: dict[str, str], receipt: dict[str, Any], *, @contextmanager def held_quota_decision(listener, capture_root: Path, *, goal_id: str, agent_id: str, - turn_instance_id: str, native_process=None, interrupt_timeout: float = 10): + turn_instance_id: str | None, native_process=None, interrupt_timeout: float = 10, + interrupt_when=None, deadline: float | None = None): """Host-side counterpart of the explicit quota pre-output transport. The existing typed owner decides whether this exact packet admits a fresh @@ -490,14 +491,24 @@ def held_quota_decision(listener, capture_root: Path, *, goal_id: str, agent_id: returns a source-only continuation body containing the original packet, rather than acknowledging a dead socket. This is not a complete workspace barrier, destination admission, or a qualified interrupted native cursor. + interrupt_when selects capture candidates through the typed intent before + interruption; false releases ordinary/duplicate guards without a native + restart. Selection is not a pre-decision proof: interrupted cursor + qualification must still reject any native work beyond the pending guard. Without native_process this is output ordering only; a slow capture can outlive the native tool's yield interval and does not stop inference. """ if not capture_root.is_absolute() or capture_root.resolve(strict=True) != capture_root: raise ValueError("decision capture root must be a physical absolute directory") + def remaining(): + limit = 120 if deadline is None else min(120, deadline - time.monotonic()) + if limit <= 0: + raise TimeoutError("original wake budget expired during decision handoff") + return limit + channel, _ = listener.accept() with channel: - channel.settimeout(120) + channel.settimeout(remaining()) with channel.makefile("rb") as stream: line = stream.readline(4097) message = json.loads(line) @@ -513,23 +524,28 @@ def held_quota_decision(listener, capture_root: Path, *, goal_id: str, agent_id: raise ValueError("decision handoff digest mismatch") quota = json.loads(raw) interrupted = None - if native_process is not None: + if native_process is not None and interrupt_when is None: from .worker import interrupt_checkpoint_process # Interrupt before the typed selector: even this read can outlive a # native tool yield. Keep original bytes for same-identity recovery. - interrupted = interrupt_checkpoint_process(native_process, timeout=interrupt_timeout) + interrupted = interrupt_checkpoint_process(native_process, timeout=min(interrupt_timeout, remaining())) intent = effect_runtime_result("runtime.checkpoint_artifacts", { "action": "select_replan", "quota": quota, "goal_id": goal_id, - "agent_id": agent_id, "turn_instance_id": turn_instance_id, - }) + "agent_id": agent_id, "turn_instance_id": (turn_instance_id if turn_instance_id is not None + else (quota.get("heartbeat_receipt") or {}).get("turn_instance_id")), + }, timeout=remaining()) + if native_process is not None and interrupt_when is not None and interrupt_when(intent): + from .worker import interrupt_checkpoint_process + + interrupted = interrupt_checkpoint_process(native_process, timeout=min(interrupt_timeout, remaining())) held = {"path": path, "sha256": message["sha256"], "quota": quota, "intent": intent} - if native_process is not None: + if interrupted is not None: held["native_process_return_code"] = interrupted yield held if path.read_bytes() != raw: raise ValueError("original decision changed while held") - if native_process is None: + if interrupted is None: channel.sendall(json.dumps({"released": True, "sha256": message["sha256"]}).encode() + b"\n") else: # Expose only after capture/readback succeeds. A failing consumer diff --git a/benchmark/runtime/checkpoint_wake.py b/benchmark/runtime/checkpoint_wake.py new file mode 100644 index 0000000000..1df2eac37c --- /dev/null +++ b/benchmark/runtime/checkpoint_wake.py @@ -0,0 +1,146 @@ +"""Opt-in native CLI rendezvous inside an existing heartbeat wake. + +This adapter owns process/transport ordering, not replan admission or complete +research-point qualification. Typed quota/artifact owners retain those rules. +""" +from __future__ import annotations + +import hashlib +import json +import select +import socket +import subprocess +import tempfile +import time +from contextlib import ExitStack +from dataclasses import replace +from pathlib import Path + +from loopx.control_plane.effect_runtime import effect_runtime_result +from .checkpoint import _mark_replan_attempt, held_quota_decision, read_checkpoint_request + + +class CheckpointWake: + """One private listener survives all source-session resumptions in a wake.""" + + def __init__(self, env, receipt): + self.env, self.receipt = env, receipt + self.stack = ExitStack() + self.listener = None + self.socket_path = None + self.request = None + self.raw = None + self.root = Path(env["LOOPX_WAKE_LOG_DIR"]).parent / "checkpoints" + self.capture_root = Path(env["LOOPX_WAKE_LOG_DIR"]) / receipt["turn_id"] / "decisions" + + def __enter__(self): + path = self.env.get("LOOPX_CHECKPOINT_REQUEST") + if not path or self.receipt["mode"] != "heartbeat" or self.receipt["stage"] != "execute": + return self + try: + self.request, self.raw = read_checkpoint_request(path) + except (OSError, ValueError): + return self # The existing preflight records declaration failure. + if self.request.get("trigger") != "fresh_replan": + return self + if Path(self.request["destination"]) != self.root or self.root.resolve() != self.root: + raise ValueError("fresh_replan destination must be this trial's checkpoints directory") + try: + # AF_UNIX paths are short; trial roots need not be. No ambient socket + # or socket authority is inherited from the caller's environment. + directory = self.stack.enter_context(tempfile.TemporaryDirectory(prefix="loopx-ckpt-", dir="/tmp")) + self.socket_path = str(Path(directory).resolve() / "host.sock") + self.listener = self.stack.enter_context(socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)) + self.listener.bind(self.socket_path) + self.listener.listen(1) + return self + except BaseException: + self.stack.close() + raise + + def __exit__(self, *args): + return self.stack.__exit__(*args) + + def _first_replan(self, intent): + return intent["capture"] and _mark_replan_attempt( + self.root, intent["obligation_id"], self.receipt["turn_id"], + hashlib.sha256(self.raw).hexdigest()) + + def _capture(self, held, deadline): + """Capture declared inert files; never downgrade a full-state request.""" + if any(self.request.get(key) for key in ("native_session", "loopx_state", "evaluation_opportunity")): + raise ValueError("Interrupted full-state capture requires qualified native pending history and the controller writer barrier") + destination = self.root / held["intent"]["obligation_id"] + metadata = {"operator_metadata": self.request["point_metadata"], + "request_sha256": hashlib.sha256(self.raw).hexdigest(), + "session": self.receipt["session"], "replan": held["intent"], + "decision_sha256": held["sha256"], + "capture_order": "after_native_interruption", + "writer_barrier": "owned_native_process_group_only"} + window = min(time.time() + 30, time.time() + deadline - time.monotonic()) + if window <= time.time(): + raise TimeoutError("original wake budget expired before capture") + components = self.request["components"] + probe = effect_runtime_result("runtime.checkpoint_artifacts", { + "action": "probe", "components": components, "deadline_epoch_ms": window * 1000, + }, timeout=max(.1, window - time.time()), retry_safe=False) + self.root.mkdir(mode=0o700, parents=True, exist_ok=True) + return effect_runtime_result("runtime.checkpoint_artifacts", { + "action": "capture", "components": components, "destination": str(destination), + "expected_source_sha256": probe["source_sha256"], "point_metadata": metadata, + "deadline_epoch_ms": window * 1000, + }, timeout=max(.1, window - time.time()), retry_safe=False) + + def run(self, execution, body, wake, session_wake, stdout, stderr, timeout): + from .worker import child_process, native_command + from .sessions import BenchmarkSessionWake + + deadline = time.monotonic() + timeout if timeout is not None else float("inf") + while True: + remaining = deadline - time.monotonic() + if remaining <= 0: + raise subprocess.TimeoutExpired("checkpoint source wake", timeout) + command = native_command(self.env, execution, "execute", wake, session_wake) + continuation = None + with child_process(command, env=self.env, stdout=stdout, stderr=stderr) as process: + process.stdin.write(body) + process.stdin.close() + while process.poll() is None: + remaining = deadline - time.monotonic() + if remaining <= 0: + raise subprocess.TimeoutExpired(command, timeout) + if not select.select([self.listener], [], [], min(.1, remaining))[0]: + continue + with held_quota_decision(self.listener, self.capture_root, + goal_id=self.env["LOOPX_GOAL_ID"], agent_id=self.env["LOOPX_AGENT_ID"], + turn_instance_id=None, native_process=process, + interrupt_timeout=min(10, remaining), interrupt_when=self._first_replan, + deadline=deadline) as held: + if "native_process_return_code" not in held: + continue + stdout.flush() + session_wake.observe(wake / "stdout.jsonl") + event = {"obligation_id": held["intent"]["obligation_id"], + "decision_path": str(held["path"]), "decision_sha256": held["sha256"], + "captured": False, "research_point_qualified": False} + self.receipt.setdefault("checkpoint_handoffs", []).append(event) + try: + event.update(self._capture(held, deadline), captured=True) + except Exception as error: + # A missed optional point cannot discard the original + # committed guard or turn a later retry into its capture. + event.update(error_kind=type(error).__name__, error=str(error)[:500]) + continuation = held.get("continuation_body") + if continuation is not None: + break + return_code = process.wait(timeout=max(.1, deadline - time.monotonic()) if timeout is not None else None) + if continuation is None: + return return_code + # A fresh-context profile still resumes THIS source session after an + # interruption. Only the next ordinary Host wake uses its own policy. + source_session_id = self.receipt["session"]["session_id"] + execution = replace(execution, context="resume") + session_wake = BenchmarkSessionWake(self.env, execution, self.receipt) + if not source_session_id or session_wake.session_id != source_session_id: + raise ValueError("checkpoint source continuation lost its native session") + body = continuation diff --git a/benchmark/runtime/worker.py b/benchmark/runtime/worker.py index 6678db2d85..6ec291afaf 100644 --- a/benchmark/runtime/worker.py +++ b/benchmark/runtime/worker.py @@ -10,7 +10,7 @@ import subprocess import time import uuid -from contextlib import contextmanager +from contextlib import ExitStack, contextmanager from dataclasses import replace from pathlib import Path @@ -38,7 +38,8 @@ def loopx_command(env: dict[str, str]) -> list[str]: ] -def heartbeat_body(env: dict[str, str], turn_id: str, *, native_goal: bool) -> str: +def heartbeat_body(env: dict[str, str], turn_id: str, *, native_goal: bool, + decision_handoff_socket: str | None = None) -> str: command = loopx_command(env) + [ "heartbeat-prompt", "--thin", @@ -59,13 +60,15 @@ def heartbeat_body(env: dict[str, str], turn_id: str, *, native_goal: bool) -> s command += ["--turn-instance-id", turn_id] if env.get("LOOPX_TURN_ENVELOPE", "0") not in {"0", "1"}: raise ValueError("LOOPX_TURN_ENVELOPE must be 0 or 1") - if env.get("LOOPX_TURN_ENVELOPE") == "1": + if env.get("LOOPX_TURN_ENVELOPE") == "1" or decision_handoff_socket is not None: if native_goal: raise ValueError("TurnEnvelope capture requires a host-owned heartbeat Turn") # Wake identity comes from this worker, never from model-authored text. root = Path(env["LOOPX_WAKE_LOG_DIR"]) / turn_id / "decisions" root.mkdir(mode=0o700, exist_ok=False) command += ["--decision-output-root", str(root)] + if decision_handoff_socket is not None: + command += ["--decision-handoff-socket", decision_handoff_socket] result = subprocess.run( command, cwd=env["LOOPX_PROJECT"], @@ -301,6 +304,7 @@ def run_once(env: dict[str, str]) -> dict: "timed_out": False, } session_wake = None + checkpoint_context = ExitStack() pending_path = ( Path(env.get("LOOPX_RUNTIME_ROOT", str(home))) / "benchmark-pending-turn.json" ) @@ -327,6 +331,9 @@ def run_once(env: dict[str, str]) -> dict: wire_api=env.get("CODEX_WIRE_API", "responses"), skills=Path(env["LOOPX_SHARED_SKILLS"]) if execution.uses_loopx else None, ) + from benchmark.runtime.checkpoint_wake import CheckpointWake + + checkpoint_wake = checkpoint_context.enter_context(CheckpointWake(env, receipt)) body = "Finish the task." if stage == "plan": from benchmark.runtime.planning import task_plan_packet @@ -336,7 +343,8 @@ def run_once(env: dict[str, str]) -> dict: (wake / "planning-input.json").write_text(body, encoding="utf-8") (wake / "planning-schema.json").write_text(json.dumps(planning_before["result_schema"])) elif execution.mode in {"heartbeat", "loopx-goal"}: - body = heartbeat_body(env, turn_id, native_goal=execution.native_goal) + body = heartbeat_body(env, turn_id, native_goal=execution.native_goal, + decision_handoff_socket=checkpoint_wake.socket_path) elif execution.mode == "plain": body = Path(env["LOOPX_TASK_DOC"]).read_text(encoding="utf-8") session_wake = None @@ -371,24 +379,25 @@ def run_once(env: dict[str, str]) -> dict: else native_command(env, execution, stage, wake, session_wake) ) with (wake / "stdout.jsonl").open("w") as stdout: - with child_process( - command, env=env, stdout=stdout, stderr=stderr - ) as process: - allowance = 150 if execution.mode == "turn" and stage == "execute" else 0 - if stage == "plan": - limit = env.get("LOOPX_PLANNING_TIMEOUT_SEC") - # SForge planning has no stage cap: the same absolute - # trial deadline bounds this subprocess and its handoff. - timeout = (float(limit) if limit is not None else - max(0, float(env["LOOPX_PHASE_DEADLINE_EPOCH"]) - time.time())) - else: - timeout = (None if execution.timeout_seconds is None - else execution.timeout_seconds + allowance) - process.communicate( - input=body, timeout=timeout - ) - receipt["return_code"] = process.returncode - receipt["ok"] = process.returncode == 0 + allowance = 150 if execution.mode == "turn" and stage == "execute" else 0 + if stage == "plan": + limit = env.get("LOOPX_PLANNING_TIMEOUT_SEC") + # SForge planning has no stage cap: the same absolute + # trial deadline bounds this subprocess and its handoff. + timeout = (float(limit) if limit is not None else + max(0, float(env["LOOPX_PHASE_DEADLINE_EPOCH"]) - time.time())) + else: + timeout = (None if execution.timeout_seconds is None + else execution.timeout_seconds + allowance) + if checkpoint_wake.listener is not None: + receipt["return_code"] = checkpoint_wake.run( + execution, body, wake, session_wake, stdout, stderr, timeout) + receipt["ok"] = receipt["return_code"] == 0 + else: + with child_process(command, env=env, stdout=stdout, stderr=stderr) as process: + process.communicate(input=body, timeout=timeout) + receipt["return_code"] = process.returncode + receipt["ok"] = process.returncode == 0 if stage == "plan" and receipt["ok"]: from benchmark.runtime.planning import task_plan_packet, validate_plan_readback @@ -421,6 +430,7 @@ def run_once(env: dict[str, str]) -> dict: receipt["error_kind"] = type(exc).__name__ raise finally: + checkpoint_context.close() try: if session_wake is not None: session_wake.observe(wake / "stdout.jsonl") diff --git a/benchmark/tests/test_native_checkpoint_handoff.py b/benchmark/tests/test_native_checkpoint_handoff.py index 189ab52460..2e89578821 100644 --- a/benchmark/tests/test_native_checkpoint_handoff.py +++ b/benchmark/tests/test_native_checkpoint_handoff.py @@ -341,3 +341,128 @@ def test_checkpoint_interrupt_timeout_reaps_without_qualifying_capture(tmp_path) with pytest.raises(subprocess.TimeoutExpired): interrupt_checkpoint_process(process, timeout=.1) assert process.poll() is not None + + +@pytest.mark.skipif(not os.environ.get("LOOPX_NATIVE_CHECKPOINT_BIN") or os.name != "posix", + reason="Requires the real native CLI and private Unix transport") +@pytest.mark.parametrize("capture", ["files", "full", "off"]) +def test_actual_worker_handles_midwake_replan_and_duplicate_without_guard_replay(tmp_path, capture): + """Use run_once, its renderer/listener/session owner and the actual quota CLI.""" + import sys + import tomllib + from pathlib import Path + from benchmark.runtime.worker import run_once + from tests.control_plane.test_quota_settlement_cli import ( + AGENT_ID, GOAL_ID, TURN_ID, _write_fixture, _configure_autonomous_replan_fixture, + _heartbeat_receipt_count, _run_cli, + ) + + project, runtime, registry = _write_fixture(tmp_path / "authority") + logs = tmp_path / "trial" / "wakes" + candidate = project / "candidate.txt" + candidate.write_text("original candidate") + request = tmp_path / "request.json" + request.write_text(json.dumps({"destination": str(logs.parent / "checkpoints"), + "components": [{"id": "code", "source_path": str(candidate)}], + "point_metadata": {}, "trigger": "fresh_replan", "native_session": capture == "full"})) + skills = tmp_path / "skills" + skills.mkdir() + cli = tmp_path / "loopx-cli" + cli.write_text("#!/bin/sh\nexec " + shlex.quote(sys.executable) + ' -m loopx.cli "$@"\n') + cli.chmod(0o700) + requests, guards = [], [] + + def texts(value): + if isinstance(value, str): + yield value + elif isinstance(value, list): + for child in value: + yield from texts(child) + elif isinstance(value, dict): + for child in value.values(): + yield from texts(child) + + def response(payload, index, message): + requests.append(payload) + if index == 2: + # The Host preflight was ordinary. A new admitted replan arises + # later in this same native wake; only the fixture state is synthetic. + preflights = list(logs.glob("*/checkpoint-start-guard.json")) + if preflights: + packet = json.loads(preflights[0].read_text()) + identity = packet["heartbeat_receipt"]["settlement_identity"] + rc, refreshed = _run_cli(registry, runtime, "refresh-state", "--goal-id", GOAL_ID, + "--agent-id", AGENT_ID, "--todo-id", identity["todo_id"], + "--turn-instance-id", identity["turn_instance_id"], + "--classification", "validated_progress", "--delivery-batch-scale", "implementation", + "--delivery-outcome", "outcome_progress", "--delivery-boundary", "in_flight_continuation", + "--no-global-sync", "--suppress-external-sinks", cwd=project) + assert rc == 0, json.dumps(refreshed) + rc, settled = _run_cli(registry, runtime, *shlex.split(refreshed["settlement_owed"]["command"])[1:], cwd=project) + assert rc == 0, settled + index_path = runtime / "goals" / GOAL_ID / "runs" / "index.jsonl" + prior_runs = index_path.read_text() if index_path.exists() else "" + _configure_autonomous_replan_fixture(project, runtime, registry) + # Preserve the actual preflight settlement evidence. The reusable + # fixture replaces the index; its two later synthetic observations + # must not erase the already settled original Host Turn. + from datetime import datetime, timezone, timedelta + later = [] + for offset, line in enumerate(index_path.read_text().splitlines(), 1): + row = json.loads(line) + row["generated_at"] = (datetime.now(timezone.utc) + timedelta(seconds=offset)).isoformat() + Path(row["json_path"]).write_text(json.dumps(row)) + later.append(json.dumps(row) + "\n") + index_path.write_text(prior_runs + "".join(later)) + body = next(t for t in texts(payload) if "quota should-run" in t and "```sh" in t) + guard = next(line for line in body.splitlines() if "quota should-run" in line) + guard = guard.replace('${LOOPX_TURN:?}', TURN_ID) + guards.append(guard) + return {"type": "function_call", "id": "fc_midwake", "call_id": "call_midwake", + "name": "exec_command", "arguments": json.dumps({"cmd": guard, + "yield_time_ms": 10000, "max_output_tokens": 100})} + if index == 3 and capture != "off": + # The resumed source deliberately rereads the same obligation. It + # must be released without another capture or native interruption. + return {"type": "function_call", "id": "fc_duplicate", "call_id": "call_duplicate", + "name": "exec_command", "arguments": json.dumps({"cmd": guards[0] + + "\nprintf continued > continued.txt", "yield_time_ms": 10000, + "max_output_tokens": 100})} + message["content"][0]["text"] = "Source research." if index == 1 else "Continued research." + return message + + with real_native_source(tmp_path / "provider", os.environ["LOOPX_NATIVE_CHECKPOINT_BIN"], project, + close_source=True, response_builder=response, sandbox="danger-full-access") as (env, _, _, calls): + base_url = tomllib.loads((Path(env["CODEX_HOME"]) / "config.toml").read_text())["model_providers"]["fixture"]["base_url"] + env.update(LOOPX_CLI=str(cli), LOOPX_REGISTRY=str(registry), LOOPX_RUNTIME_ROOT=str(runtime), + LOOPX_GOAL_ID=GOAL_ID, LOOPX_AGENT_ID=AGENT_ID, LOOPX_WAKE_LOG_DIR=str(logs), + LOOPX_CODEX_HOME=str(tmp_path / "worker-home"), LOOPX_SHARED_SKILLS=str(skills), + MODEL_NAME="fixture", REASONING_EFFORT="high", OPENAI_BASE_URL=base_url, + OPENAI_API_KEY="synthetic-local-only", LOOPX_CODEX_TURN_TIMEOUT_SEC="60") + if capture != "off": + env["LOOPX_CHECKPOINT_REQUEST"] = str(request) + result = run_once(env) + assert result["ok"] and not result["timed_out"], result + assert _heartbeat_receipt_count(runtime, TURN_ID) == 1 + if capture == "off": + assert "checkpoint_handoffs" not in result + assert "--decision-handoff-socket" not in guards[0] + assert not list(logs.glob("*/decisions")) + return + assert len(result["checkpoint_handoffs"]) == 1 + event = result["checkpoint_handoffs"][0] + assert event["captured"] is (capture == "files") + assert event["research_point_qualified"] is False + assert len(calls) == 4 and (project / "continued.txt").read_text() == "continued" + assert "heartbeat_receipt" in json.dumps(requests[2]) + assert any(item.get("type") == "function_call_output" and item.get("call_id") == "call_midwake" + and "aborted" in item.get("output", "") for item in requests[2]["input"]) + if capture == "files": + point = logs.parent / "checkpoints" / event["obligation_id"] + assert (point / "components" / "code").read_text() == "original candidate" + else: + assert "full-state capture" in event["error"] + assert not list((logs.parent / "checkpoints").glob("*/checkpoint.json")) + # The private endpoint belongs to this wake and is gone after readback. + argv = shlex.split(guards[0]) + assert not Path(argv[argv.index("--decision-handoff-socket") + 1]).exists() diff --git a/loopx/cli_commands/quota_capture.py b/loopx/cli_commands/quota_capture.py index 2e45ec8e34..29da4a23fe 100644 --- a/loopx/cli_commands/quota_capture.py +++ b/loopx/cli_commands/quota_capture.py @@ -7,11 +7,11 @@ import os import shlex import socket -import stat import tempfile from pathlib import Path from ..control_plane.quota.error_codes import QuotaCommandValidationError +from ..control_plane.turn_driver.decision_capture import validate_handoff_socket def prepare_decision_capture(args: argparse.Namespace) -> Path | None: @@ -92,21 +92,6 @@ def bind_capture_selection_transport(payload: dict, args: argparse.Namespace) -> selection["command_args_template"] = template + suffix -def validate_handoff_socket(value: str) -> Path: - """A caller-owned local transport, not an authority or admission endpoint.""" - path = Path(value) - try: - if (os.name != "posix" or not hasattr(socket, "AF_UNIX") or not path.is_absolute() or - path.resolve(strict=True) != path or - not stat.S_ISSOCK(path.stat().st_mode) or - path.parent.stat().st_mode & 0o077 or - path.stat().st_uid != os.getuid()): - raise OSError("expected a same-user socket in a private physical directory") - except OSError as exc: - raise QuotaCommandValidationError(f"invalid decision handoff socket: {exc}") from exc - return path - - def handoff_decision(directory: Path | None, socket_path: str | None) -> None: """Withhold the original output until the Host releases these exact bytes. diff --git a/loopx/cli_commands/support_control.py b/loopx/cli_commands/support_control.py index df8ea7a8c6..cdf0135e69 100644 --- a/loopx/cli_commands/support_control.py +++ b/loopx/cli_commands/support_control.py @@ -391,6 +391,7 @@ def handle_support_control_command( turn_granularity=turn_granularity, turn_instance_id=args.turn_instance_id, decision_output_root=args.decision_output_root, + decision_handoff_socket=args.decision_handoff_socket, reward_memory_enabled=reward_memory_enabled, ) if args.bootstrap and payload.get("ok"): diff --git a/loopx/cli_commands/support_control_heartbeat_registration.py b/loopx/cli_commands/support_control_heartbeat_registration.py index 679eaf3ffe..d313de0235 100644 --- a/loopx/cli_commands/support_control_heartbeat_registration.py +++ b/loopx/cli_commands/support_control_heartbeat_registration.py @@ -162,6 +162,11 @@ def register_heartbeat_control_commands( help="Opt into the short TurnEnvelope with full local captures; requires thin mode and an explicit host-owned Turn ID.", ) + heartbeat_prompt_parser.add_argument( + "--decision-handoff-socket", + help="Retain the explicit private Host handoff on generated guards; requires --decision-output-root.", + ) + heartbeat_prequota_parser = subparsers.add_parser( "heartbeat-prequota", help=( diff --git a/loopx/control_plane/heartbeat/builder.py b/loopx/control_plane/heartbeat/builder.py index d4bd8da9a1..41de3a7e2c 100644 --- a/loopx/control_plane/heartbeat/builder.py +++ b/loopx/control_plane/heartbeat/builder.py @@ -175,6 +175,7 @@ def _heartbeat_prompt_commands( capability_args: str, turn_identity_arg: str, decision_output_root: Path | None, + decision_handoff_socket: str | None, ) -> dict[str, str | None]: quota_guard_command = render_quota_guard_command( goal_id, @@ -276,6 +277,8 @@ def _heartbeat_prompt_commands( } if decision_output_root is not None: capture_arg = " --decision-output-root " + shlex.quote(str(decision_output_root)) + if decision_handoff_socket is not None: + capture_arg += " --decision-handoff-socket " + shlex.quote(decision_handoff_socket) for key in ("quota_guard_command", "task_body_quota_guard_command"): commands[key] = str(commands[key]) + " --turn-envelope" + capture_arg commands["thin_prompt_command"] = str(commands["thin_prompt_command"]) + capture_arg @@ -309,6 +312,7 @@ def build_heartbeat_prompt( turn_instance_id: str | None = None, reward_memory_enabled: bool = True, decision_output_root: str | Path | None = None, + decision_handoff_socket: str | None = None, ) -> dict[str, Any]: if not (full or compact or brief or thin): thin = True @@ -372,6 +376,12 @@ def build_heartbeat_prompt( native_goal_host=native_goal_host, turn_instance_id=normalized_turn_instance_id, ) + if decision_handoff_socket is not None: + from ..turn_driver.decision_capture import validate_handoff_socket + + if decision_output_root is None: + raise ValueError("decision handoff requires an explicit decision capture root") + validate_handoff_socket(decision_handoff_socket) explicit_agent_scopes = normalize_agent_scopes(agent_scopes) if explicit_agent_scopes: normalized_agent_scopes = explicit_agent_scopes @@ -458,6 +468,7 @@ def build_heartbeat_prompt( capability_args=capability_args, turn_identity_arg=turn_identity_arg, decision_output_root=decision_output_root, + decision_handoff_socket=decision_handoff_socket, ) cli_preflight = render_cli_preflight(cli_bin=cli_bin) task_body_renderer = _select_task_body_renderer( diff --git a/loopx/control_plane/heartbeat/task_body.py b/loopx/control_plane/heartbeat/task_body.py index a569f70094..2467fb9225 100644 --- a/loopx/control_plane/heartbeat/task_body.py +++ b/loopx/control_plane/heartbeat/task_body.py @@ -678,8 +678,9 @@ def render_thin_heartbeat_task_body( Read `detail_ref.full_decision` for selection, replan, capability context or missing/truncated commands: same-invocation observation, not fresh admission. Selection uses the saved `interaction_contract.cli_channel.selection_command`. -Keep the guard's --turn-envelope and --decision-output-root on every fresh -selection/repair/replan guard; each call saves its own immutable observation. +Keep all supplied guard transport flags (--turn-envelope, --decision-output-root, +and --decision-handoff-socket when present) on every fresh selection/repair/replan +guard; each call saves its own immutable observation. After a workspace or state change, re-enter the guard before continuing work. ```sh diff --git a/loopx/control_plane/turn_driver/decision_capture.py b/loopx/control_plane/turn_driver/decision_capture.py index 44f1e7119a..72adf33ff4 100644 --- a/loopx/control_plane/turn_driver/decision_capture.py +++ b/loopx/control_plane/turn_driver/decision_capture.py @@ -3,12 +3,15 @@ import json import os +import socket +import stat import tempfile from collections.abc import Mapping from pathlib import Path from typing import Any from ..quota.turn_envelope import bind_turn_decision_capture +from ..quota.error_codes import QuotaCommandValidationError def write_decision_capture( @@ -64,3 +67,18 @@ def prepare_turn_decision_capture( path.parent.mkdir(parents=True, exist_ok=True, mode=0o700) write_decision_capture(path, observation) return {**plan, "turn_envelope": bound} + + +def validate_handoff_socket(value: str) -> Path: + """A caller-owned local transport, not an authority or admission endpoint.""" + path = Path(value) + try: + if (os.name != "posix" or not hasattr(socket, "AF_UNIX") or not path.is_absolute() or + path.resolve(strict=True) != path or + not stat.S_ISSOCK(path.stat().st_mode) or + path.parent.stat().st_mode & 0o077 or + path.stat().st_uid != os.getuid()): + raise OSError("expected a same-user socket in a private physical directory") + except OSError as exc: + raise QuotaCommandValidationError(f"invalid decision handoff socket: {exc}") from exc + return path diff --git a/tests/control_plane/test_heartbeat_envelope_transport.py b/tests/control_plane/test_heartbeat_envelope_transport.py index be2a6864f5..03d7a4ef8c 100644 --- a/tests/control_plane/test_heartbeat_envelope_transport.py +++ b/tests/control_plane/test_heartbeat_envelope_transport.py @@ -130,3 +130,26 @@ def test_native_worker_and_stable_bootstrap_cannot_use_capture(tmp_path): assert rc != 0 and rejected['ok'] is False assert _heartbeat_receipt_count(runtime, TURN_ID) == 0 assert not list(tmp_path.glob('decision-*')) + + +@pytest.mark.skipif(os.name != 'posix', reason='Requires a private Unix socket') +def test_renderer_retains_owned_handoff_without_enabling_it_by_default(tmp_path): + import socket + import tempfile + args = dict(goal_id=GOAL_ID, agent_id=AGENT_ID, runtime_profile='generic_cli', + turn_instance_id=TURN_ID, thin=True) + assert build_heartbeat_prompt(**args) == build_heartbeat_prompt(**args, decision_handoff_socket=None) + with tempfile.TemporaryDirectory(prefix='hb-handoff-', dir='/tmp') as directory: + endpoint = str(Path(directory).resolve() / 'host.sock') + with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as listener: + listener.bind(endpoint) + packet = build_heartbeat_prompt(**args, decision_output_root=tmp_path, + decision_handoff_socket=endpoint) + for field in ('quota_guard_command', 'thin_prompt_command'): + argv = shlex.split(packet[field]) + assert argv[argv.index('--decision-handoff-socket') + 1] == endpoint + assert '--decision-handoff-socket when present' in packet['task_body'] + with pytest.raises(ValueError, match='capture root'): + build_heartbeat_prompt(**args, decision_handoff_socket=endpoint) + with pytest.raises(ValueError, match='invalid decision handoff socket'): + build_heartbeat_prompt(**args, decision_output_root=tmp_path, decision_handoff_socket=endpoint) From cb0caa7ffa56e1cc4b39c85fa979381a2cb56a45 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Wed, 14 Oct 2026 01:10:15 +0800 Subject: [PATCH 56/69] Document automatic checkpoint rendezvous and remaining joint qualification Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/README.md | 30 ++++++++++++------- .../hierarchical-agent-stride-control-v0.md | 11 +++++-- ...rarchical-agent-stride-control-v0.zh-CN.md | 8 +++-- 3 files changed, 34 insertions(+), 15 deletions(-) diff --git a/benchmark/README.md b/benchmark/README.md index 5204a389ca..6882a1557a 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -221,16 +221,26 @@ and one guard receipt. This qualifies interruption/recovery after the Host receives the packet, not absence of earlier inference, independent writer quiescence, or an automatically captured research checkpoint. -This transport is a prerequisite, **not automatic mid-wake checkpoint capture**. -The actual native Host/CLI test binds the pending command event, forks while the -real guard output is withheld, and verifies no duplicate guard receipt. It does -not pause writers, reconcile branch effects or certify an interrupted cursor. -Native tools can yield control before their subprocess exits; withholding stdout -alone is not a model or writer barrier. The production worker still needs the -Host rendezvous connected to its controller-owned writer barrier, canonical state -adoption and isolated useful continuation before this profile can launch research -runs. Omit the option to disable the transport; no socket, hold or extra read is -then performed. +The heartbeat worker now installs that listener when its explicit checkpoint +request uses `fresh_replan`. The renderer carries the private handoff endpoint +into guard, selection and reentry commands; this path also uses the existing +captured TurnEnvelope. With checkpointing off the renderer, process lifecycle +and model-visible output stay unchanged. Ordinary guards and repeated replan +obligations are released without restarting the native CLI. On a first replan, +the worker selects through the typed owner, interrupts the source, records its +session, and captures explicitly declared inert files. Failed optional capture +still resumes the source with its original packet; altered packet bytes or lost +session identity fail closed. All resumptions share the original wake deadline. + +This is **not a qualified complete research checkpoint**. Selection before +interruption is not proof that the native tool has never yielded. The worker +records each handoff and currently rejects mid-wake requests for native session, +canonical research or evaluation participants rather than silently reducing them +to files. The completed-cursor path remains strict. The next joint acceptance +must bind the pending native history, the controller's all-writer barrier and +isolated useful continuation before research runs use this profile. Remove the +checkpoint request to disable automatic handoff; a direct caller can omit the +socket option to disable only the transport. Add `"native_session": true` to include Codex's completed native cursor, all retained history pages, rollout/compaction records and native index/history SQLite diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md index 744e63c99a..bc31870ad8 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md @@ -764,9 +764,14 @@ returns the retained original guard only after successful context exit. A real CLI test exceeds the tool-yield interval and resumes the same source session without another quota receipt. The aborted tool remains visible; this is a new native turn, not transparent live-stack recovery. The adapter does not yet prove -that no model action preceded its interruption. Automatic worker wiring, -interrupted-cursor validation and the controller's all-writer barrier remain the -next joint acceptance; completed-cursor checks remain strict. +that no model action preceded its interruption. The actual heartbeat worker now +installs the rendezvous for an explicit fresh-replan request, deduplicates the +original obligation, and resumes the same source session with the original guard +and remaining wake budget. Ordinary/duplicate guards do not restart the native +CLI. Declared file capture and failed optional capture use this real entrypoint; +full-state mid-wake requests remain explicitly unqualified. Interrupted-cursor +validation, the controller's all-writer barrier and isolated joint continuation +remain the next acceptance; completed-cursor checks remain strict. The Host profile must also declare path portability. A native index can retain absolute guest paths: qualification must either preserve those paths inside a diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md index c40f13ce14..5ab2b05693 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md @@ -636,8 +636,12 @@ Host 拥有的原生进程组，要求主进程正常响应中断退出并清理 捕获；只有捕获上下文成功退出才返回原 guard 的接续正文。真实 CLI 测试等待 超过工具 yield 周期后，仍能在同一来源 session 显式接续且不新增 quota receipt。 被中断的工具保留 aborted 记录；接续是新的原生 turn，不是透明恢复执行栈。 -它尚未证明中断之前没有模型动作；自动 worker 接线、interrupted cursor 验证及 -controller 的全部 writer 屏障仍属下一步联合验收，completed cursor 检查不放宽。 +它尚未证明中断之前没有模型动作。实际 heartbeat worker 现已在显式 fresh-replan +请求下安装 rendezvous，按原义务去重，并用原 guard 和剩余 wake 预算接续同一 +来源 session；普通或重复 guard 不重启原生 CLI。声明文件的捕获与可选捕获失败 +均经过该实际入口；中途的完整状态请求仍明确不具备资格。interrupted cursor +验证、controller 的全部 writer 屏障和隔离联合接续仍属下一步验收，completed +cursor 检查不放宽。 Host profile 还须声明路径可移植性。原生索引可能保留绝对 guest 路径：资格化 须在新隔离环境保留这些路径，或使用 Host 支持的 import/rebinding API；手改 From 4b6894fb507eff90dfb2d9459f69bb789e1df9f9 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Wed, 14 Oct 2026 01:46:07 +0800 Subject: [PATCH 57/69] feat(benchmark): retain interrupted native guard checkpoint history Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/runtime/checkpoint_wake.py | 37 ++++- benchmark/runtime/native_checkpoint.py | 130 ++++++++++++++++-- benchmark/tests/test_native_checkpoint.py | 61 ++++++++ .../tests/test_native_checkpoint_handoff.py | 43 +++++- 4 files changed, 250 insertions(+), 21 deletions(-) diff --git a/benchmark/runtime/checkpoint_wake.py b/benchmark/runtime/checkpoint_wake.py index 1df2eac37c..422f7ed99d 100644 --- a/benchmark/runtime/checkpoint_wake.py +++ b/benchmark/runtime/checkpoint_wake.py @@ -6,7 +6,6 @@ from __future__ import annotations import hashlib -import json import select import socket import subprocess @@ -17,7 +16,7 @@ from pathlib import Path from loopx.control_plane.effect_runtime import effect_runtime_result -from .checkpoint import _mark_replan_attempt, held_quota_decision, read_checkpoint_request +from .checkpoint import _mark_replan_attempt, _participant_id, held_quota_decision, read_checkpoint_request class CheckpointWake: @@ -67,9 +66,9 @@ def _first_replan(self, intent): hashlib.sha256(self.raw).hexdigest()) def _capture(self, held, deadline): - """Capture declared inert files; never downgrade a full-state request.""" - if any(self.request.get(key) for key in ("native_session", "loopx_state", "evaluation_opportunity")): - raise ValueError("Interrupted full-state capture requires qualified native pending history and the controller writer barrier") + """Capture declared files/native history; never downgrade research/effects.""" + if any(self.request.get(key) for key in ("loopx_state", "evaluation_opportunity")): + raise ValueError("Interrupted full-state capture requires the controller writer barrier") destination = self.root / held["intent"]["obligation_id"] metadata = {"operator_metadata": self.request["point_metadata"], "request_sha256": hashlib.sha256(self.raw).hexdigest(), @@ -80,10 +79,36 @@ def _capture(self, held, deadline): window = min(time.time() + 30, time.time() + deadline - time.monotonic()) if window <= time.time(): raise TimeoutError("original wake budget expired before capture") - components = self.request["components"] + components = list(self.request["components"]) + native = None + if self.request.get("native_session"): + from .native_checkpoint import native_session_checkpoint + guard = {"cli": self.env["LOOPX_CLI"], "workspace": self.env["LOOPX_PROJECT"], + "registry": self.env["LOOPX_REGISTRY"], "runtime_root": self.env["LOOPX_RUNTIME_ROOT"], + "goal_id": self.env["LOOPX_GOAL_ID"], "agent_id": self.env["LOOPX_AGENT_ID"], + "turn_instance_id": held["quota"]["heartbeat_receipt"]["turn_instance_id"], + "capture_root": str(self.capture_root), "socket_path": self.socket_path, + "decision_sha256": held["sha256"]} + native = native_session_checkpoint(self.env, self.receipt["session"]["session_id"], + timeout=max(.1, window - time.time()), interrupted_guard=guard) + allocated = {} + for entry in native["components"]: + component_id = _participant_id(components, entry["id"]) + allocated[entry["id"]] = component_id + components.append({**entry, "id": component_id}) + decision_id = _participant_id(components, "held_decision") + components.append({"id": decision_id, "source_path": str(held["path"])}) + metadata["native_session"] = {**native["metadata"], "store_members": [ + {**member, "id": allocated[member["id"]]} for member in native["metadata"]["store_members"]]} + metadata["held_decision_component_id"] = decision_id probe = effect_runtime_result("runtime.checkpoint_artifacts", { "action": "probe", "components": components, "deadline_epoch_ms": window * 1000, }, timeout=max(.1, window - time.time()), retry_safe=False) + if native is not None: + confirmed = native_session_checkpoint(self.env, self.receipt["session"]["session_id"], + timeout=max(.1, window - time.time()), interrupted_guard=guard) + if confirmed != native: + raise ValueError("interrupted native cursor moved before sealing") self.root.mkdir(mode=0o700, parents=True, exist_ok=True) return effect_runtime_result("runtime.checkpoint_artifacts", { "action": "capture", "components": components, "destination": str(destination), diff --git a/benchmark/runtime/native_checkpoint.py b/benchmark/runtime/native_checkpoint.py index 77c15bad60..ea80944c2c 100644 --- a/benchmark/runtime/native_checkpoint.py +++ b/benchmark/runtime/native_checkpoint.py @@ -7,23 +7,26 @@ import argparse import hashlib +import io import json import math import os import re import shutil +import shlex import sqlite3 import subprocess import time import tomllib from pathlib import Path +from contextlib import redirect_stdout, redirect_stderr from typing import Any from loopx.capabilities.benchmark_toolkit.native_codex_goal import StdioNativeGoalTransport from .codex import Execution -def _complete_history(request, session_id: str) -> list[dict[str, Any]]: +def _complete_history(request, session_id: str, *, interrupted_guard: dict | None = None) -> list[dict[str, Any]]: turns = [] cursor = None seen = set() @@ -43,15 +46,93 @@ def _complete_history(request, session_id: str) -> list[dict[str, Any]]: seen.add(cursor) if not turns or any(t.get("status") not in {"completed", "failed", "interrupted"} or not t.get("id") for t in turns): raise ValueError("native checkpoint requires complete retained turns") - if turns[-1].get("status") != "completed": - raise ValueError("native checkpoint requires a completed final cursor") + if interrupted_guard is None: + if turns[-1].get("status") != "completed": + raise ValueError("native checkpoint requires a completed final cursor") + else: + _verify_interrupted_guard(turns, interrupted_guard) ids = [t["id"] for t in turns] if len(set(ids)) != len(ids): raise ValueError("native checkpoint history contains duplicate turns") return turns -def native_session_checkpoint(env: dict[str, str], session_id: str, *, timeout: float) -> dict[str, Any]: +def _guard_cli_arguments(command: str, guard: dict) -> list[str]: + """Parse the provider shell wrapper plus the renderer's optional Turn assignment.""" + outer = shlex.split(command) + if len(outer) != 3 or Path(outer[0]).name not in {"bash", "zsh", "sh"} or outer[1] != "-lc": + raise ValueError("native pending guard is not a supported single shell command") + lines = outer[2].strip().splitlines() + if len(lines) == 2 and shlex.split(lines[0]) == ["LOOPX_TURN=" + guard["turn_instance_id"]]: + script = lines[1].replace('"${LOOPX_TURN:?}"', shlex.quote(guard["turn_instance_id"])) + elif len(lines) == 1: + script = lines[0] + else: + raise ValueError("native pending guard contains an unsupported shell program") + # Conservative provider syntax profile, not Goal/state classification. + # Only the exact renderer assignment above is expanded; never execute shell. + if not re.fullmatch(r"[A-Za-z0-9_./:=+@ '\"-]+", script): + raise ValueError("native pending guard contains unsupported shell syntax") + argv = shlex.split(script) + if not argv or argv[0] != guard["cli"]: + raise ValueError("native pending guard executable differs") + return argv[1:] + + +def _verify_interrupted_guard(turns: list[dict], guard: dict) -> None: + """Provider-specific stopped-tool cursor, never a relaxed completed cursor. + + Only the single simple command emitted by the heartbeat renderer is eligible. + Other than the exact renderer Turn assignment, shell programs and expansions + are unsupported, even if they eventually call quota. CLI parsing owns option + semantics; this adapter binds the native command item to the Host's held packet and rejects any later native item. + """ + from loopx.cli import build_parser + + tail = turns[-1] + items = tail.get("items") or [] + if tail.get("status") != "interrupted" or not items: + raise ValueError("native checkpoint requires an interrupted pending guard") + item = items[-1] + if (item.get("type") != "commandExecution" or item.get("status") != "failed" or + item.get("aggregatedOutput") is not None or not item.get("id") or + item.get("cwd") != guard["workspace"]): + raise ValueError("native checkpoint has output or work beyond its pending guard") + parser = build_parser() + def arguments(command): + argv = _guard_cli_arguments(command, guard) + try: + with redirect_stdout(io.StringIO()), redirect_stderr(io.StringIO()): + return vars(parser.parse_args(argv)) + except (SystemExit, ValueError) as error: + raise ValueError("native pending guard arguments are invalid") from error + args = arguments(item.get("command", "")) + expected = {"command": "quota", "quota_command": "should-run", "format": "json", + "registry": guard["registry"], "runtime_root": guard["runtime_root"], + "goal_id": guard["goal_id"], "agent_id": guard["agent_id"], + "turn_instance_id": guard["turn_instance_id"], "turn_envelope": True, + "decision_output_root": guard["capture_root"], + "decision_output_dir": None, "decision_handoff_socket": guard["socket_path"]} + if any(args.get(key) != value for key, value in expected.items()): + raise ValueError("native pending guard differs from the held packet routes or identity") + # A second matching guard after a tool yield cannot stand in for the first + # held call. Compare parsed routes, not only command spelling or item count. + for prior_turn in turns: + for previous in prior_turn.get("items", []): + if previous is item or previous.get("type") != "commandExecution": + continue + try: + prior_args = arguments(previous.get("command", "")) + except ValueError: + continue + if all(prior_args.get(key) == value for key, value in expected.items()): + raise ValueError("native pending guard is ambiguous after an earlier matching call") + if guard.get("native_call_id") is not None and item["id"] != guard["native_call_id"]: + raise ValueError("native pending guard call identity differs") + + +def native_session_checkpoint(env: dict[str, str], session_id: str, *, timeout: float, + interrupted_guard: dict | None = None) -> dict[str, Any]: home = Path(env["CODEX_HOME"]) if not session_id or timeout <= 0: raise ValueError("native checkpoint requires a session and an open capture window") @@ -81,7 +162,7 @@ def request(method, params): if (not workspace.is_absolute() or workspace.resolve(strict=True) != workspace or readback["thread"].get("cwd") != str(workspace)): raise ValueError("native checkpoint session workspace differs from the declared code basis") - turns = _complete_history(request, session_id) + turns = _complete_history(request, session_id, interrupted_guard=interrupted_guard) ids = [t["id"] for t in turns] members = [] for entry in sorted(home.iterdir()): @@ -93,7 +174,10 @@ def request(method, params): raise ValueError("native checkpoint session index is unavailable") encoded = json.dumps(turns, sort_keys=True, separators=(",", ":")).encode() return {"components": members, "metadata": { - "session_id": session_id, "completed_turn_id": ids[-1], "turn_ids": ids, + "session_id": session_id, "turn_ids": ids, + **({"completed_turn_id": ids[-1]} if interrupted_guard is None else { + "interrupted_turn_id": ids[-1], "interrupted_guard": { + **interrupted_guard, "native_call_id": turns[-1]["items"][-1]["id"]}}), "visible_history_sha256": hashlib.sha256(encoded).hexdigest(), "host_version": version, "guest_home": str(home), "guest_workspace": env["LOOPX_PROJECT"], "path_profile": "preserve_guest_paths_in_isolated_environment", @@ -157,6 +241,20 @@ def _fork_native_checkpoint(env: dict[str, str], unit: Path, expected_sha256: st }, timeout=timeout) manifest = json.loads((unit / "checkpoint.json").read_text()) native = manifest["point_metadata"]["native_session"] + if native.get("interrupted_guard") is not None: + component_id = manifest["point_metadata"].get("held_decision_component_id") + if component_id not in {entry["id"] for entry in manifest["components"]}: + raise ValueError("interrupted checkpoint lacks its held decision participant") + raw = (unit / "components" / component_id).read_bytes() + if hashlib.sha256(raw).hexdigest() != native["interrupted_guard"]["decision_sha256"]: + raise ValueError("interrupted checkpoint held decision digest differs") + guard = native["interrupted_guard"] + intent = effect_runtime_result("runtime.checkpoint_artifacts", { + "action": "select_replan", "quota": json.loads(raw), "goal_id": guard["goal_id"], + "agent_id": guard["agent_id"], "turn_instance_id": guard["turn_instance_id"], + }, timeout=max(.1, deadline - time.monotonic())) + if not intent["capture"] or intent != manifest["point_metadata"].get("replan"): + raise ValueError("interrupted checkpoint original replan binding differs") home = Path(env["CODEX_HOME"]) if str(home) != native["guest_home"] or env["LOOPX_PROJECT"] != native["guest_workspace"]: raise ValueError("native checkpoint requires its qualified fixed guest paths") @@ -187,8 +285,17 @@ def _fork_native_checkpoint(env: dict[str, str], unit: Path, expected_sha256: st raise ValueError("native checkpoint member inventory is invalid") names.add(name) component_ids.add(component_id) + interrupted_guard = native.get("interrupted_guard") + if interrupted_guard is not None: + if ("completed_turn_id" in native or not native.get("interrupted_turn_id") or + not native.get("turn_ids") or native["turn_ids"][-1] != native["interrupted_turn_id"]): + raise ValueError("native checkpoint mixes completed and interrupted cursors") + cursor_basis = {"interrupted_turn_id": native["interrupted_turn_id"], + "interrupted_guard": interrupted_guard} + else: + cursor_basis = {"completed_turn_id": native["completed_turn_id"]} basis = {"checkpoint_sha256": expected_sha256, "source_session_id": native["session_id"], - "completed_turn_id": native["completed_turn_id"], "config_sha256": native["config_sha256"], + **cursor_basis, "config_sha256": native["config_sha256"], "destination_goal_id": env.get("LOOPX_GOAL_ID"), "destination_agent_id": env.get("LOOPX_AGENT_ID"), "guest_workspace": env["LOOPX_PROJECT"], "guest_home": str(home)} attempt = None @@ -248,7 +355,8 @@ def request(method, params): finally: os.close(directory) fork = request("thread/fork", {"threadId": native["session_id"], - "lastTurnId": native["completed_turn_id"], "cwd": env["LOOPX_PROJECT"], + **({"lastTurnId": native["completed_turn_id"]} if interrupted_guard is None else {}), + "cwd": env["LOOPX_PROJECT"], "sandbox": "read-only", "approvalPolicy": "never", "deferGoalContinuation": True, "excludeTurns": True}) session = fork.get("thread", {}).get("id") @@ -257,12 +365,12 @@ def request(method, params): if (fork.get("thread", {}).get("forkedFromId") != native["session_id"] or fork["thread"].get("cwd") != env["LOOPX_PROJECT"]): raise ValueError("native fork provenance differs from the original attempt") - turns = _complete_history(request, session) + turns = _complete_history(request, session, interrupted_guard=interrupted_guard) encoded = json.dumps(turns, sort_keys=True, separators=(",", ":")).encode() if ([t["id"] for t in turns] != native["turn_ids"] or hashlib.sha256(encoded).hexdigest() != native["visible_history_sha256"]): raise ValueError("native checkpoint reconstructed history differs") - return {"session_id": session, "completed_turn_id": native["completed_turn_id"], + return {"session_id": session, **cursor_basis, "visible_history_sha256": native["visible_history_sha256"], "history_verified": True, "model_calls": 0, "execution_authority_granted": False, "continuation_qualified": False} @@ -292,6 +400,8 @@ def admit_native_checkpoint_session(env: dict[str, str], unit: Path, expected_sh "action": "verify", "root": str(unit), "expected_sha256": expected_sha256, }, timeout=timeout) metadata = json.loads((unit / "checkpoint.json").read_bytes())["point_metadata"] + if metadata.get("native_session", {}).get("interrupted_guard") is not None: + raise ValueError("interrupted session admission requires the joint controller continuation contract") source_profile = (metadata.get("session") or {}).get("session_profile_digest") if not isinstance(source_profile, str) or not re.fullmatch(r"[0-9a-f]{64}", source_profile): raise ValueError("checkpoint session admission requires its captured source session profile") diff --git a/benchmark/tests/test_native_checkpoint.py b/benchmark/tests/test_native_checkpoint.py index 7b6cade42e..a0e5264fcc 100644 --- a/benchmark/tests/test_native_checkpoint.py +++ b/benchmark/tests/test_native_checkpoint.py @@ -308,6 +308,7 @@ def test_real_fork_admission_and_native_resume_use_retained_state(tmp_path, caps workspace = tmp_path / "workspace" workspace.mkdir() + monkeypatch.chdir(workspace) # Real settlement verifies the producing workspace. candidate = workspace / "candidate.txt" candidate.write_text("retained candidate: 7\n") research = workspace / "research.txt" @@ -784,3 +785,63 @@ def forbidden_fork(*args, **kwargs): settled = read_heartbeat_settlement(runtime, goal_id=destination_goal, agent_id=destination_agent, todo_id=destination_todo, turn_instance_id=turn_id, registry_path=registry) assert settled.progress["state"] == "settled" + + +def test_interrupted_cursor_requires_exact_unreleased_final_guard(tmp_path, monkeypatch): + """A stopped command is eligible only at the declared pre-output boundary.""" + import copy + import shlex + from benchmark.runtime.native_checkpoint import _verify_interrupted_guard + + guard = {"cli": "/fixture/loopx", "workspace": str(tmp_path), "registry": "/fixture/registry.json", + "runtime_root": "/fixture/runtime", "goal_id": "research", "agent_id": "worker", + "turn_instance_id": "turn-1", "capture_root": "/fixture/decisions", "socket_path": "/tmp/fixture/socket", + "decision_sha256": "a" * 64} + argv = [guard["cli"], "--registry", guard["registry"], "--runtime-root", guard["runtime_root"], + "--format", "json", "quota", "should-run", "--goal-id", "research", "--agent-id", "worker", + "--turn-instance-id", "turn-1", "--turn-envelope", "--decision-output-root", guard["capture_root"], + "--decision-handoff-socket", guard["socket_path"]] + command = shlex.join(["/bin/bash", "-lc", shlex.join(argv)]) + item = {"type": "commandExecution", "id": "guard-call", "command": command, "cwd": str(tmp_path), + "status": "failed", "exitCode": 137, "aggregatedOutput": None} + turns = [{"id": "retained-compaction", "status": "completed", "items": [{"type": "contextCompaction"}]}, + {"id": "pending", "status": "interrupted", "items": [item]}] + env, calls = fixture(tmp_path, monkeypatch, [{"data": turns[:1], "nextCursor": "next"}, + {"data": turns[1:], "nextCursor": None}]) + captured = native_session_checkpoint(env, "fixture-session", timeout=10, interrupted_guard=guard) + assert captured["metadata"]["turn_ids"] == ["retained-compaction", "pending"] + assert "completed_turn_id" not in captured["metadata"] + assert captured["metadata"]["interrupted_guard"]["native_call_id"] == "guard-call" + assert "thread/fork" not in calls + + rendered = copy.deepcopy(turns) + rendered[-1]["items"][-1]["command"] = shlex.join(["/bin/bash", "-lc", + "LOOPX_TURN=turn-1\n" + shlex.join(argv).replace("--turn-instance-id turn-1", '--turn-instance-id "${LOOPX_TURN:?}"')]) + _verify_interrupted_guard(rendered, guard) + variants = [] + for field, value in [("aggregatedOutput", "score already exposed"), ("cwd", "/elsewhere"), + ("status", "completed"), ("command", shlex.join(["/bin/bash", "-lc", shlex.join(argv) + "; touch later"]))]: + changed = copy.deepcopy(turns) + changed[-1]["items"][-1][field] = value + variants.append(changed) + for option, value in [("--turn-instance-id", "wrong-turn"), ("--decision-handoff-socket", "/tmp/other"), + ("--registry", "/other.json"), ("--goal-id", "other")]: + altered = list(argv) + altered[altered.index(option) + 1] = value + changed = copy.deepcopy(turns) + changed[-1]["items"][-1]["command"] = shlex.join(["/bin/bash", "-lc", shlex.join(altered)]) + variants.append(changed) + changed = copy.deepcopy(turns) + changed[-1]["items"].append({"type": "reasoning", "text": "later decision"}) + variants.append(changed) + changed = copy.deepcopy(turns) + changed[-1]["status"] = "completed" + variants.append(changed) + changed = copy.deepcopy(turns) + changed[-1]["items"].insert(0, copy.deepcopy(item)) + variants.append(changed) + for changed in variants: + with pytest.raises(ValueError): + _verify_interrupted_guard(changed, guard) + with pytest.raises(ValueError, match="call identity"): + _verify_interrupted_guard(turns, {**guard, "native_call_id": "different"}) diff --git a/benchmark/tests/test_native_checkpoint_handoff.py b/benchmark/tests/test_native_checkpoint_handoff.py index 2e89578821..b8bcbfa2fe 100644 --- a/benchmark/tests/test_native_checkpoint_handoff.py +++ b/benchmark/tests/test_native_checkpoint_handoff.py @@ -345,7 +345,7 @@ def test_checkpoint_interrupt_timeout_reaps_without_qualifying_capture(tmp_path) @pytest.mark.skipif(not os.environ.get("LOOPX_NATIVE_CHECKPOINT_BIN") or os.name != "posix", reason="Requires the real native CLI and private Unix transport") -@pytest.mark.parametrize("capture", ["files", "full", "off"]) +@pytest.mark.parametrize("capture", ["files", "native", "full", "off"]) def test_actual_worker_handles_midwake_replan_and_duplicate_without_guard_replay(tmp_path, capture): """Use run_once, its renderer/listener/session owner and the actual quota CLI.""" import sys @@ -364,7 +364,7 @@ def test_actual_worker_handles_midwake_replan_and_duplicate_without_guard_replay request = tmp_path / "request.json" request.write_text(json.dumps({"destination": str(logs.parent / "checkpoints"), "components": [{"id": "code", "source_path": str(candidate)}], - "point_metadata": {}, "trigger": "fresh_replan", "native_session": capture == "full"})) + "point_metadata": {}, "trigger": "fresh_replan", "native_session": capture in {"native", "full"}, "loopx_state": capture == "full"})) skills = tmp_path / "skills" skills.mkdir() cli = tmp_path / "loopx-cli" @@ -418,8 +418,9 @@ def response(payload, index, message): guard = next(line for line in body.splitlines() if "quota should-run" in line) guard = guard.replace('${LOOPX_TURN:?}', TURN_ID) guards.append(guard) + rendered_guard = ("LOOPX_TURN=" + TURN_ID + "\n" + guard.replace(TURN_ID, "${LOOPX_TURN:?}")) if capture == "native" else guard return {"type": "function_call", "id": "fc_midwake", "call_id": "call_midwake", - "name": "exec_command", "arguments": json.dumps({"cmd": guard, + "name": "exec_command", "arguments": json.dumps({"cmd": rendered_guard, "yield_time_ms": 10000, "max_output_tokens": 100})} if index == 3 and capture != "off": # The resumed source deliberately rereads the same obligation. It @@ -451,15 +452,47 @@ def response(payload, index, message): return assert len(result["checkpoint_handoffs"]) == 1 event = result["checkpoint_handoffs"][0] - assert event["captured"] is (capture == "files") + assert event["captured"] is (capture in {"files", "native"}) assert event["research_point_qualified"] is False assert len(calls) == 4 and (project / "continued.txt").read_text() == "continued" assert "heartbeat_receipt" in json.dumps(requests[2]) assert any(item.get("type") == "function_call_output" and item.get("call_id") == "call_midwake" and "aborted" in item.get("output", "") for item in requests[2]["input"]) - if capture == "files": + if capture in {"files", "native"}: point = logs.parent / "checkpoints" / event["obligation_id"] assert (point / "components" / "code").read_text() == "original candidate" + if capture == "native": + manifest = json.loads((point / "checkpoint.json").read_text()) + native = manifest["point_metadata"]["native_session"] + assert "completed_turn_id" not in native + assert native["interrupted_guard"]["native_call_id"] == "call_midwake" + assert native["interrupted_guard"]["decision_sha256"] == event["decision_sha256"] + # Reconstruct the earlier captured store after the source has + # already continued. Fixed guest paths require an isolated home; + # retain and restore the source bytes around this synthetic test. + from benchmark.runtime.native_checkpoint import fork_native_checkpoint, admit_native_checkpoint_session + from benchmark.runtime.codex import Execution + home = Path(native["guest_home"]) + saved_home = home.with_name(home.name + "-retained") + config = tmp_path / "captured-config.toml" + config.write_bytes((home / "config.toml").read_bytes()) + with pytest.raises(ValueError, match="joint controller"): + admit_native_checkpoint_session({**env, "CODEX_HOME": str(home)}, + point, event["sha256"], config, execution=Execution(context="resume"), timeout=20) + home.rename(saved_home) + try: + forked = fork_native_checkpoint({**env, "CODEX_HOME": str(home)}, + point, event["sha256"], config, timeout=20) + assert forked["history_verified"] and forked["model_calls"] == 0 + assert forked["interrupted_turn_id"] == native["interrupted_turn_id"] + assert forked["session_id"] != native["session_id"] + recovered = fork_native_checkpoint({**env, "CODEX_HOME": str(home)}, + point, event["sha256"], config, timeout=20) + assert recovered == forked # No second fork after a lost reply. + finally: + if home.exists(): + home.rename(home.with_name(home.name + "-receiver")) + saved_home.rename(home) else: assert "full-state capture" in event["error"] assert not list((logs.parent / "checkpoints").glob("*/checkpoint.json")) From e6df21086e26e7bc8d2ffbf37729873fff592137 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Wed, 14 Oct 2026 01:46:13 +0800 Subject: [PATCH 58/69] docs(benchmark): distinguish interrupted capture from joint admission Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/README.md | 30 ++++++++++++------- .../hierarchical-agent-stride-control-v0.md | 10 +++++-- ...rarchical-agent-stride-control-v0.zh-CN.md | 8 +++-- 3 files changed, 32 insertions(+), 16 deletions(-) diff --git a/benchmark/README.md b/benchmark/README.md index 6882a1557a..0ebed6361a 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -228,19 +228,29 @@ captured TurnEnvelope. With checkpointing off the renderer, process lifecycle and model-visible output stay unchanged. Ordinary guards and repeated replan obligations are released without restarting the native CLI. On a first replan, the worker selects through the typed owner, interrupts the source, records its -session, and captures explicitly declared inert files. Failed optional capture +session, and captures explicitly declared inert files and optional native history. Failed optional capture still resumes the source with its original packet; altered packet bytes or lost session identity fail closed. All resumptions share the original wake deadline. -This is **not a qualified complete research checkpoint**. Selection before -interruption is not proof that the native tool has never yielded. The worker -records each handoff and currently rejects mid-wake requests for native session, -canonical research or evaluation participants rather than silently reducing them -to files. The completed-cursor path remains strict. The next joint acceptance -must bind the pending native history, the controller's all-writer barrier and -isolated useful continuation before research runs use this profile. Remove the -checkpoint request to disable automatic handoff; a direct caller can omit the -socket option to disable only the transport. +This is **not a qualified complete research checkpoint**. With `native_session` +requested, the actual worker can seal an explicit interrupted cursor: all retained +pages, the failed final guard command with no released output or later visible +item, its parsed CLI routes/Turn identity, and the original held decision bytes. +A second matching guard is ambiguous and rejects. The exact renderer Turn +assignment is supported; other shell programs and expansions are unsupported +by this conservative provider profile. History is +rechecked against the byte witness before publication. A native fork reconstructs +that entire captured prefix, including the aborted tool, without using a completed +Turn as a substitute; completed-cursor checks stay unchanged. Lost fork replies +reuse the existing one-attempt recovery path. + +Canonical research/evaluation requests still reject at this mid-wake path. Native +history plus declared files is not a controller-wide writer barrier or authority +admission. Interrupted sessions cannot enter destination Goal admission until the +joint controller continuation contract is qualified. The next acceptance composes +that barrier, research/evaluator/budget adoption and isolated useful continuation. +Remove the checkpoint request to disable automatic handoff; a direct caller can +omit the socket option to disable only the transport. Add `"native_session": true` to include Codex's completed native cursor, all retained history pages, rollout/compaction records and native index/history SQLite diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md index bc31870ad8..6f30e9dbb3 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md @@ -769,9 +769,13 @@ installs the rendezvous for an explicit fresh-replan request, deduplicates the original obligation, and resumes the same source session with the original guard and remaining wake budget. Ordinary/duplicate guards do not restart the native CLI. Declared file capture and failed optional capture use this real entrypoint; -full-state mid-wake requests remain explicitly unqualified. Interrupted-cursor -validation, the controller's all-writer barrier and isolated joint continuation -remain the next acceptance; completed-cursor checks remain strict. +full-state mid-wake requests remain explicitly unqualified. The native participant +now binds an explicit interrupted final guard, its parsed routes/Turn identity and +original packet digest, rejects released output/later visible items or duplicate +matching calls, and preserves the full prefix in a native fork. Completed-cursor +checks stay strict. Controller-wide writer exclusion and isolated joint research, +evaluator and budget admission remain open; participant reconstruction grants no +new work or destination authority. The Host profile must also declare path portability. A native index can retain absolute guest paths: qualification must either preserve those paths inside a diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md index 5ab2b05693..491073ef57 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md @@ -639,9 +639,11 @@ Host 拥有的原生进程组，要求主进程正常响应中断退出并清理 它尚未证明中断之前没有模型动作。实际 heartbeat worker 现已在显式 fresh-replan 请求下安装 rendezvous，按原义务去重，并用原 guard 和剩余 wake 预算接续同一 来源 session；普通或重复 guard 不重启原生 CLI。声明文件的捕获与可选捕获失败 -均经过该实际入口；中途的完整状态请求仍明确不具备资格。interrupted cursor -验证、controller 的全部 writer 屏障和隔离联合接续仍属下一步验收，completed -cursor 检查不放宽。 +均经过该实际入口；中途的完整状态请求仍明确不具备资格。原生参与者已显式绑定 +interrupted 末尾 guard、解析后的路由/Turn 身份及原包摘要，拒绝已释放输出、 +后续可见条目或重复匹配调用，并通过原生 fork 保留完整前缀。completed cursor +检查不放宽。controller 全部 writer 屏障与隔离的研究/评测/预算联合准入仍待验收； +参与者重建不授予新的工作或目标执行权限。 Host profile 还须声明路径可移植性。原生索引可能保留绝对 guest 路径：资格化 须在新隔离环境保留这些路径，或使用 Host 支持的 import/rebinding API；手改 From d12b051fcb3aa4d2c4e40c07a924700136664f56 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Wed, 14 Oct 2026 02:19:53 +0800 Subject: [PATCH 59/69] feat(benchmark): capture held research state through the native controller Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/edgebench/checkpoint_supervisor.py | 118 ++++++++++++++++ benchmark/edgebench/checkpoint_worker.py | 50 ++++++- benchmark/edgebench/run.py | 14 +- benchmark/runtime/checkpoint.py | 130 +++++++++++------- benchmark/runtime/checkpoint_handoff.py | 56 ++++++++ benchmark/runtime/checkpoint_wake.py | 25 +++- benchmark/runtime/sforge_backend.py | 11 +- benchmark/tests/test_checkpoint_handoff.py | 57 ++++++++ benchmark/tests/test_checkpoint_worker.py | 63 +++++++++ .../tests/test_native_checkpoint_handoff.py | 24 +++- benchmark/tests/test_sforge_runtime.py | 47 +++++++ 11 files changed, 536 insertions(+), 59 deletions(-) create mode 100644 benchmark/edgebench/checkpoint_supervisor.py create mode 100644 benchmark/runtime/checkpoint_handoff.py create mode 100644 benchmark/tests/test_checkpoint_handoff.py diff --git a/benchmark/edgebench/checkpoint_supervisor.py b/benchmark/edgebench/checkpoint_supervisor.py new file mode 100644 index 0000000000..fc7a6778d2 --- /dev/null +++ b/benchmark/edgebench/checkpoint_supervisor.py @@ -0,0 +1,118 @@ +"""Host-only adapter for the worker's explicitly enabled checkpoint mailbox.""" +from __future__ import annotations + +import hashlib +import io +import json +import re +import tarfile +import tempfile +import threading +import time +from pathlib import Path + +from .checkpoint_controller import _controller +from .checkpoint_worker import _archive, capture_worker + + +class CheckpointSupervisor: + def __init__(self, controller_file: Path, destination: Path): + _controller(controller_file) # Fail before native registration. + self.controller_file, self.destination = controller_file, destination + self.token = None + self.stop = threading.Event() + self.thread = None + self.failure = None + + def bind(self, environment): + if self.token is not None or not (environment or {}).get("SFORGE_TOKEN"): + raise ValueError("Checkpoint controller requires one native worker registration") + self.token = environment["SFORGE_TOKEN"] + + def start(self, container): + if self.thread is not None or self.token is None: + raise ValueError("Checkpoint supervisor requires one bound worker execution") + self.destination.mkdir(mode=0o700, parents=True, exist_ok=False) + self.container = container + self.thread = threading.Thread(target=self._run, args=(container,), daemon=True) + self.thread.start() + + def close(self): + self.stop.set() + if self.thread is not None: + self.thread.join(60) + if self.thread.is_alive(): + raise RuntimeError("Checkpoint writer barrier is still active; preserve the worker for recovery") + if self.failure is not None: + # A missed optional point does not cancel otherwise useful source + # work. An uncertain/unreleased writer barrier still needs recovery. + self.container.reload() + if self.container.attrs["State"]["Paused"]: + raise RuntimeError("Checkpoint writer barrier requires controller recovery") from self.failure + + def _reply(self, container, directory, nonce, response): + container.reload() + if container.attrs["State"]["Paused"] or not container.attrs["State"]["Running"]: + raise RuntimeError("Checkpoint reply requires the released, running source worker") + raw = json.dumps(response).encode() + pending, final = nonce + ".pending", nonce + ".json" + stream = io.BytesIO() + with tarfile.open(fileobj=stream, mode="w") as archive: + entry = tarfile.TarInfo(pending) + entry.size, entry.mode = len(raw), 0o444 + archive.addfile(entry, io.BytesIO(raw)) + if not container.put_archive(directory, stream.getvalue()): + raise RuntimeError("Checkpoint reply transport was not accepted") + result = container.exec_run(["/bin/mv", directory + "/" + pending, directory + "/" + final]) + if result.exit_code: + raise RuntimeError("Checkpoint reply could not be published atomically") + + def _run(self, container): + from docker.errors import NotFound + + mailbox = "/logs/agent/checkpoint-controller" + remote = mailbox + "/ready.json" + seen = set() + try: + while not self.stop.is_set(): + with tempfile.TemporaryDirectory(prefix=".controller-poll-", dir=self.destination) as temp: + stage = Path(temp) + try: + raw = _archive(container, remote, stage / "ready").read_bytes() + except NotFound: + self.stop.wait(5) + continue + digest = hashlib.sha256(raw).hexdigest() + if digest in seen: + self.stop.wait(5) + continue + seen.add(digest) # Never retry a later state under an earlier point. + value = json.loads(raw) + nonce = value.get("capture_id") + if not isinstance(nonce, str) or not re.fullmatch(r"[0-9a-f]{32}", nonce): + raise ValueError("Malformed checkpoint request identity") + response = {"request_sha256": digest, "ok": False} + try: + request = stage / "request.json" + request.write_text(json.dumps({"container_id": container.id, + "components": value["components"], + "phase_deadline_path": "/opt/loopx-benchmark/control/phase-deadline", + "handoff": {"source_path": remote, "sha256": digest}})) + remaining = value["deadline_epoch"] - time.time() + if remaining <= 0: + raise TimeoutError("Checkpoint handoff deadline expired") + captured = capture_worker(self.controller_file, token=self.token, capture_id=nonce, + destination=self.destination / nonce, request_file=request, timeout=min(30, remaining)) + response.update(ok=True, capture=captured) + except Exception as error: + # Optional capture failure resumes the original source; + # credentials and raw provider errors never enter it. + response["error_kind"] = type(error).__name__ + diagnostic = self.destination / (nonce + ".error.txt") + diagnostic.write_text(str(error).replace(self.token, "[redacted]")[:2000]) + diagnostic.chmod(0o600) + (self.destination / (nonce + ".json")).write_text(json.dumps(response)) + self._reply(container, mailbox, nonce, response) + except Exception as error: + self.failure = error + (self.destination / "supervisor-failure.json").write_text(json.dumps({"error_kind": type(error).__name__})) diff --git a/benchmark/edgebench/checkpoint_worker.py b/benchmark/edgebench/checkpoint_worker.py index 2f66fd58fb..8d299c917d 100644 --- a/benchmark/edgebench/checkpoint_worker.py +++ b/benchmark/edgebench/checkpoint_worker.py @@ -13,6 +13,7 @@ import re import tarfile import tempfile +import time from pathlib import Path, PurePosixPath from loopx.control_plane.effect_runtime import effect_runtime_result @@ -25,7 +26,8 @@ def _request(path: Path) -> dict: value = json.loads(_physical(path).read_bytes()) if (not isinstance(value, dict) or - set(value) != {"container_id", "components", "phase_deadline_path"} or + set(value) not in ({"container_id", "components", "phase_deadline_path"}, + {"container_id", "components", "phase_deadline_path", "handoff"}) or not re.fullmatch(r"[0-9a-f]{64}", value.get("container_id", "")) or not isinstance(value["components"], list) or not value["components"]): raise ValueError("Worker request requires a full container ID, explicit components and phase_deadline_path") @@ -39,6 +41,12 @@ def _request(path: Path) -> dict: ids.add(entry["id"]) paths.append(_guest_path(entry["source_path"])) paths.append(_guest_path(value["phase_deadline_path"])) + if "handoff" in value: + handoff = value["handoff"] + if (not isinstance(handoff, dict) or set(handoff) != {"source_path", "sha256"} or + not re.fullmatch(r"[0-9a-f]{64}", handoff.get("sha256", ""))): + raise ValueError("Worker handoff requires the exact private request digest") + paths.append(_guest_path(handoff["source_path"])) if any(a == b or a in b.parents or b in a.parents for index, a in enumerate(paths) for b in paths[index + 1:]): raise ValueError("Worker components and phase clock must not overlap") @@ -80,6 +88,13 @@ def _archive(container, remote: str, destination: Path) -> Path: raise ValueError("Worker archive contains undeclared, linked or special members") names.add(str(path)) archive.extractall(destination, members=members, filter="data") + # The data filter intentionally drops directory modes. This is a + # private raw transport witness, so restore only ordinary permission + # bits after safe extraction; never restore owners or special bits. + # Otherwise a private 0700 authority directory becomes 0755 and a + # matching frozen source is falsely rejected as having changed. + for member in sorted(members, key=lambda entry: len(PurePosixPath(entry.name).parts), reverse=True): + (destination / member.name).chmod(member.mode & 0o777) return _physical(destination / root) @@ -169,6 +184,7 @@ def capture_worker(controller_file: Path, *, token: str, capture_id: str, stage = Path(directory) (stage / "workers").mkdir(mode=0o700) components = [] + prepared = None pause_may_have_committed = True try: try: @@ -182,9 +198,35 @@ def capture_worker(controller_file: Path, *, token: str, capture_id: str, container.reload() if not container.attrs["State"]["Paused"]: raise RuntimeError("Worker writer barrier was not established") + if "handoff" in request: + held = _archive(container, request["handoff"]["source_path"], stage / "held") + raw = held.read_bytes() + if hashlib.sha256(raw).hexdigest() != request["handoff"]["sha256"]: + raise ValueError("Worker handoff changed before its writer barrier") + prepared = json.loads(raw) + if (set(prepared) != {"capture_id", "components", "point_metadata", "source_sha256", "source_witness", "deadline_epoch"} or + prepared["capture_id"] != capture_id or prepared["components"] != request["components"] or + not isinstance(prepared["point_metadata"], dict) or not isinstance(prepared["source_witness"], dict) or + type(prepared["deadline_epoch"]) not in (int, float) or + not math.isfinite(prepared["deadline_epoch"]) or time.time() >= prepared["deadline_epoch"]): + raise ValueError("Worker handoff is expired or differs from the original capture") for entry in request["components"]: copied = _archive_component(container, entry["source_path"], stage / "workers" / entry["id"]) components.append({"id": entry["id"], "source_path": str(copied)}) + if prepared is not None: + observed = effect_runtime_result("runtime.checkpoint_artifacts", { + "action": "probe", "components": components, + }, retry_safe=False) + if observed["source_sha256"] != prepared["source_sha256"]: + changed = [] + for key, rows in observed["source_witness"].items(): + before = {row["path"]: row for row in prepared["source_witness"].get(key, [])} + after = {row["path"]: row for row in rows} + changed.extend(key + ":" + path for path in sorted(before.keys() | after.keys()) + if before.get(path) != after.get(path)) + raise ValueError("Worker source changed after native/research preparation: " + ", ".join(changed)) + if time.time() >= prepared["deadline_epoch"]: + raise TimeoutError("Worker handoff expired before evaluator capture") phase = _archive(container, request["phase_deadline_path"], stage / "phase") native = capture(controller_file, token=token, capture_id=capture_id, destination=stage / "native", timeout=timeout, phase_deadline_file=phase) @@ -203,14 +245,18 @@ def capture_worker(controller_file: Path, *, token: str, capture_id: str, raise RuntimeError("Worker was not restored to running; inspect the original container before retry") native_component = _participant_id(components, "native_capsule") components.append({"id": native_component, "source_path": str(stage / "native")}) - metadata = {"worker_file_snapshot": { + metadata = {**(prepared["point_metadata"] if prepared is not None else {}), "worker_file_snapshot": { "container_id": container.id, "image_id": container.attrs["Image"], "components": request["components"], "phase_deadline_path": request["phase_deadline_path"], "native_capsule_sha256": native["sha256"], "native_capsule_component_id": native_component, "writer_barrier": "docker_pause", "process_memory_captured": False, "native_cursor_qualified": False, + **({"handoff_sha256": request["handoff"]["sha256"], + "prepared_source_verified": True} if prepared is not None else {}), }} + if prepared is not None: + metadata["writer_barrier"] = "docker_pause" probe = effect_runtime_result("runtime.checkpoint_artifacts", {"action": "probe", "components": components}) return effect_runtime_result("runtime.checkpoint_artifacts", { "action": "capture", "components": components, "destination": str(destination), diff --git a/benchmark/edgebench/run.py b/benchmark/edgebench/run.py index 73208368f0..7de47b8ec4 100644 --- a/benchmark/edgebench/run.py +++ b/benchmark/edgebench/run.py @@ -119,6 +119,8 @@ def main(argv=None): help="Explicit completed-Todo cadence ablation for heartbeat profiles") parser.add_argument("--stride-strategy", choices=stride_strategy_options(), help="Explicit heartbeat Goal-review strategy; omitted keeps the ordinary off treatment") + parser.add_argument("--checkpoint-controller-file", type=Path, + help="Host-private native checkpoint controller; requires full fresh-replan handoff") parser.add_argument("--checkpoint-request", type=Path, help="Explicit host JSON declaration staged into a heartbeat worker; omitted disables capture") parser.add_argument("--eval-interval", type=int, @@ -149,7 +151,10 @@ def main(argv=None): if not args.worker.startswith("heartbeat-"): parser.error("--checkpoint-request requires a heartbeat worker") from benchmark.runtime.checkpoint import read_checkpoint_request - _, checkpoint_request = read_checkpoint_request(str(args.checkpoint_request)) + declaration, checkpoint_request = read_checkpoint_request(str(args.checkpoint_request)) + handoff = checkpoint_request is not None and declaration.get("controller_handoff", False) + if bool(args.checkpoint_controller_file) != handoff or (handoff and args.feedback != "native"): + parser.error("controller file and full controller_handoff must be enabled together with native feedback") # One directory is one attempt: never reuse native registration or overwrite # source/profile evidence after an ambiguous launch. trial = args.log_dir / "runs" / args.run_id / args.task @@ -224,9 +229,14 @@ def main(argv=None): judge_url=args.judge_url.replace("host.docker.internal", "127.0.0.1"), admin_secret=get_admin_secret(args.log_dir), logger=logger, sampler=sampler, ) if args.feedback == "best-only" else None) + checkpoint = None + if args.checkpoint_controller_file is not None: + from benchmark.edgebench.checkpoint_supervisor import CheckpointSupervisor + checkpoint = CheckpointSupervisor(args.checkpoint_controller_file, trial / "checkpoints") backend = RecordingDockerBackend(log_dir=trial / "collected", logger=logger, oauth_proxy=bool(args.api_proxy_url), - blind_api_endpoint=blind_endpoint, feedback=feedback) + blind_api_endpoint=blind_endpoint, feedback=feedback, + **({"checkpoint": checkpoint} if checkpoint is not None else {})) for image in (task.work_image_key, task.judge_image_key): if not backend.image_exists(image): raise RuntimeError(f"Missing native image: {image}") diff --git a/benchmark/runtime/checkpoint.py b/benchmark/runtime/checkpoint.py index 75cbd6e7cc..8cb5d6b494 100644 --- a/benchmark/runtime/checkpoint.py +++ b/benchmark/runtime/checkpoint.py @@ -38,7 +38,7 @@ def parse_checkpoint_request(raw: bytes) -> dict[str, Any]: """Validate the same explicit declaration before staging and before capture.""" request = json.loads(raw) if (not isinstance(request, dict) or - set(request) - {"destination", "components", "point_metadata", "native_session", "evaluation_opportunity", "loopx_state", "trigger"} or + set(request) - {"destination", "components", "point_metadata", "native_session", "evaluation_opportunity", "loopx_state", "trigger", "controller_handoff"} or not {"destination", "components", "point_metadata"}.issubset(request)): raise ValueError("checkpoint request requires destination, components and point_metadata") if "native_session" in request and type(request["native_session"]) is not bool: @@ -47,6 +47,12 @@ def parse_checkpoint_request(raw: bytes) -> dict[str, Any]: raise ValueError("checkpoint evaluation_opportunity must be an explicit boolean") if "loopx_state" in request and type(request["loopx_state"]) is not bool: raise ValueError("checkpoint loopx_state must be an explicit boolean") + if "controller_handoff" in request and type(request["controller_handoff"]) is not bool: + raise ValueError("checkpoint controller_handoff must be an explicit boolean") + if request.get("controller_handoff") and ( + request.get("trigger") != "fresh_replan" or not request.get("native_session") or + not request.get("loopx_state") or not request.get("evaluation_opportunity")): + raise ValueError("controller handoff requires fresh_replan and all three state participants") if "trigger" in request and request["trigger"] != "fresh_replan": raise ValueError("checkpoint trigger must be fresh_replan or omitted") if not isinstance(request["components"], list) or not request["components"]: @@ -130,6 +136,68 @@ def _loopx_state_components(env, components, output_dir): "execution_authority_granted": False} +def native_binding_participant(env, components, metadata): + from .codex import Execution + from .sessions import BenchmarkSessionWake + source_wake = BenchmarkSessionWake(env, Execution(context="resume", + sandbox=env.get("LOOPX_CODEX_SANDBOX", "danger-full-access")), {}) + if source_wake.session_id != metadata["session"].get("session_id"): + raise ValueError("native checkpoint differs from the current source session binding") + # This sideband record belongs to the existing session owner. It + # must participate even when loopx_state was not requested: native + # store bytes alone cannot witness a binding/profile replacement. + from loopx.control_plane.turn_driver.codex_sessions import _session_path + binding_path = _session_path(source_wake.root, source_wake.lineage, "agent") + if not any(binding_path.is_relative_to(Path(entry["source_path"])) for entry in components): + components = [*components, {"id": _participant_id(components, "source_session_binding"), + "source_path": str(binding_path)}] + metadata["session"] = {**metadata["session"], "session_profile_digest": source_wake.digest} + return components, source_wake + + +def research_participants(env, components, metadata, destination, evidence_path, deadline): + """Use the same inventory and historical context at either owned boundary.""" + research = None + components, metadata["loopx_state"] = _loopx_state_components(env, components, destination) + from loopx.control_plane.coordination.local_authority import local_authority_is_promoted + if local_authority_is_promoted(runtime_root=Path(env["LOOPX_RUNTIME_ROOT"]), goal_id=env["LOOPX_GOAL_ID"]): + from .research_checkpoint import research_context, _encoded + research = research_context(env, deadline=deadline) + evidence = evidence_path + research_raw = _encoded(research) + with os.fdopen(os.open(evidence, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600), "wb") as file: + file.write(research_raw) + parent = next((entry for entry in components if evidence.is_relative_to(Path(entry["source_path"]))), None) + if parent is None: + parent = {"id": _participant_id(components, "research_context"), "source_path": str(evidence)} + components = [*components, parent] + metadata["research_context"] = {"component_id": parent["id"], + "member_path": evidence.relative_to(Path(parent["source_path"])).as_posix() if evidence != Path(parent["source_path"]) else "", + "sha256": hashlib.sha256(research_raw).hexdigest(), + "provider_revision": research["snapshot"]["provider_revision"], "store_identity": research["snapshot"]["store_identity"]} + else: + metadata["canonical_context_unavailable"] = "Source coordination is unpromoted; its retained inventory remains inert." + return components, research + + +def verify_research_participants(env, components, metadata, research, witness, deadline): + from .research_checkpoint import research_context, verify_context_sources + if research is not None: + if research_context(env, deadline=deadline) != research: + raise ValueError("canonical research context moved before sealing") + verify_context_sources(research, components, witness) + retained = metadata["research_context"] + member = next((entry for entry in witness[retained["component_id"]] + if entry["path"] == retained["member_path"]), None) + if member is None or member.get("sha256") != retained["sha256"]: + raise ValueError("canonical research context bytes moved before sealing") + declared = metadata["loopx_state"] + witness = witness[declared["registry_component_id"]] + member = next((entry for entry in witness if entry["path"] == declared["registry_member_path"]), None) + if member is None or member.get("sha256") != declared["registry_sha256"]: + raise ValueError("LoopX registry differs from the scoped participant inventory") + + def _mark_replan_attempt(root: Path, obligation_id: str, turn_id: str, request_sha256: str | None) -> bool: """The first boundary wins even when its optional declaration was lost.""" if not root.is_absolute() or root != root.resolve(): @@ -347,28 +415,10 @@ def capture_after_wake(env: dict[str, str], receipt: dict[str, Any], *, } components = request["components"] if request.get("loopx_state"): - components, metadata["loopx_state"] = _loopx_state_components(env, components, destination) - from loopx.control_plane.coordination.local_authority import local_authority_is_promoted - research = None - if local_authority_is_promoted(runtime_root=Path(env["LOOPX_RUNTIME_ROOT"]), goal_id=env["LOOPX_GOAL_ID"]): - from .research_checkpoint import research_context, verify_context_sources, _encoded - research = research_context(env, deadline=deadline) - evidence = Path(env["LOOPX_WAKE_LOG_DIR"]) / receipt["turn_id"] / "checkpoint-research-context.json" - research_raw = _encoded(research) - with os.fdopen(os.open(evidence, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600), "wb") as file: - file.write(research_raw) - parent = next((entry for entry in components if evidence.is_relative_to(Path(entry["source_path"]))), None) - if parent is None: - parent = {"id": _participant_id(components, "research_context"), "source_path": str(evidence)} - components = [*components, parent] - metadata["research_context"] = {"component_id": parent["id"], - "member_path": evidence.relative_to(Path(parent["source_path"])).as_posix() if evidence != Path(parent["source_path"]) else "", - "sha256": hashlib.sha256(research_raw).hexdigest(), - "provider_revision": research["snapshot"]["provider_revision"], "store_identity": research["snapshot"]["store_identity"]} - else: - metadata["canonical_context_unavailable"] = "Source coordination is unpromoted; its retained inventory remains inert." + evidence = Path(env["LOOPX_WAKE_LOG_DIR"]) / receipt["turn_id"] / "checkpoint-research-context.json" + components, research = research_participants(env, components, metadata, destination, evidence, deadline) opportunity = None - if request.get("evaluation_opportunity"): + if request.get("evaluation_opportunity") and not request.get("controller_handoff"): from .sforge_checkpoint import capture_opportunity evidence = Path(env["LOOPX_WAKE_LOG_DIR"]) / receipt["turn_id"] / "checkpoint-opportunity.json" @@ -392,19 +442,7 @@ def capture_after_wake(env: dict[str, str], receipt: dict[str, Any], *, from .sessions import BenchmarkSessionWake session = receipt.get("session") or {} - source_wake = BenchmarkSessionWake(env, Execution(context="resume", - sandbox=env.get("LOOPX_CODEX_SANDBOX", "danger-full-access")), {}) - if source_wake.session_id != session.get("session_id"): - raise ValueError("native checkpoint differs from the current source session binding") - # This sideband record belongs to the existing session owner. It - # must participate even when loopx_state was not requested: native - # store bytes alone cannot witness a binding/profile replacement. - from loopx.control_plane.turn_driver.codex_sessions import _session_path - binding_path = _session_path(source_wake.root, source_wake.lineage, "agent") - if not any(binding_path.is_relative_to(Path(entry["source_path"])) for entry in components): - components = [*components, {"id": _participant_id(components, "source_session_binding"), - "source_path": str(binding_path)}] - metadata["session"] = {**session, "session_profile_digest": source_wake.digest} + components, source_wake = native_binding_participant(env, components, metadata) native = native_session_checkpoint(env, session.get("session_id", ""), timeout=max(.1, deadline - time.time())) # Native names identify provider files, not reserved artifact IDs. @@ -437,20 +475,7 @@ def capture_after_wake(env: dict[str, str], receipt: dict[str, Any], *, (source_wake.session_id, source_wake.goal_ref, source_wake.digest)): raise ValueError("native checkpoint source session binding moved before sealing") if request.get("loopx_state"): - if research is not None: - if research_context(env, deadline=deadline) != research: - raise ValueError("canonical research context moved before sealing") - verify_context_sources(research, components, probe["source_witness"]) - retained = metadata["research_context"] - member = next((entry for entry in probe["source_witness"][retained["component_id"]] - if entry["path"] == retained["member_path"]), None) - if member is None or member.get("sha256") != retained["sha256"]: - raise ValueError("canonical research context bytes moved before sealing") - declared = metadata["loopx_state"] - witness = probe["source_witness"][declared["registry_component_id"]] - member = next((entry for entry in witness if entry["path"] == declared["registry_member_path"]), None) - if member is None or member.get("sha256") != declared["registry_sha256"]: - raise ValueError("LoopX registry differs from the scoped participant inventory") + verify_research_participants(env, components, metadata, research, probe["source_witness"], deadline) if opportunity is not None and opportunity["native_incumbent"] is not None: selected = opportunity["native_incumbent"] witness = probe["source_witness"].get(metadata["evaluation_opportunity"]["incumbent_component_id"], []) @@ -459,6 +484,13 @@ def capture_after_wake(env: dict[str, str], receipt: dict[str, Any], *, witness[0].get("size") != selected["archive_bytes"]): raise ValueError("native incumbent participant differs from its captured result") destination.parent.mkdir(parents=True, exist_ok=True) + if request.get("controller_handoff"): + if research is None: + raise ValueError("controller checkpoint requires complete canonical research context") + from .checkpoint_handoff import await_controller_capture + captured = await_controller_capture(root, components, metadata, probe, deadline) + receipt["checkpoint"] = {"captured": True, **captured} + return captured = effect_runtime_result("runtime.checkpoint_artifacts", { "action": "capture", "destination": str(destination), "components": components, "point_metadata": metadata, diff --git a/benchmark/runtime/checkpoint_handoff.py b/benchmark/runtime/checkpoint_handoff.py new file mode 100644 index 0000000000..753ab6fc95 --- /dev/null +++ b/benchmark/runtime/checkpoint_handoff.py @@ -0,0 +1,56 @@ +"""Private file rendezvous while the owned native process remains interrupted. + +Only the external controller can freeze all worker writers and capture native +evaluator authority. This transport grants neither capability to the worker. +""" +from __future__ import annotations + +import hashlib +import json +import os +import time +import uuid + + +def await_controller_capture(root, components, metadata, probe, deadline): + directory = root.parent / "checkpoint-controller" + directory.mkdir(mode=0o700, exist_ok=True) + if directory.resolve() != directory: + raise ValueError("checkpoint controller mailbox must be physical") + nonce = uuid.uuid4().hex + ready = directory / "ready.json" + reply = directory / (nonce + ".json") + value = {"capture_id": nonce, "components": components, + "point_metadata": metadata, "source_sha256": probe["source_sha256"], + "source_witness": probe["source_witness"], + "deadline_epoch": deadline} + raw = json.dumps(value, sort_keys=True, separators=(",", ":")).encode() + # A surviving older request is an explicit recovery problem, not authority + # to replace a missed point with this later one. + pending = directory / (nonce + ".request") + try: + with pending.open("xb") as file: + os.chmod(pending, 0o600) + file.write(raw) + file.flush() + os.fsync(file.fileno()) + os.link(pending, ready) # Atomic publication without replacing an old request. + finally: + pending.unlink() + try: + while time.time() < deadline: + if reply.exists(): + if reply.is_symlink(): + raise ValueError("checkpoint controller reply must be physical") + result = json.loads(reply.read_bytes()) + if result.get("request_sha256") != hashlib.sha256(raw).hexdigest(): + raise ValueError("checkpoint controller reply belongs to another boundary") + if result.get("ok") is not True: + raise ValueError("controller checkpoint failed: " + str(result.get("error_kind"))) + return result["capture"] + time.sleep(.05) + raise TimeoutError("controller did not capture before the original checkpoint deadline") + finally: + # The controller must reread this exact file under its pause barrier. + # Once absent, a late controller cannot capture a resumed source. + ready.unlink() diff --git a/benchmark/runtime/checkpoint_wake.py b/benchmark/runtime/checkpoint_wake.py index 422f7ed99d..74bf252a4f 100644 --- a/benchmark/runtime/checkpoint_wake.py +++ b/benchmark/runtime/checkpoint_wake.py @@ -67,7 +67,8 @@ def _first_replan(self, intent): def _capture(self, held, deadline): """Capture declared files/native history; never downgrade research/effects.""" - if any(self.request.get(key) for key in ("loopx_state", "evaluation_opportunity")): + if (any(self.request.get(key) for key in ("loopx_state", "evaluation_opportunity")) + and not self.request.get("controller_handoff")): raise ValueError("Interrupted full-state capture requires the controller writer barrier") destination = self.root / held["intent"]["obligation_id"] metadata = {"operator_metadata": self.request["point_metadata"], @@ -81,8 +82,17 @@ def _capture(self, held, deadline): raise TimeoutError("original wake budget expired before capture") components = list(self.request["components"]) native = None + research = None + if self.request.get("loopx_state"): + from .checkpoint import research_participants + evidence = held["path"].with_name(held["path"].stem + "-research.json") + components, research = research_participants(self.env, components, metadata, destination, evidence, window) + if research is None: + raise ValueError("controller checkpoint requires complete canonical research context") if self.request.get("native_session"): from .native_checkpoint import native_session_checkpoint + from .checkpoint import native_binding_participant + components, source_wake = native_binding_participant(self.env, components, metadata) guard = {"cli": self.env["LOOPX_CLI"], "workspace": self.env["LOOPX_PROJECT"], "registry": self.env["LOOPX_REGISTRY"], "runtime_root": self.env["LOOPX_RUNTIME_ROOT"], "goal_id": self.env["LOOPX_GOAL_ID"], "agent_id": self.env["LOOPX_AGENT_ID"], @@ -107,8 +117,21 @@ def _capture(self, held, deadline): if native is not None: confirmed = native_session_checkpoint(self.env, self.receipt["session"]["session_id"], timeout=max(.1, window - time.time()), interrupted_guard=guard) + from .sessions import BenchmarkSessionWake + from .codex import Execution + confirmed_wake = BenchmarkSessionWake(self.env, Execution(context="resume", + sandbox=self.env.get("LOOPX_CODEX_SANDBOX", "danger-full-access")), {}) + if ((confirmed_wake.session_id, confirmed_wake.goal_ref, confirmed_wake.digest) != + (source_wake.session_id, source_wake.goal_ref, source_wake.digest)): + raise ValueError("interrupted source session binding moved before sealing") if confirmed != native: raise ValueError("interrupted native cursor moved before sealing") + if self.request.get("loopx_state"): + from .checkpoint import verify_research_participants + verify_research_participants(self.env, components, metadata, research, probe["source_witness"], window) + if self.request.get("controller_handoff"): + from .checkpoint_handoff import await_controller_capture + return await_controller_capture(self.root, components, metadata, probe, window) self.root.mkdir(mode=0o700, parents=True, exist_ok=True) return effect_runtime_result("runtime.checkpoint_artifacts", { "action": "capture", "components": components, "destination": str(destination), diff --git a/benchmark/runtime/sforge_backend.py b/benchmark/runtime/sforge_backend.py index 71c095a0d3..aa301db308 100644 --- a/benchmark/runtime/sforge_backend.py +++ b/benchmark/runtime/sforge_backend.py @@ -15,11 +15,12 @@ class RecordingDockerBackend(DockerBackend): def __init__(self, *, log_dir: Path, logger, oauth_proxy: bool = False, - blind_api_endpoint: tuple[str, int] | None = None, feedback=None, client=None): + blind_api_endpoint: tuple[str, int] | None = None, feedback=None, checkpoint=None, client=None): super().__init__(client) self.log_dir, self.logger = log_dir, logger self.blind_api_endpoint = blind_api_endpoint self.feedback = feedback + self.checkpoint = checkpoint self.execution_command = None self.execution_start_marker = None self.execution_receipt = None @@ -34,6 +35,8 @@ def create_container(self, image, name, **kwargs): kwargs["extra_hosts"] = hosts if self.feedback is not None: self.feedback.bind(kwargs.get("environment")) + if self.checkpoint is not None: + self.checkpoint.bind(kwargs.get("environment")) kwargs["environment"] = self._agent_environment(kwargs.get("environment")) return super().create_container(image, name, **kwargs) @@ -46,6 +49,8 @@ def _agent_environment(self, environment): def exec_run_with_timeout(self, handle, cmd, timeout=60, **kwargs): kwargs["environment"] = self._agent_environment(kwargs.get("environment")) is_solver = cmd == ["/bin/bash", "-c", getattr(self, "execution_command", None)] + if is_solver and self.checkpoint is not None: + self.checkpoint.start(handle.raw) try: result = super().exec_run_with_timeout(handle, cmd, timeout, **kwargs) if is_solver: @@ -64,6 +69,8 @@ def exec_run_with_timeout(self, handle, cmd, timeout=60, **kwargs): pending.replace(target) return result finally: + if is_solver and self.checkpoint is not None: + self.checkpoint.close() if is_solver and self.feedback is not None: self.feedback.pause() @@ -92,6 +99,8 @@ def start_feedback(self, handle): self.feedback.start(self, handle) def cleanup_container(self, handle, logger=None): + if self.checkpoint is not None: + self.checkpoint.close() if self.feedback is not None: self.feedback.close() if handle is not None: diff --git a/benchmark/tests/test_checkpoint_handoff.py b/benchmark/tests/test_checkpoint_handoff.py new file mode 100644 index 0000000000..0e2dacc8fa --- /dev/null +++ b/benchmark/tests/test_checkpoint_handoff.py @@ -0,0 +1,57 @@ +"""Private source rendezvous never substitutes a late or foreign point.""" +import hashlib +import json +import threading +import time + +import pytest + +from benchmark.runtime.checkpoint_handoff import await_controller_capture + + +@pytest.mark.parametrize("response", ["captured", "failed", "foreign", "timeout"]) +def test_original_request_only_and_cleanup_after_missed_point(tmp_path, response): + mailbox = tmp_path / "checkpoint-controller" + ready = mailbox / "ready.json" + observed = [] + def controller(): + until = time.time() + 2 + while not ready.exists() and time.time() < until: + time.sleep(.005) + raw = ready.read_bytes() + request = json.loads(raw) # Publication exposes a complete request. + observed.append(request) + if response == "timeout": + return + result = {"request_sha256": hashlib.sha256(raw).hexdigest(), + "ok": response == "captured", "error_kind": "SyntheticFailure", + "capture": {"research_point_qualified": False, "sha256": "retained"}} + if response == "foreign": + result["request_sha256"] = "another-point" + pending = mailbox / "reply.tmp" + pending.write_text(json.dumps(result)) + pending.rename(mailbox / (request["capture_id"] + ".json")) + thread = threading.Thread(target=controller) + thread.start() + try: + args = (tmp_path / "checkpoints", [{"id": "code", "source_path": "/task/code"}], {}, + {"source_sha256": "prepared", "source_witness": {}}, time.time() + .5) + if response == "captured": + assert await_controller_capture(*args)["sha256"] == "retained" + else: + with pytest.raises(TimeoutError if response == "timeout" else ValueError): + await_controller_capture(*args) + finally: + thread.join(3) + assert not thread.is_alive() and not ready.exists() and len(observed) == 1 + + +def test_surviving_request_is_not_overwritten(tmp_path): + mailbox = tmp_path / "checkpoint-controller" + mailbox.mkdir() + ready = mailbox / "ready.json" + ready.write_bytes(b"original uncertain request") + with pytest.raises(FileExistsError): + await_controller_capture(tmp_path / "checkpoints", [], {}, + {"source_sha256": "later", "source_witness": {}}, time.time() + 1) + assert ready.read_bytes() == b"original uncertain request" diff --git a/benchmark/tests/test_checkpoint_worker.py b/benchmark/tests/test_checkpoint_worker.py index 7a64a4cf8e..404542a0fd 100644 --- a/benchmark/tests/test_checkpoint_worker.py +++ b/benchmark/tests/test_checkpoint_worker.py @@ -76,6 +76,69 @@ def counter(container): return int(archive.extractfile("counter").read()) +def test_archive_keeps_private_directory_modes_without_special_bits(writer, tmp_path): + from benchmark.edgebench.checkpoint_worker import _archive + container, _ = writer + assert container.exec_run(["python", "-c", "from pathlib import Path; " + "p=Path('/private-state'); p.mkdir(mode=0o700); " + "(p/'child').mkdir(mode=0o700); (p/'child').chmod(0o2700)"]).exit_code == 0 + copied = _archive(container, "/private-state", tmp_path / "copy") + assert copied.stat().st_mode & 0o7777 == 0o700 + assert (copied / "child").stat().st_mode & 0o7777 == 0o700 + + +@pytest.mark.parametrize("fault", [None, "source_moved", "expired"]) +def test_supervisor_captures_only_the_prepared_worker_boundary(controllers, writer, tmp_path, fault): + from loopx.control_plane.effect_runtime import effect_runtime_result + from benchmark.edgebench.checkpoint_worker import _archive + from benchmark.edgebench.checkpoint_supervisor import CheckpointSupervisor + + source, _, first, _, files, grades = controllers + container, _ = writer + token = register(source, first, submission_cooldown=3600) + assert submit(first, token, 7).status_code == 200 + assert container.exec_run(["python", "-c", "from pathlib import Path; " + "Path('/candidate').write_text('retained'); " + "Path('/logs/agent/checkpoint-controller').mkdir(parents=True); " + "Path('/opt/loopx-benchmark/control').mkdir(parents=True); " + "Path('/opt/loopx-benchmark/control/phase-deadline').write_bytes(Path('/phase').read_bytes())"]).exit_code == 0 + prepared = _archive(container, "/candidate", tmp_path / "prepared") + probe = effect_runtime_result("runtime.checkpoint_artifacts", {"action": "probe", + "components": [{"id": "code", "source_path": str(prepared)}]}) + nonce = "e" * 32 + request = {"capture_id": nonce, "components": [{"id": "code", "source_path": "/candidate"}], + "point_metadata": {"synthetic_boundary": True}, "source_sha256": probe["source_sha256"], + "source_witness": probe["source_witness"], + "deadline_epoch": time.time() + (-1 if fault == "expired" else 30)} + if fault == "source_moved": + assert container.exec_run(["python", "-c", "from pathlib import Path; Path('/candidate').write_text('later')"]).exit_code == 0 + assert container.exec_run(["python", "-c", "from pathlib import Path; " + "Path('/logs/agent/checkpoint-controller/ready.json').write_text(" + repr(json.dumps(request)) + ")"]).exit_code == 0 + supervisor = CheckpointSupervisor(files[0], tmp_path / "points") + supervisor.bind({"SFORGE_TOKEN": token}) + supervisor.start(container) + try: + wait(lambda: (tmp_path / "points" / (nonce + ".json")).exists()) + wait(lambda: container.exec_run(["test", "-f", "/logs/agent/checkpoint-controller/" + nonce + ".json"]).exit_code == 0) + finally: + supervisor.close() + response = json.loads((tmp_path / "points" / (nonce + ".json")).read_bytes()) + assert response["ok"] is (fault is None) + unit = tmp_path / "points" / nonce + if fault is None: + manifest = json.loads((unit / "checkpoint.json").read_bytes()) + metadata = manifest["point_metadata"] + assert metadata["synthetic_boundary"] is True + assert metadata["worker_file_snapshot"]["prepared_source_verified"] is True + assert (unit / "components/code").read_text() == "retained" + assert response["capture"]["research_point_qualified"] is False + else: + assert not unit.exists() + assert not (Path(json.loads(files[0].read_bytes())["bank_root"]) / nonce).exists() + container.reload() + assert not container.attrs["State"]["Paused"] and len(grades) == 1 + + @pytest.mark.parametrize("directory", [False, True]) def test_paused_sqlite_capture_retains_committed_wal_not_pending_transaction( controllers, writer, tmp_path, directory): diff --git a/benchmark/tests/test_native_checkpoint_handoff.py b/benchmark/tests/test_native_checkpoint_handoff.py index b8bcbfa2fe..d36a146a66 100644 --- a/benchmark/tests/test_native_checkpoint_handoff.py +++ b/benchmark/tests/test_native_checkpoint_handoff.py @@ -345,7 +345,7 @@ def test_checkpoint_interrupt_timeout_reaps_without_qualifying_capture(tmp_path) @pytest.mark.skipif(not os.environ.get("LOOPX_NATIVE_CHECKPOINT_BIN") or os.name != "posix", reason="Requires the real native CLI and private Unix transport") -@pytest.mark.parametrize("capture", ["files", "native", "full", "off"]) +@pytest.mark.parametrize("capture", ["files", "native", "full", "off", "controller"]) def test_actual_worker_handles_midwake_replan_and_duplicate_without_guard_replay(tmp_path, capture): """Use run_once, its renderer/listener/session owner and the actual quota CLI.""" import sys @@ -357,14 +357,18 @@ def test_actual_worker_handles_midwake_replan_and_duplicate_without_guard_replay _heartbeat_receipt_count, _run_cli, ) + if capture == "controller" and os.environ.get("LOOPX_CHECKPOINT_EXTERNAL_CONTROLLER") != "1": + pytest.skip("Requires an explicitly attached host checkpoint supervisor") project, runtime, registry = _write_fixture(tmp_path / "authority") - logs = tmp_path / "trial" / "wakes" + logs = Path("/logs/agent/wakes") if capture == "controller" else tmp_path / "trial" / "wakes" candidate = project / "candidate.txt" candidate.write_text("original candidate") request = tmp_path / "request.json" request.write_text(json.dumps({"destination": str(logs.parent / "checkpoints"), "components": [{"id": "code", "source_path": str(candidate)}], - "point_metadata": {}, "trigger": "fresh_replan", "native_session": capture in {"native", "full"}, "loopx_state": capture == "full"})) + "point_metadata": {}, "trigger": "fresh_replan", "native_session": capture in {"native", "full", "controller"}, + "loopx_state": capture in {"full", "controller"}, + "evaluation_opportunity": capture == "controller", "controller_handoff": capture == "controller"})) skills = tmp_path / "skills" skills.mkdir() cli = tmp_path / "loopx-cli" @@ -414,6 +418,15 @@ def response(payload, index, message): Path(row["json_path"]).write_text(json.dumps(row)) later.append(json.dumps(row) + "\n") index_path.write_text(prior_runs + "".join(later)) + if capture == "controller": + from tests.control_plane.canonical_authority_fixture import initialize_canonical_authority + from loopx.control_plane.coordination.runtime_shadow import build_todo_runtime_shadow_projection + rc, listed = _run_cli(registry, runtime, "todo", "list", "--goal-id", GOAL_ID) + assert rc == 0, listed + initialize_canonical_authority(runtime, GOAL_ID, + build_todo_runtime_shadow_projection(goal_id=GOAL_ID, todos=listed["todos"], + handoff_mode="hard_lease", leases=[]), + state_path=project / f".codex/goals/{GOAL_ID}/ACTIVE_GOAL_STATE.md", provider="sqlite") body = next(t for t in texts(payload) if "quota should-run" in t and "```sh" in t) guard = next(line for line in body.splitlines() if "quota should-run" in line) guard = guard.replace('${LOOPX_TURN:?}', TURN_ID) @@ -452,12 +465,15 @@ def response(payload, index, message): return assert len(result["checkpoint_handoffs"]) == 1 event = result["checkpoint_handoffs"][0] - assert event["captured"] is (capture in {"files", "native"}) + assert event["captured"] is (capture in {"files", "native", "controller"}), json.dumps(event) assert event["research_point_qualified"] is False assert len(calls) == 4 and (project / "continued.txt").read_text() == "continued" assert "heartbeat_receipt" in json.dumps(requests[2]) assert any(item.get("type") == "function_call_output" and item.get("call_id") == "call_midwake" and "aborted" in item.get("output", "") for item in requests[2]["input"]) + if capture == "controller": + (tmp_path / "controller-result.json").write_text(json.dumps(event)) + return # The external controller verifies the combined artifact. if capture in {"files", "native"}: point = logs.parent / "checkpoints" / event["obligation_id"] assert (point / "components" / "code").read_text() == "original candidate" diff --git a/benchmark/tests/test_sforge_runtime.py b/benchmark/tests/test_sforge_runtime.py index 10eebe86f3..597da27ac9 100644 --- a/benchmark/tests/test_sforge_runtime.py +++ b/benchmark/tests/test_sforge_runtime.py @@ -224,6 +224,7 @@ def test_artifact_collection_preserves_both_session_homes_without_auth(tmp_path, pytest.importorskip("harbor") from benchmark.runtime.sforge_backend import RecordingDockerBackend, DockerBackend backend = object.__new__(RecordingDockerBackend) + backend.checkpoint = None backend.log_dir = tmp_path / "artifacts" backend.feedback = None copied, cleaned = [], [] @@ -277,6 +278,7 @@ def test_blind_policy_removes_judge_route_and_credentials_native_is_unchanged(mo pytest.importorskip("harbor") from benchmark.runtime.sforge_backend import RecordingDockerBackend, DockerBackend, AllowedEndpoint backend = object.__new__(RecordingDockerBackend) + backend.checkpoint = None backend.auth_ips = [] backend.blind_api_endpoint = None judge = AllowedEndpoint(ip="192.0.2.1", port=8080, hostname="judge") @@ -494,6 +496,50 @@ def test_edgebench_rejects_invalid_profile_settings_before_creating_trial(tmp_pa assert not (tmp_path / 'runs').exists() +@pytest.mark.parametrize("missing", ["controller", "declaration", "native_feedback"]) +def test_controller_handoff_cannot_be_partially_enabled(tmp_path, missing): + pytest.importorskip("sforge") + pytest.importorskip("harbor") + from benchmark.edgebench.run import main + request = tmp_path / "checkpoint.json" + request.write_text(json.dumps({"destination": "/logs/agent/checkpoints", "point_metadata": {}, + "components": [{"id": "code", "source_path": "/task/code"}], "trigger": "fresh_replan", + "native_session": True, "loopx_state": True, "evaluation_opportunity": True, + "controller_handoff": True})) + options = [] if missing == "controller" else ["--checkpoint-controller-file", str(tmp_path / "private.json")] + if missing != "declaration": + options.extend(["--checkpoint-request", str(request)]) + if missing == "native_feedback": + options.extend(["--feedback", "best-only"]) + with pytest.raises(SystemExit) as error: + main(["--task", "fixture", "--tasks-dir", str(tmp_path), "--log-dir", str(tmp_path), + "--run-id", "invalid", "--worker", "heartbeat-resume", "--model", "fixture", + "--effort", "xhigh", "--judge-url", "http://127.0.0.1:9999", *options]) + assert error.value.code == 2 and not (tmp_path / "runs").exists() + + +def test_checkpoint_supervisor_is_bound_to_container_and_only_solver_execution(tmp_path, monkeypatch): + pytest.importorskip("sforge") + pytest.importorskip("harbor") + from types import SimpleNamespace + from benchmark.runtime.sforge_backend import RecordingDockerBackend, DockerBackend + calls = [] + controller = SimpleNamespace(bind=lambda env: calls.append(("bind", env["SFORGE_TOKEN"])), + start=lambda raw: calls.append(("start", raw)), close=lambda: calls.append(("close",))) + backend = RecordingDockerBackend(log_dir=tmp_path / "collected", logger=None, + oauth_proxy=True, checkpoint=controller) + handle = SimpleNamespace(raw="exact-container") + monkeypatch.setattr(DockerBackend, "create_container", lambda *a, **kw: handle) + backend.create_container("image", "name", environment={"SFORGE_TOKEN": "synthetic"}) + backend.execution_command = "solver" + monkeypatch.setattr(DockerBackend, "exec_run_with_timeout", lambda *a, **kw: + SimpleNamespace(exit_code=0, timed_out=False, elapsed_seconds=1)) + backend.exec_run_with_timeout(handle, ["setup"]) + assert calls == [("bind", "synthetic")] + backend.exec_run_with_timeout(handle, ["/bin/bash", "-c", "solver"]) + assert calls == [("bind", "synthetic"), ("start", "exact-container"), ("close",)] + + @pytest.mark.parametrize("protect_fails", [False, True]) def test_host_checkpoint_declaration_is_staged_verbatim_and_protected(tmp_path, monkeypatch, protect_fails): pytest.importorskip("sforge") @@ -729,6 +775,7 @@ def test_recorded_solver_exit_does_not_take_status_from_output(tmp_path, monkeyp from types import SimpleNamespace from benchmark.runtime.sforge_backend import RecordingDockerBackend, DockerBackend backend = object.__new__(RecordingDockerBackend) + backend.checkpoint = None backend.log_dir = tmp_path / "collected" backend.blind_api_endpoint = None backend.feedback = None From 67d2d590081376a786d4bd286a74a6bf4433f956 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Wed, 14 Oct 2026 02:19:59 +0800 Subject: [PATCH 60/69] docs(benchmark): define joint source capture and remaining continuation acceptance Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/README.md | 88 ++++++++++++------- .../hierarchical-agent-stride-control-v0.md | 23 ++--- ...rarchical-agent-stride-control-v0.zh-CN.md | 16 ++-- 3 files changed, 79 insertions(+), 48 deletions(-) diff --git a/benchmark/README.md b/benchmark/README.md index 0ebed6361a..6e790ebd6f 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -244,11 +244,9 @@ that entire captured prefix, including the aborted tool, without using a complet Turn as a substitute; completed-cursor checks stay unchanged. Lost fork replies reuse the existing one-attempt recovery path. -Canonical research/evaluation requests still reject at this mid-wake path. Native -history plus declared files is not a controller-wide writer barrier or authority -admission. Interrupted sessions cannot enter destination Goal admission until the -joint controller continuation contract is qualified. The next acceptance composes -that barrier, research/evaluator/budget adoption and isolated useful continuation. +Full capture uses the external controller path below. Native-only history plus +declared files remains an inert subset. Interrupted sessions still cannot enter +destination Goal admission until joint controller continuation is qualified. Remove the checkpoint request to disable automatic handoff; a direct caller can omit the socket option to disable only the transport. @@ -497,17 +495,11 @@ caller IDs. The sealed `native_session.store_members` mapping binds those IDs to the existing allowlisted native filenames. Reconstruction accepts earlier fixed-ID artifacts, rejects duplicate/undeclared mappings and keeps source-history checks. -The current Host hook cannot capture a new replan created **inside an already -running model wake** before that decision. Those boundaries still need a qualified -Host handoff; a later snapshot must not be labeled as the missed point. A real -native qualification now demonstrates interrupting at a synthetic tool boundary, -forking the exact retained history and producing a new artifact while preserving -the source history. The existing checkpoint correctly rejects that interrupted -cursor: native resumability does not prove settled LoopX work, a pre-decision -replan boundary or joint evaluator consistency. The probe does not yet wire the -worker handoff or reconstruct a separate solver environment. Complete -LoopX research-state adoption, joint evaluator/Host admission and useful continued work -remain open. No automatic retention or extra evaluation opportunity is introduced. +Fresh replan can arise at Host preflight or inside a running native wake. The +explicit handoff retains the original guard and rejects replacement by a later +point. Full-state capture additionally requires the controller writer barrier +below. Capturing that point does not admit a destination session or prove useful +continued research. ### Controller-only native capsule transport @@ -579,15 +571,43 @@ later files with an older native point. Missing paths, archive links, traversal Declare only the intended private code/state stores: a whole home can contain credentials, and neither archive filtering nor a freeze is privacy certification. -This is an **inert joint file snapshot**, still `research_point_qualified=false`. -It does not snapshot process memory, qualify a pending/interrupted Codex cursor, -verify externally hosted writers or activate canonical research state. Ordinary -native restore rejects this outer unit. Worker/evaluator pairing is explicitly -caller-declared here, not a verified automatic SForge run binding. A real Docker writer plus native HTTP -test verifies unchanged file bytes throughout evaluator capture, retained used -rounds/cooldown/phase, source resumption and failure cleanup. Automatic quota -handoff before the replan decision and isolated useful continuation remain the -next integration boundary; this command alone does not qualify experiment restart. +This explicit command produces an **inert joint file snapshot**, still +`research_point_qualified=false`. It does not snapshot process memory, freeze +external writers or activate canonical research state. Ordinary native restore +rejects the outer unit; callers must use the declared native capsule and existing +fresh-authority admission paths. + +For automatic capture, the heartbeat declaration must include `"trigger": +"fresh_replan"`, `"native_session": true`, `"loopx_state": true`, +`"evaluation_opportunity": true`, and `"controller_handoff": true`, in addition +to its authorized code components, destination and metadata. Pass both +`--checkpoint-request REQUEST.json` and `--checkpoint-controller-file CONTROLLER.json` +to `benchmark.edgebench.run`, with native feedback. The controller file remains +host-private; only the declaration enters the worker. Missing or mismatched opt-ins +reject before trial creation. The backend binds the supervisor to its actual +container and native registration, rather than accepting a solver-supplied token. + +The worker prepares native history, the original decision, canonical research +context and its file witness while holding the model boundary. The controller +pauses that worker, rereads the exact live handoff and copies declared sources +through Docker archive. A differing source witness or expired/removed request +rejects **before** native evaluator capture. Ordinary permission bits survive safe +archive extraction so private authority directories do not falsely appear changed. +The controller seals the frozen files with incumbent, used opportunities, cooldown +and remaining Host phase. It releases the barrier before publishing a private reply; +the source continues from the original packet and deadline. Optional capture +failure records a missed point, never a later replacement or a new model/eval budget. +A barrier that cannot be released remains an explicit controller recovery problem. + +A real native CLI/worker plus isolated Docker/native-HTTP qualification covers the +mid-wake path, complete canonical context, conserved evaluator opportunity and +original source continuation. It uses a deterministic local model and synthetic +grading. The artifact remains unqualified for scientific restart: isolated joint +destination admission, useful work, original-effect recovery and subsequent grading +must still be verified together. No old lease, quota, timer or token is activated. +Omit the controller file and `controller_handoff` on a new run to disable automatic +controller capture; removing the checkpoint request disables capture entirely. +Existing artifacts remain inert, and the source trial is not restarted by disabling it. An explicit **SForge phase-clock participant** preserves a declared Host file's remaining duration alongside native opportunities. Add @@ -673,8 +693,16 @@ runtime receipt 绑定摘要；programmatic adapter 也可指定已放入 guest 物化后仍是惰性历史。缺少来源、共享注册表、路径别名或重叠声明会拒绝，捕获 不授予新的租约、配额、效果或执行权，不能替代完整接续验收。 -运行中才出现的 fresh replan 仍缺 Host 交接。真实 native 探针已证明：在合成工具 -边界 interrupt 后，可 fork 完整原生历史、继续读原候选并产出新文件，原历史不变。 -现有 checkpoint 仍正确拒绝该 interrupted cursor；可继续原生会话不等于已结算 -LoopX 工作、replan 决策前边界或 evaluator 联合一致性。该探针没有接入 worker -自动捕获，也没有证明独立 solver 环境恢复，不能据此宣称每个 replan 已覆盖。 +运行中出现的 fresh replan 可通过显式交接捕获。完整捕获需同时声明 +`controller_handoff`、`native_session`、`loopx_state`、`evaluation_opportunity` +为 true，使用 `fresh_replan`，并在 host CLI 提供 `--checkpoint-controller-file`。 +controller 配置及 admin 权限不进入 solver；backend 绑定真实 container 与原生注册。 +worker 保留原 guard，controller 暂停全部容器写入者，核对仍有效的请求和准备时 +文件 witness，再封存原生评测机会、incumbent 及剩余预算。源变化、过期或失去 +交接会拒绝；可恢复的捕获失败不取消原研究，也不得改用稍后的状态。解除暂停后， +源会话继续使用原 packet 和 deadline。 +真实 native worker、Docker 与 native HTTP 的合成验证覆盖了联合捕获和源接续； +产物仍是 `research_point_qualified=false`。独立目标环境的新权限准入、有效工作、 +联合丢回复恢复及后续评测尚待一体验证，不能据此重启研究实验。新运行省略 +controller 文件及 handoff 开关即可关闭；移除 checkpoint 请求关闭全部捕获， +已有产物保持惰性。 diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md index 6f30e9dbb3..f2a0ebfb18 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md @@ -895,17 +895,18 @@ Model inference is excluded. Full Host provenance/clock alignment and joint session/research/effect recovery still need their profile's qualification; this participant transport cannot certify them. -The composed local continuation qualification now carries the evaluator capsule -inside the verified conversation/code/context artifact. The actual resumed Codex -tool reads retained research and public feedback, computes a synthetic candidate -from captured code, submits it through the native HTTP owner, and observes the -inherited round limit. The original cooldown rejects an immediate submission; -SForge's phase entry uses the remaining duration instead of a new trial budget. -Source future evidence is excluded and its later state is unchanged. Participants -are explicitly quiesced by the fixture; inference and grading are synthetic. -This narrows the integration gap but does not prove an automatic joint barrier, -canonical graph/vision/effect activation, original activation recovery, isolated -Linux solver restoration, or useful research. +The automatic source-capture boundary now composes the real worker's held +fresh-replan decision with its native cursor, canonical research inventory and +controller-owned Docker pause/archive barrier. The controller verifies the exact +pending request and prepared source witness before sealing native incumbent, +opportunity and remaining phase budget. A changed or expired boundary is missed, +not recaptured after the source resumes. Synthetic native CLI/Docker/HTTP evidence +covers this capture and original source continuation; it does not qualify a joint +destination activation. The next acceptance remains one isolated destination: +consume the captured context/code, obtain fresh Goal/session/evaluator authority, +perform useful work, settle its original effect and grade without refilling budget. +Preserve the existing lost-reply and wrong-source negatives in that composed path. +No new research cells are admitted by the capture milestone alone. Committed native session admission now retains the captured artifact digest in the original atomic session binding. Exact controller retries after a lost binding diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md index 491073ef57..51dc05b95c 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md @@ -740,13 +740,15 @@ phase 文件参与者还将声明的 Host deadline 与 native 捕获时间绑定 完整 Host 来源/时钟对齐及联合会话/研究/effect 恢复仍须按 profile 资格化， 参与者传输本身不能证明这些结果。 -组合式本地接续验证现将 evaluator capsule 放入同一已验证的会话/代码/上下文产物。 -实际恢复的 Codex 工具读取保留研究与公开反馈，从捕获代码计算合成候选，经原生 -HTTP owner 提交并观察继承的轮次上限。恢复后的首次立即提交仍被原冷却拒绝； -SForge phase 入口使用剩余时长，不重置整场预算。来源的后续证据被排除，其后续 -状态保持不变。参与者由 fixture 显式静置；模型响应与评分计算均为合成。 -这缩小了集成缺口，但不证明自动联合 barrier、canonical graph/vision/effect -激活、原激活操作恢复、隔离 Linux solver 恢复或有效研究。 +自动源捕获已将真实 worker 的 fresh-replan 原始决策、原生 cursor、canonical +研究 inventory 与 controller 的 Docker pause/archive 屏障组合起来。controller +先验证仍有效的请求及准备时的 source witness，再封存 incumbent、评测机会和 +剩余 phase 预算。变化或过期的边界记为未捕获，不能在源恢复后补作原点。 +真实 native CLI/Docker/HTTP 的合成验证覆盖联合捕获及原源接续；仍不代表目标 +联合激活。下一验收是独立目标环境消费保留上下文和代码、取得新权限、开展有用 +工作、结算原 effect 并完成后续评测，同时不补预算并保留丢回复/错误源负例。 +仅捕获里程碑不准入新的研究组。 + inert verification unit 不授予执行权。目标侧使用新的实验/Host 身份，通过既有 Goal/session/state admission 进入。历史租约、quota、timer、pending effect 和 From 21830cda44d0bb8da6917875f018845fd3d337ea Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Wed, 14 Oct 2026 02:39:11 +0800 Subject: [PATCH 61/69] fix(benchmark): admit captured interrupted history under fresh authority Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/runtime/native_checkpoint.py | 12 +- benchmark/runtime/research_checkpoint.py | 6 +- .../tests/test_checkpoint_joint_receiver.py | 439 ++++++++++++++++++ .../tests/test_native_checkpoint_handoff.py | 2 +- 4 files changed, 451 insertions(+), 8 deletions(-) create mode 100644 benchmark/tests/test_checkpoint_joint_receiver.py diff --git a/benchmark/runtime/native_checkpoint.py b/benchmark/runtime/native_checkpoint.py index ea80944c2c..841f9a09fb 100644 --- a/benchmark/runtime/native_checkpoint.py +++ b/benchmark/runtime/native_checkpoint.py @@ -382,6 +382,8 @@ def admit_native_checkpoint_session(env: dict[str, str], unit: Path, expected_sh Existing native fork, Goal lifetime and session owners retain their rules. Historical research/effects and evaluator/whole-trial budget are not adopted by this participant. Admission alone never launches work or a model. + An interrupted cursor is retained as history, not as an executable source + guard. The destination must obtain its own current work/replan admission. """ from loopx.control_plane.effect_runtime import effect_runtime_result from loopx.agent_registry import load_goal_from_registry, require_registered_agent_id @@ -400,8 +402,6 @@ def admit_native_checkpoint_session(env: dict[str, str], unit: Path, expected_sh "action": "verify", "root": str(unit), "expected_sha256": expected_sha256, }, timeout=timeout) metadata = json.loads((unit / "checkpoint.json").read_bytes())["point_metadata"] - if metadata.get("native_session", {}).get("interrupted_guard") is not None: - raise ValueError("interrupted session admission requires the joint controller continuation contract") source_profile = (metadata.get("session") or {}).get("session_profile_digest") if not isinstance(source_profile, str) or not re.fullmatch(r"[0-9a-f]{64}", source_profile): raise ValueError("checkpoint session admission requires its captured source session profile") @@ -438,12 +438,14 @@ def admit_native_checkpoint_session(env: dict[str, str], unit: Path, expected_sh raise ValueError("checkpoint source session profile changed") native = metadata["native_session"] observed = native_session_checkpoint(env, binding["session_id"], - timeout=max(.1, deadline - time.monotonic()))["metadata"] - keys = ("completed_turn_id", "turn_ids", "visible_history_sha256", "host_version", + timeout=max(.1, deadline - time.monotonic()), + interrupted_guard=native.get("interrupted_guard"))["metadata"] + cursor_key = "interrupted_turn_id" if native.get("interrupted_guard") is not None else "completed_turn_id" + keys = (cursor_key, "turn_ids", "visible_history_sha256", "host_version", "guest_home", "guest_workspace", "config_sha256") if binding["session_id"] == native["session_id"] or any(observed[key] != native[key] for key in keys): raise ValueError("checkpoint admitted native session advanced or differs; use its existing continuation") - return {"session_id": binding["session_id"], "completed_turn_id": observed["completed_turn_id"], + return {"session_id": binding["session_id"], cursor_key: observed[cursor_key], "visible_history_sha256": observed["visible_history_sha256"], "history_verified": True, "model_calls": 0, "execution_authority_granted": False, "continuation_qualified": False, "destination_goal_ref": wake.goal_ref, "destination_agent_id": lineage["agent_id"], diff --git a/benchmark/runtime/research_checkpoint.py b/benchmark/runtime/research_checkpoint.py index 730efe3cb9..44bc934284 100644 --- a/benchmark/runtime/research_checkpoint.py +++ b/benchmark/runtime/research_checkpoint.py @@ -144,8 +144,10 @@ def _context_session(env, metadata): if (not wake.session_id or wake.session_id == native["session_id"] or wake.digest != metadata.get("session", {}).get("session_profile_digest")): raise ValueError("context adoption requires the admitted destination session profile") - observed = native_session_checkpoint(env, wake.session_id, timeout=30)["metadata"] - for key in ("visible_history_sha256", "turn_ids", "completed_turn_id", "host_version", "config_sha256"): + observed = native_session_checkpoint(env, wake.session_id, timeout=30, + interrupted_guard=native.get("interrupted_guard"))["metadata"] + cursor_key = "interrupted_turn_id" if native.get("interrupted_guard") is not None else "completed_turn_id" + for key in ("visible_history_sha256", "turn_ids", cursor_key, "host_version", "config_sha256"): if observed[key] != native[key]: raise ValueError("destination native history or configuration moved beyond the captured context") return wake.session_id diff --git a/benchmark/tests/test_checkpoint_joint_receiver.py b/benchmark/tests/test_checkpoint_joint_receiver.py new file mode 100644 index 0000000000..6c3d65448d --- /dev/null +++ b/benchmark/tests/test_checkpoint_joint_receiver.py @@ -0,0 +1,439 @@ +"""Qualification of an actual controller-captured point in a fresh Linux guest. + +The external fixture supplies a sealed point and its matching synthetic native +configuration. No source authority directories are mounted into this receiver. +""" + +import hashlib +import json +import os +from pathlib import Path +import shlex +import subprocess +import sys +import threading +import tomllib +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer +from urllib.parse import urlsplit + +import pytest + +from benchmark.tests.test_checkpoint_controller import controllers # noqa: F401 + + +@pytest.mark.skipif( + not os.environ.get("LOOPX_JOINT_RECEIVER_POINT"), + reason="Requires a real controller point in a fresh isolated guest", +) +def test_captured_interrupted_point_continues_with_fresh_authority( + tmp_path, monkeypatch, capsys, request +): + from benchmark.edgebench.checkpoint_controller import restore + from benchmark.runtime.codex import Execution + from benchmark.runtime.native_checkpoint import ( + admit_native_checkpoint_session, + native_session_checkpoint, + ) + from benchmark.runtime.research_checkpoint import adopt_todo_context + from benchmark.runtime.sessions import BenchmarkSessionWake + from benchmark.runtime.worker import native_command + from loopx.cli import main + from loopx.control_plane.effect_runtime import ( + effect_runtime_result, + EffectRuntimeRemoteError, + ) + from loopx.control_plane.quota.settlement import read_heartbeat_settlement + + unit = Path(os.environ["LOOPX_JOINT_RECEIVER_POINT"]) + manifest = json.loads((unit / "checkpoint.json").read_bytes()) + digest, metadata = manifest["sha256"], manifest["point_metadata"] + native, snapshot = metadata["native_session"], metadata["worker_file_snapshot"] + assert snapshot["prepared_source_verified"] and native["interrupted_guard"] + config = Path(os.environ["LOOPX_JOINT_RECEIVER_CONFIG"]) + workspace, home = Path(native["guest_workspace"]), Path(native["guest_home"]) + assert not workspace.exists() and not home.exists(), ( + "receiver must have fresh fixed guest paths" + ) + workspace.mkdir(parents=True) + monkeypatch.chdir(workspace) + candidate = workspace / "candidate.txt" + candidate.write_bytes((unit / "components/code").read_bytes()) + registry, runtime = tmp_path / "registry.json", tmp_path / "runtime" + goal, agent, turn = "joint-receiver", "joint-researcher", "joint-work" + prefix = [ + "--format", + "json", + "--registry", + str(registry), + "--runtime-root", + str(runtime), + ] + + def cli(*args): + code = main([*prefix, *args]) + output = capsys.readouterr().out + assert code == 0, output + return json.loads(output) + + cli( + "bootstrap", + "--project", + str(workspace), + "--goal-id", + goal, + "--objective", + "Continue the captured synthetic experiment.", + "--adapter-kind", + "read_only_project_map_v0", + "--adapter-status", + "connected-read-only", + "--write-scope", + "**", + "--no-global-sync", + ) + cli( + "configure-goal", + "--goal-id", + goal, + "--registered-agent", + agent, + "--agent-work-mode", + agent + "=active", + "--execute", + ) + todo = cli( + "todo", + "add", + "--goal-id", + goal, + "--role", + "agent", + "--claimed-by", + agent, + "--required-capability", + "shell", + "--task-class", + "advancement_task", + "--text", + "Use retained code and evidence for one synthetic candidate, then verify its native result.", + )["todo_id"] + execution = Execution(context="resume") + env = { + **os.environ, + "CODEX_BIN": os.environ["LOOPX_NATIVE_CHECKPOINT_BIN"], + "CODEX_HOME": str(home), + "LOOPX_PROJECT": str(workspace), + "LOOPX_REGISTRY": str(registry), + "LOOPX_RUNTIME_ROOT": str(runtime), + "LOOPX_GOAL_ID": goal, + "LOOPX_AGENT_ID": agent, + "MODEL_NAME": "fixture", + "REASONING_EFFORT": "high", + "LOOPX_CODEX_SANDBOX": execution.sandbox, + "SYNTHETIC_PROVIDER_KEY": "synthetic-local-only", + "OPENAI_API_KEY": "synthetic-local-only", + "LOOPX_USAGE_PING": "0", + } + with pytest.raises(ValueError, match="distinct destination"): + admit_native_checkpoint_session( + {**env, "LOOPX_GOAL_ID": metadata["loopx_state"]["goal_id"]}, + unit, + digest, + config, + execution=execution, + timeout=30, + ) + with pytest.raises(EffectRuntimeRemoteError): + admit_native_checkpoint_session( + env, unit, "f" * 64, config, execution=execution, timeout=30 + ) + assert not home.exists() + original_bind = BenchmarkSessionWake._bind_verified_fork + + def lose_binding_reply(self, *args): + original_bind(self, *args) + raise TimeoutError("synthetic loss after committed native binding") + + with monkeypatch.context() as lost: + lost.setattr(BenchmarkSessionWake, "_bind_verified_fork", lose_binding_reply) + with pytest.raises(TimeoutError, match="committed native binding"): + admit_native_checkpoint_session( + env, unit, digest, config, execution=execution, timeout=30 + ) + admitted = admit_native_checkpoint_session( + env, unit, digest, config, execution=execution, timeout=30 + ) + assert ( + admitted["session_id"] != native["session_id"] and admitted["model_calls"] == 0 + ) + replay = admit_native_checkpoint_session( + env, unit, digest, config, execution=execution, timeout=30 + ) + assert replay["session_id"] == admitted["session_id"] and replay["history_verified"] + lease = cli( + "task-lease", + "acquire", + "--goal-id", + goal, + "--todo-id", + todo, + "--owner", + agent, + "--idempotency-key", + "receiver-work", + ) + revision = cli("todo", "list", "--goal-id", goal, "--todo-id", todo)[ + "authority_read" + ]["provider_revision"] + carrier = json.loads((unit / "components/research_context").read_bytes()) + context_args = dict( + source_todo_id=carrier["snapshot"]["todos"][0]["todo_id"], + todo_id=todo, + operation_id="receiver-context", + expected_revision=revision, + lease_key="receiver-work", + lease_version=lease["lease"]["version"], + ) + adopted = adopt_todo_context(env, unit, digest, **context_args) + assert ( + adopt_todo_context(env, unit, digest, **context_args)["continuation"][ + "adoption" + ]["status"] + == "replayed" + ) + binding = [ + "--goal-id", + goal, + "--agent-id", + agent, + "--todo-id", + todo, + "--turn-instance-id", + turn, + ] + guard = cli("quota", "should-run", "--runtime-profile", "generic_cli", *binding) + assert guard["interaction_contract"]["agent_channel"]["delivery_allowed"] + _, branch, _, url, controller_files, grades = request.getfixturevalue("controllers") + capsule = unit / "components" / snapshot["native_capsule_component_id"] + phase = tmp_path / "phase-deadline" + fresh = restore( + controller_files[1], + root=capsule, + expected_sha256=snapshot["native_capsule_sha256"], + run_id="isolated-receiver", + phase_deadline_file=phase, + ) + original_phase = phase.read_bytes() + replay_activation = restore( + controller_files[1], + root=capsule, + expected_sha256=snapshot["native_capsule_sha256"], + run_id="isolated-receiver", + phase_deadline_file=phase, + recover=True, + ) + assert ( + replay_activation == fresh + and not grades + and phase.read_bytes() == original_phase + ) + from benchmark.tests.test_checkpoint_controller import host_phase_command + + phase_shell = host_phase_command(tmp_path, monkeypatch).replace( + "/opt/loopx-benchmark/control", str(tmp_path) + ) + (workspace / "judge.json").write_text( + json.dumps({"url": url, "token": fresh["token"]}) + ) + (workspace / "judge.json").chmod(0o600) + script = workspace / "continue.py" + script.write_text("""import json,time,requests +from pathlib import Path +b=json.loads(Path('judge.json').read_text()); url,token=b['url'],b['token'] +h=requests.get(url+'/api/v1/history',params={'token':token},timeout=5).json() +assert h['best_score']==7 and h['agent_submissions']==1 +assert Path('candidate.txt').read_text()=='original candidate' +Path('candidate.txt').write_text('8') +r=requests.post(url+'/api/v1/submit',data={'token':token},files={'archive':('candidate.tar.gz',b'8')},timeout=5) +r.raise_for_status(); sid=r.json()['submission_id'] +for _ in range(100): + result=requests.get(url+'/api/v1/result/'+sid,timeout=5).json() + if result['status']=='completed':break + time.sleep(.01) +assert result['status']=='completed' +print('Retained candidate and public best 7 used; candidate 8 graded.') +""") + requests = [] + + class Provider(BaseHTTPRequestHandler): + def log_message(self, *args): + pass + + def do_POST(self): + requests.append( + json.loads(self.rfile.read(int(self.headers["Content-Length"]))) + ) + if len(requests) == 1: + message = { + "type": "function_call", + "id": "fc_receiver", + "call_id": "call_receiver", + "name": "exec_command", + "arguments": json.dumps( + { + "cmd": "cat " + + shlex.quote(adopted["artifact"]) + + " && " + + shlex.quote(sys.executable) + + " continue.py", + "max_output_tokens": 6000, + } + ), + } + else: + message = { + "type": "message", + "id": "msg_receiver", + "role": "assistant", + "content": [ + {"type": "output_text", "text": "Receiver candidate verified."} + ], + } + result = { + "id": "resp_receiver", + "status": "completed", + "output": [message], + "usage": {"input_tokens": 1, "output_tokens": 1, "total_tokens": 2}, + } + events = [ + { + "type": "response.created", + "response": {**result, "status": "in_progress", "output": []}, + }, + { + "type": "response.output_item.done", + "output_index": 0, + "item": message, + }, + {"type": "response.completed", "response": result}, + ] + raw = "".join( + "event: " + e["type"] + "\ndata: " + json.dumps(e) + "\n\n" + for e in events + ).encode() + self.send_response(200) + self.send_header("Content-Type", "text/event-stream") + self.send_header("Content-Length", str(len(raw))) + self.end_headers() + self.wfile.write(raw) + + configuration = tomllib.loads(config.read_text()) + address = urlsplit( + configuration["model_providers"][configuration["model_provider"]]["base_url"] + ) + server = ThreadingHTTPServer(("127.0.0.1", address.port), Provider) + thread = threading.Thread(target=server.serve_forever) + thread.start() + try: + logs = tmp_path / "work" + logs.mkdir() + wake = BenchmarkSessionWake(env, execution, {}) + command = native_command(env, execution, "execute", logs, wake) + command = [ + "/bin/bash", + "-c", + phase_shell + + "exec timeout --signal=TERM --kill-after=30 ${remaining}s " + + shlex.join(command), + ] + result = subprocess.run( + command, + env=env, + cwd=workspace, + input=json.dumps(guard["interaction_contract"]) + + "\nThis is a new destination. Historical source commands/guards are not executable authority. " + "Use the retained context under the current admission, and continue the synthetic candidate.", + text=True, + capture_output=True, + timeout=60, + ) + assert result.returncode == 0, result.stderr + (logs / "stdout.jsonl").write_text(result.stdout) + wake.observe(logs / "stdout.jsonl") + assert len(requests) == 2 and "candidate 8 graded" in json.dumps(requests[1]) + assert len(grades) == 1 and candidate.read_text() == "8" + retained = native_session_checkpoint(env, admitted["session_id"], timeout=30)[ + "metadata" + ] + assert retained["turn_ids"][:-1] == native["turn_ids"] + with pytest.raises(ValueError): + admit_native_checkpoint_session( + env, unit, digest, config, execution=execution, timeout=30 + ) + vision = tmp_path / "vision.json" + vision.write_text( + json.dumps( + { + "schema_version": "goal_vision_replan_contract_v0", + "state": "vision_patch_proposed", + "vision_patch": { + "acceptance_summary": "Complete research continuation qualification." + }, + "path_delta": { + "outcome": "continue", + "prior_assumption": "Joint source point can be resumed.", + "observed_reality": "Isolated receiver used retained context and code, and graded a new candidate.", + "retained": ["Complete qualification"], + "evidence_refs": ["evidence:joint-receiver"], + }, + } + ) + ) + cli( + "refresh-state", + *binding, + "--classification", + "joint_receiver_verified", + "--delivery-batch-scale", + "implementation", + "--delivery-outcome", + "outcome_progress", + "--delivery-boundary", + "in_flight_continuation", + "--agent-vision-json", + str(vision), + "--no-global-sync", + "--suppress-external-sinks", + ) + first = cli( + "quota", "spend-slot", *binding, "--source", "heartbeat", "--execute" + ) + second = cli( + "quota", "spend-slot", *binding, "--source", "heartbeat", "--execute" + ) + assert first["appended"] and not second["appended"] + assert ( + read_heartbeat_settlement( + runtime, + goal_id=goal, + agent_id=agent, + todo_id=todo, + turn_instance_id=turn, + registry_path=registry, + ).progress["state"] + == "settled" + ) + effect_runtime_result( + "runtime.checkpoint_artifacts", + {"action": "verify", "root": str(unit), "expected_sha256": digest}, + ) + assert ( + hashlib.sha256( + (unit / "components/research_context").read_bytes() + ).hexdigest() + == metadata["research_context"]["sha256"] + ) + finally: + server.shutdown() + thread.join() + server.server_close() diff --git a/benchmark/tests/test_native_checkpoint_handoff.py b/benchmark/tests/test_native_checkpoint_handoff.py index d36a146a66..fe912f27b2 100644 --- a/benchmark/tests/test_native_checkpoint_handoff.py +++ b/benchmark/tests/test_native_checkpoint_handoff.py @@ -492,7 +492,7 @@ def response(payload, index, message): saved_home = home.with_name(home.name + "-retained") config = tmp_path / "captured-config.toml" config.write_bytes((home / "config.toml").read_bytes()) - with pytest.raises(ValueError, match="joint controller"): + with pytest.raises(ValueError, match="distinct destination"): admit_native_checkpoint_session({**env, "CODEX_HOME": str(home)}, point, event["sha256"], config, execution=Execution(context="resume"), timeout=20) home.rename(saved_home) From 67f081bb5b4e367be17b36ad9f7aeaed4177a4f5 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Wed, 14 Oct 2026 02:39:18 +0800 Subject: [PATCH 62/69] docs(benchmark): record isolated joint continuation qualification Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/README.md | 20 +++++++--- .../hierarchical-agent-stride-control-v0.md | 38 +++++++++++-------- ...rarchical-agent-stride-control-v0.zh-CN.md | 13 ++++--- 3 files changed, 45 insertions(+), 26 deletions(-) diff --git a/benchmark/README.md b/benchmark/README.md index 6e790ebd6f..a80e8b8703 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -245,8 +245,11 @@ Turn as a substitute; completed-cursor checks stay unchanged. Lost fork replies reuse the existing one-attempt recovery path. Full capture uses the external controller path below. Native-only history plus -declared files remains an inert subset. Interrupted sessions still cannot enter -destination Goal admission until joint controller continuation is qualified. +declared files remains an inert subset. An interrupted cursor can now enter the +existing explicit session-binding admission for a distinct registered destination +Goal/Agent. Its pending source guard remains historical input; the receiver must +obtain a new work admission and lease. Binding alone grants no execution or +evaluation opportunity. Remove the checkpoint request to disable automatic handoff; a direct caller can omit the socket option to disable only the transport. @@ -602,9 +605,16 @@ A barrier that cannot be released remains an explicit controller recovery proble A real native CLI/worker plus isolated Docker/native-HTTP qualification covers the mid-wake path, complete canonical context, conserved evaluator opportunity and original source continuation. It uses a deterministic local model and synthetic -grading. The artifact remains unqualified for scientific restart: isolated joint -destination admission, useful work, original-effect recovery and subsequent grading -must still be verified together. No old lease, quota, timer or token is activated. +grading. A fresh isolated Linux receiver can bind that interrupted native prefix, +adopt captured research context on its currently leased Todo, restore native +evaluation/Host allowances and run a new admitted Turn. Qualification covers a +resumed tool consuming retained code/context, producing a synthetic candidate, +receiving its next native grade and settling the destination Turn exactly once. +Lost binding replies and native activation readback retain their original +identities; wrong-source/digest and advanced-history admission reject. No old +lease, quota, timer or token is activated. Scientific restart still requires the +chosen installed source/profile and experiment inputs to be admitted; this +mechanical fixture proves neither model judgment nor policy benefit. Omit the controller file and `controller_handoff` on a new run to disable automatic controller capture; removing the checkpoint request disables capture entirely. Existing artifacts remain inert, and the source trial is not restarted by disabling it. diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md index f2a0ebfb18..363f00217c 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md @@ -773,9 +773,9 @@ full-state mid-wake requests remain explicitly unqualified. The native participa now binds an explicit interrupted final guard, its parsed routes/Turn identity and original packet digest, rejects released output/later visible items or duplicate matching calls, and preserves the full prefix in a native fork. Completed-cursor -checks stay strict. Controller-wide writer exclusion and isolated joint research, -evaluator and budget admission remain open; participant reconstruction grants no -new work or destination authority. +checks stay strict. The integrated controller/receiver qualification below now +covers the joined source and isolated destination path; participant reconstruction +by itself still grants no work or destination authority. The Host profile must also declare path portability. A native index can retain absolute guest paths: qualification must either preserve those paths inside a @@ -814,7 +814,8 @@ Replacement, changed configuration/profile and native automatic Goal activation are refused. Real native CLI/resume qualification uses a local static provider to carry retained conversation into a new Turn, verify materialized code/negative evidence and settle the original LoopX guard/writeback/spend. The quiesced fixture -swaps fixed guest paths; full environment isolation remains unqualified. This proves +swaps fixed guest paths; the newer joint receiver fixture below uses a fresh Linux +guest instead. This earlier test proves the conversation participant's adoption and synthetic continuation, not full research/effect adoption, remaining whole-trial budget, installed frontend recovery or scientific benefit. An explicit checkpoint-aware native judge also exports released public result @@ -895,18 +896,23 @@ Model inference is excluded. Full Host provenance/clock alignment and joint session/research/effect recovery still need their profile's qualification; this participant transport cannot certify them. -The automatic source-capture boundary now composes the real worker's held -fresh-replan decision with its native cursor, canonical research inventory and -controller-owned Docker pause/archive barrier. The controller verifies the exact -pending request and prepared source witness before sealing native incumbent, -opportunity and remaining phase budget. A changed or expired boundary is missed, -not recaptured after the source resumes. Synthetic native CLI/Docker/HTTP evidence -covers this capture and original source continuation; it does not qualify a joint -destination activation. The next acceptance remains one isolated destination: -consume the captured context/code, obtain fresh Goal/session/evaluator authority, -perform useful work, settle its original effect and grade without refilling budget. -Preserve the existing lost-reply and wrong-source negatives in that composed path. -No new research cells are admitted by the capture milestone alone. +The automatic source-capture boundary composes the held fresh-replan decision, +native cursor and canonical research inventory under the controller Docker writer +barrier. It verifies the exact pending request and prepared source witness before +sealing incumbent, opportunity and remaining phase budget. A changed or expired +boundary is missed, never recaptured after source continuation. + +The joint receiver qualification now consumes an actual controller-captured +interrupted point in a fresh Linux guest with no mounted source authority. +Explicit native binding and leased Todo-context adoption retain the pending call +as history; a fresh destination guard owns subsequent work. The same nested +native capsule supplies opportunities and the Host phase deadline. A resumed +native tool reads retained context/code, creates a synthetic candidate and receives +its next grade under that deadline, followed by exactly-once destination settlement. +Lost binding acknowledgment and activation readback reuse their committed identities; +wrong source/digest and advanced history reject. This closes the joined mechanical +continuation gap for the tested profile, not installed campaign admission, packaged +recovery UX, autonomous research quality or a stride-effect claim. Committed native session admission now retains the captured artifact digest in the original atomic session binding. Exact controller retries after a lost binding diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md index 51dc05b95c..8b6dc8d6a1 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md @@ -642,7 +642,7 @@ Host 拥有的原生进程组，要求主进程正常响应中断退出并清理 均经过该实际入口；中途的完整状态请求仍明确不具备资格。原生参与者已显式绑定 interrupted 末尾 guard、解析后的路由/Turn 身份及原包摘要，拒绝已释放输出、 后续可见条目或重复匹配调用，并通过原生 fork 保留完整前缀。completed cursor -检查不放宽。controller 全部 writer 屏障与隔离的研究/评测/预算联合准入仍待验收； +检查不放宽。下文的集成 controller/receiver 验证已覆盖联合源捕获和隔离目标路径； 参与者重建不授予新的工作或目标执行权限。 Host profile 还须声明路径可移植性。原生索引可能保留绝对 guest 路径：资格化 @@ -744,10 +744,13 @@ phase 文件参与者还将声明的 Host deadline 与 native 捕获时间绑定 研究 inventory 与 controller 的 Docker pause/archive 屏障组合起来。controller 先验证仍有效的请求及准备时的 source witness，再封存 incumbent、评测机会和 剩余 phase 预算。变化或过期的边界记为未捕获，不能在源恢复后补作原点。 -真实 native CLI/Docker/HTTP 的合成验证覆盖联合捕获及原源接续；仍不代表目标 -联合激活。下一验收是独立目标环境消费保留上下文和代码、取得新权限、开展有用 -工作、结算原 effect 并完成后续评测，同时不补预算并保留丢回复/错误源负例。 -仅捕获里程碑不准入新的研究组。 +真实 native CLI/Docker/HTTP 的合成验证已从联合捕获接到全新 Linux guest： +目标绑定原生历史，在当前已租用 Todo 上采用保留上下文，恢复同一 capsule 的 +评测机会与 Host phase 预算。源 pending call 只作历史；目标重新取得 guard 后， +原生工具读取保留代码/上下文、产生合成候选、取得下一次评分，并对目标原 Turn +单次结算。binding 回复丢失、激活 readback 复用原身份，错误源/digest 和已推进 +历史拒绝。此项关闭测试 profile 的机械联合接续缺口；安装版本与实验输入准入、 +packaged 恢复体验、模型判断及 stride 收益仍需各自验证。 inert verification unit 不授予执行权。目标侧使用新的实验/Host 身份，通过既有 From 153dd85fb28b69215a1547bf8c4c2b47b368580b Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Wed, 14 Oct 2026 05:45:06 +0800 Subject: [PATCH 63/69] fix(checkpoint): capture fresh replan before work selection Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- .../tests/test_checkpoint_joint_receiver.py | 6 ++--- loopx/control_plane/heartbeat/builder.py | 7 +----- loopx/control_plane/heartbeat/host.py | 7 ++++++ .../runtime/checkpoint_artifacts.ts | 21 +++++++++++++--- .../test_research_checkpoint_capture.py | 17 ++++++++++++- .../checkpoint_artifacts.test.ts | 25 +++++++++++++++++++ 6 files changed, 69 insertions(+), 14 deletions(-) diff --git a/benchmark/tests/test_checkpoint_joint_receiver.py b/benchmark/tests/test_checkpoint_joint_receiver.py index 6c3d65448d..83e87dfd9d 100644 --- a/benchmark/tests/test_checkpoint_joint_receiver.py +++ b/benchmark/tests/test_checkpoint_joint_receiver.py @@ -249,12 +249,12 @@ def lose_binding_reply(self, *args): script = workspace / "continue.py" script.write_text("""import json,time,requests from pathlib import Path -b=json.loads(Path('judge.json').read_text()); url,token=b['url'],b['token'] -h=requests.get(url+'/api/v1/history',params={'token':token},timeout=5).json() +b=json.loads(Path('judge.json').read_text()); url,submission_token=b['url'],b['token'] +h=requests.get(url+'/api/v1/history',params={'token':submission_token},timeout=5).json() assert h['best_score']==7 and h['agent_submissions']==1 assert Path('candidate.txt').read_text()=='original candidate' Path('candidate.txt').write_text('8') -r=requests.post(url+'/api/v1/submit',data={'token':token},files={'archive':('candidate.tar.gz',b'8')},timeout=5) +r=requests.post(url+'/api/v1/submit',data={'token':submission_token},files={'archive':('candidate.tar.gz',b'8')},timeout=5) r.raise_for_status(); sid=r.json()['submission_id'] for _ in range(100): result=requests.get(url+'/api/v1/result/'+sid,timeout=5).json() diff --git a/loopx/control_plane/heartbeat/builder.py b/loopx/control_plane/heartbeat/builder.py index 41de3a7e2c..9e8f35fdba 100644 --- a/loopx/control_plane/heartbeat/builder.py +++ b/loopx/control_plane/heartbeat/builder.py @@ -375,13 +375,8 @@ def build_heartbeat_prompt( thin=thin, full=full, compact=compact, brief=brief, native_goal_host=native_goal_host, turn_instance_id=normalized_turn_instance_id, + decision_handoff_socket=decision_handoff_socket, ) - if decision_handoff_socket is not None: - from ..turn_driver.decision_capture import validate_handoff_socket - - if decision_output_root is None: - raise ValueError("decision handoff requires an explicit decision capture root") - validate_handoff_socket(decision_handoff_socket) explicit_agent_scopes = normalize_agent_scopes(agent_scopes) if explicit_agent_scopes: normalized_agent_scopes = explicit_agent_scopes diff --git a/loopx/control_plane/heartbeat/host.py b/loopx/control_plane/heartbeat/host.py index 0645b9a22e..afcf51cda3 100644 --- a/loopx/control_plane/heartbeat/host.py +++ b/loopx/control_plane/heartbeat/host.py @@ -29,15 +29,22 @@ def resolve_heartbeat_capture_root( brief: bool, native_goal_host: bool, turn_instance_id: str | None, + decision_handoff_socket: str | None = None, ) -> Path | None: """Validate capture transport for an already resolved host-owned Turn.""" if root is None: + if decision_handoff_socket is not None: + raise ValueError("decision handoff requires an explicit decision capture root") return None if not thin or full or compact or brief or native_goal_host or not turn_instance_id: raise ValueError("--decision-output-root requires thin mode and an explicit host-owned Turn ID") directory = Path(root).expanduser().absolute() if directory.is_symlink() or not directory.is_dir(): raise ValueError("--decision-output-root must be an existing directory, not a symlink") + if decision_handoff_socket is not None: + from ..turn_driver.decision_capture import validate_handoff_socket + + validate_handoff_socket(decision_handoff_socket) return directory diff --git a/loopx/control_plane/runtime/checkpoint_artifacts.ts b/loopx/control_plane/runtime/checkpoint_artifacts.ts index f7d0ec18fc..fc799a856e 100644 --- a/loopx/control_plane/runtime/checkpoint_artifacts.ts +++ b/loopx/control_plane/runtime/checkpoint_artifacts.ts @@ -181,13 +181,26 @@ export async function checkpointArtifacts(value: unknown): Promise { const packet = requireJsonObject(quota.replan_action_packet, "checkpoint replan packet"); const id = normalizeHeartbeatReplanObligationId(packet.obligation_id); const receipt = requireJsonObject(quota.heartbeat_receipt, "checkpoint heartbeat receipt"); + if (packet.schema_version !== "replan_action_packet_v0" || packet.decision !== "replan_required" || + !id || (receipt.status !== "committed" && receipt.status !== "replayed") || receipt.turn_instance_id !== turn) { + throw new Error("checkpoint replan does not match the current admitted identity"); + } + const semanticReplan = normalizeHeartbeatReplanObligationId(receipt.semantic_replan_obligation_id); + if (receipt.settlement_identity === undefined && receipt.settlement_binding_owed === true) { + // The original guard owns the semantic node before the Agent chooses + // work. An inert pre-decision capture does not invent that later work's + // causal settlement identity or grant permission to execute it. + const actor = requireJsonObject(quota.agent_identity, "checkpoint source Agent"); + requireNonEmptyString(receipt.event_id, "checkpoint committed guard event"); + if (actor.agent_id !== agent || actor.registered !== true || semanticReplan !== id) { + throw new Error("checkpoint replan does not match the current admitted identity"); + } + return {capture: true, obligation_id: id, execution_authority_granted: false}; + } const identity = requireJsonObject(receipt.settlement_identity, "checkpoint settlement identity"); const todo = normalizeHeartbeatTodoId(identity.todo_id); const boundReplan = normalizeHeartbeatReplanObligationId(identity.replan_obligation_id); - const semanticReplan = normalizeHeartbeatReplanObligationId(receipt.semantic_replan_obligation_id); - if (packet.schema_version !== "replan_action_packet_v0" || packet.decision !== "replan_required" || - !id || (receipt.status !== "committed" && receipt.status !== "replayed") || receipt.turn_instance_id !== turn || - identity.goal_id !== goal || identity.agent_id !== agent || identity.turn_instance_id !== turn || + if (identity.goal_id !== goal || identity.agent_id !== agent || identity.turn_instance_id !== turn || (todo ? boundReplan !== null || semanticReplan !== id : boundReplan !== id) || identity.effect_id !== settlementIdentity({goal_id: goal, agent_id: agent, todo_id: todo, replan_obligation_id: boundReplan, turn_instance_id: turn}).effect_id) { diff --git a/tests/control_plane/test_research_checkpoint_capture.py b/tests/control_plane/test_research_checkpoint_capture.py index 11080c2de1..e26b85a551 100644 --- a/tests/control_plane/test_research_checkpoint_capture.py +++ b/tests/control_plane/test_research_checkpoint_capture.py @@ -381,7 +381,9 @@ def test_native_binding_participant_preserves_legal_caller_id(tmp_path, monkeypa @pytest.mark.parametrize("capture_fails", [False, True]) -def test_real_pre_replan_capture_is_before_decision_and_same_node_is_not_recaptured(tmp_path, monkeypatch, capture_fails): +@pytest.mark.parametrize("selection_pending", [False, True]) +def test_real_pre_replan_capture_is_before_decision_and_same_node_is_not_recaptured( + tmp_path, monkeypatch, capture_fails, selection_pending): import os import sys @@ -407,6 +409,14 @@ def run(*args): index = runtime / "goals" / cli.GOAL_ID / "runs/index.jsonl" original_settlement = index.read_bytes() cli._configure_autonomous_replan_fixture(project, runtime, registry) + if selection_pending: + # Keep executable work, as a continuing research worker does. The + # original guard commits the semantic replan before choosing its Todo; + # no settlement identity exists for the new Turn yet. + state_path = project / f".codex/goals/{cli.GOAL_ID}/ACTIVE_GOAL_STATE.md" + state_path.write_text(state_path.read_text().replace("## Agent Todo\n\n", + "## Agent Todo\n\n- [ ] [P1] Continue the retained experiment.\n" + f" \n")) # This helper supplies synthetic repeated observations; retain the real # source settlement history instead of replacing its authoritative readback. index.write_bytes(original_settlement + index.read_bytes()) @@ -437,6 +447,11 @@ def fail_probe(method, params, **kwargs): return effect_runtime_result(method, params, **kwargs) monkeypatch.setattr("benchmark.runtime.checkpoint.effect_runtime_result", fail_probe) body = capture_before_replan(env, receipt, "original instructions") + guard = json.loads((wakes / receipt["turn_id"] / "checkpoint-start-guard.json").read_text()) + if selection_pending: + assert guard["heartbeat_receipt"]["settlement_binding_owed"] is True + assert "settlement_identity" not in guard["heartbeat_receipt"] + assert guard["interaction_contract"]["agent_channel"]["delivery_allowed"] is False if capture_fails: assert receipt["checkpoint"]["reason"] == "capture_failed", receipt solution.write_text("later_decision = 11\n") diff --git a/tests/control_plane_ts/checkpoint_artifacts.test.ts b/tests/control_plane_ts/checkpoint_artifacts.test.ts index 129440d5ab..89330e678f 100644 --- a/tests/control_plane_ts/checkpoint_artifacts.test.ts +++ b/tests/control_plane_ts/checkpoint_artifacts.test.ts @@ -207,3 +207,28 @@ test("replan capture selection requires the original admitted identity and grant {...quota.heartbeat_receipt, settlement_identity: {...identity, todo_id: "todo_conflict"}}}}), /admitted identity/); await assert.rejects(checkpointArtifacts({...request, quota: {...quota, ok: false}}), /matching current/); }); + +test("committed semantic replan can capture before work selection without inventing settlement authority", async () => { + const id = "replan-0123456789abcdef", turn = "fixture-turn"; + const quota = {ok: true, goal_id: "fixture-goal", effective_action: "autonomous_replan_required", + agent_identity: {agent_id: "fixture-agent", registered: true}, + replan_action_packet: {schema_version: "replan_action_packet_v0", decision: "replan_required", obligation_id: id}, + heartbeat_receipt: {status: "committed", turn_instance_id: turn, event_id: "fixture-event", + semantic_replan_obligation_id: id, settlement_binding_owed: true}}; + const request = {action: "select_replan", quota, goal_id: "fixture-goal", agent_id: "fixture-agent", turn_instance_id: turn}; + const before = structuredClone(quota); + assert.deepEqual(await checkpointArtifacts(request), { + capture: true, obligation_id: id, execution_authority_granted: false, + }); + assert.deepEqual(quota, before); + for (const receipt of [ + {...quota.heartbeat_receipt, status: "pending"}, + {...quota.heartbeat_receipt, settlement_binding_owed: false}, + {...quota.heartbeat_receipt, semantic_replan_obligation_id: "replan-fedcba9876543210"}, + {...quota.heartbeat_receipt, turn_instance_id: "another-turn"}, + {...quota.heartbeat_receipt, settlement_identity: null}, + ]) await assert.rejects(checkpointArtifacts({...request, quota: {...quota, heartbeat_receipt: receipt}})); + await assert.rejects(checkpointArtifacts({...request, agent_id: "another-agent"})); + await assert.rejects(checkpointArtifacts({...request, quota: {...quota, + agent_identity: {...quota.agent_identity, registered: false}}})); +}); From 75c8550df54a135f24bb4a953f64953bc344723e Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Wed, 14 Oct 2026 05:45:06 +0800 Subject: [PATCH 64/69] docs(checkpoint): explain pending-selection capture boundary Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/README.md | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/benchmark/README.md b/benchmark/README.md index a80e8b8703..762215db20 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -169,8 +169,12 @@ and evaluator cooldown/opportunity inheritance must pass before matched continua For pre-decision capture at a **fresh replan projected at the start of a heartbeat wake**, add `"trigger": "fresh_replan"` and set `destination` to the trial's `checkpoints` directory itself. The Host preloads that wake's original quota guard -once, the typed owner validates its replan/settlement binding, and the adapter +once, the typed owner validates its committed semantic replan identity, and the adapter captures from the latest successful, settled work wake before invoking the model. +Work selection may still be owed on this new guard: the exact registered Agent, +Goal, Turn and semantic obligation must match, but inert capture does not invent +a causal settlement identity or authorize delivery before selection. An already +selected guard retains its existing settlement-identity checks. The model reads the complete preloaded guard and follows its existing selection, required-read, recovery and settlement contract. Each obligation has one immutable destination; a repeated read verifies that point instead of replacing it with later @@ -703,6 +707,11 @@ runtime receipt 绑定摘要；programmatic adapter 也可指定已放入 guest 物化后仍是惰性历史。缺少来源、共享注册表、路径别名或重叠声明会拒绝，捕获 不授予新的租约、配额、效果或执行权，不能替代完整接续验收。 +Host 预检已提交 fresh replan，但尚待 Agent 选择工作时，仍可在模型决策前捕获。 +类型化 owner 核对原 Goal、已注册 Agent、Turn 和语义义务；不会伪造尚未形成的 +settlement identity，也不授予选择前的交付权限。源边界仍须来自上轮已结算的成功工作； +已选 guard 的原结算身份检查保持。 + 运行中出现的 fresh replan 可通过显式交接捕获。完整捕获需同时声明 `controller_handoff`、`native_session`、`loopx_state`、`evaluation_opportunity` 为 true，使用 `fresh_replan`，并在 host CLI 提供 `--checkpoint-controller-file`。 From 82bbb82287495aa3b2ec57d19ebc402dec780efb Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Wed, 14 Oct 2026 06:11:58 +0800 Subject: [PATCH 65/69] test(checkpoint): qualify completed replan capture and pending cooldown Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- .../tests/test_checkpoint_joint_receiver.py | 49 ++++++-- .../tests/test_native_checkpoint_handoff.py | 116 ++++++++++++++++++ 2 files changed, 153 insertions(+), 12 deletions(-) diff --git a/benchmark/tests/test_checkpoint_joint_receiver.py b/benchmark/tests/test_checkpoint_joint_receiver.py index 83e87dfd9d..ac2db35e96 100644 --- a/benchmark/tests/test_checkpoint_joint_receiver.py +++ b/benchmark/tests/test_checkpoint_joint_receiver.py @@ -25,7 +25,7 @@ not os.environ.get("LOOPX_JOINT_RECEIVER_POINT"), reason="Requires a real controller point in a fresh isolated guest", ) -def test_captured_interrupted_point_continues_with_fresh_authority( +def test_captured_point_continues_with_fresh_authority( tmp_path, monkeypatch, capsys, request ): from benchmark.edgebench.checkpoint_controller import restore @@ -48,7 +48,12 @@ def test_captured_interrupted_point_continues_with_fresh_authority( manifest = json.loads((unit / "checkpoint.json").read_bytes()) digest, metadata = manifest["sha256"], manifest["point_metadata"] native, snapshot = metadata["native_session"], metadata["worker_file_snapshot"] - assert snapshot["prepared_source_verified"] and native["interrupted_guard"] + assert snapshot["prepared_source_verified"] + assert ("interrupted_guard" in native) != ("completed_turn_id" in native) + if "completed_turn_id" in native: + assert native["completed_turn_id"] == native["turn_ids"][-1] + else: + assert native["interrupted_guard"]["decision_sha256"] == metadata["decision_sha256"] config = Path(os.environ["LOOPX_JOINT_RECEIVER_CONFIG"]) workspace, home = Path(native["guest_workspace"]), Path(native["guest_home"]) assert not workspace.exists() and not home.exists(), ( @@ -215,6 +220,15 @@ def lose_binding_reply(self, *args): assert guard["interaction_contract"]["agent_channel"]["delivery_allowed"] _, branch, _, url, controller_files, grades = request.getfixturevalue("controllers") capsule = unit / "components" / snapshot["native_capsule_component_id"] + opportunity = json.loads((capsule / "checkpoint.json").read_bytes())["point_metadata"][ + "native_reference" + ]["opportunity"] + remaining_cooldown = opportunity["remaining_cooldown_seconds"] + # Explicit fixture boundaries: either already eligible at capture, or a + # cooldown longer than this entire 60-second native work window. A short + # partial cooldown would make a timing-dependent expectation ambiguous. + assert remaining_cooldown == 0 or remaining_cooldown > 60 + cooldown_pending = remaining_cooldown > 60 phase = tmp_path / "phase-deadline" fresh = restore( controller_files[1], @@ -255,14 +269,20 @@ def lose_binding_reply(self, *args): assert Path('candidate.txt').read_text()=='original candidate' Path('candidate.txt').write_text('8') r=requests.post(url+'/api/v1/submit',data={'token':submission_token},files={'archive':('candidate.tar.gz',b'8')},timeout=5) -r.raise_for_status(); sid=r.json()['submission_id'] -for _ in range(100): - result=requests.get(url+'/api/v1/result/'+sid,timeout=5).json() - if result['status']=='completed':break - time.sleep(.01) -assert result['status']=='completed' -print('Retained candidate and public best 7 used; candidate 8 graded.') -""") +if EXPECTED_COOLDOWN_PENDING: + assert r.status_code==429 + h=requests.get(url+'/api/v1/history',params={'token':submission_token},timeout=5).json() + assert h['agent_submissions']==1 and h['best_score']==7 + print('Candidate 8 validated locally; captured cooldown still blocks submission.') +else: + r.raise_for_status(); sid=r.json()['submission_id'] + for _ in range(100): + result=requests.get(url+'/api/v1/result/'+sid,timeout=5).json() + if result['status']=='completed':break + time.sleep(.01) + assert result['status']=='completed' + print('Retained candidate and public best 7 used; candidate 8 graded.') +""".replace("EXPECTED_COOLDOWN_PENDING", repr(cooldown_pending))) requests = [] class Provider(BaseHTTPRequestHandler): @@ -360,8 +380,13 @@ def do_POST(self): assert result.returncode == 0, result.stderr (logs / "stdout.jsonl").write_text(result.stdout) wake.observe(logs / "stdout.jsonl") - assert len(requests) == 2 and "candidate 8 graded" in json.dumps(requests[1]) - assert len(grades) == 1 and candidate.read_text() == "8" + assert len(requests) == 2 + expected = "cooldown still blocks submission" if cooldown_pending else "candidate 8 graded" + assert expected in json.dumps(requests[1]), json.dumps([ + item for item in requests[1].get("input", []) + if item.get("type") == "function_call_output" + ]) + assert len(grades) == (0 if cooldown_pending else 1) and candidate.read_text() == "8" retained = native_session_checkpoint(env, admitted["session_id"], timeout=30)[ "metadata" ] diff --git a/benchmark/tests/test_native_checkpoint_handoff.py b/benchmark/tests/test_native_checkpoint_handoff.py index fe912f27b2..089d7ac9bb 100644 --- a/benchmark/tests/test_native_checkpoint_handoff.py +++ b/benchmark/tests/test_native_checkpoint_handoff.py @@ -343,6 +343,122 @@ def test_checkpoint_interrupt_timeout_reaps_without_qualifying_capture(tmp_path) assert process.poll() is not None +@pytest.mark.skipif(not os.environ.get("LOOPX_NATIVE_CHECKPOINT_BIN") or + os.environ.get("LOOPX_CHECKPOINT_EXTERNAL_CONTROLLER") != "1", + reason="Requires the real native CLI and explicitly attached checkpoint supervisor") +def test_actual_worker_captures_pending_replan_before_native_model(tmp_path): + """Completed native source + canonical state + controller, before selection. + + Synthetic observations trigger review; the real guard, worker, native + history, SQLite and Host barrier must compose without inventing authority. + """ + import sys + import tomllib + from pathlib import Path + from benchmark.runtime.worker import run_once + from tests.control_plane import test_quota_settlement_cli as cli + from tests.control_plane.canonical_authority_fixture import initialize_canonical_authority + from loopx.control_plane.coordination.runtime_shadow import build_todo_runtime_shadow_projection + + project, runtime, registry = cli._write_fixture(tmp_path / "authority") + logs = Path("/logs/agent/wakes") + candidate = project / "candidate.txt" + candidate.write_text("original candidate") + executable = tmp_path / "loopx-cli" + executable.write_text("#!/bin/sh\nexec " + shlex.quote(sys.executable) + ' -m loopx.cli "$@"\n') + executable.chmod(0o700) + skills = tmp_path / "skills" + skills.mkdir() + requests, guards = [], [] + + def response(payload, index, message): + requests.append(payload) + if index == 3: + # These files can exist only after the actual Host capture reply; + # no model call is permitted while capture is still pending. + replies = list((logs.parent / "checkpoint-controller").glob("*.json")) + reply = next(json.loads(path.read_text()) for path in replies + if path.name != "ready.json") + assert reply["ok"] and len(reply["capture"]["sha256"]) == 64 + paths = list(logs.glob("*/checkpoint-start-guard.json")) + assert len(paths) == 1 + guard = json.loads(paths[0].read_text()) + guards.append(guard) + assert guard["heartbeat_receipt"]["settlement_binding_owed"] + assert "settlement_identity" not in guard["heartbeat_receipt"] + assert not guard["interaction_contract"]["agent_channel"]["delivery_allowed"] + # Read the original packet only: selection and research work are + # still owed. No guard invocation or historical effect is replayed. + return {"type": "function_call", "id": "fc_packet", "call_id": "call_packet", + "name": "exec_command", "arguments": json.dumps({ + "cmd": "cat " + shlex.quote(str(paths[0])), "max_output_tokens": 30000})} + message["content"][0]["text"] = "Completed source research." if index <= 2 else "Original packet read." + return message + + def run(*args): + code, result = cli._run_cli(registry, runtime, *args, cwd=project) + assert code == 0, result + return result + + with real_native_source(tmp_path / "provider", os.environ["LOOPX_NATIVE_CHECKPOINT_BIN"], + project, close_source=True, response_builder=response, + sandbox="danger-full-access") as (env, _, _, calls): + base = tomllib.loads((Path(env["CODEX_HOME"]) / "config.toml").read_text()) + env.update(LOOPX_CLI=str(executable), LOOPX_REGISTRY=str(registry), + LOOPX_RUNTIME_ROOT=str(runtime), LOOPX_GOAL_ID=cli.GOAL_ID, LOOPX_AGENT_ID=cli.AGENT_ID, + LOOPX_WAKE_LOG_DIR=str(logs), LOOPX_CODEX_HOME=str(tmp_path / "worker-home"), + LOOPX_ITERATION_CONTEXT="resume", LOOPX_SHARED_SKILLS=str(skills), MODEL_NAME="fixture", + REASONING_EFFORT="high", OPENAI_BASE_URL=base["model_providers"]["fixture"]["base_url"], + OPENAI_API_KEY="synthetic-local-only", LOOPX_CODEX_TURN_TIMEOUT_SEC="60") + prior = run_once(env) + assert prior["ok"] and len(calls) == 2 + binding = ("--goal-id", cli.GOAL_ID, "--agent-id", cli.AGENT_ID, + "--todo-id", cli.TODO_ID, "--turn-instance-id", prior["turn_id"]) + run("quota", "should-run", "--runtime-profile", "generic_cli", *binding) + vision = tmp_path / "vision.json" + vision.write_text(json.dumps({"schema_version": "goal_vision_replan_contract_v0", + "state": "vision_patch_proposed", "vision_patch": { + "acceptance_summary": "Continue synthetic research; complete checkpoint qualification remains."}, + "path_delta": {"outcome": "continue", "prior_assumption": "Source fixture is available.", + "observed_reality": "Actual native source and CLI boundary checked.", + "retained": ["Complete checkpoint qualification"], + "evidence_refs": ["evidence:completed-source-fixture"]}})) + run("refresh-state", *binding, "--classification", "source_fixture_verified", + "--delivery-batch-scale", "implementation", "--delivery-outcome", "outcome_progress", + "--delivery-boundary", "in_flight_continuation", "--agent-vision-json", str(vision)) + run("quota", "spend-slot", *binding, "--source", "heartbeat", "--execute") + index_path = runtime / "goals" / cli.GOAL_ID / "runs/index.jsonl" + original_runs = index_path.read_bytes() + cli._configure_autonomous_replan_fixture(project, runtime, registry) + state_path = project / f".codex/goals/{cli.GOAL_ID}/ACTIVE_GOAL_STATE.md" + state_path.write_text(state_path.read_text().replace("## Agent Todo\n\n", + "## Agent Todo\n\n- [ ] [P1] Continue the retained experiment.\n" + f" \n")) + # Preserve the real source settlement; synthetic trigger rows are + # fixture inputs, never claims of actual research progress. + index_path.write_bytes(original_runs + index_path.read_bytes()) + listed = run("todo", "list", "--goal-id", cli.GOAL_ID) + initialize_canonical_authority(runtime, cli.GOAL_ID, + build_todo_runtime_shadow_projection(goal_id=cli.GOAL_ID, todos=listed["todos"], + handoff_mode="hard_lease", leases=[]), + state_path=state_path, provider="sqlite") + request = tmp_path / "request.json" + request.write_text(json.dumps({"destination": str(logs.parent / "checkpoints"), + "components": [{"id": "code", "source_path": str(candidate)}], "point_metadata": {}, + "trigger": "fresh_replan", "native_session": True, "loopx_state": True, + "evaluation_opportunity": True, "controller_handoff": True})) + result = run_once({**env, "LOOPX_CHECKPOINT_REQUEST": str(request)}) + assert result["ok"] and result["checkpoint"]["captured"], result + assert result["checkpoint"]["phase"] == "before_replan" + assert result["checkpoint"]["source_turn_id"] == prior["turn_id"] + assert result["session"]["session_id"] == prior["session"]["session_id"] + assert not result.get("checkpoint_handoffs") and len(calls) == 4 + assert cli._heartbeat_receipt_count(runtime, result["turn_id"]) == 1 + assert "settlement_binding_owed" in json.dumps(requests[3]) + assert candidate.read_text() == "original candidate" and len(guards) == 1 + (tmp_path / "controller-result.json").write_text(json.dumps(result["checkpoint"])) + + @pytest.mark.skipif(not os.environ.get("LOOPX_NATIVE_CHECKPOINT_BIN") or os.name != "posix", reason="Requires the real native CLI and private Unix transport") @pytest.mark.parametrize("capture", ["files", "native", "full", "off", "controller"]) From 6458777a32bafff2bac02b24b77103ddb53f2bdf Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Wed, 14 Oct 2026 06:11:58 +0800 Subject: [PATCH 66/69] docs(stride): record completed-cursor continuation qualification Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- .../rfcs/hierarchical-agent-stride-control-v0.md | 10 ++++++++++ .../rfcs/hierarchical-agent-stride-control-v0.zh-CN.md | 7 +++++++ 2 files changed, 17 insertions(+) diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md index c93c1b2a62..b43bbc5305 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md @@ -921,6 +921,16 @@ wrong source/digest and advanced history reject. This closes the joined mechanic continuation gap for the tested profile, not installed campaign admission, packaged recovery UX, autonomous research quality or a stride-effect claim. +The completed-cursor path is also exercised through the real worker, native Host, +SQLite inventory and Docker controller before the model sees a fresh replan. +Its committed guard may still owe work selection: inert capture preserves that +prohibition instead of fabricating a causal settlement binding. An isolated +receiver resumes the retained code/context under fresh authority. When the sealed +cooldown outlasts its work window, local candidate work succeeds while native +submission returns 429 without consuming a round or invoking a grader. The +separate eligible-cursor fixture receives its next grade. These two lawful states +must remain distinct; restoring a point does not make its next submission eligible. + Committed native session admission now retains the captured artifact digest in the original atomic session binding. Exact controller retries after a lost binding reply verify that receipt, current Goal/Agent/profile and unchanged complete diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md index f7550d15b1..3e38b38634 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md @@ -758,6 +758,13 @@ phase 文件参与者还将声明的 Host deadline 与 native 捕获时间绑定 历史拒绝。此项关闭测试 profile 的机械联合接续缺口；安装版本与实验输入准入、 packaged 恢复体验、模型判断及 stride 收益仍需各自验证。 +completed cursor 路径也经真实 worker、native Host、SQLite inventory 与 Docker +controller 验证，捕获发生在模型看见 fresh replan 之前。已提交 guard 尚待选择 +工作时，惰性捕获保留选择前交付禁令，不伪造因果 settlement binding。隔离目标 +在新权限下接续保留代码/上下文；若封存冷却长于本次工作窗口，本地候选工作可 +推进，原生提交仍返回 429，不消耗轮次也不调用 grader。另一已 eligible 的 cursor +用例取得下一次评分。这两种合法状态须保持区分；恢复研究点不让下一次提交立即 eligible。 + inert verification unit 不授予执行权。目标侧使用新的实验/Host 身份，通过既有 Goal/session/state admission 进入。历史租约、quota、timer、pending effect 和 From 71f6ca2a6e12a4a4c3baba7e62ae1dd03b868bbf Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Wed, 14 Oct 2026 12:21:31 +0800 Subject: [PATCH 67/69] fix(benchmark): constrain interrupted native capture to qualified hosts Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/runtime/checkpoint_wake.py | 20 +- .../tests/test_checkpoint_joint_receiver.py | 12 +- .../tests/test_native_checkpoint_handoff.py | 224 +++++++++++++----- 3 files changed, 196 insertions(+), 60 deletions(-) diff --git a/benchmark/runtime/checkpoint_wake.py b/benchmark/runtime/checkpoint_wake.py index 74bf252a4f..3d3a68c164 100644 --- a/benchmark/runtime/checkpoint_wake.py +++ b/benchmark/runtime/checkpoint_wake.py @@ -9,6 +9,7 @@ import select import socket import subprocess +import sys import tempfile import time from contextlib import ExitStack @@ -61,9 +62,22 @@ def __exit__(self, *args): return self.stack.__exit__(*args) def _first_replan(self, intent): - return intent["capture"] and _mark_replan_attempt( - self.root, intent["obligation_id"], self.receipt["turn_id"], - hashlib.sha256(self.raw).hexdigest()) + if not intent["capture"] or not _mark_replan_attempt( + self.root, intent["obligation_id"], self.receipt["turn_id"], + hashlib.sha256(self.raw).hexdigest()): + return False + if self.request.get("native_session") and sys.platform == "darwin": + # The current Darwin CLI can persist only the user item after an + # interrupted tool, losing the cursor needed for a verified fork. + # Refuse before signalling it; neither a delay nor a relaxed history + # predicate can recover that point. Completed-wake capture is separate. + self.receipt.setdefault("checkpoint_handoffs", []).append({ + "obligation_id": intent["obligation_id"], "captured": False, + "research_point_qualified": False, "error_kind": "ValueError", + "error": "Interrupted native checkpoint capture is not qualified on Darwin; original tool continues.", + }) + return False + return True def _capture(self, held, deadline): """Capture declared files/native history; never downgrade research/effects.""" diff --git a/benchmark/tests/test_checkpoint_joint_receiver.py b/benchmark/tests/test_checkpoint_joint_receiver.py index ac2db35e96..7f6c5926bc 100644 --- a/benchmark/tests/test_checkpoint_joint_receiver.py +++ b/benchmark/tests/test_checkpoint_joint_receiver.py @@ -63,6 +63,7 @@ def test_captured_point_continues_with_fresh_authority( monkeypatch.chdir(workspace) candidate = workspace / "candidate.txt" candidate.write_bytes((unit / "components/code").read_bytes()) + captured_candidate = candidate.read_text() registry, runtime = tmp_path / "registry.json", tmp_path / "runtime" goal, agent, turn = "joint-receiver", "joint-researcher", "joint-work" prefix = [ @@ -266,7 +267,7 @@ def lose_binding_reply(self, *args): b=json.loads(Path('judge.json').read_text()); url,submission_token=b['url'],b['token'] h=requests.get(url+'/api/v1/history',params={'token':submission_token},timeout=5).json() assert h['best_score']==7 and h['agent_submissions']==1 -assert Path('candidate.txt').read_text()=='original candidate' +assert Path('candidate.txt').read_text()==CAPTURED_CANDIDATE Path('candidate.txt').write_text('8') r=requests.post(url+'/api/v1/submit',data={'token':submission_token},files={'archive':('candidate.tar.gz',b'8')},timeout=5) if EXPECTED_COOLDOWN_PENDING: @@ -282,7 +283,8 @@ def lose_binding_reply(self, *args): time.sleep(.01) assert result['status']=='completed' print('Retained candidate and public best 7 used; candidate 8 graded.') -""".replace("EXPECTED_COOLDOWN_PENDING", repr(cooldown_pending))) +""".replace("EXPECTED_COOLDOWN_PENDING", repr(cooldown_pending)) + .replace("CAPTURED_CANDIDATE", repr(captured_candidate))) requests = [] class Provider(BaseHTTPRequestHandler): @@ -407,7 +409,11 @@ def do_POST(self): "path_delta": { "outcome": "continue", "prior_assumption": "Joint source point can be resumed.", - "observed_reality": "Isolated receiver used retained context and code, and graded a new candidate.", + "observed_reality": ( + "Receiver used retained context and code; local candidate validated, native cooldown preserved." + if cooldown_pending else + "Receiver used retained context and code, and graded a new candidate." + ), "retained": ["Complete qualification"], "evidence_refs": ["evidence:joint-receiver"], }, diff --git a/benchmark/tests/test_native_checkpoint_handoff.py b/benchmark/tests/test_native_checkpoint_handoff.py index 089d7ac9bb..7118cc7e16 100644 --- a/benchmark/tests/test_native_checkpoint_handoff.py +++ b/benchmark/tests/test_native_checkpoint_handoff.py @@ -7,6 +7,7 @@ import os import shlex import subprocess +import sys import time import pytest @@ -343,14 +344,34 @@ def test_checkpoint_interrupt_timeout_reaps_without_qualifying_capture(tmp_path) assert process.poll() is not None +@pytest.mark.parametrize("platform,native,interrupt", [ + ("darwin", True, False), ("linux", True, True), ("darwin", False, True), +]) +def test_midwake_native_profile_refusal_precedes_interruption(tmp_path, monkeypatch, platform, native, interrupt): + from benchmark.runtime.checkpoint_wake import CheckpointWake + + wake = CheckpointWake({"LOOPX_WAKE_LOG_DIR": str(tmp_path / "wakes")}, {"turn_id": "fixture-turn"}) + wake.request, wake.raw = {"native_session": native}, b"fixture declaration" + monkeypatch.setattr(sys, "platform", platform) + intent = {"capture": True, "obligation_id": "replan-0123456789abcdef"} + assert wake._first_replan(intent) is interrupt + assert wake._first_replan(intent) is False # Same point never becomes a later attempt. + events = wake.receipt.get("checkpoint_handoffs", []) + assert len(events) == (0 if interrupt else 1) + if events: + assert not events[0]["captured"] and not events[0]["research_point_qualified"] + + @pytest.mark.skipif(not os.environ.get("LOOPX_NATIVE_CHECKPOINT_BIN") or os.environ.get("LOOPX_CHECKPOINT_EXTERNAL_CONTROLLER") != "1", reason="Requires the real native CLI and explicitly attached checkpoint supervisor") -def test_actual_worker_captures_pending_replan_before_native_model(tmp_path): - """Completed native source + canonical state + controller, before selection. +@pytest.mark.parametrize("explore_mode", ["off", "planning"]) +def test_actual_worker_captures_pending_replan_before_native_model(tmp_path, explore_mode): + """Six real settled work wakes -> pre-decision capture -> stride -> work. - Synthetic observations trigger review; the real guard, worker, native - history, SQLite and Host barrier must compose without inventing authority. + Inference is deterministic; native history/tools, canonical SQLite, quota, + periodic cadence and the Host writer barrier use their production paths. + No synthetic run-history rows are inserted to manufacture a review. """ import sys import tomllib @@ -361,44 +382,107 @@ def test_actual_worker_captures_pending_replan_before_native_model(tmp_path): from loopx.control_plane.coordination.runtime_shadow import build_todo_runtime_shadow_projection project, runtime, registry = cli._write_fixture(tmp_path / "authority") + document = json.loads(registry.read_text()) + document["goals"][0]["quota"]["allowed_slots"] = 18 + registry.write_text(json.dumps(document)) + state_path = project / document["goals"][0]["state_file"] logs = Path("/logs/agent/wakes") candidate = project / "candidate.txt" - candidate.write_text("original candidate") + candidate.write_text("0") executable = tmp_path / "loopx-cli" executable.write_text("#!/bin/sh\nexec " + shlex.quote(sys.executable) + ' -m loopx.cli "$@"\n') executable.chmod(0o700) skills = tmp_path / "skills" skills.mkdir() - requests, guards = [], [] + requests, guards, work_turns = [], [], [] + phase = {"kind": "initial", "values": [], "tool_sent": False} + + def run(*args): + code, result = cli._run_cli(registry, runtime, *args, cwd=project) + assert code == 0, json.dumps(result, indent=2) + return result + + listed = run("todo", "list", "--goal-id", cli.GOAL_ID) + initialize_canonical_authority(runtime, cli.GOAL_ID, + build_todo_runtime_shadow_projection(goal_id=cli.GOAL_ID, todos=listed["todos"], + handoff_mode="hard_lease", leases=[]), + state_path=state_path, provider="sqlite") + run("task-lease", "acquire", "--goal-id", cli.GOAL_ID, "--todo-id", cli.TODO_ID, + "--owner", cli.AGENT_ID, "--idempotency-key", "source-work") + configured = run("configure-goal", "--goal-id", cli.GOAL_ID, + "--execution-replan-after-turns", "6", "--execution-stride-strategy", "evidence_conditioned_v1", + "--explore-mode", explore_mode, "--execute") + assert configured["after"]["execution_profile"]["replan_after_effective_turns"] == 6 def response(payload, index, message): requests.append(payload) - if index == 3: - # These files can exist only after the actual Host capture reply; - # no model call is permitted while capture is still pending. + if phase["kind"] == "work" and not phase["tool_sent"]: + phase["tool_sent"] = True + prefix = [sys.executable, "-m", "loopx.entrypoint", "--registry", str(registry), + "--runtime-root", str(runtime), "--format", "json", "quota", "should-run", + "--goal-id", cli.GOAL_ID, "--agent-id", cli.AGENT_ID, "--todo-id", cli.TODO_ID, + "--runtime-profile", "generic_cli", "--available-capability", "shell"] + # The native tool selects its current Turn before changing code. + # Native work and validation precede the real CLI settlement below. + code = ("import os,json,subprocess; from pathlib import Path; " + f"p=subprocess.run({prefix!r}+['--turn-instance-id',os.environ['LOOPX_TURN']]," + "capture_output=True,text=True,check=True); g=json.loads(p.stdout); " + "assert not g.get('autonomous_replan_obligation'); " + "assert g['interaction_contract']['agent_channel']['delivery_allowed']; " + "Path('work-guard.json').write_text(p.stdout); " + f"values={phase['values']!r}; " + "[(Path('candidate.txt').write_text(str(v))," + "Path('evidence-'+str(v)+'.json').write_text(json.dumps({'value':v,'synthetic':True}))) for v in values]; " + "assert Path('candidate.txt').read_text()==str(values[-1]); print('CANDIDATE_VERIFIED')") + return {"type": "function_call", "id": f"fc_work_{index}", "call_id": f"call_work_{index}", + "name": "exec_command", "arguments": json.dumps({ + "cmd": shlex.quote(sys.executable) + " -c " + shlex.quote(code), + "yield_time_ms": 10000, "max_output_tokens": 100})} + if phase["kind"] == "review" and not phase["tool_sent"]: + phase["tool_sent"] = True replies = list((logs.parent / "checkpoint-controller").glob("*.json")) - reply = next(json.loads(path.read_text()) for path in replies - if path.name != "ready.json") + reply = next(json.loads(path.read_text()) for path in replies if path.name != "ready.json") assert reply["ok"] and len(reply["capture"]["sha256"]) == 64 paths = list(logs.glob("*/checkpoint-start-guard.json")) assert len(paths) == 1 guard = json.loads(paths[0].read_text()) guards.append(guard) + trigger = next(t for t in guard["autonomous_replan_obligation"]["triggers"] + if t["kind"] == "periodic_review_due") + assert trigger["run_count"] == trigger["threshold"] == 6 + control = guard["replan_action_packet"]["stride_control"] + assert set(control["actions"]) == {"grow", "switch", "verify"} + basis = control["explore_basis"] + assert bool(basis and basis["harness_enabled"]) == (explore_mode == "planning") assert guard["heartbeat_receipt"]["settlement_binding_owed"] assert "settlement_identity" not in guard["heartbeat_receipt"] assert not guard["interaction_contract"]["agent_channel"]["delivery_allowed"] - # Read the original packet only: selection and research work are - # still owed. No guard invocation or historical effect is replayed. + # The sealed point precedes the policy decision and seventh work. + assert candidate.read_text() == "6" return {"type": "function_call", "id": "fc_packet", "call_id": "call_packet", "name": "exec_command", "arguments": json.dumps({ "cmd": "cat " + shlex.quote(str(paths[0])), "max_output_tokens": 30000})} - message["content"][0]["text"] = "Completed source research." if index <= 2 else "Original packet read." + message["content"][0]["text"] = "Original packet read." if phase["kind"] == "review" else "Synthetic work verified." return message - def run(*args): - code, result = cli._run_cli(registry, runtime, *args, cwd=project) - assert code == 0, result - return result + def settle_work(result, expected): + assert result["ok"], result + assert candidate.read_text() == str(expected) + guard = json.loads((project / "work-guard.json").read_text()) + assert guard["heartbeat_receipt"]["turn_instance_id"] == result["turn_id"] + binding = ("--goal-id", cli.GOAL_ID, "--agent-id", cli.AGENT_ID, + "--todo-id", cli.TODO_ID, "--turn-instance-id", result["turn_id"]) + run("refresh-state", *binding, "--classification", "synthetic_candidate_verified", + "--delivery-batch-scale", "implementation", "--delivery-outcome", "outcome_progress", + "--delivery-boundary", "in_flight_continuation", "--progress-result-class", "advanced", + "--progress-surface-id", "fixture:candidate", "--progress-evidence-id", f"evidence:candidate-{expected}", + "--no-global-sync", "--suppress-external-sinks") + spent = run("quota", "spend-slot", *binding, "--source", "heartbeat", "--execute") + assert spent["settlement_progress"]["state"] == "settled" + work_turns.append(result["turn_id"]) + if expected == 3: + replay = run("quota", "spend-slot", *binding, "--source", "heartbeat", "--execute") + assert replay["idempotent_replay"] and not replay["appended"] with real_native_source(tmp_path / "provider", os.environ["LOOPX_NATIVE_CHECKPOINT_BIN"], project, close_source=True, response_builder=response, @@ -410,53 +494,74 @@ def run(*args): LOOPX_ITERATION_CONTEXT="resume", LOOPX_SHARED_SKILLS=str(skills), MODEL_NAME="fixture", REASONING_EFFORT="high", OPENAI_BASE_URL=base["model_providers"]["fixture"]["base_url"], OPENAI_API_KEY="synthetic-local-only", LOOPX_CODEX_TURN_TIMEOUT_SEC="60") - prior = run_once(env) - assert prior["ok"] and len(calls) == 2 - binding = ("--goal-id", cli.GOAL_ID, "--agent-id", cli.AGENT_ID, - "--todo-id", cli.TODO_ID, "--turn-instance-id", prior["turn_id"]) - run("quota", "should-run", "--runtime-profile", "generic_cli", *binding) - vision = tmp_path / "vision.json" - vision.write_text(json.dumps({"schema_version": "goal_vision_replan_contract_v0", - "state": "vision_patch_proposed", "vision_patch": { - "acceptance_summary": "Continue synthetic research; complete checkpoint qualification remains."}, - "path_delta": {"outcome": "continue", "prior_assumption": "Source fixture is available.", - "observed_reality": "Actual native source and CLI boundary checked.", - "retained": ["Complete checkpoint qualification"], - "evidence_refs": ["evidence:completed-source-fixture"]}})) - run("refresh-state", *binding, "--classification", "source_fixture_verified", - "--delivery-batch-scale", "implementation", "--delivery-outcome", "outcome_progress", - "--delivery-boundary", "in_flight_continuation", "--agent-vision-json", str(vision)) - run("quota", "spend-slot", *binding, "--source", "heartbeat", "--execute") - index_path = runtime / "goals" / cli.GOAL_ID / "runs/index.jsonl" - original_runs = index_path.read_bytes() - cli._configure_autonomous_replan_fixture(project, runtime, registry) - state_path = project / f".codex/goals/{cli.GOAL_ID}/ACTIVE_GOAL_STATE.md" - state_path.write_text(state_path.read_text().replace("## Agent Todo\n\n", - "## Agent Todo\n\n- [ ] [P1] Continue the retained experiment.\n" - f" \n")) - # Preserve the real source settlement; synthetic trigger rows are - # fixture inputs, never claims of actual research progress. - index_path.write_bytes(original_runs + index_path.read_bytes()) - listed = run("todo", "list", "--goal-id", cli.GOAL_ID) - initialize_canonical_authority(runtime, cli.GOAL_ID, - build_todo_runtime_shadow_projection(goal_id=cli.GOAL_ID, todos=listed["todos"], - handoff_mode="hard_lease", leases=[]), - state_path=state_path, provider="sqlite") + for value in range(1, 7): + phase.update(kind="work", values=[value], tool_sent=False) + prior = run_once(env) + settle_work(prior, value) + assert len(set(work_turns)) == 6 and len(calls) == 13 request = tmp_path / "request.json" request.write_text(json.dumps({"destination": str(logs.parent / "checkpoints"), "components": [{"id": "code", "source_path": str(candidate)}], "point_metadata": {}, "trigger": "fresh_replan", "native_session": True, "loopx_state": True, "evaluation_opportunity": True, "controller_handoff": True})) + phase.update(kind="review", tool_sent=False) result = run_once({**env, "LOOPX_CHECKPOINT_REQUEST": str(request)}) assert result["ok"] and result["checkpoint"]["captured"], result assert result["checkpoint"]["phase"] == "before_replan" assert result["checkpoint"]["source_turn_id"] == prior["turn_id"] assert result["session"]["session_id"] == prior["session"]["session_id"] - assert not result.get("checkpoint_handoffs") and len(calls) == 4 + assert not result.get("checkpoint_handoffs") and len(calls) == 15 assert cli._heartbeat_receipt_count(runtime, result["turn_id"]) == 1 - assert "settlement_binding_owed" in json.dumps(requests[3]) - assert candidate.read_text() == "original candidate" and len(guards) == 1 - (tmp_path / "controller-result.json").write_text(json.dumps(result["checkpoint"])) + assert "settlement_binding_owed" in json.dumps(requests[14]) and len(guards) == 1 + + # The original source now admits its review; capture has granted nothing. + selected = run("quota", "should-run", "--goal-id", cli.GOAL_ID, "--agent-id", cli.AGENT_ID, + "--todo-id", cli.TODO_ID, "--turn-instance-id", result["turn_id"], "--runtime-profile", "generic_cli") + control = selected["replan_action_packet"]["stride_control"] + decision = json.loads(json.dumps(control["minimal_example"])) + decision.update(basis="supported_progress", action="grow", axes={ + "work_slice": {"plan": "Complete candidates 7 and 8 with local validation in one work wake.", + "reason": "Six settled candidates establish the same complete synthetic work path."}, + "route_distance": {"plan": "Retain the counter increment method.", + "reason": "This checks continuation, not a scientific route improvement."}, + "validation_scope": {"plan": "Verify both candidate evidence files and final value 8.", + "reason": "A larger complete slice must preserve the tested postcondition."}}, + next_probe={"scope": "One two-candidate synthetic work slice.", "prediction": "Final candidate is 8.", + "adoption_criterion": "Both evidence files and final value agree.", + "cost_bound": "One native wake plus its validation and settlement.", + "minimum_evidence": "Native tool output, candidate bytes, evidence and accountable settlement."}) + if "explore_review" in decision: + decision["explore_review"].update(disposition="no_applicable_evidence", considered_refs=[], + reason="The delivered graph contains no findings; retain the tested synthetic method.") + vision = tmp_path / "vision.json" + vision.write_text(json.dumps({"schema_version": "goal_vision_replan_contract_v0", + "state": "vision_patch_proposed", "vision_patch": {"acceptance_summary": "Continuation qualification remains."}, + "path_delta": {"outcome": "continue", "prior_assumption": "One candidate per work wake.", + "observed_reality": "Six native work wakes and their once-only settlements succeeded.", + "changed": ["Complete two synthetic candidates in the next work wake."], + "evidence_refs": ["evidence:candidate-6"]}, "stride_decision": decision})) + binding = ("--goal-id", cli.GOAL_ID, "--agent-id", cli.AGENT_ID, + "--turn-instance-id", result["turn_id"]) + identity = selected["heartbeat_receipt"]["settlement_identity"] + # Selection may preserve a legal Todo binding. Use its actual causal + # identity instead of assuming every periodic review is semantic-only. + if identity.get("todo_id"): + binding += ("--todo-id", identity["todo_id"]) + else: + binding += ("--replan-obligation-id", identity["replan_obligation_id"]) + run("refresh-state", *binding, "--classification", "synthetic_stride_review", + "--autonomous-replan-recorded", "--delivery-outcome", "outcome_progress", + "--agent-vision-json", str(vision), + "--no-global-sync", "--suppress-external-sinks") + run("quota", "spend-slot", *binding, "--source", "heartbeat", "--execute") + phase.update(kind="work", values=[7, 8], tool_sent=False) + adopted = run_once(env) + settle_work(adopted, 8) + assert len(calls) == 17 and len(work_turns) == 7 + assert all(json.loads((project / f"evidence-{v}.json").read_text())["value"] == v for v in (7, 8)) + (tmp_path / "controller-result.json").write_text(json.dumps({**result["checkpoint"], + "natural_work_turns": work_turns[:6], "stride_action": "grow", "explore_mode": explore_mode, + "source_followthrough_verified": True, "adopted_turn_id": adopted["turn_id"]})) @pytest.mark.skipif(not os.environ.get("LOOPX_NATIVE_CHECKPOINT_BIN") or os.name != "posix", @@ -581,6 +686,17 @@ def response(payload, index, message): return assert len(result["checkpoint_handoffs"]) == 1 event = result["checkpoint_handoffs"][0] + if capture == "native" and sys.platform == "darwin": + # Unsupported native interruption is refused before any signal; + # the original tool result is delivered and no abort is fabricated. + assert not event["captured"] and "not qualified on Darwin" in event["error"] + assert len(calls) == 4 and (project / "continued.txt").read_text() == "continued" + assert any(item.get("type") == "function_call_output" and + item.get("call_id") == "call_midwake" for item in requests[2]["input"]) + assert not any(item.get("type") == "function_call_output" and + "aborted" in item.get("output", "") for item in requests[2]["input"]) + assert not list((logs.parent / "checkpoints").glob("*/checkpoint.json")) + return assert event["captured"] is (capture in {"files", "native", "controller"}), json.dumps(event) assert event["research_point_qualified"] is False assert len(calls) == 4 and (project / "continued.txt").read_text() == "continued" From f72d39ab0c036227f42953cc910aef319a221405 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Wed, 14 Oct 2026 12:22:11 +0800 Subject: [PATCH 68/69] docs(checkpoint): delimit host support and natural stride qualification Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/README.md | 17 +++++++++++++++-- .../hierarchical-agent-stride-control-v0.md | 13 +++++++++++++ ...ierarchical-agent-stride-control-v0.zh-CN.md | 9 +++++++++ 3 files changed, 37 insertions(+), 2 deletions(-) diff --git a/benchmark/README.md b/benchmark/README.md index 762215db20..61a27b4067 100644 --- a/benchmark/README.md +++ b/benchmark/README.md @@ -236,8 +236,14 @@ session, and captures explicitly declared inert files and optional native histor still resumes the source with its original packet; altered packet bytes or lost session identity fail closed. All resumptions share the original wake deadline. -This is **not a qualified complete research checkpoint**. With `native_session` -requested, the actual worker can seal an explicit interrupted cursor: all retained +This is **not a qualified complete research checkpoint**. Interrupted native +capture is qualified on the tested Linux profile. On Darwin, a first mid-wake +replan with `native_session` is explicitly refused **before** signalling the +native CLI: its original tool result continues and the missed point is recorded +once. The observed Darwin Host can omit the pending command from its interrupted +history; delayed reads and leader-only signalling did not recover that cursor. +Completed-wake capture remains separate. No history/output predicate is relaxed. +On the qualified profile, the actual worker can seal an explicit interrupted cursor: all retained pages, the failed final guard command with no released output or later visible item, its parsed CLI routes/Turn identity, and the original held decision bytes. A second matching guard is ambiguous and rejects. The exact renderer Turn @@ -619,6 +625,13 @@ identities; wrong-source/digest and advanced-history admission reject. No old lease, quota, timer or token is activated. Scientific restart still requires the chosen installed source/profile and experiment inputs to be admitted; this mechanical fixture proves neither model judgment nor policy benefit. +The completed-source qualification also runs six actual settled native work wakes +without fabricated history, with stride enabled and Explore planning separately +off/on. The resulting periodic review is captured before policy selection; a +recorded grow decision is followed by a larger synthetic work slice and normal +settlement. This proves mechanical composition, not model-selected strategy +quality. Retained research remains inert destination context: fresh graph, +vision/policy review adoption and same-point scientific comparison remain open. Omit the controller file and `controller_handoff` on a new run to disable automatic controller capture; removing the checkpoint request disables capture entirely. Existing artifacts remain inert, and the source trial is not restarted by disabling it. diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md index b43bbc5305..7c1f54ab3d 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.md @@ -931,6 +931,19 @@ submission returns 429 without consuming a round or invoking a grader. The separate eligible-cursor fixture receives its next grade. These two lawful states must remain distinct; restoring a point does not make its next submission eligible. +The completed-source fixture now reaches that review through six actual native +work wakes and exactly-once CLI settlements rather than inserted history rows. +Stride is enabled with Explore planning separately off/on. Capture precedes the +decision; the source records grow with independent axis plans and completes its +larger synthetic next slice. Fresh isolated receivers consume the sealed code and +context while preserving pending cooldown. This qualifies mechanical composition, +not model judgment, automatic destination graph/policy adoption or utility. +The tested Darwin native Host can omit the pending command after interruption; +delayed reads and leader-only signalling do not repair it. That profile refuses +mid-wake native capture before signalling and releases the original tool response. +Linux interrupted and completed-wake capture retain their separate qualification; +strict history, ambiguity, source and deadline predicates remain unchanged. + Committed native session admission now retains the captured artifact digest in the original atomic session binding. Exact controller retries after a lost binding reply verify that receipt, current Goal/Agent/profile and unchanged complete diff --git a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md index 3e38b38634..2845d51439 100644 --- a/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md +++ b/docs/architecture/rfcs/hierarchical-agent-stride-control-v0.zh-CN.md @@ -765,6 +765,15 @@ controller 验证，捕获发生在模型看见 fresh replan 之前。已提交 推进，原生提交仍返回 429，不消耗轮次也不调用 grader。另一已 eligible 的 cursor 用例取得下一次评分。这两种合法状态须保持区分；恢复研究点不让下一次提交立即 eligible。 +completed 源用例现在通过六个真实 native 工作 wake 及单次 CLI 结算自然触发 +复盘，不插入历史行。stride 开启，Explore planning 分别关闭/开启；先捕获， +再记录三个轴独立计划的 grow 决策，随后完成较大的合成工作切片。新隔离目标 +使用封存代码/上下文并保留未到期冷却。此项验证机械组合，不证明模型判断、 +目标 graph/policy 自动采用或科研收益。当前验证的 Darwin Host 在中断后会丢失 +pending command；延迟重读及仅向主进程发信号均未恢复。因此该 profile 在发信号 +前明确拒绝中途 native 捕获，继续原工具响应。Linux interrupted 与 completed-wake +路径分别保留验收，不放宽历史、歧义、源或期限检查。 + inert verification unit 不授予执行权。目标侧使用新的实验/Host 身份，通过既有 Goal/session/state admission 进入。历史租约、quota、timer、pending effect 和 From 5eb9baa5cb77afcb873e9aebcd8911c5880d982c Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Wed, 14 Oct 2026 13:59:12 +0800 Subject: [PATCH 69/69] test(checkpoint): classify Darwin refusal by requested native profile Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- benchmark/tests/test_native_checkpoint_handoff.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/benchmark/tests/test_native_checkpoint_handoff.py b/benchmark/tests/test_native_checkpoint_handoff.py index 7118cc7e16..529417f404 100644 --- a/benchmark/tests/test_native_checkpoint_handoff.py +++ b/benchmark/tests/test_native_checkpoint_handoff.py @@ -686,7 +686,7 @@ def response(payload, index, message): return assert len(result["checkpoint_handoffs"]) == 1 event = result["checkpoint_handoffs"][0] - if capture == "native" and sys.platform == "darwin": + if json.loads(request.read_text())["native_session"] and sys.platform == "darwin": # Unsupported native interruption is refused before any signal; # the original tool result is delivered and no abort is fabricated. assert not event["captured"] and "not qualified on Darwin" in event["error"]