Skip to content

test: align selected-work CLI binding with generated prompt - #7181

Merged
huangruiteng merged 1 commit into
mainfrom
codex/selected-work-command-binding
Oct 24, 2026
Merged

huangruiteng merged 1 commit into
mainfrom
codex/selected-work-command-binding

Conversation

@loopx-agent

@loopx-agent loopx-agent commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

The selected-work requirements regression compared a default heartbeat with a TurnEnvelope generated using a different CLI binding. Both File and SQLite failed at complete-context equality before the downstream Host and readback assertions could run.

Bind both callers to the same source-checkout CLI, assert that the emitted read command retains that executable and its --cli-bin binding, and execute the emitted argv unchanged. Complete-context equality, full requirement/stop text, fresh readback, missing-Todo and source-failure assertions remain in place. This changes one test file.

The exact-head code review publishes an APPROVE code verdict as a COMMENTED review and reports an independent complete-module 10 passed. This applies to the single-file repair at 7e4e04d475ae943a8ad7e386c6902020546b664f; complete repository and merge qualification remain outstanding.

The natural CI run failed at kernel static admission: the existing Stage 2C lock lacks declared tiktoken==0.12.0. The downstream test-shard job was skipped. Main shows the same missing dependency. The scoped lock repair is proposed separately in #7185, which is not yet adopted or qualified as a complete CI pass.

Author validation at 7e4e04d475ae943a8ad7e386c6902020546b664f, against the original 385771e56371315976ebe1166a3c07fea8b539df baseline:

  • Immutable main baseline 385771e56371315976ebe1166a3c07fea8b539df: both affected cases reproduce the binding mismatch. Final complete module: 10 passed, including real File/SQLite CLI and recovery paths.
  • Repository CI-scope Ruff, configured mypy (17 source files), CLI output budget/base-head differential, public candidate/diff scans, staged bilingual check and whitespace check passed. The semantic advisory found no supported new vocabulary carriers.
  • Full repository Python qualification has not been rerun at this head. The existing whole-suite failure evidence remains historical failure evidence; these focused passes do not qualify the full suite or hosted CI. The previously recorded change-quality receipt remains historical non-passing evidence; it does not qualify a newer target revision. Full required qualification is incomplete.

Goal-bound standard premerge ran five direct checks, all passing, with zero catalog selections for this test-only path. Its final aggregate is FAILED because it enforces the non-passing change-quality receipt. This is a completed author proposal for review, with merge qualification held.

Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent | gpt-6.1-sol | OpenAI | runtime_reported | reasoning_effort=max

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

动机

开发者运行默认 heartbeat 的完整需求回归时,需要验证生成的命令能从同一源码环境读回当前任务。

旧测试把默认 CLI 的 TurnEnvelope 与绑定源码 CLI 的默认提示词比较,File、SQLite 都先在完整上下文比较处失败;统一绑定后,原有比较保留,后续 Host 和真实命令读取得以执行。

已独立验证完整模块10例通过,完整要求、停止条件、后续更新、缺失任务和来源读取失败都仍有真实调用覆盖。 本次不改变运行时、用户默认调度、授权、生产状态或提示词内容,也不宣称全库测试、CI 或合并资格通过。 完整仓库资格仍未验证;最新主线上的精确范围质量回执已过期,须由原提案的维护者重新资格验证后再判断合并。

这是修复既有回归覆盖的有用增量,而不是新的产品功能。不能只删掉失败的完整比较,或在测试里代造另一条可用命令:这都会放过实际生成命令的错误。所用历史经验只帮助确认测试准备之后仍有真实 Host 和读回结果,不增加本次内部测试修复之外的交付义务。

改动思路

在既有测试中复用一个源码环境的 CLI 路径,让两种生产入口使用相同绑定;之后验证生成命令的可执行文件与绑定参数,并原样执行该参数列表。当前改动只有一个测试文件,7行增加、4行删除;不新增生产所有者、状态、CLI 选项、版本分支或烟测模块。最小修复已经留在原集成回归内,继续保留完整上下文、完整要求和失败路径断言。这样测试真正验证调用者将收到的命令,维护者也不需要维护第二份命令重写逻辑。这个简单改动用文字即可说明,无需额外流程图。

具体改动

  1. guard 与默认 heartbeat-prompt 共用 cli_bin,保留原有 envelope["work_context"] 的完整相等比较。独立在不可变主线 385771e56371315976ebe1166a3c07fea8b539df 重跑两个受影响案例,File、SQLite 均在完整 sources 命令绑定处失败;当前完整模块10例通过,并继续执行比较之后的断言。
  2. 生成命令前缀断言 要求实际可执行文件和 --cli-bin 都等于调用者指定值。要求来自已有源码环境绑定和当前原生读取合同,而非把头部实现本身当成正确性定义。
  3. read_detail 与缺失任务调用直接执行原始 argv,并保留同一 cwd、registry、runtime 与 PYTHONPATH。旧写法替换 argv 的首部,可能掩盖生成的可执行文件错误;新写法实际走完整命令。原有 Task/Goal 变化后的新鲜读取、Host prompt、缺失任务和来源文件读取失败仍保留。

合同依据为更改前 385771e56371315976ebe1166a3c07fea8b539df 的 docs/heartbeat-automation-prompt.md,对应原章节 Two Prompt Layers、Static semantics retained across hosts:使用本轮当前合同,向 Host 提供完整要求、验收及停止条件,并让来源失败可见。另核对同版本 docs/development/testing-and-quality.md 的源码环境说明;测试没有改写这些义务。检查相邻 selected-work、heartbeat、TurnEnvelope 和 Todo 覆盖后,没有新增同形副本;最近20项公开PR与完整30项当前队列也没有给本次修复建立重复脚手架的理由。

对主干的风险

最强回归是生成命令指向错误的 CLI,而测试偷偷用已知可用的模块调用替代。当前前缀断言与原样 argv 执行共同覆盖它;独立基线反例证明原失败存在,修复没有降低完整要求。完整模块使用真实 File、SQLite、CLI 子进程与 Host 构造,但 Goal 和材料为隔离合成数据,不证明实际模型采用或全库无回归。

本次独立验证:完整模块10例、仓库CI配置范围的 Ruff、配置内17个文件的 mypy,以及固定基线的 CLI 输出预算检查通过。标准 premerge 的5项直接检查通过,测试文件未选中 catalog/risk smoke;其最终聚合仍为 FAILED / quality_stale_receipt。当前实际目标主线已到 7129efcdef7a7110bd49b760b7de8359544889ec,质量回执必须按最新 base/diff 重新资格验证,不能用此评审或五个直接检查冒充合并通过。全库Python资格未重跑,旧失败证据仍只是历史证据,不声称已修好或已被独立归因为无关。当前配置不查询、不轮询、不等待远程CI。

我的整体评价

APPROVE,仅针对 7e4e04d475ae943a8ad7e386c6902020546b664f 的完整单文件测试修复。 原有集成回归已经拥有该边界,在原处修复绑定并执行真实命令,比另建测试层更直接。后续当前要求变更、旧观察保留和来源失败仍被验证,不新增重试、扣额或生命周期规则。测试准备仍复用已知 registry、runtime、CLI 和任务,没有新增用户配置、确认或补录;实际产品调用路径没有行为变化。

仅修改 tests/control_plane/test_selected_work_requirements.py;检查了生成命令、Host 和读回调用,生产 CLI、页面、Lark 与可见提示词均无改动,因此不需要新截图。 没有声称新增产品页面或实际模型验证。完整仓库资格及最新范围 CQR 仍是原提案维护者的后续门;本评审不自合并,不清除其他讨论,不把测试修复等同于整个长期目标完成。

English verdict: APPROVE — 7e4e04d475ae943a8ad7e386c6902020546b664f. The existing regression now uses one explicit source CLI binding and executes the emitted argv unchanged without weakening full-context, Host or fresh-readback assertions. The immutable baseline reproduces both provider failures; the current module passes all10 cases, with configured local static/budget checks passing. Merge remains held by stale exact-scope quality qualification; the full suite and remote CI are not claimed.

@huangruiteng
huangruiteng merged commit bef5892 into main Oct 24, 2026
18 of 23 checks passed
@huangruiteng
huangruiteng deleted the codex/selected-work-command-binding branch October 24, 2026 00:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants