Skip to content

test(quota): restore canonical settlement consumer coverage - #7183

Merged
huangruiteng merged 1 commit into
mainfrom
codex/settlement-fixture-qualification
Oct 24, 2026
Merged

huangruiteng merged 1 commit into
mainfrom
codex/settlement-fixture-qualification

Conversation

@loopx-agent

@loopx-agent loopx-agent commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Settlement integration tests were stopping before they exercised their intended contracts: the Explore callers still used unmigrated Goal fixtures, and the generated-command helper rejected the worker renderer’s explicitly bound CLI path. These failures reproduce on the current main branch and were also observed while qualifying #7174.

Initialize the existing disposable authority fixture before ordinary Explore reads, cover File and SQLite for the Codex App and generic CLI profiles, and let the shared helper recognize only the current Python environment’s explicit loopx path in addition to the bare command. The existing unmigrated-read rejection, optional-attachment parity, selection/replay identity, premature-closeout and single-debit assertions remain in place. This is test setup repair; production behavior and benchmark runs are unchanged, and other CI failures remain unassigned.

结算集成测试在进入原有语义断言前就失败:Explore 调用方没有初始化 canonical authority,共用执行器也拒绝 worker 生成的明确 CLI 路径。本变更复用既有隔离 fixture,覆盖 File/SQLite 与 Codex App/generic CLI,恢复原有结算、重放和单次扣费验证;没有恢复旧 Goal 的读取兼容,也没有修改产品或 benchmark。

Validation:

  • Before repair, both targeted cases failed on main at 7129efcde.
  • Explore attachment and heartbeat transport suites: 39 passed, covering real File/SQLite providers, both CLI profiles, full/envelope transport, retained rejection cases and replay with a single debit.
  • Shared settlement CLI suite: 157 passed.
  • Standard diff-selected premerge: passed all five direct checks and three selected quota/scheduler canaries; staged bilingual check and public/private scan passed.
  • The semantic advisory found no supported new vocabulary carriers; the provider parameters reuse existing test values. The exact-head CI run stops at kernel admission because the Stage 2C lock omits the declared tiktoken==0.12.0 dependency; downstream qualification did not run. The invariant and all three declaration/lock inputs are byte-identical on this PR, its immutable base 7129efcde, and current main 3378f28d1. The separate lock repair #7185 already owns this failure. Full-suite qualification remains outstanding; no CI exemption or duplicate lock fix is included here.

Scope review: reuse the existing authority and subprocess helpers; no new fixture framework or production vocabulary. Public/private diff scan passed. This proposal restores a focused validation prerequisite and does not establish model adoption or token savings.

远端 CI 已实际失败于共享 Stage 2C 依赖锁缺少 tiktoken,后续测试矩阵未执行。已核对该检查和依赖输入在本 PR、原基线与当前 main 完全一致;由现有 #7185 修复,保留完整 CI 未资格的边界。

Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent | gpt-6.1-sol | OpenAI | runtime_reported | reasoning_effort=max

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

动机

运行现有 Explore 与 quota 结算集成回归的维护者。 Explore 是保存可复用发现的能力,结算负责把有效写回与一次扣额关联到同一任务。

旧测试在未初始化权威存储或明确 CLI 路径校验处中断;修复后真实 File/SQLite 调用进入原有可选附件、提前完成拒绝与单次扣费断言。 这会使维护者无法从回归判断可选证据是否仍服从原结算门;恢复有效准备后,原有断言才真正保护产品调用方。当前三个完整模块共196例通过,最新主线组合另22例通过,标准 premerge 的5项直接检查和3项选中检查通过。 本次恢复现有测试消费者的有效覆盖,生产配置、授权、默认行为和结算实现均未改动。

改动思路

在原集成测试中复用既有 canonical authority fixture,先完成合成来源准备,再调用正常 CLI;File 与 SQLite 各自成为测试参数。隔离同时覆盖数据目录、临时目录和 Node SQLite 支持,没有迁移或修改正在运行的 Goal。

共用执行器只增加当前 Python 环境内的明确 loopx 路径,仍拒绝其他路径中同名程序。它继续通过 sys.executable -m loopx.entrypoint 执行其余生成参数;console 脚本启动包装本身不在这个 helper 的覆盖范围。保留严格路径判断比接受任意 basename 更符合原测试用途,无需新增执行框架。

这次14行增加、5行删除都留在已有测试所有者内;生产模块与公开协议没有变化。准备修复、真实 CLI 调用和原有语义断言构成一个完整的维护切片。这个简单先后关系用文字即可说明。

具体改动

  1. test_real_cli_settlement_exposes_optional_attachment 增加 File/SQLite 参数,使用既有隔离和初始化函数。Codex App 与 generic CLI、evidence 与 planning 的完整计划仍相等;附件保持 optional,生成的写回命令不强塞附件参数,提前完成继续被原回执门拒绝。
  2. test_ordinary_cli_settlement_without_evidence_does_not_create_graph 在原 read-only fixture 编辑之后初始化权威,再执行写回和扣额;最终仍只产生一次 spend,Explore 图保持不存在。原 off-plan、无绑定及 driver callback 分支的断言未删除。
  3. _run_generated_cli 的白名单由裸命令扩为裸命令加当前环境的精确路径。独立六输入对照证明:基线拒绝合法明确路径,新 head 接受;四个无关程序或路径在两边都在启动子进程前拒绝。该局部探针模拟子进程;另一个独立真实入口对照用 --cli-bin 生成明确路径,在 File/SQLite 的基线均于 helper 拒绝、扣额为0,新 head 均完成写回和扣额,重放后仍为一次,且未创建图。上述196例也继续走真实 CLI 与生成参数。

独立依据为 docs/development/good-smokes.md,spec_revision 7129efcdef7a7110bd49b760b7de8359544889ec。按变更前规则逐项核对:Review Checklist 的独立语义断言和真实入口保留;Consolidate Without Losing Coverage 复用原 fixture/helper,未新增同形 smoke;Public-Safe Fixtures 使用隔离合成资料;Validate The Change 已运行聚焦模块、真实主线组合和 diff premerge。四项均为 implemented,没有用新测试输出重新定义产品合同。

对主干的风险

最强风险是只让测试变绿,却跳过原结算不变量,或把 helper 放宽成任意同名可执行文件。完整 diff 保留原断言,独立拒绝探针验证精确路径边界,真实 CLI 模块继续覆盖选择、重放、提前 closeout 与单次扣额。

不可变基线 7129efcdef7a7110bd49b760b7de8359544889ec 的原两个模块实际结果为 6 failed, 163 passed in 466.11s (0:07:46);失败定位保留在基线原输出,当前三个完整模块为 196 passed in 516.92s (0:08:36)。完整集合包括实际 test_heartbeat_envelope_transport.py;两个修改文件的 Ruff 检查也通过。

当前 PR 的真实目标已为 3378f28d1210341affc96020a4f0336164f3a8a6。独立生成该主线与本次两文件改动的干净组合源码树后,Explore 附件整模块、host-owned selection/replay 和未迁移读取拒绝共 22 passed in 48.35s;组合仅用于隔离验证,没有操作 GitHub 合并。精确 head 的 premerge 五项直接检查、三项选中检查均通过,零失败、零人工 hold。完整生产树未重跑,远程 CI 按本次配置未查询、轮询或等待。

扫描了相邻 Explore、TurnEnvelope、settlement 消费者及最近20个同作者公开提案,并读了相关 #7181 的完整已发布评审。两者修复不同的现有调用边界;本 PR 复用原基础设施,没有增加一次性脚手架。没有证据支持重复违规或贡献限制升级。

我的整体评价

APPROVE,仅适用于 8a03228f34b391fea2d45080578e17377e6c0a3f 的这项测试准备修复。它恢复既有 shipped CLI 合同的有效回归,收益是可解释的维护覆盖;没有增加产品运行成本、额外用户配置或确认轮次。long_horizon 与 user_experience 均保持原有行为,测试通过没有被用作模型采用或长期收益证明。

只修改两个测试文件;核对其 CLI、fixture 和断言调用,没有页面、布局或可见产品文字变化,因此无需新增视口证据。 未重跑全库测试,未验证安装态、真实模型采用或长期净收益。 兼容变化限于同版测试 helper 接受当前环境合法路径,没有新增持久格式或跨版本 decoder。最高价值的简化已经采用:复用原 authority 与 subprocess helper,保持不同语义断言独立;无需再建一套框架。批准与独立 merge-readiness、授权及实际合并分别处理。

English verdict: APPROVE — 8a03228. This repairs setup in two existing settlement integration tests without changing product behavior or weakening their semantic assertions. The immutable baseline reproduces preparation failures; all196 current module cases and22 latest-main combination cases pass, as do5 direct and3 selected premerge checks. A six-input helper probe retains rejection of unrelated executables. Full-suite, installed behavior and model utility remain unverified; CI was not consulted.

@huangruiteng
huangruiteng merged commit eca7dfc into main Oct 24, 2026
18 of 23 checks passed
@huangruiteng
huangruiteng deleted the codex/settlement-fixture-qualification branch October 24, 2026 00:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants