diff --git a/docs/architecture/rfcs/long-horizon-harness-benchmark-research-program-v0.md b/docs/architecture/rfcs/long-horizon-harness-benchmark-research-program-v0.md index 9bae45b307..98d7a2e597 100644 --- a/docs/architecture/rfcs/long-horizon-harness-benchmark-research-program-v0.md +++ b/docs/architecture/rfcs/long-horizon-harness-benchmark-research-program-v0.md @@ -594,6 +594,37 @@ for the synthetic example, one-selection detail and separate comprehension acceptance. These are proposed analysis and presentation requirements, not a new runtime protocol, metric promotion or an authorization to expand a study. +#### Relationship collection before a new study + +The [collection and reading slice](../../product/use-cases/auto-research/long-horizon-progress-design.md#record-relationships-before-the-next-study) +is grounded in the #6648 whole-history proposal, not an already qualified +assessment service. Reuse ordinary Explore result writeback and its optional +relations to preserve the evidence behind a route change. Distinguish explicit +relations, parent grouping, and inferred analysis; retain meaningful hypothesis, +experiment and artifact identities rather than only question/finding counts. +The existing edge writer lacks direct evidence-reference and endpoint-version +binding. Preserve reproducible associations in the read-only analysis sidecar +and expose missing provenance; a prose summary cannot close that gap. + +For a study claiming relationship-informed decision quality, qualify one +permitted real case through result writeback, pre-replan checkpoint, restore, +receiver readback and the next observable action before launch. Check the +#6648-derived packaged reading path separately, including correct item selection, +missing/invalid relations, duplicate replay, delayed observations and changed +inputs. An open product defect is not a scientific pass. Native results with +weaker observability retain their narrower claim; all V12 analysis is not a +prerequisite for unrelated valid research. + +Freeze collection guidance and costs across compared arms: asking the Agent to +organize relationships is itself a possible intervention, unlike passive capture. +Checkpoint capture must not implicitly select TurnEnvelope or another input +mode. Preserve old frozen runs and unknown baseline coverage. Judge whether the +record explains a decision and permits independent verification, not whether the +graph has more edges. Research owns source/claim calibration and actual adoption; +Explore owns persistence and legal relations; product owns truthful reading. +No new relation vocabulary, runtime enforcement, paid-run authorization or +causal-effect conclusion is introduced by this documentation slice. + ### 6.4 Human attention For assisted studies only: diff --git a/docs/architecture/rfcs/long-horizon-harness-benchmark-research-program-v0.zh-CN.md b/docs/architecture/rfcs/long-horizon-harness-benchmark-research-program-v0.zh-CN.md index ec60ae6c9a..9e0170a7df 100644 --- a/docs/architecture/rfcs/long-horizon-harness-benchmark-research-program-v0.zh-CN.md +++ b/docs/architecture/rfcs/long-horizon-harness-benchmark-research-program-v0.zh-CN.md @@ -479,6 +479,28 @@ Todo/replan/Explore 只有连接实际执行和外部产物后才用于机制解 给出合成示例、一次选择的详情与独立理解验收。这些是拟议分析和展示要求,不是新的运行 协议、指标晋升或扩展实验的授权。 +#### 新实验前的关系采集 + +[采集与阅读切片](../../product/use-cases/auto-research/long-horizon-progress-design.zh-CN.md#record-relationships-before-the-next-study) +基于 #6648 的全历史提案,不是已经完成资格验证的评估服务。复用普通 Explore +结果回写及其可选关系,保留路线变化背后的证据。区分显式关系、parent 分组与推断 +分析;保留有意义的假设、实验及产物身份,不能只统计问题和 finding 数量。 +现有 edge writer 缺少直接的证据引用和端点版本绑定。只读分析 sidecar 应保留 +可复现的关联并暴露来源缺失;文字摘要不能关闭这一缺口。 + +若实验要论证关系信息改善了决策质量,启动前须用一个获准真实案例完成结果回写、 +replan 前 checkpoint、恢复、接收者读回及下一次可观察动作的资格验证。 +单独检查源自 #6648 的打包阅读路径,覆盖正确项目选择、缺失或无效关系、重复回放、 +迟到观察及输入变化。产品缺陷尚未关闭不等于科学验证通过。可观测性较弱的原生 +结果研究保留较窄的主张范围;不把全部 V12 分析设为其他有效研究的前置条件。 + +对比各臂须冻结相同的采集指导和成本口径:要求 Agent 组织关系本身可能构成实验 +干预,不能等同于被动采集。Checkpoint 捕获不得隐式选择 TurnEnvelope 或其他 +输入模式。保留旧冻结运行与未知的基线覆盖。判断记录是否解释了一项决策、能否 +独立复核,不能用更多图边替代。研究侧负责来源与主张校准及实际采用;Explore +负责持久化与合法关系;产品侧负责如实阅读。本次文档切片不引入新关系词汇、 +运行时强制、付费运行授权或因果效应结论。 + ### 6.4 人类注意力 仅适用于 assisted study: diff --git a/docs/product/use-cases/auto-research/long-horizon-progress-design.md b/docs/product/use-cases/auto-research/long-horizon-progress-design.md index 7a7015c0f2..5616447e09 100644 --- a/docs/product/use-cases/auto-research/long-horizon-progress-design.md +++ b/docs/product/use-cases/auto-research/long-horizon-progress-design.md @@ -107,6 +107,93 @@ and a version-specific independent check. An interpretation cites its basis and can abstain. A disputed interpretation remains disputed; selecting it does not promote it to a fact. These are presentation distinctions, not new protocol enums. +## Record relationships before the next study + +This refinement is based on [the whole-history proposal #6648](https://github.com/loopx-project/loopx/pull/6648) +at `f657a65c6abd03f1f6b87937c327f4ccfb2ccde2`: a default record-time view, +optional relationship view, and inline recorded evidence. That source is a +presentation proposal, not installed behavior or completed V11/V12 acceptance. +It does not manufacture missing research relationships or automatically compare +native outcomes. Its timeline target-selection defects and independent review +remain separate product work; a corrected head needs its own readback. + +The first useful improvement is to retain **why the next experiment follows +from the previous result**. At a meaningful conclusion or route change, the +research Agent should reuse the existing question and attach an evidenced +relation during ordinary result writeback. Users do not enter IDs, draw edges or +approve individual links. This is study guidance, not a new mandatory attachment +on every Turn or a graph-density quota. + +| Question the relation answers | Existing representation | Evidence needed | +| --- | --- | --- | +| What question or prerequisite does this attempt address? | Explicit question/hypothesis/experiment nodes; `answers` or `depends_on`, with direction matching the assertion | Attempt identity, tested inputs and an actual prerequisite or answer | +| Which claim does this result support or contradict? | `supports` or `refutes` between existing nodes | Scoped finding, permitted result reference, uncertainty and applicable input revision; a failed run alone is not a refutation | +| Why was this next route selected? | Explicit decision references; `leads_to` only for an evidenced consequence | The decision cites its basis; later artifact or behavior separately establishes adoption | +| Which topic contains this work? | `subtopic_of` or an existing parent projection | Topic membership only; never count it as scientific support | + +Use the [existing result attachment](../../../../loopx/capabilities/explore/README.md#results-from-ordinary-work-writeback): +`question_update` and up to four `relations` may accompany a complete result. +Each relation runs from the current question to an existing same-Goal node; +its summary states the observed basis. This hook does not create an arbitrary +experiment/artifact graph. Materialize meaningful hypothesis, experiment or +artifact objects through the existing Explore owner when the case requires +those identities; do not recast every finding as another question just to add a +line. Missing targets, wrong scope, self-links, duplicates and prohibited cycles +must retain their existing rejection behavior. A successful replay does not +append another relationship or claim another adoption. + +**The current evidence boundary is incomplete.** Findings carry evidence references +and tested input revisions, but the edge writer has no direct `evidence_refs` +or endpoint-version parameters. An edge summary is an assertion, not a durable +version binding. Keep the edge event identity and source snapshot with the +permitted finding/artifact references in the research analysis sidecar; if that +association cannot be reproduced, report it as unknown. Do not send invented +fields to the current strict writer. A future contract extension belongs to the +existing Explore typed owner and must qualify writer, reader, checkpoint replay, +CLI and packaged presentation together. It is not delivered by this document. + +The default graph shows explicit source relations and parent grouping distinctly. +Any later inferred association stays in the on-demand analysis layer with its +sources, method revision and uncertainty; it does not silently become a source +edge. Same-checkpoint lineage, matched comparison, implementation inheritance +and actual adoption are different facts. Neither `supports` nor `leads_to` is a +catch-all for them. Record existing exact identities in the sidecar until a real +caller and accepted contract justify extending the vocabulary. + +Before launching a new study that claims relationship-informed decision quality, +research owns one end-to-end collection canary; product owns the corresponding +reading check: + +1. Follow one permitted real case from question/attempt through result, an explicit + relation and the next decision. Reproduce its source, artifact/input revision, + record time and actual attempt time when available. Capture both the relation + and the relevant evidence before replan; restore the checkpoint and verify the + receiver can read the same basis without duplicate writes or future evidence. +2. Read it through the actual result owner and the packaged #6648-derived history + view: the selected item must open its own evidence. Preserve missing links, + isolated nodes, rejected writes, delayed backfills and changed-input limitations. + If that view remains defective, report product acceptance open; do not treat + screenshots or a private custom rendering as the shipped journey. +3. Inspect the receiver's next action separately. A relation being present or + cited proves neither adoption nor a more useful experiment. Keep unsupported + inferences and no-next-opportunity cases visible. Graph classification versus + runtime penalty, Portfolio proxy versus native transfer, and checkpoint capture + versus actual wake input supply three distinct diagnostic questions; select + authorized evidence, not predetermined favorable conclusions. +4. Freeze the same collection instructions, fields and overhead accounting across + compared arms. Agent-authored relationship guidance can change decisions and is + therefore part of the treatment manifest; passive recording alone is not the + same intervention. Keep TurnEnvelope separately off unless explicitly studied. + Do not retrofit the new guidance into old frozen runs. + +The launch requirement is the scoped collection and restore check for the claims +being tested, not completion of all V12 deep analysis or a general provenance +schema. Native-outcome studies with weaker observability must declare that limit. +Coverage diagnostics count unresolved associations, missing versions and erroneous +links against the declared opportunities; edge count and connectivity are not +progress metrics. Public documents use generalized cases; raw trajectories and +private study identities remain in the authorized research store. + ## Two layers, one trajectory-assessment basis The [Goal workspace contract](../../../architecture/rfcs/goal-workspace-surface-v0.md#511-research-goals-whole-history-explore-and-on-demand-analysis) diff --git a/docs/product/use-cases/auto-research/long-horizon-progress-design.zh-CN.md b/docs/product/use-cases/auto-research/long-horizon-progress-design.zh-CN.md index 063e20e7ea..b099f0f05e 100644 --- a/docs/product/use-cases/auto-research/long-horizon-progress-design.zh-CN.md +++ b/docs/product/use-cases/auto-research/long-horizon-progress-design.zh-CN.md @@ -57,6 +57,38 @@ 可靠性和成本限定上述四个视角,不能合并成一个进展总分。证据标记区分观察事实、模型解释和绑定版本的独立检查。解释须引用依据,允许弃权;有分歧就保留分歧,选中它不会将其晋升为事实。这些是展示区别,不是新的协议 enum。 + + +## 下一轮实验前先记录研究关系 + +本细化基于[全历史提案 #6648](https://github.com/loopx-project/loopx/pull/6648) 的 +`f657a65c6abd03f1f6b87937c327f4ccfb2ccde2`:默认记录时间视图、可选关系视图及行内证据。 +该源码仍是产品提案,不代表已安装或 V11/V12 完整验收。它不会补造缺失的研究关系,也没有自动比较原生指标。时间线点选目标问题和独立评审仍归产品侧修复,新头须重新读回。 + +首先要留下的是:**上一次结果为什么导向下一次实验。** 在有意义的结论或路线变化时,研究 Agent 复用已有问题,在正常结果回写中附上有依据的关系。用户不用输入 ID、手工拉线或逐条确认。这是研究指导,不是每 Turn 必填附件或图密度配额。 + +| 关系要回答的问题 | 复用已有表示 | 需要的依据 | +| --- | --- | --- | +| 这次尝试回答什么、依赖什么前提? | 明确的问题/假设/实验节点;`answers`、`depends_on` 的方向符合断言 | 尝试身份、实际测试输入、确有其事的前提或答案 | +| 这个结果支持或反驳哪个主张? | 已有节点间的 `supports`、`refutes` | 有范围的 finding、获准结果引用、不确定性和适用输入版本;运行失败不自动反驳假设 | +| 为什么选择下一条路线? | 明确决策引用;有可证后果才使用 `leads_to` | 决策确实引用依据;后续产物或行为单独证明实际采用 | +| 它属于什么主题? | `subtopic_of` 或既有 parent 投影 | 只表示主题归属,不能计作科研支持 | + +复用[已有结果附件](../../../../loopx/capabilities/explore/README.md#results-from-ordinary-work-writeback):完整结果可带 `question_update` 与至多四条 `relations`,方向是当前问题指向同一 Goal 中已有节点,summary 写明观察依据。这个 hook 不会自动创建任意实验/产物图。真实案例需要假设、实验或产物身份时,通过既有 Explore owner 建立有意义的对象;不要为了连线把所有 finding 都改成新问题。缺失目标、越域、自连、重复及不合法环路继续沿既有规则拒绝;成功回放不能重复写边或重复宣称采用。 + +**当前证据边界尚不完整。** Finding 有证据引用和测试输入版本,但 edge writer 没有直接的 `evidence_refs` 或端点版本参数。summary 是一项断言,不等于持久化版本绑定。研究分析 sidecar 保留 edge event 身份、来源快照及获准 finding/产物引用;关联无法复核就标未知,不能向严格 writer 塞入自创字段。后续合同扩展归既有 Explore 类型化 owner,需一起验证写入、读取、checkpoint 回放、CLI 和打包展示;本文没有交付该扩展。 + +默认图区分显式来源关系与 parent 分组。后续推断关联进入按需分析层,标明来源、方法版本和不确定性,不静默晋升为来源边。同 checkpoint 血缘、匹配对照、实现继承与实际采用各有含义,不能统统塞进 `supports` 或 `leads_to`。在真实调用方和已接受合同推动词汇扩展前,先在 sidecar 保留既有精确身份。 + +新实验若要论证关系信息改善研究决策,启动前由研究侧完成一条端到端采集 canary,产品侧负责对应阅读检查: + +1. 沿一个获准真实 case 追到问题/尝试、结果、显式关系和下一次决策,复核来源、产物/输入版本、记录时间及可得的实际尝试时间。replan 前捕获关系与相关证据;恢复后接收者能读到同一依据,没有重复写入或未来证据。 +2. 通过真实结果 owner 和 #6648 衍生的打包历史视图读回,选中项目应打开它自己的证据。保留缺失关系、孤立节点、拒绝写入、延后回填和输入变化的限制。视图仍有缺陷就保持产品验收开放,不把截图或本机定制渲染当成已交付旅程。 +3. 单独检查接收者的下一次动作。关系被保存或引用,不证明实际采用或实验更有价值。保留无依据推断与没有后续采用机会的情形。Graph 分类指标与运行时间罚、Portfolio 代理指标与原生迁移、checkpoint 捕获与实际唤醒输入分别提供三个诊断问题;选择获准证据,不预设有利结论。 +4. 对比两臂冻结相同采集指导、字段与开销口径。要求 Agent 主动组织关系可能改变决策,须进入 treatment manifest,不能等同被动记录。TurnEnvelope 单独保持关闭,除非显式研究它;不把新指导回填进旧冻结实验。 + +启动要求是当前研究主张所需的采集与恢复验证,不是等全部 V12 深度分析或通用溯源 schema 完成。可观测性较弱的原生结果研究须声明范围。覆盖诊断按预先声明的机会记录无法关联、缺失版本和错误链接;边数、连通度不算进展指标。公共文档使用泛化案例,原始轨迹和私有实验身份保留在获准研究存储中。 + ## 两层展示,共用轨迹评测基准 [Goal 工作区合同](../../../architecture/rfcs/goal-workspace-surface-v0.zh-CN.md#511-研究-goal全历史-explore-与按需深度分析)拥有两层组合。第一层只投影已有获准 Explore 记录和显式关系,零额外模型调用。第二层消费 benchmark 研究 owner 的深入轨迹评测:一套方法、一个可引用的版本化评测结果,再生成解释图和结论。整条 run 和预先固定的共同比较窗口负责覆盖、原生结果与边际成本;研究侧定义跨 Turn episode 及同一快照中的显式来源/提交/产物绑定。产品展示不另定义单位、口径或进展评分。