Ruiteng Huang · October 7, 2026 · Interim research note · Evidence cutoff 14:40 UTC
+Ruiteng Huang · Updated October 26, 2026 · Exploratory research
Giving long-horizon agents a reason for their next step
-Understanding LoopX’s semantic control plane through EdgeBench
-An agent can work for hours and retain its entire conversation while continuing along an unproductive route. LoopX aims to keep the original goal, accumulated evidence and next action connected in a way that can be inspected and revised.
-We have been studying this on portfolio risk calibration and Lean proof completion. Relative to our earliest attempts, both blind studies show substantive system-level progress: beyond persistent zeros, ineffective continuation and slow advancement, toward more verifiable artifacts, more reliable sustained work and markedly faster Lean progress in common time windows. LoopX repairs, prompts and benchmark execution improvements contributed together.
-Our interim assessment is positive: the semantic control plane is showing value on real tasks, and the overall system is more effective than at the start. We now need to separate contributions and make the observed progress more stable and reusable. This article retains the evidence of overall improvement alongside technical mechanisms and unresolved counterexamples.
+From reliable feedback to sustained progress and affordable context
+The useful question after a long agent run is what improved after the first good result, whether the improvement survived, and what the entire process cost.
+Our EdgeBench research has moved through three stages: make evaluation and continuation reliable; examine how accumulated evidence changes later work; then reduce the cost of delivering that evidence. Early Lean results and later Ad Placement traces show real progress. The strongest foundation-model baselines have advanced too. These observations support continued development of LoopX, while leaving the contribution and net benefit of individual mechanisms open.
+Our current judgment is encouraging but bounded. We have examples of useful late work and a more direct recovery/replanning flow. We do not yet have a general causal improvement estimate or a complete fee comparison.
1. What the control plane controls
-A long-horizon agent maintains at least two histories. Conversation history records what was said and done. Task and evidence state records what must still hold, under which conditions a finding remains valid, and why a next action is feasible. These histories complement each other. Resuming a conversation does not make an old conclusion applicable; completing a Todo does not establish that the original goal has been achieved.
-We call LoopX’s approach a long-horizon semantic control plane. The goal and acceptance conditions constrain work; current evidence supports or challenges a route; results lead to continuation, replanning, waiting or stopping. The model proposes and interprets choices. The control plane preserves provenance, checks state transitions and returns the obligations that still apply to the executor.
-- GoalAcceptance, scope and stops
- SelectionFeasible candidates and reasons
- ActionWork, validation and results
- EvidenceObservation, interpretation and scope
- RevisionContinue, switch, wait or finish
The guarantees differ across this cycle. A system can verify that work has a matching settlement, evidence names an input revision, and a rejected write leaves no partial effects. Field validation cannot establish that the model understood a qualification or chose the best route. Protocol correctness can be checked directly; cognitive benefit requires behavioral and outcome evidence.
-Runtime, Todo counts and graph size therefore cannot stand in for long-horizon capability. We want to know whether the system preserves constraints while rejecting bad routes sooner, reusing validated results and allocating its remaining budget well. Reliable evaluation, recovery and context delivery belong in the same study because each can break that chain.
-Research question: can experience improve later choices?
-Research question and comparison methods updated October 10, 2026. This revision adds no experimental scores; all numerical results below retain their stated historical versions and observation windows.
-We want to test whether, with the same model and comparable budgets, a general harness helps an agent accumulate and use experience well enough to keep making verifiable progress later in execution. A failed route should leave more than “failed before”: its conditions, what it ruled out and what new evidence would justify retrying. Recording that finding, changing a subsequent choice and improving the outcome require separate checks.
-A main session that continues to interpret the goal and evidence is one way to study this question; cross-session recovery tests reliable state transfer. Session continuity alone does not establish better use of experience. Fresh-context execution and workflows can also preserve experience through files and external state. Comparisons should record the actual session strategy, available information and handoff cost, letting results determine applicability. EdgeBench Section 5.2 and Appendix G provide relevant comparisons of sustained execution, independent retries and harnesses; they do not isolate LoopX’s incremental benefit.
-The 2026-10-16 continuation is integrated below: goals and recovery, evidence follow-through, whole-episode cost, planned/stride, pre-decision checkpoints and real contrasts, and the next study design. The opening and earlier numbers retain their original cutoff.
-2. Separate two stages of research
-Early Portfolio and Lean studies tested feedback, sustained work and evidence adoption; later studies inspect choices from mature research points. The table retains the early five modes, not a current launch matrix. Section 9 preserves the earlier stride-on comparison; Section 10 separates the October 17 stride-off study from later Sol old/new, Astra Official/LoopX and model-switched continuations.
-| Execution mode | Mechanism retained | Question | ||
|---|---|---|---|---|
| Official worker | Official stop hook and continuation | How far does the official execution path progress? | ||
| Single Codex call | One call, ending naturally | How much happens without an external continuation loop? | ||
| Native Goal | Codex goal execution and continuation | What does native long-horizon execution provide? | ||
| LoopX Resume | Same-session recovery, Todos, replanning and settlement | What do semantic control and reliable continuation change? | ||
| LoopX Explore | Resume plus optional evidence capture and exploration planning | Does structured evidence enter the next decision? |
| Stage | Question | What the evidence can establish |
|---|---|---|
| Feedback and continuation | Can we trust results and keep authorized work useful? | Evaluator defects, recovery failures and common-window system progress |
| Later decisions | Does accumulated evidence improve the next experiment? | Decision-to-work traces and changes after a mature research point |
| Context efficiency | Can we deliver the same obligations with less complete cost? | Actual model input, recovery, tool calls and outcome/cost comparisons |
The historical Portfolio design covered five workers × native/blind feedback, ten arms. Later work added selective ablations. Lean primarily uses blind runs. Native workers may obtain judge feedback. Blind workers retain public tasks, training data, local tests and validators, but receive neither judge scores nor hidden diagnostics. Background evaluation is an operator observation and is not returned to the blind solver. Blind means no judge feedback, not no feedback.
-Earlier Portfolio/Lean and October 17 studies use gpt-6.1-sol / xhigh; later cohorts state their own model and effort. Historical figures retain their original budgets, sampling, scorer and date. Graph/Portfolio continuation preserves consumed time, opportunities and cooldown within its cumulative 36-hour ceiling, not a fresh allowance per donor. A shorter observation window does not establish normal completion of a planned run.
Earlier work compared seeded phase Todos with goal-derived planned execution, review cadence, context and Explore. Later research selects planned + Explore and explicitly reviews every six settled effective work Turns. This is the current study setting, not a claim that old three-Turn runs used it or that explicit product units changed. Prompts, runners and contracts co-evolved, so early planned results do not establish a general advantage on their own.
-Early overall progress, with its original windows
-Lean provides direct common-window evidence. Under blind conditions, valid samples from early LoopX v14 and that-stage #5844 runs are shown below. More verifiable proof progress in the same time is a real observation, even before its component causes are separated.
-| Lean raw score | Early Resume v14 | #5844 Resume | Early Explore v14 | #5844 Explore |
|---|---|---|---|---|
| 180 minutes | 0.1103 | 0.3478 | 0.1134 | 0.4019 |
| 450 minutes | 0.1882 | 0.5243 | 0.1996 | 0.5605 |
In these single-run windows, the then-current LoopX paths reached roughly 2.8–3.5 times the earlier raw scores. This ratio describes observations. Prompts, task entry, runners and control mechanisms changed together; it is neither the causal gain of one feature nor a result to extrapolate directly to other tasks.
-Portfolio improved differently. Early persistent zeros included feedback-wrapper and accounting-contract defects. After repairs, blind arms produced valid ledgers, conducted local research and obtained usable evaluations. Older Resume reached a best sampled 17.32; later planned v17 reached 25.39, but its last sample was only 12.38, versus 15.57 for older Resume. Budgets and scoring revisions also differed. At roughly 7h45, that-stage #5844 Resume/Explore had best sampled scores of 15.01/12.21 and latest samples of 8.15/9.67. Evaluability, continuation and some best observed outcomes have moved beyond early obstacles; terminal quality and sustained retention of good solutions still need work.
-This supports a more effective overall system while leaving two questions open: where did the gains come from, and how do we prevent regression with a new version or more runtime? Improving benchmark execution—shared iteration guidance, correct recovery, task-specific sampling and budgets—is itself part of long-horizon systems engineering. Its contribution should be recorded rather than attributed entirely to the graph or replanning.
- -3. First make the feedback trustworthy
-The early Portfolio blind zeros exposed two different problems. Our blind wrapper had removed iterative optimization and best-version guidance along with the judge entrypoint, confounding feedback availability with behavioral instructions. We repaired the wrapper and started new attempts while retaining the old results. Independent reconciliation of completed samples then exposed differences between the evaluator’s assumptions and actual execution semantics.
-A tiny counterexample can invalidate a whole curve
-Start with 100,000 cash. An asset rises from 100 to 200. Buy 100 units only at the second day’s close and pay 10 commission. Valuing them immediately at that same close gives equity of 99,990 and NAV of 0.9999. The account did not earn the price increase before it bought.
-The old reconstruction multiplied post-close weights by the day’s return and produced approximately 1.199919992, rejecting the correct account. In the opposite direction, a true all-cash path [1, 1, 1] reported as [1, 1.01, 1] passed an endpoint-only check. The first counterexample proves false rejection, the second a local check’s blind spot. The latter does not prove that the complete judge accepts fabrication.
-
For two frozen terminal samples, independent reconciliation without importing solver calculation functions matched all 243 trading days in each: equity residual below 4×10⁻⁹, NAV residual below 5×10⁻¹⁶, and no share-count difference. This established those ledgers’ consistency, not strategy quality. Seven groups of matching trade signatures in another sample corresponded to separate cash, inventory and cost effects. A duplicate signature can trigger investigation; it cannot replace economic verification.
-Key technical conventions in ledger-v2
-The versioned task fixes initial cash at 10 million with no holdings, close-time execution, 5 bps commission and 1 bp cash slippage. Borrow cost accrues on prior-close short market value at 4% annually using ACT/365 calendar days. Crossing zero inventory becomes ordered execution legs. Cash, integer shares, costs, equity, NAV and weights are validated daily.
-Reconstruction starts from executions and prices, not reported cash or NAV. Negative cases cover missing costs, repeated execution identities, extra trades without economic effects, invalid inventory transitions, altered intermediate NAV, missing dates and non-finite numbers. Positive cases include legitimate equal-sized fills, weekend borrow and no-trade days. Rule revisions are versioned; they do not replace historical scores. See EdgeBench #14.
-Better feedback does not guarantee a better strategy
-The single blind ledger reconciled, yet removing the incorrect NAV penalty still left an approximately −1.87 subtotal, clipped to zero. Removing a duplicate-signature penalty in another sample would yield a counterfactual 18.05. We did not record that as a new official score.
-Cost attribution needs economic quantities rather than counts. Of single blind’s 475 fills, 281 concerned hedging, but their commission was only about 0.0412% of initial capital. Borrow cost was about 0.4484%, most of the total 0.7129% cost. Hedging is a task requirement, so deleting shorts is not a valid shortcut. A testable question is whether strategy selection and subsequent hedge adjustments share compatible cost and risk objectives on public training data.
-This repair has substantial value with a precise meaning: it helps separate capability from measurement error. It is not a LoopX solving gain. Nor may terminal hidden diagnostics become tailored advice to an active blind worker.
-4. Goals, work items and execution need different stopping rules
-The numerical tables above retain the initial draft’s evidence cutoff. Added mechanism and closeout observations below extend through 2026-10-08 09:33 UTC. They remain exploratory, change no formal score eligibility and do not replace complete terminal acceptance.
-One concrete failure occurred when seeded-blind wrote no_followup after roughly 34 minutes. The outer process resumed about 100 times without restoring useful work. A live scheduler is not a runnable semantic path. In another failure pattern, the system already knew replanning was needed but first offered a predictably invalid choice, forcing a failed call before revealing the next step.
No existing Todo does not imply that the original goal is complete. After finishing a work item, the agent must compare the result with the original acceptance: is it satisfied, does a reasonable next step remain within authorization, is there an external dependency, or is there truly no useful continuation? Replanning may produce work or justify staying on the same route. It should not invent tasks merely to keep a loop moving.
-Review completed work turns, not just completed Todos
-A long task can remain unfinished for hours, so review should track actual settled work. The effective-Turn repair replaced implicit Todo-completion counting while preserving explicit legacy units. The new study in Section 10 uses six effective work Turns; early three-Turn and then-product five-count settings describe their own revisions. A due review requires reconsideration, not a route change or new budget.
-A Turn here is neither a model message nor a tool call. Cadence comes from uniquely settled work: uncommitted refreshes, retries and duplicate records must not inflate the count. Reaching the interval asks for a review, not a mandatory route change, and grants no budget or authority. Real entrypoints have been tested to trigger while a Todo is still open and to resume work after a justified continuation. Net benefit depends on what the model actually decides.
-The counter must also distinguish work from observation. A validated negative result can be effective work; monitor-only polls, duplicate submissions and unsettled attempts do not count. This lets a long-open Todo receive periodic review without letting background sampling trigger pointless replans. A review should connect the original goal, candidate evidence and the next action. Choosing to continue can be correct; acknowledging that a check occurred does not establish better research direction.
-Review cost also needs a breakdown by trigger. Periodic checks, a remaining goal gap after a work item ends, and new evidence invalidating a key assumption may need different depths of review. Changing the interval affects only one category. If every small completion rebuilds the global plan, a longer interval may not solve the problem. Conversely, a replan phase can contain useful probes and validation, so its entire duration is not administrative waste. Traces should distinguish useful route revision, repeated evidence gathering and mechanical bookkeeping before testing the boundary between local continuation and heavier global review.
-Recovery and settlement are distinct chains
-Session recovery preserves execution context. Settlement connects the current Turn, selected work, validation result, durable writeback and budget use. A retry after a lost response needs to recognize existing effects rather than execute or count them twice. Restoring a session without settlement can revisit the same obstacle; correct settlement with the wrong session can lose the solving context.
-Earlier repairs addressed exact session recovery, next-action/receipt binding and invalid continuation diagnosis. Recent work adds checkpoint-selection and lifecycle recovery. Resume from the original Turn and committed effects: a lost reply must not duplicate execution or spend, and independent new work requires a fresh legal-path judgment. Correct continuation and useful work selection remain separate acceptance questions.
-Keep the original goal visible before selecting work
-The following status is updated through 2026-10-17, preserving historical windows while separating source, actual research adoption and effects.
-Recent inspection moved beyond whether an agent read a Goal to which goal actually arrived. The original task remained in prompts and files, and agents did read it; some newly created structured benchmark Goals nevertheless carried only generic research reminders. Work could satisfy that reminder while drifting away from the task’s main metric. A separate projection defect borrowed public-display filtering for scoped input: paths or secret-like strings could drop a whole goal paragraph, and research text could be truncated.
-#6649 preserves the original task objective and priorities in new Goals and complete source text in authorized same-Goal replan input; public export boundaries remain separate. #6654 makes that shared goal projection a default-off, explicit opt-in in LoopX and its benchmark runner. When enabled, the planning horizon shows the original Objective, provenance, separate acceptance and readback before work selection. A source correction appears on the next read. Disabling it preserves the existing horizon behavior; the Goal input already supplied to replan is independent of this switch. It creates no second persisted goal and does not rely on repeating a slogan every Turn.
-The original-goal repair #6649 and horizon switch #6654 are both merged, with configuration-editor, CLI readback and feature-off validation. The new same-point research in Section 10 explicitly enables the shared Goal horizon and reads the original objective in current reviews; old checkpoints are still not rewritten. An objective reaching work selection does not guarantee correct prioritization or explain every earlier low score. Follow-through must ask whether a complete experiment reduces the task’s main-metric gap.
-Why failed wakes did not end the whole experiment
-In the older Graph stride-on/off comparison, outer wakes failed 20/99 and 14/104 times; successful wakes also recorded 35 and 45 internal stage errors. Failures clustered around review, checkpoint selection and unsettled-state recovery. The outer runner could wake again, while committed candidates, native results and some checkpoints survived. Work therefore continued, but could repeatedly recover the same decision, delay useful action and consume research or capture time.
-Recorded failed-stage time was about 49 and 35 minutes, not total loss; 20/99 does not mean 20% of useful work was wasted. Unequal fault burdens confound an overall comparison. The selection-hop repair and lifecycle recovery repair address distinct breaks: preserve the original Turn, recognize committed effects and restore a legal route when independent work arrives. They add no submissions or cooldown resets and do not retroactively convert failures into passes. New studies need actual adoption of the repaired source.
+Comparison boundaries
Historical studies include Official, single-call, native Goal, LoopX Resume and LoopX Explore. Explore adds optional evidence capture and planning to continued work. Blind runs retain public tasks, training data and local validators but receive no hidden judge scores or diagnostics; background operator evaluation is not solver feedback. Later native-feedback studies use agent-requested evaluation, a one-hour accepted-submission cooldown and no periodic grading. These are different studies, not one pooled treatment.
Models, effort, prompts, runners and budgets co-evolved. Early studies primarily used Sol/xhigh; later Sol/max and Astra/xhigh cohorts retain their own conditions. One run per arm is descriptive evidence. Missing, stopped, invalid and valid-zero results stay distinct. Source time, evaluation opportunities and cooldown survive checkpoint continuation; donor history is not a new independent repetition.
5. Explore must carry evidence into an actual choice
-We observed enabled Explore hooks on all 20 reads in two groups’ first 180 minutes, while their graphs remained empty. The model was still discovering things: a counterexample to directly inheriting a tail bound from a finite prefix remained in local validation notes. The missing path was from that result to a scoped graph record, task association and later use. A higher read quota cannot fix a write that never happened.
-The response was more than a reminder to write a graph. An optional capability hook attaches capture to ordinary result writeback. The existing observation and route decision can be reused with an explicit question, applicability, input revision and finding status. A linked question can reuse its canonical scope. A low score cannot automatically imply refutation, and pausing a route does not mean it is forever wrong.
-- Capture: was a useful result saved, with recoverable rejection and retry?
- Association: did it bind to the current question and Todo with its conditions and version intact?
- Retrieval: did the later decision receive it without crowding out other relevant counterevidence?
- Adoption: did the model use it to change or explicitly retain a choice, followed by consistent action?
Node counts cannot certify this chain. A later audit of four frozen runs found one and three findings in the two Explore arms, but no graph associations on seven inspected Todos. Two first writes lacked node_id; the model tried creating nodes, encountered scope conflicts and eventually recorded findings manually. Nonempty graphs had still not established the full path.
#5889 addresses that friction. A new question derives stable identity from explicit question and applicability; conflicts identify mismatched fields. Existing scope stays immutable, rejected inputs leave no partial effects, and retries do not duplicate records. Writes remain subject to the current Todo, claim and lease. This repairs identity and recovery without inventing scientific findings. Real leased writeback and readback are validated; supplementary runs now also contain natural capture, association and delivery in a later Turn. The graph’s incremental value over files or the same conversation remains unisolated.
-Retrieval also needs progressive access: cover distinct questions first, then offer more records per question, with revision-bound pagination and question-specific expansion. Three initial entries can be a sensible context budget; a hard information ceiling of three can discard decision-relevant evidence. Selection and larger access must be assessed together.
-Counterevidence: memory can work without a graph
-Portfolio traces include retained results across 12 variants, five training years and four warm/cold checks. Another search required training Sharpe at least as high as baseline and turnover no higher, rejecting all nine candidates. That need not be a good research policy: all candidates had lower training returns and four had worse cold-start behavior, suggesting a limited search may have stayed too conservative.
-Current Lean Explore also preserves hard cases, records local counterexamples to false statements and moves to other chapters. Files, Todos and the same conversation can carry this information. To establish the graph’s incremental value, we must identify an additional avoided repetition, retained qualification or changed choice. It cannot take credit for all memory.
-Evidence, adoption and actual intervention need separate checks
-One supplementary Portfolio sample records a risk finding, then implements and tests tail protection. On this period’s actual inputs, however, constrained and unconstrained allocations remain equal. A finding reaching code and passing stress checks does not establish an intervention in this task. A Lean chain proceeds from periodic-review candidate declarations to installing proofs, a full build and later batches. That is closer to adoption evidence than a node count, but still does not exclude the same conversation and local files supplying equivalent information.
-A boundary failure is informative too. A supplementary Lean run rejects a ninth question association for one Todo because the active association set is limited to eight. It preserves existing associations, records the extra conclusion as standalone Goal-scoped evidence and resumes ordinary work. This limits one task’s association set, not the whole graph to eight findings. Recovery is not free. Reducing capture friction should preserve relevant decision evidence rather than silently deleting old counterevidence to make a write succeed.
-Review the immediately preceding decision
-Evidence existing in a checkpoint, a model having read it, and current replanning being able to use it structurally are different facts. We saw a new Goal’s Explore projection report an empty current graph while the archive retained historical research. That requires source/adoption repair, not concluding that experience is absent. Later continuation qualification checked research state, the original goal and actual Explore delivery; mature examples contain dozens of work Turns and over a hundred evidence events. This supports recoverability, not research quality by record count.
-In one historical Graph audit, five of ten subsequent reviews still cited an older decision. This can skip the experiment just completed and keep reviewing a stale route. Repair in the existing history-sequence and stride owners retrieves the immediate predecessor in all ten prefix replays and rejects five stale references. It establishes the right review target; the model must still judge execution, new information and progress toward the original goal.
-A discriminating record connects previous choice → actual candidate version → completed experiment → public feedback → next adoption or rejection. Explore’s incremental value belongs on that chain: which condition did it preserve, repetition did it avoid or choice did it change? A node, a switch label or a branching graph alone proves none of those.
6. Optimize the information path needed for a decision
-One clear redundancy was an exact todo list --todo-id … response repeating the body in detail, list and role views. But traces showed that exact reads were not the dominant path. Models also obtained Todo information from Goal documents, quota summaries, batched commands and prior conversation. Shortening one command could not explain or solve the whole cost problem.
#5844 therefore converged on one goal: deliver the effective current work once, with progressive access to remaining sources. Ordinary heartbeat/quota returns the complete selected Todo, applicable acceptance, scoped open User obligations and participating hook content, without requiring TurnEnvelope. Already delivered sources are removed from pending reads. The mixed Goal document, which contains intent, other work and history, still requires an exact full read.
-| Path | Before | After |
|---|---|---|
| Exact Todo | One body repeated across three views | One complete canonical record; inventory queries remain inventories |
| Ordinary quota | Work summary plus follow-up commands | Canonical current sources once, plus reads for remaining sources |
| Optional capability off | Generic skill could still induce useless reads | Participating hook owns the guidance; no routine off-state obligation |
| More background and evidence | Repeated dumps or only short summaries | Provenance, complete reads and expansion by question/revision |
Concrete same-fixture measurements and their limits
-Exact Todo output fell from 15,986 UTF-8 bytes and three body copies to 2,721 bytes and one copy. Ordinary quota grew from 23,088 to 27,137 bytes against the command-only version because it carried actual content. Pending sources shrank from preference + Goal + Todo to Goal. This does not establish lower total tokens or latency: if old reads were already batched, removing a CLI process may remove no model round trip.
-In 12 frozen real wakes, all received the complete Todo: nine through inline or file-based guards, three through exact reads. Eighteen inspected bodies matched their startup source digests. That establishes delivery and source consistency, not comprehension, complete constraint adherence or total cost benefit. The earlier roughly 78% synthetic compression claim likewise applies only to its measured output.
-The current study’s information organization follows one real decision: complete current-work constraints arrive directly; a same-source original goal anchors selection; additional history and evidence expand by relevant question and revision. Do not mechanically reread an already delivered source. Missing content, failed sources or changed versions require complete reads through the current contract. Authorized research input must not lose its objective or counterevidence to public-display filtering. Optimize the whole decision path rather than one JSON length.
-Optional capabilities must not leak through generic guidance. Repeated semantic-preference reads induced by a generic skill while the capability is off are a boundary defect; #5874 returns that guidance to the participating hook. Smaller packets and fewer useless prompts help, but must preserve acceptance tails, applicability, continuation conditions and failure recovery. The optimization unit is the decision path, not one JSON length.
-Tool granularity should match a meaningful operation
-A direction worth testing is to put state and recovery complexity inside semantically complete tools with clear boundaries: obtain all constraints needed for current work, for example, or submit a result with its source and applicability. A tool may perform several internal checks while the model remains responsible for interpreting evidence and choosing a route. Anthropic’s tool-design guidance likewise organizes interfaces around real work rather than exposing every underlying API to the model.
-Fewer commands alone are not the objective. Consolidation must preserve permissions, visible failures and retry semantics. Real execution must show whether an interface is easier to learn, produces fewer errors and turns saved interactions into useful work. #6038 improves action-specific CLI help and recovery from lease-argument errors. That repairs a usage path; it does not yet establish better task outcomes or lower whole-run cost.
-Measure the complete research episode
-A recent Portfolio fixed 2×2 check covered 14 local cases and 56 result rows: five public cases and nine synthetic cases. The factors were original V7 versus adaptive covariance, and truncating total target shares versus truncating the requested share change. The following means use only the five public cases, hold V7 fixed and change only the execution rule to share-change truncation. Mean fills fell from 451.2 to 235.4, about 47.8%; costs fell only from 62,683.15 to 62,682.18, about 0.00154%. Mean Sharpe rose just 0.000133. Separately, drawdown worsened in ten of all fourteen cases. A large change in operation count produced a tiny change in the goal. Fewer fills became a candidate mechanism whose value needed comparison with actual headroom.
-The solver reported about 25.25 CPU seconds for local computation; the two complete wakes containing this research took about 29.66 minutes, ran 90 commands and emitted roughly 68,000 output tokens. A separate completed covariance episode took seven wakes, about 68.05 wall-clock minutes and 144,700 output tokens. These include reasoning, control, authoring, tools and waiting, not pure decoding time. Some diagnosis carries information and is not all management waste. Reporting only a small script’s seconds nevertheless misses the main cost relevant to continuation.
-Measure the full episode, then distinguish active work, recovery, waiting and native evaluation. Difference and deduplicate cumulative session usage rather than adding ancestor history repeatedly. Faster local computation, fewer commands or smaller packets count as efficiency gains when they preserve decision information and improve useful output or complete cost.
-Fewer hints, with the actual callers using one factual source
-The October 24 work separates read facts → choose a route → read the writeback format. Tactical recipes recede; the original goal, cumulative requirements, public feedback and execution boundaries remain. A reader existing in source is insufficient: an earlier nominal treatment still received the old guard and planning prompt. A later Host audit also found a full-source instruction and duplicated settlement plans. Proposal #7269, reviewed at 0ddd08ad, converges CLI and native Host/journal delivery, keeps the core goal and current horizon explicit, and delivers the settlement plan once. Signed original source remains available for verification and recovery; old captures without a safe reader retain compatibility. It reuses the existing context-delta owner, without another cache or reading acknowledgment. This proposal has not been adopted by the frozen runs below.
Whole read-sequence transport: smaller on one path, larger on another
-Four fixed Schema/Grid historical snapshots compare the previous proposal with the reviewed revision. UTF-8 totals include entry, review, every advertised acceptance detail, Explore where applicable, and writer. Recovery additionally reads the complete original source. This is offline transport, not model tokens or a benchmark effect.
-| Read path | Before, bytes | After, bytes | Change |
|---|---|---|---|
| Captured CLI | 550,039 | 562,429 | +2.25% |
| Structured Host | 622,317 | 263,115 | −57.72% |
| Native renderer | 647,687 | 257,599 | −60.23% |
| Native with full-source recovery | 647,687 | 688,962 | +6.37% |
The native path replaces one full-source read with two or three staged CLI reads. Tool boundaries and recovery can offset a smaller normal input. Local read timings collected alongside validation are not controlled latency measurements; external evidence-tool output, model caching, output and billing remain unmeasured.
-A separate earlier settlement diagnostic alternated eight paired samples per backend after warmup. SQLite manual/composed median CLI time was 0.807/0.995 seconds; File was 0.796/1.018. Output bytes fell about 46%, while paired process time grew about 24%. One operator call still launched two complete CLIs and an authoritative readback. Possible model-interaction savings remain unmeasured; historical time between two tools also includes reasoning and validation.
-Cost coverage matters too. In the finite Sol/max continuations below, response-level reconciliation found three Schema and two Grid usage records missing from native notices next to compaction: 23,740 and 14,619 output tokens. Formal replan occupied 7.35% and 8.94% of the outer clock, but about 21% elapsed before the first observed native task. Preparation includes recovery, validation, repairs and startup gaps, not only replanning. The reference of 15% for all mechanisms combined remains unqualified: useful research, control, failures, shared resources and actual fees have not all been attributed.
+2. First make feedback and continuation trustworthy
+The first Portfolio experiments exposed measurement and execution defects before they could tell us much about strategy. A blind wrapper had removed iterative optimization guidance along with the judge entrypoint. A ledger check attributed earlier market returns to holdings bought only at the closing price. Both confounded the intended comparison; repairing them required new attempts while preserving the old results.
+One small oracle made the accounting error clear. Start with 100,000 cash, buy 100 units at a closing price of 200, and pay 10 commission. Immediate equity is 99,990 and NAV is 0.9999; the new holding did not earn the preceding rise from 100. Independent reconciliation then matched all 243 days in each of two frozen samples, with equity residual below 4×10⁻⁹ and no share-count difference. This established ledger consistency, not investment skill. The versioned repair is documented in EdgeBench #14.
+Continuation had a related failure: an outer process could wake repeatedly after a work item ended without recovering a useful next action. The governing question must return to the original goal. Completing a Todo can justify another experiment, waiting, or completion; it cannot by itself settle goal acceptance. Correct recovery and feedback make subsequent research interpretable.
+Early system progress was measurable
+| Blind Lean raw score | Early Resume v14 | Later Resume (#5844) | Early Explore v14 | Later Explore (#5844) |
|---|---|---|---|---|
| 180 minutes | 0.1103 | 0.3478 | 0.1134 | 0.4019 |
| 450 minutes | 0.1882 | 0.5243 | 0.1996 | 0.5605 |
These single-run common windows show roughly 2.8–3.5 times the earlier raw scores. Prompts, task entry, runner reliability and LoopX mechanisms changed together. The observation supports progress in the overall system; it does not identify one mechanism’s causal effect or guarantee terminal performance elsewhere.
7. Lean’s historical counterexample: slowdown and candidate allocation
-The improved #5844 Resume and Explore arms progressed quickly early on, then were overtaken by historical native Goal in the middle. We analyzed common 180–360 minute windows using solver-visible tool results and frozen source snapshots. Those arms share c0d519cf; historical Goal used c46f397c. Model and public task digest match, but dates, runners, prompts and budgets differ, with one run per cell. This is route diagnosis, not a clean causal framework comparison.
| Blind arm | Raw score at 180m | Raw score at 360m | Net sorry reduction | Files affected |
|---|---|---|---|---|
| Resume / planned / #5844 | 0.3478 | 0.4919 | 502 | 17 |
| Explore / planned / #5844 | 0.4019 | 0.5147 | 442 | 18 |
| Native Goal / historical | 0.3013 | 0.6477 | 1,015 | 41 |
| Historical cohort | Observed result | Interpretation |
|---|---|---|
| Sol/xhigh, old/new LoopX, Ad Placement · 14h | 49,589,589,573 → 49,615,495,839 | Newer run trailed at 2h and led at 14h; its terminal workspace matched its retained best |
| Astra/xhigh, Official Grid · 6h → 12h | 1862 → 1810, lower is better | A stronger foundation baseline still produced useful late improvements |
Only valid returned samples are shown. Raw scores use the 0–1 representation without multiplying by 100. Source sorry tokens exclude strings and comments; they are neither proven-theorem counts nor integrity qualification. These rows retain the original common windows regardless of later runtime status.
-Proof investment and immediate score differ
-The task checks original declarations’ axiom dependencies and types against 2,901 baseline unqualified declarations. Removing a sorry, adding a helper, compiling locally and gaining a valid scored declaration are different events. Bridging custom definitions and Mathlib may pay off downstream. If a dependency still contains sorryAx, a local edit need not earn proportional credit. Missing assumptions cannot be fixed by silently changing required signatures.
Current runs mostly advance chapter by chapter, reaching limits, series, rearrangements and extrema in the middle. A Resume Appendix B.2 Turn spent 16.6 minutes filling six holes. An Explore Section 8.5 Turn spent 31.5 minutes filling 77, leaving a hard case and recording counterexamples to two original false statements. These spans include proof work, validation and settlement; they are not pure proving time. Some useful work does not immediately become positive score.
-The historical baseline points toward candidate allocation
-At 180–360 minutes, Resume’s reductions mostly came from Chapters 5/6 (213/223), and Explore’s from Chapters 6/7/8 (223/79/140). Historical Goal also advanced Chapters 9/10, Misc and MeasureTheory, accounting for 344 reductions in those four areas. It progressed in 41 files versus 17/18. The arms were not simply doing identical proofs at different speeds; they were choosing different work.
-“LoopX never switches routes” is also contradicted. At roughly 197 minutes, Resume explicitly revised surjectivity/order-preservation bridging; around 252 minutes it recognized statement restrictions and switched from a foundations Todo to later chapters. Explore retained rearrangement problems and moved to Section 8.5 near 325 minutes, then preserved counterexamples and moved to 9.1 near 352. Review and substantive decisions occurred. What remains unproven is systematic comparison of feasible work packages and recent validation cost to improve allocation.
-The evidence does not show growing management packets
-Differences between monotonic session-usage endpoints give roughly 303k/289k output tokens for Resume’s first/second three hours, and 317k/292k for Explore. Tool events containing LoopX calls shrank from 1.14MB/0.72MB to 0.47MB/0.33MB. Those events can include other operations and are a transport proxy rather than exact control overhead. They nevertheless do not support the specific explanation that packets grew in the middle.
-Successful full-build logs also showed no increase in summed declaration-audit job duration: Resume 318/257 seconds, Explore 181/119, with samples of 14/12 and 7/5 respectively. Evaluation queues for the corresponding 180/360 minute snapshots took roughly 1.7–3.4 seconds, with evaluation lasting 7.2–11.2 minutes. Delayed display can leave temporary gaps, but cannot explain the slope between those completed capture points. Failure retries and all validation cost are not fully covered.
-The next question is therefore more specific than increasing quota or review frequency: can the agent compare routes, distinguish necessary dependencies from independently tractable work, and preserve revisit conditions for difficult cases? The control plane should make those judgments inspectable without hard-coding Lean knowledge or hidden scores into generic rules.
-A curve has three clocks
-Solver elapsed time, source capture time and score return time are different clocks. Lean’s full build and declaration audit take time, and sampling can queue. The latest returned score may therefore describe source from hours earlier. Placing it at the current runtime can make evaluation backlog look like a research plateau.
-We align common windows by original capture time, retain queue and evaluation duration, and separate automated samples, active submissions and final artifacts. A native active submission cannot fill a blind auto-sampling curve; missing is not zero, and the best sample cannot replace the final delivery. Completed common windows still show the new supplement behind #5844: queueing cannot erase that counterevidence. Conversely, returned later #5844 samples still gain credit, so mid-run slowdown alone does not establish an absence of late potential.
-Validation and model costs need separate denominators as well. The official repeat also shows concurrent full builds and probes, build interruption and reduced thread settings: validation contention can occur without LoopX. Output tokens divided by task wall time include thinking, tools and waiting; this is not a provider’s pure decoding rate. The evidence supports examining work selection, validation organization and model time together, while their independent contributions remain unresolved.
+Ad Placement provides a concrete instance of later progress rather than a universal plateau. A worse subsequent candidate was rejected and the best version survived. Its raw-score lead is modest—about 0.052%—and only that arm has the cited terminal-workspace verification. The old/new packages changed core and runner together, so this is not an isolated LoopX-feature effect.
+The Astra result expands the foundation-model frontier in our observations. Official separated speed changes from search changes and obtained successive native improvements. That progress belongs to the foundation/Official path; it cannot be credited to LoopX. Stronger models also raise the standard for a useful harness: context and review costs must earn their place against an improving baseline.
+Schema: an initial lead is not sustained improvement
+Schema is the detailed counterexample. In the Sol old/new 14-hour window, the newer run scored 0.804620 versus 0.723684. It already led at the first native result, 0.794620 versus 0.713158, received at different elapsed times. Subsequent increments were only 0.010000 and 0.010526. The larger endpoint advantage mostly reflected the initial result; continuing code changes did not close the six original target requirements.
+A separate Astra/xhigh 12-hour comparison was less favorable: Official reached 0.881813 and LoopX 0.786257, with neither improving its best between 6h and 12h. LoopX recorded 36 completed compactions versus four, and about 5.32 times the uncached input. Compaction can overlap useful work, and versions, recovery, shared load and scorer conditions differ. The trace still identifies the practical challenge: producing more changes and more control activity did not produce more accepted value.
+Conditions and limits of these windows
The Sol old/new study used planned + Explore, Sol/xhigh, TurnEnvelope off, agent-requested native feedback and a one-hour cooldown. The 14-hour window was selected after earlier observations as the largest whole hour below the earliest actual termination; it was not an 18-hour completion. The separate Astra study compares each arm’s own first 12 outer hours, including preparation. The Sol window reconciles 7,522 unique responses and 52 received native results; Astra reconciles 5,277 responses and 45 results. Cumulative counters were not repeatedly added; reasoning is included in output. Historical infrastructure failures and unknown whole-run integrity remain recorded. Different treatments, adaptive restarts and a prior operator exposure to an Official terminal aggregate prevent a clean causal verdict.
8. Assessing the improvements
-Improvement and technical depth are separate axes. A deep contract repair may eliminate a rare dead loop; a small instruction repair may immediately release substantial work time. We assess the gap removed and its evidence, not PR counts or passed-test totals.
-| Boundary | Currently verified | Still unproven | ||
|---|---|---|---|---|
| Goal and feedback | Reproducible ledger misjudgments repaired; full Goal input and same-source horizon have CLI, installed and current-model readback | Task prioritization and net benefit; old checkpoints are not migrated | ||
| Research state and recovery | Joint capture, fresh before review, original identity/opportunities/cooldown/deadline, mature-point continuation and further work are inspectable | Sustained stability as points grow, complete historical cost and a complete user-facing restoration journey | ||
| Evidence and review | Conditional capture, later delivery, immediate-predecessor review and real behavior chains; negative results and stale citations preserved | Explore beyond session/file memory; correct references yielding better choices | ||
| Planned and stride | Merged initial-quality/later-gain distinction; questions about retained best, goal bottleneck and actual implementation | Initial quality, later gain and cross-task stability after new-source adoption | ||
| Effect and efficiency | Graph quality gains outweighed by time penalty; Portfolio transfer counterexamples and a small best gain; episode costs decomposed | Repeatable net benefit in clean matched studies, beyond local speed or repair counts |
| Same-point Sol/max continuation | Baseline → revised | Boundary |
|---|---|---|
| Grid recovery/replan | 2 → 1 review wakes; 48 → 20 tools; 935 → 400 seconds | Review wakes through first joint preparation completion |
| Schema recovery/replan | 2 → 1 review wakes; 37 → 17 tools; 1055 → 520 seconds | The same milestone definition |
| Grid native result ↓ | 2171 → 2165, from inherited 2172 | Same new submission opportunity; small observed advantage |
Early system progress supports continued research. Recent work moves the question from sustaining execution to using early experience for better later choices. The changes and cases below separately assess inputs and judgment mechanisms, actual adoption, native outcomes and full cost. These levels cannot substitute for one another.
-Planned and stride: start well, then measure additional gain
-The recently merged #6621 restores an important distinction in shared planning: deliver a strong, coherent first artifact feasible within the budget. Do not manufacture a weak start and call repayment of that quality deficit long-horizon capability. Useful diagnostics remain legal, and preparation cannot postpone delivery indefinitely. Later improvement is anchored to the retained best artifact.
-Stride keeps its three axes: work scope, route distance and validation scope. Its questions more explicitly compare the remaining authoritative gap, limiting component and plausible headroom; check actual implementation of the proposed mechanism; and distinguish an unexecuted change, failed transfer and insufficient route potential. For a candidate’s own screening threshold, ask what goal it protects, how the incumbent behaves under the same control and whether the discrepancy warrants vetoing an informative native evaluation. These guide judgment rather than force a numerical algorithm switch every few Turns.
-The trajectories also contain more than small parameter changes: full covariance, class-prior and within-class ranking approaches differ in mechanism. The unresolved issue is whether they affect the factors limiting native reward and complete an informative test at proportionate cost. Better questions, correct evidence and larger experiments may help; actual adoption and later outcomes establish their value. Frozen old runs were not hot-updated, so a source merge cannot be credited with changing their choices.
+Both revised receivers completed the staged reads, writeback and original-Turn settlement in their first autonomous review. Baselines eventually completed too; their extra rounds included useful validation and additional evidence, so the entire difference is not redundant work. Grid’s revised choice led to deeper reuse experiments, independent equivalence checks and broader local validation before its six-point advantage over baseline. Schema found a real registry-lifecycle defect but its new native result stayed at the inherited 0.786257; the baseline had no new result at that cutoff.
+This is limited positive evidence for the flow, with a small Grid outcome advantage and neutral Schema feedback. It does not establish whole-task acceleration. The study used one pair per case, the same original checkpoint per pair, Sol/max, normal cadence three effective work Turns, and context differencing, TurnEnvelope and stride off. Preparation temporarily used cadence one until the restored semantic review and native Turn completed. It is not a study of repeated steady-state three-Turn reviews.
+Earlier retained/fresh-context checkpoint work on Graph and Portfolio asked a different question. Its source age, reconstruction charge and continuation phase stay within the original cumulative 36-hour ceiling. An earlier nominal “with #7197” arm never received the staged reader, so its score cannot establish that treatment’s benefit or harm. A later Astra-to-Sol continuation changes model and effort too. These histories remain useful diagnostics, but are not independent cold runs or interchangeable ablations.
9. What the earlier checkpoint comparison exposed
-2026-10-16 update: checkpoint continuation and trajectory diagnosis under frozen sources, not a new complete benchmark ranking. Historical versions and figures remain distinct. Those four earlier research workers were deliberately stopped; scores, trajectories and archives were retained. Section 10 describes the new study separately.
-A major recent engineering result is making mid/late research branchable and inspectable. Native execution history, LoopX research state, workspace code, incumbent, public feedback, submission opportunities, cooldown and cumulative budget must be saved at one safe boundary. Code alone loses the reasons for reaching this state; conversation alone loses executable state; successful snapshot creation does not establish restoration and useful continuation.
-- GoalSame source, constraints and remaining budget
- CapturePrior work settled; current decision not yet made
- ReplanRetained history, or fresh from shared evidence
- ExperimentImplement and obtain legal native feedback
- AdoptionKeep the best; measure net gain and full cost
Joint archive and recapture, complete preparation deadlines and fresh continuation before replanning progressively connect saving, restoration, evidence adoption and further work. Qualification includes isolated Linux, actual native HTTP/SQLite/Docker, ordinary installed packages and mature research points. Scripted models establish the mechanical chain, not model judgment. Concurrent receiver publication, native execution receipts and confirmed cleanup must also qualify together; one independently passing PR does not certify the final integration.
-One study-design correction was crucial: dump before replan, then let the fresh conversation make the new decision. Reviewing in the old conversation and asking a fresh one to execute that completed decision tests execution context, not fresh replanning; that option has been removed. Both conversations can access the same checkpoint research and original goal. The difference is retained native conversation history. Both arms in that earlier study enabled stride; Section 10 holds stride off. Neither comparison varies stride.
-Graph: higher classification quality did not improve the full goal
-That historical Graph continuation pair used frozen LoopX source 7b8b08f8. Native scoring also penalizes solution time. Higher classification/calibration quality need not improve the total. The three bound complete evaluations pass all four checks across ten splits, with no resource penalty:
| Graph raw score | Quality term | Time penalty | Final score | Mean solution time |
|---|---|---|---|---|
| Inherited incumbent | 0.753764 | 0.018622 | 0.735142 | 6.68 s |
| New retained-history candidate | 0.755289 | 0.022892 | 0.732397 | 10.47 s |
| New fresh candidate | 0.754762 | 0.028028 | 0.726734 | 17.89 s |
| #7269 common window | Uncached input | Cached input | Output |
|---|---|---|---|
| Grid baseline | 485,627 | 21,419,392 | 95,161 |
| Grid revised | 450,430 (−7.25%) | 28,378,752 (+32.49%) | 141,815 (+49.03%) |
| Schema baseline | 831,648 | 33,142,528 | 90,392 |
| Schema revised | 579,066 (−30.37%) | 30,620,288 (−7.61%) | 90,368 (about equal) |
Quality gains of about 0.001524 and 0.000998 were outweighed by extra time penalties of about 0.004269 and 0.009406. Both arms retained the original best. This locates the next question in the full quality–time tradeoff rather than a higher classification submetric. Single-run timing still depends on conditions; it does not show that fresh context is inherently slower.
-Portfolio: a larger route can still have insufficient headroom
-A full-covariance candidate raised mean Sharpe across five public training proxies from about 1.394 to 1.504, yet native Sharpe fell from 1.3731 to 1.3272. Its score was 16.94 against inherited best 20.53. Drawdown and turnover components already earned 15/15 and 5/5 in this sample. Tightening those constraints does not automatically create alpha or risk-adjusted return. A negative result should separate failed transfer, inactive intervention and insufficient route potential before selecting the next experiment.
-At this stop, Portfolio’s retained-history arm had best legal score 20.54 and fresh had 20.53: a 0.01 increase above the inherited best for the former. Neither Graph arm exceeded its inherited best. Real research, valid native evaluation and incumbent preservation occurred; the small difference establishes no general superiority for fresh, retained or stride. Older fault-affected cohorts, this historical continuation pair and the new study in Section 10 remain separate.
-A later audit found another input-path defect: checkpoint handoff requested a full-decision capture directory, but the heartbeat renderer also enabled the short TurnEnvelope view. All four recent arms record that implicit activation, so they cannot establish effects on the ordinary default-input path. Pre-decision preflight still retained the complete original packet and required it to be read; this does not show that historical goals or evidence were entirely lost, and shared exposure does not prove no effect. Merged repair #6661 separates capture from input projection, selecting the short view only through an explicit option. The new study in Section 10 uses the ordinary full-input path. Full capture, subsequent selection and actual native restoration still need joint validation; the fix does not erase the historical exposure.
-How these observations were checked, and what remains unreproducible
-The newer observations were checked by source type, rather than only by recomputing arithmetic printed in the article. The three Graph rows come from complete solver-visible native reports bound to candidate source. The fixed Portfolio study was recomputed from per-case, per-variant records, with digest links between the comparison file and execution engine checked. The scopes below keep denominators, time windows and validation levels separate.
-| Observation | Sample and verification scope | What it does not establish |
|---|---|---|
| Graph quality and time penalty | One incumbent and two new candidates, three complete public-feedback reports; ten splits each, quality minus time penalty equals final score, zero resource penalty | A single timing difference is not a stable causal context effect |
| Portfolio fixed 2×2 | Four variants across fourteen cases; reported means for the V7 execution-rule contrast use five public cases; drawdown counts use all fourteen | Local proxy gains do not establish native transfer |
| Old Graph failures and reviews | 203 wake receipts recounted by arm; eleven reviews minus the initial review without a predecessor leaves ten reference checks | A capture error does not prove a lost checkpoint; repair replay does not prove model adoption |
| Research-episode cost | Two complete Turns for the fixed study; seven complete Turns from covariance design through native feedback and rollback, inherited usage removed, active time and wall time distinguished | Script CPU time is not full research cost, and token counts are not prices |
The original archives and recomputation materials were supplied to an independent reviewer but are not yet published with this article. Public readers can check its numerical relationships and sample scopes, but cannot independently reconstruct every run from the article alone. Source-repair PRs and a passing page build do not close that data-access gap. A later data release should preserve these source links and negative results, rather than only the best scores.
-This observation covers the common 68m26s outer window through October 25, 15:28:30 UTC: initial recovery, review, ordinary work and persisted requests in unfinished Turns. Donor counters were reconciled at the receiver boundary and excluded. Last persisted requests differ by up to 37 seconds; later unpersisted usage is unknown. Grid spent more output and cached input on deeper work; Schema’s three recorded usage components did not increase. Neither the local review savings nor one declining token category proves net savings for the whole job.
+Reduce repeated delivery without losing obligations
+Staged reading and context differencing solve related problems. Staging orders decision evidence before writer instructions. Differencing reuses an exact baseline only when the native input observer independently verifies it in the current model input. Missing, compacted, truncated or uncertain baselines require full recovery of the same immutable capture. A file on disk or a completed page is not proof that the model retained the whole packet.
+Historical offline replay illustrates why totals matter: structured Host and native presentation bytes fell 57.72% and 60.23%, while complete CLI delivery rose 2.25%, and native entry plus full recovery rose 6.37%. These are historical byte measurements, not current-head model-token results. A 60% smaller view cannot be reported as 60% lower total cost.
+Our current efficiency work connects ordinary and staged delivery to the shared transport, reduces repeated reads and rebinding, and removes avoidable settlement round trips. The default with differencing off matters as much as the opt-in path. Enabled delivery also has to recover correctly and remain easy to activate across the product and benchmark. The reference of 15% for all mechanisms combined remains unqualified; it is neither a separate allowance per mechanism nor a reason to omit failures or reconstruction.
10. From a restorable point to actual research choices
-Later evidence added through 2026-10-24 19:37 UTC. The October 17 observations below retain their original 11:16 UTC cutoff; later cohorts are separate. Section 9 keeps its frozen versions, scores and stops. These observations do not close the whole scientific study.
-Executed comparison: hold stride fixed, vary replan context
-New Portfolio, Graph and Ad Placement donors started from the original objective with gpt-6.1-sol / xhigh, planned + Explore. The host still enforces finite budgets, while #6671 removes host countdowns and persistent runtime guidance from research input. Checkpoint capture is enabled independently, TurnEnvelope and stride are off, and Goal horizon is explicitly on. Native feedback remains agent-requested, with a one-hour cooldown from acceptance and no periodic or extra tail grading. Each frozen run keeps its own revision; cross-task outcomes are not pooled into a strategy-effect estimate.
Portfolio and Graph now each have retained-history and fresh arms branching from one mature point. Both arms are stride-off: this comparison examines replan context. The fresh arm resets before any current preparation or review, then decides from the same research evidence and original objective. The point stays immutable; model, Goal, scorer, feedback and cumulative budget are shared. Fresh-specific operational guidance, independent conversations and shared-host contention remain confounds; one point per case cannot establish a general causal context-strategy effect. An Ad same-point pair has not been admitted.
-| Research point | State entering comparison | Budget and observation boundary |
|---|---|---|
| Portfolio | Source age about 7.22 hours; seven inherited native results and best 23.27 | Common 1/2/4-hour windows from continuation activation, including installation and preparation; cumulative limit 36 hours |
| Graph | Selected by preregistered age and sealing conditions; source age about 13.59 hours; 14 inherited native results and best 0.844507 | The same 1/2/4-hour observations; an equal 496-second reconstruction/replay charge, with source age and continuation phase totaling 36 hours |
Original opportunities and remaining cooldown follow the point; copying it grants neither extra opportunities nor a refill. A relevant qualified repair can justify retiring an affected source to free capacity, retaining every score, failure and stop reason without selecting by reward. The replaced Graph source had 17 completed native results at retirement; only the first 14 captured at the selected point enter its new arms. Later source information is not backfilled into that point.
-Restored files must also be usable by the ordinary worker
-One real break was ownership: restored 0700 directories or 0600 files still belonged to root, so the worker could not read them. The selected-research-copy ownership repair has independent review and actual adoption of its candidate source in these Graph receivers. It preserves modes and imports no historical authority. That PR was still unmerged at this section’s cutoff; default installation is not thereby updated. Readability, model adoption, subsequent capture and research benefit have separate acceptance.
-The Graph point also lacked 37 temporary raw outputs. We regenerated them using only its sealed source, fixed fixture recipes and input digests, checked retained metrics, then sealed shared derived data separately for both arms. Ordinary users in both receivers read matching bytes and modes. Original output digests were absent: this is not original-byte recovery, and the old point was not backfilled. Saving research products under declared workspace coverage remains more reliable than continually repairing temporary paths. #6696 guides that placement; directory capture does not automatically discover every dynamic dependency.
-Both Graph models read 68 Explore events from the same basis and completed the current Goal’s semantic replan, preparation settlement and research handoff. Their first choice was identical: extending neighbor feature-mass means and standard deviations. Both stopped unchanged retries of rejected candidates and saved the 37 outputs in the workspace. Starting fresh did not itself produce a different first route. Complete experiments, decisions after negative results, the next natural capture and native gain remain to be observed.
-Portfolio at four hours: a small gain, with the main gap still open
-At two hours, each same-point Portfolio arm had two new native results and neither exceeded inherited best. By the preregistered four-hour cut, the retained arm submitted its frozen turnover-budget candidate and reached 23.42; fresh retained 23.27. The candidate, saved three-file digests, adopted package and public result were reconciled; no server package-digest attestation exists. This is a +0.15 continuation gain above the chosen point, also below the source donor’s later 23.48, rather than an all-time best.
-| Same four-hour continuation window | Retained history | Fresh replan |
|---|---|---|
| New completed native results | 3 | 2 |
| Best / gain above inherited best | 23.42 / +0.15 | 23.27 / 0 |
| Received uncached input tokens | 2,930,530 | 3,447,509 |
| Received cached input tokens | 122,126,336 | 130,243,584 |
| Received output tokens | 556,410 | 520,661 |
The retained arm’s Sharpe moved from 1.8804 to 1.8952 and drawdown from −1.12% to −1.08%, while excess return worsened from −19.46% to −19.49%; IR and Sortino still earned no points. Valid output, better local optimization and a small total-score gain did not close the main return gap. Nor does this justify simply blaming forgotten goals: both arms can read the objective. How they prioritize it, identify the limiting factor and choose experiments still needs inspection.
-How informative negative results change the next experiment
-Following its mean-influence audit, fresh tested one fixed Equity-observation repair. Some training averages improved, but the 2020 missing-macro stress path moved Sharpe from about 0.826 to −0.684, drawdown from about 3.00% to 5.34%, and relative terminal wealth down about 529,500. The model retained every result, rejected promotion and kept its native incumbent. This counterexample to one fixed rule does not reject all robust estimators: improved offline input metrics must still survive endogenous holdings, discrete allocation selection and costs.
-The retained arm also tested distinct mechanisms. A holding-period serial-covariance estimator failed its information gate; fully suppressing discretionary interim short adjustments helped some annual outcomes but hurt half-year performance and short-correlation requirements. Neither route was promoted. Fresh’s subsequent cone-optimization audit remained unfinished at the four-hour cut. Later replay and panel products are not retroactively counted as completed work, much less global-optimality proof. Complete experiments, informative counterexamples, candidate adoption and original-goal gain need separate records. Overlapping windows and case-session totals are not independent sample sizes.
-Cost ordering changed with the window. Fresh’s received uncached input was about 2.34% higher at one hour, 3.22% lower at two, and 17.64% higher at four; four-hour cached input was 6.65% higher and output 6.42% lower. These records remove inherited thread counters and include compaction and usage already returned for unfinished Turns. Fresh’s first task event also reported context-window metadata of 258,400 versus retained’s 475,000; subsequent events reported 475,000 in both. Event metadata does not establish an effective response limit, but the difference remains disclosed. Unreturned responses, evaluator resources, host admission and complete billing remain separate; qualification and other research on the shared host affect timing. A token vector cannot directly determine net efficiency.
-Later common windows: retain positive and negative cohorts
-The later windows are mixed: the newer Sol integration leads on Schema and Ad, while Astra LoopX trails Official on Schema and Grid. Each comparison below retains its own model, versions, clock and validity limits.
-| Cohort / task / direction | Reference → LoopX test best |
|---|---|
| Sol old/new · 14h Schema ↑ | 0.723684 → 0.804620 |
| Sol old/new · 14h Ad Placement ↑ | 49,589,589,573 → 49,615,495,839 |
| Astra Official/LoopX · 12h Schema ↑ | 0.881813 → 0.786257 |
| Astra Official/LoopX · 12h Grid ↓ | 1810 → 2118 |
The Sol test leads on both tasks, but trajectory matters. Schema already led at its first native result: 0.794620 versus 0.713158, received at different elapsed times. From first result to 14h, gains were only 0.010000 and 0.010526 respectively. Production code kept changing without higher native bests; more local fixes did not close the six original target requirements. Ad is a positive counterexample to a universal plateau: test trailed at the fixed 2h window and led at 14h. Its worse later candidate was rejected, and its available terminal workspace matched the retained best, not the last submission. The other three arms lack equivalent terminal-workspace proof.
-In the Astra cohort, neither Schema arm improved its best from 6h to 12h. Grid LoopX improved 2159→2118 while Official improved 1862→1810: research continued, but the gap did not narrow. LoopX tested deeper lookahead and broader mutation search, then rejected same-score or worse candidates. Official separated speed from search changes and produced successive native improvements. Schema changed many production paths during its plateau too. Experiment selection and transfer to the original goal matter alongside control cost; a flat score does not imply unchanged code.
-Recorded whole-window usage and comparison limits
-| Cohort / arm | Uncached input tokens | Cached input tokens | Output tokens |
|---|---|---|---|
| Sol 14h · Schema base | 5,536,252 | 381,548,928 | 1,621,142 |
| Sol 14h · Schema test | 7,444,288 | 479,636,096 | 2,078,121 |
| Sol 14h · Ad base | 5,896,042 | 328,260,864 | 1,656,988 |
| Sol 14h · Ad test | 4,069,556 | 319,238,784 | 1,341,356 |
| Astra 12h · Schema Official | 1,663,308 | 133,393,152 | 613,099 |
| Astra 12h · Schema LoopX | 8,850,328 | 298,377,600 | 1,410,510 |
| Astra 12h · Grid Official | 1,529,949 | 95,668,352 | 924,127 |
| Astra 12h · Grid LoopX | 3,479,450 | 213,635,840 | 1,203,790 |
The later Sol old/new study changes both LoopX core and benchmark runner: planned + Explore, Sol/xhigh, TurnEnvelope off, public native feedback, no periodic evaluation, and a one-hour accepted-submission cooldown. Each task has one run per arm. Its 14-hour window was selected after earlier results had been read: the largest whole hour below the earliest actual termination, not a score-selected arm or an 18-hour completion. A separate Astra/xhigh Official/LoopX study uses each arm’s own first 12 outer hours, including preparation. Different treatment packages are not one pooled effect.
The Sol window reconciles 7,522 unique response IDs and 52 received native results; Astra reconciles 5,277 responses and 45 received results. Grid LoopX includes one completed timeout result without a numerical score, not a zero. Cumulative thread counters are not repeatedly added; reasoning is included in output. Schema’s account-switch recovery, including work after the chosen point and before the old session stopped, remains charged to its original outer clock. Inherited result aliases do not create new opportunities.
-Astra Schema LoopX recorded 36 completed compactions versus Official’s four, and about 5.32 times its uncached input. Compaction can overlap work and is not all LoopX overhead; Official also spends some cooldown windows waiting. Both require attribution. Scorers, core/runner sources, evaluation concurrency, actual starts, adaptive restarts and shared-host load differ; an earlier operator read also exposed an Official terminal aggregate. One run per arm, missing billing and incomplete failure/resource accounting prevent a clean causal or net-cost verdict. Stopped or runner-invalid attempts stay historical; valid individual native results do not certify whole-run integrity.
-Actual staged review: adoption improved, decision quality is mixed
-An earlier nominal “with #7197” arm did not receive its staged reader and cannot count as treated. The later diagnostic uses actual fact reads, autonomous choices and writeback from the same source points: old Astra/xhigh control versus Sol/max continuation, with different versions and two-/four-hour windows. It is not a single-variable ablation. The two new tests stopped under their original finite clock; runner failure and unknown grading remain recorded. Their accepted native submissions can be inspected individually without promoting these runs into qualified benchmark results.
-| First explicit replan | Old control | Staged continuation |
|---|---|---|
| Schema · minutes / commands | 6.14 / 22 | 2.93 / 11 |
| Schema · uncached / cached / output tokens | 146,208 / 1,979,648 / 6,753 | 33,478 / 3,791,872 / 4,221 |
| Grid · minutes / commands | 5.40 / 20 | 14.74 / 23 |
| Grid · uncached / cached / output tokens | 70,829 / 2,204,416 / 10,636 | 36,689 / 5,723,776 / 21,904 |
Schema’s cheap first review selected a legal successor but did not complete the semantic writeback needed for another capture. Including the feedback review gives 17.86 versus 17.64 minutes, with different work in those rounds. Both initially chose CLI checks/configured statuses; a newer lifecycle test exposed stale registration options and improved 4/10→10/10 locally. Later testing of actual stateful execution addressed a more direct original configuration gap than the old compatibility detour. Native best nevertheless remained 0.786257, with one of six complete phases; three new accepted results did not improve the inherited best.
-Grid’s old control already created successors, changed priorities and compared independent alternatives. Staging did not invent that autonomy. The newer continuation removed an agent-imposed veto against any local regression, kept hard validity/resource constraints and completed deeper-search tests. Its accepted native score improved the inherited point 2172→2163, then regressed to 2165 while 2163 was retained. A later historical point already had 2104. This is a useful local decision/result chain, not proof that every replan became smarter or that extra review paid off. The next discriminating question is whether an original acceptance gap or a falsifiable search/speed hypothesis changes the next completed experiment and its full cost—not whether more reviews are emitted.
-Raw archives are not released with this article: readers get scope, methods and aggregates, not independent reconstruction of every run. Source qualification, model adoption, retained native gain and complete cost remain separate claims. The caller-convergence proposal above has functional validation, but no new model-outcome measurement here.
+6. The next validation starts from zero
+The next context study starts Grid, Schema and Ad Placement from their original tasks with Sol/max and an 18-hour upper limit per case. Historical Official or older LoopX runs provide descriptive references where usable. Their model/effort, source, scorer, opportunity and clock differences remain visible; they are not strict matched controls. A new baseline is warranted where the old material cannot answer the relevant question.
+First verify that actual delivery changes: the initial full read, a subsequent proven delta, complete fallback when retention is lost, and a full review-to-work interaction. Then compare useful progress, retained artifacts and the complete usage vector. Requested flags, receipts and smaller local packets cannot substitute for that evidence. If the feature fails its intended behavior, stop the affected attempt, preserve its costs and history, repair it and use a new attempt identity.
+Our effectiveness target is useful progress after many hours and reliable retention of it. Our efficiency target is comparable accepted value with less complete cost and human attention. The next result must address both; the newly launched study has no outcome claim in this article.