Skip to content

🤖 fix: stop the context estimate from overcounting at turn start - #7662

Merged
ethanndickson merged 8 commits into
mainfrom
context-handover-czsq
Oct 27, 2026
Merged

ethanndickson merged 8 commits into
mainfrom
context-handover-czsq

Conversation

@ethanndickson

@ethanndickson ethanndickson commented Oct 27, 2026 •

Copy link
Copy Markdown
Member

Summary

The context estimate (E) that decides the Token Budget stages was much higher than the provider's input count. This PR fixes two causes:

  • E no longer counts provider replay metadata (providerOptions on model output), such as Anthropic thinking signatures.
  • The turn-start check and the first step of each turn count E as the newest reply's provider input plus the messages after it. Before, they counted the local estimate of the whole request.

Before this fix, a 1M Opus chat requested a context handoff at about 10% of real usage.

Background

A chat on anthropic:claude-opus-5-5 (1M context, 20% handoff target) showed "Context handoff requested" at 28%. The usage meter showed about 10%. The provider reported 95,548 input tokens for the next request.

measureBudgetContent counts every string in the built request, except OpenAI reasoningEncryptedContent and Xum's rawToolSearchOutput stash. It counted each Anthropic reasoning part's providerOptions.anthropic.signature as text, and the Claude encoding factor then multiplied it by 1.6. In that chat, one window replayed 14 signatures (134k chars) worth about 150k budget tokens.

The overcount is older than this week, but it had no effect until the stages moved to the local estimate:

Before those PRs, the estimate drove only the hard stop near 99%, so nobody saw the error.

Without signatures, the local estimate and the provider count still differ (#7337). Inside a turn, each step's estimate is already the previous step's provider input plus the new messages. The turn-start estimate was not, so the stage decision used a different count from the meter and from the steps that follow it.

Implementation

Replay metadata

The walker skips the providerOptions key on assistant parts (model output). This covers these fields:

  • Anthropic signature and redactedData
  • OpenAI and xAI reasoningEncryptedContent and itemId
  • Google thoughtSignature on reasoning, text and tool-call parts
  • Bedrock signature
  • OpenRouter reasoning_details
  • cache hints

On user and tool content, and on message-level options, the walker skips only Xum's own mux namespace (for example the rawToolSearchOutput stash). Those options can reach the model as text: @ai-sdk/anthropic copies a document part's providerOptions.anthropic.context and title into the request.

Tool-output content items (output: { type: "content", value }) still count their providerOptions. The openai-compatible, openai chat and deepseek adapters send that value as JSON.stringify(output.value), so those bytes reach the wire. The existing tool-content impersonation test guards this case.

These two rules replace four special-case walker kinds and their two skip blocks.

Turn-start anchor

createTurnBudgetAnchor reads the newest assistant row of the turn's provider request rows. It returns an anchor only when all of these are true:

  • The row is a completed reply: not partial, synthetic, failed, a compaction summary or a boundary.
  • The row saved the same request identity as the request: canonical model, resolved model (metadataModel), thinking level, route provider and agent. Each saved field must equal the request's, unset included. A direct route saves no route, and a row from a build that did not save a field gives no anchor. A route or agent change re-freezes the system prompt and tools. A providers.jsonc mapping change keeps the model and changes the resolved model. Message preparation keeps or strips earlier Anthropic reasoning by the thinking level, so another level replays reasoning that the provider count does not hold.
  • No content follows a tool call in the row's last step. The rebuild splits such a step into an assistant message, its tool results and a trailing assistant message (a provider-run web_search does this). A delta from the last assistant message would miss the first two.
  • The row has contextUsage with a positive input total and systemMessageTokens, and none of these saved counts is negative or a non-integer (saved rows are unchecked JSON).

The anchor is the input, cache-read and cache-write tokens of that reply's last step, plus that step's reasoning output tokens. The request replays that reasoning after the anchored prefix. A model that keeps earlier thinking counts all of it, also redacted or encrypted reasoning that has no text for the estimator. On 172 real consecutive claude-opus-5-5 reply pairs, the input growth across a turn boundary matched the previous reply's full output, reasoning included. If the last step emitted reasoning and the provider reported no reasoning count, the row does not anchor. If the sent system text grew since that reply (a live system prompt can change between turns), the growth is added. The growth compares the system message plus its system rows with the reply's systemMessageTokens, which counted the same text.

The delta is the request messages from the last assistant message on. A tool reference in the delta, or a different model, gives the full estimate (the #5413 rule). A compaction request gets no anchor, because its trim can drop rows that the provider count holds.

assembleBudgetCheckedPromptPayload uses the anchor for the turn-start check and returns it. The stream request carries it to step zero (contextBudgetTurnAnchor, refusal fallbacks included), so step zero enforces the same count. A stage that the turn start published is not refused at step zero. A refusal fallback that keeps the refused output continues from a partial row, so it gets no anchor and uses the full estimate.

A thinking-level change can arrive after the turn start built the request. The stream then rebuilds the messages at the new level. A rebuild before the stream starts returns the anchor for the new level, and step zero uses it. A rebuild inside step zero (a change that races stream setup) gives no anchor, so step zero uses the full estimate.

The loopback fixture (tokenBudgetStages.fixture.integration.test.ts) now reports provider usage that tracks the request size, and runs the real tokenizer, as production does. R2b now checks the fixed behavior: with provider usage at 2/3 of the full estimate, the turn start publishes the handoff, and step zero sends that request although its full estimate is past the hard ceiling.

Validation

I ran the real message pipeline (prepareProviderRequestMessages, then prepareMessagesForProvider) on the persisted rows that the 28% notice measured. The 46 model messages, without system and tools, changed from 221,698 estimated tokens to 76,042. With the system and tool overhead from the notice, the new total is about 131k. That is under the 200k target, and the provider total was 95.5k.

I also ran a live chat on an isolated xum server with Token Budget on, claude-opus-5-5 with the 1M window, thinking xhigh, and a 15% handoff threshold (150,000 tokens). A temporary log, not in this PR, recorded the checked estimate at each step next to the provider's input count for that step.

Turn Estimate at turn start Provider input Ratio
2 23,486 23,271 1.009
3 32,817 32,424 1.012
4 45,291 44,576 1.016
5 64,387 63,806 1.009
6 89,901 89,477 1.005
7 117,054 116,475 1.005
8 145,790 145,484 1.002
8, continuation 181,720 178,784 1.016

No notice came while the real count was under 150,000. Turn 8's tool results pushed the next step past the target, so the settled-step check ended the turn and the continuation got the handoff notice: "Context budget ~18% used (181418 of 991808 tokens)". The provider count for that request was 178,784, also 18%. Later steps inside a turn were 0.4% to 4.5% over the provider count. The full estimate without the anchor was 16% to 25% over.

Risks

Medium. The change moves the turn-start handoff and final stages, and the hard stop before the first step of a turn.

  • E at turn start follows the provider count when the anchor applies. That count is usually lower than the local estimate, so stages come later, closer to the meter.
  • Step zero can now send a request whose full estimate is past the hard ceiling, when the provider count plus the new messages fits. If a provider under-reports its input, the provider rejects the request, and the existing context-exceeded recovery runs.
  • When the anchor does not apply (first turn, model, route, agent or thinking-level switch, a thinking change applied at step zero, a split last step, failed or partial reply, compaction), E is the full estimate, as before.
  • The openai-compatible adapter copies providerOptions.openaiCompatible from assistant parts into the wire message, and the walker now skips it. Xum never sets it.
  • The anchored estimate (inside a turn, and now at turn start) adds provider reasoning tokens to a delta that also holds the visible reasoning text. This overcount is small and safe.
  • For a model that drops earlier thinking (Opus 4.6 to 4.8), the turn anchor overcounts by the last step's reasoning. This is the safe direction.
  • Known limitation, accepted: a tool definition that grows between turns (an MCP change, a tool policy change or an assembly hook) is not counted at step zero of the next turn. From step one on, the in-turn anchor requires the same tools, so it uses a provider count that includes them. The tool block stays byte-identical inside a segment for prompt caching, and an agent switch already gives no anchor. A precise fix needs a persisted per-reply tool count, which this PR does not add.
  • Known limitation, accepted: the token budget tags each older user row with [id: N] (about 10 budget tokens). A reply sent with the budget off (continuous compaction) or by a build without tags holds no tags, so step zero of the next turn with the budget on does not count them. The reply of that turn has the tags, so the next turn's anchor holds them.
  • Known limitation, accepted: rows that a request sends and history does not save (post-compaction attachments, an agent transition with plan content) are in that request's provider count. At the turn start and step zero of the next turn, the anchor still counts them, and the delta counts any new injection. This overcount is at most the rows injected into one request (the post-compaction block is capped at 80,000 characters). The turn after that anchors on a request without those rows. A precise fix needs a persisted per-reply field, which this PR does not add.
  • Declined finding: when the anchor does not apply, the full estimate gives no cost to opaque reasoning replay (Anthropic redactedData, OpenAI ciphertext). Counting that data's JSON caused the overcount that this PR fixes. On the bug's payload (claude-opus-5-5 at xhigh, which keeps earlier thinking), the full estimate is still about 131k tokens against the provider's 95.5k.

Follow-ups, not in this PR:


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: xhigh • Cost: $66.81

The budget walker counted providerOptions on assistant parts as input text.
Anthropic thinking signatures are large base64 strings that Anthropic does not
bill, so a 1M Opus chat hit its 20% handoff target at about 10% real usage.
Skip providerOptions on every SDK message and message part. Tool-output content
items keep it because some adapters JSON-stringify output.value.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-27T12:02:24.505596Z a93f8e9 Manual request
🔒 Security Review ✅ Completed 2026-10-27T12:04:21.462274Z a93f8e9 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@ethanndickson

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 69cd900a1f

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

🟢 e2e stories (Haiku): 14 passed

22 agent steps · 7 replayed from cache · 15 model calls · 46.4k tokens (24% cached)
Spend: $0.09 of the $1.00 cap (proxy ledger)

All 14 tests in 9 files
Test Agent Time
🟢 chatRestart.e2e.ts · 1 passed 1 step · 1 call 5.5s
🟢 a chat survives a restart of the server 1 step · 1 call 5.5s
🟢 compaction.e2e.ts · 1 passed 1 step · 1 call 4.6s
🟢 /compact replaces the old chat with a summary 1 step · 1 call 4.6s
🟢 pluginTrustRestart.e2e.ts · 1 passed 4 steps · 2 calls 9.7s
🟢 a trusted plugin hook stays trusted after the server restarts 4 steps · 2 calls 9.7s
🟢 settingsRoundTrip.e2e.ts · 1 passed 2 steps · 1 call 8.8s
🟢 a chat survives trips through Settings 2 steps · 1 call 8.8s
🟢 sidebarPerWorkspace.e2e.ts · 1 passed 4 steps · 1 call 4.5s
🟢 a new workspace starts with its right sidebar collapsed while another keeps it open 4 steps · 1 call 4.5s
🟢 smoke.e2e.ts · 1 passed 2 steps · 1 call 2.6s
🟢 the agent opens the playground workspace 2 steps · 1 call 2.6s
🟢 streamErrors.e2e.ts · 1 passed 1 step · 1 call 8.3s
🟢 stream errors show in the chat, and the chat recovers 1 step · 1 call 8.3s
🟢 todoNudge.e2e.ts · 1 passed 1 step · 1 call 3.3s
🟢 the agent finishes open TODO items after the turn-end nudge 1 step · 1 call 3.3s
🟢 touchAlignment.e2e.ts · 6 passed 6 steps · 6 calls 21.6s
🟢 the Settings close button is level with the title on a touch screen 390px wide 1 step · 1 call 3.7s
🟢 the Settings close button is level with the title on a touch screen 768px wide 1 step · 1 call 4.1s
🟢 Backup settings checkboxes are small and level with their labels on a touch phone 1 step · 1 call 4.5s
🟢 the heartbeat checkbox is small and level with its label on a touch phone 1 step · 1 call 3.3s
🟢 a reply's model and time stay on one line on a touch phone 1 step · 1 call 3.3s
🟢 the workspace actions menu stays on the title row on a touch phone 1 step · 1 call 2.6s

e2e 0.20.0 · 1m 12s · web · run artifacts

🟢 e2e Electron stories (Haiku): 3 passed

3 agent steps · 3 model calls · 7.9k tokens (22% cached)
Spend: $0.02 of the $1.00 cap (proxy ledger)

All 3 tests in 3 files
Test Agent Time
🟢 electron/deepLink.e2e.ts · 1 passed 1 step · 1 call 5.9s
🟢 a deep link of a second launch starts a chat in the running app 1 step · 1 call 5.9s
🟢 electron/menuBar.e2e.ts · 1 passed 1 step · 1 call 5.7s
🟢 the workspace menu bar is the desktop title bar 1 step · 1 call 5.7s
🟢 electron/smoke.e2e.ts · 1 passed 1 step · 1 call 5.4s
🟢 Xum Electron opens the playground workspace 1 step · 1 call 5.4s

e2e 0.20.0 · 19.1s · electron · run artifacts

🟢 e2e agent repros (Haiku): no tests selected

Spend: $0.00 of the $1.00 cap (proxy ledger)

e2e 0.20.0 · 5.3s · 2 targets (web, phone) · run artifacts

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 69cd900a1f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/common/utils/compaction/contextBudget.ts Outdated
… usage

The turn-start check and step zero now count E as the newest reply's provider input plus the messages after it, when that reply is a completed one from the same model, route and agent. Growth of the sent system text since that reply is added. Otherwise they keep the full estimate.

Stage prompts are decided on the same E, so a stage published at turn start is never refused at step zero. The loopback fixture now reports provider usage that tracks the request and runs the real tokenizer.
@ethanndickson ethanndickson changed the title 🤖 fix: stop counting provider replay metadata in the context estimate 🤖 fix: stop the context estimate from overcounting at turn start Oct 27, 2026
@ethanndickson

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: ce3ccc3e0a

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ce3ccc3e0a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/node/services/contextBudgetCounting.ts
Comment thread src/node/services/contextBudgetCounting.ts Outdated
Comment thread src/node/services/contextBudgetCounting.ts Outdated
…played reasoning

The turn anchor now adds the last step's reasoning output, like the in-turn anchor. Claude Opus 5.5 replays that step's thinking at its full count, and the estimator sees at most the summary text. Redacted thinking and encrypted reasoning have no text at all. A last step that emitted reasoning without a reported count does not anchor.

A row from another resolved model (a providers.jsonc mapping change), or from a build that did not record it, does not anchor.
@ethanndickson

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 115c8c7522

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/node/services/contextBudgetCounting.ts
Comment thread src/node/services/contextBudgetCounting.ts
Comment thread src/node/services/contextBudgetCounting.ts Outdated
…rows in the turn anchor

- replayedStepTokens returns no anchor when a persisted input, cache or
  reasoning count is negative or not a safe integer.
- The turn anchor measures system growth against the sent system message plus
  its system rows, the same text the reply's systemMessageTokens counted.
@ethanndickson

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 5ccad0f70b

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5ccad0f70b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/node/services/contextBudgetCounting.ts Outdated
Comment thread src/common/utils/compaction/contextBudget.ts
Comment thread src/node/services/contextBudgetCounting.ts
@ethanndickson

Copy link
Copy Markdown
Member Author

I paused the Codex review loop for a maintainer decision. The reason is the AGENTS.md early stop: the loop does not converge.

Rounds 2, 3 and 4 each brought three new findings, and most of them are in the turn-start anchor. Round 4 on 5ccad0f70 found these issues:

  • Thinking replay: for Sonnet or Haiku 5.5, a switch from off to a thinking level restores older signed reasoning. The anchored provider count does not hold that reasoning.
  • Full-estimate fallback: the estimate no longer charges opaque reasoning replay, for example Anthropic redactedData.
  • Split last step: if the last step has a tool call followed by text, the delta starts at the trailing text. It then misses the tool call and its result.

Each finding is a real edge case, and each fix is a small full-estimate guard. But each round finds a new way that the rebuilt prefix differs from the prefix the provider counted. Before I add more guards, I need a decision: add the guards, or remove the turn anchor from this PR and keep only the replay-metadata fix.

…last step

- The turn anchor needs the reply's recorded thinking level to equal the
  request's effective level. Message preparation keeps or strips earlier
  Anthropic reasoning by that level.
- The turn anchor is skipped when content follows a tool call in the reply's
  last step. The rebuild then splits the step, and a delta from the last
  assistant message would miss the tool call and its result.
@ethanndickson

Copy link
Copy Markdown
Member Author

@codex review

Round 4 on 5ccad0f70:

  • Fixed in dd159a55d: the turn anchor now needs the same thinking level as the request, and it is skipped when content follows a tool call in the reply's last step.
  • Declined, "Preserve opaque reasoning in full-estimate fallbacks": this is intended, and it is the core change of this PR. Replay metadata (signatures, ciphertext and item IDs) is not input text, and counting its JSON caused the overcount that this PR fixes. A conservative charge for opaque data would bring back part of that overcount, and persisted reasoning counts would add a new path to the estimator. On the bug's real payload (46 model messages, claude-opus-5-5 at xhigh, which keeps earlier thinking), the full estimate after this PR is still about 131k tokens against the provider's 95.5k. Anthropic redacted thinking is rare.

Known limitations that the maintainer accepted in earlier rounds (please do not raise them again):

  • A tool definition that changes size between turns is not counted at step zero of the next turn. From step one on, the in-turn anchor requires the same tools.
  • A reply sent without [id: N] history tags (token budget off, or an older build) does not hold the tags that step zero of the next turn adds. Each tag is about 10 budget tokens, and the next reply holds them.

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: dd159a55d9

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@ethanndickson

Copy link
Copy Markdown
Member Author

@codex review

The code review of dd159a55d finished at 10:30:10Z and the summary marks it as having findings, but no review or inline comment was posted. Please review dd159a55d again and post any findings as inline comments.

Context from round 4, unchanged:

  • Fixed in dd159a55d: the turn anchor now needs the same thinking level as the request, and it is skipped when content follows a tool call in the reply's last step.
  • Declined, "Preserve opaque reasoning in full-estimate fallbacks". The maintainer reviewed this again today and kept the decline. Replay metadata (signatures, ciphertext and item IDs) is not input text, and counting its JSON caused the overcount that this PR fixes. Most turns use the provider's own count (the turn anchor), which includes the last step's reasoning tokens. The full estimate runs only when the request changed since the provider counted it, and on the bug's real payload (46 model messages, claude-opus-5-5 at xhigh) it is still about 131k tokens against the provider's 95.5k.

Known limitations that the maintainer accepted in earlier rounds (please do not raise them again):

  • A tool definition that changes size between turns is not counted at step zero of the next turn. From step one on, the in-turn anchor requires the same tools.
  • A reply sent without [id: N] history tags (token budget off, or an older build) does not hold the tags that step zero of the next turn adds. Each tag is about 10 budget tokens, and the next reply holds them.

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: dd159a55d9

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@ethanndickson

Copy link
Copy Markdown
Member Author

@codex review

The code review of dd159a55d that started at 10:40:14Z has been running for more than 20 minutes with no result. Please review dd159a55d again and post any findings as inline comments.

Context from round 4, unchanged:

  • Fixed in dd159a55d: the turn anchor now needs the same thinking level as the request, and it is skipped when content follows a tool call in the reply's last step.
  • Declined, "Preserve opaque reasoning in full-estimate fallbacks". The maintainer reviewed this again today and kept the decline. Replay metadata (signatures, ciphertext and item IDs) is not input text, and counting its JSON caused the overcount that this PR fixes. Most turns use the provider's own count (the turn anchor), which includes the last step's reasoning tokens. The full estimate runs only when the request changed since the provider counted it, and on the bug's real payload (46 model messages, claude-opus-5-5 at xhigh) it is still about 131k tokens against the provider's 95.5k.

Known limitations that the maintainer accepted in earlier rounds (please do not raise them again):

  • A tool definition that changes size between turns is not counted at step zero of the next turn. From step one on, the in-turn anchor requires the same tools.
  • A reply sent without [id: N] history tags (token budget off, or an older build) does not hold the tags that step zero of the next turn adds. Each tag is about 10 budget tokens, and the next reply holds them.

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: dd159a55d9

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dd159a55d9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/node/services/turnEngine/StepPreparation.ts Outdated
A thinking-level change during PREPARING, or one that races stream setup, rebuilds the first request at the new level. Step zero still counted that request against the anchor captured at the old level, so a high-to-off switch could reuse a count with replayed reasoning and refuse a request that fits.

- The pre-stream fold now carries the anchor of its rebuild to step zero.
- Step zero uses the full estimate when it applies a thinking override itself.
@ethanndickson

Copy link
Copy Markdown
Member Author

@codex review

Round 5 on dd159a55d:

  • Fixed in 2e70fc7a0, "Refresh the turn anchor after a thinking rebuild": the pre-stream thinking fold carries the anchor of its rebuild to step zero, and step zero uses the full estimate when it applies a thinking override itself.

Decisions from earlier rounds (please do not raise them again):

  • Declined, "Preserve opaque reasoning in full-estimate fallbacks". The maintainer kept this decline. Replay metadata (signatures, ciphertext and item IDs) is not input text, and counting its JSON caused the overcount that this PR fixes. Most turns use the provider's own count (the turn anchor), which includes the last step's reasoning tokens. The full estimate runs only when the request changed since the provider counted it, and on the bug's real payload it is still about 131k tokens against the provider's 95.5k.
  • Accepted limitation: a tool definition that changes size between turns is not counted at step zero of the next turn. From step one on, the in-turn anchor requires the same tools.
  • Accepted limitation: a reply sent without [id: N] history tags (token budget off, or an older build) does not hold the tags that step zero of the next turn adds. Each tag is about 10 budget tokens, and the next reply holds them.

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 2e70fc7a07

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2e70fc7a07

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/node/services/contextBudgetCounting.ts
Comment thread src/node/services/contextBudgetCounting.ts Outdated
Comment thread src/node/services/contextBudgetCounting.ts
…saved request identity

The turn anchor compared the route and agent only when both sides set them, so a row from a build that did not save them anchored a request on another route or agent. A negative saved system count read as system growth.

- One identity rule: the resolved model, thinking level, route and agent a reply saved must each equal the request's, unset included (a direct route saves no route).
- The saved system count goes through the same check as the usage counts.
@ethanndickson

Copy link
Copy Markdown
Member Author

@codex review

Round 6 on 2e70fc7a0:

  • Fixed in a93f8e9c0, "Validate the persisted system count before anchoring": the saved system count goes through the same non-negative integer check as the usage counts.
  • Fixed in a93f8e9c0, "Treat missing route or agent metadata as a mismatch": the resolved model, thinking level, route and agent that the reply saved must each equal the request's, unset included.
  • Declined, "Invalidate anchors after request-only context was injected". The overcount is the safe direction. It applies only at the turn start and step zero of the one turn after an injection, and it is at most the rows injected into one request. The full estimate also overcounts (about 131k against the provider's 95.5k on the bug's payload). A precise fix needs a new persisted field.

Decisions from earlier rounds (please do not raise them again):

  • Declined, "Preserve opaque reasoning in full-estimate fallbacks". Replay metadata (signatures, ciphertext and item IDs) is not input text, and counting its JSON caused the overcount that this PR fixes. Most turns use the provider's own count (the turn anchor), which includes the last step's reasoning tokens.
  • Accepted limitation: a tool definition that changes size between turns is not counted at step zero of the next turn. From step one on, the in-turn anchor requires the same tools.
  • Accepted limitation: a reply sent without [id: N] history tags (token budget off, or an older build) does not hold the tags that step zero of the next turn adds. Each tag is about 10 budget tokens, and the next reply holds them.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: a93f8e9c05

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: a93f8e9c05

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@ethanndickson
ethanndickson added this pull request to the merge queue Oct 27, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 27, 2026
@ethanndickson
ethanndickson added this pull request to the merge queue Oct 27, 2026
Merged via the queue into main with commit 26e3723 Oct 27, 2026
33 of 34 checks passed
@ethanndickson
ethanndickson deleted the context-handover-czsq branch October 27, 2026 12:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant