Skip to content

feat(agent): summarize the conversation when the context window fills - #87

Merged
winetree94 merged 1 commit into
mainfrom
context-compression
Aug 6, 2026
Merged

feat(agent): summarize the conversation when the context window fills#87
winetree94 merged 1 commit into
mainfrom
context-compression

Conversation

@winetree94

Copy link
Copy Markdown
Contributor

Why

A session that outgrew its model context window had exactly one escape: the model calling new_context, which discarded the conversation without summarizing it. Nothing acted on the token budget on its own, and a provider that refused an oversized history failed the turn outright.

This ports the local-compaction design from the Codex CLI (codex-rs/core/src/compact.rs). Codex's remote compaction variants depend on OpenAI-only endpoints, so they are out of scope.

What

ConversationCompactor replaces a spent window with a summary of the work so far, keeping past user messages within a token budget so the original asks survive. It runs:

  • before a turn that starts on an already-full window,
  • after any tool round that crosses the budget,
  • on request via /compact (session.compact RPC),
  • once as recovery when a provider refuses the history itself.

The replacement history holds user messages only, so it cannot carry a function_call_output whose function_call was dropped — a shape both provider APIs reject. It reuses the context-epoch machinery new_context already established, so the timeline still shows everything; only the model forgets.

Every policy value lives in one CompactionPolicy block: the 0.9 trigger ratio, the 20k retention budget, and both prompts.

Notable decisions

  • CompactionTarget instead of a ModelRequest prototype. The between-turns path would otherwise have to rebuild an agent's system prompt. The compactor now owns its own instructions, so the automatic and requested paths use exactly the same prompt.
  • Attachments are dropped on compaction. The retention budget can only measure text; carrying images and files over would leave the window as full as before, defeating the point. What they showed belongs in the summary.
  • A guard Codex explicitly waives. If a compaction fails to shrink usage, compacting again cannot help either, so the runner stops rather than buying one summary per round for the rest of a long turn.

Verification

  • dart run melos verify — all 12 gates pass
  • dart run melos verify:debug — all 6 Linux E2E shards pass (real Debug Flutter runner + embedded daemon)
  • New tests: 17 unit, 1 contract, 1 real-daemon vertical slice (asserts the compacted window round-trips through the daemon), 2 widget. Registered as the context.compaction feature contract.
  • No golden changes: the two context-window dividers were factored onto one shared widget, so pixels are unchanged.

A session that outgrew its model context window had one escape: the model
calling `new_context`, which discarded the conversation without summarizing
it. Nothing acted on the token budget on its own, and a provider that refused
an oversized history failed the turn outright.

Compaction replaces a spent window with a summary of the work so far, keeping
the past user messages within a token budget so the original asks survive. It
runs before a turn that starts on a full window, after any tool round that
crosses the budget, on request through `/compact`, and once as recovery when a
provider refuses the history itself.

The replacement history holds user messages only, so it cannot carry a
`function_call_output` whose `function_call` was dropped, and it reuses the
context epoch the reset tool already established.
@winetree94
winetree94 added this pull request to the merge queue Aug 6, 2026
Merged via the queue into main with commit af0fa5d Aug 6, 2026
93 of 100 checks passed
@winetree94
winetree94 deleted the context-compression branch August 6, 2026 23:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant