Skip to content

fix(skills): [Crew pulse] run the dedupe and merge judges on a fresh conversation - #30878

Closed
jasonzzj97 wants to merge 1 commit into
kirodotdev:mainfrom
jasonzzj97:crew-pulse/bg-helper-fresh-conversation
Closed

jasonzzj97 wants to merge 1 commit into
kirodotdev:mainfrom
jasonzzj97:crew-pulse/bg-helper-fresh-conversation

Conversation

@jasonzzj97

@jasonzzj97 jasonzzj97 commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: The skill dedupe and merge judges stop resending earlier judges' prompts and replies with every request.

Both judges run on the shared background session _bg. Each judge prompt is self-contained, but each call was sent as one more turn of _bg's conversation, so every request also carried every earlier judge's prompt and reply on that key until the recycle at 70% context or 40 prompts. Any _bg job that follows, such as a chat title, inherits that history too. This is the first of the two places #30197 describes; the knowledge pool is the second.

Why it matters

In live Crew traffic about 81% of the skill dedupe judge's input is resent history (#30197). It costs tokens, makes each request larger and slower, and gives the judge earlier candidates and an older skill list it was never meant to read.

Not a goal

  • The knowledge extraction pool (knowledge/llm_pool.py). Its AcpWorker has no cheap reset yet, so a per-call reset there would restart the process for every chunk; that is the second step of [Crew Pulse] Skill judges and knowledge extraction resend their conversation history on every call (~76% of their input) #30197 and a separate PR.
  • The ~6 s a new session waits for MCP readiness reports that a server-less agent never sends (see Manual verification). It already applies to every consolidation job; it is a separate fix.
  • Any other _bg caller. Channel auto-titles (messaging/auto_title.py) stack on _bg the same way, but a title is user-visible, so they keep the shared conversation until the 6 s wait below is fixed.

What changed (motivation → approach → change)

Consolidation fixed the same pattern in #22207: HistoryConsolidator._call_llm asks background_turn for fresh_conversation=True, which opens a new conversation on the running process before the permit is released. _dedupe_judge and _merge_skill_update in src/kiro_crew/history_consolidation_skills.py now pass the same flag. Each judge keeps what _bg was chosen for (#392, #2253): the same warm runtime, the same queue and the same cheap model. Only the history is dropped, and the context recycle no longer runs after a judge because the conversation is already empty. docs/system-specs/modules/history.md notes it next to the existing line that the judges stay on _bg.

#30197 asked whether the fallback reset could reload _bg's old conversation, since _bg is a persistent key. It cannot: _bg's session id is never written to the session map (session_lifecycle_parts/fleet.py, key != constants.background_key), and _ensure_background starts it with no resume id. A new test runs that fallback for real.

Backwards compatibility

Compatible: the judges' prompts, replies, model and session key are unchanged; only the conversation each one runs in is new.

Removes nothing: no field, key, config value or caller is removed or renamed; background_turn already accepted fresh_conversation.

Tests

  • test/test_consolidation_fresh_conversation.py, next to fix(memory): start every global consolidation job on an empty conversation (#20463) #22207's tests and on the same real-path harness (a real SessionManager on the packaged fake ACP backend, which records each request):
    • a dedupe judge, a merge judge and a chat-title turn on _bg each send turn 1 of their own conversation on one process, and none carries an earlier judge's text;
    • with new_conversation failing, the reset fallback on _bg (a persistent key, unlike _consolidate) starts new conversations with none of the judges' history.
  • test/test_history_coverage.py: the two judge tests that asserted recycle_background was awaited now assert a fresh conversation instead.

Proof: PROVEN test/test_consolidation_fresh_conversation.py::test_skill_judges_each_send_only_their_own_prompt_on_bg, test/test_consolidation_fresh_conversation.py::test_the_reset_fallback_on_bg_does_not_resume_the_judges_history, test/test_history_coverage.py::TestDedupeJudge::test_returns_text_and_releases_session, test/test_history_coverage.py::TestMergeSkillUpdate::test_returns_merged_body_and_prompts_with_all_inputs (head 2e8d9ef)

python3 scripts/local-gate.py: 1810 related backend tests passed. black, isort, flake8, mypy on the changed files and ./scripts/docs-lint.sh are clean.

Manual verification

Ran six dedupe judges (real build_dedupe_prompt prompts), one merge judge and one chat-title turn on _bg in an isolated Crew in Docker built from d775742d19 (Crew 0.13.0, kiro-cli 2.27.1 and the bundled KAS), on this branch and on main, through a real SessionManager. Context is what kiro-cli reports per call; KAS reports none.

arm backend sessions dedupe 1 → 6 context (tokens) title turn context later transcripts holding judge 1's prompt median reset + release
main, kiro-cli 1 1370 → 2064 2257 7 of 7 0.04 s
this PR, kiro-cli 8 1354 → 1314 1212 0 6.38 s
main, KAS 1 n/a n/a 7 of 7 0.07 s
this PR, KAS 8 n/a n/a 0 6.20 s

Verdicts and credits varied from run to run on both arms, so these runs show no effect on either.

The reset is not the 0.15 s #30197 quotes. session/new takes about 0.15 s; the new session then waits out the 6 s MCP-readiness ceiling (_MCP_DRAIN_NO_REPORT_CEILING), because kirocrew-lite has no MCP servers to report. With the ceiling at 0 the reset took 0.45-0.62 s. Until that is fixed, each judge holds _bg about 6 s longer, as each consolidation job already holds _consolidate. On kiro-cli each new session also logs one pre-turn drain discarded 1 leftover frame(s) warning.

Screenshots / video

None

Related Issues

Part of #30197

Sibling sweep

Swept: git grep -n "background_turn(" -- src/kiro_crew
Sibling: src/kiro_crew/history_consolidation_pass.py:1434 -- not affected -- consolidation already passes fresh_conversation=True
Sibling: src/kiro_crew/messaging/auto_title.py:559 -- tracked in #30197
Sibling: src/kiro_crew/knowledge/llm_pool.py -- tracked in #30197

Pattern harvest

Not generalizable: the remaining callers of background_turn on a shared key each need a judgement on whether their prompt is self-contained, which a lint cannot make.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license.

Each judge prompt is self-contained, but both judges ran as one more turn of
the shared _bg conversation, so every request resent the earlier judges'
prompts and replies until the 70% / 40-prompt recycle. Ask background_turn
for fresh_conversation=True, as consolidation already does.

Part of kirodotdev#30197
@jasonzzj97
jasonzzj97 requested review from a team and KiroCrewCaptain as code owners November 5, 2026 23:39
@jasonzzj97
jasonzzj97 requested a review from CrysisDeu November 5, 2026 23:39
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention labels Nov 5, 2026
@jasonzzj97 jasonzzj97 changed the title [Crew pulse] fix(skills): run the dedupe and merge judges on a fresh conversation fix(skills): [Crew pulse] run the dedupe and merge judges on a fresh conversation Nov 7, 2026
@bolichen97

Copy link
Copy Markdown
Contributor

[operator: bolichen97] Continuing this in #36730; thanks for the start.

KiroCrewCaptain pushed a commit that referenced this pull request Nov 13, 2026
…36730)

The skill dedupe and merge judges run on the shared background session
_bg, and each call was sent as one more turn of its conversation. Every
request therefore resent every earlier judge's prompt and reply until
the recycle at 70% context or 40 prompts, about 81% of the dedupe
judge's input, and the next _bg job inherited that history too.

Both judges now pass fresh_conversation=True to background_turn, the
same flag consolidation already uses: each judge leaves _bg on an empty
conversation of the same warm runtime before its permit is released,
and the context recycle no longer runs after it.

On _bg the flag applies only when the provider swaps conversations in
place. On a backend with no shared runtime, _bg's one-liner handles
wait on the _bg entry itself, and the reset fallback would pop it under
them, so such a _bg turn recycles as before.

Builds on #30878 by @jasonzzj97.

Part of #30197

Co-authored-by: Ziji Zhang <42743036+jasonzzj97@users.noreply.github.com>
@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Nov 13, 2026
@jasonzzj97

Copy link
Copy Markdown
Contributor Author

Superseded by #36730, which landed this change (plus a guard for backends without a shared runtime) on main. Closing in favour of it. Thanks @KiroCrewCaptain for carrying it through. The knowledge-pool half of #30197 will follow in a separate PR.

@jasonzzj97 jasonzzj97 closed this Nov 13, 2026
@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor) merge conflict Branch has merge conflicts with its base — author must resolve before merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants