Skip to content

fix(session): recycle the shared _bg conversation after one-liners - #18234

Merged
buluoray merged 1 commit into
mainfrom
fix/bg-oneliner-recycle
Oct 9, 2026
Merged

buluoray merged 1 commit into
mainfrom
fix/bg-oneliner-recycle

Conversation

@buluoray

@buluoray buluoray commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: On non-runtime background backends, background one-liners stop growing one shared conversation until every background feature fails with prompt-too-long.

On any background backend outside _bg_runtime_backends() (today claude, opencode, pi, goose and deepseek), chat titles, nav link labels, folder icons, session summaries and tips each append a turn to the one persistent _bg conversation. Nothing recycles that conversation after a one-liner, so it grows for the whole gateway uptime. Eventually the provider refuses it as too long. From then on every background caller sharing it (titles, link labels, memory consolidation) fails the same way until the gateway restarts.

Why it matters

Until it overflows, each one-liner re-sends the whole accumulated transcript, so background spend grows with uptime instead of staying flat. After it overflows, titles, nav labels and memory consolidation silently stop working. Everyone who runs Crew on a non-kiro backend for more than a short session is affected.

Not a goal

  • Changing the recycle thresholds (70%, 40 prompts, RSS) or what counts as one.
  • Changing the multiplexed _bg runtime used by kiro, kas and codex.
  • Moving history consolidation off the shared _bg session (that is fix(history): isolate consolidation sessions #9036).
  • Teaching the raise-time ACP error classifier the claude adapter's spelling; the new predicate is local to the background recycle.

What changed (motivation → approach → change)

Symptom → cause:

  • get_bg_session() gives a non-runtime backend a _ProviderBgSession over the shared BACKGROUND_KEY entry. Its destroy() only releases the turn semaphore.
  • run_bg_oneliner() ended at destroy() and never called recycle_background(), even though docs/system-specs/modules/session.md says that runs "after every background task". So no recycle criterion was ever checked for one-liner traffic.
  • recycle_background() decides from context_usage_pct(). After a prompt-too-long refusal that is still the last successful turn's figure, so even background_turn()'s recycle could keep a conversation every later turn would be refused on.

Change:

  • _ProviderBgSession.shares_background_conversation = True. run_bg_oneliner() calls sessions.recycle_background() after destroy() only when its handle carries that marker. Runtime handles carry no marker, so they are unaffected.
  • recycle_background(*, context_overflowed=False) forces a recycle on a non-runtime backend when the caller saw the refusal, or when the provider carries the mark mark_context_overflowed() leaves. The forced branch is gated on not _bg_backend_supports_runtime(), so kiro/kas/codex keep exactly their old criteria.
  • stream_and_collect() marks the provider in its AcpError branch when is_context_overflow_error() matches. History consolidation catches its own turn error before background_turn()'s recycle runs, so the verdict has to ride on the provider.
  • llm_helpers.is_context_overflow_error() returns true for the classifier's context_overflow tag, or for "Prompt is too long" / "context window overflowed" / "maximum context length". It needs the text match because the claude adapter puts its refusal in the JSON-RPC message, and the classifier only reads data. A false positive costs one provider respawn.
  • The spec's "Context Overflow Protection" section now documents the forced criterion and lists run_bg_oneliner() as a caller.

Enumeration of the paths into the shared _bg entry:

  • run_bg_oneliner with a provider-backed handle: fixed (recycle after every turn, forced on overflow).
  • run_bg_oneliner with a runtime handle (kiro/kas/codex): unchanged, because the session is ephemeral.
  • background_turn (consolidation, other stream_and_collect callers): already recycled; now forced after a swallowed overflow on non-runtime backends.
  • Task runner lesson extraction: calls recycle_background() with the default, so it gets the threshold criteria plus the provider mark.
  • Heartbeat: a separate _hb key, not touched.

Backwards compatibility

Compatible: recycle_background() gains a keyword-only argument with a default, and every existing caller and test fake keeps working. The only new behaviour is extra recycles on non-runtime backends, which spawn a fresh provider exactly as the existing criteria already do.

Removes nothing: the change only adds a recycle trigger and a call site. No key, record, validator or public function is removed or narrowed.

Tests

test/test_bg_oneliner_recycle.py (15 cases, real SessionManager with a mock provider factory):

  • 40 one-liners on claude reach the blind backstop and replace the provider; a single one-liner advances prompt_count.
  • A one-liner refused with the claude adapter's "Prompt is too long" (reported context 3%) forces the recycle, and the next one-liner succeeds on the fresh provider. An unrelated AcpError does not recycle.
  • stream_and_collect leaves the overflow mark, so the following recycle_background() replaces the provider; the same 3% without the mark is kept.
  • On "" (kiro), kas and codex, a mark plus context_overflowed=True does not recycle. A handle without the marker is not followed by any recycle_background() call.
  • is_context_overflow_error accepts the classifier tag and the three spellings, and rejects other AcpErrors and non-ACP exceptions.

Each mutation reddens the named tests:

  • dropping the one-liner recycle: blind-backstop, single-count and forced-recycle cases.
  • dropping the stream_and_collect mark: the swallowed-overflow case.
  • dropping the runtime gate: all three runtime-backend cases.
  • dropping the forced branch: both overflow cases.

Ran locally with pytest -n0 test/test_bg_oneliner_recycle.py against this worktree's src. I ran no other suite; that is left to CI.

Manual verification

N/A — the failure needs hours of real background traffic on a non-kiro backend to reach the window. The unit cases drive the real SessionManager recycle path with the provider's actual refusal text, and I did not run it against a live claude or deepseek backend. macOS/Windows: unverified; the change has no platform-specific code.

Related Issues

Closes #18230
Refs #15336

Pattern harvest

Rule candidate: review-prompt
Pattern: a handle over a SHARED persistent session whose destroy() is a no-op means the caller owns the lifecycle check, and every acquire/drive/destroy helper over it must also run the recycle.

Decision for the owner

The decision is whether a prompt-too-long refusal should force a _bg recycle on every backend or only on the shared-conversation (non-runtime) path. This draft scopes it to non-runtime backends so kiro, kas and codex are byte-for-byte unchanged, and leaves extending the forced recycle to kiro's persistent _bg entry as a follow-up if you want it.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.

@buluoray
buluoray force-pushed the fix/bg-oneliner-recycle branch from 766506e to 4118ee3 Compare October 8, 2026 21:16
On a background backend outside _bg_runtime_backends() (claude, opencode,
pi, goose, deepseek) every run_bg_oneliner turn (titles, nav labels,
folder icons, summaries) is appended to the one persistent _bg
conversation. _ProviderBgSession.destroy() only releases the turn
semaphore and run_bg_oneliner never called recycle_background(), so no
criterion bounded that conversation: it grew for the whole gateway
uptime until the provider refused it as prompt-too-long. After that the
reported percentage was still the last successful turn's, so even
background_turn()'s recycle kept the poisoned conversation and every
later background turn failed the same way until a restart.

- run_bg_oneliner runs recycle_background() after destroy() whenever the
  handle is a _ProviderBgSession (shares_background_conversation).
- A turn refused as too long forces the recycle on that path: the
  one-liner passes context_overflowed=True, and stream_and_collect leaves
  the verdict on the provider (mark_context_overflowed) because callers
  such as history consolidation swallow the error.
- is_context_overflow_error() matches the classifier tag or the backend
  text; the claude adapter's "Prompt is too long" is in the JSON-RPC
  message, which the classifier does not read.
- Runtime-capable backends (kiro, kas, codex) are unchanged: their
  one-liner handles carry no marker and the forced recycle is gated off.

Closes #18230
Refs #15336
@buluoray
buluoray force-pushed the fix/bg-oneliner-recycle branch from 4118ee3 to 76722e4 Compare October 8, 2026 21:33
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 8, 2026
@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 76722e45d12af132296430bc05fe2501eeab6268 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Advisory — does not block merge.

@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — 🟡 CONCERNS

Premise-level review of 76722e45d12af132296430bc05fe2501eeab6268 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

is_context_overflow_error adds a second overflow detector. It matches text the raise-time classifier deliberately refuses to read, and that classifier's existing consumer still misses Claude's spelling.

Not justified as shipped

  • Item 4 — duplicate of acp/transport_errors.py context_overflow tag: the classifier only reads the data field on purpose ("a phrase echoed only in JSON-RPC message cannot stamp an unrelated error"). The new predicate reads str(exc), which includes message. So the two detectors disagree on the same error. getattr(exc, "context_overflow" has 1 other reader outside this diff (subagent_manager/run.py:1020), and it still misses "Prompt is too long". The description puts the classifier change under Not a goal, so this local predicate is a symptom-level patch and the description says so.
    Subtraction: delete _RE_CONTEXT_OVERFLOW_TEXT and its text arm. Recognize the Claude adapter's spelling once, inside the classifier's context_overflow tag, and have both callers read that tag.
    Clears when: overflow detection lives in one predicate, or a human accepts the split and defers it to a follow-up issue.

What this change ships

Inventory (6 items) — 5 justified

Intent: keep background titles, labels and consolidation working on non-kiro backends however long the gateway runs. FIX.

  1. Every background one-liner on claude, opencode, pi, goose or deepseek now runs the recycle check afterwards — justified
  2. On those backends, a prompt-too-long refusal forces a fresh provider whatever percentage is reported — justified
  3. stream_and_collect marks the provider on overflow so a caller that swallowed the error still gets a recycle — justified
  4. New public predicate is_context_overflow_error that matches on message text — duplicate of transport_errors classifier tag
  5. recycle_background(context_overflowed=…) keyword argument — justified
  6. The spec's Context Overflow Protection section documents the forced criterion and the new caller — justified

[FIRST-PRINCIPLES-REVIEWED] 76722e4

Description read

[DESCRIPTION-READ] 764d2080e1723cdb4b55ecec4086d39c94d3fde8cbd005b0df47474fcac87404

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — 🟡 CONCERNS

Design-level review of 76722e45d12af132296430bc05fe2501eeab6268 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

The recycle runs inside the caller's await, outside its timeout, so every ~40th one-liner makes its caller wait for a whole provider respawn.

Watch

  • The recycle sits on the caller's critical path. The diff adds await sessions.recycle_background(context_overflowed=overflowed) to the finally of run_bg_oneliner(), so the caller gets its result only after that call returns. On a blind, overflow or RSS hit, recycle_background() holds the _bg turn semaphore while it starts a new provider and shuts the old one down (session_background.py:1020-1078). That respawn sits outside the wait_for(timeout), and its _start_sem is acquired at StartPriority.BACKGROUND even for a FOREGROUND caller. The worst case is STT endpointing (stt_stream.py:430, 5s timeout, mid-dictation). Before this PR, one-liners never triggered a respawn. Now about one in 40 holds back the user's auto-submit verdict for the length of a claude/opencode spawn, and the generation check then likely throws that verdict away as stale. Title and nav-label one-liners queued behind it on the same semaphore are delayed too.
    Clears when: the one-liner returns its result before the recycle runs, for example by scheduling recycle_background() as a tracked background task after destroy(). Alternatively, a test or measurement showing the respawn adds no latency to the caller.

[DESIGN-REVIEWED] 76722e4

Description read

[DESCRIPTION-READ] 33f5353ef6bec2305bb17387db63b665ff6e4717f7bb7bfc1fd6c6d2c3b4dddc

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 76722e45d12af132296430bc05fe2501eeab6268 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 76722e4

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 76722e45d12af132296430bc05fe2501eeab6268: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 76722e45d12af132296430bc05fe2501eeab6268 — this comment is updated in place on each push.

Review details

No findings that block the merge; two advisory findings below.

FINDING — src/kiro_crew/llm_helpers.py:1915 — The context_overflowed=overflowed flag describes the provider this turn ran on, but recycle_background applies it to whatever session.provider is once it gets the semaphore back. When the shared conversation is full, every queued one-liner overflows on P1. The first recycle swaps in P2, then each later queued recycle kills and respawns that fresh provider again, so N queued one-liners cause N−1 needless provider spawns. → Fix: either mark the turn's own provider (mark_context_overflowed(...)) instead of passing a bool, or pass the provider and force the recycle only when session.provider is that provider.

FINDING — src/kiro_crew/llm_helpers.py:1915 — await sessions.recycle_background(...) runs inline in the teardown after _run has already returned the text. So a caller such as stt_endpointing (timeout=self._timeout, FOREGROUND priority) now also waits for every other queued _bg turn to finish and gets the semaphore back. Whenever a recycle criterion fires (the blind backstop, the RSS ceiling, an overflow), it also waits for replacement.start(), a full provider cold start. Its result arrives well past its timeout. → Fix: run the recycle as a tracked background task, or don't wait on a busy semaphore (wait_if_busy=False, then skip it).

[OPUS-REVIEWED] 76722e4

Verdict parsed from the review's SHA-scoped output markers for commit 76722e45d12af132296430bc05fe2501eeab6268.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 76722e45d12af132296430bc05fe2501eeab6268: <one-sentence reason>

@buluoray
buluoray marked this pull request as ready for review October 8, 2026 23:51
@buluoray
buluoray requested review from a team and KiroCrewCaptain as code owners October 8, 2026 23:51
@buluoray
buluoray requested a review from bolichen97 October 8, 2026 23:51
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 8, 2026

@CrysisDeu CrysisDeu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving at 76722e4. Read the full diff plus the owning spec; nothing blocking. Non-blocking notes, worth a follow-up or a quick fix before merge:

  • [MINOR] src/kiro_crew/llm_helpers.py:1916 -- the recycle (which may spawn a replacement provider under _start_sem at StartPriority.BACKGROUND and await doomed.shutdown()) runs inside the caller's await, in the finally, OUTSIDE the timeout the caller passed to run_bg_oneliner. Before this PR a one-liner never triggered a respawn; now ~1 in 40 does, and a FOREGROUND caller with a tight deadline (STT endpointing, stt_stream.py:430, 5s) waits a whole claude/opencode spawn for a result it then likely discards. Already posted by the Design Review lane (CONCERNS, advisory) and not addressed. background_turn() has the same shape today, so this is consistent rather than new, but a _fire_and_forget-style tracked task for the recycle (or passing start_priority through) would clear it cheaply.
  • [NIT] src/kiro_crew/llm_helpers.py:249 -- second overflow detector reading str(exc) (incl. JSON-RPC message), while transport_errors deliberately tags context_overflow from data only; the two now disagree on the claude "Prompt is too long" error and subagent_manager/run.py:1020 (the tag's other reader) still misses it. Already posted by First Principles (CONCERNS) and explicitly declared a non-goal in the body. Acceptable for a recycle-only consumer (false positive = one respawn), but worth a follow-up issue so detection does not stay split.

@buluoray
buluoray enabled auto-merge (squash) October 9, 2026 05:27
@buluoray
buluoray merged commit 1ea2767 into main Oct 9, 2026
105 checks passed
@buluoray
buluoray deleted the fix/bg-oneliner-recycle branch October 9, 2026 05:28
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Background one-liners on non-runtime backends grow one shared _bg conversation until prompt-too-long

2 participants