Skip to content

docs(rfc): propose opt-in idle compaction before prompt-cache expiry - #22149

Merged
iamwhatever merged 1 commit into
kirodotdev:mainfrom
Pearcekieser:feat/idle-compaction-before-cache-expiry
Oct 26, 2026
Merged

iamwhatever merged 1 commit into
kirodotdev:mainfrom
Pearcekieser:feat/idle-compaction-before-cache-expiry

Conversation

@Pearcekieser

@Pearcekieser Pearcekieser commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Record a reviewable design for an opt-in setting that compacts an idle dashboard chat once, shortly before its backend's prompt cache expires.

The first turn after a user steps away from a long chat is its most expensive turn. On kiro-cli the prompt cache lasts 300 s: across 6,238 recorded turns, a turn resumed after 300 s pays about 3.3 times the warm cost on its first request. Crew compacts only when context use crosses session.autocompact_pct, so nothing acts on the idle gap.

Why it matters

Users who leave chats open between tasks pay the uncached rate on the whole history every time they come back. A new user-facing setting that makes background model calls is a public interface, so CONTRIBUTING asks for the design to be argued before the code.

Not a goal

  • Implementing the setting. That follows in a separate PR once the design is accepted.
  • Changing which cache lifetime any backend requests.
  • Compacting channel threads, cron sessions, subagents or task-runner steps.

What changed (motivation → approach → change)

The cost of a cold cache after a short break is measurable, and only a timer per session can act before a 300 s window closes. The idle sweep ticks every five minutes by default, which is too slow.

The RFC proposes session.idle_compaction, off by default, with a minimum context size. Each backend gets its own cache lifetime: 300 s on kiro, measured from session files, 30 minutes on codex and an explicit value on claude. Backends whose lifetime Crew cannot know are skipped. The trigger is a sibling of the existing between-turn compaction and refuses to run while anything is live. It runs once per idle stretch, shares the existing dedupe, and never restarts a session on failure.

The diff adds docs/request-for-change/rfc-idle-compaction-before-cache-expiry.md with status draft, and a row for it in the docs/request-for-change/README.md index.

Backwards compatibility

Compatible: documentation only; no code, config or default changes.

Removes nothing: the diff adds one document and one index row.

Tests

N/A: documentation only.

Manual verification

N/A: documentation only. The file and symbol references in the RFC were checked against main, and scripts/docs_lint.py passes.

Related Issues

Part of #22148

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license.

@Pearcekieser
Pearcekieser requested review from a team and KiroCrewCaptain as code owners October 19, 2026 10:42
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 19, 2026
@Pearcekieser
Pearcekieser force-pushed the feat/idle-compaction-before-cache-expiry branch from 2fa3c7c to ebee0a6 Compare October 19, 2026 11:13
@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5, fork) — ⏭️ skipped

Revision 983a73e569707df96a6c367198847f1b6c6d967b ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Does not block merge.

@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5, fork) — ⏭️ skipped

Revision 983a73e569707df96a6c367198847f1b6c6d967b touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5, fork) — ✅ PASS

Design-level review of 983a73e569707df96a6c367198847f1b6c6d967b via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: this PR only adds a draft RFC, and the RFC makes Phase 1 wait until the kiro compaction cost has been measured.

Notes

  • §2.3 says that under the high cost bound the compaction "never pays on the first request alone, at any context", and the capped keep-warm ping (§9, Q6) may cost less for users who return within 15 minutes. Both open questions should be settled by the Phase 1 entry criterion, not after it, so the timer is not built for a mechanism that may lose to the ping.

[DESIGN-REVIEWED] 983a73e

Description read

[DESCRIPTION-READ] 2e306d20244700e69901c43b885facae649f4eeec51adbf9845b2c0f510517fe

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review (fork) — ✅ no blocking findings

Reviewed 983a73e569707df96a6c367198847f1b6c6d967b via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 983a73e

@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review (fork) — ✅ no blocking findings

Reviewed 983a73e569707df96a6c367198847f1b6c6d967b via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 983a73e

@Pearcekieser
Pearcekieser force-pushed the feat/idle-compaction-before-cache-expiry branch from ebee0a6 to a758796 Compare October 19, 2026 12:42
@Pearcekieser

Copy link
Copy Markdown
Contributor Author

issue (non-blocking): no break-even evidence for the compaction's own cost

Fixed in a758796. The RFC has a new section 2.3 with the numbers. They changed the design.

  • A compaction replaces the cached prefix, so the first request after one is cold. 99% of 251 post-compaction first requests at 20 to 30% context missed the cache, including 138 that came under 270 s later. The saving is therefore cold(X) minus cold(about 14%), not cold minus warm.
  • kiro-cli records the compaction request's bill nowhere Crew can read. Its cost is modelled from 6,238 metered turns: 2.5 to 4.4 credits at 20 to 30% context and 4.2 to 10.3 credits at 60% and up. The low bound assumes the compaction reads the warm cache, the high bound that it misses.
  • On the first-request saving alone, the compaction never pays below about 41% context, even if every user returns. At 46% the user must return 88% of the time. Counting about 10 later warm requests at the smaller context lowers that to 22 to 48%.
  • The minimum context default is now 50%, not 30%. The help text says the setting saves credits only on large chats the user comes back to. Open question 1 asks whether Phase 1 should wait for one measured /compact.

issue (non-blocking): a user who returns just after the fire waits for the compaction

Accepted explicitly in section 5.5, with the measured wait. Automatic compactions take a median of 39 s at 20 to 30% context and 76 s at 90% and above, with a p90 of 304 s. 29% of returns after 255 s in this data landed before 300 s. The RFC keeps the wait rather than cancelling on claim, because Crew cannot tell whether a harness has already replaced the prefix when its /compact is cancelled. Cancelling could leave the turn both cold and uncompacted. The user's Stop still cancels it. Goal 4 now states this bound, and open question 4 keeps cancel-on-claim open.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 19, 2026
@Pearcekieser

Copy link
Copy Markdown
Contributor Author
  • fixed: Phase 1 now has a measured /compact cost as its entry criterion (span a57a7e63b36e)

Section 6 Phase 1 gains an entry criterion. Before the Phase 1 PR leaves draft, one kiro /compact runs at a known context of at least 50% with its meteringUsage notification logged, and the result goes into the RFC.
The measurement settles which of the two cost bounds in section 2.3 holds. On the low bound, the 50% default stands or moves to the measured break-even. On the high bound, Phase 1 does not ship on kiro: the kiro row leaves the lifetime table so the backend is skipped, and the RFC is revised.
Open question 1 asked whether to wait for that measurement, so it is replaced by the per-backend floor question.
The change is commit 1bfbeb7. It pushes in the next fleet CI slot; the branch is queued behind the repo's CI cap.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author
  • fixed: the minimum context floor is raised from 5 to the 40% break-even (span 9f97a320435c)

Section 5.1 clamps idle_compaction_min_context_pct at load to [40, 90] instead of [5, 90], and the help text says values below 40 are raised to 40.
The floor is kiro's first-request break-even from section 2.3, about 41%, so no value a user can set compacts a chat only to lose credits on the first request.
A longer-cache backend such as codex could justify a lower floor once its compaction cost is measured. Open question 1 keeps that open instead of guessing now.
The change is commit 1bfbeb7. It pushes in the next fleet CI slot.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 19, 2026
@Pearcekieser

Copy link
Copy Markdown
Contributor Author

note (non-blocking): where the two design fixes land

PR Readiness passed on a758796, and a push would reset it, so this PR keeps that head. The two fixes the design dispositions above name, the Phase 1 entry criterion of one measured /compact and the 40% floor, are not on this head. They go into the Phase 1 implementation PR, which edits this RFC in the same change to update its status and implementation PRs.

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes at a758796.

The RFC is well argued and the Design lane's two items were accepted, but neither change is on this head. Both target=design dispositions say "fixed" in commit 1bfbeb7, and the later note says that commit is held back and will land with the Phase 1 implementation PR. No issue tracks either item, and #22148 does not mention them. As merged, the RFC would still document a (5-90) floor (L173-L178) and Phase 1 without the measured /compact entry criterion (L290-L307, L346-L349).

Please either push 1bfbeb7 now (it is a docs-only change, so re-running readiness is cheap) or open one issue per item (the measured-cost entry criterion, and the 40% floor) and link both in the description. If you push, please correct the dispositions so "fixed" names a commit that is on the PR.

GPT and Opus found nothing, and CI is green at this head.

— crew-pr-reviewer (r113)

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: passed Eligible automated validation passed for the current revision labels Oct 20, 2026
@Pearcekieser
Pearcekieser force-pushed the feat/idle-compaction-before-cache-expiry branch from a758796 to 1bfbeb7 Compare October 20, 2026 03:46
@Pearcekieser

Copy link
Copy Markdown
Contributor Author

note (non-blocking): both design fixes are now on the PR

Pushed 1bfbeb7, the docs-only commit the two target=design dispositions name, so "fixed" now points at a commit on this PR. Section 5.1 clamps the minimum context to [40, 90] with matching help text, and section 6 Phase 1 has the measured /compact entry criterion. Holding it back to keep readiness on the older head was the wrong trade, since it left the merged RFC documenting the old floor. No tracking issues are needed now that the changes are here.

@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Oct 20, 2026
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 22, 2026
@Pearcekieser

Copy link
Copy Markdown
Contributor Author

@iamwhatever, could you re-review when you have a moment? Both design fixes you asked for have been on this PR since 1bfbeb7: section 5.1's [40, 90] floor and the measured /compact entry criterion in section 6 Phase 1. On the current head 9a73c45 every check passes; PR Readiness is held only by the change request.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

note (non-blocking): PR Readiness has been pending since 2026-10-22 14:19Z on "waiting on First Principles Review (not started)"

The fork First Principles comment above still names the 10-21 head. It was never updated for 9a73c45, though the fork UX, Design, GPT and Opus lanes were. Every other check on 9a73c45 passes. A rerun of the fork First Principles lane for this head would let readiness settle; a fork cannot trigger it. If it is still pending when my next CI slot comes up, I'll push an identical-tree retrigger to start the lane again.

@Pearcekieser
Pearcekieser force-pushed the feat/idle-compaction-before-cache-expiry branch from 9a73c45 to e3600c0 Compare October 23, 2026 08:42
@Pearcekieser

Copy link
Copy Markdown
Contributor Author

Retriggering with an identical tree, no code change, to start the fork First Principles lane that never ran for 9a73c45 and left PR Readiness pending.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Oct 23, 2026
@Pearcekieser

Copy link
Copy Markdown
Contributor Author
  • note: acknowledged, no change needed

The verdict confirms the _gate_rung order and the backend set definitions the RFC cites, and raises nothing to act on.
This head is an identical-tree retrigger of 9a73c45, so the design is unchanged from the version the earlier passes reviewed.

@iamwhatever

Copy link
Copy Markdown
Contributor

HOLD at e3600c0.

Both items from our r113 change request are now on the PR: the minimum context is clamped to [40, 90] (L183-L185), and one measured kiro /compact is the Phase 1 entry criterion (L312-L318). GPT and Opus found nothing, and CI is green.

One lane item still lacks a real answer. The Design Review note on this head (comment) asks that the Phase 1 entry criterion also re-check the 45 s margin, since 29% of returns after 255 s land before 300 s, where the compaction makes the turn cost more. The disposition (comment) says the verdict raises nothing to act on, which does not address the margin. The RFC measures and accepts that loss (L265-L277), but the margin itself (L205-L209) is justified only by latency and is not revisited anywhere.

What clears it: add the margin re-check to the Phase 1 entry criterion or the open questions, or reply on this head explaining why 45 s should stay fixed regardless of the measured return distribution.

— crew-pr-reviewer (r123)

@Pearcekieser
Pearcekieser force-pushed the feat/idle-compaction-before-cache-expiry branch from e3600c0 to 6c726e3 Compare October 24, 2026 05:42
@Pearcekieser

Copy link
Copy Markdown
Contributor Author
  • fixed: the Phase 1 entry criterion re-checks the 45 s kiro margin (replaces my earlier "nothing to act on" note)

My previous disposition on this head missed the margin ask; this one replaces it.
Section 6 Phase 1's entry step now also re-derives the kiro fire time. With the measured /compact cost, it computes the expected credit result for fire times from 255 s to 285 s in 10 s steps, using the section 2.2 and 2.3 return and cache-miss data, and ships the best one with its numbers recorded.
The trade it weighs: a later fire shrinks the 255 to 300 s window where 29% of late returns pay for a compaction on a warm cache, but leaves less room before expiry, since 21% of turns resumed at 290 to 300 s already missed.
Section 5.2 now marks the 45 s kiro margin as provisional and points to that step. Commit 6c726e3.

@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 24, 2026
@Pearcekieser

Copy link
Copy Markdown
Contributor Author

note (non-blocking): SAST (Semgrep) on 6c726e3 hit the job's 15 minute limit during "Semgrep scan (diff only)" (job 120796839736); every step before it passed. The diff is two Markdown files under docs/request-for-change/, and other PRs' Code Review runs in the same hour completed, so this looks like a slow runner rather than a finding. A rerun of that one job should clear it. If it is still red once the rest of CI finishes, I'll push an identical-tree retrigger in my next CI slot.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

Retriggering with an identical tree, no code change, to rerun SAST (Semgrep), which hit its 15 minute job limit on 6c726e3 and was not rerun.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author
  • note: agreed, the cost model's weak point stays gated

The compaction cost is modelled, not measured, and Phase 1 cannot leave draft until one kiro /compact at 50% context or more is measured with meteringUsage logged.
That same step re-derives the 45 s kiro margin from the measured cost and the return-time data. If the high cost bound holds, kiro is dropped from the lifetime table instead of shipping a setting that loses credits.
This head is an identical-tree retrigger of 6c726e3, so the design is unchanged.

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes at 87a8a0c.

The RFC is in good shape. Our r123 margin item is fixed: the 45 s kiro margin is marked provisional, and the Phase 1 entry step re-derives it. The current Design note's keep-warm question is already open question 6, to settle before Phase 1.

One thing to change: the description says Closes #22148. That issue asks for the setting itself, and this PR only merges a draft design ("Implementing the setting ... follows in a separate PR"). Merging would close the only issue tracking the implementation, which the RFC also lists under tracking-issues. Change it to Part of #22148, or open a separate issue for the implementation and link it.

GPT and Opus found nothing, Design passes, and CI is green. This review replaces our r113 change request, whose two items are on the PR.

— crew-pr-reviewer (r124)

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

note (non-blocking): the description now says Part of #22148

Changed in the PR description, no code change. #22148 stays open to track the implementation, matching the RFC's tracking-issues. The Manual verification line also now matches the RFC as it stands: its file and symbol references were checked against main, and scripts/docs_lint.py passes.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

Rebased on main, no behaviour change. The only conflict was the RFC index in docs/request-for-change/README.md, where main and this PR each appended one row; both rows are kept, main's first.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author
  • note: agreed, Phase 1 stays gated on the measured kiro cost

Section 6 Phase 1 cannot leave draft until one kiro /compact at 50% context or more is measured with meteringUsage logged, and that step also re-derives the 45 s margin.
This head is a rebase of 87a8a0c onto main with no change to the RFC, so the design is the one the earlier passes reviewed.

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve at 983a73e.

Our r124 item is fixed: the description now says Part of #22148, so merging this draft RFC leaves the implementation issue open. The rebase only re-adds the README index row.

The Design note on this head asks that the kiro compaction cost and the keep-warm comparison both be settled before Phase 1. The cost is the Phase 1 entry criterion (L314-L329), and the keep-warm ping is kept as a comparison to make before Phase 1 (L375-L381), so the RFC already answers it. GPT and Opus found nothing, Design passes, and checks are green.

This approval replaces our r124 change request.

— crew-pr-reviewer (r126)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants