Repository navigation
docs(rfc): propose opt-in idle compaction before prompt-cache expiry - #22149
iamwhatever merged 1 commit into
Conversation
2fa3c7c to
ebee0a6
Compare
First Principles Review (Opus 5.5, fork) — ⏭️ skippedRevision |
UX Review (Opus 5.5, fork) — ⏭️ skippedRevision |
Design Review (Opus 5.5, fork) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: this PR only adds a draft RFC, and the RFC makes Phase 1 wait until the kiro compaction cost has been measured. Notes
[DESIGN-REVIEWED] 983a73e Description read[DESCRIPTION-READ] 2e306d20244700e69901c43b885facae649f4eeec51adbf9845b2c0f510517fe That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
GPT 6.1 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
Opus 5.5 Review (fork) — ✅ no blocking findingsReviewed |
ebee0a6 to
a758796
Compare
|
issue (non-blocking): no break-even evidence for the compaction's own cost Fixed in a758796. The RFC has a new section 2.3 with the numbers. They changed the design.
issue (non-blocking): a user who returns just after the fire waits for the compaction Accepted explicitly in section 5.5, with the measured wait. Automatic compactions take a median of 39 s at 20 to 30% context and 76 s at 90% and above, with a p90 of 304 s. 29% of returns after 255 s in this data landed before 300 s. The RFC keeps the wait rather than cancelling on claim, because Crew cannot tell whether a harness has already replaced the prefix when its |
|
|
|
note (non-blocking): where the two design fixes land PR Readiness passed on a758796, and a push would reset it, so this PR keeps that head. The two fixes the design dispositions above name, the Phase 1 entry criterion of one measured |
iamwhatever
left a comment
There was a problem hiding this comment.
Request changes at a758796.
The RFC is well argued and the Design lane's two items were accepted, but neither change is on this head. Both target=design dispositions say "fixed" in commit 1bfbeb7, and the later note says that commit is held back and will land with the Phase 1 implementation PR. No issue tracks either item, and #22148 does not mention them. As merged, the RFC would still document a (5-90) floor (L173-L178) and Phase 1 without the measured /compact entry criterion (L290-L307, L346-L349).
Please either push 1bfbeb7 now (it is a docs-only change, so re-running readiness is cheap) or open one issue per item (the measured-cost entry criterion, and the 40% floor) and link both in the description. If you push, please correct the dispositions so "fixed" names a commit that is on the PR.
GPT and Opus found nothing, and CI is green at this head.
— crew-pr-reviewer (r113)
a758796 to
1bfbeb7
Compare
|
note (non-blocking): both design fixes are now on the PR Pushed 1bfbeb7, the docs-only commit the two |
|
@iamwhatever, could you re-review when you have a moment? Both design fixes you asked for have been on this PR since 1bfbeb7: section 5.1's [40, 90] floor and the measured |
|
note (non-blocking): PR Readiness has been pending since 2026-10-22 14:19Z on "waiting on First Principles Review (not started)" The fork First Principles comment above still names the 10-21 head. It was never updated for 9a73c45, though the fork UX, Design, GPT and Opus lanes were. Every other check on 9a73c45 passes. A rerun of the fork First Principles lane for this head would let readiness settle; a fork cannot trigger it. If it is still pending when my next CI slot comes up, I'll push an identical-tree retrigger to start the lane again. |
9a73c45 to
e3600c0
Compare
|
Retriggering with an identical tree, no code change, to start the fork First Principles lane that never ran for 9a73c45 and left PR Readiness pending. |
|
|
HOLD at e3600c0. Both items from our r113 change request are now on the PR: the minimum context is clamped to [40, 90] (L183-L185), and one measured kiro One lane item still lacks a real answer. The Design Review note on this head (comment) asks that the Phase 1 entry criterion also re-check the 45 s margin, since 29% of returns after 255 s land before 300 s, where the compaction makes the turn cost more. The disposition (comment) says the verdict raises nothing to act on, which does not address the margin. The RFC measures and accepts that loss (L265-L277), but the margin itself (L205-L209) is justified only by latency and is not revisited anywhere. What clears it: add the margin re-check to the Phase 1 entry criterion or the open questions, or reply on this head explaining why 45 s should stay fixed regardless of the measured return distribution. — crew-pr-reviewer (r123) |
e3600c0 to
6c726e3
Compare
|
|
note (non-blocking): |
|
Retriggering with an identical tree, no code change, to rerun |
|
iamwhatever
left a comment
There was a problem hiding this comment.
Request changes at 87a8a0c.
The RFC is in good shape. Our r123 margin item is fixed: the 45 s kiro margin is marked provisional, and the Phase 1 entry step re-derives it. The current Design note's keep-warm question is already open question 6, to settle before Phase 1.
One thing to change: the description says Closes #22148. That issue asks for the setting itself, and this PR only merges a draft design ("Implementing the setting ... follows in a separate PR"). Merging would close the only issue tracking the implementation, which the RFC also lists under tracking-issues. Change it to Part of #22148, or open a separate issue for the implementation and link it.
GPT and Opus found nothing, Design passes, and CI is green. This review replaces our r113 change request, whose two items are on the PR.
— crew-pr-reviewer (r124)
|
note (non-blocking): the description now says Changed in the PR description, no code change. #22148 stays open to track the implementation, matching the RFC's |
|
Rebased on main, no behaviour change. The only conflict was the RFC index in |
|
iamwhatever
left a comment
There was a problem hiding this comment.
Approve at 983a73e.
Our r124 item is fixed: the description now says Part of #22148, so merging this draft RFC leaves the implementation issue open. The rebase only re-adds the README index row.
The Design note on this head asks that the kiro compaction cost and the keep-warm comparison both be settled before Phase 1. The cost is the Phase 1 entry criterion (L314-L329), and the keep-warm ping is kept as a comparison to make before Phase 1 (L375-L381), so the RFC already answers it. GPT and Opus found nothing, Design passes, and checks are green.
This approval replaces our r124 change request.
— crew-pr-reviewer (r126)
Problem / Motivation
Goal: Record a reviewable design for an opt-in setting that compacts an idle dashboard chat once, shortly before its backend's prompt cache expires.
The first turn after a user steps away from a long chat is its most expensive turn. On kiro-cli the prompt cache lasts 300 s: across 6,238 recorded turns, a turn resumed after 300 s pays about 3.3 times the warm cost on its first request. Crew compacts only when context use crosses
session.autocompact_pct, so nothing acts on the idle gap.Why it matters
Users who leave chats open between tasks pay the uncached rate on the whole history every time they come back. A new user-facing setting that makes background model calls is a public interface, so CONTRIBUTING asks for the design to be argued before the code.
Not a goal
What changed (motivation → approach → change)
The cost of a cold cache after a short break is measurable, and only a timer per session can act before a 300 s window closes. The idle sweep ticks every five minutes by default, which is too slow.
The RFC proposes
session.idle_compaction, off by default, with a minimum context size. Each backend gets its own cache lifetime: 300 s on kiro, measured from session files, 30 minutes on codex and an explicit value on claude. Backends whose lifetime Crew cannot know are skipped. The trigger is a sibling of the existing between-turn compaction and refuses to run while anything is live. It runs once per idle stretch, shares the existing dedupe, and never restarts a session on failure.The diff adds
docs/request-for-change/rfc-idle-compaction-before-cache-expiry.mdwith statusdraft, and a row for it in thedocs/request-for-change/README.mdindex.Backwards compatibility
Compatible: documentation only; no code, config or default changes.
Removes nothing: the diff adds one document and one index row.
Tests
N/A: documentation only.
Manual verification
N/A: documentation only. The file and symbol references in the RFC were checked against main, and
scripts/docs_lint.pypasses.Related Issues
Part of #22148
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license.