Repository navigation
fix(cron): keep only each job's 20 newest run transcripts - #31757
Conversation
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing to check. What this change shipsInventory (6 items) — 6 justifiedIntent: stop frequent stateless cron jobs from piling up run files until the gateway stops answering. This is a FIX (#31734).
(The [FIRST-PRINCIPLES-REVIEWED] b5085b4 Description read[DESCRIPTION-READ] e29fd6a7ac4fe2b84972669f1e666ead6ad8e41a9db48f11a83c0f44f1766bf8 That sha256 names the PR description this verdict read. Recompute it with |
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks. Notes
[DESIGN-REVIEWED] b5085b4 Description read[DESCRIPTION-READ] 42472f31e1bbddd31c2edfa7b8140d15f290ce9c591295ed60960df6f093b0f3 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
|
Backend Tests (3.12, 7) and (Windows) (7) fail on |
|
Rebased: main moved in docs/system-specs/modules/learn-cron-dashboard.md since this head went green (39717a0 -> 7c364bc); CI re-running. The new base carries the #31771 fix for the pull_request_target allowlist red. The same push drops the unused keep_per_job parameter (First Principles note); behavior is unchanged. |
a4fd6cc to
c753e29
Compare
|
c753e29 to
af0fd83
Compare
af0fd83 to
c8ca52b
Compare
|
|
|
Backend Tests (3.12, 2), Windows (2) and the Coverage Gate fail on |
c8ca52b to
b5085b4
Compare
|
Rebased: main moved in src/kiro_crew/slack/gateway_runtime/cron_turn.py, cron_fire.py, cron_service/service/run_teardown.py and docs/system-specs/modules/learn-cron-dashboard.md (base b65af1a -> 20764b5). The only conflict was the TYPE_CHECKING import list in cron_turn.py: main added auto_job_approver and this PR adds agent_sequence_dispatches, so both stay. Re-ran test_cron_run_residue.py, test_cron_ephemeral_session.py, test_security_gate_liveness.py (47 passed) and the cron_turn/cron_fire/run_teardown tests (96 passed). test_app_script_cron_sibling_import.py and test_cron_run_scratch.py fail at collection on main 20764b5 too. New head b5085b4. |
Co-authored-by: Joe Guo <zejiangg@amazon.com> (cherry picked from commit 072b98f) Backport note: cron_turn.py's TYPE_CHECKING list gains only agent_sequence_dispatches (auto_job_approver is #30692, not on release). learn-cron-dashboard.md takes the prune wording without the new_chat_per_run clause and Retention bullet, which release does not have.
Co-authored-by: Joe Guo <zejiangg@amazon.com> (cherry picked from commit 072b98f) Backport note: cron_turn.py's TYPE_CHECKING list gains only agent_sequence_dispatches (auto_job_approver is #30692, not on release). learn-cron-dashboard.md takes the prune wording without the new_chat_per_run clause and Retention bullet, which release does not have.
Problem / Motivation
Goal: A stateless cron job's run files stay near 20 per job, however often it fires, so busy jobs can no longer bury the gateway in session files.
A cron job with
persistent_session: falsewrites onecron_<job>_<run>.jsonlfile per run into~/.kiro/crew/sessions/. The cleanup deletes a run file only after 7 days, and at most 200 per hourly pass. A job that fires every minute leaves about 10,000 files. A user had about 50,000, mostly from ops-mission-control's dispatch and rotation-check crons. After upgrading to 0.12.0.8 their gateway booted but never answered andkirocrew tokenkept saying port 5476 was up but busy. Deleting the files fixed it.Why it matters
Anyone running a frequent stateless cron (an app's poller, a dispatch loop) slowly fills the sessions folder. Every full session listing then pays one file read per run, until the dashboard stops loading. Users can't see why, and reinstalling doesn't help because the files stay.
Not a goal
What changed (motivation → approach → change)
The 7-day age limit can't bound the pile, because the count depends on how often a job fires. So each job now keeps only its 20 newest run files, plus the age limit. Recent runs stay for debugging, and each run's result and trace stay in cron history either way.
A keep count only holds if refused fires don't fill it. So the
cron_run="ephemeral"tag is written when the run starts, inslack/gateway_runtime/cron_turn.py_cron_run_single, right after its key is marked active, instead of incron_fire.py_cron_capturebefore the gate. A refused fire writes no file, and a tagged file is held by a live key until its run ends, so the prune can't remove it mid-run.The rules that already protected a file still hold: it must be tagged, not held by a live run, and not pinned. The prune is in
cron_service/run_residue.py(ranking and counts) andcron_service/service/run_teardown.py(the reaper's timing). The cron spec (learn-cron-dashboard.md) names the new rules.Backwards compatibility
Compatible: no setting, API or file format changes. Old installs just lose more of their old run files.
Reader: src/kiro_crew/cron_history.py:CronHistoryStore -- cron -- run summary, error and trace live in their own store, untouched by this prune
Reader: src/kiro_crew/cron_service/service/run_kill.py:_run_session_keys -- cron -- reads only a live run's key, which
active_session_keyskeeps out of the pruneReader: src/kiro_crew/dashboard/chat_utils_slots.py -- chat -- maps a live run's key to the job's tab; live runs are never pruned
Reader: src/kiro_crew/history.py:ConversationLog -- chat -- History lists what is on disk; a pruned run drops out, as a 7-day-old one already does
Reader: src/kiro_crew/slack/gateway_runtime/cron_delivery.py -- cron -- re-tags a run that opened a tab to
tab; delivery runs after the start-of-run tag, sotabstill wins (test_cron_new_chat_per_run.py)Tests
test/test_cron_run_residue.pyadds:test/test_cron_ephemeral_session.pycovers the tag site:Proof: not provable -- other: prove.py reverts the new RUN_TRANSCRIPT_KEEP_PER_JOB constant the test imports, so the run errors at collection. By hand on origin/main with the constant inlined in the test: 4 failed (the busy-job test and the three reaper tests), 12 passed. With cron_fire.py and cron_turn.py restored from the parent, 5 of the tag-site tests fail (both refusal tests, the active-before-tag test, and the two stateless capture cases).
Determinism: 10 repeats all passed, and a run in reversed test order passed.
Manual verification
N/A — unit coverage sufficient: the change is file selection and a timer, both exercised directly with real files in a temp sessions folder.
Related Issues
Closes #31734
Sibling sweep
Swept: git grep -nE "retention_days|_TTL_SECS|PRUNE_CAP" -- src/kiro_crew
Sibling: src/kiro_crew/crew_log/store_retention.py -- not affected -- one log per closed session, not one per timer fire
Sibling: src/kiro_crew/history_files.py -- not affected -- archive files are one per closed session, at the rate a person closes chats
Pattern harvest
Not generalizable: the only file written once per timer fire is this per-run transcript; other retention paths store one file per session.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)