Repository navigation
feat(apps): cap how many finished job records a kind keeps - #33196
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks. It's an opt-in, keyword-only cap with a default that keeps every record, and its first caller is the backup jobs. Every reader of the removed records is listed in the description and backed by a test. Notes
[DESIGN-REVIEWED] 3a9b1c7 Description read[DESCRIPTION-READ] 001d379cddd3d301a5b417ee68157c6f41092d7f7443237b2b1955ea7db3c4d2 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS The cap has a real caller and a maintainer decision (#7588), and the backup readers' windows (20 and 100) sit at or below the cap of 100. What's left is one second-spelling scan that pulls in a new key limit. Notes
What this change shipsInventory (7 items) — 5 justifiedIntent: stop a Job SDK kind's finished-run records from growing forever, starting with the nightly backups. This is an ADDITION.
[FIRST-PRINCIPLES-REVIEWED] 3a9b1c7 Description read[DESCRIPTION-READ] aef617c4080ff58fba7704b0a91d670721778d76ff4a530e5a51c4d1e93f9033 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findings (all downgraded on adjudication)GPT 6.1 flagged blocking issues on This comment is updated in place on each push. Review detailsBLOCKING -- src/kiro_crew/apps/job_sdk.py:1023 -- Terminal writes outside workers bypass retention
Adjudication (Opus 5.5) — is blocking on each finding proportionate?F1 — harm rung: LOW. Both conditions are confirmed. A refused thread start persists a FAILED record without sweeping ( False positive or not applicable? A repository writer can comment: |
ebab280 to
1183bb0
Compare
|
[operator: bolichen97]
AI-assisted, posted on behalf of bolichen97 |
|
[operator: bolichen97]
AI-assisted, posted on behalf of bolichen97 |
|
[operator: bolichen97]
AI-assisted, posted on behalf of bolichen97 |
1183bb0 to
75dc34c
Compare
The Job SDK writes one record file per run and removes them only when the app is disabled, so a kind's store grows with every run. register() takes an opt-in per-kind retention count: after each terminal write the sweep removes that kind's oldest finished records until at most that many remain. It never removes a run still running or one whose own sweep has not run yet, and it reads each record without following a link. aws_control's two backup kinds opt in at 100, at or above every reader's window. The liveness predicate and eviction callback are left to #33058. Builds on #22711. Closes #7588
75dc34c to
3a9b1c7
Compare
|
[operator: bolichen97]
AI-assisted, posted on behalf of bolichen97 |
|
[operator: bolichen97] The shard 6 red on 75dc34c (Linux and Windows, plus Coverage Gate) was AI-assisted, posted on behalf of bolichen97 |
Kiro Crew's Job SDK gains an opt-in per-kind retention cap (kirodotdev/KiroCrew#33196). Both backup kinds register with retention=100, at or above every reader's window, so the nightly loop's records stop growing forever. A host older than the option raises TypeError on the keyword, and the kind is then registered without it, keeping every record as before. Co-authored-by: KiroCrewCaptain <312312717+KiroCrewCaptain@users.noreply.github.com>
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Approved on head 3a9b1c7: PR Readiness is green, every AI review lane on this head passes with no blocking findings, and no change request stands.
|
[operator: bolichen97] Decision 6 of #26967 (comment) (an agent-evidenced Removing a shipped entry is refused by Fast Gate's Needs a decision from a maintainer: leave the entry as shipped, or supersede it with an entry quoting you. If it goes, the rationale to keep is one bullet in app-kit-platform.md §26.1: the cap ships with its first caller, so a consumer tests its shape; the backup kinds do no work after a run finishes, so a status-only cap is safe for them; The frozen Goal / Why it matters / Not a goal sections do not cite the entry (they cite the #7588 comment), so nothing in them changes. |
Problem / Motivation
Goal: Let an app bound how many finished job records each kind keeps, so a store that gains a run every night stops growing forever, starting with the backup jobs.
The Job SDK writes one record file per run and removes records only when the app is disabled.
list_recent'slimitcaps the response, not the store. So every kind's file count grows with every run, and every read that needs the whole set (list_active,list_recent,reconcile) opens every file the app ever wrote. The backup jobs already run on the Job SDK: the nightly loop adds a record per due kind every night, and the Backup row reads the whole store four times on each status read.Why it matters
Without a bound, a long-lived install pays a little more on every backup status read, forever, and an app that moves onto the Job SDK inherits the same growth. The cap lands with a real caller, as the maintainer decision on #7588 asks, so its shape is tested by a consumer before other apps depend on it.
Not a goal
live,on_evict). A kind whose app keeps working on a run after its status turns terminal must not set the cap; that half is Job SDK: let a kind say a finished run is still using its resources #33058.retentionkeeps every record, as today.What changed (motivation → approach → change)
Records pile up because nothing removes them, so
registergains an opt-in count cap and the backup jobs use it.register(kind, fn, retention=N)Nremain.0, the default, keeps everything.job_sdk_retention.py, a new owner behind thejob_sdkfacade)finished_at,updated_at,created_at; there is no new field. Never raises, runs one at a time per app (a run is let go before the next sweep scans), and removes nothing once the app is disabled.jobs/jsonl_util.open_regular_nofollow, so a planted link, a non-regular or oversized file, or a body naming another run is skipped. Not the agent-facing read: its path screen refuses aws-control's own data dir.dedupe_keyValueError; the start route answers 400invalid_dedupe_key). It is the one record field a caller sets with no length bound, and a record too big for the scan would never be removed.jobs.job_evicted, andreconcilewill not write back a record that is gone. The start route answers with the run as it stood at the start when a fast run's record is gone before it reads it back.retention=100, at or above every reader's window (the Backup row reads 20 of a kind,_jobs/recentat most 100).The App Kit API reference documents the option, including when not to use it, the key limit, and that an older host raises
TypeError. The app-kit-platform spec gains §26.1, anddocs/decisions/records the maintainer decision.The app repository carries the same registration in KiroCrewCaptain/kirocrew-app-aws-control#1 (commit d9243e312c): it falls back to an uncapped kind on a host without the option.
Builds on #22711 by @dodgeblaster. It keeps that PR's sweep, store delete, no-follow scan, audit event and reconcile guard, and drops
live,on_evictand theseqcounter.Backwards compatibility
Compatible:
retentionis keyword-only with a default that keeps every record, so an app that does not pass it behaves exactly as on main, longdedupe_keyvalues included.The backup jobs now remove their oldest finished records past 100 per kind, and their start route refuses a
dedupe_keyover 512 characters. No client sends one: the backup page starts runs through its ownbackup/{account}/runroute with the 12-digit account as the key, and the nightly loop does the same. Readers of those records:Reader: src/kiro_crew/apps/builtins/aws_control/backend/routes_parts/backups.py:_account_jobs -- app -- test_the_backup_row_still_reports_the_newest_failure_after_the_cap
Reader: src/kiro_crew/apps/job_routes.py:_handle_recent -- app -- test_a_run_the_retention_cap_removed_reads_as_unknown
Reader: src/kiro_crew/apps/job_routes.py:_handle_get -- app -- test_a_run_the_retention_cap_removed_reads_as_unknown (a removed run answers 404
job_not_found)Reader: src/kiro_crew/apps/job_routes.py:_handle_cancel -- app -- test_a_run_the_retention_cap_removed_reads_as_unknown (a removed run answers 404
job_not_foundinstead of 409; no page holds such an id)Reader: src/kiro_crew/apps/job_routes.py:_handle_start -- app -- when the run it reads back is already gone it answers 200 with the run as it stood at the start, instead of 500
run_unreadable: test_a_start_whose_run_the_cap_removed_still_answers_the_run; a capped kind's overlong key answers 400: test_a_capped_kind_answers_400_to_an_overlong_keyReader: src/kiro_crew/apps/job_routes.py:_handle_active -- app -- only finished records are removed: test_an_executing_run_is_never_removed
Reader: src/kiro_crew/apps/builtins/aws_control/backend/backup.py:_run -- app -- its
sdk.getreads its own executing run, which is never removed: test_an_executing_run_is_never_removedReader: src/kiro_crew/apps/job_sdk.py:JobSDK.reconcile -- app -- test_reconcile_does_not_bring_back_a_removed_record
Reader: src/kiro_crew/apps/builtins/aws_control/hooks.py:_register_job_runners -- app -- test_both_backup_kinds_register_with_the_cap, test_the_cap_is_at_least_every_readers_window
Tests
test/test_job_sdk_retention.py(new):retentionkeeps every record; a capped kind settles at the cap and keeps the newest; a cap on one kind leaves another alone;reconciledoes not bring a removed record back;test/test_job_routes.py: a run the cap removed is gone fromrecentand answers 404 togetandcancel, while the kept run is served; a start whose run the cap removed before the read-back still answers 200 with the run; a capped kind answers 400invalid_dedupe_keyto an overlong key without quoting it.test/test_job_sdk_facade.pylists the new owner.Determinism: the two files' 59 tests passed at
-n0in three shuffled orders (seeds 4242, 31337, 777), and the four key-limit tests passed 20/20 at-n0. The refusal tests fail with the length check removed and pass with it. They wait on record file names, the live table and each run's own sweep, never on a sleep, and never patch the clock.Manual verification
N/A — unit coverage is sufficient: the change is a backend module and one registration call with no screen of its own, and the sweep, its order and every reader are exercised by the tests above.
Screenshots / video
N/A — no frontend file is touched.
Related Issues
Closes #7588
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)