Repository navigation
fix(cron): keep a newer build's store fields through this build's save - #18894
bolichen97 merged 2 commits into
Conversation
UX Review (Opus 5.5, fork) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5, fork) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS The two size caps throw away the very newer-build keys this fix exists to keep, and nothing in the diff or the codebase shows what harm they prevent. Not justified as shipped
What this change shipsInventory (8 items) — 7 justifiedIntent: after a desktop rollback, the older build's ordinary cron save must not delete what a newer build wrote. FIX.
[FIRST-PRINCIPLES-REVIEWED] c0ce2b7 Description read[DESCRIPTION-READ] 53ccb119d7e656231dfebb15a98d4bd3f383fba3fef3e70fbe3ad7b75758f350 That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Opus 5.5, fork) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS When a cap drops a newer build's keys, the save still writes the newer version, so the newer build can't tell anything was removed. Watch
[DESIGN-REVIEWED] c0ce2b7 Description read[DESCRIPTION-READ] 71ad46011407f340b5fb9e06fa4f99111d92fda7a61e8fd08c4539aa60f3c9ad That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
GPT 6.1 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
Opus 5.5 Review (fork) — ✅ no blocking findingsReviewed |
3534901 to
9f5fe51
Compare
After a rollback (the desktop updater allows downgrades), the older build loaded a crons.json a newer build wrote and its first save re-encoded every job from its own field list, dropping every field the newer build added, and stamped the store back to version 2. decode_store keeps the version read when it is higher, the unknown top-level keys, and each loaded job's unknown record and schedule keys. encode_store writes them back fill-only, the rule the config file uses for unknown keys: a key this build writes is never overwritten, a job this build deleted is gone with its keys, and the version is never lowered. A store with no unknown keys encodes exactly as before. What is kept is bounded, measured as the JSON it is written back as: at most 64 KiB per job (its unknown keys with its schedule's) and for the document's unknown top-level keys, and at most 1 MiB for the whole store, the top-level keys first and then each job in file order. An item over its cap, or a job past the store's total, keeps none of them and is logged, so its next save writes it as this build knows it.
The store test's module docstring wrote the product name as one word. The brand-name gate requires "Kiro Crew". No test changes behaviour.
9f5fe51 to
c0ce2b7
Compare
bolichen97
left a comment
There was a problem hiding this comment.
Approved at head c0ce2b7: PR Readiness is green, no open change requests, and the review lanes passed. (AI-assisted review at bolichen97's request.)
Problem / Motivation
Goal: An older build's save keeps the fields a newer build wrote into
crons.json, up to 64 KiB per job and 1 MiB for the whole store, and never stamps the store's version lower.The desktop updater allows downgrades: it runs with
allowDowngrade=trueso that a channel switch-back installs the older version (docs/build/release.md). After a rollback, the older build loads acrons.jsona newer build wrote, and its first ordinary save deletes what the newer build added:CronService._save(src/kiro_crew/cron.py) re-encodes every job throughjob_record(src/kiro_crew/cron_service/store.py), an explicit list of the fields this build knows. Any field a newer build added is dropped, on every job, including jobs the user never touched."version": _STORE_VERSION(2), so a store a newer build wrote at a higher version is stamped back down. After the user upgrades again, the newer build cannot tell that its fields were removed.Why it matters
This already happened once.
managed_bywas added by merged #17683, and a build from before it deletes the field on its first save, so the installer that owns the job can no longer find it. The writer's own comment said an older build "simply ignores the extra key". That is true for reading, not for the rewrite.Not a goal
is_duereturns False for an unknown kind). That is the safe direction, and this change does not touch it._kept_sizelogs each,cron_service/store.py:651-668). Nothing limits how many jobs a store holds, so the whole store keeps at most 1 MiB of them: the top-level keys first, then each job in file order, and a job whose keys would take the total past 1 MiB keeps none of them. One warning per load counts those jobs (store.py:815-828). The next save writes each such job, or the document, as this build knows it, as main does with every unknown key today. Keeping more would mean merging them back from the file at save time under the store lock; this PR does not.What changed (motivation → approach → change)
encode_storerebuilds each job fromjob_record, which writes only this build's fields, and always writes_STORE_VERSION; nothing carried the unknown keys or the higher version from the load to the save.src/kiro_crew/cron_service/store.py:decode_store(raw)returns aStoreDocumentwith the jobs (decode_jobsis now a thin wrapper over it), the version read when it is an integer above this build's, the top-level keys other thanversionandjobs, and, for each job that loads, the keys of its record and of itsschedulethat this build does not know.encode_store(jobs, *, version, extra, unknown_fields)writes them back fill-only, the ruleKiroCrewConfig.to_dictalready applies to unknown config keys (restore_extra_section_keys,src/kiro_crew/config/loader.py): a key this build writes itself is never overwritten; only a job still injobsgets its unknown keys back, so a job this build deletes is gone with them; and the version is never written lower than the one read. A store with no unknown keys encodes exactly as before.decode_storekeeps it, by two named constants, both measured as the JSON text it is written back as (_kept_size,cron_service/store.py:651).MAX_UNKNOWN_FIELDS_BYTES(64 KiB,:642) bounds one item: a job's unknown record and schedule keys together (_kept_size,:742) and the document's unknown top-level keys (:778).MAX_UNKNOWN_FIELDS_STORE_BYTES(1 MiB,:648) bounds the store's total, the top-level keys first and then each job in file order (:815). An item over its cap, or a job past the total, keeps none of them and is logged. The version is an integer and is kept regardless._KNOWN_JOB_KEYSisjob_recordof a job that sets every conditionally written key (managed_by), and_KNOWN_SCHEDULE_KEYSisfields(CronSchedule), so the two cannot drift. A key this build knows and clears, such asmanaged_byset to "", stays cleared and is not restored.src/kiro_crew/cron.py:CronService._loadkeeps the document beside the jobs (_keep_store_document) and resets it when the file is missing or does not load (_forget_store_document);_savepasses it toencode_store. Class-level defaults hold this build's own empty document, so a service built without__init__still saves. Every load replaces the state whole, and nothing edits it in place. Main's fix(cron): keep a busy merge's run record across store reloads #18628 swaps a reloaded job list in only after it re-applies the run records a busy store refused, and its held-record save writes copies of the listed jobs. So_loadkeeps the document right after that swap (_keep_store_document,src/kiro_crew/cron.py:5101), and_savewrites the kept fields into whichever list it is given (encode_store,src/kiro_crew/cron.py:5255-5256), including the copies_save_held_run_records_lockedwrites (src/kiro_crew/cron.py:2463), since the fields are matched by job id (fields_by_job,cron_service/store.py:691).crons.json: the import sanitizer (portability._sanitize_cron_text), the snapshot merge (snapshot_merge._merge_crons) and the execution-context migration (cron_service/identity.py) all edit the parsed document and write it back withjson.dumps, so they already keep unknown keys.CronService._saveis the only writer that re-encodes jobs.restore_extra_section_keys, andjob_recorditself as the source of the known keys.Backwards compatibility
Compatible: a store this build wrote has no unknown keys and encodes byte-for-byte as before (pinned by the store-bytes fixtures in
test/test_cron_refactor_contract.py); only a store a newer build wrote keeps more than it did.Removes nothing: the change keeps keys and a version that were dropped before; it deletes no field, key or function.
Security: no redaction, auth or sandbox rule is touched.
Tests
In short: red on origin/main 64e1cbb, 7 failed and 3 passed in each of 3 runs; green on this branch, the file 10 passed in 3 of 3 runs, the tests of the two cron changes merged since pass, and the 56 neighbouring test files 4,050 passed, 36 skipped. The commands and failures are below.
New file
test/test_cron_store_newer_version.py, driving a realCronServiceon atmp_pathstore that a newer build wrote (version 3, an unknown job field, an unknown schedule field and an unknown top-level key):test_an_older_builds_save_keeps_a_newer_builds_fields_and_version: after one ordinary save, the job field, the schedule field, the top-level key and version 3 survive.test_a_known_field_this_build_changes_is_written_with_its_new_value: the new value of a known field wins, and the unknown key beside it is kept.test_a_job_this_build_deletes_is_gone_with_its_unknown_keys: a deleted job does not come back through its unknown keys.test_a_16_mib_field_from_a_newer_build_is_not_kept_in_memory: a job with a 16 MiB unknown field keeps none of it in the service's memory, the refusal is logged, the save writes that job without it, and a second job under the cap keeps its field.test_the_cap_keeps_a_job_at_the_cap_and_refuses_one_byte_over: a job whose unknown keys serialize to exactly 64 KiB keeps them through the save; one byte more keeps none.test_a_16_mib_top_level_key_from_a_newer_build_is_not_kept: a 16 MiB unknown top-level key is not kept or written back, and version 3 still is.test_many_jobs_past_the_store_total_keep_only_what_fits: 40 jobs carry a 60 KB unknown field each, every one under the per-job cap and 2.4 MB together. The store keeps at most 1 MiB, the first 17 jobs in file order keep theirs, the other 23 keep none, one warning counts the 23, and the save writes the first job's field and not the last's.test_a_store_from_this_version_round_trips_unchanged,test_a_known_field_this_build_clears_stays_cleared,test_a_job_with_an_unknown_schedule_kind_still_never_fires.Red on origin/main (64e1cbb, this test file copied into a clean checkout,
pytest -n 0, 3 of 3 serial runs): 7 failed, 3 passed each. Three fail on the defect:The two
KeyErrorfailures are the kept-field assertions of the second and third tests: on main the unknown key is gone after the save. The four cap tests read state main does not have (AttributeErroron_unknown_fieldsand_store_extra,ImportErroron the cap), so they cannot run there. The 3 passes are the controls.The cap tests are the witness for the caps. These witness runs were made on 843abe3; the rebase onto 64e1cbb changes neither
cron_service/store.pynor the test file. On this change with the per-item cap removed (the same tree before it was added, 3 of 3 serial runs), they fail, 3 failed, 6 passed:With the per-item cap and without the store total (3 of 3 serial runs), the many-jobs test fails, 1 failed, 9 passed: all 40 jobs keep their fields.
Green on this branch (rebased onto origin/main 64e1cbb): the file passes 10 of 10 in 3 of 3 serial runs. The tests of the two cron changes merged since, which the rebase keeps, pass in 3 of 3 serial runs:
test/test_cron_merge_result_reload.pyandtest/test_portability.py(#18628), 185 passed, 5 skipped;test/test_cron_refused_one_shot_retry_floor.pyandtest/test_cron_undispatched_one_shot.py(#18591), 10 passed. The 56 test files that name the store's decoder, encoder orcrons.json, the new file first, includetest_cron_refactor_contract.py, whose store-bytes fixtures pinencode_store(decode_jobs(doc)) == json.dumps(doc)for this build's own documents: 4,050 passed, 36 skipped. Seven files that read the real home directory were left out of that run. A second set, the 31 cron test files that namecancel,_force_reap,_merge_terminal_state_lockedoruser_paused(withtest/test_cron_merge_result_cascade.pyadded and two that read the home directory left out), run serially: 1,183 passed.Manual verification
N/A: unit coverage is sufficient; the tests write a newer build's store to disk, run a real
CronServicesave over it, and read the file back.Related Issues
Related: merged #17683 (added
managed_by, the field an older build drops). On the earlier base 843abe3, all 629 open PRs were checked withgit merge-tree, with and without this change. 13 touchedcron_service/store.pyorcron.py; 11 of them had the same conflicts either way. Two gained onecron.pyconflict, both on neighbouring lines, and each resolves by keeping both sides:feat(cron): per-job project directory so agent crons load repo steering #13408: both add a name to the same
typingimport list (Mappinghere,Sequencethere).fix(cron): make cron store writes crash-durable #16859: this change's
_forget_store_document()sits beside its_save_failed = Falseat the top of_load, and both edit theencode_store(...)line in_save.fix(cron): keep a busy merge's run record across store reloads #18628, merged since: both edited the
encode_store(...)call inCronService._saveand the reloaded job list in_load. This branch is rebased onto it and keeps both. That PR passes the list to write through and swaps a reloaded list in under its guard; this one adds the version and preserved-field keywords, and keeps the document after that swap. The call now reads:fix(cron): keep a one-shot whose fire never started scheduled #18591, merged alongside it: it changes other lines of
cron.py(a one-shot whose fire never started stays scheduled) and none of this change's.Pattern harvest
Rule candidate: review-prompt
Pattern: a store writer that re-encodes records from an explicit field list, so a rollback's first save deletes every field a newer build added.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)