Repository navigation
feat(skills): audit registered crons for a cheaper execution mode - #9592
Conversation
Design Review (Fable 5) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS The refusal-over-guess principle stops at script detection: Latin non-English prompts get high-confidence verdicts from English blocker lists that cannot read them. WatchThe scanner hard-refuses CJK prompts but judges German/French ones on English word lists anyway — its own comment concedes a "fasse zusammen und entscheide" job "would be cleared for script mode precisely because the blockers could not see it", yet Suggestions
[DESIGN-REVIEWED] 1f8b4ec |
Opus 4.8 Review — ✅ no blocking findingsReviewed Review detailsBased on my analysis of the diff and the single candidate: Candidate 1 (Latin-script non-English prompt cleared for script rewrite): This is an intended, documented design behavior, not a defect. I re-derived the load-bearing security claims independently: ownership scoping happens upstream of No findings. [OPUS-REVIEWED] 1f8b4ec Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
First Principles Review (Fable 5) — 🟡 CONCERNSPremise-level review of All load-bearing claims are verified: First-Principles-Verdict: CONCERNS The regex classifier ships three refusal branches patching a blindness it created itself — the consumer is a model that reads any language. Not justified as shipped
What this change shipsInventory (8 items) — 6 justifiedIntent: help a user cut token spend on already-registered cron jobs by auditing them for cheaper execution modes — an ADDITION.
Watch
Subtractions
[FIRST-PRINCIPLES-REVIEWED] 1f8b4ec |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsFINDING -- src/kiro_crew/builtin_skills/cron-cost-optimize/scripts/cron_cost_scan.py:401 -- unknown history still yields False positive or not applicable? A repository writer can comment: |
|
Superseded, and no longer a disposition record. It was written against head It also claimed
needs-a-decision: should |
|
Disposition: needs-a-decision -- Blocker (cron-history is OS-masked from every sandboxed agent shell, exactly where SKILL.md tells the agent to run the scan)
needs-a-decision: which surface should carry ownership-scoped job records + per-job history stats to a builtin-skill script running in its own sandboxed shell -- an extended |
UX Review (Fable 5) — ⏭️ skippedRevision |
|
Both blockers are resolved in The GPT finding was right about the store. Every cron MCP tool scopes a caller to the jobs it owns, via The Design finding was worse than a degradation. The fix is the one surface both reviews named.
The scanner now reads that document on stdin and opens nothing. Its folding logic moved to Also picked up 34 new tests across the three pieces. Full local run: 390 passing over the new suites plus the existing |
fd8cf8b to
72765b2
Compare
72765b2 to
44fb4e2
Compare
e26746d to
105b3c9
Compare
|
Round 3. The two advisory findings are fixed on Fixed: function-local imports. Fixed: the description contradicted the partial-fetch behaviour. It said "every job's history is null" when a partial read keeps the history it did fetch. It now says each UNFETCHED job's history is null and states that a partial read keeps the rest. My own code comment and Rebutting:
There is also a correctness reason not to drop @bolichen97 this one needs your call, since only a repository writer can clear it: Local state on |
|
/ai-review override gpt 105b3c9: --input reads with the calling shell own privileges so it grants nothing beyond cat, and .aws is bind-masked in every sandbox mode, so the stdin-only fix would relocate the same read rather than close it; authorized by bolichen as a repository writer. |
Human judgment recorded@bolichen97 marked the gpt AI finding as false positive, not applicable, or explicitly accepted for
This decision applies only to this commit. A new push requires a new judgment. |
|
Human judgment recorded, but the following reviewer lane(s) could not be re-run automatically: GPT 5.6. Re-run the lane's latest workflow run for |
105b3c9 to
31e006a
Compare
|
Rebased onto The scanner file is byte-identical across the rebase, so the previously authorized override still applies unchanged and only needs re-anchoring to the new head: /ai-review override gpt 31e006a: --input reads with the calling shell own privileges so it grants nothing beyond cat, and .aws is bind-masked in every sandbox mode, so the stdin-only fix would relocate the same read rather than close it; authorized by bolichen as a repository writer. |
A cron job dispatched to the agent pays for its whole injected context on every wake, whether or not the wake had anything to do. ASBX measured that cohort at 36 percent of all sessions, 93 percent of their cost spent re-loading static context rather than reasoning, and 43 percent of runs concluding there was nothing to do. Two cheaper modes already exist -- a script or command job takes no agent turn, and an agent job with minimal_context keeps the agent but drops the bulk of the context -- and neither reaches a job registered before its owner knew about them. Adds a cron-cost-optimize skill and the deterministic scanner it drives, so the audit itself costs no inference. Both reads go through the gateway, because a sandboxed process must not and cannot do either itself: - The job store is ownership-scoped by cron_list against a gateway-vouched session key. Reading crons.json directly would skip that, so a non-owner sharing one data home would see every participant's job metadata. It is sandbox-visible at all only as a carve-out for mcp_cron itself. - cron-history is masked outright, and the two platforms fail differently: on macOS the store resolves to disabled, but on Linux the mask is a bind-mounted EMPTY WRITABLE directory, so prepare() succeeds, enabled is True, and every read returns zero records. A job whose history was unreadable would be indistinguishable from one that had never fired -- fatal here, since the whole claim is that recorded runs rather than prompt wording are the evidence. So cron_list gains a json flag. Ownership comes free from the filter already running there. History is fetched from the gateway for the already-scoped ids only, over loopback with X-Internal-Secret, using the same port and credential resolvers as cron_trigger. Counts cross that boundary, not the run text they came from: a count cannot carry a credential, and folding gateway-side keeps the payload small. A read that cannot happen sets history_available false and leaves each job's history null, never an empty tally, because unknown and idle are different facts. The payload bounds itself and reports it, since the response cap is a blind tail slice that would hand a consumer unparseable JSON. Every free-text field is sanitized before truncation. json wins over verbose and over the verbose that ids forces on, because it serves a program rather than a reader. Both text shapes stay byte-identical and all 64 existing cron_list tests pass untouched. The scanner reads that document on stdin and opens nothing. Its history folding lives in cron_history_stats so the computation sits next to the data, leaving the script with only the judgement. An unreadable history is stated before any verdict, because every verdict then rests on wording alone. Recommend, never auto-apply. A script rewrite drops the carried previous result: build_cron_session_context prepends the last run's output only on the agent path, and the job object inside a cron script exposes just its id and message, so a job whose dedup relies on the agent recognising its own earlier answer silently starts repeating itself. minimal_context injects no skill at all, so a job naming one stops working. Judgement verbs match on stems and block a script rewrite outright, because that failure mode is the silent one; an over-block only costs a missed saving. Verified read-only by a test that compares the tree and every mtime across a scan, and the gateway fetch verified live against a running gateway.
31e006a to
1f8b4ec
Compare
…rodotdev#9592) A cron job dispatched to the agent pays for its whole injected context on every wake, whether or not the wake had anything to do. ASBX measured that cohort at 36 percent of all sessions, 93 percent of their cost spent re-loading static context rather than reasoning, and 43 percent of runs concluding there was nothing to do. Two cheaper modes already exist -- a script or command job takes no agent turn, and an agent job with minimal_context keeps the agent but drops the bulk of the context -- and neither reaches a job registered before its owner knew about them. Adds a cron-cost-optimize skill and the deterministic scanner it drives, so the audit itself costs no inference. Both reads go through the gateway, because a sandboxed process must not and cannot do either itself: - The job store is ownership-scoped by cron_list against a gateway-vouched session key. Reading crons.json directly would skip that, so a non-owner sharing one data home would see every participant's job metadata. It is sandbox-visible at all only as a carve-out for mcp_cron itself. - cron-history is masked outright, and the two platforms fail differently: on macOS the store resolves to disabled, but on Linux the mask is a bind-mounted EMPTY WRITABLE directory, so prepare() succeeds, enabled is True, and every read returns zero records. A job whose history was unreadable would be indistinguishable from one that had never fired -- fatal here, since the whole claim is that recorded runs rather than prompt wording are the evidence. So cron_list gains a json flag. Ownership comes free from the filter already running there. History is fetched from the gateway for the already-scoped ids only, over loopback with X-Internal-Secret, using the same port and credential resolvers as cron_trigger. Counts cross that boundary, not the run text they came from: a count cannot carry a credential, and folding gateway-side keeps the payload small. A read that cannot happen sets history_available false and leaves each job's history null, never an empty tally, because unknown and idle are different facts. The payload bounds itself and reports it, since the response cap is a blind tail slice that would hand a consumer unparseable JSON. Every free-text field is sanitized before truncation. json wins over verbose and over the verbose that ids forces on, because it serves a program rather than a reader. Both text shapes stay byte-identical and all 64 existing cron_list tests pass untouched. The scanner reads that document on stdin and opens nothing. Its history folding lives in cron_history_stats so the computation sits next to the data, leaving the script with only the judgement. An unreadable history is stated before any verdict, because every verdict then rests on wording alone. Recommend, never auto-apply. A script rewrite drops the carried previous result: build_cron_session_context prepends the last run's output only on the agent path, and the job object inside a cron script exposes just its id and message, so a job whose dedup relies on the agent recognising its own earlier answer silently starts repeating itself. minimal_context injects no skill at all, so a job naming one stops working. Judgement verbs match on stems and block a script rewrite outright, because that failure mode is the silent one; an over-block only costs a missed saving. Verified read-only by a test that compares the tree and every mtime across a scan, and the gateway fetch verified live against a running gateway.
What
Adds a
cron-cost-optimizeskill that audits already-registered cron jobs for a cheaper execution mode, and the two pieces of plumbing it needs to do that without reading anything it is not allowed to read.Three shipped surfaces, all of them permanent:
cron_listgrows ajson: truemode (_render_cron_list_jsoninmcp_cron.py,FieldSpec("json", bool)inCRON_LIST_SCHEMA). A machine-readable payload for a program rather than a reader, taking precedence over bothverboseand the verbose shapeidsimplies. The two text renderers are untouched and byte-identical; all 64 pre-existingcron_listtests pass unmodified._fetch_history_stats). Permanent tool behavior, not a scan-time detail.Why the plumbing exists at all
A sandboxed skill script can read neither thing an audit needs, and the two failure modes are different:
crons.jsonIS sandbox-visible (a carve-out formcp_cron), but reading it directly bypasses_owned_byownership scoping, which would disclose other participants' job metadata. So the scan goes through the tool, which scopes.cron-historyis insandbox._CREW_HIDDEN_LEAVES. On Linux the mask is a bind-mounted empty writable directory, so a naive reader succeeds, reportsenabled, and returns zero records for every job — indistinguishable from a job that never fired. That silent-zero is why the read has to happen gateway-side over loopback withX-Internal-Secret(resolve_serving_port+read_local_secret, the same pathcron_triggeruses) rather than in the script.What crosses that boundary is COUNTS, not run text: a count cannot carry a credential.
historyisNone— never an empty tally — for a job whose history could not be read, because a job with unknown history has not been shown to be idle and the consumer must tell those two apart. On a partial read, the jobs that were fetched keep their counts.The scanner is biased toward the safe direction
A cron rewritten to a script that turns out to need a model fails silently: the job stays green and quietly stops doing its work. Over-matching a blocker costs a missed saving; under-matching costs a broken job. So the scan refuses rather than guesses in three cases:
message_truncated), so a blocker could sit past the cut.Payload bounds
_JSON_MAX_JOBS(100) caps the job count and_JSON_BYTE_BUDGET(88,000) caps the payload by measured serialized size, assembled incrementally. The byte bound is not redundant with the count bound:json.dumpsescapes a non-ASCII character to a six-character\uXXXX, so 100 jobs with 400-character CJK prompts serialize to ~296,800 characters againstMAX_RESPONSE_LEN(100,000) — andsanitize_responsetruncates with a blind tail slice that appends its notice outside the JSON grammar, so an over-budget payload would not parse at all. At least one record is always kept so a single oversize job is reported rather than dropped, andtruncatedis true when either bound shed a job.Tested
157 tests across the new suites plus the existing
cron_listand caller-identity suites. Every correctness fix in this PR was negative-controlled — the test was confirmed to fail with the fix removed — including the byte budget, the non-English refusal, and the pluralskillsblocker.mypyclean over 1388 files;flake8,black,isortand the builtin-skill scope gate clean. The gateway history fetch was verified live against a running gateway.Docs: the
cron_listsection ofdocs/system-specs/modules/learn-cron-dashboard.mdcovers thejsonmode, its history semantics, and both bounds.