Repository navigation
fix(skills): honour approval_required=false for prose-only candidates - #10054
chenmingwei23 wants to merge 1 commit into
Conversation
|
Intent: Make |
Design Review (Fable 5) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS Reversing a deliberate security pin — "attempted-script candidates never auto-publish" — trades a fail-toward-review default for fail-toward-publish; a human should own that. Watch
Suggestions
[DESIGN-REVIEWED] 25f371c |
GPT 5.6 Review — ✅ human override acceptedHuman judgment by @chenmingwei23 overrides the GPT 5.6 finding for This comment is updated in place on each push. The model was not re-run because an authorized human decision supersedes it. False positive or not applicable? A repository writer can comment: |
Opus 4.8 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
First Principles Review (Fable 5) — 🟡 CONCERNSPremise-level review of All checks complete. Composing the review. First-Principles-Verdict: CONCERNS Reverses a GPT-MEDIUM security pin on evidence this PR itself rewrites — the help text; issue #10042 is the only untouched support. Verify it. Not justified as shipped
What this change shipsInventory (9 items) — 3 justifiedIntent: make
Watch
[FIRST-PRINCIPLES-REVIEWED] 25f371c |
91a4f4f to
dc6e072
Compare
|
span=e4634eec0079
|
|
span=24eba5104e94
|
|
span=894e6167ab45
|
|
span=e39790f6c35d
Which reading of a validator rejection is intended: per-script (rejected scripts dropped, prose-only remainder honours approval_required=false, as this PR implements), or candidate-level taint (any rejection forces staging/drop and the toggle stays inert for script-attempting candidates, requiring the notification/Settings UX follow-ups)? |
|
span=ba8982ff2519
|
|
span=7137a292b18c
|
|
span=1cd6fedbcda3
|
|
span=3de721520fde
|
|
span=48231c4467c6
|
dc6e072 to
3e87ff4
Compare
|
span=3ec110428e82
|
|
span=7eac2a9d5b05
|
|
span=7718fdc75f25
|
|
span=ac4768f80319
|
|
span=c119ed63b18d
|
|
span=59083af88b59
|
|
span=588520ebb14e
|
|
span=157368205335
|
|
/ai-review override gpt 25f371c: Exhaustion false-success is a pre-existing main-owned defect - the return-name-without-persisting branch is main's untouched stage_skill_candidate code (fires on main after 49 pending siblings); this PR only adds a second route to it. Tracked as issue #10252 (deferred-finding, owner, due date); the fix there returns None on exhaustion so callers take their rejection branch. |
Human judgment recorded@chenmingwei23 marked the gpt AI finding as false positive, not applicable, or explicitly accepted for
This decision applies only to this commit. A new push requires a new judgment. |
Paused: a competing fix landed on mainCommit ed1a6ed ("fix(skills): respect disabled approval setting (#7617)") merged to main while this PR was converging. It removes the same Consequences for this PR:
Held for a maintainer/operator ruling: rework this PR to carry only the surviving fixes on top of main's semantics, or close it as superseded and move the surviving fixes to the #10252 track. |
bolichen97
left a comment
There was a problem hiding this comment.
Maintainer review (agent-assisted), pinned to 25f371c. Requesting changes. #7617 is now on main and makes the owner's selected policy explicit: with approval disabled, a candidate whose supplied scripts all fail validation is rejected, not promoted as prose. Genuine prose-only candidates already auto-publish. This PR changes the same predicate/spec/test in the opposite direction, deleting the rejection pin and publishing an all-rejected-script candidate. I independently inspected that predicate change; this is not an equivalent fix or a routine UX choice.
Rebase on current main and retain its rejection rule/test. The atomic create and collision-name repairs are potentially useful independent changes: separate them from the policy reversal and review their tests. The collision-to-pending fallback also needs a clear contract for users who disabled approval. No local runtime suites run; the partial review of collision riders is not approval of them. Current-head green CI does not authorize overriding the landed policy.
|
Not approving for now: the branch has merge conflicts with main; please rebase and we'll take another look. |
|
Not approving. This reverses a security decision that already shipped, and I do not think the conflict is mechanical. The reversal.
This PR deletes that branch, removes What the reversal permits. A candidate supplies a companion script, the static validator flags it as dangerous and drops it (this PR own new test uses a recursive-delete shell payload as its example), the candidate is now considered prose-only, and with The guard for it was deleted too. Two smaller notes in the same area:
Worth keeping either way: the slug-collision hardening in Suggested path: rebase onto post-#7617 main and decide the design explicitly rather than resolving the conflict by hand. If auto-publishing an ex-script candidate is genuinely wanted, that deserves its own change that says it is relaxing #7617 and keeps a negative test for the case it still refuses. Splitting the slug-collision work out would let that half land immediately. One thing I did not chase: whether a frontmatter block embedded in |
|
Recommending closure as superseded. PR #7617 merged 2026-09-12 and landed the opposite policy for the same case: with This PR reverses that landed decision on the same predicate/spec/test: it deletes the rejection pin and auto-publishes an all-rejected-script candidate as prose-only. That is a contested reversal of a shipped, accepted policy, not a fix awaiting a rebase, so I am not resolving the conflict -- a mergeable branch here would only make a contested change look ready. Recommendation: close #10054 as superseded by #7617. If the auto-publish-prose direction is still wanted, it belongs in a fresh proposal making that case against current main, not on this branch. Note for salvage: the |
|
Superseding and closing: this PR's premise was reversed by a merged PR, so it is no longer a fix awaiting a rebase -- it is a proposal to reverse a policy that already shipped. Same code path (
That policy is on current main: Auto-publish vs reject-outright for the identical condition is a contested reversal of a shipped decision, not a mergeable fix. Reopening it would need an RFC to reverse #7617, not a rebase. Closing. |
Problem / Motivation
Setting
skills.approval_required = falsedoes not stop skill-approval notifications. Every auto-skill candidate whose generation produced scripts still lands in the pending queue, even when the static validator rejected every script and the skill on disk is prose-only (has_scripts: false, scripts: []). Becauseskills.generate_scriptsdefaults to true, the toggle is inert under default config.Why it matters
The setting's own description promises "Prose-only skills may auto-publish when this is false". The notification opt-out action ("Stop requiring skill approval...") deep-links users to this exact toggle, so users who click it keep getting the approval notifications it claims to end. The loop cannot be closed by the control offered for closing it.
What changed (motivation -> approach -> change)
The staging predicate in
history_consolidation.pywasself._approval_required or valid_scripts or scripts_supplied.scripts_suppliedwas set whenever generation returned any scripts and stayed true after the validator rejected all of them, so a candidate that ends up prose-only on disk still staged.The predicate now gates on surviving VALIDATED scripts only:
self._approval_required or valid_scripts. A candidate whose every script was validator-rejected is prose-only on disk and auto-publishes whenapproval_required=false. The safety invariant is unchanged:valid_scriptsis non-empty exactly when a validator-passing script survives, and any such candidate always stages. The auto-publish path (create_auto_skill) takes no scripts argument, so a rejected script can reach disk on neither path.scripts_suppliedbecame unused and is removed; comments, the validator docstring, and the owning spec (docs/system-specs/modules/memory-skills-hooks.md) move with the code.On the auto-publish path a slug collision is handled by detection plus deferral:
create_auto_skillclaims its directory atomically (mkdir(exist_ok=False)) and returns None on collision, and the consolidator then routes the candidate to the pending queue viastage_skill_candidate. For a NEW candidate the pending claim skips any slug whose live directory is occupied (approval refuses to promote onto a live skill, so staging there would queue an unapprovable candidate that pending TTL pruning eventually deletes); update candidates deliberately name a live target and are exempt. Sibling slugs reserve room for their suffix (the base is truncated so<base>-Nstays within the 64-char slug pattern), because an overlong pending directory is skipped by the listing and the slug-keyed approve/dismiss handlers -- an invisible, unprunable candidate; this bound applies to every collision route throughstage_skill_candidate, including the pre-existing pending-vs-pending one. A distinct candidate whose slug is already live is never silently dropped (consolidation advances the message offset regardless of per-candidate outcome, so a drop is unrecoverable), and nothing is ever published live under a synthetic name unattended. The auto-publish SELinvokedevent carriesrejected_scripts: N, so an auto-published candidate whose generation attempted validator-rejected scripts is visible as such to operators. The help text on both settings is narrowed to match the behavior:generate_scriptsandapproval_requiredeach now say skills that bundle a VALIDATED script always require approval, removing the two ambiguous clauses that read as candidate-level.One deliberate test removal:
test_script_bearing_candidate_stages_even_when_all_scripts_invalidpinned the reversed behavior (a script-SUPPLYING candidate never auto-publishes). That pin is exactly the contradiction this PR resolves; it is replaced bytest_all_rejected_scripts_auto_publish_prose_only.Tests
test_all_rejected_scripts_auto_publish_prose_only(new): withapproval_required=False, a candidate whose every generated script is validator-rejected publishes live, prose-only, and the rejected script never reaches disk.test_slug_collision_returns_none_and_leaves_existing_untouched(rewritten fromtest_refuses_duplicate): a colliding create returns None without touching the existing skill and without minting a sibling.test_auto_publish_slug_collision_stages_for_review(new): on the auto-publish path a distinct candidate whose slug is already live is staged for review under a promotable sibling slug (not dropped, not published under a synthetic name), andapprove_pending_skillthen promotes it successfully.test_max_length_slug_collision_stages_visible_sibling(new): a 64-character slug colliding with a live skill stages a truncated sibling that fits the slug pattern, appears in the pending listing, and promotes successfully.test_script_bearing_candidate_always_stages(existing, still green): a candidate with a surviving valid script stages even withapproval_required=False.test_dangerous_script_dropped_but_skill_staged(existing, still green): withapproval_requiredat its default (true), a candidate with rejected scripts still stages via the flag.test/test_history.py+test/test_skills.py(486 tests) plus composition-contract, script-validator and pending-queue suites pass locally; isort, flake8, mypy, black, subprocess-encoding and comment-history gates green.Manual verification
N/A - unit coverage sufficient: the change is a predicate plus a collision branch on a code path fully exercised by the consolidator and loader tests above, with staging, auto-publish and collision outcomes all asserted.
Related Issues
Closes #10042
Pattern harvest
Rule candidate: review-prompt
Pattern: a guard keyed on "input was supplied" instead of "validated input survived" makes an opt-out setting inert whenever any input arrives; gate on the validated survivor set, not the raw supply signal.
Checklist
feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)