Skip to content

fix(skills): honour approval_required=false for prose-only candidates - #10054

Closed
chenmingwei23 wants to merge 1 commit into
mainfrom
fix/approval-required-prose-only-10042
Closed

chenmingwei23 wants to merge 1 commit into
mainfrom
fix/approval-required-prose-only-10042

Conversation

@chenmingwei23

@chenmingwei23 chenmingwei23 commented Sep 11, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Setting skills.approval_required = false does not stop skill-approval notifications. Every auto-skill candidate whose generation produced scripts still lands in the pending queue, even when the static validator rejected every script and the skill on disk is prose-only (has_scripts: false, scripts: []). Because skills.generate_scripts defaults to true, the toggle is inert under default config.

Why it matters

The setting's own description promises "Prose-only skills may auto-publish when this is false". The notification opt-out action ("Stop requiring skill approval...") deep-links users to this exact toggle, so users who click it keep getting the approval notifications it claims to end. The loop cannot be closed by the control offered for closing it.

What changed (motivation -> approach -> change)

The staging predicate in history_consolidation.py was self._approval_required or valid_scripts or scripts_supplied. scripts_supplied was set whenever generation returned any scripts and stayed true after the validator rejected all of them, so a candidate that ends up prose-only on disk still staged.

The predicate now gates on surviving VALIDATED scripts only: self._approval_required or valid_scripts. A candidate whose every script was validator-rejected is prose-only on disk and auto-publishes when approval_required=false. The safety invariant is unchanged: valid_scripts is non-empty exactly when a validator-passing script survives, and any such candidate always stages. The auto-publish path (create_auto_skill) takes no scripts argument, so a rejected script can reach disk on neither path. scripts_supplied became unused and is removed; comments, the validator docstring, and the owning spec (docs/system-specs/modules/memory-skills-hooks.md) move with the code.

On the auto-publish path a slug collision is handled by detection plus deferral: create_auto_skill claims its directory atomically (mkdir(exist_ok=False)) and returns None on collision, and the consolidator then routes the candidate to the pending queue via stage_skill_candidate. For a NEW candidate the pending claim skips any slug whose live directory is occupied (approval refuses to promote onto a live skill, so staging there would queue an unapprovable candidate that pending TTL pruning eventually deletes); update candidates deliberately name a live target and are exempt. Sibling slugs reserve room for their suffix (the base is truncated so <base>-N stays within the 64-char slug pattern), because an overlong pending directory is skipped by the listing and the slug-keyed approve/dismiss handlers -- an invisible, unprunable candidate; this bound applies to every collision route through stage_skill_candidate, including the pre-existing pending-vs-pending one. A distinct candidate whose slug is already live is never silently dropped (consolidation advances the message offset regardless of per-candidate outcome, so a drop is unrecoverable), and nothing is ever published live under a synthetic name unattended. The auto-publish SEL invoked event carries rejected_scripts: N, so an auto-published candidate whose generation attempted validator-rejected scripts is visible as such to operators. The help text on both settings is narrowed to match the behavior: generate_scripts and approval_required each now say skills that bundle a VALIDATED script always require approval, removing the two ambiguous clauses that read as candidate-level.

One deliberate test removal: test_script_bearing_candidate_stages_even_when_all_scripts_invalid pinned the reversed behavior (a script-SUPPLYING candidate never auto-publishes). That pin is exactly the contradiction this PR resolves; it is replaced by test_all_rejected_scripts_auto_publish_prose_only.

Tests

  • test_all_rejected_scripts_auto_publish_prose_only (new): with approval_required=False, a candidate whose every generated script is validator-rejected publishes live, prose-only, and the rejected script never reaches disk.
  • test_slug_collision_returns_none_and_leaves_existing_untouched (rewritten from test_refuses_duplicate): a colliding create returns None without touching the existing skill and without minting a sibling.
  • test_auto_publish_slug_collision_stages_for_review (new): on the auto-publish path a distinct candidate whose slug is already live is staged for review under a promotable sibling slug (not dropped, not published under a synthetic name), and approve_pending_skill then promotes it successfully.
  • test_max_length_slug_collision_stages_visible_sibling (new): a 64-character slug colliding with a live skill stages a truncated sibling that fits the slug pattern, appears in the pending listing, and promotes successfully.
  • test_script_bearing_candidate_always_stages (existing, still green): a candidate with a surviving valid script stages even with approval_required=False.
  • test_dangerous_script_dropped_but_skill_staged (existing, still green): with approval_required at its default (true), a candidate with rejected scripts still stages via the flag.
  • Full test/test_history.py + test/test_skills.py (486 tests) plus composition-contract, script-validator and pending-queue suites pass locally; isort, flake8, mypy, black, subprocess-encoding and comment-history gates green.

Manual verification

N/A - unit coverage sufficient: the change is a predicate plus a collision branch on a code path fully exercised by the consolidator and loader tests above, with staging, auto-publish and collision outcomes all asserted.

Related Issues

Closes #10042

Pattern harvest

Rule candidate: review-prompt
Pattern: a guard keyed on "input was supplied" instead of "validated input survived" makes an opt-out setting inert whenever any input arrives; gate on the validated survivor set, not the raw supply signal.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@chenmingwei23
chenmingwei23 requested review from a team and CrysisDeu September 11, 2026 02:58
@chenmingwei23

Copy link
Copy Markdown
Contributor Author

Intent: Make skills.approval_required=false actually auto-publish prose-only auto-skill candidates, including candidates whose every generated script was rejected by the static validator, while any candidate with a surviving validated script still stages.
Not a goal: Notification-body rewording or Settings-page annotation (issue options 2/3); any change to the validator itself or to what scripts reach disk.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 11, 2026
@github-actions

github-actions Bot commented Sep 11, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — 🟡 CONCERNS

Design-level review of 25f371cf05e7840d62dc00d23acef3039e1ae969 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

Reversing a deliberate security pin — "attempted-script candidates never auto-publish" — trades a fail-toward-review default for fail-toward-publish; a human should own that.

Watch

  • The removed test (test_script_bearing_candidate_stages_even_when_all_scripts_invalid, tagged "GPT MEDIUM") was a review-response decision, not a gap, and the PR openly reverses it. The reversal is defensible — an adversary bypasses the old control by simply emitting no scripts, and the hard gate (validated scripts always stage, rejected scripts never reach disk) is untouched — but it removes the one pre-publication escalation for the anomalous class: a candidate whose generation attempted rm -rf / now goes live with its correlated prose unreviewed, the only trace being SEL metadata (rejected_scripts: N) that an operator who opted out of approvals will likely never read. Consequence: a dangerous-intent generation's procedure text can steer future agent behavior with zero human eyes on it.
    Clears when: a maintainer explicitly affirms reversing the pin in PR review, or the auto-publish path surfaces rejected_scripts > 0 to the operator (notification/dashboard), not only SEL.

Suggestions

  • The slug-collision fallback (atomic claim, staging reroute, sibling truncation, live-occupied skip) is ~half the diff and fixes a pre-existing drop-on-collision bug; it deserved its own commit/PR so the predicate reversal — the contentious part — could be reviewed and reverted in isolation.

[DESIGN-REVIEWED] 25f371c

@github-actions

github-actions Bot commented Sep 11, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ human override accepted

Human judgment by @chenmingwei23 overrides the GPT 5.6 finding for 25f371cf05e7840d62dc00d23acef3039e1ae969; the recorded reason is authoritative for this commit.

This comment is updated in place on each push.

The model was not re-run because an authorized human decision supersedes it.

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 25f371cf05e7840d62dc00d23acef3039e1ae969: <one-sentence reason>

@github-actions

github-actions Bot commented Sep 11, 2026 •

Copy link
Copy Markdown
Contributor

Opus 4.8 Review — ✅ no blocking findings

Reviewed 25f371cf05e7840d62dc00d23acef3039e1ae969 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 25f371c

Verdict parsed from the review's SHA-scoped output markers for commit 25f371cf05e7840d62dc00d23acef3039e1ae969.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 25f371cf05e7840d62dc00d23acef3039e1ae969: <one-sentence reason>

@github-actions

github-actions Bot commented Sep 11, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — 🟡 CONCERNS

Premise-level review of 25f371cf05e7840d62dc00d23acef3039e1ae969 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

All checks complete. Composing the review.

First-Principles-Verdict: CONCERNS

Reverses a GPT-MEDIUM security pin on evidence this PR itself rewrites — the help text; issue #10042 is the only untouched support. Verify it.

Not justified as shipped

  1. Live-slug collision staging — rides along: derived (offset advances regardless of outcome, so the fix's new auto-publish traffic would turn base's staged candidates into drops), but it is a second defect fixed under a fix title.
  2. Live-occupancy skip in the pending queue — rides along: required so item 4's output is approvable (approve_pending_skill refuses live collisions, skills.py:4584 — verified).
  3. 64-char sibling-slug truncation — rides along: required so item 4's output is visible (listing skips pattern-failing dirs, skills.py:3802 — verified); also fixes the pre-existing pending-vs-pending sibling.
  4. Atomic directory claim in create_auto_skill — rides along: closes a race the fallback path would otherwise turn into an overwrite.
    8–9. rejected_scripts count and the staged/slug_collision audit event — rides along: operator-visibility compensations for the reversal (2 write sites, operator-consumed, no code consumers — counted).

What this change ships

Inventory (9 items) — 3 justified

Intent: make skills.approval_required=false actually stop approval notifications for candidates that end up prose-only on disk — a FIX.

  1. A candidate whose every generated script was validator-rejected now auto-publishes prose-only when approval is off — justified
  2. Both settings' help text narrowed from "scripts" to "a validated script" — justified
  3. The pin test forbidding exactly this auto-publish is deleted and its inverse pinned — justified
  4. Auto-publish hitting an already-live slug now stages for review instead of dropping the candidate — rides along
  5. New candidates are never queued under a slug whose live skill exists — rides along
  6. Collision sibling slugs truncate to fit the 64-char slug bound — rides along
  7. Concurrent skill creates can no longer overwrite an existing live skill — rides along
  8. The auto-publish audit event now records the validator-rejected script count — rides along
  9. A collision emits a new staged/slug_collision audit event — rides along

Watch

  • The deleted pin said a script-supplying candidate "must never auto-publish as prose-only, even with approval disabled and every script rejected (GPT MEDIUM)" — a recorded review decision. The PR's counter-evidence is the base help-text promise, which this same diff rewrites, so the repo held two contradictory pins and the PR picked a side; the linked issue is the only support outside the diff and is unreadable from this environment. Clears when: skills.approval_required=false does not stop approval notifications (generate_scripts forces staging) #10042 is confirmed to report this defect and endorse prose-only auto-publish over the GPT-MEDIUM staging decision.

[FIRST-PRINCIPLES-REVIEWED] 25f371c

@chenmingwei23
chenmingwei23 force-pushed the fix/approval-required-prose-only-10042 branch from 91a4f4f to dc6e072 Compare September 11, 2026 03:16
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 11, 2026
@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=e4634eec0079
self-added: no

  • fixed - Same-slug auto-publication drops a distinct candidate

Fixed in dc6e072. create_auto_skill now claims its directory atomically (mkdir(exist_ok=False)) and on FileExistsError allocates a unique sibling slug (-2 .. -50), mirroring stage_skill_candidate's allocation, so a distinct NEW-verdict candidate whose slug is already live publishes under the sibling name instead of being silently dropped. Pinned by test_slug_collision_allocates_unique_sibling (second distinct candidate lands as auto/duplicate-name-2, original untouched).

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=24eba5104e94
self-added: no

  • fixed - [WATCH] auto-publish of a candidate whose generation attempted a validator-rejected script loses the human checkpoint silently

Fixed in dc6e072 by the lane's own named clearing condition: the auto-publish SEL invoked event now carries rejected_scripts: N, so an auto-published candidate whose generation attempted validator-rejected scripts is visible to operators in the audit record, at parity with the staged path's scripts count. The unvalidated-prose exposure is unchanged from the staged path: prose is redacted identically on both paths, and prose-only auto-publish at approval_required=false is the setting's documented contract.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=894e6167ab45
self-added: no

  • fixed - [SUGGESTION] add rejected_scripts to the SEL invoked metadata on the auto-publish path

Implemented as suggested in dc6e072: metadata={"name": name, "rejected_scripts": rejected_scripts} on the auto_skill_create invoked event, counted per validator rejection in the extraction loop.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Sep 11, 2026
@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=e39790f6c35d
self-added: no

  • needs-a-decision - [WATCH] reversing the pin that treated a validator rejection as a candidate-level taint signal

The two candidate designs are mutually exclusive and the choice is a product ruling, not a code fact: (A) this PR's reading - the validator verdict is per-script; rejected scripts are dropped and the surviving prose-only candidate honours approval_required=false, which is the setting's documented contract ("Prose-only skills may auto-publish when this is false") and the only reading under which the #3927 notification opt-out can ever end the loop; (B) the deleted pin's reading - any rejection taints the whole candidate, so it must stage (or drop) regardless of the toggle. Under (B) with default generate_scripts=true the toggle stays inert for script-attempting candidates and the Settings description plus the opt-out deep-link stay misleading, so (B) requires the issue's options 2/3 (notification rewording / Settings annotation) as follow-up UX work.
Maintainer: please rule (A) or (B). This PR implements (A). If (B) is the ruling, the smaller honest variant is drop-and-audit for rejected-script candidates when approval_required=false, per the sibling suggestion span.

Which reading of a validator rejection is intended: per-script (rejected scripts dropped, prose-only remainder honours approval_required=false, as this PR implements), or candidate-level taint (any rejection forces staging/drop and the toggle stays inert for script-attempting candidates, requiring the notification/Settings UX follow-ups)?

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=ba8982ff2519
self-added: no

  • rebutted - [SUGGESTION] discard all-scripts-rejected candidates instead of publishing them

Discard closes the notification loop but silently destroys the learned prose: consolidation advances the message offset regardless of per-candidate outcome, so a discarded candidate is unrecoverable, and the prose itself passed redaction and is exactly what the operator opted into auto-publishing (approval_required=false is documented as "Prose-only skills may auto-publish"). The prose is not taint-flagged by the validator - the validator's subject is scripts, and its per-script findings do not evaluate prose. Discard-and-audit is the honest fallback only under the candidate-level-taint ruling; that ruling is put to the maintainer on span e39790f6c35d, so this record keeps the two designs from being mixed.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=7137a292b18c
self-added: no

  • rebutted - [NOT JUSTIFIED AS SHIPPED] the slug-collision fix rides along

The collision branch is not opportunistic: it is reachable ONLY through the auto-publish path this PR opens up (before this PR, an all-rejected-scripts candidate never reached create_auto_skill with approval off), and the GPT lane's round-1 adjudication upheld the resulting silent candidate loss as unbounded harm on exactly that new path. A fix that creates a reachable data-loss branch and ships without closing it would be incomplete, not smaller. Findings that require splitting a defect's own reachability closure into a second PR are covered by this ruling.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=1cd6fedbcda3
self-added: no

  • fixed - [NOT JUSTIFIED AS SHIPPED] the audit field rides along and "parity with the staged path" is inexact

The wording critique holds: the staged event records scripts=len(valid_scripts), never a rejected count, so "parity" was inexact. Fixed in the PR body (dc6e072): the field is now described as what it is - the Design lane's round-1 requested clearing condition ("the auto-publish SEL event carries the rejected-script count"), making an auto-published candidate whose generation attempted rejected scripts visible to operators. It is one metadata key on an event this diff already touches, added to answer a same-PR review finding, which is the narrowest way to answer it.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=3de721520fde
self-added: no

  • needs-a-decision - [WATCH] whether the candidate-level rule or the setting description was the error

Same maintainer ruling as design span e39790f6c35d, recorded once there to keep one question from fragmenting into two threads: this PR implements the per-script reading (the setting description is correct, the pin was the error); the alternative candidate-level reading keeps the pin and requires the issue's options 2/3 UX follow-ups. The PR body's "safety invariant is unchanged" claims exactly the script-on-disk invariant (no validator-passing script auto-publishes), which is true on both readings.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=48231c4467c6
self-added: no

  • rebutted - [WATCH] live-path sibling allocation lacks the pending queue's human backstop

The comparison baseline is not "sibling mint vs human review" - approval_required=false is precisely the operator declining that backstop for prose-only skills. The baseline is what the pre-PR code did on the same inputs: return None and silently lose the distinct candidate (upheld round-1 as unrecoverable, since the message offset advances regardless). A same-skill re-detection that slips BOTH the lexical dedupe and the LLM judge now mints a visible duplicate (SEL invoked event, dashboard skill list, bounded by run_skill_lifecycle's max_auto_skills cap) instead of silently discarding a distinct skill; a visible bounded duplicate is strictly recoverable, silent loss is not. The staging twin makes the same tradeoff with the same upstream dedupe.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 11, 2026
@chenmingwei23
chenmingwei23 force-pushed the fix/approval-required-prose-only-10042 branch from dc6e072 to 3e87ff4 Compare September 11, 2026 04:36
@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Sep 11, 2026
@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=3ec110428e82
self-added: yes

  • fixed - "dropped and never reaches disk" contradicts approval-time validation

Fixed in 25f371c as prescribed: the validator docstring now scopes the drop to consolidation-time generation and states the approval-time behavior separately (a rejected script stays in the pending candidate on disk, untouched). Branch also rebased onto current main, which carries the main-side fix for the backend shard-4 fence-parity red that was holding CI.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 12, 2026
@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=7eac2a9d5b05
self-added: no

  • fixed - [WATCH] explicit affirmation of reversing the pin

Affirmed in PR review, per this lane's first clearing condition: the maintainer-operator (chenmingwei23) delegated the ruling and the per-script reading stands (recorded on spans e39790f6c35d, c97582e3ac38, ce2153c94851). The lane's own analysis this round agrees the reversal is defensible (an adversary bypasses the old control by emitting no scripts). The alternative clearing condition (notification/dashboard surfacing of rejected_scripts > 0) is product surface beyond this fix; the SEL field carries the signal today.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=7718fdc75f25
self-added: yes

  • rebutted - [NOT JUSTIFIED] live-slug collision staging rides along

The lane's own derivation states the coupling: the fix's new auto-publish traffic would turn base's staged candidates into drops. A change that creates the traffic ships the closure for it; the operator accepted the riders (span d582ad3e6522, restated for this head).

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=ac4768f80319
self-added: yes

  • rebutted - [NOT JUSTIFIED] live-occupancy skip rides along

As this lane itself verified, the skip is required so the fallback's output is approvable (approve_pending_skill refuses live collisions); it was prescribed by blocking round 4 of this PR. Covered by the operator's acceptance of the riders.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=c119ed63b18d
self-added: yes

  • rebutted - [NOT JUSTIFIED] sibling truncation rides along

As this lane itself verified, truncation is required so the fallback's output is visible to the listing; it was prescribed by blocking round 5 and the operator explicitly ruled fix-in-PR for it at the escalation. Covered by the acceptance of the riders.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=59083af88b59
self-added: yes

  • rebutted - [NOT JUSTIFIED] atomic claim + audit events ride along

The atomic claim closes the race the fallback would otherwise turn into an overwrite (this lane's own verification); the rejected_scripts count and the staged/slug_collision audit metadata are the operator-visibility compensations the Design lane requested in round 1. All covered by the operator's acceptance of the riders (span d582ad3e6522).

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=588520ebb14e
self-added: no

Confirmed: issue #10042 reports exactly this defect and its option 1 prescribes this reversal; the maintainer-operator ratified it by delegation (recorded on spans e39790f6c35d, e0a7f180a16a). On the circularity concern - the help-text promise predates this diff (the diff only NARROWS its second clause), so the shipped contract this PR restores is base's own, not one this PR invents.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=157368205335
self-added: no

  • rebutted - [SUGGESTION] collision fallback deserved its own commit/PR

The fallback cannot precede or follow this PR independently: it is reachable only through the auto-publish traffic this PR's predicate creates, and shipping the predicate without it re-opens the round-1 adjudication-upheld silent drop. CI enforces a single commit on this repo, so in-PR separation is not available; the separation that IS available - the intent comment, the round table, and per-mechanism dispositions - already lets a reviewer evaluate or revert the predicate reversal in isolation. The operator accepted the riders on span d582ad3e6522.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

/ai-review override gpt 25f371c: Exhaustion false-success is a pre-existing main-owned defect - the return-name-without-persisting branch is main's untouched stage_skill_candidate code (fires on main after 49 pending siblings); this PR only adds a second route to it. Tracked as issue #10252 (deferred-finding, owner, due date); the fix there returns None on exhaustion so callers take their rejection branch.

@github-actions

Copy link
Copy Markdown
Contributor

Human judgment recorded

@chenmingwei23 marked the gpt AI finding as false positive, not applicable, or explicitly accepted for 25f371cf05e7840d62dc00d23acef3039e1ae969.

Exhaustion false-success is a pre-existing main-owned defect - the return-name-without-persisting branch is main's untouched stage_skill_candidate code (fires on main after 49 pending siblings); this PR only adds a second route to it. Tracked as issue #10252 (deferred-finding, owner, due date); the fix there returns None on exhaustion so callers take their rejection branch.

This decision applies only to this commit. A new push requires a new judgment.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 12, 2026
@chenmingwei23

Copy link
Copy Markdown
Contributor Author

Paused: a competing fix landed on main

Commit ed1a6ed ("fix(skills): respect disabled approval setting (#7617)") merged to main while this PR was converging. It removes the same scripts_supplied disjunct this PR removes - the staging predicate on main is now the same self._approval_required or valid_scripts - but it rules the all-rejected-scripts case the opposite way: with approval disabled, such a candidate is REJECTED outright (SEL all_scripts_rejected), not auto-published as prose. The merge conflict on this branch is that semantic fork, not a textual accident.

Consequences for this PR:

Held for a maintainer/operator ruling: rework this PR to carry only the surviving fixes on top of main's semantics, or close it as superseded and move the surviving fixes to the #10252 track.

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maintainer review (agent-assisted), pinned to 25f371c. Requesting changes. #7617 is now on main and makes the owner's selected policy explicit: with approval disabled, a candidate whose supplied scripts all fail validation is rejected, not promoted as prose. Genuine prose-only candidates already auto-publish. This PR changes the same predicate/spec/test in the opposite direction, deleting the rejection pin and publishing an all-rejected-script candidate. I independently inspected that predicate change; this is not an equivalent fix or a routine UX choice.

Rebase on current main and retain its rejection rule/test. The atomic create and collision-name repairs are potentially useful independent changes: separate them from the policy reversal and review their tests. The collision-to-pending fallback also needs a clear contract for users who disabled approval. No local runtime suites run; the partial review of collision riders is not approval of them. Current-head green CI does not authorize overriding the landed policy.

@iamwhatever

Copy link
Copy Markdown
Contributor

Not approving for now: the branch has merge conflicts with main; please rebase and we'll take another look.

@buluoray

Copy link
Copy Markdown
Contributor

Not approving. This reverses a security decision that already shipped, and I do not think the conflict is mechanical.

The reversal. origin/main carries a reject branch from ed1a6ed75 (#7617, "fix(skills): respect disabled approval setting"):

  • if scripts_supplied and not valid_scripts and not self._approval_required: refuses the candidate outright, with outcome="rejected", reason="all_scripts_rejected";
  • the following elif self._approval_required or valid_scripts: even carries a comment stating why: "An all-invalid candidate with approval disabled is consumed by the reject branch above, so a bare scripts_supplied never decides this branch."

This PR deletes that branch, removes scripts_supplied, and reduces the gate to self._approval_required or valid_scripts. Its diff baseline is ... or valid_scripts or scripts_supplied, which is the pre-#7617 code, so the branch was cut before that fix and never rebased, and the body never mentions #7617. That is also why the PR is dirty: it is a semantic reversal of the same lines rather than a textual clash, so resolving the conflict forces a choice between two designs rather than a merge.

What the reversal permits. A candidate supplies a companion script, the static validator flags it as dangerous and drops it (this PR own new test uses a recursive-delete shell payload as its example), the candidate is now considered prose-only, and with approval_required=false it auto-publishes live under auto/<slug> with no human approval. The validator only ever inspects the Python scripts entries, never procedure_md, so the dangerous procedure is typically still described in the prose that was never scanned, and an agent can load and follow it later. #7617 refused exactly this class; this change publishes it.

The guard for it was deleted too. test_script_bearing_candidate_stages_even_when_all_scripts_invalid pinned that a script-supplying, all-rejected candidate must not auto-publish. It is removed and replaced with test_all_rejected_scripts_auto_publish_prose_only, which asserts the opposite. After the PR there is a positive test that valid-script candidates still stage, but no negative test standing between the higher-risk all-rejected subset and auto-publish, so the approval-skip has lost its falsification.

Two smaller notes in the same area:

  • "prose-only" is a misnomer for the predicate: frontmatter, an allowed-tools grant, shell fences and references to sibling script files are all outside what the validator reads. Largely pre-existing for genuinely prose-only candidates, but this change extends the auto-publish surface to the ex-script subset, where intent was higher.
  • Less fail-closed on malformed input than main: in the script loop a non-dict entry hits continue without incrementing rejected_scripts, so a candidate supplying one non-dict script entry yields no valid scripts, a zero rejected count, an auto-publish, and a SEL record that does not even show scripts were attempted. The scripts_supplied branch on main caught that.

Worth keeping either way: the slug-collision hardening in skills.py (atomic mkdir(exist_ok=False) claim, _live_occupied skip, 64-char sibling truncation) is solid and well-tested, and is orthogonal to the contested gate.

Suggested path: rebase onto post-#7617 main and decide the design explicitly rather than resolving the conflict by hand. If auto-publishing an ex-script candidate is genuinely wanted, that deserves its own change that says it is relaxing #7617 and keeps a negative test for the case it still refuses. Splitting the slug-collision work out would let that half land immediately.

One thing I did not chase: whether a frontmatter block embedded in procedure_md (an allowed-tools grant, say) can become the effective frontmatter of an auto-published skill. If it can, the misnomer above stops being advisory.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

Recommending closure as superseded.

PR #7617 merged 2026-09-12 and landed the opposite policy for the same case: with approval_required=false, an auto-skill candidate whose supplied scripts ALL fail validation is rejected outright (SEL audit all_scripts_rejected), not published as prose. It closed the same issue this PR targets, #10042, which is now CLOSED (2026-09-15).

This PR reverses that landed decision on the same predicate/spec/test: it deletes the rejection pin and auto-publishes an all-rejected-script candidate as prose-only. That is a contested reversal of a shipped, accepted policy, not a fix awaiting a rebase, so I am not resolving the conflict -- a mergeable branch here would only make a contested change look ready.

Recommendation: close #10054 as superseded by #7617. If the auto-publish-prose direction is still wanted, it belongs in a fresh proposal making that case against current main, not on this branch.

Note for salvage: the skills.py atomic-create / collision-name handling here is independent of the policy question and could be carried separately on top of main's rejection rule. bolichen97 flagged it as potentially useful and also noted its user-facing contract would need its own review.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

Superseding and closing: this PR's premise was reversed by a merged PR, so it is no longer a fix awaiting a rebase -- it is a proposal to reverse a policy that already shipped.

Same code path (HistoryConsolidator staging of an auto-skill candidate whose every generated script fails validation, under approval_required=false), opposite outcome:

That policy is on current main: src/kiro_crew/history_consolidation.py:2159 (reason: all_scripts_rejected) and test/test_history.py:5142. Issue #10042 -- the problem this PR addressed -- is CLOSED/COMPLETED.

Auto-publish vs reject-outright for the identical condition is a contested reversal of a shipped decision, not a mergeable fix. Reopening it would need an RFC to reverse #7617, not a rebase. Closing.

@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Sep 17, 2026
@bolichen97
bolichen97 deleted the fix/approval-required-prose-only-10042 branch September 20, 2026 08:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge conflict Branch has merge conflicts with its base — author must resolve before merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

skills.approval_required=false does not stop approval notifications (generate_scripts forces staging)

4 participants