Repository navigation
fix: name the rule that refused an environment credential command - #34992
Conversation
Sensitive Change ReviewSensitive files changed:
|
0dfab42 to
583eefa
Compare
|
Rebased: main moved in docs/system-specs/modules/security.md, test/test_security_gate_liveness.py since this head went green (fd016ef -> 7db81f6); CI re-running. The rebase clears a merge conflict in both files: the security spec keeps the sentences main added to the same bullet, and the package line budget is re-measured on the new base. The earlier run on 0dfab42 did not finish its review and test lanes because GitHub API rate limits cut it short, so its reds say nothing about the code. [operator: bolichen97] |
UX Review (Opus 5.5) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks. The change reuses the existing What this change shipsInventory (5 items) — 5 justifiedIntent: let a user whose shell command was refused by the always-on environment-credential check see which rule refused it, so they can check or report it. FIX.
[FIRST-PRINCIPLES-REVIEWED] 6e5d3f9 Description read[DESCRIPTION-READ] 2d36df44c97cce7ac83aaacc734e011c7c50324a59337dafb26137715e595cb9 That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: the refusal reuses the existing Notes
[DESIGN-REVIEWED] 6e5d3f9 Description read[DESCRIPTION-READ] be19d13bce3aefcc53967be12bd3af51d2680d821033ad5a4a37aafe481414f8 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Security Scope Review -- ✅ PASSWhich legitimate operations does Verdict basis: adjudicated, zero confirmed regressions. Security scope review -- adjudicated candidates
No candidate flipped from allowed to refused. Scope-Verdict: PASS Reason: This change only adds a diagnostic line. TIGHTENING: env-credential tier diagnostic (sensitive-bash, ADJUDICATED: 14 candidates, 0 confirmed newly-refused PLATFORM COVERAGE: posix and windows. The diff adds no assumptions about paths, separators, interpreters or executable names; the regexes are byte-identical to the base. Posix rows probe the non-secret neighbours of each rule ( [SCOPE-REVIEWED] 071a698 |
Code change approver recommendation: ApproveWhy: the decider estimates a 7% chance that this change introduces a defect, within the 12% limit for approving on its own, and every check passes. Adds a An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit |
583eefa to
5f6fb4e
Compare
|
Rebased: main moved in docs/system-specs/modules/security.md, test/test_security_gate_liveness.py since this head went green (7db81f6 -> 2df02ab); CI re-running. The rebase clears a merge conflict in the security package's line budget: main raised it three times since the last base, and this change's three lines sit on top of main's figure, re-measured on the new base. The security spec merged without a conflict and keeps this change's wording. [operator: bolichen97] |
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Reviewed on head 68b4d4d. The env-credential tier now appends one refusal diagnostic line that names the rule that fired. The first line of the refusal stays byte-identical. We checked that the four patterns are unchanged and keep their order after the list-to-dict change, that every new rule id passes the diagnostic id filter, and that every caller of is_sensitive_bash_command handles the two-line reason. We also checked that the debug view now files these rows as denied-rule refusals and that the line budget rise matches the measured +3 lines. Reverting the fix reds 7 tests. The touched test files pass locally (751 passed).
Security: this touches the denied-command tier, but only the refusal text changes. Every command that main refuses is still refused, and nothing new is refused or allowed. The added line carries offsets and character counts, never the matched bytes.
Done well: the parametrized test pins one row per rule, including both over-blocks from #34526. A later narrowing therefore has to change a test row on purpose.
Optional:
- Also assert the
span=field intest_an_env_credential_refusal_names_which_rule_fired. Droppingmatch.span()keeps every test green today. - The Sensitive Change Review check still needs the filled reasoning template for head 68b4d4d.
- One line in the body could say that the spec also stops claiming the IMDS refusal carries a diagnostic, because it never did.
68b4d4d to
57944a3
Compare
|
Rebased: main moved in docs/system-specs/modules/security.md, test/test_security_gate_liveness.py since this head went green (029b6db -> a02e90c); CI re-running. The rebase clears the merge conflict in the security package's line budget: main raised it twice, to 35,804 and then 35,824, and this PR's three lines put it at 35,827, the package's re-measured size. The security spec merged cleanly. The change itself is unchanged. [operator: bolichen97] |
|
The review found no blocking issue on this head. It cannot be approved yet because PR Readiness is red on 57944a3. What the review checked on this head:
Done well: the PR reuses the payload-free diagnostic format other refusals already use, so no second format appears. The two #34526 over-block rows are pinned on purpose and say so in the test. Why it waits: the AI review lanes, Security Scope and Sensitive Change Review failed on this head because of a GitHub API rate limit, not because of the code. The lane verdicts on record are for the previous head, which carries the same code. Those lanes need a rerun, Sensitive Change Review needs its reasoning or an approval on this head, and then readiness needs a new run. A maintainer will take it from there. Optional:
|
|
Rebased: main moved in docs/system-specs/modules/security.md, test/test_security_bash_gate.py, test/test_security_gate_liveness.py since this head went green (a02e90c -> 5672158); CI re-running. The rebase clears the merge conflict in the security package's line budget. Main now takes each change's raise as its own row under [operator: bolichen97] |
57944a3 to
8c06190
Compare
|
[operator: bolichen97] Answers to the three optional notes in the review of 57944a3, one per note:
|
|
Rebased: main moved in docs/system-specs/modules/security.md since this head went green (5672158 -> 256d5af); CI re-running. The rebase onto current main applied cleanly and the change itself is unchanged. Main also changed other files of the security package in that range; none of them is on the refusal path this PR changes, and the package's line budget still holds with this PR's +3 row. On the previous head every check was green except Sensitive Change Review, which needs a sensitive-change review recorded for the new head, and PR Readiness, which folds it in, so no code fix was needed. The targeted gates, the proof and the determinism runs were repeated on the rebased tree and pass; the description's proof and real-machine lines now name the new head. [operator: bolichen97] |
8c06190 to
071a698
Compare
|
[operator: bolichen97] Backend Tests (3.12, 1), Backend Tests (Windows) (4) and Coverage Gate are red on this head for the same reason they are red on main at this PR's base (256d5af): |
The always-on environment credential check answered with one fixed sentence whichever of its rules matched, so a user who thought a refusal was wrong could not tell which rule had fired. The refusal keeps that sentence and adds a diagnostic line naming the rule and the matched span, never the matched text. A security event that stores the refusal text, as a background turn's does, carries the rule as well; the dashboard and CLI chats keep their fixed refusal code. What the check refuses is unchanged. Part of #34526
|
Rebased: main moved in test/conftest.py, test/conftest_resets.py and files that share a directory with this PR's (docs/system-specs/modules/ among them) since this head went green (256d5af -> 3688176); CI re-running. The rebase picks up #39479, which fixed [operator: bolichen97] |
071a698 to
6e5d3f9
Compare
Sensitive change review
|
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Reviewed head 6e5d3f9. The environment-credential check refuses exactly the same commands as before, and the first line of its refusal is unchanged. The refusal now adds the existing diagnostic line naming the rule that fired, with offsets and a character-class count, never the matched text. Every reader of this refusal text already handles the extra line. The related security, cron and offload test files pass on this head, and the new test fails on the base for every rule row. Nothing blocks.
Problem / Motivation
Goal: A refusal from the always-on environment credential check names the rule that refused the command, in the refusal message and in every security event that stores the refusal text.
When this check refuses a shell command, every one of its rules answers with the same sentence: "Blocked: command reads AWS credentials from environment variables". A user who believes a refusal is wrong cannot tell which rule matched, so they cannot check it against their command or report it precisely (#34526).
Why it matters
The check cannot be switched off, so its refusal text is all a user gets. A refusal that names no rule looks the same whether it is right or wrong, and the usual reaction is to re-spell the command until it passes.
Not a goal
hook_denycode the dashboard and CLI chats record for a refusal: that audit row deliberately restates no gate reason.What changed (motivation → approach → change)
A refusal from this check says which rule refused the command, so a user who thinks it is wrong can check that rule against the command and report it by name. The first line of the refusal stays exactly as it is. Under it, the refusal adds the
Refusal diagnostic:line the size-ceiling refusal already carries: the rule id (the catalog id when the rule has one), the matched span as offsets, and a character-class count, never the matched text. A security event that stores the refusal text, as a background turn's does (a cron run, a workflow agent), names the rule as well, and the debug view files those rows as denied-rule refusals instead of unclassified. The security spec says which refusals of this gate carry that line, and a row intest/security_line_budget/raises the line budget of thesrc/kiro_crew/securitypackage by three, this change's own growth.Backwards compatibility
Compatible: the first line of the refusal is unchanged, and the added line has the format the debug view already reads.
Removes nothing: the check refuses exactly the commands it refused before.
Tests
-n0, 10 runs at-n 4 --dist loadgroupbesidetest/test_security_denied_commands.pyandtest/test_denied_commands_security.py, and one shuffled order (seed 2264350906).Proof: PROVEN test/test_security_bash_gate.py::TestARefusalNamesItsRuleAndSpan::test_an_env_credential_refusal_names_which_rule_fired[env | grep AWS_SECRET-credential-exfil-env-grep-aws], test/test_security_bash_gate.py::TestARefusalNamesItsRuleAndSpan::test_an_env_credential_refusal_names_which_rule_fired[printenv AWS_SECRET_ACCESS_KEY-credential-exfil-printenv-aws], test/test_security_bash_gate.py::TestARefusalNamesItsRuleAndSpan::test_an_env_credential_refusal_names_which_rule_fired[declare -p AWS_SECRET_ACCESS_KEY-keystone-env-cred-declare], test/test_security_bash_gate.py::TestARefusalNamesItsRuleAndSpan::test_an_env_credential_refusal_names_which_rule_fired[echo $AWS_SECRET_ACCESS_KEY-keystone-env-cred-echo], test/test_security_bash_gate.py::TestARefusalNamesItsRuleAndSpan::test_an_env_credential_refusal_names_which_rule_fired[awk 'BEGIN{print ENVIRON["AWS_SECRET_ACCESS_KEY"]}'-keystone-env-cred-awk-environ] (head ef348876ce0d)
Manual verification
On Linux (Python 3.12), through the real shell gate (
is_sensitive_bash_command), with an isolatedKIROCREW_HOME:printenv,declare -pandechoofAWS_SECRET_ACCESS_KEYnamecredential-exfil-printenv-aws,keystone-env-cred-declareandkeystone-env-cred-echothe same way, and no diagnostic carries the matched text, only its offsets and a character-class count. The two commands from #34526 are still refused, bykeystone-env-cred-interpreterandcredential-exfil-env-awk-sed-aws.ls -laandgit statusare still allowed.Real-machine verification
Machine: EC2 m7i.2xlarge, Ubuntu 24.04 x86_64, 8 vCPU / 32 GiB, Python 3.12.3, no inbound rules
Before (main c0d672b):
prove.py, which puts main's code back under the five rule-naming tests listed under Tests ->PROVEN: an assertion failed with the bug reintroduced; on main the check returns only the first line, so no refusal names its ruleAfter (tree 245987d506c7, this patch on main c0d672b; head 6e5d3f9 is the same patch on 3688176): the same five tests pass, and
env | grep AWS_SECRETthroughis_sensitive_bash_command->Blocked: command reads AWS credentials from environment variablesthenRefusal diagnostic: rule=credential-exfil-env-grep-aws component=env-credential span=0..21 ..., as under Manual verificationCleanup: the run's worktree and isolated data home were deleted; the instance is a shared short-lived verification host that keeps serving other runs and is terminated once idle, so it was not terminated for this PR
Screenshots / video
N/A — no frontend change.
Related Issues
Part of #34526
Sibling sweep
Siblings: none -- the size-ceiling refusal already names its rule, and the IMDS refusal names the spelling it matched.
Pattern harvest
Not generalizable: this was the one refusal of the gate that shares a fixed sentence across several rules.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...); PR Hygiene does not count commits