Repository navigation
fix: run the deploy-skill and ledger credential scans in linear time - #31445
Conversation
Sensitive Change ReviewSensitive files changed:
How to clear this check:
Only the author's comment counts this way. A reviewer can instead |
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: the change reuses the linear scanner Notes
[DESIGN-REVIEWED] 20e68f7 Description read[DESCRIPTION-READ] 431112fbed3d197b995dda420e684db62efb1dacb4232c9afad9f6212a7bb71a That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks: each caller moves onto the linear scanner Notes
What this change shipsInventory (7 items) — 6 justifiedIntent: stop the pre-deploy and pre-push credential scans stalling on long
[FIRST-PRINCIPLES-REVIEWED] 20e68f7 Description read[DESCRIPTION-READ] 8fb1e28f09e10e0d3552fce821776b5aee2ec5a3a9187686ed9afe97629476c0 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Security Scope Review -- ✅ PASSWhich legitimate operations does Verdict basis: adjudicated, zero confirmed regressions. Security scope review -- adjudicated candidates
No candidate flipped from allowed to refused. Scope-Verdict: PASS Reason: This diff doesn't make any shell-deny tier stricter. It adds TIGHTENING: No shell-tier mechanism got stricter. The behavioural change is in three scan sites, for example ADJUDICATED: 13 candidates, 0 confirmed newly-refused PLATFORM COVERAGE: I wrote posix candidates for the artifact-deploy skill's operator commands ( UNADJUDICATED: The changed guards are content scanners, not shell-command guards, so no row can reach them:
The inputs I would have submitted, described in words: a ledger line quoting a minified bundle with a long [SCOPE-REVIEWED] 20e68f7 |
2649c4a to
50adb2d
Compare
50adb2d to
d43a234
Compare
|
Rebased: main moved in docs/system-specs/modules/security.md, src/kiro_crew/security_posture_sinks_delivery.py and test/test_security_gate_liveness.py since this head went green (38a8ef5 -> 4d9cb58); CI re-running. The only conflict was the package line budget, which main raised to 34,753; this PR now adds its 10 on top (34,763). Targeted tests pass on the new head. |
d43a234 to
45ed54a
Compare
|
[operator: bolichen97] Rebased: main moved in test/test_security_gate_liveness.py since this head (d43a234 -> 76ec06a); CI re-running. The only conflict was the security package line budget, which main raised to 34,777; this PR adds its 10 on top (34,787). The patch is otherwise byte-identical (same patch-id). The previous head's red AI and scope lanes were rate-limit failures with no verdict, not findings. Targeted tests, isort, flake8, mypy, semgrep and local GPT and Opus reviews pass on 45ed54a. AI-assisted, posted on behalf of bolichen97 |
The artifact-deploy skill's pre-upload scans (deploy.sh, _common.sh) and the ops ledger's pre-push scan ran get_credential_patterns()'s raw search over whole files or lines. That search re-walks a long eyJ or digit run from every start, so one such file or line stalls the scan for seconds to minutes. Export the shared linear scanner as kiro_crew.security's public iter_credential_matches and move every caller outside the package to it, including the two that imported the private _credential_matches. Drop the unused get_credential_patterns re-export from the design tweak server. Findings stay those redact_credentials sees, and a scanner error still aborts the deploy.
45ed54a to
20e68f7
Compare
|
[operator: bolichen97] Rebased: main moved in src/kiro_crew/security/redaction.py and src/kiro_crew/security/redaction_patterns.py since this head went green (76ec06a -> 480d162); CI re-running. Main's change there shrank the security package, so the package now fits main's existing line budget and this PR no longer raises it (the budget hunk is dropped). The rest of the patch is unchanged. On the previous head, every red lane failed on the shared token's API rate limit; the scope lane's rerun reached PASS but was held by the per-head floor that the rate-limited first run set. Targeted tests, isort, flake8, mypy, semgrep and local GPT and Opus reviews pass on 20e68f7. AI-assisted, posted on behalf of bolichen97 |
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Approved for the on-call. Review lanes pass on 20e68f7, no conflict, no open change request.
…31445) The artifact-deploy skill's pre-upload scans (deploy.sh, _common.sh) and the ops ledger's pre-push scan ran get_credential_patterns()'s raw search over whole files or lines. That search re-walks a long eyJ or digit run from every start, so one such file or line stalls the scan for seconds to minutes. Export the shared linear scanner as kiro_crew.security's public iter_credential_matches and move every caller outside the package to it, including the two that imported the private _credential_matches. Drop the unused get_credential_patterns re-export from the design tweak server. Findings stay those redact_credentials sees, and a scanner error still aborts the deploy. (cherry picked from commit 7aaf1b1)
Problem / Motivation
Goal: The credential scans that run before an artifact deploy and before a team-ledger push finish in time linear in the file size, and still refuse the credentials they refuse today.
The artifact-deploy skill's pre-upload scans (
deploy.sh,_common.sh) and the ops ledger's pre-push scan (ledger_sync._credential_bearing_lines) hand each whole file or ledger line to the rawget_credential_patterns()regex withsearch. Thatsearchre-walks a longeyJrun or digit run from every start, so its cost grows with the square of the run. Measured on main: 60 KB ofeyJtakes 22 s, and 80 KB ofeyJatakes 46 s. Two other callers reach the shared linear scanner only through the private nameredaction._credential_matches.Why it matters
A minified bundle or data file holding such a run stalls
deploy.shbefore upload, and one long ledger line stalls a team-ledger push, both with the GIL held. With no public linear scan, the next caller copies the raw pattern again.Not a goal
redact_credentialsalready acts on.kiro_crew.redact_tokens.get_credential_patterns().What changed (motivation → approach → change)
The slow scans are a raw
searchover a whole file or line. The security package already has a scanner that finds the same credentials in linear time, the oneredact_credentialsruns, but it was private.kiro_crew.securitynow exports it asiter_credential_matches, and every caller outside the package uses it:deploy.sh,_common.sh(embedded Python)searchper fileiter_credential_matchesledger_sync._credential_bearing_linessearchper lineiter_credential_matchesdeploy/scan.py, design tweakpreview_files.py_credential_matchesiter_credential_matchesThe design tweak server drops its unused
get_credential_patternsre-export. A scanner error still aborts the deploy, and a file with a real key is still refused. The specs and the security-posture text insecurity_posture_sinks_delivery.pythat named the old accessor name the new one. The security package stays inside its existing line budget.These three gates now skip the shape-only lookalikes the redactor already rejects, such as
honeyJar.example.com(a JWT shape with no JSON header) or a CDK ARN resource name, so those no longer block a deploy or a push. The issues describe the cost as one read per repeated PEM header. For these stop-at-first-match scans that shape costs one extra read per file; the quadratic shapes are theeyJand digit runs, which the tests use.Backwards compatibility
Compatible: a credential the old scans refused is still refused; only the JWT/Telegram lookalikes
redact_credentialsalready lets through stop blocking, and the new name is additive.Reader: src/kiro_crew/apps/builtins/design_tweak/backend/server.py:get_credential_patterns -- app -- no reader: no
runtime.get_credential_patternsorserver.get_credential_patternson origin/main;test_design_tweak_backend.pypassesReader: src/kiro_crew/deploy/scan.py:_credential_matches -- app -- its only reader was the spy in
test_pem_scan_callers_linear.py, now aimed atscan.iter_credential_matchesReader: src/kiro_crew/apps/builtins/design_tweak/backend/preview_files.py:_credential_matches -- app -- no reader outside the module on origin/main
Tests
test_pem_scan_callers_linear.py::test_deploy_skill_scan_never_runs_the_full_pattern_over_a_file[deploy.sh|_common.sh]runs each script's embedded scan in-process: a file of 2,000eyJnever reaches the raw alternation, and a file with a real key still exits 2 with that file named.test_ledger_sync_coverage.py::test_credential_scan_never_runs_the_raw_search_over_a_line: a ledger line of 2,000eyJfollowed by an AWS key id is flagged with no rawsearch.test_ledger_sync_git.py::test_the_scan_stays_in_step_with_the_write_pathnow asserts the push scan callsiter_credential_matches, and the deploy-scan spy targets the public name.Proof: PROVEN src/kiro_crew/apps/builtins/ops_mission_control/tests/test_ledger_sync_coverage.py::test_credential_scan_never_runs_the_raw_search_over_a_line, src/kiro_crew/apps/builtins/ops_mission_control/tests/test_ledger_sync_git.py::TestPushRefusesCredentialMaterial::test_the_scan_stays_in_step_with_the_write_path, test/test_pem_scan_callers_linear.py::test_deploy_skill_scan_never_runs_the_full_pattern_over_a_file[deploy.sh], test/test_pem_scan_callers_linear.py::test_deploy_skill_scan_never_runs_the_full_pattern_over_a_file[_common.sh] (head 50adb2d)
Determinism: the two new tests passed 20 of 20 runs at
-n0and 10 of 10 at-n 4 --dist loadgroup --max-worker-restart=2beside the rest of both files, and a shuffled run of both files (seed 21697, 124 passed). They count calls on a patched pattern object, with no clock or ordering dependence.Manual verification
On main,
get_credential_patterns()[0].searchover 20,000eyJ(60 KB) took 22.5 s and over 20,000eyJa(80 KB) 45.8 s;iter_credential_matchesreturned on the same texts in 0.011 s and 0.015 s. Over 32,000 repeated PEM headers the rawsearchtook 0.028 s, which is why the tests use theeyJrun.Screenshots / video
N/A — no frontend change.
Related Issues
Closes #21697
Closes #21696
Sibling sweep
Swept: git grep -nE "get_credential_patterns()|_credential_matches(" -- src
Sibling: src/kiro_crew/deploy/skills/artifact-deploy/scripts/deploy.sh -- changed here
Sibling: src/kiro_crew/deploy/skills/artifact-deploy/scripts/_common.sh -- changed here
Sibling: src/kiro_crew/apps/builtins/ops_mission_control/backend/ledger_sync.py -- changed here
Sibling: src/kiro_crew/deploy/scan.py -- changed here
Sibling: src/kiro_crew/apps/builtins/design_tweak/backend/preview_files.py -- changed here
Sibling: src/kiro_crew/security/text_redaction.py -- not affected -- inside the security package, already runs the linear scanner
Sibling: src/kiro_crew/apps/builtins/aws_control/crew/packaging/tests/test_review_findings_security.py -- not affected -- a test that reads the pattern to probe token lengths, never scans a file
Pattern harvest
Rule candidate: semgrep
Pattern: a
get_credential_patterns()pattern used with.search/.finditeroutsidekiro_crew/security/, whereiter_credential_matchesis the linear scanChecklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributed under the project's Apache-2.0 license.