Repository navigation
test: use synthetic credentials in the remaining test fixtures - #36127
mbriones98 wants to merge 2 commits into
Conversation
First Principles Review (Opus 5.5) — ⏭️ skippedRevision |
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS This PR only changes test fixtures. It removes no test or assertion and touches no production code, so nothing blocks. Notes
[DESIGN-REVIEWED] 65367ee Description read[DESCRIPTION-READ] c738533d351f613e3dbfd21fa018989374f27da6590d13bad06e09bd8e85b744 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Tests that need a credential the redactor removes used the AWS documentation example key or an AWS access key ID. The structured detectors keep both visible as identifiers, so once they are on those tests would pass or fail for the wrong reason. This first half adds kiro_crew.testing.synthetic_credentials, which builds synthetic credentials of real shapes at run time, and moves the shared fixtures (test/conftest.py, test/redaction_structured_cases.py, the fake computer-use helper) and the built-in app tests onto it, along with test files from test_a* through test_mate*. The rest of test/ follows in a second change. Each swapped test passes with the detectors off, as on the base, and with them on. Production code is untouched. Where an assertion named a piece of the old key (a prefix such as "AKIAIO", a body fragment split by markup) it now names the same piece of the new credential. Where the old key was glued to padding, the padding now ends at a blank, because a GitHub token is matched at a word boundary; the cut still falls inside the credential. A test whose premise is pass 1's own reading (a credential that eats a JSON delimiter) checks that premise with the detectors off. The black baseline drops one test file that is now formatted.
The second half of the fixture swap. Tests that need a credential the redactor removes used the AWS documentation example key or an AWS access key ID. The structured detectors keep both visible as identifiers, so once they are on those tests would pass or fail for the wrong reason. These fixtures, test files from test_mcp* through test_z*, now use synthetic credentials of real shapes from kiro_crew.testing.synthetic_credentials, which the first half adds, or split literals of the same shape. Each swapped test passes with the detectors off, as on the base, and with them on. Production code is untouched. Where an assertion named a piece of the old key it now names the same piece of the new credential. Where the old key was glued to padding, the padding now ends at a blank, because a GitHub token is matched at a word boundary; the cut still falls inside the credential.
0edd187 to
3a68d2c
Compare
ff2fd5a to
65367ee
Compare
3a68d2c to
9fee228
Compare
Problem / Motivation
Goal: the rest of the redaction tests that need a credential use a synthetic one of a real shape, so they keep testing redaction once the structured detectors keep AWS key IDs visible.
Many tests use the AWS documentation example key or another AWS access key ID as "the credential the redactor must remove". #29423 turns the structured detectors on, and they treat an AWS access key ID and the documentation example values as identifiers that stay visible. Every test that relies on them being removed would then pass or fail for the wrong reason. #31863 swaps the first half of those fixtures and adds the shared helper; this PR swaps the second half.
Why it matters
In one PR, the swaps changed 416 test files that CI's New-test stability job repeats three times. On Windows that took longer than the job's 60-minute limit, so the job could never pass. Split in two, each PR repeats about 210 files.
Not a goal
What changed (motivation → approach → change)
test/, fromtest/test_mcp_*.pythroughtest/test_z*.py: fixtures that used the AWS documentation example key, or an AWS key ID, as the value to redact now use a synthetic credential fromkiro_crew.testing.synthetic_credentials(added in test: use synthetic credentials in redaction test fixtures #31863), or a split literal of the same shape.test/test_messaging_auto_title.py,test/test_workflows_service.pyandtest/test_queue_discard_sender_notice.pyname the test variables that hold synthetic values neutrally, so CodeQL does not read them as clear-text secrets. Values and assertions are unchanged.test/test_outbound_files.pyandtest/test_search_chat_history.pykeep the tests main added to them since; only the fixture values change.Backwards compatibility
Compatible: only test files change; no production code, config or data changes.
Removes nothing: no test is deleted and no assertion is removed.
Tests
Proof: not provable -- test-only change
Manual verification
N/A — unit coverage sufficient: test fixtures only.
Screenshots / video
N/A — no UI change.
Related Issues
Part of #22552
Stack: #31863 (shared helper and first half, base
main) → this PR (second half, base #31863's branch) → #29423 (turns the switch on).Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license.
Written by the Crew Tasks Bot on behalf of mbriones98.