Skip to content

test: plant a secret the redactor removes in the row-scrub tests - #37423

Merged
bolichen97 merged 1 commit into
mainfrom
fix/row-scrub-tests-use-a-masked-secret
Nov 15, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
fix/row-scrub-tests-use-a-masked-secret

Conversation

@bolichen97

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Eight backend tests that are red on main for every pull request go green again, without changing what the redactor keeps or removes.

#36640 added row-scrub tests that plant the AWS documentation example key and assert it is removed from a persisted conversation row. #29423, merged 50 minutes earlier, turned on the structured credential detectors, which keep an AWS key ID (the documentation example key included) visible as an identifier. The redactor is right and the fixtures are wrong, so Backend Tests (3.12) shards 1, 2 and 8 and Backend Tests (Windows) (8) fail on every PR (#37421).

Why it matters

Every open PR's backend shards are red for a reason none of them caused, so no PR can show a clean CI run until this lands.

Not a goal

  • Changing the redactor, the structured detectors or what an AWS key ID counts as.
  • Touching any test that passes today.

What changed (motivation → approach → change)

The failing tests plant a value the redactor deliberately keeps. They now plant a synthetic GitHub token from kiro_crew.testing.synthetic_credentials, a value the redactor removes on every path these tests cover, which is how the other redaction tests adapted to #29423. Each assertion is unchanged: the token must be gone from the stored row, the live projection and the fallback pass of a miscomposed host. Four test files change (test_autonudge_row_scrub.py, test_history.py, test_telegram_sessions_resume.py, test_one_conversation_one_session.py); no production code changes.

Backwards compatibility

Compatible: tests only, no shipped code changes.

Removes nothing: every test and assertion stays; only the planted value changes.

Tests

The eight node ids of #37421 fail on origin/main fdf7cbc2ec (8 failed) and pass with this commit; the four files pass whole (210 passed).

Proof: not provable -- test-only change

Determinism: the seven affected test classes and node ids ran three times in a row and once in reversed order with -n0: 18 passed each time. The token is built deterministically (github_classic() with a fixed seed), no clock, port or ordering is involved. test/test_flake_pattern_ratchet.py passes.

Manual verification

N/A — the change is test fixtures only, and the failing node ids are the verification.

Related Issues

Closes #37421

Sibling sweep

Swept: git grep -n "AKIAIOSFODNN7EXAMPLE" -- test/
Sibling: test/test_security_posture.py -- not affected -- its fixtures check a URL and base64 detector, which still flag the key, and pass on main
Sibling: test/test_one_conversation_one_session.py -- changed here -- the dedup test planted the same key and no longer exercised a scrub

Pattern harvest

Not generalizable: two PRs that each passed CI on their own base disagreed about one fixture value; the shared synthetic-credentials helper already exists for exactly this, and it is what the fix uses.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributed under the project's Apache-2.0 license.

The row-scrub tests planted the AWS documentation example key and
asserted it was removed from a persisted row. The structured credential
detectors keep an AWS key ID visible as an identifier, so the redactor
correctly left it and eight tests went red on main. They now plant a
synthetic GitHub token from kiro_crew.testing.synthetic_credentials, a
value the redactor removes on every path these tests cover.

Closes #37421
@bolichen97
bolichen97 requested review from a team and KiroCrewCaptain as code owners November 15, 2026 00:15
@bolichen97
bolichen97 enabled auto-merge (squash) November 15, 2026 00:15
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 15, 2026
@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision fa888655fe0b50622c1ae6b23e8cb2395df5d2ad ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision fa888655fe0b50622c1ae6b23e8cb2395df5d2ad touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of fa888655fe0b50622c1ae6b23e8cb2395df5d2ad — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing to check. The patch only swaps test fixtures to a credential the redactor removes. Every assertion and all production behaviour stay the same.

[DESIGN-REVIEWED] fa88865

Description read

[DESCRIPTION-READ] c2e2c5e4a78932991a2989928e3c394ef2cd747cb7df891d1a9e7eadafb16aaf

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed fa888655fe0b50622c1ae6b23e8cb2395df5d2ad — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] fa88865

Verdict parsed from the review's SHA-scoped output markers for commit fa888655fe0b50622c1ae6b23e8cb2395df5d2ad.

False positive or not applicable? A repository writer can comment:
/ai-review override fable fa888655fe0b50622c1ae6b23e8cb2395df5d2ad false-positive: <reason, 20 to 500 characters>

@github-actions

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of fa888655fe0b50622c1ae6b23e8cb2395df5d2ad and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] fa88865

False positive or not applicable? A repository writer can comment:
/ai-review override gpt fa888655fe0b50622c1ae6b23e8cb2395df5d2ad false-positive: <reason, 20 to 500 characters>

@KiroCrewCaptain

Copy link
Copy Markdown
Collaborator

Code change approver recommendation: Approve

Why: the decider estimates a 5% chance that this change introduces a defect, within the 12% limit for approving on its own, and every check passes.

Test-only change: the row-scrub tests used to plant the AWS documentation example key, which the structured detectors from #29423 now deliberately keep visible as an identifier. They now plant a deterministic synthetic GitHub token from kiro_crew.testing.synthetic_credentials. Every assertion is unchanged. The token goes through the composed policy or the baseline redact fallback in _redact_at_write_boundary (history_append.py:53-80), the paths these tests cover, and other redaction tests already rely on it the same way. Imports are correct (test_one_conversation_one_session.py already imports github_classic from redaction_structured_cases), and the positive control still plants a credential that gets rewritten. No production code changes, so the state, lifecycle, limits and trust sweeps don't apply.

An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit fa888655fe · 42 changed lines. If this recommendation is wrong, reply here: replies are how the approver is evaluated.

@bolichen97
bolichen97 disabled auto-merge November 15, 2026 00:57
@bolichen97
bolichen97 merged commit a049d91 into main Nov 15, 2026
94 of 95 checks passed
@bolichen97
bolichen97 deleted the fix/row-scrub-tests-use-a-masked-secret branch November 15, 2026 00:57
@github-actions github-actions Bot removed the readiness: checking Automated validation is still running label Nov 15, 2026
bolichen97 added a commit that referenced this pull request Nov 15, 2026
…37692)

A test that plants the AWS documentation example key and asserts it is
removed depends on detector policy, not on the code under test: the
structured redactor keeps AWS key IDs visible as identifiers. #36640 did
this after #29423 and turned main red on every platform until #37423
(issue #37421). Record the rule in testing-conventions and point to it
from AGENTS.md.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Eight row-scrub tests are red on main: they plant an AWS key ID the redactor keeps

2 participants