Repository navigation
fix(baseline): Redact only long tokens that could start a credential - #304
Conversation
The long-run rule replaced any token holding a run over 128 characters, so a generated test's file name quoted in a finding or the summary lost its reference and raised a false credential warning. It now applies only to a token that also holds a string every CREDENTIAL alternative starts with (eyJ, gh?_, sk-ant-, github_pat_), which is what made CREDENTIAL quadratic; the script and the workflow mirror agree. Fixes #301 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…w's copy of the rule The new tests fail when an alternative of CREDENTIAL_START, the start in another dot-separated segment, or the workflow's copy of the long-run rule is dropped, and when a CREDENTIAL alternative starts with a string CREDENTIAL_START misses. A long name that holds a start without being a credential is tested as left alone in a finding's path, and redact_path's comments say why. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Claude Review
Overview
The long-run redaction rule now fires only when a long token also holds one of the strings CREDENTIAL starts with: eyJ, gh[pousr]_, sk-ant- or github_pat_. Before, any token with a run over 128 characters was replaced, which dropped long generated test names from reviews and raised false warnings (#301). I found no blocking problem. Reading the code and tests, I checked that CREDENTIAL_START covers every CREDENTIAL alternative, that the quadratic-time guard still holds, and that the script and the workflow stay in step. I did not run the tests.
- Coverage of starts. A match of
CREDENTIALcan only begin at one of those four strings.TOKEN_CHAIN's character class holds every character they use, and none of them contains a dot, so a start can't straddle a chain boundary. - Linear time. A chain with a run over 128 characters and a start is still replaced whole. What
CREDENTIALthen scans has no such run, and a long run without a start can't begin a match. The existing linear-time tests still apply. - Script and workflow agree. The root and Template copies of the script and workflow have identical diffs. The workflow tests exec the heredoc from
CREDENTIAL = re.compile(up todef answer_in, and the newCREDENTIAL_STARTline sits inside that range. The new test asserts the patterns are equal. - Test fixtures. I traced the new fixtures by hand against the new rule and they behave as the tests expect. For example,
tests/Get_eyJ_AAAA…_Returns_401.csis untouched byredact_pathbut becomestests/[redacted]throughredact_credentials.
Changes
| File | What changed |
|---|---|
.github/scripts/post_claude_review.py and the Template copy |
Adds CREDENTIAL_START. The long-run rule in redact_credentials now also requires a start. Comments and the redact_path docstring are updated. |
.github/workflows/claude-review.yml and the Template copy |
Mirrors the same constant and condition in the answer check's heredoc. The "must match" comment now names CREDENTIAL_START. |
…/tests/test_post_claude_review.py |
Adds tests: long non-credential tokens are left alone; long tokens with each start are still replaced whole; the workflow and the script carry the same rule; every CREDENTIAL match starts with a start; a path holding eyJ is left alone. LONG_NAME is added to the script-vs-workflow fuzz. |
docs/adr/0004-…md |
Documents the narrower rule and why (#301). |
Advisory (not blocking)
- Narrower backstop for encoded secrets. Before this change the blanket rule would also replace a base64 or hex rendering of
CLAUDE_CODE_OAUTH_TOKENin the answer. A token of about 108 characters is about 144 characters in base64 and 216 in hex, both over 128. Now such a blob has no start, so it is posted and printed in the post job's log. The ADR already lists base64/hex of the token as an accepted limit of the exact-match check, so this is consistent. But the blanket rule was catching it by accident, and the ADR's "accepted limits" list doesn't say that this change removes that accidental coverage. Consider one sentence in the ADR saying so. - Pinning
CREDENTIAL_STARTby samples only.test_every_text_credential_matches_starts_with_a_credential_startchecks fixed samples. A futureCREDENTIALalternative with a new prefix would pass these tests while its long runs skip the guard and reintroduce the quadratic scan. A comment next toCREDENTIALsaying to updateCREDENTIAL_STARTwith it would help. - Line length. The new "must match" comment in
claude-review.ymlis about 124 characters, and the new condition line in the heredoc is about 116. If yamllint's line-length limit is below that,test.shwill flag them. I couldn't check the yamllint config. - Rare false positive. A long identifier that happens to hold
eyJorghp_(for example a test name witheyJin it) is still replaced whole in summaries and bodies. This is documented and tested only for paths.
No findings.
There was a problem hiding this comment.
Claude Review
Overview
This PR narrows the long-token redaction rule (#301). A token chain with a run over 128 characters is now replaced whole only if it also contains a CREDENTIAL_START string (eyJ, gh[pousr]_, sk-ant- or github_pat_). Long file names, identifiers and digests with none of those are left alone, with no false redaction warning. I found no blocking problems and would merge.
I traced the logic by hand and did not run the tests. Every CREDENTIAL match is made only of token characters and dots, and begins with one of the four start strings, which are all token characters. So any match lies inside one TOKEN_CHAIN chain that CREDENTIAL_START finds. A chain with no start therefore can't match CREDENTIAL, and leaving it alone drops nothing CREDENTIAL would have redacted.
A chain with a start and a long run is still replaced whole, as before. A chain with a start and only short runs is handled by CREDENTIAL, with each scan bounded by about 128 characters, so cost stays linear. The new tests walk through each start string, the cases where the start is in a different segment from the long run, and the plain long tokens, and these checks agree with my trace. The script and the workflow's inline copy match line for line, the dotfiles copy is identical to the Template copy, and a test pins the constants against each other.
Changes
| File | What changed |
|---|---|
.github/scripts/post_claude_review.py and the Template copy under template/owned/.github/scripts/ |
Adds CREDENTIAL_START. redact_credentials now replaces a long chain only when it also holds a start. The comments and the redact_path docstring are updated. |
.github/workflows/claude-review.yml and the Template copy |
The answer check's inline redact_credentials is mirrored with the same CREDENTIAL_START gate. The "must match" comment names the new constant. |
template/owned/.github/scripts/tests/test_post_claude_review.py |
Adds LONG_NAME to the script-versus-workflow equivalence fuzz. Adds tests that long plain tokens are untouched with no warning, that every start string is still replaced whole, and that the workflow carries the same constants. Adds a check that every CREDENTIAL match begins with a start, and a test for a long path that holds eyJ. |
docs/adr/0004-claude-review-as-copilots-backup.md |
Documents the new rule and the #301 rationale. |
Advisory (not blocking)
- Line length: the updated comment at
.github/workflows/claude-review.yml:1745(and the Template copy) is about 124 characters, up from about 106. If the repository's yamllintline-lengthmax is 120 or lower, it would fail lint. I couldn't read.yamllintin this review, so check it. - Remaining false positives: a long token in a finding body or summary that contains a start string without being a credential is still replaced whole. Examples are a 130+ character identifier containing
Highs_orlaughs_(which holdghs_) oreyJ. Onlyredact_pathis exempt. The new comments and the ADR say this is accepted, and it is rare. It means the original false-warning problem is only mostly fixed, not fully. - Docstring drift: the module docstring of
post_claude_review.py(lines 33–35) still says only that the script redacts "with the same CREDENTIAL, which must match the workflow's". It doesn't mentionCREDENTIAL_START,MAX_TOKEN_RUNorTOKEN_CHAIN, which must also match. - Weaker catch-all: a long run with no start is no longer redacted, so redaction no longer catches an unknown-format secret above 128 characters. The exact 16-character secret-window check in the answer check still covers the real secrets, and the ADR records that check as the defence. I'm noting the weakening for the record.
I read every diff piece and every context piece.
No findings.
) ## Why [dotfiles#304](#304 `Template tests` run failed once with `ENOTEMPTY: directory not empty, rmdir '.git/objects/pack'` in `checkpoint.test.mts` ("lets a branch that leaves the gate script alone through"), and passed on a rerun. The test removes its scratch git repository with a bare `rmSync(dir, { recursive: true, force: true })` while git may still be writing into it (background maintenance after a commit). Every assertion in the test had passed; the failure was in the cleanup. ## What changed - **The cleanup:** all 39 recursive `rmSync` calls in the Owned `lib/` modules (`checkpoint.test.mts`, `host-safety.test.mts`, `scan.test.mts`, `shell.test.mts`, `review-ledger.test.mts`, `git-sandbox-fake.mts`) now pass `maxRetries: 5, retryDelay: 100`, which `rmSync` uses to retry `ENOTEMPTY`. dotfiles' root copies match. - **The guard:** a dotfiles-only test in `template.test.mts` fails when a recursive `rmSync` in those modules lacks `maxRetries`, and when its scan finds no call at all. - The other `rmSync` uses (`review-ledger.mts`, one test) delete single files with `{ force: true }` and need no retries. ## Verification - `test.sh` passes; `check:sandcastle` passes (2029 tests, no failures). - With one site left bare the new check fails; with the scan broken it fails with "the scan checks nothing". - Local `/pre-push-review`: 2 rounds, one Advisory finding (the guard passing without checking anything), fixed; nothing open. - A race can't be reproduced on demand, so the fix is the retry plus the guard that keeps a bare call from coming back. - Other repos get the retried cleanup on their next re-Apply. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
## Why Re-Apply the repo-ci-baseline Template so this repo carries the Template changes from mpaulosky/dotfiles#303, mpaulosky/dotfiles#304, mpaulosky/dotfiles#305 and mpaulosky/dotfiles#306. ## What changed - `6af2035` Apply commit, `apply.sh` output alone: brings mpaulosky/dotfiles#303, mpaulosky/dotfiles#304, mpaulosky/dotfiles#305 and mpaulosky/dotfiles#306. - `.github/scripts/post_claude_review.py` - `.github/scripts/tests/test_post_claude_review.py` - `.github/workflows/claude-review.yml` - `.sandcastle/lib/agents.test.mts` - `.sandcastle/lib/build.test.mts` - `.sandcastle/lib/checkpoint.test.mts` - `.sandcastle/lib/git-sandbox-fake.mts` - `.sandcastle/lib/host-safety.test.mts` - `.sandcastle/lib/plan.test.mts` - `.sandcastle/lib/review-ledger.test.mts` - `.sandcastle/lib/scan.test.mts` - `.sandcastle/lib/shell.test.mts` - `.sandcastle/lib/ui-role-fixture.mts` (added) - `.sandcastle/lib/ui-role-fixture.test.mts` (added) ## Verification - The Apply commit is exactly `apply.sh`'s output on a branch up to date with `main`. - The pre-push gate (`scripts/gate.sh`) passed on `6af2035`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Why
Claude Review's finding on mpaulosky/TicketManager#249:
redact_credentialsreplaced any dot-joined token holding a run over 128 characters. A generated test's file name quoted in a finding body or theChangestable (tests/Get_Articles_When_<130 chars>_Returns_401.cs) becametests/[redacted], lost its extension and raised a false "credential-shaped string(s) were redacted" warning.What changed
CREDENTIALalternative starts with (eyJ,gh[pousr]_,sk-ant-,github_pat_) — the strings that madeCREDENTIALquadratic. A long token without one can't matchCREDENTIAL, so a long identifier, a digest or a generated name is left alone, and nothing credential-shaped is let through.post_claude_review.pyand its mirror in theclaude-review.ymlanswer check (CREDENTIAL_START), with the dotfiles root copies.redact_path's docstring and the workflow comment.Verification
test.shpasses (the Python suite is 242 tests). New tests: long plain tokens are left alone with no warning, in a finding body and the summary, by the script and the workflow alike; a long token that starts a credential is still replaced whole for every start, including eachgh[pousr]_letter and a start in another dot-separated segment; the workflow carries the script'sCREDENTIAL_START,MAX_TOKEN_RUN,TOKEN_CHAINandCREDENTIAL; everyCREDENTIALmatch starts with aCREDENTIAL_STARTstring.CREDENTIAL_START, 5 fail./pre-push-review: 3 rounds, 7 findings (4 Critical, 1 Warning, 2 Advisory) all fixed, none open.post_claude_review.pyandclaude-review.ymlare Owned.Fixes #301
🤖 Generated with Claude Code