Skip to content
This repository was archived by the owner on Nov 9, 2026. It is now read-only.

fix(baseline): Redact only long tokens that could start a credential - #304

Merged
mpaulosky merged 4 commits into
mainfrom
fix/301-long-run-redaction
Oct 26, 2026
Merged

mpaulosky merged 4 commits into
mainfrom
fix/301-long-run-redaction

Conversation

@mpaulosky

Copy link
Copy Markdown
Owner

Why

Claude Review's finding on mpaulosky/TicketManager#249: redact_credentials replaced any dot-joined token holding a run over 128 characters. A generated test's file name quoted in a finding body or the Changes table (tests/Get_Articles_When_<130 chars>_Returns_401.cs) became tests/[redacted], lost its extension and raised a false "credential-shaped string(s) were redacted" warning.

What changed

  • The rule: the long-run replacement now applies only to a token that also holds one of the strings every CREDENTIAL alternative starts with (eyJ, gh[pousr]_, sk-ant-, github_pat_) — the strings that made CREDENTIAL quadratic. A long token without one can't match CREDENTIAL, so a long identifier, a digest or a generated name is left alone, and nothing credential-shaped is let through.
  • Where: post_claude_review.py and its mirror in the claude-review.yml answer check (CREDENTIAL_START), with the dotfiles root copies.
  • Docs: ADR 0004's redaction paragraph, redact_path's docstring and the workflow comment.

Verification

  • test.sh passes (the Python suite is 242 tests). New tests: long plain tokens are left alone with no warning, in a finding body and the summary, by the script and the workflow alike; a long token that starts a credential is still replaced whole for every start, including each gh[pousr]_ letter and a start in another dot-separated segment; the workflow carries the script's CREDENTIAL_START, MAX_TOKEN_RUN, TOKEN_CHAIN and CREDENTIAL; every CREDENTIAL match starts with a CREDENTIAL_START string.
  • Mutation checks: with the condition removed, 7 tests fail; with a start dropped from CREDENTIAL_START, 5 fail.
  • Local /pre-push-review: 3 rounds, 7 findings (4 Critical, 1 Warning, 2 Advisory) all fixed, none open.
  • Landing it needs a re-Apply of the other repos, since post_claude_review.py and claude-review.yml are Owned.

Fixes #301

🤖 Generated with Claude Code

mpaulosky and others added 3 commits October 26, 2026 12:04
The long-run rule replaced any token holding a run over 128 characters, so a
generated test's file name quoted in a finding or the summary lost its reference
and raised a false credential warning. It now applies only to a token that also
holds a string every CREDENTIAL alternative starts with (eyJ, gh?_, sk-ant-,
github_pat_), which is what made CREDENTIAL quadratic; the script and the workflow
mirror agree.

Fixes #301

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…w's copy of the rule

The new tests fail when an alternative of CREDENTIAL_START, the start in another
dot-separated segment, or the workflow's copy of the long-run rule is dropped, and
when a CREDENTIAL alternative starts with a string CREDENTIAL_START misses. A long
name that holds a start without being a credential is tested as left alone in a
finding's path, and redact_path's comments say why.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@mpaulosky mpaulosky added the review:claude Review with Claude label Oct 26, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Review

Overview
The long-run redaction rule now fires only when a long token also holds one of the strings CREDENTIAL starts with: eyJ, gh[pousr]_, sk-ant- or github_pat_. Before, any token with a run over 128 characters was replaced, which dropped long generated test names from reviews and raised false warnings (#301). I found no blocking problem. Reading the code and tests, I checked that CREDENTIAL_START covers every CREDENTIAL alternative, that the quadratic-time guard still holds, and that the script and the workflow stay in step. I did not run the tests.

  • Coverage of starts. A match of CREDENTIAL can only begin at one of those four strings. TOKEN_CHAIN's character class holds every character they use, and none of them contains a dot, so a start can't straddle a chain boundary.
  • Linear time. A chain with a run over 128 characters and a start is still replaced whole. What CREDENTIAL then scans has no such run, and a long run without a start can't begin a match. The existing linear-time tests still apply.
  • Script and workflow agree. The root and Template copies of the script and workflow have identical diffs. The workflow tests exec the heredoc from CREDENTIAL = re.compile( up to def answer_in, and the new CREDENTIAL_START line sits inside that range. The new test asserts the patterns are equal.
  • Test fixtures. I traced the new fixtures by hand against the new rule and they behave as the tests expect. For example, tests/Get_eyJ_AAAA…_Returns_401.cs is untouched by redact_path but becomes tests/[redacted] through redact_credentials.

Changes

File What changed
.github/scripts/post_claude_review.py and the Template copy Adds CREDENTIAL_START. The long-run rule in redact_credentials now also requires a start. Comments and the redact_path docstring are updated.
.github/workflows/claude-review.yml and the Template copy Mirrors the same constant and condition in the answer check's heredoc. The "must match" comment now names CREDENTIAL_START.
…/tests/test_post_claude_review.py Adds tests: long non-credential tokens are left alone; long tokens with each start are still replaced whole; the workflow and the script carry the same rule; every CREDENTIAL match starts with a start; a path holding eyJ is left alone. LONG_NAME is added to the script-vs-workflow fuzz.
docs/adr/0004-…md Documents the narrower rule and why (#301).

Advisory (not blocking)

  • Narrower backstop for encoded secrets. Before this change the blanket rule would also replace a base64 or hex rendering of CLAUDE_CODE_OAUTH_TOKEN in the answer. A token of about 108 characters is about 144 characters in base64 and 216 in hex, both over 128. Now such a blob has no start, so it is posted and printed in the post job's log. The ADR already lists base64/hex of the token as an accepted limit of the exact-match check, so this is consistent. But the blanket rule was catching it by accident, and the ADR's "accepted limits" list doesn't say that this change removes that accidental coverage. Consider one sentence in the ADR saying so.
  • Pinning CREDENTIAL_START by samples only. test_every_text_credential_matches_starts_with_a_credential_start checks fixed samples. A future CREDENTIAL alternative with a new prefix would pass these tests while its long runs skip the guard and reintroduce the quadratic scan. A comment next to CREDENTIAL saying to update CREDENTIAL_START with it would help.
  • Line length. The new "must match" comment in claude-review.yml is about 124 characters, and the new condition line in the heredoc is about 116. If yamllint's line-length limit is below that, test.sh will flag them. I couldn't check the yamllint config.
  • Rare false positive. A long identifier that happens to hold eyJ or ghp_ (for example a test name with eyJ in it) is still replaced whole in summaries and bodies. This is documented and tested only for paths.

No findings.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Review

Overview
This PR narrows the long-token redaction rule (#301). A token chain with a run over 128 characters is now replaced whole only if it also contains a CREDENTIAL_START string (eyJ, gh[pousr]_, sk-ant- or github_pat_). Long file names, identifiers and digests with none of those are left alone, with no false redaction warning. I found no blocking problems and would merge.

I traced the logic by hand and did not run the tests. Every CREDENTIAL match is made only of token characters and dots, and begins with one of the four start strings, which are all token characters. So any match lies inside one TOKEN_CHAIN chain that CREDENTIAL_START finds. A chain with no start therefore can't match CREDENTIAL, and leaving it alone drops nothing CREDENTIAL would have redacted.

A chain with a start and a long run is still replaced whole, as before. A chain with a start and only short runs is handled by CREDENTIAL, with each scan bounded by about 128 characters, so cost stays linear. The new tests walk through each start string, the cases where the start is in a different segment from the long run, and the plain long tokens, and these checks agree with my trace. The script and the workflow's inline copy match line for line, the dotfiles copy is identical to the Template copy, and a test pins the constants against each other.

Changes

File What changed
.github/scripts/post_claude_review.py and the Template copy under template/owned/.github/scripts/ Adds CREDENTIAL_START. redact_credentials now replaces a long chain only when it also holds a start. The comments and the redact_path docstring are updated.
.github/workflows/claude-review.yml and the Template copy The answer check's inline redact_credentials is mirrored with the same CREDENTIAL_START gate. The "must match" comment names the new constant.
template/owned/.github/scripts/tests/test_post_claude_review.py Adds LONG_NAME to the script-versus-workflow equivalence fuzz. Adds tests that long plain tokens are untouched with no warning, that every start string is still replaced whole, and that the workflow carries the same constants. Adds a check that every CREDENTIAL match begins with a start, and a test for a long path that holds eyJ.
docs/adr/0004-claude-review-as-copilots-backup.md Documents the new rule and the #301 rationale.

Advisory (not blocking)

  • Line length: the updated comment at .github/workflows/claude-review.yml:1745 (and the Template copy) is about 124 characters, up from about 106. If the repository's yamllint line-length max is 120 or lower, it would fail lint. I couldn't read .yamllint in this review, so check it.
  • Remaining false positives: a long token in a finding body or summary that contains a start string without being a credential is still replaced whole. Examples are a 130+ character identifier containing Highs_ or laughs_ (which hold ghs_) or eyJ. Only redact_path is exempt. The new comments and the ADR say this is accepted, and it is rare. It means the original false-warning problem is only mostly fixed, not fully.
  • Docstring drift: the module docstring of post_claude_review.py (lines 33–35) still says only that the script redacts "with the same CREDENTIAL, which must match the workflow's". It doesn't mention CREDENTIAL_START, MAX_TOKEN_RUN or TOKEN_CHAIN, which must also match.
  • Weaker catch-all: a long run with no start is no longer redacted, so redaction no longer catches an unknown-format secret above 128 characters. The exact 16-character secret-window check in the answer check still covers the real secrets, and the ADR records that check as the defence. I'm noting the weakening for the record.

I read every diff piece and every context piece.

No findings.

@mpaulosky
mpaulosky merged commit 950efa5 into main Oct 26, 2026
5 of 6 checks passed
@mpaulosky
mpaulosky deleted the fix/301-long-run-redaction branch October 26, 2026 20:04
mpaulosky added a commit that referenced this pull request Oct 26, 2026
)

## Why

[dotfiles#304](#304
`Template tests` run failed once with `ENOTEMPTY: directory not empty,
rmdir '.git/objects/pack'` in `checkpoint.test.mts` ("lets a branch that
leaves the gate script alone through"), and passed on a rerun. The test
removes its scratch git repository with a bare `rmSync(dir, { recursive:
true, force: true })` while git may still be writing into it (background
maintenance after a commit). Every assertion in the test had passed; the
failure was in the cleanup.

## What changed

- **The cleanup:** all 39 recursive `rmSync` calls in the Owned `lib/`
modules (`checkpoint.test.mts`, `host-safety.test.mts`, `scan.test.mts`,
`shell.test.mts`, `review-ledger.test.mts`, `git-sandbox-fake.mts`) now
pass `maxRetries: 5, retryDelay: 100`, which `rmSync` uses to retry
`ENOTEMPTY`. dotfiles' root copies match.
- **The guard:** a dotfiles-only test in `template.test.mts` fails when
a recursive `rmSync` in those modules lacks `maxRetries`, and when its
scan finds no call at all.
- The other `rmSync` uses (`review-ledger.mts`, one test) delete single
files with `{ force: true }` and need no retries.

## Verification

- `test.sh` passes; `check:sandcastle` passes (2029 tests, no failures).
- With one site left bare the new check fails; with the scan broken it
fails with "the scan checks nothing".
- Local `/pre-push-review`: 2 rounds, one Advisory finding (the guard
passing without checking anything), fixed; nothing open.
- A race can't be reproduced on demand, so the fix is the retry plus the
guard that keeps a bare call from coming back.
- Other repos get the retried cleanup on their next re-Apply.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
mpaulosky added a commit to mpaulosky/Blazor-Server that referenced this pull request Oct 26, 2026
## Why

Re-Apply the repo-ci-baseline Template so this repo carries the Template
changes from mpaulosky/dotfiles#303, mpaulosky/dotfiles#304,
mpaulosky/dotfiles#305 and mpaulosky/dotfiles#306.

## What changed

- `6af2035` Apply commit, `apply.sh` output alone: brings
mpaulosky/dotfiles#303, mpaulosky/dotfiles#304, mpaulosky/dotfiles#305
and mpaulosky/dotfiles#306.
  - `.github/scripts/post_claude_review.py`
  - `.github/scripts/tests/test_post_claude_review.py`
  - `.github/workflows/claude-review.yml`
  - `.sandcastle/lib/agents.test.mts`
  - `.sandcastle/lib/build.test.mts`
  - `.sandcastle/lib/checkpoint.test.mts`
  - `.sandcastle/lib/git-sandbox-fake.mts`
  - `.sandcastle/lib/host-safety.test.mts`
  - `.sandcastle/lib/plan.test.mts`
  - `.sandcastle/lib/review-ledger.test.mts`
  - `.sandcastle/lib/scan.test.mts`
  - `.sandcastle/lib/shell.test.mts`
  - `.sandcastle/lib/ui-role-fixture.mts` (added)
  - `.sandcastle/lib/ui-role-fixture.test.mts` (added)

## Verification

- The Apply commit is exactly `apply.sh`'s output on a branch up to date
with `main`.
- The pre-push gate (`scripts/gate.sh`) passed on `6af2035`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

review:claude Review with Claude

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(baseline): Stop the long-run redaction mangling ordinary review text

1 participant