Skip to content

test(chat): give the echo-scope fixture a checksum-valid credential - #38019

Merged
KiroCrewCaptain merged 1 commit into
kirodotdev:mainfrom
jeeshofone:fix/38012-echo-scope-needle
Nov 16, 2026
Merged

KiroCrewCaptain merged 1 commit into
kirodotdev:mainfrom
jeeshofone:fix/38012-echo-scope-needle

Conversation

@jeeshofone

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: test_user_echo_reaches_only_authorized_ws_clients passes on main again, and it still proves the echo scope boundary.

test/test_chat_send_echo_scope.py::test_user_echo_reaches_only_authorized_ws_clients fails on every run on main since #37408 merged, at -n0 locally and on shard 3 of every open PR that runs the file.

Why it matters

Every open PR whose shard runs this file is red on it, so the failure blocks readiness across the queue.

Not a goal

What changed (motivation → approach → change)

  • Symptom: the final check, that the app's SSE stream never contains the credential, found the fixture's whole message in a slots frame's last_message preview.
  • Cause: the fixture token was ghp_ followed by 36 x characters, which fails the GitHub token checksum. Under feat(security): one redaction switch, rule table only, value-only masks #37408 the rule table reads a ghp_ run without a valid checksum as a benign placeholder, so nothing masks it. A real token on the same path is masked, so this is a fixture problem, not a leak.
  • Change, test only:
    • The fixture builds its token with kiro_crew.testing.synthetic_credentials.github_classic(), the helper the testing conventions name for a token that must be masked.
    • The leak checks search for the token value, which is stricter than searching for the whole message.
    • When the app may read user slots, its WebSocket row is now pinned to the owner's row with only the token masked, as feat(security): one redaction switch, rule table only, value-only masks #37408 specifies for a non-owner socket. When it may not, it still receives nothing.

Backwards compatibility

Compatible: only a test file changes.

Removes nothing: no product code or public helper is touched.

Tests

  • test/test_chat_send_echo_scope.py: 7 passed at -n0, three runs in a row. On main the same file fails.
  • Redaction fixed-point, stream-table parity and the related echo suites passed.
  • black, isort, flake8, Semgrep (repo rules) and the comment-history gate are clean.

Proof: not provable -- test-only change

Manual verification

N/A — the change is to a test, and the test run is the verification.

Related Issues

Fixes #38012

Sibling sweep

Swept: git grep -n 'ghp_" + "x"' -- test/
Siblings: none -- the other repeated-x ghp_ fixtures are placeholder cases or scan-cost filler, and none asserts the token is masked

Pattern harvest

Not generalizable: the testing conventions already require synthetic_credentials for credentials that must be masked; this fixture predated that rule.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license.

The fixture's ghp_ token had no valid checksum, so under the rule table
it reads as a benign placeholder and nothing masks it: the SSE slots
frame carried it in last_message and the leak check failed every run.
The fixture now uses the synthetic_credentials helper's checksum-valid
token, the leak checks look for the credential value, and a non-owner
socket's row is pinned to the masked form with the rest kept.

Fixes kirodotdev#38012
@jeeshofone
jeeshofone requested review from a team and KiroCrewCaptain as code owners November 16, 2026 17:07
@jeeshofone
jeeshofone requested a review from buluoray November 16, 2026 17:07
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: checking Automated validation is still running labels Nov 16, 2026
@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5, fork) — ⏭️ skipped

Revision 4cfceb853957b6e14e7b70101ea2865718c80992 ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5, fork) — ⏭️ skipped

Revision 4cfceb853957b6e14e7b70101ea2865718c80992 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5, fork) — ✅ PASS

Design-level review of 4cfceb853957b6e14e7b70101ea2865718c80992 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Test-only change: it swaps in a checksum-valid credential and makes the leak checks stricter, so nothing blocks.

[DESIGN-REVIEWED] 4cfceb8

Description read

[DESCRIPTION-READ] edeb859bc22e2f236420f44fc4051175323f44efc41193d62029dd2047b2ff8f

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

Opus 5.5 Review (fork) — ✅ no blocking findings

Reviewed 4cfceb853957b6e14e7b70101ea2865718c80992 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 4cfceb8

@github-actions

Copy link
Copy Markdown
Contributor

GPT 6.1 Review (fork) — ✅ no blocking findings

Reviewed 4cfceb853957b6e14e7b70101ea2865718c80992 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 4cfceb8

@buluoray buluoray left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve on 4cfceb8. The test planted a GitHub credential that the new rule table correctly treats as a placeholder, and the fix plants a checksum-valid one that main does mask on every path the test checks, so this repairs a test and hides no leak. Decisive lines: test/test_chat_send_echo_scope.py:18-27, test/test_chat_send_echo_scope.py:194-206, src/kiro_crew/security/redaction_structured/table.py:46-54, src/kiro_crew/security/redaction_structured/checks.py:30-35, src/kiro_crew/security/redaction_structured/checks.py:145-150

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 16, 2026
@KiroCrewCaptain
KiroCrewCaptain merged commit 8cbc84d into kirodotdev:main Nov 16, 2026
143 of 144 checks passed

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This review checked head 4cfceb8 by running the test. At this head, test/test_chat_send_echo_scope.py passes all 7 cases. With main's version of the file on the same tree, [False-ws-send-id] fails with the unmasked placeholder in the SSE slots frame, which matches #38012. The cause is confirmed in the redaction code: the old ghp_ + x run fails the checksum, so the redactor treats it as a placeholder. The changed assertions are tighter than before. The needle is now the token, which is a substring of the prompt. The observer socket must now receive exactly the owner's row with only the token replaced by [REDACTED: credential]. A socket without slots:user must still receive nothing.
Done well: the fixture uses the shared github_classic() helper, so the test gets a checksum-valid credential and no credential-shaped literal lands in the repo.
Optional:

  • #37498 rewrites the same _PROMPT line in a different way. Whichever PR merges second needs a rebase, and that rebase should keep the github_classic() token.
  • This is older than this PR: the slots frame reaches the SSE stream only in the first parametrization a process runs. A follow-up could make that check run in every case.

@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

main red: test_chat_send_echo_scope.py::test_user_echo_reaches_only_authorized_ws_clients leaks the credential after #37408

3 participants