Skip to content

fix(security): scan exfil URLs past ) ' > and uppercase schemes - #26741

Merged
KiroCrewCaptain merged 1 commit into
mainfrom
fix/exfil-scan-bypass-7666
Nov 1, 2026
Merged

KiroCrewCaptain merged 1 commit into
mainfrom
fix/exfil-scan-bypass-7666

Conversation

@bolichen97

@bolichen97 bolichen97 commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: The exfiltration URL scanner inspects a URL's whole path and query, so a payload placed after ), ' or >, or behind an uppercase scheme, is caught and redacted.

The scanner ended every URL at the first ), ' or >. Those bytes are legal inside a URL, so anything after one was never checked. On main, each of these scans clean and passes through redact_exfiltration_urls unchanged (a 56-character base64 blob stands in for the payload):

  • https://evil.example.com/a)b?data=<blob>
  • https://evil.example.com/a'b?data=<blob>
  • https://evil.example.com/a>b?data=<blob>
  • https://evil.example.com/a)<AWS key id>
  • HTTPS://evil.example.com/a?data=<blob> (the scheme was matched case-sensitively)
  • 'https://evil.example.com/a?x=1',<blob>
  • (https://evil.example.com/a?x=1)<blob>

Why it matters

Model and provider text passes through this redactor before a user sees it. One punctuation byte was enough to keep a credential-like query out of every check, so the link survived with its payload.

Not a goal

  • The browser-side mirror in website/src/utils/sanitize.ts, which still stops at ' and >.
  • Splitting a redirect URL that carries a second URL in its query (https://host/r?next=https://...). That shape is judged as one URL, as on main.
  • Retiring the provider-text pre-encoding in dashboard/source_providers/adf.py. It still works and is now redundant.

What changed (motivation → approach → change)

Root cause: the one regex served two jobs. It found where a renderer ends a link, and it decided what the scanner reads. The first job needs the stop at ), ' and >; the second must not stop there.

The scanner and redactor now use a second pattern, _URL_SCAN_RE, with the same groups. Its scheme is matched case-insensitively (ASCII only, and only the scheme, so any Unicode space still ends a URL). After the old stop byte it reads on to whitespace, a double quote, or the next http(s)://<host>. Stopping before a glued second URL keeps that URL judged under its own host, so it never rides the first host's exemptions.

Each match yields two spans: the old narrow one, then the whole match. The narrow span is judged first. If it is flagged, only it is replaced, so markup after a flagged link survives exactly as on main. If it is clean, the whole match is judged, and replaced when flagged. A tail of punctuation only is not judged, so [x](<presigned S3 URL>) stays clean. In the added tail, each run of non-ASCII text is judged as one space, so Chinese or Japanese prose glued to a link is not read as a long query. restore_allowed_links reads URLs with the same spans, and the prompt optimizer's draft check (dashboard/handlers/optimizer.py) reads them with the same walk, embedded URLs included. _URL_RE itself is unchanged.

A valid S3 presigned link and our own Slack app-create link pass by their exact parameter shape. When the narrow span passed that way, the added bytes are judged as a query of their own (the Slack alias joins them), so **[f](<presigned>)**, a table cell or a 。 after the link stays clean.

The narrow span does not stop at a second http(s)://, since a redirect carries one in its query. So when a match is clean, each URL starting inside it on a host the exempt set does not hold is judged under its own host, and the first flagged one is replaced as its own span. An exempt host's relaxed checks then cover only its own span: in https://exempt/r?x=1`[open](https://evil/?d=<blob>) only the second link is replaced. Each search resumes where the last embedded match ends, so a bare-host decoy does not end the walk, and no byte is searched twice.

This replaces the span model of #7666, which could not be rebased after main moved these rules into exfil_url_rules.py and rewrote the redact loop. It follows the two-span rule suggested in review there, which also removes the per-apostrophe tail scan.

The span walk lives in a new owner module, security/exfil_scan_span.py, listed in the exfil facade's owners (test/test_exfil_facade.py) and in the display battery's rule sources (dashboard/display_battery.py); exfil.py keeps thin adapters so it stays under the line cap. The spec paragraph in docs/system-specs/modules/security.md describes the scan span.

Backwards compatibility

Compatible: every URL main flags is replaced over the same span as before. The only new output is a redaction where main returned the text unchanged. The known cost is over-redaction of a clean URL glued, with no whitespace, to a following payload-shaped run, such as a compact field 'url':'https://host/p?a=1','sha':'<40 hex>', or Chinese or Japanese prose (which has no spaces) carrying a 40-hex SHA right after a link's ). A tail is not cut at the first non-ASCII byte, because an autolinker reads on through such text to the next space, so the blob would still be fetched. That shape is byte-identical to the attack, and the redactor fails closed.

Removes nothing: no name, field or behaviour is removed; _URL_RE keeps its pattern.

Tests

test/test_exfil_scan_span.py (new):

  • Each bypass above is flagged and its payload removed. These fail on main.

  • A flagged link in [x](...), <a href='...'>label</a> and <...> keeps the text after it, with the exact output pinned.

  • Clean links in parentheses, quotes, a balanced wiki path and an HTML attribute are left untouched.

  • A URL glued after ) to an exempt host's link is judged under its own host.

  • A second URL after a no-break, ideographic or em space is judged under its own host.

  • Chinese prose glued after a link's ) is kept; a blob glued between Chinese words is still caught.

  • An allowed host's link before ) is restored by restore_allowed_links.

  • A link glued inside an exempt host's URL is judged under its own host, with the outer scheme written HTTPS, https and HtTpS; one past a chain of exempt URLs is caught too; a clean one is left alone.

  • A payload link after a bare-host decoy inside an exempt link is caught (test_embedded_url_after_a_bare_host_decoy_is_caught).

  • A valid presigned link wrapped in **, | or followed by 。 is left alone, and our own Slack app-create link in bold is left alone; a payload glued after a presigned link is still caught. The decoy, presigned and Slack tests fail on the previous head fe59b1a5e9ae.

test/test_optimizer.py: a draft URL with an uppercase scheme that the model echoes returns the draft; a draft URL glued inside an exempt host's link returns the draft (fails on the previous head).

test/test_dynamic_dashboard_cards.py::test_markup_cannot_split_a_suspicious_url_across_tags: a URL split by a bare tag is now caught by the raw scan, so the test pins that, and its markup-split case uses a tag with a quoted attribute, which still ends the URL for the raw scan, to keep testing the second scan of the shown text.

Proof: PROVEN test/test_dynamic_dashboard_cards.py::test_markup_cannot_split_a_suspicious_url_across_tags, test/test_exfil_scan_span.py::test_payload_after_a_stop_byte_in_the_path_is_caught[)], test/test_exfil_scan_span.py::test_payload_after_a_stop_byte_in_the_path_is_caught['], test/test_exfil_scan_span.py::test_payload_after_a_stop_byte_in_the_path_is_caught[>], test/test_exfil_scan_span.py::test_path_credential_after_a_paren_is_caught (head 7d08e7bae65b)

The single-quote and closing-paren tests also fail on main.

Related suites: the exfil, redaction-parity, blocked-link, facade, optimizer, dashboard-card, Slack-manifest and chat-runner test files (459 + 189 + 152 passed). Black, isort, flake8, mypy, Semgrep on the diff and the file line cap pass.

Manual verification

N/A: the public scanner and redactor are exercised directly by the tests above.

Related Issues

Part of #7611 (the browser-side mirror is still open there).

Supersedes #7666.

Sibling sweep

Swept: git grep -n "_URL_RE" -- src website/src
Sibling: website/src/utils/sanitize.ts -- tracked in #7611

The backend readers of _URL_RE: kiro_crew.security.exfil and kiro_crew.dashboard.handlers.optimizer are changed here to read the scan spans, and kiro_crew.dashboard.source_providers.adf pre-encodes the stop bytes before the same redactor, which still works.

Pattern harvest

Rule candidate: review-prompt
Pattern: a scanner reused a renderer's link boundaries, so bytes legal inside a URL ended inspection before a payload.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Sensitive Change Review

Sensitive files changed:

  • src/kiro_crew/security/exfil.py

  • src/kiro_crew/security/exfil_scan_span.py

  • src/kiro_crew/security/exfil_url_rules.py

  • @bolichen97 posted a complete reasoning template for 1d76e8d.

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 31, 2026
@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 1d76e8d383756e476a948e3040e8f198a66b9e6b touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 1d76e8d383756e476a948e3040e8f198a66b9e6b — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the bypass is real, and splitting "where a renderer ends a link" from "what the scanner reads" fixes the root cause. Nothing is taken away, and the only cost is over-redaction, which fails closed.

Notes

  • The backend scan span now differs from website/src/utils/sanitize.ts, which still stops at ' and >. Any surface the browser renders without the backend redacting it first still has the ' bypass. The fix is tracked under Exfiltration URL scan stops at a ')' in the path, leaving the query unscanned #7611 as the stated not-goal, and it should land before any client-only render path relies on parity.
  • The test change in test_markup_cannot_split_a_suspicious_url_across_tags is not a weakened pin. The old _redact(markup) == markup line only set up the test's second scan. The new quoted-attribute variant keeps that scan under test, and the bare-tag case is now caught by the raw scan.

[DESIGN-REVIEWED] 1d76e8d

Description read

[DESCRIPTION-READ] 5493804656503940fa54e5ddef71d4015603243116a6bbdce174f53b55f4bcc7

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 1d76e8d383756e476a948e3040e8f198a66b9e6b — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks. The fix is at mechanism level: one regex was doing two jobs and is now split. The tests show the defect on main. The old per-surface workaround in adf.py stays behind as a second spelling.

Notes

  • Item 10 — duplicate of dashboard/source_providers/adf.py (_URL_SCAN_ESCAPES, the _md_redact_untruncated pre-encoding). The description admits this: "It still works and is now redundant." Grep for _URL_RE consumers: 1 surface-specific workaround is left beside the general scan. Subtraction: delete the pre-encoding in adf.py (deferred to a follow-up is fine, but it should not live on forever).
  • Item 1 — point patch: 1 sibling with the same root cause is unfixed, website/src/utils/sanitize.ts (stops at ' and >). The PR declares it under ## Not a goal and it stays open under Exfiltration URL scan stops at a ')' in the path, leaving the query unscanned #7611. Accepted and deferred.

What this change ships

Inventory (10 items) — 8 justified

Intent: stop an exfil payload from getting past the URL redactor just because a ), ' or > sits in the URL or the scheme is uppercase. This is a FIX.

  1. A payload after ), ' or > in a URL is now flagged and redacted — justified
  2. URLs with an uppercase or mixed-case scheme are now scanned — justified
  3. A URL glued inside an exempt host's link is judged under its own host — justified
  4. Presigned S3 and Slack app links followed by markdown punctuation stay clean — justified
  5. CJK prose glued after a link is not read as a payload — justified
  6. The optimizer's draft-URL check uses the redactor's spans — justified
  7. Allowed-host link restore reads links with the same spans — justified
  8. A clean URL glued to a payload-shaped run is now over-redacted (declared cost) — justified
  9. New owner module exfil_scan_span, registered in the facade and in the display battery — justified
  10. The adf.py pre-encoding is left in place, now redundant — duplicate of the new scan span

[FIRST-PRINCIPLES-REVIEWED] 1d76e8d

Description read

[DESCRIPTION-READ] 4d704f9cc3a62a23ba85255750ece9f75dbc3c6601f540ab6893320b404f8350

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 1d76e8d383756e476a948e3040e8f198a66b9e6b and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 1d76e8d

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 1d76e8d383756e476a948e3040e8f198a66b9e6b false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 1d76e8d383756e476a948e3040e8f198a66b9e6b — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 1d76e8d

Verdict parsed from the review's SHA-scoped output markers for commit 1d76e8d383756e476a948e3040e8f198a66b9e6b.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 1d76e8d383756e476a948e3040e8f198a66b9e6b false-positive: <reason, 20 to 500 characters>

@bolichen97
bolichen97 enabled auto-merge (squash) October 31, 2026 14:09
@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Security Scope Review -- ✅ PASS

Which legitimate operations does 1d76e8d383756e476a948e3040e8f198a66b9e6b newly refuse? Updated in place on each push. A script-confirmed regression blocks; the model proposes candidates and never decides a refusal.

Verdict basis: adjudicated, zero confirmed regressions.

Security scope review -- adjudicated candidates

  • macos-latest (platform posix): 9 classified, 0 newly refused, 9 still allowed.
  • ubuntu-latest (platform posix): 9 classified, 0 newly refused, 9 still allowed.
  • windows-latest (platform windows): 5 classified, 0 newly refused, 5 still allowed.

No candidate flipped from allowed to refused.

Scope-Verdict: PASS

Reason: The stricter rule is in the exfil URL redactor and scanner (scan_exfiltration_urls, redact_exfiltration_urls_with_records, restore_allowed_links) and the optimizer's draft-URL check. None of the three shell deny tiers calls these functions: exfil_bash.audit_bash_exfiltration, deny_gate/denied_rules and paths.is_sensitive_bash_command never reference them, and _URL_RE itself is unchanged. So no confirmed regression is possible on the shell composite. The real tightening is in output redaction, which the classifier cannot judge, so the gap is named below and does not block.

TIGHTENING: The URL span the exfil scan judges and replaces now reaches past ), ' and >, matches the scheme without regard to case, and checks each URL embedded inside an exempt host's match under that URL's own host. Hunk: +_URL_SCHEME = r"(?ai:https?)://" / +_URL_SCAN_RE = re.compile(... r"([/?][^\s)\"'>]*(?:[)'>](?:(?!" + _URL_SCHEME + ... and - for match in _URL_RE.finditer(text): → + for match in _URL_SCAN_RE.finditer(text): flagged = _flagged_url_span(match, exempt_hosts, rules). The one threat it refuses: an exfiltration payload placed after a stop byte (https://host/a)b?data=<blob>), behind an uppercase HTTPS:// scheme, or glued inside an exempt host's link, slipping past the redactor.

ADJUDICATED: 14 candidates, 0 confirmed newly-refused

PLATFORM COVERAGE: posix and windows. The diff makes no assumption about paths, executables or environment variables. Its platform-sensitive inputs are quoting bytes (' and " end or extend the span differently) and the scheme's letter case. The posix rows probe single-quoted URLs, which put ' right after the URL. The windows rows use curl.exe, start "", %VAR% and double-quote-only cmd quoting, where a " ends the span. The any spelling is not used, because every row names an executable.

UNADJUDICATED:

  • Guard: exfil_scan_span._walk_flagged_span / _walk_span_urls with exfil_url_rules._URL_SCAN_RE, _url_scan_candidates and _embedded_url_matches.

  • Call paths: model or agent text → redact_exfiltration_urls (for example acp/_dispatch_parts/surfaced_text.py:241, acp/client_turn.py:1315, crew_log/emit_text.py:82, cli_chat.py:178, context_assembly/session_start.py:474) → redact_exfiltration_urls_with_records → _flagged_url_span. Also dashboard handle_optimize → _scan_span_urls(prompt) for draft-URL recognition, and restore_allowed_links re-matching placed links against _url_scan_candidates.

  • Inputs that would need a verdict here. In each case the wide span now reaches past a )/'/> and the text after it becomes part of what is judged:

    • A clean doc link followed by a long ASCII identifier glued right after the ), such as a markdown link immediately followed by a long hex commit SHA or base64-looking build ID with no space.
    • A single-quoted URL in a Python or JSON snippet followed by , and a long token-like literal.
    • An HTML <a href='...'> whose label text is a long base64-looking string.
    • A clean URL followed by ) and then a second long URL on a non-exempt host, which is now judged as its own match.
    • The OAuth consent URL whose redirect_uri carries an embedded https://localhost callback, now judged under its own host.
    • A presigned S3 link inside a table cell followed by a long tail of non-punctuation text.

    Narrowing I can argue but cannot confirm: the tail runs on to whitespace, a double quote or the next scheme, so a long glued ASCII run after ) (such as a SHA) can now push a clean link over the 200-character query or 40-character blob threshold. When that happens the whole wide span is replaced, and ordinary agent output loses the link and the identifier that follows it.

[SCOPE-REVIEWED] 1d76e8d

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 31, 2026
@bolichen97
bolichen97 force-pushed the fix/exfil-scan-bypass-7666 branch from 2a92480 to dabc6e9 Compare October 31, 2026 14:24
@bolichen97

Copy link
Copy Markdown
Contributor Author

Dispositions for the review of 2a92480. Both are fixed in dabc6e9.

GPT 6.1: Unicode space no longer ends the URL (exfil_url_rules.py:78): fixed.

re.ASCII applied to the whole pattern, so \s stopped matching a no-break space and a second URL merged into an exempt host's match.
The case fold is now scoped to the scheme only, (?ai:https?)://, in both the match and the glued-URL lookahead. \s is Unicode again, as in _URL_RE.
Test: test_unicode_space_ends_the_url with no-break, ideographic and em spaces under an exempt first host.

Opus 5.5: glued CJK prose read as a long query (exfil_url_rules.py:78): fixed.

In the bytes the wide span adds past the stop byte, each run of non-ASCII text is now judged as one space. CJK prose glued to a link no longer trips the length check, and an ASCII payload glued between CJK words is still caught.
Tests: test_cjk_prose_glued_after_a_link_is_kept, test_payload_glued_between_cjk_prose_is_still_caught.
The minified-markup case (src='...?v=1'integrity='sha384-...') is the same bytes as the single-quote bypass this PR closes ('https://host/a?x=1',<blob>); no syntactic rule separates them, so that over-redaction is the stated fail-closed cost.

Opus 5.5 note: optimizer draft check: fixed.

dashboard/handlers/optimizer.py now reads draft URLs with the same scan spans the redactor removes. Test: test_redacted_uppercase_scheme_draft_url_returns_the_draft.

@bolichen97

Copy link
Copy Markdown
Contributor Author

Sensitive change review

  • Head commit: dabc6e9
  • What rule changed: the exfiltration URL scanner and redactor read each URL past ), ' and > (up to whitespace, a double quote or the next glued URL) and match an uppercase scheme. The old narrow span is judged first and replaced when flagged; otherwise the whole match is judged and replaced when flagged.
  • Before -> after (what was allowed/blocked, now): before, a payload after ), ' or > in a URL, or behind HTTPS://, scanned clean and passed through unredacted. After, it is flagged and redacted. Every URL flagged before is still replaced over the same span.
  • Worst case if wrong (breaks commands / leaks secret / locks user out): over-redaction of a clean URL glued, with no whitespace, to a payload-shaped ASCII run, or a missed payload in a shape the tests do not cover (which is the state on main). No command or tool is blocked by this change; it only rewrites text.
  • Evidence checked (which sandbox result proves it): N/A, this is a text redactor, not a sandbox rule. The new tests in test/test_exfil_scan_span.py fail on main and pass on this head (prove.py reports PROVEN).
  • No regression:
    • Tools still work (which tools/commands were run, result): N/A, no tool gate or command rule is touched; the scanner runs on model and provider text only.
    • Backward compatible (old config / old data / old callers still work, how checked): _URL_RE is unchanged; saved blocked-link records still restore (test_redaction_allow.py and the new restore test pass).
    • Existing tests (which suites passed on this head): every test file that calls the exfil scanner, redactor, blocked-link records, restore path or optimizer, 10,152 passed, 4 skipped.
  • How to undo: revert this commit.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 31, 2026
@bolichen97

Copy link
Copy Markdown
Contributor Author

Disposition for the GPT 6.1 finding on dabc6e9 (exfil_url_rules.py:79, a glued link inheriting an exempt host's exemption): fixed in fe59b1a.

The narrow span keeps running past a second http(s)://, because a redirect carries one in its query, so the inner link was judged under the outer host. The class predates this PR for a lowercase outer scheme; the scheme fold made it reachable with HTTPS:// too.
Now, when a match is clean, _embedded_url_match finds the first URL starting inside it on a host the exempt set does not hold, and that URL is judged under its own host and replaced as its own span. The outer host's exemption covers only the outer span.
One embedded URL is enough: it spans every later one and its query holds theirs. Exempt starts are skipped with a scheme-and-host pattern, so the pass stays linear (32,000 glued exempt URLs redact in about 0.3 s).
The restore and display paths both go through redact_exfiltration_urls, so the restored outer link is re-checked with the same rule.
Tests: test_glued_url_inside_an_exempt_link_is_judged_under_its_own_host for HTTPS, https and HtTpS outer schemes, test_embedded_url_past_a_chain_of_exempt_ones_is_caught, and test_clean_url_inside_an_exempt_link_is_left_alone. The first four fail on dabc6e9.

@bolichen97

Copy link
Copy Markdown
Contributor Author

Sensitive change review

  • Head commit: fe59b1a
  • What rule changed: the exfiltration URL scanner and redactor read each URL past ), ' and > (up to whitespace, a double quote or the next glued URL) and match an uppercase scheme. A URL embedded inside a clean match is judged under its own host, so an exempt host covers only its own span. The old narrow span is judged first and replaced when flagged; otherwise the whole match is judged and replaced when flagged.
  • Before -> after (what was allowed/blocked, now): before, a payload after ), ' or > in a URL, or behind HTTPS://, scanned clean and passed through unredacted. After, it is flagged and redacted. Every URL flagged before is still replaced over the same span.
  • Worst case if wrong (breaks commands / leaks secret / locks user out): over-redaction of a clean URL glued, with no whitespace, to a payload-shaped ASCII run, or a missed payload in a shape the tests do not cover (which is the state on main). No command or tool is blocked by this change; it only rewrites text.
  • Evidence checked (which sandbox result proves it): N/A, this is a text redactor, not a sandbox rule. The new tests in test/test_exfil_scan_span.py fail on main and pass on this head (prove.py reports PROVEN).
  • No regression:
    • Tools still work (which tools/commands were run, result): N/A, no tool gate or command rule is touched; the scanner runs on model and provider text only.
    • Backward compatible (old config / old data / old callers still work, how checked): _URL_RE is unchanged; saved blocked-link records still restore (test_redaction_allow.py and the new restore test pass).
    • Existing tests (which suites passed on this head): every test file that calls the exfil scanner, redactor, blocked-link records, restore path or optimizer, 10,175 passed, 4 skipped.
  • How to undo: revert this commit.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 31, 2026
@bolichen97
bolichen97 force-pushed the fix/exfil-scan-bypass-7666 branch from fe59b1a to 45615f5 Compare October 31, 2026 19:52
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 31, 2026
@bolichen97

bolichen97 commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor Author
  • span=df1a82b5c3ff: a clean embedded match that ends early stopped the embedded walk

fixed in 45615f5: _embedded_url_matches now yields every embedded URL on a non-exempt host, resuming where the last one ends, so a bare-host decoy no longer ends the walk. No byte is searched twice, so it stays linear.
Test: test_embedded_url_after_a_bare_host_decoy_is_caught (fails on fe59b1a).

@bolichen97

bolichen97 commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor Author
  • span=f304b4e59d80: the optimizer's draft URL set missed embedded spans

fixed in 45615f5: the draft check now reads URLs through _scan_span_urls, the redactor's own walk (scan match, then embedded URLs, each narrow and whole), so no span logic is copied into the handler.
Test: TestOptimizerDraftUrls::test_draft_url_embedded_in_an_exempt_link_returns_the_draft (fails on fe59b1a).

@bolichen97

bolichen97 commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor Author
  • span=bc940be32bb4: markdown after a presigned link undid its exemption

fixed in 45615f5: when the narrow span passed as a valid S3 presigned query or our own Slack app-create link, _tail_payload_after_exempt_span judges the added bytes as a query of their own (the Slack alias joins them). Every other URL is still judged as narrow plus tail, so a payload split across the stop byte stays whole.
Tests: test_presigned_link_in_markdown_is_left_alone (**, |, 。), test_slack_app_link_in_markdown_bold_is_left_alone, and test_payload_glued_after_a_presigned_link_is_still_caught.
The single-quote plus 200-character case is the over-redaction cost this PR states under Backwards compatibility: the shape matches the attack byte for byte, and the redactor fails closed.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 31, 2026
@bolichen97
bolichen97 force-pushed the fix/exfil-scan-bypass-7666 branch 2 times, most recently from fceda74 to d52ac1d Compare October 31, 2026 23:53
@bolichen97

Copy link
Copy Markdown
Contributor Author
  • exfil.py over the 2,000-line cap (src/kiro_crew/security/exfil.py:467)

fixed in d52ac1d: the span walk moved to a new owner module security/exfil_scan_span.py, listed in the facade's _OWNERS; exfil.py keeps two thin adapters and is 1,992 lines. check_file_line_cap.py passes, and the facade guards (test_exfil_facade.py, test_facade_mirror_guards.py) pass.

The exfil URL scanner stopped a URL at ), ' or >. Those bytes are
legal in a URL, so a payload placed after one escaped every check:
https://host/a)b?data=<blob>, 'https://host/a?x=1',<blob> and
(https://host/a?x=1)<blob> all scanned clean. An uppercase HTTPS://
scheme was not matched at all.

The scanner and redactor now iterate a scan pattern that is case
insensitive and runs past those bytes to whitespace, a double quote
or the next glued URL. Each match is judged on the old narrow span
first, so a flagged link is replaced as before and the markup after
it survives; only when the narrow span is clean is the whole match
judged and, if flagged, replaced. A tail of punctuation only is not
judged, and in the bytes the whole match adds, each run of non-ASCII
text is judged as one space, so prose glued to a link is not a
payload. The case fold is scoped to the scheme, so any Unicode space
still ends a URL.

When the narrow span passed as a valid S3 presigned query or our own
Slack app-create link, the added bytes are judged as a query of their
own (the Slack alias joins them), so markdown such as ** or | after
the link does not break the exact parameter shape and undo the
exemption.

A match does not stop at a second http(s):// in its narrow span, since
a redirect carries one in its query. So when a match is clean, each
URL starting inside it on a host the exempt set does not hold is
judged under its own host and, when flagged, replaced as its own
span. The walk resumes where each embedded match ends, so a bare-host
decoy does not end it, and stays linear. The allowed-link restore and
the prompt optimizer's draft check read back with the same spans.

Supersedes #7666.

Co-authored-by: patrigao <112447010+patrigao@users.noreply.github.com>
@bolichen97
bolichen97 force-pushed the fix/exfil-scan-bypass-7666 branch from d52ac1d to 1d76e8d Compare November 1, 2026 00:51
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Nov 1, 2026
@bolichen97

Copy link
Copy Markdown
Contributor Author

Sensitive change review

  • Head commit: 1d76e8d
  • What rule changed: the exfiltration URL scanner and redactor read each URL past ), ' and > (up to whitespace, a double quote or the next glued URL) and match an uppercase scheme. The old narrow span is judged first and replaced when flagged; otherwise the whole match is judged. A URL embedded inside a clean match on a non-exempt host is judged under its own host, so an exempt host covers only its own span. A valid S3 presigned or Slack app-create link keeps its exemption when markdown such as **, | or 。 follows it.
  • Before -> after (what was allowed/blocked, now): before, a payload after ), ' or > in a URL, behind HTTPS://, or in a link glued inside an exempt host's URL scanned clean and passed through unredacted. After, it is flagged and redacted. Every URL flagged before is still replaced over the same span.
  • Worst case if wrong (breaks commands / leaks secret / locks user out): over-redaction of a clean URL glued, with no whitespace, to a payload-shaped ASCII run, or a missed payload in a shape the tests do not cover (which is the state on main). No command or tool is blocked by this change; it only rewrites text.
  • Evidence checked (which sandbox result proves it): N/A, this is a text redactor, not a sandbox rule. The tests in test/test_exfil_scan_span.py cover each bypass and each benign wrapper case.
  • No regression:
    • Tools still work (which tools/commands were run, result): N/A, no tool gate or command rule is touched; the scanner runs on model and provider text only.
    • Backward compatible (old config / old data / old callers still work, how checked): _URL_RE is unchanged; saved blocked-link records still restore (test_redaction_allow.py passes).
    • Existing tests (which suites passed on this head): test_exfil_scan_span.py, test_optimizer.py, test_security_exfil_urls.py and test_redaction_allow.py, 242 passed locally; the CI backend test shards are green on this head.
  • How to undo: revert this commit.

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved on behalf of the watch holder (bolichen). Lanes pass on 1d76e8d, no conflict, no open change request.

@KiroCrewCaptain
KiroCrewCaptain merged commit 2e2d275 into main Nov 1, 2026
102 of 111 checks passed
@KiroCrewCaptain
KiroCrewCaptain deleted the fix/exfil-scan-bypass-7666 branch November 1, 2026 02:15
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants