Repository navigation
fix(security): scan exfil URLs past ) ' > and uppercase schemes - #26741
Conversation
Sensitive Change ReviewSensitive files changed:
|
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: the bypass is real, and splitting "where a renderer ends a link" from "what the scanner reads" fixes the root cause. Nothing is taken away, and the only cost is over-redaction, which fails closed. Notes
[DESIGN-REVIEWED] 1d76e8d Description read[DESCRIPTION-READ] 5493804656503940fa54e5ddef71d4015603243116a6bbdce174f53b55f4bcc7 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks. The fix is at mechanism level: one regex was doing two jobs and is now split. The tests show the defect on main. The old per-surface workaround in Notes
What this change shipsInventory (10 items) — 8 justifiedIntent: stop an exfil payload from getting past the URL redactor just because a
[FIRST-PRINCIPLES-REVIEWED] 1d76e8d Description read[DESCRIPTION-READ] 4d704f9cc3a62a23ba85255750ece9f75dbc3c6601f540ab6893320b404f8350 That sha256 names the PR description this verdict read. Recompute it with |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
Security Scope Review -- ✅ PASSWhich legitimate operations does Verdict basis: adjudicated, zero confirmed regressions. Security scope review -- adjudicated candidates
No candidate flipped from allowed to refused. Scope-Verdict: PASS Reason: The stricter rule is in the exfil URL redactor and scanner ( TIGHTENING: The URL span the exfil scan judges and replaces now reaches past ADJUDICATED: 14 candidates, 0 confirmed newly-refused PLATFORM COVERAGE: posix and windows. The diff makes no assumption about paths, executables or environment variables. Its platform-sensitive inputs are quoting bytes ( UNADJUDICATED:
[SCOPE-REVIEWED] 1d76e8d |
2a92480 to
dabc6e9
Compare
|
Dispositions for the review of 2a92480. Both are fixed in dabc6e9. GPT 6.1: Unicode space no longer ends the URL (exfil_url_rules.py:78): fixed.
Opus 5.5: glued CJK prose read as a long query (exfil_url_rules.py:78): fixed.
Opus 5.5 note: optimizer draft check: fixed.
|
Sensitive change review
|
|
Disposition for the GPT 6.1 finding on dabc6e9 (exfil_url_rules.py:79, a glued link inheriting an exempt host's exemption): fixed in fe59b1a.
|
Sensitive change review
|
fe59b1a to
45615f5
Compare
|
|
|
fceda74 to
d52ac1d
Compare
|
The exfil URL scanner stopped a URL at ), ' or >. Those bytes are legal in a URL, so a payload placed after one escaped every check: https://host/a)b?data=<blob>, 'https://host/a?x=1',<blob> and (https://host/a?x=1)<blob> all scanned clean. An uppercase HTTPS:// scheme was not matched at all. The scanner and redactor now iterate a scan pattern that is case insensitive and runs past those bytes to whitespace, a double quote or the next glued URL. Each match is judged on the old narrow span first, so a flagged link is replaced as before and the markup after it survives; only when the narrow span is clean is the whole match judged and, if flagged, replaced. A tail of punctuation only is not judged, and in the bytes the whole match adds, each run of non-ASCII text is judged as one space, so prose glued to a link is not a payload. The case fold is scoped to the scheme, so any Unicode space still ends a URL. When the narrow span passed as a valid S3 presigned query or our own Slack app-create link, the added bytes are judged as a query of their own (the Slack alias joins them), so markdown such as ** or | after the link does not break the exact parameter shape and undo the exemption. A match does not stop at a second http(s):// in its narrow span, since a redirect carries one in its query. So when a match is clean, each URL starting inside it on a host the exempt set does not hold is judged under its own host and, when flagged, replaced as its own span. The walk resumes where each embedded match ends, so a bare-host decoy does not end it, and stays linear. The allowed-link restore and the prompt optimizer's draft check read back with the same spans. Supersedes #7666. Co-authored-by: patrigao <112447010+patrigao@users.noreply.github.com>
d52ac1d to
1d76e8d
Compare
Sensitive change review
|
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Approved on behalf of the watch holder (bolichen). Lanes pass on 1d76e8d, no conflict, no open change request.
Problem / Motivation
Goal: The exfiltration URL scanner inspects a URL's whole path and query, so a payload placed after
),'or>, or behind an uppercase scheme, is caught and redacted.The scanner ended every URL at the first
),'or>. Those bytes are legal inside a URL, so anything after one was never checked. On main, each of these scans clean and passes throughredact_exfiltration_urlsunchanged (a 56-character base64 blob stands in for the payload):https://evil.example.com/a)b?data=<blob>https://evil.example.com/a'b?data=<blob>https://evil.example.com/a>b?data=<blob>https://evil.example.com/a)<AWS key id>HTTPS://evil.example.com/a?data=<blob>(the scheme was matched case-sensitively)'https://evil.example.com/a?x=1',<blob>(https://evil.example.com/a?x=1)<blob>Why it matters
Model and provider text passes through this redactor before a user sees it. One punctuation byte was enough to keep a credential-like query out of every check, so the link survived with its payload.
Not a goal
website/src/utils/sanitize.ts, which still stops at'and>.https://host/r?next=https://...). That shape is judged as one URL, as on main.dashboard/source_providers/adf.py. It still works and is now redundant.What changed (motivation → approach → change)
Root cause: the one regex served two jobs. It found where a renderer ends a link, and it decided what the scanner reads. The first job needs the stop at
),'and>; the second must not stop there.The scanner and redactor now use a second pattern,
_URL_SCAN_RE, with the same groups. Its scheme is matched case-insensitively (ASCII only, and only the scheme, so any Unicode space still ends a URL). After the old stop byte it reads on to whitespace, a double quote, or the nexthttp(s)://<host>. Stopping before a glued second URL keeps that URL judged under its own host, so it never rides the first host's exemptions.Each match yields two spans: the old narrow one, then the whole match. The narrow span is judged first. If it is flagged, only it is replaced, so markup after a flagged link survives exactly as on main. If it is clean, the whole match is judged, and replaced when flagged. A tail of punctuation only is not judged, so
[x](<presigned S3 URL>)stays clean. In the added tail, each run of non-ASCII text is judged as one space, so Chinese or Japanese prose glued to a link is not read as a long query.restore_allowed_linksreads URLs with the same spans, and the prompt optimizer's draft check (dashboard/handlers/optimizer.py) reads them with the same walk, embedded URLs included._URL_REitself is unchanged.A valid S3 presigned link and our own Slack app-create link pass by their exact parameter shape. When the narrow span passed that way, the added bytes are judged as a query of their own (the Slack alias joins them), so
**[f](<presigned>)**, a table cell or a。after the link stays clean.The narrow span does not stop at a second
http(s)://, since a redirect carries one in its query. So when a match is clean, each URL starting inside it on a host the exempt set does not hold is judged under its own host, and the first flagged one is replaced as its own span. An exempt host's relaxed checks then cover only its own span: inhttps://exempt/r?x=1`[open](https://evil/?d=<blob>)only the second link is replaced. Each search resumes where the last embedded match ends, so a bare-host decoy does not end the walk, and no byte is searched twice.This replaces the span model of #7666, which could not be rebased after main moved these rules into
exfil_url_rules.pyand rewrote the redact loop. It follows the two-span rule suggested in review there, which also removes the per-apostrophe tail scan.The span walk lives in a new owner module,
security/exfil_scan_span.py, listed in the exfil facade's owners (test/test_exfil_facade.py) and in the display battery's rule sources (dashboard/display_battery.py);exfil.pykeeps thin adapters so it stays under the line cap. The spec paragraph indocs/system-specs/modules/security.mddescribes the scan span.Backwards compatibility
Compatible: every URL main flags is replaced over the same span as before. The only new output is a redaction where main returned the text unchanged. The known cost is over-redaction of a clean URL glued, with no whitespace, to a following payload-shaped run, such as a compact field
'url':'https://host/p?a=1','sha':'<40 hex>', or Chinese or Japanese prose (which has no spaces) carrying a 40-hex SHA right after a link's). A tail is not cut at the first non-ASCII byte, because an autolinker reads on through such text to the next space, so the blob would still be fetched. That shape is byte-identical to the attack, and the redactor fails closed.Removes nothing: no name, field or behaviour is removed;
_URL_REkeeps its pattern.Tests
test/test_exfil_scan_span.py(new):Each bypass above is flagged and its payload removed. These fail on main.
A flagged link in
[x](...),<a href='...'>label</a>and<...>keeps the text after it, with the exact output pinned.Clean links in parentheses, quotes, a balanced wiki path and an HTML attribute are left untouched.
A URL glued after
)to an exempt host's link is judged under its own host.A second URL after a no-break, ideographic or em space is judged under its own host.
Chinese prose glued after a link's
)is kept; a blob glued between Chinese words is still caught.An allowed host's link before
)is restored byrestore_allowed_links.A link glued inside an exempt host's URL is judged under its own host, with the outer scheme written
HTTPS,httpsandHtTpS; one past a chain of exempt URLs is caught too; a clean one is left alone.A payload link after a bare-host decoy inside an exempt link is caught (
test_embedded_url_after_a_bare_host_decoy_is_caught).A valid presigned link wrapped in
**,|or followed by。is left alone, and our own Slack app-create link in bold is left alone; a payload glued after a presigned link is still caught. The decoy, presigned and Slack tests fail on the previous headfe59b1a5e9ae.test/test_optimizer.py: a draft URL with an uppercase scheme that the model echoes returns the draft; a draft URL glued inside an exempt host's link returns the draft (fails on the previous head).test/test_dynamic_dashboard_cards.py::test_markup_cannot_split_a_suspicious_url_across_tags: a URL split by a bare tag is now caught by the raw scan, so the test pins that, and its markup-split case uses a tag with a quoted attribute, which still ends the URL for the raw scan, to keep testing the second scan of the shown text.Proof: PROVEN test/test_dynamic_dashboard_cards.py::test_markup_cannot_split_a_suspicious_url_across_tags, test/test_exfil_scan_span.py::test_payload_after_a_stop_byte_in_the_path_is_caught[)], test/test_exfil_scan_span.py::test_payload_after_a_stop_byte_in_the_path_is_caught['], test/test_exfil_scan_span.py::test_payload_after_a_stop_byte_in_the_path_is_caught[>], test/test_exfil_scan_span.py::test_path_credential_after_a_paren_is_caught (head 7d08e7bae65b)
The single-quote and closing-paren tests also fail on main.
Related suites: the exfil, redaction-parity, blocked-link, facade, optimizer, dashboard-card, Slack-manifest and chat-runner test files (459 + 189 + 152 passed). Black, isort, flake8, mypy, Semgrep on the diff and the file line cap pass.
Manual verification
N/A: the public scanner and redactor are exercised directly by the tests above.
Related Issues
Part of #7611 (the browser-side mirror is still open there).
Supersedes #7666.
Sibling sweep
Swept: git grep -n "_URL_RE" -- src website/src
Sibling: website/src/utils/sanitize.ts -- tracked in #7611
The backend readers of
_URL_RE:kiro_crew.security.exfilandkiro_crew.dashboard.handlers.optimizerare changed here to read the scan spans, andkiro_crew.dashboard.source_providers.adfpre-encodes the stop bytes before the same redactor, which still works.Pattern harvest
Rule candidate: review-prompt
Pattern: a scanner reused a renderer's link boundaries, so bytes legal inside a URL ended inspection before a payload.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)