Skip to content

fix(messaging): keep a link's parenthesised URL whole on every channel - #14314

Merged
bolichen97 merged 1 commit into
kirodotdev:mainfrom
unstablebrainiac:fix/markdown-link-parenthesised-url
Sep 29, 2026
Merged

bolichen97 merged 1 commit into
kirodotdev:mainfrom
unstablebrainiac:fix/markdown-link-parenthesised-url

Conversation

@unstablebrainiac

@unstablebrainiac unstablebrainiac commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A Markdown link whose URL holds a balanced pair of parentheses reaches Slack, Telegram, WhatsApp and iMessage readers whole, and the display-safety screen still collapses every link those channels show, so a credential split around such a link is scanned as the reader sees it.

A link to a URL that contains a pair of parentheses arrives broken on chat channels. An answer citing [Python](https://en.wikipedia.org/wiki/Python_(programming_language)) reaches Slack as a link to https://en.wikipedia.org/wiki/Python_(programming_language, a page that does not exist, followed by a stray ). Telegram puts the same cut URL in its <a href>, and iMessage shows the raw markdown instead of the link.

Channel What the reader got What the reader gets now
Slack <https://en.wikipedia.org/wiki/Python_(programming_language|Python>) <https://en.wikipedia.org/wiki/Python_(programming_language)|Python>
Telegram <a href="…/Python_(programming_language">Python</a>) <a href="…/Python_(programming_language)">Python</a>
iMessage [Python](https://…/Python_(programming_language)) Python (https://…/Python_(programming_language))

The same happens to every URL with a balanced (...): Wikipedia disambiguation pages, MSDN's ms123401(v=vs.85) pages, many docs anchors.

Why it matters

Agents cite Wikipedia and vendor docs all the time, and Slack and Telegram are the main chat channels. The reader clicks the link, lands on "page does not exist" or the wrong page, and nothing on screen says why.

Not a goal

  • Changing link handling outside the chat renderers: session previews, voice replies, artifact snippets and deploy pages, the web app's own link regexes.
  • Nesting deeper than one level.
  • Backslash-escaped parentheses.

What changed (motivation → approach → change)

Links whose URL contains a (...) pair now reach Slack, Telegram, WhatsApp and iMessage whole, so they open the right page. Each renderer's link regex stopped the URL at the first ) ([^)]+), and iMessage's refused parentheses outright; they now accept a balanced pair, as CommonMark does.

File Change
kiro_crew/constants.py new md_link_destination(): one URL character, or a (...) group of them, one level deep; a backslash and the character after it are one token, so an escaped [ or backslash stays whole and a backslash before a parenthesis is refused; a [ that opens a nested [label]( is refused
slack/format.py link URL built from it (any character but a parenthesis, `
telegram/renderer.py, whatsapp/renderer.py same, for an http(s) URL with no whitespace; Telegram's plain-text fallback passes are imported from the screen, one definition for both
imessage/plaintext.py same, for links and images, no whitespace
messaging/display_safety.py the screen's link pattern built from it, plus the readings listed below; text it rewrites is collapsed again until it settles
messaging/split.py a cut, a rotation boundary and a cut's repair are graded under the same readings as one message; the repair always runs with the channel's own cutter
messaging/outbound_files.py docstring only: names the helper as this walker's grammar in regex form
docs/system-specs/modules/messaging.md the shared link grammar, the plain reading and the settling bound

An unbalanced (, a second level of nesting, or a backslash-escaped \) is left as written instead of being linked to a cut URL. A [label]( inside a URL opens a link of its own, so that inner link renders on its own and the outer text stays as written. A ]( that opens no link, as in https://x/a](b), stays part of the URL.

The helper spells the grammar messaging/outbound_files.py walks to find local image files, limited to one level of nesting. Wherever the helper matches, it ends at the same ) as that walker, and a test pins this. It stays a regex because four renderers compose it into one re.sub pass, and the display-safety screen must stay linear-time.

The display-safety screen had to change with the renderers. It collapses each link to its label so the credential scan reads what the reader will see, and a wider link match drops more URL text from that scan. So besides the literal text and the renderers' own grammar, it always scans four readings that model no single client:

  • Telegram's plain-text fallback, applying that fallback's own fence, inline-code, heading, **, __ and link passes in its order, lone delimiters kept;
  • each link closed at its balanced ) at any depth, as a CommonMark reader such as Discord closes it, with a backslash escape read as one character and an <...> destination taken whole;
  • each link closed at the first ), as a lazy reader closes it;
  • no link parsing at all, so a key split by markup inside a URL is still scanned.

Each converted Slack block, and the joined message, is also scanned as the Slack client displays it and settled until that view shows no key, and so is every part a long message is cut into; a block dirty only as written keeps its links and emphasis.

A key found in any reading is answered with the collapsed, redacted form, which carries no URL. That answer is itself scanned under every reading, because one message can carry two keys that different readings join. Whatever a reading still shows is redacted and settled, for at most four passes, then the strip below. A cut splitting a long message, and each boundary of a message rotated into several, is refused when any reading joins a key across it.

Collapsing can itself close a link ([AKIA]*(u) reads as [AKIA](u) once the * is gone), so rewritten text is collapsed and scanned again until a pass changes nothing, for at most four passes; past that it strips markup outside redactor tags and collapses and scans once more. Each renderer's link label excludes [, ] and line breaks, as the screen's does.

Backwards compatibility

Compatible: no input is refused. A link renders exactly as before unless its URL holds parentheses: a balanced pair is kept whole, and an unbalanced (, deeper nesting, an escaped \) or a nested [label]( is left as written instead of being linked to a cut URL. A message hiding a key that any reading shows joined (Telegram's plain-text fallback, a balanced or first-) link close, or the URL text once markup is dropped) is redacted and sent in its collapsed form, as a message hiding a key elsewhere already is. A lone or unpaired delimiter in a URL leaves the link as it is. That collapsed form is collapsed again until it settles, and one that still changes after four passes also loses its markup characters, redaction tags excepted. A link whose label holds [, ] or a line break is left as written on every channel; for [x[y](u) only [y](u) becomes a link. On Slack a link whose URL holds | or > is left as written.

Tests

  • test/test_markdown_link_parentheses.py: the exact text each channel receives for Wikipedia and MSDN URLs (Slack, Telegram HTML and plain, WhatsApp, iMessage), an aside's ) staying outside the link, two links on one line, an unbalanced URL left unlinked, an escaped parenthesis left as written, and images left raw on Slack. One contract test runs every renderer over the same inputs and fails if a renderer shows a link the display screen does not collapse. Another checks the helper against the outbound_files walker: for every character class a renderer passes, each row states whether the helper matches, a match ends at the walker's ), and deeper nesting, an escaped parenthesis or a nested [label]( gives no match while https://x/a](b) matches. A key split around a link nested in another link's URL is redacted, with and without a space before it, and so is one split around a URL ending in an escaped backslash; titled, angle-bracketed and spaced destinations still collapse on the screen. A key split by ** inside a link's URL is redacted, with and without parentheses and in Slack's <url|label> form, and Telegram's plain-text fallback never shows it; bold text beside a key-free link is left untouched. A differential test runs Telegram's real fallback over 130 generated splits of a key inside a URL and requires the screen to redact exactly the ones the fallback shows joined; a lone or unpaired delimiter, ~~ and single * or _ pairs leave the link untouched, and the reading's inline-code, ** and __ patterns are pinned to the fallback's own. Its fence pattern gives the fallback's result on all 29,524 strings of up to nine backticks, letters and newlines and on 20,000 seeded longer ones, and a key joined by an empty fence or by a fence's info string, inside a URL or in plain text, is redacted and never shown by the fallback. Text the screen rewrites never renders into a key on Slack, on screen or in Telegram's fallback: a link closed once *, a format character or a Slack link is dropped, the same shape on the canonical branch, one nested a level deeper, and nesting past the pass bound, which leaves no ]( behind; the construct alone is sent as written. Slack links https://x/a](b) whole and still links an IPv6 http://[::1]:8080/. Every renderer's label class is pinned to the screen's, and a key split around a link whose label holds [ or a line break is never shown joined on Slack, Telegram, WhatsApp or iMessage. Past the pass bound, a redaction tag stays whole and the output is its own redaction. The screen's plain reading is pinned pass by pass to Telegram's fallback, so a pass added to or reordered in the fallback fails a test. The bound itself is pinned: four rewrites settle, a fifth reaches the fallback, and the fallback removes every markup character it lists.
  • test/test_slack_render_pipeline.py: a key split around a parenthesised link is redacted; a plain parenthesised link renders whole with no redaction; the ReDoS structure guard expects seven negated classes in the screen's link pattern, each still excluding its own delimiters; the controls that prove Slack's own display scan and settling pass are needed narrow the shared screen first, so removing either Slack pass still reopens the leak.
  • test/test_markdown_link_parentheses.py also covers each of the screen's readings: a key split around a link a CommonMark reader closes at its balanced ) is redacted at nesting depths 2, 3, 4 and 8, behind an escaped \), inside an <...> destination holding ), before a stray ), and around a link written inside a destination; titled destinations and a [label]( nested in a URL stay caught; [AKIA](a)(b)IOSFODNN7EXAMPLE, Wikipedia and MSDN URLs, a nested balanced URL and unclosed openers are returned byte-identical; five hostile inputs (unclosed openers, 3,000-deep nesting, [ runs, 400 openers against 400 closers, ](< runs) stay linear; no renderer links two-level nesting or an escaped \); AKIA[IOSFODNN7EXAMPLE]**(https://x)** and 192 generated link-and-emphasis shapes against the real Telegram fallback (110 redacted, 82 unchanged); a <url|label> inside a raw Markdown link, with a generator over key cuts; a model-written [REDACTED: AKIA](https://v) past the pass bound; ~~, ||, * and _ pairs inside a parenthesised URL through the Discord path; a redaction tag followed by (, *(, _( or ||( keeping its brackets, its count and the text after it through the Discord path, the settling passes, the last resort and the split's repairs; a credential or suspicious-URL tag inside a link destination still delivered and counted when a reading joins a key across that link; and the Telegram renderer and the screen sharing the same fallback pattern objects in order.
  • test/test_markdown_link_parentheses.py pins what the balanced and first-close walks return on escaped brackets and parentheses, a trailing backslash, open and closed angle destinations, and openers with no ). test/test_display_split_safety.py pins that a window with no ]( reads the same under every link reading, which lets the split grading render such a window once.
  • test/test_display_split_safety.py: a rotation split into two or three messages is refused when the heading-marker reading or the balanced-link reading joins a key across a boundary, and a clean heading sequence is not. test/test_split_credential_boundary.py: the repair refuses to run without the channel's cutter, and the last resort strips the markup of a tag an author wrote to look like the redactor's, so no cut of it rejoins a key. test/test_discord.py: the control proving a seal grades its own split narrows the shared screen first.
  • Positive controls: undoing only the screen change reds the smuggle case, the contract test and the structure guard; undoing only Slack's pattern reds the Slack cases; dropping the escape check reds the escaped-parenthesis cases and the walker test; dropping the nested-opener rule reds both nested-link cases, its walker row and the structure guard; restoring a blanket refusal of a group after ] reds the ]( URL cases and the structure guard; running a *** pass before ** reds the case where a URL's ** pairs with a later ***; adding a ~~ pass reds the ~~ case; dropping only the fence pass reds the three code-fence cases; a greedy one-alternative fence pattern reds the fence equivalence check; sending the rewritten text without collapsing it again reds seven of the eight new rewrite cases, all but the construct alone; reading \\ as two separate backslashes reds the escaped-backslash cases; restoring the old [^\]]+ label class on Slack and Telegram reds the label cases and their speed tests; stripping tag brackets in the fallback reds the tag cases; an extra or swapped pass in Telegram's fallback reds the pass-list pin; a bound of three or five reds the bound tests.

Manual verification

Ran each renderer on the examples above and got the "now" column verbatim. Not checked in a live Slack, Telegram or Discord workspace; the tests pin the exact wire text each platform receives. The screen does not model a client that closes a link at a ) that is neither the balanced one nor the first.

The changed destination unit scales linearly on hostile input: unclosed [label]( openers, ]( runs, nested [label]( openers, ** runs, backslashes and parentheses, up to 320,000 characters. Doubling the input about doubles the time. The screen's fence pass is linear too, about 3 ms on an unclosed fence at 320,000 characters, where Telegram's own fence pattern takes about 2.8 s at 20,000 and grows with the square of the length. Measured end to end with the real redactor on messages that hide a key, the screen takes at most about 0.4 s at 320,000 characters (links nested past the pass bound) against about 0.1 s on main, and under 50 ms at Slack's 40,000-character message cap. On the same unclosed openers, main's Telegram and WhatsApp renderers take about 45 s, because their URL pattern rescans to the end of the message from every opener. A long unclosed run of [b took about 4.7 s at 20,000 repeats in the old Slack and Telegram label class; the narrowed class takes under 2 ms. The degenerate rotation test (a 5,000-backtick run cut into thousands of messages) takes about 4.6 s here against 2.7 s on main, under its 30 s limit. With one link in that run, grading the split takes about 4.8 s of CPU against 1.7 s on main, because every window is read under each reading; the link walks skip ordinary text, and a window with no ]( is rendered once for all four link readings. When a cut would rejoin a key, _under_a_safe_budget walks back through at most 135 budgets (at Teams' 16,000-character budget, the largest a channel passes), and can run once more after the span that rejoins is redacted. With every budget refused, one walk over 16,000 characters takes about 0.7 s of CPU with no ](, 2 s with one link and 4 s when a link with nested parentheses in its URL comes every 30 characters, against 0.4 s, 0.4 s and 1 s on main. It grows linearly: the link-dense walk takes about 16 s at 64,000 characters against 4 s on main. End to end through the call pair Teams makes at that budget (split_markdown_safe then bounded_for_delivery, best of three), a markdown reply with a plain access-key pair after every sixth section takes about 0.06 s of CPU at 32,000 characters and 0.13 s at 64,000, against 0.02 s and 0.04 s on main; render_for_slack takes 0.04 s and 0.08 s on it against 0.02 s and 0.03 s. With every cut of a 32,000-character reply forced to rejoin a key, so both walks run to the end, delivery takes about 2.8 s with no ](, 8.2 s with one link and 16.7 s on the link-dense text, against 1.7 s, 1.7 s and 4.3 s on main.

Related Issues

no linked issue: found by reading the channel renderers; no report exists yet.

Pattern harvest

Rule candidate: semgrep
Pattern: a Markdown-link regex whose destination is spelled [^)]… instead of built from constants.md_link_destination. The session preview, voice reply, artifact snippet and artifact-deploy page still spell it that way; there it only leaves a stray ) or a cut href outside chat, so they are left for a follow-up.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable): docs/system-specs/modules/messaging.md
  • No secrets, credentials, or internal references in the diff

@unstablebrainiac
unstablebrainiac requested review from a team and iamwhatever September 26, 2026 23:39
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention labels Sep 26, 2026
@unstablebrainiac

Copy link
Copy Markdown
Contributor Author

Intent: A Markdown link whose URL holds a balanced pair of parentheses reaches Slack, Telegram, WhatsApp and iMessage readers whole, and the display-safety screen still collapses every link those channels show, so a credential split around such a link is scanned as the reader sees it.
Not a goal: changing link handling outside the chat renderers (session previews, voice replies, artifact snippets and deploy pages, the web app's own link regexes), nesting deeper than one level, or backslash-escaped parentheses.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 26, 2026
@github-actions

github-actions Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review (fork) — ✅ no blocking findings

Reviewed 94648a91ddc7b9f7c74e02b82263debfec31d591 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 94648a9

@github-actions

github-actions Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5, fork) — 🟡 CONCERNS

Design-level review of 94648a91ddc7b9f7c74e02b82263debfec31d591 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

Worst-case delivery latency is asserted in comments and micro-tests, never measured end-to-end at the largest channel budgets.

Watch

The seam grade grew from 2 readings to up to 10 per chunk (_rejoins_a_key over _RENDERINGS), the dense budget walk from ~270 to a self-declared "at most 2,700 whole-text readings" (_DENSE_PROBES comment), and Slack adds per-piece settle/re-cut loops (_settled_pieces, DISPLAY_SETTLING_PASSES). Keyless fast paths are pinned byte-identical and the linearity tests cap adversarial regex time at <1s, but nothing times a realistic credential-bearing 16,000-character message through bounded_for_delivery/render_for_slack, where these multipliers compound on the outbound hot path a user is waiting on. The repo's own prior comment called a full-budget scan "minutes rather than milliseconds" territory; a 10× reading multiplier deserves one measured number before this ships.
Clears when: a stated timing (test or PR comment) for the worst-case redacting split at the 16,000-char budget shows delivery stays within the ~seconds bound the _DENSE_PROBES comment claims.

Suggestions

The _narrow_generic_screen monkeypatching of FURTHER_READINGS/_RENDERINGS in test_discord.py and test_slack_render_pipeline.py couples regression tests to private module tuples; a small fixture or injection point in display_safety would keep those historical pins from breaking every time a reading is added.

[DESIGN-REVIEWED] 94648a9

Description read

[DESCRIPTION-READ] 1d0cec88740c2e42171a7e97737a78a8977021ef4e5aa65fb8666f8b0f969460

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5, fork) — ✅ PASS

Premise-level review of 94648a91ddc7b9f7c74e02b82263debfec31d591 via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: PASS

Verify the first-) "lazy reader" reading against a real client (Discord's masked-link parser is the candidate) — the description names no client that closes there.

Not justified as shipped

  • Item 9 — rides along: the heading-marker seam (SecretAccessKey / # : value across a cut) is a leak class that predates and is orthogonal to parenthesised URLs; it enters because modelling Telegram's _strip_md faithfully requires its heading pass, and it carries its own named harm.

What this change ships

Inventory (10 items) — 9 justified

Intent: FIX — a Markdown link whose URL holds a balanced (...) pair reaches Slack, Telegram, WhatsApp and iMessage whole, with the display-safety screen still collapsing every link those channels show.

  1. Balanced-parenthesis URLs now link whole on Slack, Telegram (HTML and plain), WhatsApp — justified
  2. iMessage now links/renders images for parenthesised URLs it previously refused outright — justified
  3. Slack leaves raw a link whose URL holds |/> or whose label holds [, ] or a newline — justified
  4. Unbalanced, deeper-nested, escaped-parenthesis or nested-link destinations stay as written instead of linking a cut URL — justified
  5. The screen scans four further readings (Telegram fallback, balanced close, first-) close, link-free) and redacts a key any of them joins — justified
  6. Redacted output is re-rendered until it settles (4-pass bound), then stripped of markup outside redaction tags — justified
  7. Slack blocks, the joined message, split pieces and the truncation cut are scanned as Slack displays them; a display-only match costs the block its markup — justified
  8. Cuts, rotation boundaries and repairs are graded under the same readings, with repairs judged by the caller's own cutter (repaired_for_delivery now requires cut) and a new last-resort flatten — justified
  9. Heading-marker seams across a cut are now refused or redacted — rides along (leak class predates the parenthesis fix; required once the plain fallback is modelled)
  10. One shared destination grammar (constants.md_link_destination), pinned to outbound_files._walk_destination's close; 6 composition sites across 5 modules — justified

The 2 unfixed first-) siblings on base (deploy/render.py:75, voice_reply.py:414; grepped \[\^\)(\\s)?\]\+) are exactly the surfaces ## Not a goal declares out of scope, so the fix states its level and what is left.

[FIRST-PRINCIPLES-REVIEWED] 94648a9

Description read

[DESCRIPTION-READ] 05305a5a5c24324d64f7f11e5ae1cbdec04d3e5bff3b7465156a840bae042770

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5 Review (fork) — ✅ no blocking findings

Reviewed 94648a91ddc7b9f7c74e02b82263debfec31d591 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 94648a9

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Sep 27, 2026
@unstablebrainiac
unstablebrainiac force-pushed the fix/markdown-link-parenthesised-url branch from 13095b3 to 3cd48e8 Compare September 27, 2026 01:48
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: passed Eligible automated validation passed for the current revision labels Sep 27, 2026
@unstablebrainiac

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: md_link_destination refuses a backslash before a parenthesis, and a contract test pins its closing parenthesis to the outbound_files walker

  • span=ad3e984db40c fixed in 3cd48e8: md_link_destination is derived from the walker grammar and pinned to it, not deleted

The unit is defined as the parenthesis grammar of messaging/outbound_files.py _walk_destination, limited to one nesting level, with a backslash directly before a parenthesis refused.
Invariant: wherever the unit matches, it closes at the same ")" as the walker; otherwise it does not match. Only the captured text can differ (the walker unescapes other characters, then _finish_destination rewrites or rejects the text), and neither moves the end.
Pinned by test/test_markdown_link_parentheses.py::TestTheRegexUnitIsTheWalkerBoundedToOneLevel for every class a caller passes ([^()], [^()\s], [^()\n]). Removing the escape check turns it red.
Kept as a regex, not deleted: four renderers compose it into one re.sub pass, and display_safety._MD_LINK must stay linear-time (test_link_patterns_exclude_their_own_delimiters). A walk from every "[label](" opener is quadratic, and a regex cannot express the walker's unbounded depth.
Each helper's docstring names the other, so a change to one grammar points at the other and at the test.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 27, 2026
@unstablebrainiac
unstablebrainiac force-pushed the fix/markdown-link-parenthesised-url branch from 3cd48e8 to 48a53b2 Compare September 27, 2026 04:27
@unstablebrainiac

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: the character-class strings passed to md_link_destination are named module constants, so no f-string replacement field holds a backslash

  • span=b0c3fbbe921b fixed in 48a53b2: every backslash-bearing class string is hoisted out of the f-string fields

imessage/plaintext.py, telegram/renderer.py, whatsapp/renderer.py and messaging/display_safety.py pass named class constants to md_link_destination, and constants.py keeps its escape unit and nested-opener probe as module constants.
No f-string replacement field in the diff contains a backslash.
All 9 Python files this PR touches compile under Python 3.10 and 3.11 with compile(), without importing them.

@unstablebrainiac

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: md_link_destination refuses a "[" that opens a nested "[label](", and redact_for_display also scans a links-kept reading so a key inside a url is not dropped with the collapsed link

  • span=dbb61ed86f19 fixed in 48a53b2: a nested link opener ends the destination, so the inner link collapses and the key is scanned whole

In [x](a [AKIA](u)REST) the unit refuses the [ of [AKIA](, so the outer span does not match, the inner link collapses to AKIA, and the scan sees the joined key and redacts it, with or without the space.
A ]( that opens no link, as in https://x/a](b), stays part of the URL, so that destination collapses and a key split around it is redacted too.
Excluding whitespace from the screen's class was rejected: the screen would stop collapsing titled, <...> and spaced destinations that CommonMark or Slack link, and reopen those splits.
test_markdown_link_parentheses.py pins both nested cases, the walker rows and the five-class structure guard; removing the rule fails four of them.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 27, 2026
@unstablebrainiac
unstablebrainiac force-pushed the fix/markdown-link-parenthesised-url branch from 48a53b2 to c5d44d1 Compare September 27, 2026 06:02
@unstablebrainiac

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: the tag shape narrowed to the redactor's own tags in the previous round

  • span=3d3d343cb3c0 fixed in 1e8b246

_REDACTION_TAG now also matches the bracketed IPv6 literal exfil writes (\[[0-9A-Fa-f:.]+\], the class its host group admits), so [REDACTED: suspicious URL to [fd00::1]] and the IPv4-mapped form stay whole through both last-resort paths and are still counted by count_redaction_tags.
A DNS name holding _ stays out. A span of the tag shape is kept whoever wrote it, so the shape admits nothing a reading could act on, and _ is emphasis markup. Such a tag can lose its brackets and its count on the last-resort paths; the URL stays redacted.
Tests pin both IPv6 tags through redacted_without_markup, _flattened_for_any_cut and count_redaction_tags, and fail without the fix.

@unstablebrainiac

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: the docstring sentence added in the previous round

  • span=72fbb1083572 fixed in 1e8b246

The provenance sentence is gone. The flatten docstring now says a span of the tag shape is kept whoever wrote it, and that the shape admits nothing a reading acts on, so a lookalike holding markup, a heading marker or other whitespace is ordinary text.
The comment above _REDACTION_TAG says the same: an author-written exact [REDACTED: credential] is kept, which is safe because the shape holds nothing a reading could act on.

@unstablebrainiac

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: none added this round for this item

  • span=31a4c3f5a995 rebutted: a long keyless reply was timed through render_for_slack and bounded_for_delivery, before and after

A markdown reply with no key (headings, lists, a code block, inline code, and links including a parenthesised Wikipedia URL), CPU seconds, best of five, main 7525f1f99 against this head:

  • 4,000 characters: render_for_slack 0.002 against 0.005; split_markdown_safe then bounded_for_delivery at 4,000, 0.001 against 0.003
  • 16,000 characters: 0.006 against 0.018; 0.008 against 0.031
  • 64,000 characters: 0.024 against 0.072; 0.030 against 0.121
    The extra readings cost about three to four times main's time on a reply with no key. At 64,000 characters that is about 0.07 s for the Slack render and 0.12 s for the split, and every redacting split runs in a worker thread.

@unstablebrainiac

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: none added this round for this item

  • span=f0a5b8eb2dc6 rebutted: _RENDERINGS is built from FURTHER_READINGS, so the two cannot drift in code

split._RENDERINGS is (canonicalize_display, *FURTHER_READINGS), evaluated when the module loads, and LINK_READINGS filters it where a probe holds no ](. A reading added to FURTHER_READINGS lands in the split's set with no second edit.
The two differ only when a test replaces one binding at runtime. test_discord.py patches both on purpose, to narrow the screen to the readings that existed when its regression was added and prove the seal's own split grading still catches the key.

@unstablebrainiac

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: none added this round for this item

  • span=7b9484dfc322 needs-a-decision: the heading-seam and settling readings stay with the widening unless a maintainer asks to split them out

This is the bundling question already put to the maintainer on this PR (spans 8408dff56e6f, 06a29b72456d, 3ba9bf8df72f, 8864bb2bf86b, 41df4c109b90, fe1b744a8c26 and 2f635428d9f8).
A maintainer ruling to move the display-safety hardening into its own PR, to merge first, clears this item; either shape is ready on request.

@unstablebrainiac

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: none added this round for this item

  • span=b18d099b3a0f needs-a-decision: the heading-marker seam guard stays with the widening unless a maintainer asks to split it out

This is the bundling question already put to the maintainer on this PR (spans 8408dff56e6f, 06a29b72456d, 3ba9bf8df72f, 8864bb2bf86b, 41df4c109b90, fe1b744a8c26 and 2f635428d9f8).
A maintainer ruling to move the display-safety hardening into its own PR, to merge first, clears this item; either shape is ready on request.

@unstablebrainiac

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: none added this round for this item

  • span=daedc87bae48 needs-a-decision: settling every Slack block, piece and overflow cut stays with the widening unless a maintainer asks to split it out

This is the bundling question already put to the maintainer on this PR (spans 8408dff56e6f, 06a29b72456d, 3ba9bf8df72f, 8864bb2bf86b, 41df4c109b90, fe1b744a8c26 and 2f635428d9f8).
A maintainer ruling to move the display-safety hardening into its own PR, to merge first, clears this item; either shape is ready on request.

@unstablebrainiac

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: none added this round for this item

  • span=cd31d3969827 rebutted: the keyless streaming rotation and the worst-case split are timed below

Keyless streaming rotation: Discord's _rotate_on_length sealing a markdown reply with no key (headings, lists, code, links including a parenthesised URL) at DISCORD_MAX_TEXT, process CPU seconds including its worker threads, best of five, main 3a770911e against this head:

  • 4,000 characters, two frames: 0.012 against 0.040
  • 16,000 characters handed over at once, eight frames: 0.12 against 0.48
  • 64,000 characters handed over at once, 34 frames: 1.27 against 5.4
    The renderer rotates each time its buffer passes the limit, so a streamed reply pays about the first row's cost per frame (0.006 s against 0.02 s). The larger rows are a whole reply arriving in one piece, and every rotation runs in a worker thread.
    Worst-case redacting split, every budget refused, one walk at Teams' 16,000-character budget, the largest a channel passes (from the PR description): 0.67 against 0.41 with no ](, 2.00 against 0.41 with one link, and 4.05 against 1.04 on link-dense text. One redacting split runs at most two walks.
    A keyless reply through render_for_slack and through split_markdown_safe plus bounded_for_delivery is timed in the reply to span 31a4c3f5a995.

@unstablebrainiac

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: none added this round for this item

  • span=a7a897d15e1d rebutted: the tests that patch the reading sets are controls, and the shipped set is pinned without them

Those tests change the screen on purpose. Some narrow it to the readings that predate this PR, to prove a later defence (Slack's own display pass, a seal's own split grading) still catches the key; others take one reading out to prove it is needed. Either kind has to run a screen production does not, or it would prove nothing.
Every other test in those files runs with the module's own readings, which pins the shipped set directly. A readings parameter on the grade would widen its signature for tests alone.

@unstablebrainiac

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: none added this round for this item

  • span=4afd98767460 needs-a-decision: the balanced and first-) readings stay with the widening unless a maintainer asks to split them out

This is the bundling question already put to the maintainer on this PR (spans 8408dff56e6f, 06a29b72456d, 3ba9bf8df72f, 8864bb2bf86b, 41df4c109b90, fe1b744a8c26 and 2f635428d9f8).
A maintainer ruling to move the display-safety hardening into its own PR, to merge first, clears this item; either shape is ready on request.

@unstablebrainiac

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: none added this round for this item

  • span=96732b5ff7d8 needs-a-decision: the heading-marker seam stays with the widening unless a maintainer asks to split it out

This is the bundling question already put to the maintainer on this PR (spans 8408dff56e6f, 06a29b72456d, 3ba9bf8df72f, 8864bb2bf86b, 41df4c109b90, fe1b744a8c26 and 2f635428d9f8).
A maintainer ruling to move the display-safety hardening into its own PR, to merge first, clears this item; either shape is ready on request.

@unstablebrainiac

Copy link
Copy Markdown
Contributor Author

Rebased: main moved by 13 commits since 1e8b24606 was built (3a770911e -> 244afb1d7), with no conflict and no change to the patch. One of them narrows the exfiltration base64 heuristic; the suspicious-URL tag and its host shape, which this PR's tag pattern mirrors, are unchanged. The PR's related tests and main's changed tests pass on 15dd8ea6d.

@unstablebrainiac

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: none added this round for this item

  • span=98e70c70a869 needs-a-decision: the display-safety hardening stays with the fix unless a maintainer asks to split it out

This is the bundling question already put to the maintainer on this PR (spans 8408dff56e6f, 06a29b72456d, 3ba9bf8df72f, 8864bb2bf86b, 41df4c109b90, fe1b744a8c26, 2f635428d9f8, 7b9484dfc322, b18d099b3a0f, daedc87bae48, 4afd98767460 and 96732b5ff7d8).
A maintainer ruling to move the display-safety hardening into its own PR, to merge first, clears this item; either shape is ready on request.

@unstablebrainiac

Copy link
Copy Markdown
Contributor Author

The red Backend Tests (Windows) (1) on 15dd8ea6d is two tests main added in #13568, both in test/test_apps_registry_coverage.py: TestSubdirGates::test_contained_join_degrades_to_none_on_a_symlink_loop and TestRunAppBuild::test_a_self_pointing_directory_link_under_a_deep_declared_path_is_judged_in_bounded_time[60]. On this runner a self-pointing link resolved instead of raising, and a 60-link path was accepted. This PR does not touch apps/, and both tests pass on Linux on this head. Main's own run passed them on its Windows Server 2022 runner; this fork run used GitHub's hosted windows-2025-vs2026 image (Windows Server 2025), with the same Python 3.12.10.

@unstablebrainiac

Copy link
Copy Markdown
Contributor Author

Rebased: main moved by 3 commits since 15dd8ea6d was built (244afb1d7 -> 48b040c54), touching none of this PR's files. The PR's related tests and main's changed tests pass on 2160efdbb.

@unstablebrainiac

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: one helper, redacted_rendering, through which every emitted rendering runs outside redactor-owned tags

  • span=dbb61ed86f19 fixed in 2160efd

Reproduced on 15dd8ea: AKIA||IOSFODNN7EXAMPLE||(rotated 2026-09-01) and its ** variant came out as REDACTED: credential, counted (0, 0), on the Discord path and through redact_for_display. Main keeps [REDACTED: credential](rotated 2026-09-01), counted (1, 0).
A redactor-owned tag is now an atom on every path that emits a rendering: the settling passes, the every-reading loop, the last resort, Slack's settled emission and the split's repairs all render outside tags through redacted_rendering. What the screen scans is unchanged: every form is still read whole, tags included, and text is emitted only once its literal form and its whole-text rendering are both clean.
Where a reading joins a key only across a tag, that reading's redacted form is emitted, because it is what that reader sees, followed by every tag that reading hid, so the delivered count still names each redaction. AKIA[](<suspicious URL>)IOSFODNN7EXAMPLE delivers both tags, counted (1, 1); main delivers only the credential tag.
DISPLAY_MARKUP keeps its parentheses. With [ and ] gone outside tags, the only link left past the bound is a tag's own ](, which the last resort now scans under every reading; stripping parentheses would take them from every word a last-resort reply keeps.
Tests: TestATagFollowedByAParenthesisIsNotALink (26 cases, 24 fail on 15dd8ea) and TestTheWholeTextFallbackKeepsRemovedRedactionTags (3 cases, all fail on 15dd8ea). Opus's input now comes out as [REDACTED: credential](rotated 2026-09-01), counted (1, 0), as on main.

@unstablebrainiac

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: none added this round for this item

  • span=ca568111a6cc needs-a-decision: the heading-marker seam and Slack's settled cuts stay with the fix unless a maintainer asks to split them out

This is the bundling question already put to the maintainer on this PR (spans 8408dff56e6f, 06a29b72456d, 3ba9bf8df72f, 8864bb2bf86b, 41df4c109b90, fe1b744a8c26, 2f635428d9f8, 7b9484dfc322, b18d099b3a0f, daedc87bae48, 4afd98767460, 96732b5ff7d8 and 98e70c70a869).
A maintainer ruling to move those two fixes into their own PRs, to merge first, clears this item; either shape is ready on request.

@unstablebrainiac

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: none added this round for this item

  • span=8a043a5e2857 needs-a-decision: the five-reading model and its named residual need the display-safety owner's sign-off as the screen's coverage contract

The question for the maintainer: is the screen's coverage the literal bytes plus the five display forms the messaging/display_safety.py row of docs/system-specs/modules/messaging.md lists (the shared one-level link grammar, Telegram's plain fallback, a depth-unbounded balanced close, a lazy first-) close and a link-free reading), with the residual the module names, a client that closes a destination at another ), outside it?
Where a reading models one of this repo's renderers it cannot drift silently: Telegram's _strip_md imports the TELEGRAM_FALLBACK_* pattern objects from the screen, and TestPlainReadingSharesTelegramFallbacksPasses fails when _strip_md gains, loses or reorders a pass or copies a pattern. The balanced, first-) and link-free readings model readers outside this repo, so no test can pin them to a renderer here.
A maintainer sign-off on that contract clears this item.

@unstablebrainiac

Copy link
Copy Markdown
Contributor Author

Backend Tests (Windows) (1) is red on 2160efdbb on the same two tests as on 15dd8ea6d: main's #13568 symlink tests in test/test_apps_registry_coverage.py. Upstream tracks this as #15006, which sees it on every PR since #13568. This PR does not touch apps/. The branch gets rebased once main's fix lands.

A Markdown link such as
[Python](https://en.wikipedia.org/wiki/Python_(programming_language))
is valid CommonMark: a destination may hold a balanced pair of
parentheses. The Slack, Telegram and WhatsApp renderers ended the URL
at its first ")", so the link opened a page that does not exist and a
stray ")" trailed the label. iMessage refused parentheses and left the
markup raw.

Every renderer now builds its destination from one shared unit,
constants.md_link_destination(): one URL character, or a "(...)" group
of them, one level deep. A backslash and the character after it are one
token, so an escaped "[" or backslash stays whole and a backslash before
a parenthesis is refused. A "[" that opens a nested "[label](" is
refused, so the inner link renders on its own and the outer text stays
as written, while a "](" that opens no link stays part of the URL. An
unbalanced "(", a second level of nesting or an escaped "\)" leaves the
link as written instead of linking a cut URL. The unit is the grammar
outbound_files walks to find local image files, bounded to one level;
wherever it matches it ends at the walker's ")", and a test pins this.
Slack's destination also excludes "|" and ">", which would end or split
its <url|label> form, and every renderer's link label excludes "[", "]"
and line breaks, as the screen's does.

The display-safety screen collapses each link to its label so the
credential scan reads what the reader sees, and a wider link match
drops more URL text from that scan. So besides the literal text and the
renderers' own grammar, the screen always scans four readings that
model no single client: Telegram's plain-text fallback, applying that
fallback's own fence, inline-code, heading, "**", "__" and link passes
in its order with lone delimiters kept; each link closed at its
balanced ")" at any depth, as a CommonMark reader such as Discord
closes it; each link closed at the first ")", as a lazy reader closes
it; and no link parsing at all, so a key split by markup inside a URL
is still scanned. Telegram's renderer imports the fallback passes from
the screen, one definition for both. Each converted Slack block, and
the joined message, is also scanned as the Slack client displays it
and settled until that view shows no key.

A key found in any reading is answered with the collapsed, redacted
form, which carries no URL, and that answer is itself scanned under
every reading. Collapsing can itself close a link ("[AKIA]*(u)" reads
as "[AKIA](u)" once the "*" is gone), so rewritten text is collapsed and
scanned again until a pass changes nothing, for at most four passes;
past that it strips markup outside redaction tags and collapses and
scans once more.

A cut splitting a long message, each boundary of a message rotated into
several, and a cut's repair are graded under the same readings as one
message, and the repair always runs with the channel's own cutter. Text
with no "](" is graded under the canonical form and the plain fallback
only, since each link reading equals the canonical form there.
@unstablebrainiac

Copy link
Copy Markdown
Contributor Author

Rebased: 48b040c54 -> 199532b50, new head 94648a91d. Main's #14986 (601decce7) makes the two symlink tests in test/test_apps_registry_coverage.py platform-aware, which should clear the Backend Tests (Windows) (1) failure tracked in #15006. The patch is unchanged (same patch-id). The PR's test files plus the one changed test on main that imports its modules pass (2,650), and flake8 and mypy are clean.

@unstablebrainiac

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: none added this round for this item

  • span=3655973633f7 rebutted: a credential-bearing reply and the worst-case redacting split at Teams' 16,000-character budget are timed below

Every case runs the call pair teams/renderer.py makes at its 16,000-character limit, split_markdown_safe then bounded_for_delivery with _default_redactor, each in a worker thread there. Process CPU seconds, best of three, main 199532b50 against this head.
A markdown reply (headings, lists, a code block, links including a parenthesised URL) with one plain AWS access-key pair after every sixth section:

  • 16,000 characters, 4 pairs, one message: 0.01 against 0.01, and render_for_slack 0.01 against 0.02
  • 32,000 characters, 8 pairs, two messages: 0.02 against 0.06, and render_for_slack 0.02 against 0.04
  • 64,000 characters, 16 pairs, four messages: 0.04 against 0.13, and render_for_slack 0.03 against 0.08
    Worst case: a 32,000-character reply with every seam grade refused after the real grade has run, and the span repair forced to report a repair, so both budget walks of the redacting split run to the end (273 grades here, 271 on main) before the bound repairs the answer:
  • no ](: 1.69 against 2.82
  • one link: 1.70 against 8.16
  • a link with nested parentheses in its URL every 30 characters: 4.26 against 16.68
    That case is an upper bound: it needs every one of those cuts to rejoin a key. The docstring's "minutes rather than milliseconds" is about grading every budget one at a time, which the walk avoids by doubling its step after 128 budgets. The PR description's Manual verification section now carries these figures.

@unstablebrainiac

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: none added this round for this item

  • span=f781d618139d rebutted: the tests that narrow the reading sets are controls, and a reading added later does not break them

This is the suggestion answered as span a7a897d15e1d. Both tests narrow the screen on purpose, to the two readings that existed when their regression was added, to prove a later defence (Slack's own display pass, a seal's own split grading) still catches the key. A control has to run a screen production does not.
Each sets FURTHER_READINGS and _RENDERINGS outright to those two readings, so a reading added to the module leaves them unchanged. Every other test in those files runs the module's own readings, which pins the shipped set directly, and a readings parameter on the grade would widen its signature for tests alone.

@unstablebrainiac

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: none added this round for this item

  • span=a6f2515f3e1b needs-a-decision: the heading-marker seam stays with the fix unless a maintainer asks to split it out

This is the bundling question already put to the maintainer on this PR (spans 8408dff56e6f, 06a29b72456d, 3ba9bf8df72f, 8864bb2bf86b, 41df4c109b90, fe1b744a8c26, 2f635428d9f8, 7b9484dfc322, b18d099b3a0f, daedc87bae48, 4afd98767460, 96732b5ff7d8, 98e70c70a869 and ca568111a6cc).
A maintainer ruling to move the heading-marker seam into its own PR, to merge first, clears this item; either shape is ready on request.

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed on behalf of the maintainers (sweep 2026-09-29): readiness passed, required checks green on 94648a9, no blocking review threads, security posture unchanged. Ruling on the author's needs-a-decision item: the bundled shape (link grammar + display-safety readings in one PR) is accepted as-is. Approving and merging.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants