Skip to content

fix(channels): grade the delivered sequence so a split cannot rejoin a key - #13984

Merged
iamwhatever merged 2 commits into
mainfrom
fix/cross-delivery-redaction-13743-v12
Sep 25, 2026
Merged

iamwhatever merged 2 commits into
mainfrom
fix/cross-delivery-redaction-13743-v12

Conversation

@chenmingwei23

Copy link
Copy Markdown
Contributor

What is the problem?

Supersedes #13968, and before it #13944, #13889, #13864, #13858, #13852, #13844,
#13839, #13836, #13828 and #13758. Each replacement exists for the
same mechanical reason: taking a review fix while staying inside the two-commit limit
needs the commits reshaped, and a reshaped branch cannot be published by a
fast-forward push. Cutting a fresh branch publishes the same two commits as a plain
create instead. No review had been left on either predecessor.

Six channel delivery paths cut model text on a length budget and scan each piece on its
own. A credential the cut severs matches nothing in either piece, so both are reported
clean and the reader rejoins the halves reading one message and then the next.

Five of the six already redact the whole buffer before splitting, and that closes a cut
that lands mid-line: the credential is one marker before any boundary can reach it. It
does not close a cut at a line boundary. Sealing a chunk trims the whitespace that
ended it, and the next chunk starts at the first character after that whitespace, so the
two halves end up flush against each other on screen while the whole text still holds the
break between them. That form is invisible to a scan of the text as written and to a scan
of either piece.

Why this issue matters to the user

The reader sees the key and nothing reports it. Because each piece scans clean, no
redaction notice is emitted and the delivery is recorded as a clean send: the operator has
no signal at all. This is the audit half of the problem, not only the disclosure half.

How our fix solves it

The boundary is where the defect lives, so the fix goes into the two shared splitters
rather than into six renderers. split_markdown_safe and chunk_utf8_bytes take an
optional redactor and, when given one, reduce the text before any boundary is chosen:

  1. the text is redacted against the rendered form first, via the existing
    redact_for_display, which closes a cut landing mid-line and the markup-hidden variant;
  2. the boundaries it then chooses are graded on the RENDERED SEQUENCE. A platform drops
    the whitespace at a message's edges, so the last visible character of one message sits
    against the first visible character of the next and a trailing newline separates
    nothing. Stripping every chunk's edges before the grade is what makes the predicate see
    what a reader sees, and it is what carries the guarantee to the byte splitter, which
    stores the break rather than trimming it.

The whole sequence, not each neighbouring pair, because a key needs no more than a narrow
budget to span three chunks: split a twenty-character key over four and every pair holds a
fragment matching nothing while the screen holds the key whole. The same gap swallows a
chunk that renders to nothing at all, which the lossless byte splitter can place between
the halves. A fragment is only ever part of a key, so no pair of neighbours can be asked
about it. Both readings joins_to_a_credential names are kept, and each is scanned once
over the joined text rather than per boundary, which is what keeps the search affordable --
measured at 10.5 ms against 160 ms on a 74 KB reply, so the ordinary path is 15 times
cheaper than a per-boundary grade.

When a boundary does rejoin a key the cut is MOVED before any text is given up. Candidate
budgets step back one at a time for the first 128 and exponentially after that; the dense
prefix is what stops a budget clean at 6997 being stepped over by a walk that jumps 4, 8,
16, and the exponential tail keeps the reach without a probe per byte. Worst case 1.49 s on
a 74 KB reply, below the 2.2 s a purely exponential walk cost with the old grade.

A STREAMING caller cannot use that search at all, and this is the sharpest thing the change
had to get right. Such a caller re-splits its growing body every frame and treats all but
the last chunk as delivered, which whatsapp/turn_renderer.py states as its own
requirement: chunk i never changes once chunk i+1 exists. A search that reads the whole
body breaks it -- appending ONE character moved a boundary under an already-sent message in
a measured case, and the final pass slices the new list from the delivered count, so the
span is duplicated or dropped with no way back. That leg therefore takes a prefix-stable
split: the text is redacted, and every boundary stays where the caller's own budget puts
it. The same search space that produced a revision produces none under it.

Every path that SHOWS a message repairs its own seam and records only what went out, and the
enumeration is the enforcement. Three gaps closed here, each found by review on the previous head.
WhatsApp's seal loop sealed its chunk RAW: the forward grade asks whether that chunk severs a key
and says nothing about the message already sent, so once a reply passed two chunks the first seam
was graded by nothing and the final pass started past it. It now repairs against the recorded
predecessor and records what it actually sealed. Telegram recorded the predecessor BEFORE any
delivery confirmed, so a segment whose every send and edit failed became the predecessor and the
next delivered message gave up a leading span for a key no reader ever saw; the repair is now pure
and _record_sent runs at each confirmed delivery. And the degraded table path re-splits AFTER
the segment's seam was graded, so its tail sits under a chunk that did not exist at that point and
is graded again there.

The seam guard lives in the two SINKS, not at the call sites. Five paths seal a Telegram segment
as its own message -- a length rotation, an upload-hold rotation, a steer boundary, a degraded
table chunk and the end of the turn -- and a guard written at one of them is not a guard on the
channel: on the previous head only the length rotation carried it, so the final seal re-rendered
the same text unrepaired and put the key back into the persistent chat, and the upload-hold seal
recorded nothing, leaving every later grade pointed at a message that was no longer the one above
it. One helper now redacts, repairs the seam and records the predecessor, and both sinks call it,
so no sealing path can forget and none repeats the rule. A test asserts the record has exactly
one writer.

A boundary graded when a chunk SEALS says nothing about text that arrives afterwards, and that
is the sharpest case in the change. A chunk whose tail is a credential PREFIX matches nothing, so
every scan passes it and the message goes out; the characters completing the key arrive later and
the reader scrolling the two messages reads it whole while neither message holds it. A sent message
cannot be recalled, so both streaming legs remember the chunk they last showed and grade the next
thing shown against it, giving up exactly the leading span that completes the key -- everything
after that span is the reply as written, sliced rather than reassembled. Reproduced on the previous
head through the rotation itself: a buffer ending AKIA sealed, the rest of the key arrived, and
the key stood whole on screen; with the record in place it does not.

Prefix stability says nothing about whether a BOUNDARY is safe either, so the streaming renderer
grades each one against everything that follows before counting the chunk final, using
joins_to_a_credential, which main already has. That grade reads the RENDERED text, every
chunk's edges stripped, for the same reason the splitter's own predicate strips them: a
platform drops the whitespace at a message's edges, so a tail beginning with a tab separates
nothing once two messages sit on screen. Asking about the characters as stored approves
exactly that pair, and the case is reachable -- the splitter never strips leading whitespace
and a hard cut can leave a remainder starting with one. Both grades now answer the same
question about the same boundary, and a test pins that they agree. A chunk that does not pass stays in the
live bubble, where a later frame may still revise it; at finalization every chunk must go,
so the still-unsealed tail is repaired through repaired_for_delivery rather than held.

Only when no budget at or below the caller's cuts safely is text given up, and then only
the credential's own span. There is no wider fallback that strips a whole message: when no
span names the key, the splitter DECLINES TO CUT and returns the text whole. A boundary is
the only thing that rejoins a key across messages, so creating none is fail-closed, and it
costs the caller a chunk over its limit instead of costing the reader a reply stripped of
its whitespace and markup. A caller that bounds that chunk again has
repaired_for_delivery to grade what it produced. The span is found by SHRINKING -- a span known to collapse into
a key is trimmed from each end by half its width, the trim kept while what remains still
collapses into one -- so a message carrying two keys keeps the paragraphs between them. The
whitespace inside that span is closed up, which makes the key contiguous for the ordinary
redaction, and the loop returns only once the form with EVERY run removed reads clean. That
last condition is the promise to a caller that re-cuts these chunks under its own limit, and
it is why one grade at the end of such a caller is enough. For a key only the rendering
assembles, which no span of the literal text names, the answer is the canonical form -- the
one-directional trade redact_for_display already makes, giving up markup and keeping every
space and every break.

Grading chosen boundaries rather than every candidate is what keeps ordinary wrapped content
intact -- two lines that read as one key when glued are never joined by a cut, so they are
left exactly as written, and a fixture pins that.

The parameter is optional because a caller whose chunks land inside one message severs
nothing a reader can rejoin across messages. It is not optional for the seven paths that do
turn one chunk into one message: those are enumerated in the tests, and any splitter call
there that omits the redactor fails. The check walks the AST and counts both shapes,
including the four sites that hand the splitter to asyncio.to_thread as a reference --
a check reading direct calls only would see four of the eight sites and silently pass the
rest.

Per path, all eight call sites:

path call before now
slack/renderer.py:875 split_markdown_safe, offloaded no whole-buffer redaction at the cut passes _redact_all
webex/renderer.py:396 chunk_utf8_bytes display-safe via webex_display_safe line-boundary seam also closed
whatsapp/renderer.py:302 split_markdown_safe display-safe before the split line-boundary seam also closed
teams/renderer.py:383 split_markdown_safe, offloaded display-safe via _display_text line-boundary seam also closed
teams/renderer.py:743 split_markdown_safe, offloaded display-safe before the split line-boundary seam also closed
wecom/renderer.py:306 split_markdown_safe, offloaded display-safe before the split line-boundary seam also closed
dashboard/chat_mirror.py:588 split_markdown_safe display-safe before the split line-boundary seam also closed
telegram/renderer.py:413 split_markdown_safe no redaction at the cut, each segment redacted alone passes _default_redactor

WeCom's streaming push already picks a credential-safe offset for the frame it is about to
send, and that path is untouched.

Two legs needed more than the redactor argument, both found by review and both reproduced
before they were fixed.

Slack bounds the splitter's output AFTER it has been graded. A logical line placed whole
carries its fence scaffolding on top, so a chunk can exceed the limit, and chunk_text then
blind-slices it and each slice is posted as its own message -- an ungraded boundary at the
very last step. _split_for_slack now grades that final sequence through the new
repaired_for_delivery and repairs it when it rejoins a key. One grade suffices because the
repair is only returned once its fully-collapsed form reads clean, which the tests check by
re-cutting the repaired text at five different caps.

WhatsApp's own sender is the sixth capped path and had no bound. Its non-stable split is what
the channel's client uses, posting each chunk as its own message with no length rule of its own,
and giving that split a redactor made it able to decline to cut and answer with the whole reply
as one over-cap chunk -- a regression this change introduced and now closes with the same shared
bound. The stable split is deliberately left unbounded: its caller treats all but the last chunk
as delivered, so a bound there would move a boundary under a message already sent, and a test
pins that its boundaries stay the budget's.

FIVE paths have a HARD cap, and the splitter's fail-closed answer collides with all of them. When no
budget cuts without rejoining a key the splitter declines to cut and answers with the text
whole, which costs the caller a chunk over its budget; this client truncates a larger payload,
so the answer's tail would go with no notice. Each now bounds and grades EACH CHUNK on its own
through one shared bounded_for_delivery, which applies the budget BEFORE it grades. That order
is the point: the splitter's fail-closed answer is a list of ONE chunk, a one-element list holds
no boundary, and a grade asked about it reports that nothing rejoins -- so the oversized chunk
would travel on unbounded, which is the very case the bound exists for. Cutting to the budget
first makes the grade see the sequence the transport will really deliver. A chunk already inside
the budget is returned unchanged, so the pass is free. Re-cutting a repair needs no redactor and
must NOT take one, since a credential-aware re-cut could decline to cut and leave the budget
unmet. Webex, Teams, WeCom, the dashboard mirror and Telegram all run it; a test walks those five
and fails if any sends the splitter's answer unbounded, another asserts no chunk exceeds the byte
cap, and a third that ordinary text still comes back whole.

What a caller re-cuts without a grade is a collapse fixed point, and the splitter makes it
so rather than claiming it.
A text whose whitespace-free canonical form holds no key cannot
produce one under any later cut, because closing a boundary's whitespace is the most a cut can
do to bring two characters together, and no substring of a clean collapse holds a key either.
That is the promise a re-cutting caller relies on, and it was previously asserted rather than
established: measured on the previous head, the delivery fallback returned text whose collapse
still read as a key. _made_collapse_clean now establishes it in three steps, least destructive
first -- text already clean is returned byte for byte, then the canonical form is redacted, and
only where a key is named by no span and found by no markup reading does whitespace go at all,
and then only the whitespace of the SPAN that hides it: the canonical form's smallest such span is
found the way the literal reading finds one, closed up and redacted. Measured on a reply with two
paragraphs, a bullet list and a key written in code spans, eight of its nine breaks survive and
every line of prose is intact, where a blanket collapse leaves none of them. The splitter applies it to its FINAL fallback, the answer reached when the budget search
fails, and the delivery helper applies it to its own fallback. The successful search path is
untouched and still returns every character, because those chunks are re-cut by callers that
grade what they cut; a gate placed before the search was measured to strip the spaces out of an
ordinary reply, which is why it sits at the fallback.

A repair is made on the text its caller HOLDS, never on a rebuilt sequence. The chunk list is
graded and never reassembled, because its concatenation is not the reply: sealing a chunk outside
a fence trims the whitespace that ended it, so a join deletes a break at every seam, and a fence
spanning a seam contributes its synthetic closer while the next chunk opens with the reopener, and
the two meet as one longer delimiter run that reads as a NEW fence -- after which the rest of the
message renders as code. Measured on the previous head: four chunks of a five-break source joined
to two breaks, and a fenced body whose source held only three-backtick runs produced a six-backtick
one. So each chunk is its own repair subject, the boundaries between chunks stay the splitter's own
where the whole source was available, and the one seam no splitter call covers -- Telegram's, between
two blocks it cut separately -- is repaired from the body that function holds.

Webex was the one path that kept the grade on the loop thread. Its final answer calls the
byte splitter directly inside on_done, and with no cut clean the search and the span repair
hold that thread for seconds -- long enough for the liveness watchdog to exit the gateway and
take every other channel's in-flight turn with it. It now offloads like the others. A
test walks all seven paths and fails if any of them stops handing its splitter call to a
thread, and Telegram carries a stricter version of that check described below.

The span repair is also bounded now. Its passes were capped only by the number of whitespace
runs in the text, and each pass costs a logarithmic number of whole-text redactions, so a
reply whose every break split a key-shaped token would cost thousands of them and the send
would never finish. The cap is the same constant the budget search uses, and past it the
answer is to decline to cut, which gives up nothing.

The dashboard mirror's backfill composed its units inline on the loop thread. An imported
history row carries no size cap, the grade redacts and rescans each candidate boundary, and
the liveness watchdog exits the process after seconds of loop silence -- so one large row
could terminate the gateway. The units are now composed inside asyncio.to_thread, the same
offload the Slack twin already uses and this handler already uses for its selection read.

Telegram was the remaining path whose cut chose boundaries on length alone. It rotates: a
segment that outgrows one message is split, every chunk but the last is sealed as its own
message, and the redaction runs per segment -- so the halves of a severed key are redacted in
neither message while the reader scrolling the two reads it whole. Reproduced at the channel's
own 400-character budget floor: the plain cut hands the key over, the credential-aware cut
moves the boundary instead and every character still ships.

Three things that leg needed beyond the argument.

It asks for no prefix-stable mode, and the reason is a structural difference from the
WhatsApp streaming leg: a rotation REPLACES the buffer with the retained tail, so text
already sealed is never part of a later cut. That is the bookkeeping whose absence is the
residual below, and Telegram has it.

Its rendered budget is enforced by measuring the real HTML and shrinking the source budget,
which means the cut is made more than once; each candidate answer now runs the shared bound,
so a declined cut cannot reach the client's own truncation with the reply's tail inside it.

Its table-aware path cuts each block on its own, so a boundary BETWEEN two blocks belongs to
no single cut and was graded by none of them. The assembled sequence is graded once at the end,
against the body that function holds. Every block is also redacted BEFORE it enters the list,
which the grade requires: the sequence reading is sound only on chunks already a fixed point of
its own scan, and a table run bypasses the splitter, so without it a cell holding a whole
credential fires the repair for something no seam severed -- measured on the previous head, that
cost the message its **bold** and its `code` and collapsed the table and the prose into
one message. The reachable case is a table whose last cell ends its line followed by prose, and a
fixture pins both halves of it: the per-block pair is shown to hand the key over, the
function's answer is shown not to.

Four call sites on that channel run the cut from a coroutine, and a scan on the gateway's
event loop is what the liveness watchdog exits the process for. All four now hand their work
to a thread. The file-wide check here is stricter than the one the other six paths get: "one
offload exists somewhere in the file" is satisfied while three sites stay on the loop, so the
Telegram check walks every coroutine's own body -- nested closures excluded, since those are
themselves what gets offloaded -- and fails if any of them calls a scanning splitter directly.

The redaction-tag tally on WhatsApp's finalization is fixed rather than deferred, which the
last round recorded as advisory. That pass repairs the unsealed tail and sends the repair, but
the notice counted the pre-repair chunk list, so a reply whose only placeholder came from the
repair announced none. The branch now re-points the list the notice reads at the sequence it
actually sends.

What tests we did

test/test_split_credential_boundary.py, 140 passed and 5 skipped, every run scoped with
an explicit -n0:

  • the seam, measured rather than assumed: the fixture's cut is shown to fall between the
    halves of a key, the two pieces are shown to scan clean individually, and the joined
    sequence is shown to hold the key;
  • a key spanning three and four chunks, where every neighbouring pair reads clean, on both
    splitters and across five narrow budgets;
  • a chunk that renders to nothing placed between the halves by the lossless splitter;
  • a key inside a link, which the canonicalised join loses to the label and only the
    per-side reading sees;
  • the dense probe: a budget off the doubling ladder is reached, asserted on the probe order;
  • the terminal, on text no budget can cut safely: the prose around the span survives byte
    for byte, every break and delimiter is kept, two keys keep the paragraph between them,
    the fully-collapsed form of the result reads clean, and the wider flush is shown to keep
    neither -- plus the end-to-end case through the byte splitter, where all seven breaks
    survive;
  • the post-bounding grade: a blind slice is shown to reopen the seam, the repair fires, a
    clean sequence needs none, and the repair stays safe when re-cut at five caps;
  • structural checks that the Slack path runs that grade and that the mirror composes its
    units in a thread;
  • prefix stability, which is the streamed-reply case: the searching path is shown to revise
    an already-sealed chunk after a single appended character, the stable path is shown not to
    across seven append lengths, and the stable path is shown to still redact;
  • structural checks that the streaming renderer asks for the stable split, grades its seam
    and repairs its tail, and that no global whitespace flush remains in the module;
  • all seven delivery paths, asserting each hands its splitter call to a thread, so none of
    them can put the grade back on the event loop, plus the stricter Telegram check that no
    coroutine in that file calls a scanning splitter directly;
  • the repair loop's bound, asserting the constant is in force;
  • the hard cap on all four capped paths: none of them sends the splitter's answer unbounded,
    no chunk exceeds the byte cap, and ordinary text is returned whole;
  • a SINGLE oversized chunk, the shape a one-element grade skips: it is cut inside the budget,
    every character still ships, a key across one of the new boundaries is still caught, and
    text already inside the budget is returned untouched;
  • the seam's edge whitespace: the stored reading is shown to approve a tab-separated pair
    the reader sees whole, the rendered reading to refuse it, and the two grades to agree;
  • Telegram's rotation at its real budget floor: the plain cut is shown to hand the reader
    the key, the credential-aware cut to move the boundary instead, every character to still
    ship, the bounded splitter to carry the same guard, a declined cut to stay inside the
    budget, and ordinary prose to come back unchanged;
  • Telegram's table-aware seam, with its own control: the per-block sequence is shown to
    rejoin the key and the function's answer is shown not to;
  • the sealed prefix, end to end: the sealed chunk is shown to be clean on its own, the pair to
    hand the reader the key, the offset to cover only the completing span, the repair to close the
    seam and to keep every character after it, and the rotation itself to stop handing it over;
  • that the repair is made on the source and not on the join: a source whose join loses breaks
    keeps them in the repair, and both its paragraph breaks survive;
  • the bound's own grade, on a blind slice INSIDE one chunk, which is the boundary no splitter
    call saw;
  • Telegram's table run: the key never reaches the screen, the message keeps its markup, and the
    table still seals as its own message;
  • structural checks that both streaming legs offload the repair and record what they sent, that
    each of WhatsApp's two send paths does so rather than one, and that the notice counts the
    sequence that shipped;
  • the collapse guarantee: the splitter's fallback and the delivery fallback are each shown to be
    a collapse fixed point on text where no span names the key, and the successful search path is
    shown to lose no character;
  • an enumeration of whitespace rewrites, asserting all three sit in named helpers, so a fourth
    way for a valid span to lose its formatting fails here;
  • the sealing sinks: both are shown to run the seam helper, the record is shown to have exactly
    one writer, that writer is shown to run only where a delivery confirms, and no sealing caller is
    allowed to carry a seam rule of its own;
  • the degraded tail, shown to be graded again after its re-split;
  • WhatsApp's seal loop, shown to repair before it seals and to record what it sealed rather than
    the original chunk;
  • the narrowed last resort: the premise that neither the literal nor the canonical reading closes
    the case, then the answer shown to be a collapse fixed point, every break but the key's own
    shown to survive, and a blanket collapse shown as the control that keeps none;
  • an enumeration of the five delivery tallies, each asserted to read the text that shipped, with
    the premise that a repair really does add a placeholder;
  • WhatsApp's own send path: the splitter alone is shown to decline and answer over the cap, the
    channel path to bound it with no key on screen, the stable split to keep the budget's
    boundaries, and ordinary prose to come back as its converted form exactly;
  • one channel end to end, through the function WhatsApp's client sends from, asserting on
    the frames it would deliver;
  • backward compatibility: prose and a table row are returned byte for byte with a redactor
    passed, so existing callers are unaffected.

Mutation proof, thirty-eight ways, each against a pristine copy kept outside the repo and each
verified by hash to have applied and been restored, with a passing control before and after.
Eight cover this round: sealing the raw chunk again fails 2, collapsing everything again fails 2,
recording before delivery confirms fails 2, dropping the degraded tail's seam pass fails 1, and
each of the four tallies reverting to its source fails 1.
Seven cover this round: taking the collapse fix off the splitter's fallback fails 1, gating the
guarantee on the stored reading instead of the rendered one fails 1, letting the delivery fallback
assert the promise again fails 1, dropping the seam repair from either sink fails 1 each, stopping
the predecessor record fails 1, and removing WhatsApp's bound fails 1. The third and the gating one
each caught a weak test first: the gate measured clean on a key written in code spans, which is
exactly the shape the guarantee exists for.
Eight cover this round: rebuilding the repair from the join fails 1, making the sent-tail offset
always answer clean fails 4, grading the flattened sequence again in the shared bound fails 1 and
in Slack fails 1, dropping Telegram's repair against its sealed predecessor fails 1, letting its
table blocks in unredacted fails 2, and dropping WhatsApp's repair fails 1 while dropping its
record fails 1. Three of those measured weak tests first and are the reason for three of the new
cases: the shared bound, WhatsApp's repair and WhatsApp's record each failed NOTHING under a
structural check that only asked whether the file mentioned the guard somewhere. Three cover the streaming and terminal
work: stopping the renderer from asking for the stable split fails 1, making the stable mode
fall back to the search fails 2, and sealing without grading the seam fails 1. Four cover the
grade and the repair. Dropping the redactor from one of the
seven call sites fails the enumeration test. Replacing the dense probe with a doubling walk
fails the two probe-order cases. Sending the terminal straight to the wider flush fails the
end-to-end case, where the breaks drop from seven to zero. Widening the repair's span to the
whole text fails four cases, including the two-key one. A fifth mutation, dropping one of the
grade's two readings, initially failed NOTHING -- that gap is what the link case was added
for, and it now fails.

Regression on the rebased tree, each file scoped with -n0: test_teams_renderer.py 22 passed,
test_webex_renderer.py 34 passed, test_wecom_renderer.py 33 passed,
test_credential_redaction_notice.py 27 passed, test_whatsapp.py 26 passed,
test_telegram_parity.py 360 passed, test_teams_renderer.py 22 passed,
test_wecom_renderer.py 33 passed, test_telegram.py 331 passed,
test_telegram_parity.py 360 passed, test_options_cap_contract.py 71 passed,
test_channel_control_tag_strip.py 39 passed, test_credential_redaction_notice.py
27 passed, test_redaction_mirror_parity.py 10 passed, test_whatsapp.py 26 passed,
test_messaging_split.py 138 passed,
test_channel_table_rendering.py 82 passed (the reassembly invariant the byte splitter
documents), test_chat_mirror.py 82 passed, test_display_split_safety.py 50 passed,
test_whatsapp_renderer.py 46 passed, test_whatsapp_turn_renderer.py 39 passed,
test_security_posture.py 53 passed, test_slack_renderer.py 73 passed,
test_webex_renderer.py 34 passed. Gates: black, isort, flake8, mypy on the changed files,
and the comment-history gate, all clean.

messaging/split.py is listed in NON_EGRESS_REDACTION_MODULES with the reason the
display-safety module gives: it applies a redactor its caller injects, owns no output, and
the modules calling it are the registered sinks.

The fixture uses the AWS-documented example key the repo's own display-safety tests
already use, so no new credential shape is introduced.

Any other suggestions on the work

This does NOT close the issue, and the issue stays open on purpose. Five of the six delivery
paths it names are closed end to end, and Telegram -- a seventh path beyond the six, which
seals splitter chunks as separate messages the same way -- is closed here too: every boundary they choose is graded, and the
enumeration test fails if any of them stops passing a redactor. The sixth, WhatsApp's
streaming leg, keeps the graded cut but carries a residual that is present on current main
independently of this change -- that leg re-renders a growing reply on every frame while
treating earlier messages as delivered, so a delivery tracked by a count cannot notice when
an earlier message would be rendered differently. Closing it properly means grading a
boundary against what is already on screen, which is the channel-neutral work in #13494, not
something this change can reach. The residual is recorded on #13743 rather than a duplicate
issue. What it does NOT add is predecessor bookkeeping at each point a message stops
being editable -- a streaming leg that has already delivered a message cannot re-cut it, so
that piece has to be graded against what is already on screen. The channel-neutral
primitives for that are in #13494 and are not on main yet, so this change reuses only what
main already has and touches none of the files that PR owns. That bookkeeping belongs to a
follow-up once #13494 lands, not to the six paths this issue is about.

Two things found while measuring, worth recording. The splitter has callers beyond the six
paths named in the issue: discord/renderer.py (five calls), telegram/renderer.py,
feishu/client.py, messaging/renderer.py and teams/session_resume.py. Discord seals
splitter chunks as separate messages and carries the same open seam; that file belongs to
#13494, which is rewriting it with the predecessor primitives, so it is stage-two scope here
by construction and this PR does not touch it. Telegram carried the same seam and is closed
here: its rotation discards sealed text from the buffer, so it needs none of the predecessor
primitives and can be fixed with what main already has.
Feishu, the shared renderer and the Teams resume path are unchanged, and none of them
belongs to #13494. Their own code answers whether their chunks become separate messages
rather than leaving it open: feishu/client.py sends every chunk as its own reply to the
same inbound anchor, and chunk_for_transport hands its parts to callers that send them
one message at a time. Neither passes a redactor, so the seam is open on those legs too.
They sit outside this stage by scope, not by another PR's ownership. And the issue's per-file grade was measured by grepping for the new primitives'
names, which cannot see that five of the six paths already close the mid-line cut by
redacting the whole buffer first; the line-boundary seam is what was actually open at all
of them.

Two advisory findings are recorded rather than fixed. The spec below is one. The other asks that
the enumeration of delivery paths be DERIVED by walking every module rather than curated, and that
Discord and Feishu then pass a redactor too; the reviewer marks it advisory itself, on the grounds
that its remedy lands in files this change does not touch, and that is the disposition taken here.
The tally finding from the previous round is fixed: the notice now counts the sequence that
shipped.

One review finding is recorded rather than fixed. The owning spec,
docs/system-specs/modules/messaging.md, still calls the Webex splitter lossless and the
character splitter free of modes, and the redactor ladder falsifies both. That file belongs
to #13494, which this change deliberately does not touch, so the correction travels with the
follow-up that lands after it rather than creating a conflict here.

Pattern harvest

Rule candidate: enumeration test, added in this PR as
test_split_credential_boundary.py::TestEveryDeliveryPathIsEnumerated.

Pattern: a text is cut on a length budget and then scanned one piece at a time, so the scan
runs after the boundary is chosen and cannot see what the cut separated.

The shape generalises past credentials: any check that reads one piece of a text the caller
will deliver in several is blind to whatever the boundary split. It resists a purely
syntactic rule, because "these chunks each become their own message" is not visible at the
call site -- which is why the enforceable half is an enumeration: the delivery paths are
listed, and every splitter call in them must pass a redactor. The list is checked in both
directions, so a path that stops reaching a splitter fails rather than passing by having
nothing left to check, and the AST pass counts the offloaded asyncio.to_thread shape as
well as the direct call.

Refs #13743

Six delivery paths cut model text on a length budget and scan each piece
on its own. A key the cut severs matches nothing in either piece, so both
are reported clean, and the reader rejoins the halves reading one message
and then the next. The per-piece scan emits no notice, so the operator
sees a clean send.

Five of those paths already redact the whole buffer before splitting,
which closes a mid-line hard cut. It cannot close a cut at a line
boundary: sealing a chunk trims the whitespace that ended it, so the two
halves sit flush against each other on screen while the whole text still
holds the break between them, invisible to a scan of the text as written
and to a scan of either piece.

The shared splitters now take the redactor themselves. split_markdown_safe
and chunk_utf8_bytes reduce the text before any boundary is chosen: the
form the platform renders, and the form the seal's trim produces once
every break is collapsed. A credential visible in either is a marker
before a boundary can land inside it. The parameter stays optional, so a
caller whose chunks stay inside one message is unaffected, and an
enumeration test over the six delivery paths fails when any splitter call
there omits it.

Refs #13743
Grade the boundaries the ladder chooses, one at a time, rather than every
break at once. A scan of all of them together answers a wider question than
any cut asks: two lines that read as one key when glued would cost the message
every break and its markup, though no cut ever joins them. Wrapped content in a
fence is that shape.

The grade reads the RENDERED pair. A platform drops the whitespace at a
message's edges, so the last visible character of one message sits against the
first visible character of the next: neither a trailing newline nor a trailing
space separates anything. That is also what carries the guarantee to the byte
splitter, which stores the whitespace rather than trimming it -- lossless where
it is kept, invisible where it is shown.

The repair moves the CUT before it gives up any text. Budgets step back
exponentially and every candidate is verified, so a key straddling a boundary
ends up whole inside one chunk with every character and every break intact,
where the space between its halves is on screen and the scanner's own reading
of such text stands. Only when no budget at or below the caller's cuts safely
is the whitespace flushed and the text redacted -- every run, because the
whitespace a cut consumes can be a single space -- and that result is graded as
well, since a repair returned without a grade is a repair nobody checked.
Flushing every run is deliberately wider than one cut produces: that width is
what lets a caller bound the chunks again under its own platform limit without
a grade of its own.

The splitter applies a caller's redactor, so it is classified in
security_posture's NON_EGRESS_REDACTION_MODULES. It owns no output; the modules
that call it are the registered sinks. Same reason display_safety.py sits
there, and the posture drift guard fails without the entry.

Refs #13743
@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — 🟡 CONCERNS

Design-level review of 40418eec7a984744495372e3105c31d7bd03fed3 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

Splitter safety stays opt-in: Discord, Feishu and both session-resume paths still ship ungraded seams, guarded only by a curated enumeration test.

Watch

  • The redactor parameter is optional on a security-relevant primitive, so the shared splitters are unsafe-by-default; the enforcement is a hand-curated path list (TestEveryDeliveryPathIsEnumerated), and the PR's own audit names live callers outside it — discord/renderer.py (5 calls), feishu/client.py, teams/session_resume.py, discord/session_resume.py, chunk_for_transport — each sending chunks as separate messages, so the exact disclosure this PR fixes remains reachable on those channels, and a future channel added off the list inherits the ungraded cut silently.
    Clears when: the residual paths pass a redactor (or the enumeration is derived by walking modules) in the recorded follow-up after #13494, tracked on #13743.
  • The owning spec is now false in the same commit that falsifies it: docs/system-specs/modules/messaging.md:611 still says the splitter is "stdlib-only and pure — no config objects, no modes", while the diff adds a display_safety import, a redactor ladder and a stable mode; AGENTS.md requires the spec update in the SAME commit, and the deferral (to avoid conflicting with fix(channels): grade the rotation boundary the splitter cannot see #13494) leaves the doc actively misleading for anyone routed to it meanwhile.
    Clears when: a maintainer accepts the recorded deferral or the spec correction lands with the #13494 follow-up.

[DESIGN-REVIEWED] 40418ee

@github-actions

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of 40418eec7a984744495372e3105c31d7bd03fed3 and found no blocking issues.

This comment is updated in place on each push.

Review details

FINDING -- src/kiro_crew/wecom/renderer.py:320 -- with _carried > 0, overwriting the whole-answer tally with remainder-only counts omits redaction notices for earlier bubbles -> Fix: preserve prior counts and add only placeholders introduced by boundary repair.
[GPT-REVIEWED] 40418ee

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 40418eec7a984744495372e3105c31d7bd03fed3: <one-sentence reason>

@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — 🟡 CONCERNS

Premise-level review of 40418eec7a984744495372e3105c31d7bd03fed3 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

Discord and Feishu still cut model text into per-message chunks with no redactor — the defect this PR closes lives on, unlisted, in two channels.

Not justified as shipped

  • Item 1 — unfixed siblings: the description claims "Every path that SHOWS a message repairs its own seam … and the enumeration is the enforcement," but grepping split_markdown_safe|chunk_utf8_bytes|chunk_text finds redactor-less per-message splits at discord/renderer.py:941,962,966,1219,1244 (each chunk sealed via _seal_current + _open_new_message, the Telegram rotation pattern) and feishu/client.py:213 (each chunk posted as its own reply) — 2 sibling channels, plus the discord/session_resume.py:106 / teams/session_resume.py:343 resume splits, none in DELIVERY_PATHS.
    Clears when: those paths pass the redactor (Discord's streaming legs also the sent-tail record), or the description names them as counted and deferred.
  • Item 7 — one consumer, generalized: stable=True has exactly one consumer (whatsapp/turn_renderer.py:334; grep stable=True = 1 delivery path) and its branch is two existing calls — redact_for_display then the plain split.
    Subtraction: delete the stable kwarg from split_markdown_safe/render_chunks/render_chunks_off_loop; the one caller composes the two existing calls.
    Clears when: a second streaming caller needs the mode, or the kwarg is deleted.

What this change ships

Inventory (10 items) — 7 justified

Intent: FIX — a credential severed by a message-length split reads whole across adjacent messages while every per-piece scan reports a clean send; defect reproduced by tests in this PR against the base splitter.

  1. Splits on Slack/Teams/Telegram/Webex/WeCom/WhatsApp/dashboard mirror now move the cut or redact instead of severing a key — unfixed siblings (discord/renderer.py, feishu/client.py)
  2. Streaming messages give up the leading span completing a key ending the previous sent message (Telegram, WhatsApp) — justified
  3. Redaction notices count what shipped, so a repair-only redaction is announced — justified
  4. Declined cuts are re-bounded to hard transport caps instead of silent client truncation — justified
  5. When no safe cut exists, only the key-hiding span loses whitespace/markup — justified
  6. Splitting/composition moved off the event loop (dashboard backfill, Telegram rotations, Webex) — rides along: required by the grade's new cost; the liveness watchdog exits the process on a held loop
  7. stable mode on the shared splitter — one consumer, generalized
  8. New shared surface: redactor kwargs, bounded_for_delivery (6 consumers), repaired_for_delivery (3), repaired_after_a_sent_tail (5 call sites) — justified
  9. messaging/split.py registered in NON_EGRESS_REDACTION_MODULES — justified
  10. Telegram records the seam predecessor only on confirmed delivery, checking a previously fire-and-forget plain send — justified

[FIRST-PRINCIPLES-REVIEWED] 40418ee

@github-actions

Copy link
Copy Markdown
Contributor

Opus 5 Review — ✅ no blocking findings

Reviewed 40418eec7a984744495372e3105c31d7bd03fed3 — this comment is updated in place on each push.

Review details

Nothing blocks; the redaction-notice tally was re-pointed at text that is not the whole answer on two channels, so a reply the reader received redacted is announced as clean.

FINDING — src/kiro_crew/wecom/renderer.py:320 — count_redaction_tags("\n".join(chunks)) overwrites the whole-answer tally from line 274 with a count over remainder = answer[self._carried:] only, so after any bubble rotation (_roll_if_sealed sets _carried on seal/unroutable/age) a placeholder in the already-delivered prefix is dropped from _notice_creds/_notice_urls and close() posts no notice at all → Fix: combine instead of replacing — tally answer[: self._carried] + "\n" + "\n".join(chunks), or take the per-counter max() of the two readings.

FINDING — src/kiro_crew/slack/renderer.py:1786 — count_redaction_tags(self._delivered or clean_text) reads a ledger that _render_fallback writes only for chunks[0] (line 1168; the chunks[1:] posts at 1169–1173 record nothing), so on the no-stream path an over-limit answer whose placeholders sit in a continuation chunk under-counts, and counts 0 when every placeholder is past chunk 0 → Fix: also record the continuations (self._delivered += part after each successful post_message), or reconcile per counter against count_redaction_tags(clean_text).

FINDING — src/kiro_crew/telegram/renderer.py:846 — on a repair the function returns _split_markdown_bounded(repaired, rendered_limit), a splitter that knows nothing about _table_blocks or rich_limit, and both callers only reach it with text already past their budget, so the table run is cut mid-body: the continuation chunks carry no header/separator, fail _has_table at line 1939 and _table_blocks, and arrive as unframed literal pipes — the outcome _split_table_rows exists to prevent → Fix: re-enter _split_markdown_table_aware(repaired, rendered_limit, rich_limit) with a flag that forbids a second repair, so table runs keep their atomicity and their rich budget after a repair.

[OPUS-REVIEWED] 40418ee

Verdict parsed from the review's SHA-scoped output markers for commit 40418eec7a984744495372e3105c31d7bd03fed3.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 40418eec7a984744495372e3105c31d7bd03fed3: <one-sentence reason>

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Sep 25, 2026
@iamwhatever
iamwhatever enabled auto-merge (squash) September 25, 2026 23:11
@iamwhatever
iamwhatever merged commit 3fe931b into main Sep 25, 2026
93 checks passed
@iamwhatever
iamwhatever deleted the fix/cross-delivery-redaction-13743-v12 branch September 25, 2026 23:13
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Sep 25, 2026
chenmingwei23 added a commit that referenced this pull request Sep 27, 2026
`_rotate_on_length` cuts the raw buffer on a length budget and each chunk is
sealed as its own message, redacted alone. #13984 closed most of this by giving
the shared splitters the redactor, so they reduce the text before choosing any
boundary. Two boundaries are outside that fix.

Discord's rotation splitter takes no redactor at all -- `split_markdown_safe` is
called with text and a limit on both its paths -- so its chunk boundaries are
ungraded. And on Telegram the pre-split pass reads the text AS WRITTEN: a
horizontal rule standing between the halves of a key keeps them apart there,
while the seal strips horizontal rules, so the halves sit flush on screen.

`severs_a_credential` is the n-piece case of `joins_to_a_credential`: it grades a
delivered SEQUENCE, over the whole join and over each boundary against the rest,
because neither reading subsumes the other. Both renderers gate their rotation on
it and fall back to `safe_split_offset` to move the cut, withholding the buffer
when no sampled offset is safe.

Both take an optional `present` and grade BOTH the raw and the delivered form. A
delivered-only grade is unsound in one direction that measured here: trimming
reveals a join where whitespace kept the halves apart, and CONCEALS one where the
pattern needs that whitespace, so `-----BEGIN ` beside `RSA PRIVATE KEY-----`
reads clean trimmed and is an intact PEM anchor as sent. Three of nine straddle
shapes failed on the delivered-only rule and pass on both.

No gate on Telegram's held-image tail. That tail begins AT the ref span, so its
first characters are always `![`, and `!` survives canonicalising between the
sealed chunk and anything the tail contributes; measured over alt-text,
link-target, trailing-text and bare-adjacency shapes, no join reaches a pattern.

Tests: 99 pass -- the two rotation-seam classes, the primitive's own unit tests
including the both-forms rule, and the `TurnFooter` and table-streaming cases that
were red before.

Refs #13743
Fixes #12293
iamwhatever pushed a commit that referenced this pull request Sep 28, 2026
…13494)

* fix(channels): grade the rotation boundary the splitter cannot see

`_rotate_on_length` cuts the raw buffer on a length budget and each chunk is
sealed as its own message, redacted alone. #13984 closed most of this by giving
the shared splitters the redactor, so they reduce the text before choosing any
boundary. Two boundaries are outside that fix.

Discord's rotation splitter takes no redactor at all -- `split_markdown_safe` is
called with text and a limit on both its paths -- so its chunk boundaries are
ungraded. And on Telegram the pre-split pass reads the text AS WRITTEN: a
horizontal rule standing between the halves of a key keeps them apart there,
while the seal strips horizontal rules, so the halves sit flush on screen.

`severs_a_credential` is the n-piece case of `joins_to_a_credential`: it grades a
delivered SEQUENCE, over the whole join and over each boundary against the rest,
because neither reading subsumes the other. Both renderers gate their rotation on
it and fall back to `safe_split_offset` to move the cut, withholding the buffer
when no sampled offset is safe.

Both take an optional `present` and grade BOTH the raw and the delivered form. A
delivered-only grade is unsound in one direction that measured here: trimming
reveals a join where whitespace kept the halves apart, and CONCEALS one where the
pattern needs that whitespace, so `-----BEGIN ` beside `RSA PRIVATE KEY-----`
reads clean trimmed and is an intact PEM anchor as sent. Three of nine straddle
shapes failed on the delivered-only rule and pass on both.

No gate on Telegram's held-image tail. That tail begins AT the ref span, so its
first characters are always `![`, and `!` survives canonicalising between the
sealed chunk and anything the tail contributes; measured over alt-text,
link-target, trailing-text and bare-adjacency shapes, no join reaches a pattern.

Tests: 99 pass -- the two rotation-seam classes, the primitive's own unit tests
including the both-forms rule, and the `TurnFooter` and table-streaming cases that
were red before.

Refs #13743
Fixes #12293

* fix(channels): grade every rotation cut, including the seal's own

The presentation fallback took one safe offset and retained the whole
remainder, which is parked as `_delivery_text` where the only cut left is
`_seal_current`'s ungraded length re-split. It now carves until the
remainder fits the budget and reads the finished list once as a whole.

The sequence grader read each boundary forwards only, so every reading ran
to the end of the list and carried the next message's first letter, which
the patterns anchor on. It now reads prefixes alongside suffixes.

It also reads BOUNDED interior runs. A credential can sit with a spoiling
message on each side, which no whole, prefix or suffix reading reaches. A
window of at most four consecutive messages is slid across the interior,
which is a linear number of readings over bounded-size strings; reading
every contiguous run is quadratic in a piece count the author controls.

The rotation's re-assignment handed back the source remainder plus the held
image reference without re-grading, and an image starting at or before the
budget makes the source shorter than the budget, so the offset search
early-returns the whole string and the refused pair ships unchanged. The
pair is now re-graded in its delivered form before it is assigned.

Telegram's upload-hold branch sealed its whole prefix and returned before
the length path's sequence grade. The splitter reads the raw pieces, where
a horizontal rule still stands between two credential fragments, and the
seam repair reads the delivered form against one predecessor only. The
prefix is now graded as a sequence and carved when it severs.

Discord's `_seal_current` split its over-cap payload with no redactor,
where Telegram's splitter call has carried one all along. That is the last
cut before the wire, with no rotation behind it, and a buffer the rotation
withheld as unsafe is parked whole and arrives there over the cap -- so the
withhold routed text TO the one ungraded cut. Both of its
`split_markdown_safe` calls now pass one.

Closes #14587

---------

Co-authored-by: wave1 verify <worker@localhost>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants