Skip to content

fix(channels): grade the delivered sequence so a split cannot rejoin a key - #13968

Closed
chenmingwei23 wants to merge 2 commits into
mainfrom
fix/cross-delivery-redaction-13743-v11
Closed

chenmingwei23 wants to merge 2 commits into
mainfrom
fix/cross-delivery-redaction-13743-v11

Conversation

@chenmingwei23

Copy link
Copy Markdown
Contributor

What is the problem?

Supersedes #13944, and before it #13889, #13864, #13858, #13852, #13844, #13839,
#13836, #13828 and #13758. Each replacement exists for the
same mechanical reason: taking a review fix while staying inside the two-commit limit
needs the commits reshaped, and a reshaped branch cannot be published by a
fast-forward push. Cutting a fresh branch publishes the same two commits as a plain
create instead. No review had been left on either predecessor.

Six channel delivery paths cut model text on a length budget and scan each piece on its
own. A credential the cut severs matches nothing in either piece, so both are reported
clean and the reader rejoins the halves reading one message and then the next.

Five of the six already redact the whole buffer before splitting, and that closes a cut
that lands mid-line: the credential is one marker before any boundary can reach it. It
does not close a cut at a line boundary. Sealing a chunk trims the whitespace that
ended it, and the next chunk starts at the first character after that whitespace, so the
two halves end up flush against each other on screen while the whole text still holds the
break between them. That form is invisible to a scan of the text as written and to a scan
of either piece.

Why this issue matters to the user

The reader sees the key and nothing reports it. Because each piece scans clean, no
redaction notice is emitted and the delivery is recorded as a clean send: the operator has
no signal at all. This is the audit half of the problem, not only the disclosure half.

How our fix solves it

The boundary is where the defect lives, so the fix goes into the two shared splitters
rather than into six renderers. split_markdown_safe and chunk_utf8_bytes take an
optional redactor and, when given one, reduce the text before any boundary is chosen:

  1. the text is redacted against the rendered form first, via the existing
    redact_for_display, which closes a cut landing mid-line and the markup-hidden variant;
  2. the boundaries it then chooses are graded on the RENDERED SEQUENCE. A platform drops
    the whitespace at a message's edges, so the last visible character of one message sits
    against the first visible character of the next and a trailing newline separates
    nothing. Stripping every chunk's edges before the grade is what makes the predicate see
    what a reader sees, and it is what carries the guarantee to the byte splitter, which
    stores the break rather than trimming it.

The whole sequence, not each neighbouring pair, because a key needs no more than a narrow
budget to span three chunks: split a twenty-character key over four and every pair holds a
fragment matching nothing while the screen holds the key whole. The same gap swallows a
chunk that renders to nothing at all, which the lossless byte splitter can place between
the halves. A fragment is only ever part of a key, so no pair of neighbours can be asked
about it. Both readings joins_to_a_credential names are kept, and each is scanned once
over the joined text rather than per boundary, which is what keeps the search affordable --
measured at 10.5 ms against 160 ms on a 74 KB reply, so the ordinary path is 15 times
cheaper than a per-boundary grade.

When a boundary does rejoin a key the cut is MOVED before any text is given up. Candidate
budgets step back one at a time for the first 128 and exponentially after that; the dense
prefix is what stops a budget clean at 6997 being stepped over by a walk that jumps 4, 8,
16, and the exponential tail keeps the reach without a probe per byte. Worst case 1.49 s on
a 74 KB reply, below the 2.2 s a purely exponential walk cost with the old grade.

A STREAMING caller cannot use that search at all, and this is the sharpest thing the change
had to get right. Such a caller re-splits its growing body every frame and treats all but
the last chunk as delivered, which whatsapp/turn_renderer.py states as its own
requirement: chunk i never changes once chunk i+1 exists. A search that reads the whole
body breaks it -- appending ONE character moved a boundary under an already-sent message in
a measured case, and the final pass slices the new list from the delivered count, so the
span is duplicated or dropped with no way back. That leg therefore takes a prefix-stable
split: the text is redacted, and every boundary stays where the caller's own budget puts
it. The same search space that produced a revision produces none under it.

The seam guard lives in the two SINKS, not at the call sites. Five paths seal a Telegram segment
as its own message -- a length rotation, an upload-hold rotation, a steer boundary, a degraded
table chunk and the end of the turn -- and a guard written at one of them is not a guard on the
channel: on the previous head only the length rotation carried it, so the final seal re-rendered
the same text unrepaired and put the key back into the persistent chat, and the upload-hold seal
recorded nothing, leaving every later grade pointed at a message that was no longer the one above
it. One helper now redacts, repairs the seam and records the predecessor, and both sinks call it,
so no sealing path can forget and none repeats the rule. A test asserts the record has exactly
one writer.

A boundary graded when a chunk SEALS says nothing about text that arrives afterwards, and that
is the sharpest case in the change. A chunk whose tail is a credential PREFIX matches nothing, so
every scan passes it and the message goes out; the characters completing the key arrive later and
the reader scrolling the two messages reads it whole while neither message holds it. A sent message
cannot be recalled, so both streaming legs remember the chunk they last showed and grade the next
thing shown against it, giving up exactly the leading span that completes the key -- everything
after that span is the reply as written, sliced rather than reassembled. Reproduced on the previous
head through the rotation itself: a buffer ending AKIA sealed, the rest of the key arrived, and
the key stood whole on screen; with the record in place it does not.

Prefix stability says nothing about whether a BOUNDARY is safe either, so the streaming renderer
grades each one against everything that follows before counting the chunk final, using
joins_to_a_credential, which main already has. That grade reads the RENDERED text, every
chunk's edges stripped, for the same reason the splitter's own predicate strips them: a
platform drops the whitespace at a message's edges, so a tail beginning with a tab separates
nothing once two messages sit on screen. Asking about the characters as stored approves
exactly that pair, and the case is reachable -- the splitter never strips leading whitespace
and a hard cut can leave a remainder starting with one. Both grades now answer the same
question about the same boundary, and a test pins that they agree. A chunk that does not pass stays in the
live bubble, where a later frame may still revise it; at finalization every chunk must go,
so the still-unsealed tail is repaired through repaired_for_delivery rather than held.

Only when no budget at or below the caller's cuts safely is text given up, and then only
the credential's own span. There is no wider fallback that strips a whole message: when no
span names the key, the splitter DECLINES TO CUT and returns the text whole. A boundary is
the only thing that rejoins a key across messages, so creating none is fail-closed, and it
costs the caller a chunk over its limit instead of costing the reader a reply stripped of
its whitespace and markup. A caller that bounds that chunk again has
repaired_for_delivery to grade what it produced. The span is found by SHRINKING -- a span known to collapse into
a key is trimmed from each end by half its width, the trim kept while what remains still
collapses into one -- so a message carrying two keys keeps the paragraphs between them. The
whitespace inside that span is closed up, which makes the key contiguous for the ordinary
redaction, and the loop returns only once the form with EVERY run removed reads clean. That
last condition is the promise to a caller that re-cuts these chunks under its own limit, and
it is why one grade at the end of such a caller is enough. For a key only the rendering
assembles, which no span of the literal text names, the answer is the canonical form -- the
one-directional trade redact_for_display already makes, giving up markup and keeping every
space and every break.

Grading chosen boundaries rather than every candidate is what keeps ordinary wrapped content
intact -- two lines that read as one key when glued are never joined by a cut, so they are
left exactly as written, and a fixture pins that.

The parameter is optional because a caller whose chunks land inside one message severs
nothing a reader can rejoin across messages. It is not optional for the seven paths that do
turn one chunk into one message: those are enumerated in the tests, and any splitter call
there that omits the redactor fails. The check walks the AST and counts both shapes,
including the four sites that hand the splitter to asyncio.to_thread as a reference --
a check reading direct calls only would see four of the eight sites and silently pass the
rest.

Per path, all eight call sites:

path call before now
slack/renderer.py:875 split_markdown_safe, offloaded no whole-buffer redaction at the cut passes _redact_all
webex/renderer.py:396 chunk_utf8_bytes display-safe via webex_display_safe line-boundary seam also closed
whatsapp/renderer.py:302 split_markdown_safe display-safe before the split line-boundary seam also closed
teams/renderer.py:383 split_markdown_safe, offloaded display-safe via _display_text line-boundary seam also closed
teams/renderer.py:743 split_markdown_safe, offloaded display-safe before the split line-boundary seam also closed
wecom/renderer.py:306 split_markdown_safe, offloaded display-safe before the split line-boundary seam also closed
dashboard/chat_mirror.py:588 split_markdown_safe display-safe before the split line-boundary seam also closed
telegram/renderer.py:413 split_markdown_safe no redaction at the cut, each segment redacted alone passes _default_redactor

WeCom's streaming push already picks a credential-safe offset for the frame it is about to
send, and that path is untouched.

Two legs needed more than the redactor argument, both found by review and both reproduced
before they were fixed.

Slack bounds the splitter's output AFTER it has been graded. A logical line placed whole
carries its fence scaffolding on top, so a chunk can exceed the limit, and chunk_text then
blind-slices it and each slice is posted as its own message -- an ungraded boundary at the
very last step. _split_for_slack now grades that final sequence through the new
repaired_for_delivery and repairs it when it rejoins a key. One grade suffices because the
repair is only returned once its fully-collapsed form reads clean, which the tests check by
re-cutting the repaired text at five different caps.

WhatsApp's own sender is the sixth capped path and had no bound. Its non-stable split is what
the channel's client uses, posting each chunk as its own message with no length rule of its own,
and giving that split a redactor made it able to decline to cut and answer with the whole reply
as one over-cap chunk -- a regression this change introduced and now closes with the same shared
bound. The stable split is deliberately left unbounded: its caller treats all but the last chunk
as delivered, so a bound there would move a boundary under a message already sent, and a test
pins that its boundaries stay the budget's.

FIVE paths have a HARD cap, and the splitter's fail-closed answer collides with all of them. When no
budget cuts without rejoining a key the splitter declines to cut and answers with the text
whole, which costs the caller a chunk over its budget; this client truncates a larger payload,
so the answer's tail would go with no notice. Each now bounds and grades EACH CHUNK on its own
through one shared bounded_for_delivery, which applies the budget BEFORE it grades. That order
is the point: the splitter's fail-closed answer is a list of ONE chunk, a one-element list holds
no boundary, and a grade asked about it reports that nothing rejoins -- so the oversized chunk
would travel on unbounded, which is the very case the bound exists for. Cutting to the budget
first makes the grade see the sequence the transport will really deliver. A chunk already inside
the budget is returned unchanged, so the pass is free. Re-cutting a repair needs no redactor and
must NOT take one, since a credential-aware re-cut could decline to cut and leave the budget
unmet. Webex, Teams, WeCom, the dashboard mirror and Telegram all run it; a test walks those five
and fails if any sends the splitter's answer unbounded, another asserts no chunk exceeds the byte
cap, and a third that ordinary text still comes back whole.

What a caller re-cuts without a grade is a collapse fixed point, and the splitter makes it
so rather than claiming it.
A text whose whitespace-free canonical form holds no key cannot
produce one under any later cut, because closing a boundary's whitespace is the most a cut can
do to bring two characters together, and no substring of a clean collapse holds a key either.
That is the promise a re-cutting caller relies on, and it was previously asserted rather than
established: measured on the previous head, the delivery fallback returned text whose collapse
still read as a key. _made_collapse_clean now establishes it in three steps, least destructive
first -- text already clean is returned byte for byte, then the canonical form is redacted, and
only where a key is named by no span and found by no markup reading does the whitespace itself
go. The splitter applies it to its FINAL fallback, the answer reached when the budget search
fails, and the delivery helper applies it to its own fallback. The successful search path is
untouched and still returns every character, because those chunks are re-cut by callers that
grade what they cut; a gate placed before the search was measured to strip the spaces out of an
ordinary reply, which is why it sits at the fallback.

A repair is made on the text its caller HOLDS, never on a rebuilt sequence. The chunk list is
graded and never reassembled, because its concatenation is not the reply: sealing a chunk outside
a fence trims the whitespace that ended it, so a join deletes a break at every seam, and a fence
spanning a seam contributes its synthetic closer while the next chunk opens with the reopener, and
the two meet as one longer delimiter run that reads as a NEW fence -- after which the rest of the
message renders as code. Measured on the previous head: four chunks of a five-break source joined
to two breaks, and a fenced body whose source held only three-backtick runs produced a six-backtick
one. So each chunk is its own repair subject, the boundaries between chunks stay the splitter's own
where the whole source was available, and the one seam no splitter call covers -- Telegram's, between
two blocks it cut separately -- is repaired from the body that function holds.

Webex was the one path that kept the grade on the loop thread. Its final answer calls the
byte splitter directly inside on_done, and with no cut clean the search and the span repair
hold that thread for seconds -- long enough for the liveness watchdog to exit the gateway and
take every other channel's in-flight turn with it. It now offloads like the others. A
test walks all seven paths and fails if any of them stops handing its splitter call to a
thread, and Telegram carries a stricter version of that check described below.

The span repair is also bounded now. Its passes were capped only by the number of whitespace
runs in the text, and each pass costs a logarithmic number of whole-text redactions, so a
reply whose every break split a key-shaped token would cost thousands of them and the send
would never finish. The cap is the same constant the budget search uses, and past it the
answer is to decline to cut, which gives up nothing.

The dashboard mirror's backfill composed its units inline on the loop thread. An imported
history row carries no size cap, the grade redacts and rescans each candidate boundary, and
the liveness watchdog exits the process after seconds of loop silence -- so one large row
could terminate the gateway. The units are now composed inside asyncio.to_thread, the same
offload the Slack twin already uses and this handler already uses for its selection read.

Telegram was the remaining path whose cut chose boundaries on length alone. It rotates: a
segment that outgrows one message is split, every chunk but the last is sealed as its own
message, and the redaction runs per segment -- so the halves of a severed key are redacted in
neither message while the reader scrolling the two reads it whole. Reproduced at the channel's
own 400-character budget floor: the plain cut hands the key over, the credential-aware cut
moves the boundary instead and every character still ships.

Three things that leg needed beyond the argument.

It asks for no prefix-stable mode, and the reason is a structural difference from the
WhatsApp streaming leg: a rotation REPLACES the buffer with the retained tail, so text
already sealed is never part of a later cut. That is the bookkeeping whose absence is the
residual below, and Telegram has it.

Its rendered budget is enforced by measuring the real HTML and shrinking the source budget,
which means the cut is made more than once; each candidate answer now runs the shared bound,
so a declined cut cannot reach the client's own truncation with the reply's tail inside it.

Its table-aware path cuts each block on its own, so a boundary BETWEEN two blocks belongs to
no single cut and was graded by none of them. The assembled sequence is graded once at the end,
against the body that function holds. Every block is also redacted BEFORE it enters the list,
which the grade requires: the sequence reading is sound only on chunks already a fixed point of
its own scan, and a table run bypasses the splitter, so without it a cell holding a whole
credential fires the repair for something no seam severed -- measured on the previous head, that
cost the message its **bold** and its `code` and collapsed the table and the prose into
one message. The reachable case is a table whose last cell ends its line followed by prose, and a
fixture pins both halves of it: the per-block pair is shown to hand the key over, the
function's answer is shown not to.

Four call sites on that channel run the cut from a coroutine, and a scan on the gateway's
event loop is what the liveness watchdog exits the process for. All four now hand their work
to a thread. The file-wide check here is stricter than the one the other six paths get: "one
offload exists somewhere in the file" is satisfied while three sites stay on the loop, so the
Telegram check walks every coroutine's own body -- nested closures excluded, since those are
themselves what gets offloaded -- and fails if any of them calls a scanning splitter directly.

The redaction-tag tally on WhatsApp's finalization is fixed rather than deferred, which the
last round recorded as advisory. That pass repairs the unsealed tail and sends the repair, but
the notice counted the pre-repair chunk list, so a reply whose only placeholder came from the
repair announced none. The branch now re-points the list the notice reads at the sequence it
actually sends.

What tests we did

test/test_split_credential_boundary.py, 126 passed and 5 skipped, every run scoped with
an explicit -n0:

  • the seam, measured rather than assumed: the fixture's cut is shown to fall between the
    halves of a key, the two pieces are shown to scan clean individually, and the joined
    sequence is shown to hold the key;
  • a key spanning three and four chunks, where every neighbouring pair reads clean, on both
    splitters and across five narrow budgets;
  • a chunk that renders to nothing placed between the halves by the lossless splitter;
  • a key inside a link, which the canonicalised join loses to the label and only the
    per-side reading sees;
  • the dense probe: a budget off the doubling ladder is reached, asserted on the probe order;
  • the terminal, on text no budget can cut safely: the prose around the span survives byte
    for byte, every break and delimiter is kept, two keys keep the paragraph between them,
    the fully-collapsed form of the result reads clean, and the wider flush is shown to keep
    neither -- plus the end-to-end case through the byte splitter, where all seven breaks
    survive;
  • the post-bounding grade: a blind slice is shown to reopen the seam, the repair fires, a
    clean sequence needs none, and the repair stays safe when re-cut at five caps;
  • structural checks that the Slack path runs that grade and that the mirror composes its
    units in a thread;
  • prefix stability, which is the streamed-reply case: the searching path is shown to revise
    an already-sealed chunk after a single appended character, the stable path is shown not to
    across seven append lengths, and the stable path is shown to still redact;
  • structural checks that the streaming renderer asks for the stable split, grades its seam
    and repairs its tail, and that no global whitespace flush remains in the module;
  • all seven delivery paths, asserting each hands its splitter call to a thread, so none of
    them can put the grade back on the event loop, plus the stricter Telegram check that no
    coroutine in that file calls a scanning splitter directly;
  • the repair loop's bound, asserting the constant is in force;
  • the hard cap on all four capped paths: none of them sends the splitter's answer unbounded,
    no chunk exceeds the byte cap, and ordinary text is returned whole;
  • a SINGLE oversized chunk, the shape a one-element grade skips: it is cut inside the budget,
    every character still ships, a key across one of the new boundaries is still caught, and
    text already inside the budget is returned untouched;
  • the seam's edge whitespace: the stored reading is shown to approve a tab-separated pair
    the reader sees whole, the rendered reading to refuse it, and the two grades to agree;
  • Telegram's rotation at its real budget floor: the plain cut is shown to hand the reader
    the key, the credential-aware cut to move the boundary instead, every character to still
    ship, the bounded splitter to carry the same guard, a declined cut to stay inside the
    budget, and ordinary prose to come back unchanged;
  • Telegram's table-aware seam, with its own control: the per-block sequence is shown to
    rejoin the key and the function's answer is shown not to;
  • the sealed prefix, end to end: the sealed chunk is shown to be clean on its own, the pair to
    hand the reader the key, the offset to cover only the completing span, the repair to close the
    seam and to keep every character after it, and the rotation itself to stop handing it over;
  • that the repair is made on the source and not on the join: a source whose join loses breaks
    keeps them in the repair, and both its paragraph breaks survive;
  • the bound's own grade, on a blind slice INSIDE one chunk, which is the boundary no splitter
    call saw;
  • Telegram's table run: the key never reaches the screen, the message keeps its markup, and the
    table still seals as its own message;
  • structural checks that both streaming legs offload the repair and record what they sent, that
    each of WhatsApp's two send paths does so rather than one, and that the notice counts the
    sequence that shipped;
  • the collapse guarantee: the splitter's fallback and the delivery fallback are each shown to be
    a collapse fixed point on text where no span names the key, and the successful search path is
    shown to lose no character;
  • an enumeration of whitespace rewrites, asserting all three sit in named helpers, so a fourth
    way for a valid span to lose its formatting fails here;
  • the sealing sinks: both are shown to run the seam helper, the record is shown to have exactly
    one writer, and no sealing caller is allowed to carry a seam rule of its own;
  • WhatsApp's own send path: the splitter alone is shown to decline and answer over the cap, the
    channel path to bound it with no key on screen, the stable split to keep the budget's
    boundaries, and ordinary prose to come back as its converted form exactly;
  • one channel end to end, through the function WhatsApp's client sends from, asserting on
    the frames it would deliver;
  • backward compatibility: prose and a table row are returned byte for byte with a redactor
    passed, so existing callers are unaffected.

Mutation proof, thirty ways, each against a pristine copy kept outside the repo and each
verified by hash to have applied and been restored, with a passing control before and after.
Seven cover this round: taking the collapse fix off the splitter's fallback fails 1, gating the
guarantee on the stored reading instead of the rendered one fails 1, letting the delivery fallback
assert the promise again fails 1, dropping the seam repair from either sink fails 1 each, stopping
the predecessor record fails 1, and removing WhatsApp's bound fails 1. The third and the gating one
each caught a weak test first: the gate measured clean on a key written in code spans, which is
exactly the shape the guarantee exists for.
Eight cover this round: rebuilding the repair from the join fails 1, making the sent-tail offset
always answer clean fails 4, grading the flattened sequence again in the shared bound fails 1 and
in Slack fails 1, dropping Telegram's repair against its sealed predecessor fails 1, letting its
table blocks in unredacted fails 2, and dropping WhatsApp's repair fails 1 while dropping its
record fails 1. Three of those measured weak tests first and are the reason for three of the new
cases: the shared bound, WhatsApp's repair and WhatsApp's record each failed NOTHING under a
structural check that only asked whether the file mentioned the guard somewhere. Three cover the streaming and terminal
work: stopping the renderer from asking for the stable split fails 1, making the stable mode
fall back to the search fails 2, and sealing without grading the seam fails 1. Four cover the
grade and the repair. Dropping the redactor from one of the
seven call sites fails the enumeration test. Replacing the dense probe with a doubling walk
fails the two probe-order cases. Sending the terminal straight to the wider flush fails the
end-to-end case, where the breaks drop from seven to zero. Widening the repair's span to the
whole text fails four cases, including the two-key one. A fifth mutation, dropping one of the
grade's two readings, initially failed NOTHING -- that gap is what the link case was added
for, and it now fails.

Regression on the rebased tree, each file scoped with -n0: test_whatsapp.py 26 passed,
test_telegram_parity.py 360 passed, test_teams_renderer.py 22 passed,
test_wecom_renderer.py 33 passed, test_telegram.py 331 passed,
test_telegram_parity.py 360 passed, test_options_cap_contract.py 71 passed,
test_channel_control_tag_strip.py 39 passed, test_credential_redaction_notice.py
27 passed, test_redaction_mirror_parity.py 10 passed, test_whatsapp.py 26 passed,
test_messaging_split.py 138 passed,
test_channel_table_rendering.py 82 passed (the reassembly invariant the byte splitter
documents), test_chat_mirror.py 82 passed, test_display_split_safety.py 50 passed,
test_whatsapp_renderer.py 46 passed, test_whatsapp_turn_renderer.py 39 passed,
test_security_posture.py 53 passed, test_slack_renderer.py 73 passed,
test_webex_renderer.py 34 passed. Gates: black, isort, flake8, mypy on the changed files,
and the comment-history gate, all clean.

messaging/split.py is listed in NON_EGRESS_REDACTION_MODULES with the reason the
display-safety module gives: it applies a redactor its caller injects, owns no output, and
the modules calling it are the registered sinks.

The fixture uses the AWS-documented example key the repo's own display-safety tests
already use, so no new credential shape is introduced.

Any other suggestions on the work

This does NOT close the issue, and the issue stays open on purpose. Five of the six delivery
paths it names are closed end to end, and Telegram -- a seventh path beyond the six, which
seals splitter chunks as separate messages the same way -- is closed here too: every boundary they choose is graded, and the
enumeration test fails if any of them stops passing a redactor. The sixth, WhatsApp's
streaming leg, keeps the graded cut but carries a residual that is present on current main
independently of this change -- that leg re-renders a growing reply on every frame while
treating earlier messages as delivered, so a delivery tracked by a count cannot notice when
an earlier message would be rendered differently. Closing it properly means grading a
boundary against what is already on screen, which is the channel-neutral work in #13494, not
something this change can reach. The residual is recorded on #13743 rather than a duplicate
issue. What it does NOT add is predecessor bookkeeping at each point a message stops
being editable -- a streaming leg that has already delivered a message cannot re-cut it, so
that piece has to be graded against what is already on screen. The channel-neutral
primitives for that are in #13494 and are not on main yet, so this change reuses only what
main already has and touches none of the files that PR owns. That bookkeeping belongs to a
follow-up once #13494 lands, not to the six paths this issue is about.

Two things found while measuring, worth recording. The splitter has callers beyond the six
paths named in the issue: discord/renderer.py (five calls), telegram/renderer.py,
feishu/client.py, messaging/renderer.py and teams/session_resume.py. Discord seals
splitter chunks as separate messages and carries the same open seam; that file belongs to
#13494, which is rewriting it with the predecessor primitives, so it is stage-two scope here
by construction and this PR does not touch it. Telegram carried the same seam and is closed
here: its rotation discards sealed text from the buffer, so it needs none of the predecessor
primitives and can be fixed with what main already has.
Feishu, the shared renderer and the Teams resume path are unchanged, and none of them
belongs to #13494. Their own code answers whether their chunks become separate messages
rather than leaving it open: feishu/client.py sends every chunk as its own reply to the
same inbound anchor, and chunk_for_transport hands its parts to callers that send them
one message at a time. Neither passes a redactor, so the seam is open on those legs too.
They sit outside this stage by scope, not by another PR's ownership. And the issue's per-file grade was measured by grepping for the new primitives'
names, which cannot see that five of the six paths already close the mid-line cut by
redacting the whole buffer first; the line-boundary seam is what was actually open at all
of them.

Two advisory findings are recorded rather than fixed. The spec below is one. The other asks that
the enumeration of delivery paths be DERIVED by walking every module rather than curated, and that
Discord and Feishu then pass a redactor too; the reviewer marks it advisory itself, on the grounds
that its remedy lands in files this change does not touch, and that is the disposition taken here.
The tally finding from the previous round is fixed: the notice now counts the sequence that
shipped.

One review finding is recorded rather than fixed. The owning spec,
docs/system-specs/modules/messaging.md, still calls the Webex splitter lossless and the
character splitter free of modes, and the redactor ladder falsifies both. That file belongs
to #13494, which this change deliberately does not touch, so the correction travels with the
follow-up that lands after it rather than creating a conflict here.

Pattern harvest

Rule candidate: enumeration test, added in this PR as
test_split_credential_boundary.py::TestEveryDeliveryPathIsEnumerated.

Pattern: a text is cut on a length budget and then scanned one piece at a time, so the scan
runs after the boundary is chosen and cannot see what the cut separated.

The shape generalises past credentials: any check that reads one piece of a text the caller
will deliver in several is blind to whatever the boundary split. It resists a purely
syntactic rule, because "these chunks each become their own message" is not visible at the
call site -- which is why the enforceable half is an enumeration: the delivery paths are
listed, and every splitter call in them must pass a redactor. The list is checked in both
directions, so a path that stops reaching a splitter fails rather than passing by having
nothing left to check, and the AST pass counts the offloaded asyncio.to_thread shape as
well as the direct call.

Refs #13743

Six delivery paths cut model text on a length budget and scan each piece
on its own. A key the cut severs matches nothing in either piece, so both
are reported clean, and the reader rejoins the halves reading one message
and then the next. The per-piece scan emits no notice, so the operator
sees a clean send.

Five of those paths already redact the whole buffer before splitting,
which closes a mid-line hard cut. It cannot close a cut at a line
boundary: sealing a chunk trims the whitespace that ended it, so the two
halves sit flush against each other on screen while the whole text still
holds the break between them, invisible to a scan of the text as written
and to a scan of either piece.

The shared splitters now take the redactor themselves. split_markdown_safe
and chunk_utf8_bytes reduce the text before any boundary is chosen: the
form the platform renders, and the form the seal's trim produces once
every break is collapsed. A credential visible in either is a marker
before a boundary can land inside it. The parameter stays optional, so a
caller whose chunks stay inside one message is unaffected, and an
enumeration test over the six delivery paths fails when any splitter call
there omits it.

Refs #13743
Grade the boundaries the ladder chooses, one at a time, rather than every
break at once. A scan of all of them together answers a wider question than
any cut asks: two lines that read as one key when glued would cost the message
every break and its markup, though no cut ever joins them. Wrapped content in a
fence is that shape.

The grade reads the RENDERED pair. A platform drops the whitespace at a
message's edges, so the last visible character of one message sits against the
first visible character of the next: neither a trailing newline nor a trailing
space separates anything. That is also what carries the guarantee to the byte
splitter, which stores the whitespace rather than trimming it -- lossless where
it is kept, invisible where it is shown.

The repair moves the CUT before it gives up any text. Budgets step back
exponentially and every candidate is verified, so a key straddling a boundary
ends up whole inside one chunk with every character and every break intact,
where the space between its halves is on screen and the scanner's own reading
of such text stands. Only when no budget at or below the caller's cuts safely
is the whitespace flushed and the text redacted -- every run, because the
whitespace a cut consumes can be a single space -- and that result is graded as
well, since a repair returned without a grade is a repair nobody checked.
Flushing every run is deliberately wider than one cut produces: that width is
what lets a caller bound the chunks again under its own platform limit without
a grade of its own.

The splitter applies a caller's redactor, so it is classified in
security_posture's NON_EGRESS_REDACTION_MODULES. It owns no output; the modules
that call it are the registered sinks. Same reason display_safety.py sits
there, and the posture drift guard fails without the entry.

Refs #13743
@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — 🟡 CONCERNS

Design-level review of 8352d83d946785f574ab5ddb56773b6a5829256c — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

Feishu and chunk_for_transport legs keep the exact seam this PR closes, and the curated enumeration test cannot fail toward closing them.

Watch

  • The description concedes "the seam is open on those legs too" for feishu/client.py and the shared chunk_for_transport callers, and TestEveryDeliveryPathIsEnumerated pins only the curated seven — so a credential rejoined across Feishu messages ships silently forever, with no test, no owner, and no tracked follow-up beyond "the issue stays open." The derived-enumeration remedy was itself recorded as advisory.
    Clears when: a follow-up (issue or PR) is filed that passes a redactor on the Feishu and shared-renderer legs, or the enumeration is derived by walking delivery modules.
  • docs/system-specs/modules/messaging.md is now knowingly false — the PR admits it "still calls the Webex splitter lossless and the character splitter free of modes" — deferred past fix(channels): grade the rotation boundary the splitter cannot see #13494, against the same-commit spec rule. A caller trusting documented chunk_utf8_bytes losslessness under a redactor builds on a broken contract in the interim.
    Clears when: the spec correction lands (with the fix(channels): grade the rotation boundary the splitter cannot see #13494 follow-up, or sooner if fix(channels): grade the rotation boundary the splitter cannot see #13494 stalls).

[DESIGN-REVIEWED] 8352d83

@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — 🟡 CONCERNS

Premise-level review of 8352d83d946785f574ab5ddb56773b6a5829256c — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

Discord and Feishu run the identical budget-blind per-message split this PR fixes on seven other paths, and Slack respells the PR's own bounded_for_delivery inline.

Not justified as shipped

  • Item 1 — point patch, 2 unfixed siblings: the description opens "Six channel delivery paths cut model text on a length budget", but grepping split_markdown_safe|chunk_utf8_bytes finds discord/renderer.py (5 call sites — the rotation seals chunks[:-1] as their own messages, the multipart seal posts each slice) and feishu/client.py:213 (each chunk its own reply) passing no redactor and absent from the new DELIVERY_PATHS gate; both already redact the whole buffer first, which is exactly the state the description says fails at a line boundary.
    Clears when: discord/renderer.py and feishu/client.py pass the redactor and join DELIVERY_PATHS, or a linked issue records the deferral.
  • Item 5 — duplicate of messaging/split.py:bounded_for_delivery: slack/renderer.py's _bounded hand-rolls cut → repaired_for_delivery → re-cut per chunk, the exact loop this same diff ships as bounded_for_delivery (counted: 6 paths call the helper, Slack alone respells it).
    Subtraction: replace _bounded's inner loop with bounded_for_delivery(chunks, SLACK_MSG_LIMIT - reserve, _redact_all, chunk_text).
    Clears when: the inline loop is deleted in favour of the shared helper.

What this change ships

Inventory (10 items) — 7 justified

Intent: stop a message-length cut from handing the reader a credential neither adjacent message holds, and the silent clean-send audit record with it — a FIX.

  1. A severed key no longer reads whole across adjacent messages on seven delivery paths — point patch, 2 unfixed siblings (Discord, Feishu)
  2. When no budget cuts safely the splitter declines to cut; hard-capped transports re-bound the oversized answer — justified
  3. Streaming channels give up the span of a new message that completes a key already sent (Telegram, WhatsApp) — justified
  4. WhatsApp holds a chunk in the live bubble until its boundary grades clean — justified
  5. Slack re-implements the shared re-bound loop inline — duplicate of messaging/split.py:bounded_for_delivery
  6. Splitting/grading now runs off the event loop on six paths — rides along, derived from the liveness watchdog
  7. Telegram table cells are redacted before blocks assemble — justified
  8. WhatsApp's redaction notice counts repair-introduced placeholders — justified
  9. messaging/split.py registered as a non-egress redaction module — justified
  10. A test gate fails any enumerated delivery path whose splitter call lacks the guard — justified

[FIRST-PRINCIPLES-REVIEWED] 8352d83

@github-actions

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — 🔴 changes requested (blocking)

GPT 5.6 found at least one blocking issue that must be resolved before merging 8352d83d946785f574ab5ddb56773b6a5829256c. 3 of 3 blocking finding(s) are security-class and were withheld from adjudication, so the blocking verdict stands.

This comment is updated in place on each push.

BLOCKING -- src/kiro_crew/whatsapp/turn_renderer.py:274 -- sealing restores a credential repaired in the live bubble
await self._seal_chunk(chunks[index])
Sealed prefix + repaired live suffix + another chunk -> original suffix replaces the repair -> credential becomes visible.
Anchor: residual/security
Fix: Seal and record the suffix repaired against _sent_tail, not the original chunk.

BLOCKING -- src/kiro_crew/messaging/split.py:579 -- terminal repair corrupts unrelated formatting
return [_made_collapse_clean(safe, redactor)]
Long prose containing repeated markup-split keys -> no safe budget or literal span -> all whitespace and markup are collapsed.
Anchor: residual/crash-data-loss-corruption
Fix: Repair only the credential-bearing rendered span; never collapse unrelated text.

BLOCKING -- src/kiro_crew/telegram/renderer.py:1806 -- failed deliveries become seam predecessors
self._sent_tail = sealed
All send/edit fallbacks fail -> unsent text is recorded -> the next delivered message loses a leading credential-shaped span.
Anchor: residual/crash-data-loss-corruption
Fix: Record _sent_tail only after a delivery path confirms success.

FINDING -- src/kiro_crew/teams/renderer.py:386 -- boundary repair can add placeholders while Teams, Slack, Webex, and WeCom tally their pre-split source, so the promised redaction notice is omitted -> Fix: tally the chunks actually delivered on each changed path.

[BLOCK-MERGE] 8352d83
[GPT-REVIEWED] 8352d83

Adjudication (Opus 5) — is blocking on each finding proportionate?

F1 — src/kiro_crew/whatsapp/turn_renderer.py:274
UNBOUNDED (credential exposure). Confirmed: the seal loop grades rendered[index] only against text after it (turn_renderer.py:267-272) and never against _sent_tail, so the c0|c1 seam repair computed at turn_renderer.py:288-291 is applied to the live tail only and is never written back to _buf; the next frame seals the un-repaired chunks[index] (274) and records the un-repaired rendered[index] (275). Recovery: none — _seal_chunk (390-410) posts through _show and a sent message cannot be recalled.
Real fix: seal the repaired suffix and record that as _sent_tail — one substitution in an existing two-line span, no new state or ordering contract; far below the harm.

F2 — src/kiro_crew/messaging/split.py:579
Corruption of the whole delivered reply. Confirmed: reached only when _under_a_safe_budget exhausts the dense-then-exponential walk (529-536) and _redact_only_the_rejoined_span returns None (577), whose own None paths include the _DENSE_PROBES pass cap (361, 371) — i.e. text with >128 whitespace-split key-shaped tokens, which model or tool output can produce. _made_collapse_clean's third step then strips every whitespace run in the message (291). Recovery: none in-band; the collapsed text is what ships.
The rarity argument cannot be completed: the conditions are independent, not contradictory, and the code's own step-4 contract (546-571) promises "the text goes back whole", not a whole-message collapse, so no pre-drafted acceptance is available.

F3 — src/kiro_crew/telegram/renderer.py:1806
Visible, fail-closed over-redaction bounded to one turn. Conditions confirmed: _seam_safe records the predecessor before any transport call (1806, invoked at 1896 ahead of the sends at 1920-1990); every fallback must fail — rich send None (1929), HTML edit and plaintext edit both false (1953-1967), HTML send None (1984), plaintext send None (1985-1990); and the next sealed segment's leading text must complete a key with that undelivered tail (split.py:441). Recovery/bounds: the bad predecessor is overwritten by the next seal (1806) and is per-renderer-instance state (1220), so it cannot outlive the turn; the loss is marked with a visible tag (messaging/renderer.py:437) and the direction is strictly stricter, never an exposure.
Real fix: thread delivery success from _seal_text's five exits back into the off-loop _seam_safe, inverting the sink-owns-the-seam centralization stated at 1888-1895.

[ADJUDICATION] 8352d83 total=0 uphold=0 downgrade=0
[GPT-ADJUDICATED] 8352d83
[ADJUDICATION-FENCED] 8352d83 fenced=3 flagged=1
UPHOLD-FENCED F1 src/kiro_crew/whatsapp/turn_renderer.py:274 -- The seal path replaces the repaired live suffix with the original chunk and the loop's forward-only grade never re-checks _sent_tail, so an ordinary multi-chunk reply puts a credential whole on screen across two unrecallable messages.
UPHOLD-FENCED F2 src/kiro_crew/messaging/split.py:579 -- The terminal collapse is reached by independent, producible conditions (no safe budget plus the span repair's pass cap) and destroys every space and break in the message, which the function's own documented step 4 does not authorize.
FLAG F3 src/kiro_crew/telegram/renderer.py:1806 -- Override rationale: reaching this needs four transport fallbacks to all fail on one segment and then recover for the next, conjoined with a credential-shaped seam on exactly that next segment; the outcome is a visible, fail-closed leading-span redaction that cannot outlive the turn (_sent_tail is per-instance at 1220 and overwritten at the next seal) and is strictly smaller than the delivery loss that caused it, while the fix inverts the sink-owns-the-seam contract stated at 1888-1895.
[GPT-ADJUDICATED-FENCED] 8352d83

🏷️ Fenced finding(s) machine-flagged as likely edge case

The security fence keeps these findings blocking regardless of adjudication; the only clearance path is a human override. The annotate-only pass judged the condition combination each one requires extreme, and pre-drafted the override rationale below. A repository writer must independently verify a rationale before posting it — it is machine-authored, and a wrong override on a security-class finding ships exactly the class the fence exists to stop.

  • F3 src/kiro_crew/telegram/renderer.py:1806 — Override rationale: reaching this needs four transport fallbacks to all fail on one segment and then recover for the next, conjoined with a credential-shaped seam on exactly that next segment; the outcome is a visible, fail-closed leading-span redaction that cannot outlive the turn (_sent_tail is per-instance at 1220 and overwritten at the next seal) and is strictly smaller than the delivery loss that caused it, while the fix inverts the sink-owns-the-seam contract stated at 1888-1895.

    /ai-review override gpt 8352d83d946785f574ab5ddb56773b6a5829256c: Override rationale: reaching this needs four transport fallbacks to all fail on one segment and then recover for the next, conjoined with a credential-shaped seam on exactly that next segment; the outcome is a visible, fail-closed leading-span redaction that cannot outlive the turn (_sent_tail is per-instance at 1220 and overwritten at the next seal) and is strictly smaller than the delivery loss that caused it, while the fix inverts the sink-owns-the-seam contract stated at 1888-1895.
    

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 8352d83d946785f574ab5ddb56773b6a5829256c: <one-sentence reason>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 25, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Opus 5 Review — 🔴 changes requested (blocking)

Reviewed 8352d83d946785f574ab5ddb56773b6a5829256c — this comment is updated in place on each push.

BLOCKING — the WhatsApp seal loop seals a chunk without grading it against the message already sent, so a credential split across the 4096-char boundary reaches two adjacent messages whole.

BLOCKING — src/kiro_crew/whatsapp/turn_renderer.py:274

            await self._seal_chunk(chunks[index])
            self._sent_tail = rendered[index]

A streamed reply whose chunk boundary lands between …AKIA (line end) and IOSFODNN7EXAMPLE… — AWS_KEY_ID is (?:AKIA|ASIA)[A-Z0-9]{16} contiguous, so the whole-body redact_for_display in the stable=True path never sees it → an early frame where the tail chunk is still shorter than the 16-char completion passes joins_to_a_credential(rendered[0], forward) and chunk 0 seals → once the reply grows past two chunks, the loop seals chunks[1] raw, and repaired_after_a_sent_tail(self._sent_tail, …) runs only on chunks[-1] (line 288), so the c0|c1 seam is graded by nothing and on_done starts at _sealed_count, past it.
Fix: in the loop, repaired = await asyncio.to_thread(repaired_after_a_sent_tail, self._sent_tail, chunks[index], _redact_all), seal repaired or chunks[index], and record _sent_tail from what was actually sealed.

FINDING — src/kiro_crew/telegram/renderer.py:1896 — text = await asyncio.to_thread(self._seam_safe, text) records _sent_tail for the whole segment, but _seal_without_rich then re-splits it and each _seal_chunk_html overwrites the record, so after the degraded-table path _sent_tail is chunks[-2] while the message the reader is looking at is chunks[-1] (which itself never passes _seam_safe) — the next segment's seam is graded against the wrong predecessor → Fix: re-record _sent_tail from the tail _seal_without_rich returns, or move the _seam_safe call after the re-split chooses the final text.

FINDING — src/kiro_crew/whatsapp/turn_renderer.py:273 — the comment says "A chunk that does not pass stays in the live bubble, where a later frame may still revise it", but the refused boundary does return before tail = chunks[-1] / _show(tail), and because stable=True pins the boundary every later frame returns at the same point, so the bubble keeps whatever stale prefix the previous frame put there and live output stops for the rest of the turn → Fix: correct the comment to say the frame is dropped and the text waits for on_done.

FINDING — src/kiro_crew/security_posture.py:1708 — the new NON_EGRESS_REDACTION_MODULES justification says the splitter "redacts the text it hands back only when no budget cuts it safely", but _cut_where_no_key_rejoins runs safe = redact_for_display(text, redactor)[0] unconditionally and cuts every returned chunk from safe, so an auditor reading the registry is told the module is pass-through in the common case when it is not → Fix: state that the splitter redacts against the rendered form before every cut and gives up whitespace or markup only when no budget cuts safely.

[BLOCK-MERGE] 8352d83
[OPUS-REVIEWED] 8352d83

Verdict parsed from the review's SHA-scoped output markers for commit 8352d83d946785f574ab5ddb56773b6a5829256c.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 8352d83d946785f574ab5ddb56773b6a5829256c: <one-sentence reason>

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

Superseded by #13984. The replacement was independently read back at exact head 40418ee: two commits on current main, matching file set, no closing reference for #13743, and a clean matching local/remote tree. This PR is closed unmerged; no approval or merge was performed.

@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Sep 25, 2026
@bolichen97
bolichen97 deleted the fix/cross-delivery-redaction-13743-v11 branch November 6, 2026 07:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant