Repository navigation
fix(channels): grade the delivered sequence so a split cannot rejoin a key - #13852
chenmingwei23 wants to merge 2 commits into
Conversation
Six delivery paths cut model text on a length budget and scan each piece on its own. A key the cut severs matches nothing in either piece, so both are reported clean, and the reader rejoins the halves reading one message and then the next. The per-piece scan emits no notice, so the operator sees a clean send. Five of those paths already redact the whole buffer before splitting, which closes a mid-line hard cut. It cannot close a cut at a line boundary: sealing a chunk trims the whitespace that ended it, so the two halves sit flush against each other on screen while the whole text still holds the break between them, invisible to a scan of the text as written and to a scan of either piece. The shared splitters now take the redactor themselves. split_markdown_safe and chunk_utf8_bytes reduce the text before any boundary is chosen: the form the platform renders, and the form the seal's trim produces once every break is collapsed. A credential visible in either is a marker before a boundary can land inside it. The parameter stays optional, so a caller whose chunks stay inside one message is unaffected, and an enumeration test over the six delivery paths fails when any splitter call there omits it. Refs #13743
Grade the boundaries the ladder chooses, one at a time, rather than every break at once. A scan of all of them together answers a wider question than any cut asks: two lines that read as one key when glued would cost the message every break and its markup, though no cut ever joins them. Wrapped content in a fence is that shape. The grade reads the RENDERED pair. A platform drops the whitespace at a message's edges, so the last visible character of one message sits against the first visible character of the next: neither a trailing newline nor a trailing space separates anything. That is also what carries the guarantee to the byte splitter, which stores the whitespace rather than trimming it -- lossless where it is kept, invisible where it is shown. The repair moves the CUT before it gives up any text. Budgets step back exponentially and every candidate is verified, so a key straddling a boundary ends up whole inside one chunk with every character and every break intact, where the space between its halves is on screen and the scanner's own reading of such text stands. Only when no budget at or below the caller's cuts safely is the whitespace flushed and the text redacted -- every run, because the whitespace a cut consumes can be a single space -- and that result is graded as well, since a repair returned without a grade is a repair nobody checked. Flushing every run is deliberately wider than one cut produces: that width is what lets a caller bound the chunks again under its own platform limit without a grade of its own. The splitter applies a caller's redactor, so it is classified in security_posture's NON_EGRESS_REDACTION_MODULES. It owns no output; the modules that call it are the registered sinks. Same reason display_safety.py sits there, and the posture drift guard fails without the entry. Refs #13743
Design Review (Fable 5) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS Three delivery legs the PR itself names — Feishu, Watch
Suggestions
[DESIGN-REVIEWED] e1984a1 |
First Principles Review (Fable 5) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS The test claims to enumerate "every path that turns one chunk into one message," yet Discord, Telegram, and Feishu deliver splitter chunks as messages with no redactor. Not justified as shipped
What this change shipsInventory (10 items) — 9 justifiedIntent: stop a length-budget cut from delivering a credential as two clean-scanning message halves a reader rejoins on screen — a FIX.
[FIRST-PRINCIPLES-REVIEWED] e1984a1 |
GPT 5.6 Review — 🔴 changes requested (blocking)GPT 5.6 found at least one blocking issue that must be resolved before merging This comment is updated in place on each push. BLOCKING -- src/kiro_crew/whatsapp/turn_renderer.py:257 -- Streaming can seal an incomplete credential prefix BLOCKING -- src/kiro_crew/teams/renderer.py:382, src/kiro_crew/wecom/renderer.py:305, src/kiro_crew/dashboard/chat_mirror.py:588 -- Whole fallback is truncated after its final security scan (origin: validation) [BLOCK-MERGE] e1984a1 Adjudication (Opus 5) — is blocking on each finding proportionate?F1 — src/kiro_crew/whatsapp/turn_renderer.py:257 F2 — src/kiro_crew/teams/renderer.py:382, src/kiro_crew/wecom/renderer.py:305, src/kiro_crew/dashboard/chat_mirror.py:588 [ADJUDICATION] e1984a1 total=0 uphold=0 downgrade=0 False positive or not applicable? A repository writer can comment: |
Opus 5 Review — ✅ no blocking findingsReviewed Review detailsNo blocking defect, but the new seam guard has reachable holes on WhatsApp, Webex and three ungated channels. FINDING — src/kiro_crew/whatsapp/turn_renderer.py:257 — the seam is graded against only the text that has arrived, so a frame where FINDING — test/test_split_credential_boundary.py:47 — FINDING — src/kiro_crew/webex/renderer.py:132 — FINDING — src/kiro_crew/whatsapp/turn_renderer.py:567 — FINDING — src/kiro_crew/whatsapp/turn_renderer.py:258 — the refusal FINDING — src/kiro_crew/whatsapp/turn_renderer.py:541 — [OPUS-REVIEWED] e1984a1 Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
What is the problem?
Six channel delivery paths cut model text on a length budget and scan each piece on its
own. A credential the cut severs matches nothing in either piece, so both are reported
clean and the reader rejoins the halves reading one message and then the next.
Five of the six already redact the whole buffer before splitting, and that closes a cut
that lands mid-line: the credential is one marker before any boundary can reach it. It
does not close a cut at a line boundary. Sealing a chunk trims the whitespace that
ended it, and the next chunk starts at the first character after that whitespace, so the
two halves end up flush against each other on screen while the whole text still holds the
break between them. That form is invisible to a scan of the text as written and to a scan
of either piece.
Why this issue matters to the user
The reader sees the key and nothing reports it. Because each piece scans clean, no
redaction notice is emitted and the delivery is recorded as a clean send: the operator has
no signal at all. This is the audit half of the problem, not only the disclosure half.
How our fix solves it
The boundary is where the defect lives, so the fix goes into the two shared splitters
rather than into six renderers.
split_markdown_safeandchunk_utf8_bytestake anoptional
redactorand, when given one, reduce the text before any boundary is chosen:redact_for_display, which closes a cut landing mid-line and the markup-hidden variant;the whitespace at a message's edges, so the last visible character of one message sits
against the first visible character of the next and a trailing newline separates
nothing. Stripping every chunk's edges before the grade is what makes the predicate see
what a reader sees, and it is what carries the guarantee to the byte splitter, which
stores the break rather than trimming it.
The whole sequence, not each neighbouring pair, because a key needs no more than a narrow
budget to span three chunks: split a twenty-character key over four and every pair holds a
fragment matching nothing while the screen holds the key whole. The same gap swallows a
chunk that renders to nothing at all, which the lossless byte splitter can place between
the halves. A fragment is only ever part of a key, so no pair of neighbours can be asked
about it. Both readings
joins_to_a_credentialnames are kept, and each is scanned onceover the joined text rather than per boundary, which is what keeps the search affordable --
measured at 10.5 ms against 160 ms on a 74 KB reply, so the ordinary path is 15 times
cheaper than a per-boundary grade.
When a boundary does rejoin a key the cut is MOVED before any text is given up. Candidate
budgets step back one at a time for the first 128 and exponentially after that; the dense
prefix is what stops a budget clean at 6997 being stepped over by a walk that jumps 4, 8,
16, and the exponential tail keeps the reach without a probe per byte. Worst case 1.49 s on
a 74 KB reply, below the 2.2 s a purely exponential walk cost with the old grade.
A STREAMING caller cannot use that search at all, and this is the sharpest thing the change
had to get right. Such a caller re-splits its growing body every frame and treats all but
the last chunk as delivered, which
whatsapp/turn_renderer.pystates as its ownrequirement: chunk i never changes once chunk i+1 exists. A search that reads the whole
body breaks it -- appending ONE character moved a boundary under an already-sent message in
a measured case, and the final pass slices the new list from the delivered count, so the
span is duplicated or dropped with no way back. That leg therefore takes a prefix-stable
split: the text is redacted, and every boundary stays where the caller's own budget puts
it. The same search space that produced a revision produces none under it.
Prefix stability says nothing about whether a BOUNDARY is safe, so the streaming renderer
grades each one against everything that follows before counting the chunk final, using
joins_to_a_credential, which main already has. That grade reads the RENDERED text, everychunk's edges stripped, for the same reason the splitter's own predicate strips them: a
platform drops the whitespace at a message's edges, so a tail beginning with a tab separates
nothing once two messages sit on screen. Asking about the characters as stored approves
exactly that pair, and the case is reachable -- the splitter never strips leading whitespace
and a hard cut can leave a remainder starting with one. Both grades now answer the same
question about the same boundary, and a test pins that they agree. A chunk that does not pass stays in the
live bubble, where a later frame may still revise it; at finalization every chunk must go,
so the still-unsealed tail is repaired through
repaired_for_deliveryrather than held.Only when no budget at or below the caller's cuts safely is text given up, and then only
the credential's own span. There is no wider fallback that strips a whole message: when no
span names the key, the splitter DECLINES TO CUT and returns the text whole. A boundary is
the only thing that rejoins a key across messages, so creating none is fail-closed, and it
costs the caller a chunk over its limit instead of costing the reader a reply stripped of
its whitespace and markup. A caller that bounds that chunk again has
repaired_for_deliveryto grade what it produced. The span is found by SHRINKING -- a span known to collapse intoa key is trimmed from each end by half its width, the trim kept while what remains still
collapses into one -- so a message carrying two keys keeps the paragraphs between them. The
whitespace inside that span is closed up, which makes the key contiguous for the ordinary
redaction, and the loop returns only once the form with EVERY run removed reads clean. That
last condition is the promise to a caller that re-cuts these chunks under its own limit, and
it is why one grade at the end of such a caller is enough. For a key only the rendering
assembles, which no span of the literal text names, the answer is the canonical form -- the
one-directional trade
redact_for_displayalready makes, giving up markup and keeping everyspace and every break.
Grading chosen boundaries rather than every candidate is what keeps ordinary wrapped content
intact -- two lines that read as one key when glued are never joined by a cut, so they are
left exactly as written, and a fixture pins that.
The parameter is optional because a caller whose chunks land inside one message severs
nothing a reader can rejoin across messages. It is not optional for the six paths that do
turn one chunk into one message: those are enumerated in the tests, and any splitter call
there that omits the redactor fails. The check walks the AST and counts both shapes,
including the four sites that hand the splitter to
asyncio.to_threadas a reference --a check reading direct calls only would see three of the seven sites and silently pass the
rest.
Per path, all seven call sites:
slack/renderer.py:875split_markdown_safe, offloaded_redact_allwebex/renderer.py:396chunk_utf8_byteswebex_display_safewhatsapp/renderer.py:302split_markdown_safeteams/renderer.py:383split_markdown_safe, offloaded_display_textteams/renderer.py:743split_markdown_safe, offloadedwecom/renderer.py:306split_markdown_safe, offloadeddashboard/chat_mirror.py:588split_markdown_safeWeCom's streaming push already picks a credential-safe offset for the frame it is about to
send, and that path is untouched.
Two legs needed more than the redactor argument, both found by review and both reproduced
before they were fixed.
Slack bounds the splitter's output AFTER it has been graded. A logical line placed whole
carries its fence scaffolding on top, so a chunk can exceed the limit, and
chunk_textthenblind-slices it and each slice is posted as its own message -- an ungraded boundary at the
very last step.
_split_for_slacknow grades that final sequence through the newrepaired_for_deliveryand repairs it when it rejoins a key. One grade suffices because therepair is only returned once its fully-collapsed form reads clean, which the tests check by
re-cutting the repaired text at five different caps.
Webex also has a HARD cap, and the splitter's fail-closed answer collides with it. When no
budget cuts without rejoining a key the splitter declines to cut and answers with the text
whole, which costs the caller a chunk over its budget; this client truncates a larger payload,
so the answer's tail would go with no notice. That path now grades the sequence once more
through
repaired_for_deliveryand cuts the repair back to the budget, which is sound becausethe repair is safe to bound again by contract. A test asserts no chunk it produces exceeds the
cap, and another asserts ordinary text still comes back whole.
Webex was the one path that kept the grade on the loop thread. Its final answer calls the
byte splitter directly inside
on_done, and with no cut clean the search and the span repairhold that thread for seconds -- long enough for the liveness watchdog to exit the gateway and
take every other channel's in-flight turn with it. It now offloads like the other five. A
test walks all six paths and fails if any of them stops handing its splitter call to a
thread.
The span repair is also bounded now. Its passes were capped only by the number of whitespace
runs in the text, and each pass costs a logarithmic number of whole-text redactions, so a
reply whose every break split a key-shaped token would cost thousands of them and the send
would never finish. The cap is the same constant the budget search uses, and past it the
answer is to decline to cut, which gives up nothing.
The dashboard mirror's backfill composed its units inline on the loop thread. An imported
history row carries no size cap, the grade redacts and rescans each candidate boundary, and
the liveness watchdog exits the process after seconds of loop silence -- so one large row
could terminate the gateway. The units are now composed inside
asyncio.to_thread, the sameoffload the Slack twin already uses and this handler already uses for its selection read.
What tests we did
test/test_split_credential_boundary.py, 82 cases, every run scoped with an explicit-n0:halves of a key, the two pieces are shown to scan clean individually, and the joined
sequence is shown to hold the key;
splitters and across five narrow budgets;
per-side reading sees;
for byte, every break and delimiter is kept, two keys keep the paragraph between them,
the fully-collapsed form of the result reads clean, and the wider flush is shown to keep
neither -- plus the end-to-end case through the byte splitter, where all seven breaks
survive;
clean sequence needs none, and the repair stays safe when re-cut at five caps;
units in a thread;
an already-sealed chunk after a single appended character, the stable path is shown not to
across seven append lengths, and the stable path is shown to still redact;
and repairs its tail, and that no global whitespace flush remains in the module;
them can put the grade back on the event loop;
whole, and the re-bound is shown to run after the grade;
the reader sees whole, the rendered reading to refuse it, and the two grades to agree;
the frames it would deliver;
passed, so existing callers are unaffected.
Mutation proof, ten ways, each against a pristine copy kept outside the repo and each
verified by hash to have applied and been restored. Three cover the streaming and terminal
work: stopping the renderer from asking for the stable split fails 1, making the stable mode
fall back to the search fails 2, and sealing without grading the seam fails 1. Four cover the
grade and the repair. Dropping the redactor from one of the
seven call sites fails the enumeration test. Replacing the dense probe with a doubling walk
fails the two probe-order cases. Sending the terminal straight to the wider flush fails the
end-to-end case, where the breaks drop from seven to zero. Widening the repair's span to the
whole text fails four cases, including the two-key one. A fifth mutation, dropping one of the
grade's two readings, initially failed NOTHING -- that gap is what the link case was added
for, and it now fails.
Regression, each file scoped with
-n0:test_messaging_split.py138 passed,test_channel_table_rendering.py82 passed (the reassembly invariant the byte splitterdocuments),
test_chat_mirror.py82 passed,test_display_split_safety.py50 passed,test_whatsapp_renderer.py46 passed,test_whatsapp_turn_renderer.py39 passed,test_security_posture.py53 passed,test_slack_renderer.py73 passed,test_webex_renderer.py34 passed. Gates: black, isort, flake8, mypy on the changed files,and the comment-history gate, all clean.
messaging/split.pyis listed inNON_EGRESS_REDACTION_MODULESwith the reason thedisplay-safety module gives: it applies a redactor its caller injects, owns no output, and
the modules calling it are the registered sinks.
The fixture uses the AWS-documented example key the repo's own display-safety tests
already use, so no new credential shape is introduced.
Any other suggestions on the work
This does NOT close the issue, and the issue stays open on purpose. Five of the six delivery
paths it names are closed end to end: every boundary they choose is graded, and the
enumeration test fails if any of them stops passing a redactor. The sixth, WhatsApp's
streaming leg, keeps the graded cut but carries a residual that is present on current main
independently of this change -- that leg re-renders a growing reply on every frame while
treating earlier messages as delivered, so a delivery tracked by a count cannot notice when
an earlier message would be rendered differently. Closing it properly means grading a
boundary against what is already on screen, which is the channel-neutral work in #13494, not
something this change can reach. The residual is recorded on #13743 rather than a duplicate
issue. What it does NOT add is predecessor bookkeeping at each point a message stops
being editable -- a streaming leg that has already delivered a message cannot re-cut it, so
that piece has to be graded against what is already on screen. The channel-neutral
primitives for that are in #13494 and are not on main yet, so this change reuses only what
main already has and touches none of the files that PR owns. That bookkeeping belongs to a
follow-up once #13494 lands, not to the six paths this issue is about.
Two things found while measuring, worth recording. The splitter has callers beyond the six
paths named in the issue:
discord/renderer.py(five calls),telegram/renderer.py,feishu/client.py,messaging/renderer.pyandteams/session_resume.py. Discord andTelegram seal splitter chunks as separate messages and carry the same open seam, and both
files belong to #13494, which is rewriting them with the predecessor primitives -- so they
are stage-two scope here by construction, not an oversight, and this PR touches neither.
Feishu, the shared renderer and the Teams resume path are unchanged, and none of them
belongs to #13494. Their own code answers whether their chunks become separate messages
rather than leaving it open:
feishu/client.pysends every chunk as its own reply to thesame inbound anchor, and
chunk_for_transporthands its parts to callers that send themone message at a time. Neither passes a redactor, so the seam is open on those legs too.
They sit outside this stage by scope, not by another PR's ownership. And the issue's per-file grade was measured by grepping for the new primitives'
names, which cannot see that five of the six paths already close the mid-line cut by
redacting the whole buffer first; the line-boundary seam is what was actually open at all
of them.
Two advisory findings are recorded rather than fixed. One asks that the redaction-tag tally
be taken from the chunks actually delivered rather than before the split, which changes a
notice rather than a boundary; it is worth doing and is not in this change's scope. The other
is the spec below.
One review finding is recorded rather than fixed. The owning spec,
docs/system-specs/modules/messaging.md, still calls the Webex splitter lossless and thecharacter splitter free of modes, and the
redactorladder falsifies both. That file belongsto #13494, which this change deliberately does not touch, so the correction travels with the
follow-up that lands after it rather than creating a conflict here.
Pattern harvest
Rule candidate: enumeration test, added in this PR as
test_split_credential_boundary.py::TestEveryDeliveryPathIsEnumerated.Pattern: a text is cut on a length budget and then scanned one piece at a time, so the scan
runs after the boundary is chosen and cannot see what the cut separated.
The shape generalises past credentials: any check that reads one piece of a text the caller
will deliver in several is blind to whatever the boundary split. It resists a purely
syntactic rule, because "these chunks each become their own message" is not visible at the
call site -- which is why the enforceable half is an enumeration: the delivery paths are
listed, and every splitter call in them must pass a redactor. The list is checked in both
directions, so a path that stops reaching a splitter fails rather than passing by having
nothing left to check, and the AST pass counts the offloaded
asyncio.to_threadshape aswell as the direct call.
Refs #13743