Skip to content

fix(channels): grade the delivered sequence so a split cannot rejoin a key - #13889

Closed
chenmingwei23 wants to merge 2 commits into
mainfrom
fix/cross-delivery-redaction-13743-v9
Closed

chenmingwei23 wants to merge 2 commits into
mainfrom
fix/cross-delivery-redaction-13743-v9

Conversation

@chenmingwei23

Copy link
Copy Markdown
Contributor

What is the problem?

Supersedes #13864, and before it #13858, #13852, #13844, #13839, #13836, #13828
and #13758. Each replacement exists for the
same mechanical reason: taking a review fix while staying inside the two-commit limit
needs the commits reshaped, and a reshaped branch cannot be published by a
fast-forward push. Cutting a fresh branch publishes the same two commits as a plain
create instead. No review had been left on either predecessor.

Six channel delivery paths cut model text on a length budget and scan each piece on its
own. A credential the cut severs matches nothing in either piece, so both are reported
clean and the reader rejoins the halves reading one message and then the next.

Five of the six already redact the whole buffer before splitting, and that closes a cut
that lands mid-line: the credential is one marker before any boundary can reach it. It
does not close a cut at a line boundary. Sealing a chunk trims the whitespace that
ended it, and the next chunk starts at the first character after that whitespace, so the
two halves end up flush against each other on screen while the whole text still holds the
break between them. That form is invisible to a scan of the text as written and to a scan
of either piece.

Why this issue matters to the user

The reader sees the key and nothing reports it. Because each piece scans clean, no
redaction notice is emitted and the delivery is recorded as a clean send: the operator has
no signal at all. This is the audit half of the problem, not only the disclosure half.

How our fix solves it

The boundary is where the defect lives, so the fix goes into the two shared splitters
rather than into six renderers. split_markdown_safe and chunk_utf8_bytes take an
optional redactor and, when given one, reduce the text before any boundary is chosen:

  1. the text is redacted against the rendered form first, via the existing
    redact_for_display, which closes a cut landing mid-line and the markup-hidden variant;
  2. the boundaries it then chooses are graded on the RENDERED SEQUENCE. A platform drops
    the whitespace at a message's edges, so the last visible character of one message sits
    against the first visible character of the next and a trailing newline separates
    nothing. Stripping every chunk's edges before the grade is what makes the predicate see
    what a reader sees, and it is what carries the guarantee to the byte splitter, which
    stores the break rather than trimming it.

The whole sequence, not each neighbouring pair, because a key needs no more than a narrow
budget to span three chunks: split a twenty-character key over four and every pair holds a
fragment matching nothing while the screen holds the key whole. The same gap swallows a
chunk that renders to nothing at all, which the lossless byte splitter can place between
the halves. A fragment is only ever part of a key, so no pair of neighbours can be asked
about it. Both readings joins_to_a_credential names are kept, and each is scanned once
over the joined text rather than per boundary, which is what keeps the search affordable --
measured at 10.5 ms against 160 ms on a 74 KB reply, so the ordinary path is 15 times
cheaper than a per-boundary grade.

When a boundary does rejoin a key the cut is MOVED before any text is given up. Candidate
budgets step back one at a time for the first 128 and exponentially after that; the dense
prefix is what stops a budget clean at 6997 being stepped over by a walk that jumps 4, 8,
16, and the exponential tail keeps the reach without a probe per byte. Worst case 1.49 s on
a 74 KB reply, below the 2.2 s a purely exponential walk cost with the old grade.

A STREAMING caller cannot use that search at all, and this is the sharpest thing the change
had to get right. Such a caller re-splits its growing body every frame and treats all but
the last chunk as delivered, which whatsapp/turn_renderer.py states as its own
requirement: chunk i never changes once chunk i+1 exists. A search that reads the whole
body breaks it -- appending ONE character moved a boundary under an already-sent message in
a measured case, and the final pass slices the new list from the delivered count, so the
span is duplicated or dropped with no way back. That leg therefore takes a prefix-stable
split: the text is redacted, and every boundary stays where the caller's own budget puts
it. The same search space that produced a revision produces none under it.

Prefix stability says nothing about whether a BOUNDARY is safe, so the streaming renderer
grades each one against everything that follows before counting the chunk final, using
joins_to_a_credential, which main already has. That grade reads the RENDERED text, every
chunk's edges stripped, for the same reason the splitter's own predicate strips them: a
platform drops the whitespace at a message's edges, so a tail beginning with a tab separates
nothing once two messages sit on screen. Asking about the characters as stored approves
exactly that pair, and the case is reachable -- the splitter never strips leading whitespace
and a hard cut can leave a remainder starting with one. Both grades now answer the same
question about the same boundary, and a test pins that they agree. A chunk that does not pass stays in the
live bubble, where a later frame may still revise it; at finalization every chunk must go,
so the still-unsealed tail is repaired through repaired_for_delivery rather than held.

Only when no budget at or below the caller's cuts safely is text given up, and then only
the credential's own span. There is no wider fallback that strips a whole message: when no
span names the key, the splitter DECLINES TO CUT and returns the text whole. A boundary is
the only thing that rejoins a key across messages, so creating none is fail-closed, and it
costs the caller a chunk over its limit instead of costing the reader a reply stripped of
its whitespace and markup. A caller that bounds that chunk again has
repaired_for_delivery to grade what it produced. The span is found by SHRINKING -- a span known to collapse into
a key is trimmed from each end by half its width, the trim kept while what remains still
collapses into one -- so a message carrying two keys keeps the paragraphs between them. The
whitespace inside that span is closed up, which makes the key contiguous for the ordinary
redaction, and the loop returns only once the form with EVERY run removed reads clean. That
last condition is the promise to a caller that re-cuts these chunks under its own limit, and
it is why one grade at the end of such a caller is enough. For a key only the rendering
assembles, which no span of the literal text names, the answer is the canonical form -- the
one-directional trade redact_for_display already makes, giving up markup and keeping every
space and every break.

Grading chosen boundaries rather than every candidate is what keeps ordinary wrapped content
intact -- two lines that read as one key when glued are never joined by a cut, so they are
left exactly as written, and a fixture pins that.

The parameter is optional because a caller whose chunks land inside one message severs
nothing a reader can rejoin across messages. It is not optional for the seven paths that do
turn one chunk into one message: those are enumerated in the tests, and any splitter call
there that omits the redactor fails. The check walks the AST and counts both shapes,
including the four sites that hand the splitter to asyncio.to_thread as a reference --
a check reading direct calls only would see four of the eight sites and silently pass the
rest.

Per path, all eight call sites:

path call before now
slack/renderer.py:875 split_markdown_safe, offloaded no whole-buffer redaction at the cut passes _redact_all
webex/renderer.py:396 chunk_utf8_bytes display-safe via webex_display_safe line-boundary seam also closed
whatsapp/renderer.py:302 split_markdown_safe display-safe before the split line-boundary seam also closed
teams/renderer.py:383 split_markdown_safe, offloaded display-safe via _display_text line-boundary seam also closed
teams/renderer.py:743 split_markdown_safe, offloaded display-safe before the split line-boundary seam also closed
wecom/renderer.py:306 split_markdown_safe, offloaded display-safe before the split line-boundary seam also closed
dashboard/chat_mirror.py:588 split_markdown_safe display-safe before the split line-boundary seam also closed
telegram/renderer.py:413 split_markdown_safe no redaction at the cut, each segment redacted alone passes _default_redactor

WeCom's streaming push already picks a credential-safe offset for the frame it is about to
send, and that path is untouched.

Two legs needed more than the redactor argument, both found by review and both reproduced
before they were fixed.

Slack bounds the splitter's output AFTER it has been graded. A logical line placed whole
carries its fence scaffolding on top, so a chunk can exceed the limit, and chunk_text then
blind-slices it and each slice is posted as its own message -- an ungraded boundary at the
very last step. _split_for_slack now grades that final sequence through the new
repaired_for_delivery and repairs it when it rejoins a key. One grade suffices because the
repair is only returned once its fully-collapsed form reads clean, which the tests check by
re-cutting the repaired text at five different caps.

FIVE paths have a HARD cap, and the splitter's fail-closed answer collides with all of them. When no
budget cuts without rejoining a key the splitter declines to cut and answers with the text
whole, which costs the caller a chunk over its budget; this client truncates a larger payload,
so the answer's tail would go with no notice. Each now grades the sequence once more and cuts the
repair back to its budget through one shared bounded_for_delivery, which applies the budget
BEFORE it grades. That order is the point: the splitter's fail-closed answer is a list of ONE
chunk, a one-element list holds no boundary, and a grade asked about it reports that nothing
rejoins -- so the oversized chunk would travel on unbounded, which is the very case the bound
exists for. Cutting to the budget first makes the grade see the sequence the transport will
really deliver. A chunk already inside the budget is returned unchanged, so the pass is free. which is sound because the
repair is safe to bound again by contract -- and which must NOT take a redactor, since a
credential-aware re-cut could decline to cut and leave the budget unmet. Webex, Teams, WeCom, the dashboard
mirror and Telegram all run it; a test walks those five and fails if any sends the splitter's
answer unbounded, another asserts no chunk exceeds the byte cap, and a third that ordinary text
still comes back whole.

Webex was the one path that kept the grade on the loop thread. Its final answer calls the
byte splitter directly inside on_done, and with no cut clean the search and the span repair
hold that thread for seconds -- long enough for the liveness watchdog to exit the gateway and
take every other channel's in-flight turn with it. It now offloads like the others. A
test walks all seven paths and fails if any of them stops handing its splitter call to a
thread, and Telegram carries a stricter version of that check described below.

The span repair is also bounded now. Its passes were capped only by the number of whitespace
runs in the text, and each pass costs a logarithmic number of whole-text redactions, so a
reply whose every break split a key-shaped token would cost thousands of them and the send
would never finish. The cap is the same constant the budget search uses, and past it the
answer is to decline to cut, which gives up nothing.

The dashboard mirror's backfill composed its units inline on the loop thread. An imported
history row carries no size cap, the grade redacts and rescans each candidate boundary, and
the liveness watchdog exits the process after seconds of loop silence -- so one large row
could terminate the gateway. The units are now composed inside asyncio.to_thread, the same
offload the Slack twin already uses and this handler already uses for its selection read.

Telegram was the remaining path whose cut chose boundaries on length alone. It rotates: a
segment that outgrows one message is split, every chunk but the last is sealed as its own
message, and the redaction runs per segment -- so the halves of a severed key are redacted in
neither message while the reader scrolling the two reads it whole. Reproduced at the channel's
own 400-character budget floor: the plain cut hands the key over, the credential-aware cut
moves the boundary instead and every character still ships.

Three things that leg needed beyond the argument.

It asks for no prefix-stable mode, and the reason is a structural difference from the
WhatsApp streaming leg: a rotation REPLACES the buffer with the retained tail, so text
already sealed is never part of a later cut. That is the bookkeeping whose absence is the
residual below, and Telegram has it.

Its rendered budget is enforced by measuring the real HTML and shrinking the source budget,
which means the cut is made more than once; each candidate answer now runs the shared bound,
so a declined cut cannot reach the client's own truncation with the reply's tail inside it.

Its table-aware path cuts each block on its own, so a boundary BETWEEN two blocks belongs to
no single cut and was graded by none of them. The assembled sequence is graded once at the
end. The reachable case is a table whose last cell ends its line followed by prose, and a
fixture pins both halves of it: the per-block pair is shown to hand the key over, the
function's answer is shown not to.

Four call sites on that channel run the cut from a coroutine, and a scan on the gateway's
event loop is what the liveness watchdog exits the process for. All four now hand their work
to a thread. The file-wide check here is stricter than the one the other six paths get: "one
offload exists somewhere in the file" is satisfied while three sites stay on the loop, so the
Telegram check walks every coroutine's own body -- nested closures excluded, since those are
themselves what gets offloaded -- and fails if any of them calls a scanning splitter directly.

The redaction-tag tally on WhatsApp's finalization is fixed rather than deferred, which the
last round recorded as advisory. That pass repairs the unsealed tail and sends the repair, but
the notice counted the pre-repair chunk list, so a reply whose only placeholder came from the
repair announced none. The branch now re-points the list the notice reads at the sequence it
actually sends.

What tests we did

test/test_split_credential_boundary.py, 102 passed and 5 skipped, every run scoped with
an explicit -n0:

  • the seam, measured rather than assumed: the fixture's cut is shown to fall between the
    halves of a key, the two pieces are shown to scan clean individually, and the joined
    sequence is shown to hold the key;
  • a key spanning three and four chunks, where every neighbouring pair reads clean, on both
    splitters and across five narrow budgets;
  • a chunk that renders to nothing placed between the halves by the lossless splitter;
  • a key inside a link, which the canonicalised join loses to the label and only the
    per-side reading sees;
  • the dense probe: a budget off the doubling ladder is reached, asserted on the probe order;
  • the terminal, on text no budget can cut safely: the prose around the span survives byte
    for byte, every break and delimiter is kept, two keys keep the paragraph between them,
    the fully-collapsed form of the result reads clean, and the wider flush is shown to keep
    neither -- plus the end-to-end case through the byte splitter, where all seven breaks
    survive;
  • the post-bounding grade: a blind slice is shown to reopen the seam, the repair fires, a
    clean sequence needs none, and the repair stays safe when re-cut at five caps;
  • structural checks that the Slack path runs that grade and that the mirror composes its
    units in a thread;
  • prefix stability, which is the streamed-reply case: the searching path is shown to revise
    an already-sealed chunk after a single appended character, the stable path is shown not to
    across seven append lengths, and the stable path is shown to still redact;
  • structural checks that the streaming renderer asks for the stable split, grades its seam
    and repairs its tail, and that no global whitespace flush remains in the module;
  • all seven delivery paths, asserting each hands its splitter call to a thread, so none of
    them can put the grade back on the event loop, plus the stricter Telegram check that no
    coroutine in that file calls a scanning splitter directly;
  • the repair loop's bound, asserting the constant is in force;
  • the hard cap on all four capped paths: none of them sends the splitter's answer unbounded,
    no chunk exceeds the byte cap, and ordinary text is returned whole;
  • a SINGLE oversized chunk, the shape a one-element grade skips: it is cut inside the budget,
    every character still ships, a key across one of the new boundaries is still caught, and
    text already inside the budget is returned untouched;
  • the seam's edge whitespace: the stored reading is shown to approve a tab-separated pair
    the reader sees whole, the rendered reading to refuse it, and the two grades to agree;
  • Telegram's rotation at its real budget floor: the plain cut is shown to hand the reader
    the key, the credential-aware cut to move the boundary instead, every character to still
    ship, the bounded splitter to carry the same guard, a declined cut to stay inside the
    budget, and ordinary prose to come back unchanged;
  • Telegram's table-aware seam, with its own control: the per-block sequence is shown to
    rejoin the key and the function's answer is shown not to;
  • a structural check that the finalization repair branch re-points the chunk list its
    redaction notice counts;
  • one channel end to end, through the function WhatsApp's client sends from, asserting on
    the frames it would deliver;
  • backward compatibility: prose and a table row are returned byte for byte with a redactor
    passed, so existing callers are unaffected.

Mutation proof, fifteen ways, each against a pristine copy kept outside the repo and each
verified by hash to have applied and been restored, with a passing control before and after.
Five cover this round: taking the redactor off the Telegram cut fails 2, dropping the
delivery bound from its bounded splitter fails 1, putting one rotation site back on the event
loop fails 1, dropping the table-aware seam grade fails 1, and leaving the notice tally on the
pre-repair chunks fails 1. The third of those is the one that measured a weak test: with the
file-wide check alone it failed NOTHING, because three other sites still offloaded, which is
what the per-coroutine check was added for. Three cover the streaming and terminal
work: stopping the renderer from asking for the stable split fails 1, making the stable mode
fall back to the search fails 2, and sealing without grading the seam fails 1. Four cover the
grade and the repair. Dropping the redactor from one of the
seven call sites fails the enumeration test. Replacing the dense probe with a doubling walk
fails the two probe-order cases. Sending the terminal straight to the wider flush fails the
end-to-end case, where the breaks drop from seven to zero. Widening the repair's span to the
whole text fails four cases, including the two-key one. A fifth mutation, dropping one of the
grade's two readings, initially failed NOTHING -- that gap is what the link case was added
for, and it now fails.

Regression on the rebased tree, each file scoped with -n0: test_telegram.py 331 passed,
test_telegram_parity.py 360 passed, test_options_cap_contract.py 71 passed,
test_channel_control_tag_strip.py 39 passed, test_credential_redaction_notice.py
27 passed, test_redaction_mirror_parity.py 10 passed, test_whatsapp.py 26 passed,
test_messaging_split.py 138 passed,
test_channel_table_rendering.py 82 passed (the reassembly invariant the byte splitter
documents), test_chat_mirror.py 82 passed, test_display_split_safety.py 50 passed,
test_whatsapp_renderer.py 46 passed, test_whatsapp_turn_renderer.py 39 passed,
test_security_posture.py 53 passed, test_slack_renderer.py 73 passed,
test_webex_renderer.py 34 passed. Gates: black, isort, flake8, mypy on the changed files,
and the comment-history gate, all clean.

messaging/split.py is listed in NON_EGRESS_REDACTION_MODULES with the reason the
display-safety module gives: it applies a redactor its caller injects, owns no output, and
the modules calling it are the registered sinks.

The fixture uses the AWS-documented example key the repo's own display-safety tests
already use, so no new credential shape is introduced.

Any other suggestions on the work

This does NOT close the issue, and the issue stays open on purpose. Five of the six delivery
paths it names are closed end to end, and Telegram -- a seventh path beyond the six, which
seals splitter chunks as separate messages the same way -- is closed here too: every boundary they choose is graded, and the
enumeration test fails if any of them stops passing a redactor. The sixth, WhatsApp's
streaming leg, keeps the graded cut but carries a residual that is present on current main
independently of this change -- that leg re-renders a growing reply on every frame while
treating earlier messages as delivered, so a delivery tracked by a count cannot notice when
an earlier message would be rendered differently. Closing it properly means grading a
boundary against what is already on screen, which is the channel-neutral work in #13494, not
something this change can reach. The residual is recorded on #13743 rather than a duplicate
issue. What it does NOT add is predecessor bookkeeping at each point a message stops
being editable -- a streaming leg that has already delivered a message cannot re-cut it, so
that piece has to be graded against what is already on screen. The channel-neutral
primitives for that are in #13494 and are not on main yet, so this change reuses only what
main already has and touches none of the files that PR owns. That bookkeeping belongs to a
follow-up once #13494 lands, not to the six paths this issue is about.

Two things found while measuring, worth recording. The splitter has callers beyond the six
paths named in the issue: discord/renderer.py (five calls), telegram/renderer.py,
feishu/client.py, messaging/renderer.py and teams/session_resume.py. Discord seals
splitter chunks as separate messages and carries the same open seam; that file belongs to
#13494, which is rewriting it with the predecessor primitives, so it is stage-two scope here
by construction and this PR does not touch it. Telegram carried the same seam and is closed
here: its rotation discards sealed text from the buffer, so it needs none of the predecessor
primitives and can be fixed with what main already has.
Feishu, the shared renderer and the Teams resume path are unchanged, and none of them
belongs to #13494. Their own code answers whether their chunks become separate messages
rather than leaving it open: feishu/client.py sends every chunk as its own reply to the
same inbound anchor, and chunk_for_transport hands its parts to callers that send them
one message at a time. Neither passes a redactor, so the seam is open on those legs too.
They sit outside this stage by scope, not by another PR's ownership. And the issue's per-file grade was measured by grepping for the new primitives'
names, which cannot see that five of the six paths already close the mid-line cut by
redacting the whole buffer first; the line-boundary seam is what was actually open at all
of them.

One advisory finding is recorded rather than fixed, the spec below. The other -- that the
redaction-tag tally be taken from the text actually delivered rather than from the pre-repair
chunks -- is fixed in this round and described above.

One review finding is recorded rather than fixed. The owning spec,
docs/system-specs/modules/messaging.md, still calls the Webex splitter lossless and the
character splitter free of modes, and the redactor ladder falsifies both. That file belongs
to #13494, which this change deliberately does not touch, so the correction travels with the
follow-up that lands after it rather than creating a conflict here.

Pattern harvest

Rule candidate: enumeration test, added in this PR as
test_split_credential_boundary.py::TestEveryDeliveryPathIsEnumerated.

Pattern: a text is cut on a length budget and then scanned one piece at a time, so the scan
runs after the boundary is chosen and cannot see what the cut separated.

The shape generalises past credentials: any check that reads one piece of a text the caller
will deliver in several is blind to whatever the boundary split. It resists a purely
syntactic rule, because "these chunks each become their own message" is not visible at the
call site -- which is why the enforceable half is an enumeration: the delivery paths are
listed, and every splitter call in them must pass a redactor. The list is checked in both
directions, so a path that stops reaching a splitter fails rather than passing by having
nothing left to check, and the AST pass counts the offloaded asyncio.to_thread shape as
well as the direct call.

Refs #13743

Six delivery paths cut model text on a length budget and scan each piece
on its own. A key the cut severs matches nothing in either piece, so both
are reported clean, and the reader rejoins the halves reading one message
and then the next. The per-piece scan emits no notice, so the operator
sees a clean send.

Five of those paths already redact the whole buffer before splitting,
which closes a mid-line hard cut. It cannot close a cut at a line
boundary: sealing a chunk trims the whitespace that ended it, so the two
halves sit flush against each other on screen while the whole text still
holds the break between them, invisible to a scan of the text as written
and to a scan of either piece.

The shared splitters now take the redactor themselves. split_markdown_safe
and chunk_utf8_bytes reduce the text before any boundary is chosen: the
form the platform renders, and the form the seal's trim produces once
every break is collapsed. A credential visible in either is a marker
before a boundary can land inside it. The parameter stays optional, so a
caller whose chunks stay inside one message is unaffected, and an
enumeration test over the six delivery paths fails when any splitter call
there omits it.

Refs #13743
Grade the boundaries the ladder chooses, one at a time, rather than every
break at once. A scan of all of them together answers a wider question than
any cut asks: two lines that read as one key when glued would cost the message
every break and its markup, though no cut ever joins them. Wrapped content in a
fence is that shape.

The grade reads the RENDERED pair. A platform drops the whitespace at a
message's edges, so the last visible character of one message sits against the
first visible character of the next: neither a trailing newline nor a trailing
space separates anything. That is also what carries the guarantee to the byte
splitter, which stores the whitespace rather than trimming it -- lossless where
it is kept, invisible where it is shown.

The repair moves the CUT before it gives up any text. Budgets step back
exponentially and every candidate is verified, so a key straddling a boundary
ends up whole inside one chunk with every character and every break intact,
where the space between its halves is on screen and the scanner's own reading
of such text stands. Only when no budget at or below the caller's cuts safely
is the whitespace flushed and the text redacted -- every run, because the
whitespace a cut consumes can be a single space -- and that result is graded as
well, since a repair returned without a grade is a repair nobody checked.
Flushing every run is deliberately wider than one cut produces: that width is
what lets a caller bound the chunks again under its own platform limit without
a grade of its own.

The splitter applies a caller's redactor, so it is classified in
security_posture's NON_EGRESS_REDACTION_MODULES. It owns no output; the modules
that call it are the registered sinks. Same reason display_safety.py sits
there, and the posture drift guard fails without the entry.

Refs #13743
@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — 🟡 CONCERNS

Design-level review of 6fd276ad938e2a0f0ed16032f2db914645d2dd60 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

Credential-safe splitting is opt-in per call site, held together only by a hand-maintained path list a new channel will not be on.

Watch

[DESIGN-REVIEWED] 6fd276a

@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — 🟡 CONCERNS

Premise-level review of 6fd276ad938e2a0f0ed16032f2db914645d2dd60 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

The general fix stops at seven enumerated paths: Discord, Feishu, and four chunk_for_transport egress legs still cut model text ungraded — 10 counted call sites.

Not justified as shipped

  • Item 7 — point patch, counted unfixed siblings: the description says "the seven paths that do turn one chunk into one message: those are enumerated in the tests", but grepping split_markdown_safe|chunk_utf8_bytes plus chunk_for_transport consumers finds 6 more model-text-per-message delivery paths, 10 call sites, none graded: discord/renderer.py (941, 962, 966, 1219, 1244 — including a streaming rotation shaped exactly like the WhatsApp one this PR fixes), feishu/client.py:213, dashboard/chat_runner.py:4120 (the live mirror leg, while its backfill twin chat_mirror.py IS fixed), slack/gateway.py:3931 (proactive cron/subagent egress), dashboard/handlers/messaging.py:2192, :2484 ("This text can come from the model").
    Clears when: those sites pass the redactor (the streaming ones the stable+seam grade) and join DELIVERY_PATHS, or the description names them deliberately deferred and states the residual.
  • Item 3 — duplicate of src/kiro_crew/messaging/split.py:bounded_for_delivery: slack's _bounded closure re-implements it line for line (cut each chunk, repaired_for_delivery, re-cut the repair) — 5 call sites use the shared helper, slack is the one inline copy.
    Subtraction: delete _bounded's loop; call bounded_for_delivery(chunks, SLACK_MSG_LIMIT - reserve, _redact_all, chunk_text).
    Clears when: the deletion is made.

What this change ships

Inventory (10 items) — 8 justified

Intent: stop a length-budget message split from handing the reader a credential neither chunk shows to any scan — a FIX (control tests added here reproduce the defect on base).

  1. A key severed at a chunk boundary no longer reads whole across adjacent messages on seven delivery paths — justified
  2. When no budget cuts cleanly the splitter declines to cut; capped transports re-cut via new bounded_for_delivery (5 consumers) — justified
  3. Slack re-grades its final blind slices — duplicate of messaging/split.py:bounded_for_delivery (inline copy of its loop)
  4. WhatsApp streaming holds a chunk whose seam grades dirty in the live bubble; finalization repairs the tail — justified
  5. WhatsApp streaming split becomes prefix-stable (stable=), so a sent message is never revised — justified
  6. Grading splits moved off the event loop on all seven paths (watchdog exits on a held loop) — justified
  7. AST-enforced enumeration of redactor-passing delivery paths — point patch: 6 sibling paths / 10 call sites uncounted
  8. Ordinary replies pay one extra whole-text redaction scan at the cut (measured 10.5 ms / 74 KB) — justified
  9. messaging/split.py added to NON_EGRESS_REDACTION_MODULES (mandated by the registry pin) — justified
  10. New permanent surface: repaired_for_delivery (3 consumers), redactor kwarg (8 sites) — justified

[FIRST-PRINCIPLES-REVIEWED] 6fd276a

@github-actions

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — 🔴 changes requested (blocking)

GPT 5.6 found at least one blocking issue that must be resolved before merging 6fd276ad938e2a0f0ed16032f2db914645d2dd60. 1 of 1 blocking finding(s) are security-class and were withheld from adjudication, so the blocking verdict stands.

This comment is updated in place on each push.

BLOCKING -- src/kiro_crew/telegram/renderer.py:413, src/kiro_crew/whatsapp/turn_renderer.py:257 -- Streaming seals a credential prefix before its suffix arrives

return split_markdown_safe(text, limit, redactor=_default_redactor) / if joins_to_a_credential(rendered[index], "".join(rendered[index + 1 :]), _redact_all):
LLM streams 4,091 padding chars + AKIA\nI, then the remaining key -> rotation seals the currently clean prefix -> later text completes the credential across messages without rechecking the sealed boundary.
Anchor: backend-security-controls
Fix: Retain credential-prefix suffixes in the mutable tail, or otherwise prove a boundary remains safe under future appends; final repair must include the sealed predecessor.
[BLOCK-MERGE] 6fd276a
[GPT-REVIEWED] 6fd276a

Adjudication (Opus 5) — is blocking on each finding proportionate?

F1 — harm rung: UNBOUNDED (credential rendered whole to a chat reader across two messages).

Conditions I confirmed in this run: Telegram seals chunks[:-1] as their own messages and retains only chunks[-1] as the new buffer (src/kiro_crew/telegram/renderer.py:1467-1475), and the next rotation re-splits only that retained buffer (renderer.py:1387), so the sealed boundary is graded once — by _rejoins_a_key over the chunks that existed at that moment (src/kiro_crew/messaging/split.py:434, split.py:240-245) — and never against text that arrives afterwards. The WhatsApp path has the same shape: grading starts at index = self._sealed_count and walks forward only (src/kiro_crew/whatsapp/turn_renderer.py:255-259), and prefix-stable splitting keeps the already-sealed chunk byte-identical on every later frame (split.py:536-549), so no later frame revisits it. A cut can land inside a key without any newline of its own: the ladder falls through to a hard cut at the budget for a line longer than a quarter of it (split.py:498-503), and the suffix that completes the key reads as nothing on its own, so no per-chunk redaction fires.

Recovery path: none. A sealed Telegram message is posted via _seal_current before _open_new_message (renderer.py:1473-1474) and the WhatsApp comment states the same property at turn_renderer.py:242-243; the end-of-turn pass can only redact text not yet sent. The conditions are ordinary streaming timing over model-authored text this module already treats as adversarial (renderer.py:1400), not contradicting timing or an input the system's own writer cannot produce, so I cannot complete a rarity record for FLAG.

[ADJUDICATION] 6fd276a total=0 uphold=0 downgrade=0
[GPT-ADJUDICATED] 6fd276a
[ADJUDICATION-FENCED] 6fd276a fenced=1 flagged=0
UPHOLD-FENCED F1 src/kiro_crew/telegram/renderer.py:413 -- Both renderers grade a boundary only against text present at seal time and never revisit it, a sealed message cannot be recalled, and the required input is plain streamed model text, so the unbounded-harm claim stands as written.
[GPT-ADJUDICATED-FENCED] 6fd276a

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 6fd276ad938e2a0f0ed16032f2db914645d2dd60: <one-sentence reason>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 25, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Opus 5 Review — 🔴 changes requested (blocking)

Reviewed 6fd276ad938e2a0f0ed16032f2db914645d2dd60 — this comment is updated in place on each push.

BLOCKING — repaired_for_delivery rebuilds its repair from "".join(chunks), but the character splitter's chunks are not a lossless partition, so every repair ships text that differs from the answer.

BLOCKING — src/kiro_crew/messaging/split.py:357

    joined = "".join(chunks)
    repaired = _redact_only_the_rejoined_span(joined, redactor)

A WhatsApp reply over max_message_chars whose chunk boundary severs a key (exactly the case the new seal loop at whatsapp/turn_renderer.py:257 refuses) → on_done → repaired_for_delivery(pending, _redact_all) at whatsapp/turn_renderer.py:541 → the join runs over _seal output (split.py:958 return body.rstrip() outside a fence; body + fence.closer inside one, with the next chunk opening on the synthetic f"{fence.opener}\n" reopener at split.py:630), so the repair that is then split and sent glues the last line of each chunk onto the first line of the next — a paragraph or list break at every seam is silently deleted — and a fenced block spanning a seam becomes closer+opener, a ≥6-backtick run that _BACKTICK_OPEN_RE reads as a NEW fence, so the remainder of the reply renders as one code block. The _redact_only_the_rejoined_span branch's promise that "every other space, every line break and all the markup of the message survive" is already broken before it runs.
Fix: repair the pre-split source, not the concatenation — give repaired_for_delivery (and bounded_for_delivery) the source text as a parameter and use chunks only for the grade; every caller this diff adds still holds it (text in slack _bounded and telegram _split_markdown_table_aware, content/remainder in teams/wecom, pending's source body in on_done).

FINDING — src/kiro_crew/telegram/renderer.py:828 — the table branch appends out.append(block) / _split_table_rows(lines, rich_limit) unredacted while only the elif prose branch reaches a redactor, so repaired_for_delivery(kept, _default_redactor) at :834 violates the fixed-point precondition _rejoins_a_key documents and fires for a credential sitting wholly inside one table cell; the span then holds no whitespace, closed == repaired, the canonical fallback flattens every **/`/link in the whole message, and the table is re-cut by _split_markdown_bounded(repaired, rendered_limit) under the HTML cap instead of rich_cap — the header-less pipe-text continuations _rotate_on_length picked rich_cap to avoid → Fix: block = redact_for_display(block, _default_redactor)[0] on both table branches before it enters out, or grade only the seams between blocks.

FINDING — src/kiro_crew/whatsapp/turn_renderer.py:541 — repaired_for_delivery(pending, _redact_all) runs synchronously in async def on_done (and joins_to_a_credential likewise at :257 in _render_live), while the very next statement offloads split_markdown_safe and every sibling site in this diff offloads its grade; _redact_only_the_rejoined_span runs up to _DENSE_PROBES (128) passes, each doing ~2·log₂(n) _collapses_to_a_key calls that re-run the 4-pass redact_credentials plus the exfil scan over the whole reply, all on the loop that carries every channel and the liveness heartbeat → Fix: wrap both in asyncio.to_thread and add ("whatsapp/turn_renderer.py", …) to the new OFFLOADED table, which omits this file.

FINDING — test/test_split_credential_boundary.py:54 — DELIVERY_PATHS is curated with no derived-omission guard and omits discord/renderer.py (split_markdown_safe with no redactor= at :941, :962, :966, :1219, :1244, each followed by a per-chunk _seal_current()/_open_new_message()) and feishu/client.py:213 (one _sync_reply per chunk), so the exact defect this PR closes stays reachable on two shipped one-chunk-per-message channels while the suite reports full coverage → Fix: derive the enumeration by walking SRC.rglob("*.py") with _splitter_calls and asserting every module reached is in DELIVERY_PATHS or an explicitly-reasoned non-delivery allowlist (the shape security_posture.NON_EGRESS_REDACTION_MODULES already uses), then pass redactor= at the Discord and Feishu sites — a remedy that lands in files this PR does not touch, so it is advisory rather than blocking.

[BLOCK-MERGE] 6fd276a
[OPUS-REVIEWED] 6fd276a

Verdict parsed from the review's SHA-scoped output markers for commit 6fd276ad938e2a0f0ed16032f2db914645d2dd60.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 6fd276ad938e2a0f0ed16032f2db914645d2dd60: <one-sentence reason>

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

Superseded by #13944.

The replacement was cut from current main and read back exactly before this closure. It has two commits, the expected 12 files, a clean local and remote head at 032ee346afd3bc009e0d3517cbf0985967a23b62, Refs #13743, and no closing issue reference. It preserves chunk seams and never rebuilds a delivery sequence. This PR is closed unmerged; Issue #13743 stays open.

@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Sep 25, 2026
@bolichen97
bolichen97 deleted the fix/cross-delivery-redaction-13743-v9 branch November 6, 2026 07:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant