Repository navigation
fix(channels): cut model text where no key can be rejoined - #13758
chenmingwei23 wants to merge 2 commits into
Conversation
Six delivery paths cut model text on a length budget and scan each piece on its own. A key the cut severs matches nothing in either piece, so both are reported clean, and the reader rejoins the halves reading one message and then the next. The per-piece scan emits no notice, so the operator sees a clean send. Five of those paths already redact the whole buffer before splitting, which closes a mid-line hard cut. It cannot close a cut at a line boundary: sealing a chunk trims the whitespace that ended it, so the two halves sit flush against each other on screen while the whole text still holds the break between them, invisible to a scan of the text as written and to a scan of either piece. The shared splitters now take the redactor themselves. split_markdown_safe and chunk_utf8_bytes reduce the text before any boundary is chosen: the form the platform renders, and the form the seal's trim produces once every break is collapsed. A credential visible in either is a marker before a boundary can land inside it. The parameter stays optional, so a caller whose chunks stay inside one message is unaffected, and an enumeration test over the six delivery paths fails when any splitter call there omits it. Refs #13743
GPT 5.6 Review — 🔴 changes requested (blocking)GPT 5.6 found at least one blocking issue that must be resolved before merging This comment is updated in place on each push. BLOCKING -- src/kiro_crew/dashboard/chat_mirror.py:588 -- large backfill splitting can terminate the gateway BLOCKING -- src/kiro_crew/slack/renderer.py:875 -- hard-cap fallback reopens the credential seam (origin: validation) [BLOCK-MERGE] abba1b6 Adjudication (Opus 5) — is blocking on each finding proportionate?F1 — UNBOUNDED (crash / in-flight work loss, and the fence is authoritative here). F2 — UNBOUNDED (security-class: credential halves rejoined on screen). [ADJUDICATION] abba1b6 total=0 uphold=0 downgrade=0 [ADJUDICATION-FENCED] abba1b6 fenced=2 flagged=0 False positive or not applicable? A repository writer can comment: |
First Principles Review (Fable 5) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS The seam this PR calls a silent disclosure stays open on Feishu and both Not justified as shipped
What this change shipsInventory (6 items) — 4 justifiedIntent: FIX — stop a length-budget cut from handing a reader a credential neither delivered message contains, and stop the scan reporting that send clean.
[FIRST-PRINCIPLES-REVIEWED] abba1b6 |
Design Review (Fable 5) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS Fail-open by default: the credential-aware cut is opt-in, enforced only by a hand-maintained six-file allowlist that a new delivery path never enters. Watch
Suggestions
[DESIGN-REVIEWED] abba1b6 |
e824c69 to
c20a0e6
Compare
Opus 5 Review — ✅ no blocking findingsReviewed Review detailsAdvisory only — the credential-aware cut is not wired to every one-chunk-one-message path, and the Webex site runs its new grading on the event loop. FINDING — src/kiro_crew/webex/renderer.py:396 — FINDING — test/test_split_credential_boundary.py:39 — FINDING — src/kiro_crew/messaging/split.py:227 — [OPUS-REVIEWED] abba1b6 Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
Grade the boundaries the ladder chooses, one at a time, rather than every break at once. A scan of all of them together answers a wider question than any cut asks: two lines that read as one key when glued would cost the message every break and its markup, though no cut ever joins them. Wrapped content in a fence is that shape. The grade reads the RENDERED pair. A platform drops the whitespace at a message's edges, so the last visible character of one message sits against the first visible character of the next: neither a trailing newline nor a trailing space separates anything. That is also what carries the guarantee to the byte splitter, which stores the whitespace rather than trimming it -- lossless where it is kept, invisible where it is shown. The repair moves the CUT before it gives up any text. Budgets step back exponentially and every candidate is verified, so a key straddling a boundary ends up whole inside one chunk with every character and every break intact, where the space between its halves is on screen and the scanner's own reading of such text stands. Only when no budget at or below the caller's cuts safely is the whitespace flushed and the text redacted -- every run, because the whitespace a cut consumes can be a single space -- and that result is graded as well, since a repair returned without a grade is a repair nobody checked. Flushing every run is deliberately wider than one cut produces: that width is what lets a caller bound the chunks again under its own platform limit without a grade of its own. The splitter applies a caller's redactor, so it is classified in security_posture's NON_EGRESS_REDACTION_MODULES. It owns no output; the modules that call it are the registered sinks. Same reason display_safety.py sits there, and the posture drift guard fails without the entry. Refs #13743
c20a0e6 to
abba1b6
Compare
|
Superseded by #13828, which carries the same two commits plus the follow-up fixes review asked for on this head. This branch needed a history rewrite to take them while staying inside the two-commit limit, so the replacement was cut as a fresh branch and pushed without one. No review had been left here. |
What is the problem?
Six channel delivery paths cut model text on a length budget and scan each piece on its
own. A credential the cut severs matches nothing in either piece, so both are reported
clean and the reader rejoins the halves reading one message and then the next.
Five of the six already redact the whole buffer before splitting, and that closes a cut
that lands mid-line: the credential is one marker before any boundary can reach it. It
does not close a cut at a line boundary. Sealing a chunk trims the whitespace that
ended it, and the next chunk starts at the first character after that whitespace, so the
two halves end up flush against each other on screen while the whole text still holds the
break between them. That form is invisible to a scan of the text as written and to a scan
of either piece.
Why this issue matters to the user
The reader sees the key and nothing reports it. Because each piece scans clean, no
redaction notice is emitted and the delivery is recorded as a clean send: the operator has
no signal at all. This is the audit half of the problem, not only the disclosure half.
How our fix solves it
The boundary is where the defect lives, so the fix goes into the two shared splitters
rather than into six renderers.
split_markdown_safeandchunk_utf8_bytestake anoptional
redactorand, when given one, reduce the text before any boundary is chosen:redact_for_display, which closes a cut landing mid-line and the markup-hidden variant;platform drops the whitespace at a message's edges, so the last visible character of one
message sits against the first visible character of the next and a trailing newline
separates nothing. Stripping those edges before the grade is what makes the predicate
see the boundary a reader sees, and it is what carries the guarantee to the byte
splitter, which stores the break rather than trimming it.
Only a boundary that really does rejoin a key is repaired, and the repair collapses the
breaks and redacts, so that message loses its line breaks and its markup. That is the same
one-directional trade
redact_for_displayalready makes when it gives up markup alone:formatting is worth less than a key. Grading the chosen boundaries rather than every
candidate is what keeps ordinary wrapped content intact -- two lines that read as one key
when glued are never joined by a cut, so they are left exactly as written, and a fixture
pins that.
The parameter is optional because a caller whose chunks land inside one message severs
nothing a reader can rejoin across messages. It is not optional for the six paths that do
turn one chunk into one message: those are enumerated in the tests, and any splitter call
there that omits the redactor fails. The check walks the AST and counts both shapes,
including the four sites that hand the splitter to
asyncio.to_threadas a reference --a check reading direct calls only would see three of the seven sites and silently pass the
rest.
Per path, all seven call sites:
slack/renderer.py:875split_markdown_safe, offloaded_redact_allwebex/renderer.py:396chunk_utf8_byteswebex_display_safewhatsapp/renderer.py:302split_markdown_safeteams/renderer.py:383split_markdown_safe, offloaded_display_textteams/renderer.py:743split_markdown_safe, offloadedwecom/renderer.py:306split_markdown_safe, offloadeddashboard/chat_mirror.py:588split_markdown_safeWeCom's streaming push already picks a credential-safe offset for the frame it is about to
send, and that path is untouched.
What tests we did
test/test_split_credential_boundary.py, 25 cases, run scoped with-n0:halves of a key, the two pieces are shown to scan clean individually, and the joined
pair is shown to hold the key;
from the joined chunks and from each chunk read alone;
the frames it would deliver;
passed, so existing callers are unaffected.
Mutation proof, three ways. Dropping the redactor from one of the seven call sites fails
the enumeration test (1 failed). Disabling the redactor branch in both splitters fails five
behaviour tests while the cases that assert the unguarded behaviour keep passing, so the
suite discriminates on the fix and not on the fixture. Removing the edge strip from the
boundary grade -- the part that models what the client renders -- fails the byte-splitter
case alone (1 failed), which is the case that exists for it.
Regression, each file scoped with
-n0:test_messaging_split.py138 passed,test_channel_table_rendering.py82 passed (this is the reassembly invariant the bytesplitter documents),
test_chat_mirror.py82 passed,test_display_split_safety.py50passed,
test_whatsapp.py26 passed. Gates: black, isort, flake8, mypy on the changedfiles, and the comment-history gate, all clean.
The fixture uses the AWS-documented example key the repo's own display-safety tests
already use, so no new credential shape is introduced.
Any other suggestions on the work
This is the first of two stages and deliberately does not close the issue. It makes the
cut safe at every boundary these paths choose. It does not add predecessor bookkeeping at
each point a message stops being editable, which is the remaining scope: a streaming leg
that has already delivered a message cannot re-cut it, so that piece has to be graded
against what is already on screen. The channel-neutral primitives for that are in #13494
and are not on main yet, so this stage reuses only what main already has and touches none
of the files that PR owns.
Two things found while measuring, worth recording. The splitter has callers beyond the six
paths named in the issue:
discord/renderer.py(five calls),telegram/renderer.py,feishu/client.py,messaging/renderer.pyandteams/session_resume.py. Discord andTelegram seal splitter chunks as separate messages and carry the same open seam, and both
files belong to #13494, which is rewriting them with the predecessor primitives -- so they
are stage-two scope here by construction, not an oversight, and this PR touches neither.
Feishu, the shared renderer and the Teams resume path are unchanged, and none of them
belongs to #13494. Their own code answers whether their chunks become separate messages
rather than leaving it open:
feishu/client.pysends every chunk as its own reply to thesame inbound anchor, and
chunk_for_transporthands its parts to callers that send themone message at a time. Neither passes a redactor, so the seam is open on those legs too.
They sit outside this stage by scope, not by another PR's ownership. And the issue's per-file grade was measured by grepping for the new primitives'
names, which cannot see that five of the six paths already close the mid-line cut by
redacting the whole buffer first; the line-boundary seam is what was actually open at all
of them.
Pattern harvest
Rule candidate: enumeration test, added in this PR as
test_split_credential_boundary.py::TestEveryDeliveryPathIsEnumerated.Pattern: a text is cut on a length budget and then scanned one piece at a time, so the scan
runs after the boundary is chosen and cannot see what the cut separated.
The shape generalises past credentials: any check that reads one piece of a text the caller
will deliver in several is blind to whatever the boundary split. It resists a purely
syntactic rule, because "these chunks each become their own message" is not visible at the
call site -- which is why the enforceable half is an enumeration: the delivery paths are
listed, and every splitter call in them must pass a redactor. The list is checked in both
directions, so a path that stops reaching a splitter fails rather than passing by having
nothing left to check, and the AST pass counts the offloaded
asyncio.to_threadshape aswell as the direct call.
Refs #13743