Repository navigation
feat(dashboard): raise the hand-written page cap to 3 MiB - #20790
Conversation
Sensitive Change ReviewSensitive files changed:
|
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS Every member-dashboard read now re-scrubs and re-frames a page that can be 3 MiB, taking 0.1–1.7 s of GIL-bound regex time. Moving that work to a thread does not take it off the GIL. Watch
[DESIGN-REVIEWED] fa3d217 Description read[DESCRIPTION-READ] 11c4a97ceb245934a00eea395d3bbdedca89a12e78cd7e4da70cdbf42c39ad7f That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS The description says this PR makes the no-script binding fill linear and changes two of its edge cases, but the diff only adds tests for a fix the base already has. Not justified as shipped
What this change shipsInventory (9 items) — 8 justifiedIntent: let someone save, preview, apply and draw a hand-written dashboard page of up to 3 MiB (#20634). ADDITION.
[FIRST-PRINCIPLES-REVIEWED] fa3d217 Description read[DESCRIPTION-READ] 113666b9da98d8fa6306e6d63baaace26ff7dd127a560a403eb8dea1a374f3a6 That sha256 names the PR description this verdict read. Recompute it with |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsFINDING -- src/kiro_crew/dashboard/handlers/sandbox_doc.py:69 -- A document containing 5,242,880 False positive or not applicable? A repository writer can comment: |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
b640377 to
37ee5c6
Compare
Sensitive change review
|
|
span=83b0acf97324
|
|
Security Scope Review -- 🟡 CONCERNSWhich legitimate operations does Verdict basis: the reviewer's own judgement; no confirmed regression. Security scope review -- adjudicated candidates
No candidate flipped from allowed to refused. Scope-Verdict: CONCERNS TIGHTENING: Nothing in the shell deny path got stricter, and the change names no threat that it newly refuses. Every size cap is raised. ADJUDICATED: 18 candidates, 0 confirmed newly-refused PLATFORM COVERAGE: The diff adds no shell, path, separator, interpreter or environment-variable assumption. Its only platform-sensitive spellings are the line endings in the PEM body (
No assumed platform spelling is left without a candidate. UNADJUDICATED: The changed matcher is the output credential redactor,
If any of these redacted more than before, that would be a new refusal: a blocked link stripped of its URL, or a page or diff chip with text replaced. These are the inputs I would have submitted, none of which I could get a verdict for:
The changed branch is reached only at a private-key header with no END marker at or after the header's end. The full-block alternative can never succeed in that position, so in principle nothing should flip. The confirmation is the repo's own randomized comparison with the plain [SCOPE-REVIEWED] fa3d217 |
|
The only red check is This PR changes no |
37ee5c6 to
46b50de
Compare
Sensitive change review
|
|
|
ff0eef6 to
8643e60
Compare
Sensitive change review
|
|
Rebased: main's #19866 (the suggestions page for a crewmate with no adopted page) edited the same lines of member_dashboard.py as this PR. Both changes are kept. The git range-diff against the previous head shows the first commit differing only in the context lines around that spot, and the second commit identical. The security package line budget needed no change: the measured count is still 29,492. The related tests pass on this head together with the tests main added for #19866. CI re-running. |
8643e60 to
33c036c
Compare
Sensitive change review
|
33c036c to
02e0786
Compare
Sensitive change review
|
A hand-written dashboard page that inlines a client-side library, such as a 3D renderer, together with its own scene code is several hundred KB to a few MB of layout with no data in it. The 128 KiB page cap in a dashboard package refused every such page, and the frame drew nothing over 1,000,000 bytes. The page cap is now 3 MiB in the package and in the frame. The package cap is twice that plus 128 KiB, because the stored JSON writes a quote or a line break in the page as two bytes. The page cap bounds what a page costs to store, send and draw; it no longer claims to detect values baked into the layout, which a size cannot tell from a library. The sandbox document stash that holds the composed page is 8 MiB instead of 4 MiB, so a page at the cap keeps room for the largest data island a package can hold. Two scans that were harmless at the old caps now see pages 24 times as large. The member dashboard route scrubs and frames a stored page off the event loop. The no-script binding fill reads each start tag once, where it used to read on to the end of the page from every unclosed tag, which was quadratic. Fixes #20634
…r time The credential pattern spells a PEM private key as a header followed by a full block, the body up to the first END marker, or a truncated tail. The full block fails only after reading to the end of the text, so a header that no END marker follows costs one such read, and a text that repeats the header costs one per header: quadratic, with the GIL held. With the dashboard page cap at 3 MiB, a page of repeated headers could stop the gateway for hours while it was redacted. The scan now looks up the text's last END marker once and matches a header past it with the same alternation minus the full-block tail, which cannot succeed there. The pattern text and every match are unchanged. The new module is a rule source of the display battery, so its bytes join the rules version. Part of #20634
Sensitive change review
|
Problem / Motivation
Goal: A hand-written dashboard page of up to 3 MiB can be saved, previewed, applied and drawn.
Issue #20634 asks for a 3 MiB page cap, with the frame and package caps raised to match. A dashboard package refuses a page (
view.html) over 128 KiB, and the frame refuses any page over 1,000,000 bytes. A page that inlines a client-side library, such as a 3D renderer, together with its own scene code is several hundred KB to a few MB of layout with no data in it. Pages like that are common, and today none of them can be saved as a package. For example, the unminified three.js r170 build, its OrbitControls, Chart.js 4 and D3 7, with a 3 KB scene, is 1,848,313 bytes, and the same build with TensorFlow.js 4.22 instead is 2,830,086 bytes: over the old frame cap and the old package cap, under the new one.Why it matters
Anyone who writes their own dashboard page meets this cap as soon as the page inlines a library. As a package the page cannot be saved at all. As a template page it stops working at 1,000,000 bytes.
Not a goal
What changed (motivation -> approach -> change)
A hand-written page of up to 3 MiB can be saved, previewed, applied and drawn. The caps now bound only what a page costs to store, send and draw, because size cannot tell an inlined library from values baked into a layout.
MAX_PAGE_BYTESindashboard_frame.py)MAX_VIEW_HTML_BYTESinartifact_store/dashboard_package.py)MAX_PACKAGE_BYTES, same file)_MAX_BYTESindashboard/handlers/sandbox_doc.py)A page that passes the page check also passes the package check. The stored package is JSON, which writes a quote or a line break as two bytes, and real minified libraries grow 1 to 7.5% that way, so the package cap is 2 x the page cap + 128 KiB.
The frame cap measures the page, before the host adds about 17 KB of bootstrap and the data island. The sandbox-document stash (
dashboard/handlers/sandbox_doc.py) holds the whole document, so it grows from 4 MiB to 8 MiB: room for a page at the cap and a 4 MiB data island, 64 fields of 64 KiB. The gateway's 60 MiB request cap and the artifact store's 25 MiB content cap already fit, and so does the preview tool's 64 KiBhtmlargument (src/kiro_crew/validation.py), which refuses every page whatever its size.The old refusal said a large page "usually has values baked into it". Size does not show that, so the message now says to inline less and keep values in the fold path. The checks that keep values out are unchanged: every field names its source, every binding is a declared field, the controller pushes only declared fields, and value-shaped keys are refused.
Three scans that were harmless at the old caps now see pages 24 times larger.
dashboard/handlers/member_dashboard.py) ran the credential scrub and the framing check on the event loop. At 3 MiB the scrub takes between 0.1 s and 1.7 s, by page shape, so both now run in a worker thread.fill_bindings_as_textindashboard_frame.py) re-read text in three ways: from every<to the end of a page with no>after it, from every-of a tag name such asa-a-a-, and from every repeat of the binding attribute in one start tag. Each is quadratic and holds the GIL: 16 to 31 KiB took one to two seconds. Each start tag is now read once, and a 3 MiB page of any of them takes under 0.1 s.security/redaction.py, with the newsecurity/pem_scan.py) is changed here because the new cap lets a page of legal size stop it. A page that repeats a PEM private-key header with no END marker after it sent the scan to the end of the page once per header: 64 KiB took 2 s, 256 KiB took 40 s, and a page at the cap would stop the gateway for hours. The scan now looks up the last END marker once and matches a header past it without the full-block tail, which cannot succeed there. Every match is unchanged; 3 MiB of repeated headers takes about 0.4 s.Backwards compatibility
Compatible: all four limits go up, so every page, package and document accepted today is still accepted. A page over the new cap is refused with the same error, which now names the new limit.
The credential scan returns the same matches. The alternation text is unchanged, and in a differential run the new scan found the same matches as the previous one on 270,000 random texts built from PEM headers, END markers, newlines and other credentials.
The no-script binding fill changes in two edge cases. It no longer reads a start tag that contains a literal
<; such a cell keeps the author's own text. And a hyphenated tag name is read whole, so<my-el ...>is closed only by</my-el>, never by</my>, as a browser reads it. All 134 cells in the 14 shipped and fixture pages that carry bindings fill the same as before, and so do 40,000 random well-formed pages.The package cap can be raised later but not lowered while packages stored under the larger one remain, because the reader applies it too.
The frame cap was pinned equal to another tool's own default; #20634 asks the frame to follow the page cap, so that a page that saves also draws, and
test_the_page_cap_is_the_frames_ownpins that equality in the pin's place.The display battery's rules version changes, because
security/pem_scan.pyis one of its rule sources.Reader:
dashboard_frame.py:fill_bindings_as_text-- crew page, a package page served with scripts withheld -- its one caller isrender_package_page; ordinary cells still fill (test_dashboard_package_html_view.py, classTestAScriptsOffPageStillShowsItsValues).Tests
test_dashboard_frame.py: the cap is 3 MiB. A page exactly at it is accepted, and one byte over is refused with the cap in the message.test_dashboard_package_html_view.py: a package whose page is exactly the cap saves, and one byte over is refused naming the cap. A page of quotes at the cap, the 2x worst case, still saves. A stored package exactly atMAX_PACKAGE_BYTESsaves and one byte more is refused. The page cap equals the frame's. The no-script fill finishes a page at the cap built from each of five quadratic shapes (unclosed tags, unclosed bound tags, a hyphenated tag name, one tag repeating the attribute, and that with a wrong close tag), under a hang guard and with no time asserted. A literal<in a start tag leaves the cell as written, and a hyphenated tag name is closed only by its own end tag.test_dashboard_templates_builtin.py: a template page exactly at the shared ceiling passes the check that preview, apply and rollback run, and one byte more is refused naming it.test_member_dashboard_package_route.py: a package page at the cap is served through the frame, and the scrub and framing run off the event loop.test_member_dashboard_routes.py: a template page at the cap previews, applies and draws through the real routes, with its scrub off the loop.test_sandbox_doc_channel.py: a document built from a page at the cap plus a 4 MiB data island (64 fields of 64 KiB), which is over the old 4 MiB, is minted and served back unchanged.test_artifact_dashboard_package.py: the stored-form test sets its own cap just under its layout, because the real cap is now sized for a library page. Small pages and every older case are covered by the existing tests, which pass unchanged.test_redaction_pem_scan_linear.py: 26 PEM shapes (full and encrypted blocks, unterminated and nested headers, several blocks, newlines spelled LF, CRLF or a lone carriage return, overlapping markers) keep the spans and the redacted text they had. 30,000 random texts match the plainsearchloop, and the last-END-marker lookup matches a scan of every offset. A text of 8,000 headers with no END marker is matched without once reaching the full-block tail, which is a count and not a timing, and 3 MiB of repeated headers is redacted under a hang guard.test_security_gate_liveness.py: the security package line budget now covers the 81 new lines, the new module and the choice between the two patterns.test_redaction_jwt_scan_linear.pypasses the scan state its helper now takes.Manual verification
Not run in a browser. The page document was built with the dashboard's own
buildSrcdocin a Node DOM over a 3 MiB page: 3.17 MB in about 0.5 s. To see it draw, put a page of about 3 MiB in thepanel-templatesfolder, preview it and press Keep.Related Issues
Fixes #20634
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)