Repository navigation
feat(dashboard): check an agent-written page on a synthetic copy of its crewmate's data - #20747
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS The agent can now trigger its own script to run outside its OS sandbox, in a gateway-host Chromium. Only Chromium's own sandbox contains it, and the security model docs don't record that. Watch
Suggestions
[DESIGN-REVIEWED] 96bd6f0 Description read[DESCRIPTION-READ] a50378252698b56bb463c36c328de1068ae33818e3a029921405934c1d73fbad That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Before merge, run one save on a real Linux gateway host with a distro-installed Chromium and confirm it returns What this change shipsInventory (10 items) — 10 justifiedIntent: an agent finds out that its staged dashboard page throws on data shaped like its crewmate's real record, as the tab serves it, before a person opens it. ADDITION.
[FIRST-PRINCIPLES-REVIEWED] 96bd6f0 Description read[DESCRIPTION-READ] 119c02c8d314748790396792467e19aaa800d17d49c75d5156fba152fb2f555e That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsFINDING -- src/kiro_crew/dashboard_synth.py:552 -- With labels False positive or not applicable? A repository writer can comment: |
50a8482 to
af466ea
Compare
|
span=16ccd70fe79d
|
|
span=fba84b4f33b9
|
|
span=9f57dcc94fce
|
|
span=ab22a5e34ca0
|
|
span=46c1e6db9e48
|
|
span=46c1e6db9e48
|
|
span=24606ee37fc7
|
|
af466ea to
a4c549d
Compare
|
span=d23892af4bbf
|
|
span=26361e3ff963
|
|
span=d8fbd05e59a1
|
|
span=424cf3fecaa7
|
|
|
span=e541306d9acf
|
5966370 to
c1f35c3
Compare
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
span=7199f259df34
|
|
span=f133aeafa4bc
|
|
span=969766f5b4af
|
|
span=60d2419074b2
|
…ts crewmate's data An agent composes a dashboard page without being able to open it, so a page that works on sample data can still throw on data shaped like the crewmate's record, and the person meets an error instead of the page. After dashboard_save stages a page written as view.html, the gateway runs it once in a headless Chromium on the gateway host and returns a verification block in the save reply: passed, failed with each script error and its line and column in the agent's own view.html, timed_out, or skipped with the reason (no system-installed browser, for one). The page stays staged either way. The page never receives the record. The gateway reads the record once, as the tab is served it, and dashboard_synth builds a copy of the same shape with invented values: it keeps the declared fields, list lengths, nulls, types, string lengths and character kinds, number sizes, date and id formats, booleans and the words of the closed vocabularies, and replaces every other value, from a fresh random seed. An enum field shows one of the labels its package declares. Each replacement is drawn without looking at the value it replaces, since leaving that value out would make it the one value a page saved again and again never sees. A key inside a value is kept only when the type catalog lists it, or the page itself names it and it is not an id, since a page reaches a key only by spelling it. Only a work item id keeps the word in front of it (it_ and eight hex digits). The browser is handed the copy only, and the block says its data is synthetic. The page is checked as the tab is served it: its text goes through the same secret filter the dashboard route applies first, and the block says how many places that rewrote and the line of the first. What the check returns crosses the redaction the tab's values cross, because a page chooses the text of its errors. The check runs only for a save of a view.html page, one at a time across the gateway, and is skipped at once when another is running or the machine is short of memory. The copy is two linear walks on a thread. One 8 second cap covers the whole run: at it the browser's process group is killed and the result is timed_out. A page whose script stops at a library from the CDN allowlist, which the offline run cannot fetch, is skipped with that reason and not failed. Nothing leaves the machine. DevTools runs over an inherited pipe, so nothing listens. The page policy stays, a request filter answers only the page, no name resolves, every connection goes to a closed local port, and WebRTC is removed. The browser keeps its own sandbox on, runs on a throwaway profile and a scrubbed environment, and is killed with its process group. The browser and the Python that starts it must be ones an administrator installed (root-owned, not a link, not writable by the gateway's account). They run outside the agent's OS sandbox and a save is the agent's to trigger, so a program the agent could write would let it choose what the gateway runs. For that reason the check runs on Linux only. Fixes #20643
c1f35c3 to
96bd6f0
Compare
|
span=e38e95300bc4
|
|
span=e8b9a40b6137
|
|
|
span=282867d86f40
|
|
HOLD at 96bd6f0. This PR runs agent-written page script in a gateway-host Chromium outside What needs a person:
The GPT finding (27 one-letter labels into 26 replacements) is answered as a documented best-effort case. I accept that answer. CI on this head is green. I read the code but did not run it, and I did not read every line of the 2,189 new production lines. — crew-pr-reviewer (r105) |
|
Withdrawing the versions-gallery plan and API contract above (now hidden as outdated). We are not building it; the direction is the 2026-10-15 RFC amendment in #20963 (a page goes live on apply). No split of work is needed on this PR. |
Problem / Motivation
Goal: An agent finds out that its dashboard page throws on data shaped like the crewmate's real record, and as the dashboard really serves it, before the person ever opens the page.
An agent writes a dashboard page but cannot open it, and it only sees sample data. The crewmate's real record can have a different shape (a field that is null, a list that is empty, a string where a number was expected), so a page that works for the agent can throw for the person, who then sees an error instead of the page. The dashboard route also rewrites part of a page's text with its secret filter before the page is shown, which can break a page on the gateway alone. Nothing in save, stage or preview runs a page's script today, so nothing could have told the agent.
Why it matters
Every agent-written page has been tested only on data the agent had to guess, and only as written rather than as served. The first real run happens in front of the person and costs them a round trip.
Not a goal
What changed (motivation -> approach -> change)
A save now runs the page once, over a synthetic copy of the record, and tells the agent what broke.
After
dashboard_savestages a page written asview.html, the gateway loads it once in a headless Chromium on its own host, in the frame's own document, and returns averificationblock. A page the tab would mount without script is not run, and that decision is audited like the tab's.The page never receives the record. The gateway reads the crewmate's data once, as the tab is served it, and
dashboard_synth.pybuilds a copy with the same shape: the declared fields, list lengths, nulls, types, string lengths and character kinds, number sizes and signs, date and id formats, booleans and the words of the closed vocabularies. Every other value is invented from a fresh random seed, drawn without looking at the value it replaces, so a page saved again and again cannot find the real value as the one never shown (links becomehttps://example.invalid/..., anenumfield shows one of its declared labels). A key is kept only when the type catalog lists it or the page itself names it and it is not an id. Equal values stay equal so a join on an id still joins, and the browser is handed the copy only. This fits the RFC rule that a staged page never receives a value (line 743 ofrfc-crewmate-dynamic-dashboard.md): the page gets the record's shape, booleans and the schema's own fixed words, and nothing a person or a crewmate wrote. No reader is shown the run and its result goes only to the agent that wrote the page, so the approval gate that protects a reader is untouched. The block saysdata: synthetic.The page is checked as the tab is served it: its text goes through the route's own secret filter first. When that rewrote anything, the block says how many places and the line of the first, never what was replaced. The reply crosses the same redaction as the tab's values.
The browser, and the Python that starts it, must be ones an administrator installed (root-owned, not a link, not writable by the gateway's account), because they run outside the agent's OS sandbox and the agent triggers the save. The browser keeps its sandbox on and DevTools runs over an inherited pipe. Linux only: a macOS app directory is writable by every administrator account, so a Mac answers
skipped.Limits: only that save runs a check, one at a time across the gateway; a save that finds one running, or a machine short of memory, is
skippedat once. The copy is two linear walks on a thread, and one 8-second cap kills the browser's process group and answerstimed_out.statuspassedfailedview.htmltimed_outskippedThe page stays staged whatever the run finds. Nothing leaves the machine, and no one layer carries that alone:
Code:
dashboard_verify.py(the runner),dashboard_synth.py(the copy),member_dashboard_fields.py,agent_panel.pyandmcp_panel.py;builtin_skills/dashboard/SKILL.mdand two docs pages explain it.Backwards compatibility
Compatible: the save reply gains a
verificationkey (nullfor a block layout) and the tool text gains a block. Nothing that worked is refused, and a host with no system-installed browser or Python, or a host that is not Linux, behaves as it did, with askippedblock.Where it runs: a Linux host with a Chrome or Chromium and
/usr/bin/python3installed by an administrator (for example from the distribution's package manager), whose browser can start its own sandbox. That is the intended reach. Every other host (macOS, Windows, a Playwright-cache browser, a container without user namespaces) answersskippedwith the reason.A browser that fails to start is not started again for 15 minutes: checks in that time answer
skippedwith the start error, so a broken install costs one failed start and not one per save. A browser that dies mid-run or runs slow is tried again on the next save.To keep the check off today, leave out the administrator-installed browser or Python, or restrict the crewmate's egress in the governance ceiling (
network.egress), which also withholds script from its live page. Either way the save answersskippedand says why.Removes nothing: no route, field, argument or tool is removed or renamed, and the page is staged whatever the run finds.
Tests
test_dashboard_synth.py: the copy has exactly the real read's shape; no real string, number or id appears in it (booleans and closed-vocabulary words excepted); a new random seed each run; a draw is independent of the value it replaces; which keys are kept; equal values stay equal; the size bounds.test_dashboard_verify.py, without a browser: where a browser position lands in the agent's own HTML; which DevTools events are errors; the note on what the secret filter rewrote; which browser is chosen under the strict provenance check; every way a run is skipped, including busy and short of memory.test_dashboard_verify.py, against a fake browser (fake_devtools_browser.py) over the real inherited pipe: the spawn, the framing, the request filter, the limit passed by turning the runner's clock and the process-group kill, what the browser was handed (a document built from the copy, never a record value), and two checks at once starting one browser.test_dashboard_verify.py, real Chromium, run only where a plain page first passes in the host's browser and skipped otherwise: a page that crashes only on a null field also crashes on the copy, a clean page passes, and a page that tries every way out finds a loopback listener silent.test_dashboard_save_tool.py: the route, the glue and the tool text; the page is run as the route serves it (its secret filter, same functions); the event loop keeps turning while a check runs; two saves at once start one browser; a timeout kills the process and answerstimed_out; the save stops waiting at its bound; only a save can start a check.test_spawn_audit.py: the spawn is listed with its reason.Manual verification
Ran the real-browser tests, plus a probe of eleven page shapes (clean, thrown error, rejection, syntax error, timer error,
console.error, a CDN script, WebRTC, storage, self-navigation,fetch) against a Chromium headless shell. The authoring host cannot nest user namespaces, so those runs started the browser with its sandbox off through a local test plugin that is not in the diff; a host whose browser runs a plain page runs the tests as written. The CI container's Chromium does not run reliably, so these tests are skipped in CI and the fake-browser tests carry the coverage there. The loopback listener saw 0 connections with the layers on and 2 with them removed.Screenshots / video
N/A: no UI change: this is a gateway and tool-result change, and no
website/file is touched.Related Issues
Fixes #20643
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)