Skip to content

feat(dashboard): check an agent-written page on a synthetic copy of its crewmate's data - #20747

Merged
chenmingwei23 merged 1 commit into
mainfrom
feat/20643-dashboard-page-verification
Oct 16, 2026
Merged

chenmingwei23 merged 1 commit into
mainfrom
feat/20643-dashboard-page-verification

Conversation

@chenmingwei23

@chenmingwei23 chenmingwei23 commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: An agent finds out that its dashboard page throws on data shaped like the crewmate's real record, and as the dashboard really serves it, before the person ever opens the page.

An agent writes a dashboard page but cannot open it, and it only sees sample data. The crewmate's real record can have a different shape (a field that is null, a list that is empty, a string where a number was expected), so a page that works for the agent can throw for the person, who then sees an error instead of the page. The dashboard route also rewrites part of a page's text with its secret filter before the page is shown, which can break a page on the gateway alone. Nothing in save, stage or preview runs a page's script today, so nothing could have told the agent.

Why it matters

Every agent-written page has been tested only on data the agent had to guess, and only as written rather than as served. The first real run happens in front of the person and costs them a round trip.

Not a goal

  • Giving an unapproved page any value from the record. The run uses a synthetic copy.
  • Hiding the record's shape. By decision of the RFC owner the copy keeps the field set, list lengths, nulls, types, string lengths and scripts, number size and sign, date and id formats, enums and booleans, and the fact that these stay the same across saves. Only content is replaced.
  • Showing or keeping a staged page. A staged page still cannot be previewed or applied.
  • Checking later refills, other languages or other window sizes. Only the first paint is run.
  • Running libraries from the CDN allowlist. The run is offline, so they are not fetched, and a page that cannot work without one is not verified.
  • Downloading or bundling a browser. It uses one an administrator already installed.
  • Running the page in the agent's own sandboxed browser. The issue asks the gateway to run the page and return the errors in the save result. The agent's browser is a separate install that a policy can deny outright, it would need the composed document (up to 1 MB plus the frame's scripts) in the agent's context, and it has the network this check must not have.
  • Pages that ship with the product. They are not agent-written and are not run.

What changed (motivation -> approach -> change)

A save now runs the page once, over a synthetic copy of the record, and tells the agent what broke.

After dashboard_save stages a page written as view.html, the gateway loads it once in a headless Chromium on its own host, in the frame's own document, and returns a verification block. A page the tab would mount without script is not run, and that decision is audited like the tab's.

The page never receives the record. The gateway reads the crewmate's data once, as the tab is served it, and dashboard_synth.py builds a copy with the same shape: the declared fields, list lengths, nulls, types, string lengths and character kinds, number sizes and signs, date and id formats, booleans and the words of the closed vocabularies. Every other value is invented from a fresh random seed, drawn without looking at the value it replaces, so a page saved again and again cannot find the real value as the one never shown (links become https://example.invalid/..., an enum field shows one of its declared labels). A key is kept only when the type catalog lists it or the page itself names it and it is not an id. Equal values stay equal so a join on an id still joins, and the browser is handed the copy only. This fits the RFC rule that a staged page never receives a value (line 743 of rfc-crewmate-dynamic-dashboard.md): the page gets the record's shape, booleans and the schema's own fixed words, and nothing a person or a crewmate wrote. No reader is shown the run and its result goes only to the agent that wrote the page, so the approval gate that protects a reader is untouched. The block says data: synthetic.

The page is checked as the tab is served it: its text goes through the route's own secret filter first. When that rewrote anything, the block says how many places and the line of the first, never what was replaced. The reply crosses the same redaction as the tab's values.

The browser, and the Python that starts it, must be ones an administrator installed (root-owned, not a link, not writable by the gateway's account), because they run outside the agent's OS sandbox and the agent triggers the save. The browser keeps its sandbox on and DevTools runs over an inherited pipe. Linux only: a macOS app directory is writable by every administrator account, so a Mac answers skipped.

Limits: only that save runs a check, one at a time across the gateway; a save that finds one running, or a machine short of memory, is skipped at once. The copy is two linear walks on a thread, and one 8-second cap kills the browser's process group and answers timed_out.

status Means What the agent does
passed the page ran and raised nothing describes the page
failed each error, with its message and its line and column in the agent's own view.html fixes the page and saves again
timed_out the run hit its cap looks for a loop or a wait that never ends
skipped the check did not run, and why (not Linux, no system-installed browser or Python, busy, short of memory, a sandbox that cannot start, a network policy that withholds script, a script that stopped at an unfetched CDN library) nothing was checked

The page stays staged whatever the run finds. Nothing leaves the machine, and no one layer carries that alone:

Layer What it closes
The frame's page policy fetch, XHR, WebSocket, beacons, images, forms
A request filter in the browser every request but the page, the CDN origins included
No name resolution, and a proxy at a closed port a request the filter misses, such as a preconnect
WebRTC removed the one channel a page policy does not govern

Code: dashboard_verify.py (the runner), dashboard_synth.py (the copy), member_dashboard_fields.py, agent_panel.py and mcp_panel.py; builtin_skills/dashboard/SKILL.md and two docs pages explain it.

Backwards compatibility

Compatible: the save reply gains a verification key (null for a block layout) and the tool text gains a block. Nothing that worked is refused, and a host with no system-installed browser or Python, or a host that is not Linux, behaves as it did, with a skipped block.

Where it runs: a Linux host with a Chrome or Chromium and /usr/bin/python3 installed by an administrator (for example from the distribution's package manager), whose browser can start its own sandbox. That is the intended reach. Every other host (macOS, Windows, a Playwright-cache browser, a container without user namespaces) answers skipped with the reason.

A browser that fails to start is not started again for 15 minutes: checks in that time answer skipped with the start error, so a broken install costs one failed start and not one per save. A browser that dies mid-run or runs slow is tried again on the next save.

To keep the check off today, leave out the administrator-installed browser or Python, or restrict the crewmate's egress in the governance ceiling (network.egress), which also withholds script from its live page. Either way the save answers skipped and says why.

Removes nothing: no route, field, argument or tool is removed or renamed, and the page is staged whatever the run finds.

Tests

  • test_dashboard_synth.py: the copy has exactly the real read's shape; no real string, number or id appears in it (booleans and closed-vocabulary words excepted); a new random seed each run; a draw is independent of the value it replaces; which keys are kept; equal values stay equal; the size bounds.
  • test_dashboard_verify.py, without a browser: where a browser position lands in the agent's own HTML; which DevTools events are errors; the note on what the secret filter rewrote; which browser is chosen under the strict provenance check; every way a run is skipped, including busy and short of memory.
  • test_dashboard_verify.py, against a fake browser (fake_devtools_browser.py) over the real inherited pipe: the spawn, the framing, the request filter, the limit passed by turning the runner's clock and the process-group kill, what the browser was handed (a document built from the copy, never a record value), and two checks at once starting one browser.
  • test_dashboard_verify.py, real Chromium, run only where a plain page first passes in the host's browser and skipped otherwise: a page that crashes only on a null field also crashes on the copy, a clean page passes, and a page that tries every way out finds a loopback listener silent.
  • test_dashboard_save_tool.py: the route, the glue and the tool text; the page is run as the route serves it (its secret filter, same functions); the event loop keeps turning while a check runs; two saves at once start one browser; a timeout kills the process and answers timed_out; the save stops waiting at its bound; only a save can start a check.
  • test_spawn_audit.py: the spawn is listed with its reason.

Manual verification

Ran the real-browser tests, plus a probe of eleven page shapes (clean, thrown error, rejection, syntax error, timer error, console.error, a CDN script, WebRTC, storage, self-navigation, fetch) against a Chromium headless shell. The authoring host cannot nest user namespaces, so those runs started the browser with its sandbox off through a local test plugin that is not in the diff; a host whose browser runs a plain page runs the tests as written. The CI container's Chromium does not run reliably, so these tests are skipped in CI and the fake-browser tests carry the coverage there. The loopback listener saw 0 connections with the layers on and 2 with them removed.

Screenshots / video

N/A: no UI change: this is a gateway and tool-result change, and no website/ file is touched.

Related Issues

Fixes #20643

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 96bd6f02bd2a29daf4218197d15c3df0decfcb8d touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Advisory — does not block merge.

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 15, 2026
@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — 🟡 CONCERNS

Design-level review of 96bd6f02bd2a29daf4218197d15c3df0decfcb8d — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

The agent can now trigger its own script to run outside its OS sandbox, in a gateway-host Chromium. Only Chromium's own sandbox contains it, and the security model docs don't record that.

Watch

  • New way out of the sandbox, not recorded in the security model. The patch exempts "dashboard_verify.py::_spawn" in BENIGN_SPAWNS and says it is "Not routed through the sandbox chokepoint, on purpose: Chromium's own sandbox is the containment". So an agent that is untrusted against its own ceiling writes view.html, and dashboard_save then runs that page as the gateway's account, outside sandbox.py. If Chromium's renderer sandbox is escaped, the agent's code runs unsandboxed and can reach HIDDEN leaves and the keystone files that the OS layer otherwise fences. The network layers block data leaving the machine. Nothing else stands between the agent and the host. That may be the right trade, but only artifacts.md and a test comment state it. It is absent from security-deep-dive and from security.md, where a later reviewer would look for things the sandbox does not cover.
    Clears when: security-deep-dive.md (or security.md) lists the verify spawn as a boundary outside sandbox.py, names what contains it (Chromium sandbox, admin-installed root-owned binaries, offline flags), and says how an operator turns it off; or the spawn goes through the sandbox chokepoint.

Suggestions

  • Add an explicit off-switch: a governed config key, or a policy scope in SCOPE_CATALOG. Today the only ways to turn the check off are uninstalling the browser or Python, or restricting network.egress. An administrator who accepts the unsandboxed run would then have a clean way to refuse it.

[DESIGN-REVIEWED] 96bd6f0

Description read

[DESCRIPTION-READ] a50378252698b56bb463c36c328de1068ae33818e3a029921405934c1d73fbad

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 96bd6f02bd2a29daf4218197d15c3df0decfcb8d — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: PASS

Before merge, run one save on a real Linux gateway host with a distro-installed Chromium and confirm it returns passed/failed, not skipped.

What this change ships

Inventory (10 items) — 10 justified

Intent: an agent finds out that its staged dashboard page throws on data shaped like its crewmate's real record, as the tab serves it, before a person opens it. ADDITION.

  1. dashboard_save runs a view.html page once in a headless browser and returns a verification block — justified
  2. A save now waits for the run, up to REPLY_SECONDS (~12s) — justified
  3. The page gets a synthetic, same-shape copy of the record, never the record itself — justified
  4. Keys are kept only when the catalog lists them or the page spells them; all others are invented — justified
  5. The page is checked after the route's secret filter, with a note giving how many places it rewrote and the line of the first — justified
  6. Only a root-owned browser and Python are trusted, on Linux only; every other host answers skipped — justified
  7. One check runs at a time across the gateway, and a machine short of memory answers skipped — justified
  8. A browser that fails to start is not tried again for 15 minutes — justified
  9. A script that stops at a CDN library's undefined name answers skipped, not failed — justified
  10. The staged page's script-grant decision is audited as (staged), and an egress-denied page is not run — justified

[FIRST-PRINCIPLES-REVIEWED] 96bd6f0

Description read

[DESCRIPTION-READ] 119c02c8d314748790396792467e19aaa800d17d49c75d5156fba152fb2f555e

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 96bd6f02bd2a29daf4218197d15c3df0decfcb8d — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 96bd6f0

Verdict parsed from the review's SHA-scoped output markers for commit 96bd6f02bd2a29daf4218197d15c3df0decfcb8d.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 96bd6f02bd2a29daf4218197d15c3df0decfcb8d: <one-sentence reason>

Comment thread test/test_dashboard_verify.py Fixed
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 15, 2026
@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 96bd6f02bd2a29daf4218197d15c3df0decfcb8d and found no blocking issues.

This comment is updated in place on each push.

Review details

FINDING -- src/kiro_crew/dashboard_synth.py:552 -- With labels A–Z plus Ā, return pick merges 27 distinct strings into 26 replacements, contradicting the documented distinctness guarantee (recurring-defect-patterns) -> Fix: raise SynthesisRefused when retries and _nearby_new cannot produce an unused replacement.
[GPT-REVIEWED] 96bd6f0

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 96bd6f02bd2a29daf4218197d15c3df0decfcb8d: <one-sentence reason>

@chenmingwei23
chenmingwei23 force-pushed the feat/20643-dashboard-page-verification branch from 50a8482 to af466ea Compare October 15, 2026 08:58
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 15, 2026
@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=16ccd70fe79d

fixed in af466ea: dashboard_verify.py imports _SANDBOX_MARKERS from kiro_crew.browser_cli.launcher and keeps no copy of its own.
One definition of the sandbox-failure strings remains, shared with the Browser panel's launcher.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=fba84b4f33b9

fixed in af466ea: verify_package_page(slug, crew, slot, package) no longer takes session_unit.
It reads the newest unit through member_dashboard._newest_unit(slot), the call the tab itself makes, so the checked document and the served one come from the same unit.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=9f57dcc94fce

fixed in af466ea: Verification.wire() no longer emits elapsed_ms. The value stays on the dataclass for the log line only.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=ab22a5e34ca0

fixed in af466ea: an error past the cap is looked up in a bounded set of messages already seen (at most 8 x MAX_ERRORS entries), so a repeating omitted error is counted once.
The reply's "more" count now counts distinct errors. test_an_error_past_the_cap_that_repeats_is_counted_once throws the ninth error 100 times and asserts omitted == 1.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=46c1e6db9e48

  • The launcher interpreter bypasses executable provenance checks

fixed in af466ea: the launcher no longer runs under sys.executable. find_launcher_python() takes the first of a fixed list of system interpreters, resolves it, and requires validate_provider_executable(require_protected=True): root-owned, not a link, not writable by this account.
With none present the check returns skipped and starts nothing. Tests pin that a writable interpreter and the gateway's own interpreter are never chosen.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=46c1e6db9e48

  • Refused-request overflow is silently discarded

fixed in af466ea: a refused URL past the 20 kept is counted in blocked_overflow, and the note reads "and N more" with that count included.
test_a_refused_url_is_clipped_where_it_is_stored sends three times the cap, asserts blocked_overflow is twice the cap, and asserts the total in the note.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=24606ee37fc7

fixed in af466ea: verify_package_page keeps the structured decision from member_dashboard._egress_decision and calls the existing member_dashboard._audit_page_script_grant for both outcomes before it branches.
Two tests in test_dashboard_save_tool.py assert one audit row for a granted page and one for a withheld page, with the decision and its reason.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

rebutted: a user-owned Chrome in /Applications fails the root-owned check, so on such a Mac the verdict is skipped and the reply says why. That is the intended result.
The browser runs outside the agent's OS sandbox on a save the agent triggers. A build the agent could write must not become the program the gateway runs.
A Mac with an administrator-installed (root-owned) Chrome or Chromium is checked.
Reaching the other Macs takes a sealed browser install or a run inside the sandbox chokepoint. Each is its own design and is left out of this change.

@chenmingwei23
chenmingwei23 force-pushed the feat/20643-dashboard-page-verification branch from af466ea to a4c549d Compare October 15, 2026 09:25
@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=d23892af4bbf

fixed in a4c549d: when a script request to a CDN origin the frame allows was refused, an undefined-name error (Chart is not defined) is the run's doing and no longer makes the page failed.
If it is the only error the result is skipped, with the name and the reason. If the page has other errors, those are failed and the name is listed in a note. A script that throws on the data before the library call is still reported on its line.
Tests: unit cases for each of those, a fake-browser case, and two real-Chromium cases (a library page whose own code throws on the data is failed on its line; one that stops at the library is skipped).
The dashboard skill, the tool text and the artifacts spec say the same.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=26361e3ff963

fixed in a4c549d: the dashboard skill now says the save waits for the run (usually about a second, at most about twenty) and that a check begun while another runs is skipped. The dashboard_save tool text says the save waits for the run.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=d8fbd05e59a1

fixed in a4c549d: the PR description, the dashboard skill and the artifacts spec now say that one check runs at a time, and that a save arriving during another waits up to 3 seconds for it and then skips its check.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=424cf3fecaa7

fixed in a4c549d: exception and console messages now go through the same URL cleaning as the policy entries (the query and fragment are cut, then the message is clipped), so a URL's query string never reaches the reply.
test_a_url_in_an_error_loses_its_query_and_its_fragment asserts it for an uncaught error and for a console.error.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

rebutted: on a host whose browser is user-owned (the Ubuntu snap Chromium, a Chrome dragged into /Applications) the check returns skipped, and the reply names the reason. That is the intended result.
The page document carries the crewmate's real data and the browser runs outside the agent's OS sandbox. A program the agent could have written would receive that data, so the rule is not loosened to suit an install location.
The check runs where an administrator installed the browser: Google Chrome or Edge from the vendor package, the distribution's chromium package, or a root-owned Chrome on macOS.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=e541306d9acf

rebutted: os.posix_spawn closes no inherited descriptor on Python 3.12 (its closefrom action arrived in 3.13, and this repo supports 3.12), so any descriptor the gateway holds without close-on-exec would reach a browser that runs the page. subprocess.Popen(close_fds=True) guarantees none does.
It also returns a bare pid. The kill and reap path (platform_compat.kill_popen_tree) takes a Popen, and the spawn audit would stop listing this spawn.
The cost of the launcher Python is one kind of host: an administrator-installed browser with no administrator-installed Python. It is skipped with a reason that says so.

@chenmingwei23
chenmingwei23 force-pushed the feat/20643-dashboard-page-verification branch from 5966370 to c1f35c3 Compare October 15, 2026 20:32
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 15, 2026
@buluoray

This comment has been minimized.

@buluoray

This comment has been minimized.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=7199f259df34

needs-a-decision: this is the off-switch question asked on earlier heads (spans bc48d1458abc, cb119353ddfd, d722bfaf6a28, ca037f2c4489). Both ways to clear it are the maintainers' to choose, and neither is in this PR.
What is true today: the check runs in a host process outside sandbox.py, and test_spawn_audit lists it with that reason. Its containment is the browser's own sandbox, which is never turned off, plus a root-owned browser and Python, a scrubbed environment, no network, one check at a time and an 8 second cap. The page holds no value of the record. The residual is a renderer escape.
Option 1: a boolean setting (default on, or off while the agent sandbox is enforcing) read by dashboard_verify.preflight, with the skipped reason naming it, plus a registry entry, the UI index and the docs. Option 2: a maintainer's word here that the renderer sandbox is an acceptable boundary, which the description then records.
Question for the maintainer: which one, and in this PR or as its own change?

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=f133aeafa4bc

rebutted: the numbers are not separate. REPLY_SECONDS is derived from RUN_TIMEOUT_SECONDS in one expression in dashboard_verify.py, and the run's deadline and the route's wait both read those two constants.
A test pins the derivation and that the wait stays under the tool's 30 second limit (test_a_save_never_waits_on_the_check_past_a_bound_inside_what_the_tool_waits).
The tool text says "about twelve" for 11.5 seconds, and a test checks those words. Tying the words to the constant is a test-only follow-up and changes no behavior.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=969766f5b4af

rebutted: the finding is right that no producer reaches the else branch today. Every site that runs a page passes data=DATA_SYNTHETIC, and the save-tool tests assert "synthetic" on a passed and a failed block. The branch only guards a reply that lacks the key, and it changes no output the agent sees.
Deleting it is two lines and a push that restarts all four lanes, with no change in what an agent is told, so it stays. If a maintainer prefers it gone, it is a one-commit cleanup.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=60d2419074b2

rebutted: the copy holds no value of the record, so the RFC line "a staged page is never fed, so a page nobody approved never receives a value" holds as written, and the RFC owner chose to leave the RFC unchanged. The owner is also the author of issue 20643 and of this PR, and this comment records the choice.
The choice: the copy keeps the field set, list lengths, nulls, types, string lengths and scripts, number size and sign, date and id formats, enums and booleans, and that these stay the same across saves. Every other value is replaced. No amendment is pending.
The run-at-save shape is the issue's: issue 20643 (label decision) asks the gateway to run a render and return the errors in the save result. Nobody is shown the run, and the approval gate is untouched (test_the_apply_of_a_package_is_refused_until_a_renderer_lands, test_an_apply_with_nothing_staged_writes_nothing).
Same item as spans 0e81255b1a17 and 4a77011b3b63 on earlier heads.

…ts crewmate's data

An agent composes a dashboard page without being able to open it, so a
page that works on sample data can still throw on data shaped like the
crewmate's record, and the person meets an error instead of the page.

After dashboard_save stages a page written as view.html, the gateway
runs it once in a headless Chromium on the gateway host and returns a
verification block in the save reply: passed, failed with each script
error and its line and column in the agent's own view.html, timed_out,
or skipped with the reason (no system-installed browser, for one). The
page stays staged either way.

The page never receives the record. The gateway reads the record once,
as the tab is served it, and dashboard_synth builds a copy of the same
shape with invented values: it keeps the declared fields, list lengths,
nulls, types, string lengths and character kinds, number sizes, date and
id formats, booleans and the words of the closed vocabularies, and
replaces every other value, from a fresh random seed. An enum field
shows one of the labels its package declares. Each replacement is drawn
without looking at the value it replaces, since leaving that value out
would make it the one value a page saved again and again never sees. A
key inside a value is kept only when the type catalog lists it, or the
page itself names it and it is not an id, since a page reaches a key
only by spelling it. Only a work item id keeps the word in front of it
(it_ and eight hex digits). The browser is handed the copy only, and the
block says its data is synthetic.

The page is checked as the tab is served it: its text goes through the
same secret filter the dashboard route applies first, and the block says
how many places that rewrote and the line of the first. What the check
returns crosses the redaction the tab's values cross, because a page
chooses the text of its errors.

The check runs only for a save of a view.html page, one at a time across
the gateway, and is skipped at once when another is running or the
machine is short of memory. The copy is two linear walks on a thread. One
8 second cap covers the whole run: at it the browser's process group is
killed and the result is timed_out. A page whose script stops at a
library from the CDN allowlist, which the offline run cannot fetch, is
skipped with that reason and not failed.

Nothing leaves the machine. DevTools runs over an inherited pipe, so
nothing listens. The page policy stays, a request filter answers only
the page, no name resolves, every connection goes to a closed local
port, and WebRTC is removed. The browser keeps its own sandbox on, runs
on a throwaway profile and a scrubbed environment, and is killed with
its process group.

The browser and the Python that starts it must be ones an administrator
installed (root-owned, not a link, not writable by the gateway's
account). They run outside the agent's OS sandbox and a save is the
agent's to trigger, so a program the agent could write would let it
choose what the gateway runs. For that reason the check runs on Linux
only.

Fixes #20643
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 15, 2026
@chenmingwei23
chenmingwei23 force-pushed the feat/20643-dashboard-page-verification branch from c1f35c3 to 96bd6f0 Compare October 15, 2026 21:15
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 15, 2026
@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=e38e95300bc4

needs-a-decision: this is the security-docs half of the question asked on earlier heads (spans 7199f259df34, bc48d1458abc, ed07433c150b). The boundary is recorded in the artifacts spec, in test_spawn_audit with its reason, and in the PR description under Where it runs. Whether it also belongs in security.md and security-deep-dive.md is the maintainers' call.
The paragraph is ready: the check runs in a host process outside sandbox.py. Its containment is the browser's own sandbox (never turned off), a root-owned browser and Python, a scrubbed environment, no network (page policy, request filter, no name resolution, a closed proxy, no WebRTC), one check at a time and an 8 second cap. It is off when the browser or Python is absent, or when the crewmate's egress is restricted. The page holds no value of the record. The residual is a renderer escape.
Question for the maintainer: add that paragraph to the security docs in this PR, or as its own change?

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=e8b9a40b6137

needs-a-decision: this is the off-switch question asked on earlier heads (spans 7199f259df34, bc48d1458abc, cb119353ddfd). A governed key or a SCOPE_CATALOG scope touches the settings registry, the governance catalog, the UI index and the docs, and the default (on, off, or off while the agent sandbox is enforcing) is a product choice.
In code it is one boolean read by dashboard_verify.preflight, with the skipped reason naming it. Today the check is off when the browser or Python is absent, or when the crewmate's egress is restricted, and the description says so under Where it runs.
Question for the maintainer: which form and which default, and in this PR or as its own change?

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

needs-a-decision: this save has not been run on a real gateway host. The authoring host has no root-owned browser and cannot start the browser's sandbox, so the real-browser tests ran against a Chromium with its sandbox off through a local plugin that is not in the diff. They skip in CI because the CI container's Chromium does not run. The fake-browser tests cover the runner, and the PR description says so under Manual verification.
Question for the maintainer: could someone with a Linux host that has a distro-installed Chromium and user namespaces save one view.html and confirm the block says passed or failed, not skipped?

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

span=282867d86f40

rebutted: there are 26 uppercase single-letter replacements, so a 27th distinct label has no unused one of the same kind and shares a replacement. This is the one place the guarantee is best-effort, and the wording says so: a text another text already has is drawn again. A short text first gets up to 64 draws and a search for a nearby unused one.
Refusing is the alternative, and it costs more: SynthesisRefused skips the whole check for the save, while a merge makes two one-character labels equal in the copy. Widening the class instead would change the kind of character the copy keeps.
Numbers never merge (test_small_distinct_numbers_and_codes_never_merge). A map keyed by colliding values is refused, because a merged key would drop an entry. General case: test_distinct_values_never_become_one.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 15, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor

HOLD at 96bd6f0.

This PR runs agent-written page script in a gateway-host Chromium outside sandbox.py, kills that browser's process group on timeout (dashboard_verify.py#L879-L929), and reads the crewmate's real record to build the copy. Changes like that are kept for a human security reviewer. In the parts I read, the spawn looks carefully fenced: fixed argv, scrubbed env, root-owned binaries, the browser sandbox never disabled, and four network layers.

What needs a person:

  1. The maintainer's decision removes the approval step and asks whether this check still has a job. There is no reply yet. That answer comes first.
  2. If it stays: Design CONCERNS asks for the unsandboxed spawn to be recorded in security.md / security-deep-dive.md, and for an explicit off-switch. Both of your dispositions are needs-a-decision, and no maintainer has ruled.
  3. First Principles asks for one save on a real Linux host with a distro-installed Chromium. You note it has never returned passed/failed outside a sandbox-off local plugin, and CI skips the real-browser tests.

The GPT finding (27 one-letter labels into 26 replacements) is answered as a documented best-effort case. I accept that answer.

CI on this head is green. I read the code but did not run it, and I did not read every line of the 2,189 new production lines.

— crew-pr-reviewer (r105)

@buluoray

Copy link
Copy Markdown
Contributor

Withdrawing the versions-gallery plan and API contract above (now hidden as outdated). We are not building it; the direction is the 2026-10-15 RFC amendment in #20963 (a page goes live on apply). No split of work is needed on this PR.

@chenmingwei23
chenmingwei23 enabled auto-merge (squash) October 16, 2026 01:45
@chenmingwei23
chenmingwei23 merged commit 333061b into main Oct 16, 2026
103 checks passed
@chenmingwei23
chenmingwei23 deleted the feat/20643-dashboard-page-verification branch October 16, 2026 01:45
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dashboard: verify an agent-written page against the crewmate's own record before the person previews it

4 participants