Repository navigation
Conversation
Sensitive Change ReviewSensitive files changed:
How to clear this check:
Only the author's comment counts this way. A reviewer can instead |
UX Review (Opus 5.5) — ✅ PASSUX-level review of UX-Verdict: PASS Nothing to check. The only file under [UX-REVIEWED] 7fc1f2b Description read[DESCRIPTION-READ] 87c5a5862745e96cae5663c22a9153a8093ffb33a7f4a825ceab9835dabdb694 That sha256 names everything this verdict read: the PR description, plus the 3 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: Notes
[DESIGN-REVIEWED] 7fc1f2b Description read[DESCRIPTION-READ] 4a52eb98b52a6d6f2b2b6c266e7ecd23e3ff499e2cd905a01102061ae247cac3 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks. The new Notes
What this change shipsInventory (8 items) — 7 justifiedIntent: let the agent read a page's full rendered text without scroll-and-screenshot loops. ADDITION.
[FIRST-PRINCIPLES-REVIEWED] 7fc1f2b Description read[DESCRIPTION-READ] f17feb0ffb04c3c3755ac76b6dfd6c1c821ae6882a701f8fcdbcccb2da192e6d That sha256 names the PR description this verdict read. Recompute it with |
GPT 6.1 Review —
|
Add a `read` op to the browser tool. It returns the page's text for the whole page (not only the screen), plus open shadow roots and same-origin frames, or a scrubbed copy of its HTML. Results page at 18,000 characters with `next_offset` and a `digest`. It runs through `run_op`, so the URL gate and the takeover fence apply as for `snapshot`, and it needs no preview setting. The tool description tells agents to prefer it over screenshots and scrolling. `snapshot` lists only controls and headings, so a reading-heavy page (prose, code blocks) came back almost empty and agents fell back to screenshot loops. Carried from #36027 onto current main: the description conflict is merged with main's `type` steps text, the redaction tests use a synthetic GitHub token with the owner's switch on (AWS key IDs stay visible since #37408), and a walker test reads a tutorial-like page. Co-authored-by: smeyffret <smeyffret@gmail.com>
7d79556 to
7fc1f2b
Compare
Problem / Motivation
Goal: Let the agent read what a web page says as text, instead of scrolling and screenshotting it one screen at a time.
The
browsertool'ssnapshotlists only controls and headings, andscreenshotreturns pixels. A user testing tutorial pages reported that the outline was missing almost all of the page: no paragraphs, no code blocks. Their agent fell back to scroll-and-screenshot loops of about 550 px each, which were slow and which the host flagged as a loop.Why it matters
Each frame costs image tokens, and text read off pixels cannot be quoted or searched exactly. A page that only renders in a signed-in browser cannot fall back to
web_fetch, so today the screenshot loop is the only way to read it. Thereflexloop's text is no substitute: it needs the opt-in decision model, reads only the screen and returns about 1,500 characters.Not a goal
snapshot: its outline stays a list of things to act on.queryread by selector (Browser tool: let the agent act by DOM/CSS selector, not snapshot-refs + screenshots #18933).evaluatestays out of the op list.What changed (motivation → approach → change)
This carries #36027 by @smeyffret onto current main, with credit in the commit trailer. It adds one op,
read, with aformatoftextorhtml. Text is the whole page's rendered text, plus open shadow roots and same-origin frames, whichinnerTextskips. HTML is a scrubbed copy: typed values, secret attributes, credential URL parameters, scripts and comments are blanked, and the live DOM is untouched.The walker side is in
browser_harness/snapshot_walker.js, the handler inbrowser_harness/ops.py, and the shim rendering inmcp_tools/browser.py. The op runs throughrun_op, so the URL gate and the takeover fence apply as they do forsnapshot. It needs no preview setting. Results page at 18,000 characters, withnext_offsetand adigestthat changes when the page changes. Redaction runs on the full content before paging, so a credential cannot be split across two pages, and it follows the owner's redaction switch like the other owner-only paths. A new fence pair incontext_assembly/markers.pywraps the page text as untrusted data.security_posture.pylists the handler as an inbound redaction site.The tool description (
mcp_tools/catalog/browser.py),config/prompt.md, the built-in skillsbuiltin_skills/web-browse/SKILL.mdandbuiltin_skills/kirocrew-commands/SKILL.md, and the browser spec tell agents to usereadinstead of screenshots and scrolling. The tool description, theweb-browseskill and the spec also say thattextholds only what is shown. An unselected tab and a closeddetailssection are left out, so the agent clicks or opens them first, or readsformat: html, which keeps them. A note was chosen over an opt-in "include hidden text" mode: it adds no new read path, and hidden text is often stale or for another platform.Changes since #36027: the description conflict is merged with main's
typesteps text. The redaction tests now use a synthetic GitHub token with the switch on, because AWS key IDs stay visible since #37408. A new walker test reads a tutorial-like page.Backwards compatibility
Compatible: a new op and a new optional walker call; every existing op, argument and result is unchanged.
Removes nothing: the diff adds an op, a fence pair and an optional
scanFramesparameter whose default keeps today's walk.Tests
website/src/test/snapshotWalker.read.test.tsruns the real walker source in happy-dom. The new case uses a tutorial page with headings, prose, aprecode block, tabs, a collapseddetailssection and a same-origin frame.snapshotleaves out the prose and the code there.readreturns the prose, the code, the active tab's text, the summary and the frame's text. The other cases cover text below the fold, shadow text, hidden roots and frames giving nothing, the HTML scrub, and the live DOM staying unchanged.test/test_browser_read_op.py: paging gives back the full content. A credential that spans a page boundary is redacted. Badformat,offsetandrefare refused, and a stale ref givesref_stale. A refused URL refusesread. The digest is stable for the same page and changes with it. A forged fence stays inside the fence. The result fits the shim's 20,000-character cap.test/test_browser_guard_isolated_count.py: the source pins match the newscanFrames(signature.test/*browser*.pyfile (2,173 passed) and the walker vitest files (754 passed).Proof: not provable -- other: a new op; its tests have nothing to fail against before it exists
Manual verification
On #36027, the walker's
readran in real headless Chromium on a page with an open shadow root, a direct shadow text node, a same-origin iframe, a hidden iframe, a hidden script and a textarea.textreturned the light, shadow and frame text once each, and nothing from the hidden parts. On this branch, the walker'sreadran in headless Chromium 154 (--dump-domon a local file, no network) on a page with an active tab, ahiddentab panel, adisplay:nonetab panel, a closeddetailsand an open one.textreturned the active tab, both summaries and the open section, and left out both hidden panels and the closed body.htmlkept all of them. No end-to-end run through a live gateway was done.Real-machine verification
N/A -- a new read-only op with no base behaviour to show red; the walker and handler are covered by the tests above, and no gateway was started on this shared host.
Screenshots / video
Why no screenshot: the only file under
website/is a unit test; the op has no UI and changes nothing the dashboard renders.Related Issues
Closes #35742
Carries #36027
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...); PR Hygiene does not count commitsContribution License Agreement