Repository navigation
Conversation
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: the feature is additive and off by default, the run reuses the probe's sandboxed spawn and teardown, and governance is still checked under the dashboard surface. [DESIGN-REVIEWED] 5aa6a6c Description read[DESCRIPTION-READ] d145f6441717d2d4ccc3d62f26ab091a905cf13bd951fa504e8acac9ed524ec7 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
UX Review (Opus 5.5) — ✅ PASS (a BLOCK with no Trigger and no Outcome does not block)UX-level review of UX-Verdict: PASS cannot evaluate: missing the panel's empty, limit, truncation, field-error, config-error and in-flight states. The diff adds strings for each of these, and none of the 12 screenshots shows them. Evidence gaps
Notes
[UX-REVIEWED] 5aa6a6c Description read[DESCRIPTION-READ] 6ede0898c08391205533efb7e7cbd6d0d64dbcb7cddd82433a8eafd3452e86d7 That sha256 names everything this verdict read: the PR description, plus the 15 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks: the panel is opt-in, owner-only and new, with no existing owner route that calls a tool. One part goes further than the Goal needs: the panel builds a form from each tool's schema. Notes
What this change shipsInventory (8 items) — 7 justifiedIntent: the owner checks that one tool of a local MCP server works with exact arguments, without a chat turn. ADDITION.
[FIRST-PRINCIPLES-REVIEWED] 5aa6a6c Description read[DESCRIPTION-READ] 73372bf4f6c9136a15506ad6bee56e8c29972448b2d3330f05d839d71ccc2003 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — 🔴 changes requested (blocking)GPT 6.1 found at least one blocking issue that must be resolved before merging This comment is updated in place on each push. BLOCKING -- website/src/pages/overview/McpTryTool.tsx:208 -- Prototype-named arguments crash the tool form
BLOCKING -- website/src/pages/overview/McpTryTool.tsx:23 -- Dynamic argument schemas become unusable empty forms
[BLOCK-MERGE] 5aa6a6c Adjudication (Opus 5.5) — is blocking on each finding proportionate?F2 (MEDIUM): this is a common path that fails without crashing. Conditions confirmed: F1 (fenced): the panel crashes, but no data is lost or corrupted. Conditions confirmed: 🏷️ Fenced finding(s) machine-flagged as likely edge caseThe security fence keeps these findings blocking regardless of adjudication; the only clearance path is a human override. The annotate-only pass judged the condition combination each one requires extreme, and pre-drafted the override rationale below. A repository writer must independently verify a rationale before posting it — it is machine-authored, and a wrong override on a security-class finding ships exactly the class the fence exists to stop.
Replace False positive or not applicable? A repository writer can comment: |
21c1105 to
7d2f148
Compare
|
Round 1 answers, head 7d2f148. 1. Static Ratchet Gate (error code, redaction sink). Fixed in 7d2f148. Every refusal and failure of 2. UX block: the confirm step. Fixed in 7d2f148. The warning reads: This runs "tool" on the real "server" server and can change its data. This panel cannot undo it. The button reads Run "tool" for real, and the quoted names are pinned in 3. McpTryTool.tsx:49 (server starts as ''). Fixed in 7d2f148. The pick is kept only while it is still listed, otherwise the first server is used, for the select, the Load button and every request. 4. McpTryTool.tsx:67 (stale listing). Fixed in 7d2f148. A listing carries the server it was requested for and shows only for that server, and an answer for a server the user has left is dropped before it sets tools, a result or an error. "drops a listing that lands after the user chose another server" fails on the old head. 5. McpTryTool.tsx:33 (trimmed strings). Fixed in 7d2f148. Strings and enums go out as typed; only numbers are trimmed. A field never touched is left out (and an empty enum or boolean choice reads as not set), while a string emptied on purpose is sent as 6. mcp_try_tool.py:164 (process tree). Fixed in 7d2f148. 7. McpTryTool.tsx:120 (error surface). Fixed in 7d2f148. A tool's 8. mcp_try_tool.py:68 (silent overflow). Fixed in 7d2f148. A tool past 200, past the listing budget, or with a name too long to show is counted. The listing returns Also. Round 0's Bengali, Hindi and Russian strings used the formal register, which |
Sensitive change review
|
7d2f148 to
ef52b82
Compare
|
Round 2 answers, head ef52b82. 1. Process teardown (Opus and GPT 2. Opus BLOCKING 3. GPT 4. CI 5. Frontend. Fixed in ef52b82.
6. Whole suites and size. The whole backend suite on this patch fails only where plain main fails on this machine, plus one timing test ( |
Sensitive change review
|
ef52b82 to
919949b
Compare
b1f2f9c to
4a70ac0
Compare
|
Round 6 answers, head 4a70ac0 (rebased on main 5ff3fed). 1. Opus 2. Opus 3. GPT 4. GPT 5. UX lane: the inline confirm state is not shown. Done. The light still of the inline confirm (after pressing "Run tool…", with "Run … for real" and Cancel) is back in the description, which stays under 8,000 bytes after upload. 6. CI. The two failing tests, Whole suites: the backend suite fails only where plain main fails on this machine; the one id not in the earlier known sets (a find-UI test) fails the same on a clean main with the website deps installed. The website vitest passes in four shards (71,396 tests). |
Sensitive change review
|
4a70ac0 to
d70295a
Compare
|
Round 7 (head d70295a), on current main c87ea0c. Answers the three blocking GPT findings for 4a70ac0; nothing else changed (no website file).
|
Sensitive change review
|
d70295a to
24afa04
Compare
|
Round 8 (head 24afa04), rebased on main cb6cffb. Answers the two blocking GPT findings for d70295a; nothing else changed.
Maintainer review: the whole-design, first-principles, UX and Opus lanes have passed for several rounds, and each recent GPT round has found a new edge case in a large surface. Could a maintainer look at this PR? From here, further automated findings will be answered in one batch after a maintainer has looked, rather than round by round. |
Sensitive change review
|
The MCP Servers page lists a server's tools but gives the owner no way to run one and see what it returns. With the new mcp.tool_tester setting on (off by default, applies at restart), POST /api/mcp/try-tool is registered. It is owner-only. It runs one local server through the discovery probe, which gains an optional one-call step after its listing (ToolCallStep), so the run shares the probe's sandboxed spawn on a private loop, its handshake and its process-group teardown; a plain probe is unchanged. The step's answer is read against the run's 30-second limit, not the shorter handshake timeout. A muted or remote server is refused before it starts, a tool not declared readOnlyHint needs a confirm, a switched-off tool is refused, and so is a tool the governance ceiling or the dashboard's profile denies (the same mcp-scope check an agent's call meets; a check that cannot run refuses). Every run is written to the security event log ahead of the call, which is refused when that write fails. The result is redacted and then cut to 65,536 characters with a note. The panel builds a form from the tool's input schema for simple types, with a multi-line box for strings, and falls back to a raw JSON box.
24afa04 to
5aa6a6c
Compare
|
Round 9 (head 5aa6a6c), rebased on main 1eb2dd7. It fixes this PR's own Windows test failure and answers the two blocking findings for 24afa04; nothing else changed. Windows CI,
First-state stills. The description now shows the panel before "Load tools" is pressed, in light and dark: the title, the one-line explanation, the server picker and the Load tools button. No string or look changed. The request in the round 8 reply for a maintainer review still stands. |
Sensitive change review
|
Problem / Motivation
Goal: the owner can run one tool of a configured MCP server from the MCP Servers page and see its raw result, without a chat turn.
POST /api/mcp/probelists a server's tools, but no route calls one. Todaytools/callruns only through an agent turn, behind approvals, and the owner reads the model's summary instead of the result.Why it matters
Someone who has just added or fixed a server needs to know whether a tool works with exact arguments. A chat turn costs model time and may pick others.
Not a goal
url) servers: the run starts a server's command, so only local ones are listed.What changed (motivation → approach → change)
Round 9 (head 5aa6a6c): a timed-out run's backstop waits 20 s past the run's limit, so the probe's own teardown (kill, group reap, temp sweep) is never cancelled partway, and the run's limit also bounds the handshake reads; raw JSON refuses a number too large to be finite (
1e400would be sent asnull); a Windows line-ending assertion is fixed. Product code is about 850 lines; a split puts the setting, probe step, route and sink row (about 590) first and the panel (about 260) second.mcp.tool_tester(config/integration_sections.py,section_builders.py,config_all_settings.py): off by default, only a realtrueturns it on, read per request, so no restart is needed.mcp_discovery_probe.py:probe_servertakes an optionaltool_callstep that sends onetools/callafter a goodtools/liston the same pipe, so a run shares the probe's sandboxed spawn, handshake, cancel forwarding andfinally, which reaps the whole process group. A cancel queued while the step chooses lands before anything is written. The handshake and the answer are read against the run's 30-second limit; meanwhile stderr is drained, keeping its first 32 KiB as a probe does, so a verbose server still answers. A plain probe is unchanged (a test that passes on main pins it).dashboard/routes/agent_config.py:POST /api/mcp/try-toolis always registered through a small owner-gated handler that reads the setting off the loop, answers 404 while it is off, and imports the tester module only once it is on, so start-up reads no config.dashboard/handlers/mcp_try_tool.py(new): lists tools or calls one once through that step. A tool name the redactor would change is counted as withheld, never listed or run (403tool_name_withheldwhen asked for). A schema that is not an object, is over the size cap or stops being JSON once redacted is sent asnull, not{}. A tool not markedreadOnlyHint: trueneedsconfirm: true; a tool or server switched off on the page is refused. A tool the governance ceiling or the profile bound to the dashboard surface denies is refused with 403governance_denied(gate_decisionfor@server/toolin themcpscope, asked underdashboard:mcp-try-tool, which classifies asdashboard); a check that cannot run refuses with 503. Every run writes a critical security event before the call and one with the outcome; if the first cannot be written nothing runs. Every error carries acode; output is dumped, redacted and then cut to 65,536 characters in a worker thread; the module is a registered redaction sink.website/src/pages/overview/McpTryTool.tsx(new, mounted inMcpTab.tsx): the panel. For a tool that can change data the button reads "Run tool…" with "Can change data. You'll be asked to confirm." beside it; a read-only tool keeps "Run tool". A run in flight shows the button busy, not switched off. A tool error leads with the server's own text. A schema usingallOf,anyOf,oneOf,notor$ref, or anullone, gets the raw JSON box. A line counts withheld tools; a failed run shows "The run did not finish." above the server's text. The first tool not switched off is selected (none when all are). An error, hint, answer or confirm does not outlive its tool or server.docs/configuration.md; regeneratedconfig-baseline.json.Backwards compatibility
With the setting off (the default) the route answers as an unknown path does (a test that passes on main pins it), start-up does no new work, and the panel is not drawn. A probe without the new step behaves as on main.
Tests
{"value":1e400}goes out asnull.src/i18n/: 1,262 passed.python -m pytest test/test_mcp_try_tool.py -n0andnpx vitest run src/pages/overview/McpTryTool.test.tsxManual verification
The real panel on a loopback vite page with fake answers, light and dark: every state above.
Sibling sweep
Sibling:
dashboard/handlers/mcp_probe.py-- not affected -- no tool stepSibling:
mcp_gateway/app_call.py-- not affected -- it asks the same governance questionPattern harvest
Rule candidate: review-prompt
Pattern: a new way to invoke a tool asks the governance question under its own surface's session key.
Related Issues
None filed. Searched open PRs and issues for "try a tool", "tool tester" and "mcp probe".
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement