Skip to content

docs(mochi): tell the watch checker an auth wall is not an outage - #13020

Merged
chenmingwei23 merged 1 commit into
mainfrom
fix/mesh-3761-watchlist-auth-reason
Sep 23, 2026
Merged

chenmingwei23 merged 1 commit into
mainfrom
fix/mesh-3761-watchlist-auth-reason

Conversation

@iamwhatever

Copy link
Copy Markdown
Contributor

What changed

A watched page that wants a login is no longer reported as a page that is down.

Where Before After
url check, mochi-watch/SKILL.md one tool: web_fetch fetch first; a 401, or a 403 pointing at sign-in, sends the check looking for a page reader among the mounted tools
failure reasons, mochi-watch/SKILL.md one bucket needs-auth for an auth refusal, access-denied for a bare 403 (bot, rate or region block)
3-strike nudge, mochi-watch/SKILL.md one line: "the target may be down" one wording per reason; the outage sentence is kept only for a target that really failed to answer
nudge payload, mochi-watch/SKILL.md everything in summary summary stays inside its 100-char bubble, the guidance moves to chatMessage
Allowed tools, mochi-watch/SKILL.md fetch plus Slack readers adds the session-carrying page reader the fallback asks for, and one read-only rule for every server
Watch item creation, agents/context/prompt.md target MUST be fetchable target SHOULD be openable, so a page behind a login is still worth watching

The checker is an agent reading a skill, so the bug was in the words. The url
row named one tool, so a 401 had nowhere to go. The nudge had one sentence for
every kind of failure, so the user was told the site might be down when the site
was fine and only the reader was anonymous. Three strikes later that is the worst
sentence to send: it points at the wrong thing to fix. A 403 gets its own reason
for the same reason — a bot or region block is not cured by signing in either.

Nothing is granted. The background agent's tool list in mochi-bg.json is
byte-identical. A user who wants an authenticated reader there still grants it in
Settings, and build_policy already maps that grant onto the background agent —
this change only makes the grant usable once it exists. Picking a tool by
description is also where a model could reach for a write tool, so the read-only
rule now covers every server instead of Slack alone.

test/test_mochi_agent_contract.py pins each half: the url row must name a
fallback path rather than a single tool, a bare 403 must not be called a login
wall, every nudge bubble must fit the 100-char cap, the auth wording must not
claim an outage, and the background grant is asserted byte-for-byte so a later
round cannot widen it quietly.

Backwards compatibility

No behaviour change for a page that answers anonymously, and no schema change:
the new reasons ride the free-text result field historyEntry already carries.

Testing

Related backend set green (134 tests). Two mutants, each caught: restoring the
single-tool url row fails the fallback tests, and restoring the over-long
bubble fails the cap test. Under both, the grant test still passes — so the tests
measure the skill prose, not the authorisation.

Why no screenshot: agent instruction text only — no rendered surface changes.

Mesh-3761

@iamwhatever
iamwhatever requested review from a team and krishdhasmana September 23, 2026 06:45
@iamwhatever

Copy link
Copy Markdown
Contributor Author

Intent: A watch item whose page needs a login must be told it needs a login. The background checker had one tool for a url item and one sentence for every failure, so an auth-walled page failed three times and was announced to the user as a possible outage — pointing them at the wrong repair.

Not a goal: Granting any tool to the unattended mochi-bg agent — its JSON stays byte-identical, and a user-granted reader reaches it through Settings as it already did. No auth branch in product code: the checker is an agent reading a skill, so the fix is the instruction. No schema change for the new reasons; they ride the free-text result that historyEntry already carries.

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Sep 23, 2026
@github-actions

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of f1ec25cb083abd607783e229d7af537554afdc83 and found no blocking issues.

This comment is updated in place on each push.

Review details

FINDING -- src/kiro_crew/apps/builtins/mochi/agents/context/prompt.md:52 -- "if the user has granted one" accepts the default chat-only MCP grant, but mochi-bg remains denied and every scheduled authenticated check records needs-auth -> Fix: require granting the reader to the background agent and say so in the nudge. (origin: validation)
[GPT-REVIEWED] f1ec25c

False positive or not applicable? A repository writer can comment:
/ai-review override gpt f1ec25cb083abd607783e229d7af537554afdc83: <one-sentence reason>

@github-actions

github-actions Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — ✅ PASS

Design-level review of e4de7f501477418b1965e0684331124f94194218 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Confirm kiro-cli's web_fetch result surfaces HTTP status/headers — the new 401/403 branch never fires on an opaque error string.

[DESIGN-REVIEWED] e4de7f5

@github-actions

github-actions Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — ✅ PASS

Premise-level review of e4de7f501477418b1965e0684331124f94194218 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: PASS

Verify end-to-end that a Settings grant scoped to bg actually mounts a page reader in a live background check — the new tests pin prose, not mounting.

What this change ships

Inventory (8 items) — 8 justified

Intent: stop reporting a login-walled watched page as an outage, and let the checker use a user-granted signed-in reader — a FIX.

  1. A 401/sign-in-403 watch check retries with a user-granted session-carrying page reader — justified
  2. Failure reason recorded as needs-auth vs access-denied, as free text — justified
  3. Three-strike nudge wording now matches the failure cause; the outage sentence survives only for real failures — justified
  4. Nudge guidance moved out of the 100-char bubble into chatMessage — justified
  5. The session-carrying page reader is added to the skill's allowed-tools list — justified
  6. The read-only rule now covers every server, not only Slack — justified
  7. Watch creation accepts a login-walled target (MUST fetchable → SHOULD openable) — justified
  8. The background agent's tool grant is now byte-pinned by a test — justified

[FIRST-PRINCIPLES-REVIEWED] e4de7f5

@github-actions

github-actions Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5 Review — ✅ no blocking findings

Reviewed e4de7f501477418b1965e0684331124f94194218 — this comment is updated in place on each push.

Review details

A chat-audience MCP grant — the Settings default — never reaches the background checker, so the newly permitted login-walled target can never be checked and the nudge asks for a grant the user already made.

FINDING — src/kiro_crew/apps/builtins/mochi/agents/context/prompt.md:52 — an authenticated page reader if the user has granted one is true for the chat agent but not the checker: SettingsPanel.tsx:1092 pushes a newly enabled server as agents: ['chat'], agent_policy.build_policy maps that audience to CHAT_AGENT only, and mochi-bg.json ships "includeMcpJson": false, "mcpServers": {}, so _apply_agent_mcp_policy never adds @reader to bg's tools — the chat agent stores a login-walled target it believes is checkable, every check records needs-auth, and the 3-strike chatMessage tells the user to "Grant a page reader that carries your session", which they did → Fix: name the audience in both changed lines — prompt.md "…if the user has granted one to the background agent", and the needs-auth chatMessage "…grant it to the background agent in Settings → MCP".

[OPUS-REVIEWED] e4de7f5

Verdict parsed from the review's SHA-scoped output markers for commit e4de7f501477418b1965e0684331124f94194218.

False positive or not applicable? A repository writer can comment:
/ai-review override fable e4de7f501477418b1965e0684331124f94194218: <one-sentence reason>

@iamwhatever

Copy link
Copy Markdown
Contributor Author

Inherited red, not this PR: Backend Tests (Windows) (7) fails on
test/test_crew_log_exc_info_sites.py::test_the_exc_info_sites_left_in_crew_log_are_exactly_the_vetted_ones.

The assertion names new exc_info sites in crew_log/emit.py and
crew_log/session_tree_projection.py. This PR changes neither file — its whole
diff is two agent instruction files under mochi/agents/ plus
test/test_mochi_agent_contract.py.

Evidence: the test fails on the base commit fe71a59d6140 with none of this
branch's changes applied, so the failure is deterministic and inherited through
the merge ref rather than introduced here.

It is already being fixed in #13022, which vets exactly those sites. No change
here, and no new push: once that lands, the merge ref picks it up and the failed
shard can be re-run on this same head.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 23, 2026
The url kind bound the check to one tool, and the 3-strike nudge had one
sentence for every failure. A page behind a login therefore failed three
times and was announced to the user as a possible outage.

The url row is now a two-step rule: fetch first, and on 401/403 look for a
page reader among the tools actually mounted. The nudge takes its wording
from the failure reason, so needs-auth asks for a login instead of
reporting the site down. The chat prompt no longer refuses to create an
item whose target needs a session.

No grant changes: mochi-bg's tool list is byte-identical.
@iamwhatever
iamwhatever force-pushed the fix/mesh-3761-watchlist-auth-reason branch from f1ec25c to e4de7f5 Compare September 23, 2026 07:33
@github-actions

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ⚠️ review incomplete

GPT 5.6 did not produce a complete verdict for e4de7f501477418b1965e0684331124f94194218; inspect the workflow logs and re-run it.

This comment is updated in place on each push.

See the GPT 5.6 Review job logs; this commit has no completed GPT verdict.

False positive or not applicable? A repository writer can comment:
/ai-review override gpt e4de7f501477418b1965e0684331124f94194218: <one-sentence reason>

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 23, 2026
@github-actions

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of e4de7f501477418b1965e0684331124f94194218 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] e4de7f5

False positive or not applicable? A repository writer can comment:
/ai-review override gpt e4de7f501477418b1965e0684331124f94194218: <one-sentence reason>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 23, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor Author

Status on head e4de7f5014: 91 of 93 checks pass. The two reds are one
infrastructure failure and its rollup.

Opus 5 Review fails in its Resolve human override step, before any model runs:
reading the PR's comments returns API rate limit exceeded for installation
(HTTP 403) on all three of the step's attempts, so the lane fails closed and the
review step is skipped. PR Readiness is red as a consequence of that one lane.

This is not a verdict on the change. The four sibling lanes — Design Review,
First Principles Review, GPT 5.6 Review, UX Review — all pass on this exact
head, and Security Scope Review passes too.

Reruns tried on this same head, no new push: a --failed rerun of all seven
jobs the earlier rate-limit wave took out (six recovered, this one did not), then
two full reruns of this run. Each landed in a still-exhausted window: 07:34,
08:31, 09:24. The budget is the installation's and is shared across the
repository, so it is not something this PR can clear.

The earlier inherited failure is gone: test_crew_log_exc_info_sites.py was red
on the old base and is green since #13007 landed, which this branch is now
rebased onto.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 23, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor Author
  • rebutted — span=fecc79f2d5c2 — prompt.md:52 says a granted reader, not a default one

The finding is right that agents: ['chat'] is the DEFAULT at
SettingsPanel.tsx:1092, and wrong that the checker therefore cannot be
granted. The same server row carries two MiniToggles, chat and bg
(SettingsPanel.tsx:1172-1174); toggleAgent stages the bg audience and
agent_policy._AUDIENCE_TO_AGENT maps bg to BG_AGENT, so
build_policy grants that server to the background checker.
The sentence claims a reader is reachable "if the user has granted one", which
is exactly that path — it does not claim the default reaches the checker.
mochi-bg.json staying byte-identical is the point: the static floor is
unchanged and the user's own grant is what opens the reader. The skill's
needs-auth nudge tells the user to grant one rather than assuming one.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: passed Eligible automated validation passed for the current revision readiness: checking Automated validation is still running labels Sep 23, 2026
@chenmingwei23
chenmingwei23 merged commit b4dbc7c into main Sep 23, 2026
98 of 108 checks passed
@chenmingwei23
chenmingwei23 deleted the fix/mesh-3761-watchlist-auth-reason branch September 23, 2026 16:07
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Sep 23, 2026
wknight1 pushed a commit to wknight1/KiroCrew that referenced this pull request Oct 5, 2026
…rodotdev#13020)

The url kind bound the check to one tool, and the 3-strike nudge had one
sentence for every failure. A page behind a login therefore failed three
times and was announced to the user as a possible outage.

The url row is now a two-step rule: fetch first, and on 401/403 look for a
page reader among the tools actually mounted. The nudge takes its wording
from the failure reason, so needs-auth asks for a login instead of
reporting the site down. The chat prompt no longer refuses to create an
item whose target needs a session.

No grant changes: mochi-bg's tool list is byte-identical.

Co-authored-by: Zejiang Guo <zejiangg@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants