Skip to content

fix: tighten guide observe, find_ui and search_docs from review - #18782

Merged
buluoray merged 1 commit into
mainfrom
fix/guides-review-followups
Oct 10, 2026
Merged

buluoray merged 1 commit into
mainfrom
fix/guides-review-followups

Conversation

@buluoray

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Apply the review follow-ups left open when the guides PR (#18443) merged, so a guide's tab observation, find_ui and search_docs behave as the review asked.

The review on #18443 raised points after its head was already final:

  • any turn, including a loop wake, could ask the person's tab for an observation;
  • a not_observed answer was never audited;
  • find_ui made two calls (observe, then language);
  • findByName could match a control inside agent-authored content;
  • search_docs re-read every page on every call;
  • the committed ui-index went stale on locale edits that change nothing in it.

Why it matters

An observation that times out marks the person's running guide stale, so a turn the person did not send must not trigger one. A control inside agent-written text is not something the guide should point at. The other points cost round trips, file reads and spurious gen:ui --check failures for other PRs.

Not a goal

What changed (motivation → approach → change)

Observe is a user-turn action. guide.observe joins _USER_TURN_OPERATIONS, so a wake, cron, app or sub-agent turn gets 403 not_user_turn before any tab is asked. Every observe answer, observed or not_observed, carries ui_lang, and every not_observed outcome is audited with its reason.

One round trip for find_ui. mcp_guide searches, observes once, and relabels from the observation's ui_lang; it calls /api/guide/agent/language only when no observation was made. In findByName.ts, interactive() drops anything inUntrustedContent. search_docs reads pages through an lru_cache keyed by path, mtime and size, so a page is read once until it changes; there is no module-level mutable state.

A stable index digest. In gen-ui-index.mjs, input_digest is the sha256 of the generated index with its own digest field blanked, so only a change to the index content moves it. docs/architecture/mcp.md says so.

Backwards compatibility

Compatible: the observe response only gains a ui_lang field. The not_user_turn refusal applies to callers that were never meant to observe; the guide tools call observe only from a user's turn.

Reader: src/kiro_crew/mcp_guide.py:_find_with_live -- chat -- reads ui_lang from the observation; test_find_ui_reads_the_dashboard_language_from_the_observation_in_one_call
Reader: src/kiro_crew/mcp_guide.py:guide observe tool -- chat -- a user turn still observes; test_guide_observe.py

Tests

  • test_a_turn_the_user_did_not_send_cannot_observe_their_tab: a wake turn gets 403 and no tab is asked.
  • The test_guide_observe.py not-observed cases assert ui_lang and the audit row.
  • test_find_ui_reads_the_dashboard_language_from_the_observation_in_one_call: one observe, and no separate language call.
  • findByName.test.tsx: a control inside agent-authored content is never matched, whatever predicate the caller passes.
  • test_search_docs_reads_each_page_once_until_it_changes.

Local run: 1246 backend guide, find_ui and search_docs tests passed, plus vitest findByName (72). mypy, flake8, black, tsc, eslint, docs-lint, comment-history and gen:ui --check are all clean.

Manual verification

N/A -- unit coverage is sufficient: there is no visual change, and every behaviour above is pinned by the tests listed.

Screenshots / video

Why no screenshot: findByName only filters which elements the guide may point at; nothing renders differently.

Related Issues

Part of #18300

Pattern harvest

Not generalizable: these are review points specific to the guide surface, not a recurring defect class.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Observe now needs a turn the person sent, audits every not-observed
answer and carries the dashboard language so find_ui needs one round
trip. findByName never matches a control inside agent-authored content.
search_docs reads each page once until it changes. The ui-index
input_digest is the hash of the generated content, so an unrelated
locale edit no longer marks the committed index stale.
@buluoray
buluoray requested review from a team and KiroCrewCaptain as code owners October 10, 2026 08:05
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 10, 2026
@buluoray

Copy link
Copy Markdown
Contributor Author

Sensitive change review

  • Head commit: c8b05eb
  • What rule changed:
    • guide.observe joins _USER_TURN_OPERATIONS in dashboard/handlers/guide.py. Asking the person's tab for an observation now needs a turn the person sent.
    • Every not_observed observe answer is now written to the audit log with its reason. Before, only successes were audited.
    • No deny rule, sandbox scope, credential path or approval default changes.
  • Before -> after:
    • Before: any turn whose caller resolved to the slot could ask for an observation. That included a loop wake, a cron or app injection, and a sub-agent completion, and its timeout marked the person's running guide stale.
    • After: those turns get 403 not_user_turn and no tab is asked.
    • Nothing that was blocked before is allowed now.
  • Worst case if wrong: a legitimate observation from a user-sent turn is refused. The guide then reports it could not see the tab and still points. The gate reuses the same _turn_user_sent check that guide.start already passes.
  • Evidence checked: these pass on this head: test_a_turn_the_user_did_not_send_cannot_observe_their_tab, test_guide_observe.py, test_guide_routes.py, test_find_ui*.py, test_mcp_managed_caller_identity.py.
  • No regression:
    • Tools still work: find_ui and the guide observe tool are covered by the find_ui and observe suites, 1246 tests in all.
    • Backward compatible: there are no new config keys. The observe response only gains ui_lang.
    • Existing tests: the guide, find_ui and search_docs suites, flake8, mypy, tsc, eslint, docs-lint, and gen:ui --check.
  • How to undo: revert the commit.

@github-actions

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ✅ PASS

UX-level review of c8b05eb09287652385f34c757dd893f3d2730bf1 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

UX-Verdict: PASS

Check what the guide says when someone asks about a button that sits inside agent-written content, which findByName can no longer match.

[UX-REVIEWED] c8b05eb

Description read

[DESCRIPTION-READ] 3232ae71326e28125fd72574cafb2768064b3884a6724f5fff6c19b078b3f7f1

That sha256 names everything this verdict read: the PR description, plus the 3 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of c8b05eb09287652385f34c757dd893f3d2730bf1 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Check one thing before merge: find_ui called from a wake, cron or sub-agent turn now gets a 403 on observe. Confirm it still returns results, marked not_observed, and still asks for the language separately.

[DESIGN-REVIEWED] c8b05eb

Description read

[DESCRIPTION-READ] fbf08ef52cd974033b2a3e870aea90c1d9728f90654743c27d401a183bac2726

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — 🟡 CONCERNS

Premise-level review of c8b05eb09287652385f34c757dd893f3d2730bf1 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

Moving the untrusted-content check into findByName's interactive() repeats a check that every production shown predicate already makes. Nobody gains from it today.

Not justified as shipped

  • Item 5 — duplicate of website/src/guide/liveRegistry.ts:60 (isDisplayed: if (inUntrustedContent(el)) return false): the description says "findByName could match a control inside agent-authored content". I counted 9 production call sites (searchByName, resolveFind, findOpenerLocation, resolveContainerPath, probeForName). Each one gets either isDisplayed as the default or isGuideTargetVisible, which is export const isGuideTargetVisible = isDisplayed (useGuideStepTracker.ts:331), or a predicate built from it. No caller can reach an untrusted control on base. The new test only fails because it passes everything as never, which is not a real predicate.
    Subtraction: drop !inUntrustedContent(el) and its import from findByName.ts, plus the whatever predicate the caller passes test. isDisplayed remains the one place this check lives.
    Clears when: the extra check is removed, or a production caller is named that passes a shown predicate not built from isDisplayed.

What this change ships

Inventory (8 items) — 6 justified

Intent: close the review follow-ups left open on the guides PR #18443, so a guide observes the person's tab only when they asked, and searches cost fewer round trips. FIX.

  1. A turn the person did not send (a wake, cron, app or sub-agent turn) gets 403 when it tries to observe their tab — justified
  2. Every unanswered observation now writes an audit row with its reason — justified
  3. Observe answers carry the dashboard's ui_lang — justified
  4. find_ui makes one gateway call instead of two — justified
  5. Name matching skips controls inside agent-written content — duplicate of liveRegistry.ts:60 isDisplayed
  6. search_docs reads each page once until the file changes — justified
  7. The ui-index digest hashes the index's content, not its input bytes — justified
  8. The committed ui-index gains the "Interrupted dictation" setting in all locales — undeclared: it catches up with settingsManual.ts:213

[FIRST-PRINCIPLES-REVIEWED] c8b05eb

Description read

[DESCRIPTION-READ] 9a352505824aeb9c227adba35bc99cafea211bd0f73b03f937e743244adeff69

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@buluoray
buluoray enabled auto-merge (squash) October 10, 2026 08:16
@github-actions

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed c8b05eb09287652385f34c757dd893f3d2730bf1 — this comment is updated in place on each push.

Review details

No findings. Nothing here blocks the merge; there are two advisory findings.

FINDING — src/kiro_crew/mcp_guide.py:343 — probe = find_ui(query, lang, surface) searches without label_lang, but find_ui adds the label locale to searched. Example: the dashboard shows zh-CN, lang="en", and the user quotes the Chinese label. The probe then matches nothing, _observe([]) returns {}, and the relabelled d does return results. _with_live(d, obs={}) skips the observe because obs is not None. Every result comes back live: not_observed with no reason, where the old code observed them. The same thing happens to any result in d that was not in the probe's ids. → Fix: in _find_with_live, when _observable_ids(d) is not a subset of the ids the probe observed, pass obs=None (or observe the missing ids) instead of reusing obs.
FINDING — docs/architecture/mcp.md:1585 — the diff adds "guide.observe" to _USER_TURN_OPERATIONS, so find_ui's live observation is now refused with 403 not_user_turn on wake, cron, session_send and subagent turns. The owning spec still lists only guide_start/rename_self and says those turns' "reads still answer". The api_guide_agent_language docstring (guide.py:502) still says "find_ui asks it once per call". → Fix: add the live observation to that mcp.md sentence, note that the observe answer carries ui_lang, and update the docstring in this commit.

[OPUS-REVIEWED] c8b05eb

Verdict parsed from the review's SHA-scoped output markers for commit c8b05eb09287652385f34c757dd893f3d2730bf1.

False positive or not applicable? A repository writer can comment:
/ai-review override fable c8b05eb09287652385f34c757dd893f3d2730bf1: <one-sentence reason>

@github-actions

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of c8b05eb09287652385f34c757dd893f3d2730bf1 and found no blocking issues.

This comment is updated in place on each push.

Review details

FINDING -- src/kiro_crew/mcp_guide.py:965 -- “ids to observe do not depend on the label language” contradicts execution (recurring-defect-patterns): querying "mcp" with lang="en" on a Chinese dashboard adds observable mcp.add-custom and mcp.add-server after observation, returning both as not_observed without asking the answering tab about them -> Fix: correct the docstring to acknowledge that the localized re-search can add unobserved results.
[GPT-REVIEWED] c8b05eb

False positive or not applicable? A repository writer can comment:
/ai-review override gpt c8b05eb09287652385f34c757dd893f3d2730bf1: <one-sentence reason>

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 10, 2026
@buluoray
buluoray merged commit d32c084 into main Oct 10, 2026
102 checks passed
@buluoray
buluoray deleted the fix/guides-review-followups branch October 10, 2026 09:11
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants