Skip to content

fix(skills): say a scoped skill_search miss is out of scope, not absent - #17559

Closed
Premshay wants to merge 1 commit into
kirodotdev:mainfrom
Premshay:fix/skill-search-scoped-no-match
Closed

Premshay wants to merge 1 commit into
kirodotdev:mainfrom
Premshay:fix/skill-search-scoped-no-match

Conversation

@Premshay

@Premshay Premshay commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: when skill_search finds nothing in an agent's mapped skill scope, the message says so, instead of reading as "this skill is not installed".

A session whose agent declares skill:// resources searches only those (session_skill_globs). A miss returned:

No skills matched 'babysit'. Try broader keywords, browse action='list', or load a known full key with action='read'.

babysit was installed. It was simply outside the agent's mapping. Agents read the message as "the skill does not exist" and either gave up or tried more keywords, none of which can match.

Why it matters

The message is the only signal the agent gets, and it points the wrong way: "try broader keywords" cannot succeed for a skill outside the mapping. One session searched for babysit three times and got the same message each time, with the skill installed throughout.

Not a goal

  • Changing what a mapped scope can find. It stays bounded; test_unmapped_custom_agent_still_gets_nothing is untouched.
  • The unscoped message.

What changed (motivation → approach → change)

  • api_skills search and list responses carry scope_size: the number of mapped skill:// resources, or null when the session is unscoped.
  • skill_search names that scope on a miss: No skills matched 'babysit' among the 3 skill(s) mapped to this agent. Search covers only the agent's skill:// resources, so an installed skill outside them is not found; browse action='list' to see the mapped set.

Backwards compatibility

Compatible. scope_size is an added response field. Unscoped sessions and the CLI path get the old message.

Tests

  • New: test_mcp_core_coverage.py::TestSkillSearch::test_no_matches_in_a_mapped_scope_names_the_scope.
  • Extended: test_gateway_list_and_exact_read_share_the_session_mapping asserts scope_size for a 3-skill mapping and for a cleared one (0).
  • Both fail with the source change reverted. 56 tests pass across the two files. black and flake8 are clean on the changed files; ruff and mypy report nothing new.

Pattern harvest

When a scoped skill_search or list comes back empty, the message must name the scope it searched (how many skill:// resources were mapped) and point at the one action that can still succeed (action='list'), instead of suggesting broader keywords that can never match outside the mapping.

@Premshay
Premshay requested a review from a team as a code owner October 6, 2026 23:49
@Premshay
Premshay requested a review from bolichen97 October 6, 2026 23:49
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 7, 2026
@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5, fork) — 🟡 CONCERNS

Premise-level review of e75149421aa949d2366268d2e6973bc8ff8dccb0 via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

The search miss now carries a number, scope_size = len(only). That counts skill:// globs, not skills, and the fix doesn't need the number at all.

Not justified as shipped

  • Item 2 — oversized: the hunk "scope_size": None if only is None else len(only) counts what session_skill_globs returns, which is a list[str] of globs. The message then calls that count "the {scope_size} skill(s) mapped to this agent", so one skill://team/* entry would read as "1 skill". The only consumer, skill_search, needs to know whether the search was scoped, not how big the scope is. The field is also sent on list responses, but a list miss returns at skills.py:292 before it is read, so nothing uses it there.
    Subtraction: replace the integer with presence (scoped or not). Drop the count from the miss message and stop adding the field to list responses.
    Clears when: the number is removed from the message and the field, or it counts resolved skills rather than globs.

What this change ships

Inventory (3 items) — 2 justified

Intent: when an agent has a mapped skill scope, a skill_search miss should say "out of scope" instead of reading as "not installed". This is a FIX.

  1. A search miss in a scoped session names the scope and points to action='list' instead of "try broader keywords" — justified
  2. The discover route's search and list responses gain a numeric scope_size field — oversized: counts globs, and only its presence is used
  3. Unscoped sessions and the CLI keep the old miss message — justified

[FIRST-PRINCIPLES-REVIEWED] e751494

Description read

[DESCRIPTION-READ] 1e02f2768a75c7158e8d9f6fd3040f333523bfd1a8df88232be4422f2fc97be9

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5, fork) — 🟡 CONCERNS

Design-level review of e75149421aa949d2366268d2e6973bc8ff8dccb0 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

scope_size is len(only), a count of glob patterns rather than skills, so a single wildcard skill:// mapping makes the miss message say "among the 1 skill(s)".

Watch

  • The new count is wrong for wildcard mappings. The patch sets "scope_size": None if only is None else len(only), where only comes from session_skill_globs. That function returns the fnmatch globs expanded from each skill:// URI (agent_skill_globs → expand_skill_uri), one glob per URI, not one per skill. Take an agent mapped to skill://.kiro/skills/** that covers 40 skills: a miss returns "No skills matched 'x' among the 1 skill(s) mapped to this agent". The agent reads that as a one-skill scope. The PR exists to fix a misleading message, and this sends the agent to the wrong conclusion again. The description makes the same slip: "the number of mapped skill:// resources" becomes "3 skill(s)" in the message text. The test passes only because its fixture maps three literal skills.
    Clears when: scope_size counts the skills the scope actually resolves to (the same set action='list' pages through), or the message says "N skill:// resource(s)", plus a test with one wildcard URI covering several skills.

[DESIGN-REVIEWED] e751494

Description read

[DESCRIPTION-READ] 6712e05a38050a93d8523b46e158d4554c3dc31f55ebc16609d66a2ca170f890

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review (fork) — ✅ no blocking findings

Reviewed e75149421aa949d2366268d2e6973bc8ff8dccb0 via the fork AI-review pipeline; updated in place on each push.

Review details

One advisory finding; nothing blocks the merge.

FINDING — src/kiro_crew/dashboard/handlers/prompts.py:2380 — "scope_size": None if only is None else len(only) counts the fnmatch globs from session_skill_globs, not the skills. A standard wildcard mapping such as skill://.kiro/skills/*/SKILL.md covers 40 installed skills, yet the model is told "No skills matched 'q' among the 1 skill(s) mapped to this agent". The new test hides this because its fixture maps one literal path per skill. → Fix: either count the skills the scoped catalog actually resolves, or change the wording to "N skill:// resource pattern(s)" and stop calling it a skill count.

[OPUS-REVIEWED] e751494

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review (fork) — ✅ no blocking findings

Reviewed e75149421aa949d2366268d2e6973bc8ff8dccb0 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] e751494

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 7, 2026
A session whose agent maps skill:// resources searches only those, but a
miss read 'No skills matched', which agents took to mean the skill is not
installed. The gateway route now returns scope_size for a mapped scope, and
the tool names it in the no-match message.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@Premshay
Premshay force-pushed the fix/skill-search-scoped-no-match branch from 6a7ea55 to e751494 Compare October 8, 2026 07:52
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 8, 2026
@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5, fork) — ⏭️ skipped

Revision e75149421aa949d2366268d2e6973bc8ff8dccb0 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Advisory — does not block merge.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 8, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor

Thank you @Premshay for this fix and the clear write-up.

Your commit now lands through #19299, a branch on the main repository so CI could run with repository permissions. It is the same change, rebased onto current main, with you kept as the commit author (plus a Co-authored-by: trailer), so the credit stays in git history. All checks on #19299 are green, including PR Readiness; it tracks issue #19294.

Closing this one in favour of #19299. Your branch is left untouched.

@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Oct 12, 2026
@Premshay
Premshay deleted the fix/skill-search-scoped-no-match branch November 13, 2026 11:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants