Skip to content

Install user-scope telemetry hooks for project-scope agent init - #22093

Merged
Ella Hathaway (ellahathaway) merged 83 commits into
ellahathaway-telemetry-hook-matcher-scopingfrom
ellahathaway-telemetry-hooks-project-scope
Nov 13, 2026
Merged

Ella Hathaway (ellahathaway) merged 83 commits into
ellahathaway-telemetry-hook-matcher-scopingfrom
ellahathaway-telemetry-hooks-project-scope

Conversation

@ellahathaway

@ellahathaway Ella Hathaway (ellahathaway) commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Description

Stack 3/3, on #22084. aspire agent init --scope project now installs telemetry hooks too. The hooks are written at user scope (~/.copilot/hooks and ~/.claude/settings.json) no matter which scope was requested. Skills and MCP configuration still follow the requested scope, and #22078's per-client gating still decides which clients get hooks.

Because the hooks live in user-level configuration, they apply in every repository where that client runs, not only the one where aspire agent init was run. The PostToolUse matcher from #22084 makes the client launch aspire only for matching tool calls, and only Aspire skill, MCP tool, and reference-file usage is reported. Users can opt out with ASPIRE_CLI_TELEMETRY_OPTOUT=true. The install message now says the hooks were written to user-level configuration.

Why we can't install project-level hooks today

Every hook launches a separate aspire process, so it must fire exactly once per event. Today a project-level hook would make it fire more than once (see #20480):

  • Copilot runs hooks from every source and doesn't dedupe them. It loads the repo's .github/hooks/*.json, ~/.copilot/hooks, .github/copilot/settings(.local).json, ~/.copilot/settings.json, and plugins, then runs all of them. A project hook in .github/hooks plus the user hook that global init installs would launch aspire twice for each event. Claude Code dedupes identical handlers across its settings files, so this is a Copilot problem.
  • Copilot also runs Claude's project hooks. Copilot reads the repo's .claude/settings(.local).json and runs those hooks too, ignoring Claude-only fields such as if. A Claude project hook would fire in Copilot sessions on top of Copilot's own hook, even if the user set up only Claude for the project.
  • Install time can't avoid the duplicates. Project hook files are committed and shared, but user hooks belong to each machine. We can't delete a user hook to make room for a project hook, because it also serves the user's other repos. Skipping the project hook when a user hook exists only helps the developer who ran init, not teammates who clone the repo with their own user hooks.
  • Runtime dedupe is heavy. Each invocation is an independent process with no shared event ID. Deduping would mean hook processes coordinating through shared state on every tool call, which is the hot path.

Installing hooks only at user scope gives exactly one hook per client on each machine. We can revisit project hooks if Copilot starts deduping identical hook commands or applying precedence across hook sources.

Validation: dotnet test --project tests\Aspire.Cli.Tests\Aspire.Cli.Tests.csproj --no-launch-profile -- --filter-class "*.AgentInit*" --filter-class "*.TelemetryHook*" --filter-class "*.AgentTelemetry*" --filter-not-trait "quarantined=true" --filter-not-trait "outerloop=true" passed (464/464). The E2E tests were not run.

Checklist

  • Is this feature complete?
    • Yes. Ready to ship.
    • No. Follow-up changes expected.
  • Are you including unit tests for the changes and scenario tests if relevant?
    • Yes
    • No
  • Did you add public API?
    • Yes
      • If yes, did you have an API Review for it?
        • Yes
        • No
      • Did you add <remarks /> and <code /> elements on your triple slash comments?
        • Yes
        • No
    • No
  • Does the change make any security assumptions or guarantees?
    • Yes
      • If yes, have you done a threat model and had a security review?
        • Yes
        • No
    • No

Fixes #20014

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Fixes #20014

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Each agent client declares the environments that make it eligible for a telemetry hook, and the configurator registers one hook per eligible client instead of per detected environment.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
VS Code's Copilot harness reads user hooks from ~/.copilot/hooks, the same location the Copilot App and CLI use.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Pass the selected clients to the telemetry hook configurator so detected but
unselected clients are not modified, and cover the per-client skip messages
at the command level.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@ellahathaway Ella Hathaway (ellahathaway) changed the title Install agent telemetry hooks at project scope Install telemetry hooks at project scope Nov 10, 2026
@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

Install hooks only for selected clients whose aspire-skills plugin registered or MCP setup succeeded, keep the global-only rule, and drop the unverified VS Code agent marker.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Register the Copilot user hook with a tool-name matcher for skill and Aspire
MCP tools, and register Claude PostToolUse hooks for the Skill and Aspire MCP
tools plus Read calls limited to Aspire skill paths through an if rule. Share
settings merge helpers through TelemetryHookSettings.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Add a test that every canonical skill and reference path matches ClaudeReadIf, and split the merged matcher paragraph in the telemetry spec.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@ellahathaway
Ella Hathaway (ellahathaway) force-pushed the ellahathaway-telemetry-hooks-project-scope branch from d6dd47c to 477bd0e Compare November 11, 2026 00:03
@github-actions

Copy link
Copy Markdown
Contributor

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 22093

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 22093"

Move the Copilot and Claude matcher constants into their agent clients, drop the shared settings helper, and restore the clients to their base shape plus the matcher behavior.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ook-message' into ellahathaway-telemetry-hook-matcher-scoping
…her-scoping' into ellahathaway-telemetry-hook-dedupe-events
…pe-events' into ellahathaway-telemetry-hooks-project-scope

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@github-actions

This comment has been minimized.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ook-message' into ellahathaway-telemetry-hook-matcher-scoping
…ook-message' into ellahathaway-telemetry-hook-matcher-scoping
…elemetry-hooks-project-scope

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@github-actions

This comment has been minimized.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ook-message' into ellahathaway-telemetry-hook-matcher-scoping

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…her-scoping' into ellahathaway-telemetry-hooks-project-scope
@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…her-scoping' into ellahathaway-telemetry-hooks-project-scope
@github-actions

This comment has been minimized.

@IEvangelist David Pine (IEvangelist) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No actionable findings.

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: b39716fb-e8ff-45f3-a048-a710d4f3052e
@github-actions

Copy link
Copy Markdown
Contributor

Tests selector

3 / 101 PR test projects · 4 PR jobs, from 23 changed files.

Selected PR test projects (3 / 101)

Aspire.Cli.EndToEnd.Tests, Aspire.Cli.Tests, Infrastructure.Tests

Selected PR jobs (4)

cli-starter-validation, extension-e2e, polyglot, typescript-api-compat


How these were chosen — grouped by what changed

📦 affected project Aspire.Cli
→ 1 test: Aspire.Cli.EndToEnd.Tests

🔧 src/Aspire.Cli/Resources/AgentCommandStrings.resx (changed source)
→ 1 directly: Infrastructure.Tests

🧪 tests/Aspire.Cli.EndToEnd.Tests/AgentCommandTests.cs (changed test)
→ 1 directly: Aspire.Cli.EndToEnd.Tests

🧪 tests/Aspire.Cli.Tests/Commands/AgentInitClientSelectionTests.cs (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Commands/AgentInitCommandTests.cs (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_InstallationLocations_ShowSelectedAssetsAndSharedConsumers_scope=global_installSkills=False_configureMcp=True.verified.txt (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_InstallationLocations_ShowSelectedAssetsAndSharedConsumers_scope=global_installSkills=True_configureMcp=True.verified.txt (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_InstallationLocations_ShowSelectedAssetsAndSharedConsumers_scope=project_installSkills=False_configureMcp=True.verified.txt (changed test)
→ 1 directly: Aspire.Cli.Tests

🧪 tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_InstallationLocations_ShowSelectedAssetsAndSharedConsumers_scope=project_installSkills=True_configureMcp=True.verified.txt (changed test)
→ 1 directly: Aspire.Cli.Tests

Job reasons

Job Triggered by
cli-starter-validation affected project Aspire.Cli
extension-e2e • src/Aspire.Cli/Commands/AgentInitCommand.cs, src/Aspire.Cli/Resources/AgentCommandStrings.Designer.cs, src/Aspire.Cli/Resources/AgentCommandStrings.resx, src/Aspire.Cli/Resources/xlf/AgentCommandStrings.cs.xlf, src/Aspire.Cli/Resources/xlf/AgentCommandStrings.de.xlf, src/Aspire.Cli/Resources/xlf/AgentCommandStrings.es.xlf, src/Aspire.Cli/Resources/xlf/AgentCommandStrings.fr.xlf, src/Aspire.Cli/Resources/xlf/AgentCommandStrings.it.xlf, src/Aspire.Cli/Resources/xlf/AgentCommandStrings.ja.xlf, src/Aspire.Cli/Resources/xlf/AgentCommandStrings.ko.xlf, src/Aspire.Cli/Resources/xlf/AgentCommandStrings.pl.xlf, src/Aspire.Cli/Resources/xlf/AgentCommandStrings.pt-BR.xlf, src/Aspire.Cli/Resources/xlf/AgentCommandStrings.ru.xlf, src/Aspire.Cli/Resources/xlf/AgentCommandStrings.tr.xlf, src/Aspire.Cli/Resources/xlf/AgentCommandStrings.zh-Hans.xlf, src/Aspire.Cli/Resources/xlf/AgentCommandStrings.zh-Hant.xlf
• affected project Aspire.Cli
polyglot affected project Aspire.Cli
typescript-api-compat affected project Aspire.Cli

Selection computed for commit c02fdf5.

@ellahathaway
Ella Hathaway (ellahathaway) enabled stacked auto-merge (squash) November 12, 2026 23:15
@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

@ellahathaway
Ella Hathaway (ellahathaway) merged commit a67ef19 into main Nov 13, 2026
433 of 437 checks passed
@aspire-repo-bot

Copy link
Copy Markdown
Contributor

Pull request created: #2140

Generated by PR Documentation Check · copilot · auto · 20.1 AIC · ⌖ 1.18 AIC · ⊞ 7.4K

@aspire-repo-bot

Copy link
Copy Markdown
Contributor

📝 Documentation has been drafted in microsoft/aspire.dev#2140 targeting release/17.0.

Documents that telemetry hooks are always written at user scope even with aspire agent init --scope project. Updated aspire-agent-init.mdx and microsoft-collected-cli-telemetry.mdx. Signals: cli_command_file_changed (AgentInitCommand.cs), cli_resource_strings_changed (AgentCommandStrings.resx), pr_body_has_cli_flag_mention (--scope project in PR body).

Note

This draft PR needs human review before merging.

Jose Perez Rodriguez (joperezr) pushed a commit that referenced this pull request Nov 13, 2026
## Description

Backports of the PRs from
https://github.com/microsoft/aspire/stack/22118:
- #22078
- #22084
- #22093

## Checklist

- Is this feature complete?
  - [ ] Yes. Ready to ship.
  - [ ] No. Follow-up changes expected.
- Are you including unit tests for the changes and scenario tests if
relevant?
  - [ ] Yes
  - [ ] No
- Did you add public API?
  - [ ] Yes
    - If yes, did you have an API Review for it?
      - [ ] Yes
      - [ ] No
- Did you add `<remarks />` and `<code />` elements on your triple slash
comments?
      - [ ] Yes
      - [ ] No
  - [ ] No
- Does the change make any security assumptions or guarantees?
  - [ ] Yes
    - If yes, have you done a threat model and had a security review?
      - [ ] Yes
      - [ ] No
  - [ ] No

---------

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: b39716fb-e8ff-45f3-a048-a710d4f3052e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants