Skip to content

Test the telemetry consent path in release builds, and enable the telemetry smoke tests on arm64 - #1249

Merged
Elliot (theelliotm) merged 9 commits into
mainfrom
user/emichlin/telemetry-smoke-arm64
Sep 23, 2026
Merged

Elliot (theelliotm) merged 9 commits into
mainfrom
user/emichlin/telemetry-smoke-arm64

Conversation

@theelliotm

@theelliotm Elliot (theelliotm) commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

📖 Description

TLDR; Current telemetry consent store / policy key tests use the debug build with a fake environment. This test will actually mutate the machine using the release build of MXC. Runs during build in CI.

--

The telemetry consent store and policy key both have debug-only overrides gated on cfg(any(test, all(feature = "test-support", debug_assertions))). A shipped binary resolves the store through SHGetKnownFolderPath and reads policy from HKLM, and that release-only shape had no coverage. The existing smoke tests refuse release binaries, and cargo test --release still compiles the override branch in because cfg(test) is on.

Adds tests/scripts/run_telemetry_consent_release_test.ps1, which drives a release wxc-exec.exe against the real per-user store and the real HKLM policy key. It covers CLI exit codes, fresh-store resolution, fail-closed behavior on presenter EOF, corrupt and stale-prompt records, and the policy ceiling. It also sets both override environment variables and asserts the release binary ignores them, which is the property the cfg gating exists for.

Wired into the Windows build job, which already produces the release executor, so it adds about a second and no extra build time. The test mutates the machine, so it requires -AcceptRealMachineMutation and runs only on the ephemeral GitHub-hosted runners that job declares. Prior state is backed up and restored, and -RequirePolicyCeiling makes CI fail rather than silently skip the HKLM section if the runner ever stops being elevated.

Also enables the two existing telemetry smoke tests on arm64. They were gated to x64 with no arch-specific reason: there is no target_arch cfg in the telemetry code, and logman and tracerpt are native on ARM64. While enabling them, fixed the ETW test printing a blank exit code, since Start-Process -PassThru does not retain the process handle unless it is dereferenced first.

🔗 References

Resolves #691

🔍 Validation

  • New release-path test passed on both windows-2025 and windows-11-arm, with all seven checks reporting and the HKLM section running elevated rather than skipping.
  • Both existing telemetry smoke tests pass on arm64, the consent one on a native ARM64 dev box and the ETW one elevated.
  • Verified the override assertion actually discriminates: the environment variable names match the constants in consent.rs and policy.rs, and the owner-PID check requires the direct parent, which the script satisfies. A debug binary would honor the override, so the release result is meaningful.
  • Confirmed the script leaves the consent store and the HKLM value as it found them.

✅ Checklist

📋 Issue Type

  • Bug fix
  • Feature
  • Task
Microsoft Reviewers: Open in CodeFlow

Elliot (theelliotm) and others added 5 commits September 22, 2026 18:07
The consent store location and policy key both have a test-only override
branch gated on `any(test, all(feature = "test-support", debug_assertions))`,
so the shipped release shape -- real %LocalAppData% store and real HKLM
policy key -- is never executed. `cargo test --release` does not close this:
`cfg(test)` is on there, so the compiled function still contains the override.

Add a test that drives a release wxc-exec.exe through the CLI and asserts the
overrides are inert: it seeds a temp store saying granted and a registry key
saying blocked, sets both override variables, and requires the binary to
ignore them. It also covers real store creation and atomic replace, corruption
and stale-prompt recovery, piped-EOF fail-closed, and the HKLM policy ceiling.

Because it mutates real machine state it refuses to run without
-AcceptRealMachineMutation, and refuses debug binaries outright. The temporary
workflow runs it on ephemeral GitHub-hosted runners only and asserts
RUNNER_ENVIRONMENT, since self-hosted runners also set GITHUB_ACTIONS.

Refs #691

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings September 23, 2026 21:00
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@theelliotm Elliot (theelliotm) changed the title Cover the telemetry consent path in release builds, and enable the telemetry smoke tests on arm64 Test the telemetry consent path in release builds, and enable the telemetry smoke tests on arm64 Sep 23, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Cleanup can leave a newly created consent lock file on the real machine.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
What changed in this PR

Adds release-build coverage for telemetry consent and broadens ARM64 smoke-test coverage.

Changes:

  • Adds release-path consent and HKLM policy testing.
  • Enables telemetry smoke tests on ARM64.
  • Preserves ETW process exit-code access and updates documentation.
File Description
.github/​workflows/​Build.Windows.Job.yml Runs telemetry tests across Windows architectures.
tests/​scripts/​run_telemetry_consent_release_test.ps1 Adds release consent-path testing.
tests/​scripts/​run_telemetry_etw_smoke_test.ps1 Retains the process handle for exit-code reporting.
tests/​scripts/​README.md Documents telemetry test requirements.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread tests/scripts/run_telemetry_consent_release_test.ps1
Copilot AI review requested due to automatic review settings September 23, 2026 21:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The release test does not fully isolate or restore pre-existing consent-marker and policy state.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 High severity

Open (1)
Resolved since last review (1)
Previously missed (2)

In code that hasn't changed since last review

Medium severity Clear ambient AllowTelemetry before unrestricted-policy tests

tests/​scripts/​run_telemetry_consent_release_test.ps1:379

The ambient AllowTelemetry value is backed up but never removed before tests that require policy = unrestricted. Any machine already configured with 0, 1, or 3 fails Test-FreshStore for reasons unrelated to the release path. Clear an existing value after backing it up (or reject a non-elevated run that cannot do so), then restore it in finally.

Medium severity Preserve original registry value kind during policy restore

tests/​scripts/​run_telemetry_consent_release_test.ps1:428

This restores every pre-existing policy value as REG_DWORD, but wrong-type values are explicitly valid fail-closed input for this subsystem. For example, a pre-existing REG_SZ is converted (or cleanup throws), so the script does not restore the machine state it backed up. Record the original RegistryValueKind and restore the value with that exact kind.

Comment thread tests/scripts/run_telemetry_consent_release_test.ps1
Copilot AI review requested due to automatic review settings September 23, 2026 21:19

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The release test does not reliably isolate or exactly restore pre-existing machine policy state.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 High severity

Open (1)
Resolved since last review (1)

Comment thread tests/scripts/run_telemetry_consent_release_test.ps1 Outdated
Copilot AI review requested due to automatic review settings September 23, 2026 21:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Existing machine state can invalidate tests, and timeout cleanup can race a surviving process.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 High severity

Open (1)
Resolved since last review (1)
Previously missed (2)

In code that hasn't changed since last review

Medium severity Remove withdrawal marker when resetting consent state

tests/​scripts/​run_telemetry_consent_release_test.ps1:234

Remove-RealConsentFile leaves a pre-existing withdrawal marker in place. Once that marker is at least five seconds old, the first status call performs withdrawal recovery by writing a denied record, so this “fresh store” check fails instead of exercising the absent-store path. The marker is already backed up for restoration; remove it as part of this reset.

Medium severity Normalize AllowTelemetry before policy assertions

tests/​scripts/​run_telemetry_consent_release_test.ps1:410

The test backs up an existing AllowTelemetry value but never clears it before running checks that require policy == 'unrestricted'. On a machine already configured with either 0 or 3, the fresh/lifecycle assertions fail before the policy-ceiling section, despite the script claiming to preserve and restore prior state. Normalize the backed-up value while elevated, and fail early with a clear prerequisite when a non-elevated session cannot do so.

Comment thread tests/scripts/run_telemetry_consent_release_test.ps1
Ensure the process waits for exit after killing it.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 23, 2026 21:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The release-path coverage is comprehensive, explicitly guarded, and restores the affected consent and policy state.

Review effort: Balanced
Findings: None

Resolved since last review (1)

@theelliotm
Elliot (theelliotm) marked this pull request as ready for review September 23, 2026 21:47
@theelliotm
Elliot (theelliotm) requested a review from a team as a code owner September 23, 2026 21:47

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@theelliotm
Elliot (theelliotm) merged commit dd0ea25 into main Sep 23, 2026
31 checks passed
@theelliotm
Elliot (theelliotm) deleted the user/emichlin/telemetry-smoke-arm64 branch September 23, 2026 22:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Telemetry consent path has no automated coverage in release builds

3 participants