MSGuide: live Teams recovery, local voice, and question-first UI - #1
Open
sadediwura wants to merge 33 commits into
Open
sadediwura wants to merge 33 commits into
sadediwura wants to merge 33 commits into
Conversation
Use the installed Copilot CLI runtime when configured to avoid bundled-runtime cache races. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add controls-only UIA sensing, private multi-frame verification, Copilot SDK lifecycle wiring, and launcher modes for live and fixture testing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Treat a no-exception URI activation as success because shell launches may not return a process handle. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Refresh stale window choices and hide completed-session controls so each state exposes only its useful next action. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use opaque accessible theme resources and replace the stock window picker with a compact Teams-only selector. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep MSGuide capture-excluded by default while allowing the companion and overlay to appear during full-screen demos. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ensure Start over begins from the simulated permission-off state instead of retaining counters from the previous fixture journey. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Detect the exact Teams camera-on control, add Guide me and Fix it for me modes, invoke one freshly verified action after approval, and require separate approval before restarting Teams. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add scoped camera permission approvals and current-state verification. Replace interactive legacy dictation with local Whisper and explicit microphone/draft controls. Simplify the product shell, preserve consent boundaries, and document the live camera and voice test/presentation scenario. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add continuous same-window Copilot planning, visible task progress, stronger UIA grounding, resilient speech handling, local diagnostics, and reproducible CI dependency locks. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Move the Windows-logo companion to the grounded UI target during execution, retain the overlay badge as a fallback, and tighten browser resource revalidation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep camera discovery, scoped approvals, local voice, and verified completion in one compact companion. Add explicit Guide/Fix choices, remembered placement, accessible hover help, and DPI-safe follow rendering. Revalidate stable camera controls without action retries and retain separate permission/restart consent. Add regression coverage and refresh demo/setup documentation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Test and presentation instructions
Teams camera and voice walkthrough
The runbook covers live camera-off recovery, device-wide permission blocks, Guide me, already-working/Start over, voice input, New voice question/Add more, Settings and screen-context consent, fixture rehearsal, and regression commands.
Validation
Safety and limits
No coordinate clicks, SendInput, automatic action retries, implicit permission changes, or automatic speech submission. Each permission scope and Teams restart requires its own approval. Generic shared-context guidance retains its explicit capture/review/consent boundary and public/synthetic-data restriction. This is not unrestricted autonomous desktop control.