Repository navigation
programs.vscode: VS Code Copilot Chat on GitHub's models through the gateway (copilotChat: githubModels) - #7
Closed
LutzLange wants to merge 8 commits into
Conversation
… models) Baseline test suite for specs/PR-3b.md, written before the implementation. Adds compile-only todo!() stubs for the interface contract's not-yet-existing functions (provider/vscode/settings.rs: settings_path, managed_settings, plan; llm_proxy.rs: split_path_pairing) plus the real, non-stub CAPI_ROUTE entry in ROUTES, core config (VsCodeCopilotChat::GithubModels, VsCodeStartupConfig), and validation rules, so the tests below compile and fail only on the stubs or on genuinely unimplemented routing/tunnel behaviour. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…thubModels)
With `copilotChat: githubModels` the daemon manages the VS Code user
`settings.json` instead of `chatLanguageModels.json`: it merges
`github.copilot.advanced.debug.overrideCapiUrl` pointing at the loopback
proxy's new `/vscode-copilot-capi/<pairing>` route and adds that key and
`github.copilot.internal.capiUrl` to `settingsSync.ignoredSettings` (by value,
user entries kept). The file is written 0600 (an existing looser mode is
tightened, since the URL carries the pairing), removal keeps the mode it
finds, reports are redacted, the sidecar is `.settings.json.agentdesktop`,
comments/trailing commas are a conflict. Removal triggers: program absent,
`ownModels`, `useLlmGateway: false`, no gateway, no `proxyUrl`, no proxy (with
a warning). Under `githubModels` the managed `chatLanguageModels.json` entry is
removed. `daemon.vscode.settings` overrides the path (error in system mode).
Proxy: the CAPI route takes the pairing as the first path segment (VS Code
cannot add a header there); the header is ignored on that route, the segment
is compared in constant time and stripped, and never forwarded or logged.
The client's bearer token moves to `x-llm-token` as on the pass-through
route; a credential-less `GET /_ping` is forwarded without `x-llm-token`
(`POST /_ping` without a credential stays 401). WebSocket upgrades on the CAPI
route are tunnelled: `serve_with_cache` serves `with_upgrades()`, `forward`
takes the inbound `OnUpgrade` before buffering the body, keeps
`Connection: Upgrade`/`Upgrade: websocket` on the request and returns the
upstream 101 with its `Connection`/`Upgrade`/`Sec-WebSocket-*` headers; the
tunnel is awaited by the connection task (under the server's JoinSet, so
dropping the server closes it) and ends when either side closes or the gateway
credential it was opened with reaches its deadline (cache `valid_until`, or the
freshly fetched credential's expiry). Behaviour change for existing callers: a
WebSocket upgrade on any other route is refused with `400 upgrade_not_supported`
(before, it was stripped and forwarded as plain HTTP); other `Upgrade` tokens
are still stripped and forwarded.
Config: `VsCodeCopilotChat::GithubModels`; `validate()` rejects it with
`models`; `validate_daemon` rejects it without `llmGateway.proxyUrl` when the
gateway is used; `VsCodeStartupConfig { config, settings }`. Wiring:
`VsCode { chat_models_path, settings_path }`, `Reconciler::new` gains
`vscode_settings_path`, `ResolvedDaemonArgs.vscode_settings`,
`default_vscode_settings_path(home)`; `--once` and the system-mode provider
message name `settings.json` as well. Workspace tokio gains `io-util` for
`copy_bidirectional`.
Frontend: the VS Code program placeholder shows `copilotChat: githubModels`;
the daemon-config builder has a `proxyUrl` field under the gateway section
(emitted only when set); `DaemonConfigDocument.llmGateway.proxyUrl`.
README: routes-table row, pairing-in-path paragraph, `upgrade_not_supported`,
the Requests paragraph on tunnels (outside the 600 s header limit, outlive a
revocation until the credential expires), and a new section "VS Code Copilot
Chat on GitHub's models through the gateway" (what VS Code sends, `_ping`, the
WebSocket authorisation window, settings rules, caveats: undocumented setting
and HTTP-proxy fallback, stale override, single-user devices, model policy;
upgrade note; gateway route example for the Business host with the Enterprise
host named). Schema regenerated.
Tests. Baseline (test author, f1f8967, unchanged except one struct literal):
13 proxy tests (`llm_proxy.rs`), 18 settings tests (`settings_tests.rs`, 16 on
Linux: the per-OS `settings_path` tests are one per platform), 3 config tests
(`vscode/tests.rs`). Writer: `system_mode_rejects_a_vscode_settings_path`
(daemon.rs) and the Docker test `managed_settings_github_models_lifecycle`
(settings written with the path-pairing URL and both entries, 644 -> 600, a
curl with `Authorization: Bearer tid=x` recorded upstream as `x-llm-token`
with `authorization` null and no pairing in the path, a WebSocket upgrade
through the proxy to a host-side RFC 6455 echo behind `proxyUrl` reaching 101
and echoing one frame with `x-llm-token` seen at the upgrade, removal keeping
a user key and 640). The stub gateway records `x-llm-token` as `llmToken`
(additive). The PR 3a test `system_mode_rejects_a_configured_program_before_writing_anything`
gained the new `settings_path` field in its two `VsCode` literals (ledger).
Agent lib: 217 tests; core: 18.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…n under runuser `Container::write` removes an existing file before writing it: with `fs.protected_regular=2` root cannot open a file another user owns in a sticky world-writable directory such as `/tmp`, so the second config write of the lifecycle tests failed with "Permission denied". `Container::stop_process` also signals the children of the recorded PID: the PID is the `runuser` launcher, which ignores SIGINT while its child runs (as `su` does), so the daemon never stopped and the 10 s wait timed out. Children are read from /proc (the image has no pgrep); a launcher that has already exited is not an error. Both defects made `managed_chat_models_lifecycle` (PR 3a) and `managed_settings_github_models_lifecycle` fail locally at the restart step; both pass now (Linux 6.17, Docker, protected_regular=2). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…iry; review fixes Review round 1 (conformance, adversarial, maintainer) on pr3b-v1. Tunnel lifetime (adversarial B1 + M1, maintainer M1): the deadline of a WebSocket tunnel opened with a cached credential was the cache's reuse window (`valid_until`, at most 60 s), not the credential's expiry, and it was looked up again after the upstream answered, so an entry that expired or was invalidated meanwhile left the tunnel without a deadline. `CachedCredential` now carries the credential's own expiry on both clocks (`expires`, `expires_unix`), `CredentialCache::lookup` returns credential and deadline in one read (`get` stays for tests), and `run_tunnel` takes a `Deadline` checked on the monotonic clock and, at most every 10 s, on the wall clock (adversarial m2: the monotonic clock does not advance across a suspend). A freshly fetched credential gives `Deadline::from_unix(expires_at_unix_seconds)`. Proxy (maintainer minors 1, 2; conformance minor 3): the CAPI route's behaviour is three fields on `ProxyRoute` (`pairing_in_path`, `tunnels_upgrades`, `credential_less_ping`) instead of a prefix comparison; the 101 response has its hop headers stripped and `Connection: Upgrade` plus the upstream's `Upgrade` value put back. Docs (adversarial M2, maintainer M2, minors 3, 4, 5, 9, nit; conformance minors 1, 2): the README states the stale-override exposure (while the daemon is down VS Code keeps sending `tid=`, `gho_` and the pairing to the loopback port, which another local user could bind), the single-user recommendation and the stop order; the plain-JSON rewrite (sorted keys, two-space indent), the comment/trailing-comma conflict as the most likely one with its fix, `github.copilot.internal.capiUrl` as the newer alias; the upgrade note no longer calls `proxyUrl` new; the gateway example names only the lab-verified Business host and tells readers where to find theirs; JWKS URL over https. `ProgramsConfig.vscode` doc comment covers both variants (schema regenerated); the provider README names `settings.json` in the user-mode and dry-run paragraphs. Config (maintainer nit): the `githubModels` `proxyUrl` rule checks `llm_gateway` once. Tests: writer test `github_models_through_the_provider_removes_the_chat_models_entry_and_writes_settings` (adversarial m1: `VsCode::plan` switching ownModels -> githubModels -> ownModels through the provider); the Docker test asserts the stub's 401 passed through (maintainer minor 10); `plan_remove_orphaned` logs when it leaves a non-plain-JSON file alone. Baseline test `capi_tunnel_closes_when_the_cached_credential_expires` gained the two new cache-entry fields in its hand-inserted literal (ledger); assertions unchanged. Agent lib: 218 tests; core: 18. Left for follow-ups (maintainer minors 6, 7, 8): extracting the shared reconcile helpers (third copy), the `Reconciler::new` argument list, and the provider requiring both VS Code paths. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ine; one system-mode check Delta review 1 minors. `forward` matches the route once and reuses the result for the pairing placement, the upgrade refusal, the near-miss check, the ping exemption and the dispatch (maintainer minor 1, remaining part). New writer test `credential_cache_deadline_is_the_credential_expiry_not_the_reuse_window` inserts through `CredentialCache::insert` and asserts the deadline `lookup` returns follows the credential's `exp` (3600 s) while the reuse window stays within `CREDENTIAL_CACHE_TTL`; this is the regression test for round 1's blocker. `daemon.rs` rejects `daemon.vscode.config`/`.settings` in system mode with one check naming whichever is set (maintainer nit); messages unchanged. Left as is, with the reason: the per-connection tunnel slot stays a mutex around an `Option` (one upgrade per HTTP/1.1 connection; a oneshot would add a receiver to poll for no behaviour change); the core doc comments keep naming the setting and the route so the schema tells an operator what the value does; the adversarial nit on the hand-inserted cache entry in the baseline expiry test is covered by the new test going through `insert`. Agent lib: 219 tests; core: 18. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ttings mode caveat
Lab first run on pr3b-v2 (Linux VM, VS Code 1.139, Business seat, Enterprise
standalone gateway).
Stale override after a hand edit (lab G3 -> G5): the three-way rollback treats
a changed value of a managed key as the user's change, so a hand-edited
`overrideCapiUrl` restored by the daemon on restart was recorded in the
sidecar as the user's original and came back on removal, and VS Code got 403
`pairing_invalid`. `MergeOptions::owned_keys` names top-level keys the daemon
owns once a sidecar exists: on re-apply and on removal their current value is
replaced by the value the key had before the first merge (or removed when it
had none), so a hand edit is drift and never the user's value; a value the
user had before the first apply still comes back on removal. The settings
merge owns `github.copilot.advanced.debug.overrideCapiUrl`; every other caller
passes no owned keys and is unchanged. Test
`hand_edited_override_is_replaced_on_reapply_and_not_restored_on_removal`
(settings_tests.rs, writer-added) reproduces the lab sequence.
Gateway example (lab G2): with the default ALPN the gateway negotiates HTTP/2
to GitHub and GitHub answers the forwarded upgrade `400 websocket: the client
is not using the websocket protocol`; VS Code then falls back to plain
`POST /responses`. `backendTLS: {alpn: [http/1.1]}` on the GitHub backend
gives 101 and tunnels of 256 s and 300 s. The README example and paragraph
carry it; the daemon side was correct throughout (capture: upgrade headers
forwarded, no pairing in the path).
README (lab G2c): VS Code saves `settings.json` as 664 with a new inode when a
setting is changed in the UI; the URL is readable to others until the next
apply tightens it; the owned-key rule is stated.
Agent lib: 220 tests; core: 18.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ADME on pass-through headers Delta review 3 minors. Writer test `user_override_from_before_the_first_apply_survives_a_hand_edit_and_returns_on_removal` covers the owned-key rule's other half: a value the user had before the first apply comes back on removal even after a hand edit of the managed URL and a re-apply. README: VS Code's other request headers, including `Copilot-Session-Token` and its machine and device ids, reach the gateway and GitHub unchanged (lab observation). Agent lib: 221 tests; core: 18. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`programs.copilot.models` became a map keyed by model ID when the Copilot CLI program was revised; the Docker test still pushed the earlier list shape, so the daemon refused the config at parse time and the dry-run check failed on the wrong message. The test had not run to completion locally until the harness fixes (cbcc8f1). The user's own `providers.json` fixture keeps its list, which is the file's real shape. With this, the whole Docker integration suite passes locally (9/9: VS Code discovery and both VS Code lifecycles, Copilot CLI, Claude Code, Codex, OpenCode, Cursor, Grok). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Owner
Author
|
Superseded: landed upstream in agentdesktop-dev#135 (VS Code githubModels). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on this fork so the diff is this PR only (base:
copilot/pr3a-vscode-own-models). Sixth change in the series; James gets one integration PR fromcopilot/integration(the tip of the stack) when the series is ready.What it does.
programs.vscodegainscopilotChat: githubModels: VS Code keeps GitHub's own models and its own Copilot sign-in, and the daemon routes Copilot Chat's traffic to GitHub through the gateway. The daemon merges two keys into the VS Code usersettings.json:github.copilot.advanced.debug.overrideCapiUrlpointing at the loopback proxy's new/vscode-copilot-capi/<pairing>route, and that key plusgithub.copilot.internal.capiUrlinsettingsSync.ignoredSettings(by value; user entries kept). The managedchatLanguageModels.jsonentry is removed under this value. User mode only; requiresllmGateway.proxyUrl(a gateway route that forwards to GitHub with the client's own token; example in the README)./vscode-copilot-capi/<pairing>/…: the pairing is the first path segment because VS Code cannot add a header to these requests; compared in constant time, stripped, never forwarded or logged; missing or wrong segment 403pairing_invalidwith the header ignored. The client's bearer token (tid=session token,gho_on/agents/*) moves tox-llm-tokenas on the pass-through route; a credential-lessGET /_ping(Copilot Chat's health check) is forwarded withoutx-llm-token,POST /_pingwithout a credential stays 401. The route's behaviour is three flags onProxyRoute(pairing_in_path,tunnels_upgrades,credential_less_ping).GET /responsesas a WebSocket):serve_with_cacheserveswith_upgrades();forwardtakes the inboundOnUpgradebefore buffering the body, keepsConnection: Upgrade/Upgrade: websocketon the request and returns the upstream 101 withConnection/Upgrade/Sec-WebSocket-*after hop stripping; the tunnel is awaited by the connection task under the server'sJoinSet(dropping the server closes it) and ends when either side closes or the gateway credential it was opened with expires (its own expiry on both clocks, read together with the credential; not the cache's 60 s reuse window). Behaviour change: a WebSocket upgrade on any other route is refused with400 upgrade_not_supported(before, it was stripped and forwarded as plain HTTP); otherUpgradetokens are still stripped and forwarded..settings.json.agentdesktop, plain-JSON rewrite (sorted keys), comments/trailing commas are a conflict. Removal on program absent,ownModels,useLlmGateway: false, no gateway, noproxyUrl, no proxy (with a warning); sidecar-less removal by this daemon's own URL; a user's ownoverrideCapiUrlis overwritten while active and restored on removal.daemon.vscode.settingsoverrides the path (error in system mode).--onceand the system-mode message namesettings.json.VsCodeCopilotChat::GithubModels;githubModelswithmodelsrejected;githubModelswithoutllmGateway.proxyUrlrejected when the gateway is used;VsCodeStartupConfig { config, settings }; schema regenerated.proxyUrlfield under the gateway section of the daemon-config builder; the VS Code program placeholder showscopilotChat: githubModels._ping, the WebSocket authorisation window, settings rules, the stale-override token exposure while the daemon is down and the single-user recommendation and stop order, undocumented-setting caveat with the HTTP-proxy fallback, model policy, restart VS Code, no--dry-runpreview, upgrade note), routes-table row, pairing paragraph,upgrade_not_supported, the Requests paragraph on tunnels, gateway route example (Business host, lab-verified), provider README.json_merge):MergeOptions::owned_keys. A hand edit of the managed override after the merge is replaced on the next apply and never becomes the value removal restores; a value the user had before the first apply still comes back. Found by the lab: before this, a hand-edited URL came back as a stale override on removal and Copilot Chat got 403. Other callers pass no owned keys and are unchanged.Container::writeremoves an existing file first (fs.protected_regular=2stopped root rewriting a tester-owned/tmpfile), andContainer::stop_processalso signals the children of the recorded PID (runuserignores SIGINT while its child runs). The Copilot CLI Docker test pushedmodelsas a list, a shape PR 2's revision replaced with a map; fixed. The whole Docker suite now passes locally, 9 of 9.Process. Spec reviewed (Opus) before the test author, then frozen (
specs/PR-3b.mdrev 2); tests first by a separate agent in their own baseline commit (f1f8967: 13 proxy tests with a hand-rolled RFC 6455 echo, 18 settings tests, 3 config tests); test guard with a two-entry ledger (two hand-written struct literals gained new fields; no assertion changed); gate (218 agent + 18 core tests, clippy, schema, frontend build + lint); three review lenses on two models; delta review; independent lab run. Round 1 findings folded in: the tunnel deadline was the cache's 60 s reuse window instead of the credential expiry and a second lookup could leave a tunnel without a deadline (adversarial blocker + major, confirmed by probe); the README lacked the stale-override exposure text; route behaviour keyed on a prefix string; the 101 skipped hop stripping; a staleProgramsConfig.vscodedoc comment in the schema; provider README; misleading upgrade note; unverified Enterprise host claim. Follow-ups noted: extracting the duplicated reconcile helpers, theReconciler::newargument list, override removal on shutdown.Delta reviews (Opus): delta 1 on the fixes was "pass with minors" (the blocker fix re-probed: a credential with
exp = now + 3600now gives a tunnel deadline of 3599.9999 s, was 59.9997 s); the minors (a regression test for the deadline throughCredentialCache::insert, onematch_routecall inforward, one system-mode check) went into the next commit, and delta 2 was a clean pass. After the lab run, delta 3 on the owned-key change was "pass with minors" (everyMergeOptionscaller checked; a probe confirmed a user value from before the first apply survives); its minors (a test for that case, README note on the headers VS Code sends) are in the last code commit, which adds a test and README text only, so no further delta pass was run. Two cosmetic nits stay: the_pingexemption is tied to the path-pairing branch (correct for the only route that sets both flags), and thesettings/reconcilemodule pair could be renamedchat_models/settingslater.Lab (Linux VM, user mode, VS Code 1.139, Business seat, Solo Enterprise for AgentGateway standalone v2026.9.1 with the
/copilot-proxyroute, controller-pushed config, Playwright over CDP for the GUI steps):chatLanguageModels.jsonentry removed, chat turns and_ping200 through the gateway withllm.client="vscode-copilot"and the enrolled user, pairing absent from VS Code's logs, agent turns over 120 s complete, hand edit restored, re-pairing makes the old URL 403 even with a valid header, comment conflict blocks every managed file and the daemon restart exits, controller rejectsgithubModelswithmodelsand withoutproxyUrl,--once,--dry-runand system-mode messages. Three findings went into code or docs: GitHub refused the forwarded WebSocket upgrade until the gateway backend was pinned to HTTP/1.1 (alpn, now in the README example); the stale override after a hand edit (fixed with owned keys); VS Code savingsettings.jsonas 664 (documented). VS Code's upgrade carries noOrigin.lab/checks/pr3b-vscode.sh: 21 of 21 checks pass, including the hand-edit-then-ownModelssequence that failed before. GUI: an agent turn of 2m 15s over the WebSocket, gatewayGET /copilot-proxy/responses http.status=101 duration=259532ms llm.client="vscode-copilot"; the close at credential expiry is one error line in VS Code's log and the next agent turn answers in 5 s without falling back.Not covered by a test: Enterprise Cloud (
api.enterprise.githubcopilot.com) and other plans' hosts; macOS and Windows settings paths beyond unit tests; Settings Sync across two devices; a VS Code release withoutoverrideCapiUrl.Follow-ups: extract the reconcile helpers now copied three times; a
ProviderPathsstruct forReconciler::new; removing the override on graceful daemon shutdown (bounds the stale-override exposure); the reconcile tick (PR 5), which would also bound the window after VS Code rewrites the file as 664.🤖 Generated with Claude Code