Skip to content

programs.vscode: VS Code Copilot Chat on GitHub's models through the gateway (copilotChat: githubModels) - #7

Closed
LutzLange wants to merge 8 commits into
copilot/pr3a-vscode-own-modelsfrom
copilot/pr3b-vscode-github-models
Closed

LutzLange wants to merge 8 commits into
copilot/pr3a-vscode-own-modelsfrom
copilot/pr3b-vscode-github-models

Conversation

@LutzLange

Copy link
Copy Markdown
Owner

Stacked on this fork so the diff is this PR only (base: copilot/pr3a-vscode-own-models). Sixth change in the series; James gets one integration PR from copilot/integration (the tip of the stack) when the series is ready.

What it does. programs.vscode gains copilotChat: githubModels: VS Code keeps GitHub's own models and its own Copilot sign-in, and the daemon routes Copilot Chat's traffic to GitHub through the gateway. The daemon merges two keys into the VS Code user settings.json: github.copilot.advanced.debug.overrideCapiUrl pointing at the loopback proxy's new /vscode-copilot-capi/<pairing> route, and that key plus github.copilot.internal.capiUrl in settingsSync.ignoredSettings (by value; user entries kept). The managed chatLanguageModels.json entry is removed under this value. User mode only; requires llmGateway.proxyUrl (a gateway route that forwards to GitHub with the client's own token; example in the README).

  • Proxy route /vscode-copilot-capi/<pairing>/…: the pairing is the first path segment because VS Code cannot add a header to these requests; compared in constant time, stripped, never forwarded or logged; missing or wrong segment 403 pairing_invalid with the header ignored. The client's bearer token (tid= session token, gho_ on /agents/*) moves to x-llm-token as on the pass-through route; a credential-less GET /_ping (Copilot Chat's health check) is forwarded without x-llm-token, POST /_ping without a credential stays 401. The route's behaviour is three flags on ProxyRoute (pairing_in_path, tunnels_upgrades, credential_less_ping).
  • WebSocket tunnel (Auto and agent mode use GET /responses as a WebSocket): serve_with_cache serves with_upgrades(); forward takes the inbound OnUpgrade before buffering the body, keeps Connection: Upgrade/Upgrade: websocket on the request and returns the upstream 101 with Connection/Upgrade/Sec-WebSocket-* after hop stripping; the tunnel is awaited by the connection task under the server's JoinSet (dropping the server closes it) and ends when either side closes or the gateway credential it was opened with expires (its own expiry on both clocks, read together with the credential; not the cache's 60 s reuse window). Behaviour change: a WebSocket upgrade on any other route is refused with 400 upgrade_not_supported (before, it was stripped and forwarded as plain HTTP); other Upgrade tokens are still stripped and forwarded.
  • Settings file: written 0600 (an existing looser mode is tightened, since the URL carries the pairing), removal keeps the mode it finds, reports redacted, sidecar .settings.json.agentdesktop, plain-JSON rewrite (sorted keys), comments/trailing commas are a conflict. Removal on program absent, ownModels, useLlmGateway: false, no gateway, no proxyUrl, no proxy (with a warning); sidecar-less removal by this daemon's own URL; a user's own overrideCapiUrl is overwritten while active and restored on removal. daemon.vscode.settings overrides the path (error in system mode). --once and the system-mode message name settings.json.
  • Config: VsCodeCopilotChat::GithubModels; githubModels with models rejected; githubModels without llmGateway.proxyUrl rejected when the gateway is used; VsCodeStartupConfig { config, settings }; schema regenerated.
  • Frontend: proxyUrl field under the gateway section of the daemon-config builder; the VS Code program placeholder shows copilotChat: githubModels.
  • Docs: README section (what VS Code sends, pairing-in-URL and why, _ping, the WebSocket authorisation window, settings rules, the stale-override token exposure while the daemon is down and the single-user recommendation and stop order, undocumented-setting caveat with the HTTP-proxy fallback, model policy, restart VS Code, no --dry-run preview, upgrade note), routes-table row, pairing paragraph, upgrade_not_supported, the Requests paragraph on tunnels, gateway route example (Business host, lab-verified), provider README.
  • Settings ownership (json_merge): MergeOptions::owned_keys. A hand edit of the managed override after the merge is replaced on the next apply and never becomes the value removal restores; a value the user had before the first apply still comes back. Found by the lab: before this, a hand-edited URL came back as a stale override on removal and Copilot Chat got 403. Other callers pass no owned keys and are unchanged.
  • Test harness (separate commits): Container::write removes an existing file first (fs.protected_regular=2 stopped root rewriting a tester-owned /tmp file), and Container::stop_process also signals the children of the recorded PID (runuser ignores SIGINT while its child runs). The Copilot CLI Docker test pushed models as a list, a shape PR 2's revision replaced with a map; fixed. The whole Docker suite now passes locally, 9 of 9.

Process. Spec reviewed (Opus) before the test author, then frozen (specs/PR-3b.md rev 2); tests first by a separate agent in their own baseline commit (f1f8967: 13 proxy tests with a hand-rolled RFC 6455 echo, 18 settings tests, 3 config tests); test guard with a two-entry ledger (two hand-written struct literals gained new fields; no assertion changed); gate (218 agent + 18 core tests, clippy, schema, frontend build + lint); three review lenses on two models; delta review; independent lab run. Round 1 findings folded in: the tunnel deadline was the cache's 60 s reuse window instead of the credential expiry and a second lookup could leave a tunnel without a deadline (adversarial blocker + major, confirmed by probe); the README lacked the stale-override exposure text; route behaviour keyed on a prefix string; the 101 skipped hop stripping; a stale ProgramsConfig.vscode doc comment in the schema; provider README; misleading upgrade note; unverified Enterprise host claim. Follow-ups noted: extracting the duplicated reconcile helpers, the Reconciler::new argument list, override removal on shutdown.

Delta reviews (Opus): delta 1 on the fixes was "pass with minors" (the blocker fix re-probed: a credential with exp = now + 3600 now gives a tunnel deadline of 3599.9999 s, was 59.9997 s); the minors (a regression test for the deadline through CredentialCache::insert, one match_route call in forward, one system-mode check) went into the next commit, and delta 2 was a clean pass. After the lab run, delta 3 on the owned-key change was "pass with minors" (every MergeOptions caller checked; a probe confirmed a user value from before the first apply survives); its minors (a test for that case, README note on the headers VS Code sends) are in the last code commit, which adds a test and README text only, so no further delta pass was run. Two cosmetic nits stay: the _ping exemption is tied to the path-pairing branch (correct for the only route that sets both flags), and the settings/reconcile module pair could be renamed chat_models/settings later.

Lab (Linux VM, user mode, VS Code 1.139, Business seat, Solo Enterprise for AgentGateway standalone v2026.9.1 with the /copilot-proxy route, controller-pushed config, Playwright over CDP for the GUI steps):

  • First run on a85f77d by an independent agent: settings written and tightened 644 to 600 with both ignore entries next to the user's, chatLanguageModels.json entry removed, chat turns and _ping 200 through the gateway with llm.client="vscode-copilot" and the enrolled user, pairing absent from VS Code's logs, agent turns over 120 s complete, hand edit restored, re-pairing makes the old URL 403 even with a valid header, comment conflict blocks every managed file and the daemon restart exits, controller rejects githubModels with models and without proxyUrl, --once, --dry-run and system-mode messages. Three findings went into code or docs: GitHub refused the forwarded WebSocket upgrade until the gateway backend was pinned to HTTP/1.1 (alpn, now in the README example); the stale override after a hand edit (fixed with owned keys); VS Code saving settings.json as 664 (documented). VS Code's upgrade carries no Origin.
  • Repeat on 11e1de4 with lab/checks/pr3b-vscode.sh: 21 of 21 checks pass, including the hand-edit-then-ownModels sequence that failed before. GUI: an agent turn of 2m 15s over the WebSocket, gateway GET /copilot-proxy/responses http.status=101 duration=259532ms llm.client="vscode-copilot"; the close at credential expiry is one error line in VS Code's log and the next agent turn answers in 5 s without falling back.

Not covered by a test: Enterprise Cloud (api.enterprise.githubcopilot.com) and other plans' hosts; macOS and Windows settings paths beyond unit tests; Settings Sync across two devices; a VS Code release without overrideCapiUrl.

Follow-ups: extract the reconcile helpers now copied three times; a ProviderPaths struct for Reconciler::new; removing the override on graceful daemon shutdown (bounds the stale-override exposure); the reconcile tick (PR 5), which would also bound the window after VS Code rewrites the file as 664.

🤖 Generated with Claude Code

LutzLange and others added 8 commits September 30, 2026 04:33
… models)

Baseline test suite for specs/PR-3b.md, written before the implementation.
Adds compile-only todo!() stubs for the interface contract's not-yet-existing
functions (provider/vscode/settings.rs: settings_path, managed_settings, plan;
llm_proxy.rs: split_path_pairing) plus the real, non-stub CAPI_ROUTE entry in
ROUTES, core config (VsCodeCopilotChat::GithubModels, VsCodeStartupConfig),
and validation rules, so the tests below compile and fail only on the stubs
or on genuinely unimplemented routing/tunnel behaviour.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…thubModels)

With `copilotChat: githubModels` the daemon manages the VS Code user
`settings.json` instead of `chatLanguageModels.json`: it merges
`github.copilot.advanced.debug.overrideCapiUrl` pointing at the loopback
proxy's new `/vscode-copilot-capi/<pairing>` route and adds that key and
`github.copilot.internal.capiUrl` to `settingsSync.ignoredSettings` (by value,
user entries kept). The file is written 0600 (an existing looser mode is
tightened, since the URL carries the pairing), removal keeps the mode it
finds, reports are redacted, the sidecar is `.settings.json.agentdesktop`,
comments/trailing commas are a conflict. Removal triggers: program absent,
`ownModels`, `useLlmGateway: false`, no gateway, no `proxyUrl`, no proxy (with
a warning). Under `githubModels` the managed `chatLanguageModels.json` entry is
removed. `daemon.vscode.settings` overrides the path (error in system mode).

Proxy: the CAPI route takes the pairing as the first path segment (VS Code
cannot add a header there); the header is ignored on that route, the segment
is compared in constant time and stripped, and never forwarded or logged.
The client's bearer token moves to `x-llm-token` as on the pass-through
route; a credential-less `GET /_ping` is forwarded without `x-llm-token`
(`POST /_ping` without a credential stays 401). WebSocket upgrades on the CAPI
route are tunnelled: `serve_with_cache` serves `with_upgrades()`, `forward`
takes the inbound `OnUpgrade` before buffering the body, keeps
`Connection: Upgrade`/`Upgrade: websocket` on the request and returns the
upstream 101 with its `Connection`/`Upgrade`/`Sec-WebSocket-*` headers; the
tunnel is awaited by the connection task (under the server's JoinSet, so
dropping the server closes it) and ends when either side closes or the gateway
credential it was opened with reaches its deadline (cache `valid_until`, or the
freshly fetched credential's expiry). Behaviour change for existing callers: a
WebSocket upgrade on any other route is refused with `400 upgrade_not_supported`
(before, it was stripped and forwarded as plain HTTP); other `Upgrade` tokens
are still stripped and forwarded.

Config: `VsCodeCopilotChat::GithubModels`; `validate()` rejects it with
`models`; `validate_daemon` rejects it without `llmGateway.proxyUrl` when the
gateway is used; `VsCodeStartupConfig { config, settings }`. Wiring:
`VsCode { chat_models_path, settings_path }`, `Reconciler::new` gains
`vscode_settings_path`, `ResolvedDaemonArgs.vscode_settings`,
`default_vscode_settings_path(home)`; `--once` and the system-mode provider
message name `settings.json` as well. Workspace tokio gains `io-util` for
`copy_bidirectional`.

Frontend: the VS Code program placeholder shows `copilotChat: githubModels`;
the daemon-config builder has a `proxyUrl` field under the gateway section
(emitted only when set); `DaemonConfigDocument.llmGateway.proxyUrl`.

README: routes-table row, pairing-in-path paragraph, `upgrade_not_supported`,
the Requests paragraph on tunnels (outside the 600 s header limit, outlive a
revocation until the credential expires), and a new section "VS Code Copilot
Chat on GitHub's models through the gateway" (what VS Code sends, `_ping`, the
WebSocket authorisation window, settings rules, caveats: undocumented setting
and HTTP-proxy fallback, stale override, single-user devices, model policy;
upgrade note; gateway route example for the Business host with the Enterprise
host named). Schema regenerated.

Tests. Baseline (test author, f1f8967, unchanged except one struct literal):
13 proxy tests (`llm_proxy.rs`), 18 settings tests (`settings_tests.rs`, 16 on
Linux: the per-OS `settings_path` tests are one per platform), 3 config tests
(`vscode/tests.rs`). Writer: `system_mode_rejects_a_vscode_settings_path`
(daemon.rs) and the Docker test `managed_settings_github_models_lifecycle`
(settings written with the path-pairing URL and both entries, 644 -> 600, a
curl with `Authorization: Bearer tid=x` recorded upstream as `x-llm-token`
with `authorization` null and no pairing in the path, a WebSocket upgrade
through the proxy to a host-side RFC 6455 echo behind `proxyUrl` reaching 101
and echoing one frame with `x-llm-token` seen at the upgrade, removal keeping
a user key and 640). The stub gateway records `x-llm-token` as `llmToken`
(additive). The PR 3a test `system_mode_rejects_a_configured_program_before_writing_anything`
gained the new `settings_path` field in its two `VsCode` literals (ledger).
Agent lib: 217 tests; core: 18.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…n under runuser

`Container::write` removes an existing file before writing it: with
`fs.protected_regular=2` root cannot open a file another user owns in a
sticky world-writable directory such as `/tmp`, so the second config write
of the lifecycle tests failed with "Permission denied".

`Container::stop_process` also signals the children of the recorded PID:
the PID is the `runuser` launcher, which ignores SIGINT while its child
runs (as `su` does), so the daemon never stopped and the 10 s wait timed
out. Children are read from /proc (the image has no pgrep); a launcher that
has already exited is not an error.

Both defects made `managed_chat_models_lifecycle` (PR 3a) and
`managed_settings_github_models_lifecycle` fail locally at the restart
step; both pass now (Linux 6.17, Docker, protected_regular=2).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…iry; review fixes

Review round 1 (conformance, adversarial, maintainer) on pr3b-v1.

Tunnel lifetime (adversarial B1 + M1, maintainer M1): the deadline of a
WebSocket tunnel opened with a cached credential was the cache's reuse
window (`valid_until`, at most 60 s), not the credential's expiry, and it was
looked up again after the upstream answered, so an entry that expired or was
invalidated meanwhile left the tunnel without a deadline. `CachedCredential`
now carries the credential's own expiry on both clocks (`expires`,
`expires_unix`), `CredentialCache::lookup` returns credential and deadline in
one read (`get` stays for tests), and `run_tunnel` takes a `Deadline` checked
on the monotonic clock and, at most every 10 s, on the wall clock (adversarial
m2: the monotonic clock does not advance across a suspend). A freshly fetched
credential gives `Deadline::from_unix(expires_at_unix_seconds)`.

Proxy (maintainer minors 1, 2; conformance minor 3): the CAPI route's
behaviour is three fields on `ProxyRoute` (`pairing_in_path`,
`tunnels_upgrades`, `credential_less_ping`) instead of a prefix comparison;
the 101 response has its hop headers stripped and `Connection: Upgrade` plus
the upstream's `Upgrade` value put back.

Docs (adversarial M2, maintainer M2, minors 3, 4, 5, 9, nit; conformance
minors 1, 2): the README states the stale-override exposure (while the
daemon is down VS Code keeps sending `tid=`, `gho_` and the pairing to the
loopback port, which another local user could bind), the single-user
recommendation and the stop order; the plain-JSON rewrite (sorted keys,
two-space indent), the comment/trailing-comma conflict as the most likely one
with its fix, `github.copilot.internal.capiUrl` as the newer alias; the
upgrade note no longer calls `proxyUrl` new; the gateway example names only
the lab-verified Business host and tells readers where to find theirs; JWKS
URL over https. `ProgramsConfig.vscode` doc comment covers both variants
(schema regenerated); the provider README names `settings.json` in the
user-mode and dry-run paragraphs.

Config (maintainer nit): the `githubModels` `proxyUrl` rule checks
`llm_gateway` once.

Tests: writer test
`github_models_through_the_provider_removes_the_chat_models_entry_and_writes_settings`
(adversarial m1: `VsCode::plan` switching ownModels -> githubModels -> ownModels
through the provider); the Docker test asserts the stub's 401 passed through
(maintainer minor 10); `plan_remove_orphaned` logs when it leaves a
non-plain-JSON file alone. Baseline test
`capi_tunnel_closes_when_the_cached_credential_expires` gained the two new
cache-entry fields in its hand-inserted literal (ledger); assertions
unchanged. Agent lib: 218 tests; core: 18.

Left for follow-ups (maintainer minors 6, 7, 8): extracting the shared
reconcile helpers (third copy), the `Reconciler::new` argument list, and the
provider requiring both VS Code paths.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ine; one system-mode check

Delta review 1 minors. `forward` matches the route once and reuses the result
for the pairing placement, the upgrade refusal, the near-miss check, the ping
exemption and the dispatch (maintainer minor 1, remaining part). New writer
test `credential_cache_deadline_is_the_credential_expiry_not_the_reuse_window`
inserts through `CredentialCache::insert` and asserts the deadline `lookup`
returns follows the credential's `exp` (3600 s) while the reuse window stays
within `CREDENTIAL_CACHE_TTL`; this is the regression test for round 1's
blocker. `daemon.rs` rejects `daemon.vscode.config`/`.settings` in system mode
with one check naming whichever is set (maintainer nit); messages unchanged.

Left as is, with the reason: the per-connection tunnel slot stays a mutex
around an `Option` (one upgrade per HTTP/1.1 connection; a oneshot would add
a receiver to poll for no behaviour change); the core doc comments keep naming
the setting and the route so the schema tells an operator what the value
does; the adversarial nit on the hand-inserted cache entry in the baseline
expiry test is covered by the new test going through `insert`.

Agent lib: 219 tests; core: 18.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ttings mode caveat

Lab first run on pr3b-v2 (Linux VM, VS Code 1.139, Business seat, Enterprise
standalone gateway).

Stale override after a hand edit (lab G3 -> G5): the three-way rollback treats
a changed value of a managed key as the user's change, so a hand-edited
`overrideCapiUrl` restored by the daemon on restart was recorded in the
sidecar as the user's original and came back on removal, and VS Code got 403
`pairing_invalid`. `MergeOptions::owned_keys` names top-level keys the daemon
owns once a sidecar exists: on re-apply and on removal their current value is
replaced by the value the key had before the first merge (or removed when it
had none), so a hand edit is drift and never the user's value; a value the
user had before the first apply still comes back on removal. The settings
merge owns `github.copilot.advanced.debug.overrideCapiUrl`; every other caller
passes no owned keys and is unchanged. Test
`hand_edited_override_is_replaced_on_reapply_and_not_restored_on_removal`
(settings_tests.rs, writer-added) reproduces the lab sequence.

Gateway example (lab G2): with the default ALPN the gateway negotiates HTTP/2
to GitHub and GitHub answers the forwarded upgrade `400 websocket: the client
is not using the websocket protocol`; VS Code then falls back to plain
`POST /responses`. `backendTLS: {alpn: [http/1.1]}` on the GitHub backend
gives 101 and tunnels of 256 s and 300 s. The README example and paragraph
carry it; the daemon side was correct throughout (capture: upgrade headers
forwarded, no pairing in the path).

README (lab G2c): VS Code saves `settings.json` as 664 with a new inode when a
setting is changed in the UI; the URL is readable to others until the next
apply tightens it; the owned-key rule is stated.

Agent lib: 220 tests; core: 18.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ADME on pass-through headers

Delta review 3 minors. Writer test
`user_override_from_before_the_first_apply_survives_a_hand_edit_and_returns_on_removal`
covers the owned-key rule's other half: a value the user had before the
first apply comes back on removal even after a hand edit of the managed URL
and a re-apply. README: VS Code's other request headers, including
`Copilot-Session-Token` and its machine and device ids, reach the gateway and
GitHub unchanged (lab observation).

Agent lib: 221 tests; core: 18.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`programs.copilot.models` became a map keyed by model ID when the Copilot
CLI program was revised; the Docker test still pushed the earlier list shape,
so the daemon refused the config at parse time and the dry-run check failed
on the wrong message. The test had not run to completion locally until the
harness fixes (cbcc8f1). The user's own `providers.json` fixture keeps its
list, which is the file's real shape.

With this, the whole Docker integration suite passes locally (9/9: VS Code
discovery and both VS Code lifecycles, Copilot CLI, Claude Code, Codex,
OpenCode, Cursor, Grok).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@LutzLange

Copy link
Copy Markdown
Owner Author

Superseded: landed upstream in agentdesktop-dev#135 (VS Code githubModels).

@LutzLange LutzLange closed this Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant