From ae8bc6fd95f4d6e53eb9ec640e30d5c037ccb5d2 Mon Sep 17 00:00:00 2001 From: Lily Shen <115414357+lilyshen0722@users.noreply.github.com> Date: Tue, 27 Oct 2026 13:06:12 -0700 Subject: [PATCH 1/6] docs(plans): D81 bring-your-own-key design note Model-layer keys per user and team routed per request at the gateway, harness layer ruled out for hosted agents, write-only UI, spend and caps, visible failure with no fallback onto our key, extension path from agentProviderKey.ts, and a PR-by-PR estimate. Every current-code claim cites file:line at origin/main 9823d2bb. Co-Authored-By: Claude Haiku 5.5 Co-authored-by: wisp Agent-Model: claude/claude-haiku-5-5/default --- docs/plans/byok-design-2026-10-27.md | 95 ++++++++++++++++++++++++++++ 1 file changed, 95 insertions(+) create mode 100644 docs/plans/byok-design-2026-10-27.md diff --git a/docs/plans/byok-design-2026-10-27.md b/docs/plans/byok-design-2026-10-27.md new file mode 100644 index 0000000000..fa35d99d0d --- /dev/null +++ b/docs/plans/byok-design-2026-10-27.md @@ -0,0 +1,95 @@ +# Bring-your-own-key design (D81) + +**Status:** proposed 2026-10-27 +**Owner:** wisp (design). Tier 2 (keys and money): Opus code gate on every implementation PR; ux-lead gates the UI section (write-only key entry, team scope). +**Ledger:** D81 (10-25, Sam), un-parked 10-27 ~12:00 PT by the Cofounder. + +Line references are to `origin/main` at `9823d2bb`. Every claim about current code cites a file and line range. Anything marked **to verify** is a design assumption with no spike yet. + +## Current state (what exists today) + +- **One key setter, one harness, one sandbox.** `ownKeyCapability` only offers the own-key setter for `runtimeType === 'wrapper'` with `harness === 'opencode'`, `sandbox.trust` public, a confined mode (`workspace`, `read-only` or `bwrap`) and an https destination (`backend/services/agentProviderKey.ts:13-24`). The comment there records Lily's confined-modes ruling: the other harnesses expose their bearer to tool env and config, so they get no setter. +- **Metadata-only reads.** `GET /agents/:name/provider-key` returns `{set, setAt, setBy, lastUsedAt}` and nothing else, with `Cache-Control: no-store` (`agentProviderKey.ts:43-53`, `metadata()` at `:30-31`). The write path is `PUT` with `{key}` and validation at `:65-68`, and `DELETE` at `:52-63`. +- **Per-generation immutable secrets.** Each set writes a new `ConnectorSecret` (`connectorSecrets.ts:164-171`, `put`), bumps `generation`, and revokes the previous ref only after the row commits (`agentProviderKey.ts:69-82`). The ref is the only thing on `AgentProviderKey` (`backend/models/AgentProviderKey.ts:4-11`), keyed by `agentUserId` with a unique index. +- **The key does leave the server, to the daemon.** `launchProviderKey` returns the decrypted key to an authenticated machine daemon (`agentProviderKey.ts:97-125`; route at `backend/routes/agentsRuntime.ts:525-527`). That is the opencode BYO path. It is not a model-layer path and must not become one (see §1). +- **Hosted agents use our key, through LiteLLM.** The provisioner issues per-agent LiteLLM virtual keys and stores them on the agent's PVC (`backend/services/agentProvisionerServiceK8s.ts:1576` read, `:1599` `issueLiteLLMVirtualKey`, `:1676` `issueLiteLLMOpenRouterKey`, `:1753` injection into auth profiles). So an agent process **already holds a gateway bearer**, though not a vendor key. The vendor key stays at the gateway. +- **Native calls disable fallbacks.** The native runtime sends `disable_fallbacks: true` (`backend/services/nativeRuntimeService.ts:2092`). The LiteLLM config has no fallback to another model or vendor for the hosted DeepSeek default, and a test pins that (`backend/__tests__/unit/helm/litellmHostedFallback.test.js:8-30`, which also asserts Workers AI is not a route or fallback target). +- **Run ceiling is per user.** `userRunCeilingForOwner` returns 120 for free and 300 for pro, from env (`backend/services/hostedRuntimeService.ts:83-89`), and applies to the owner (`backend/routes/registry/manage-settings.ts:111`). +- **Spend is tokens, not money.** `AgentRun` records prompt, completion and total tokens and a `liteLLMCallId` (`backend/models/AgentRun.ts:61-67`, `:135-136`). The header says there are "no cost/budget fields" (`:11`). `servedModel` and `fallbacks` come from LiteLLM headers (`backend/services/nativeRuntimeService.ts:1436-1452`, `AgentRun.ts:70-72`). +- **Stale copy.** The public legal page still says "Cloudflare Workers AI is the fallback" (`frontend/src/v2/legal/V2LegalPage.tsx:54`). That contradicts D81(a) and the removal in #2718, which is in prod. It is a copy fix, out of scope for this note, and listed in §7. + +## Constraints that bind the design + +1. **Confined-modes ruling (Lily):** a harness's own credential path exposes its bearer to tool env and config. A key must never reach an agent process or tool env. (`agentProviderKey.ts:15-17`.) +2. **No brokering of Claude usage (Stack L6, Anthropic's Claude Code terms, Cofounder ruling B, 10-25):** we never pay for, resell or intermediate a user's Claude subscription, and we never hold a subscription or session token. Hosted agents reach Claude only through the model API, with the user's own Anthropic API key or ours. +3. **Our key is the default, not the fallback.** A failed or exhausted user key fails the turn visibly. It never falls back onto our key (D81(a)). + +## 1. Model-layer keys, per user and per team, routed per request + +**Recommendation:** the vendor key is registered at the gateway and never leaves it. The agent keeps only its own gateway bearer, as it does today. + +- Store the vendor key the way `AGENT_PROVIDER_KEY` already stores material: an encrypted `ConnectorSecret` with a generation and a revoke-after-commit (`connectorSecrets.ts:164-183`, `connectorSecretKinds.ts:111-113`). Add a new kind, e.g. `MODEL_PROVIDER_KEY`, so the audit and rewrap paths stay separate. +- At call time, the gateway resolves the caller's virtual key to its owner and team, then selects the owner's or team's vendor key for that one request. The mechanism is **to verify**: LiteLLM team- or key-scoped model deployments are the likely route. A spike must show that a request carrying team A's virtual key cannot resolve team B's deployment, and that the key string never appears in the response, the logs or the agent's env. +- Do **not** extend `launchProviderKey` (`agentsRuntime.ts:525`) to model-layer keys. That path hands the plaintext to a daemon. The model layer needs the key to stay in the gateway's process. + +## 2. Harness layer: a person's own subscription + +**Recommendation: out of scope for hosted agents. Ruled out explicitly.** + +- Hosted agents cannot hold a subscription or session token (constraint 2). +- BYO CLI seats already run the person's own Claude Code or Codex binary, signed in on their own machine. No grant is needed, and we hold nothing. +- The only harness-layer key path that exists (opencode, `agentProviderKey.ts:13-24`) passes a **vendor API key**, not a subscription. Extending it to another harness re-opens the confined-modes problem. Keep it confined to `public` plus a confined mode, as today. + +**Decision for Sam/Lily:** confirm that "harness layer = out of scope for hosted; BYO CLI stays as today" closes D81(2). If Sam wants a per-run harness grant for API keys in a confined sandbox, that is a separate design, and it is the only way the harness layer returns. + +## 3. Write-only UI + +- The contract already exists. The UI sends `PUT {key}` and reads only metadata (`frontend/src/v2/components/V2AgentProviderKey.tsx:28`, `:45` a `type="password"` input with `autoComplete="new-password"`). Nothing reads the key back. +- Keep that invariant for every new surface: no route returns key material to a browser, no "show key" control, and `no-store` on every key response (`agentProviderKey.ts:44`). +- New surfaces: a personal key in account settings, and a team key in the team admin view. Both show `set`, `setAt`, `setBy` and last-used only. Replacing a key is a new PUT, not an edit. +- **ux-lead gates this section.** + +## 4. Spend attribution and caps + +- **Attribution:** every BYO call already has a `liteLLMCallId` (`AgentRun.ts:67`). Attribute spend by joining that to the gateway's spend log per key, not by parsing our token counts. Token counts in `AgentRun` are not money, and there is no cost field to fill (`:11`). +- **Caps:** a per-key budget set at the gateway (LiteLLM's key or team budget, **to verify**). The budget is enforced by the vendor-side path, so a runaway agent cannot exceed it. +- **Run ceiling: recommendation.** A team key does **not** change the run ceiling. Runs still count against the invoking user's ceiling (`hostedRuntimeService.ts:83-89`, `manage-settings.ts:111`). The ceiling protects our abuse surface; the key cap protects the money. Two separate limits. +- **Team key cap:** a team key carries its own cap, set by a team admin. It does not draw from any member's personal cap. + +## 5. Failure is visible, with no fallback onto our key + +- A BYO turn sets `disable_fallbacks: true` as native calls do today (`nativeRuntimeService.ts:2092`). The BYO deployment is bound only to the user's key, so the router has no chain that reaches our default. +- Map gateway errors to named turn errors: `auth_failed`, `rate_limited`, `budget_exceeded`, `key_unavailable`. The turn ends with that code shown in the pod. It does not retry on our key. +- Test (new): a BYO turn with a revoked key returns the named error, `AgentRun.fallbacks` is 0 (`AgentRun.ts:70-72`), and `servedModel` equals the requested model. + +## 6. How it extends `agentProviderKey.ts` + +Today the setter is keyed by **seat** (`AgentProviderKey.agentUserId`, unique) and gated by `ownKeyCapability` (`agentProviderKey.ts:13-24`). Extend it in four steps: + +1. **Scope.** Change the row from `agentUserId` to `scope: {kind: 'user' | 'team', id}`, keeping the seat row for the existing opencode path. Migrate existing rows to `kind: 'user'` with the seat as a sub-key. The unique index moves with it. +2. **Capability.** Keep `ownKeyCapability` for the opencode harness. Add a second capability for the model layer, which requires only an https provider, the gateway being reachable, and an owner or team role check. It does **not** depend on the harness, the sandbox trust or `runtimeType`, because the key never reaches the agent. +3. **Resolution.** Do not reuse `launchProviderKey` for the model layer. Add a gateway-side resolver that the LiteLLM router calls, and keep `launchProviderKey` for opencode only. +4. **Generations and revocation.** Keep the immutable per-generation secret and revoke-after-commit pattern (`agentProviderKey.ts:69-82`). Keep the 409 on a changed destination. + +## 7. PR-by-PR estimate + +My estimate, one owner, not yet reviewed by CTO: + +| PR | Scope | Size | Gate | +|---|---|---|---| +| 0 | Spike: LiteLLM team/key-scoped deployment routing, key-not-in-response proof, budget field behaviour | S (2–3 days) | Opus | +| 1 | Schema: `scope` on `AgentProviderKey`, migration of seat rows, index change | S | Opus | +| 2 | Gateway resolver and per-request routing for user keys | M | Opus | +| 3 | Named turn errors, `disable_fallbacks` on BYO turns, `fallbacks === 0` test | S | Opus | +| 4 | Spend attribution: `liteLLMCallId` joined to gateway spend log; per-key budget | M | Opus | +| 5 | Write-only personal key UI | S | Opus + ux-lead | +| 6 | Team key: admin role, team cap, team UI | M | Opus + ux-lead | +| 7 | Copy: drop Cloudflare from `V2LegalPage.tsx:54`, add a BYO section to the legal and privacy pages | XS | ux-lead | + +Total: about 7 PRs, roughly 3–4 weeks for one owner, with PR 0 deciding whether PRs 2 and 4 keep their shape. PR 7 is copy-only and does not need the prod freeze lifted, so it can go before launch if Growth wants it. + +## Open questions + +- **Harness layer (§2):** Sam or Lily to confirm "out of scope for hosted" closes D81(2). +- **Team cap semantics (§4):** a team key carries its own cap, separate from member run ceilings. Confirm. +- **Spike result (§1, §4):** if LiteLLM cannot scope deployments per key, PR 2 changes shape. This note would then go back to review. From 25eb5577a0268a22dca24066aa475bb3c5fde0f5 Mon Sep 17 00:00:00 2001 From: Lily Shen <115414357+lilyshen0722@users.noreply.github.com> Date: Wed, 28 Oct 2026 04:08:39 -0700 Subject: [PATCH 2/6] docs(plans): BYOK scopes, bounded gateway bearer and per-PR done-tests Adds the key-source scopes (agent, team, user, ours) recorded once per turn as AgentRun.keySource, the design rule that BYOK keys bind to the in-cluster proxy with client-side overrides disabled, the bound on the agent's gateway bearer, the metering split against the run ceiling, and a done-test for each PR in the plan. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01MDs9DRK3nqxjeAV7xL8uhg --- docs/plans/byok-design-2026-10-27.md | 43 ++++++++++++++++++++-------- 1 file changed, 31 insertions(+), 12 deletions(-) diff --git a/docs/plans/byok-design-2026-10-27.md b/docs/plans/byok-design-2026-10-27.md index fa35d99d0d..4955d37cfb 100644 --- a/docs/plans/byok-design-2026-10-27.md +++ b/docs/plans/byok-design-2026-10-27.md @@ -31,6 +31,22 @@ Line references are to `origin/main` at `9823d2bb`. Every claim about current co - Store the vendor key the way `AGENT_PROVIDER_KEY` already stores material: an encrypted `ConnectorSecret` with a generation and a revoke-after-commit (`connectorSecrets.ts:164-183`, `connectorSecretKinds.ts:111-113`). Add a new kind, e.g. `MODEL_PROVIDER_KEY`, so the audit and rewrap paths stay separate. - At call time, the gateway resolves the caller's virtual key to its owner and team, then selects the owner's or team's vendor key for that one request. The mechanism is **to verify**: LiteLLM team- or key-scoped model deployments are the likely route. A spike must show that a request carrying team A's virtual key cannot resolve team B's deployment, and that the key string never appears in the response, the logs or the agent's env. - Do **not** extend `launchProviderKey` (`agentsRuntime.ts:525`) to model-layer keys. That path hands the plaintext to a daemon. The model layer needs the key to stay in the gateway's process. +- **Design rule: BYOK keys are bound to the in-cluster proxy, and client-side overrides are disabled.** A request can't carry its own `api_base` or `api_key`, so it can't redirect a stored key, and a key that resolves to a user's vendor key is accepted only on the in-cluster path. +- **The agent's gateway bearer is bounded.** The per-agent virtual key stays in the agent's environment, and the agent's tools can read it. So any virtual key that can resolve to a user's vendor key gets its own model allowlist, budget and rate limit, and is rotated on reprovision. The vendor key is never exposed. The bearer that spends it is exposed, which is why it must be bounded. +- LiteLLM's credential routing selects credentials by team and project, not by user. A personal key maps to a one-member team and an agent key to a project. That mapping is **to verify** in the spike. + +## 1a. Scopes and which key a turn uses + +There are three scopes, plus our default: + +| Scope | Set by | Serves | Precedence | +|---|---|---|---| +| Agent | the agent's owner | one hosted agent | 1 | +| Team | a team admin | hosted agents run in that team's pods | 2 | +| User | the person | hosted agents that person owns | 3 | +| Ours | — | everything else (the default, never a fallback) | 4 | + +The source is resolved once, at turn start. It is recorded on `AgentRun` as `keySource: ours | user | team | agent` and never changes mid-turn. Open: whether team should beat user (as here) or user beat team, and whether "team" means the workspace rather than the pod. ## 2. Harness layer: a person's own subscription @@ -55,6 +71,9 @@ Line references are to `origin/main` at `9823d2bb`. Every claim about current co - **Caps:** a per-key budget set at the gateway (LiteLLM's key or team budget, **to verify**). The budget is enforced by the vendor-side path, so a runaway agent cannot exceed it. - **Run ceiling: recommendation.** A team key does **not** change the run ceiling. Runs still count against the invoking user's ceiling (`hostedRuntimeService.ts:83-89`, `manage-settings.ts:111`). The ceiling protects our abuse surface; the key cap protects the money. Two separate limits. - **Team key cap:** a team key carries its own cap, set by a team admin. It does not draw from any member's personal cap. +- **Metering split.** A BYOK turn removes our inference cost, not our compute cost. It still counts toward the run ceiling and any agent-hour meter, but not toward anything that meters our inference spend. `AgentRun.keySource` makes that split computable. +- **Plans (proposed).** Personal and agent keys on Free and Pro. Team keys on Pro, because they need admin roles and a team cap. No markup on inference that runs on a user's key. +- **Labels.** A run on someone's key shows "on your key" or "on the team key". Settings show last use, month-to-date spend from the gateway, and the cap, never the key. ## 5. Failure is visible, with no fallback onto our key @@ -75,18 +94,18 @@ Today the setter is keyed by **seat** (`AgentProviderKey.agentUserId`, unique) a My estimate, one owner, not yet reviewed by CTO: -| PR | Scope | Size | Gate | -|---|---|---|---| -| 0 | Spike: LiteLLM team/key-scoped deployment routing, key-not-in-response proof, budget field behaviour | S (2–3 days) | Opus | -| 1 | Schema: `scope` on `AgentProviderKey`, migration of seat rows, index change | S | Opus | -| 2 | Gateway resolver and per-request routing for user keys | M | Opus | -| 3 | Named turn errors, `disable_fallbacks` on BYO turns, `fallbacks === 0` test | S | Opus | -| 4 | Spend attribution: `liteLLMCallId` joined to gateway spend log; per-key budget | M | Opus | -| 5 | Write-only personal key UI | S | Opus + ux-lead | -| 6 | Team key: admin role, team cap, team UI | M | Opus + ux-lead | -| 7 | Copy: drop Cloudflare from `V2LegalPage.tsx:54`, add a BYO section to the legal and privacy pages | XS | ux-lead | - -Total: about 7 PRs, roughly 3–4 weeks for one owner, with PR 0 deciding whether PRs 2 and 4 keep their shape. PR 7 is copy-only and does not need the prod freeze lifted, so it can go before launch if Growth wants it. +| PR | Scope | Size | Gate | Done when | +|---|---|---|---|---| +| 0 | Spike: LiteLLM credential routing as the isolation boundary | S | Opus, security | Team A's virtual key never resolves team B's credential. A client-side `api_base` or `api_key` is refused for an agent key, and the stored key is never sent to that base. The key appears in no response or log. A one-member team and a project carry the user and agent scopes. Results are posted on the row. | +| 1 | Schema: `scope` on `AgentProviderKey`, seat-row migration, index change | S | Opus | Existing opencode rows migrate to `kind: 'user'` and still resolve. The unique index holds per scope. | +| 2 | Key-source resolution and the per-request gateway resolver | M | Opus | Unit tests cover each precedence pair. The source never changes mid-turn. `AgentRun.keySource` is written. | +| 3 | Bound the agent's gateway bearer | S | Opus, security | A BYOK-resolving virtual key gets 403 for a model off its allowlist, `budget_exceeded` when its budget is spent, and is refused outside the in-cluster path. Reprovision invalidates the old key. | +| 4 | Named turn errors, `disable_fallbacks` on BYO turns | S | Opus | A revoked key ends the turn with `auth_failed`, `AgentRun.fallbacks === 0`, and `keySource` unchanged. | +| 5 | Spend attribution and the metering split | M | Opus | A service test with mixed turns gives correct run-ceiling, agent-hour and inference totals per `keySource`. Per-key spend comes from the gateway log joined on `liteLLMCallId`. | +| 6 | Write-only key UI for personal, agent and team keys, plus plan gating | M | Opus + ux-lead | Entitlement tests show Free can't set a team key. No route returns key material. Screenshots at 390 and 1440 of each scope and of the "on your key" label. zh-CN pass. | +| 7 | Copy: drop the Cloudflare line from `V2LegalPage.tsx:54`; add a BYOK section to the legal and privacy pages | XS | ux-lead | Every claim on the pages is backed by PRs 0–6 as each lands. | + +Total: about 8 PRs, roughly 3–4 weeks for one owner, with PR 0 deciding whether PRs 2 and 4 keep their shape. PR 7 is copy-only and does not need the prod freeze lifted, so it can go before launch if Growth wants it. ## Open questions From 00ac4b4d6adf3b01fe1e2499626879a722d90bec Mon Sep 17 00:00:00 2001 From: Lily Shen <115414357+lilyshen0722@users.noreply.github.com> Date: Wed, 28 Oct 2026 04:09:05 -0700 Subject: [PATCH 3/6] docs(plans): additive BYOK scope migration; required call id for BYO spend Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01MDs9DRK3nqxjeAV7xL8uhg --- docs/plans/byok-design-2026-10-27.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/plans/byok-design-2026-10-27.md b/docs/plans/byok-design-2026-10-27.md index 4955d37cfb..8fb5b3d578 100644 --- a/docs/plans/byok-design-2026-10-27.md +++ b/docs/plans/byok-design-2026-10-27.md @@ -67,7 +67,7 @@ The source is resolved once, at turn start. It is recorded on `AgentRun` as `key ## 4. Spend attribution and caps -- **Attribution:** every BYO call already has a `liteLLMCallId` (`AgentRun.ts:67`). Attribute spend by joining that to the gateway's spend log per key, not by parsing our token counts. Token counts in `AgentRun` are not money, and there is no cost field to fill (`:11`). +- **Attribution:** `IAgentRunTurn.liteLLMCallId` is optional today (`AgentRun.ts:67`), so not every call is guaranteed to carry it. For BYO turns it becomes required: every metered BYO call, including a failed call that was still charged, records the gateway's call id. A BYO call with no id is counted as unattributed in the spend report, never dropped. The gateway spend log is reconciled against `AgentRun` daily. Attribute spend by joining the call id to the gateway's spend log per key, not by parsing our token counts. Token counts in `AgentRun` are not money, and there is no cost field to fill (`:11`). - **Caps:** a per-key budget set at the gateway (LiteLLM's key or team budget, **to verify**). The budget is enforced by the vendor-side path, so a runaway agent cannot exceed it. - **Run ceiling: recommendation.** A team key does **not** change the run ceiling. Runs still count against the invoking user's ceiling (`hostedRuntimeService.ts:83-89`, `manage-settings.ts:111`). The ceiling protects our abuse surface; the key cap protects the money. Two separate limits. - **Team key cap:** a team key carries its own cap, set by a team admin. It does not draw from any member's personal cap. @@ -85,7 +85,7 @@ The source is resolved once, at turn start. It is recorded on `AgentRun` as `key Today the setter is keyed by **seat** (`AgentProviderKey.agentUserId`, unique) and gated by `ownKeyCapability` (`agentProviderKey.ts:13-24`). Extend it in four steps: -1. **Scope.** Change the row from `agentUserId` to `scope: {kind: 'user' | 'team', id}`, keeping the seat row for the existing opencode path. Migrate existing rows to `kind: 'user'` with the seat as a sub-key. The unique index moves with it. +1. **Scope.** Add `scope: {kind: 'user' | 'team' | 'agent', id}` **beside** `agentUserId`, not in place of it. `launchProviderKey` looks rows up by `agentUserId` and `ownerUserId`, so it keeps working unchanged. Existing rows are backfilled with `kind: 'user'` and keep `agentUserId`. The new unique index is added next to the old one. `agentUserId` is retired only after `launchProviderKey` reads `scope`, and that happens in its own PR with a rollback path. 2. **Capability.** Keep `ownKeyCapability` for the opencode harness. Add a second capability for the model layer, which requires only an https provider, the gateway being reachable, and an owner or team role check. It does **not** depend on the harness, the sandbox trust or `runtimeType`, because the key never reaches the agent. 3. **Resolution.** Do not reuse `launchProviderKey` for the model layer. Add a gateway-side resolver that the LiteLLM router calls, and keep `launchProviderKey` for opencode only. 4. **Generations and revocation.** Keep the immutable per-generation secret and revoke-after-commit pattern (`agentProviderKey.ts:69-82`). Keep the 409 on a changed destination. @@ -97,11 +97,11 @@ My estimate, one owner, not yet reviewed by CTO: | PR | Scope | Size | Gate | Done when | |---|---|---|---|---| | 0 | Spike: LiteLLM credential routing as the isolation boundary | S | Opus, security | Team A's virtual key never resolves team B's credential. A client-side `api_base` or `api_key` is refused for an agent key, and the stored key is never sent to that base. The key appears in no response or log. A one-member team and a project carry the user and agent scopes. Results are posted on the row. | -| 1 | Schema: `scope` on `AgentProviderKey`, seat-row migration, index change | S | Opus | Existing opencode rows migrate to `kind: 'user'` and still resolve. The unique index holds per scope. | +| 1 | Schema: additive `scope` on `AgentProviderKey` beside `agentUserId`, backfill, new index next to the old one | S | Opus | Before and after the backfill, an existing opencode key still resolves through `launchProviderKey` (test). The unique index holds per scope. `agentUserId` is not removed in this PR. | | 2 | Key-source resolution and the per-request gateway resolver | M | Opus | Unit tests cover each precedence pair. The source never changes mid-turn. `AgentRun.keySource` is written. | | 3 | Bound the agent's gateway bearer | S | Opus, security | A BYOK-resolving virtual key gets 403 for a model off its allowlist, `budget_exceeded` when its budget is spent, and is refused outside the in-cluster path. Reprovision invalidates the old key. | | 4 | Named turn errors, `disable_fallbacks` on BYO turns | S | Opus | A revoked key ends the turn with `auth_failed`, `AgentRun.fallbacks === 0`, and `keySource` unchanged. | -| 5 | Spend attribution and the metering split | M | Opus | A service test with mixed turns gives correct run-ceiling, agent-hour and inference totals per `keySource`. Per-key spend comes from the gateway log joined on `liteLLMCallId`. | +| 5 | Spend attribution and the metering split | M | Opus | A service test with mixed turns gives correct run-ceiling, agent-hour and inference totals per `keySource`. Every metered BYO call, including a charged failure, records a `liteLLMCallId`. A call missing one appears as unattributed, not omitted. | | 6 | Write-only key UI for personal, agent and team keys, plus plan gating | M | Opus + ux-lead | Entitlement tests show Free can't set a team key. No route returns key material. Screenshots at 390 and 1440 of each scope and of the "on your key" label. zh-CN pass. | | 7 | Copy: drop the Cloudflare line from `V2LegalPage.tsx:54`; add a BYOK section to the legal and privacy pages | XS | ux-lead | Every claim on the pages is backed by PRs 0–6 as each lands. | From 81621b2f28f4ecfa630b7a0dd213158adf85f623 Mon Sep 17 00:00:00 2001 From: Lily Shen <115414357+lilyshen0722@users.noreply.github.com> Date: Wed, 28 Oct 2026 04:37:13 -0700 Subject: [PATCH 4/6] docs(plans): BYOK design: agent-scope backfill, model keys in their own model (HOLD fixes) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Backfill existing seat keys as scope agent, not user. As user scope one seat's key would serve every agent its owner has. - agentUserId made optional, partial and scope unique indexes, with a done-test beside an existing agent row. - Model-layer keys get a separate ModelProviderKey model, so launchProviderKey cannot reach them. - §4: the budget is enforced by the gateway, not "vendor-side". - Scope precedence moves to Open questions; PR 7 (the Cloudflare copy) moves to the front. Written by Wisp, a Commonly agent — https://commonly.me/v2/pods/6a692a1be833c668acdb84cf Co-Authored-By: Claude Haiku 5.5 Co-authored-by: wisp Agent-Model: claude/claude-haiku-5-5/default --- docs/plans/byok-design-2026-10-27.md | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/docs/plans/byok-design-2026-10-27.md b/docs/plans/byok-design-2026-10-27.md index 8fb5b3d578..da03ae82f1 100644 --- a/docs/plans/byok-design-2026-10-27.md +++ b/docs/plans/byok-design-2026-10-27.md @@ -68,7 +68,7 @@ The source is resolved once, at turn start. It is recorded on `AgentRun` as `key ## 4. Spend attribution and caps - **Attribution:** `IAgentRunTurn.liteLLMCallId` is optional today (`AgentRun.ts:67`), so not every call is guaranteed to carry it. For BYO turns it becomes required: every metered BYO call, including a failed call that was still charged, records the gateway's call id. A BYO call with no id is counted as unattributed in the spend report, never dropped. The gateway spend log is reconciled against `AgentRun` daily. Attribute spend by joining the call id to the gateway's spend log per key, not by parsing our token counts. Token counts in `AgentRun` are not money, and there is no cost field to fill (`:11`). -- **Caps:** a per-key budget set at the gateway (LiteLLM's key or team budget, **to verify**). The budget is enforced by the vendor-side path, so a runaway agent cannot exceed it. +- **Caps:** a per-key budget set at the gateway (LiteLLM's key or team budget, **to verify**). The gateway enforces the budget (the LiteLLM key or team budget, **to verify**), so a runaway agent cannot exceed it. - **Run ceiling: recommendation.** A team key does **not** change the run ceiling. Runs still count against the invoking user's ceiling (`hostedRuntimeService.ts:83-89`, `manage-settings.ts:111`). The ceiling protects our abuse surface; the key cap protects the money. Two separate limits. - **Team key cap:** a team key carries its own cap, set by a team admin. It does not draw from any member's personal cap. - **Metering split.** A BYOK turn removes our inference cost, not our compute cost. It still counts toward the run ceiling and any agent-hour meter, but not toward anything that meters our inference spend. `AgentRun.keySource` makes that split computable. @@ -83,10 +83,13 @@ The source is resolved once, at turn start. It is recorded on `AgentRun` as `key ## 6. How it extends `agentProviderKey.ts` -Today the setter is keyed by **seat** (`AgentProviderKey.agentUserId`, unique) and gated by `ownKeyCapability` (`agentProviderKey.ts:13-24`). Extend it in four steps: +Today the setter is keyed by **seat** (`AgentProviderKey.agentUserId`, required and unique, `backend/models/AgentProviderKey.ts:3`) and gated by `ownKeyCapability` (`agentProviderKey.ts:13-24`). Extend it in four steps: -1. **Scope.** Add `scope: {kind: 'user' | 'team' | 'agent', id}` **beside** `agentUserId`, not in place of it. `launchProviderKey` looks rows up by `agentUserId` and `ownerUserId`, so it keeps working unchanged. Existing rows are backfilled with `kind: 'user'` and keep `agentUserId`. The new unique index is added next to the old one. `agentUserId` is retired only after `launchProviderKey` reads `scope`, and that happens in its own PR with a rollback path. -2. **Capability.** Keep `ownKeyCapability` for the opencode harness. Add a second capability for the model layer, which requires only an https provider, the gateway being reachable, and an owner or team role check. It does **not** depend on the harness, the sandbox trust or `runtimeType`, because the key never reaches the agent. +1. **Scope.** Add `scope: {kind: 'user' | 'team' | 'agent', id}` to `AgentProviderKey`. Existing rows are backfilled as `kind: 'agent'`, `id: agentUserId`. They are seat keys, one per agent, set through the opencode harness. Read as `user` scope, one seat's key would resolve for every agent its owner has. `agentUserId` becomes optional, because a user or team row has no seat. Its non-partial unique index is replaced by a partial unique index on `agentUserId` where `scope.kind` is `agent`, and a unique index on `(scope.kind, scope.id)` covers user and team rows. `launchProviderKey` looks rows up by `agentUserId` and `ownerUserId`, so it keeps finding only agent rows and works unchanged. `agentUserId` is retired only after `launchProviderKey` reads `scope`, in its own PR with a rollback path. +2. **Model-layer keys get their own collection.** They do not share `AgentProviderKey`. A separate `ModelProviderKey` model means `launchProviderKey` cannot reach a model-layer row by construction. The shared-collection alternative needs a `layer` field and a pin on the daemon lookup. We reject it: one missed filter would send a plaintext model key to a daemon, which §1 says must never happen. +3. **Capability.** Keep `ownKeyCapability` for the opencode harness. Add a second capability for the model layer, which requires only an https provider, the gateway being reachable, and an owner or team role check. It does **not** depend on the harness, the sandbox trust or `runtimeType`, because the key never reaches the agent. +4. **Resolution.** Do not reuse `launchProviderKey` for the model layer. Add a gateway-side resolver that the LiteLLM router calls, and keep `launchProviderKey` for opencode only. +5. **Generations and revocation.** Keep the immutable per-generation secret and revoke-after-commit pattern (`agentProviderKey.ts:69-82`), in both collections. Keep the 409 on a changed destination. 3. **Resolution.** Do not reuse `launchProviderKey` for the model layer. Add a gateway-side resolver that the LiteLLM router calls, and keep `launchProviderKey` for opencode only. 4. **Generations and revocation.** Keep the immutable per-generation secret and revoke-after-commit pattern (`agentProviderKey.ts:69-82`). Keep the 409 on a changed destination. @@ -97,7 +100,7 @@ My estimate, one owner, not yet reviewed by CTO: | PR | Scope | Size | Gate | Done when | |---|---|---|---|---| | 0 | Spike: LiteLLM credential routing as the isolation boundary | S | Opus, security | Team A's virtual key never resolves team B's credential. A client-side `api_base` or `api_key` is refused for an agent key, and the stored key is never sent to that base. The key appears in no response or log. A one-member team and a project carry the user and agent scopes. Results are posted on the row. | -| 1 | Schema: additive `scope` on `AgentProviderKey` beside `agentUserId`, backfill, new index next to the old one | S | Opus | Before and after the backfill, an existing opencode key still resolves through `launchProviderKey` (test). The unique index holds per scope. `agentUserId` is not removed in this PR. | +| 1 | Schema: additive `scope` on `AgentProviderKey`, existing rows backfilled as `kind: 'agent'`, `agentUserId` made optional, partial and scope indexes; the separate `ModelProviderKey` model | S | Opus | Before and after the backfill, an existing opencode key still resolves through `launchProviderKey` (test). A user-scoped row and a team-scoped row can be created beside an existing agent row (test). The unique indexes hold per scope. `agentUserId` is not removed in this PR. | | 2 | Key-source resolution and the per-request gateway resolver | M | Opus | Unit tests cover each precedence pair. The source never changes mid-turn. `AgentRun.keySource` is written. | | 3 | Bound the agent's gateway bearer | S | Opus, security | A BYOK-resolving virtual key gets 403 for a model off its allowlist, `budget_exceeded` when its budget is spent, and is refused outside the in-cluster path. Reprovision invalidates the old key. | | 4 | Named turn errors, `disable_fallbacks` on BYO turns | S | Opus | A revoked key ends the turn with `auth_failed`, `AgentRun.fallbacks === 0`, and `keySource` unchanged. | @@ -105,10 +108,11 @@ My estimate, one owner, not yet reviewed by CTO: | 6 | Write-only key UI for personal, agent and team keys, plus plan gating | M | Opus + ux-lead | Entitlement tests show Free can't set a team key. No route returns key material. Screenshots at 390 and 1440 of each scope and of the "on your key" label. zh-CN pass. | | 7 | Copy: drop the Cloudflare line from `V2LegalPage.tsx:54`; add a BYOK section to the legal and privacy pages | XS | ux-lead | Every claim on the pages is backed by PRs 0–6 as each lands. | -Total: about 8 PRs, roughly 3–4 weeks for one owner, with PR 0 deciding whether PRs 2 and 4 keep their shape. PR 7 is copy-only and does not need the prod freeze lifted, so it can go before launch if Growth wants it. +Total: about 8 PRs, roughly 3–4 weeks for one owner, with PR 0 deciding whether PRs 2 and 4 keep their shape. PR 7 is copy-only and does not need the prod freeze lifted. Move it to the front: the Cloudflare line is live on the public legal page today, and #2718 removed that fallback on 10-25 (Rhea's gate). ## Open questions +- **Scope precedence (§1a):** whether team beats user or user beats team, and whether "team" means the pod or the workspace. - **Harness layer (§2):** Sam or Lily to confirm "out of scope for hosted" closes D81(2). - **Team cap semantics (§4):** a team key carries its own cap, separate from member run ceilings. Confirm. - **Spike result (§1, §4):** if LiteLLM cannot scope deployments per key, PR 2 changes shape. This note would then go back to review. From 8ab37ca5e8d4caa7fd22ec532db94dda4b7fce5a Mon Sep 17 00:00:00 2001 From: Lily Shen <115414357+lilyshen0722@users.noreply.github.com> Date: Wed, 28 Oct 2026 05:22:16 -0700 Subject: [PATCH 5/6] docs(plans): BYOK design: drop duplicate steps, fix step count, one to-verify (Lumen HOLD) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Written by Wisp, a Commonly agent — https://commonly.me/v2/pods/6a692a1be833c668acdb84cf Co-Authored-By: Claude Haiku 5.5 Co-authored-by: wisp Agent-Model: claude/claude-haiku-5-5/default --- docs/plans/byok-design-2026-10-27.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/docs/plans/byok-design-2026-10-27.md b/docs/plans/byok-design-2026-10-27.md index da03ae82f1..fa613fc9ed 100644 --- a/docs/plans/byok-design-2026-10-27.md +++ b/docs/plans/byok-design-2026-10-27.md @@ -68,7 +68,7 @@ The source is resolved once, at turn start. It is recorded on `AgentRun` as `key ## 4. Spend attribution and caps - **Attribution:** `IAgentRunTurn.liteLLMCallId` is optional today (`AgentRun.ts:67`), so not every call is guaranteed to carry it. For BYO turns it becomes required: every metered BYO call, including a failed call that was still charged, records the gateway's call id. A BYO call with no id is counted as unattributed in the spend report, never dropped. The gateway spend log is reconciled against `AgentRun` daily. Attribute spend by joining the call id to the gateway's spend log per key, not by parsing our token counts. Token counts in `AgentRun` are not money, and there is no cost field to fill (`:11`). -- **Caps:** a per-key budget set at the gateway (LiteLLM's key or team budget, **to verify**). The gateway enforces the budget (the LiteLLM key or team budget, **to verify**), so a runaway agent cannot exceed it. +- **Caps:** a per-key budget set at the gateway (LiteLLM's key or team budget, **to verify**). The gateway enforces the budget, so a runaway agent cannot exceed it. - **Run ceiling: recommendation.** A team key does **not** change the run ceiling. Runs still count against the invoking user's ceiling (`hostedRuntimeService.ts:83-89`, `manage-settings.ts:111`). The ceiling protects our abuse surface; the key cap protects the money. Two separate limits. - **Team key cap:** a team key carries its own cap, set by a team admin. It does not draw from any member's personal cap. - **Metering split.** A BYOK turn removes our inference cost, not our compute cost. It still counts toward the run ceiling and any agent-hour meter, but not toward anything that meters our inference spend. `AgentRun.keySource` makes that split computable. @@ -83,15 +83,13 @@ The source is resolved once, at turn start. It is recorded on `AgentRun` as `key ## 6. How it extends `agentProviderKey.ts` -Today the setter is keyed by **seat** (`AgentProviderKey.agentUserId`, required and unique, `backend/models/AgentProviderKey.ts:3`) and gated by `ownKeyCapability` (`agentProviderKey.ts:13-24`). Extend it in four steps: +Today the setter is keyed by **seat** (`AgentProviderKey.agentUserId`, required and unique, `backend/models/AgentProviderKey.ts:3`) and gated by `ownKeyCapability` (`agentProviderKey.ts:13-24`). Extend it in five steps: 1. **Scope.** Add `scope: {kind: 'user' | 'team' | 'agent', id}` to `AgentProviderKey`. Existing rows are backfilled as `kind: 'agent'`, `id: agentUserId`. They are seat keys, one per agent, set through the opencode harness. Read as `user` scope, one seat's key would resolve for every agent its owner has. `agentUserId` becomes optional, because a user or team row has no seat. Its non-partial unique index is replaced by a partial unique index on `agentUserId` where `scope.kind` is `agent`, and a unique index on `(scope.kind, scope.id)` covers user and team rows. `launchProviderKey` looks rows up by `agentUserId` and `ownerUserId`, so it keeps finding only agent rows and works unchanged. `agentUserId` is retired only after `launchProviderKey` reads `scope`, in its own PR with a rollback path. 2. **Model-layer keys get their own collection.** They do not share `AgentProviderKey`. A separate `ModelProviderKey` model means `launchProviderKey` cannot reach a model-layer row by construction. The shared-collection alternative needs a `layer` field and a pin on the daemon lookup. We reject it: one missed filter would send a plaintext model key to a daemon, which §1 says must never happen. 3. **Capability.** Keep `ownKeyCapability` for the opencode harness. Add a second capability for the model layer, which requires only an https provider, the gateway being reachable, and an owner or team role check. It does **not** depend on the harness, the sandbox trust or `runtimeType`, because the key never reaches the agent. 4. **Resolution.** Do not reuse `launchProviderKey` for the model layer. Add a gateway-side resolver that the LiteLLM router calls, and keep `launchProviderKey` for opencode only. 5. **Generations and revocation.** Keep the immutable per-generation secret and revoke-after-commit pattern (`agentProviderKey.ts:69-82`), in both collections. Keep the 409 on a changed destination. -3. **Resolution.** Do not reuse `launchProviderKey` for the model layer. Add a gateway-side resolver that the LiteLLM router calls, and keep `launchProviderKey` for opencode only. -4. **Generations and revocation.** Keep the immutable per-generation secret and revoke-after-commit pattern (`agentProviderKey.ts:69-82`). Keep the 409 on a changed destination. ## 7. PR-by-PR estimate From 2121e577a6f3d8e7f9d5a342162371cb83fa6921 Mon Sep 17 00:00:00 2001 From: Lily Shen <115414357+lilyshen0722@users.noreply.github.com> Date: Sat, 31 Oct 2026 04:16:12 -0700 Subject: [PATCH 6/6] docs(plans): name the BYOK persistence owner and bearer binding as open (#2893) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Open questions section now carries the two questions that must be decided before PR 1 ships: which collection owns user and team model keys (§6.1 and §6.2 disagree), and how a gateway bearer is bound to the one key it may resolve. The team-admin authorization gap in §6.3 is listed with them. None of the three is settled here. Co-Authored-By: Claude Haiku 5.5 Co-authored-by: wisp Agent-Model: claude/claude-haiku-5-5/default --- docs/plans/byok-design-2026-10-27.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/plans/byok-design-2026-10-27.md b/docs/plans/byok-design-2026-10-27.md index fa613fc9ed..59a30b746f 100644 --- a/docs/plans/byok-design-2026-10-27.md +++ b/docs/plans/byok-design-2026-10-27.md @@ -110,6 +110,12 @@ Total: about 8 PRs, roughly 3–4 weeks for one owner, with PR 0 deciding whethe ## Open questions +Both of the first two must be decided before PR 1 ships. Neither is settled in this note. + +- **Persistence owner for user and team model keys (§6.1 vs §6.2):** §6.1 puts user and team rows in `AgentProviderKey`. §6.2 puts model-layer keys in a separate `ModelProviderKey` and says they never share `AgentProviderKey`. The note holds both contracts. Decide which collection owns user- and team-scoped model keys. +- **Gateway bearer binding (§1, PR 3):** the note does not say how a gateway bearer is bound to the one user's or team's key it may resolve, or which check the gateway applies. A bearer that can resolve more than that key lets an agent spend another user's or team's vendor key within its limits. +- **Team-admin authorization for model keys (§6.3):** the "owner or team role check" is named but not defined. Decide it together with the persistence owner. + - **Scope precedence (§1a):** whether team beats user or user beats team, and whether "team" means the pod or the workspace. - **Harness layer (§2):** Sam or Lily to confirm "out of scope for hosted" closes D81(2). - **Team cap semantics (§4):** a team key carries its own cap, separate from member run ceilings. Confirm.