Repository navigation
feat(proxy): per-app route mode with weighted provider groups - #517
Open
AtaraxiaEpiphany wants to merge 17 commits into
Open
AtaraxiaEpiphany wants to merge 17 commits into
AtaraxiaEpiphany wants to merge 17 commits into
Conversation
Store per-app route-mode groups (weighted provider pools) as JSON
documents in the existing settings KV table under route_mode_{app};
this fork's schema stays synced with upstream, so no tables or columns
are added and SCHEMA_VERSION is unchanged.
- RouteModeGroup/RouteModeMember with camelCase serde and defaults
- get_route_mode_group fails closed to a disabled default on missing,
corrupt, or structurally invalid data
- save_route_mode_group validates structure and provider existence
under one connection lock
- route_mode_enabled_for_app / set_route_mode_enabled flag helpers
- unit tests: roundtrip, flag-flip isolation, every validation
rejection, corrupt-JSON fail-closed
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Route mode branch in ProviderRouter::select_providers, consulted before the failover queue: while the app's route group is enabled the queue is never used (mutually exclusive modes). - SmoothWrrState: nginx smooth weighted round-robin; ties keep the earlier member, stale member state is garbage-collected per pick - select_route_providers resolves members against stored providers, drops disabled and dangling members, applies the same circuit-breaker filter as the failover branch, picks the WRR winner, and orders the rest in stored fallback order for in-request failover - group doc is re-read per request so edits are hot - unit tests: canonical 3:1 cycle, exact 30/10 over 40 picks, equal weights, single member, GC, mid-stream weight change; router tests for queue exclusion, disabled/dangling drops, breaker-open error, empty group error, winner-first ordering Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Exact request-model to actual-model mapping per route member, applied before the existing role-keyword mappers so the mapped name inherits normal [1m] handling and upstream override logic. - apply_route_model_mapping rewrites body.model only on an exact table hit; unmatched models keep existing behavior - RequestForwarder gains an optional route_model_mappings table (provider_id -> request -> actual) via with_route_model_mappings; absent mappings are a no-op - request_builder applies the member table immediately before apply_model_mapping - tests: rewrite-only-matched, empty-table no-op, route mapping feeding the role-keyword mapper Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
HandlerContext now loads route mode state per request: the active flag plus the member model-mapping table, copied into both forwarder construction sites. - health_checks_engaged() = auto failover or route mode; replaces the failover-only predicate for bypass_circuit_breaker and the streaming/ non-streaming timeout getters so route-mode apps get breakers and hung-member timeouts - sync_successful_provider_selection returns early under route mode: no current-provider rewrite, no failover snapshot refresh, no failover_count bump; in-memory observability target is still recorded - test: sync no-op assertions under route mode Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Service-level lifecycle for route mode, restricted to the takeover apps (claude/codex/gemini) like the failover flows it mirrors. - enable_route_mode_for_app: requires at least one enabled member and no active automatic failover, reuses the failover always-on primitive for proxy routing, flips the flag last - enable_proxy_and_route_mode_for_app: attaches the daemon-backed managed session and enables route mode in one step - disable_route_mode_for_app flips the flag only; member config and proxy takeover stay untouched - exclusivity: enable_auto_failover_for_app and the combined proxy+failover activation refuse while route mode is active, before any live-config mutation; the managed-session disable path refuses while route mode is active (startup recovery bypasses it) - proxy show reports route mode with the enabled member count - tests: every guard refusal, unsupported-app rejection, flag-only disable Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ProxySnapshot gains route_mode_enabled plus route_members projections (provider id and display name, weight, enabled, mapping count) read from the per-app route group doc on every snapshot load, so the Route Mode screen reflects live edits. Dangling members fall back to their raw provider id as the display name. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New Route Mode screen as a top-level nav item for the proxy-capable apps (Claude/Codex/Gemini), right after Providers. - Screen: status row toggles route mode with Enter, member table shows name / enabled / weight / mapping count in stored fallback order, read-only endpoint footer; all long-form guidance lives in ? help - Keys: w edits the member weight (1-1000), m edits the exact-match model-mapping JSON, a opens the member picker, d removes a member - Member picker overlay lists all app providers with group membership - Enable flow mirrors automatic failover: warns without an enabled member, and confirms ProxyEnableAndRouteMode when the app is not routed yet - Runtime actions persist through the validating DAO so per-member model mappings survive edits; exclusivity and managed-session guards surface as error toasts - ProxySnapshot members now carry the full mapping table for editor prefill - Tests: nav order, intent-to-action coverage, runtime-action DAO effects (add/weight/toggle/mappings/remove + unknown-provider rejection), render assertions Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
End-to-end tests over a live proxy with one mock upstream per route member: - 3:1 weights over 40 requests hit upstreams exactly 30/10 - mapped member receives the rewritten model, unmapped member receives the request model verbatim - upstream 500 fails over in-request to the backup and the breaker records the failure - failover-queued non-member receives zero requests while route mode is on - lifecycle guards surface their text errors (empty group, unsupported app, failover conflict, failover-enable refusal, proxy-disable refusal) - stale-takeover startup recovery keeps the route mode flag and member config - hung member times out under non_streaming_timeout and the backup completes the request Also exports RouteModeGroup/RouteModeMember from the library root for integration tests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Blind-review findings, verified and fixed: - set_route_mode_enabled no longer re-checks provider existence: a member referencing a deleted provider could previously wedge route mode on (disable refused, and proxy disable refused in turn). The flag flip now skips the existence check; direct member edits still validate. Unit test covers disable/re-enable over a dangling member. - enabling route mode no longer reports automatic-failover wording when the daemon routing precondition fails: the shared precondition takes a feature label, so errors say what the user asked for. - weight input pre-validates the 1..=1000 range so the toast matches the DAO rule instead of a differently-worded save error. - removed the dead always-true wrapper around the route-mode guard in the managed-session disable path. - daemon DropTakeover handler carries a comment pinning why the route-mode guard lives in the foreground caller only. - route mode ? help documents that model mappings rewrite the request body's model field: they apply to Claude/Codex traffic while Gemini passthrough carries the model in the URL path (same scope as the existing role-keyword mapping). - new tests: structurally-invalid object fails closed (read path), e2e proves a live group edit is visible to the very next request. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Round-two blind review follow-ups: - member edits no longer fail because of a different, externally deleted member: the provider-existence check exempts members carried over unchanged from the stored doc (new or changed ids are still validated), so removing p2 no longer errors on dangling p1 - corrupt-doc warnings log once per app per process instead of at request rate (the request path re-reads the group 3-4x per request) - dormant state is visible on primary surfaces: the Route Mode screen shows 'enabled (proxy stopped)' and proxy status shows 'route mode on (N members; proxy stopped)' when the flag is on but the proxy isn't running - new unit test pins the route-mode wording of the daemon-routing refusal Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- member mutations warn when a saved group is enabled but has no enabled member left, so the 'route mode on, zero members, all traffic failing' state is visible at the keypress that creates it - the Route Mode screen's dormant qualifier now keys off per-app routing (routes_current_app_through_proxy) instead of the global proxy running flag, matching the codex-worker-only scenario - new e2e: failover between two members with different mapping tables proves each attempt derives the body from the original request with its own member's mappings (no chaining across attempts) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- streaming e2e: a streaming request with a hung winner times out on
the first byte and completes as SSE from the backup, covering the
changed streaming timeout/breaker gates
- Codex e2e: route mode on the codex app rewrites body.model via the
member mapping and passes unmapped models through verbatim
- the proxy-disable guard message no longer implies the proxy is
currently running ('disable route mode first')
- proxy status pluralizes members and labels the section
'Auto failover / route mode' now that the section shows both
- router unit test pins route-branch priority when both modes are on
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the keyboard-dead, freezing route-mode screen with a proper interactive flow: - Fix cursor navigation: Up and Down move independently (j/k via the vim normalizer); weight/mappings/delete bindings appear once a member row is selected. - The member picker closes after add/remove and the cursor lands on the newly added row. - Weight editing (w) validates 1..=1000 input and short-circuits equal-weight resubmission without a DB write or reload-token bump. - Multi-row mappings editor (m): add, edit, and delete multiple request->actual pairs with duplicate/empty rejection (1-based row toasts), an empty-table clear, and Esc applying the whole table as one SetRouteModeMemberMappings action. - Upstream model fetch (Ctrl+F) on the actual-model field supports Anthropic, Gemini-native, OpenAI-compatible Bearer, and Codex OAuth providers through a target-tagged model-fetch pipeline (ModelFetchTarget), keeping the existing form-field flow behavior-preserving; failures keep the editor and its draft. - Disabling or removing the last enabled member is blocked with a warning toast while route mode is ON; free when OFF. - Member mutations take the fast path: Database::init + targeted in-place refresh of route-mode data with mark-before-refresh and a best-effort targeted read, instead of a synchronous full UiData load. Reload-token staleness machinery discards stale full/snapshot loads, retargets pending warm loads on app switch, and drops stale snapshot refreshes so background reloads cannot revert visible edits. - New i18n strings, contextual ? help entries, render cue for no-enabled-member groups, and 60+ tests across keymap, picker, editor, fetch, staleness, DAO persistence, and proxy integration. Blind-reviewed through 13 rounds per the repository protocol; final round reports all 9 acceptance criteria satisfied with only accepted nit-level findings. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add Ctrl+R on the mapping editor's request field to open the model-fetch picker pre-filled locally with request models read from the route app's active provider config (Claude env model keys with [1m] markers stripped, Codex top-level TOML model, Gemini GEMINI_MODEL). No network involved. Adding a route member now seeds passthrough mappings (candidate -> candidate) from the same pool, insert-if-absent only so existing keys are never clobbered. Empty candidate pools keep the editor open with a toast. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Route mode's help was one prose blob shared by the route screen and the model-mappings editor, and it never mentioned the new Ctrl+R/Ctrl+F row keys or the passthrough auto-seed. Give the editor its own HelpTarget that carries the provider name into the title, keep the member picker on the screen help, and rewrite both blobs in the `· key — purpose` bullet style used by the Codex catalog help (EN " — ", ZH "——"). Screen help keeps the rules (last enabled member, failover mutual exclusion, proxy takeover, dormant persistence) and drops the nginx analogy. Editor help covers mapping semantics, the Gemini URL-path caveat, Esc apply-all with the stays-open-on-error report, and an indented row-editing section. Help wording is aligned one-way to the keymap chip vocabulary (delete member / add member / apply); shared t! key labels are unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Route-model mapping: exact hit keeps priority; on miss retry the [1m]-stripped name so picker-seeded bare keys match suffixed requests - Route-mode flag flip mutates the stored doc raw so structurally invalid members survive instead of being erased by a default-collapse rewrite - Remove unused route_mode keymap help_items (help.rs uses a static block); add pinning tests for the mapping fallback, the raw flag flip, and the route-mode enable success path with a live managed worker Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resolve tests-module overlap in src-tauri/src/services/proxy.rs by keeping both the route-mode test helpers/tests and the upstream gemini/claude recovery tests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an opt-in route mode for each proxy-capable app (claude, codex, gemini, opencode, hermes, openclaw): instead of forwarding every request to the single current provider, the proxy routes to a member of a weighted provider group using smooth weighted round-robin.
[1m]handling run. Keys seeded by the TUI picker are bare, soname[1m]requests hit the barenameentry (exact keys keep priority).route_mode_{app}, camelCase JSON); no schema changes, keeping this fork's schema synced with upstream. Missing or corrupt data fails closed to a disabled group.?help carries the behavioral caveats.Testing
proxy_route_modeplus unit tests across the DAO, service, router, forwarder, and TUI layers: lifecycle, weighted selection distribution, exact mapping (hit/miss/[1m]priority), exclusivity both directions, corrupt-storage fail-closed, and test-home isolation viatests/support.rs.cargo fmt --checkpasses; branch rebased/merged onto currentmain(conflict inservices/proxy.rstests module resolved by keeping both sides).Known pre-existing failures on
main(not introduced here)Verified present on
origin/mainand left untouched (upstream parity):import_export_sync: 2 failures (auth.json not written under the test HOME)prompt_commands: 2 failures (external-editor resolution is environment-dependent)provider_commands: 3 failures (usage-query script handling)proxy_service::managed_session_disable_last_app_does_not_terminate_when_status_probe_fails(introduced by fix(proxy): stop daemon workers when runtime metadata is missing #508's area)🤖 Generated with Claude Code