Repository navigation
Merged
Conversation
…ro new deps (item #438) Replaces the hardcoded 3-provider ProviderKind match (NvidiaNim/OpenRouter/ LmStudio) with a small provider-kind registry: OpenAiCompatible (covers NVIDIA NIM, OpenRouter, LM Studio, and any future OpenAI-compatible endpoint via config alone), Anthropic, and Gemini. Patterns vendored (not depended on) from litellm-rust (avivsinai, MIT) -- see items #436 (rig-core, rejected: streaming not raw-JSON-shaped) and #437 (litellm-rs rejected for dependency weight; litellm-rust's design is the reference). Zero new Cargo.toml/Cargo.lock entries: reuses reqwest/ serde_json/futures already in flare-proxy instead of importing an SSE macro crate, an error crate, or a trait-object abstraction. - providers/openai_compat.rs: header/auth wiring only -- request/response translation stays shape_xlat's existing Anthropic<->OpenAI logic. - providers/anthropic.rs: near-passthrough. Since flare-proxy's own wire format already IS Anthropic Messages, native Anthropic upstream needs no shape_xlat at all -- forward.rs proxies the byte stream unchanged. - providers/gemini.rs: new Anthropic<->Gemini translator, scoped to text + tool_calls (no image/audio/video/file parts -- out of scope for this proxy's actual traffic). New shape_xlat::gemini_chunk_to_anthropic_sse / gemini_finish_stream read Gemini's candidates[0].content.parts shape instead of OpenAI's choices[0].delta. forward.rs restructured to dispatch per ProviderKind before body translation (each upstream needs a different request shape and URL path, not just different headers), with a shared stream_translated_sse() helper for the two paths that do need chunk-by-chunk translation. 65/65 tests pass, clippy clean (-A unsafe_code -A clippy::pedantic), fmt clean. Agentflare-Agent: claude-code_2-1-225_agent Agentflare-Branch: task/438 Agentflare-Item: 438
…opic/cf_gateway_openai (item #438) Cloudflare AI Gateway is a URL prefix in front of a provider's own wire protocol, not a distinct protocol -- gateway.ai.cloudflare.com/v1/{acct}/ {gw}/anthropic still speaks Anthropic's real Messages API shape, .../compat still speaks OpenAI's /chat/completions shape. So this needed no new ProviderKind: just two new provider prefixes whose base_url is built from CF_AI_GATEWAY_ACCOUNT_ID/CF_AI_GATEWAY_ID, reusing the Anthropic and OpenAiCompatible paths added earlier in this item unchanged. cf-aig-authorization (Cloudflare's own gateway-level auth, separate from the underlying provider's API key) is optional, set from CF_AI_GATEWAY_TOKEN via the extra_headers mechanism already added for OpenRouter. Missing CF_AI_GATEWAY_ACCOUNT_ID/ID fails fast to default_free() at startup rather than shipping an unusable base_url. Closes the Cloudflare AI Gateway gap identified earlier this session (flare-proxy previously had zero Cloudflare support of any kind). 68/68 tests pass, clippy clean, fmt clean. Zero new dependencies (single file changed). Agentflare-Agent: claude-code_2-1-225_agent Agentflare-Branch: task/438 Agentflare-Item: 438
…ded match arms (item #438) Per appz-cli's registry/toolchains.toml pattern (crates/appz-core/src/ toolchain_registry.rs) -- scoped down to just what flare-proxy needs: no HTTP-fetch-with-cache-TTL layer, just an embedded TOML file (include_str!) parsed once via OnceLock. Adding a provider -- another OpenAI-compatible endpoint, another Cloudflare AI Gateway route -- is now a registry/providers.toml edit, not a Rust change. providers/registry.rs owns the load/lookup/resolve logic: - RegistrySpec mirrors ProviderEntry's shape (prefix, id, kind, base_url, api_key_env, extra_headers) plus two fields the hardcoded version didn't need: base_url_template ({ENV_VAR} placeholders, for Cloudflare's per-user account_id/gateway_id) and gateway_auth_env/gateway_auth_header (an extra header sourced from an env var, distinct from the provider's own api_key_env -- Cloudflare's cf-aig-authorization). - resolve() returns None when a template references an unset env var, so from_env()'s existing 'fail fast to default_free()' behavior is preserved without each provider needing its own bespoke resolver function (deleted cloudflare_gateway_base_url/_headers and openrouter_headers -- all three are now registry.toml entries). default_free() now also builds its 3 built-in providers by resolving them out of the same registry, instead of duplicating base_url/api_key_env/ extra_headers as separate hardcoded ProviderEntry literals -- one source of truth for provider identity, whether reached via MODEL=... or the zero-config default path. toml crate added to flare-proxy's Cargo.toml -- already a workspace dependency (root Cargo.toml), so this adds zero new entries to Cargo.lock. 74/74 tests pass, clippy clean, fmt clean. Agentflare-Agent: claude-code_2-1-225_agent Agentflare-Branch: task/438 Agentflare-Item: 438
… embed it (item #438) The embedded-only registry.toml from the previous commit still required a new agentflare release to add a provider -- it was baked into the binary at compile time. This closes that gap the same way appz-cli's toolchain registry does (crates/appz-core/src/{toolchain_registry,registry_cache}.rs), scoped down to flare-proxy's single call site: no offline-mode CLI flag, no generic <T> cache helper, since there's nothing else to share it with. registry() -- used by find()/resolve()/known_prefixes(), and by this module's own tests -- reads only the embedded TOML plus whatever's already cached on disk. It never touches the network, so it stays fast and deterministic in cargo test (77 tests in 0.02s, unchanged from before this commit). ensure_fresh() is the explicit 'download on demand' step: called once from router() before the first ProviderConfig::from_env() read, it checks the local cache (~/.agentflare/flare-proxy-providers-registry.json) against a 24h TTL and, if stale, attempts one 3s-timeout live fetch of registry/providers.toml's latest published version, merging remote entries over the embedded ones by prefix. Any failure -- offline, DNS, non-2xx, malformed TOML -- silently falls back to the embedded copy; existing installs never break because of a network hiccup, they just don't get the newest provider until the next successful refresh. dirs and ureq added to flare-proxy's Cargo.toml -- both already workspace dependencies via the main agentflare crate, so this adds zero new entries to Cargo.lock (confirmed via diff). 77/77 tests pass, clippy clean (including a result_large_err fix -- ureq::Error is >270 bytes, fetch_remote() now maps to a plain String error immediately, matching appz-cli's own Result<String, String> convention for the same kind of fetch function), fmt clean. Agentflare-Agent: claude-code_2-1-225_agent Agentflare-Branch: task/438 Agentflare-Item: 438
…le provider set (item #438) Adds 17 more openai_compatible entries alongside the original NVIDIA NIM/ OpenRouter/LM Studio/Cloudflare-Gateway set: OpenAI itself, DeepSeek, Groq, Mistral, xAI, Together, Perplexity, Fireworks, DeepInfra, Moonshot, Novita, Hyperbolic, Cerebras, SambaNova, GitHub Models, Cohere (compatibility API), and local Ollama. base_url values are each provider's own documented OpenAI-SDK-compatible endpoint as of this session -- best-effort from public docs/conventions, not individually live-verified against every provider's current docs given how many there are. Flagged as such directly in the TOML file. If one drifts, that's exactly what the on-demand refresh added in the previous commit exists to fix -- correcting registry/providers.toml upstream reaches every existing install on its next refresh, no new release needed. Two new regression tests: every_registry_entry_has_a_resolvable_static_or_ gateway_url (catches a future copy-paste mistake -- a provider block missing both base_url and base_url_template -- at test time instead of a confusing request-time 'no route' error) and broad_openai_compatible_providers_are_present_and_resolve (locks in that all 17 new prefixes parse, resolve, and produce a well-formed URL). 79/79 tests pass, clippy clean, fmt clean. Agentflare-Agent: claude-code_2-1-225_agent Agentflare-Branch: task/438 Agentflare-Item: 438
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThe proxy now loads provider definitions from an embedded and cached registry. It supports native Anthropic and Gemini requests, generic OpenAI-compatible providers, Cloudflare Gateway routes, provider authentication, and Gemini-to-Anthropic SSE translation. ChangesProvider proxy expansion
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant AnthropicClient
participant FlareProxy
participant ProviderRegistry
participant UpstreamProvider
AnthropicClient->>FlareProxy: Send Messages request
FlareProxy->>ProviderRegistry: Resolve provider and endpoint
ProviderRegistry-->>FlareProxy: Return provider metadata
FlareProxy->>UpstreamProvider: Send translated request
UpstreamProvider-->>FlareProxy: Stream provider response
FlareProxy-->>AnthropicClient: Stream Anthropic SSE response
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 8
🧹 Nitpick comments (3)
crates/flare-proxy/src/forward.rs (2)
273-290: 🩺 Stability & Availability | 🔵 Trivial | 🏗️ Heavy liftEmit stream termination when the upstream stream ends without a finish marker.
finishruns only when a chunk carrieschoices[0].finish_reasonorcandidates[0].finishReason. If the upstream connection closes after the last content chunk, or the transport error branch at line 242 triggers, the proxy sends nocontent_block_stop,message_delta, ormessage_stop. The Anthropic client then waits for an incomplete message.Track whether
finishalready ran, and emit it once from a stream-end step.Also applies to: 329-330
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/flare-proxy/src/forward.rs` around lines 273 - 290, Update the stream handling around the finish-marker detection and the stream-end/error branches to track whether the termination sequence has already been emitted. Reuse the existing finish logic to emit content_block_stop, message_delta, and message_stop exactly once when the upstream ends without a finish marker, including the transport error path, while preserving normal marker-triggered termination.
317-327: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueRemove the discarded
strip_think_tagscall.The result is dropped, so the call allocates a new
Stringand aVec<String>on every finish with no effect on output. The comment already explains why suppression is not implemented. Keep the comment and delete the call until buffering lands.♻️ Proposed change
- if needs_think && !accumulated_text.is_empty() { - let _ = crate::think::strip_think_tags(&accumulated_text); - }
needs_thinkthen becomes unused in this function; drop the parameter or keep it with#[allow(unused)]and a TODO.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/flare-proxy/src/forward.rs` around lines 317 - 327, Remove the discarded crate::think::strip_think_tags call from the accumulated_text handling block while preserving the explanatory comment. Then update the surrounding function signature and call sites to remove the now-unused needs_think parameter, unless the existing API requires retaining it; do not add replacement processing.crates/flare-proxy/Cargo.toml (1)
19-21: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueRemove the redundant
ureqjsonfeature fromcrates/flare-proxy/Cargo.toml.
flare-proxyuses only.call()andResponse::into_string(). Keepureq2.x for the current API. Other workspace crates still require the JSON feature.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/flare-proxy/Cargo.toml` around lines 19 - 21, Update the flare-proxy ureq dependency declaration to remove the redundant json feature while retaining the existing ureq 2.x version and dependency. Do not change ureq feature configuration in other workspace crates.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/flare-proxy/registry/providers.toml`:
- Around line 76-83: Remove the github_models provider entry from the additional
OpenAI-compatible providers registry in providers.toml, including its obsolete
endpoint, while leaving all other provider definitions unchanged.
In `@crates/flare-proxy/src/forward.rs`:
- Around line 96-110: The inline request builders in proxy_anthropic and
proxy_gemini drop provider.extra_headers, including gateway authentication. Add
a shared helper or equivalent loop that applies every provider.extra_headers
entry to both request builders; update crates/flare-proxy/src/forward.rs lines
96-110 and lines 140-147. crates/flare-proxy/registry/providers.toml lines 58-74
requires no direct change; re-verify cf_gateway_anthropic against an
authenticated gateway.
In `@crates/flare-proxy/src/lib.rs`:
- Around line 82-90: Remove the function-local environment mutex in the
providers registry tests and update every test there that mutates process-global
environment variables to acquire the shared crate::test_env_lock() helper from
lib.rs. Ensure all MODEL, MODEL_OPUS, AGENTFLARE_HOME_OVERRIDE, and related env
mutations use this single lock.
In `@crates/flare-proxy/src/providers/gemini.rs`:
- Around line 60-75: Update the tool conversion logic in the declarations
builder so each input_schema is recursively filtered to Gemini-supported schema
fields before assigning functionDeclarations.parameters. Remove unsupported
fields such as $schema and additionalProperties at every nested level while
preserving nullable, format, anyOf, $ref, $defs, and the other supported fields
defined by the existing schema filtering rules; keep the object-type fallback
when input_schema is absent.
In `@crates/flare-proxy/src/providers/mod.rs`:
- Around line 128-152: Update from_env() route construction so
parse_model_string’s raw provider prefixes are resolved through registry::find
and stored as the matching spec.id for the default, opus, sonnet, and haiku
routes; keep providers entries keyed by spec.id. Add a regression test using
MODEL="nvidia_nim/meta/llama-3.1-405b-instruct" that verifies the resulting
route resolves to an existing provider.
In `@crates/flare-proxy/src/providers/registry.rs`:
- Around line 30-31: Harden registry merging so untrusted remote entries cannot
alter embedded providers’ base_url or api_key_env and redirect API-key traffic.
Update the merge logic around parse_registry_toml and the registry refresh path
to permit remote entries only for new prefixes, or otherwise enforce an embedded
host/credential allowlist; ensure cached remote data receives the same
protection before use.
- Around line 317-328: Update the test
resolve_template_substitutes_multiple_placeholders to acquire
crate::test_env_lock() before modifying environment variables, retaining the
guard for the entire test until all variables are removed.
In `@crates/flare-proxy/src/shape_xlat.rs`:
- Around line 582-605: Update gemini_finish_stream to track whether
buffer.open_indices contained an emitted tool-use block, then derive stop_reason
as "tool_use" for that case even when finishReason is "STOP"; preserve the
existing MAX_TOKENS and explicit TOOL_CALLS mappings otherwise. Adjust the
corresponding test fixture and expectation to use Gemini finishReason "STOP"
with the tool block.
---
Nitpick comments:
In `@crates/flare-proxy/Cargo.toml`:
- Around line 19-21: Update the flare-proxy ureq dependency declaration to
remove the redundant json feature while retaining the existing ureq 2.x version
and dependency. Do not change ureq feature configuration in other workspace
crates.
In `@crates/flare-proxy/src/forward.rs`:
- Around line 273-290: Update the stream handling around the finish-marker
detection and the stream-end/error branches to track whether the termination
sequence has already been emitted. Reuse the existing finish logic to emit
content_block_stop, message_delta, and message_stop exactly once when the
upstream ends without a finish marker, including the transport error path, while
preserving normal marker-triggered termination.
- Around line 317-327: Remove the discarded crate::think::strip_think_tags call
from the accumulated_text handling block while preserving the explanatory
comment. Then update the surrounding function signature and call sites to remove
the now-unused needs_think parameter, unless the existing API requires retaining
it; do not add replacement processing.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 0f549718-9d57-4b92-991d-9ada11c4c38b
⛔ Files ignored due to path filters (1)
Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (10)
crates/flare-proxy/Cargo.tomlcrates/flare-proxy/registry/providers.tomlcrates/flare-proxy/src/forward.rscrates/flare-proxy/src/lib.rscrates/flare-proxy/src/providers/anthropic.rscrates/flare-proxy/src/providers/gemini.rscrates/flare-proxy/src/providers/mod.rscrates/flare-proxy/src/providers/openai_compat.rscrates/flare-proxy/src/providers/registry.rscrates/flare-proxy/src/shape_xlat.rs
Comment on lines
+82
to
+90
|
|
||
| /// Serializes tests that mutate process-global env vars (`MODEL`, | ||
| /// `MODEL_OPUS`, ...) — `cargo test` runs tests in parallel by default, and | ||
| /// unsynchronized `env::set_var` calls across tests race. | ||
| #[cfg(test)] | ||
| pub(crate) fn test_env_lock() -> std::sync::MutexGuard<'static, ()> { | ||
| static LOCK: std::sync::Mutex<()> = std::sync::Mutex::new(()); | ||
| LOCK.lock().unwrap_or_else(|e| e.into_inner()) | ||
| } |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Use one shared environment lock.
crates/flare-proxy/src/providers/registry.rs defines another function-local static LOCK. That mutex is distinct from the mutex in crates/flare-proxy/src/lib.rs, so registry tests do not synchronize with tests that call crate::test_env_lock(). Parallel tests can observe incorrect MODEL or AGENTFLARE_HOME_OVERRIDE values.
Remove the duplicate helper and make every environment-mutating test call this shared helper.
Proposed fix
- let _guard = test_env_lock();
+ let _guard = crate::test_env_lock();🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@crates/flare-proxy/src/lib.rs` around lines 82 - 90, Remove the
function-local environment mutex in the providers registry tests and update
every test there that mutates process-global environment variables to acquire
the shared crate::test_env_lock() helper from lib.rs. Ensure all MODEL,
MODEL_OPUS, AGENTFLARE_HOME_OVERRIDE, and related env mutations use this single
lock.
…roviders (item #438) Three more openai_compatible registry entries, picked for coding use: siliconflow (Qwen3-Coder-480B, DeepSeek-V3.2/R1, permanently free after identity verification), llm7 (Qwen2.5-Coder-32B, DeepSeek-R1, no-cost daily tier), requesty (300+ model router, cleanest ToS rating of the free aggregators surveyed). base_url values verified against each provider's own docs (SiliconFlow quickstart, LLM7 live /v1/models endpoint, Requesty gateway docs), not guessed. Extended broad_openai_compatible_providers_are_present_and_resolve to cover all three. 11/11 flare-proxy registry tests pass, clippy clean, fmt clean. Agentflare-Agent: claude-code_2-1-225_agent Agentflare-Branch: task/438 Agentflare-Item: 438
…(item #438) New optional RegistrySpec fields -- free_type, monthly_tokens, credit_tokens -- record each provider's documented free-tier budget: recurring-daily/monthly (renews), recurring-uncapped (rate-limited, no token cap), one-time-initial (signup credit), or unmetered (local). Populated for all 21 providers with a documented free tier; omitted for paid-key/pass-through providers (openai, xai, together's paid tier, perplexity, moonshot, anthropic, cf_gateway_*) with no free-tier data. Data pulled from OmniRoute's freeModelCatalog.data.ts (pool-deduped to each provider's max), not estimated -- source noted in the TOML header. Metadata only: no selection/load-balancing logic reads these fields yet. That's the natural follow-up once this shape has been used for a bit, scoped separately per plan. New test (every_registry_free_type_is_a_known_value) catches a typo'd free_type value against a known set. 12/12 flare-proxy registry tests pass, clippy clean, fmt clean. Agentflare-Agent: claude-code_2-1-225_agent Agentflare-Branch: task/438 Agentflare-Item: 438
…tem #438) New providers::quota module: tracks per-provider token consumption against the free_type/monthly_tokens/credit_tokens registry metadata added in the previous commit, and picks the first available provider from a priority-ordered candidate list. Design ported (not copied) from two real implementations, researched and cloned locally for reference: - LiteLLM's provider budget limiter (BerriAI/litellm, MIT, litellm/router_strategy/budget_limiter.py) -- the get-or-init window start / reset-if-expired / increment-if-current three-way branch, without its Redis multi-instance sync (flare-proxy is one process). - OmniRoute's emergency fallback (diegosouzapw/OmniRoute, MIT, open-sse/services/emergencyFallback.ts) -- fail-open on tracking errors, walk a short ordered candidate list rather than re-rank a large pool. Usage persists to flare-proxy-usage.json next to the existing registry cache (agentflare_home(), now pub(super) for quota.rs to reuse rather than duplicating the home-dir helper). Scope boundary, called out in the module doc: this is the tracking/ selection primitive only, not wired into forward.rs's request path yet. ProviderConfig/ModelRoute resolve to exactly one provider per request today; using select_best for real routing needs a candidate list there, and record_consumption needs a real token count, which the streaming response path doesn't currently extract. Both are separate design decisions, tracked as the next item #438 follow-up rather than bolted on here unreviewed. 7 new tests (budget exhaustion, window expiry reset, one-time credits never resetting, priority-ordered fallback selection). 87/87 flare-proxy tests pass (stable across repeated runs -- fixed a test-isolation race by reusing crate::test_env_lock() instead of a module-local mutex, since AGENTFLARE_HOME_OVERRIDE is a process-global env var shared with registry.rs's own tests). clippy clean, fmt clean. Agentflare-Agent: claude-code_2-1-225_agent Agentflare-Branch: task/438 Agentflare-Item: 438
- forward.rs: apply provider.extra_headers on the Anthropic/Gemini paths too, not just OpenAI-compat -- Cloudflare AI Gateway's cf-aig-authorization was silently dropped for cf_gateway_anthropic - mod.rs: from_env() route construction now resolves MODEL prefixes (e.g. nvidia_nim) to their registry id (nvidia-nim) before storing provider_id, fixing a 400 "unknown provider" for any prefix whose id differs from its prefix - gemini.rs: recursively strip JSON-Schema fields Gemini's function-calling API rejects ($schema, etc.) from tool input_schema before sending - shape_xlat.rs: gemini_finish_stream now reports stop_reason "tool_use" whenever a tool block was opened, even when Gemini's finishReason is "STOP" (its common case for a turn containing a functionCall) - registry.rs: merge() no longer lets a remote/cached registry entry override an existing provider's base_url/api_key_env, only its quota metadata -- a compromised publish path could otherwise redirect API traffic to an attacker host - providers.toml: remove github_models -- GitHub retired the Models inference endpoint (410) - registry.rs: add the missing test_env_lock() guard to resolve_template_substitutes_multiple_placeholders - Cargo.toml: drop ureq's unused json feature - forward.rs: remove a dead strip_think_tags() call whose result was discarded Agentflare-Agent: claude-code_2-1-225_agent Agentflare-Branch: task/438 Agentflare-Item: 438
This was referenced Aug 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ProviderKindmatch (NVIDIA NIM/OpenRouter/LM Studio) with a small provider-kind system:openai_compatible(any/chat/completions-shaped endpoint),anthropic(near-passthrough, since flare-proxy's own inbound wire format already is Anthropic Messages), andgemini(new Anthropic<->Gemini translator, scoped to text + tool_calls).cf_gateway_anthropic,cf_gateway_openai) — a URL prefix in front of the same wire protocols already spoken, not a new protocol.registry/providers.toml, loaded via embedded defaults + a locally-cached, on-demand-refreshed remote copy (mirrors appz-cli'sregistry/toolchains.tomlpattern, scoped to this crate's single call site). Adding or fixing a provider is now a TOML edit published upstream, not a new agentflare release.Cargo.lock—toml,dirs, andureqwere all already resolved via the mainagentflarecrate; adding them toflare-proxy/Cargo.tomlreuses those exact resolutions.Design notes
providers/anthropic.rs's near-passthrough means native Anthropic upstream doesn't needshape_xlattranslation at all —forward.rsproxies the byte stream unchanged, since input and output are both Anthropic's own SSE shape.providers/gemini.rs+ newshape_xlat::gemini_chunk_to_anthropic_sse/gemini_finish_streamhandle Gemini's distinctcandidates[0].content.partsstreaming shape.providers/registry.rs'sregistry()accessor (used byfind/resolve/known_prefixes, and all tests) never touches the network — it only reads the embedded TOML plus whatever's already cached on disk, socargo teststays fast and deterministic (79 tests in 0.02s).ensure_fresh()is the separate, explicit "download on demand" step, called once at proxy startup (router()): if the local cache (~/.agentflare/flare-proxy-providers-registry.json) is missing or older than 24h, it attempts one 3-second-timeout live fetch of the latest publishedregistry/providers.tomland merges remote entries over the embedded ones by prefix. Any failure (offline, DNS, non-2xx, malformed TOML) silently falls back to the embedded copy.base_urlvalues are each provider's documented OpenAI-compat endpoint as of this session — best-effort from public docs, not individually live-verified for every entry given how many there are. Flagged directly in the TOML file; a wrong one is a config fix via the refresh mechanism above, not a code change.Test plan
cargo test -p flare-proxy— 79/79 passcargo clippy -p flare-proxy --all-targets -- -D warnings -A unsafe_code -A clippy::pedantic— cleancargo fmt -p flare-proxy --check— cleangit diff Cargo.lockshows zero new crate entriesSummary by CodeRabbit
New Features
Bug Fixes