diff --git a/AGENTS.md b/AGENTS.md index d6cc08a..914a1c9 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -58,14 +58,21 @@ settings for future launches; manual settings changes clear its active-profile marker. Profiles are stored separately in `/profiles.json`. See `core::paths` for how the data directory is resolved. -`build_providers` in `crates/alan/src/main.rs` lists the registered providers. -The `providers` crate also has a Google provider that is not registered. Sign in -with `/login`, or set the provider's API key variable: +`build_providers` in `crates/alan/src/main.rs` lists the built-in providers. +The `providers` crate also has a Google provider that is not registered. Sign +in with `/login`, or set the provider's API key variable: ```bash OPENROUTER_API_KEY=... cargo run -p alan ``` +`ProviderRegistry` holds the built-ins from `build_providers`, then the custom +providers added with `/provider add`; a built-in wins a name clash. +`core::custom_providers::CustomProviderStore` keeps a custom provider's name, +URL and API standard (`ApiId`) in `/providers.json` and its key in +`auth.json`. `/provider` saves there first, then registers a +`providers::CustomProvider`. Only `ApiId::ChatCompletions` is implemented. + Optional model override: ```bash @@ -143,7 +150,7 @@ auto-includes it in release archives. `ChatView` (`views/components/chat_view/`) owns the chat session: the `ChatController` and the agent stream. `PromptEditor` owns input. Overlays -such as login and local models live beside `root.rs`. +such as login and providers live beside `root.rs`. Keep UI simple. Avoid borders, unnecessary widgets, and premature abstraction. diff --git a/CHANGELOG.md b/CHANGELOG.md index 6c89bca..aabdcd4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Added + +- `/provider add`, `/provider edit` and `/provider remove` to use any Chat Completions server by name. Alan checks the server's `/models` before saving, lists all of its models in `/models`, and keeps the key in `auth.json` so `/login` can change it. + ### Changed - Update default system prompt to include output formatting. diff --git a/README.md b/README.md index 2309413..647d136 100644 --- a/README.md +++ b/README.md @@ -13,7 +13,7 @@ A minimal coding agent in your terminal. - [Providers](#providers) - [Usage](#usage) - [Permissions](#permissions) -- [Local models](#local-models) +- [Custom providers](#custom-providers) - [Skills](#skills) - [Configuration and data locations](#configuration-and-data-locations) - [Building from source](#building-from-source) @@ -21,13 +21,13 @@ A minimal coding agent in your terminal. ## Features -- **First-class OpenRouter support** — full model catalog, per-model custom provider pinning (`/model-providers`), and reasoning effort control (`/effort`). +- **First-class OpenRouter support** — full model catalog, per-model upstream provider pinning (`/model-providers`), and reasoning effort control (`/effort`). - **Paste images** — drop a screenshot from your clipboard straight into the conversation and ask about it. - **Compact with focus** — `/summarize-new "focus hint"` summarizes the context and restarts with just what matters. - **Steering** — queue a message mid-run and the agent picks it up without waiting for the response to finish. - **Plan and review modes** — cycle through them with `Shift+Tab` when you want the agent to think before it acts. -- **Local models** — point Alan at any OpenAI-compatible endpoint, no API key required. - **Skills** — type `#name` in the prompt to attach a skill's full instructions to that message. +- **Custom providers** — add any Chat Completions server, hosted or on your machine, with `/provider add`; all of its models appear in `/models`. ## Installation @@ -79,7 +79,9 @@ Alan currently supports: Use the `/login` command to sign in to any of the available providers. -For OpenRouter, you can pin a custom provider for the selected model with `/model-providers`, followed by the provider name (no quotes): +Any other Chat Completions server can be added with `/provider add` (see [Custom providers](#custom-providers)). + +For OpenRouter, you can pin an upstream provider for the selected model with `/model-providers`, followed by the provider name (no quotes): - `/model-providers deepseek` - `/model-providers xiaomi/fp8` @@ -92,6 +94,9 @@ For OpenRouter, you can pin a custom provider for the selected model with `/mode | `/profile` | Open the profile picker to switch profiles (`Ctrl+P`, or `Cmd+Option+P` where the terminal passes it through) | | `/profile save ` | Save current model/provider, reasoning, and web settings as a new profile | | `/profile delete` | Pick a saved profile to delete | +| `/provider add` | Add a Chat Completions provider: name, URL and optional API key | +| `/provider edit` | Change a provider's URL or API key | +| `/provider remove` | Remove a provider and its API key | | `/plan` | Toggle plan mode (also `Shift+Tab`) | | `/review` | Toggle review mode (also `Shift+Tab`) | | `/normal` | Turn off plan and review mode | @@ -105,8 +110,7 @@ For OpenRouter, you can pin a custom provider for the selected model with `/mode | `/tool-free` | Allow all tool calls without asking | | `/tool-slip` | Approve a command family once, allow its siblings | | `/tool-strict` | Allow only exact commands already approved | -| `/local` | Manage local models (add, remove, edit) | -| `/model-providers ` | Pin a custom provider for the current OpenRouter model | +| `/model-providers ` | Pin an upstream provider for the current OpenRouter model | | `/help` | List available commands | | `/quit` | Exit Alan | @@ -148,15 +152,13 @@ Alan starts in strict policy and remembers the policy you pick across runs. Swit - `/tool-free` - `/tool-strict` -## Local models +## Custom providers -Alan supports OpenAI-compatible local models. +Any server that speaks the Chat Completions API can be a provider: hosted APIs such as Groq, xAI or Mistral, or a model server on your machine such as Ollama or LM Studio. -```bash -/local add -``` +`/provider add` asks for a name (not one a built-in provider already uses), the base URL (the part before `/chat/completions`, e.g. `https://api.groq.com/openai/v1` or `http://localhost:11434/v1`) and an API key if the server needs one. Alan calls `/models` to check the URL and key, and saves the provider only when that call works. Every model the server lists then appears in `/models`, including ones it adds later. -This opens an overlay with fields for URL, API, API Key, and Model ID. Alan queries the server's `/models` endpoint and offers the detected models in a searchable picker. Use `/local remove` and `/local edit` to manage your local models. No API key or `/login` is needed for local models. +The key is stored in `auth.json` with the other providers' keys, so `/login` can change it too. `/provider edit` changes the URL or key; the name is fixed. `/provider remove` deletes the provider and its key. If one of its models is in use, Alan first asks which model to switch to. ## Configuration and data locations @@ -164,8 +166,8 @@ Alan stores everything under `~/.alan/` by default: - `settings.json` — your current settings and active-profile marker. - `profiles.json` — saved named model/reasoning/web settings profiles. -- `auth.json` — provider credentials saved by `/login`. -- `local_models.json` — local models added with `/local`. +- `auth.json` — provider credentials saved by `/login` and `/provider`. +- `providers.json` — providers added with `/provider add`. - `projects/` — per-project permission grants. - `sessions/` — conversation history (append-only JSONL, one file per session). - `skills/` — personal skills, available in every project. @@ -180,7 +182,7 @@ Settings variables override `settings.json` for that run without changing it. St | Variable | Effect | | --- | --- | | `ALAN_HOME` | Parent of the data directory: `ALAN_HOME=/tmp/x` puts everything in `/tmp/x/.alan`. | -| `ALAN_PROVIDER` | Provider id: `openrouter`, `deepseek`, `zai`, or `local`. | +| `ALAN_PROVIDER` | Provider id: `openrouter`, `deepseek`, `zai`, `chatgpt`, or a name added with `/provider add`. | | `ALAN_MODEL` | Model id for that provider, e.g. `ALAN_MODEL=openai/gpt-4o-mini alan`. | | `ALAN_PROFILE` | Select a saved profile by name for this run without changing `settings.json`. | | `ALAN_REASONING_EFFORT` | `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, or `max`. | @@ -247,4 +249,4 @@ cargo run -p alan ## Requirements - Rust 1.88 or newer (only for building from source) -- An API key for your chosen provider (not needed for local models) +- An API key for your chosen provider, if it needs one diff --git a/crates/alan/src/core/command.rs b/crates/alan/src/core/command.rs index 0cacdca..205f7ec 100644 --- a/crates/alan/src/core/command.rs +++ b/crates/alan/src/core/command.rs @@ -32,9 +32,8 @@ pub enum SlashCommand { ToolFree, ToolSlip, ToolStrict, - // Manage local models (add, remove, edit). - Local, Profile, + Provider, Quit, } @@ -87,8 +86,8 @@ impl SlashCommand { | Self::ModelProviders | Self::SummarizeNew | Self::Rename - | Self::Local | Self::Profile + | Self::Provider ) } @@ -107,13 +106,13 @@ impl SlashCommand { "fork the whole session into a new file and print a command to open it" } Self::Help => "list the available commands", - Self::ModelProviders => "pick a provider from openrouter for the selected model", + Self::ModelProviders => "openrouter only: pick which upstream serves this model", Self::Rename => "rename the current session", Self::ToolFree => "allow all tool calls without asking", Self::ToolSlip => "approve a command family once, allow its siblings", Self::ToolStrict => "allow only exact commands already approved", - Self::Local => "manage local models (add, remove, edit)", Self::Profile => "save and switch named model profiles", + Self::Provider => "add, edit or remove your own Chat Completions server", Self::Quit => "abort and quit", } } @@ -347,7 +346,6 @@ mod tests { assert!(!SlashCommand::Normal.takes_args()); assert!(!SlashCommand::Help.takes_args()); // Argument-taking commands. - assert!(SlashCommand::Local.takes_args()); assert!(SlashCommand::Effort.takes_args()); assert!(SlashCommand::ModelProviders.takes_args()); assert!(SlashCommand::SummarizeNew.takes_args()); @@ -369,31 +367,13 @@ mod tests { } #[test] - fn parses_local_command() { - assert_eq!(SlashCommand::parse("/local"), Some(SlashCommand::Local)); - } - - #[test] - fn parses_local_subcommands() { - assert_eq!( - SlashCommand::parse_with_args("/local add"), - Some((SlashCommand::Local, "add")) - ); - assert_eq!( - SlashCommand::parse_with_args("/local remove"), - Some((SlashCommand::Local, "remove")) - ); - assert_eq!( - SlashCommand::parse_with_args("/local edit"), - Some((SlashCommand::Local, "edit")) - ); - } - - #[test] - fn parses_local_with_empty_args() { - assert_eq!( - SlashCommand::parse_with_args("/local"), - Some((SlashCommand::Local, "")) - ); + fn parses_provider_subcommands() { + for sub in ["add", "edit", "remove"] { + assert_eq!( + SlashCommand::parse_with_args(&format!("/provider {sub}")), + Some((SlashCommand::Provider, sub)) + ); + } + assert!(SlashCommand::Provider.takes_args()); } } diff --git a/crates/alan/src/core/custom_providers.rs b/crates/alan/src/core/custom_providers.rs new file mode 100644 index 0000000..9297547 --- /dev/null +++ b/crates/alan/src/core/custom_providers.rs @@ -0,0 +1,656 @@ +//! Providers added with `/provider add`: any server that speaks Chat +//! Completions. Each is a name, a base URL and an API standard in +//! `providers.json`. Its key, if any, lives in `auth.json` under the same +//! name so `/login` can change it. + +use anyhow::Result; +use providers::{ + ApiId, Credential, CredentialStore, CustomProvider, ModelInfo, Provider, ProviderError, + ProviderId, ProviderRegistry, fetch_catalog, +}; +use serde::{Deserialize, Serialize}; +use serde_json::Value; +use std::fmt; +use std::path::PathBuf; +use std::sync::Arc; + +use crate::core::paths; +use crate::core::store::JsonStore; + +const PROVIDERS_KEY: &str = "providers"; + +/// A custom provider's name, and its id in `/models`, `auth.json`, +/// `settings.json` and `ALAN_PROVIDER`. +#[derive(Clone, Debug, Serialize, Deserialize)] +#[serde(try_from = "String")] +pub struct ProviderName(String); + +impl ProviderName { + pub fn parse(name: &str) -> Result { + let name = name.trim(); + anyhow::ensure!(!name.is_empty(), "name is required"); + anyhow::ensure!( + name.chars() + .all(|c| c.is_ascii_lowercase() || c.is_ascii_digit() || c == '-'), + "name may only contain lowercase letters, digits and -" + ); + Ok(Self(name.to_owned())) + } + + pub fn id(&self) -> ProviderId { + ProviderId::new(&self.0) + } +} + +impl TryFrom for ProviderName { + type Error = anyhow::Error; + + fn try_from(name: String) -> Result { + Self::parse(&name) + } +} + +impl fmt::Display for ProviderName { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.write_str(&self.0) + } +} + +#[derive(Clone, Debug, Serialize, Deserialize)] +pub struct CustomProviderEntry { + pub name: ProviderName, + pub url: String, + pub api: ApiId, +} + +/// The saved custom providers: entries in `providers.json`, in the order +/// they were added, and their keys in `auth.json`. +/// +/// Entries are edited as raw JSON, matched by name, so one Alan cannot read +/// (a hand edit, or an API standard from a newer Alan) is kept and does not +/// block saving the others. +pub struct CustomProviderStore { + store: JsonStore, + credentials: Arc, +} + +impl CustomProviderStore { + pub fn new(path: impl Into, credentials: Arc) -> Self { + Self { + store: JsonStore::new(path), + credentials, + } + } + + /// The entries Alan can read. + pub async fn entries(&self) -> Result> { + Ok(self.load_each().await?.into_iter().flatten().collect()) + } + + /// The saved key, to prefill the edit form. + pub async fn key(&self, name: &ProviderName) -> Result> { + Ok(self + .credentials + .read(&name.id()) + .await? + .and_then(|credential| match credential { + Credential::ApiKey { key } => Some(key), + // `/provider` only saves API keys; sign-in tokens belong to built-ins. + Credential::OAuth { .. } => None, + })) + } + + /// Save a new provider and its key, or neither. + pub async fn add(&self, entry: &CustomProviderEntry, key: Option) -> Result<()> { + self.put(entry.clone()).await?; + if let Err(error) = self.save_key(&entry.name, key).await { + if let Err(undo) = self.remove_entry(&entry.name).await { + return Err(error.context(format!( + "{} is still in providers.json: {undo:#}", + entry.name + ))); + } + return Err(error); + } + Ok(()) + } + + /// Save a changed provider. No key removes the saved one. + pub async fn update(&self, entry: &CustomProviderEntry, key: Option) -> Result<()> { + self.put(entry.clone()).await?; + self.save_key(&entry.name, key).await + } + + pub async fn remove(&self, name: &ProviderName) -> Result<()> { + self.remove_entry(name).await?; + self.credentials.delete(&name.id()).await?; + Ok(()) + } + + /// The provider to register for `entry`; it reads its key from this + /// store's `auth.json` on every request. + pub fn provider( + &self, + entry: &CustomProviderEntry, + models: Vec, + ) -> Result> { + Ok(Arc::new(CustomProvider::new( + entry.name.id(), + entry.api.clone(), + entry.url.clone(), + Arc::clone(&self.credentials), + models, + )?)) + } + + /// Each saved entry on its own, with the reason for any it cannot read. + async fn load_each(&self) -> Result>> { + let values = raw_entries(self.store.get(PROVIDERS_KEY).await?)?; + Ok(values.into_iter().map(serde_json::from_value).collect()) + } + + /// Save `entry`, replacing a saved provider with the same name. + async fn put(&self, entry: CustomProviderEntry) -> Result<()> { + let name = entry.name.to_string(); + let value = serde_json::to_value(entry)?; + self.store + .update(PROVIDERS_KEY, move |saved| { + let mut values = raw_entries(saved)?; + match values.iter_mut().find(|saved| has_name(saved, &name)) { + Some(saved) => *saved = value, + None => values.push(value), + } + Ok((Some(Value::Array(values)), ())) + }) + .await + } + + async fn remove_entry(&self, name: &ProviderName) -> Result<()> { + let name = name.to_string(); + self.store + .update(PROVIDERS_KEY, move |saved| { + let mut values = raw_entries(saved)?; + values.retain(|saved| !has_name(saved, &name)); + Ok(((!values.is_empty()).then_some(Value::Array(values)), ())) + }) + .await + } + + async fn save_key(&self, name: &ProviderName, key: Option) -> Result<()> { + match key { + Some(key) => { + self.credentials + .put(&name.id(), Credential::ApiKey { key }) + .await? + } + None => self.credentials.delete(&name.id()).await?, + } + Ok(()) + } +} + +fn raw_entries(value: Option) -> Result> { + Ok(value + .map(serde_json::from_value) + .transpose()? + .unwrap_or_default()) +} + +fn has_name(entry: &Value, name: &str) -> bool { + entry.get("name").and_then(Value::as_str) == Some(name) +} + +/// Default location of providers.json inside the Alan data directory. +pub fn default_providers_path() -> Result { + paths::data_file("providers.json") +} + +/// Register every saved provider Alan can use; their models are fetched the +/// next time `/models` opens. Entries it cannot read or bind are skipped. +/// Call it once the built-ins are registered: returns a notice for each entry +/// removed because a built-in now takes its name. +pub async fn load(store: &CustomProviderStore, registry: &ProviderRegistry) -> Result> { + let mut notices = Vec::new(); + for entry in store.load_each().await? { + let Ok(entry) = entry.inspect_err(|error| { + tracing::warn!(%error, "skipping an unreadable entry in providers.json"); + }) else { + continue; + }; + // The name, and its key slot in `auth.json`, now belong to the + // built-in, so only the entry goes. + if registry.get(&entry.name.id()).is_some() { + store.remove_entry(&entry.name).await?; + notices.push(format!( + "custom provider {} ({}) was removed: a built-in provider now uses that name; add it again under another name", + entry.name, entry.url + )); + continue; + } + match store.provider(&entry, Vec::new()) { + Ok(provider) => registry.insert(provider), + Err(error) => { + tracing::warn!(%error, provider = %entry.name, "skipping a custom provider") + } + } + } + Ok(notices) +} + +/// Refuse a name `/provider add` cannot take, before any request. +pub async fn ensure_new_name( + store: &CustomProviderStore, + registry: &ProviderRegistry, + name: &ProviderName, +) -> Result<()> { + let saved = store.entries().await?; + anyhow::ensure!( + saved.iter().all(|entry| entry.name.id() != name.id()), + "{name} already exists; change it with /provider edit" + ); + // A built-in's key in `auth.json` sits under the same name. + anyhow::ensure!( + registry.get(&name.id()).is_none(), + "{name} is a built-in provider; pick another name, e.g. {name}-2" + ); + Ok(()) +} + +/// Check `url` with a `/models` call. Returns the entry to save and the +/// models the server lists. +pub async fn check( + name: ProviderName, + api: ApiId, + url: &str, + key: Option<&str>, +) -> Result<(CustomProviderEntry, Vec)> { + let url = url.trim().trim_end_matches('/').to_owned(); + // The reason already names the URL; the error's own prefix would only + // repeat that the check failed. + let models = fetch_catalog(&name.id(), &api, &url, key) + .await + .map_err(|error| match error { + ProviderError::Fetch(reason) => anyhow::anyhow!(reason), + other => other.into(), + })?; + Ok((CustomProviderEntry { name, url, api }, models)) +} + +#[cfg(test)] +pub(crate) mod tests { + use super::*; + use crate::core::store::JsonStore; + use providers::{CredentialError, CredentialInfo, InMemoryCredentialStore}; + use serde_json::json; + use std::env::temp_dir; + + const OK_TWO_MODELS: &str = + "HTTP/1.1 200 OK\r\nconnection: close\r\n\r\n{\"data\":[{\"id\":\"m1\"},{\"id\":\"m2\"}]}"; + + fn temp_path(name: &str) -> PathBuf { + let path = temp_dir().join(format!( + "alan-custom-providers-test-{}-{name}.json", + std::process::id() + )); + let _ = std::fs::remove_file(&path); + path + } + + fn cleanup(name: &str) { + let path = temp_path(name); + let _ = std::fs::remove_file(format!("{}.lock", path.display())); + } + + fn name(value: &str) -> ProviderName { + ProviderName::parse(value).unwrap() + } + + fn entry(value: &str, url: &str) -> CustomProviderEntry { + CustomProviderEntry { + name: name(value), + url: url.into(), + api: ApiId::ChatCompletions, + } + } + + pub(crate) fn provider(id: &str, models: &[&str]) -> Arc { + let models = models + .iter() + .map(|model| ModelInfo::new(model, ProviderId::new(id))) + .collect(); + Arc::new( + CustomProvider::new( + ProviderId::new(id), + ApiId::ChatCompletions, + "http://localhost:1/v1", + Arc::new(InMemoryCredentialStore::new()), + models, + ) + .unwrap(), + ) + } + + fn store(file: &str) -> (CustomProviderStore, Arc) { + let credentials = Arc::new(InMemoryCredentialStore::new()); + let store = CustomProviderStore::new(temp_path(file), credentials.clone()); + (store, credentials) + } + + async fn names(store: &CustomProviderStore) -> Vec { + store + .entries() + .await + .unwrap() + .into_iter() + .map(|e| e.name.to_string()) + .collect() + } + + // Replies `response` to one connection, hands back the request. + async fn serve(response: &'static str) -> (String, tokio::task::JoinHandle) { + use tokio::io::{AsyncReadExt, AsyncWriteExt}; + let listener = tokio::net::TcpListener::bind(("127.0.0.1", 0)) + .await + .unwrap(); + let address = listener.local_addr().unwrap(); + let server = tokio::spawn(async move { + let (mut socket, _) = listener.accept().await.unwrap(); + let mut request = Vec::new(); + let mut buffer = [0; 4096]; + while !request.windows(4).any(|window| window == b"\r\n\r\n") { + let read = socket.read(&mut buffer).await.unwrap(); + assert!(read > 0); + request.extend_from_slice(&buffer[..read]); + } + socket.write_all(response.as_bytes()).await.unwrap(); + String::from_utf8(request).unwrap() + }); + (format!("http://{address}/v1/"), server) + } + + #[test] + fn provider_name_accepts_lowercase_digits_and_dashes() { + assert_eq!(name(" my-groq2 ").to_string(), "my-groq2"); + } + + #[test] + fn provider_name_rejects_bad_characters() { + for bad in ["", "Grok", "my groq", "groq!"] { + assert!(ProviderName::parse(bad).is_err(), "{bad:?} was accepted"); + } + } + + #[tokio::test] + async fn ensure_new_name_refuses_saved_and_builtin_names() { + let (store, _credentials) = store("names"); + store.add(&entry("grok", "http://x"), None).await.unwrap(); + let registry = ProviderRegistry::new([provider("deepseek", &[])]); + + let saved = ensure_new_name(&store, ®istry, &name("grok")) + .await + .unwrap_err(); + let builtin = ensure_new_name(&store, ®istry, &name("deepseek")) + .await + .unwrap_err(); + + assert_eq!( + saved.to_string(), + "grok already exists; change it with /provider edit" + ); + assert_eq!( + builtin.to_string(), + "deepseek is a built-in provider; pick another name, e.g. deepseek-2" + ); + assert!( + ensure_new_name(&store, ®istry, &name("groq")) + .await + .is_ok() + ); + cleanup("names"); + } + + #[tokio::test] + async fn check_trims_the_url_and_lists_the_models() { + let url = serve(OK_TWO_MODELS).await.0; + + let (entry, models) = check(name("grok"), ApiId::ChatCompletions, &url, None) + .await + .unwrap(); + + assert_eq!(entry.url, url.trim_end_matches('/')); + assert_eq!(models.len(), 2); + } + + #[tokio::test] + async fn check_reports_only_the_reason() { + let url = serve("HTTP/1.1 401 Unauthorized\r\nconnection: close\r\n\r\n{}") + .await + .0; + + let error = check(name("grok"), ApiId::ChatCompletions, &url, Some("bad")) + .await + .unwrap_err(); + + assert_eq!( + format!("{error:#}"), + "the API key was rejected (HTTP 401 Unauthorized)" + ); + } + + #[tokio::test] + async fn put_replaces_by_name_and_keeps_order() { + let (store, _credentials) = store("put"); + for (n, url) in [ + ("a", "http://one"), + ("b", "http://two"), + ("a", "http://three"), + ] { + store.put(entry(n, url)).await.unwrap(); + } + let urls: Vec<_> = store + .entries() + .await + .unwrap() + .into_iter() + .map(|e| e.url) + .collect(); + assert_eq!(urls, ["http://three", "http://two"]); + cleanup("put"); + } + + #[tokio::test] + async fn store_keeps_entries_it_cannot_read_and_still_saves_others() { + let path = temp_path("unreadable"); + JsonStore::new(path.clone()) + .set( + PROVIDERS_KEY, + json!([{"name": "Bad Name", "url": "http://x", "api": "ChatCompletions"}]), + ) + .await + .unwrap(); + let store = + CustomProviderStore::new(path.clone(), Arc::new(InMemoryCredentialStore::new())); + + store.add(&entry("grok", "http://one"), None).await.unwrap(); + store.remove(&name("grok")).await.unwrap(); + store.add(&entry("groq", "http://two"), None).await.unwrap(); + + assert_eq!(names(&store).await, ["groq"]); + let raw = JsonStore::new(path) + .get(PROVIDERS_KEY) + .await + .unwrap() + .unwrap(); + assert_eq!(raw.as_array().unwrap().len(), 2, "{raw}"); + cleanup("unreadable"); + } + + #[tokio::test] + async fn add_saves_the_entry_and_key_and_remove_deletes_both() { + let (store, _credentials) = store("add-remove"); + + store + .add(&entry("grok", "http://x/v1"), Some("sk".into())) + .await + .unwrap(); + assert_eq!(names(&store).await, ["grok"]); + assert_eq!( + store.key(&name("grok")).await.unwrap().as_deref(), + Some("sk") + ); + + store.remove(&name("grok")).await.unwrap(); + assert!(names(&store).await.is_empty()); + assert!(store.key(&name("grok")).await.unwrap().is_none()); + cleanup("add-remove"); + } + + #[tokio::test] + async fn add_leaves_no_key_when_providers_json_cannot_be_written() { + // A directory where the file should be: every read and write fails. + let path = temp_path("unwritable"); + std::fs::create_dir_all(&path).unwrap(); + let credentials = Arc::new(InMemoryCredentialStore::new()); + let store = CustomProviderStore::new(path.clone(), credentials.clone()); + + let result = store + .add(&entry("grok", "http://x"), Some("sk".into())) + .await; + + assert!(result.is_err()); + assert!( + credentials + .read(&name("grok").id()) + .await + .unwrap() + .is_none() + ); + let _ = std::fs::remove_dir_all(&path); + cleanup("unwritable"); + } + + #[tokio::test] + async fn add_takes_the_entry_back_when_the_key_cannot_be_saved() { + let store = CustomProviderStore::new(temp_path("key-fails"), Arc::new(FailingKeys)); + + let result = store + .add(&entry("grok", "http://x"), Some("sk".into())) + .await; + + assert!(result.is_err()); + assert!(names(&store).await.is_empty()); + cleanup("key-fails"); + } + + /// A key store whose writes always fail. + struct FailingKeys; + + #[async_trait::async_trait] + impl CredentialStore for FailingKeys { + async fn read(&self, _: &ProviderId) -> Result, CredentialError> { + Ok(None) + } + + async fn list(&self) -> Result, CredentialError> { + Ok(Vec::new()) + } + + async fn put(&self, _: &ProviderId, _: Credential) -> Result<(), CredentialError> { + Err(CredentialError::LockBusy) + } + + async fn delete(&self, _: &ProviderId) -> Result<(), CredentialError> { + Ok(()) + } + } + + #[tokio::test] + async fn update_without_a_key_deletes_the_saved_one() { + let (store, _credentials) = store("update"); + store + .add(&entry("grok", "http://one"), Some("old".into())) + .await + .unwrap(); + + store + .update(&entry("grok", "http://two"), None) + .await + .unwrap(); + + assert!(store.key(&name("grok")).await.unwrap().is_none()); + assert_eq!(store.entries().await.unwrap()[0].url, "http://two"); + cleanup("update"); + } + + #[tokio::test] + async fn load_skips_entries_it_cannot_use_and_keeps_the_rest() { + let path = temp_path("load-skips"); + JsonStore::new(path.clone()) + .set( + PROVIDERS_KEY, + json!([ + {"name": "grok", "url": "http://localhost:1/v1", "api": "ChatCompletions"}, + {"name": "Bad Name", "url": "http://x", "api": "ChatCompletions"}, + {"name": "newer", "url": "http://x", "api": "SomeFutureStandard"}, + {"name": "claude", "url": "http://x", "api": "AnthropicMessages"} + ]), + ) + .await + .unwrap(); + let registry = ProviderRegistry::new([]); + let store = CustomProviderStore::new(path, Arc::new(InMemoryCredentialStore::new())); + + let notices = load(&store, ®istry).await.unwrap(); + + assert!(notices.is_empty(), "{notices:?}"); + let ids: Vec<_> = registry.providers().iter().map(|p| p.id().0).collect(); + assert_eq!(ids, ["grok"]); + // Models come with the first `/models`, not at startup. + assert!( + registry + .get(&name("grok").id()) + .unwrap() + .models() + .is_empty() + ); + cleanup("load-skips"); + } + + #[tokio::test] + async fn load_drops_an_entry_a_builtin_now_names_and_keeps_the_builtins_key() { + let (store, credentials) = store("load-clash"); + for n in ["openai", "grok"] { + store.put(entry(n, "http://localhost:1/v1")).await.unwrap(); + } + credentials + .put( + &name("openai").id(), + Credential::ApiKey { key: "sk".into() }, + ) + .await + .unwrap(); + let registry = ProviderRegistry::new([provider("openai", &["gpt"])]); + + let notices = load(&store, ®istry).await.unwrap(); + + assert_eq!( + notices, + [ + "custom provider openai (http://localhost:1/v1) was removed: a built-in provider now uses that name; add it again under another name" + ] + ); + assert_eq!(names(&store).await, ["grok"]); + // The built-in keeps its place and its key. + let openai = registry.get(&name("openai").id()).unwrap(); + assert_eq!(openai.models()[0].id, "gpt"); + assert!(registry.get(&name("grok").id()).is_some()); + assert!( + credentials + .read(&name("openai").id()) + .await + .unwrap() + .is_some() + ); + cleanup("load-clash"); + } +} diff --git a/crates/alan/src/core/mod.rs b/crates/alan/src/core/mod.rs index 96a2f62..3d17efe 100644 --- a/crates/alan/src/core/mod.rs +++ b/crates/alan/src/core/mod.rs @@ -4,6 +4,7 @@ pub mod action; pub mod chat; pub mod command; pub mod completion; +pub mod custom_providers; pub mod paths; pub mod permissions; pub mod permissions_store; diff --git a/crates/alan/src/local_model_overlay.rs b/crates/alan/src/local_model_overlay.rs deleted file mode 100644 index e081a16..0000000 --- a/crates/alan/src/local_model_overlay.rs +++ /dev/null @@ -1,675 +0,0 @@ -use crate::root::AlanAction; -use crate::views::theme; -use crate::views::{SearchListEvent, SearchListOverlay}; -use alan_tui::context::Context; -use alan_tui::{ActionStatus, Component, RenderContext, TaskError}; -use crossterm::event::{Event, KeyCode, KeyEventKind, KeyModifiers}; -use providers::{LocalApi, LocalModelEntry, LocalProvider, list_local_models}; -use ratatui::Frame; -use ratatui::layout::Alignment; -use ratatui::layout::Rect; -use ratatui::style::{Color, Modifier, Style}; -use ratatui::text::{Line, Span}; -use ratatui::widgets::{Block, BorderType, Clear, Paragraph}; -use std::sync::Arc; - -const FIELD_COUNT: usize = 4; -// model last so url + key are known when we auto-detect it. -const FIELD_LABELS: [&str; FIELD_COUNT] = ["URL", "API", "API Key", "Model ID"]; -const URL: usize = 0; -const API: usize = 1; -const API_KEY: usize = 2; -const MODEL: usize = 3; - -#[derive(Debug, Clone)] -struct FormFields { - model_id: String, - url: String, - api_key: String, -} - -impl FormFields { - fn field_mut(&mut self, index: usize) -> Option<&mut String> { - match index { - URL => Some(&mut self.url), - API_KEY => Some(&mut self.api_key), - MODEL => Some(&mut self.model_id), - _ => None, - } - } - - fn field(&self, index: usize) -> &str { - match index { - URL => &self.url, - API_KEY => &self.api_key, - MODEL => &self.model_id, - _ => "", - } - } - - fn label(index: usize) -> &'static str { - FIELD_LABELS[index] - } - - fn to_entry(&self) -> Option { - let model_id = self.model_id.trim().to_owned(); - let url = self.url.trim().to_owned(); - if model_id.is_empty() || url.is_empty() { - return None; - } - let api_key = self.api_key.trim().to_owned(); - Some(LocalModelEntry { - model_id, - url, - api: LocalApi::ChatCompletions, - api_key: if api_key.is_empty() { - None - } else { - Some(api_key) - }, - }) - } -} - -#[derive(Debug, Clone, PartialEq, Eq)] -enum OverlayState { - Editing, - Saving, - Success, - Error(String), -} - -/// `GET /models` result for `detect_source`. -#[derive(Debug, Clone, PartialEq, Eq)] -enum Detect { - Idle, - Loading, - Found(Vec), - Failed(String), -} - -pub struct LocalModelOverlay { - provider: Arc, - fields: FormFields, - focused: usize, - state: OverlayState, - edit_mode: bool, - detect: Detect, - // (url, key) the current `detect` belongs to. - detect_source: (String, String), -} - -impl LocalModelOverlay { - pub fn new(provider: Arc, edit_entry: Option) -> Self { - let (fields, edit_mode) = match edit_entry { - Some(entry) => ( - FormFields { - model_id: entry.model_id, - url: entry.url, - api_key: entry.api_key.unwrap_or_default(), - }, - true, - ), - None => ( - FormFields { - model_id: String::new(), - url: String::new(), - api_key: String::new(), - }, - false, - ), - }; - Self { - provider, - fields, - focused: 0, - state: OverlayState::Editing, - edit_mode, - detect: Detect::Idle, - detect_source: (String::new(), String::new()), - } - } - - fn source(&self) -> (String, String) { - ( - self.fields.url.trim().to_owned(), - self.fields.api_key.trim().to_owned(), - ) - } - - /// Fetch the model list when focus lands on Model ID. Reuses a result for - /// the same url/key; a failed one retries. - fn maybe_detect(&mut self, cx: &mut Context<'_, Self, AlanAction>) { - if self.edit_mode || self.focused != MODEL { - return; - } - let source = self.source(); - if source.0.is_empty() - || (source == self.detect_source - && matches!(self.detect, Detect::Loading | Detect::Found(_))) - { - return; - } - self.detect_source = source.clone(); - self.detect = Detect::Loading; - let requested = source.clone(); - cx.spawn( - async move { - let (url, key) = source; - list_local_models(&url, Some(&key)) - .await - .map_err(|e| match e { - // drop the "failed to fetch models:" prefix, hint says it. - providers::ProviderError::Fetch(reason) => TaskError(reason.into()), - e => TaskError(e.to_string().into()), - }) - }, - move |result, overlay, cx| { - // url/key changed mid-flight. - if overlay.detect_source != requested { - return; - } - match result { - Ok(ids) => { - overlay.detect = Detect::Found(ids); - // don't yank focus if they already typed or moved on. - if overlay.focused == MODEL - && overlay.state == OverlayState::Editing - && overlay.fields.model_id.trim().is_empty() - { - overlay.open_picker(cx); - } - } - Err(e) => overlay.detect = Detect::Failed(e.to_string()), - } - cx.notify(); - }, - ); - } - - fn open_picker(&mut self, cx: &mut Context<'_, Self, AlanAction>) { - let Detect::Found(ids) = &self.detect else { - return; - }; - let ids = ids.clone(); - let picker = cx.open_overlay(SearchListOverlay::new("Pick Model", ids.clone())); - cx.subscribe_once::( - picker, - move |event, overlay, _picker, cx| { - if let SearchListEvent::Chosen(index) = event - && let Some(id) = ids.get(*index) - { - overlay.fields.model_id = id.clone(); - cx.notify(); - } - }, - ); - } - - fn on_enter(&mut self, cx: &mut Context<'_, Self, AlanAction>) { - if !self.fields.model_id.trim().is_empty() || self.fields.url.trim().is_empty() { - self.submit(cx); - return; - } - // url set, model missing: detect/pick instead of erroring out the form. - if self.focused != MODEL { - self.focused = MODEL; - self.maybe_detect(cx); - cx.notify(); - } else if matches!(self.detect, Detect::Found(_)) { - self.open_picker(cx); - } - } - - fn move_focus(&mut self, delta: isize, cx: &mut Context<'_, Self, AlanAction>) { - self.focused = ((self.focused as isize + delta).rem_euclid(FIELD_COUNT as isize)) as usize; - self.maybe_detect(cx); - cx.notify(); - } - - fn submit(&mut self, cx: &mut Context<'_, Self, AlanAction>) { - let Some(entry) = self.fields.to_entry() else { - self.state = OverlayState::Error("Model ID and URL are required".into()); - cx.notify(); - return; - }; - - let provider = Arc::clone(&self.provider); - let edit_mode = self.edit_mode; - - cx.spawn( - async move { - if edit_mode { - provider - .update_model(entry) - .await - .map_err(|e| alan_tui::TaskError(e.to_string().into()))?; - } else { - provider - .add_model(entry) - .await - .map_err(|e| alan_tui::TaskError(e.to_string().into()))?; - } - Ok::<(), alan_tui::TaskError>(()) - }, - move |result, overlay, cx| { - match result { - Ok(()) => { - overlay.state = OverlayState::Success; - } - Err(e) => { - overlay.state = OverlayState::Error(e.to_string()); - } - } - cx.notify(); - }, - ); - - self.state = OverlayState::Saving; - cx.notify(); - } - - fn dismiss(&self, cx: &mut Context<'_, Self, AlanAction>) { - cx.close_overlay(); - } -} - -impl Component for LocalModelOverlay { - fn handle_action( - &mut self, - action: &AlanAction, - cx: &mut Context<'_, Self, AlanAction>, - ) -> ActionStatus { - match action { - AlanAction::Paste(text) => { - if let OverlayState::Editing = &self.state - && let Some(field) = self.fields.field_mut(self.focused) - { - field.push_str(text); - cx.notify(); - } - ActionStatus::Handled - } - AlanAction::Raw(event) => { - let Event::Key(key) = event else { - return ActionStatus::Handled; - }; - if key.kind != KeyEventKind::Press { - return ActionStatus::Handled; - } - match &self.state { - OverlayState::Editing => match key.code { - KeyCode::Esc => { - self.dismiss(cx); - } - KeyCode::Tab => self.move_focus(1, cx), - KeyCode::BackTab => self.move_focus(-1, cx), - KeyCode::Down => self.move_focus(1, cx), - KeyCode::Up => self.move_focus(-1, cx), - // re-pick after a wrong choice. - KeyCode::Left | KeyCode::Right - if self.focused == MODEL && matches!(self.detect, Detect::Found(_)) => - { - self.open_picker(cx) - } - KeyCode::Enter => self.on_enter(cx), - KeyCode::Backspace => { - if let Some(field) = self.fields.field_mut(self.focused) { - field.pop(); - cx.notify(); - } - } - KeyCode::Char(c) - if key.modifiers.is_empty() || key.modifiers == KeyModifiers::SHIFT => - { - if let Some(field) = self.fields.field_mut(self.focused) { - field.push(c); - cx.notify(); - } - } - KeyCode::Char('c') if key.modifiers.contains(KeyModifiers::CONTROL) => { - self.dismiss(cx); - } - _ => {} - }, - OverlayState::Saving => {} - OverlayState::Success | OverlayState::Error(_) => { - if matches!(key.code, KeyCode::Enter | KeyCode::Esc) { - self.dismiss(cx); - } - } - } - ActionStatus::Handled - } - _ => ActionStatus::Continue, - } - } - - fn render(&self, frame: &mut Frame, area: Rect, _cx: &RenderContext<'_, '_, AlanAction>) { - render_local_model_overlay( - frame, - area, - &self.fields, - self.focused, - &self.state, - self.edit_mode, - &self.detect, - ); - } -} - -fn render_local_model_overlay( - frame: &mut Frame, - area: Rect, - fields: &FormFields, - focused: usize, - state: &OverlayState, - edit_mode: bool, - detect: &Detect, -) { - let width = area.width.saturating_sub(4).min(88); - let height = area.height.saturating_sub(2).min(18); - if width < 12 || height < 6 { - return; - } - let popup = Rect::new( - area.x + (area.width - width) / 2, - area.y + (area.height - height) / 2, - width, - height, - ); - frame.render_widget(Clear, popup); - let accent = Style::default().fg(theme::PROMPT_FG); - let muted = Style::default().fg(theme::TOOL_FG); - - let title_label = if edit_mode { "Edit" } else { "Add" }; - let (heading, footer) = match state { - OverlayState::Editing => ( - format!("{} Local Model", title_label), - if width >= 60 { - " Tab move Enter save Esc cancel " - } else { - " Tab · Enter · Esc " - }, - ), - OverlayState::Saving => ("Saving…".to_owned(), ""), - OverlayState::Success => ("Saved".to_owned(), " Enter / Esc close "), - OverlayState::Error(_) => ("Error".to_owned(), " Enter / Esc close "), - }; - - let title = vec![Span::styled( - format!(" {} ", heading), - accent.add_modifier(Modifier::BOLD), - )]; - let block = Block::bordered() - .border_type(BorderType::Rounded) - .border_style(accent) - .style(Style::default().bg(theme::EDITOR_BG).fg(theme::USER_FG)) - .title(Line::from(title).style(accent)) - .title_alignment(Alignment::Center) - .title_bottom(Line::from(footer).style(muted)); - - let inner = block.inner(popup); - frame.render_widget(block, popup); - - match state { - OverlayState::Editing => { - let max_label = (0..FIELD_COUNT) - .map(|i| FormFields::label(i).len()) - .max() - .unwrap_or(0); - for i in 0..FIELD_COUNT { - let y = inner.y + i as u16 * 2; - if y + 2 > inner.bottom() { - break; - } - let is_focused = i == focused; - let label_style = if is_focused { - accent.add_modifier(Modifier::BOLD) - } else { - muted - }; - let line_style = if is_focused { - Style::default().bg(theme::SELECTION_BG) - } else { - Style::default() - }; - - // Value (API is a fixed read-only label so the user - // knows local models use the chat-completions API). - let value = if i == API { - "Chat Completions" - } else { - fields.field(i) - }; - let value_style = if is_focused { - Style::default() - .fg(theme::SELECTION_FG) - .bg(theme::SELECTION_BG) - } else { - Style::default().fg(theme::EDITOR_FG) - }; - let display = if value.is_empty() && i == API_KEY { - Span::styled("optional", muted) - } else if value.is_empty() && i != API { - Span::styled("required", muted) - } else { - Span::styled(value, value_style) - }; - - let mut spans = vec![ - Span::styled(if is_focused { " › " } else { " " }, accent), - Span::styled( - format!("{:>width$} ", FormFields::label(i), width = max_label), - label_style, - ), - Span::styled(": ", muted), - ]; - // detected list available: ◀ ▶ reopen the picker. - if i == MODEL && matches!(detect, Detect::Found(_)) { - spans.extend([ - Span::styled("◀ ", accent), - display, - Span::styled(" ▶", accent), - ]); - } else { - spans.push(display); - } - let line = Line::from(spans); - frame.render_widget( - Paragraph::new(line).style(line_style), - Rect::new(inner.x, y, inner.width, 1), - ); - let hint = if i == MODEL { - detect_hint(detect, fields.model_id.trim().is_empty()) - } else { - String::new() - }; - frame.render_widget( - Paragraph::new(Span::styled(hint, muted)), - Rect::new(inner.x, y + 1, inner.width, 1), - ); - } - } - OverlayState::Saving => { - frame.render_widget(Paragraph::new(" Saving…").style(accent), inner); - } - OverlayState::Success => { - let msg = if edit_mode { - "Model updated." - } else { - "Model added." - }; - frame.render_widget(Paragraph::new(format!(" {msg}")).style(accent), inner); - } - OverlayState::Error(err) => { - frame.render_widget( - Paragraph::new(format!(" {err}")).style(Style::default().fg(Color::Red)), - inner, - ); - } - } -} - -fn detect_hint(detect: &Detect, model_empty: bool) -> String { - let indent = " ".repeat(5 + FIELD_LABELS.iter().map(|l| l.len()).max().unwrap_or(0) + 2); - match detect { - Detect::Idle => String::new(), - Detect::Loading => format!("{indent}detecting models…"), - Detect::Found(ids) if model_empty => { - format!("{indent}{} found, Enter or ◀ ▶ to pick", ids.len()) - } - Detect::Found(_) => String::new(), - Detect::Failed(reason) => { - format!("{indent}auto-detect failed ({reason}), type the model ID") - } - } -} - -#[cfg(test)] -mod tests { - use super::*; - use ratatui::{Terminal, backend::TestBackend}; - - fn render_grid(width: u16, height: u16, detect: &Detect) -> Vec { - let fields = FormFields { - model_id: "gpt-4o-mini".to_owned(), - url: "http://localhost:11434".to_owned(), - api_key: "sk-abc".to_owned(), - }; - let mut terminal = Terminal::new(TestBackend::new(width, height)).unwrap(); - terminal - .draw(|frame| { - render_local_model_overlay( - frame, - frame.area(), - &fields, - 0, - &OverlayState::Editing, - false, - detect, - ); - }) - .unwrap(); - let buffer = terminal.backend().buffer(); - buffer - .content - .iter() - .map(|cell| cell.symbol()) - .collect::() - .chars() - .collect::>() - .chunks(width as usize) - .map(|row| row.iter().collect::()) - .collect() - } - - #[test] - fn form_field_labels_align_to_a_fixed_column() { - let rows = render_grid(80, 18, &Detect::Idle); - let mut colon_cols = Vec::new(); - for row in &rows { - if let Some(col) = row.chars().position(|character| character == ':') { - colon_cols.push(col); - } - } - assert_eq!( - colon_cols.len(), - FIELD_COUNT, - "expected one colon per field, got {colon_cols:?}" - ); - let first = colon_cols[0]; - assert!( - colon_cols.iter().all(|&c| c == first), - "label colon columns are not aligned: {colon_cols:?}" - ); - } - - #[test] - fn model_field_comes_last() { - let rows = render_grid(80, 18, &Detect::Idle); - let labels: Vec<&str> = rows - .iter() - .filter_map(|row| row.split(':').next()) - .map(str::trim) - .filter(|label| FIELD_LABELS.iter().any(|l| label.ends_with(l))) - .collect(); - assert_eq!(labels.len(), FIELD_COUNT); - assert!(labels[MODEL].ends_with("Model ID"), "{labels:?}"); - } - - #[test] - fn model_arrows_only_when_detected() { - let model_row = |detect: &Detect| { - render_grid(80, 18, detect) - .into_iter() - .find(|row| row.contains("Model ID")) - .unwrap() - }; - assert!(!model_row(&Detect::Idle).contains('◀')); - assert!(!model_row(&Detect::Failed("x".into())).contains('◀')); - assert!(model_row(&Detect::Found(vec!["gpt-4o-mini".into()])).contains("◀ gpt-4o-mini ▶")); - } - - #[test] - fn detect_hint_per_state() { - assert_eq!(detect_hint(&Detect::Idle, true), ""); - assert!(detect_hint(&Detect::Loading, true).ends_with("detecting models…")); - let found = Detect::Found(vec!["a".into(), "b".into()]); - assert!(detect_hint(&found, true).ends_with("2 found, Enter or ◀ ▶ to pick")); - // already typed: nothing to nag about. - assert_eq!(detect_hint(&found, false), ""); - let failed = Detect::Failed("HTTP 401 Unauthorized".into()); - assert!( - detect_hint(&failed, false) - .ends_with("auto-detect failed (HTTP 401 Unauthorized), type the model ID") - ); - } - - #[test] - fn to_entry_rejects_empty_model_id() { - let fields = FormFields { - model_id: String::new(), - url: "http://localhost:11434".to_owned(), - api_key: String::new(), - }; - assert!(fields.to_entry().is_none()); - } - - #[test] - fn to_entry_rejects_empty_url() { - let fields = FormFields { - model_id: "llama3".to_owned(), - url: String::new(), - api_key: String::new(), - }; - assert!(fields.to_entry().is_none()); - } - - #[test] - fn to_entry_accepts_valid_fields() { - let fields = FormFields { - model_id: "llama3".to_owned(), - url: "http://localhost:11434".to_owned(), - api_key: "sk-abc".to_owned(), - }; - let entry = fields.to_entry().unwrap(); - assert_eq!(entry.model_id, "llama3"); - assert_eq!(entry.url, "http://localhost:11434"); - assert_eq!(entry.api, LocalApi::ChatCompletions); - assert_eq!(entry.api_key, Some("sk-abc".to_owned())); - } - - #[test] - fn to_entry_strips_whitespace() { - let fields = FormFields { - model_id: " llama3 ".to_owned(), - url: " http://localhost:11434 ".to_owned(), - api_key: String::new(), - }; - let entry = fields.to_entry().unwrap(); - assert_eq!(entry.model_id, "llama3"); - assert_eq!(entry.url, "http://localhost:11434"); - assert_eq!(entry.api_key, None); - } -} diff --git a/crates/alan/src/local_model_store.rs b/crates/alan/src/local_model_store.rs deleted file mode 100644 index 08868c8..0000000 --- a/crates/alan/src/local_model_store.rs +++ /dev/null @@ -1,101 +0,0 @@ -use crate::core::store::JsonStore; -use anyhow::Result; -use providers::{LocalModelEntry, LocalModelStore, ProviderError}; -use std::path::PathBuf; - -/// `LocalModelStore` backed by `JsonStore` with the same -/// `.lock` sidecar protection that `JsonStore` already provides. -pub struct JsonLocalModelStore { - store: JsonStore, -} - -impl JsonLocalModelStore { - pub fn new(path: PathBuf) -> Self { - Self { - store: JsonStore::new(path), - } - } -} - -#[async_trait::async_trait] -impl LocalModelStore for JsonLocalModelStore { - async fn load(&self) -> Result, ProviderError> { - let value = self - .store - .get("local_models") - .await - .map_err(|e| ProviderError::Fetch(e.to_string()))?; - match value { - Some(serde_json::Value::Array(arr)) => { - serde_json::from_value::>(serde_json::Value::Array(arr)) - .map_err(|e| ProviderError::Fetch(e.to_string())) - } - Some(_) | None => Ok(Vec::new()), - } - } - - async fn save(&self, entries: &[LocalModelEntry]) -> Result<(), ProviderError> { - let value = - serde_json::to_value(entries).map_err(|e| ProviderError::Fetch(e.to_string()))?; - self.store - .set("local_models", value) - .await - .map_err(|e| ProviderError::Fetch(e.to_string())) - } -} - -#[cfg(test)] -mod tests { - use super::*; - use crate::core::store::JsonStore; - use providers::LocalApi; - use serde_json::json; - use std::env::temp_dir; - - fn temp_path(name: &str) -> PathBuf { - let mut path = temp_dir(); - path.push(format!( - "alan-local-store-test-{}-{name}.json", - std::process::id() - )); - let _ = std::fs::remove_file(&path); - path - } - - #[tokio::test] - async fn load_missing_file_returns_empty() { - let store = JsonLocalModelStore::new(temp_path("missing")); - assert!(store.load().await.unwrap().is_empty()); - } - - #[tokio::test] - async fn save_then_load_roundtrips() { - let store = JsonLocalModelStore::new(temp_path("roundtrip")); - let entries = vec![LocalModelEntry { - model_id: "llama3".into(), - url: "http://localhost:11434".into(), - api: LocalApi::ChatCompletions, - api_key: None, - }]; - store.save(&entries).await.unwrap(); - assert_eq!(store.load().await.unwrap(), entries); - } - - #[tokio::test] - async fn load_rejects_malformed_entries() { - let path = temp_path("corrupt"); - // A valid JSON array containing one well-formed and one malformed entry. - JsonStore::new(path.clone()) - .set( - "local_models", - json!([ - {"model_id": "ok", "url": "http://x", "api": "ChatCompletions", "api_key": null}, - {"model_id": "bad"} - ]), - ) - .await - .unwrap(); - let store = JsonLocalModelStore::new(path); - assert!(store.load().await.is_err()); - } -} diff --git a/crates/alan/src/main.rs b/crates/alan/src/main.rs index a5f9732..471f271 100644 --- a/crates/alan/src/main.rs +++ b/crates/alan/src/main.rs @@ -1,22 +1,23 @@ mod core; mod keymap; -mod local_model_overlay; -mod local_model_store; mod logging; mod login_overlay; +mod provider_overlay; mod root; mod views; +use crate::core::custom_providers::{self, CustomProviderStore}; use crate::core::permissions::AlanPermissionManager; use crate::core::permissions::ToolPolicy; use crate::core::permissions_store; use crate::core::server_tools::server_tools; use crate::core::settings::patch_from_profile; -use crate::core::settings::{DEFAULT_MODEL, PatchSettings, Settings, SettingsStore}; +use crate::core::settings::{ + DEFAULT_MODEL, DEFAULT_PROVIDER, PatchSettings, Settings, SettingsStore, +}; use crate::core::update::current_version; use crate::core::{ChatController, SlashCommand}; use crate::keymap::AlanKeyMapper; -use crate::local_model_store::JsonLocalModelStore; use crate::logging::init; use crate::root::AlanRoot; use agent::{ @@ -25,8 +26,8 @@ use agent::{ use alan_tui::Runtime; use llm::ReasoningEffort; use providers::{ - ChatGptProvider, DeepSeekProvider, FileCredentialStore, LocalProvider, ModelOptions, - OpenRouterProvider, Provider, ProviderRegistry, ZaiProvider, bind_local_model, bind_model, + ChatGptProvider, DeepSeekProvider, FileCredentialStore, ModelOptions, OpenRouterProvider, + Provider, ProviderId, ProviderRegistry, ZaiProvider, bind_model, }; use std::path::PathBuf; use std::sync::Arc; @@ -69,20 +70,22 @@ async fn main() -> anyhow::Result<()> { let credential_store = Arc::new(FileCredentialStore::new(auth_path()?)); let credential_store_for_providers = credential_store.clone() as Arc; - let local_provider = load_local_provider().await?; - let providers = build_providers(&credential_store_for_providers, &local_provider)?; + let providers = build_providers(&credential_store_for_providers)?; + let registry = Arc::new(ProviderRegistry::new(providers)); + + let custom_provider_store = CustomProviderStore::new( + custom_providers::default_providers_path()?, + Arc::clone(&credential_store_for_providers), + ); + let startup_notices = custom_providers::load(&custom_provider_store, ®istry) + .await + .unwrap_or_else(|error| { + tracing::warn!(%error, "failed to load custom providers"); + Vec::new() + }); let session_manager = Arc::new(SessionManager::new(sessions_path()?)); let resumed_session = load_resumed_session(&session_manager).await?; - let model = select_model( - &providers, - &local_provider, - &settings, - resumed_session.as_ref(), - )?; - let registry = Arc::new(ProviderRegistry::with_local_provider( - providers, - Arc::clone(&local_provider), - )); + let model = select_model(®istry, &settings, resumed_session.as_ref())?; let current_dir = std::env::current_dir()?; @@ -133,6 +136,9 @@ async fn main() -> anyhow::Result<()> { if was_resumed { controller.restore_session_history().await; } + for notice in startup_notices { + controller.push_info(notice); + } // `Runtime::run` consumes the root, so keep the agent for the saved-session // message printed after the TUI exits. @@ -199,20 +205,8 @@ async fn load_or_create_settings(store: &SettingsStore) -> anyhow::Result anyhow::Result> { - let store = Arc::new(JsonLocalModelStore::new( - core::paths::alan_data_dir()?.join("local_models.json"), - )); - let provider = Arc::new(LocalProvider::new(store)); - if let Err(error) = provider.load().await { - tracing::warn!("failed to load local models: {error}"); - } - Ok(provider) -} - fn build_providers( credential_store: &Arc, - local_provider: &Arc, ) -> anyhow::Result>> { Ok(vec![ Arc::new(ZaiProvider::from_store(Arc::clone(credential_store)).build()?), @@ -221,7 +215,6 @@ fn build_providers( Arc::new(OpenRouterProvider::from_store(Arc::clone(credential_store)).build()?), Arc::new(DeepSeekProvider::from_store(Arc::clone(credential_store)).build()?), Arc::new(ChatGptProvider::from_store(Arc::clone(credential_store)).build()), - Arc::clone(local_provider) as Arc, ]) } @@ -236,24 +229,22 @@ async fn load_resumed_session( } fn select_model( - providers: &[Arc], - local_provider: &LocalProvider, + registry: &ProviderRegistry, settings: &Settings, resumed_session: Option<&agent::Session>, ) -> anyhow::Result { let (provider_id, model_id) = match resumed_session { Some(session) => (session.provider.as_str(), session.model.clone()), None => ( - settings.provider.as_deref().unwrap_or("openrouter"), + settings.provider.as_deref().unwrap_or(DEFAULT_PROVIDER), settings .model .clone() .unwrap_or_else(|| DEFAULT_MODEL.into()), ), }; - let provider = providers - .iter() - .find(|provider| provider.id().0 == provider_id) + let provider = registry + .get(&ProviderId::new(provider_id)) .ok_or_else(|| anyhow::anyhow!("Provider not found: {provider_id}"))?; let options = ModelOptions { server_tools: server_tools( @@ -264,12 +255,6 @@ fn select_model( reasoning_effort: settings.reasoning.unwrap_or_default(), extensions: settings.extensions(&model_id), }; - if provider_id == "local" { - let entry = local_provider.find_entry(&model_id).ok_or_else(|| { - anyhow::anyhow!("Local model not found: {model_id} (manage it with /local)") - })?; - return Ok(bind_local_model(&entry, options)?); - } Ok(bind_model(provider.as_ref(), &model_id, options)?) } diff --git a/crates/alan/src/provider_overlay.rs b/crates/alan/src/provider_overlay.rs new file mode 100644 index 0000000..0000b30 --- /dev/null +++ b/crates/alan/src/provider_overlay.rs @@ -0,0 +1,646 @@ +//! `/provider add` and `/provider edit`: the form for a custom provider. + +use crate::core::custom_providers::{self, CustomProviderEntry, CustomProviderStore, ProviderName}; +use crate::root::AlanAction; +use crate::views::theme; +use alan_tui::context::Context; +use alan_tui::{ActionStatus, Component, RenderContext, TaskError}; +use crossterm::event::{Event, KeyCode, KeyEvent, KeyEventKind, KeyModifiers}; +use providers::{ApiId, ProviderId, ProviderRegistry}; +use ratatui::Frame; +use ratatui::layout::{Alignment, Rect}; +use ratatui::style::{Modifier, Style}; +use ratatui::text::{Line, Span}; +use ratatui::widgets::{Block, BorderType, Clear, Paragraph, Wrap}; +use std::sync::Arc; +use unicode_width::UnicodeWidthStr; + +const FIELD_COUNT: usize = 4; +const FIELD_LABELS: [&str; FIELD_COUNT] = ["Name", "URL", "API", "API Key"]; +const NAME: usize = 0; +const URL: usize = 1; +const API: usize = 2; +const API_KEY: usize = 3; + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum Mode { + Add, + Edit, +} + +#[derive(Default)] +struct FormFields { + name: String, + url: String, + api_key: String, +} + +impl FormFields { + fn submission(&self) -> anyhow::Result { + let name = ProviderName::parse(&self.name)?; + let url = self.url.trim(); + anyhow::ensure!(!url.is_empty(), "URL is required"); + let key = self.api_key.trim(); + Ok(Submission { + name, + url: url.to_owned(), + key: (!key.is_empty()).then(|| key.to_owned()), + }) + } +} + +/// What the form sends once its fields parse. +struct Submission { + name: ProviderName, + url: String, + key: Option, +} + +#[derive(Debug, PartialEq, Eq)] +enum OverlayState { + Editing, + Checking, + /// Saved; the server lists this many models. + Saved(usize), + Error(String), +} + +/// Emitted once a provider is saved and registered under this id. +pub struct ProviderSaved(pub ProviderId); + +/// What a key press needs from the host beyond the overlay's own state. +#[derive(Debug, PartialEq, Eq)] +enum KeyOutcome { + Redraw, + Submit, + Close, + Ignore, +} + +pub struct ProviderOverlay { + store: Arc, + registry: Arc, + mode: Mode, + api: ApiId, + fields: FormFields, + focused: usize, + state: OverlayState, +} + +impl ProviderOverlay { + pub fn add(store: Arc, registry: Arc) -> Self { + Self { + store, + registry, + mode: Mode::Add, + api: ApiId::ChatCompletions, + fields: FormFields::default(), + focused: NAME, + state: OverlayState::Editing, + } + } + + /// The name is fixed: `auth.json`, `settings.json` and profiles refer to + /// the provider by it. + pub fn edit( + store: Arc, + registry: Arc, + entry: CustomProviderEntry, + key: Option, + ) -> Self { + Self { + mode: Mode::Edit, + api: entry.api, + fields: FormFields { + name: entry.name.to_string(), + url: entry.url, + api_key: key.unwrap_or_default(), + }, + focused: URL, + ..Self::add(store, registry) + } + } + + fn on_key(&mut self, key: KeyEvent) -> KeyOutcome { + match &self.state { + OverlayState::Editing => self.on_editing_key(key), + OverlayState::Checking => KeyOutcome::Ignore, + OverlayState::Saved(_) => match key.code { + KeyCode::Enter | KeyCode::Esc => KeyOutcome::Close, + _ => KeyOutcome::Ignore, + }, + // Back to the form with everything typed, to fix the URL or key. + OverlayState::Error(_) => match key.code { + KeyCode::Enter | KeyCode::Esc => { + self.state = OverlayState::Editing; + KeyOutcome::Redraw + } + _ => KeyOutcome::Ignore, + }, + } + } + + fn on_editing_key(&mut self, key: KeyEvent) -> KeyOutcome { + match key.code { + KeyCode::Esc => KeyOutcome::Close, + KeyCode::Char('c') if key.modifiers.contains(KeyModifiers::CONTROL) => { + KeyOutcome::Close + } + KeyCode::Enter => KeyOutcome::Submit, + KeyCode::Tab | KeyCode::Down => self.move_focus(1), + KeyCode::BackTab | KeyCode::Up => self.move_focus(-1), + KeyCode::Backspace => self.edit_focused(|field| { + field.pop(); + }), + KeyCode::Char(c) + if key.modifiers.is_empty() || key.modifiers == KeyModifiers::SHIFT => + { + self.edit_focused(|field| field.push(c)) + } + _ => KeyOutcome::Ignore, + } + } + + fn move_focus(&mut self, delta: isize) -> KeyOutcome { + self.focused = (self.focused as isize + delta).rem_euclid(FIELD_COUNT as isize) as usize; + KeyOutcome::Redraw + } + + fn edit_focused(&mut self, edit: impl FnOnce(&mut String)) -> KeyOutcome { + let field = match self.focused { + NAME if self.mode == Mode::Add => &mut self.fields.name, + URL => &mut self.fields.url, + API_KEY => &mut self.fields.api_key, + _ => return KeyOutcome::Ignore, + }; + edit(field); + KeyOutcome::Redraw + } + + fn submit(&mut self, cx: &mut Context<'_, Self, AlanAction>) { + let submission = match self.fields.submission() { + Ok(submission) => submission, + Err(error) => { + self.state = OverlayState::Error(error.to_string()); + cx.notify(); + return; + } + }; + let store = Arc::clone(&self.store); + let registry = Arc::clone(&self.registry); + let mode = self.mode; + let api = self.api.clone(); + cx.spawn( + async move { + save(&store, ®istry, mode, api, submission) + .await + .map_err(|error| TaskError(format!("{error:#}").into())) + }, + |result, overlay, cx| { + overlay.state = match result { + Ok((id, count)) => { + cx.emit(ProviderSaved(id)); + OverlayState::Saved(count) + } + Err(error) => OverlayState::Error(error.to_string()), + }; + cx.notify(); + }, + ); + self.state = OverlayState::Checking; + cx.notify(); + } +} + +/// Check the server, save the provider, then register it. Nothing is saved +/// when the check fails. Returns how many models the server lists. +async fn save( + store: &CustomProviderStore, + registry: &ProviderRegistry, + mode: Mode, + api: ApiId, + submission: Submission, +) -> anyhow::Result<(ProviderId, usize)> { + let Submission { name, url, key } = submission; + if mode == Mode::Add { + custom_providers::ensure_new_name(store, registry, &name).await?; + } + let (entry, models) = custom_providers::check(name, api, &url, key.as_deref()).await?; + match mode { + Mode::Add => store.add(&entry, key).await?, + Mode::Edit => store.update(&entry, key).await?, + } + let count = models.len(); + registry.insert(store.provider(&entry, models)?); + Ok((entry.name.id(), count)) +} + +impl Component for ProviderOverlay { + fn handle_action( + &mut self, + action: &AlanAction, + cx: &mut Context<'_, Self, AlanAction>, + ) -> ActionStatus { + match action { + AlanAction::Paste(text) => { + if self.state == OverlayState::Editing + && self.edit_focused(|field| field.push_str(text)) == KeyOutcome::Redraw + { + cx.notify(); + } + ActionStatus::Handled + } + AlanAction::Raw(Event::Key(key)) => { + if key.kind == KeyEventKind::Press { + match self.on_key(*key) { + KeyOutcome::Redraw => cx.notify(), + KeyOutcome::Submit => self.submit(cx), + KeyOutcome::Close => cx.close_overlay(), + KeyOutcome::Ignore => {} + } + } + ActionStatus::Handled + } + AlanAction::Raw(_) => ActionStatus::Handled, + _ => ActionStatus::Continue, + } + } + + fn render(&self, frame: &mut Frame, area: Rect, _cx: &RenderContext<'_, '_, AlanAction>) { + render_provider_overlay( + frame, + area, + &self.fields, + self.focused, + &self.state, + self.mode, + ); + } +} + +fn render_provider_overlay( + frame: &mut Frame, + area: Rect, + fields: &FormFields, + focused: usize, + state: &OverlayState, + mode: Mode, +) { + let width = area.width.saturating_sub(4).min(88); + let height = area.height.saturating_sub(2).min(18); + if width < 12 || height < 6 { + return; + } + let popup = Rect::new( + area.x + (area.width - width) / 2, + area.y + (area.height - height) / 2, + width, + height, + ); + frame.render_widget(Clear, popup); + let accent = Style::default().fg(theme::PROMPT_FG); + let muted = Style::default().fg(theme::TOOL_FG); + + let (heading, footer) = match state { + OverlayState::Editing => ( + match mode { + Mode::Add => "Add Provider", + Mode::Edit => "Edit Provider", + }, + if width >= 60 { + " Tab move Enter check and save Esc cancel " + } else { + " Tab · Enter · Esc " + }, + ), + OverlayState::Checking => ("Checking…", ""), + OverlayState::Saved(_) => ("Saved", " Enter / Esc close "), + OverlayState::Error(_) => ("Not saved", " Enter / Esc back to the form "), + }; + let block = Block::bordered() + .border_type(BorderType::Rounded) + .border_style(accent) + .style(Style::default().bg(theme::EDITOR_BG).fg(theme::USER_FG)) + .title( + Line::from(Span::styled( + format!(" {heading} "), + accent.add_modifier(Modifier::BOLD), + )) + .style(accent), + ) + .title_alignment(Alignment::Center) + .title_bottom(Line::from(footer).style(muted)); + let inner = block.inner(popup); + frame.render_widget(block, popup); + + let url = fields.url.trim().trim_end_matches('/'); + match state { + OverlayState::Editing => render_fields(frame, inner, fields, focused, mode, accent, muted), + OverlayState::Checking => frame.render_widget( + Paragraph::new(format!(" Checking {url}/models…")).style(accent), + inner, + ), + OverlayState::Saved(count) => frame.render_widget( + Paragraph::new(format!( + " Saved {}: {count} model{}. Pick one with /models.", + fields.name.trim(), + if *count == 1 { "" } else { "s" } + )) + .style(accent), + inner, + ), + OverlayState::Error(error) => frame.render_widget( + Paragraph::new(format!(" {error}")) + .style(Style::default().fg(theme::TOOL_ERROR_FG)) + .wrap(Wrap { trim: false }), + inner, + ), + } +} + +fn render_fields( + frame: &mut Frame, + inner: Rect, + fields: &FormFields, + focused: usize, + mode: Mode, + accent: Style, + muted: Style, +) { + let value_style = Style::default().fg(theme::EDITOR_FG); + let max_label = FIELD_LABELS + .iter() + .map(|label| label.width()) + .max() + .unwrap_or(0); + for (i, label) in FIELD_LABELS.iter().enumerate() { + let y = inner.y + i as u16 * 2; + if y + 2 > inner.bottom() { + break; + } + let is_focused = i == focused; + let text = if i == NAME { &fields.name } else { &fields.url }; + let value = match i { + API => Span::styled("Chat Completions", value_style), + API_KEY if fields.api_key.is_empty() => Span::styled("optional", muted), + API_KEY => Span::styled("•".repeat(fields.api_key.chars().count()), value_style), + _ if text.is_empty() => Span::styled("required", muted), + _ => Span::styled(text.as_str(), value_style), + }; + let mut spans = vec![ + Span::styled(if is_focused { " › " } else { " " }, accent), + Span::styled( + format!("{}{label} ", " ".repeat(max_label - label.width())), + if is_focused { + accent.add_modifier(Modifier::BOLD) + } else { + muted + }, + ), + Span::styled(": ", muted), + ]; + let value_column: usize = spans.iter().map(Span::width).sum(); + spans.push(value); + frame.render_widget( + Paragraph::new(Line::from(spans)), + Rect::new(inner.x, y, inner.width, 1), + ); + let typed = match i { + NAME if mode == Mode::Add => Some(fields.name.width()), + URL => Some(fields.url.width()), + API_KEY => Some(fields.api_key.chars().count()), + _ => None, + }; + if is_focused && let Some(typed) = typed { + let x = (inner.x as usize + value_column + typed).min(inner.right() as usize - 1); + frame.set_cursor_position((x as u16, y)); + } + // No colon in the hint: the alignment test counts one per field row. + if i == URL && fields.url.is_empty() { + frame.render_widget( + Paragraph::new(Span::styled( + format!( + "{}the part before /chat/completions, e.g. https…/v1", + " ".repeat(value_column) + ), + muted, + )), + Rect::new(inner.x, y + 1, inner.width, 1), + ); + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + use providers::InMemoryCredentialStore; + use ratatui::{Terminal, backend::TestBackend}; + + fn store() -> Arc { + Arc::new(CustomProviderStore::new( + std::env::temp_dir().join(format!( + "alan-provider-overlay-test-{}.json", + std::process::id() + )), + Arc::new(InMemoryCredentialStore::new()), + )) + } + + fn overlay() -> ProviderOverlay { + ProviderOverlay::add(store(), Arc::new(ProviderRegistry::new([]))) + } + + fn press(overlay: &mut ProviderOverlay, code: KeyCode) -> KeyOutcome { + overlay.on_key(KeyEvent::new(code, KeyModifiers::NONE)) + } + + fn type_text(overlay: &mut ProviderOverlay, text: &str) { + for c in text.chars() { + press(overlay, KeyCode::Char(c)); + } + } + + fn screen(fields: &FormFields, state: &OverlayState) -> String { + let mut terminal = Terminal::new(TestBackend::new(80, 18)).unwrap(); + terminal + .draw(|frame| { + render_provider_overlay(frame, frame.area(), fields, NAME, state, Mode::Add) + }) + .unwrap(); + terminal + .backend() + .buffer() + .content + .iter() + .map(|cell| cell.symbol()) + .collect() + } + + #[test] + fn typing_fills_the_focused_field_and_tab_moves_on() { + let mut overlay = overlay(); + type_text(&mut overlay, "grok"); + press(&mut overlay, KeyCode::Tab); + type_text(&mut overlay, "https://api.x.ai/v1"); + press(&mut overlay, KeyCode::Tab); + assert_eq!(press(&mut overlay, KeyCode::Char('x')), KeyOutcome::Ignore); + press(&mut overlay, KeyCode::Tab); + type_text(&mut overlay, "xai-1"); + + assert_eq!(overlay.fields.name, "grok"); + assert_eq!(overlay.fields.url, "https://api.x.ai/v1"); + assert_eq!(overlay.fields.api_key, "xai-1"); + } + + #[test] + fn the_name_is_fixed_when_editing() { + let entry = CustomProviderEntry { + name: ProviderName::parse("grok").unwrap(), + url: "https://api.x.ai/v1".into(), + api: ApiId::ChatCompletions, + }; + let mut overlay = ProviderOverlay::edit( + store(), + Arc::new(ProviderRegistry::new([])), + entry, + Some("xai-1".into()), + ); + assert_eq!(overlay.focused, URL); + press(&mut overlay, KeyCode::BackTab); + assert_eq!(press(&mut overlay, KeyCode::Char('x')), KeyOutcome::Ignore); + assert_eq!(overlay.fields.name, "grok"); + assert_eq!(overlay.fields.api_key, "xai-1"); + } + + #[test] + fn an_error_goes_back_to_the_form_with_fields_kept() { + let mut overlay = overlay(); + type_text(&mut overlay, "grok"); + overlay.state = + OverlayState::Error("the API key was rejected (HTTP 401 Unauthorized)".into()); + + assert_eq!(press(&mut overlay, KeyCode::Enter), KeyOutcome::Redraw); + assert_eq!(overlay.state, OverlayState::Editing); + assert_eq!(overlay.fields.name, "grok"); + } + + #[test] + fn saved_closes_and_esc_cancels() { + let mut overlay = overlay(); + assert_eq!(press(&mut overlay, KeyCode::Esc), KeyOutcome::Close); + overlay.state = OverlayState::Saved(3); + assert_eq!(press(&mut overlay, KeyCode::Enter), KeyOutcome::Close); + overlay.state = OverlayState::Checking; + assert_eq!(press(&mut overlay, KeyCode::Esc), KeyOutcome::Ignore); + } + + #[test] + fn submission_trims_and_treats_a_blank_key_as_none() { + let fields = FormFields { + name: " grok ".into(), + url: " https://api.x.ai/v1 ".into(), + api_key: " ".into(), + }; + let submission = fields.submission().unwrap(); + assert_eq!(submission.name.to_string(), "grok"); + assert_eq!(submission.url, "https://api.x.ai/v1"); + assert_eq!(submission.key, None); + } + + #[test] + fn submission_requires_a_url_and_a_valid_name() { + let no_url = FormFields { + name: "grok".into(), + ..FormFields::default() + }; + assert!(no_url.submission().is_err()); + let bad_name = FormFields { + name: "Grok".into(), + url: "http://x".into(), + ..FormFields::default() + }; + assert!(bad_name.submission().is_err()); + } + + #[test] + fn labels_align_and_the_key_is_masked() { + let fields = FormFields { + name: "grok".into(), + url: "https://api.x.ai/v1".into(), + api_key: "xai-secret".into(), + }; + let screen = screen(&fields, &OverlayState::Editing); + assert!(!screen.contains("xai-secret")); + assert!(screen.contains("••••••••••")); + let rows: Vec = screen + .chars() + .collect::>() + .chunks(80) + .map(|row| row.iter().collect()) + .collect(); + // Char positions, not bytes: the focus marker `›` is multibyte. + let colons: Vec = rows + .iter() + .filter_map(|row| row.chars().position(|c| c == ':')) + .collect(); + assert_eq!(colons.len(), FIELD_COUNT, "{rows:#?}"); + assert!(colons.iter().all(|&c| c == colons[0]), "{colons:?}"); + } + + #[test] + fn the_focused_field_has_a_cursor_not_a_highlighted_row() { + let fields = FormFields { + name: "grok".into(), + ..FormFields::default() + }; + let mut terminal = Terminal::new(TestBackend::new(80, 18)).unwrap(); + terminal + .draw(|frame| { + render_provider_overlay( + frame, + frame.area(), + &fields, + NAME, + &OverlayState::Editing, + Mode::Add, + ) + }) + .unwrap(); + + let buffer = terminal.backend().buffer(); + assert!( + buffer + .content + .iter() + .all(|cell| cell.bg != theme::SELECTION_BG) + ); + let rows: Vec> = buffer + .content + .iter() + .flat_map(|cell| cell.symbol().chars()) + .collect::>() + .chunks(80) + .map(<[char]>::to_vec) + .collect(); + let (y, x) = rows + .iter() + .enumerate() + .find_map(|(y, row)| { + let x = row.windows(4).position(|w| w == ['g', 'r', 'o', 'k'])?; + Some((y as u16, x as u16 + 4)) + }) + .unwrap(); + let cursor = terminal.get_cursor_position().unwrap(); + assert_eq!((cursor.x, cursor.y), (x, y)); + } + + #[test] + fn an_error_shows_the_reason() { + let screen = screen( + &FormFields::default(), + &OverlayState::Error("the API key was rejected (HTTP 401 Unauthorized)".into()), + ); + assert!(screen.contains("HTTP 401")); + } +} diff --git a/crates/alan/src/root.rs b/crates/alan/src/root.rs index 5449ecb..a70ec21 100644 --- a/crates/alan/src/root.rs +++ b/crates/alan/src/root.rs @@ -143,6 +143,7 @@ impl Component for AlanRoot { .take() .expect("chat component installed once"), Arc::clone(&self.providers), + Arc::clone(&self.credentials), self.permission_handler.clone(), self.policy.clone(), self.skill_roots.clone(), diff --git a/crates/alan/src/views/components/chat_view/mod.rs b/crates/alan/src/views/components/chat_view/mod.rs index f013119..e0a6c5e 100644 --- a/crates/alan/src/views/components/chat_view/mod.rs +++ b/crates/alan/src/views/components/chat_view/mod.rs @@ -13,6 +13,7 @@ mod session; mod steering; use crate::core::chat::ChatController; +use crate::core::custom_providers::{self, CustomProviderStore}; use crate::core::permissions::PermissionHandler; use crate::core::permissions::PermissionRequest; use crate::core::permissions::{Answer, Policy, ToolPolicy}; @@ -29,7 +30,7 @@ use alan_tui::entity::Entity; use alan_tui::{Subscription, SubscriptionEvent}; use crossterm::event::{Event, KeyCode, KeyEventKind}; use futures_util::Stream; -use providers::ProviderRegistry; +use providers::{CredentialStore, ProviderRegistry}; use ratatui::Frame; use ratatui::layout::{Constraint, Layout, Rect}; use ratatui::style::Style; @@ -50,12 +51,6 @@ const EDITOR_BOTTOM_PAD: u16 = 1; /// Fixed-rate repaint interval while an agent stream is in flight. const STREAM_REPAINT_INTERVAL: std::time::Duration = std::time::Duration::from_millis(32); -/// Action chosen from the local-model picker. -pub(crate) enum LocalPick { - Remove, - Edit, -} - #[derive(Debug, Clone, Copy)] pub struct LoginRequested; @@ -71,6 +66,7 @@ pub struct ChatView { permission: Option>, providers: Arc, + credentials: Arc, permission_handler: PermissionHandler, policy: ToolPolicy, /// Subscription to the permission-request stream. @@ -89,6 +85,7 @@ impl ChatView { pub fn new( controller: ChatController, providers: Arc, + credentials: Arc, permission_handler: PermissionHandler, policy: ToolPolicy, skill_roots: Vec, @@ -101,6 +98,7 @@ impl ChatView { pending_permission: None, permission: None, providers, + credentials, permission_handler, policy, permission_subscription: None, @@ -112,6 +110,17 @@ impl ChatView { } } + /// What `/provider` saves through. It holds only handles, so a fresh one + /// sees the same files as the one `main.rs` loaded them from. + fn custom_provider_store(&self) -> Arc { + let path = custom_providers::default_providers_path() + .expect("main.rs resolved the data directory at startup"); + Arc::new(CustomProviderStore::new( + path, + Arc::clone(&self.credentials), + )) + } + fn set_tool_policy(&mut self, policy: Policy, cx: &mut Context<'_, Self, AlanAction>) { let policy_changed = self.policy.policy() != policy; cx.spawn( @@ -250,7 +259,7 @@ impl ChatView { Cmd::Help => controller.push_info(crate::core::SlashCommand::help()), Cmd::New => session::start_new_session(self, cx), Cmd::Models => models::open_models_picker(self, cx), - Cmd::Local => models::handle_local_command(self, &submission.text, cx), + Cmd::Provider => models::handle_provider_command(self, &submission.text, cx), Cmd::Profile => profiles::handle_profile_command(self, &submission.text, cx), Cmd::Quit => cx.quit(), } diff --git a/crates/alan/src/views/components/chat_view/models.rs b/crates/alan/src/views/components/chat_view/models.rs index f049b0a..a02aaed 100644 --- a/crates/alan/src/views/components/chat_view/models.rs +++ b/crates/alan/src/views/components/chat_view/models.rs @@ -1,5 +1,5 @@ -//! Model, provider, and local-model commands: `/models`, `/providers`, -//! `/local`, and the model picker overlays. +//! Model and provider commands: `/models`, `/model-providers`, +//! `/provider`, and the model picker overlays. use crate::core::SlashCommand; use crate::core::server_tools::server_tools; @@ -8,31 +8,40 @@ use crate::root::AlanAction; use alan_tui::TaskError; use alan_tui::context::Context; use providers::{ - Model, ModelInfo, ModelOptions, OpenRouterOptions, Provider, ProviderId, ProviderRegistry, - bind_local_model, bind_model, + Model, ModelInfo, ModelOptions, OpenRouterOptions, ProviderId, ProviderRegistry, bind_model, }; use std::sync::Arc; -use super::LocalPick; use crate::views::components::{SearchListEvent, SearchListOverlay}; use super::{ChatView, parse_provider_order, persist_provider_order}; -use crate::local_model_overlay::LocalModelOverlay; - -/// Provider id reserved for locally-hosted models. -const LOCAL_PROVIDER_ID: &str = "local"; +use crate::core::custom_providers::{CustomProviderEntry, ProviderName}; +use crate::provider_overlay::{ProviderOverlay, ProviderSaved}; /// `/models`: open the model picker overlay. pub(crate) fn open_models_picker(view: &mut ChatView, cx: &mut Context<'_, ChatView, AlanAction>) { + open_model_picker(view, cx, "Select Model".to_owned(), None); +} + +/// The model picker. With `replacing`, that provider's models are hidden and +/// the provider is removed once the switch to the chosen model succeeds. +fn open_model_picker( + view: &mut ChatView, + cx: &mut Context<'_, ChatView, AlanAction>, + title: String, + replacing: Option, +) { let providers = Arc::clone(&view.providers); + let hidden = replacing.as_ref().map(ProviderName::id); let picker = cx.open_overlay(SearchListOverlay::new( - "Select Model", - model_labels(&providers), + title, + model_labels(&picker_models(&providers, hidden.as_ref())), )); let providers_for_fetch = Arc::clone(&providers); let providers_for_items = Arc::clone(&providers); + let hidden_for_items = hidden.clone(); let _ = cx.spawn( async move { fetch_all_models(&providers_for_fetch).await; @@ -40,7 +49,10 @@ pub(crate) fn open_models_picker(view: &mut ChatView, cx: &mut Context<'_, ChatV }, move |result, _view, cx| { if result.is_ok() { - let items = model_labels(&providers_for_items); + let items = model_labels(&picker_models( + &providers_for_items, + hidden_for_items.as_ref(), + )); let _ = cx.update(picker, |p| p.set_items(items)); } }, @@ -52,11 +64,15 @@ pub(crate) fn open_models_picker(view: &mut ChatView, cx: &mut Context<'_, ChatV let SearchListEvent::Chosen(index) = event else { return; }; - let Some(model_info) = all_models(&providers).into_iter().nth(*index) else { + let Some(model_info) = picker_models(&providers, hidden.as_ref()) + .into_iter() + .nth(*index) + else { return; }; let agent = view.controller.agent(); let providers = Arc::clone(&providers); + let replacing = replacing.clone(); let _ = cx.spawn( async move { let mut options = agent.model_options().await; @@ -85,6 +101,9 @@ pub(crate) fn open_models_picker(view: &mut ChatView, cx: &mut Context<'_, ChatV view.controller.set_max_context(max_context); view.controller.set_reasoning_effort(reasoning_effort); view.controller.apply_model_switch(name); + if let Some(replaced) = replacing { + remove_idle_provider(view, cx, replaced); + } } Err(e) => view.controller.apply_model_switch_failed(e.to_string()), } @@ -157,135 +176,193 @@ pub(crate) fn apply_model_provider( ); } -/// `/local`: dispatch local model sub-commands. -pub(crate) fn handle_local_command( +/// `/provider`: add, edit or remove a custom provider. +pub(crate) fn handle_provider_command( view: &mut ChatView, text: &str, cx: &mut Context<'_, ChatView, AlanAction>, ) { let args = SlashCommand::parse_with_args(text).map(|(_, a)| a.trim().to_owned()); match args.as_deref() { - Some("add") => open_local_model_overlay(view, cx, None), - Some("remove") => open_local_remove_picker(view, cx), - Some("edit") => open_local_edit_picker(view, cx), + Some("add") => { + let form = + ProviderOverlay::add(view.custom_provider_store(), Arc::clone(&view.providers)); + open_provider_form(cx, form); + } + Some("edit") => open_provider_picker(view, cx, ProviderPick::Edit), + Some("remove") => open_provider_picker(view, cx, ProviderPick::Remove), _ => view .controller - .push_info("usage: /local ".to_owned()), + .push_info("usage: /provider ".to_owned()), } } -/// Open the local model overlay (for adding or editing). -pub(crate) fn open_local_model_overlay( +/// List saved providers in a picker, then edit or remove the chosen one. +fn open_provider_picker( view: &mut ChatView, cx: &mut Context<'_, ChatView, AlanAction>, - edit_entry: Option, + pick: ProviderPick, ) { - let Some(local_provider) = view.providers.local() else { - view.controller - .push_info("Local provider not available".to_owned()); - return; + let title = match pick { + ProviderPick::Edit => "Edit Provider", + ProviderPick::Remove => "Remove Provider", }; - cx.open_overlay(LocalModelOverlay::new(local_provider, edit_entry)); + let store = view.custom_provider_store(); + cx.spawn( + async move { store.entries().await.map_err(super::to_task) }, + move |result, _view, cx| match result { + Ok(entries) => { + let labels = entries + .iter() + .map(|entry| format!("{} — {}", entry.name, entry.url)) + .collect(); + let mut picker = SearchListOverlay::new(title, labels); + if entries.is_empty() { + picker.set_empty_message("No saved providers"); + } + let picker = cx.open_overlay(picker); + cx.subscribe_once::( + picker, + move |event, view, _picker, cx| { + let SearchListEvent::Chosen(index) = event else { + return; + }; + let Some(entry) = entries.get(*index).cloned() else { + return; + }; + match pick { + ProviderPick::Edit => edit_provider(view, cx, entry), + ProviderPick::Remove => remove_provider(view, cx, entry.name), + } + }, + ); + } + Err(error) => { + let mut picker = SearchListOverlay::new(title, Vec::new()); + picker.set_empty_message(format!("unable to load providers: {error}")); + cx.open_overlay(picker); + } + }, + ); } -/// Open a picker to remove a local model. -fn open_local_remove_picker(view: &mut ChatView, cx: &mut Context<'_, ChatView, AlanAction>) { - open_local_picker( - view, - cx, - "Remove Local Model", - "No local models to remove.", - LocalPick::Remove, +/// Action chosen from the custom-provider picker. +#[derive(Clone, Copy)] +enum ProviderPick { + Edit, + Remove, +} + +fn edit_provider( + view: &mut ChatView, + cx: &mut Context<'_, ChatView, AlanAction>, + entry: CustomProviderEntry, +) { + let store = view.custom_provider_store(); + cx.spawn( + async move { + let key = store.key(&entry.name).await.map_err(super::to_task)?; + Ok((store, entry, key)) + }, + move |result, view, cx| match result { + Ok((store, entry, key)) => { + let form = ProviderOverlay::edit(store, Arc::clone(&view.providers), entry, key); + open_provider_form(cx, form); + } + Err(error) => { + view.controller + .push_info(format!("failed to read the provider's key: {error}")); + cx.notify(); + } + }, ); } -/// Open a picker to edit a local model. -fn open_local_edit_picker(view: &mut ChatView, cx: &mut Context<'_, ChatView, AlanAction>) { - open_local_picker( - view, - cx, - "Edit Local Model", - "No local models to edit.", - LocalPick::Edit, +fn open_provider_form(cx: &mut Context<'_, ChatView, AlanAction>, form: ProviderOverlay) { + let form = cx.open_overlay(form); + cx.subscribe_once::(form, |saved, view, _form, cx| { + rebind_model(view, cx, saved.0.clone()); + }); +} + +/// A bound model keeps the provider it was bound from, so after an edit the +/// running model is bound again from the registered replacement. +fn rebind_model( + view: &mut ChatView, + cx: &mut Context<'_, ChatView, AlanAction>, + provider: ProviderId, +) { + let agent = view.controller.agent(); + let providers = Arc::clone(&view.providers); + cx.spawn( + async move { + let info = agent.info().await; + if info.provider != provider { + return Ok(()); + } + let model = bind_model_info(&providers, &info, agent.model_options().await)?; + agent + .set_model(model) + .await + .map_err(|e| TaskError(e.into())) + }, + |result, view, cx| { + if let Err(error) = result { + view.controller.push_info(format!( + "saved, but the running model still uses the old settings: {error}" + )); + cx.notify(); + } + }, ); } -/// Open a `SearchListOverlay` over the local catalog and run `on_pick` with the -/// chosen entry's index. Shared by the remove and edit flows. -fn open_local_picker( +/// The provider of the running model is swapped out first: the model picker +/// opens without its models, and it goes once the switch succeeds. +fn remove_provider( view: &mut ChatView, cx: &mut Context<'_, ChatView, AlanAction>, - title: &str, - empty_message: &str, - pick: LocalPick, + name: ProviderName, ) { - let Some(local) = view.providers.local() else { - view.controller.push_info(empty_message.to_owned()); - return; - }; - let models: Vec = local - .models() - .iter() - .map(|m| format!("local — {}", m.name)) - .collect(); - if models.is_empty() { - view.controller.push_info(empty_message.to_owned()); - return; - } - let picker = cx.open_overlay(SearchListOverlay::new(title, models)); - let local = local.clone(); - cx.subscribe_once::( - picker, - move |event, _view, _picker, cx| { - let SearchListEvent::Chosen(index) = event else { - return; - }; - let Some(model_info) = local.models().into_iter().nth(*index) else { + let agent = view.controller.agent(); + cx.spawn( + async move { Ok::<_, TaskError>(agent.info().await.provider) }, + move |result, view, cx| { + let Ok(active) = result else { return; }; - match &pick { - LocalPick::Remove => { - let local = Arc::clone(&local); - let model_id = model_info.id.clone(); - let display_id = model_id.clone(); - cx.spawn( - async move { - local - .remove_model(&model_id) - .await - .map_err(|e| TaskError(e.to_string().into()))?; - Ok::<(), TaskError>(()) - }, - move |result, view, _cx| match result { - Ok(()) => { - view.controller - .push_info(format!("Removed local model: {display_id}")); - } - Err(e) => view.controller.push_info(format!("Failed to remove: {e}")), - }, - ); - } - LocalPick::Edit => { - let local = Arc::clone(&local); - cx.spawn( - async move { - local.find_entry(&model_info.id).ok_or_else(|| { - TaskError( - format!("Local model not found: {}", model_info.id).into(), - ) - }) - }, - move |result, view, cx| match result { - Ok(entry) => open_local_model_overlay(view, cx, Some(entry)), - Err(e) => view.controller.push_info(e.to_string()), - }, - ); - } + if active == name.id() { + open_model_picker(view, cx, format!("Replace {name} with"), Some(name)); + } else { + remove_idle_provider(view, cx, name); } }, ); } +fn remove_idle_provider( + view: &mut ChatView, + cx: &mut Context<'_, ChatView, AlanAction>, + name: ProviderName, +) { + let store = view.custom_provider_store(); + let providers = Arc::clone(&view.providers); + cx.spawn( + async move { + store.remove(&name).await.map_err(super::to_task)?; + providers.remove(&name.id()); + Ok(name) + }, + |result, view, cx| { + view.controller.push_info(match result { + Ok(name) => format!("removed provider: {name}"), + Err(error) => format!("failed to remove provider: {error}"), + }); + cx.notify(); + }, + ); +} + pub(crate) async fn fetch_all_models(providers: &ProviderRegistry) { for provider in providers.providers() { if let Err(e) = provider.fetch_models().await { @@ -306,9 +383,17 @@ pub(crate) fn all_models(providers: &ProviderRegistry) -> Vec { .collect() } -pub(crate) fn model_labels(providers: &ProviderRegistry) -> Vec { +/// The models a picker offers: every provider's, minus `hidden`'s. +fn picker_models(providers: &ProviderRegistry, hidden: Option<&ProviderId>) -> Vec { all_models(providers) .into_iter() + .filter(|model| Some(&model.provider) != hidden) + .collect() +} + +fn model_labels(models: &[ModelInfo]) -> Vec { + models + .iter() .map(|m| format!("{} — {}", m.provider, m.name)) .collect() } @@ -326,23 +411,11 @@ pub(crate) fn server_tools_for_provider( }) } -/// Bind a catalog entry to a live model. Local entries bind through the -/// registry's local provider; everything else through the named provider. pub(crate) fn bind_model_info( providers: &ProviderRegistry, model_info: &ModelInfo, options: ModelOptions, ) -> Result { - if model_info.provider.0 == LOCAL_PROVIDER_ID { - let local = providers - .local() - .ok_or_else(|| TaskError("local provider not available".into()))?; - let entry = local - .find_entry(&model_info.id) - .ok_or_else(|| TaskError(format!("local model not found: {}", model_info.id).into()))?; - return bind_local_model(&entry, options).map_err(|e| TaskError(e.into())); - } - let provider = providers .get(&model_info.provider) .ok_or_else(|| TaskError(format!("provider not found: {}", model_info.provider).into()))?; @@ -357,3 +430,28 @@ pub(crate) async fn persist_model(model_id: &str, provider_id: &ProviderId) -> a }) .await } + +#[cfg(test)] +mod tests { + use super::*; + use crate::core::custom_providers::tests::provider; + + #[test] + fn picker_models_hides_only_the_replaced_provider() { + let registry = ProviderRegistry::new([]); + registry.insert(provider("grok", &["grok-4"])); + registry.insert(provider("groq", &["llama"])); + + let all: Vec<_> = picker_models(®istry, None) + .into_iter() + .map(|m| m.id) + .collect(); + let without_grok: Vec<_> = picker_models(®istry, Some(&ProviderId::new("grok"))) + .into_iter() + .map(|m| m.id) + .collect(); + + assert_eq!(all, ["grok-4", "llama"]); + assert_eq!(without_grok, ["llama"]); + } +} diff --git a/crates/alan/src/views/mod.rs b/crates/alan/src/views/mod.rs index 19c8eed..7f57709 100644 --- a/crates/alan/src/views/mod.rs +++ b/crates/alan/src/views/mod.rs @@ -1,4 +1,4 @@ mod components; pub mod theme; -pub(crate) use components::{ChatView, Header, LoginRequested, SearchListEvent, SearchListOverlay}; +pub(crate) use components::{ChatView, Header, LoginRequested}; diff --git a/crates/providers/src/auth.rs b/crates/providers/src/auth.rs index 78eb925..15df3f3 100644 --- a/crates/providers/src/auth.rs +++ b/crates/providers/src/auth.rs @@ -66,18 +66,6 @@ impl AuthResolver for ApiKeyAuth { } } -pub struct NoAuth; - -#[async_trait] -impl AuthResolver for NoAuth { - async fn resolve(&self) -> Result { - Ok(Resolved { - credential: RequestCredential::None, - headers: Vec::new(), - }) - } -} - pub struct CredentialAuth { provider: ProviderId, store: Arc, diff --git a/crates/providers/src/custom.rs b/crates/providers/src/custom.rs new file mode 100644 index 0000000..57c8367 --- /dev/null +++ b/crates/providers/src/custom.rs @@ -0,0 +1,532 @@ +use crate::{ + ApiId, Auth, AuthError, AuthMethod, AuthResolver, AuthResult, Credential, CredentialStore, + ModelInfo, Provider, ProviderError, ProviderId, Resolved, ServerToolInfo, +}; +use async_trait::async_trait; +use llm::{ChatCompletionsApi, Credential as RequestCredential, HttpClient, LlmApi}; +use serde::Deserialize; +use std::collections::HashMap; +use std::sync::{Arc, RwLock}; + +/// Where servers report a model's context window. No field is standard, so +/// the first one holding a positive integer wins. +const CONTEXT_FIELDS: [&str; 4] = [ + "context_length", + "context_window", + "max_context_length", + "max_model_len", +]; + +/// A provider the user adds at runtime: any server that speaks Chat +/// Completions, known by a name and a base URL. +pub struct CustomProvider { + id: ProviderId, + api: ApiId, + base_url: String, + models: RwLock>, + apis: HashMap>, + auth: Arc, + fetch_lock: tokio::sync::Mutex<()>, +} + +impl CustomProvider { + /// `models` seeds the catalog so a provider checked a moment ago needs no + /// second `/models` call; an empty list is filled on the next fetch. + pub fn new( + id: ProviderId, + api: ApiId, + base_url: impl Into, + credentials: Arc, + models: Vec, + ) -> Result { + let base_url = base_url.into(); + let client: Arc = match &api { + ApiId::ChatCompletions => Arc::new(ChatCompletionsApi::new( + base_url.clone(), + Arc::new(HttpClient::new()), + )), + other => return Err(ProviderError::UnsupportedApi(other.clone())), + }; + Ok(Self { + auth: Arc::new(StoredKeyAuth { + provider: id.clone(), + credentials, + }), + apis: HashMap::from([(api.clone(), client)]), + api, + models: RwLock::new(models), + fetch_lock: tokio::sync::Mutex::new(()), + base_url, + id, + }) + } +} + +#[async_trait] +impl Provider for CustomProvider { + fn id(&self) -> ProviderId { + self.id.clone() + } + + fn models(&self) -> Vec { + self.models + .read() + .unwrap_or_else(|e| e.into_inner()) + .clone() + } + + fn server_tools(&self) -> &[ServerToolInfo] { + &[] + } + + fn apis(&self) -> &HashMap> { + &self.apis + } + + async fn fetch_models(&self) -> Result<(), ProviderError> { + let _guard = self.fetch_lock.lock().await; + let credential = self + .auth + .resolve() + .await + .map_err(|error| ProviderError::Fetch(format!("authentication failed: {error}")))? + .credential; + let key = match credential { + RequestCredential::ApiKey(key) => Some(key), + _ => None, + }; + let models = fetch_catalog(&self.id, &self.api, &self.base_url, key.as_deref()).await?; + *self.models.write().unwrap_or_else(|e| e.into_inner()) = models; + Ok(()) + } +} + +#[async_trait] +impl Auth for CustomProvider { + fn auth_methods(&self) -> Vec { + vec![AuthMethod::ApiKey] + } + + fn auth(&self) -> Arc { + self.auth.clone() + } + + async fn validate_auth(&self, auth_result: &AuthResult) -> Result<(), AuthError> { + match auth_result { + AuthResult::ApiKey(key) => { + fetch_catalog(&self.id, &self.api, &self.base_url, Some(key)) + .await + .map(drop) + .map_err(|error| { + AuthError::Validation(match error { + ProviderError::Fetch(reason) => reason, + other => other.to_string(), + }) + }) + } + AuthResult::OAuth => Err(AuthError::Missing), + } + } +} + +/// Reads the provider's key from the credential store. Without one, requests +/// carry no `Authorization` header, which keyless local servers expect. +struct StoredKeyAuth { + provider: ProviderId, + credentials: Arc, +} + +#[async_trait] +impl AuthResolver for StoredKeyAuth { + async fn resolve(&self) -> Result { + let credential = match self.credentials.read(&self.provider).await? { + Some(Credential::ApiKey { key }) if !key.is_empty() => RequestCredential::ApiKey(key), + _ => RequestCredential::None, + }; + Ok(Resolved { + credential, + headers: Vec::new(), + }) + } +} + +/// The models a server lists. No API standard has a health endpoint, so this +/// is how a URL and key are checked before a provider is saved. +pub async fn fetch_catalog( + provider: &ProviderId, + api: &ApiId, + base_url: &str, + api_key: Option<&str>, +) -> Result, ProviderError> { + match api { + ApiId::ChatCompletions => fetch_chat_completions_catalog(provider, base_url, api_key).await, + other => Err(ProviderError::UnsupportedApi(other.clone())), + } +} + +/// `GET {base_url}/models` in the Chat Completions format. +async fn fetch_chat_completions_catalog( + provider: &ProviderId, + base_url: &str, + api_key: Option<&str>, +) -> Result, ProviderError> { + let base = base_url.trim().trim_end_matches('/'); + let url = format!("{base}/models"); + let mut request = reqwest::Client::new() + .get(&url) + .timeout(std::time::Duration::from_secs(10)); + if let Some(key) = api_key.filter(|key| !key.is_empty()) { + request = request.bearer_auth(key); + } + // reqwest's own message ("error sending request") hides why. + let response = request.send().await.map_err(|error| { + ProviderError::Fetch(if error.is_timeout() { + format!("{base} did not answer within 10 seconds") + } else if error.is_connect() { + format!("could not reach {base}; is the server running?") + } else if error.is_builder() { + format!("\"{base}\" is not a valid URL; it should look like http://localhost:11434/v1") + } else { + format!("could not reach {base}: {error}") + }) + })?; + let status = response.status(); + match status { + reqwest::StatusCode::UNAUTHORIZED => { + return Err(ProviderError::Fetch(format!( + "the API key was rejected (HTTP {status})" + ))); + } + // The usual mistake: a base URL without the `/v1` the API lives under. + reqwest::StatusCode::NOT_FOUND => { + return Err(ProviderError::Fetch(format!( + "{url} was not found; the URL should end where /chat/completions would start, e.g. …/v1" + ))); + } + status if !status.is_success() => { + return Err(ProviderError::Fetch(format!( + "{url} answered HTTP {status}" + ))); + } + _ => {} + } + let payload = response + .text() + .await + .map_err(|error| ProviderError::Fetch(format!("could not read {url}: {error}")))?; + parse_catalog(provider, base, &payload) +} + +/// `source` is the base URL, for the error messages. +fn parse_catalog( + provider: &ProviderId, + source: &str, + payload: &str, +) -> Result, ProviderError> { + #[derive(Deserialize)] + struct Listing { + data: Vec, + } + + let listing: Listing = serde_json::from_str(payload).map_err(|error| { + ProviderError::Fetch(format!( + "{source}/models did not return a Chat Completions model list: {error}" + )) + })?; + let mut models: Vec = listing + .data + .iter() + .filter_map(|entry| { + let id = entry.get("id")?.as_str()?.trim(); + (!id.is_empty()).then(|| ModelInfo { + provider: provider.clone(), + id: id.to_owned(), + name: id.to_owned(), + pricing: None, + context_length: context_length(entry), + }) + }) + .collect(); + models.sort_by(|a, b| a.id.cmp(&b.id)); + models.dedup_by(|a, b| a.id == b.id); + if models.is_empty() { + return Err(ProviderError::Fetch(format!("{source} lists no models"))); + } + Ok(models) +} + +/// A zero window would paint the context badge full from the first token, so +/// it counts as missing, like a string or a float. +fn context_length(entry: &serde_json::Value) -> Option { + CONTEXT_FIELDS + .iter() + .find_map(|field| entry.get(field)?.as_u64().filter(|&tokens| tokens > 0)) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::InMemoryCredentialStore; + + const PROVIDER: &str = "grok"; + + const SOURCE: &str = "http://x/v1"; + + fn id() -> ProviderId { + ProviderId::new(PROVIDER) + } + + // one-shot server: replies `response`, hands back the raw request. + async fn serve_once(response: &'static str) -> (String, tokio::task::JoinHandle) { + use tokio::io::{AsyncReadExt, AsyncWriteExt}; + let listener = tokio::net::TcpListener::bind(("127.0.0.1", 0)) + .await + .unwrap(); + let address = listener.local_addr().unwrap(); + let server = tokio::spawn(async move { + let (mut socket, _) = listener.accept().await.unwrap(); + let mut request = Vec::new(); + let mut buffer = [0; 4096]; + while !request.windows(4).any(|window| window == b"\r\n\r\n") { + let read = socket.read(&mut buffer).await.unwrap(); + assert!(read > 0); + request.extend_from_slice(&buffer[..read]); + } + socket.write_all(response.as_bytes()).await.unwrap(); + String::from_utf8(request).unwrap() + }); + (format!("http://{address}/v1/"), server) + } + + const TWO_MODELS: &str = "HTTP/1.1 200 OK\r\nconnection: close\r\n\r\n{\"data\":[{\"id\":\"m2\"},{\"id\":\"m1\",\"context_window\":8192}]}"; + + #[test] + fn parse_catalog_reads_each_known_context_field() { + let models = parse_catalog( + &id(), + SOURCE, + r#"{"data":[ + {"id":"a","context_length":128000}, + {"id":"b","context_window":131072}, + {"id":"c","max_context_length":32768}, + {"id":"d","max_model_len":8192}, + {"id":"e"}]}"#, + ) + .unwrap(); + let contexts: Vec<_> = models + .iter() + .map(|m| (m.id.as_str(), m.context_length)) + .collect(); + assert_eq!( + contexts, + [ + ("a", Some(128000)), + ("b", Some(131072)), + ("c", Some(32768)), + ("d", Some(8192)), + ("e", None) + ] + ); + assert!(models.iter().all(|m| m.provider == id() && m.name == m.id)); + } + + #[test] + fn context_length_takes_the_first_field_present() { + let models = parse_catalog( + &id(), + SOURCE, + r#"{"data":[{"id":"a","context_length":1000,"context_window":2000}]}"#, + ) + .unwrap(); + assert_eq!(models[0].context_length, Some(1000)); + } + + #[test] + fn context_length_ignores_zero_strings_and_floats() { + let models = parse_catalog(&id(), SOURCE, r#"{"data":[ + {"id":"a","context_length":0,"context_window":"4096","max_context_length":4096.5,"max_model_len":2048}, + {"id":"b","context_length":"128k"}]}"#, + ) + .unwrap(); + assert_eq!(models[0].context_length, Some(2048)); + assert_eq!(models[1].context_length, None); + } + + #[test] + fn parse_catalog_sorts_dedupes_and_skips_blank_ids() { + let models = parse_catalog(&id(), SOURCE, r#"{"data":[{"id":"b"},{"id":"a","owned_by":"x"},{"id":"b"},{"id":" "},{"object":"model"}]}"#, + ) + .unwrap(); + let ids: Vec<_> = models.iter().map(|m| m.id.as_str()).collect(); + assert_eq!(ids, ["a", "b"]); + } + + #[test] + fn parse_catalog_rejects_empty_and_other_shapes() { + let empty = parse_catalog(&id(), SOURCE, r#"{"data":[]}"#).unwrap_err(); + assert!( + empty.to_string().contains("http://x/v1 lists no models"), + "{empty}" + ); + assert!(parse_catalog(&id(), SOURCE, r#"{"models":[{"name":"models/gemini"}]}"#).is_err()); + let error = parse_catalog(&id(), SOURCE, "404").unwrap_err(); + assert!( + error + .to_string() + .contains("http://x/v1/models did not return a Chat Completions model list: "), + "{error}" + ); + } + + #[tokio::test] + async fn unsupported_apis_are_refused_before_any_request() { + let provider = CustomProvider::new( + id(), + ApiId::AnthropicMessages, + "http://localhost:11434/v1", + Arc::new(InMemoryCredentialStore::new()), + Vec::new(), + ); + assert!(matches!( + provider, + Err(ProviderError::UnsupportedApi(ApiId::AnthropicMessages)) + )); + + // "not a url" would fail as "invalid URL" if a request were attempted. + let error = fetch_catalog(&id(), &ApiId::OpenAiResponses, "not a url", None) + .await + .unwrap_err(); + assert!(matches!( + error, + ProviderError::UnsupportedApi(ApiId::OpenAiResponses) + )); + } + + #[tokio::test] + async fn fetch_catalog_sends_no_auth_without_key() { + let (url, server) = serve_once(TWO_MODELS).await; + fetch_catalog(&id(), &ApiId::ChatCompletions, &url, None) + .await + .unwrap(); + assert!( + !server + .await + .unwrap() + .to_lowercase() + .contains("authorization") + ); + } + + #[tokio::test] + async fn fetch_catalog_reports_unreachable() { + // bind then drop: port is closed. + let address = std::net::TcpListener::bind(("127.0.0.1", 0)) + .unwrap() + .local_addr() + .unwrap(); + let error = fetch_catalog( + &id(), + &ApiId::ChatCompletions, + &format!("http://{address}/v1"), + None, + ) + .await + .unwrap_err(); + assert!( + error.to_string().contains(&format!( + "could not reach http://{address}/v1; is the server running?" + )), + "{error}" + ); + } + + #[tokio::test] + async fn fetch_catalog_reports_invalid_url() { + let error = fetch_catalog(&id(), &ApiId::ChatCompletions, "not a url", None) + .await + .unwrap_err(); + assert!( + error.to_string().contains( + "\"not a url\" is not a valid URL; it should look like http://localhost:11434/v1" + ), + "{error}" + ); + } + + #[tokio::test] + async fn fetch_catalog_explains_a_wrong_path() { + let (url, _server) = + serve_once("HTTP/1.1 404 Not Found\r\nconnection: close\r\n\r\n").await; + let error = fetch_catalog(&id(), &ApiId::ChatCompletions, &url, None) + .await + .unwrap_err(); + assert!( + error.to_string().contains( + "/v1/models was not found; the URL should end where /chat/completions would start, e.g. …/v1" + ), + "{error}" + ); + } + + #[tokio::test] + async fn fetch_models_uses_the_stored_key() { + let (url, server) = serve_once(TWO_MODELS).await; + let credentials = Arc::new(InMemoryCredentialStore::new()); + credentials + .put( + &id(), + Credential::ApiKey { + key: "sk-stored".into(), + }, + ) + .await + .unwrap(); + let provider = + CustomProvider::new(id(), ApiId::ChatCompletions, url, credentials, Vec::new()) + .unwrap(); + provider.fetch_models().await.unwrap(); + assert_eq!(provider.models().len(), 2); + let request = server.await.unwrap(); + assert!(request.starts_with("GET /v1/models HTTP/1.1"), "{request}"); + assert!( + request + .to_lowercase() + .contains("authorization: bearer sk-stored") + ); + } + + #[tokio::test] + async fn provider_without_a_key_sends_no_credential() { + let provider = CustomProvider::new( + id(), + ApiId::ChatCompletions, + "http://localhost:11434/v1", + Arc::new(InMemoryCredentialStore::new()), + Vec::new(), + ) + .unwrap(); + let credential = provider.auth().resolve().await.unwrap().credential; + assert!(matches!(credential, RequestCredential::None)); + } + + #[tokio::test] + async fn validate_auth_checks_the_key_against_models() { + let (url, _server) = + serve_once("HTTP/1.1 401 Unauthorized\r\nconnection: close\r\n\r\n{}").await; + let provider = CustomProvider::new( + id(), + ApiId::ChatCompletions, + url, + Arc::new(InMemoryCredentialStore::new()), + Vec::new(), + ) + .unwrap(); + let result = provider + .validate_auth(&AuthResult::ApiKey("bad".into())) + .await; + assert!( + matches!(&result, Err(AuthError::Validation(reason)) if reason == "the API key was rejected (HTTP 401 Unauthorized)"), + "{result:?}" + ); + } +} diff --git a/crates/providers/src/lib.rs b/crates/providers/src/lib.rs index 7aeaea5..c7ca93b 100644 --- a/crates/providers/src/lib.rs +++ b/crates/providers/src/lib.rs @@ -2,20 +2,20 @@ mod auth; mod catalog; mod chatgpt; mod credentials; +mod custom; mod deepseek; mod google; -mod local; mod model; mod oauth; mod openrouter; mod provider; -mod store; mod zai; +pub use custom::{CustomProvider, fetch_catalog}; pub use deepseek::{DeepSeekBuilder, DeepSeekProvider}; pub use auth::{ - ApiKeyAuth, AuthError, AuthMethod, AuthResolver, AuthResult, CredentialAuth, NoAuth, Resolved, + ApiKeyAuth, AuthError, AuthMethod, AuthResolver, AuthResult, CredentialAuth, Resolved, }; pub use catalog::{ApiId, ModelCapabilities, ModelInfo, ModelPricing, ProviderId, ServerToolInfo}; pub use chatgpt::{ @@ -27,7 +27,6 @@ pub use credentials::{ FileCredentialStore, InMemoryCredentialStore, SharedCredentialStore, }; pub use google::{GoogleBuilder, GoogleProvider}; -pub use local::{LocalApi, LocalModelEntry, LocalProvider, bind_local_model, list_local_models}; pub use model::{Model, ModelError, ModelOptions}; pub use oauth::{ OAuthConfig, OAuthError, SignInStart, Tokens as OAuthTokens, exchange_code, @@ -36,5 +35,4 @@ pub use oauth::{ }; pub use openrouter::{OpenRouterBuilder, OpenRouterProvider, Options as OpenRouterOptions}; pub use provider::{Auth, Provider, ProviderError, ProviderRegistry, bind_model}; -pub use store::LocalModelStore; pub use zai::{ZaiBuilder, ZaiProvider}; diff --git a/crates/providers/src/local.rs b/crates/providers/src/local.rs deleted file mode 100644 index 2653db7..0000000 --- a/crates/providers/src/local.rs +++ /dev/null @@ -1,597 +0,0 @@ -use crate::store::LocalModelStore; -use crate::{ - ApiId, ApiKeyAuth, Auth, AuthResolver, ModelInfo, ModelOptions, NoAuth, Provider, - ProviderError, ProviderId, ServerToolInfo, -}; -use async_trait::async_trait; -use llm::{ChatCompletionsApi, HttpClient, LlmApi}; -use serde::{Deserialize, Serialize}; -use std::collections::HashMap; -use std::sync::{Arc, LazyLock, RwLock}; - -#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)] -pub enum LocalApi { - ChatCompletions, -} - -#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)] -pub struct LocalModelEntry { - pub model_id: String, - pub url: String, - pub api: LocalApi, - pub api_key: Option, -} - -pub struct LocalProvider { - store: Arc, - entries: RwLock>, - models: RwLock>, -} - -impl LocalProvider { - pub fn new(store: Arc) -> Self { - Self { - store, - entries: RwLock::new(Vec::new()), - models: RwLock::new(Vec::new()), - } - } - - pub async fn load(&self) -> Result<(), ProviderError> { - let entries = self.store.load().await?; - let models = Self::rebuild(&entries); - *self.entries.write().unwrap_or_else(|e| e.into_inner()) = entries; - *self.models.write().unwrap_or_else(|e| e.into_inner()) = models; - Ok(()) - } - - pub async fn add_model(&self, entry: LocalModelEntry) -> Result<(), ProviderError> { - { - let mut entries = self.entries.write().unwrap_or_else(|e| e.into_inner()); - if entries - .iter() - .any(|existing| existing.model_id == entry.model_id) - { - return Err(ProviderError::Fetch(format!( - "local model already exists: {}", - entry.model_id - ))); - } - entries.push(entry); - let models = Self::rebuild(&entries); - *self.models.write().unwrap_or_else(|e| e.into_inner()) = models; - } - self.persist().await - } - - pub async fn remove_model(&self, model_id: &str) -> Result { - let removed; - { - let mut entries = self.entries.write().unwrap_or_else(|e| e.into_inner()); - let len_before = entries.len(); - entries.retain(|e| e.model_id != model_id); - removed = entries.len() < len_before; - let models = Self::rebuild(&entries); - *self.models.write().unwrap_or_else(|e| e.into_inner()) = models; - } - if removed { - self.persist().await?; - } - Ok(removed) - } - - pub async fn update_model(&self, entry: LocalModelEntry) -> Result<(), ProviderError> { - { - let mut entries = self.entries.write().unwrap_or_else(|e| e.into_inner()); - let Some(existing) = entries.iter_mut().find(|e| e.model_id == entry.model_id) else { - return Err(ProviderError::ModelNotFound(entry.model_id)); - }; - *existing = entry; - let models = Self::rebuild(&entries); - *self.models.write().unwrap_or_else(|e| e.into_inner()) = models; - } - self.persist().await - } - - pub fn find_entry(&self, model_id: &str) -> Option { - self.entries - .read() - .unwrap_or_else(|e| e.into_inner()) - .iter() - .find(|e| e.model_id == model_id) - .cloned() - } - - fn rebuild(entries: &[LocalModelEntry]) -> Vec { - entries - .iter() - .map(|e| ModelInfo { - provider: ProviderId::new("local"), - id: e.model_id.clone(), - name: e.model_id.clone(), - pricing: None, - context_length: None, - }) - .collect() - } - - async fn persist(&self) -> Result<(), ProviderError> { - let entries = self - .entries - .read() - .unwrap_or_else(|e| e.into_inner()) - .clone(); - self.store.save(&entries).await - } -} - -#[async_trait] -impl Provider for LocalProvider { - fn id(&self) -> ProviderId { - ProviderId::new("local") - } - - fn models(&self) -> Vec { - self.models - .read() - .unwrap_or_else(|e| e.into_inner()) - .clone() - } - - fn server_tools(&self) -> &[ServerToolInfo] { - &[] - } - - fn apis(&self) -> &HashMap> { - // Local models bind per-entry via `bind_local_model`, which reads the - // URL from the chosen `LocalModelEntry`. This provider-level entry is a - // placeholder so the catalog advertises chat-completions support. - static APIS: LazyLock>> = LazyLock::new(|| { - HashMap::from([( - ApiId::ChatCompletions, - Arc::new(ChatCompletionsApi::new("", Arc::new(HttpClient::new()))) - as Arc, - )]) - }); - &APIS - } - - async fn fetch_models(&self) -> Result<(), ProviderError> { - self.load().await - } -} - -#[async_trait] -impl Auth for LocalProvider { - fn auth_methods(&self) -> Vec { - vec![] - } - - fn auth(&self) -> Arc { - Arc::new(NoAuth) - } - - async fn validate_auth( - &self, - _auth_result: &crate::auth::AuthResult, - ) -> Result<(), crate::auth::AuthError> { - Ok(()) - } -} - -/// Bind a [`crate::Model`] to a local model entry, using the entry's URL and -/// optional API key. Local models are chat-completions compatible. -pub fn bind_local_model( - entry: &LocalModelEntry, - options: ModelOptions, -) -> Result { - let api: Arc = Arc::new(ChatCompletionsApi::new( - &entry.url, - Arc::new(HttpClient::new()), - )); - let auth: Arc = match entry.api_key.as_deref() { - Some(key) if !key.is_empty() => Arc::new(ApiKeyAuth::new(key)), - _ => Arc::new(NoAuth), - }; - let info = ModelInfo { - provider: ProviderId::new("local"), - id: entry.model_id.clone(), - name: entry.model_id.clone(), - pricing: None, - context_length: None, - }; - Ok(crate::Model::new_with_options(info, api, auth, options)) -} - -/// `GET {url}/models` on an OpenAI-compatible server. Sorted, deduped ids. -pub async fn list_local_models( - url: &str, - api_key: Option<&str>, -) -> Result, ProviderError> { - let mut request = reqwest::Client::new() - .get(format!("{}/models", url.trim().trim_end_matches('/'))) - .timeout(std::time::Duration::from_secs(10)); - if let Some(key) = api_key.filter(|key| !key.is_empty()) { - request = request.bearer_auth(key); - } - // reqwest's own message ("error sending request") hides why. - let response = request.send().await.map_err(|error| { - ProviderError::Fetch(if error.is_timeout() { - "timed out".into() - } else if error.is_connect() { - "could not connect".into() - } else if error.is_builder() { - "invalid URL".into() - } else { - format!("request failed: {error}") - }) - })?; - let status = response.status(); - if !status.is_success() { - return Err(ProviderError::Fetch(format!("HTTP {status}"))); - } - let payload = response - .text() - .await - .map_err(|error| ProviderError::Fetch(format!("request failed: {error}")))?; - parse_model_ids(&payload) -} - -fn parse_model_ids(payload: &str) -> Result, ProviderError> { - #[derive(Deserialize)] - struct Listing { - data: Vec, - } - - #[derive(Deserialize)] - struct Listed { - id: String, - } - - let listing: Listing = serde_json::from_str(payload) - .map_err(|_| ProviderError::Fetch("not an OpenAI-style model list".into()))?; - let mut ids: Vec = listing - .data - .into_iter() - .map(|model| model.id) - .filter(|id| !id.trim().is_empty()) - .collect(); - ids.sort(); - ids.dedup(); - if ids.is_empty() { - return Err(ProviderError::Fetch("server listed no models".into())); - } - Ok(ids) -} - -#[cfg(test)] -mod tests { - use super::*; - use crate::store::LocalModelStore; - - fn make_entry(model_id: &str) -> LocalModelEntry { - LocalModelEntry { - model_id: model_id.into(), - url: format!("http://localhost:8080/{model_id}"), - api: LocalApi::ChatCompletions, - api_key: None, - } - } - - #[derive(Default)] - struct MockStore { - entries: RwLock>, - } - - impl MockStore { - fn shared() -> Arc { - Arc::new(Self::default()) - } - } - - #[async_trait::async_trait] - impl LocalModelStore for MockStore { - async fn load(&self) -> Result, ProviderError> { - Ok(self - .entries - .read() - .unwrap_or_else(|e| e.into_inner()) - .clone()) - } - - async fn save(&self, entries: &[LocalModelEntry]) -> Result<(), ProviderError> { - *self.entries.write().unwrap_or_else(|e| e.into_inner()) = entries.to_vec(); - Ok(()) - } - } - - #[tokio::test] - async fn load_missing_file_returns_empty() { - let store = MockStore::shared(); - let provider = LocalProvider::new(store); - provider.load().await.unwrap(); - assert!(provider.models().is_empty()); - } - - #[tokio::test] - async fn add_model_appears_in_catalog() { - let store = MockStore::shared(); - let provider = LocalProvider::new(Arc::clone(&store)); - provider.add_model(make_entry("llama3")).await.unwrap(); - let models = provider.models(); - assert_eq!(models.len(), 1); - assert_eq!(models[0].id, "llama3"); - assert_eq!(models[0].provider, ProviderId::new("local")); - // Verify persistence via store - let persisted = store.load().await.unwrap(); - assert_eq!(persisted.len(), 1); - } - - #[tokio::test] - async fn add_multiple_models() { - let store = MockStore::shared(); - let provider = LocalProvider::new(Arc::clone(&store)); - provider.add_model(make_entry("a")).await.unwrap(); - provider.add_model(make_entry("b")).await.unwrap(); - assert_eq!(provider.models().len(), 2); - let persisted = store.load().await.unwrap(); - assert_eq!(persisted.len(), 2); - } - - #[tokio::test] - async fn remove_model() { - let store = MockStore::shared(); - let provider = LocalProvider::new(Arc::clone(&store)); - provider.add_model(make_entry("a")).await.unwrap(); - provider.add_model(make_entry("b")).await.unwrap(); - let removed = provider.remove_model("a").await.unwrap(); - assert!(removed); - assert_eq!(provider.models().len(), 1); - assert_eq!(provider.models()[0].id, "b"); - let persisted = store.load().await.unwrap(); - assert_eq!(persisted.len(), 1); - assert_eq!(persisted[0].model_id, "b"); - } - - #[tokio::test] - async fn remove_nonexistent_returns_false() { - let store = MockStore::shared(); - let provider = LocalProvider::new(store); - assert!(!provider.remove_model("nope").await.unwrap()); - } - - #[tokio::test] - async fn update_model() { - let store = MockStore::shared(); - let provider = LocalProvider::new(Arc::clone(&store)); - provider.add_model(make_entry("a")).await.unwrap(); - let updated = LocalModelEntry { - model_id: "a".into(), - url: "http://new-url:8080/v1".into(), - api: LocalApi::ChatCompletions, - api_key: Some("key".into()), - }; - provider.update_model(updated).await.unwrap(); - let found = provider.find_entry("a").unwrap(); - assert_eq!(found.url, "http://new-url:8080/v1"); - let persisted = store.load().await.unwrap(); - assert_eq!(persisted[0].url, "http://new-url:8080/v1"); - } - - #[tokio::test] - async fn find_entry_returns_none_for_missing() { - let store = MockStore::shared(); - let provider = LocalProvider::new(store); - assert!(provider.find_entry("nope").is_none()); - } - - #[tokio::test] - async fn find_entry_returns_correct_entry() { - let store = MockStore::shared(); - let provider = LocalProvider::new(Arc::clone(&store)); - provider.add_model(make_entry("x")).await.unwrap(); - let entry = provider.find_entry("x").unwrap(); - assert_eq!(entry.model_id, "x"); - } - - #[tokio::test] - async fn persists_to_disk_and_reloads() { - let store = MockStore::shared(); - { - let provider = LocalProvider::new(Arc::clone(&store)); - provider.add_model(make_entry("a")).await.unwrap(); - } - let provider = LocalProvider::new(Arc::clone(&store)); - provider.load().await.unwrap(); - assert_eq!(provider.models().len(), 1); - assert_eq!(provider.models()[0].id, "a"); - } - - #[tokio::test] - async fn remove_persists_to_disk() { - let store = MockStore::shared(); - { - let provider = LocalProvider::new(Arc::clone(&store)); - provider.add_model(make_entry("a")).await.unwrap(); - provider.add_model(make_entry("b")).await.unwrap(); - provider.remove_model("a").await.unwrap(); - } - let provider = LocalProvider::new(Arc::clone(&store)); - provider.load().await.unwrap(); - assert_eq!(provider.models().len(), 1); - assert_eq!(provider.models()[0].id, "b"); - } - - #[tokio::test] - async fn update_persists_to_disk() { - let store = MockStore::shared(); - { - let provider = LocalProvider::new(Arc::clone(&store)); - provider.add_model(make_entry("a")).await.unwrap(); - let updated = LocalModelEntry { - model_id: "a".into(), - url: "http://updated:9090".into(), - api: LocalApi::ChatCompletions, - api_key: None, - }; - provider.update_model(updated).await.unwrap(); - } - let provider = LocalProvider::new(Arc::clone(&store)); - provider.load().await.unwrap(); - let entry = provider.find_entry("a").unwrap(); - assert_eq!(entry.url, "http://updated:9090"); - assert_eq!(entry.api, LocalApi::ChatCompletions); - } - - #[tokio::test] - async fn fetch_models_populates_catalog() { - let store = MockStore::shared(); - { - let provider = LocalProvider::new(Arc::clone(&store)); - provider.add_model(make_entry("x")).await.unwrap(); - } - let provider = LocalProvider::new(Arc::clone(&store)); - provider.fetch_models().await.unwrap(); - assert_eq!(provider.models().len(), 1); - assert_eq!(provider.models()[0].id, "x"); - } - - #[tokio::test] - async fn bind_chat_completions() { - let entry = LocalModelEntry { - model_id: "llama3".into(), - url: "http://localhost:11434/v1".into(), - api: LocalApi::ChatCompletions, - api_key: None, - }; - let model = bind_local_model(&entry, ModelOptions::default()).unwrap(); - assert_eq!(model.info().provider, ProviderId::new("local")); - assert_eq!(model.info().id, "llama3"); - } - - #[tokio::test] - async fn bind_uses_entry_url() { - let entry = LocalModelEntry { - model_id: "gemini-local".into(), - url: "http://localhost:8080".into(), - api: LocalApi::ChatCompletions, - api_key: None, - }; - let model = bind_local_model(&entry, ModelOptions::default()).unwrap(); - assert_eq!(model.info().id, "gemini-local"); - assert_eq!(model.info().provider, ProviderId::new("local")); - } - - #[tokio::test] - async fn provider_id_is_local() { - let store = MockStore::shared(); - let provider = LocalProvider::new(store); - assert_eq!(provider.id(), ProviderId::new("local")); - } - - // one-shot server: replies `response`, hands back the raw request. - async fn serve_once(response: &'static str) -> (String, tokio::task::JoinHandle) { - use tokio::io::{AsyncReadExt, AsyncWriteExt}; - let listener = tokio::net::TcpListener::bind(("127.0.0.1", 0)) - .await - .unwrap(); - let address = listener.local_addr().unwrap(); - let server = tokio::spawn(async move { - let (mut socket, _) = listener.accept().await.unwrap(); - let mut request = Vec::new(); - let mut buffer = [0; 4096]; - while !request.windows(4).any(|window| window == b"\r\n\r\n") { - let read = socket.read(&mut buffer).await.unwrap(); - assert!(read > 0); - request.extend_from_slice(&buffer[..read]); - } - socket.write_all(response.as_bytes()).await.unwrap(); - String::from_utf8(request).unwrap() - }); - (format!("http://{address}/v1/"), server) - } - - #[test] - fn parse_model_ids_sorts_and_dedupes() { - let ids = parse_model_ids( - r#"{"data":[{"id":"b"},{"id":"a","owned_by":"x"},{"id":"b"},{"id":" "}]}"#, - ) - .unwrap(); - assert_eq!(ids, ["a", "b"]); - } - - #[test] - fn parse_model_ids_rejects_empty_and_junk() { - assert!(parse_model_ids(r#"{"data":[]}"#).is_err()); - assert!(parse_model_ids(r#"{"models":[{"name":"x"}]}"#).is_err()); - assert!(parse_model_ids("404").is_err()); - } - - #[tokio::test] - async fn list_local_models_sends_key_to_models_path() { - let (url, server) = serve_once( - "HTTP/1.1 200 OK\r\ncontent-type: application/json\r\nconnection: close\r\n\r\n{\"data\":[{\"id\":\"m1\"}]}", - ) - .await; - let ids = list_local_models(&url, Some("sk-test")).await.unwrap(); - assert_eq!(ids, ["m1"]); - let request = server.await.unwrap(); - assert!(request.starts_with("GET /v1/models HTTP/1.1"), "{request}"); - assert!( - request - .to_lowercase() - .contains("authorization: bearer sk-test") - ); - } - - #[tokio::test] - async fn list_local_models_skips_empty_key() { - let (url, server) = - serve_once("HTTP/1.1 200 OK\r\nconnection: close\r\n\r\n{\"data\":[{\"id\":\"m1\"}]}") - .await; - list_local_models(&url, Some("")).await.unwrap(); - assert!( - !server - .await - .unwrap() - .to_lowercase() - .contains("authorization") - ); - } - - #[tokio::test] - async fn list_local_models_reports_http_status() { - let (url, _server) = serve_once( - "HTTP/1.1 401 Unauthorized\r\nconnection: close\r\n\r\n{\"error\":\"Invalid API key\"}", - ) - .await; - let error = list_local_models(&url, Some("bad")).await.unwrap_err(); - assert!(error.to_string().contains("401"), "{error}"); - } - - #[tokio::test] - async fn list_local_models_reports_unreachable() { - // bind then drop: port is closed. - let address = std::net::TcpListener::bind(("127.0.0.1", 0)) - .unwrap() - .local_addr() - .unwrap(); - let error = list_local_models(&format!("http://{address}/v1"), None) - .await - .unwrap_err(); - assert!(error.to_string().contains("could not connect"), "{error}"); - } - - #[tokio::test] - async fn list_local_models_reports_invalid_url() { - let error = list_local_models("not a url", None).await.unwrap_err(); - assert!(error.to_string().contains("invalid URL"), "{error}"); - } - - #[tokio::test] - async fn empty_provider_has_no_models() { - let store = MockStore::shared(); - let provider = LocalProvider::new(store); - assert!(provider.models().is_empty()); - } -} diff --git a/crates/providers/src/provider.rs b/crates/providers/src/provider.rs index 60d6e04..efbead7 100644 --- a/crates/providers/src/provider.rs +++ b/crates/providers/src/provider.rs @@ -1,24 +1,23 @@ use crate::Model; use crate::auth::AuthResolver; use crate::catalog::{ApiId, ModelInfo, ProviderId, ServerToolInfo}; -use crate::local::LocalProvider; use crate::model::ModelOptions; use async_trait::async_trait; use llm::LlmApi; use std::collections::HashMap; -use std::sync::Arc; +use std::sync::{Arc, RwLock}; use thiserror::Error; #[derive(Debug, Error)] pub enum ProviderError { - #[error("model not found: {0}")] - ModelNotFound(String), #[error("API is not configured for model {model}")] MissingApi { model: String }, #[error("authentication is not configured")] MissingAuth, #[error("failed to fetch models: {0}")] Fetch(String), + #[error("{0:?} is not supported for custom providers yet")] + UnsupportedApi(ApiId), #[error("issued OAuth tokens did not contain a usable credential: {0}")] SignIn(String), } @@ -79,40 +78,49 @@ pub trait Provider: Auth + Send + Sync { } } +/// Every provider a model can bind to: the built-ins, then the custom +/// providers `/provider add` registers at runtime. pub struct ProviderRegistry { - providers: Vec>, - local_provider: Option>, + providers: RwLock>>, } impl ProviderRegistry { pub fn new(providers: impl IntoIterator>) -> Self { Self { - providers: providers.into_iter().collect(), - local_provider: None, + providers: RwLock::new(providers.into_iter().collect()), } } - pub fn with_local_provider( - providers: impl IntoIterator>, - local_provider: Arc, - ) -> Self { - Self { - providers: providers.into_iter().collect(), - local_provider: Some(local_provider), - } + pub fn providers(&self) -> Vec> { + self.providers + .read() + .unwrap_or_else(|e| e.into_inner()) + .clone() } - pub fn providers(&self) -> &[Arc] { - &self.providers + pub fn get(&self, id: &ProviderId) -> Option> { + self.providers + .read() + .unwrap_or_else(|e| e.into_inner()) + .iter() + .find(|p| p.id() == *id) + .cloned() } - pub fn get(&self, id: &ProviderId) -> Option> { - self.providers.iter().find(|p| p.id() == *id).cloned() + /// Add `provider`, or replace the one with its id in place. + pub fn insert(&self, provider: Arc) { + let mut providers = self.providers.write().unwrap_or_else(|e| e.into_inner()); + match providers.iter_mut().find(|p| p.id() == provider.id()) { + Some(saved) => *saved = provider, + None => providers.push(provider), + } } - /// Return an `Arc` clone of the local provider, if registered. - pub fn local(&self) -> Option> { - self.local_provider.clone() + pub fn remove(&self, id: &ProviderId) { + self.providers + .write() + .unwrap_or_else(|e| e.into_inner()) + .retain(|provider| provider.id() != *id); } } @@ -140,3 +148,51 @@ pub fn bind_model( options, )) } + +#[cfg(test)] +mod tests { + use super::*; + use crate::{CustomProvider, InMemoryCredentialStore}; + + fn provider(name: &str, url: &str) -> Arc { + Arc::new( + CustomProvider::new( + ProviderId::new(name), + ApiId::ChatCompletions, + url, + Arc::new(InMemoryCredentialStore::new()), + vec![ModelInfo::new(url, ProviderId::new(name))], + ) + .unwrap(), + ) + } + + fn ids_and_models(registry: &ProviderRegistry) -> Vec<(String, String)> { + registry + .providers() + .iter() + .map(|p| (p.id().0, p.models()[0].id.clone())) + .collect() + } + + #[test] + fn insert_replaces_a_provider_in_place_and_remove_drops_it() { + let registry = ProviderRegistry::new([provider("openrouter", "a")]); + registry.insert(provider("grok", "b")); + registry.insert(provider("groq", "c")); + + registry.insert(provider("grok", "d")); + assert_eq!( + ids_and_models(®istry), + [ + ("openrouter".into(), "a".into()), + ("grok".into(), "d".into()), + ("groq".into(), "c".into()) + ] + ); + + registry.remove(&ProviderId::new("grok")); + assert!(registry.get(&ProviderId::new("grok")).is_none()); + assert!(registry.get(&ProviderId::new("groq")).is_some()); + } +} diff --git a/crates/providers/src/store.rs b/crates/providers/src/store.rs deleted file mode 100644 index 1799356..0000000 --- a/crates/providers/src/store.rs +++ /dev/null @@ -1,15 +0,0 @@ -use crate::{LocalModelEntry, ProviderError}; - -/// Persists local-model entries to disk. -/// -/// `LocalProvider` holds a `Arc` and delegates all -/// file I/O here. The concrete implementation lives in the `alan` crate -/// (which owns the `fs2` file-lock pattern and `JsonStore`). -#[async_trait::async_trait] -pub trait LocalModelStore: Send + Sync { - /// Load entries from disk. Missing file => empty vec. - async fn load(&self) -> Result, ProviderError>; - - /// Overwrite the entire entry list on disk. - async fn save(&self, entries: &[LocalModelEntry]) -> Result<(), ProviderError>; -}