Repository navigation
Conversation
…istry in test - Only remove wildcard path from openai_routes when the route entry has type="subpath", avoiding accidental removal when two endpoints share the same base path but differ in include_subpath - Clean up _registered_pass_through_routes in the test finally block to prevent stale entries from polluting subsequent tests on failure
…-call-type Fix/proxy only failure call type
…age-metadata fix(langsmith): populate usage_metadata in outputs for Cost column
…ssage-detection-performance Fix model repetition detection performance
fix: fix logging for response incomplete streaming + custom pricing on /v1/messages and /v1/responses
…idebar - Add 'Contributing to Guardrails' category with links to: - Generic Guardrail API (integrate without PR) - Adding a New Guardrail Integration tutorial - Adding Guardrail Support to Endpoints - Add 'Team Bring-Your-Own Guardrails' link for team BYOG workflow These docs existed but were only accessible from the 'LiteLLM AI Gateway' sidebar. Now they're also accessible when browsing the 'Guardrail Providers' section. Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
…ls-docs-143b docs: add Contributing to Guardrails section to Guardrail Providers sidebar
Adds `default_api_key_tpm_limit` and `default_api_key_rpm_limit` to `GenericLiteLLMParams` so operators can set per-deployment rate limit defaults in config.yaml. When a key has no model-specific tpm/rpm limit configured, the proxy falls back to these deployment defaults (Case 2 in spec). Key-level limits always take priority (Case 1). - Extends `get_key_model_tpm_limit` / `get_key_model_rpm_limit` with a `model_name` param and a priority-4 deployment-default fallback - Passes `model_name=requested_model` in the parallel request limiter so the fallback is triggered at enforcement time - Adds `"limit"` to `SensitiveDataMasker` non-sensitive overrides so `*_limit` fields are not masked in `/model/info` responses - Adds 17 unit tests covering both spec cases and the `/model/info` path Co-Authored-By: Claude (claude-sonnet-4-6) <noreply@anthropic.com>
…82.3 changelog Helicone (PRs BerriAI#19288, BerriAI#22603) and Langfuse (BerriAI#22390) were present in the v1.82.0-stable...v1.82.3-stable diff but omitted from the AI Integrations logging section. Also updates the AI Integrations diff summary count from 2 to 4. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Use min() across all matching deployments instead of first-wins when resolving default_api_key_tpm/rpm_limit for a model group, so load-balanced setups with different per-deployment limits always apply the most conservative value - Replace the global SensitiveDataMasker non_sensitive_overrides change with a targeted excluded_keys set at the remove_sensitive_info_from_deployment call site, avoiding unintended suppression of other fields - Update the v1 parallel request limiter to pass model_name to get_key_model_tpm/rpm_limit so deployment defaults apply there too - Add 4 tests covering multi-deployment min semantics Co-Authored-By: Claude (claude-sonnet-4-6) <noreply@anthropic.com>
Compute get_key_model_tpm/rpm_limit once before the guard condition instead of calling each function twice (once to check non-None, once to retrieve). Removes 2 extra llm_router.get_model_list() calls per request when deployment defaults are active. Co-Authored-By: Claude (claude-sonnet-4-6) <noreply@anthropic.com>
…ult limits async_log_success_event only updated the per-model cache counter when model_rpm_limit / model_tpm_limit were present in key metadata or model_max_budget was set. For the new deployment-default path (default_api_key_tpm_limit / default_api_key_rpm_limit), none of those conditions held, so current_tpm stayed at zero and tpm enforcement was never applied across multiple requests. Extend the guard condition to also trigger when the model group has a deployment-default tpm or rpm limit, and import the two helpers at module level. Co-Authored-By: Claude (claude-sonnet-4-6) <noreply@anthropic.com>
Replace bare _get_deployment_default_tpm/rpm_limit calls in the async_log_success_event condition with get_key_model_tpm/rpm_limit (model_name=model_group). The higher-level getters short-circuit on key/team metadata hits before ever reaching the router, so requests that don't use deployment defaults incur no extra router lookup. Remove the now-unused bare helper imports. Also fix invalid `int = None` type hints in test helper signatures to `Optional[int] = None`. Co-Authored-By: Claude (claude-sonnet-4-6) <noreply@anthropic.com>
…ures Full audit of 371 PRs in v1.82.0-stable...v1.82.3-stable range. Adds previously undocumented user-facing changes: - Key Highlights: Hashicorp Vault, Responses WebSocket, Org Admin RBAC, guardrail mode defaults - New Providers: Google Search API, Bedrock Mantle (7 total, was 5) - LLM API: Anthropic Files API, Mistral Voxtral transcription, WebRTC, Responses WebSocket, litellm.acount_tokens() public API, OpenRouter image edit, Vertex AI VIDEO token tracking, input_fidelity image edit, model cost aliases, per-request json schema validation, 15+ bug fixes - Management: RBAC expansion for Org Admins, Vector Store CRUD, MCP token auth + team scoping, BYOK key precedence, virtual key spend reset, batch expiry for teams, Admin Viewer audit log access, 12+ bug fixes - Guardrails: mode default list, tag-based modes, presidio fix, OTEL fix - Secret Managers: Hashicorp Vault (was "no changes") - Spend Tracking: new section — budget-linked reset fix, flex pricing, spend log cleanup, WebSearch dedup fix - Performance: 4 additional reliability fixes - Diff summary counts updated to reflect actual scope Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add MCP Gateway section (moved from Management per guide rule §11) - Rename Spend Tracking → Spend Tracking, Budgets and Rate Limiting - Fix Hashicorp Vault doc link: docs/secret → docs/secret_managers - Fix LLM API section: #### Bug Fixes → #### Bugs (matches guide) - Add Documentation Updates section (required by guide §11) - Update Diff Summary: correct section names, add MCP Gateway and Documentation Updates counts Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
BerriAI#23931) - Change chunk["id"] to chunk.get("id") for compatibility with MiniMax - ModelResponseStream auto-generates id when None is passed - Add regression test test_chunk_parser_without_id_field
Move pre-call checks (rate limits, guardrails, budget) to run BEFORE polling ID creation in the background streaming flow. This prevents the edge case where a rate-limited request receives a polling ID that immediately fails. Changes: - Add skip_pre_call_logic parameter to base_process_llm_request to allow skipping pre-call checks (avoiding double-counting of RPM/parallel requests) - Run common_processing_pre_call_logic before generating polling ID in the responses API endpoint. If rate limits/guardrails fail, return error immediately without creating a polling ID - Background streaming task passes skip_pre_call_logic=True to avoid re-running pre-call checks that were already done before polling ID creation - Add tests verifying skip_pre_call_logic parameter works correctly Fixes the edge case where polling_via_cache would return a polling ID for a request that immediately fails due to rate limiting.
- Guard logging_obj for None when skip_pre_call_logic=True: raise ValueError if litellm_logging_obj not in data, preventing AttributeError downstream - Add model=None to common_processing_pre_call_logic call in endpoints.py to match style of other call sites - Add test verifying rate-limited request never receives polling ID
…) directly Previously the test called common_processing_pre_call_logic in isolation, making generate_polling_id.assert_not_called() vacuously true. Now the test calls responses_api() end-to-end so it actually verifies that a rate-limited request never receives a polling ID. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Enable deployment_affinity, responses_api_deployment_check, and session_affinity to be configured per model group via router_settings.model_group_affinity_config, falling back to global settings for unconfigured groups. - Add model_group_affinity_config parameter to Router and DeploymentAffinityCheck - Add _get_effective_flags helper to resolve flags per model group - Update async_filter_deployments and async_pre_call_deployment_hook to use per-group config - Add 4 comprehensive tests covering per-group config, fallback, and override scenarios This allows fine-grained control of affinity behavior across model groups, e.g., enabling stickiness only for cross-provider deployments while leaving other groups free to load-balance. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
…n on unknown affinity flags Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
merge main
…fault Aligns proxy default with litellm.AZURE_DEFAULT_API_VERSION (2025-02-01-preview) so Azure response_format + json_schema works without tools fallback. Made-with: Cursor
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Addresses Greptile feedback that test assertions were weakened when removing summary: "detailed" expectations — now every default-behavior test explicitly asserts that "summary" is absent from the result. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…17_2026 fix(fireworks): skip #transform=inline for base64 data URLs (BerriAI#23729)
…-activity-entity-breakdown fix(proxy): restore per-entity breakdown in aggregated daily activity endpoint
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Extract multiline `or` chain from LiteLLM_AuditLogs constructor to fix pydantic mypy plugin field-type misattribution, and add explicit Optional[bool] annotation to avoid variable name shadowing conflict.
…05_2026 Litellm oss staging 03 05 2026
…arch_week Litellm dev sameer 16 march week
New docs page covering the HA control plane architecture where each worker instance has its own DB, Redis, and master key. Includes a React component diagram, setup configs, SSO notes, and local testing instructions.
litellm ryan march 20
[Infra] Build UI for release
…activity endpoint" This reverts commit 9c3fab2.
…hboard routes - OldTeams: refresh table via fetchTeamsV2 after team create instead of appending - TeamDropdown: rewrite with useInfiniteTeams for paginated fetch, scroll-to-load, and debounced search - Update all TeamDropdown consumers to use the new self-fetching API - Dashboard layout: switch from Sidebar2 to SidebarProvider (leftnav) - Leftnav: add MIGRATED_PAGES routing for path-based navigation (api-reference) - Navbar: remove chat button Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Use Select.Option with font-medium alias + Text secondary ID to match OrganizationDropdown - Default page size to 20 - Add useInfiniteTeams mock to AddModelForm tests Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
[Fix] UI - Teams: Table refresh, infinite dropdown, leftnav migration
Collaborator
Author
|
Closing duplicate PR; keeping #3 for the upstream sync. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
main1.82.6and the upstream responses/reasoning fixes landed before and around 2026-03-22Verification
git rev-list --left-right --count HEAD...upstream/main->0 0