feat(elevenlabs): add ElevenLabs voice provider - #672
Conversation
Adds a voice-only provider exposing OpenAI-compatible text-to-speech (/v1/audio/speech) and speech-to-text (/v1/audio/transcriptions); chat, /v1/responses, and embeddings are not supported since ElevenLabs has no such APIs. Voice IDs pass through the OpenAI "voice" field directly, xi-api-key auth is used instead of Bearer, and the transcription model catalog (scribe_v1/scribe_v2) is merged in since ElevenLabs' /v1/models only lists TTS models. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughAdds ElevenLabs as a registered voice provider. It supports text-to-speech, speech-to-text, passthrough requests, model discovery, configuration, documentation, dashboard links, and comprehensive tests. ChangesElevenLabs provider
Estimated code review effort: 4 (Complex) | ~60 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 7
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@internal/providers/elevenlabs/audio_test.go`:
- Around line 68-127: Update TestCreateSpeech_MapsFormatsAndSpeed and
TestCreateSpeech_SupportsWAV into a table-driven
TestCreateSpeech_MapsResponseFormats covering default MP3, opus, pcm, and wav
query mappings, while preserving the speed assertion separately if needed. Add
TestCreateTranscription_WordGranularityFromRequest to verify CreateTranscription
sends TimestampGranularities []string{"word"} as timestamps_granularity=word,
using multipart request inspection.
In `@internal/providers/elevenlabs/audio.go`:
- Around line 40-50: Update speechSpeed to clamp nonzero speed values to
ElevenLabs’ supported 0.7–1.2 range instead of returning an error for
out-of-range OpenAI values; preserve the nil result for speed == 0 and return
the clamped value.
- Around line 179-185: Update the transcription request flow around
transcriptionMultipart and DoRaw to construct the multipart payload as buffered
bytes and assign it to RawBody instead of passing the pipe-backed RawBodyReader.
Preserve the existing content type and request endpoint, and propagate any
payload-construction error so retries can replay the same body safely.
In `@internal/providers/elevenlabs/elevenlabs_test.go`:
- Around line 110-128: Refactor
TestUnsupportedCapabilities_ReturnInvalidRequestErrors into a table-driven test
covering the five unsupported provider methods, with each case storing its
invocation and expected error substring. Iterate over the cases and apply one
shared assertion for the returned error while preserving the existing method
calls and message expectations.
In `@internal/providers/elevenlabs/elevenlabs.go`:
- Around line 47-58: Update the ElevenLabs New constructor to store the shared
keyring from opts.Keys on Provider instead of creating one with
opts.Keyring(cfg.APIKey). Preserve the existing llmclient.Config setup and
client initialization so key rotation uses the factory-provided keyring.
- Around line 32-43: Update the comment above Provider to accurately state that
it implements the audio and core.PassthroughProvider surfaces, while not
providing chat, Responses, or Embeddings endpoints; keep the existing type
assertions and Passthrough implementation unchanged.
In `@run/providers_test.go`:
- Around line 171-172: Add an ElevenLabs table entry to
TestDefaultProviderFactoryCredentialForms that verifies its credential fields,
required API key, default base URL, and provider-specific parameter mapping,
while matching the existing test-case structure and expectations.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: d5300ba2-4eba-48e9-9056-6a506b387da8
⛔ Files ignored due to path filters (2)
internal/admin/dashboard/static/dist/assets/index-D05Km9Si.jsis excluded by!**/dist/**internal/admin/dashboard/static/dist/index.htmlis excluded by!**/dist/**
📒 Files selected for processing (14)
.env.templateCLAUDE.mdREADME.mdconfig/config.example.yamldocs/docs.jsondocs/providers/elevenlabs.mdxdocs/providers/overview.mdxinternal/providers/elevenlabs/audio.gointernal/providers/elevenlabs/audio_test.gointernal/providers/elevenlabs/elevenlabs.gointernal/providers/elevenlabs/elevenlabs_test.gorun/providers.gorun/providers_test.goweb/dashboard/src/pages/overview/providersLogic.js
| body, contentType := transcriptionMultipart(req, model, content) | ||
| resp, err := p.client.DoRaw(ctx, llmclient.Request{ | ||
| Method: http.MethodPost, | ||
| Endpoint: "/v1/speech-to-text", | ||
| RawBodyReader: body, | ||
| Headers: http.Header{"Content-Type": {contentType}}, | ||
| }) |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -nP -C8 'RawBodyReader' internal/llmclient
rg -nP -C10 'func .*DoRaw\(' internal/llmclient
rg -nP -C6 'GetBody|retr(y|ies)|Attempt' internal/llmclient
# Do other providers stream multipart bodies through a pipe?
rg -nP -C4 'io\.Pipe\(\)' internal/providersRepository: ENTERPILOT/GoModel
Length of output: 50374
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== internal/providers/elevenlabs/audio.go relevant sections =="
wc -l internal/providers/elevenlabs/audio.go
sed -n '1,80p' internal/providers/elevenlabs/audio.go
sed -n '160,245p' internal/providers/elevenlabs/audio.go
sed -n '290,350p' internal/providers/elevenlabs/audio.go
echo "== llmclient request building raw reader refs =="
sed -n '120,170p' internal/llmclient/client.go
sed -n '376,420p' internal/llmclient/client.go
sed -n '680,755p' internal/llmclient/client.go
sed -n '1835,1900p' internal/llmclient/client_test.go
sed -n '402,428p' internal/llmclient/client_test.go
echo "== static verifier: Request.Body/RawBody/RawBodyReader usage and closing =="
python3 - <<'PY'
from pathlib import Path
import re
files = {p.name: p.read_text(errors='replace') for p in Path('internal').rglob('*.go')}
print("audio.go line counts with transcriptionMultipart caller:", {
"caller_line": next(i+1 for i,l in enumerate(files.get('audio.go','')) if 'transcriptionMultipart(req, model, content)' in l),
"reader_type": next(i+1 for i,l in enumerate(files.get('audio.go','')) if 'transcriptionMultipart(req, model, content)' in l and 'io.Reader' in files.get('audio.go',''.join(files['audio.go'].splitlines()[:i+5][1:])))
})
for name, text in files.items():
if 'RawBodyReader' in text or 'transcriptionMultipart' in text:
print(f"--- {name} ---")
for i,l in enumerate(text.splitlines(),1):
if 'RawBodyReader' in l or 'transcriptionMultipart' in l or 'Close()' in l:
print(f"{i}: {l}")
PYRepository: ENTERPILOT/GoModel
Length of output: 15298
Use a buffered multipart payload for transcription requests.
This pipe-backed body is not retry-safe, and if the build/circuit-breaker path rejects it, the producer can remain blocked until RawBodyReader is closed. Build the payload into RawBody so DoRaw can replay the same body on retry; build errors already close RawBodyReader, which removes this leak for streaming readers.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@internal/providers/elevenlabs/audio.go` around lines 179 - 185, Update the
transcription request flow around transcriptionMultipart and DoRaw to construct
the multipart payload as buffered bytes and assign it to RawBody instead of
passing the pipe-backed RawBodyReader. Preserve the existing content type and
request endpoint, and propagate any payload-construction error so retries can
replay the same body safely.
| func New(cfg providers.ProviderConfig, opts providers.ProviderOptions) core.Provider { | ||
| p := &Provider{keys: opts.Keyring(cfg.APIKey)} | ||
| clientCfg := llmclient.Config{ | ||
| ProviderName: "elevenlabs", | ||
| BaseURL: providers.ResolveBaseURL(cfg.BaseURL, defaultBaseURL), | ||
| Retry: opts.Resilience.Retry, | ||
| Hooks: opts.Hooks, | ||
| CircuitBreaker: opts.Resilience.CircuitBreaker, | ||
| } | ||
| p.client = llmclient.New(clientCfg, p.setHeaders) | ||
| return p | ||
| } |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Keyring/ProviderOptions definitions
ast-grep run --pattern 'func ($_ $_) Keyring($$$) $$$' --lang go internal/providers || true
rg -nP -C4 'func .*Keyring\(|func NewKeyring\(|SessionStickyKeys' internal/providers
# How other providers construct their keyring
rg -nP -C2 'opts\.Keyring\(' internal/providersRepository: ENTERPILOT/GoModel
Length of output: 31981
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== elevenlabs.go relevant sections =="
sed -n '1,260p' internal/providers/elevenlabs/elevenlabs.go | cat -n
echo
echo "== keyring.go relevant sections =="
sed -n '1,180p' internal/providers/keyring.go | cat -n
echo
echo "== factory.go relevant section =="
sed -n '1,170p' internal/providers/factory.go | cat -n
echo
echo "== ProviderConfig relevant section =="
sed -n '1,60p' internal/providers/config.go | cat -n
echo
echo "== config env/APIKs resolver section =="
sed -n '182,240p' internal/providers/config.go | cat -nRepository: ENTERPILOT/GoModel
Length of output: 28453
Use the factory keyring for ElevenLabs rotation.
New builds opts.Keyring(cfg.APIKey), so factory-created ElevenLabs providers use only the primary key instead of the shared keyring built from cfg.APIKeys in ProviderFactory.Create. Store opts.Keys on the provider (as sibling providers do for shared clients) instead of creating a single-key fallback.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@internal/providers/elevenlabs/elevenlabs.go` around lines 47 - 58, Update the
ElevenLabs New constructor to store the shared keyring from opts.Keys on
Provider instead of creating one with opts.Keyring(cfg.APIKey). Preserve the
existing llmclient.Config setup and client initialization so key rotation uses
the factory-provided keyring.
Source: Coding guidelines
Confidence Score: 4/5Not safe to merge until static Scribe transcription models remain available when live ElevenLabs model discovery fails. The failure was reproduced with a local HTTP server that returned a catalog error while successfully serving transcription. A comparison run that retained the static model inventory restored registry initialization and Scribe model resolution under the same upstream responses. Files Needing Attention:
 T-Rex LogsWhat T-Rex did
Comments Outside Diff (1)
Reviews (1): Last reviewed commit: "feat(elevenlabs): add ElevenLabs voice p..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| }, &upstream); err != nil { | ||
| return nil, err | ||
| } |
There was a problem hiding this comment.
When GET /v1/models is temporarily unavailable, ListModels returns the catalog error before appending the fixed Scribe models. Registry initialization therefore cannot resolve elevenlabs/scribe_v2, even though the independent /v1/speech-to-text endpoint remains healthy. Return the known static transcription inventory when catalog discovery fails so transcription remains available during a TTS catalog outage.
Artifacts
Focused httptest validation source for ElevenLabs catalog failure and transcription success
- Authored test source used one local httptest server to return HTTP 503 for the model catalog and HTTP 200 for transcription, showing whether model resolution remains available.
Current-code request results when ElevenLabs model catalog is unavailable
- Executed current-code focused test showing GET /v1/models returned HTTP 503 Service Unavailable while POST /v1/speech-to-text returned HTTP 200 OK, but scribe_v2 model resolution failed.
Overlay comparison results preserving Scribe models on catalog failure
- Executed the same focused test through a non-persistent ListModels fallback overlay, showing the same HTTP 503 catalog and HTTP 200 transcription responses now leave scribe_v2 resolvable.
ElevenLabs provider package test suite result
- Executed the existing ElevenLabs provider package suite after the focused validation cleanup, and it passed.
- ListModels: a live TTS catalog failure on the very first fetch now still returns the static Scribe transcription models (which don't depend on that call) instead of leaving the registry with nothing to resolve elevenlabs/scribe_v2 against. Once a fetch has succeeded, later failures propagate normally so the registry's existing stale-inventory carry-forward keeps the larger prior list instead of this call shrinking it (Greptile). - speechSpeed: clamp to ElevenLabs' 0.7-1.2 range instead of rejecting values OpenAI clients legitimately send (0.25-4.0), per Postel's Law (CodeRabbit). - Fix a stale doc comment claiming the provider doesn't implement core.PassthroughProvider when it does (CodeRabbit). - Add missing test coverage: pcm/word-granularity parameter mapping, wav response format, speed clamping, table-driven unsupported- capability checks, an elevenlabs credential-schema case in run/providers_test.go, and the new catalog-fallback behavior. Skipped two CodeRabbit suggestions after verification: buffering the transcription multipart body instead of streaming it through io.Pipe (llmclient.DoRaw already forces maxAttempts=1 for RawBodyReader and closes the reader on early failure, so it's neither a retry-replay risk nor a goroutine leak — the same pattern cohere's audio.go already uses), and storing opts.Keys directly instead of opts.Keyring(cfg.APIKey) (opts.Keyring already returns opts.Keys when the factory set it, only falling back to a single-key ring outside the factory — same pattern cohere.go uses). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Reviewed and verified each finding against the actual code/behavior; pushed fixes for the valid ones. Fixed:
Skipped after verification (false positives):
Also independently double-checked the original implementation against ElevenLabs' actual API surface (via their Fern-generated SDK source, not just prose docs) before this review pass and found two real bugs in the original PR body, already fixed in an earlier commit: |
… e2e testing
E2E-tested TTS, STT, and passthrough against the real ElevenLabs API
(round-tripped synthesized audio through transcription, exercised every
supported format/model, error paths, and speed clamping). Found and
fixed one real code bug along the way:
- CreateSpeech/CreateTranscription had a dead status-code check copied
from a Passthrough-style pattern: llmclient.Client.DoRaw already
parses any non-2xx response into an error before returning, so
`resp.StatusCode` can only ever be 200 by the time that check ran —
it never executed. Replaced it with refineElevenLabsError, which
unwraps ElevenLabs' actual error shape ({"detail": "..."} or
{"detail": {"message": "...", ...}}) from the GatewayError's
preserved ResponseBody, since the generic client-level parser only
recognizes {"message": ...}/{"error": {...}} and previously fell
back to dumping the raw JSON body as the message. Verified against
live 400/401/403 responses.
Documented two out-of-scope findings rather than fixing them here,
since both are shared-infrastructure issues, not ElevenLabs-specific
code:
- The provider-passthrough router's ALLOW_PASSTHROUGH_V1_ALIAS
handling strips a leading "v1/" path segment for every provider
uniformly, assuming the provider's own base URL already embeds
"/v1" (true for OpenAI-shaped providers). ElevenLabs' base URL does
not, so every /p/elevenlabs/v1/... passthrough call 404s today;
/v2/... paths are unaffected and verified working. This blocks
voice listing via the v1 endpoint and makes speech-to-speech
(voice changer) entirely unreachable, since it has no /v2 path.
Documented in docs/providers/elevenlabs.mdx with the exact
mechanism and impact.
- Audio endpoint audit log entries don't populate the top-level
`provider` field (confirmed against internal/server/audio_service.go,
which doesn't set it for any audio-capable provider) — a
pre-existing gap, not introduced here.
Also added the "Not implemented" section requested in review: speech-
to-speech, dubbing, voice cloning/design, projects, conversational
agents, and realtime streaming all have no OpenAI-compatible shape to
translate to and are candidates for native passthrough once the v1
alias issue above is fixed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
E2E test results against the live ElevenLabs APIRan the gateway locally with a real Found and fixed one real bug (pushed): Found a real bug outside this PR's scope (documented, not fixed here): the provider-passthrough router's Also noted (not fixed, pre-existing, confirmed via |
2 participants
Summary
elevenlabsprovider type: a voice-only provider exposing OpenAI-compatible/v1/audio/speech(text-to-speech) and/v1/audio/transcriptions(speech-to-text). Chat,/v1/responses, and embeddings returninvalid_request_errorsince ElevenLabs has no such APIs.voicefield carries the ElevenLabsvoice_iddirectly (ElevenLabs has no named voices); auth uses thexi-api-keyheader rather than Bearer.ListModelsmerges ElevenLabs' live TTS catalog (GET /v1/models, filtered tocan_do_text_to_speech) with the fixed speech-to-text model list (scribe_v2,scribe_v1), which is not included in that listing.mp3/opus/pcm/wavresponse formats andspeed0.7–1.2 (ElevenLabs' voice-setting range); transcription supportsjson/text/verbose_jsonwith word-level timestamps mapped from ElevenLabs'wordsarray.run/providers.go, dashboard help-icon doc link,.env.template,config/config.example.yaml, README, CLAUDE.md, and a new dedicateddocs/providers/elevenlabs.mdxguide (added to nav).Test plan
go build ./...go test ./...(full suite green, including newinternal/providers/elevenlabspackage tests)go vet ./internal/providers/elevenlabs/...node --test tests/*.test.js, 477 passing)🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Documentation