Skip to content

Integration request from aimlapi.com - #424

Closed
hugoaimlapi wants to merge 2 commits into
vxcontrol:mainfrom
aimlapi:pr/aimlapi-provider
Closed

hugoaimlapi wants to merge 2 commits into
vxcontrol:mainfrom
aimlapi:pr/aimlapi-provider

Conversation

@hugoaimlapi

Copy link
Copy Markdown

Hi! I'm Hugo from aimlapi.com — an AI aggregator that gives access to 1000+ models in one API, trusted by 400k+ users.

We'd love to be available as a verified provider option inside PentAGI — so we went ahead and did all the technical work on our side, in our fork: https://github.com/aimlapi/pentagi-aimlapi

To build our partnership, we offer a 50/50 revenue share on all traffic from this integration.

My contacts: hugo@aimlapi.com (email / Slack), Telegram: @hug0the


Description of the Change

Problem

aimlapi.com is an OpenAI-compatible gateway in front of many vendors' chat models. Today the only way to use it is the generic custom provider (LLM_SERVER_URL/LLM_SERVER_KEY plus a hand-written LLM_SERVER_CONFIG_PATH for per-role models): no built-in role defaults, no model catalog in Settings, no installer screen, and it takes the single custom slot. Details in #417.

Solution

A first-class aimlapi provider on the existing openaicompat base. No new client code, no new dependencies.

  • backend/pkg/providers/aimlapi/: aimlapi.go (fallback model deepseek/deepseek-v4-flash), embedded config.yml with defaults for all 13 agent roles, and models.yml with a 6-model catalog and prices.
    • Roles: DeepSeek V4 Flash (utility roles, enricher, pentester), GLM 5 Turbo (primary_agent, assistant), GLM 5.2 at effort: max (generator, refiner), MiniMax M3 (adviser), Kimi K2.7 Code (coder, installer).
    • OpenAI/Anthropic/Google models are left out for the same reason as in examples/configs/openrouter.provider.yml.
    • The enricher stays on deepseek-v4-flash because minimax-m3 ignores "thinking off" through this gateway.
  • Env vars in config.go, .env.example and docker-compose.yml: AIMLAPI_API_KEY, AIMLAPI_SERVER_URL (default https://api.aimlapi.com/v1), AIMLAPI_PROVIDER (LiteLLM prefix). The key is added to GetSecretPatterns().
  • Registration: ProviderAIMLAPI in provider.go (including AllProviderTypes), an entry in registry.go, ProviderTypeAimlapi in database/models.go.
  • Migration 20260903_120000_add_aimlapi_provider.sql adds aimlapi to PROVIDER_TYPE, same pattern as the MiniMax one. Down deletes aimlapi rows before restoring the enum.
  • GraphQL: aimlapi in ProviderType, ProvidersModelsList, ProvidersReadinessStatus and DefaultProvidersConfig, wired in SettingsProviders. Frontend types regenerated; new icon, label and e2e cassette entries.
  • Installer: llm_provider_form§aimlapi screen (base URL, key, provider name) with help text; the key is masked.
  • ctester -type aimlapi and ftester -provider aimlapi.

Attribution: attribution.go wraps a copy of the shared HTTP client (the global one isn't modified) and adds these headers to requests going to aimlapi.com:

HTTP-Referer: https://github.com/vxcontrol/pentagi
X-Title: PentAGI
X-AIMLAPI-Source: agent/pentagi
X-AIMLAPI-Partner-ID: part_6bffrRIYBS8OtYbQhsEPi0SS

They tell aimlapi.com the traffic comes from PentAGI; the partner ID is what the revenue share is counted on. No user data in them.

  • Sent only when the AIMLAPI_SERVER_URL host is aimlapi.com or a subdomain, re-checked on every request, so they don't follow a redirect to another host.
  • A header already present on the request is never overwritten.
  • Pointing AIMLAPI_SERVER_URL at a proxy or self-hosted gateway turns them off. Otherwise removing them means dropping the withAttribution(...) wrapper in aimlapi.New.

Closes #417

Type of Change

  • 🐛 Bug fix (non-breaking change which fixes an issue)
  • 🚀 New feature (non-breaking change which adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 📚 Documentation update
  • 🔧 Configuration change
  • 🧪 Test update
  • 🛡️ Security update

Areas Affected

  • Core Services (Frontend UI/Backend API)
  • AI Agents (Researcher/Developer/Executor)
  • Security Tools Integration
  • Memory System (Vector Store/Knowledge Base)
  • Monitoring Stack (Grafana/OpenTelemetry)
  • Analytics Platform (Langfuse)
  • External Integrations (LLM/Search APIs)
  • Documentation
  • Infrastructure/DevOps

Testing and Verification

Test Configuration

PentAGI Version: main @ ea66530 + this branch
Docker Version: n/a (backend tests and ctester run outside Docker)
Host OS: not recorded
LLM Provider: aimlapi (AIMLAPI_SERVER_URL=https://api.aimlapi.com/v1)
Enabled Features: n/a

Test Steps

  1. cd backend && go test ./pkg/providers/aimlapi/ ./pkg/providers/ ./pkg/config/ ./pkg/server/models/
  2. With AIMLAPI_API_KEY set: go run cmd/ctester/*.go -type aimlapi -report ../examples/tests/aimlapi-report.md
  3. Settings → Providers: aimlapi.com is listed and a provider can be created with the default role models. The installer shows the aimlapi.com screen.

Test Results

New unit tests in backend/pkg/providers/aimlapi/aimlapi_test.go: role models present in models.yml, partner-id format, headers only for the aimlapi.com host and never over caller headers, shared client not mutated, unset top_p/seed and cleared tools omitted from the JSON body instead of sent as null (the gateway answers 400 to null). The aimlapi rows were added to the existing tables in providers_test.go, config_test.go and server/models/providers_test.go.

ctester against the live API (2026-09-03) is committed as examples/tests/aimlapi-report.md: 292/299 (97.66%). The 7 failures:

  • generate_report not called in "Penetration Testing Memory with Tool Call": simple, searcher, enricher (deepseek-v4-flash).
  • Empty or wrong diff in "Read a file, then edit it via unified diff": primary_agent, assistant (glm-5-turbo), adviser (minimax-m3).
  • One 504 on assistant.

All role models are in the live catalog.

Security Considerations

  • AIMLAPI_API_KEY is masked in the installer and added to GetSecretPatterns().
  • The only additions to requests are the attribution headers above; they go only to the aimlapi.com host and are not forwarded on a redirect to another host.
  • No new dependencies.

Performance Impact

None for existing providers. The aimlapi transport adds four headers to requests going to aimlapi.com.

Documentation Updates

  • README.md updates
  • API documentation updates
  • Configuration documentation updates
  • GraphQL schema updates
  • Other: backend/docs/config.md, backend/docs/database.md, installer help text

Deployment Notes

  • New optional env vars: AIMLAPI_API_KEY, AIMLAPI_SERVER_URL, AIMLAPI_PROVIDER. Nothing changes unless AIMLAPI_API_KEY is set.
  • Includes a goose migration that adds aimlapi to PROVIDER_TYPE. Rolling back deletes aimlapi providers, flows and assistants.

Checklist

Code Quality

  • My code follows the project's coding standards
  • I have added/updated necessary documentation
  • I have added tests to cover my changes
  • All new and existing tests pass
  • I have run go fmt and go vet (for Go code)
  • I have run pnpm run lint (for TypeScript/JavaScript code)

Security

  • I have considered security implications
  • Changes maintain or improve the security model
  • Sensitive information has been properly handled

Compatibility

  • Changes are backward compatible
  • Breaking changes are clearly marked and documented
  • Dependencies are properly updated

Documentation

  • Documentation is clear and complete
  • Comments are added for non-obvious code
  • API changes are documented

Additional Notes

Other aggregators here ship as examples/configs/*.provider.yml for the custom provider. This one is first-class so it gets its own Settings/installer entry and per-role defaults, and can run next to a separate custom endpoint. If you'd rather have just an example config, I can cut it down to that.

Lookoff-AIMLAPI and others added 2 commits September 3, 2026 06:11
PentAGI already advertises an aggregator tier next to its per-vendor
providers, but reaching aimlapi.com meant configuring the generic `custom`
provider by hand: one endpoint, one key, no model catalog, no per-agent
defaults and no entry in the settings UI. Since the gateway is
OpenAI-compatible it fits the existing openaicompat base exactly, so making
it first-class costs a registry entry and a config rather than a new client.

The default roster spreads the 13 agent roles over four model families and
deliberately skips the OpenAI/Anthropic/Google models the gateway also
carries: this product runs offensive-security workloads and those vendors'
guardrails false-positive on legitimate exploit-development content. That is
the same reasoning already recorded in examples/configs/openrouter.provider.yml,
and the role/model mapping mirrors it so a reviewer can compare them directly.

The enricher is the one place the mapping diverges. Through this gateway
minimax/minimax-m3 ignores both reasoning_effort:"none" and
extra_body.thinking.type:"disabled" and still thinks, returning the chain
inline in `content` wrapped in <think> tags instead of in reasoning_content;
the enricher is the role that explicitly wants thinking off, so it stays on
deepseek-v4-flash, which honours it.

Requests to api.aimlapi.com carry HTTP-Referer, X-Title, X-AIMLAPI-Source and
X-AIMLAPI-Partner-ID identifying PentAGI as the calling application. They are
attached by a RoundTripper that wraps a copy of the shared HTTP client, so the
process-wide client is never mutated, a caller's own header always wins, and
the host is re-checked per request — pointing AIMLAPI_SERVER_URL at a LiteLLM
proxy or a self-hosted gateway disables attribution rather than tagging
another operator's traffic through a redirect.

Two regression tests exist because the failures they catch are invisible
otherwise: a malformed partner id is accepted by the gateway and silently
earns nothing, and the gateway rejects `null` for temperature, top_p, seed,
tools and most other optional fields with a 400 while accepting them absent —
so a client that serialises cleared tools as null passes turn one of an agent
loop and fails every turn two.
Signed-off-by: Hugo <hugo@aimlapi.com>
@sirozha

sirozha commented Sep 23, 2026

Copy link
Copy Markdown
Collaborator

Thanks for the work. We are not taking this: the provider duplicates what the custom OpenAI-compatible endpoint already covers, and we cannot carry vendor attribution headers in the product. Closing this PR.

@sirozha sirozha closed this Sep 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Enhancement]: Add aimlapi.com as a first-class LLM provider

3 participants