Skip to content

litellm.md

Latest commit

 

History

History
78 lines (58 loc) · 2.82 KB

litellm.md

File metadata and controls

78 lines (58 loc) · 2.82 KB
Raw

Use LiteLLM with CometAPI

Canonical docs: https://apidoc.cometapi.com/integrations/litellm

Use this guide to configure LiteLLM with CometAPI by setting the base URL, API key, and model or provider options. LiteLLM provides a unified Python API for 100+ LLM providers. CometAPI is natively supported — use the cometapi/ prefix to route requests through CometAPI's model catalog.

Prerequisites

  • Python 3.6+
  • A CometAPI account with an active API key — get yours here

Install LiteLLM

pip install litellm

Set your API key

Set the API key as an environment variable (recommended) or pass it inline:

import os
from litellm import completion

# Recommended: environment variable
os.environ["COMETAPI_KEY"] = "<COMETAPI_KEY>"

# Alternative: pass inline
api_key = "<COMETAPI_KEY>"

Note Use environment variables to avoid hardcoding sensitive credentials in your scripts.

Make a completion call

Use the cometapi/<model-name> format to specify models. You can pass the key via environment variable or explicitly:

messages = [{"content": "Hello, how are you?", "role": "user"}]

# Method 1: environment variable (recommended)
response = completion(model="cometapi/your-model-id", messages=messages)

# Method 2: explicit API key
response = completion(model="cometapi/your-model-id", messages=messages, api_key=api_key)

print(response.choices[0].message.content)

Async and streaming calls

Use acompletion with stream=True for non-blocking, real-time responses:

from litellm import acompletion
import asyncio, traceback

async def stream_call():
    try:
        response = await acompletion(
      model="cometapi/your-model-id",
            messages=[{"content": "Hello, how are you?", "role": "user"}],
            stream=True,
        )
        async for chunk in response:
            print(chunk)
    except Exception:
        print(f"Error: {traceback.format_exc()}")

asyncio.run(stream_call())

Tips and troubleshooting

  • Model format: CometAPI models use the prefix cometapi/<model-name>, e.g. cometapi/your-model-id. See the CometAPI Models page for available models.
  • Fine-tuning responses: LiteLLM supports temperature, max_tokens, and top_p — add them to any completion() call, e.g. completion(..., temperature=0.7).
  • Error handling: Wrap calls in try/except to catch invalid key errors or network issues.
  • Security: Never commit API keys to version control. Use environment variables or a secrets manager.
  • Rate limits: Monitor usage in the CometAPI console.
  • More docs: LiteLLM documentation — CometAPI quick start