Skip to content

Send max_output_tokens as max_completion_tokens to OpenAI - #600

Open
aksg87 wants to merge 2 commits into
mainfrom
aksg87/openai-max-completion-tokens
Open

aksg87 wants to merge 2 commits into
mainfrom
aksg87/openai-max-completion-tokens

Conversation

@aksg87

@aksg87 aksg87 commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Summary

  • The OpenAI provider sends max_output_tokens as max_tokens. OpenAI
    deprecated that parameter in favor of max_completion_tokens and documents
    it as incompatible with o-series models
    (Chat Completions reference).
    GPT-5 reasoning models reject it as well
    (openai/openai-dotnet#737).
    Callers who set max_output_tokens with those models get a 400 today.
  • The provider now sends max_completion_tokens, the documented replacement,
    when requests go to OpenAI's API: no base_url, https://api.openai.com,
    or a regional *.api.openai.com host. It keeps max_tokens for any other
    base_url, because some OpenAI-compatible servers accept only the older
    name. OPENAI_BASE_URL is checked too, since the OpenAI SDK falls back to
    it when base_url is unset. The Batch API path uses the same request
    builder.
  • Rewrites the skill doc caveat that told users to omit max_output_tokens
    for o-series models. It still applies to reasoning models behind a custom
    base_url, such as Azure OpenAI deployments.
  • Stacked on Route gpt-6 model IDs to the OpenAI provider #599, which rewraps the same skill-doc paragraph. Merge that
    first; this branch then rebases to one commit.

Bug fix

How Has This Been Tested?

Tests

  • New: test_max_output_tokens_request_parameter with openai_api,
    custom_base_url, openai_base_url_env, explicit_openai_base_url,
    regional_openai_base_url and lookalike_openai_host cases. The openai_api case fails on main
    (KeyError: 'max_completion_tokens').
  • pytest -m 'not live_api and not requires_pip and not integration':
    991 passed, 25 deselected.
  • isort 5.13.2 and pyink 24.3.0 checks are clean; pylint is clean on the
    changed files.
  • Live: in CI, the only OpenAI live test that sets max_output_tokens is
    test_medication_extraction_with_schema_constraints (512). On this PR it sends max_completion_tokens to
    gpt-4o-mini; on Move OpenAI live tests off retiring GPT-4o models #601 it sends it to gpt-6-luna. The opt-in Batch
    live test is skipped in CI. Pending; added once this run finishes.

Checklist

  • I have made any needed documentation changes.
  • I have added tests, or I have ensured existing tests cover the changes.
  • I have followed Google's Python Style Guide and ran pylint over the
    affected code.

@github-actions github-actions Bot added the size/S Pull request with 50-150 lines changed label Oct 30, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/S Pull request with 50-150 lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant