Skip to content

Move OpenAI live tests off retiring GPT-4o models - #601

Merged
aksg87 merged 1 commit into
mainfrom
aksg87/live-tests-gpt-6-luna
Nov 2, 2026
Merged

aksg87 merged 1 commit into
mainfrom
aksg87/live-tests-gpt-6-luna

Conversation

@aksg87

@aksg87 aksg87 commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Summary

  • OpenAI shuts down gpt-4o-2024-08-06, gpt-4o-2024-11-20 and
    gpt-4o-mini-2024-07-18 on January 14, 2027. The gpt-4o and
    gpt-4o-mini aliases point at those snapshots. The OpenAI live tests run on
    every push to main and on same-repository PRs, so they would then fail
    with the same 404 that o4-mini returned after its October 23 shutdown
    (fixed in Replace retired o4-mini in the live reasoning-effort test #594).
  • Moves DEFAULT_OPENAI_MODEL and the two inline gpt-4o-mini uses (the
    schema-constraints test and the opt-in Batch test) to gpt-6-luna.
  • gpt-6-luna defaults to medium reasoning effort
    (model page)
    and accepts temperature only when reasoning_effort is none
    (GPT-6 guide).
    OPENAI_MODEL_PARAMS now sends reasoning_effort: "none" with
    temperature: 0.0, which keeps the runs cheap and close to deterministic,
    and test_explicit_provider_selection passes the same pair. The
    output-schema live test sends neither, so it exercises the model's default
    effort.
  • Test-only. It needs Route gpt-6 model IDs to the OpenAI provider #599 (routing) and Send max_output_tokens as max_completion_tokens to OpenAI #600 (max_completion_tokens),
    which have both merged.

Testing

How Has This Been Tested?

Tests

  • pytest -m 'not live_api and not requires_pip and not integration':
    993 passed, 25 deselected, the same as main.
  • pytest --co tests/test_live_api.py collects all 16 tests.
  • isort 5.13.2 and pyink 24.3.0 checks are clean; pylint is clean on the
    changed files.
  • With a mocked OpenAI client, the schema-constraints
    request sends model="gpt-6-luna", reasoning_effort="none",
    temperature=0.0, max_completion_tokens=512 and a json_schema response
    format.
  • Skipped in CI: test_batch_extraction_uses_openai_batch_api is opt-in
    (LANGEXTRACT_RUN_OPENAI_BATCH_LIVE_TESTS=1) and CI does not set it, so the
    Batch API path on gpt-6-luna is not verified by this PR's live run.
  • Live results: 12 passed, 4 skipped (the Batch test above and 3 Vertex AI
    tests, which CI has no credentials for).

Checklist

  • I have followed Google's Python Style Guide and ran pylint over the
    affected code.

@aksg87
aksg87 marked this pull request as ready for review November 2, 2026 08:06
@aksg87
aksg87 force-pushed the aksg87/live-tests-gpt-6-luna branch from 6f7b5d0 to e59e95d Compare November 2, 2026 08:06
@github-actions github-actions Bot added size/S Pull request with 50-150 lines changed size/XS Pull request with less than 50 lines changed and removed size/S Pull request with 50-150 lines changed labels Nov 2, 2026
@aksg87
aksg87 merged commit 7548803 into main Nov 2, 2026
25 checks passed
@aksg87
aksg87 deleted the aksg87/live-tests-gpt-6-luna branch November 2, 2026 08:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/XS Pull request with less than 50 lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant