Repository navigation
Move OpenAI live tests off retiring GPT-4o models - #601
Merged
Merged
Conversation
This was referenced Oct 30, 2026
aksg87
marked this pull request as ready for review
November 2, 2026 08:06
aksg87
force-pushed
the
aksg87/live-tests-gpt-6-luna
branch
from
November 2, 2026 08:06
6f7b5d0 to
e59e95d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Summary
gpt-4o-2024-08-06,gpt-4o-2024-11-20andgpt-4o-mini-2024-07-18on January 14, 2027. Thegpt-4oandgpt-4o-minialiases point at those snapshots. The OpenAI live tests run onevery push to
mainand on same-repository PRs, so they would then failwith the same 404 that
o4-minireturned after its October 23 shutdown(fixed in Replace retired o4-mini in the live reasoning-effort test #594).
DEFAULT_OPENAI_MODELand the two inlinegpt-4o-miniuses (theschema-constraints test and the opt-in Batch test) to
gpt-6-luna.gpt-6-lunadefaults tomediumreasoning effort(model page)
and accepts
temperatureonly whenreasoning_effortisnone(GPT-6 guide).
OPENAI_MODEL_PARAMSnow sendsreasoning_effort: "none"withtemperature: 0.0, which keeps the runs cheap and close to deterministic,and
test_explicit_provider_selectionpasses the same pair. Theoutput-schema live test sends neither, so it exercises the model's default
effort.
max_completion_tokens),which have both merged.
Testing
How Has This Been Tested?
Tests
pytest -m 'not live_api and not requires_pip and not integration':993 passed, 25 deselected, the same as
main.pytest --co tests/test_live_api.pycollects all 16 tests.pylintis clean on thechanged files.
request sends
model="gpt-6-luna",reasoning_effort="none",temperature=0.0,max_completion_tokens=512and ajson_schemaresponseformat.
test_batch_extraction_uses_openai_batch_apiis opt-in(
LANGEXTRACT_RUN_OPENAI_BATCH_LIVE_TESTS=1) and CI does not set it, so theBatch API path on
gpt-6-lunais not verified by this PR's live run.tests, which CI has no credentials for).
Checklist
pylintover theaffected code.