Skip to content

fix(ai): gemini default model is dead for new API keys - #171

Merged
shossain786 merged 2 commits into
masterfrom
fix/gemini-default-model
Oct 19, 2026
Merged

shossain786 merged 2 commits into
masterfrom
fix/gemini-default-model

Conversation

@shossain786

Copy link
Copy Markdown
Contributor

Summary

  • Live-checked ai.runSummary against a real Gemini key (the one thing feat(ai): opt-in AI run summary on the HTML dashboard #168 left unverified) and found the default model, gemini-2.5-flash, returns HTTP 404 — Google retired it for new API keys, pointing callers at gemini-3.8-flash instead.
  • Any ai.provider: gemini config that doesn't set an explicit ai.model is broken out of the box today. Switched the default and the docs table to gemini-3.8-flash.

Verification

  • curl directly against the Gemini endpoint: gemini-2.5-flash → 404 before, gemini-3.8-flash → 200 after.
  • End-to-end through the actual framework code path (AiRunSummarizer.buildPrompt → AiFailureAnalyzer.callApi → response parsing) against the real API with a realistic two-failure run summary prompt — returned a correct plain-English summary.
  • Note for anyone tuning ai.timeoutSeconds with Gemini: observed latency for this model was highly variable in testing (as low as ~5s, as high as ~95s for the same prompt), driven by the model's internal "thinking" tokens. The default ai.timeoutSeconds: 20 may be too tight for Gemini in practice — not changed here since it's a latency/cost tradeoff for the user to tune, not a correctness bug, but worth knowing.
  • mvn test: 740/740 green.

Test plan

  • mvn test passes
  • Live API call confirms the old default 404s and the new default works
  • Live end-to-end call through the actual AiRunSummarizer prompt-building and AiFailureAnalyzer HTTP/parsing code

🤖 Generated with Claude Code

shossain786 and others added 2 commits October 19, 2026 08:15
Live-checking ai.runSummary with a real Gemini key (per #168 follow-up)
found the default gemini-2.5-flash returns HTTP 404 "no longer available
to new users" — every ai.provider: gemini config without an explicit
ai.model is broken out of the box. Switch the default to gemini-3.8-flash,
confirmed against the real API (prompt -> HTTP call -> response parse all
verified end-to-end, including a realistic run-summary prompt).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Pinned Gemini model names get retired (gemini-2.5-flash already was); the
-latest alias does not. Confirmed against the real Gemini API. Adds a test
pinning the default endpoint.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@shossain786
shossain786 merged commit e142d85 into master Oct 19, 2026
2 checks passed
@shossain786
shossain786 deleted the fix/gemini-default-model branch October 19, 2026 04:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant