Prune the AI assistant model list to 15 models - #719
Merged
Conversation
Keep the frontier models and one older model per lab, keep Sonnet 4.6 as the Goldilocks default with Gemini 3.5 Flash as the cheap alternative, and drop Sonnet 5. Labs whose models all failed the 2026-09-07 smoke sweep (xAI, MiniMax, Mistral, Meta) leave the list entirely. The ModelConfiguration cards stay in the realm; only the system card's list changes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Staging Submissions PreviewThis PR's content is pushed to the staging submissions realm: https://realms-staging.stack.cards/submissions/ Changed folders:
Updated at 2026-09-10 11:53:05 UTC for commit |
New ModelConfiguration card pointing at the OpenRouter card, listed after GPT-5.6 Terra. No reasoning effort is set, the same as Terra. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
jurgenwerk
changed the title
xAI keeps its latest model and one older one, the same rule as the other labs. Grok 4.6 gets a new ModelConfiguration card pointing at the OpenRouter card; Grok 4.5 already had one. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
jurgenwerk
changed the title
Z.ai leaves the list. The ModelConfiguration cards stay in the realm. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
jurgenwerk
changed the title
jurgenwerk
marked this pull request as ready for review
September 10, 2026 12:10
jurgenwerk
changed the title
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
The PR metadata/title indicates 16 models, but the updated modelConfigurations list contains 15 entries, so the PR intent and implementation need to be aligned.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR updates the default system configuration to reduce the set of selectable AI assistant models, removing models that are failing basic evals and adding two new model configurations (Grok 4.6 and GPT‑5.6 Luna).
Changes:
- Pruned
SystemCard/default.jsonmodel configuration list down to the intended smaller set. - Added
ModelConfigurationcards for xAI: Grok 4.6 and OpenAI: GPT‑5.6 Luna. - Removed references to dropped labs/models from the default configuration.
File summaries
| File | Description |
|---|---|
| SystemCard/default.json | Updates the default system’s modelConfigurations list to the reduced set and includes the new Grok 4.6 + GPT‑5.6 Luna entries. |
| ModelConfiguration/x-ai-grok-46.json | Adds the new Grok 4.6 model configuration card pointing to the OpenRouter model id. |
| ModelConfiguration/openai-gpt-56-luna.json | Adds the new GPT‑5.6 Luna model configuration card pointing to the OpenRouter model id. |
Review details
- Files reviewed: 3/3 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
jurgenwerk
changed the title
habdelra
approved these changes
Sep 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In the last meeting with Chris where I presented the initial model eval results in our AI assistant, we discussed we want to reduce the set of models we have in our AI assistant. The reason is that some of them are failing our basic eval, and we want to reduce the set for simplicity.
We discussed we want to keep the following models:
And we want to add the following model, due to its efficiency and price: