Skip to content

feat(models): add input-modality overrides for catalog-gap vision models - #92

Closed
eibednejo wants to merge 1 commit into
patlux:mainfrom
eibednejo:feat/input-modalities-overrides
Closed

eibednejo wants to merge 1 commit into
patlux:mainfrom
eibednejo:feat/input-modalities-overrides

Conversation

@eibednejo

Copy link
Copy Markdown

Related to #90 — same root cause as #91, the other capability. #91 covers the missing reasoning flag; this covers missing input modalities.

Problem

src/commandcode-catalog.ts is generated from a single Command Code CLI release. A model published afterwards is absent from MODEL_INPUT_MODALITIES, so inputModalitiesForModel() falls back to ["text"] — and both transports honor that absence:

  • index.ts:127 publishes input: [...inputModalitiesForModel(model.id)] to pi and Oh My Pi, so the host never offers image attachments for the model.
  • src/core.ts:543 calls assertTextOnlyMessages() on the generate transport, so an image in a user message fails the request outright with Selected Command Code model does not support image content in user messages.

commandcode-catalog-overrides.ts only exposed MODEL_EFFORT_OVERRIDES, so there was no supported way to restore image input for a catalog-gap model.

Evidence

command-code@1.53.0 (dist/cli.mjs) declares image input for both models:

id:"deepseek/deepseek-v4.1-flash",inputModalities:["text","image"],,description:"V4.1 hybrid-attention reasoning with vision",
id:"xai/grok-4.6",inputModalities:["text","image"],,reasoning:!0,

The generated catalog (from command-code@1.44.0) has no entry for deepseek/deepseek-v4.1-flash at all, and xai/grok-4.6 appears only in MODEL_REASONING and MODEL_EFFORTS — neither is in MODEL_INPUT_MODALITIES. Both ids are served by GET /provider/v1/models.

Change

  • src/commandcode-catalog-overrides.ts — add MODEL_INPUT_MODALITIES_OVERRIDES with its own doc block (when to add, when to drop) and the two entries.
  • src/models.ts — merge the overrides over CATALOG_MODEL_INPUT_MODALITIES, mirroring how MODEL_EFFORTS already merges MODEL_EFFORT_OVERRIDES.
  • tests/test-models.ts — test mirroring the existing effort-override test: each override must not be upstream yet (drop signal), must include image, must reach inputModalitiesForModel()/modelSupportsImageInput(), and generated entries must still pass through unchanged.
  • CHANGELOG.md — Unreleased entry.

Verification

  • npm run test:unit217/217 pass; typecheck and prettier --check clean.
  • Live on a Go subscription, with the generated catalog left untouched and no user-level models.json override in play: deepseek/deepseek-v4.1-flash correctly described a test image (red circle, blue square, green triangle, text ZEBRA-42). Before this change the same request fails with the text-only assertion quoted above.

Notes

  • xai/grok-4.6 could not be exercised end to end: it returns MODEL_NOT_IN_PLAN on a Go plan (GOAT and above required). Its entry rests on the CLI registry declaration — the same source npm run sync:commandcode-catalog reads.
  • Two further ids declare image upstream but are absent from GET /provider/v1/models, so they are deliberately not added: gpt-6-astra and minimax/minimax-m3-free.
  • This touches the same files as feat(models): enable reasoning for deepseek-v4.1-flash #91 (src/models.ts, src/commandcode-catalog-overrides.ts, tests/test-models.ts, CHANGELOG.md). Happy to rebase on whichever lands first.
  • npm test exits 1 in my environment on tests/test-pi-local.mjs ("Claude request through Anthropic Messages endpoint", 404). It reproduces identically on unmodified main, so it is unrelated to this change; CI is authoritative for the suites that need the full toolchain.

MODEL_INPUT_MODALITIES comes straight from the generated catalog, so a model
published after the pinned CLI release falls back to ["text"] and image input is
withheld on both transports: index.ts publishes input: ["text"] to the host and
the generate transport rejects image blocks with "does not support image
content". The override file only exposed an efforts hook, so there was no
supported way to re-enable vision for such a model.

Add MODEL_INPUT_MODALITIES_OVERRIDES next to the existing effort overrides and
merge it the same way, with entries for deepseek/deepseek-v4.1-flash and
xai/grok-4.6. Both are served by the Provider API and both are declared
inputModalities:["text","image"] by command-code@1.53.0.

Verified live on a Go subscription with the generated catalog untouched:
deepseek/deepseek-v4.1-flash described a test image correctly (red circle, blue
square, green triangle, text ZEBRA-42). xai/grok-4.6 requires GOAT or above
(MODEL_NOT_IN_PLAN on Go), so its entry rests on the CLI registry declaration.
@patlux

patlux commented Sep 15, 2026

Copy link
Copy Markdown
Owner

Thanks for identifying the missing image metadata. Merged #98 now includes upstream image modalities for DeepSeek V4.1 Flash and Grok 4.6, and #95 repairs the sync automation. This supersedes the per-model override entries without adding another maintenance path. The independent host-input behavior in #94 remains planned. Replacement CI passed before merge.

@patlux patlux closed this Sep 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants