Skip to content

fix(core): honor the host input modalities on the generate transport - #94

Merged
patlux merged 2 commits into
patlux:mainfrom
djymike:feat/honor-host-input-modalities
Sep 15, 2026
Merged

patlux merged 2 commits into
patlux:mainfrom
djymike:feat/honor-host-input-modalities

Conversation

@djymike

@djymike djymike commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Problem

The generate transport gated image content on the pinned capability snapshot alone:

const allowImages = modelSupportsImageInput(model.id)
if (!allowImages) assertTextOnlyMessages(context.messages)

src/commandcode-catalog.ts is generated from a single CLI release, so any model published upstream after the last sync is absent from MODEL_INPUT_MODALITIES and inputModalitiesForModel() falls back to ["text"]. The result is a transport split: the same model accepts images on the Provider API transport (which resolves image support from the host) and rejects them with Selected Command Code model does not support image content in user messages once the endpoint returns 403 upgrade_required and the request falls back to generate.

This is the mirrored counterpart of #91: the missing catalog entry degrades both capabilities, and neither has an override hook.

Change

inputModalitiesForModel(modelId, hostInput?) and modelSupportsImageInput(modelId, hostInput?) now prefer the host's resolved model.input and fall back to the snapshot when it is absent, and src/core.ts passes model.input through.

Both directions are honored: a host that advertises ["text", "image"] for a model the catalog does not list yet unlocks image content, and a host that narrows a catalogued vision model back to ["text"] keeps the rejection.

This does not replace the catalog sync (#93) — the snapshot stays the offline floor for hosts that supply no input — it removes the coupling that makes a capability override impossible without a release.

Verification

  • npm run typecheck
  • npx tsx tests/test-models.ts — 23 pass
  • npx tsx tests/test-stream.ts — 41 pass
  • npx tsx tests/test-transport.ts, npx tsx tests/test-pure-functions.ts — 5 + 49 pass

Both new tests fail on main (2 stream, 1 model) and pass with the change.

Reproduced end to end on Oh My Pi against deepseek/deepseek-v4.1-flash with a local mock endpoint: before the change the generate transport rejects the image, after it the request body carries params.messages[0].content[1].image.

Related: #90, #91, #92

The generate transport gated image content on the pinned capability
snapshot alone. A model published upstream after the last catalog sync is
absent from it, so the transport rejected images for a model the host
already advertises as vision-capable -- the same model works on the
Provider API transport, which resolves image support from the host.

Read the host's resolved model.input first and fall back to the snapshot,
so models.yml/modelOverrides can unlock (or narrow) image support before
the generated catalog catches up.
@patlux

patlux commented Sep 15, 2026

Copy link
Copy Markdown
Owner

Integrated current main; replaced the unchecked string-array cast with a type-guard filter. Unsupported host modalities cannot accidentally regain catalog image support. Added mixed/unsupported modality assertions. Confirmed 1 model test and both transport stream regressions fail on the original implementation and pass with the fix. Full npm test passes with real Pi and OMP plus format/diff checks. Waiting for CI.

@patlux
patlux merged commit 4de55c4 into patlux:main Sep 15, 2026
15 checks passed
@patlux

patlux commented Sep 15, 2026

Copy link
Copy Markdown
Owner

Shipped in pi-commandcode-provider@0.7.0: https://github.com/patlux/pi-commandcode-provider/releases/tag/v0.7.0 . Public npm package model discovery and GOAT live smoke verified.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants