fix(core): honor the host input modalities on the generate transport - #94
Merged
Merged
Conversation
The generate transport gated image content on the pinned capability snapshot alone. A model published upstream after the last catalog sync is absent from it, so the transport rejected images for a model the host already advertises as vision-capable -- the same model works on the Provider API transport, which resolves image support from the host. Read the host's resolved model.input first and fall back to the snapshot, so models.yml/modelOverrides can unlock (or narrow) image support before the generated catalog catches up.
Owner
|
Integrated current main; replaced the unchecked string-array cast with a type-guard filter. Unsupported host modalities cannot accidentally regain catalog image support. Added mixed/unsupported modality assertions. Confirmed 1 model test and both transport stream regressions fail on the original implementation and pass with the fix. Full npm test passes with real Pi and OMP plus format/diff checks. Waiting for CI. |
Owner
|
Shipped in |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The generate transport gated image content on the pinned capability snapshot alone:
src/commandcode-catalog.tsis generated from a single CLI release, so any model published upstream after the last sync is absent fromMODEL_INPUT_MODALITIESandinputModalitiesForModel()falls back to["text"]. The result is a transport split: the same model accepts images on the Provider API transport (which resolves image support from the host) and rejects them withSelected Command Code model does not support image content in user messagesonce the endpoint returns403 upgrade_requiredand the request falls back to generate.This is the mirrored counterpart of #91: the missing catalog entry degrades both capabilities, and neither has an override hook.
Change
inputModalitiesForModel(modelId, hostInput?)andmodelSupportsImageInput(modelId, hostInput?)now prefer the host's resolvedmodel.inputand fall back to the snapshot when it is absent, andsrc/core.tspassesmodel.inputthrough.Both directions are honored: a host that advertises
["text", "image"]for a model the catalog does not list yet unlocks image content, and a host that narrows a catalogued vision model back to["text"]keeps the rejection.This does not replace the catalog sync (#93) — the snapshot stays the offline floor for hosts that supply no
input— it removes the coupling that makes a capability override impossible without a release.Verification
npm run typechecknpx tsx tests/test-models.ts— 23 passnpx tsx tests/test-stream.ts— 41 passnpx tsx tests/test-transport.ts,npx tsx tests/test-pure-functions.ts— 5 + 49 passBoth new tests fail on
main(2 stream, 1 model) and pass with the change.Reproduced end to end on Oh My Pi against
deepseek/deepseek-v4.1-flashwith a local mock endpoint: before the change the generate transport rejects the image, after it the request body carriesparams.messages[0].content[1].image.Related: #90, #91, #92