[Bug] Custom-provider vision capability is ambiguous when model metadata is missing

Client or integration

Direct HTTP/API client through the OpenCodex proxy (/v1/responses, /v1/models).

Area

Proxy and routing · Catalog / models · Vision sidecar · Custom providers

Summary

This issue now tracks only the remaining TokenRouter case:

TR/moonshotai/kimi-k3-free

The previously reported OpenCode Zen vision cases were investigated and resolved by #1070. That PR classified the Zen models individually, including confirming that mimo-v2.5-free is natively vision-capable and that nemotron-3-ultra-free is text-only and should use the vision fallback.

The remaining failure is different because TR is a custom TokenRouter provider in the reporter configuration rather than a built-in OpenCodex registry provider. OpenCodex therefore has no canonical registry metadata from which to determine whether moonshotai/kimi-k3-free supports image input or requires the vision sidecar.

In the reporter's configuration, the model returns HTTP 200 but does not correctly identify a deterministic image, while another model on the same TokenRouter provider succeeds with the same request.

Reproduction

Configure the custom TR provider for TokenRouter.
Start OpenCodex.
Send a /v1/responses request using:

TR/moonshotai/kimi-k3-free
Include a 16×16 opaque red PNG as the only input_image.
Ask the model to identify the dominant color.
Observe that the request returns HTTP 200 but does not correctly answer RED.
Repeat the same request with:

TR/qwen/qwen3.8-max
Observe that the control model correctly processes the same image.

Version

Originally reproduced on @bitkyc08/opencodex 2.10.0.

The OpenCode Zen portion of the original report has since been resolved on dev by #1070. This issue remains open only for the custom TokenRouter route.

Operating system

Windows 11, using the Codex runtime through the local OpenCodex proxy.

The remaining behavior is expected to be provider/routing-specific rather than Windows-specific.

Provider and model

Custom provider:

TR → TokenRouter

Remaining affected model:

moonshotai/kimi-k3-free

Known working control on the same custom provider:

qwen/qwen3.8-max

Remaining problem

Custom-provider model rows do not necessarily carry authoritative modality metadata.

For a built-in provider, OpenCodex can maintain model-specific capability information such as:

native image-input support;
known text-only models;
vision-sidecar requirements.

For a custom provider such as TR, OpenCodex cannot safely infer those properties from the provider name or model lineage alone.

The remaining question is therefore not whether the already-fixed Zen classifications should be expanded. It is how OpenCodex should handle missing or uncertain image capability metadata for custom-provider models without either:

sending images directly to a text-only upstream, or
unnecessarily replacing native image input with a sidecar for a model that actually supports vision.

Expected behavior

OpenCodex should have a deterministic policy for custom-provider models whose image-input capability is unknown.

A valid solution could include one or more of:

preserving authoritative modality metadata returned by the custom provider's model catalog;
allowing explicit per-model inputModalities / vision capability configuration;
allowing explicit per-model vision-sidecar classification for custom providers;
exposing unknown modality state clearly rather than treating unknown as proven vision support;
ensuring the runtime and /v1/models expose the same effective capability decision.

The implementation must not globally classify models by name or vendor lineage when the actual custom endpoint may expose different capabilities.

Acceptance criteria

TR/moonshotai/kimi-k3-free receives deterministic image handling based on actual or explicitly configured capability evidence.
A custom-provider model with confirmed native vision support continues receiving the original image directly.
A custom-provider model explicitly classified as text-only uses the vision fallback rather than receiving unsupported image content.
Unknown capability is not silently converted into false capability certainty.
/v1/models and request-time routing agree on the model's effective image capability.
Regression coverage includes at least:
- one custom-provider text-only model;
- one custom-provider native-vision model;
- one custom-provider model with unknown capability.

Resolved portions of the original report

OpenCode Zen model classification: resolved by fix(zen): strip images for the Zen models that reject them (#1043) #1070.
opencode-zen/mimo-v2.5-free: verified vision-capable.
opencode-zen/nemotron-3-ultra-free: classified text-only.
The broader Zen noVisionModels gap: resolved by fix(zen): strip images for the Zen models that reject them (#1043) #1070.

Those cases should not be considered acceptance criteria for this issue anymore.

Cross-references

[Bug]: Vision sidecar never triggers for NVIDIA NIM text-only models (registry nvidia entry missing noVisionModels) #956 / stack 7/7: NIM vision classification, service repair, and the qwen3.8-max rename #980 — NVIDIA NIM modality classification work.
fix(zen): strip images for the Zen models that reject them (#1043) #1070 — resolves the OpenCode Zen portion of this report.
[Bug]: No management-plane support for provider headers — custom providers needing client fingerprints (Agent Router) 401 after headers are lost #959 / feat(providers): manage provider headers via PATCH and ocx provider edit --headers (#959) #1033 — custom-provider header management; related management surface but separate from modality capability.

Checks

I searched existing issues and documentation.
I removed secrets, tokens, account details, request credentials, and personal data.

_{Automated translation bookkeeping — detected language: English.}

Issue reopened

The report now contains the information required by the automated check. Thanks for updating it.

Client or integration

Codex runtime via the local OpenCodex proxy: /v1/responses for execution and /v1/models for catalog visibility.

Area

Proxy and routing · Catalog / models · Vision sidecar · Provider adapter

Summary

Following the NVIDIA-specific fix recorded in #956 (PR #980 merged into dev as 7343f0ba5914a96aa9eab0a1397bfa5454c286bc), a similar class of regression persists for other text-only or capability-uncertain models. These models do not advertise reliable modality coverage, yet they occasionally attempt to answer image requests without any sidecar guidance and therefore deliver either empty responses, hallucinated confidence, or explicit inability statements.

Four models in my environment are confirmed to produce HTTP 200 responses without recognizing the deterministic image:

Model	Failure mode
`nvidia/nvidia/nemotron-3-ultra-550b-a55b`	Explicit cannot-see answer after image payload
`opencode-zen/mimo-v2.5-free`	Empty response body
`opencode-zen/nemotron-3-ultra-free`	Same cannot-see pattern as NVIDIA
`TR/moonshotai/kimi-k3-free`	Empty response body

Controls on the same payload and agents answer "RED" correctly: anthropic/claude-sonnet-5, anthropic/claude-opus-5, DW/devworld/deepseek-v4-pro-r, TR/qwen/qwen3.8-max, google-antigravity/gemini-3.6-flash, plus both Agent Router transports AGR-OAI and AGR. The general request-build, authentication, and proxy routing paths work; the divergence is isolated to capability attribution and sidecar invocation for the emerging providers.

This sits alongside #956 rather than duplicating it: the NVIDIA path was repaired, but the corresponding decision surface now depends on per-provider hints that are not visibly synchronized with catalog state, registry confidence, or the exported configuration consumed by the Codex client.

Reproduction

Install @bitkyc08/opencodex 2.10.0 and start the proxy (ocx start).
Feed any flailing model this red 16×16 PNG via data:image/png;base64, inside a /v1/responses call and ask for the dominant color.
Observe the four failure cases above return readable or blank text with no vision sidecar acknowledgement.
Repeat against the controls above and note the correct RED response under the same conditions.

Version and runtime

OpenCodex: 2.10.0.
Local proxy confirmed healthy on port 10100 with upstreams reachable.
Reference points to [Bug]: Vision sidecar never triggers for NVIDIA NIM text-only models (registry nvidia entry missing noVisionModels) #956 merged and tested on the dev branch — this issue therefore tracks what remains uncovered after that stack.

Evidence supporting root-cause hypothesis

/v1/models rows for the four failing models show inputModalities: ["text"] or omit the field, so Codex grips only text-side affordances before any sidecar can intervene.
Exported custom rows also lack inputModalities unless explicitly written, while the server-side registry keeps hint fields private to its import path — causing an off-by-one between runtime behavior and what users see in dashboards or clients.
Logs during the failing probes contain no [vision]/sidecar traces, so no arbitration agent is invoked before the upstream rejection or hallucination.

Impact

Failed models misbehave silently or confidently wrong without operator visibility.
Automations that assume “image input supported” route around or ignore the sidecar but hit text-only ceilings.
[Bug]: No management-plane support for provider headers — custom providers needing client fingerprints (Agent Router) 401 after headers are lost #959’s header work resolves transport for secrets, but does not address this capability-versus-catalog shadow state.

Suggested fix

Introduce a canonical per-model capability source of truth that merges registry hints, manual noVisionModels, and catalog export outputs, and make the runtime and export interfaces both read from it.
Extend persistent e2e coverage beyond NVIDIA: at least one text-only, one native-vision, and one unknown-provider probe per adapter, asserting non-empty, accurate answer or explicit 4xx.
Add an integration smoke test that walks /v1/models and /v1/responses for every known provider family in the local catalog build, so missing modality flags are caught before release.

Cross-references

[Bug]: Vision sidecar never triggers for NVIDIA NIM text-only models (registry nvidia entry missing noVisionModels) #956 — NVIDIA/NIM vision classification fix (26 text-only, 16 vision-capable audited models).
[Bug]: No management-plane support for provider headers — custom providers needing client fingerprints (Agent Router) 401 after headers are lost #959 — Provider headers management gap merged under feat: manage provider custom headers via PATCH and ocx provider edit --headers #961; excludes vision capability semantics discussed here.

Checks

I searched existing issues and documentation.
I removed secrets, tokens, headers, and prompt content.

docs(devlog): triage every open bug issue and PR against dev

docs(devlog): fold four audit rounds into the triage unit

docs(devlog): move three inferred PR-issue links out of the literal l…

fix(zen): strip images for the Zen models that reject them (lidge-jun…

Sorry for the delay on your follow-up — it deserved a per-model answer rather than a general one, so here it is against what actually shipped in v2.10.2 (now current on npm).

Your four probes have three different dispositions:

opencode-zen/nemotron-3-ultra-free — fixed. It is in the text-only exception list now, so the proxy strips the image and the sidecar describes it instead of forwarding a part the endpoint rejects. That came from a live probe against opencode.ai/zen/v1 on 2026-08-05, recorded in devlog/_plan/260805_bug_fix_stack/002_zen_modality_probe.md. Please re-test on v2.10.2.

opencode-zen/mimo-v2.5-free — deliberately excluded, and I want to check your reading against ours. That same probe found mimo-v2.5-free accepts images, so it is intentionally absent from the list, with a negative assertion in tests/provider-registry-parity.test.ts to keep it that way. The reasoning: adding it would silently replace a real image with a caption, which is worse than a loud 400.

Your report describes it returning 200 while not recognizing the red-image probe — that is a different failure from rejecting the image. If the model accepts the part but answers as though it were blind, the honest classification is "accepts images, poor vision", and stripping the image would make our behavior worse, not better. If you disagree, the thing that would change my mind is a capture showing the raw request we sent and the model's response — specifically whether the image part was present on the wire.

nvidia/nvidia/nemotron-3-ultra-550b-a55b — the NVIDIA member was addressed by the #956 fix you cited; re-test on v2.10.2 and tell me if it still fails.

TR/moonshotai/kimi-k3-free — still blocked on you. TR (TokenRouter) is not a built-in registry provider, so its behavior comes entirely from your own provider config. Without that block, redacted, I cannot tell whether the models list, noVisionModels, or the input modality metadata is the gap. This is the one item where I genuinely cannot progress without you.

To be direct about the state: this issue is now partly fixed, partly a disagreement about classification, and partly waiting on one config paste. Keeping it open for the TR half. Your probe methodology — deterministic red image, text control first — is what made the Zen list possible, so thank you for that.

Retested on v2.10.2 (npm, Windows 11, Codex 0.146.1, same 16×16 opaque red PNG, text control first, 5 trials each). Three of the four are resolved, and one of them resolves against me — details below, including the wire capture you asked for.

1. `opencode-zen/nemotron-3-ultra-free` — confirmed fixed

Text control RED, image probe 5/5 RED. Closing this one from my side.

2. `nvidia/nvidia/nemotron-3-ultra-550b-a55b` — confirmed fixed by #956

Image probe 5/5 RED. My first control run returned a transient upstream 503 (ResourceExhausted: Worker local total request limit reached (32/32)) — NIM capacity, not modality; a clean re-run gave control RED, image 1/1 RED. Closing this one too.

3. `opencode-zen/mimo-v2.5-free` — you were right, and my original report was wrong

You asked what would change your mind: a capture showing whether the image part was on the wire. Here it is, and it vindicates your classification.

The cause was on my side. My config.json carried a local override:

"opencode-zen": {
  "noVisionModels": ["deepseek-v4-flash-free", "mimo-v2.5-free", "nemotron-3-ultra-free"]
}

That entry predates the report — it is byte-identical in my 2026-08-03 config backup, one day before I filed this. So on the original probe my own config stripped the image before it ever reached MiMo, and the sidecar caption is what the model answered from. "200 but doesn't recognize the red image" was my configuration, not the model being blind. I should have diffed my local overrides against the registry before filing that half.

After removing the local override, mimo-v2.5-free returns 5/5 RED with the image forwarded natively.

Wire capture from ocx debug provider, same probe, two models side by side:

[ocx:openai-chat:request] host=opencode.ai model=mimo-v2.5-free        stream=false bodyBytes=190   <- text control
[ocx:openai-chat:request] host=opencode.ai model=mimo-v2.5-free        stream=false bodyBytes=407   <- with image
[ocx:openai-chat:request] host=opencode.ai model=nemotron-3-ultra-free stream=false bodyBytes=197   <- text control
[ocx:openai-chat:request] host=opencode.ai model=nemotron-3-ultra-free stream=false bodyBytes=307   <- with image

The probe's data: URL is exactly 130 characters. The cleanest comparison is the two image rows against each other — identical prompt, identical messageCount=1, same adapter, same host: 407 vs 307 bytes. That ~100-byte gap is the image payload. MiMo carries it; nemotron-3-ultra-free does not, because it was stripped and a caption put in its place. (Comparing each model to its own control is muddier — my image prompt is ~22 characters longer than my text control prompt — so the model-to-model comparison is the one to read.)

So: the registry is right, the negative assertion in tests/provider-registry-parity.test.ts should stay, and mimo-v2.5-free should remain absent from the text-only list. No disagreement left here.

4. `TR/moonshotai/kimi-k3-free` — config, plus a diagnostic that narrows it

Full TR block, redacted:

"TR": {
  "adapter": "openai-chat",
  "baseUrl": "https://api.tokenrouter.com/v1",
  "authMode": "key",
  "noVisionModels": [
    "moonshotai/kimi-k3-free",
    "qwen/qwen3.8-max"
  ],
  "apiKey": "<REDACTED>",
  "apiKeyPool": "<REDACTED>"
}

There is no models key and no modelInputModalities — the TR roster is discovered live, so the only modality metadata in play is that noVisionModels list.

Two things narrow this considerably:

The sidecar path demonstrably works on TokenRouter. TR/qwen/qwen3.8-max is in the same noVisionModels array as kimi-k3-free, and it was one of the six healthy controls that passed in my original report. So was DW/devworld/deepseek-v4-pro-r, which is also a noVisionModels entry on its provider. Two of my six "healthy controls" were themselves sidecar-routed — one of them on TokenRouter, through this exact block. Whatever is wrong with kimi-k3-free is not the shape of my TR config, because the identical config makes qwen3.8-max pass.

The proxy is doing the right thing on the wire for kimi-k3-free too. Same debug capture:

[ocx:openai-chat:request] host=api.tokenrouter.com model=moonshotai/kimi-k3-free stream=false bodyBytes=199   <- text control
[ocx:openai-chat:request] host=api.tokenrouter.com model=moonshotai/kimi-k3-free stream=false bodyBytes=309   <- with image

309 bytes against nemotron-3-ultra-free's 307 for the same prompt — the same stripped-and-captioned signature, ~100 bytes below the 407 of a request that actually carries the image. Classification and strip are both correct here; the gap is in what TokenRouter's kimi-k3-free does with the resulting text-only request.

And the failure mode has changed since I filed this — it is now a hang, not a bad answer. v2.10.2, 5 trials:

TR/moonshotai/kimi-k3-free   text control      200  3264ms   RED
TR/moonshotai/kimi-k3-free   image trial 1..5    -  180000ms  client timeout, no response
                                                              => 0/5

TR/qwen/qwen3.8-max          text control      200  3833ms   RED
TR/qwen/qwen3.8-max          image trial 1..5  200  ~2400ms  RED  => 5/5

The text-only control on kimi-k3-free answers in 3.2 s. The moment the request carries the sidecar caption instead, TokenRouter accepts the POST and then never responds — five for five, hard client timeout at 180 s. My original report said "HTTP 200 but never answers correctly"; for this model that no longer reproduces on v2.10.2, and I am correcting it: there is no 200, there is no response at all.

Since the same TR block, same key pool and same sidecar make qwen/qwen3.8-max answer in ~2.4 s, my current reading is that this is TokenRouter-side behavior for the moonshotai/kimi-k3-free route rather than an opencodex classification gap — the proxy sends a well-formed text-only request and the upstream stalls on it. Tell me what else would be useful: I can capture the full outbound body (not just bodyBytes), try the same request against TokenRouter directly with curl to take opencodex out of the path, or drop kimi-k3-free from my noVisionModels to see whether the native image request also stalls. Happy to run whichever you'd trust most.

Config cleanup on my side

For the record, since it affects any future report I file: I removed the local noVisionModels entries that v2.10.2's registry already covers, so my config stops shadowing yours.

nvidia:       ["deepseek-ai/deepseek-v4-flash", "minimaxai/minimax-m3",
               "nvidia/nemotron-3-ultra-550b-a55b", "z-ai/glm-5.2"]
           -> ["minimaxai/minimax-m3"]              # only the id NVIDIA_NIM_NO_VISION_MODELS lacks

opencode-zen: ["deepseek-v4-flash-free", "mimo-v2.5-free", "nemotron-3-ultra-free"]
           -> (key removed)                          # 2 now in OPENCODE_ZEN_TEXT_ONLY_MODELS,
                                                     # 1 deliberately excluded by you

Both retests above were re-run after this cleanup and still pass 5/5, which confirms the union in router.ts (mergeStringArray(registryEntry.noVisionModels, provider.noVisionModels)) is doing the work — the registry seed alone is sufficient. Worth noting for other users: a stale local noVisionModels never breaks a registry entry, but it silently overrides a deliberate registry exclusion, which is precisely how I produced a false bug report. A startup warning when a config noVisionModels id collides with a registry text-only decision would have caught this in one line of output.

Happy for you to keep this open for the TR half only.

fix(catalog): advertise image input for modalities-declared sidecar rows

fix(catalog): advertise image input for modalities-declared sidecar r…

docs(devlog): triage every open bug issue and PR against dev

docs(devlog): fold four audit rounds into the triage unit

docs(devlog): move three inferred PR-issue links out of the literal l…

fix(zen): strip images for the Zen models that reject them (lidge-jun…

docs(devlog): 260808 bug campaign - dispositions, execution, and faults

docs(devlog): add decade docs for lidge-jun#1562 GUI preflight and li…

fix(vision): respect modelInputModalities for text-only detection (li…

fix(catalog): advertise image input for modalities-declared sidecar r…

Navigation Menu

Repository navigation

[Bug] Custom-provider vision capability is ambiguous when model metadata is missing #1024

Client or integration

Area

Summary

Reproduction

Version

Operating system

Provider and model

Remaining problem

Expected behavior

Acceptance criteria

Resolved portions of the original report

Cross-references

Checks

Issue reopened

Client or integration

Area

Summary

Reproduction

Version and runtime

Evidence supporting root-cause hypothesis

Impact

Suggested fix

Cross-references

Checks

1. `opencode-zen/nemotron-3-ultra-free` — confirmed fixed

2. `nvidia/nvidia/nemotron-3-ultra-550b-a55b` — confirmed fixed by #956

3. `opencode-zen/mimo-v2.5-free` — you were right, and my original report was wrong

4. `TR/moonshotai/kimi-k3-free` — config, plus a diagnostic that narrows it

Config cleanup on my side

5 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

[Bug] Custom-provider vision capability is ambiguous when model metadata is missing #1024

Description

Client or integration

Area

Summary

Reproduction

Version

Operating system

Provider and model

Remaining problem

Expected behavior

Acceptance criteria

Resolved portions of the original report

Cross-references

Checks

Activity

github-actions commented on Aug 4, 2026

github-actions commented on Aug 4, 2026

Issue reopened

brunoflma commented on Aug 4, 2026

Client or integration

Area

Summary

Reproduction

Version and runtime

Evidence supporting root-cause hypothesis

Impact

Suggested fix

Cross-references

Checks

lidge-jun commented on Aug 6, 2026

brunoflma commented on Aug 6, 2026

1. opencode-zen/nemotron-3-ultra-free — confirmed fixed

2. nvidia/nvidia/nemotron-3-ultra-550b-a55b — confirmed fixed by #956

3. opencode-zen/mimo-v2.5-free — you were right, and my original report was wrong

4. TR/moonshotai/kimi-k3-free — config, plus a diagnostic that narrows it

Config cleanup on my side

5 remaining items

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

1. `opencode-zen/nemotron-3-ultra-free` — confirmed fixed

2. `nvidia/nvidia/nemotron-3-ultra-550b-a55b` — confirmed fixed by #956

3. `opencode-zen/mimo-v2.5-free` — you were right, and my original report was wrong

4. `TR/moonshotai/kimi-k3-free` — config, plus a diagnostic that narrows it