Repository navigation
[Feature]: Auto-advertise image inputModalities for noVisionModels so Codex App lets images reach the vision sidecar #344
Description
Goal / Problem
Codex App users cannot use the vision sidecar feature for text-only routed models (GLM, DeepSeek, etc.). When a user pastes an image while a noVisionModels-tagged model is selected, Codex App rejects the upload at the client side before the request ever reaches the OpenCodex proxy. The sidecar never runs.
Concrete workflow:
- User selects a routed text-only model (e.g.
GLM.MacAgent/glm-5.2) in Codex App - User pastes an image into the conversation
- Codex App checks the model's catalog
inputModalities-> sees["text"]only - App refuses the image upload immediately ("model does not support image input")
- No request is sent to the OpenCodex proxy
- Vision sidecar (
gpt-5.4-minivia ChatGPT login) never activates
This means the "Give any model superpowers - image understanding via gpt-5.4-mini sidecar" feature is dead for Codex App users, even though it works correctly in Claude Code (which does not perform client-side inputModalities pre-checks).
Expected behaviour
When a provider has noVisionModels configured AND the vision sidecar is enabled, OpenCodex should automatically advertise inputModalities: ["text", "image"] in the catalog entry for those models. This way:
- Codex App sees the model supports image input -> lets the image through
- Request reaches OpenCodex proxy with the image
- Vision sidecar activates:
gpt-5.4-minidescribes the image -> feeds text to the upstream model - User gets image understanding end-to-end
The noVisionModels field already tells OpenCodex "this model can't see images, use the sidecar." But the catalog metadata contradicts this by declaring text-only, blocking the image at the App layer before the sidecar can help. The two configs are fighting each other.
Summary
OpenCodex has a vision sidecar (gpt-5.4-mini over ChatGPT login) that can describe images for text-only routed models. However, this sidecar never gets a chance to run in Codex App because the app rejects image uploads at the client side before the request reaches the proxy.
Root cause
When a model is registered in the catalog with inputModalities: ["text"] (e.g. GLM, Kimi, DeepSeek), Codex App checks this metadata locally before sending. If "image" is not in the list, the App refuses the paste/upload and never sends the request to the proxy. The sidecar intercept only works after the request reaches OpenCodex, so it is dead code for App users.
Current workaround attempt
Manually setting modelInputModalities: ["text", "image"] on the provider config to trick the App into letting the image through. This is fragile, undocumented, and may break on sync.
Environment
- opencodex: 2.7.36
- macOS, Apple Silicon
- Codex App
- Confirmed working: Claude Code + same GLM provider + same sidecar config (image successfully described by sidecar, GLM answered based on the description)
Checks
- I searched existing issues for vision sidecar / noVisionModels / image input / inputModalities.
- This is a feature request to make the existing vision sidecar actually usable from Codex App.
- Confirmed the sidecar works end-to-end in Claude Code, proving the issue is specifically the App's client-side inputModalities pre-check.
Activity
github-actions commentedon Jul 23, 2026
Maintainer decision respected
A maintainer has reopened this issue. The automated closure has been deactivated.
Wibias commentedon Jul 23, 2026
Sorry for the automatically closing. I am working on that.
fix(catalog): advertise image input for noVisionModels custom rows (#349
lidge-jun commentedon Jul 24, 2026
Implemented — the injected catalog now auto-advertises image
inputModalitiesfornoVisionModels-tagged models so the Codex app lets images through to the vision sidecar (standard path already ondev; the customModels-override residual is fixed in PR #416). See the detail on #349.fix(web-search): bound sidecar search deadline to 60s + graceful degr…
lidge-jun commentedon Jul 24, 2026
Implemented and merged in PR #416 (dev 6a0f5b5): the injected catalog auto-advertises image inputModalities for noVisionModels-tagged models (discovered + customModels rows) so Codex App lets images through to the vision sidecar.
fix(web-search): bound sidecar search deadline to 60s + graceful degr…