Citation-marker filter strips Codex App visualize directives #6039

Client or integration

Codex App

Area

Streaming

Summary

stripCitationMarkers / createCitationMarkerFilter in src/responses/citation-markers.ts remove every \uE200 … \uE201 span from assistant text, whatever keyword follows \uE200. #3150 was about \uE200cite\uE202…\uE201 citation spans, but the filter never checks the keyword. The Codex App uses the same private-use delimiters for other inline directives, and the bundled visualize plugin (1.0.39) renders its visual from one of them:

\uE200visualize\uE202{"path":"/abs/path/to/fragment.html"}\uE201

Because the filter runs on both the streaming path (src/bridge/sse.ts, citationFilter.push and stripCitationMarkers in closeCurrentMessage) and the non-streaming path (src/bridge/response-json.ts flushText), a correctly formed visualize reference gets deleted from the response. The Codex App never sees it, and no inline visualization is rendered.

Expected: only citation spans (\uE200cite\uE202…\uE201) are stripped, and other directive spans such as visualize pass through unchanged.

A likely minimal fix is to require the segment to start with \uE200cite\uE202 before treating it as a span, in both the whole-string and streaming paths. Other keywords would then be emitted verbatim. The streaming filter has to hold back text until it has seen enough characters to decide, since \uE200 and cite can arrive in separate deltas.

This is one of two problems we hit together and is the one that is clearly OpenCodex's. The other one: with the routed model we used, the private-use characters were also missing from the model's own output, so that turn printed plain visualize{"path":...} text. That text passed through untouched, which is consistent with the filter only removing delimited spans. Earlier reports #1349 / #1350 describe the visible symptom (visualize degrading to plain text) and were auto-closed for a missing OS field. This issue is about the proxy-side stripping, which would still break visualize for models that emit the delimiters correctly.

Reproduction

Proxy-side, no model needed:

import { stripCitationMarkers, createCitationMarkerFilter } from "./src/responses/citation-markers";

const viz = "before\n\uE200visualize\uE202{\"path\":\"/tmp/x.html\"}\uE201\nafter";
console.log(JSON.stringify(stripCitationMarkers(viz)));
// actual:   "before\n\nafter"
// expected: input unchanged

const f = createCitationMarkerFilter();
console.log(JSON.stringify(f.push(viz) + f.flush()));
// actual:   "before\n\nafter"

End to end:

1. Run Codex App (Desktop, remote SSH host) through OpenCodex with a routed model.
2. Invoke the bundled visualize skill. Its SKILL.md tells the model to reply with the \uE200visualize\uE202{...}\uE201 line.
3. Any delimited span the model emits is removed by the proxy before it reaches the app, so nothing renders.

Version

2.59.0 installed. Same code on main at 4bc9229 (package.json 2.67.0); citation-markers.ts is byte-identical.

Operating system

Ubuntu 24.04.3 LTS (kernel 7.0.0-31-generic), Codex Desktop 0.155.1 connected over SSH

Provider and model

Not provider-specific for the stripping itself. Observed with a routed Claude model (klb/claude-opus-5.5).

Logs or error output

# visualize skill source, line 46 (bytes above 0x7F on that line):
$ sed -n '46p' .../visualize/1.0.39/skills/visualize/SKILL.md | python3 -c "import sys; print([hex(ord(c)) for c in sys.stdin.read() if ord(c)>127])"
['0xe200', '0xe202', '0xe201']

Checks

• I searched existing issues and documentation.
• I removed secrets, tokens, account details, request credentials, and personal data.