fix(cli): decode JSON-escaped newlines in --content argv - #6566
Conversation
Agents (especially Codex) JSON-escape a multiline body then pass `--content 'para1\n\n- item'`. Bash does not expand `\n` in quotes, so the two-character sequence was signed and rendered literally. Prompt guidance (block#2121) did not hold: 49/100 Codex events on a production nest still carried literal `\n`. Decode those escapes at Markdown paragraph/list boundaries in argv `--content` before signing. `--content -` stays byte-exact. Isolated `\n` (Windows paths, prose) and `\n` inside fenced/inline code are unchanged. Fail-closed (block#5114) would turn the same turns into missing replies. Fixes block#4891. Co-authored-by: crgallego <chris.cokerconstruction@gmail.com> Signed-off-by: crgallego <chris.cokerconstruction@gmail.com>
Chessing234
left a comment
There was a problem hiding this comment.
the scoping here is more careful than i expected — code-span masking, the boundary requirement, isolated \n left alone, and stdin byte-exact. that's the right set of constraints and the doc comments say why.
my question is scope rather than correctness: this is wired into cmd_create_issue and cmd_set_canvas as well as messages. the doc comment says "do not use this for YAML, diffs, or other non-Markdown payloads" — but an issue body is one of the most likely places for someone to paste a diff or a shell snippet without fencing it, and once it's unfenced the code mask can't help. grep -P 'a\\n\\nb' written inline in an issue would come out rewritten.
the part that makes that uncomfortable is that the rewrite happens before signing, so the published event no longer matches what the user typed and there's no way back except knowing to use stdin. is an opt-out worth having (--raw / --no-decode-escapes), or restricting this to messages where the agent-callback motivation actually lives?
not arguing the default is wrong for chat — agents JSON-escaping bodies is a real problem and this fixes it well.
Summary
Messages built with argv
--content 'para1\n\n- item'reach the relay carrying the two-character sequence\+ninstead of LF. Desktop then correctly renders those characters literally. This is the Codex/OpenCode quoting path: the model JSON-escapes a body, bash does not expand\nin quotes, andread_or_stdincopies argv verbatim.#2121 taught agents to use
printf … | --content -. That is still the right manual path; it is not a mechanism. After it merged, 49/100 Codex events on a production nest still stored literal\n(#4891).Why decode, not reject
#5114 uses the same structural-boundary detector and fail-closes. On that nest that is ~half of Codex turns, which converts an ugly-but-present reply into a missing one (
docs/welcome-kickoff-silent-failures.md). Codex does not rewrite to stdin on retry.This PR inverts that polarity: decode
\n\n/\n-/\n*/\n#/\n>/ ordered-list /\n```outside Markdown code regions into real LF before signing.--content -stays byte-exact so intentional\nstill has an escape hatch. Isolated\n(C:\new\tmp, prose) is left intact.Wired through
read_markdown_or_stdinon Markdown bodies (messages send/edit,issuescreate/status,canvas set). YAML, diffs, patches, and agent system-prompts stay on rawread_or_stdin.Fixes #4891. Complementary to #2121. Alternative polarity to #5114.
Testing
At
4c7126366b14e3837d2a971e77d075c0446bbf1f:372 passed, 0 failed. New cases cover decode, real-LF passthrough, inline and fenced code preservation, isolated
\n, ordered lists/headings, mixed real+escaped, and that rawread_or_stdinstill does not decode.No UI change.
Reproduction
Before:
buzz messages send --channel <uuid> --content $'not this'is not the bug; the bug is--content 'First.\n\n- item'(single-quoted two-character\n).After: that argv form publishes real LF and Desktop GFM renders a paragraph plus a list.
printf 'a\\nb' | buzz messages send --content -still stores literal backslash-n.