Skip to content

fix(mcpserver): base64-encode bytes tool output instead of crashing on non-UTF-8 - #3667

Closed
udsy19 wants to merge 1 commit into
modelcontextprotocol:mainfrom
udsy19:fix/bytes-tool-output-base64
Closed

udsy19 wants to merge 1 commit into
modelcontextprotocol:mainfrom
udsy19:fix/bytes-tool-output-base64

Conversation

@udsy19

@udsy19 udsy19 commented Oct 9, 2026

Copy link
Copy Markdown

Closes #3554.

Problem

A tool annotated -> bytes advertises outputSchema {"result": {"type": "string", "format": "binary"}}, but when it returns non-UTF-8 binary data (PNG magic bytes, a zip, a PDF, …) the call crashes with PydanticSerializationError inside _convert_to_content; the client only gets is_error: true with a generic message and the payload is lost. UTF-8-decodable bytes didn't crash but came back as a raw string rather than base64.

Root cause

_convert_to_content (func_metadata.py) fed the result to pydantic_core.to_json(result, fallback=str, …), which JSON-encodes bytes by UTF-8-decoding them — fallback only applies to types pydantic can't serialize, so non-UTF-8 bytes raise. The generated structured-output model also serialized bytes with the default ser_json_bytes="utf8", hitting the same failure on the structured path.

Fix

Base64-encode bytes for JSON, matching the convention already used by Image/Audio (utilities/types.py) and the lowlevel server (server.py, base64.b64encode(...).decode()):

  1. _convert_to_content base64-encodes bytes before the to_json branch, so all bytes become a consistent base64 string instead of crashing or leaking a raw decode.
  2. The generated output models (_create_wrapped_model, _create_model_from_class) set ser_json_bytes="base64" so the structured-content path serializes bytes as base64.
  3. StrictJsonSchema.bytes_schema pins the advertised schema to {"type": "string", "format": "binary"}, so enabling ser_json_bytes="base64" does not leak format: base64url into outputSchema — the advertised schema is unchanged.

Scope note: this covers the direct -> bytes return (the issue's primary case) and ordinary classes converted via _create_model_from_class. A user-defined BaseModel/TypedDict with a bytes field is intentionally not changed here: pydantic forbids a TypeAdapter(config=...) override for those types (PydanticUserError: Cannot use config when the type is a BaseModel, dataclass or TypedDict), so fixing it would require mutating/rebuilding the user's model — a larger, separate change. Happy to follow up if maintainers want that path addressed too.

Tests

Added 4 regression tests in tests/server/mcpserver/test_func_metadata.py: non-UTF-8 bytes base64-encode in _convert_to_content; UTF-8-decodable bytes also return base64 (consistent); a -> bytes tool returns base64 in both unstructured and structured output with the format: binary schema asserted; a bytes field in an ordinary-class output model serializes to base64 and decodes back. The 4 tests fail on main and pass with the fix. Full tests/server/mcpserver/ suite: 646 passed; ruff + pyright clean.

…n non-UTF-8

A tool annotated `-> bytes` advertises output schema {"type":"string","format":"binary"}, but returning non-UTF-8 binary data (e.g. PNG magic bytes) crashed with PydanticSerializationError.

Root cause: _convert_to_content fed bytes to pydantic_core.to_json, which UTF-8-decodes them (fallback=str only applies to types pydantic cannot serialize), and the generated structured-output model serialized bytes with the default ser_json_bytes='utf8'. Both raise on non-UTF-8 bytes; UTF-8-decodable bytes also came back as a raw JSON string rather than base64.

Fix:
- _convert_to_content now base64-encodes bytes before the to_json branch, matching Image/Audio and the lowlevel server, so all bytes become a consistent base64 string.
- Generated output models (_create_wrapped_model, _create_model_from_class) set ser_json_bytes='base64' so the structured-content path serializes bytes as base64 instead of crashing.
- StrictJsonSchema.bytes_schema pins the advertised schema to format: binary, so ser_json_bytes='base64' does not leak base64url into outputSchema.

Only generated output models are covered; a user-defined BaseModel/TypedDict with a bytes field is unchanged (pydantic forbids a TypeAdapter config override there).

Signed-off-by: Udaya Tejas <udayatejas2004@gmail.com>
@github-actions github-actions Bot added the missing-issue-link Auto-closed: PR needs a linked issue assigned to its author (see CONTRIBUTING.md) label Oct 9, 2026
@github-actions

github-actions Bot commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

This PR has been closed automatically. This repo only keeps pull requests open when they come from a maintainer, or from a contributor a maintainer has assigned to the linked issue, and you aren't currently assigned to #3554.

If a maintainer assigns you to #3554, this PR reopens on its own and there's nothing more you need to do here. Assignment is a maintainer call based on capacity; comments that only ask to be assigned don't factor in. What does help is engaging on the issue itself by confirming the repro, explaining why it matters for your use case, or describing the approach you'd take.

You're welcome to keep pushing commits here (just avoid force-pushing, since GitHub can't reopen a rewritten branch), but that on its own won't get the PR reviewed or the issue assigned, and realistically most auto-closed PRs stay closed. There's no need to open a new PR either way.

CONTRIBUTING.md has the full reasoning, but in short:

  • We're a small team with very little capacity to review community PRs right now.
  • Many recent PRs are AI-generated with little human review, and reviewing one carefully still costs a maintainer as much time as it ever did. A well-described issue is usually more useful to us than the code.

Maintainers: reopen, remove missing-issue-link, or add bypass-issue-check to override.

@github-actions github-actions Bot closed this Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

missing-issue-link Auto-closed: PR needs a linked issue assigned to its author (see CONTRIBUTING.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Structured tool output: -> bytes return crashes with a generic error on non-UTF-8 payloads instead of returning the advertised base64/binary string

1 participant