fix(deepagents): keep surrogate pairs intact when chunking long lines

[Anton Nak (antonnak)]

Problem

formatContentWithLineNumbers chunks lines longer than MAX_LINE_LENGTH with line.substring(start, end) at fixed UTF-16 code-unit boundaries. When an astral-plane character (emoji, some CJK) straddles a boundary, its surrogate pair is split: a lone high surrogate ends one chunk and a lone low surrogate begins the continuation line.

read_file returns this formatted text as tool output, so the lone surrogate reaches the model provider in the request body. Anthropic's strict JSON parser rejects it:

400 invalid_request_error: The request body is not valid JSON:
no low surrogate in string: line 1 column N (char N-1)

OpenAI accepts the same content, so the bug only surfaces on Anthropic models and silently "works" behind a cross-provider fallback. The same unsafe slicing exists in truncateIfTooLong.

Fixes #631.

Fix

Add surrogateSafeCutIndex(text, max): when a cut would fall directly after a high surrogate (U+D800-U+DBFF), back it off by one so the whole pair stays on one side. Used in:

• formatContentWithLineNumbers — a running cursor replaces fixed MAX_LINE_LENGTH strides, so a straddling pair moves whole into the continuation chunk. Lossless: the character is preserved, just rendered on the next continuation line. Output is unchanged for content with no astral character at a chunk boundary (the back-off only triggers when the code unit before the cut is a high surrogate).
• truncateIfTooLong — the string cut no longer splits a pair.

Tests

Three regression tests (utils.test.ts) assert the output is well-formed UTF-16 by checking that encodeURIComponent does not throw on it — it throws on a lone surrogate, the same way Anthropic's request-body parser rejects one. Without the fix they fail with URIError: URI malformed; with it, all 91 utils.test.ts tests pass.

(encodeURIComponent rather than String.prototype.isWellFormed() because the repo targets lib: ["ES2023"], which has no ES2024 typings.)

Scope / follow-ups

#631 also notes the same code-unit slicing in the read_file eviction truncation, compactToolResults, and truncateArgs. Those can reuse surrogateSafeCutIndex (it would need to be exported). I kept this PR to the two utils.ts functions for reviewability — happy to extend it to cover the rest in this PR if you'd prefer.

[changeset-bot]

⚠️ No Changeset found

Latest commit: b044c93

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[vercel]

Anton Nak (@antonnak) is attempting to deploy a commit to the LangChain Team on Vercel.

A member of the Team first needs to authorize it.

[pkg-pr-new]

Open in StackBlitz

deepagents-acp

npm i https://pkg.pr.new/deepagents-acp@632

deepagents

npm i https://pkg.pr.new/deepagents@632

@langchain/sandbox-standard-tests

npm i https://pkg.pr.new/@langchain/sandbox-standard-tests@632

@langchain/daytona

npm i https://pkg.pr.new/@langchain/daytona@632

@langchain/deno

npm i https://pkg.pr.new/@langchain/deno@632

@langchain/modal

npm i https://pkg.pr.new/@langchain/modal@632

@langchain/node-vfs

npm i https://pkg.pr.new/@langchain/node-vfs@632

@langchain/quickjs

npm i https://pkg.pr.new/@langchain/quickjs@632

commit: b044c93