Skip to content

fix(core): report the tool return value on ToolResultEndEvent - #3331

Open
King52HerTz wants to merge 2 commits into
agentscope-ai:mainfrom
King52HerTz:fix/toolemitter-result-on-end-event
Open

King52HerTz wants to merge 2 commits into
agentscope-ai:mainfrom
King52HerTz:fix/toolemitter-result-on-end-event

Conversation

@King52HerTz

Copy link
Copy Markdown

Problem

ToolEmitter documents that "the emitted chunks do NOT affect what the LLM receives - only the tool method's return value is sent to the LLM." The event stream disagrees. Once a tool has emitted chunks its id lands in chunkedToolIds, emitToolResultDelta then skips the return value, and ToolResultEndEvent carries no result either — so a consumer rebuilding the tool message from the stream sees the concatenated progress text.

For AG-UI that means TOOL_CALL_RESULT holds progress text while the model received the return value:

  • a client resending full history without server-side memory feeds progress text back as the tool result, contradicting the emitter contract;
  • anything emitted only for the UI (a one-time code, say) leaks into model-visible history;
  • a client showing the operation's outcome shows the wrong thing.

Reproduced from the issue with the reporter's scenario in AguiAgentAdapterV2Test.

Change

ToolResultEndEvent gains finalResultText, populated by ReActAgent from the tool result's text blocks, and AguiStreamContext.endToolResult prefers it over the accumulated delta buffer. Progress chunks still stream as deltas, so UIs that want them can render them — the difference is that the closed result is now the value the model saw.

Why a field and not metadata. My first attempt put this under AgentEvent.getMetadata(). That map passes the tool result's own metadata through unchanged, and ReActAgentNewLoopE2ETest.toolResultMetadataPropagatesToEvents asserts it by exact equality — injecting a framework key there breaks it. That test failed on the first draft, so the value now travels on a dedicated field and the metadata map is untouched.

The @JsonCreator gained an optional finalResultText property; payloads without it deserialize to null, and the two existing constructors delegate unchanged. When no return value is reported the previous buffer behaviour is preserved, so other producers are unaffected.

Scope

Results with no text blocks yield null and fall back to the delta buffer rather than reporting an empty result.

Testing

Two cases added: progress deltas plus a reported return value yields the return value; a bare ToolResultEndEvent still yields the buffered deltas.

mvn test -pl :agentscope-core,:agentscope-harness,:agentscope-extensions-agui -am
# core 2492 / harness 509 / agui 1062 tests, 0 failures

Partially addresses #3312

ToolEmitter documents that emitted chunks do not affect what the LLM receives;
only the tool method's return value does. The event stream disagrees: once a
tool has emitted chunks, ReActAgent records its id in chunkedToolIds and
emitToolResultDelta then skips the return value, and ToolResultEndEvent carries
no result either. A consumer reconstructing the tool message from the stream
therefore gets the concatenated progress text.

For AG-UI that means TOOL_CALL_RESULT holds progress text while the model saw
the return value, so the two histories diverge: a client resending full history
feeds progress text back as the tool result, anything emitted only for the UI
leaks into it, and the UI shows the wrong outcome.

Put the return value on the end event as an explicit field and let AguiStreamContext
prefer it over the delta buffer. It is not stored under AgentEvent.getMetadata(),
which passes the tool result's own metadata through unchanged; ReActAgentNewLoopE2ETest
asserts that map exactly, so adding a framework key there would be a contract
change for unrelated consumers. Producers that report no return value keep the
previous behaviour.
@CLAassistant

CLAassistant commented Sep 28, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@codecov

codecov Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

oss-maintainer

This comment was marked as abuse.

@oss-maintainer

This comment was marked as abuse.

Review follow-up for agentscope-ai#3331.

null on getFinalResultText() meant two different things: "the producer reports
nothing" and "the tool returned content with no text in it". The second read as the
first, so an image-only or blank return value fell back to the delta buffer and put
progress text straight back into TOOL_CALL_RESULT — the leak this field exists to
close. A result that carries content now reports its joined text even when that join
is empty; only a result with no content blocks at all stays unreported.

finalResultText joins text blocks, while an image or structured part reaches a
client as ToolResultDataDeltaEvent, so replacing the whole buffer dropped it.
AguiStreamContext now keeps the non-text fragments beside the buffer and re-appends
them after the return value. The fallback path is untouched, including the
arrival-order rendering the existing tests pin.

service-dataplane was the second first-party consumer and still persisted the
buffer, so the two disagreed about what a tool returned. SessionEventMapper applies
the same rule with the same fallback, reuses the shared payload cap, and re-appends
non-text fragments the way it already glued them.

Also closes the two Info points: JSON round-trip for the new @JsonCreator property
(the value survives polymorphic decode, "" decodes as "" rather than null, and a
payload written before the field existed decodes to null), and the ReActAgent
population path end to end — with an assertion that the progress chunk really
reached the stream, so it cannot pass by the tool never emitting.

Checked by mutation: reverting the empty-vs-null rule fails one test, never
populating the field fails two, dropping the re-append in the AG-UI adapter fails
two, reverting the dataplane rule fails two. Full reactor mvn test green,
spotless:check green.
@King52HerTz

Copy link
Copy Markdown
Author

All three warnings are addressed in 0e94997, along with both [Info] points. One limitation is called out at the bottom rather than papered over.

1. ReActAgent.java:3525 — null conflated two states.

finalToolResultText now reports the join of the text blocks whenever the result carries content blocks — including when that join is empty — and returns null only when there is nothing to report from. So an image-only or blank return value no longer reads as "the producer reported nothing" and no longer falls back to the delta buffer, which is the case you identified as re-opening the leak. ToolResultEndEvent.getFinalResultText()'s contract is written down: "" is a reported value, null is the absence of one.

2. AguiStreamContext.java:311 — authoritative text dropped non-text blocks.

appendToolResultData now records each serialised block alongside the buffer, and endToolResult re-appends those blocks after the authoritative text instead of discarding them. The fallback path is unchanged byte-for-byte, including the arrival-order rendering pinned by testToolResultTextAndDataDeltasAreJoinedInArrivalOrder ("hel\nstructuredlo"). Two new cases cover the multimodal shape: text + image with a reported return value, and image-only with "".

3. ToolResultEndEvent.java:40 — the two first-party consumers disagreed.

service-dataplane's SessionEventMapper applies the same rule through ToolResultBuffer.applyFinalResultText(...): the return value replaces the accumulated text, non-text fragments are re-appended in the way that module already glued them, null keeps the previous buffered output untouched, and the shared 64K payload cap still decides truncation so truncated / originalSize describe what was actually persisted.

[Info] — JSON round trip. Three cases in AgentEventStreamTest exercise the polymorphic decode that RemoteEventCodec and session persistence use: the property survives, "" decodes as "" rather than null, and a payload written before the field existed decodes to null.

[Info] — population path untested. Two ReActAgentNewLoopE2ETest cases run a real loop with a tool that emits one progress chunk and then returns a value; the first asserts the progress chunk actually reached the stream before asserting the result excludes it, so the test cannot pass by the tool never emitting.

One limit I did not resolve. A streaming tool that emits progress as image/data blocks will still have those blocks appear in the result. ToolResultDataDeltaEvent is shared by the result-delta path (emitToolResultDelta) and the chunk-callback path, and nothing on the wire distinguishes them, so the adapter cannot tell "result part" from "progress part" today. Between dropping every multimodal result and keeping the pre-PR behaviour for that one shape, I kept the latter. If you would rather distinguish them at the source, the natural seam is a marker on the chunk-path events — which is a larger change than this PR and I did not want to fold it in silently.

Evidence. Four mutations each fail exactly the tests that guard them: reverting the empty-vs-null rule → 1; never populating the field → 2; dropping the re-append in the AG-UI adapter → 2; reverting the dataplane rule → 2. Full reactor mvn test green and spotless:check green locally.

@jujn

jujn commented Oct 7, 2026

Copy link
Copy Markdown
Collaborator

从可维护性角度来说,我们是否可以新增一个ToolProgressEvent事件传递工具进度
(ToolResult*DeltaEvent本就只应传递工具结果):

ToolResultStartEvent
ToolProgressEvent          // ToolEmitter 的 progress
ToolProgressEvent
ToolResultTextDeltaEvent   // 只表示最终结果
ToolResultDataDeltaEvent   // 只表示最终结果
ToolResultEndEvent

@jujn jujn self-assigned this Oct 7, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants