llama-atmosphere-agent: assert the plain-chat integration test on behaviour, not wording - #446
Merged
Merged
Conversation
…aviour, not wording The first model-backed CI run (35600558852) proved the loop -- the tool-call and read->write->read tests passed against Qwen2.5-1.5B -- while plainChatStreamsAnAnswer failed because the model answered "OK" to "reply with exactly ATMOSPHERE_OK". That assertion pinned prose a 1.5B model does not produce deterministically, which the class javadoc already said not to do. The test now asks a question with one unambiguous fact (2 + 2), asserts a non-empty streamed answer containing "4" that arrived in several chunks, and documents why. TODO.md drops the "read the first run" item that run resolved. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UVwj2UuMPybiK1bHyG9toH
bernardladenthin
had a problem deploying
to
maven-central
September 21, 2026 13:15 — with 
GitHub Actions
Failure
GitHub Actions
Failure
bernardladenthin
had a problem deploying
to
maven-central
September 21, 2026 13:15 — with
GitHub Actions
Failure
GitHub Actions
Failure
bernardladenthin
had a problem deploying
to
startgate
September 21, 2026 13:15 — with
GitHub Actions
Error
GitHub Actions
Error
|
This was referenced Sep 21, 2026
This branch had an error being deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Integration Test llama-atmosphere-agent (model-backed)). That run proved the loop:toolCallResultIsFedBackAndAnsweredandmultiRoundReadWriteReadLoopChangesTheFilepassed against the real Qwen2.5-1.5B model throughOpenAiCompatServer. OnlyplainChatStreamsAnAnswerfailed, because the model answeredOKto "reply with exactly ATMOSPHERE_OK" — a working loop failing a prose assertion the class javadoc already said not to write.2 + 2) and asserts what the wire contract guarantees: a non-empty streamed answer containing4that arrived in several SSE chunks, with no session error. The javadoc records why.TODO.mddrops the "read the first model-backed run before trusting it" item that this run resolved, and notes the result.Test-only change (validation-only job, not a publish gate).
Test plan
mvn verify -Dllama.version=5.2.0-SNAPSHOT: 23 tests, 20 run green, 3 model-backed self-skip (no GGUF in the sandbox; the model-backed run is the CI job itself)Integration Test llama-atmosphere-agent (model-backed)job is the check that matters hereRelated issues / PRs
Refs #444, #445
Checklist
CONTRIBUTING.mdandCODE_OF_CONDUCT.mdSECURITY.md)🤖 Generated with Claude Code
https://claude.ai/code/session_01UVwj2UuMPybiK1bHyG9toH
Generated by Claude Code