Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 2 additions & 5 deletions TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,9 @@ so everything below is genuinely still open.

### Atmosphere coding agent (`llama-atmosphere-agent/`) — follow-ups

The headless loop is verified (see CLAUDE.md "Local coding agent with Atmosphere"). Still open:
The headless loop is verified, including the model-backed CI job (run 35600558852: tool call
answered, read→write→read loop changed the file). Still open:

- **First model-backed CI run.** `test-java-llama-atmosphere-agent-integration` was added without a
run on GitHub's runners; the three assertions are about the loop (tool invoked, result answered,
file changed), but a 1.5B model on a CPU runner may still need a prompt or budget tweak. Read its
first run before trusting it as a signal.
- **Tool rounds are not carried across REPL turns** — only `user`/`assistant` text is replayed, so a
second question cannot refer to a tool result of the first. Keep the full Atmosphere
`ChatMessage` list (incl. `tool_calls`/`tool` messages) per turn instead.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -116,16 +116,23 @@ private ConsoleSession session() {
return new ConsoleSession(new PrintStream(new ByteArrayOutputStream(), true, StandardCharsets.UTF_8), fs);
}

/**
* Plain chat: a non-empty answer that arrives as several streamed chunks. The first CI run (run
* 35600558852) showed why this must not pin wording: asked to "reply with exactly ATMOSPHERE_OK", the
* 1.5B model streamed {@code OK} — a perfectly working loop failing a prose assertion. What the wire
* contract guarantees is that the question reaches the model and its answer streams back, so that
* is what is asserted; the one content check is a fact no instruct model gets wrong.
*/
@Test
void plainChatStreamsAnAnswer() throws Exception {
ConsoleSession session = session();

runner(List.of()).run("Reply with exactly this word and nothing else: ATMOSPHERE_OK", List.of(), session);
runner(List.of()).run("What is 2 + 2? Answer with one short sentence.", List.of(), session);

assertThat(session.await(TURN_TIMEOUT), is(true));
assertThat(session.failure(), is(nullValue()));
assertThat(
"streamed text: " + session.text(), session.text().toUpperCase().contains("ATMOSPHERE_OK"), is(true));
assertThat("streamed text: " + session.text(), session.text().trim().isEmpty(), is(false));
assertThat("the answer must contain the number 4: " + session.text(), session.text(), containsString("4"));
assertThat(
"the answer must arrive as several SSE chunks, not one blob",
session.chunks().size(),
Expand Down
Loading