diff --git a/CLAUDE.md b/CLAUDE.md index 93b0fe83..62148c82 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -2003,9 +2003,10 @@ missed again.) release version now appears in only ~4 spots here, not ~20 — the runtime details live once in the classifier table.) - **`llama-langchain4j/README.md`** — its own `` snippet. -- **`llama-atmosphere-agent/pom.xml`** — the `llama.version` property default (standalone project, - outside the reactor, so `versions:set` skips it), plus the `-Dllama.version=` snippets - in the root README's "Local coding agent" section and the project's own README. +- **`llama-atmosphere-agent/pom.xml`** — the `llama.version` property (the **release** version; + standalone project outside the reactor, so `versions:set` skips it), plus the fat-jar filename + `llama--jar-with-dependencies.jar` in the root README's "Local coding agent" section and + the project's own README. - **`llama-android/README.md`** and **`llama-kotlin/README.md`** — their Gradle dependency snippets, plus the `llama-android`/`llama-kotlin` snippets in the root README's "Importing in Android" section. @@ -2142,8 +2143,10 @@ that pairs [Atmosphere](https://github.com/Atmosphere/atmosphere)'s built-in Ope agent runtime with this project's `OpenAiCompatServer`. Like `android-llmservice/` it is a **standalone Maven project, NOT a reactor module and NOT published** — it is an application, and it needs Java 21 (Atmosphere's floor) while the core stays Java 8. CI builds it against the core it just -installed (`-Dllama.version=`); a user copies the folder, sets a released -`llama.version`, and runs `mvn compile exec:java -Dexec.args="…"`. +installed (`-Dllama.version=`); a user copies the folder and runs +`mvn compile exec:java -Dexec.args="…"` with no `-D` at all — the pom's `llama.version` names the +**released** core the READMEs describe (currently `5.2.0`, written as if released so the docs are +right the moment the release lands). **What Atmosphere is, for this purpose.** `org.atmosphere:atmosphere-ai` (4.0.70) ships `BuiltInAgentRuntime` + `OpenAiCompatibleClient`: a zero-framework OpenAI client that *always* @@ -2192,10 +2195,13 @@ lines: `AiConfig.configure` → `BuiltInAgentRuntime` → `AgentExecutionContext with `enableJinja()`, one-shot `--prompt` or a `you>` REPL with `/clear` `/exit`). Spotless (palantir) is configured in its own pom; the model-free CI job runs `spotless:check`. -**Version bump note.** The pom's `llama.version` property defaults to the current reactor version -(CI always overrides it). `versions:set` does not touch this standalone pom, so bump the default by -hand together with the two README snippets (`README.md` "Local coding agent" + the project's own -README) — the same class as the `llama-langchain4j/README.md` snippet. +**Version bump note.** The pom's `llama.version` property is the **release** version, not the +reactor's `-SNAPSHOT` (CI always overrides it, so a not-yet-published default never breaks CI). +`versions:set` does not touch this standalone pom, so at release time bump it by hand together with +the fat-jar filename `llama--jar-with-dependencies.jar` in the two READMEs (`README.md` +"Local coding agent" + the project's own README) — the same class as the `llama-langchain4j/README.md` +snippet. Before the release, run it against a pre-release core with `-Dllama.version=-SNAPSHOT` +(the pom keeps the Sonatype snapshot repository for exactly that). ## Android app "LLM Service" (`android-llmservice/`) diff --git a/README.md b/README.md index ec983ffb..aafa47cd 100644 --- a/README.md +++ b/README.md @@ -1025,14 +1025,23 @@ A copy-and-run **terminal coding agent on the JVM** — Claude Code / OpenCode r essentials, fully offline — built from [Atmosphere](https://github.com/Atmosphere/atmosphere)'s built-in OpenAI-compatible agent runtime (streaming, tool loop, workspace file tools) driven **headless** against this project's OpenAI-compatible server. It is a standalone Maven project (not a -reactor module, not published); you copy the folder and run it: +reactor module, not published); you copy the folder and run it. With java-llama.cpp already running +(`--jinja` is required for tool calling): ```bash -# against a server you started (java-llama.cpp fat jar with --jinja, or llama-server) ... -mvn -q compile exec:java -Dllama.version= \ +# 1. the server, e.g. from the release fat jar +java -jar llama-5.2.0-jar-with-dependencies.jar -m /models/Qwen2.5-7B-Instruct-Q4_K_M.gguf --jinja --port 8080 + +# 2. the agent, from the llama-atmosphere-agent/ folder — a you> prompt appears (/clear, /exit) +mvn -q compile exec:java \ -Dexec.args="--base-url http://127.0.0.1:8080/v1 --workspace /path/to/project --allow-shell" -# ... or with the GGUF loaded in-process -mvn -q compile exec:java -Dllama.version= \ + +# a single turn instead of the prompt loop +mvn -q compile exec:java \ + -Dexec.args="--base-url http://127.0.0.1:8080/v1 --workspace /path/to/project --prompt 'Read the README and summarize it'" + +# or without a separate server: load the GGUF in-process +mvn -q compile exec:java \ -Dexec.args="--model /models/Qwen2.5-7B-Instruct-Q4_K_M.gguf --ngl 99 --workspace /path/to/project" ``` diff --git a/llama-atmosphere-agent/README.md b/llama-atmosphere-agent/README.md index 2cdefae6..126fb257 100644 --- a/llama-atmosphere-agent/README.md +++ b/llama-atmosphere-agent/README.md @@ -19,42 +19,57 @@ essentials) that runs entirely on the JVM and entirely offline: - **Shell:** an opt-in `run_command` tool (`--allow-shell`) so the model can build and test. This folder is a **standalone Maven project**, deliberately *not* a reactor module and *not* -published: CI builds and tests it against the core of the same checkout; you copy the folder, set -`llama.version`, and run it. +published: CI builds and tests it against the core of the same checkout; you copy the folder and +run it. Its `pom.xml` pins `llama.version` to the release these instructions describe (**5.2.0**); +pass `-Dllama.version=…` to run against another core, e.g. a `-SNAPSHOT` before a release. ## Quick start Requirements: JDK 21+ and Maven. No native toolchain: the core jar ships the natives. -**A. Against a server you run yourself** (you keep every llama.cpp flag): +**A. Against a java-llama.cpp server that is already running** (you keep every llama.cpp flag): + +1. java-llama.cpp is running, for example started from the release fat jar like this + (`--jinja` is required for tool calling: it enables the model's tool-call chat template): + + ```bash + java -jar llama-5.2.0-jar-with-dependencies.jar -m /models/Qwen2.5-7B-Instruct-Q4_K_M.gguf --jinja --port 8080 + ``` + + Upstream `llama-server` with the same flags works too; any OpenAI-compatible endpoint does. + +2. Start the agent from this folder (`llama-atmosphere-agent/`): + + ```bash + mvn -q compile exec:java \ + -Dexec.args="--base-url http://127.0.0.1:8080/v1 --workspace /path/to/project --allow-shell" + ``` + +A `you>` prompt appears. Type a request; the answer streams as it is generated, and every tool call +and its result are printed as `⚙ read_file {path=…}` / `↳ …` lines. `/clear` drops the history, +`/exit` quits. + +A single turn without the REPL: ```bash -# 1. start java-llama.cpp's full upstream server (WebUI included) from the fat jar of a release; -# --jinja enables the model's tool-call template, which tool calling needs -java -jar llama--jar-with-dependencies.jar -m /models/Qwen2.5-7B-Instruct-Q4_K_M.gguf \ - --jinja --port 8080 --api-key sk-local -# (or upstream llama-server with the same flags — any OpenAI-compatible endpoint works) - -# 2. run the agent from this folder -mvn -q compile exec:java -Dllama.version= \ - -Dexec.args="--base-url http://127.0.0.1:8080/v1 --workspace /path/to/project --allow-shell" +mvn -q compile exec:java \ + -Dexec.args="--base-url http://127.0.0.1:8080/v1 --workspace /path/to/project --prompt 'Read the README and summarize it'" ``` +If the server was started with `--api-key `, add `--api-key ` to the agent's arguments. + **B. In-process** (one command, the GGUF is loaded into the agent's JVM and served over a loopback -`OpenAiCompatServer`): +`OpenAiCompatServer` — no separately running server): ```bash -mvn -q compile exec:java -Dllama.version= \ +mvn -q compile exec:java \ -Dexec.args="--model /models/Qwen2.5-7B-Instruct-Q4_K_M.gguf --ngl 99 --workspace /path/to/project" ``` GPU natives: pick the core classifier, e.g. `-Dllama.classifier=cuda13-linux-x86-64` or `vulkan-windows-x86-64` (the vendor runtime must be installed — see the root README's classifier -table). Without it the default CPU jar (incl. macOS Metal) is used. - -Then type a request at the `you>` prompt (`/clear` drops the history, `/exit` quits), or run a single -turn with `--prompt "…"`. Streamed text appears as it is generated; every tool call and its result -are printed as `⚙ read_file {path=…}` / `↳ …` lines. +table). Without it the default CPU jar (incl. macOS Metal) is used. In mode A the classifier is +irrelevant: inference stays in the running server, the agent's JVM loads no model. ### Options diff --git a/llama-atmosphere-agent/pom.xml b/llama-atmosphere-agent/pom.xml index d6feebc9..34c19ba1 100644 --- a/llama-atmosphere-agent/pom.xml +++ b/llama-atmosphere-agent/pom.xml @@ -13,7 +13,8 @@ SPDX-License-Identifier: MIT Standalone (NOT a reactor module, NOT published): a copy-and-run local coding agent that wires Atmosphere's built-in OpenAI-compatible agent runtime to java-llama.cpp. CI builds and tests it against the freshly installed core (-Dllama.version=); a user copies - the folder, picks a released llama.version and runs it with `mvn compile exec:java`. + the folder and runs it with `mvn compile exec:java` against the released core the + llama.version default names. See README.md. --> net.ladenthin @@ -38,9 +39,11 @@ SPDX-License-Identifier: MIT UTF-8 21 - - 5.2.0-SNAPSHOT + + 5.2.0 @@ -57,7 +60,7 @@ SPDX-License-Identifier: MIT net.ladenthin.llama.atmosphere.LocalAgent - + sonatype-snapshots