Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 15 additions & 9 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -2003,9 +2003,10 @@ missed again.)
release version now appears in only ~4 spots here, not ~20 — the runtime details live once in the
classifier table.)
- **`llama-langchain4j/README.md`** — its own `<dependency>` snippet.
- **`llama-atmosphere-agent/pom.xml`** — the `llama.version` property default (standalone project,
outside the reactor, so `versions:set` skips it), plus the `-Dllama.version=<version>` snippets
in the root README's "Local coding agent" section and the project's own README.
- **`llama-atmosphere-agent/pom.xml`** — the `llama.version` property (the **release** version;
standalone project outside the reactor, so `versions:set` skips it), plus the fat-jar filename
`llama-<version>-jar-with-dependencies.jar` in the root README's "Local coding agent" section and
the project's own README.
- **`llama-android/README.md`** and **`llama-kotlin/README.md`** — their Gradle dependency
snippets, plus the `llama-android`/`llama-kotlin` snippets in the root README's
"Importing in Android" section.
Expand Down Expand Up @@ -2142,8 +2143,10 @@ that pairs [Atmosphere](https://github.com/Atmosphere/atmosphere)'s built-in Ope
agent runtime with this project's `OpenAiCompatServer`. Like `android-llmservice/` it is a
**standalone Maven project, NOT a reactor module and NOT published** — it is an application, and it
needs Java 21 (Atmosphere's floor) while the core stays Java 8. CI builds it against the core it just
installed (`-Dllama.version=<reactor version>`); a user copies the folder, sets a released
`llama.version`, and runs `mvn compile exec:java -Dexec.args="…"`.
installed (`-Dllama.version=<reactor version>`); a user copies the folder and runs
`mvn compile exec:java -Dexec.args="…"` with no `-D` at all — the pom's `llama.version` names the
**released** core the READMEs describe (currently `5.2.0`, written as if released so the docs are
right the moment the release lands).

**What Atmosphere is, for this purpose.** `org.atmosphere:atmosphere-ai` (4.0.70) ships
`BuiltInAgentRuntime` + `OpenAiCompatibleClient`: a zero-framework OpenAI client that *always*
Expand Down Expand Up @@ -2192,10 +2195,13 @@ lines: `AiConfig.configure` → `BuiltInAgentRuntime` → `AgentExecutionContext
with `enableJinja()`, one-shot `--prompt` or a `you>` REPL with `/clear` `/exit`). Spotless (palantir)
is configured in its own pom; the model-free CI job runs `spotless:check`.

**Version bump note.** The pom's `llama.version` property defaults to the current reactor version
(CI always overrides it). `versions:set` does not touch this standalone pom, so bump the default by
hand together with the two README snippets (`README.md` "Local coding agent" + the project's own
README) — the same class as the `llama-langchain4j/README.md` snippet.
**Version bump note.** The pom's `llama.version` property is the **release** version, not the
reactor's `-SNAPSHOT` (CI always overrides it, so a not-yet-published default never breaks CI).
`versions:set` does not touch this standalone pom, so at release time bump it by hand together with
the fat-jar filename `llama-<version>-jar-with-dependencies.jar` in the two READMEs (`README.md`
"Local coding agent" + the project's own README) — the same class as the `llama-langchain4j/README.md`
snippet. Before the release, run it against a pre-release core with `-Dllama.version=<x>-SNAPSHOT`
(the pom keeps the Sonatype snapshot repository for exactly that).

## Android app "LLM Service" (`android-llmservice/`)

Expand Down
19 changes: 14 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1025,14 +1025,23 @@ A copy-and-run **terminal coding agent on the JVM** — Claude Code / OpenCode r
essentials, fully offline — built from [Atmosphere](https://github.com/Atmosphere/atmosphere)'s
built-in OpenAI-compatible agent runtime (streaming, tool loop, workspace file tools) driven
**headless** against this project's OpenAI-compatible server. It is a standalone Maven project (not a
reactor module, not published); you copy the folder and run it:
reactor module, not published); you copy the folder and run it. With java-llama.cpp already running
(`--jinja` is required for tool calling):

```bash
# against a server you started (java-llama.cpp fat jar with --jinja, or llama-server) ...
mvn -q compile exec:java -Dllama.version=<version> \
# 1. the server, e.g. from the release fat jar
java -jar llama-5.2.0-jar-with-dependencies.jar -m /models/Qwen2.5-7B-Instruct-Q4_K_M.gguf --jinja --port 8080

# 2. the agent, from the llama-atmosphere-agent/ folder — a you> prompt appears (/clear, /exit)
mvn -q compile exec:java \
-Dexec.args="--base-url http://127.0.0.1:8080/v1 --workspace /path/to/project --allow-shell"
# ... or with the GGUF loaded in-process
mvn -q compile exec:java -Dllama.version=<version> \

# a single turn instead of the prompt loop
mvn -q compile exec:java \
-Dexec.args="--base-url http://127.0.0.1:8080/v1 --workspace /path/to/project --prompt 'Read the README and summarize it'"

# or without a separate server: load the GGUF in-process
mvn -q compile exec:java \
-Dexec.args="--model /models/Qwen2.5-7B-Instruct-Q4_K_M.gguf --ngl 99 --workspace /path/to/project"
```

Expand Down
53 changes: 34 additions & 19 deletions llama-atmosphere-agent/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,42 +19,57 @@ essentials) that runs entirely on the JVM and entirely offline:
- **Shell:** an opt-in `run_command` tool (`--allow-shell`) so the model can build and test.

This folder is a **standalone Maven project**, deliberately *not* a reactor module and *not*
published: CI builds and tests it against the core of the same checkout; you copy the folder, set
`llama.version`, and run it.
published: CI builds and tests it against the core of the same checkout; you copy the folder and
run it. Its `pom.xml` pins `llama.version` to the release these instructions describe (**5.2.0**);
pass `-Dllama.version=…` to run against another core, e.g. a `-SNAPSHOT` before a release.

## Quick start

Requirements: JDK 21+ and Maven. No native toolchain: the core jar ships the natives.

**A. Against a server you run yourself** (you keep every llama.cpp flag):
**A. Against a java-llama.cpp server that is already running** (you keep every llama.cpp flag):

1. java-llama.cpp is running, for example started from the release fat jar like this
(`--jinja` is required for tool calling: it enables the model's tool-call chat template):

```bash
java -jar llama-5.2.0-jar-with-dependencies.jar -m /models/Qwen2.5-7B-Instruct-Q4_K_M.gguf --jinja --port 8080
```

Upstream `llama-server` with the same flags works too; any OpenAI-compatible endpoint does.

2. Start the agent from this folder (`llama-atmosphere-agent/`):

```bash
mvn -q compile exec:java \
-Dexec.args="--base-url http://127.0.0.1:8080/v1 --workspace /path/to/project --allow-shell"
```

A `you>` prompt appears. Type a request; the answer streams as it is generated, and every tool call
and its result are printed as `⚙ read_file {path=…}` / `↳ …` lines. `/clear` drops the history,
`/exit` quits.

A single turn without the REPL:

```bash
# 1. start java-llama.cpp's full upstream server (WebUI included) from the fat jar of a release;
# --jinja enables the model's tool-call template, which tool calling needs
java -jar llama-<version>-jar-with-dependencies.jar -m /models/Qwen2.5-7B-Instruct-Q4_K_M.gguf \
--jinja --port 8080 --api-key sk-local
# (or upstream llama-server with the same flags — any OpenAI-compatible endpoint works)

# 2. run the agent from this folder
mvn -q compile exec:java -Dllama.version=<version> \
-Dexec.args="--base-url http://127.0.0.1:8080/v1 --workspace /path/to/project --allow-shell"
mvn -q compile exec:java \
-Dexec.args="--base-url http://127.0.0.1:8080/v1 --workspace /path/to/project --prompt 'Read the README and summarize it'"
```

If the server was started with `--api-key <key>`, add `--api-key <key>` to the agent's arguments.

**B. In-process** (one command, the GGUF is loaded into the agent's JVM and served over a loopback
`OpenAiCompatServer`):
`OpenAiCompatServer` — no separately running server):

```bash
mvn -q compile exec:java -Dllama.version=<version> \
mvn -q compile exec:java \
-Dexec.args="--model /models/Qwen2.5-7B-Instruct-Q4_K_M.gguf --ngl 99 --workspace /path/to/project"
```

GPU natives: pick the core classifier, e.g. `-Dllama.classifier=cuda13-linux-x86-64` or
`vulkan-windows-x86-64` (the vendor runtime must be installed — see the root README's classifier
table). Without it the default CPU jar (incl. macOS Metal) is used.

Then type a request at the `you>` prompt (`/clear` drops the history, `/exit` quits), or run a single
turn with `--prompt "…"`. Streamed text appears as it is generated; every tool call and its result
are printed as `⚙ read_file {path=…}` / `↳ …` lines.
table). Without it the default CPU jar (incl. macOS Metal) is used. In mode A the classifier is
irrelevant: inference stays in the running server, the agent's JVM loads no model.

### Options

Expand Down
13 changes: 8 additions & 5 deletions llama-atmosphere-agent/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@ SPDX-License-Identifier: MIT
Standalone (NOT a reactor module, NOT published): a copy-and-run local coding agent that
wires Atmosphere's built-in OpenAI-compatible agent runtime to java-llama.cpp. CI builds and
tests it against the freshly installed core (-Dllama.version=<reactor version>); a user copies
the folder, picks a released llama.version and runs it with `mvn compile exec:java`.
the folder and runs it with `mvn compile exec:java` against the released core the
llama.version default names.
See README.md.
-->
<groupId>net.ladenthin</groupId>
Expand All @@ -38,9 +39,11 @@ SPDX-License-Identifier: MIT
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<!-- Atmosphere requires Java 21; the core net.ladenthin:llama stays Java 8. -->
<maven.compiler.release>21</maven.compiler.release>
<!-- The java-llama.cpp version to run against. CI overrides this with the reactor version it
just installed; a user picks a released version, e.g. -Dllama.version=5.2.0. -->
<llama.version>5.2.0-SNAPSHOT</llama.version>
<!-- The java-llama.cpp release this project is documented against (the README's commands
run without any -D). CI overrides it with the reactor version it just installed; anyone
else can override it too, e.g. -Dllama.version=5.3.0-SNAPSHOT for a pre-release core.
Bump it as part of every release. -->
<llama.version>5.2.0</llama.version>
<!-- Native backend of the core jar: empty = default CPU jar (incl. macOS Metal). GPU users pass
e.g. -Dllama.classifier=cuda13-linux-x86-64 or vulkan-windows-x86-64 (runtime on PATH). -->
<llama.classifier></llama.classifier>
Expand All @@ -57,7 +60,7 @@ SPDX-License-Identifier: MIT
<agent.main>net.ladenthin.llama.atmosphere.LocalAgent</agent.main>
</properties>

<!-- Lets the default -SNAPSHOT llama.version resolve without a local install of the core. -->
<!-- Only consulted for a -SNAPSHOT llama.version override (releases come from Maven Central). -->
<repositories>
<repository>
<id>sonatype-snapshots</id>
Expand Down
Loading