diff --git a/.github/verify-patches-applied.sh b/.github/verify-patches-applied.sh index 5797b41e..64ec0c88 100755 --- a/.github/verify-patches-applied.sh +++ b/.github/verify-patches-applied.sh @@ -13,7 +13,6 @@ # # 0003, 0006, 0007, 0008 -> jllama.cpp / native_server.cpp call the symbols they add, # so dropping one is a compile or link error. -# 0011 -> the ContentOnlyParseUtf8 tests in src/test/cpp/test_utils.cpp. # 0012 -> src/test/cpp/test_model_split.cpp. # 0001, 0002 -> model-gated Java jobs (Windows argv, LoadProgressCallbackTest). # diff --git a/CHANGELOG.md b/CHANGELOG.md index 45d98733..f7cd31bf 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -58,6 +58,21 @@ from version 5.0.0 onward. Pre-fork releases (`1.x`–`4.2.0`) were authored by where the backend cannot provide it, `OFF` disables it. ### Changed +- **llama.cpp `b11062` → `b11069`, and local patch `0011` dropped — malformed UTF-8 in a + completion is now replaced, not truncated.** Seven upstream commits, 57 KiB, no project-source + change; the one that matters is llama.cpp #29161 ("common/peg : handle invalid utf-8 sequences in + the AST", first tagged b11063). It fixes the failure `0011` had carried since 5.1.0 — a single + undecodable byte in the model's output made the content-only parse `FAIL` and the request 500 — + independently and more broadly than the patch did, so the patch no longer applies and was dropped + rather than refreshed (the `0009`/`0013` precedent). The observable difference: `0011` returned the + text *up to* the bad byte, whereas upstream consumes every undecodable run and substitutes exactly + one U+FFFD for it (the Unicode "maximal subpart" rule — `\xE4\xB8` followed by `c` is one run, + `\xFF\xFE` is two), so the text *after* the byte is now delivered too. A trailing sequence that is + still incomplete at the end of the input keeps being withheld, as before. The `ContentOnlyParseUtf8` + C++ tests that guarded the patch now pin upstream's replacement contract on every platform. The + rest of the range is CUDA/Metal/WebGPU kernel tuning and a converter flag; `tools/server/`, + `common/arg.*`, `src/llama-model.*` and `ggml/include` are byte-identical across it, so the other + eight patches apply unchanged and the server wire contract cannot have moved. - **llama.cpp `b10731` → `b10850`.** No project-source change: every header move in the range is additive or a **widening** const-qualification, and the server wire contract is byte-identical (request-field set, `set_hard_limits` bounds and response keys all verified mechanically, which is diff --git a/CLAUDE.md b/CLAUDE.md index 397e3ea0..743f3755 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI. -Current llama.cpp pinned version: **b11062** +Current llama.cpp pinned version: **b11069** ## Upgrading CUDA Version @@ -510,7 +510,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi ships no UI): ```bash # needs node/npm + network for the asset build; the embed step is plain cmake -P -git clone --depth 1 --branch b11062 https://github.com/ggml-org/llama.cpp /tmp/lc +git clone --depth 1 --branch b11069 https://github.com/ggml-org/llama.cpp /tmp/lc ( cd /tmp/lc/tools/ui && npm ci && npm run build ) mkdir -p webui-generated /tmp/ui-gen cmake -DUI_SOURCE_DIR=/tmp/lc/tools/ui -DUI_BINARY_DIR=/tmp/ui-gen \ @@ -550,7 +550,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend: - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored as the repo secret **`DEPOT_TOKEN`**. -Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b11062`), the +Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b11069`), the ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's per-branch `actions/cache`), so every branch builds incrementally; a `b` version bump @@ -754,10 +754,33 @@ Current patches: | `0007-server-attach-http-frontend.patch` | **Adds `llama_server_attach(argc, argv, server_context&)`** so the `NativeServer` *attach mode* can serve an **already-loaded `LlamaModel`** over the upstream HTTP frontend — no second model load, no `start_loop()`; the LlamaModel's worker keeps driving the shared `server_context` and the HTTP routes post tasks to its queue (the queue is the synchronization point). Mechanically: (1) extracts the **pure core route table** (`health` … `slots`) out of `llama_server()` into `static void llama_server_register_common_routes(ctx_http, routes)` (shared, so the two entry points cannot drift on the core endpoint set). **Scope note (narrowed at the b10154 bump):** the helper deliberately carries **only** the stable, state-independent route table — **not** the resumable-streaming routes (their handlers differ between router / non-router), the GCP-compat shim, or the experimental **CORS-proxy / MCP-server / built-in-tools** wiring. b10154 (upstream MCP-server support) moved the streaming routes into the middle of that block and coupled tools/CORS to a per-call `server_mcp mcp_mgr` lifecycle, so the earlier contiguous "route-table + CORS-proxy + tools" extraction is no longer possible; `llama_server()` keeps all of that inline, **byte-identical to upstream b10154** (only the route-table block is factored out). (2) adds `llama_server_attach`, which parses only the HTTP-side argv via `common_params_parse`, starts the stream-session GC + `server_http_context`, registers the common route table, the **non-router** resumable-streaming handlers (upstream b10154 paths `/v1/stream` GET/DEL + `/v1/streams/lookup` POST), the GCP-compat shim, and **403 "disabled" stubs for `/cors-proxy` + `/tools`** (attach mode does not wire the experimental CORS-proxy / MCP / built-in-tools host — those belong to a full `llama-server`, not an embedded model), marks ready immediately (model already loaded), and blocks on the HTTP thread until `llama_server_request_shutdown()` — never calling `common_init()`, backend init, `ctx_server.terminate()` or `llama_backend_free()` (the embedding caller owns those). Applies after `0001`+`0006` (same file); closes the "NativeServer — reuse an already-loaded LlamaModel" TODO. Upstream-submittable ("server: let embedding callers attach the HTTP frontend to an existing server_context"). **Refreshed at the b10519 bump:** upstream #26347 dropped the API key from the `/models` + `/v1/models` public-endpoint set and deleted the two trailing `// public endpoint (no API key check)` comments on those route registrations. Those two lines sit inside this patch's route-table removal block, so `git apply` failed ("patch does not apply", `server.cpp:258`) at **every** tag from b10519 on; the fix was to drop the now-wrong comment from all four affected lines (2 on the `-` side, 2 in the extracted helper on the `+` side), keeping the helper byte-identical to the block it replaces. **This is the invariant to re-check on every bump:** the `+` side of `llama_server_register_common_routes()` must stay a verbatim copy of the route table it factors out of `llama_server()`. | | `0008-server-models-worker-cmd-override.patch` | **Makes router mode usable in-JVM.** The router (`server-models.cpp`) spawns each model worker by re-executing its own binary (`get_server_exec_path()` = `/proc/self/exe` & friends) — inside a JVM that binary is `java`, not a llama-server, so embedded router workers could never start. The patch adds env `LLAMA_SERVER_WORKER_CMD` (whitespace-split; read in `server_model_meta::update_args`) which replaces only the leading binary-path token of the rendered worker args, letting an embedding host relaunch workers through its own bootstrap — e.g. `java -cp app.jar net.ladenthin.llama.server.NativeServer` (each worker is then a fresh JVM running the classic single-model `NativeServer`). Exposed in Java as `NativeServer.setWorkerCommand(String...)` (JNI `setenv`); exercised by `RouterModeIntegrationTest` (Linux CI). Upstream-submittable (also useful for containerized/wrapped deployments). | | `0010-server-cast-vocab-type-for-common-json.patch` | **Upstream regression from the b10585 `common_json` switch (#27511), one line.** `get_res_model_info()` (`tools/server/server-context.cpp`) builds the `GET /models` + `GET /v1/models` payload and emits `{"vocab_type", meta.model_vocab_type}` — an **unscoped enum**. `common_json_value`'s integral constructor template is `std::is_integral`-gated, which *excludes* enums, so the value binds to `common_json_value(bool)` and serialises as `true`/`false` instead of the numeric vocab type. It was correct while the alias was `nlohmann::ordered_json` (nlohmann serialises an enum as an integer), so upstream regressed it silently when they flipped the alias. The project ships this: `server-context.cpp` is compiled into `libjllama` and both routes are served by `NativeServer` — the default fat-jar `Main-Class` — in full **and** attach mode (`patches/0007`'s common route table registers them). The patch casts the value to `int` at the emit site, mirroring what `jllama.cpp` does for its own two `"vocab_type"` sites. Upstream-submittable; **not yet filed upstream**. Applies after `0002`/`0003` (same file) — numbered `0010` because `0009` is burned: it names the subprocess.h patch dropped at the b10280 bump (see the note below this table), and reusing the number would make that note read as if it were about this patch. **On every bump, check whether upstream cast the value themselves; if they did, DROP this patch rather than refreshing it** — the fail-loud applier only detects "does not apply", never "upstream already fixed this", and no test can catch a redundant carry here because `get_res_model_info` is `static` inside `server-context.cpp` and unreachable from `jllama_test`. See the `CommonJsonEnumTrap` tests in `test_json_helpers.cpp` for the mechanism the cast defends against. | -| `0011-peg-parser-lenient-invalid-utf8.patch` | **A model that emits one malformed UTF-8 byte turns a finished generation into an HTTP 500.** The server parses *every* completion through `common_chat_parse()`; with no chat parser configured (plain `/completion`) that is the content-only fallback `content(rest()) + end()`, whose scan is `common_peg_until_parser` (`common/peg-parser.cpp`). `common_chat_peg_parse()` always parses in **lenient** mode, and that scan tolerates an `INCOMPLETE` trailing UTF-8 sequence by keeping the text before it — but the `INVALID` branch right below it returns `FAIL` unconditionally, ignoring leniency. One stray byte anywhere in the generated text therefore throws `"The model produced output that does not match the expected Content-only format"` and the request 500s even though generation completed normally (`stop processing: n_tokens = 4, truncated = 0`). The patch makes the `INVALID` branch respect `ctx.is_lenient()` exactly like the `INCOMPLETE` branch — keep the text up to the malformed byte — and adds an upstream `tests/peg-parser/test-unicode.cpp` case pinning both the lenient and the still-failing strict behavior. **Strict mode is unchanged**, which is what keeps upstream's own tests green: `tests/peg-parser/test-unicode.cpp` *does* assert `FAIL` on invalid UTF-8 through the *until* parser (a `malformed UTF-8` block with three `p.until("")` cases), but each builds a bare `common_peg_parse_context` with no `COMMON_PEG_PARSE_FLAG_LENIENT`, so the lenient-only change cannot reach them. This patch adds its case inside that same block. Found by `NativeServerAttachIntegrationTest.completion_overHttp_served`, which 500s on all six Java CI platforms. Upstream-submittable; **not yet filed upstream**. Touches only `common/peg-parser.cpp` + that test, which no other patch touches, so it is independent of `0001`/`0006`/`0007`. Runnable guard: the `ContentOnlyParseUtf8` tests in `src/test/cpp/test_utils.cpp` — unlike the upstream test they are compiled and run in CI on every platform, so a bump that drops this patch reds `C++ Tests` instead of one Java job. | | `0006-server-embed-native-server-jni.patch` | **Makes `server.cpp`'s `llama_server` embeddable in the JVM** so the `NativeServer` JNI bridge can run the full upstream HTTP server (WebUI included) inside `libjllama` — see "Two server modes" below. b9870 already exposes `int llama_server(int, char**)` (non-static; no `main` in the file), so the patch only adds embedded-mode support: (1) a `g_llama_server_embedded` flag + `llama_server_set_embedded()` / `llama_server_request_shutdown()` (declared in the committed `src/main/cpp/native_server_bridge.h`); (2) skips installing the process-wide SIGINT/SIGTERM handlers when embedded (they would hijack the JVM's); (3) in embedded mode parses the **forwarded** argv via `common_params_parse` instead of `common_params_parse_main` (whose `GetCommandLineW` recovery would pick up `java.exe`'s command line — the same Windows class of bug `0001` fixes). `llama_server_request_shutdown()` mirrors the SIGTERM path (invokes the installed `shutdown_handler` → `ctx_server.terminate()` unblocks `start_loop()`), giving JNI an out-of-band stop since `ctx_server` is loop-local. Applies **after `0001`** (which flips this call site to `common_params_parse_main`), so its context is the post-`0001` tree; regenerate against `0001`+source on a bump. Only touches `tools/server/server.cpp`. | | `0012-model-guard-zero-split-sum-and-name-the-device-index.patch` | **A GPU that reports zero free memory makes every model load fail with the unactionable `error loading model: vector`.** `llama_model_base::load_tensors` (`src/llama-model.cpp`) weights the per-device layer split by `ggml_backend_dev_memory()`'s `free`, then normalises: `splits[i] /= split_sum`. With a single device reporting `free == 0` that is `0/0` → **NaN** in every split point; NaN compares false against everything, so the `std::upper_bound` below returns the end iterator, `layer_gpu == n_devices()`, and `devices.at(layer_gpu)` throws `std::out_of_range` — whose libc++ `what()` is the bare string `"vector"`, which `llama.cpp`'s `catch (const std::exception &)` prints verbatim. Upstream's `free == 0 && total == 0` host-memory fallback does **not** fire, because `total` is `recommendedMaxWorkingSetSize` and is non-zero. **Reachable since b10618..b10797**: upstream `8c0b9cd04` ("metal : fix memory query under low-memory conditions", [#27701](https://github.com/ggml-org/llama.cpp/pull/27701)) changed `ggml-metal-device.m` to `*free = *total > cur ? *total - cur : 0`; before that clamp an over-committed device (`currentAllocatedSize > recommendedMaxWorkingSetSize`) *underflowed* to a huge `size_t`, which normalised fine, so the same precondition was harmless. That is why the `Java Tests macOS …` jobs went red at the b10792→b10797 step while every Linux/Windows job stayed green — **and why only a GPU build can fail this way at all**: `act_gpu_layers` is `devices.empty() ? 0 : …`, so with no GPU backend `devices` is empty, every layer returns early on `cpu_dev`, and the `.at()` line is unreachable. **Shape:** the two blocks are lifted out of `load_tensors` into free functions declared in `src/llama-model.h`, purely so they can be driven by a test — the failing state needs a real over-committed GPU and cannot be arranged through any public API. `llama_model_splits_normalize()` carries **the fix**: on `split_sum == 0` it `LLAMA_LOG_WARN`s and falls back to an even split (`splits[i] = float(i+1)/splits.size()`), the only neutral choice when no device can be preferred and exactly right for a single device. `llama_model_splits_select_device()` carries **the diagnostic**: it bounds-checks the index and throws a `std::runtime_error` naming the function, the offloaded layer, the device index, the split-point count **and the split points themselves** — with NaN splits that message prints `nan` and names the cause outright, which is precisely what was missing when this had to be diagnosed by reading source. **A second, backend-independent trigger reaches the same line**, found while writing this up and verified against the unfixed library: `--tensor-split` values are parsed with `std::stof` and never range-checked (`common/arg.cpp`), so `-ts 1,-1` cancels out, `split_sum` is 0 again, the split points become `[inf, -nan]`, and every layer maps one past the last device — on CUDA, Vulkan or ROCm just as much as on Metal, with no memory pressure involved. That is what makes this an ordinary upstream defect rather than a Metal edge case, and the warning names both causes rather than only the memory one. Also adds upstream `tests/test-model-split.cpp` (5 cases in upstream's `testing.h` style) + its `llama_build_and_test` registration. Touches `src/llama-model.{cpp,h}`, `tests/test-model-split.cpp` and `tests/CMakeLists.txt` — **none** of which any other patch touches, so it is independent of all of them. Upstream-submittable ("model: fall back to an even split when no device reports free memory"); **not yet filed upstream**. **Runnable guard: `src/test/cpp/test_model_split.cpp`** — a FetchContent subproject builds with `LLAMA_BUILD_TESTS=OFF`, so the upstream test above is applied-but-never-compiled here (same as `0001`'s test). That file drives the same two functions from `jllama_test`, which runs on **every** platform in `C++ Tests`, so a bump that drops this patch fails the build at link time everywhere instead of surfacing as one red macOS Java job. **Verification limit — read before assuming this can be dropped:** the *failing path* still cannot be reached without a GPU backend, so the guard pins the arithmetic (what actually broke), not the end-to-end load; the end-to-end proof is the macOS CI job. On a bump, re-check whether upstream added its own `split_sum == 0` guard (grep `split_sum` in `src/llama-model.cpp`) and **drop this patch rather than refreshing it** if they did — the fail-loud applier detects "does not apply", never "upstream already fixed this". | +**`0011` was dropped at the b11069 bump.** Upstream merged +[ggml-org/llama.cpp#29161](https://github.com/ggml-org/llama.cpp/pull/29161) +("common/peg : handle invalid utf-8 sequences in the AST", commit `3d82ef62d`, first tagged at +**b11063**) — an independent and broader fix for the same defect; the patch itself had never been +filed upstream. Where `0011` made the until-parser's `INVALID` branch honour `ctx.is_lenient()` by +returning what was scanned *before* the bad byte (so the text after it was lost), upstream now +**consumes** every undecodable run, in strict mode too, records its `{pos, len}` on the AST node +(`common_peg_invalid_utf8`, carried up through `common_peg_parse_result`), and +`common_chat_peg_mapper` emits `node.sanitized_text()`, which substitutes exactly one U+FFFD per run +(the Unicode "maximal subpart" rule: `\xE4\xB8` followed by `c` is one run, `\xFF\xFE` is two). +`common/unicode.cpp` now reports the valid-prefix length in `bytes_consumed` for `INVALID` / +`INCOMPLETE` results to make that possible. The lenient incomplete-at-end-of-input branch (withhold +the trailing bytes, more may arrive on a stream) is unchanged. The applier failed loud with "does not +apply" at `common/peg-parser.cpp:680` exactly as designed — the `INVALID` branch it patched no longer +exists. Dropped, not refreshed, per the `0009`/`0013` precedent. **The runnable guard stays, +re-pointed:** the `ContentOnlyParseUtf8` tests in `src/test/cpp/test_utils.cpp` now pin upstream's +replacement contract (invalid byte → U+FFFD with the text after it kept; a trailing incomplete +sequence still withheld), so a future upstream revert to `FAIL`, or a change to the replacement rule, +still reds `C++ Tests` on every platform. The strict-mode `FAIL` on invalid UTF-8 that `0011` +deliberately preserved is gone upstream as well — `tests/peg-parser/test-unicode.cpp`'s +`malformed UTF-8` block now asserts `SUCCESS` plus the sanitized text. **Behavioural note for +consumers:** a completion containing a stray byte now returns the *full* text with U+FFFD in place of +the byte, where the patched builds returned the text up to the byte. + **`0013` was dropped at the b10948 bump.** Upstream merged this project's own PR [ggml-org/llama.cpp#28775](https://github.com/ggml-org/llama.cpp/pull/28775) ("ggml-cpu(s390x): guard VXE-only repack helpers", commit `6978052`, first tagged at **b10948**): @@ -1587,7 +1610,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson" | File | Tests | Scope | |------|-------|-------| -| `src/test/cpp/test_utils.cpp` | 167 | Upstream helpers: `server_tokens`, `server_grammar_trigger`, `gen_tool_call_id`, `json_value`, `json_get_nested_values`, UTF-8 helpers, `format_response_rerank`, `format_embeddings_response_oaicompat`, `oaicompat_completion_params_parse`, `oaicompat_chat_params_parse`, `are_lora_equal`, `strip_flag_from_argv`, `token_piece_value`, `json_is_array_and_contains_numbers`, `format_oai_sse`, `format_oai_resp_sse`, `format_anthropic_sse`, `parse_lora_request`, `common_chat_parse` over malformed UTF-8 (the `ContentOnlyParseUtf8` guard for `patches/0011`) | +| `src/test/cpp/test_utils.cpp` | 168 | Upstream helpers: `server_tokens`, `server_grammar_trigger`, `gen_tool_call_id`, `json_value`, `json_get_nested_values`, UTF-8 helpers, `format_response_rerank`, `format_embeddings_response_oaicompat`, `oaicompat_completion_params_parse`, `oaicompat_chat_params_parse`, `are_lora_equal`, `strip_flag_from_argv`, `token_piece_value`, `json_is_array_and_contains_numbers`, `format_oai_sse`, `format_oai_resp_sse`, `format_anthropic_sse`, `parse_lora_request`, `common_chat_parse` over malformed UTF-8 (the `ContentOnlyParseUtf8` guard, which pins upstream #29161's one-U+FFFD-per-invalid-run contract — formerly the guard for the dropped `patches/0011`) | | `src/test/cpp/test_server.cpp` | 206 | Upstream result types: `server_slot_stats` (the `timings` JSON payload; replaced `result_timings` in b10408), `task_params::to_json()` (incl. `dry_sequence_breakers`, `preserved_tokens`, `timings_per_token`), `completion_token_output`, `server_task_result_cmpl_partial` (non-oaicompat + `to_json_oaicompat` + logprobs + `to_json_oaicompat_chat` + `to_json_anthropic` + dispatcher), `server_task_result_cmpl_final` (non-oaicompat + `to_json_oaicompat` + `to_json_oaicompat_chat` + `to_json_oaicompat_chat_stream` + `to_json_anthropic` + `to_json_anthropic_stream` + tool_calls + dispatcher), `server_task_result_embd`, `server_task_result_rerank`, `server_task_result_metrics` (`to_metrics()` = the `/metrics` Prometheus exposition text; its `to_json()` has been unused since b10519 and returns `json{}` = JSON null), `server_task_result_slots` (`to_json()` = the `/slots` array, fed by the b10519 `SERVER_TASK_TYPE_SLOT_GET` task), `server_task_result_slot_save_load`, `server_task_result_slot_erase`, `server_task_result_apply_lora`, `server_task_result_get_lora`, `server_task_result_error`, `format_error_response`, `server_task::need_sampling()`, `server_task::n_tokens()`, `server_schema::eval_llama_cmpl_schema()` (parsing pipeline + grammar routing + error paths + per-request `dry_*` and `sse_ping_interval` field round-trips incl. hard-limit + server-default inheritance), `response_fields` projection | | `src/test/cpp/test_json_helpers.cpp` | 63 | All functions in `json_helpers.hpp`: `get_result_error_message`, `results_to_json`, `rerank_results_to_json` (incl. missing/out-of-range `index` rejection), `parse_encoding_format`, `extract_embedding_prompt`, `is_infill_request`, `parse_slot_prompt_similarity`, `parse_positive_int_config`, `wrap_stream_chunk`, `server_metrics_to_json` | | `src/test/cpp/test_log_helpers.cpp` | 13 | All functions in `log_helpers.hpp`: `log_level_name`, `format_log_as_json` | @@ -1598,11 +1621,11 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson" | `src/test/cpp/test_model_flags.cpp` | 4 | **The contract between the Java CLI-flag registries and llama.cpp's server argument parser.** CMake reads `ModelFlag.java` + `ModelOption.java` (`cmake/extract-java-wire-names.cmake` → a generated header of `{name, contract}` pairs), and this file asserts every `SERVER_PARSER` name is in `common_params_parser_init(params, LLAMA_EXAMPLE_SERVER).options`. It exists because **no Java test can catch this class**: `ModelFlagTest`/`ModelParametersExtendedTest` pin the *string mapping* (`hasKey("--mlock")`), never that llama.cpp still accepts the string, so they stay green forever while the flag is dead — and `common_params_parse` treats an unregistered option as a hard error, so the affected builder method makes the model **unloadable**, not merely ineffective. **A grep over `arg.cpp` is not a substitute**: `--grp-attn-n`/`-w` are present there at every pinned tag but `set_examples()`-scoped to `LLAMA_EXAMPLE_COMPLETION`/`PASSKEY`, so the server parser rejects them exactly like a deleted flag — only the real option table sees that. `--vocab-only` is the one exemption, and it declares itself `CliContract.PROJECT_PSEUDO` on its own constant rather than appearing in a list inside this file; the test asserts such a name is **still unknown** to the parser (an exemption upstream later registers would be hiding a real check) and that the exempt set is non-empty. | | `src/test/cpp/test_wire_contracts.cpp` | 6 | **The same contract for the two quieter surfaces.** `RequestField` against `server_schema::make_llama_cmpl_schema(...)` (5 tests) and `TrainingField` against `jllama_train::config_keys()` (1 test). Both receivers *silently ignore* an unknown key — the schema skips it, `train_engine.cpp` reads with `j.value(key, default)` and falls back — so a dead field produces no error anywhere and every string-mapping test keeps passing. `OAI_LAYER`-declared keys (consumed by `oaicompat_*_params_parse` before the schema) are exempt from the schema check, and are checked **both** ways: still unknown to the schema (the inverted check), and read by at least one upstream reader-shaped site (the configure-time sweep — this is what caught `chat_template`, a key a public builder wrote and nothing read). See [`docs/history/parameter-wire-surface.md`](docs/history/parameter-wire-surface.md). | -**Current total: 551 tests (all passing).** +**Current total: 552 tests (all passing).** #### Upstream source location (in CMake build tree) -llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b11062`. +llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b11069`. **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.18.0`), used solely by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the diff --git a/README.md b/README.md index e6c80980..7b8573f0 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ **Build:** ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational) ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey) -[![llama.cpp b11062](https://img.shields.io/badge/llama.cpp-%23b11062-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11062) +[![llama.cpp b11069](https://img.shields.io/badge/llama.cpp-%23b11069-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b11069) [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/) ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162) [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev) diff --git a/TODO.md b/TODO.md index 0420720f..5927fc88 100644 --- a/TODO.md +++ b/TODO.md @@ -118,10 +118,6 @@ be described here as "drops automatically when that merges"; it will not.) - **`0010` cast `vocab_type` for `common_json`** (one line; upstream regressed `GET /models` + `GET /v1/models` to emit `true`/`false` instead of the numeric vocab type when they flipped the `json` alias to `common_json` at b10585/#27511). **Not yet filed upstream.** -- **`0011` lenient invalid-UTF-8 in the PEG parser** (one malformed byte from the model turns a - finished generation into an HTTP 500; the `INVALID` branch ignores leniency while the `INCOMPLETE` - branch beside it honours it). Ships an upstream `tests/peg-parser/test-unicode.cpp` case. - **Not yet filed upstream.** - **`0012` guard the zero split-sum and name the device index** (a GPU reporting zero free memory — or a cancelling `--tensor-split` such as `-ts 1,-1` on any backend — makes every model load fail with the unactionable `error loading model: vector`). Ships an upstream `tests/test-model-split.cpp`. @@ -129,8 +125,11 @@ be described here as "drops automatically when that merges"; it will not.) (`0009` is **not** in this list and the number is burned: upstream merged the subprocess.h fix via ggml-org/llama.cpp#26606, so the patch was dropped at the b10280 bump. `0013` is likewise gone — -upstream merged this project's own PR ggml-org/llama.cpp#28775 and it was dropped at b10948. Both -drops are recorded in `CLAUDE.md` under the patch table.) +upstream merged this project's own PR ggml-org/llama.cpp#28775 and it was dropped at b10948. `0011` +went the same way at b11069: upstream fixed the invalid-UTF-8 PEG-parser failure independently and +more broadly via ggml-org/llama.cpp#29161 (one U+FFFD per undecodable run, text after it kept) before +the patch was ever filed, so the `ContentOnlyParseUtf8` guard now pins upstream's contract instead. +All three drops are recorded in `CLAUDE.md` under the patch table.) ### llama.cpp upstream feature exposure (queued, deferred by policy) diff --git a/docs/history/llama-cpp-breaking-changes.md b/docs/history/llama-cpp-breaking-changes.md index 7a218e25..eba1e6da 100644 --- a/docs/history/llama-cpp-breaking-changes.md +++ b/docs/history/llama-cpp-breaking-changes.md @@ -740,3 +740,5 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r | b11012–b11018 | patches + upstream verification | **Nine patches, none touched and none droppable.** No patch-target file appears anywhere in the range, so `common/arg.{cpp,h}`, `common/peg-parser.cpp`, every `tools/server/*.cpp`, `src/llama-model.{cpp,h}` and `tests/CMakeLists.txt` are byte-unchanged. **All six standing drop-checks still say "still required"**, run against the pristine tag because the fail-loud applier detects "does not apply" but never "upstream already fixed this": `0001` (`common_params_parse_main` 0 occurrences in `b11018:common/arg.h`; WIN32 override still at `common/arg.cpp:1282`), `0002` (`params_base.load_progress_callback` still unguarded at `server-context.cpp:1095`), `0010` (`{"vocab_type", meta.model_vocab_type}` still uncast at `server-context.cpp:4554`), `0012` (bare `splits[i] /= split_sum;` still at `src/llama-model.cpp:1518` — unmoved from b11012), and `0003`/`0006`/`0008` (`get_slot_prompt_similarity`, `llama_server_set_embedded`, `LLAMA_SERVER_WORKER_CMD` all absent from `b11018:tools/server/`). Verified from a fresh configure: stamp at head `c9a5eeeb3` with nine SHA-256 lines, `verify-patches-applied.sh` green, extraction unchanged at **138 CLI / 57 request / 15 trainer** names, Release build clean with zero errors and zero warnings, `ctest` **551/551**, `nm -D` **40** `Java_*` exports and **0** mangled, `NativeLibraryLoadSmokeTest` **4/4, 0 skipped** after a `clean`, full `mvn test` **1763/0**, SpotBugs **0**, spotless clean. **Context worth recording: the previous range's PR run (#958, the b11012 PR) was the first full-matrix execution since b10948** — 66 jobs, **58 success / 2 failure / 6 skipped**, the two failures being the `Verify GPG signing key` pair that `publish.yml` documents as an expected red on a `pull_request` event (the `maven-central` environment withholds secrets there). That run is what first exercised the trainer-model wiring, `verify-test-counts.sh` and both aarch64 fat-jar smoke jobs added earlier in the same session; all passed. | | b11018–b11062 | 36 commits, **1 347 KiB**, and the chunking is the first thing worth recording: the range was walked in **nine** steps — `b11018→b11020` (10 KiB / 2 commits), `→b11022` (579 / 2), `→b11024` (224 / 2), `→b11042` (87 / 18), `→b11045` (98 / 3), `→b11050` (57 / 5), `→b11052` (115 / 2), `→b11055` (97 / 3) and `→b11062` (84 / 7). **Three steps break the 100 KiB rule and all three are irreducible**: b11021, b11023 and b11051 do not exist as tags, so each of those steps is a *single* upstream commit with no smaller step available — **#28732** (Vulkan: split `ggml-vulkan.cpp` into buffers/debug translation units plus three shared headers, ~5.4k lines moved, `ggml-vulkan/CMakeLists.txt` gains exactly the five new files), **#29009** (OpenVINO update to 2026.4, entirely inside `ggml/src/ggml-openvino/**`), and **#28948** (Metal MoE + SSM_CONV fusion, new `argsort.metal`). **The review surface is 43 files, all additive or implementation-only.** `include/llama.h` gains two things and loses nothing: `LLAMA_VOCAB_TYPE_TEST = 7` (a tail append — no existing enumerator renumbers, and this project reads `vocab_type` as a raw int in `ModelMeta.getVocabType()` and emits it `static_cast`-ed in `jllama.cpp`, so no Java-side constant can go stale) and `llama_adapter_lora_init_from_file_ptr` (#28993, additive; adapters are loaded by path here, never by `FILE*`). `common/chat.cpp` picks up a Ling 3.0 / Bailing V3 detection arm and `common/parsers/ling3.cpp` (#28682), and `common/parsers/gemma4.cpp` **fixes a real bug on a path this project serves**: with `tool_choice == required` the grammar now terminates at the tool call instead of falling through to the content scan (#29115). `common/json-schema-to-grammar.cpp` fixes a second one — `gbnf_escape_length()` now accepts `\-`, so a JSON-schema `pattern` containing an escaped hyphen no longer produces a grammar the parser rejects (#29127). `src/llama-model.{cpp,h}` gain `load_swa_pattern()` with 20 `src/models/*.cpp` architectures rewritten onto it and `TENSOR_SKIP` honoured in `create_tensor_gate_up_exps()` (#29042, #29014); `tools/mtmd/clip.cpp` returns false instead of proceeding when `ggml_backend_sched_alloc_graph()` fails (#28149 / #26070). **`ggml/include` is byte-identical across the whole range**, so no ggml public API moved at all. | | b11018–b11062 | patches + upstream verification | **Nine patches still, none dropped — but two needed a refresh, the first in several ranges.** One upstream commit is responsible: **#29125** ("server : improve startup log messages", first tagged b11053) adds an `SRV_INF("initializing ...")` line immediately above `llama_server()`'s argv parse and a two-line `TODO` comment above `common_params_parse()` in `common/arg.h`. `0001` anchors hunks on both spots and `0006` replaces the very line `0001` flips, so both went stale **on context only** — the refresh changes `@@` line numbers, three context lines and the index blob hashes, and not one added or removed line. Replayed in filename order against pristine **b11055 and b11062**: all nine apply clean at both. **`0007`'s standing invariant is intact and provably so** — its `-` side is a verbatim copy of the route table it factors out of `llama_server()`, so a clean apply *is* the proof upstream did not touch that block; #29125's edits sit above it (the CORS warning) and below it (the `warn_names` loop), never inside. **The three mechanical `tools/server/` contract greps have no input** despite `tools/server/` being touched: `server-schema.cpp`, `server-task.cpp` and `server-context.cpp` are byte-identical b11018→b11062, verified by blob hash rather than by reading a diff, so the request-field set, the field bounds and the response-key set cannot have moved. **All standing drop-checks still say "still required"**, run against the pristine tag because the fail-loud applier detects "does not apply" but never "upstream already fixed this": `0001` (`common_params_parse_main` 0 occurrences in `b11062:common/arg.h`, WIN32 override still at `common/arg.cpp:1282`), `0002` (`params_base.load_progress_callback = load_progress_callback` still unguarded at `server-context.cpp:1095`), `0010` (`{"vocab_type", meta.model_vocab_type}` still uncast at `server-context.cpp:4554`), `0011` (the `common_peg_until_parser` `INVALID` branch still returns `FAIL` unconditionally, ignoring `ctx.is_lenient()`, while the `INCOMPLETE` branch right above it honours it), `0012` (bare `splits[i] /= split_sum` at `llama-model.cpp:1518`, no zero guard), and `0003`/`0006`/`0007`/`0008` absent upstream. **Verified at the target from a fresh configure**: stamp head `3cf03257f` with nine SHA-256 lines, `verify-patches-applied.sh` green (9 applied, 0010 cast present), extraction unchanged at 138 CLI / 57 request / 15 trainer names, Release build clean (0 errors, 0 warnings), `ctest` 551/551, `nm -D` 40 `Java_*` exports and 0 mangled, `NativeLibraryLoadSmokeTest` 4/4 with 0 skipped after a `mvn clean`, `mvn test` 1763/0 (269 model-gated skips in a HF-blocked sandbox), `verify-test-counts.sh` 1763 across 119 classes, SpotBugs 0, spotless clean. **The OpenVINO SDK pin moved with it**: #29009 takes upstream's own `OPENVINO_VERSION_MAJOR`/`OPENVINO_VERSION_FULL` to 2026.4, and this project's two OpenVINO classifier jobs — which had drifted two releases behind at 2026.2.1 — now install `2026.4` / `2026.4.0.22959.99c81491cc3` from the same URL template upstream's `{linux,windows}-setup-openvino` actions use. ggml-openvino is developed against whatever pair upstream pins, so tracking it is the cheaper end of the trade: a lagging pin does not fail on the bump that introduces the drift, it fails on some later one, in a job whose runner has no Intel GPU to reproduce on. **Not verifiable from the bump sandbox** — `storage.openvinotoolkit.org` is blocked by the network policy, so neither archive URL could be HEAD-checked here; the evidence they resolve is that upstream's own release jobs download exactly these two URLs at b11062. Per the classifier policy the step is fail-loud, so a wrong URL reds the job rather than shipping a backend-less jar. Both jobs now carry a keep-in-sync note naming upstream's two variables as the source of truth, so the next bump has somewhere to look instead of rediscovering the coupling. | +| b11062–b11069 | Seven commits, 21 files, **57 KiB** (`tools/ui` untouched, so the WebUI-excluded figure is the same). One touches a priority-list neighbourhood and a patch target: **#29161** ("common/peg : handle invalid utf-8 sequences in the AST", first tagged **b11063**) rewrites `common/peg-parser.{cpp,h}`, `common/unicode.{cpp,h}`, `common/chat-peg-parser.cpp` and the two upstream tests. Everything else is backend-internal: CUDA (#28912 MMVQ→MMQ crossover for SM70, #29152 FA tuning for Gemma 4 on Ampere+), Metal (#29169 arbitrary `hc` in `dsv4_hc_pre`, #29136 macOS 27 SDK deprecation warnings — warnings only, no API), WebGPU (#28976 fused GDN + cpy; not a backend this project builds), and the Python converter (#29203, not compiled). | **No project-source change, but one patch dropped.** `common/peg-parser.h` grows additively — a new `common_peg_invalid_utf8 {pos, len}` record, an `invalid_utf8` vector plus `sanitized_text()` on `common_peg_ast_node`, and a defaulted trailing parameter on `common_peg_ast_arena::add_node` and the four-argument `common_peg_parse_result` constructor — and it is on the *safe-to-skip* list anyway (nothing in `src/main/cpp` names a PEG type; `jllama.cpp` reaches the parser only through `common_chat_parse`). `common/unicode.h` changes one *comment*: `utf8_parse_result::bytes_consumed` now carries the valid-prefix length on `INVALID`/`INCOMPLETE` results instead of `0`. Neither `common/chat.h` nor any other priority row moves. **`tools/server/` is untouched** (no file in the range), so the three mechanical contract greps have no input. **`ggml/include` is byte-identical.** The behavioural change is the one `patches/0011` existed for — see the patch row below. | +| b11062–b11069 | patches + upstream verification | **Eight patches now: `0011` dropped, the other eight apply unchanged.** #29161 deleted the very `INVALID` branch of `common_peg_until_parser` that `0011` patched: the until-parser now consumes an undecodable run in every mode (strict included), records it on the result, and `common_chat_peg_mapper` renders the node through `sanitized_text()` — one U+FFFD per run, per the Unicode "maximal subpart" rule (`\xE4\xB8` + `c` → one replacement, `\xFF\xFE` → two), with the text after the run kept. `0011` had returned only the text *before* the byte. The lenient incomplete-at-end branch is unchanged (trailing bytes withheld). So the applier failed loud — `patch failed: common/peg-parser.cpp:680` — exactly as designed, and the patch was **dropped, not refreshed**, per the `0009`/`0013` precedent; it had never been filed upstream, so nothing to close. Its runnable guard was kept and re-pointed: `ContentOnlyParseUtf8` in `src/test/cpp/test_utils.cpp` now pins upstream's replacement contract (six cases, one more than before, the extra one pinning the run boundary `\xFF\xFE` → two U+FFFD), so a future upstream revert to `FAIL` still reds `C++ Tests` everywhere. **Replayed in filename order against pristine b11069**: `0001` `0002` `0003` `0006` `0007` `0008` `0010` `0012` apply clean, `0011` is the only failure. **All standing drop-checks still say "still required"** at the pristine tag: `0001` (`common_params_parse_main` 0 occurrences in `b11069:common/arg.h`; the count-guarded `argv = utf8.ptrs.data()` override still at `common/arg.cpp:1281` — i.e. [ggml-org/llama.cpp#26416](https://github.com/ggml-org/llama.cpp/issues/26416) remains open upstream and is **not** what this range fixed), `0002` (`params_base.load_progress_callback = load_progress_callback` still unguarded at `server-context.cpp:1095`), `0010` (`{"vocab_type", meta.model_vocab_type}` still uncast at `server-context.cpp:4554`), `0012` (bare `splits[i] /= split_sum` at `llama-model.cpp:1518`), and `0003`/`0006`/`0007`/`0008` absent upstream. `.github/verify-patches-applied.sh`'s header comment no longer lists `0011`. **Verified at the target from a fresh configure** (`rm -rf build && cmake -B build -DBUILD_TESTING=ON`, the real `FetchContent` path): stamp head `68d9053af` with **eight** SHA-256 lines, `verify-patches-applied.sh` green (8 applied, tree dirty, 0010 cast present), extraction unchanged at 138 CLI / 57 request / 15 trainer names, Release build clean, `ctest` **552/552** (551 → 552: the re-pointed guard gained one case), `nm -D` 40 `Java_*` exports and 0 mangled, `NativeLibraryLoadSmokeTest` **4/4, 0 skipped** after a `mvn clean` — `nativeBuildInfoMatchesPinnedVersionConstant` confirms `LlamaCppVersion.LLAMA_CPP_VERSION` (`b11069`) against the linked `build-info`. `test_utils.cpp` is clean under the CI-pinned clang-format 23.1.1; `spotless:check` clean. Model-backed Java tests were not run (HF-blocked sandbox); `NativeServerAttachIntegrationTest.completion_overHttp_served`, the test that first surfaced the `0011` failure, is the CI-side confirmation for this drop. | diff --git a/llama/CMakeLists.txt b/llama/CMakeLists.txt index c936f016..edc703d4 100644 --- a/llama/CMakeLists.txt +++ b/llama/CMakeLists.txt @@ -173,7 +173,7 @@ set(LLAMA_BUILD_APP OFF CACHE BOOL "" FORCE) FetchContent_Declare( llama.cpp GIT_REPOSITORY https://github.com/ggerganov/llama.cpp.git - GIT_TAG b11062 + GIT_TAG b11069 PATCH_COMMAND ${CMAKE_COMMAND} -DPATCH_DIR=${CMAKE_CURRENT_SOURCE_DIR}/patches -DLLAMA_SRC= diff --git a/llama/patches/0011-peg-parser-lenient-invalid-utf8.patch b/llama/patches/0011-peg-parser-lenient-invalid-utf8.patch deleted file mode 100644 index 0f2d96d0..00000000 --- a/llama/patches/0011-peg-parser-lenient-invalid-utf8.patch +++ /dev/null @@ -1,74 +0,0 @@ -diff --git a/common/peg-parser.cpp b/common/peg-parser.cpp -index 46fc29bf2..3f2008217 100644 ---- a/common/peg-parser.cpp -+++ b/common/peg-parser.cpp -@@ -680,7 +680,16 @@ struct parser_executor { - - if (utf8_result.status == utf8_parse_result::INVALID) { - // Malformed UTF-8 -- return common_peg_parse_result(COMMON_PEG_PARSE_RESULT_FAIL, start_pos); -+ if (!ctx.is_lenient()) { -+ return common_peg_parse_result(COMMON_PEG_PARSE_RESULT_FAIL, start_pos); -+ } -+ // Lenient: keep what was scanned before the malformed byte instead of failing the -+ // whole parse, mirroring the INCOMPLETE branch above. Failing here loses a result -+ // that was produced successfully: common_chat_peg_parse() always parses in lenient -+ // mode, and the server runs it over every completion (content-only when the request -+ // configures no chat parser), so a single stray byte anywhere in the generated text -+ // turns a finished generation into an HTTP 500 instead of a response. -+ return common_peg_parse_result(COMMON_PEG_PARSE_RESULT_NEED_MORE_INPUT, start_pos, last_valid_pos); - } - - // Check if a delimiter starts at this position -diff --git a/tests/peg-parser/test-unicode.cpp b/tests/peg-parser/test-unicode.cpp -index 24663d701..9ba64b519 100644 ---- a/tests/peg-parser/test-unicode.cpp -+++ b/tests/peg-parser/test-unicode.cpp -@@ -238,6 +238,47 @@ void test_unicode(testing &t) { - } - }); - -+ t.test("invalid UTF-8 is tolerated when lenient", [](testing &t) { -+ // A malformed byte must not fail the whole parse in lenient mode: common_chat_peg_parse() -+ // always parses leniently and the server runs it over every completion, so failing here -+ // would turn a finished generation into an error response. Keep what was scanned before -+ // the bad byte instead, exactly like the incomplete-sequence case above. -+ std::vector test_cases { -+ // Lone continuation byte in the middle -+ {std::string("Hello\x80World"), "Hello", COMMON_PEG_PARSE_RESULT_NEED_MORE_INPUT}, -+ -+ // Truncated CJK sequence followed by more bytes -+ {std::string("ab\xE4\xB8cd"), "ab", COMMON_PEG_PARSE_RESULT_NEED_MORE_INPUT}, -+ -+ // Invalid lead byte -+ {std::string("abc\xFF" "d"), "abc", COMMON_PEG_PARSE_RESULT_NEED_MORE_INPUT}, -+ }; -+ -+ auto parser = build_peg_parser([](common_peg_parser_builder& p) { -+ return p.until(""); -+ }); -+ -+ for (size_t i = 0; i < test_cases.size(); i++) { -+ const auto & tc = test_cases[i]; -+ std::string test_name = "case " + std::to_string(i) + ": " + hex_dump(tc.input); -+ -+ t.test(test_name, [&](testing &t) { -+ common_peg_parse_context lenient(tc.input, COMMON_PEG_PARSE_FLAG_LENIENT); -+ auto result = parser.parse(lenient); -+ -+ assert_result_equal(t, tc.expected_result, result.type); -+ -+ std::string matched = tc.input.substr(result.start, result.end - result.start); -+ t.assert_equal(tc.expected_text, matched); -+ -+ // Strict mode still rejects malformed input. -+ common_peg_parse_context strict(tc.input); -+ auto strict_result = parser.parse(strict); -+ assert_result_equal(t, COMMON_PEG_PARSE_RESULT_FAIL, strict_result.type); -+ }); -+ } -+ }); -+ - t.test("incomplete UTF-8 at end", [](testing &t) { - std::vector test_cases { - // Incomplete emoji at end, no delimiter diff --git a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java index 7936ed6b..ea561992 100644 --- a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java +++ b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java @@ -10,13 +10,13 @@ * library was compiled against, exposed as a compile-time constant so callers can render a badge or * emit a startup log line without loading the native library. * - *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11062"}) that mirrors the + *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b11069"}) that mirrors the * {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is * absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a * lightweight version badge in Android or other UIs.

* *

For the authoritative value that is baked into the native binary — the build number - * plus the resolved upstream commit, e.g. {@code "b11062-"} — call + * plus the resolved upstream commit, e.g. {@code "b11069-"} — call * {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own * {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires * the native library to be loaded).

@@ -24,14 +24,14 @@ public final class LlamaCppVersion { /** - * The pinned llama.cpp release tag this library was built against, e.g. {@code "b11062"}. + * The pinned llama.cpp release tag this library was built against, e.g. {@code "b11069"}. * *

Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the * "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the * compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the * value actually linked into the native binary.

*/ - public static final String LLAMA_CPP_VERSION = "b11062"; + public static final String LLAMA_CPP_VERSION = "b11069"; // Constants holder — not instantiable. private LlamaCppVersion() {} diff --git a/llama/src/test/cpp/test_utils.cpp b/llama/src/test/cpp/test_utils.cpp index a389d225..708a484b 100644 --- a/llama/src/test/cpp/test_utils.cpp +++ b/llama/src/test/cpp/test_utils.cpp @@ -1438,21 +1438,26 @@ TEST(FormatAnthropicSse, Array_EachElementDispatchedCorrectly) { // ============================================================ // common_chat_parse — the content-only path over malformed UTF-8 // -// Guards patches/0011. The server parses *every* completion through -// common_chat_parse(); with no chat parser configured that is the -// content-only fallback (`content(rest()) + end()`), whose scan is -// common_peg_until_parser. Upstream lets that scan tolerate an +// Pins upstream #29161 (first tagged b11063), which replaced this +// project's former patches/0011. The server parses *every* completion +// through common_chat_parse(); with no chat parser configured that is +// the content-only fallback (`content(rest()) + end()`), whose scan is +// common_peg_until_parser. Before b11069 that scan tolerated an // INCOMPLETE trailing UTF-8 sequence in lenient mode (and -// common_chat_peg_parse always parses leniently) but hard-fails on an -// INVALID byte, which turns a generation that finished normally into +// common_chat_peg_parse always parses leniently) but hard-failed on an +// INVALID byte, which turned a generation that finished normally into // an HTTP 500 — "The model produced output that does not match the // expected Content-only format" — for output the model really did -// produce. The patch makes the INVALID branch respect leniency the -// same way, keeping the text up to the bad byte. +// produce. Patch 0011 kept the text up to the bad byte; upstream went +// further: the until-parser now consumes the undecodable run, records +// it on the AST node, and common_chat_peg_mapper emits the node's +// sanitized_text(), so every invalid run becomes exactly one U+FFFD +// (the "maximal subpart" rule) and the text after it survives. // // These tests are the runnable half of that guard: if a llama.cpp bump -// drops the patch, or upstream reverts to failing, they go red here -// rather than in a model-backed Java integration test on one platform. +// reverts to failing, or changes the replacement contract, they go red +// here rather than in a model-backed Java integration test on one +// platform. // ============================================================ namespace { @@ -1465,6 +1470,10 @@ std::string parse_content_only(const std::string &raw) { return common_chat_parse(raw, /*is_partial=*/false, params).content; } +// U+FFFD REPLACEMENT CHARACTER, the byte sequence upstream substitutes for +// each undecodable run. +constexpr const char *REPLACEMENT = "\xEF\xBF\xBD"; + } // namespace TEST(ContentOnlyParseUtf8, ValidMultiByteContent_SurvivesByteForByte) { @@ -1472,28 +1481,42 @@ TEST(ContentOnlyParseUtf8, ValidMultiByteContent_SurvivesByteForByte) { EXPECT_EQ(parse_content_only(in), in); } -TEST(ContentOnlyParseUtf8, LoneContinuationByte_DoesNotThrow) { +TEST(ContentOnlyParseUtf8, LoneContinuationByte_ReplacedAndTextAfterItKept) { // The failure that reached CI: a stray continuation byte in the middle of - // the generated text made the whole request 500. + // the generated text made the whole request 500. Upstream now replaces the + // byte and keeps everything after it (patch 0011 used to stop at "Hello"). std::string content; EXPECT_NO_THROW(content = parse_content_only(std::string("Hello\x80World"))); - EXPECT_EQ(content, "Hello"); + EXPECT_EQ(content, std::string("Hello") + REPLACEMENT + "World"); } -TEST(ContentOnlyParseUtf8, TruncatedSequenceFollowedByMoreBytes_DoesNotThrow) { +TEST(ContentOnlyParseUtf8, TruncatedSequenceFollowedByMoreBytes_ReplacedOnce) { + // \xE4\xB8 is a valid two-byte prefix of a three-byte sequence; the 'c' + // that follows is not a continuation byte. The whole prefix is one + // undecodable run, so it becomes a single U+FFFD, not two. std::string content; EXPECT_NO_THROW(content = parse_content_only(std::string("ab\xE4\xB8") + "cd")); - EXPECT_EQ(content, "ab"); + EXPECT_EQ(content, std::string("ab") + REPLACEMENT + "cd"); } -TEST(ContentOnlyParseUtf8, InvalidLeadByte_DoesNotThrow) { +TEST(ContentOnlyParseUtf8, InvalidLeadByte_ReplacedAndTextAfterItKept) { std::string content; EXPECT_NO_THROW(content = parse_content_only(std::string("abc\xFF") + "d")); - EXPECT_EQ(content, "abc"); + EXPECT_EQ(content, std::string("abc") + REPLACEMENT + "d"); +} + +TEST(ContentOnlyParseUtf8, TwoAdjacentInvalidBytes_ReplacedOneEach) { + // Two lone bytes are two runs of length one, hence two replacements — + // pins the run boundary rather than a single "something was replaced". + std::string content; + EXPECT_NO_THROW(content = parse_content_only(std::string("Hello\xFF\xFE"))); + EXPECT_EQ(content, std::string("Hello") + REPLACEMENT + REPLACEMENT); } -TEST(ContentOnlyParseUtf8, IncompleteTrailingSequence_DoesNotThrow) { - // Upstream already tolerated this one; pinned so the two malformed-UTF-8 +TEST(ContentOnlyParseUtf8, IncompleteTrailingSequence_DroppedNotReplaced) { + // The one branch #29161 left alone: in lenient mode a sequence that is + // still incomplete at the very end of the input is *withheld* (more bytes + // may follow in a stream), not replaced. Pinned so the two malformed-UTF-8 // branches cannot drift apart again. std::string content; EXPECT_NO_THROW(content = parse_content_only(std::string("abc\xE2\x82")));