Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 15 additions & 20 deletions .github/verify-patches-applied.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,21 +7,26 @@
# Asserts that every llama/patches/*.patch really reached the fetched llama.cpp tree.
#
# WHY THIS EXISTS. The patch applier (llama/cmake/apply-llama-patches.cmake) is fail-loud on
# "does not apply", so a *stale* patch cannot ship silently. What it cannot detect is a patch
# that stops having an effect while still applying, and most patches do not need this check
# because they have a runnable guard that reds CI on every platform if they go missing:
# "does not apply", so a *stale* patch cannot ship silently. What it cannot detect is the
# applier never having run at all, or a patched tree being reverted after the fact — the stamp
# bookkeeping and the tree's dirty state are the only evidence of that, and this script asserts
# both. It runs in the always-on `C++ Tests` job, needs no model, and costs milliseconds.
#
# Every patch in the set does also have a runnable guard that reds CI on every platform if it
# goes missing, so these checks are a second line rather than the only one:
#
# 0003, 0006, 0007, 0008 -> jllama.cpp / native_server.cpp call the symbols they add,
# so dropping one is a compile or link error.
# 0012 -> src/test/cpp/test_model_split.cpp.
# 0014 -> src/test/cpp/test_common_log_callback.cpp (link error).
# 0001, 0002 -> model-gated Java jobs (Windows argv, LoadProgressCallbackTest).
#
# `0010` is the exception and the reason for this script. It casts one enum to int inside
# upstream's `get_res_model_info()`, which is `static` in server-context.cpp and therefore
# unreachable from jllama_test; reverting it leaves `ctest` completely green. Its only guard is
# NativeServerAttachIntegrationTest.models_reportNumericVocabType, which is model-gated — so the
# day a platform stops downloading models, the regression ships. This check runs in the
# always-on `C++ Tests` job, needs no model, and costs milliseconds.
# It used to carry a third, patch-specific check for `0010`, the one patch with no runnable
# guard (it cast an enum inside upstream's `static get_res_model_info()`, unreachable from
# jllama_test). That patch was DROPPED at the b11080 bump — upstream #28518 gave
# `common_json_value` an enum constructor, fixing the defect at its root — so the check retired
# with it. If a future patch is ever added that likewise cannot be reached from `ctest`, add a
# check for it here rather than relying on a model-gated Java test.
#
# Usage: .github/verify-patches-applied.sh [<llama.cpp-src-dir>]
# Exit codes: 0 all good, 1 a check failed.
Expand Down Expand Up @@ -67,14 +72,4 @@ if git -C "$SRC" rev-parse --git-dir >/dev/null 2>&1; then
fi
fi

# --- 3. the one patch with no runnable guard ------------------------------------------------------
VOCAB_CAST='(int) meta.model_vocab_type'
SERVER_CONTEXT="$SRC/tools/server/server-context.cpp"
[ -f "$SERVER_CONTEXT" ] || fail "not found: $SERVER_CONTEXT"
grep -qF "$VOCAB_CAST" "$SERVER_CONTEXT" \
|| fail "patches/0010 is not present in $SERVER_CONTEXT: expected '$VOCAB_CAST'.
Without the cast, common_json binds the unscoped enum to its bool constructor and
GET /models + GET /v1/models report vocab_type as true/false instead of a number.
If upstream added the cast themselves, DROP patch 0010 and update this check."

echo "patches verified: $on_disk applied, tree dirty, patches/0010 cast present"
echo "patches verified: $on_disk applied, tree dirty"
8 changes: 4 additions & 4 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2356,10 +2356,10 @@ jobs:
run: |
mvn -q --no-transfer-progress -f llama/pom.xml compile
.github/build.sh -DBUILD_TESTING=ON
# Most patches have a runnable guard that reds this job if they go missing (a link error, or
# test_utils.cpp / test_model_split.cpp). patches/0010 has none — it casts one enum inside a
# `static` function unreachable from jllama_test, so reverting it leaves ctest fully green and
# only a model-gated Java test notices. This is the always-on, model-free check for it.
# Every patch has a runnable guard that reds this job if it goes missing (a link error, or
# test_utils.cpp / test_model_split.cpp / test_common_log_callback.cpp). This is the second
# line: it asserts the applier actually ran and nothing reverted the patched tree, which no
# per-patch guard covers directly. Model-free, milliseconds.
- name: Verify llama.cpp patches are applied
run: .github/verify-patches-applied.sh
- name: Run C++ unit tests
Expand Down
27 changes: 27 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,33 @@ from version 5.0.0 onward. Pre-fork releases (`1.x`–`4.2.0`) were authored by
where the backend cannot provide it, `OFF` disables it.

### Changed
- **llama.cpp `b11069` → `b11080`, and local patch `0010` dropped — upstream fixed the
enum-to-JSON-boolean trap at its root.** Eleven upstream commits, 1244 KiB, no project-source
change. The size is one commit that does not concern this project (llama.cpp #29197 rewrites 46
files under `ggml/src/ggml-hexagon/`; no hexagon classifier is built here); the rest of `ggml/src`
is additive ARM repack kernels, a Metal fusion-list simplification and a SYCL softmax tweak, and
`ggml/include` is byte-identical. The one that matters is llama.cpp #28518 ("json: Fixed json enum
handling"): `common_json_value` gains an `std::is_enum`-gated constructor delegating to the
underlying type, so an unscoped enum no longer binds to `common_json_value(bool)` and serialises
as `true`/`false`. That is exactly the defect `patches/0010` cast around in upstream's own
`get_res_model_info()`, so the patch became a redundant carry and was dropped rather than kept
(the `0009`/`0011`/`0013` precedent). **Nothing observable changes for consumers** —
`GET /models` and `GET /v1/models` reported a numeric `vocab_type` with the patch and still do
without it — but the drop is worth flagging because `0010` *still applied cleanly*: the fail-loud
applier can only detect "does not apply", never "upstream already fixed this", which is why that
patch carries a by-hand drop-check on every bump. Its guard was kept and re-pointed: the
`CommonJsonEnumTrap` pair in `test_json_helpers.cpp` is now the `CommonJsonEnum` trio and pins
upstream's contract (uncast enum is numeric, an explicit cast is equivalent, a real `bool` is
still a boolean), so a bump that loses the constructor reds `C++ Tests` everywhere instead of
shipping a boolean. `jllama.cpp` keeps its own two `"vocab_type"` casts — correct either way.
Also in range: six existing sampling flags gained environment defaults (#27380 —
`LLAMA_ARG_TEMPERATURE`, `_TOP_P`, `_MIN_P`, `_REPEAT_PENALTY`, `_PRESENCE_PENALTY`,
`_FREQUENCY_PENALTY`), which adds no option but does mean a host with those variables set now
inherits them; and a router no longer forwards `LLAMA_ARG_API_KEY_FILE` to spawned children
(#28938). `tools/server/`'s schema, task and context translation units are byte-identical, and
the request-field set (68), bounded-field set (23) and response-key set (142) were all verified
unchanged mechanically, so the server wire contract cannot have moved. The other eight patches
apply unchanged and all four remaining drop-checks still say "still required".
- **llama.cpp `b11062` → `b11069`, and local patch `0011` dropped — malformed UTF-8 in a
completion is now replaced, not truncated.** Seven upstream commits, 57 KiB, no project-source
change; the one that matters is llama.cpp #29161 ("common/peg : handle invalid utf-8 sequences in
Expand Down
Loading
Loading