Build on official llama.cpp; vendor the decision engine

.github/workflows/release.yml

@@ -56,8 +56,8 @@ jobs:
       - name: Test multilingual
         run: LEV_TEST_CHECKPOINTS=multilingual jolt -M:test lev.checkpoints-test
 
-      # jolt binary first runs the llama task: llama.cpp (thecodacus's
-      # parallel-decision fork) is fetched at its pinned commit and built static (cmake; Metal on mac, CPU elsewhere), so
+      # jolt binary first runs the llama task: llama.cpp is fetched at its
+      # pinned release and built static (cmake; Metal on mac, CPU elsewhere), so
       # the binary carries the thinker
       - name: Build and self-test the binary
         run: jolt binary

README.md

@@ -144,7 +144,7 @@ models' templates use. Another model family needs its own template, which lives
 
 ```
 jolt kernels             # compile native/lev_kernels.c
-jolt llama               # fetch + build llama.cpp (the parallel-decision fork, pinned commit) into native/liblev_llm.*, the thinker's native
+jolt llama               # fetch + build llama.cpp (pinned release) with the vendored decision engine into native/liblev_llm.*, the thinker's native
 jolt mlx                 # mac: clone + build mlx-c (pinned tag) into native/liblev_mlx.*, the encoders' GPU backend
 jolt prepare             # every checkpoint under ../laya -> data/, data/typed-decisions, ...
 jolt -M:test             # parity suites vs golden/
@@ -333,8 +333,10 @@ a 503.
 ### Jev mode
 
 With thinking off, a thinker answers every question of a call in one
-pass. `native/llama.cpp` is [thecodacus/llama.cpp's](https://github.com/thecodacus/llama.cpp/tree/parallel-decision) `parallel-decision`
-branch, and its decision engine does the work. The chat up to the state
+pass. The decision engine of [thecodacus/llama.cpp's](https://github.com/thecodacus/llama.cpp/tree/parallel-decision)
+`parallel-decision` branch does the work. It is vendored in
+`native/decision-engine.{h,cpp}` and built against official llama.cpp
+until upstream takes it ([#29752](https://github.com/ggml-org/llama.cpp/pull/29752)). The chat up to the state
 is decoded once and kept for later calls. The state is decoded once per
 call. Each question is then a branch forked from the state in the KV
 cache, and every branch is decoded in the same batch. Each branch holds
@@ -967,13 +969,13 @@ entries, and the build task branches on OS.
   `LEV_THREADS`. Attention calls `cblas_sgemm` through a pointer the
   Clojure side hands it, so the library links against no BLAS.
 - **llama.cpp**: `native/liblev_llm.dylib` / `.so` plus `liblev_llm.a`,
-  built by `jolt llama`. It fetches thecodacus/llama.cpp's
-  `parallel-decision` branch at its pinned commit (upstream b10435 plus
-  the decision engine) into `native/llama.cpp` and builds `llama` and
+  built by `jolt llama`. It fetches official llama.cpp at its pinned
+  release into `native/llama.cpp` and builds `llama` and
   `llama-common` static with cmake, with no https or subprocess support,
   Metal with the shader library embedded on mac and CPU elsewhere. They
-  sit behind `native/lev_llm.c` and `native/lev_decision.cpp` (the
-  engine's C face), which jolt.ffi binds as `lev.llm`. A checkout at
+  sit behind `native/lev_llm.c` and `native/lev_decision.cpp` (the C
+  face of the decision engine vendored in `native/decision-engine.cpp`),
+  which jolt.ffi binds as `lev.llm`. A checkout at
   another commit is replaced. `LLAMA_CPP_SRC=path` fetches from a local
   clone that has the commit. It is
   optional, since without it the encoders run and thinkers are

bench/README.md

@@ -704,3 +704,34 @@ the held-out score are what separated 0.733 from 0.905.
 
 Timing was taken with other benches sharing the machine, so it is
 indicative only.
+
+## Official llama.cpp, the decision engine vendored (2026-10-15)
+
+The parallel-decision fork changed nothing in libllama: its decision
+engine runs on the stock API (`llama_memory_seq_cp`, `seq_rm`,
+`llama_decode`, `llama_get_logits_ith`). The engine is now vendored in
+`native/decision-engine.{h,cpp}` and built against official llama.cpp
+b11630, 589 commits past the fork's base. Only the class lev drives is
+kept, and the two `common/` batch helpers upstream dropped are local.
+
+Same machine, back to back: upstream, the fork's library, upstream again.
+The workflow rows use Qwen3.5-4B Q8 in Jev mode (thinking off), p50 over
+20 iterations. The authored144 rows use MiniCPM5-2B Q8, thinking off.
+
+| | upstream b11630 | fork | upstream b11630 again |
+|---|---|---|---|
+| 4 questions, short state | 689 ms | 746 ms | 653 ms |
+| 4 questions, long state (1.6k tokens) | 3,165 ms | 3,561 ms | 3,439 ms |
+| 1 question, long state | 2,786 ms | 3,010 ms | 2,923 ms |
+| 8 short states, one call each | 5,218 ms | 6,134 ms | 5,234 ms |
+| 8 short states, one batch | 5,260 ms | 5,403 ms | 4,672 ms |
+| authored144 (MiniCPM5) | 108/144, ECE 0.174, 116 ms | 108/144, ECE 0.174, 125 ms | 108/144, ECE 0.174, 112 ms |
+
+The answers do not move: authored144 is the same 108 cases with the same
+calibration, and llm-test, think-test and clef-test pass against their
+goldens. Upstream is 3–15% faster on every row.
+
+Qwen3.5-4B pays more per call than MiniCPM5 did in the Jev-mode section.
+It is a hybrid model with recurrent layers, and llama.cpp splits its
+batches by sequence, so its branches take several passes, not one. On a
+long state the prefill dominates either way.

deps.edn

@@ -109,14 +109,15 @@
                (println "built:" kern-lib kern-archive)))}
 
   llama
-  {:doc "fetch llama.cpp at the pinned commit of thecodacus/llama.cpp's parallel-decision branch (upstream b10435+617 plus its decision engine) into native/llama.cpp, build it static, wrap it and the engine in native/liblev_llm.* (the thinker's native; LLAMA_CPP_SRC=path fetches from a local checkout holding that commit)"
-   :task (let [rev "14d04e755fa28653e87b9a07072892265bdc0fad"
-               src (or (System/getenv "LLAMA_CPP_SRC") "https://github.com/thecodacus/llama.cpp")
+  {:doc "fetch llama.cpp at the pinned release into native/llama.cpp, build it static, wrap it and the decision engine (native/decision-engine.cpp, vendored until llama.cpp takes it) in native/liblev_llm.* (the thinker's native; LLAMA_CPP_SRC=path fetches from a local checkout holding that commit)"
+   :task (let [release "b11630"
+               rev "6868a52cb337450658006ad2f9281c2eb63f1666"
+               src (or (System/getenv "LLAMA_CPP_SRC") "https://github.com/ggml-org/llama.cpp")
                dir "native/llama.cpp"
                head (str dir "/.git/HEAD")
                lib (str "native/liblev_llm." (if mac? "dylib" "so"))]
-           ;; a checkout of any other commit (the upstream tag this used to
-           ;; pin) is replaced, build and all
+           ;; a checkout of any other commit (the parallel-decision fork this
+           ;; used to pin) is replaced, build and all
            (when-not (and (fs/exists? head) (= rev (.trim (slurp head))))
              (fs/delete-tree dir)
              (fs/create-dirs dir)
@@ -139,9 +140,9 @@
              (shell "cmake" "--build" (str dir "/build") "--target" "llama" "llama-common"
                     "--parallel" (str (.availableProcessors (Runtime/getRuntime)))))
            (if (and (fs/exists? lib) (fs/exists? "native/liblev_llm.a")
-                    (empty? (fs/modified-since lib ["native/lev_llm.c" "native/lev_llm.h" "native/lev_decision.cpp" "native/lev_clef.cpp" "native/build_llm.sh" head])))
+                    (empty? (fs/modified-since lib ["native/lev_llm.c" "native/lev_llm.h" "native/lev_decision.cpp" "native/decision-engine.cpp" "native/decision-engine.h" "native/lev_clef.cpp" "native/build_llm.sh" head])))
              (println "llm: up to date -" lib "native/liblev_llm.a")
-             (shell "sh" "native/build_llm.sh" rev)))}
+             (shell "sh" "native/build_llm.sh" release)))}
 
   mlx
   {:doc "clone mlx-c at the pinned tag into native/mlx-c (it fetches mlx itself), build both static, wrap them in native/liblev_mlx.* with the Metal kernels beside it (the encoders' optional GPU backend, mac only; MLX_C_SRC=path clones from a local checkout)"

native/build_llm.sh

@@ -1,16 +1,15 @@
 #!/bin/sh
 # Build native/lev_llm.c and native/lev_decision.cpp (with the decision
-# engine of the vendored fork, tools/parallel-decision) against llama.cpp
-# (native/llama.cpp, fetched by the `jolt llama` task at the pinned commit)
+# engine vendored in native/decision-engine.cpp) against llama.cpp
+# (native/llama.cpp, fetched by the `jolt llama` task at the pinned release)
 # into liblev_llm.{dylib,so} for jolt run/test and liblev_llm.a for jolt
-# build. Run from the project root; $1 is the commit, for lev_llm_version.
+# build. Run from the project root; $1 is the release tag, for lev_llm_version.
 set -e
 L=native/llama.cpp
-REV=${1:-$(cd "$L" && git rev-parse HEAD 2>/dev/null || echo unknown)}
-TAG="thecodacus/parallel-decision@$(echo "$REV" | cut -c1-9)"
+TAG="${1:-$(cd "$L" && git rev-parse --short HEAD 2>/dev/null || echo unknown)}"
 LIBS="$L/build/common/libllama-common.a $L/build/common/libllama-common-base.a $L/build/vendor/cpp-httplib/libcpp-httplib.a
       $L/build/src/libllama.a $L/build/ggml/src/libggml.a $L/build/ggml/src/libggml-cpu.a $L/build/ggml/src/libggml-base.a"
-INC="-I$L/include -I$L/ggml/include -I$L/common -I$L/vendor -I$L/tools/parallel-decision -Inative"
+INC="-I$L/include -I$L/ggml/include -I$L/common -I$L/vendor -Inative"
 CFLAGS="-O2 -std=c11 -Wall -Wextra -DLEV_LLAMA_BUILD=\"$TAG\" $INC"
 CXXFLAGS="-O2 -std=c++17 -Wall $INC"
 # lev_clef.cpp reads hidden states through the staging header in src/
@@ -23,7 +22,7 @@ case "$(uname -s)" in
     cc $CFLAGS -c native/lev_llm.c -o native/lev_llm.o
     c++ $CXXFLAGS -c native/lev_decision.cpp -o native/lev_decision.o
     c++ $CXXFLAGS $CLEF_INC -c native/lev_clef.cpp -o native/lev_clef.o
-    c++ $CXXFLAGS -w -c $L/tools/parallel-decision/decision-engine.cpp -o native/decision-engine.o
+    c++ $CXXFLAGS -c native/decision-engine.cpp -o native/decision-engine.o
     c++ -dynamiclib $OBJS $LIBS $FRAMEWORKS -o native/liblev_llm.dylib
     # one archive with every llama.cpp member inside, for jolt build's force-load
     libtool -static -o native/liblev_llm.a $OBJS $LIBS
@@ -39,7 +38,7 @@ case "$(uname -s)" in
     cc $CFLAGS -fPIC -c native/lev_llm.c -o native/lev_llm.o
     c++ $CXXFLAGS -fPIC -c native/lev_decision.cpp -o native/lev_decision.o
     c++ $CXXFLAGS $CLEF_INC -fPIC -c native/lev_clef.cpp -o native/lev_clef.o
-    c++ $CXXFLAGS -w -fPIC -c $L/tools/parallel-decision/decision-engine.cpp -o native/decision-engine.o
+    c++ $CXXFLAGS -fPIC -c native/decision-engine.cpp -o native/decision-engine.o
     c++ -shared $OBJS -Wl,--whole-archive $LIBS -Wl,--no-whole-archive -lm -lpthread -o native/liblev_llm.so
     printf 'create native/liblev_llm.a\n' > native/llm.mri
     for o in $OBJS; do printf 'addmod %s\n' "$o" >> native/llm.mri; done