Repository navigation
fix: give the vendored linux ggml libraries private sonames - #39455
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS The fix is at cause level: the public ggml sonames are what let glibc reuse whichever ggml loaded first, and this removes them. The Windows sibling is declared and deferred to #39002. Not checked: whether the patched aarch64 libraries load at runtime. Only x86_64 ran Notes
What this change shipsInventory (7 items) — 6 justifiedIntent: let a user's source-built (GPU) whisper.cpp and Kiro Crew's embedding runtime each load their own ggml in one Linux process. FIX.
[FIRST-PRINCIPLES-REVIEWED] 67e02db Description read[DESCRIPTION-READ] 7aa2de1f0c8f4debe3376f8d946ca1de97e24c4dcb53d955ae18099c2f2cbb10 That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: renaming to private sonames fixes the cause (the glibc soname reuse), and every reader of the old names in this tree goes through Notes
[DESIGN-REVIEWED] 67e02db Description read[DESCRIPTION-READ] 870352047cd5fbdc2ce9e0f5ddb3e43350fae2fed92d330a68db396fa6358ff5 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
|
[operator: bolichen97] CI triage for head 016fc78:
The failed jobs get one re-run on a later pass. If shard 7 is still red after it, this PR rebases once #39475 lands. |
The Linux llama.cpp closure shipped ggml under upstream's public names (libggml.so.0, libggml-base.so.0, libggml-cpu.so.0). glibc reuses an already-loaded object whose soname matches a DT_NEEDED entry, so a source-built whisper.cpp (pywhispercpp, the GPU route for speech) in the same process and the embedding runtime shared whichever ggml loaded first. Whisper first: libllama.so fails on ggml_lightning_indexer, which whisper's ggml lacks, and memory search falls back to keywords. Embeddings first: whisper binds to our CPU-only ggml and its GPU or BLAS backend never loads. Rename the three files in linux_x86_64, linux_aarch64 and linux_aarch64_dotprod to auditwheel-style private names (libggml-<sha256 prefix>.so.0), set each soname to match and rewrite the dependency entries of libllama.so and the ggml libraries, with pinned patchelf 0.17.2. patchelf moves the dynamic strings into an added load segment; .text and .rodata are byte-identical and the RPATH is unchanged. libllama.so and libgomp keep their names. The required-libs declaration and the vendor manifest name the new files, the bounded-decode proof digest is re-proven against the shipped model, and the vendoring notes carry the rename recipe so an upgrade re-applies it. A new payload test reads each Linux file's dynamic section and fails if a public ggml name, a mismatched soname or a RUNPATH appears. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Rebased: main moved in [operator: bolichen97] |
016fc78 to
67e02db
Compare
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
On head 67e02db I inspected the dynamic sections of every vendored Linux library. Each ggml file now carries a private, hash-named soname that matches its file name. libllama.so and the ggml files list only those private names as dependencies, all present in the same directory, and RPATH is preserved. A section comparison against main showed .text and .rodata unchanged. The payload, bundled-libs, manifest, lib-path, doctor and bounded-decode tests pass, and so does the real-model embedding test against the shipped GGUF, which confirms the re-pinned runtime digest. A clean-process load maps only the renamed files. Packaging selects these files by directory, so nothing blocks this change.
Problem / Motivation
Goal: A source-built (GPU) pywhispercpp and Kiro Crew's embedding runtime each load their own ggml in one process on Linux, whichever loads first.
On Linux, a user who builds pywhispercpp from source for GPU speech gets one of two silent failures, depending on which side loads first in the gateway:
libllama.sohas anundefined symbol: ggml_lightning_indexer;Why it matters
Someone who builds whisper.cpp for their GPU, which is the route the STT docs point to, ends up with either speech silently on the CPU or memory search silently on keywords. Which one they get depends on load order, and they can't fix either from their side short of patching their own build.
Not a goal
win_amd64; it needs a Windows build and run to prove, tracked in Vendored Windows llama.cpp DLLs share base names with a source-built pywhispercpp's ggml #39002).kirocrew doctoror the Voice panel.What changed (motivation → approach → change)
The vendored Linux ggml libraries now have private names, so the dynamic loader can no longer mix them up with a user's own ggml.
Root cause: our
linux_x86_64,linux_aarch64andlinux_aarch64_dotprodclosures shippedlibggml.so.0,libggml-base.so.0andlibggml-cpu.so.0under upstream's public sonames. glibc reuses an already-loaded object whose soname matches aDT_NEEDEDentry, whatever the dlopen mode. A source-built whisper.cpp installs the same public names, so whichever side loaded first supplied ggml for both.Approach (the maintainer decision on the issue): auditwheel-style private names. The vendored libgomp files already use this scheme.
libggml-base.so.0libggml-base-<first 8 hex of the unpatched sha256>.so.0(likewiselibggml,libggml-cpu)libllama.so,libggml*,libggml-cpu*libllama.sofile name and soname, libgomp, RPATH.text/.rodataof every patched fileThe patch tool is pinned (patchelf 0.17.2,
pip install patchelf==0.17.2.1), and two runs from the unpatched files give byte-identical output. Each patched file grows by 32–290 KB (patchelf adds a page-aligned load segment for the longer dynamic strings), about 1.7 MB across the three dirs.Around the binaries:
_REQUIRED_VENDORED_LIBS(insrc/kiro_crew/_llama_lib_path.py) names the new files.scripts/vendor_manifest.sha256is regenerated._PROVEN_RUNTIME_DIGESTis re-proven:TestRealModelpassed against the shipped GGUF on this tree._vendor/README.mdgets a "Private ggml names on Linux" subsection with the exact recipe, the check, and a before/after sha256 table. It also adds the rename step to the upgrade and local-divergence notes.Nothing in packaging changes:
setup.cfgandMANIFEST.inselect these files by directory, and the new names still end.so.0, soglobal-exclude *.sodoes not strip them.Backwards compatibility
Compatible: the loader opens
libllama.soby path and every other file is reached throughlibllama.so's ownDT_NEEDEDentries, which name the new files; an operator'sLLAMA_CPP_LIB_PATHdirectory is not touched.The diff renames shipped files, so the readers of the old names:
Reader: src/kiro_crew/embeddings_native.py:verify_vendored_libs -- release -- reads
_REQUIRED_VENDORED_LIBS, which names the new files;test_source_tree_carries_every_required_libReader: src/kiro_crew/embeddings_native.py:_linux_aarch64_libs_dir -- chat -- dotprod completeness check iterates the same declaration;
test_an_incomplete_dotprod_dir_loads_todays_dirReader: src/kiro_crew/embeddings_native.py:_load_llama_class -- chat -- missing-file check iterates the same declaration; probe below loads and embeds in a clean process
Reader: src/kiro_crew/doctor_checks/features.py:verify_vendored_libs -- release -- same declaration;
test/test_cli_stt_memory_doctor.pyReader: scripts/verify_vendored_payload.py:_REQUIRED_VENDORED_LIBS -- release -- reads the literal with
ast.literal_eval;TestPayloadVerifierWithoutRuntimeDependenciesReader: setup.cfg:package_data / MANIFEST.in -- release -- per-directory
*globs, names still end.so.0;test_package_data_declares_the_libs_explicitly,test_manifest_rules_keep_every_required_libReader: src/kiro_crew/_vendor/llama_cpp/llama_cpp.py:load_shared_library -- chat -- opens
libllama.soby base name, which keeps its name; probe belowTests
test/test_vendored_llama_payload.py::test_linux_ggml_libraries_carry_private_sonames(new): a stdlib ELF64 reader opens every committed file in the three Linux dirs. It fails if a public ggml file name, soname orNEEDEDentry comes back, if alibggml*soname differs from its file name or is not hash-named, if alibggml/libgompdependency names a file the dir lacks, or ifRPATHis lost orRUNPATHappears. It needs no readelf, so it runs on the Windows and macOS lanes too. Red on the base commit (linux_aarch64 ships a public ggml file name), green here.test/test_embeddings_bundled_libs.py::…::test_an_incomplete_dotprod_dir_loads_todays_dirreads the dotprod ggml-cpu name from the declaration instead of hard-coding the public name.test/test_embed_bounded_decode.py:_PROVEN_RUNTIME_DIGESTupdated afterTestRealModelpassed with the shipped model.Proof: not provable -- other: the production change is binary (patchelf-rewritten .so files) and prove.py cannot reverse-apply binary hunks; the pin test was run on the base commit instead and fails there
Manual verification
How it was verified, on Linux x86_64 (Ubuntu 24.04, glibc 2.39), with pywhispercpp 1.5.1 (f7bf621, whisper.cpp f24588a) built from source twice: a CPU build, and an OpenBLAS build (
-DGGML_BLAS=ON). The BLAS build stood in for a GPU build, since it is a non-CPU ggml backend that needs no GPU. No CUDA or Vulkan GPU build was run. One fresh process per row;/proc/self/mapsis the oracle._load_llama_class()/proc/self/maps)undefined symbol: ggml_lightning_indexerundefined symbol: ggml_lightning_indexerEvery embed is 1024 dims and bit-identical (max abs diff 0) to a clean process on either tree, as expected:
.textand.rodataare unchanged.detect()readswhisper_print_system_info(), which in this whisper.cpp lists only registries that exportggml_backend_get_features. ggml-blas does not export it, sodetect()reportscpueven for the BLAS build loaded alone. The BLAS oracle is therefore the registry list of the object that holds sonamelibggml.so.0, i.e. what whisper binds to. In this ggml, only the CPU, CUDA, Metal and zDNN registries export it. A Vulkan build therefore reads ascputhere too. That limitation already exists and is outside this goal.aarch64: under
qemu-aarch64-static8.2.2 (CPUmax) with an arm64python:3.12-slimrootfs,libllama.sowas loaded by full path from copies oflinux_aarch64/andlinux_aarch64_dotprod/on both trees, followed by one embed through the vendored bindings. The fix maps the renamed ggml files,DOTPROD = 1is reported for the dotprod build, and the vectors are bit-identical to base's.Real-machine verification
Machine: EC2 m7i.2xlarge, Ubuntu 24.04.5 LTS (glibc 2.39), x86_64, 8 vCPU / 32 GiB; aarch64 under
qemu-aarch64-static8.2.2 on the same machineBefore (main 433d99b; main has not changed the vendored libraries or the loader since):
env -u LLAMA_CPP_LIB_PATH PYTHONPATH=<tree>/src python probe.py <whisper-first|llama-first> <pywhispercpp build> <model>, one fresh process per row -> whisper-first:_load_llama_class()returns None withlibllama.so: undefined symbol: ggml_lightning_indexer(CPU and BLAS builds); llama-first: only our ggml is mapped and the BLAS build's registries drop to['CPU']After (head 67e02db; the machine ran 80064f7e622, whose binaries, code and tests are identical, only README wording and its manifest line differ): the same command -> both orders load the Llama class, map both ggml sets, embed 1024 dims bit-identical to a clean process, and the BLAS build keeps
['BLAS', 'CPU']; aarch64 and aarch64_dotprod load and embed with the renamed files mapped; the pin test is red on the base and green on the headCleanup: the checkouts and builds lived in the job's own scratch directory; the machine was a tagged instance from a verification pool (no inbound rules, terminate-on-shutdown) and is terminated: it no longer appears in the pool's tagged-instance listing
Screenshots / video
No user-visible UI change: the diff touches no frontend path.
Related Issues
Closes #37142
Sibling sweep
Swept:
git ls-files 'src/kiro_crew/_vendor/llama_cpp_libs/*'andreadelf -dof every Linux file for a publiclibggml*nameSibling: src/kiro_crew/_vendor/llama_cpp_libs/win_amd64/ggml.dll -- tracked in #39002
Sibling: src/kiro_crew/_vendor/llama_cpp_libs/macos_arm64/libggml.0.dylib -- not affected -- macOS is outside this goal (Not a goal), and the report and decision are Linux-only
Sibling: src/kiro_crew/_vendor/llama_cpp_libs/linux_x86_64/libllama.so -- not affected -- keeps soname
libllama.so.0; whisper.cpp ships no libllama, and ctypes opens it by pathPattern harvest
Rule candidate: lint
Pattern: a vendored native library that keeps upstream's public soname collides with a user's own copy loaded in the same process. The new payload test pins this for the Linux ggml files; a later vendored ELF closure should get the same soname/NEEDED pin.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...); PR Hygiene does not count commitsContribution License Agreement
Contributed under the project's Apache-2.0 license.