Repository navigation
docs: run IFM K2-Horizon with custom llama.cpp builds - #3
Conversation
mvillmow
left a comment
There was a problem hiding this comment.
[AI-assisted review] Required findings are in the inline comments.
|
[AI-assisted Athena PR review] Verdict: CONDITIONAL GO for this source update; not GO for merge or promotion. I reviewed exact head Verification: 152/152 focused local tests passed; pre-commit, workflow validation, Black, ShellCheck, Bash syntax, Python compilation, generated-doc drift, and diff checks passed. Exact-head GitHub CI finished with 40 passing and 28 intentionally skipped checks and no failures; the transient ARM download stall passed on rerun attempt 2. The PR must remain draft. External gates are still open: an accepted current-upstream K2 commit; immutable K2-capable artifacts and coordinated pins for every managed backend; the complete protected packaged-runtime/protocol matrix; default-branch trusted-workflow/ruleset/CODEOWNERS/merge-queue and runner bootstrap; and either enabling Actions-created PRs or configuring a scoped App/PAT with matching publisher identity. The existing major review threads remain unresolved for those gates. |
|
Follow-up The fixture now computes the advertised length before the move. The unchanged strict assertion passes 100/100 on macOS, once in the exact x86_64 CI image (GCC 13.3, Release/O3), and as part of the local 69/69 |
|
Exact-head validation is complete for Final exact-head review is technical GO (grade A) with no actionable P0–P2 findings and all six review threads resolved. The PR remains draft; production model catalogs and backend pins are unchanged, and the documented upstream artifact/platform/governance promotion gates still apply. |
mvillmow
left a comment
There was a problem hiding this comment.
[AI-assisted review] One required documentation-consistency finding and one simplification suggestion are inline. Exact-head CI is also incomplete because six checks failed for unrelated infrastructure or existing workflow reasons.
mvillmow
left a comment
There was a problem hiding this comment.
One required documentation finding.
Summary
Document the minimum IFM K2-Horizon integration through existing bring-your-own-backend and custom-model support. Add the complete macOS source-build flow and installation commands for every non-Uno GGUF supported by the IFM llama.cpp branch: 0.9B, 3.7B, 7B, 32B, and MoVA 36B-A4B, including their model-card context limits. Related to #1; this deliberately does not implement managed-backend distribution or close that broader issue.
Scope
The net diff is limited to two documentation files: 131 additions and 4 deletions. The second file aligns the canonical
*_binconfiguration contract with the executable-path examples. No server, model catalog, backend pin, workflow, dependency, or test changes. The 375B release is excluded because its architecture is intentionally not implemented in the IFM llama.cpp branch.Testing
Testing details: Validated the macOS Metal walkthrough on an Apple M4 Pro using the source-built
lemondandlemonadebinaries, the IFMmodel/K2Horizonllama-server, anduser.K2-Horizon-0.9B.POST /v1/loadsucceeded with a 4096-token context andPOST /v1/chat/completionsreturned2 + 2 = 4withreasoning_effort=high. Verified all five repository and GGUF filenames against the official IFM Hugging Face API, and verified maximum context metadata as 131072 tokens for 0.9B and 524288 for every other listed model. Also rangit diff --check, shell syntax validation for every added Bash block, and local-link checks. The full context limits, larger models, and optionalBUILD_WEB_APP=ONpath were not runtime-tested.Documentation
Breaking Changes
AI-assisted contribution
If AI tools were used: