Skip to content

docs: run IFM K2-Horizon with custom llama.cpp builds - #3

Merged
mvillmow merged 14 commits into
mainfrom
codex/k2-horizon-integration
Sep 9, 2026
Merged

mvillmow merged 14 commits into
mainfrom
codex/k2-horizon-integration

Conversation

@mvillmow

@mvillmow mvillmow commented Sep 5, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Document the minimum IFM K2-Horizon integration through existing bring-your-own-backend and custom-model support. Add the complete macOS source-build flow and installation commands for every non-Uno GGUF supported by the IFM llama.cpp branch: 0.9B, 3.7B, 7B, 32B, and MoVA 36B-A4B, including their model-card context limits. Related to #1; this deliberately does not implement managed-backend distribution or close that broader issue.

Scope

  • This PR addresses one clear issue or change.
  • I reviewed the full diff myself before submitting.
  • I removed unrelated local changes.
  • I kept refactoring separate unless it is required for this change.

The net diff is limited to two documentation files: 131 additions and 4 deletions. The second file aligns the canonical *_bin configuration contract with the executable-path examples. No server, model catalog, backend pin, workflow, dependency, or test changes. The 375B release is excluded because its architecture is intentionally not implemented in the IFM llama.cpp branch.

Testing

  • Code builds without errors locally.
  • I tested this change locally.
  • I described the testing performed below.

Testing details: Validated the macOS Metal walkthrough on an Apple M4 Pro using the source-built lemond and lemonade binaries, the IFM model/K2Horizon llama-server, and user.K2-Horizon-0.9B. POST /v1/load succeeded with a 4096-token context and POST /v1/chat/completions returned 2 + 2 = 4 with reasoning_effort=high. Verified all five repository and GGUF filenames against the official IFM Hugging Face API, and verified maximum context metadata as 131072 tokens for 0.9B and 524288 for every other listed model. Also ran git diff --check, shell syntax validation for every added Bash block, and local-link checks. The full context limits, larger models, and optional BUILD_WEB_APP=ON path were not runtime-tested.

Documentation

  • Documentation is not affected by this change.
  • Documentation is affected and has been updated.
  • Documentation is affected but will be handled in a separate PR or issue.

Breaking Changes

  • This PR introduces breaking changes.
  • This PR does not introduce breaking changes.

AI-assisted contribution

  • I used AI tools for this PR.
  • I did not use AI tools for this PR.

If AI tools were used:

  • I verified that I understand the changes.
  • I checked for hallucinated APIs, unrelated changes, and incorrect assumptions.

@mvillmow mvillmow left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[AI-assisted review] Required findings are in the inline comments.

Comment thread src/cpp/resources/server_models.json Outdated
Comment thread src/cpp/resources/server_models.json Outdated
Comment thread .github/workflows/validate_llamacpp_core.yml Outdated
Comment thread test/utils/validation_model_selection.py Outdated
Comment thread .github/workflows/validate_llamacpp.yml Outdated
Comment thread docs/embeddable/backends.md Outdated
@mvillmow

mvillmow commented Sep 6, 2026

Copy link
Copy Markdown
Collaborator Author

[AI-assisted Athena PR review]

Verdict: CONDITIONAL GO for this source update; not GO for merge or promotion.

I reviewed exact head 7817bd3119e6cad0ea1c87f4736b79cf7e897402 against main at bc8d99ccfee301742f1dd86a3f6ec406bb8a863d. Three independent exact-diff audits report no remaining P0-P2 findings and grade the source A. The last required fixes centralized managed-pin applicability in the base-owned verifier and bound every PR/branch mutation to the published OID, with executable two-repository and race regressions.

Verification: 152/152 focused local tests passed; pre-commit, workflow validation, Black, ShellCheck, Bash syntax, Python compilation, generated-doc drift, and diff checks passed. Exact-head GitHub CI finished with 40 passing and 28 intentionally skipped checks and no failures; the transient ARM download stall passed on rerun attempt 2.

The PR must remain draft. External gates are still open: an accepted current-upstream K2 commit; immutable K2-capable artifacts and coordinated pins for every managed backend; the complete protected packaged-runtime/protocol matrix; default-branch trusted-workflow/ruleset/CODEOWNERS/merge-queue and runner bootstrap; and either enabling Actions-created PRs or configuring a scoped App/PAT with matching publisher identity. The existing major review threads remain unresolved for those gates.

@mvillmow mvillmow changed the title Stage K2-Horizon llama.cpp integration Add K2-Horizon validation and promotion gates Sep 7, 2026
@mvillmow

mvillmow commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator Author

Follow-up 09e83c9c fixes the failed C++ unit-test fixture. C++17 permits the response-length argument to be evaluated after the lambda moves its source string, so GCC advertised 128 bytes and cut the intended interrupted SSE event mid-JSON; the production parser was behaving correctly.

The fixture now computes the advertised length before the move. The unchanged strict assertion passes 100/100 on macOS, once in the exact x86_64 CI image (GCC 13.3, Release/O3), and as part of the local 69/69 cpp-ci suite. Replacement GitHub CI is running on the exact head.

@mvillmow

mvillmow commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator Author

Exact-head validation is complete for 09e83c9c0cb26a4cbd708b92cb0b74057be76820: all 68 head-bound checks settled with 40 successes, 28 intentional skips, and 0 failures/cancellations. The C++ build/test/release run passed, including the previously failing C++ unit-test job.

Final exact-head review is technical GO (grade A) with no actionable P0–P2 findings and all six review threads resolved. The PR remains draft; production model catalogs and backend pins are unchanged, and the documented upstream artifact/platform/governance promotion gates still apply.

@mvillmow mvillmow changed the title Add K2-Horizon validation and promotion gates docs: run IFM K2-Horizon with custom llama.cpp builds Sep 8, 2026

@mvillmow mvillmow left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[AI-assisted review] One required documentation-consistency finding and one simplification suggestion are inline. Exact-head CI is also incomplete because six checks failed for unrelated infrastructure or existing workflow reasons.

Comment thread docs/embeddable/backends.md
Comment thread docs/embeddable/backends.md
@mvillmow
mvillmow marked this pull request as ready for review September 9, 2026 19:01

@mvillmow mvillmow left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One required documentation finding.

Comment thread docs/embeddable/backends.md
@mvillmow
mvillmow merged commit 2133804 into main Sep 9, 2026
63 checks passed
@mvillmow
mvillmow deleted the codex/k2-horizon-integration branch September 9, 2026 20:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant