ComfyLocalModel

A small ComfyUI node that talks to your local llama.cpp server (LLM and VLM) through its OpenAI-compatible endpoint http://<host>:<port>/v1/chat/completions and hands the response to the rest of your workflow — most naturally into a CLIPTextEncode prompt.

It's a personal project, built for my own workflows and shaped by my own habits, with no claims or pretensions of any kind. That said, it's shared in good faith — if it fits your setup too, I'm genuinely glad.

What it does

• Sends a system prompt + user prompt to your llama.cpp server; host and port are node inputs (default 127.0.0.1:8080)
• Optional IMAGE input for vision models: images go as base64 PNG, batch capped at 4. The server needs a multimodal model with --mmproj
• Returns the model's text as a STRING output
• Shows a read-only status field right on the node: endpoint, model, finish reason, token usage
• Seed widget with control_after_generate, so ComfyUI caching never quietly swallows a request
• model and api_key are optional: llama.cpp serves whatever was loaded at startup, and the key matters only if you run the server with --api-key

Features

• Nothing to install. Python stdlib plus requests, numpy and PIL — all already shipped with ComfyUI.
• Fully local. No accounts, no rate limits, no keys leaking into shared workflow files unless you set one yourself.
• Honest failures. Errors show up visibly in the UI; connection problems say plainly "is the server running at this address?". Transient errors retry with backoff, count is configurable.
• Patient. A 600-second timeout, because large models on modest hardware deserve time to think.

Downsides

• No streaming, no temperature or max_tokens knobs
• You run and manage the llama.cpp server yourself
• Tested exactly on one machine: mine

Install

Extract into custom_nodes/, restart ComfyUI, hard-refresh the browser. The node lives under text/llm.