Strip to a local LLM hub: Ollama, a UI, logs and metrics - #18
Merged
Conversation
The research-corpus RAG stack is removed at the operator's instruction, along with all of its data. What remains is the machine's actual purpose: run local models on the GPU, talk to them, and see what the box is doing. Removed from Compose: research-hub, research-worker, searxng, crawl4ai, qdrant, redis, uptime-kuma. Removed from the repository: the research-hub application, PRDs, the research documentation set, searxng config, scripts, bootstrap. All recoverable from git history. Deleted irrecoverably, as instructed: 679 crawled documents, 54 reports, 68,072 vectors, the lexical index and the SQLite backups. Only the model store (hub_ollama_data, ~28 GB) was kept. Open WebUI no longer carries the four OPENAI_API_* variables that served the corpus as a pseudo-model, and talks to Ollama alone. Auth is off by operator decision for a localhost single-user box. Metrics are rebuilt around what this machine actually does: GPU utilization, VRAM, temperature, power and clocks from nvidia-smi; WSL2 VM CPU, memory and disk; HTTP liveness probes for Ollama and the UI; six alerts covering the three ways this box fails (out of VRAM, out of disk, Ollama stops answering). A Grafana dashboard leads with VRAM used against total, because that is the number that decides whether a model is usable. Three findings worth recording, all measured rather than assumed: - `name: hub` in Compose is load-bearing. Dropping it renamed every volume to hub-stack_* and brought the stack up healthy against an empty model store. - The GPU exporter cannot take explicit /dev/nvidia* mappings under WSL2, and panics on field auto-detection against driver 610.88. Both are now pinned. - cAdvisor returns no per-container metrics at all under Docker Desktop, tested with and without --docker_only. Dropped rather than shipped emitting one meaningless series. Measured on the rebuilt stack: qwen3.5:9b at 103 tok/s fitting in 12 GB of VRAM, qwen3.6:27b at 2.8 tok/s spilling to CPU. That 37x gap is the operating constraint of the machine and is documented as such. CI now validates the Compose file, the Prometheus config and the dashboard JSON, since there is no application source left to lint. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…files prometheus.yml refers to its rule file by absolute container path, so promtool only resolves it when the mount layout matches the container's. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tears down the research-corpus RAG stack and rebuilds the box around what it's actually for: running local models.
140 files changed, 764 insertions, 74,019 deletions.
What runs now
8 containers, all on
127.0.0.1:hub-ollamahub-open-webuihub-dozzlehub-prometheus/hub-grafanahub-gpu-exporter/hub-node-exporter/hub-blackbox-exporterDeleted, as instructed
679 crawled documents, 54 reports, 68,072 vectors, the 71k-row lexical index, SQLite backups, and the Redis/Crawl4AI/Uptime-Kuma/Postgres volumes. Only
hub_ollama_data(~28 GB of models) was kept. Source is recoverable from git history; the data is not.Measured, not assumed
qwen3.5:9bqwen3.6:27bA 37× gap on a 12 GB RTX 3080 Ti. Nothing warns you at request time — it just gets slow. The dashboard leads with VRAM used vs total for exactly this reason.
Three things that broke, now documented
name: hubis load-bearing. Dropping it renamed every volume tohub-stack_*; the stack came up healthy with{"models":[]}while the real volume sat untouched. Caught by end-to-end inference, not by health checks./dev/nvidia*mappings under WSL2 — those devices don't exist. The nvidia driver reservation withcapabilities: [gpu, utility]is what works.--query-field-names=AUTOagainst driver 610.88, which exposes a field that isn't a valid Prometheus metric name. Fields are now pinned.Tried and abandoned
cAdvisor. Under Docker Desktop it returns one root-cgroup series with
--docker_only, and zero series without it (tested privileged, with/var/lib/dockerand/dev/diskmounted). Removed rather than shipped emitting a meaningless number; blackbox probes cover liveness anddocker statscovers the rest.Verified
Compose validates · all 5 Prometheus targets
up· both liveness probes returning 1 · 6 alert rules parsingok· Grafana serving the dashboard · inference confirmed at 101–103 tok/s after the final config.Left alone deliberately
TODO/is untracked hand-written healthcare LLM evaluation research notes — not in git, so deleting it would be unrecoverable. Flagged rather than removed.🤖 Generated with Claude Code