slugvision

Tiny vision-language models that turn an uploaded image (plus an optional article title) into a 3–5 word kebab-case permalink slug, e.g. golden-retriever-beach-sunset — small enough to serve cheaply on CPU as editors upload images.

Built by distillation: a large local teacher VLM labeled a ~32k-image corpus, and SmolVLM2 students were LoRA fine-tuned on the pairs. The published artifacts live on the HF Hub:

Artifact	Link
500M model (CPU pick: ~1.6s/img cold, 6 threads)	kelnei/slugvision-500m
2.2B model (quality pick: best untitled-photo slugs)	kelnei/slugvision-2.2b
Training corpus (5 slices, labels + generated images)	kelnei/slugvision (dataset)
Everything	slugvision collection

Both model cards document the input contract (two fixed prompt templates, 1024px image cap) and ship GGUF quants for llama.cpp serving.

Results

Blind pairwise LLM-judge vs the 26B-class teacher on a frozen 1,000-image holdout (details and full v1–v5 ladder in docs/exploration.md):

• Titled inputs are near teacher parity even at 500M — when the article headline is in context, the small model matches the teacher's usefulness.
• Untitled photos are capacity-bound: 4x data, extra slices, and higher LoRA rank all left the 500M's gap at ~1.0 points; the 2.2B cut it to 0.63.
• Quantization (Q8/Q4 GGUF) is quality-neutral; image encoding dominates CPU latency (500M ≈ 1.6s, 2.2B ≈ 16s cold per image at 6 threads).

Pipeline

Everything runs locally (vLLM teacher + FLUX renderer sharing one GPU):

1. Corpus — COCO photos, plus fully generated slices: the teacher authors FLUX prompts (illustrations, products, clinical) and complete HTML pages (screenshots via Playwright); FLUX.1-schnell renders images.
2. Labels — the teacher slugs every image (scripts/generate_slugs.py, resumable); titled slices are labeled with their source headline in context, which trains the titled mode.
3. Train — LoRA via TRL/PEFT (scripts/train_student.py).
4. Evaluate — format/uniqueness/token-F1 (scripts/eval_student.py), blind pairwise LLM-judge (scripts/judge_eval.py).
5. Serve — merge + GGUF export (scripts/merge_lora.py), CPU latency benchmark simulating serverless cold starts (scripts/bench_cpu.py).

Layout

src/slugvision/   library code (slug normalization, prompt contract)
scripts/          runnable pipeline steps, in pipeline order above
docs/             research, decision notes, full results
data/             images + generated pairs (gitignored)
runs/             training outputs, merged checkpoints (gitignored)
gguf/             exported GGUF models (gitignored)

License and attribution

Code is licensed under Apache 2.0; redistributions must retain the attribution in NOTICE. The pipeline builds on several models and datasets with their own terms:

• Slug labels are generated with Gemma 4 (gemma-4-26B-A4B-it), subject to the Gemma Terms of Use — the published models are distilled from these outputs and carry the gemma license tag.
• Illustration/product/clinical images are rendered with FLUX.1-schnell (Apache-2.0) by Black Forest Labs.
• Photo images come from COCO 2017; images are Flickr-sourced under varying licenses and are not redistributed here or in the published dataset (labels are keyed by COCO filename).
• The student models fine-tune SmolVLM2 (Apache-2.0) by Hugging Face.