Switch openCodeMlx default model to Qwen3.8-27B-8bit (mlx-vlm)

[perNyfelt]

Summary

• Switches demo/openCodeMlx's default MLX_MODEL to mlx-community/Qwen3.8-27B-8bit, a vision-language checkpoint that plain mlx_lm.server can't load (unsupported architecture).
• Adds detect_model_backend(), which inspects the downloaded model's config.json for a vision_config key to auto-pick mlx_vlm vs mlx_lm as the serving backend — so future MLX_MODEL swaps between text-only and vision-capable checkpoints keep working with the existing "swap one line" UX, no second env var to keep in sync.
• Adds ensure_mlx_vlm_current() (mirrors ensure_mlx_lm_current's not-installed→hard-error / already-installed→warn-and-continue pattern), only invoked when the detected backend needs it.
• start_mlx_server_instance() now branches between mlx_lm.server (--prompt-cache-bytes) and python3 -m mlx_vlm.server (--max-kv-size, --trust-remote-code) based on the detected backend. The small background model (Qwen2.5-Coder-1.5B-Instruct-4bit) is explicitly pinned to mlx_lm and unaffected.

Known tradeoff (accepted): mlx-vlm's prompt-prefix caching is limited, so opencode's system prompt may get reprocessed every turn instead of cached like the mlx_lm.server path — a documented upstream opencode+mlx-vlm behavior. This is a real latency cost for the coding-agent workflow, not fixed here.

Test plan

• bash -n demo/openCodeMlx syntax check
• demo/test_updates_opencode.sh — 8/8 passing (no regression)
• Ran the real script end-to-end on Apple Silicon: full ~27GB ModelScope download succeeded, backend auto-detected as mlx_vlm, mlx-vlm auto-installed
• Confirmed real mlx_vlm.server --help output matches the flags used in the script
• Started the actual server against the real downloaded model; /v1/models responds (what the script polls)
• Sent a real tool-calling chat-completions request and got back a correct tool_calls response, confirming the agentic use case works end-to-end

🤖 Generated with Claude Code

[perNyfelt]

Reviewed PR #54 ✅

Verified against the actual environment:

• \ bash -n demo/openCodeMlx — syntax OK
• Ran all demo test suites: test_updates_opencode 8/8, test_updates_model 10/10, test_updates_mlxlm 8/8, test_main_update 6/6, test_main_guard 4/4 — no regressions
• Confirmed real python3 -m mlx_vlm.server --help exposes exactly the flags the new branch passes: --model, --host, --port, --prefill-step-size, --max-kv-size, --trust-remote-code
• Small model path correctly pinned to mlx_lm with --prompt-cache-bytes unchanged

Review notes (non-blocking):

1. Double opt-in is clumsy: the new default makes the active line MLX_MODEL="${MLX_MODEL:-mlx-community/Qwen3.8-27B-8bit} and the very next line a commented-out #MLX_MODEL="${MLX_MODEL:-...Coder-30B...}". The documented pattern is one active MLX_MODEL=line (the other model lines below are already bare#MLX_...comments). The new Qwen3 line should be the active one and the old default commented out as a plain#MLX_MODEL\u003d...— as written, this file has twoMLX_MODEL=` assignments at lines 13–14.
2. ensure_mlx_vlm_current pip line: pip install --upgrade pip mlx-vlm — consider matching the pip install ... spelling used two lines above in ensure_mlx_lm_current for consistency (cosmetic).
3. The name is ensure_mlx_vlm_current but the body mirrors ensure_mlx_lm_current; fine, just noting it's deliberately the same install-or-update shape.
4. detect_model_backend falling back to mlx_lm on any read/parse failure is a sensible fail-safe for existing text-only models — agreed with the design.
5. --max-kv-size reuses `` (128k tokens) as the token-count analogue of the byte cap — a reasonable mapping given mlx-vlm exposes no byte-based knob.

[perNyfelt]

Follow-up: the 8-bit model was too slow in practice, so this branch now switches to mlx-community/Qwen3.8-27B-4bit with real speculative decoding via the mlx-community/Qwen3.8-27B-MTP-4bit draft head (--draft-model/--draft-kind mtp).

Also fixed model downloads to bind to a specific network interface (MLX_DOWNLOAD_INTERFACE, default en7) — GlobalProtect VPN was resetting every connection to huggingface.co outright.

Live-verified on real hardware:

• Both models download successfully via en7 (bypassing the VPN block)
• mlx_vlm.server starts with the draft model attached, /v1/models responds
• Tool-calling smoke test works correctly with speculative decoding active
• Decode throughput: 17.8 tok/s (195-token generation, ~77% draft-token acceptance) vs the previous ~12 tok/s 8-bit baseline