Repository navigation
Description
Part of #84.
Owner: Anders (@andersalm).
Depends on LA-09 (#103) core results. Reuse verified weights where available. This issue owns qualification only. Catalogue signing, component pins and seed publication belong to #105.
Done means
Release acceptance: Each exposed feature works clearly from start to end, including recovery. Keep unfinished or confusing features hidden until their complete journey passes. Hiding a feature does not complete its task; source checks alone do not establish installed acceptance.
- Exact model, quantization, licence and provenance are recorded for both packages. The 4B pin is bartowski/Qwen_Qwen3.5-4B-GGUF revision 4168f45a16a1290d65a4ec0fa312ae917a4c15d6, file Qwen_Qwen3.5-4B-Q4_K_M.gguf, SHA-256 13c16f426047e2de38cd075bdade4a7bcbc8c774384876f677740cda65f8a983. Proof: hash check of the downloaded file.
- Each package passes at least 8 of the fixed 10 questions, including arithmetic, extraction and two-turn recall, in two complete runs out of three.
- Cold/warm start, first token, generation rate, memory and Stop are measured. The rubric is fixed before the runs.
- At least one qualified Qwen works in Assistant on each target. Capacity refusals explain why another package cannot run.
Local AI plan reconciliation — 2026-10-08
Reuse the fixed U1–U10 rubric and scorer controls after #103's accepted core milestone. Both package identities and measurements remain required.
- Verify the actual downloaded 4B and 9B file bytes, GGUF architecture, licence and complete provenance. An upstream advertised hash is a preparation fact; the local file hash supplies artifact proof.
- Use LA-04: Bound memory and local execution #98's accepted admission interface to implement and measure the required Qwen3.5 profile on the pinned engine. The engine's Qwen3.5 mapping is
QWEN35; validate the actual hybrid layout instead of assuming the ordinary Qwen2/Qwen3 attention profile. The proposed 4096 MiB 4B requirement stays provisional until measured. - Validate each scorer rule using valid and misleading replies before model runs. Keep all three complete runs, every raw displayed reply, required-question scores and the existing semantic checks. Record effective sampling settings, context/answer limits and any displayed reasoning defect.
- Measure cold/warm load, first token, rate, peak provider/engine and Home memory, resource pressure and observed Stop/descendant settlement. Record whether each model fits each actual target.
- Both packages qualify on capable hardware, and at least one qualified Qwen completes Assistant use on each required Mac/Linux x86-64/Jetson target.
#105 owns deterministic packaging, signed catalogue and production publication. This supersedes the older comment's instruction to put a model catalogue head back into release components.json. Qualification leaves model selection and hosted/remote scope explicit.
Activity
andersalm commentedon Sep 29, 2026
Qualification input retained from the preparation notes. Verify these source facts before use: bartowski/Qwen_Qwen3.5-4B-GGUF revision 4168f45a16a1290d65a4ec0fa312ae917a4c15d6, file Qwen_Qwen3.5-4B-Q4_K_M.gguf, 3,013,027,808 bytes, SHA-256 13c16f426047e2de38cd075bdade4a7bcbc8c774384876f677740cda65f8a983. Base revision: 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a. The proposed 4096 MiB requirement remains provisional until measured.
The fixed screening questions and scoring rules follow. Validate the scorer before running qualification; preserve full runs and all replies.
4.1 Items
Retain every raw reply. Score the text that Assistant actually displays; unexpected
<think>output is a separate product defect and cannot be removed to hide it. Trim outer whitespace for scoring. U3 checks the requested JSON-only response; a code fence is a format failure. In the table,\|is a plain|in the regex. Start each item in a new chat. U7 uses two turns in one chat. All regexes are case-insensitive.What is the capital city of Australia? Reply with the city name only.^\**canberra\**\.?$A shop sells pens for 3 euros each. Anna buys 17 pens and pays with a 100 euro note. How much change does she get? Reply with the number only.^(€\s*)?49(\.00)?(\s*(euros?|€))?\.?$Extract the facts from the text below. Reply with only one JSON object that has exactly these keys: "name" (string), "city" (string), "year" (integer). Text: Maria Lindqvist moves to Gothenburg in 2019 and opens a small bakery near the harbour.{"type":"object","additionalProperties":false,"required":["name","city","year"],"properties":{"name":{"const":"Maria Lindqvist"},"city":{"enum":["Gothenburg","Göteborg"]},"year":{"const":2019}}}Rewrite the sentence below in a polite, formal tone. Keep it to one sentence and reply with the new sentence only. Sentence: Send me the report now, I have waited long enough.reportand(please|kindly|could you|would you|grateful|appreciate); does not matchwaited long enough; no match for[.!?]\s+\S; ≤ 250 charactersSummarise the text below in one sentence of at most 25 words. Reply with the summary only. Text: The village of Norrby gets its drinking water from one well. The old diesel pump often fails in summer, and then families carry water from the lake. The council now has a new solar-powered pump and a tank that holds water for three days. The water supply is now reliable, and the council no longer buys diesel fuel. Before winter, the council plans to add a second tank.[.!?]\s+\S; matchessolarand(pump|water)Name exactly three fruits that are usually yellow. Use a numbered list: one fruit per line, numbers 1 to 3, and no other text.Remember this: my locker code is 4817 and my favourite colour is green. Reply with OK only.Turn 2, after turn 1 ends:What is my locker code? Reply with the number only.^\D*4817\D*$. Turn 1 is not scoredAnswer from the text only. Text: In the town of Velmora, the weekly market takes place on Tuesdays from 07:00 to 13:00. The town hall is closed on Mondays. Question: On which day of the week is the market in Velmora? Reply with the day only.^\W*tuesdays?\W*$Give me three short tips for better sleep. Use a bulleted list.^\s*([-*•]|\d+[.)])\s+\S; no match for(I can(no|')t|I am unable|I'm unable|as an AI); 40–1500 charactersTranslate into French and reply with the translation only: "The train leaves at eight o'clock."\btrain\b,\b(part|quitte)\band(huit heures|\b8\s*h|\b8[:.]00\b|\b20\s*h\b); no match for\bleaves\bSet rule: a complete run passes when at least 8 of 10 items pass, including U2, U3 and U7. Run the complete set three times; at least two complete runs must pass. Do not construct a passing result by mixing successful items from different failing runs. Record all replies and failures.
The table is a screening rubric. U4, U5, U6, U9 and U10 also need a semantic check: the answer must satisfy the request and preserve the supplied meaning. Keyword-only matches are insufficient. Check the scorer with at least one valid and one misleading reply per rule before qualification. Fix the rubric before scoring model outputs. Record effective engine sampling settings, model/engine hashes and profile; b10516 uses non-greedy defaults when the provider sends none. Adding a new sampling API is outside this preparation task.
andersalm commentedon Sep 29, 2026
DGX kit for Sasha (@SashaMIT): script, pinned inputs and run instructions. Use that single kit for the SmolLM2 ARM64 engine check and post its result in #90. It stops its engine and preserves the result.
For Qwen, reuse the fixed U1-U10 rubric in this issue's qualification comment. Keep both packages' exact identities, all three complete runs, required-question scores, cold/warm timing, first token, generation rate, memory and Stop results. Complete the installed Assistant path required here after #103's core results. The SmolLM2 script rejects Qwen inputs by design; it is the initial engine check, not a Qwen runner.
DGX measurements describe DGX only. Mac, seed and Jetson keep their own acceptance. This kit handoff leaves #104 in the backlog until its dependency and turn in the Local AI lane are ready.
andersalm commentedon Sep 30, 2026
Adding Qwen3.5-4B to the signed model list belongs to #104; #102 owns the catalogue-update and saved-state checks.
At source 77c83c56, model-catalog.json contains Qwen3.5-9B and SmolLM2, but no 4B entry.
First reconcile the prepared bartowski 4B pin in the qualification input with the older unsloth 4B component entry; they identify different files. Use one verified, revision-pinned package.
Then verify its weights, licence and provenance; measure memory, speed, Stop and the fixed usefulness rubric on the intended targets.
Package the verified files through the existing Content path, obtain approval to sign the catalogue successor, and update the catalogue head in components.json while preserving current entries.
Verify Marketplace Get → Assistant reply and catalogue-update recovery on an isolated Home; publish and pin the package on the seed only after exact approval.
This is preparation for #104, which keeps its #103 dependency. No model download, production signing or seed change was made.
andersalm commentedon Oct 10, 2026
Moved to After 0.8.0: Local AI scope for 0.8.0 is SmolLM2 and Qwen2.5 1.5B (decision 8 Oct, #84).