[raiseCatError]

/btw (alias /ask) can now answer general and terminal questions with the local model, offline. Every existing path stays as it was: NMSh actions and their confirmations, refusals, and Copy/Insert-only command suggestions. No existing issue covers this; related to #432 (command suggestions), which this does not change.

Design: docs/design/ask-conversation.md.

Before and after

Before: Ask mapped requests onto NMSh capabilities. The model only chose a capability (intent), hinted folds, or suggested one command. A question it could not place got "I'm not sure what you mean yet", and in Auto the 0.6B intent model often read it into the wrong capability. Recorded with the real model:

• "I have 3 apples and eat one…" and "Correct the grammar: …" → This session runs Fish
• "What is in my clipboard right now?" → a folder listing
• "Why would I use git stash?" → Git status
• "show me my api keys" → tried to open a file named "api keys"

After:

• A question Ask cannot place, or a follow-up to a model answer, gets an answer from the local model.
• The answer streams into a turn labelled Ask · local model · <model>.
• Esc stops it without closing Ask, and you can type the next question while it generates.
• Each answer offers Copy answer, Copy code and Think it through. The last re-asks with a bounded Qwen3 thinking pass and replaces the quick answer.

Routing (deterministic; the model never routes)

1. Fail closed. A request that looks like it contains a secret is not sent. A request to show the environment or credentials is refused.
2. Terminal context only when asked. "Explain the last error" / "summarise the last output" use the block chosen with Explain, else the newest failed or finished block.
   • NMSh's own diagnosis comes first, as its own turn, with its own next steps.
   • The model then gets the redacted command and at most 40 lines / 2,000 characters (ANSI removed, each line redacted).
   • The answer says which context was sent.
3. Ask's own resolver answers what it can place: proposals keep their confirmations, refusals stand, and shell tasks keep the suggestion path. Questions never go to the intent model.
4. Questions it can't place go to the model. That includes a match on a command word the question merely contains ("HTTP status 404", "today's date", "a Python list"), a question about another language ("sort a list in Python"), and a follow-up. "What does chmod 755 mean" keeps NMSh's exact catalog answer.
5. Unclear requests get a clarifying question.
   • "Fix it" offers to explain the last error, summarise the last output, answer it as a question, or open a file.
   • In Auto, the intent model's reading of an unclear request becomes one choice in that question instead of being acted on.
   • Always mode is unchanged.

The answer task

One more task in the existing user-global model service; no second model or backend.

• Request: the question, a bounded history, the shell, the OS family and optional context; any other field is rejected.
• History: at most 6 turns, 900 characters each, 2,400 in all, oldest whole exchanges dropped first. It lives per Ask in this window, in memory only, and is never persisted or shared; a new /ask starts empty. The service is stateless, with the prompt cache off.
• Qwen3 settings: non-thinking by default with the model card's sampling (0.7 / 0.8 / 20, min-p 0, presence penalty 1.5). Thinking uses 0.6 / 0.95 / 20 and runs only on request. Reasoning is never shown, streamed or kept.
• Bounds: 512 output tokens (900 thinking) in the 2,048-token context; 45 s generation limit (67 s thinking).
• Streaming: llama.cpp and LM Studio over server-sent events, Ollama over NDJSON; sanitized partial text at most every 50 ms.
• Cancellation: per request. It drops the request if queued and aborts it if running; closing Ask or a window also stops it.
• Older services started by a previous NMSh are never sent the task, and Ask says so.
• Model text is untrusted:
  • escape sequences (CSI, OSC including hyperlinks and clipboard writes, DCS, C1), controls and bidi/zero-width characters are removed;
  • the answer must be exactly {text, truncated};
  • NMSh renders it itself: code indented and hard-wrapped, lists, narrow widths, NO_COLOR;
  • shell code blocks get NMSh's own risk label only; nothing becomes an action, and "run it" has nothing to run.
• Recording follows the existing setting: off persists nothing; on records plain text marked Local model (…).

Thinking and non-thinking

Both modes were tested on the real model; results are in the examples below.

Prompt tuning (measured, real model)

A/B on 10 weak cases × 3 runs: adding "rewrite: reply with only the new text" and "say you cannot see the user's files" took loose checks from 18/30 to 22/30. It stopped answers like "No, there is no typo on line 12 of your zshrc" and invented dates.

Quoting the text apart from the instruction: this turned "Rewrite this more politely: send me the report now" from "Sure! Let me know when you're ready…" into "Please send me the report." (5/5). Corrections measured 5/5 unquoted against 4/5 quoted, so only rewrite/shorten/translate requests are quoted.

Real Qwen3 0.6B examples (final run, verbatim)

Good:

• Water boiling point: "Water boils at 100°C at sea level."
• 404: "…the requested resource was not found."
• Polite rewrite: "Please send the report now."
• Translation: "Buenos días."
• Follow-up chain: a Python list → sorted() example → sorted(my_list, reverse=True) with code
• git stash follow-up: "use git stash pop"
• Weather, bitcoin, news, today's date, clipboard, zshrc: all say it has no access (for example "I cannot view your clipboard as I don't have access to your files or environment.")

Wrong or weak (the loose checks miss some of these; I read every answer):

• "What is the capital of Australia?" → "Sydney" (Canberra in 1 of 3 A/B runs)
• "Is 91 prime?" → "Yes, 91 is not a prime number. It is an even number and divisible by 7." Thinking: "No, 91 is not a prime number."
• Bat and ball → "$0.50". Thinking: "$0.05".
• 9.11 vs 9.9 → non-thinking "9.9 is larger"; thinking "9.11 is larger than 9.9" (wrong; thinking is not always better)
• git stash → "save your changes to a remote repository" (wrong)
• merge vs rebase, closures, hard vs symbolic links: muddled or partly wrong
• Grammar correction: one run gave "Go to school yesterday." (5/5 correct in a separate run)

Routing in the final run: 48 of 49 as expected. The one miss is a known gap kept in the set: "In a shell, what is the difference between > and >>?" is claimed by NMSh's shell guide.

Performance (Apple M2, 8 GB, llama.cpp b11146 Metal, Qwen3-0.6B-Q8_0)

	measured
Cold start (load + first answer)	first text 3.6–4.7 s, done 4.1–5.5 s
Warm, first text	0.25–0.9 s (prompt ≈ 165 tokens)
Generation	30–50 tok/s on a quiet machine (4–47 under load); answers 8–60 tokens, 0.4–2 s
Max history (≈ 500 prompt tokens)	first text 0.6–0.8 s vs 0.27–0.34 s without
Largest allowed request	1,430–1,473 prompt tokens, leaving 575–618 for the answer (budget 512)
Thinking pass	2–17 s (92–284 tokens)
Memory (llama-server RSS)	≈ 840 MB steady, 1.08–1.14 GB peak; 902 MB at context 2,048 vs 1,128 MB at 4,096, so it stays 2,048
Cancel	resolved in ≈ 1 ms; next answer's first text 0.4–0.7 s; service ready
Unload	0.4–0.5 s, process exits; reload first text ≈ 3 s

No larger model was evaluated: none is installed, and NMSh never downloads one.

Tests

• New tests/askConversation.test.ts (fake runtime): validation and sanitizing, request bounds, history trimming, prompt, ChatML and soft switch, stream parsing, service streaming, thinking budget, cancel (queued, running, other window untouched), closing window, timeouts, older service, eligibility, signals, context excerpt, shell checks, rendering at 24/40/80 columns, NO_COLOR.
• New tests/askConversationApp.test.ts (real TerminalApp, stub model): every route in the evaluation set; questions never reach the intent model; actions and refusals kept; streaming labelled turn with nothing actionable; follow-up history, bounds and a new Ask starting empty; Esc stops without closing; typing during generation; close stops it; context only when asked, redacted; fail-closed cases; recording off/on; Think it through; copy; intent guess becomes a choice; follow-up and other-language routing.
• Focused run on the final code: all tests/ask*.test.ts, tests/understanding.test.ts and the two new files — 174 passed, 0 failed.
• Real model: scripts/local-model/askEval.ts with askEvalSet.ts (49 cases across general knowledge, programming, shell and git, reasoning, rewriting, follow-ups, ambiguous, current events, can't-know, command-like and security).
• npm run verify: see the comment below; the first local run was under load averages up to 32.

Needs human terminal testing (not done)

Ghostty / Terminal.app with a real model:

• streaming redraw and the generating line
• Esc to stop, and typing while it generates
• narrow widths, reduced motion and NO_COLOR in the real panel
• Copy answer and Copy code through the real clipboard
• Explain on a failed block
• Ollama and LM Studio streaming and soft switches (only llama.cpp was run for real)

Unsuitable for 0.6B

Factual recall is unreliable (it said "Sydney"). Arithmetic and reasoning need the thinking pass and still fail sometimes. Its git explanations are often wrong. It is fine for short rewrites, translations of simple phrases, common programming idioms, and saying what it cannot know. The labels and the design say so; a larger, user-chosen model was not measured.