Repository navigation
feat(stt): allow a custom pinned whisper model - #14918
Conversation
UX Review (Opus 5.5) — 🟡 CONCERNSUX-level review of UX-Verdict: CONCERNS If you pick Custom before both fields are filled, the Model picker jumps back to "base", and a file-name error shows up under the fingerprint box, not the file box. Watch
Suggestions
[UX-REVIEWED] 123542d Description read[DESCRIPTION-READ] 9882423918534311abfe1d658799e125459c9926a05c98f728385f4adcc9e014 That sha256 names everything this verdict read: the PR description, plus the 9 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Design Review (Opus 5.5) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS
Watch
[DESIGN-REVIEWED] 123542d Description read[DESCRIPTION-READ] ce48a8708ee129b3171c09fcdddde6af888354314ee7009f3e1f41a9651b1aad That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS No live load of a real third-party ggml file ran ("A live load of a real ggml model was not run here"). Check that one loads through Not justified as shipped
What this change shipsInventory (9 items) — 8 justifiedIntent: let a user run local speech-to-text on their own whisper.cpp model file. ADDITION.
[FIRST-PRINCIPLES-REVIEWED] 123542d Description read[DESCRIPTION-READ] 4d9c7b1d379bce5e7ed2002472033574ffc0fd0beb3945c3eb97072e4ba93756 That sha256 names the PR description this verdict read. Recompute it with |
1a3fbfd to
d77967a
Compare
Opus 5.5 Review — ✅ no blocking findingsReviewed Review detailsNo findings block this merge. Three advisory issues below. FINDING — src/kiro_crew/stt/models.py:1034 — FINDING — website/src/pages/settings/SttSettings.tsx:1343 — The custom rows open whenever FINDING — src/kiro_crew/stt/models.py:151 — [OPUS-REVIEWED] 123542d Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
d77967a to
8493ca5
Compare
8493ca5 to
3880ed0
Compare
3880ed0 to
5f324c2
Compare
b2b1457 to
b9036af
Compare
|
Dispositions for the reviews on UX blocker 1 (file row). The setup is now a real UX blocker 2 (SHA-256 row). The description now leads with what it is for ("A fingerprint that proves the file is the one you meant."). Under the box: "Copy it from the download page, or run the command for your system and paste what it prints:" with UX watch. A refusal now starts with "Not saved:" and ends by naming the model still in use. The half-pair line now says "…once both are filled in, unless you pick another model." The status badge is lowercase ("model file problem") to match "ready". GPT 6.1 (hash-then-load race, Updated screenshots (built SPA, |
f287e37 to
d0c1c15
Compare
d0c1c15 to
c3ee66f
Compare
|
Dispositions for the UX review on Blocker (SHA-256 row). Each command now has its system as a label ("macOS / Linux terminal", "Windows PowerShell"). The text says the download page usually lists the SHA-256 next to the file, and says to open the terminal in the folder from step 2. Watch. A refused row is now marked invalid ( Suggestion. The idle line now reads "Not used while {{model}} is selected. Pick Custom to use this file." Also fixed: Screenshots (built SPA, |
|
Dispositions for the reviews on GPT 6.1 F1: copied hash commands interpret the file name as shell syntax ( GPT 6.1 F2: native load reopens a file the agent can write, after the hash ( UX blocker (SHA-256 step). The first thing under the box is now a direct link, "Copy the SHA-256 from the file's page on Hugging Face (open the file, it is listed as SHA256)." The terminal commands sit behind an "Or work it out yourself" disclosure. Screenshot (built SPA, frame 1): |
|
/ai-review override gpt b08ec5e: F1 (unquoted copy commands) is fixed and tested; the UX round is fixed. The remaining F2 hash-then-load race at models.py:847 needs an attacker who can already write the models directory on the host, and it is the same hash-then-reopen pattern the model catalog path already ships on main (#16054); this PR adds no new load path. The fd-pinned hash-and-load fix covers both paths and goes in a separate follow-up. |
Human judgment recorded@iamwhatever marked the gpt AI finding as false positive, not applicable, or explicitly accepted for
This decision applies only to this commit. A new push requires a new judgment. |
|
Dispositions for the UX review on Blocker. The second row is now labelled "File fingerprint (SHA-256)". Its description is the instruction: "Paste the SHA256 value shown on the model file's download page." The link now reads "Open the file's page on Hugging Face, the model download site, and copy its SHA256." Watch. The missing-file notice now renders under the file-name row, not after the SHA-256 row. With a built-in model selected, the "Not used while … is selected" note comes before the rows, and the rows are dimmed. The "Ask the agent" wording is unchanged: it is the shared Suggestion. Step 2 now reads "Put it in the speech models folder below.", matching the wording in the errors. Screenshots (built SPA, |
|
Disposition for the UX review on Blocker. The SHA-256 row's visible description now carries the purpose and the safety line, not only the ⓘ: "Checks the file is the one you downloaded. Paste the SHA256 value shown on its download page. A wrong value is refused; nothing else changes." Screenshots (built SPA): |
|
/ai-review override ux fa81bbe: eight UX rounds on copy and layout only, each fixed and dispositioned; round 8 reverses round 7 (show the hash commands by default vs hide them behind a disclosure), so the lane is not converging; no behavior, accessibility or safety finding is open, and GPT and Opus are clean on this head. |
Human judgment recorded@iamwhatever marked the ux AI finding as false positive, not applicable, or explicitly accepted for
This decision applies only to this commit. A new push requires a new judgment. |
|
/ai-review override ux 8c6e463: same copy finding as the override on fa81bbe; this head only rebases and adds sensitive=True with a test, no UI copy changed. Eight prior UX rounds on copy/layout were fixed and dispositioned, and the lane reversed itself (hide vs show the hash commands), so it is not converging; no behavior, accessibility or safety finding is open, and GPT and Opus are clean on this head. |
Human judgment recorded@iamwhatever marked the ux AI finding as false positive, not applicable, or explicitly accepted for
This decision applies only to this commit. A new push requires a new judgment. |
|
New head
|
|
New head
|
|
New head
|
stt.model = "custom" runs a whisper.cpp ggml model file the user has placed in the speech models folder (models/whisper under the Kiro Crew home). Two settings select it: stt.custom_model_path names the file and stt.custom_model_sha256 pins it. Nothing is downloaded. The path must resolve, symlinks followed, to a regular file strictly inside the models folder. `..`, an absolute path elsewhere and a symlink pointing out are refused by models.custom_model_file, which runs when the dashboard saves the field and again on every load. PUT /api/config/stt answers a refused file name or a malformed digest with a 400 naming the rule, and Settings > Voice shows it under the rows with the agent hand-off. The file is hashed on every load and not used unless it matches the digest. A refused file is left in place, because it is the user's own. Both fields are owner-only on GET and PUT; config.json is read-only in the agent sandbox, so the pin cannot be rewritten to match a file an agent placed in the folder. The catalog's models seal checks apply to the custom file too. The models and whisper folders must be real directories, and the hash reads one O_NOFOLLOW descriptor, so a second hard link or a symlinked leaf refuses the load. A half-configured pair degrades to the default model with a warning. A complete pair whose file is missing does not degrade: it fails at load with the reason shown, so a different model is never run in its place. The digest field accepts a `sha256:` value and a `sha256sum` line, and still requires exactly 64 hex characters. Co-authored-by: LuisBrel <193017400+LuisBrel@users.noreply.github.com>
|
New head
|

























Supersedes #8281 — in-repo replacement of the fork PR by @LuisBrel (authorship preserved via
Co-authored-by), rebased onto freshmain.Problem / Motivation
Goal: Let a user run local speech-to-text on their own whisper.cpp ggml model instead of only the four catalog models.
Voice input is locked to a fixed catalog (
tiny,base,small,large-v3-turbo). There is no way to run a fine-tune, a quantised build, or a language-specific model.memory.embed_model_pathalready solves the same shape for embeddings; STT had no equivalent.Why it matters
Users who need a model outside the catalog (accuracy on an accent or jargon, a smaller quantised build for a constrained machine, an offline copy) cannot use local STT with it at all today.
Not a goal
customselects a different model for the bundled whisper.cpp engine.What changed
SttConfiggainscustom_model_pathandcustom_model_sha256;stt.modelacceptscustom.stt/models.py:custom_model_fileresolves the path, symlinks followed, and refuses anything not a regular file strictly inside the models folder (models/whisperunder the Kiro Crew home): any..component, an absolute path elsewhere, a symlink pointing out, the folder itself, a built-in model's file name in any case, or the same file under another name (ggml-base.binetc., which the catalog download would delete or overwrite), and a.partstaging file.~is not expanded. The engine loads the exact real path that was hashed. It runs when the dashboard saves the field and on every load.modelsand the whisper folder must be real directories (sandbox.require_real_model_path), and the hash reads oneO_NOFOLLOWdescriptor (_sha256_verified_file), so a second hard link or a symlinked leaf refuses the load with afailedstatus flaggedrefused. A link below the folder is still followed and must stay inside it.PUT /api/config/sttanswers a refused name or a malformed digest with a 400 naming the rule (custom_model_path_outside/_missing/_not_file/_empty,custom_model_sha256_invalid). The message never quotes the path.GET/PUT /api/config/stt, audited on both branches.config.jsonis read-only in the agent sandbox, so the pin cannot be rewritten to match a file an agent placed in the folder.shasum -a 256/Get-FileHashcommands sit as copy blocks under "Work out the SHA-256 yourself". Status reads "Model file problem" while a configured custom file is unusable. Refusals show under the rows as anErrorNoticewith the agent hand-off; a missing configured file shows its reason instead of a download button.customreadsconfig.json.Known limit
The hash and the native load are two opens of the same path, so a writer that swaps the file between them is not caught. Catalog weights have the same race. The models seal from #16054 keeps a sandboxed agent from being that writer, and the custom path now goes through the same load checks (above); an unsandboxed process on the host can still swap the file between the two opens, as it can for catalog weights.
Backwards compatibility
Compatible: both fields default to
""andstt.modelstill defaults tobase. Purely additive.Tests
test/test_stt_custom_model.py: containment (.., absolute elsewhere, symlink leaf out, symlinked folder out, the folder itself), missing / folder / blank, digest carriers, resolve and degrade, load-time hash refusal that keeps the file, swap-after-load refusal, no download ever, and (POSIX) a hard-linked file and a linked whisper folder refused at load. Each of those two fails with its check removed.test/test_dashboard_handlers_core_coverage.py: owner-only gate, and the 400s for a path outside the models folder, a missing file, a folder, and a bad digest.test/test_config_loader.py,test/test_stt_stream.py: loader wiring and off-loop resolve.website/src/test/SttSettings.customModelGuard.test.tsx: the incomplete-pair line, the refusal notice with hand-off, the missing-file notice, the draft hand-off guard, and the setup copy: the SHA-256 row's first sentence names where the value comes from, it gives the refusal reassurance once (in its ⓘ hint), and its folded section is named after what it works out; the model site is linked once, from step 1, which says the file page lists the SHA-256; and step 3 shows a real file name. Each of those fails without its fix.Manual verification
Screenshots below come from the built SPA with fixture-backed
/api/**via the committed harnesswebsite/scripts/capture-stt-custom-model.mjs(no gateway, no network). A live load of a real ggml model was not run here.Screenshots / video
1. Custom selected, file in the models folder

2. Configured file missing: Status says so, and the reason shows with the agent hand-off, no download button

3. Half-configured pair: the picker reads

base, and the line says why4. A name outside the models folder is refused under the rows

5. A complete pair with a built-in model selected: the rows say they are idle

6. "Work out the SHA-256 yourself" opened under the SHA-256 box: the command for each system, with the file name quoted for that shell

Related Issues
Supersedes #8281.
Checklist
feat: ...)docs/configuration.md, system-specs)