Repository navigation
feat(stt): make the local recogniser decode thread count configurable - #19648
Conversation
Design Review (Opus 5.5, fork) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing to check. [DESIGN-REVIEWED] f9c4874 Description read[DESCRIPTION-READ] 815de78777945244f1ea6b85dd132d2572b9063b02e493d28c6f2a78c6edce0b That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5, fork) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS
Not justified as shipped
What this change shipsInventory (8 items) — 6 justifiedIntent: let an operator on a large host use more CPU threads for local dictation so it finishes sooner. This is an ADDITION.
[FIRST-PRINCIPLES-REVIEWED] f9c4874 Description read[DESCRIPTION-READ] 432633c3394e9814b54c000b712de133f3f11e2051b93c314f46d198723500a2 That sha256 names the PR description this verdict read. Recompute it with |
UX Review (Opus 5.5, fork) — 🟡 CONCERNSUX-level review of UX-Verdict: CONCERNS The disabled picker shows "16" while the note under it says 64 threads are running, so the screen shows two numbers for one setting. Watch
Suggestions
[UX-REVIEWED] f9c4874 Description read[DESCRIPTION-READ] 8f10c2e080012c66417b18b6cf1016513233267b046fa0e7bfeecdf0856880d1 That sha256 names everything this verdict read: the PR description, plus the 6 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review (fork) — ✅ no blocking findingsReviewed |
GPT 6.1 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
f7765d3 to
27ee908
Compare
|
Rebased on main 365d87c and regenerated the find_ui index with npm run gen:ui, which fixes the stale-index failure in Frontend Lint & Type Check. No behaviour change. The earlier Publish readiness signal failure was a CI-app rate limit (HTTP 403 at 01:49Z), not a code finding. |
27ee908 to
126d385
Compare
|
issue (blocking, security): oversized digit strings crash config loading. Fixed in 126d385. parse_threads now refuses an ASCII digit string with more digits than MAX_THREADS has, after stripping leading zeros, before it calls int(). A 5,000-digit stt.threads in config.json therefore loads as auto with a warning, and the CLI and dashboard refuse the write. New test cases cover a 5,000-digit value in the parser, the config loader and kirocrew config set, and check that 0048 still reads as 48. |
126d385 to
eeeadd7
Compare
Security Scope Review (fork) — 🔴 BLOCK (blocking)Which legitimate operations does no leg reported, and this was not the nothing-to-adjudicate short circuit No |
|
mechanism: LocalSession and transcribe_pcm now default threads to None, the same "no opinion" value shared_engine and prewarm use.
|
|
|
|
mechanism: the override note now leads with the outcome and says how to undo it.
|
|
|
mechanism: parse_threads now strips leading zeros once and both the length check and int() read that stripped string.
|
eeeadd7 to
9abd9ba
Compare
|
chore: Build Wheel, Build Desktop and both Build image jobs failed on 5e19fd6 at gen-ui-index. Main had renamed the generated settings field from |
|
Pushed head 7812eb2, still one commit, rebased on current main. note (non-blocking): every review lane on 33e0315 failed before producing a verdict because the CI app ran out of API quota between 07:42Z and 07:46Z. Publish readiness signal, Fork Internal Content Scan and Screenshot Evidence logged "API rate limit exceeded for installation", Sensitive Change Review could not read the PR, and the GPT 6.1 and Opus 5.5 fork lanes reported "could not run". Dependency License Gate reports that dependency review is not enabled on the repository, which this PR does not change. Backend Lint & Type Check and Backend Tests shard 2 (Linux and Windows) failed on chore: the new head also applies the Opus 5.5 advisory from 5e19fd6. The constants table in docs/system-specs/modules/stt-streaming.md now lists |
|
note (non-blocking): CI on 7812eb2 has finished, and none of the five red checks touch the STT change. Could a maintainer rerun the failed jobs? The fork account cannot.
|
|
Rebased on main, no behaviour change. Head a7c3fa9 is still one commit, and range-diff against 7812eb2 shows the patch unchanged. 329 tests across the STT engine and session suites and the two dashboard core-coverage files pass, and flake8 and gen-ui-index --check are clean. note (non-blocking): this push retriggers CI. The five reds on 7812eb2 were E2E (stub ACP backend, offline) at the 1020 s bound tracked in #18185, PR Scope cancelled during checkout, one case in test_agent_scratch.py on backend shard 8, two tests in test_app_mcp_tool_policy_scope.py on Windows shard 7, and Coverage Gate failing on those shards. None of those files is in this diff. The rerun ask from 09:22Z had no answer after 4 hours, and the fork account cannot rerun jobs. |
|
Rebased on main, no behaviour change. Head bc7256a is still one commit, and range-diff against a7c3fa9 shows the patch unchanged. 207 tests across test_stt_threads.py, the two dashboard core-coverage files and test_app_mcp_tool_policy_scope.py pass, and flake8 is clean. note (non-blocking): the Windows shard 7 red on a7c3fa9 was test_app_mcp_tool_policy_scope.py failing on a cross-drive |
|
Rebased on main 93617e9 to clear the merge conflict, no behaviour change. Head a0bfae9 is still one commit. #20555 split src/kiro_crew/transcribe.py into owner modules, so this PR's one added line, note (non-blocking): the backend shard 6 red on bc7256a, on Linux and Windows, was test_the_lazy_imports_stay_inside_the_functions_that_need_them asserting 84 == 83. #19700 added a page_cache entry to the lazy-import table without raising the pinned count, and #20930 fixed that on main at 20:35Z. This diff does not touch that file. Coverage Gate failed only because that shard did. The E2E (stub ACP backend, offline) red was the 1020 s suite bound tracked in #18185, with 322 specs passed and 7 not run; this push gives it a fresh run. |
|
No code change. Head 2ef00e2 has the same tree as a0bfae9 and only a new commit date, to rerun CI. note (non-blocking): the one blocking readiness item on a0bfae9 was Backend Tests (Windows) shard 2. It failed on test_member_dashboard_package_route.py::test_the_gallery_pages_own_subscriber_receives_the_tree, where a node subprocess hit its timeout, and on test_md_notebook.py::test_a_sync_while_the_temp_is_still_staging_never_commits_it with an asyncio CancelledError. Neither file is in this diff. Every other check passed on that head, including E2E (stub ACP backend, offline), and the fork account cannot rerun a job. |
|
Rebased on main 7e77836 to clear the merge conflict, no behaviour change. Head 48aa321 is still one commit. #21129 split src/kiro_crew/config/sections.py into sections_* owner modules behind a facade. This PR's two additions there moved with the STT owner: 3491 tests across the sections facade and config contract suites, the STT and transcribe suites, the dashboard core-coverage files and the app MCP tool policy suite pass. flake8 is clean on the changed files, and gen:settings and gen-ui-index --check report nothing to regenerate. |
|
Rebased on main 2245fd4 to clear the merge conflict, no behaviour change. Head 540e1c0 is still one commit. The only conflict was src/kiro_crew/docs/ui-index.generated.json, digests and counts only; I took main's copy and regenerated it with gen:settings and gen:ui, and gen-ui-index --check reports it up to date. 1265 tests across test_stt_threads.py, the sections facade suite, the STT and transcribe suites and the app MCP tool policy suite pass, and flake8 is clean on the changed files. note (non-blocking): CI never ran on the previous head 48aa321. All 20 of its fork workflow runs stayed at action_required waiting for maintainer approval, so there were no check results on it. |
|
Rebased on main ee95877 to clear the merge conflict, no behaviour change. Head ce4fb3e is still one commit. The only conflict was again src/kiro_crew/docs/ui-index.generated.json, digests and counts only; I took main's copy and regenerated it with gen:settings and gen:ui, and gen-ui-index --check reports it up to date. 1346 tests across test_stt_threads.py, the sections facade suite, the STT and transcribe suites and the app MCP tool policy suite pass, and flake8 is clean on the changed files. note (non-blocking): CI has not run on this PR since 08:12Z on 10-16. The fork workflow runs on 48aa321 and 540e1c0 all stayed at action_required waiting for maintainer approval. |
|
Rebased on main 85dcaee, no behaviour change. Head 3be2e7c is still one commit and rebased without conflicts. 1346 tests across test_stt_threads.py, the sections facade suite, the STT and transcribe suites and the app MCP tool policy suite pass, flake8 is clean on the changed files, and gen-ui-index --check reports the index up to date. note (non-blocking): Real Adapter Contract Tests failed on ce4fb3e in test_codex_session_mcp.py::test_the_installed_adapter_still_builds_the_frames_the_refusal_reads, because the installed codex-acp no longer contains |
|
Rebased on main 273a50b to clear the merge conflict, no behaviour change. Head 52f4009 is still one commit. The only conflict was again src/kiro_crew/docs/ui-index.generated.json, digests and counts only; I took main's copy and regenerated it with gen:settings and gen:ui, and gen-ui-index --check reports it up to date. 1347 tests across test_stt_threads.py, the sections facade suite, the STT and transcribe suites and the app MCP tool policy suite pass, and flake8 is clean on the changed files. |
iamwhatever
left a comment
There was a problem hiding this comment.
Requesting changes at 52f4009. The r99 cold-import concern is fixed: _stt_threads_env now resolves through stt.limits and cpu_affinity and never imports the engine (
KiroCrew/src/kiro_crew/dashboard/handlers/core.py
Lines 1124 to 1146 in 52f4009
Remaining blockers:
- The three deferred items (Auto resolved-count label, reload note, thread-options comment) are tracked only in Pearcekieser#36 on your fork, which is not linked in the PR description and cannot carry this repository's deferred-finding label, assignee and Due line. Please fix them here, or open one issue per item in kirodotdev/KiroCrew and link each in the description.
- The UX Review suggestion on this head about the zh-CN hint is unanswered. decode_threads_desc still describes threads as CPU cores (https://github.com/kirodotdev/KiroCrew/blob/52f40098fbb7b11465e27dfb4bdfe0ebe502f1a2/website/src/i18n/locales/zh-CN.json#L17260). Fix the wording or post a disposition.
- The First Principles CONCERNS on this head have replies only on older heads. Your #19647 reasoning stands; a current-head disposition is enough.
— crew-pr-reviewer (r111)
|
@iamwhatever thanks for r111. Head 75d0c17 addresses all three items. It is one commit rebased on main 45d21f5; the only conflict was the generated ui-index digests, regenerated. todo (blocking): deferred items tracked only on the fork.
issue (non-blocking): UX Review suggestion on the zh-CN hint. note (non-blocking): First Principles CONCERNS on the current head, about Checks run locally: 1456 backend tests across the STT, transcribe, sections facade, dashboard core-coverage, settings registry and app MCP tool policy suites, plus 1169 frontend tests including the i18n suite. tsc -p tsconfig.app.json, flake8, |
|
HOLD at 75d0c17. The r111 blockers are cleared: the three deferred items are now fixed in this PR (Auto (n) label from threads_auto, the reload note, the thread_options comment), the zh-CN hint no longer says CPU cores, and your reply on this head answers the First Principles env-var concern with the per-process reason. Checks are green. Two current-head AI items still have no reply:
Either a fix (which needs a fresh review) or a current-head disposition per item clears this hold. No PR code was run locally. — crew-pr-reviewer (r112) |
|
@iamwhatever thanks for r112. Dispositions for the two current-head items on 75d0c17, with no push so readiness stays as it is. note (non-blocking): Opus advisory, "this machine's cores" in the stt.threads help, config-baseline.json and decode_threads_desc. note (non-blocking): UX watch item, the disabled picker shows the saved value while KIROCREW_STT_THREADS is set. |
|
Rebased on main 8e686e5 to clear the merge conflict, no behaviour change. Head 3712064 is still one commit. Main added note (non-blocking): the two r112 items have their dispositions on 75d0c17 in issuecomment-6268967889. This rebase leaves those lines unchanged. |
|
Rebased on main 02eeb00, no behaviour change. Head 71be0cc is still one commit, and the rebase had no conflicts. note (non-blocking): the Harness Parity Gate red on 3712064, which also failed Await Fast Gate and Coverage Gate, came from main. The x-capture check reported "ACP_BACKENDS_MEMBER_CAPABILITIES [claude]: allowlisted but now proven; delete the entry", and the same error reproduced on a clean checkout of main 6257f69. #22178 removed that entry at 17:19Z. On this head 1170 frontend tests and 1430 backend tests across the STT, transcribe, sections facade, dashboard core-coverage and settings registry suites pass. tsc, |
|
HOLD at 71be0cc. Our r111 and r112 items are answered, and the replies on the env override and the Two current-head AI items still have no reply:
The replies also say each speech MCP server reads CI is green on this head. — crew-pr-reviewer (r114) |
|
@iamwhatever thanks for r114. Head 992c8db is still one commit on main 02eeb00. issue (non-blocking): the override note passes the raw note (non-blocking): UX Watch 2, the reload sentence sits in the tooltip. question (non-blocking): which process reads |
|
Rebased on main 5ccd492 to clear the merge conflict, no behaviour change. Head 9b3f563 is still one commit. #22362 split src/kiro_crew/dashboard/handlers/core.py into core_* owner modules behind a facade. I took main's core.py and moved this PR's additions into the owners that now hold that code:
The tests that patch |
|
fix: Backend Tests (3.12, 2) on 9b3f563 failed in test_facade_mirror_guards.py::test_every_mirrored_facade_manifest_names_what_it_forwards. The core facade served note (non-blocking): the same shard also hit the 120 s timeout on test_every_name_the_test_trees_patch_through_a_facade_reaches_its_readers. Locally that test takes 59 s and passes on this head, so I am treating the CI timeout as runner load. If it times out again on this head, I will look at it as a real failure. test_facade_mirror_guards.py, test_core_handlers_facade.py and the dashboard core-coverage file pass, 171 tests in all, and flake8 is clean. |
|
Rebased on main dea18bc to clear the merge conflict, no behaviour change. Head 310213e is still one commit. The only conflict was src/kiro_crew/docs/ui-index.generated.json, in its digests and counts. I took main's copy and regenerated it with gen:settings and gen:ui, and gen-ui-index --check reports it up to date. 1645 backend tests pass, covering the facade mirror guards, the core facade, the dashboard core-coverage files, the settings registry, the sections facade, and the STT and transcribe suites. 1170 frontend tests also pass. tsc, |
|
Rebased on main 4d5cb47 to clear the merge conflict, no behaviour change. Head f9c4874 is still one commit. The only conflict was src/kiro_crew/docs/ui-index.generated.json, in its digests and counts. I took main's copy and regenerated it with gen:settings and gen:ui, and gen-ui-index --check reports it up to date. 1645 backend tests pass, covering the facade mirror guards, the core facade, the dashboard core-coverage files, the settings registry, the sections facade, and the STT and transcribe suites. 1170 frontend tests also pass. tsc, |
|
HOLD at f9c4874. The r114 items are handled: the override note now formats the count with fmtNumber, the reload sentence stays in the tooltip with a stated reason, and the THREADS_ENV comment and description now say the gateway is the only process that loads the model. The move into core_stt_settings and core_stt_status after the core.py split matches main's layout and keeps the GET path free of engine imports. CI is green on this head. The First Principles items (env override, auto spelling) stay answered by #19647, and the UX two-numbers item by your disposition in issuecomment-6268967889. One current-head AI item has no reply:
A disposition with no push clears this hold. A fix needs a fresh review. No PR code was run locally. — crew-pr-reviewer (r116) |
|
@iamwhatever thanks for r116. Disposition for the one current-head item on f9c4874, with no push so readiness stays as it is. note (non-blocking): UX Review suggestion, the override note does not mention the restart. |
iamwhatever
left a comment
There was a problem hiding this comment.
Approving f9c4874.
Your disposition on the override note's restart step answers the last open item from r116. The note appears only to an operator who set KIROCREW_STT_THREADS in the gateway's environment, and configuration.md documents the restart.
The change is sound. parse_threads refuses bools, non-ASCII digits and over-long digit strings before int() runs, and an explicit count is clamped to the cores the process may use. The config GET resolves the counts without importing the engine (core_stt_settings.py#L152-L174), and callers that pass no thread setting keep the operator's count. #19647 asks for both the env override and the auto spelling, which answers First Principles. Every other lane item has a reply, the description defers nothing, and all checks on this head are green.
No PR code was run locally. Any new push needs a fresh review.
— crew-pr-reviewer (r117)
Problem / Motivation
Goal: let an operator choose how many CPU threads the local speech-to-text recogniser decodes with, so a large host can transcribe faster.
The recogniser derives its thread count from
THREAD_CEILING = 16insrc/kiro_crew/stt/engine.py: half the usable cores, never more than 16. Nothing lets an operator change that. On a 128-core host, large-v3-turbo decodes the 11 s JFK sample in 3.7 to 4.2 s at 16 threads, 2.0 s at 32, 1.6 s at 48, 1.25 s at 64 and 1.1 s at 96.Why it matters
Dictation latency is the time between stopping speech and seeing text. On a large host the fixed cap leaves most of that latency on the table: a 64-thread decode is about 2.6 times faster than the default on the host above, measured under load.
Not a goal
autokeeps half the cores capped at 16, because the ceiling exists for small hosts where wide pools varied badly under load.What changed (motivation → approach → change)
The count is a host property the operator can measure and the code cannot, so it becomes a setting with the current rule as its default.
stt.threadsis a new integer setting.0, or"auto"on write, keeps today's rule. A count from 1 to 1024 is used as given and clamped at model load to the cores the process may run on. An explicit count is not capped at 16: the ceiling bounds a guess on hosts nobody measured, and an operator who sets a count has measured theirs. The 1024 bound only stops a typo like6400from being stored.KIROCREW_STT_THREADSoutranks the setting for one process, the same way otherKIROCREW_*variables outrank their settings. It is read from the environment of the process that loads the model, which today is always the gateway: dictation, Slack and messaging attachments, and the meetings audio import all decode there. A value that does not parse is logged once per process and ignored, and the setting applies.LoadedKey. A change through Settings orkirocrew config settakes effect at the next dictation with one model reload, so the setting is notrestart_required. Changing the env var needs a process restart, like any env var.threads_autofield the config GET computes from the CPU affinity count without importing the engine. The hint says a change reloads the model once at the next dictation. WhenKIROCREW_STT_THREADSoverrides the setting, the picker is disabled and a note gives the count in effect first: "This gateway uses 64 threads, set by KIROCREW_STT_THREADS." It mentions the variable only when its value parses, because an unparseable value does not override anything.kirocrew config set stt.threads auto|<n>writes the setting. Validation refuses junk on write. A bad value hand-edited into config.json loads as auto with a warning.Backwards compatibility
Compatible: the new key defaults to
0, which resolves exactly as the current code does, and no existing key, validator or caller changes shape.Removes nothing: the change only adds a setting, an env var and a picker.
Tests
test/test_stt_threads.py, 48 cases: auto on small and large hosts, an explicit count, clamping to the usable cores, rejected spellings, env precedence over the setting, a bad env value falling back to the setting and logging once,LoadedKeychanging with the count, config load and validation, and the CLIautospelling.website/src/test/SttSettings.threads.test.tsx, 6 cases: the picker stays behind the Fine-tuning disclosure, offers the served choices with 0 labelled "Auto (16)" and saves a number, labels Auto alone whenthreads_autois missing, names the env override and disables the picker when the gateway reports one, stays enabled without an override, and is absent for a provider that does not run the local model.test/test_dashboard_handlers_core_coverage.py: the config GET and PUT round tripthreads, reject bad values, report the env override only when it parses, includingKIROCREW_STT_THREADS=lots, and reportthreads_autoas half the cores capped at 16 on 1, 20 and 128 cores.test/test_stt_boot_prewarm.py,test/test_stt_session.pyandtest/test_stt_endpointing.py: their fake engines gain the new method.Locally, the related backend set passed 30,145 tests in its first batch, and the i18n suite (768) and settings registry tests (394) pass. The 25 tests that failed locally fail the same way on a clean checkout of main at 38eb463. isort, flake8, mypy, tsc, semgrep, the flake ratchet and
npm run i18n:checkare clean.Manual verification
Run on an isolated dev gateway with its own data home on a 128-core host:
KIROCREW_STT_THREADS=64set: the engine loaded with 64 threads.JFK benchmark, large-v3-turbo, 11 s clip, run in-process against the real engine while the host load average was between 124 and 140:
Screenshots / video
Settings > Voice > Fine-tuning, rendered from the built SPA with fixture data for a 128-core host.
Auto, the default, labelled with the count it resolves to:
64 selected:
KIROCREW_STT_THREADSset to 64 with 16 saved, with the picker disabled and the override note ("This gateway uses 64 threads, set by KIROCREW_STT_THREADS. Remove that variable to use this choice."):Related Issues
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license.
Fixes #19647