Repository navigation
fix(dashboard): serialize the auto-update toggle against both config writers - #7167
Conversation
|
The failure is Fixed separately in #7176 rather than folded in here. No SHA churn on this branch for it. |
|
Full CI triage on this head — three distinct failures, none attributable to the diff. This PR touches only
No commit, no rebase, and no empty push to reroll — none of these is a signal about this branch. |
|
🤖 Kiro Crew [operator: bolichen97#66809557]: This PR has been inactive for 3+ days with failing CI. I've assessed the blockers and they appear resolvable — I'll push fixes directly to this branch as a co-author. Assessment: The failing Backend Tests are not in this PR's diff. Shard-1 (3.10 & 3.12) fails on If you'd prefer I don't touch this PR, add the |
bcbaecb to
586e55d
Compare
|
🤖 Kiro Crew [operator: bolichen97#66809557]: Rebased this branch onto current main and force-pushed (
Local gates before push, all green: |
Design Review (Fable 5, fork) — ✅ PASSDesign-level review of Design-Verdict: PASS Legacy writers still hold only the loop-side lock against the CLI/second gateway — confirm that mirror-image gap is tracked as a follow-up. [DESIGN-REVIEWED] 156e2b0 |
Opus 4.8 Review (fork) — ✅ no blocking findingsReviewed |
First Principles Review (Fable 5, fork) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS One residual, declared: legacy handlers holding only the asyncio lock still race the CLI/second-gateway flock — the description scopes that out explicitly, nothing to hold here. I verified the sibling claim myself: grepping What this change shipsInventory (4 items) — 4 justifiedIntent: stop a routine dashboard settings save from silently reverting the auto-update toggle (or vice versa) — a FIX.
[FIRST-PRINCIPLES-REVIEWED] 156e2b0 |
GPT 5.6 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
|
Rebased onto main Clean rebase: no conflicts. Your Gates run locally on the changed files only: Please review the rebased result. A maintainer push makes the maintainer the last pusher, so under this repo's last-push rule a second approver is needed. Reply here if anything looks wrong. |
…writers `POST /api/update/auto` persisted the flag with a bare `asyncio.to_thread(update_config_locked, ...)`. That holds only the sidecar advisory flock, which excludes the CLI and a second gateway but not the legacy dashboard writers -- core.py's theme/settings PUT, the agents endpoint, security.py, messaging.py, mcp.py, computer_use.py -- which read-modify-write the same config.json holding only the loop-side `_get_config_lock`. So a theme save landing between this endpoint's read and its write commits from a snapshot taken before it and silently reverts the auto-update flag the user just toggled, or this write reverts their theme. Nothing errors; the response is built from `enabled`, not from a re-read of what landed. Route it through `run_config_write`, the one entry point that holds both generations. It takes the loop-side lock on the event loop and runs the blocking writer in a worker, so the flock wait still never stalls the loop. Nothing here holds a config lock already, so no nesting is introduced, and the helper propagates the writer's exceptions unchanged so the fail-closed 500 arm is untouched. Sibling of kirodotdev#6984, which made the same correction in apps/routes.py. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Rebased onto main by Kiro Crew (conflict in test_config_rmw_preserves_settings.py resolved by keeping both appended test classes); original work by Leon (leonlaiyc). Co-authored-by: Kiro Crew <noreply@kirodotdev.github.io>
3fbbd88 to
156e2b0
Compare
|
Rebased onto current Conflicts resolved (two, not one)1. 2. One formatting fix on top of the rebaseThe repo's black gate ( Tests and gates runAgainst the pushed head, using the repo's own config:
The 1 skip is Anti-revert verificationRebases of this kind can silently drop upstream work while the file count stays the same, so both checks were run explicitly:
CI has not been polled from here; the conductor is watching the checks on |
|
|
iamwhatever
left a comment
There was a problem hiding this comment.
Tier 1 auto-approve: fix (2 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, SAST annotations clean (the single Semgrep annotation on this head is GitHub's own ubuntu-latest runner-migration notice on .github, annotation_level=notice, not a finding), security checklist all-NO, AI reviewers green. Category: the auto-update toggle held only the sidecar advisory flock, which does not exclude the legacy dashboard writers that read-modify-write the same config.json under the loop-side lock, so a concurrent theme or settings save could silently revert the toggled flag -- swapping the bare asyncio.to_thread for run_config_write holds both locks while still keeping the blocking wait off the event loop, and the fail-closed unreadable-config contract is unchanged. CodeQL is not applicable on this fork PR (default-setup emits no check-run); SAST coverage is Semgrep only, latest run success with 0 real annotations.
bolichen97
left a comment
There was a problem hiding this comment.
Approved after a live re-verification: readiness: passed label, PR Readiness status success on this head, no required check pending/failed/cancelled, mergeable, no effective CHANGES_REQUESTED, and no unresolved human blocking request found in comments or review threads. Native auto-merge (squash) enabled; no admin override used.
…writers (kirodotdev#7167) `POST /api/update/auto` persisted the flag with a bare `asyncio.to_thread(update_config_locked, ...)`. That holds only the sidecar advisory flock, which excludes the CLI and a second gateway but not the legacy dashboard writers -- core.py's theme/settings PUT, the agents endpoint, security.py, messaging.py, mcp.py, computer_use.py -- which read-modify-write the same config.json holding only the loop-side `_get_config_lock`. So a theme save landing between this endpoint's read and its write commits from a snapshot taken before it and silently reverts the auto-update flag the user just toggled, or this write reverts their theme. Nothing errors; the response is built from `enabled`, not from a re-read of what landed. Route it through `run_config_write`, the one entry point that holds both generations. It takes the loop-side lock on the event loop and runs the blocking writer in a worker, so the flock wait still never stalls the loop. Nothing here holds a config lock already, so no nesting is introduced, and the helper propagates the writer's exceptions unchanged so the fail-closed 500 arm is untouched. Sibling of kirodotdev#6984, which made the same correction in apps/routes.py. Rebased onto main by Kiro Crew (conflict in test_config_rmw_preserves_settings.py resolved by keeping both appended test classes); original work by Leon (leonlaiyc). Co-authored-by: Kiro Crew <noreply@kirodotdev.github.io>
…writers (kirodotdev#7167) `POST /api/update/auto` persisted the flag with a bare `asyncio.to_thread(update_config_locked, ...)`. That holds only the sidecar advisory flock, which excludes the CLI and a second gateway but not the legacy dashboard writers -- core.py's theme/settings PUT, the agents endpoint, security.py, messaging.py, mcp.py, computer_use.py -- which read-modify-write the same config.json holding only the loop-side `_get_config_lock`. So a theme save landing between this endpoint's read and its write commits from a snapshot taken before it and silently reverts the auto-update flag the user just toggled, or this write reverts their theme. Nothing errors; the response is built from `enabled`, not from a re-read of what landed. Route it through `run_config_write`, the one entry point that holds both generations. It takes the loop-side lock on the event loop and runs the blocking writer in a worker, so the flock wait still never stalls the loop. Nothing here holds a config lock already, so no nesting is introduced, and the helper propagates the writer's exceptions unchanged so the fail-closed 500 arm is untouched. Sibling of kirodotdev#6984, which made the same correction in apps/routes.py. Rebased onto main by Kiro Crew (conflict in test_config_rmw_preserves_settings.py resolved by keeping both appended test classes); original work by Leon (leonlaiyc). Co-authored-by: Kiro Crew <noreply@kirodotdev.github.io>
Problem / Motivation
POST /api/update/auto— the auto-update toggle in dashboard settings — persists the flag with a bare offload:That is correct about the event loop and wrong about exclusion.
config.jsonhas two writer generations that do not exclude each other:update_config_lockedtakes the sidecar advisory flock — covering the CLI, the boot refresh, and a second gateway process.core.py's theme/settings PUT, the agents endpoint,security.py,messaging.py,mcp.py,computer_use.py— do their own read-modify-write of the same file while holding only the loop-side_get_config_lockasyncio lock.Holding just the flock excludes nothing that second family respects.
core.py:424is the plainest counterpart:async with _get_config_lock():→KiroCrewConfig.load→ mutate → save, with no flock anywhere. A theme save landing between this endpoint's read and its write commits from a snapshot taken before it, and silently reverts the auto-update flag the user just toggled — or this write reverts their theme.config/loader.pystates the rule directly: a caller running while the dashboard serves requests must also hold the in-process asyncio lock.Why it matters
The lost update is silent and bidirectional, and it is reachable by ordinary use rather than by a race a user has to provoke — toggling auto-update and saving a setting are both routine dashboard actions. Nothing errors and nothing is logged; the endpoint reports success in both directions because the response is built from
enabledrather than from a re-read of what landed on disk. The write window is not narrow either: it includes a filesystem lock wait plus, on Windows, an owner-only DACL application that can cost an unbounded SMB round-trip on a network-homed data home.Auto-update is also the wrong setting to lose silently. A user who turned it off and finds it back on has an install that will update itself against their explicit decision.
What changed (motivation → approach → change)
Symptom → a config write that can be silently reverted, in either direction. Root cause → the dispatch holds one of the two locks that guard
config.json. Change → route it through the entry point that holds both.src/kiro_crew/dashboard/handlers/updates.py:run_config_write(dashboard/chat_utils.py) acquires_get_config_lockon the event loop, then runs the blocking writer in a worker — so the flock wait still never stalls the loop, which is the property the bareto_threadwas there for, unchanged. It also shields and drains that worker across cancellation, so a client disconnecting mid-toggle cannot unwind the lock while the write is still in flight.Three properties checked before making the swap rather than assumed, because a canonical-helper swap on a persistence path is not mechanical:
config_path()here is the mainconfig.jsonthe legacy family mutates, not a sidecar.api_update_autois a flat coroutine with noasync withof its own, sorun_config_writeis its only lock acquisition and this introduces no nesting. (This is what distinguishes the site fromapps/routes.py, where the same swap in fix(apps): serialize the builtin config sync against both config writers #6984 sat insideapp_lifecycle_lockand needed an explicit lock-ordering argument. There is no second lock here to order against.)run_config_writeawaits the worker throughasyncio.shieldand re-raises, soConfigReadErrorstill reaches the handler's 500 arm and an unreadable config is still never overwritten with a one-key file. Pinned by a test rather than asserted.The import is module-level, unlike #6984's call-time one:
updates.pyalready lives insidekiro_crew.dashboard, so there is no layering inversion to avoid and no import cycle (verified by importing both modules in either order).The comment block above the call is rewritten. The old one explained why the flock replaced
_get_config_lockand read as though that were the end of the story; it now says why both are needed.Scope. This is the one
update_config_lockedsite inupdates.py. It is the sibling the First Principles review on #6984 counted as the remaining same-class one-lock config writer, and #6984 (merged) namesupdates.pyin its own problem statement. No other handler is touched.Tests
New in
test/test_config_rmw_preserves_settings.py, classTestAutoUpdateToggleHoldsBothConfigLocks— the file that already owns this endpoint's config-write contract:test_the_loop_side_lock_is_held_across_the_write— the defect, pinned behaviourally. Spiesupdate_config_lockedand, from inside the worker, records_get_config_lock().locked()and the thread identity. Probing from inside is what makes this a test of the property rather than of the spelling of the dispatch. It also asserts the write is still off the loop, so the fix cannot pass by moving the blocking call back onto it.test_the_loop_side_lock_is_released_afterwards— holding it is only correct if the handler gives it back; also asserts the toggle actually landed on disk.test_an_unreadable_config_still_fails_closed_and_releases— the 500 arm survives the new dispatch, the torn file is byte-identical afterwards, and the lock is not stranded on the exception path.test_the_dispatch_cannot_regress_to_a_one_lock_offload— an AST ratchet (not a substring search, so a reformat cannot defeat it) that names the offendingupdates.py:<line>ifupdate_config_lockedis ever dispatched with a bareasyncio.to_threadagain.test_the_ratchet_can_actually_fail— a scan that matches nothing passes vacuously; this plants a violating source and asserts the scan finds exactly one.Red-before, production change reverted with the tests in place: 2 failed, 3 passed.
Green after:
test_config_rmw_preserves_settings.py+test_dashboard_updates_coverage.py— 99 passed, 3 skipped, 0 failed (Python 3.10.6, Windows).Gates green: flake8, isort,
scripts/check_black_formatting.py,mypyon the changed handler (no issues in it), andscripts/check_loop_bound_locks.pyincluding its own--testself-check (21/21 probes).Manual verification
N/A — unit coverage sufficient: the defect is which lock is held around a write, and the test observes that lock's state from inside the worker doing the write, which is the exact moment a manual toggle could not show. Reproducing it by hand would require winning a millisecond-scale interleaving between two dashboard requests.
Related Issues
Sibling of #6984 (merged), which made the same correction at the two
apps/routes.pycall sites and whose First Principles review counted this one as the remaining member of the class.Checklist
feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Pattern harvest
Rule candidate: semgrep / AST ratchet — when one file has two writer families guarded by different locks (here
config.json: a sidecar advisory flock and the loop-side_get_config_lock), holding only one of them excludes nothing the other family respects, so every writer must go through the entry point that takes both; a bareasyncio.to_thread(update_config_locked, ...)is the detectable shape of the defect, and the resulting lost update is silent and bidirectional.Section appended by a maintainer to satisfy the
PR Hygienegate, which began requiring a Pattern harvest section onfix:PRs after this PR was opened. No other part of this description, and nothing on the branch, was touched.