Repository navigation
Conversation
5feabb2 to
c58fe84
Compare
|
Hi @leonlaiyc, a maintainer nudge on this one: it is carrying the Current state:
Could you rebase onto the latest git fetch upstream # or: git fetch origin, if this branch lives here
git rebase upstream/main
# resolve any conflicts, run the local gates below, then update the branch with
# a force-with-lease so the rebase lands without clobbering anyone else's workThe local gates to run before updating the branch: black src/kiro_crew test && isort src/kiro_crew test
flake8 src/kiro_crew test && mypy src/kiro_crew
python -m pytest
cd website && npm run build && npm run testA good part of this branch's redness is likely stale rather than a real defect: it predates a lot of what is now on |
Every config.json update in dashboard/handlers/memory.py reads the file, changes one key and writes it all back on the event loop. The read, the JSON parse, and write_config_atomically -- a tmp-file write plus a rename, which can fsync -- all block, stalling every other session while they run. Three sites do it: api_memory_settings, _write_embed_model_config, and _set_migrated, which runs on EVERY boot while migrated is false and so lands the stall exactly when the gateway is bringing sessions up. This is the class the repo has been closing site by site (kirodotdev#4118, kirodotdev#3803, kirodotdev#4550), and kirodotdev#4946's review named this module. The whole transaction crosses over, never just the read. Offloading the read alone would leave the write on the loop and insert a suspension point between the read and the write-back while the file is unguarded on disk: an external editor, a CLI command or another process landing in that gap would be silently overwritten by a write derived from state nobody re-checked. That gap is zero today because the sequence is synchronous, and it stays zero because the worker performs the whole thing without yielding. The existing per-config lock is held across the hop, so two coroutines still cannot interleave. Two of the three sites also hand-rolled a reader this module already imports. read_config_for_update is the documented companion to write_config_atomically with 27 call sites, and api_memory_settings uses it 200 lines above; _set_migrated and _write_embed_model_config instead caught Exception around json.loads. The helper additionally refuses a non-object top level, where the hand-rolled version accepted a list and then raised AttributeError from setdefault -- a crash where a fail-closed refusal was intended. ConfigReadError is not swallowed by the helper: what to tell the user differs per site, and each keeps exactly the behaviour it had -- skip and retry next boot, raise ValueError, or answer 500 config_unreadable. api_memory_settings now validates its body before the transaction. None of that reads the config, and a 400 previously took the lock and abandoned it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
_write_env_updates stats and reads the whole .env, re-parses it line by line, then creates a 0600 temp file, chmods it, writes and renames. All synchronous file I/O, and six async channel config-save handlers call it. Three already reached it through asyncio.to_thread -- telegram, teams, wecom -- and three called it inline on the gateway loop: slack, discord and webex, stalling every other session for the duration of a token save. So this is not a missing convention but an existing one applied to half the call sites. messaging.py already uses asyncio.to_thread 29 times, and with three siblings doing it correctly nothing in the file said which half was right, or stopped the next channel from copying the wrong one. It is the class the repo has been closing site by site (kirodotdev#4118, kirodotdev#3803, kirodotdev#4550), and kirodotdev#4946's review named this module. The WHOLE call is offloaded, never a part of it: the read-modify-write is one transaction, and a suspension point between the read and the rename would let a concurrent writer's keys be dropped by a write derived from lines nobody re-read. Keeping _write_env_updates one synchronous function on one worker preserves that without depending on the caller, which is now said on the function itself so the next channel inherits the reason and not just the shape. The regression pins all six channels rather than the three that moved, since the defect was the split and not any one site. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every config.json update in dashboard/handlers/memory.py reads the file, changes one key and writes it all back on the event loop. The read, the JSON parse, and write_config_atomically -- a tmp-file write plus a rename, which can fsync -- all block, stalling every other session while they run. Three sites do it: api_memory_settings, _write_embed_model_config, and _set_migrated, which runs on EVERY boot while migrated is false and so lands the stall exactly when the gateway is bringing sessions up. This is the class the repo has been closing site by site (kirodotdev#4118, kirodotdev#3803, kirodotdev#4550), and The whole transaction crosses over, never just the read. Offloading the read alone would leave the write on the loop and insert a suspension point between the read and the write-back while the file is unguarded on disk: an external editor, a CLI command or another process landing in that gap would be silently overwritten by a write derived from state nobody re-checked. That gap is zero today because the sequence is synchronous, and it stays zero because the worker performs the whole thing without yielding. The existing per-config lock is held across the hop, so two coroutines still cannot interleave. Two of the three sites also hand-rolled a reader this module already imports. read_config_for_update is the documented companion to write_config_atomically with 27 call sites, and api_memory_settings uses it 200 lines above; _set_migrated and _write_embed_model_config instead caught Exception around json.loads. The helper additionally refuses a non-object top level, where the hand-rolled version accepted a list and then raised AttributeError from setdefault -- a crash where a fail-closed refusal was intended. ConfigReadError is not swallowed by the helper: what to tell the user differs per site, and each keeps exactly the behaviour it had -- skip and retry next boot, raise ValueError, or answer 500 config_unreadable. api_memory_settings now validates its body before the transaction. None of that reads the config, and a 400 previously took the lock and abandoned it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every config.json update in dashboard/handlers/memory.py reads the file, changes one key and writes it all back on the event loop. The read, the JSON parse, and write_config_atomically -- a tmp-file write plus a rename, which can fsync -- all block, stalling every other session while they run. Three sites do it: api_memory_settings, _write_embed_model_config, and _set_migrated, which runs on EVERY boot while migrated is false and so lands the stall exactly when the gateway is bringing sessions up. This is the class the repo has been closing site by site (kirodotdev#4118, kirodotdev#3803, kirodotdev#4550), and The whole transaction crosses over, never just the read. Offloading the read alone would leave the write on the loop and insert a suspension point between the read and the write-back while the file is unguarded on disk: an external editor, a CLI command or another process landing in that gap would be silently overwritten by a write derived from state nobody re-checked. That gap is zero today because the sequence is synchronous, and it stays zero because the worker performs the whole thing without yielding. The existing per-config lock is held across the hop, so two coroutines still cannot interleave. Two of the three sites also hand-rolled a reader this module already imports. read_config_for_update is the documented companion to write_config_atomically with 27 call sites, and api_memory_settings uses it 200 lines above; _set_migrated and _write_embed_model_config instead caught Exception around json.loads. The helper additionally refuses a non-object top level, where the hand-rolled version accepted a list and then raised AttributeError from setdefault -- a crash where a fail-closed refusal was intended. ConfigReadError is not swallowed by the helper: what to tell the user differs per site, and each keeps exactly the behaviour it had -- skip and retry next boot, raise ValueError, or answer 500 config_unreadable. api_memory_settings now validates its body before the transaction. None of that reads the config, and a 400 previously took the lock and abandoned it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every config.json update in dashboard/handlers/memory.py reads the file, changes one key and writes it all back on the event loop. The read, the JSON parse, and write_config_atomically -- a tmp-file write plus a rename, which can fsync -- all block, stalling every other session while they run. Three sites do it: api_memory_settings, _write_embed_model_config, and _set_migrated, which runs on EVERY boot while migrated is false and so lands the stall exactly when the gateway is bringing sessions up. This is the class the repo has been closing site by site (kirodotdev#4118, kirodotdev#3803, kirodotdev#4550), and The whole transaction crosses over, never just the read. Offloading the read alone would leave the write on the loop and insert a suspension point between the read and the write-back while the file is unguarded on disk: an external editor, a CLI command or another process landing in that gap would be silently overwritten by a write derived from state nobody re-checked. That gap is zero today because the sequence is synchronous, and it stays zero because the worker performs the whole thing without yielding. The existing per-config lock is held across the hop, so two coroutines still cannot interleave. Two of the three sites also hand-rolled a reader this module already imports. read_config_for_update is the documented companion to write_config_atomically with 27 call sites, and api_memory_settings uses it 200 lines above; _set_migrated and _write_embed_model_config instead caught Exception around json.loads. The helper additionally refuses a non-object top level, where the hand-rolled version accepted a list and then raised AttributeError from setdefault -- a crash where a fail-closed refusal was intended. ConfigReadError is not swallowed by the helper: what to tell the user differs per site, and each keeps exactly the behaviour it had -- skip and retry next boot, raise ValueError, or answer 500 config_unreadable. api_memory_settings now validates its body before the transaction. None of that reads the config, and a 400 previously took the lock and abandoned it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Open PR relationship auditThis is a consolidated, point-in-time code-level audit note. It compares complete merge-base diffs and current/merged code; it does not treat a shared topic as duplication or partial coverage as completion. Relationship findings
No PR, Issue, label, branch, or review state was changed by the relationship-note portion of this audit. |
8161898 to
4a63f79
Compare
|
Rebased onto main Conflicts (1 file):
Gates run locally on changed files: black, isort, flake8 all clean; Please review the resolution. A maintainer push makes the maintainer the last pusher, so under the repo's last-push rule a second approver is needed. Reply if anything looks wrong. |
Head branch was pushed to by a user without write access
4a63f79 to
01eec1a
Compare
PR triage (2026-10-05): DISTINCT, needs a rebaseI compared the full merge-base diff (
Recommendation: REBASE onto current |
|
Thank you, @leonlaiyc, for this fix. To get it merged against current |
Problem / Motivation
Goal: Eager spawn must load its agent-binding configuration without blocking the gateway event loop.
The current speculative spawn synchronously stats, reads, merges and validates configuration before its off-loop binding resolution.
Why it matters
Slow configuration storage or validation stalls all sessions sharing the gateway loop during speculative startup.
Not a goal
Changing loader migrations, scheduler gating, binding resolution or eager-session ownership policy.
What changed (motivation → approach → change)
Integrates onto upstream main 38d5688. Offloads the single current _eager_spawn agent-binding load through asyncio.to_thread. Preserves session_key resolution, captured bindings, app self-heal, explicit Crew Member handling, memory assignment, deferred resets, V2 refusal, model resolution and pre-allocation ownership guards.
The scheduler uses live.snapshot() and no longer performs a preceding synchronous load. This change therefore does not rely on migrations having already run. The loader still writes migrations: #7937 is merged as 752d08a and is an ancestor of this base. Current _persist_config_migration re-reads the document and applies only migration-owned deltas, using update_config_locked for contained paths and deferring a contended lock. It no longer serializes the stale whole cfg snapshot through cfg.save(). Redirected foreign configs retain their existing immediate-fresh-read delta behavior without creating a lock sidecar. Existing loader global publication semantics are unchanged, including ticket-ordered autocompact publication.
A real first eager worker load is held immediately before migration persistence; a concurrent locked setting update lands after its original read. The test confirms both the unrelated setting and migration survive. The source comment records only the current delta-write invariant.
Backwards compatibility
Compatible: the same configuration snapshot feeds the same binding and ownership flow.
Removes nothing: this changes the execution thread of one load without narrowing accepted configuration or sessions.
Tests
Measured on Python 3.13 on Windows. Fresh CI supplies broader cross-platform and repository gate results; no whole-tree clean claim is made.
Manual verification
N/A — deterministic bounded worker handshakes exercise thread identity, stale-work refusal and a real migration/config-writer interleave.
Related Issues
Closes #4117
Retains the original eager config offload intent while incorporating the merged config write-back ordering fix.
Pattern harvest
Rule candidate: review-prompt
Pattern: offloading a configuration read requires checking its migration writes; fresh-document deltas and writer-lock ordering must preserve concurrent settings.
Checklist
Contribution License Agreement