Skip to content

fix(memory): honor configured episodic store options - #12856

Merged
bolichen97 merged 1 commit into
kirodotdev:mainfrom
jeeshofone:fix/10889-named-store-dedup-threshold
Sep 28, 2026
Merged

bolichen97 merged 1 commit into
kirodotdev:mainfrom
jeeshofone:fix/10889-named-store-dedup-threshold

Conversation

@jeeshofone

@jeeshofone jeeshofone commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Every memory store honors the configured episodic dedup threshold and cap, at boot and after a live reload.

Memory stores can ignore configured episodic settings. Named stores miss memory.episodic_dedup_threshold, and several production constructors miss memory.episodic_max_count, so they run on the built-in defaults.

Why it matters

A store can deduplicate or evict episodic memories with defaults that disagree with the operator's configuration. Boot behavior can then differ from live reconfigure() behavior for the same store.

Not a goal

  • Changing any default episodic setting.
  • Plumbing config into onboarding import's foreign data_home store, the eval/ harnesses or the read-only injection audit, which keep constructor defaults.
  • Recovering a duplicate refusal that could succeed after its colliding row is tombstoned.

What changed (motivation → approach → change)

Every store built from the install's config takes its memory.* tuning from VectorMemoryStore.reconfigure(cfg), the method the live reload calls. In src/kiro_crew/vector_memory.py the constructor takes config= and applies it BEFORE subscribing to the live watcher, so a reload landing during construction is not overwritten by the caller's older copy. The store then calls ConfigWatch.replay(sub), added in src/kiro_crew/config/live.py, on its new subscription, only when it was built with config= (a config=None store, such as onboarding import's foreign data_home, keeps its constructor defaults at construction): a reload adopts its config before it snapshots the registry, so one dispatched between the caller's load and the registration never reached the store. replay hands the new subscription the adopted snapshot when the watcher's fingerprint matches the file, re-reading it after each apply, and otherwise marks the subscription stale so the next tick delivers the file.

Named stores in src/kiro_crew/context.py (_build_store_vectors, V1 and V2 via open_member_database) pass config=cfg, replacing a hand-copied options dict that dropped episodic_dedup_threshold and episodic_max_count. So do src/kiro_crew/slack/gateway.py, src/kiro_crew/cli_server.py, src/kiro_crew/dashboard/handlers/memory.py, the lesson, carve and import/migrate stores in src/kiro_crew/cli_commands.py, and the ops-mission-control routes.py and dispatch.py, several of which dropped some or all of the memory tunables.

Honoring a lower cap exposed two existing hazards, fixed here:

  • ledger_index.import_pending cursored every refused write as a duplicate, so entries a full V1 store refused were never indexed after the cap was raised. write_episodic_outcome() returns WRITTEN/REFUSED/AT_CAPACITY (write_episodic is its bool wrapper), decided inside the transaction that refused the row. On AT_CAPACITY the indexer stops, logs how many entries stay pending, and retries them next import. REFUSED is cursored as final; only the text refusals are final in fact, and a duplicate refusal that could succeed after its colliding row is tombstoned is an accepted loss.
  • _build_memory_config (in src/kiro_crew/config/section_builders.py since refactor(config): give each config domain a named owner module #14558, re-exported by config/loader.py) accepted memory.episodic_max_count: 0; it now floors the value at 1.

onboarding_import, the eval/ harnesses and the read-only injection audit keep constructor defaults, each with a stated reason. An inventory test fails on a new construction site that is neither plumbed nor exempted, and a structural guard fails on a site that omits config=, passes a literal, or sets a tunable on the constructor. The memory, config and ops-mission-control system specs describe the change.

Backwards compatibility

Compatible: defaults stay unchanged, and existing supported configuration values now reach every applicable constructor. A configured cap of 0 or below now loads as 1; the old value evicted every V1 episode on the next write. The ledger-index result keys are unchanged.

Tests

The named-store regression writes the dedup threshold, cap, confidence threshold and result limit, constructs the store through _build_store_vectors, and checks the live settings. The dashboard regression constructs a real throwaway store and checks the configured dedup threshold, cap, confidence threshold, result limit and semantic keys. The structural guard verifies every plumbed construction call passes config=, proves it catches a missing config= (including a construction followed by a late reconfigure), a literal config=None and a constructor tunable, and the inventory test covers every construction site in the package. test_the_gateway_passes_the_configured_count_into_the_store now pins the gateway's config=self._cfg. test_a_reload_during_construction_is_not_overwritten delivers a watcher reload (cap 900) at the moment the store subscribes and checks the caller's older config (cap 10) does not overwrite it. test_a_reload_dispatched_before_the_store_subscribed_is_replayed primes the watcher with an adopted cap of 900 and builds the store from a cap-10 config: the store ends at 900. test_a_store_built_without_a_config_is_not_replayed primes the same cap-900 watcher and builds a config=None store with episodic_max=77: it stays at 77. TestReplayOnRegistration in test/test_config_live.py covers a reload that missed the registration (a control subscriber that is not replayed keeps the old value), a snapshot behind the file (not applied, delivered on the next tick even when that tick changes only a field outside the prefix), a reload landing mid-replay, and the no-op on an unstarted watcher.

TestCapacityRefusal in test_ledger_index.py covers a fake full store and a real V1 store at its cap: refused entries stay pending and are written after the cap is raised, while duplicates are still cursored. test_capacity_verdict_survives_a_cap_raise_racing_the_refusal raises the cap on a real V1 store the instant a write is refused (what reconfigure does from the watcher thread) and checks the entry is still retried. test_write_outcome_names_the_refusal_cause pins each outcome, and test_episodic_max_count_is_floored_at_one pins the loader floor. write_episodic_outcome is registered as a write primitive in test_persistence_writer_inventory.py, and the generation-check guard in test_embedding_model_apply.py now reads the method that holds the write body.

On this head d5bcf912b (rebased on 64b0bdeda; #14558 moved _build_memory_config from config/loader.py to config/section_builders.py, so the floor hunk moved with it and range-diff shows no other change), the touched test modules, test_config_module_boundaries.py and the embeddings/slack-thread suites pass (1930 passed, 14 skipped), the 129 test files that import slack.gateway or dashboard.handlers.memory pass (7097 passed, 27 skipped), and mypy src/kiro_crew/ is clean. On 96f89e1f6 (same diff on 174614135) the 560 test files that import config.loader or section_builders ran 41975 passed, 3 failed: two fail identically on origin/main on this host (test_home_pin_survives_monkeypatch_undo.py::test_the_guard_accepts_a_scratch_home, test_auto_improvement_members.py::test_assignments_build_real_context_from_only_their_private_store; host environment), and test_remote_crew_chaining.py::TestLentHopReservation::test_a_torn_down_lent_hop_is_owned_not_merely_skipped failed once under -n 24 and its file passed 3/3 in isolation (163 passed). Earlier heads: on d057475d6 the touched modules plus the watcher suites and test_dashboard_origin.py passed (1285 passed, 14 skipped). TestReplayOnRegistration::test_a_degraded_section_is_deferred_on_replay_not_logged_as_a_failure pins replay's ConfigDeferred arm: with the arm deleted it fails ('failed on replay' is contained). On the previous base 31c291de1, scripts/local-gate.py --base origin/main (backend, related): 25313 passed, 7 failed that fail identically on origin/main on the same host (aws_control/.../test_producer.py x3, code_review_sage/.../test_store_ancestor_chain.py::...test_a_link_planted_inside_the_window_is_followed_outside_every_root, test_agent_import_hoist.py x2, test_auto_improvement_members.py::test_assignments_build_real_context_from_only_their_private_store; host environment). black --check (changed files clean or baselined in .github/black-baseline.txt), isort, flake8, the fast-gate scripts/check_*.py guards, verify_vendor_manifest.py and docs_lint.py pass.

Manual verification

N/A — unit coverage drives the real configuration and constructor paths. Against the previous source the new regressions fail at pytest call phase: the capacity tests (refused entries cursored as duplicates and never retried; on eb3d37748 the cap-raise race test fails with 1 != 0 : a capacity refusal is not a duplicate), the loader floor (0 != 1) and the structural guard (three unplumbed cli_commands.py sites). On 0c102253d the reconfigure guard fails (stores built without reconfigure(cfg)), the dashboard regression fails on the confidence threshold (0.8 == 0.61); on 111a2fe71 the guard fails with context.py:line 454/461: VectorMemoryStore(...) is never reconfigured; on 9363051bb test_capacity_pause_is_logged fails with no warning logged; and on 0159afca8 the construct-then-reconfigure order leaves the cap at 10 after a watcher reload to 900 lands during construction (reproduced with that head's own call sequence); on 0e04f488e the replay regression fails with assert 10 == 900; on 58eb5b699 the config=None regression fails with assert 900 == 77.

Related Issues

Fixes #10889

Pattern harvest

Rule candidate: review-prompt
Pattern: Memory-store construction must pass every live reconfigure() tuning field through each applicable constructor, with behavioral and structural parity coverage.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@jeeshofone
jeeshofone requested review from a team and patrigao September 22, 2026 16:51
@github-actions github-actions Bot added the fork Pull request from a fork (external contributor) label Sep 22, 2026
@jeeshofone

Copy link
Copy Markdown
Contributor Author

Maintainers: please approve the fork workflows for this PR when convenient. The branch is locally green, including both extracted GPT 5.6 and Opus 4.8 review contracts with no findings.

@jeeshofone

Copy link
Copy Markdown
Contributor Author

Intent: Named memory stores must honor the configured episodic deduplication threshold, matching the other memory construction paths.
Not a goal: Change the default threshold or the deduplication algorithm.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 22, 2026
@github-actions

github-actions Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review (fork) — ✅ no blocking findings

Reviewed d5bcf912b6d4e984057861624a35c0ee1811fdca via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] d5bcf91

@github-actions

github-actions Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5 Review (fork) — ✅ no blocking findings

Reviewed d5bcf912b6d4e984057861624a35c0ee1811fdca via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] d5bcf91

@github-actions

github-actions Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5, fork) — 🟡 CONCERNS

Design-level review of d5bcf912b6d4e984057861624a35c0ee1811fdca via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

Ledger indexing pauses permanently once the episodic store sits at its cap — eviction never frees a slot, so retry-forever is the steady state on mature installs.

Watch

The PR's own doc states the trap: "cap eviction by other writers always leaves the count AT the cap, so it never frees a slot for the importer." Once any install accrues episodic_max_count episodes (default 10,000, and now honored at smaller configured caps), every import_pending cycle scans, refuses one merge-only write, warns, and breaks — forever, since the named remedy (raise the cap) is a treadmill and the only organic escape is an operator delete or stale retirement. This strictly improves the prior behavior (silent permanent loss, cursored as duplicates), but the chosen shape — unbounded pending backlog plus a warning per cycle with no importer escape — is a quiet feature death on exactly the installs that use memory most.
Clears when: a recorded decision or follow-up issue on the importer's steady state at cap (e.g. the importer may evict its own SOURCE_TAG rows, or the pause is explicitly accepted with a rate-limited warning).

Suggestions

  • ConfigWatch.replay is generic but wired to one of the six owners with the identical registration gap (config.md now names the other five as out of scope); file the follow-up so the doc's "keep the registration gap" line has an owner rather than becoming permanent.

[DESIGN-REVIEWED] d5bcf91

@github-actions

github-actions Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5, fork) — ✅ PASS

Premise-level review of d5bcf912b6d4e984057861624a35c0ee1811fdca via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: PASS

Verify the operational trade: at a configured low cap, ledger indexing now pauses indefinitely, signalled only by a warning log until the operator raises the cap.

What this change ships

Inventory (9 items) — 9 justified

Intent: FIX — every config-built memory store honors the configured episodic dedup threshold and cap, at boot and after a live reload. Defect verified on base: cli_server.py:2575 and dashboard/handlers/memory.py:695 pass only dedup_threshold, context.py passes neither key; reconfigure (vector_memory.py:1525) covers all six tunables, so config= is a strict superset of every deleted keyword list. The _stale next-tick machinery replay reuses exists on base (live.py:351–814), and the adopt-before-dispatch ordering it rests on is pinned at live.py:700. write_episodic keeps 9 production callers, so the bool wrapper stands.

  1. Named, CLI, dashboard, Slack and ops-mission-control stores now honor configured episodic settings — justified
  2. Stores take the loaded config through the same path the live reload uses — justified
  3. A reload dispatched before a store subscribes still reaches it — justified
  4. Episodic writes report why they were refused (written / refused / at-capacity) — justified
  5. Ledger import pauses at a full store and retries later, instead of dropping entries forever — justified
  6. The pause is logged with the pending-entry count — justified
  7. A configured episodic cap of 0 or below now loads as 1 — justified
  8. A new store construction site must pass config= or be exempted (guard) — justified
  9. Config, memory and ops-mission-control specs updated in the same commit — justified

The five other watch_object owners keeping the registration gap are counted and deferred by name in the config spec — accepted-and-deferred, not a demand.

[FIRST-PRINCIPLES-REVIEWED] d5bcf91

@jeeshofone jeeshofone changed the title fix(memory): honor named-store dedup threshold fix(memory): honor configured episodic store options Sep 22, 2026
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 22, 2026
@jeeshofone
jeeshofone force-pushed the fix/10889-named-store-dedup-threshold branch from 7fe8a57 to 46ead19 Compare September 22, 2026 17:41
@jeeshofone

Copy link
Copy Markdown
Contributor Author

fixed
Named-store, CLI, and dashboard constructors now pass memory.episodic_max_count; Slack already did.
Behavioral checks cover named/dashboard stores, and the structural guard covers CLI/dashboard/Slack call sites for both episodic options.
Fixed in 46ead196eb84dfbaa617d2d75953d2d6afdc3f49.

@jeeshofone

Copy link
Copy Markdown
Contributor Author

fixed
context._build_store_vectors now passes episodic_max=mem.episodic_max_count through the shared V1/V2 options.
The real named-store regression asserts the non-default live value 321.
Fixed in 46ead196eb84dfbaa617d2d75953d2d6afdc3f49.

@jeeshofone

Copy link
Copy Markdown
Contributor Author

rebutted
A shared production helper is not required to close this defect and would widen a bounded constructor-plumbing fix.
The strengthened AST guard checks both configured options at every direct production call site and its seeded meta-test proves missing or hard-coded values are detected.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 22, 2026
@jeeshofone
jeeshofone force-pushed the fix/10889-named-store-dedup-threshold branch from 46ead19 to c9c64e5 Compare September 22, 2026 18:21
@jeeshofone

Copy link
Copy Markdown
Contributor Author

Current head c9c64e5a27a02597ccea3246965dffa61f2f899c is locally green. Three setup workflows failed only because the repository GitHub App installation hit its API rate limit while publishing their already-computed results:

  • Fork PR Description: computed success, then check-run API returned rate-limit 403 (run)
  • PR Readiness: failed resolving the PR after three rate-limited API attempts (run)
  • Fork Workflow-Change Guard: failed removing/checking an absent override label due the same installation limit (run)

My failed-job-only rerun request was refused because contributors lack repository admin rights. Maintainers: please rerun these failed jobs after the installation rate limit resets. No new head is needed.

@jeeshofone
jeeshofone force-pushed the fix/10889-named-store-dedup-threshold branch from c9c64e5 to 45b14ae Compare September 25, 2026 00:07
@jeeshofone

Copy link
Copy Markdown
Contributor Author

Rebased onto current main and pushed a fresh head: 45b14aed3870bb26730662070263d9167a99f909 (was c9c64e5a2).

Blocking item — resolved by new head. The PR Readiness failure on the prior head was not a code finding. The failing lanes on c9c64e5a2 were infrastructure, not verdicts:

  • GPT 5.6 Review, Opus 4.8 Review, Internal Content Scan — all reported "could not run … failed before producing a verdict; re-run it".
  • Security Scope Review — [scope-floor:unsettled] (no leg reported; adjudication did not complete).

The prior head was also 328 commits behind main, so those setup/review workflows had gone stale. The last lanes that actually produced verdicts (46ead196e) were all PASS / no-blocking, and the two Design/First-Principles CONCERNS spans were already dispositioned (fixed / rebutted-and-accepted).

Rebase is a no-op on behaviour. One commit, kept as one; git range-diff shows only the 328 intervening main commits ahead of an otherwise-identical patch. main moved the three touched production files (cli_server.py, context.py, dashboard/handlers/memory.py); the two episodic_max / dedup_threshold insertions re-applied cleanly at the correct constructor call sites (surrounding episodic_limit / embedding_dim context intact).

Verified on 45b14aed3:

  • VectorMemoryStore.__init__ accepts dedup_threshold and episodic_max (vector_memory.py:1391-1392); both are consumed in the dedup (4033), eviction (4088/5226) and search (4368) paths — the fix is real.
  • Targeted pytest: test/test_context_store_admission_order.py + test/test_vector_memory.py → 231 passed, 14 skipped.
  • Gates on changed files: black --check clean, isort --check-only clean, flake8 clean, mypy clean (3 source files, no issues).

Independent-reviewer spawn (contract step 6, gpt-5.6-sol) was refused by host governance (capability disabled), so it was not run; constructor-signature and call-site sweep were verified directly instead.

Remaining work is maintainer-side only: this fork PR's re-triggered workflows need maintainer approval to execute, after which PR Readiness should recompute green.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 25, 2026
@github-actions github-actions Bot added the readiness: action required A blocking check or review needs attention label Sep 25, 2026
@jeeshofone
jeeshofone force-pushed the fix/10889-named-store-dedup-threshold branch from 0159afc to 0e04f48 Compare September 25, 2026 22:46
@jeeshofone

Copy link
Copy Markdown
Contributor Author

span=dc2d7de999a7
mechanism: construction-time config= applied before the live-watcher subscription; per-call structural guard

fixed in 0e04f488e6c5 (rebased on 3f16157ba)
The race is real on 0159afca8: each site built the store, which subscribed to the live watcher inside __init__, and only then called store.reconfigure(cfg) with the caller's earlier snapshot, so a reload delivered in between was put back. Reproduced with that head's own call order: a watcher delivery of cap 900 at subscribe time, followed by the site's reconfigure(cap 10), leaves _episodic_max == 10.
VectorMemoryStore.__init__ now takes config= and applies it through reconfigure BEFORE live.watch_object(...) registers the subscription (src/kiro_crew/vector_memory.py). Every plumbed site passes config=cfg and no longer calls reconfigure after construction: both branches of context._build_store_vectors (V2 via open_member_database(..., config=cfg), which forwards to the constructor, and V1), the gateway, cli_server.py, cli_commands.py (lesson, carve, import/migrate), the dashboard fallback, and the ops-mission-control routes.py/dispatch.py sites. The class-level fix therefore covers all sibling sites, not just context.py:458.
Regression: test_services_config_hot_reload.py::TestEpisodicMaxCountIsConsumed::test_a_reload_during_construction_is_not_overwritten delivers a cap-900 reload at the moment the store subscribes and asserts the constructor's cap-10 config does not overwrite it; it fails on 0159afca8. The structural guard in test_vector_memory.py now checks each construction call for config= (a literal config=None is flagged), and its seeded case pins that a construct-then-reconfigure sequence is still reported, because that late call is the race.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 25, 2026
@jeeshofone
jeeshofone force-pushed the fix/10889-named-store-dedup-threshold branch from 0e04f48 to a751ac0 Compare September 26, 2026 00:14
@jeeshofone

Copy link
Copy Markdown
Contributor Author

span=3f8cfc777e5f
mechanism: ConfigWatch.replay(sub) via watch_object(..., replay=True), run right after registration

fixed in a751ac0db (rebased on 7f1f4fb1d)
Upheld: _cycle adopts its config (self._cfg = cfg) and only then _dispatch snapshots the registry under _subs_lock, so a reload that snapshotted before the store appended its subscription never reaches it, and a later reload that leaves memory.* unchanged does not correct it.
The handoff is now in src/kiro_crew/config/live.py: watch_object(..., replay=True) registers and then calls ConfigWatch.replay(sub). Because every reload that can have missed the subscription adopted its config before reading the registry, the snapshot read after registration is at least as new as any of them, and every later reload includes the subscription. When the watcher's fingerprint matches the file, the snapshot is applied through the subscription's own applier (degraded sections still defer), and re-read after each apply so a reload landing mid-replay is not undone by the older one. When the file has moved past the snapshot, or nothing is fingerprinted yet, the snapshot may be OLDER than the caller's own load, so it is not applied; the subscription is marked stale for its prefixes and the next tick delivers the file even where that tick leaves them unchanged. VectorMemoryStore.__init__ registers with replay=True.
Regressions: test_services_config_hot_reload.py::TestEpisodicMaxCountIsConsumed::test_a_reload_dispatched_before_the_store_subscribed_is_replayed (watcher adopted cap 900, store built from cap 10) fails on 0e04f488e with assert 10 == 900. test_config_live.py::TestReplayOnRegistration covers the missed reload (with a no-replay control that keeps the old value), a snapshot behind the file, a reload landing mid-replay, and the unstarted-watcher no-op.

@jeeshofone

Copy link
Copy Markdown
Contributor Author

span=04181484b0a3

fixed in a751ac0db
Upheld: the prefix/exact dedup query selects is_deleted = 0, so once the colliding row is tombstoned the same write can succeed. The REFUSED comment in src/kiro_crew/vector_memory.py and the matching memory-skills-hooks.md sentence now say only the text refusals are final in fact, and that the ledger indexer cursoring a duplicate or similarity refusal accepts losing that later retry. Behaviour is unchanged; this was a prose contract.

@jeeshofone

Copy link
Copy Markdown
Contributor Author

span=c35f877c0707

fixed in a751ac0db
Upheld: result["scanned"] = len(entries) counts every entry before any write. The import_pending docstring now says capacity-refused entries count in scanned but in none of written, skipped or embedded.

@jeeshofone
jeeshofone force-pushed the fix/10889-named-store-dedup-threshold branch from a751ac0 to f97d117 Compare September 26, 2026 01:40
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 26, 2026
@jeeshofone
jeeshofone force-pushed the fix/10889-named-store-dedup-threshold branch from f97d117 to 58eb5b6 Compare September 26, 2026 05:46
@jeeshofone

Copy link
Copy Markdown
Contributor Author

span=6afa6d02c389

fixed in 99a514a66
Subtraction made: the replay= kwarg is gone from ConfigWatch.watch_object and the module-level live.watch_object. The one consumer, VectorMemoryStore, calls live.watch().replay(sub) on the subscription watch_object returned. TestReplayOnRegistration (test/test_config_live.py) now calls w.replay(w.watch_object(...)). The config spec records the five sibling watch_object owners (cron_history.py, subagent.py, history_consolidation.py, adaptive/controller.py, slack/gateway.py) as keeping the registration gap, out of scope for #10889.

@jeeshofone

Copy link
Copy Markdown
Contributor Author

span=708455be55bb

fixed in 99a514a66
Upheld in the form the Watch asked for: the exemption is now true by construction in the constructor (config is None means no apply and no replay), so no per-site flag is needed at the exempted call sites. test_every_construction_site_is_plumbed_or_exempted still fails on a new construction site in neither list.

@jeeshofone

Copy link
Copy Markdown
Contributor Author

span=d2d549431dc0

fixed in 99a514a66
Upheld: on 58eb5b699 the constructor replayed unconditionally, so a config=None store (onboarding import's foreign data_home, eval harnesses) adopted the install's live memory.* tuning at construction. VectorMemoryStore.__init__ now replays only when it was built with config=. New regression test_a_store_built_without_a_config_is_not_replayed (test/test_services_config_hot_reload.py) primes a cap-900 watcher and builds a config=None store with episodic_max=77: it fails on 58eb5b699 with assert 900 == 77 and passes on 99a514a66. The memory spec now says exempt stores keep constructor defaults at construction and, like every store, still follow later live reloads through the subscription (unchanged from main).

@jeeshofone

Copy link
Copy Markdown
Contributor Author

span=15f027fccf46

fixed in 65d731d15
Upheld: EpisodicWriteOutcome and VectorMemoryStore are now imported once at module scope in src/kiro_crew/apps/builtins/ops_mission_control/tests/test_ledger_index.py; all five method-local from kiro_crew.vector_memory import ... lines are removed, including the one that predates this PR. The file's 35 tests pass.

@jeeshofone

Copy link
Copy Markdown
Contributor Author

First-principles, span=3753a3dfe4b9 (replay's ConfigDeferred and awaitable arms): rebutted, with a pinning test on d057475d6.

The item rests on "that subscription's applier is sync and never raises ConfigDeferred". The sole caller does raise it:

  • vector_memory.py registers with live.watch_object(self, "memory", name="VectorMemoryStore") (the line before live.watch().replay(self._config_sub)).
  • watch_object uses _object_applier in config/live.py. When memory is in change.new.degraded_sections it raises ConfigDeferred(...), the fail-closed rule every owned subscription follows, so a degraded document's defaults never reach reconfigure.

So on replay against a snapshot with a degraded memory section, the except ConfigDeferred arm is the path taken. If it were deleted, the generic except Exception would still mark the paths stale, but it would log config applier 'VectorMemoryStore' failed on replay at ERROR with a traceback for an expected deferral. dispatch (_apply_one) keeps the same two arms apart for the same reason.

New test TestReplayOnRegistration::test_a_degraded_section_is_deferred_on_replay_not_logged_as_a_failure in test/test_config_live.py primes a watcher with a snapshot whose agent section is degraded, then replays a watch_object owner. It asserts reconfigure is not called, the paths go stale, and no "failed on replay" error is logged. With the except ConfigDeferred arm removed locally it fails: 'failed on replay' is contained here.

The awaitable arm is not covered by the generic except Exception either. An async applier returns a coroutine and raises nothing, so without the arm replay would record the snapshot as applied, never run it, and leave a "coroutine was never awaited" warning. That is the silent lost reload this PR exists to close. replay accepts any Subscription, and the registry supports async appliers (test_a_sync_and_an_async_applier_both_run). The arm is five lines and fails safe (close the coroutine, mark stale, and the next tick awaits it). I am keeping it and deferring any narrowing of replay's contract to the first owner that needs one.

@jeeshofone

Copy link
Copy Markdown
Contributor Author

GPT, span=5f43642af427 (docs/system-specs/modules/memory-skills-hooks.md, V1 cap paragraph: "for a normal write 0 and 1 behave the same"): rebutted, no change on d057475d6.

The parenthetical explains why the loader floors memory.episodic_max_count at 1. It does not describe a behavior a configured store can reach: _build_memory_config in config/loader.py clamps 0 and negatives to 1 before any store sees the value. test/test_config_loader.py::test_episodic_max_count_is_floored_at_one pins that clamp (0 -> 1, -5 -> 1, 7 kept). The part of the floor's reason that matters, a cap of 1 still admitting the first merge-only write, is pinned by the EpisodicWriteOutcome contract test in test/test_vector_memory.py (episodic_max=1: first write WRITTEN, a distinct second AT_CAPACITY).

Spec prose is not test input in this repo. A test that string-matches this paragraph would fail on a rewording and pass on a behavior change, which is the reverse of what it would be for. If the floor moved, the loader test fails, and that is the trigger to revise the sentence.

@jeeshofone

Copy link
Copy Markdown
Contributor Author

Design, span=eb9d6193a40c (a pending count in import_pending's result): rebutted as disproportional, no change on 2cea2ad47.

The only production consumer is the hygiene route in routes.py, which calls _index_ledger_safely, derives changed from indexed.get("written") alone, and echoes the dict as index; nothing acts on a paused import, and the next import retries every refused entry because they stay off the cursor.
The pause is pinned by test_capacity_pause_is_logged (asserts "3 ledger entries left pending" at WARNING) and by the cursor tests in ops_mission_control/tests/test_ledger_index.py. A pending key would add result surface with no reader today; it belongs with the first caller that must act on a pause.

@jeeshofone

Copy link
Copy Markdown
Contributor Author

First-principles, span=16d6ebf51ecb (ConfigWatch.replay's apply-now loop): needs-a-decision on 2cea2ad47.

The premise holds as read this session: _cycle calls _retry_stale(self._cfg) on an unchanged-fingerprint tick and on a no-diff reload (config/live.py), so _mark_stale(sub, frozenset(sub.prefixes)) alone delivers the adopted config within one DEFAULT_POLL_INTERVAL_SECS (2.0s) tick.
I cannot name a production loss for the Clears-when. All three config-built sites call KiroCrewConfig.load() immediately before constructing the store (context.py _prepare, ops-mission-control dispatch._attach_similar_safely, cli_commands.py), and the apply arm only runs when the watcher's fingerprint equals the file's, so it can differ from the caller's own load only for a reload landing in that gap.
The one residual: a per-call store (_attach_similar_safely) that closes inside one tick never receives a stale-marked delivery; it loaded the file itself milliseconds earlier.
Shrinking replay to its _mark_stale arm deletes the loop plus the mid-replay and deferred-replay tests, and reverses the d057475d6 disposition (first-principles span 3753a3dfe4b9) to keep those arms. Maintainer: shrink it in this PR, or keep the immediate apply? I will make the subtraction on your word.

@jeeshofone

Copy link
Copy Markdown
Contributor Author

Design, span=568b3691042a (ledger indexing pauses at the episodic cap): rebutted, the pause is the recorded steady state for #10889, no change on d5bcf912b.

Decision: at a full V1 store the ledger importer pauses and keeps its backlog off the cursor. That is the intended steady state for this PR, not an oversight. The goal is that every store honors the configured cap; whether the importer may make room for itself is a retention policy this PR does not set.
The trap is not new. On origin/main the merge-only branch of write_episodic returns False at active_count >= self._episodic_max (vector_memory.py:4350), and import_pending cursors every False as skipped. So an install at the default 10,000 cap already stops indexing today, and it loses those entries silently. This head makes the same state visible and recoverable: write_episodic_outcome returns AT_CAPACITY, the entries stay pending, and they index once the cap is raised or an unrelated delete frees a slot (TestCapacityRefusal, test_capacity_verdict_survives_a_cap_raise_racing_the_refusal).
The warning is already rate-limited by its caller. The only production path is routes.py:_index_ledger_safely, reached through POST /ledger/hygiene, which the app's ledger-hygiene cron calls once a day (app.json, cron_expr: "17 3 * * *"). A paused install logs one WARNING per day naming the pending count and the remedy, and the loop breaks at the first refusal, so a paused run costs one refused write.
Letting the importer evict its own ops-ledger rows would drop older ledger knowledge to admit newer entries and would break the merge-only contract that preserve_existing exists for. That belongs in a separate change with its own retention decision. I am flagging it to the maintainer as a follow-up candidate and am not claiming it is tracked.

@jeeshofone

Copy link
Copy Markdown
Contributor Author

Design, span=84667e35b901 (ConfigWatch.replay wired to one of six owners): rebutted as out of scope, no change on d5bcf912b.

The other five watch_object owners (cron_history.py, subagent.py, history_consolidation.py, adaptive/controller.py, slack/gateway.py) are named in docs/system-specs/modules/config.md as keeping the registration gap and out of scope for #10889. None of them is a memory store, so wiring them is outside this PR's goal. The First Principles lane on this head counted them as deferred by name.
replay is generic and each owner opts in with one call after watch_object, so the follow-up is mechanical per owner. It needs a per-owner check that replaying the adopted snapshot is safe for that applier. I am flagging it to the maintainer as a follow-up candidate and am not claiming it is tracked.

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed on behalf of the maintainers (sweep 2026-09-28): readiness passed, required checks green on d5bcf91, no blocking review threads, security posture unchanged. Approving and merging.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

memory: named-store construction omits dedup_threshold, so the configured value is ignored

2 participants