diff --git a/docs/system-specs/modules/memory-skills-hooks.md b/docs/system-specs/modules/memory-skills-hooks.md index 31af001c6b42..0e597eec4808 100644 --- a/docs/system-specs/modules/memory-skills-hooks.md +++ b/docs/system-specs/modules/memory-skills-hooks.md @@ -779,7 +779,7 @@ outlive the incognito or temporary boundary that prohibits derived memory. ### Lesson Extraction from Chat -The history consolidation prompt includes a `"lessons"` key that extracts only implicit correction patterns — corrections the user made without explicitly saying "remember" (those are already saved immediately via `learn_add`). V1 lesson writes use `write_lesson()` with substring and topic-overlap dedup (shared keywords ≥ 50% of the LARGER of the two keyword sets → newer replaces older), or `LessonStore.save()` when vector memory is inactive. V2 publishes lessons inside the consolidation transaction and never constructs a JSONL fallback. +The history consolidation prompt includes a `"lessons"` key that extracts only implicit correction patterns — corrections the user made without explicitly saying "remember" (those are already saved immediately via `learn_add`). V1 lesson writes use `write_lesson()` with substring and topic-overlap dedup (shared keywords ≥ 50% of the LARGER of the two keyword sets → newer replaces older, unless the stored row outranks the write), or `LessonStore.save()` when vector memory is inactive. V2 publishes lessons inside the consolidation transaction and never constructs a JSONL fallback. ### Configuration @@ -3839,6 +3839,13 @@ whose `repo_scope` is present but unusable counts as neither. makes its text longer and its guidance NARROWER, so the row that survives can be the one that applies in fewer cases. - Topic-overlap dedup: "use light mode" replaces "use dark mode" (shared keywords ≥ 50% of the LARGER keyword set → newer wins) +- Neither lexical rule deletes a stored row that outranks the write (`user_explicit` over + another source, or strictly higher stored confidence): the submission is stored beside it. + The cost is that a consolidation correction of a user rule ("use light mode" over a + `user_explicit` "use dark mode") leaves both in effect until a `user_explicit` write or a + delete retires the old one. The authority pre-pass declines a pure semantic + near-duplicate instead: above 0.85 cosine it carries no new text, while a lexical match + can carry a new condition the user may want - Allowlist validation, injection scanning, audit logging Substring-delete and topic-overlap are not independent: verbatim containment at word @@ -3848,7 +3855,7 @@ did. Suppressing either one alone does not keep both lessons — which is why `write_lesson` REPORTS its deletions (below) rather than declining to make them, and why a caller that must never replace an existing lesson routes to `set_semantic_if_absent` instead (see `onboarding_import`, whose comment records that a -foreign directive could otherwise delete a correction the user taught the agent). +foreign directive could otherwise delete a lesson already in the store). **A call that stores nothing deletes nothing.** Every branch above QUEUES its supersedes; the queue drains only after the semantic write commits the submission. The @@ -4046,7 +4053,7 @@ lesson beat a contradicting preference in the same prompt. | Lesson contradicts a preference | Lesson wins via the `[Learned corrections]` framing | `context.py` | | Two semantic writes to one key in V1 | User-explicit writes win; automated writes cannot replace a user-explicit fact, and otherwise use the existing confidence precedence | `vector_memory._write_semantic()` | | Two semantic writes to one key in V2 | Owner correction or verified transcript correction with matching revision replaces the fact; other changed automated assertions remain reviewable proposals | `vector_memory._write_semantic()` | -| Duplicate lessons in V1 | Substring dedup (contained-in-stored declines; contains-a-stored-one deletes it, longer wins, regardless of source), then topic-overlap dedup (shared keywords cover at least 50% of the LARGER keyword set; newer replaces older regardless of source), then embedding dedup (cosine > 0.85; newer replaces older unless a stored near-duplicate outranks the write: `user_explicit` over a lower-authority source, or strictly higher stored confidence). A non-mutating authority pre-pass decides semantic-match refusals before the scan; it mirrors the earlier substring/topic branches, which remain source-blind. Every branch's deletions are deferred and execute only after the semantic write commits, so a call that stores nothing deletes nothing: a `deduped` verdict from any branch and a `refused` from the semantic write each preserve every live lesson row and report empty `superseded` (a lazy embedding backfill for a row the call read may still have flushed, which changes a vector and no lesson). `delete_semantic` takes an optional `expect_value_json`, carrying the comparison inside its own UPDATE, so a queued row is tombstoned only while its stored body is still the version the scan read and one a competing writer changed is skipped -- the same compare-and-write contract the lazy embedding backfill applies, atomic against another process, and what covers the write's own key once `set_semantic` has committed under it. A drain that stops partway (raised error, killed process) keeps the submission and leaves the rows it had not reached: the residue is a duplicate, never a lost lesson, and the next write matching those rows retires them. Every completed deletion is named in `LessonWriteResult.superseded` | `vector_memory.write_lesson()` | +| Duplicate lessons in V1 | Substring dedup (contained-in-stored declines; contains-a-stored-one deletes it, longer wins), then topic-overlap dedup (shared keywords cover at least 50% of the LARGER keyword set; newer replaces older), then embedding dedup (cosine > 0.85; newer replaces older unless a stored near-duplicate outranks the write: `user_explicit` over a lower-authority source, or strictly higher stored confidence). No branch deletes a row that outranks the write by that same rule: the two lexical branches store the submission beside it, so a consolidation rewording never retires the user's own lesson. A non-mutating authority pre-pass decides semantic-match refusals before the scan; it skips the rows the earlier substring/topic branches claim. Every branch's deletions are deferred and execute only after the semantic write commits, so a call that stores nothing deletes nothing: a `deduped` verdict from any branch and a `refused` from the semantic write each preserve every live lesson row and report empty `superseded` (a lazy embedding backfill for a row the call read may still have flushed, which changes a vector and no lesson). `delete_semantic` takes an optional `expect_value_json`, carrying the comparison inside its own UPDATE, so a queued row is tombstoned only while its stored body is still the version the scan read and one a competing writer changed is skipped -- the same compare-and-write contract the lazy embedding backfill applies, atomic against another process, and what covers the write's own key once `set_semantic` has committed under it. A drain that stops partway (raised error, killed process) keeps the submission and leaves the rows it had not reached: the residue is a duplicate, never a lost lesson, and the next write matching those rows retires them. Every completed deletion is named in `LessonWriteResult.superseded` | `vector_memory.write_lesson()` | | Distinct lessons in V2 | Different rule text coexists without substring, topic or embedding deduplication. Exact-rule enrichment remains; key-targeted corrections use the owner/revision machinery | `vector_memory.write_lesson()` | | Contradicting episodic fragments | No explicit resolution: time decay plus MMR surfaces the newer/more relevant fragment | `VectorMemoryStore.search_episodic()` (`vector_memory_runtime/episodic_search.py`) | | A semantic value is superseded | `_retire_stale_episodic()` tombstones episodic rows that quote the old value | the publication tail of `vector_memory._write_semantic()` (`vector_memory_runtime/semantic.py`), after the write commits | diff --git a/docs/system-specs/modules/onboarding-import.md b/docs/system-specs/modules/onboarding-import.md index ff5f8c68c2dd..489ad95fe0f5 100644 --- a/docs/system-specs/modules/onboarding-import.md +++ b/docs/system-specs/modules/onboarding-import.md @@ -229,7 +229,7 @@ Durable tiers only: and receives the write instead — via `set_semantic_if_absent`, NOT `write_lesson`: that writer deletes an existing lesson on exact-substring or >50% topic overlap ("newer replaces older"), which for an import would let a - foreign directive delete a correction the *user* taught the agent. Import + foreign directive delete a lesson already in the store. Import recognizes the same overlap and reports `existing` instead of replacing. `ContextBuilder` reads vector lessons first and never falls back to the JSONL when they exist, so a JSONL-only write would be recorded as imported yet stay invisible to the agent. diff --git a/src/kiro_crew/onboarding_import.py b/src/kiro_crew/onboarding_import.py index ab0b07fa8e21..f0d1162bdabb 100644 --- a/src/kiro_crew/onboarding_import.py +++ b/src/kiro_crew/onboarding_import.py @@ -235,8 +235,8 @@ def _write_instruction( if vector_store is not None: # NOT ``write_lesson``: it deletes an existing lesson on exact-substring # OR >50% topic overlap ("newer replaces older"), which for an import - # means a foreign directive can delete a correction the USER taught the - # agent. Import is merge-only, so overlap yields ``existing`` (nothing + # means a foreign directive can delete a lesson already in the store. + # Import is merge-only, so overlap yields ``existing`` (nothing # written, nothing deleted) and only a genuinely new rule is inserted -- # via the absent-only writer, which cannot replace anything. for existing in vector_store.get_lessons(): diff --git a/src/kiro_crew/vector_memory_runtime/lesson_write.py b/src/kiro_crew/vector_memory_runtime/lesson_write.py index b2d15cd4fb75..9840d259be6c 100644 --- a/src/kiro_crew/vector_memory_runtime/lesson_write.py +++ b/src/kiro_crew/vector_memory_runtime/lesson_write.py @@ -47,6 +47,21 @@ def _lesson_writer(raw: object) -> dict[str, str] | None: return stamp_of(raw) +def _outranks_write(existing: dict, source: str, confidence: float) -> bool: + """Whether stored row *existing* outranks a ``source`` write at *confidence*. + + A ``user_explicit`` row outranks every other source, and a strictly higher stored + confidence outranks a lower one. Nothing outranks a ``user_explicit`` write. + """ + if source == "user_explicit": + return False + try: + existing_confidence = float(existing.get("confidence") or 0.0) + except (TypeError, ValueError): + existing_confidence = 0.0 + return existing.get("source") == "user_explicit" or existing_confidence > confidence + + def write_lesson( self: VectorMemoryStore, rule: str, @@ -82,6 +97,9 @@ def write_lesson( over a lower-authority source, or strictly higher stored confidence), which reports ``deduped`` / ``semantic_similarity``. + No rule deletes a stored row that outranks this write: the two lexical rules + store the submission beside it instead. + **A call that stores nothing deletes nothing.** Every supersede the scan decides on is QUEUED, and the queue drains only once ``set_semantic`` has committed the submission. So a ``deduped`` verdict from any branch, and a @@ -370,7 +388,6 @@ def _flush_backfills() -> None: # skipped for it entirely. The pass is non-mutating for its own reason, # independent of the scan's deferral below: an authority decline must not # spend the call's embed budget rewriting rows it is about to refuse. - # Scan-wide authority ordering is a tracked follow-up decision. if ( self.algorithm_version != "v2" and similarity is not None @@ -383,11 +400,10 @@ def _flush_backfills() -> None: continue # PURE semantic matches only. A row the mutating scan's # substring or topic-overlap branch would claim FIRST (per-row - # branch order) keeps main's outcome for it -- those branches - # are source-blind by main's design, and the pre-pass must not - # decline a write that main would have resolved lexically - # before the semantic test ever ran. The predicates mirror the - # scan's own, on the same normalized text. + # branch order) is left to that branch: it declines a submission + # the row covers, as before, and otherwise stores the submission + # beside an outranking row. The predicates mirror the scan's own, + # on the same normalized text. pre_lower = pre_text.lower() if rule_lower in pre_lower or pre_lower in rule_lower: continue @@ -446,11 +462,7 @@ def _flush_backfills() -> None: # equal and lets the newer write win): near-duplicates are # DIFFERENT rows with no same-key freshness to prefer, and # equal confidence falls through to newest-wins below. - try: - existing_confidence = float(existing.get("confidence") or 0.0) - except (TypeError, ValueError): - existing_confidence = 0.0 - if existing.get("source") == "user_explicit" or existing_confidence > confidence: + if _outranks_write(existing, source, confidence): logger.info( "Lesson semantic dedup: higher-authority %r kept over %s", existing["key"], @@ -489,6 +501,11 @@ def _flush_backfills() -> None: may_retire_existing, covering_row_arrives_less_often = _lessons.tier_permissions( existing, applies ) + # No branch below retires a row that outranks this write: a consolidation + # rewording that overlaps the user's own lesson is stored beside it. + may_retire_existing = may_retire_existing and not _outranks_write( + existing, source, confidence + ) # Substring dedup if rule_lower in existing_lower and not covering_row_arrives_less_often: @@ -588,8 +605,7 @@ def _flush_backfills() -> None: sim = similarity({"embedding": row_blob}) if sim > 0.85: # Newest wins, matching the substring and topic-overlap - # branches above -- both supersede the stored row - # unconditionally. A length tie-break on + # branches above. A length tie-break on # ``len(rule) > len(existing_text)`` would DROP the # submission when it loses, making character count # decide which of two near-identical rules is current. @@ -600,9 +616,8 @@ def _flush_backfills() -> None: # effect, the one outcome that actively misleads the # agent rather than merely losing information. # - # No authority check HERE, by construction: the - # non-mutating pre-pass above already returned DEDUPED - # if any purely-semantic row outranks the write. Like + # The non-mutating pre-pass above already returned + # DEDUPED if any purely-semantic row outranks the write. Like # both lexical branches, this one only QUEUES its # supersede -- a later row can still decline the write # via ``substring_covered``, and a declined write diff --git a/test/test_vector_memory_coverage_vector_space.py b/test/test_vector_memory_coverage_vector_space.py index 5050e4c1504b..70735af7c743 100644 --- a/test/test_vector_memory_coverage_vector_space.py +++ b/test/test_vector_memory_coverage_vector_space.py @@ -412,10 +412,12 @@ def test_a_declined_write_deletes_no_row_it_scanned_before_the_decliner( def test_the_authority_pre_pass_skips_rows_the_lexical_branches_claim( self, tmp_path: Path ) -> None: - """A row substring/topic-overlap would resolve keeps main's outcome. + """A consolidation write containing a ``user_explicit`` rule keeps both. - The pre-pass covers pure semantic matches only; the source-blind substring branch - still deletes a contained ``user_explicit`` row ("longer wins"). + The pre-pass covers pure semantic matches only, so the substring + branch claims this pair first (per-row branch order). That branch + must not turn it into an authority decline, and must not retire the + user's own rule either: the submission is stored beside it. """ taught = "Zebra crossings need beacons" submitted = "Zebra crossings need beacons after dusk" @@ -425,8 +427,51 @@ def test_the_authority_pre_pass_skips_rows_the_lexical_branches_claim( result = store.write_lesson(submitted, source="consolidation") assert result # stored, not declined assert result.outcome is LessonWriteOutcome.INSERTED - assert len(result.superseded) == 1 # the contained row was superseded - assert _lesson_texts(store) == [submitted] + assert result.superseded == () + assert sorted(_lesson_texts(store)) == sorted([taught, submitted]) + + @pytest.mark.parametrize("stored_source", ["user_explicit", "import"]) + def test_a_topic_overlap_write_keeps_an_outranking_row( + self, tmp_path: Path, stored_source: str + ) -> None: + """A consolidation write sharing half its keywords never retires a higher row. + + ``user_explicit`` outranks by source; the onboarding import's 1.0 outranks + the write's 0.9 by confidence. + """ + taught = "Quokkas patrol the deck" + submitted = "Quokkas patrol the western observation deck at dawn" + store = _store(tmp_path) + if stored_source == "user_explicit": + assert store.write_lesson(taught) + else: + key = f"lesson.{hashlib.sha256(taught.encode()).hexdigest()[:16]}" + value = {"rule": taught, "category": "preference", "negative": None} + assert store.set_semantic_if_absent(key, value, 1.0, "import") == "imported" + result = store.write_lesson(submitted, source="consolidation") + assert result.outcome is LessonWriteOutcome.INSERTED + assert result.superseded == () + assert sorted(_lesson_texts(store)) == sorted([taught, submitted]) + + def test_a_semantic_match_the_pre_pass_skips_keeps_an_outranking_row( + self, tmp_path: Path + ) -> None: + """The scan's semantic branch keeps a user rule the pre-pass left to the scan. + + The stored row is an ``on_topic`` finding that contains the standing + submission, so the pre-pass skips it as lexical and the substring branch + does not decline it; the keyword ratio is under 0.5, so only the cosine + branch matches. + """ + taught = "Quokkas patrol the western observation deck at dawn every single morning" + submitted = "patrol the western observation deck" + store = _store(tmp_path) + store.embed_fn = _TableEmbedder({taught: _unit(0), submitted: _unit(0)}) + assert store.write_lesson(taught, applies="on_topic") # user_explicit + result = store.write_lesson(submitted, source="consolidation") + assert result.outcome is LessonWriteOutcome.INSERTED + assert result.superseded == () + assert sorted(_lesson_texts(store)) == sorted([taught, submitted]) def test_a_substring_covered_refusal_drops_the_deferred_semantic_deletes( self, tmp_path: Path