Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions docs/system-specs/modules/memory-skills-hooks.md
Original file line number Diff line number Diff line change
Expand Up @@ -779,7 +779,7 @@ outlive the incognito or temporary boundary that prohibits derived memory.

### Lesson Extraction from Chat

The history consolidation prompt includes a `"lessons"` key that extracts only implicit correction patterns — corrections the user made without explicitly saying "remember" (those are already saved immediately via `learn_add`). V1 lesson writes use `write_lesson()` with substring and topic-overlap dedup (shared keywords ≥ 50% of the LARGER of the two keyword sets → newer replaces older), or `LessonStore.save()` when vector memory is inactive. V2 publishes lessons inside the consolidation transaction and never constructs a JSONL fallback.
The history consolidation prompt includes a `"lessons"` key that extracts only implicit correction patterns — corrections the user made without explicitly saying "remember" (those are already saved immediately via `learn_add`). V1 lesson writes use `write_lesson()` with substring and topic-overlap dedup (shared keywords ≥ 50% of the LARGER of the two keyword sets → newer replaces older, unless the stored row outranks the write), or `LessonStore.save()` when vector memory is inactive. V2 publishes lessons inside the consolidation transaction and never constructs a JSONL fallback.

### Configuration

Expand Down Expand Up @@ -3839,6 +3839,13 @@ whose `repo_scope` is present but unusable counts as neither.
makes its text longer and its guidance NARROWER, so the row that survives can be the
one that applies in fewer cases.
- Topic-overlap dedup: "use light mode" replaces "use dark mode" (shared keywords ≥ 50% of the LARGER keyword set → newer wins)
- Neither lexical rule deletes a stored row that outranks the write (`user_explicit` over
another source, or strictly higher stored confidence): the submission is stored beside it.
The cost is that a consolidation correction of a user rule ("use light mode" over a
`user_explicit` "use dark mode") leaves both in effect until a `user_explicit` write or a
delete retires the old one. The authority pre-pass declines a pure semantic
near-duplicate instead: above 0.85 cosine it carries no new text, while a lexical match
can carry a new condition the user may want
- Allowlist validation, injection scanning, audit logging

Substring-delete and topic-overlap are not independent: verbatim containment at word
Expand All @@ -3848,7 +3855,7 @@ did. Suppressing either one alone does not keep both lessons — which is why
`write_lesson` REPORTS its deletions (below) rather than declining to make them, and
why a caller that must never replace an existing lesson routes to
`set_semantic_if_absent` instead (see `onboarding_import`, whose comment records that a
foreign directive could otherwise delete a correction the user taught the agent).
foreign directive could otherwise delete a lesson already in the store).

**A call that stores nothing deletes nothing.** Every branch above QUEUES its
supersedes; the queue drains only after the semantic write commits the submission. The
Expand Down Expand Up @@ -4046,7 +4053,7 @@ lesson beat a contradicting preference in the same prompt.
| Lesson contradicts a preference | Lesson wins via the `[Learned corrections]` framing | `context.py` |
| Two semantic writes to one key in V1 | User-explicit writes win; automated writes cannot replace a user-explicit fact, and otherwise use the existing confidence precedence | `vector_memory._write_semantic()` |
| Two semantic writes to one key in V2 | Owner correction or verified transcript correction with matching revision replaces the fact; other changed automated assertions remain reviewable proposals | `vector_memory._write_semantic()` |
| Duplicate lessons in V1 | Substring dedup (contained-in-stored declines; contains-a-stored-one deletes it, longer wins, regardless of source), then topic-overlap dedup (shared keywords cover at least 50% of the LARGER keyword set; newer replaces older regardless of source), then embedding dedup (cosine > 0.85; newer replaces older unless a stored near-duplicate outranks the write: `user_explicit` over a lower-authority source, or strictly higher stored confidence). A non-mutating authority pre-pass decides semantic-match refusals before the scan; it mirrors the earlier substring/topic branches, which remain source-blind. Every branch's deletions are deferred and execute only after the semantic write commits, so a call that stores nothing deletes nothing: a `deduped` verdict from any branch and a `refused` from the semantic write each preserve every live lesson row and report empty `superseded` (a lazy embedding backfill for a row the call read may still have flushed, which changes a vector and no lesson). `delete_semantic` takes an optional `expect_value_json`, carrying the comparison inside its own UPDATE, so a queued row is tombstoned only while its stored body is still the version the scan read and one a competing writer changed is skipped -- the same compare-and-write contract the lazy embedding backfill applies, atomic against another process, and what covers the write's own key once `set_semantic` has committed under it. A drain that stops partway (raised error, killed process) keeps the submission and leaves the rows it had not reached: the residue is a duplicate, never a lost lesson, and the next write matching those rows retires them. Every completed deletion is named in `LessonWriteResult.superseded` | `vector_memory.write_lesson()` |
| Duplicate lessons in V1 | Substring dedup (contained-in-stored declines; contains-a-stored-one deletes it, longer wins), then topic-overlap dedup (shared keywords cover at least 50% of the LARGER keyword set; newer replaces older), then embedding dedup (cosine > 0.85; newer replaces older unless a stored near-duplicate outranks the write: `user_explicit` over a lower-authority source, or strictly higher stored confidence). No branch deletes a row that outranks the write by that same rule: the two lexical branches store the submission beside it, so a consolidation rewording never retires the user's own lesson. A non-mutating authority pre-pass decides semantic-match refusals before the scan; it skips the rows the earlier substring/topic branches claim. Every branch's deletions are deferred and execute only after the semantic write commits, so a call that stores nothing deletes nothing: a `deduped` verdict from any branch and a `refused` from the semantic write each preserve every live lesson row and report empty `superseded` (a lazy embedding backfill for a row the call read may still have flushed, which changes a vector and no lesson). `delete_semantic` takes an optional `expect_value_json`, carrying the comparison inside its own UPDATE, so a queued row is tombstoned only while its stored body is still the version the scan read and one a competing writer changed is skipped -- the same compare-and-write contract the lazy embedding backfill applies, atomic against another process, and what covers the write's own key once `set_semantic` has committed under it. A drain that stops partway (raised error, killed process) keeps the submission and leaves the rows it had not reached: the residue is a duplicate, never a lost lesson, and the next write matching those rows retires them. Every completed deletion is named in `LessonWriteResult.superseded` | `vector_memory.write_lesson()` |
| Distinct lessons in V2 | Different rule text coexists without substring, topic or embedding deduplication. Exact-rule enrichment remains; key-targeted corrections use the owner/revision machinery | `vector_memory.write_lesson()` |
| Contradicting episodic fragments | No explicit resolution: time decay plus MMR surfaces the newer/more relevant fragment | `VectorMemoryStore.search_episodic()` (`vector_memory_runtime/episodic_search.py`) |
| A semantic value is superseded | `_retire_stale_episodic()` tombstones episodic rows that quote the old value | the publication tail of `vector_memory._write_semantic()` (`vector_memory_runtime/semantic.py`), after the write commits |
Expand Down
2 changes: 1 addition & 1 deletion docs/system-specs/modules/onboarding-import.md
Original file line number Diff line number Diff line change
Expand Up @@ -229,7 +229,7 @@ Durable tiers only:
and receives the write instead — via `set_semantic_if_absent`, NOT
`write_lesson`: that writer deletes an existing lesson on exact-substring or
>50% topic overlap ("newer replaces older"), which for an import would let a
foreign directive delete a correction the *user* taught the agent. Import
foreign directive delete a lesson already in the store. Import
recognizes the same overlap and reports `existing` instead of replacing. `ContextBuilder` reads vector lessons first and
never falls back to the JSONL when they exist, so a JSONL-only write would be
recorded as imported yet stay invisible to the agent.
Expand Down
4 changes: 2 additions & 2 deletions src/kiro_crew/onboarding_import.py
Original file line number Diff line number Diff line change
Expand Up @@ -235,8 +235,8 @@ def _write_instruction(
if vector_store is not None:
# NOT ``write_lesson``: it deletes an existing lesson on exact-substring
# OR >50% topic overlap ("newer replaces older"), which for an import
# means a foreign directive can delete a correction the USER taught the
# agent. Import is merge-only, so overlap yields ``existing`` (nothing
# means a foreign directive can delete a lesson already in the store.
# Import is merge-only, so overlap yields ``existing`` (nothing
# written, nothing deleted) and only a genuinely new rule is inserted --
# via the absent-only writer, which cannot replace anything.
for existing in vector_store.get_lessons():
Expand Down
47 changes: 31 additions & 16 deletions src/kiro_crew/vector_memory_runtime/lesson_write.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,21 @@ def _lesson_writer(raw: object) -> dict[str, str] | None:
return stamp_of(raw)


def _outranks_write(existing: dict, source: str, confidence: float) -> bool:
"""Whether stored row *existing* outranks a ``source`` write at *confidence*.

A ``user_explicit`` row outranks every other source, and a strictly higher stored
confidence outranks a lower one. Nothing outranks a ``user_explicit`` write.
"""
if source == "user_explicit":
return False
try:
existing_confidence = float(existing.get("confidence") or 0.0)
except (TypeError, ValueError):
existing_confidence = 0.0
return existing.get("source") == "user_explicit" or existing_confidence > confidence


def write_lesson(
self: VectorMemoryStore,
rule: str,
Expand Down Expand Up @@ -82,6 +97,9 @@ def write_lesson(
over a lower-authority source, or strictly higher stored confidence),
which reports ``deduped`` / ``semantic_similarity``.

No rule deletes a stored row that outranks this write: the two lexical rules
store the submission beside it instead.

**A call that stores nothing deletes nothing.** Every supersede the scan
decides on is QUEUED, and the queue drains only once ``set_semantic`` has
committed the submission. So a ``deduped`` verdict from any branch, and a
Expand Down Expand Up @@ -370,7 +388,6 @@ def _flush_backfills() -> None:
# skipped for it entirely. The pass is non-mutating for its own reason,
# independent of the scan's deferral below: an authority decline must not
# spend the call's embed budget rewriting rows it is about to refuse.
# Scan-wide authority ordering is a tracked follow-up decision.
if (
self.algorithm_version != "v2"
and similarity is not None
Expand All @@ -383,11 +400,10 @@ def _flush_backfills() -> None:
continue
# PURE semantic matches only. A row the mutating scan's
# substring or topic-overlap branch would claim FIRST (per-row
# branch order) keeps main's outcome for it -- those branches
# are source-blind by main's design, and the pre-pass must not
# decline a write that main would have resolved lexically
# before the semantic test ever ran. The predicates mirror the
# scan's own, on the same normalized text.
# branch order) is left to that branch: it declines a submission
# the row covers, as before, and otherwise stores the submission
# beside an outranking row. The predicates mirror the scan's own,
# on the same normalized text.
pre_lower = pre_text.lower()
if rule_lower in pre_lower or pre_lower in rule_lower:
continue
Expand Down Expand Up @@ -446,11 +462,7 @@ def _flush_backfills() -> None:
# equal and lets the newer write win): near-duplicates are
# DIFFERENT rows with no same-key freshness to prefer, and
# equal confidence falls through to newest-wins below.
try:
existing_confidence = float(existing.get("confidence") or 0.0)
except (TypeError, ValueError):
existing_confidence = 0.0
if existing.get("source") == "user_explicit" or existing_confidence > confidence:
if _outranks_write(existing, source, confidence):
logger.info(
"Lesson semantic dedup: higher-authority %r kept over %s",
existing["key"],
Expand Down Expand Up @@ -489,6 +501,11 @@ def _flush_backfills() -> None:
may_retire_existing, covering_row_arrives_less_often = _lessons.tier_permissions(
existing, applies
)
# No branch below retires a row that outranks this write: a consolidation
# rewording that overlaps the user's own lesson is stored beside it.
may_retire_existing = may_retire_existing and not _outranks_write(
existing, source, confidence
)

# Substring dedup
if rule_lower in existing_lower and not covering_row_arrives_less_often:
Expand Down Expand Up @@ -588,8 +605,7 @@ def _flush_backfills() -> None:
sim = similarity({"embedding": row_blob})
if sim > 0.85:
# Newest wins, matching the substring and topic-overlap
# branches above -- both supersede the stored row
# unconditionally. A length tie-break on
# branches above. A length tie-break on
# ``len(rule) > len(existing_text)`` would DROP the
# submission when it loses, making character count
# decide which of two near-identical rules is current.
Expand All @@ -600,9 +616,8 @@ def _flush_backfills() -> None:
# effect, the one outcome that actively misleads the
# agent rather than merely losing information.
#
# No authority check HERE, by construction: the
# non-mutating pre-pass above already returned DEDUPED
# if any purely-semantic row outranks the write. Like
# The non-mutating pre-pass above already returned
# DEDUPED if any purely-semantic row outranks the write. Like
# both lexical branches, this one only QUEUES its
# supersede -- a later row can still decline the write
# via ``substring_covered``, and a declined write
Expand Down
55 changes: 50 additions & 5 deletions test/test_vector_memory_coverage_vector_space.py
Original file line number Diff line number Diff line change
Expand Up @@ -412,10 +412,12 @@ def test_a_declined_write_deletes_no_row_it_scanned_before_the_decliner(
def test_the_authority_pre_pass_skips_rows_the_lexical_branches_claim(
self, tmp_path: Path
) -> None:
"""A row substring/topic-overlap would resolve keeps main's outcome.
"""A consolidation write containing a ``user_explicit`` rule keeps both.

The pre-pass covers pure semantic matches only; the source-blind substring branch
still deletes a contained ``user_explicit`` row ("longer wins").
The pre-pass covers pure semantic matches only, so the substring
branch claims this pair first (per-row branch order). That branch
must not turn it into an authority decline, and must not retire the
user's own rule either: the submission is stored beside it.
"""
taught = "Zebra crossings need beacons"
submitted = "Zebra crossings need beacons after dusk"
Expand All @@ -425,8 +427,51 @@ def test_the_authority_pre_pass_skips_rows_the_lexical_branches_claim(
result = store.write_lesson(submitted, source="consolidation")
assert result # stored, not declined
assert result.outcome is LessonWriteOutcome.INSERTED
assert len(result.superseded) == 1 # the contained row was superseded
assert _lesson_texts(store) == [submitted]
assert result.superseded == ()
assert sorted(_lesson_texts(store)) == sorted([taught, submitted])

@pytest.mark.parametrize("stored_source", ["user_explicit", "import"])
def test_a_topic_overlap_write_keeps_an_outranking_row(
self, tmp_path: Path, stored_source: str
) -> None:
"""A consolidation write sharing half its keywords never retires a higher row.

``user_explicit`` outranks by source; the onboarding import's 1.0 outranks
the write's 0.9 by confidence.
"""
taught = "Quokkas patrol the deck"
submitted = "Quokkas patrol the western observation deck at dawn"
store = _store(tmp_path)
if stored_source == "user_explicit":
assert store.write_lesson(taught)
else:
key = f"lesson.{hashlib.sha256(taught.encode()).hexdigest()[:16]}"
value = {"rule": taught, "category": "preference", "negative": None}
assert store.set_semantic_if_absent(key, value, 1.0, "import") == "imported"
result = store.write_lesson(submitted, source="consolidation")
assert result.outcome is LessonWriteOutcome.INSERTED
assert result.superseded == ()
assert sorted(_lesson_texts(store)) == sorted([taught, submitted])

def test_a_semantic_match_the_pre_pass_skips_keeps_an_outranking_row(
self, tmp_path: Path
) -> None:
"""The scan's semantic branch keeps a user rule the pre-pass left to the scan.

The stored row is an ``on_topic`` finding that contains the standing
submission, so the pre-pass skips it as lexical and the substring branch
does not decline it; the keyword ratio is under 0.5, so only the cosine
branch matches.
"""
taught = "Quokkas patrol the western observation deck at dawn every single morning"
submitted = "patrol the western observation deck"
store = _store(tmp_path)
store.embed_fn = _TableEmbedder({taught: _unit(0), submitted: _unit(0)})
assert store.write_lesson(taught, applies="on_topic") # user_explicit
result = store.write_lesson(submitted, source="consolidation")
assert result.outcome is LessonWriteOutcome.INSERTED
assert result.superseded == ()
assert sorted(_lesson_texts(store)) == sorted([taught, submitted])

def test_a_substring_covered_refusal_drops_the_deferred_semantic_deletes(
self, tmp_path: Path
Expand Down
Loading