Repository navigation
fix(memory): consolidation states the lesson applies tier - #12875
Conversation
Design Review (Fable 5) — ✅ PASSDesign-level review of Design-Verdict: PASS Verify one live consolidation run emits [DESIGN-REVIEWED] 10a9b6b |
Opus 4.8 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
First Principles Review (Fable 5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Verify issue #12495's scope covers the member-store path too — the second commit extends the fix there beyond the first commit's two paths. What this change shipsInventory (8 items) — 8 justifiedIntent: Consolidated lessons should carry the authored startup tier so the two-budget split actually governs the highest-volume lesson writer — a FIX (issue #12495, 6 of 7 new tests red on base).
[FIRST-PRINCIPLES-REVIEWED] 10a9b6b |
e3d5d89 to
6e2c860
Compare
GPT 5.6 Review —
|
GPT 5.6 Review —
|
…2495) The extraction prompt now asks the model for "applies": "always|on_topic" per lesson with the same decide-from-what-the-user-said instruction that learn_add carries, and _save_lessons forwards the normalized tier to both stores. An omitted or blank tier lands the row unstated; a misspelled one is logged at warning and still lands unstated rather than dropping the correction. Closes #12495
VectorMemoryStore.apply_consolidation builds the lesson value itself and dropped the model-stated applies tier. It now normalizes it with the same policy as _save_lessons, via one shared extracted_lesson_applies helper: absent stays absent, a misspelling is logged and lands unstated. The tier instruction moves to one shared constant read by both learn_add's schema and the consolidation prompt, and the spec names the writers that still state no tier.
6e2c860 to
10a9b6b
Compare
Base branch was modified
bolichen97
left a comment
There was a problem hiding this comment.
Approved after verified readiness: passed, PR Readiness success on this head, mergeable, no open change requests. [operator: bolichen97 batch review 2026-09-23]
…#12875) * fix(memory): history consolidation states the lesson applies tier (kirodotdev#12495) The extraction prompt now asks the model for "applies": "always|on_topic" per lesson with the same decide-from-what-the-user-said instruction that learn_add carries, and _save_lessons forwards the normalized tier to both stores. An omitted or blank tier lands the row unstated; a misspelled one is logged at warning and still lands unstated rather than dropping the correction. Closes kirodotdev#12495 * fix(memory): member-store consolidation persists the lesson tier too VectorMemoryStore.apply_consolidation builds the lesson value itself and dropped the model-stated applies tier. It now normalizes it with the same policy as _save_lessons, via one shared extracted_lesson_applies helper: absent stays absent, a misspelling is logged and lands unstated. The tier instruction moves to one shared constant read by both learn_add's schema and the consolidation prompt, and the spec names the writers that still state no tier.
What
History consolidation now states the startup lesson tier (
applies) it extracts, on both lesson write paths.src/kiro_crew/history_consolidation.pylessonskey of the consolidation prompt asks for"applies": "always|on_topic"per item, with the same instructionlearn_addcarries: decide it from what the user actually said;alwaysis a standing rule,on_topica past finding; omit the key when you cannot tell._lesson_tier(item)callsextracted_lesson_applies(inlesson_validation.py, shared with the member-store path) once per item: it runsnormalize_lesson_applies; absent / blank ->None(unstated); a value that is not one of the two literals -> warning log (reason only, never the untrusted value) and the row is still written unstated, never dropped._save_lessonspasses the tier tovector_store.write_lesson(applies=...)and toLesson(applies=...).docs/system-specs/modules/memory-skills-hooks.md: the sentence saying consolidation states no tier, and the compatibility note, now describe the new behaviour.src/kiro_crew/vector_memory.py:apply_consolidation(member-store path) persists the tier through the sameextracted_lesson_applieshelper, so the two paths cannot drift.src/kiro_crew/lesson_validation.py+src/kiro_crew/mcp_tools/learn.py: the tier instruction lives in one constant,LESSON_APPLIES_INSTRUCTION, read by bothlearn_add's schema and the consolidation prompt.test/test_history.py:TestConsolidationLessonApplies(7 tests);test/test_member_memory_database.py:test_consolidation_persists_the_authored_lesson_tier.Why
Closes #12495. #12431 introduced the authored
appliestier and the two startup budgets, but consolidation, the highest-volume lesson writer, never reached it: its prompt asked only for rule / negative / category / repo_scope, and_save_lessonswrotesource="consolidation"rows with no tier. Every consolidated row therefore landed unstated and was served as a standing rule, so the budget split protected nothing for the rows that make up most of a store.The tier is authored, never derived (spec:
applies is AUTHORED, never derived), so the only correct place to state it is the extracting model, and the instruction text is lifted frommcp_tools/learn.pyso both writers ask the same question.How
Writers of consolidation lessons, enumerated (every
write_lesson/Lesson(...)construction reachable from consolidation):_save_lessonsvector path ->VectorMemoryStore.write_lesson(source="consolidation", ...)applies=self._lesson_tier(item)_save_lessonsJSONL fallback ->Lesson(...)->LessonStore.saveapplies=self._lesson_tier(item)VectorMemoryStore.apply_consolidation(member-store / V2 path, builds the lesson value itself)applieswith the same policy, key absent when unstatedtaskrunner.pyLLM lesson extraction (source="task_runner") andcli_commands.pydirectwrite_lessonskills.py:4783source="consolidation"vector_memory.py:3869source="consolidation"Misspelling policy: the write path's
normalize_lesson_appliesraises by design so a buggy writer is audible. Here the writer is a model, so the raise is caught per item, logged at warning, and the row lands unstated. Dropping it would lose a correction the user actually made over a one-word slip; unstated is the documented safe direction (a demoted standing rule is the costlier mistake). The value is normalized beforewrite_lesson, so its own raising check cannot fire on it.Tests
test/test_history.py::TestConsolidationLessonApplies:alwaysandon_topicround-trip through the vector store (value_json["applies"]).applieskey absent on the vector row (byte-identical to a pre-field row)."Directive") -> warning logged, row lands unstated, clean sibling keeps its tier, the untrusted value never appears in the log.LessonStore.load_all(); the unstated row carries noapplieskey on disk."ALWAYS "canonicalises without a warning; a non-string (7) warns and lands unstated.consolidate_sessionwith_call_llmpatched) names"applies": "always|on_topic", "YOU decide it from what the user actually said", and "OMIT the key when you genuinely cannot tell".Red on base: 6 of 7 fail on
e46967b5d(the omitted-key test passes there by construction, as the invariant guard). Mutation: removing the warning log fails both misspelled tests; droppingapplies=from the JSONL path fails the JSONL round-trip. Run locally by path with-n0; suites run in CI.test/test_member_memory_database.py::test_consolidation_persists_the_authored_lesson_tier: on the member-store path both tiers round-trip, an omitted key is absent, a misspelling warns and lands unstated (red on base:KeyError: 'applies').Static gates run locally on the changed files: black (ratchet), isort, flake8, mypy, comment-history.
Pattern harvest
Rule candidate: when a new authored field is added to a write API, grep every writer of that row type by
source=and list each with a verdict in the PR; the highest-volume writer is the one most likely to be missed because it does not call the new surface by name.Not generalizable: the catch-and-write-unstated policy is specific to a model-authored value where the alternative (drop) loses user data; a human-facing surface should keep the raise.
Other
No product decision remains: the spec fixes that the tier is authored by the extracting model and that an undecided row stays unstated. No existing row is rewritten.