Repository navigation
feat(memory): add matching lessons to follow-up messages - #15517
Conversation
|
Intent: Let a session receive a stored lesson when its conversation reaches that lesson's topic, not only when its first message did, behind |
Design Review (Fable 5, fork) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS Admission rule may rarely fire on the motivating case: topic words shared by more than max(1, 1%·N) lessons can never count as rare. Watch
[DESIGN-REVIEWED] fe7069d Description read[DESCRIPTION-READ] c0d862c1945f56906196aac61026a01b7c7395a516c691c3abb6c3087316cb2e That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Fable 5, fork) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS With the flag on, every follow-up message scans and stems the whole lesson store lexically — verify that cost stays negligible on a store of thousands of lessons. What this change shipsInventory (9 items) — 9 justifiedIntent: let a session receive a stored lesson when the conversation reaches its topic, not only at session start — an ADDITION, gated off by default (explicitly not product shape: a new flag nobody gets without choosing, with the default change deferred to an RFC).
Counts run: [FIRST-PRINCIPLES-REVIEWED] fe7069d Description read[DESCRIPTION-READ] 6fa4a8cae915b1e33772fd6c26be4b36c3db2ac24523f2ea930d9db97cb336f8 That sha256 names the PR description this verdict read. Recompute it with |
GPT 5.6 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
Opus 5 Review (fork) — ✅ no blocking findingsReviewed |
0fdfb51 to
5cb3704
Compare
|
Design Review on 0fdfb51 asked: "Verify the 2-term/5% admission bar against a real lesson store -- the precision thresholds are pinned only on synthetic fixtures." Verified, and the bar was too loose; 5cb3704 replaces it.
|
|
self-added: no
|
|
self-added: no
|
5cb3704 to
1713e2e
Compare
|
Pushed 1713e2e: it fixes the three advisory GPT findings on 5cb3704 (dispositions below) and rebases onto main 7460284, which carries the Windows lock-file sweep fix (#15633). Main touched none of this PR's files. CI re-running. The one red check on 5cb3704, Windows shard 2, fails the same two |
|
|
|
1713e2e to
8b66512
Compare
|
Rebased: main moved in files this PR touches or sits beside ( The red checks on 1713e2e were outside this PR:
|
…y.inject_lessons_per_turn With the flag on (off by default), each follow-up message can carry up to three stored lessons that match it on words rare among stored lessons and that the session has not been shown: the session-start block holds only what fits its budget. A per-session record of the startup block and a bounded hash of each lesson sent since keeps a lesson from repeating; it is checked again after the store read, reset on a compaction re-injection, and each lesson line is scrubbed of boundary, member-authority, lesson-frame and skill-frame markers.
8b66512 to
fe7069d
Compare
|
Rebased onto main ecdc59a (from 309d005) to re-run the review lanes. This PR's diff is unchanged (patch-id equal), and main touched none of its files. CI re-running. On 8b66512 every other check passed, but no lane verdict reached the PR:
|
|
self-added: yes
|
bolichen97
left a comment
There was a problem hiding this comment.
Approved: PR Readiness passed on this head, no open change requests.
kirodotdev#15688 replaced lessons.py's by-name imports from text_scoring with reads through the owner module. kirodotdev#15517, which merged after it, added turn_lessons with one bare _row_stem_tokens_for_scan call, so main raises NameError: name '_row_stem_tokens_for_scan' is not defined on every turn_lessons call (flake8 F821). Read it through _text_scoring like the other callers in the file.
…15744) * fix(crew-log): regenerate the fold catalogue for the subagents fold #15078 added a `subagents` projection fold but did not add its answer sentence to the catalogue generator or regenerate the committed catalogue, so test_fold_catalogue.py reddened on main d5bdff6 (shard 4, Linux + Windows): the generator refused to build (no `_ANSWERS` sentence for `subagents`) and the committed folds.json / FOLDS.md no longer matched the registry. Add the `subagents` sentence to `_ANSWERS` and regenerate both artifacts with `python3 scripts/fold_catalogue.py --write`. * fix(memory): read turn_lessons' row-token helper through text_scoring #15688 replaced lessons.py's by-name imports from text_scoring with reads through the owner module. #15517, which merged after it, added turn_lessons with one bare _row_stem_tokens_for_scan call, so main raises NameError: name '_row_stem_tokens_for_scan' is not defined on every turn_lessons call (flake8 F821). Read it through _text_scoring like the other callers in the file. --------- Co-authored-by: kirocrew-agent <agent@kirocrew.local> Co-authored-by: Sajal Narang <sajalnarang@gmail.com>
kirodotdev#15688 replaced lessons.py's by-name imports from text_scoring with reads through the owner module. kirodotdev#15517, which merged after it, added turn_lessons with one bare _row_stem_tokens_for_scan call, so main raises NameError: name '_row_stem_tokens_for_scan' is not defined on every turn_lessons call (flake8 F821). Read it through _text_scoring like the other callers in the file.
Problem / Motivation
Goal: A session can receive a stored lesson when its conversation reaches the lesson's topic, not only when its first message did.
Lessons reach a session once, at session start, ranked against the first message and cut to a 37,000-character budget. A store holding more rules than that sends the rest to no session that did not open on their topic. When the conversation moves on, a rule it now needs never arrives, and a rule saved with
learn_addmid-session is not seen until the next session.Why it matters
The costliest mistakes happen after the first message: a session that opens on one task and moves to another runs the second task without the corrections the user taught for it. On a store past the budget, most rules are in this position on any given session.
Not a goal
memory.inject_lessons_per_turn, off by default. Turning it on by default is a product decision for an RFC.What changed (motivation → approach → change)
With
memory.inject_lessons_per_turnon, a follow-up message that shares at least two rare words with a stored lesson gets that lesson, once per session, in a short block before the message.[Learned corrections — relevant to this message …]blockinject_lessonsoffThe match reads the user's own text (the
user_text_rangeslice, or a transform hook's output), because every lesson it picks is recorded as shown.VectorMemoryStore.turn_lessons(avector_memory.pyfacade overvector_memory_runtime/lessons.py) drops message words on the lesson stop list and words of two letters or fewer, then matches each remaining word once, through its stem. A lesson is admitted only when it shares at least two distinct stems with the message that each appear in at most 1% of the stored lessons, and never fewer than one, so a small store still admits through words no other lesson carries. The rule counts rare words rather than summing their rarity weights, because a sum lets one word found in a single lesson admit that lesson alone, and in a store of a few hundred lessons it counts ordinary words as rare. Admitted lessons are ordered by summed rarity weight over the square root of their number of distinct words (the #15161 scorer, split into_shared_lesson_stemsand_rarity_weights); rules and findings are both eligible.context.pykeeps, per session and keyed by a fixed-size digest of the session key, the session-start block as sent and a hash of each lesson sent since (_ShownLessons: 256 sessions of 256 lessons, oldest dropped first). It reads the same store the session-start block reads under the same gates, checks its choice again against the record once the store has been read, and resets the record when a compaction re-injection runs. A stored lesson can come from an inferred or imported rule, so each line is scrubbed of boundary, member-authority, lesson-frame and skill-frame markers (_scrub_turn_lesson). The record is in memory and bounded: a gateway restart, turning the flag on mid-session, 256 newer lessons in the same session, or the record dropping the whole session can send a lesson once more. The 2,000-character cap counts the scrubbed lesson lines the block shows, not its header and footer, and a lesson left out for size is not counted as shown. A lesson counts as shown when its block is built, so a cancelled or failed turn withholds it until the next compaction.The flag is declared in
config/memory_sections.pyand parsed inconfig/section_builders.py;config-baseline.jsoncarries the new key. The memory spec, the config spec,docs/configuration.mdanddocs/architecture/context-management.mddocument it.Backwards compatibility
Compatible: the flag defaults to off, and with it off no code path changes what a session receives.
_lexical_lesson_weights, built from_shared_lesson_stemsand_rarity_weights, and the shared_renderable_entrieshelper return exactly what the code they came from returned, so the session-start ranking is unchanged.Tests
test/test_lesson_turn_injection.py:inject_lessonsis off, for a temporary session and for a minimal-context turn.test/test_vector_memory_composition_contract.pypins the facade's signature, with the two limits keyword-only and no default.test/test_config_baseline.pychecks the regeneratedconfig-baseline.json. Each gate was checked by removing it, and the test written for it failed every time; the one-rare-word and six-lesson tests both fail against a rule that sums rarity weights instead of counting rare words.Manual verification
N/A. Unit coverage exercises the builder end to end (
build_messageagainst a real vector store); the flag is off by default, so no existing session changes.Screenshots / video
Why no screenshot: this PR changes no file under
website/. The Screenshot Evidence check diffs against the base GitHub recorded before the rebase, so it would count frontend files that main changed in between.Related Issues
no linked issue: this adds an opt-in capability; #14243 (fixed by #15161) is about the session-start ranking it builds on.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)