Repository navigation
fix(memory): count a lesson's length in distinct words - #15397
iamwhatever merged 1 commit into
Conversation
|
Intent: Make the startup lesson scorer's length divisor count a row's distinct words, so inflected wording no longer reads as extra length and costs a rule its rank. |
Design Review (Fable 5, fork) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing to check. [DESIGN-REVIEWED] 7cd5a82 Description read[DESCRIPTION-READ] 6f7606fc0cf2c9f65042d86667e1c0dd742246c755ca8486962e2f4993b83d4a That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
GPT 5.6 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
First Principles Review (Fable 5, fork) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS The new length term inlines the exact body of Not justified as shipped
What this change shipsInventory (3 items) — 2 justifiedIntent: FIX — a rule written in ordinary inflected English should not rank below an equally long base-form rule in startup lesson selection. The new test fails on base, so the defect's provenance is checkable; grep of
[FIRST-PRINCIPLES-REVIEWED] 7cd5a82 Description read[DESCRIPTION-READ] c96d8c97f2d3554dd4dd45cfc23f2d3050495a39d1246f1b46e8c902c9b5ef74 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5 Review (fork) — ✅ no blocking findingsReviewed |
dac7b0d to
ba60cfd
Compare
|
|
|
The rarity score behind startup lesson ranking divides by the square root of a
row's length. It measured that length as the row's token set, which holds every
word plus every stem that differs from its word, so a rule written with
inflections ("tests", "pushing") read as longer and ranked below an older rule
of the same length. The divisor is now the row's number of distinct words.
ba60cfd to
7cd5a82
Compare
|
Rebased: main moved in The patch is unchanged ( |
iamwhatever
left a comment
There was a problem hiding this comment.
Reviewed. The length divisor now counts distinct words of the row instead of the stemmed token set, so an inflected newer row no longer takes a bigger penalty than its plain twin; the new test pins the tie-then-recency case with equal distinct-word counts.
|
Problem / Motivation
Goal: A stored rule written with inflected words ranks as high as an equally long rule written without them.
Startup lesson ranking (#15161) divides each row's rarity score by the square root of the row's length. That length was the size of the row's token set, which holds every word plus every stem that differs from its word. A rule written with "tests" and "pushing" therefore read as longer than a rule of the same length written with "test" and "push". When a first message matched both, the newer inflected rule ranked below the older one instead of keeping its recency.
Why it matters
The divisor exists so a long row cannot win on incidental overlap. Counting stems as extra length turns it into a penalty on ordinary English: plurals and "-ing" or "-ed" forms. Rules written in plain prose rank below rules written in base forms, and on a store that overflows the startup budget the lower rank is the one that gets cut.
Not a goal
What changed (motivation → approach → change)
A lesson's length now counts each of its distinct words once, so a rule written with plurals or "-ing" forms no longer reads as longer than the same rule in base forms.
_lexical_lesson_scoresinvector_memory_runtime/lessons.pydivides by the square root of that count, instead of the size of the row's word-and-stem token set. Matching still runs on the stemmed token set, so only the length term changes. The owning spec paragraph indocs/system-specs/modules/memory-skills-hooks.mdnames the new divisor, and no longer promises that rules which fit within both budgets keep their old order. The test module's docstring names the divisor too.Backwards compatibility
Compatible: no API, config or stored data changes. Only the order of rows on the no-vector ranking path changes, and only between rows whose words differ in how many have a separate stem.
Tests
test/test_lesson_keyword_scoring.py: a newer rule written with inflections ("tests", "pushing") keeps its recency over an older rule with the same number of distinct words when the request matches both. The newer rule repeats one word, so the test fails both on main and with a divisor that counts every word. The existing ranking tests pass unchanged.Manual verification
N/A: unit coverage is sufficient. The change is one term of a pure scoring function, and the new test pins it against main's behaviour.
Related Issues
no linked issue: follow-up to #15161, which added this scorer for #14243.
Pattern harvest
Rule candidate: review-prompt
Pattern: a length normaliser measured on an expanded token set (words plus stems), so the expansion itself is counted as length
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)