Skip to content

perf(memory): bound startup lesson injection with authored tiers - #12431

Merged
iamwhatever merged 1 commit into
mainfrom
perf/memory-startup-budget
Sep 21, 2026
Merged

iamwhatever merged 1 commit into
mainfrom
perf/memory-startup-budget

Conversation

@bolichen97

@bolichen97 bolichen97 commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Why it matters

A new session that says nothing but hi carried every stored lesson.

The startup path computes the lessons cap, passes it, and then ignores it:
get_lessons_context's background branch reads only hard_cap, which is
whatever protected room is left. At a 1M window that room is about 500,000
characters
. So startup injection grew with two things a user never chose: the
number of lessons they had accumulated, and the size of the model window. Moving
from a 200K to a 1M model multiplied it roughly fivefold for a user who changed
nothing. An empty query changed nothing either, because the background branch
reorders by relevance without dropping anything.

The existing code says why it kept everything, and the reason is real:

Extraction provenance cannot distinguish advice from a user's explicit safety
correction.

That is true, and it is why this PR does not try to infer the difference. It
stores it.

What changed

Two blocks, two separate window-independent budgets, split on an authored
applies field.

applies Block Budget
"always" [Learned corrections] the existing lessons cap (22.6% of the 33,000 base)
"on_topic" [Learned experience] a separate, smaller lesson_experience cap (5%)
absent / unrecognized [Learned corrections] shares the rule budget

Separate budgets rather than one shared allowance, so a user with many findings
cannot crowd out their own rules and vice versa. They are not summed into a
larger total, and a wider window enlarges neither. The model-safe protected
ceiling still bounds the two blocks together, and the rule block is served
first, so a finding yields to a rule rather than the two sharing a shortfall.

Authored rules are ordered ahead of unstated rows inside the rule block.
This is not cosmetic. Ranked together on recency, six authored directives all
fell out of the budget behind forty newer untagged rows — caught by a probe
while building this, and now pinned by a mutation-verified test.

Every overflow is announced. Each block's notice names its own
shown/omitted counts, names the budget it hit, states that the limit is a budget
rather than a judgement that the rules stopped applying, and points at
learn_list (rules) or memory_recall (findings). A budget too small for even
one entry still renders the labelled block with that notice, because an empty
block is indistinguishable from "this user has no rules".

The JSONL prune is now tier-aware (findings, then unstated, then authored;
oldest-first within each class). A tier-blind [-200:] deletes the user's
oldest authored rules from disk as cheap findings accumulate, which is worse
than a prompt omission: an omission is announced and readable through
learn_list, a pruned row is gone.

Why applies is authored and never derived

No property of a stored row separates "a rule the user wants enforced in every
session" from "something we worked out once":

  • source does not — the consolidator can extract a real safety correction, and
    a human can type a note about last week's outage.
  • category does not — tool/preference/knowledge describe subject matter.
  • Keyword shape does not — "always" appears in both classes and reliably in
    neither.

A reader that guessed from any of those would re-decide the user's intent on
every turn, and would fail silently in the direction that costs most. So
normalize_lesson_applies accepts only the two literals and raises on a
misspelling at the write path — there is no safe clamp, since one choice injects
a note into every session and the other demotes a standing rule, and both are
silent. The read path never raises: anything unrecognized reads as unstated, so
one hand-edited row cannot abort prompt assembly for every other lesson.

The field is additive and absent when unset, so an unstated row is byte-identical
to what the writer produced before the field existed. On the enrich path it is
write-once like category and repo_scope, so enrichment cannot silently demote
a rule; re-tiering is a delete + re-add.

The compatibility limit, stated rather than papered over

A store whose lessons all predate applies gets its reduction from the budget
alone
, not from classification: every row is unstated, so they all compete for
the rule budget and the overflow notice is what tells the user which ones did not
fit. Classification only starts helping as new lessons are written with a stated
value.

"Retain an unbounded number of unstated rules" and "a small fixed startup budget"
cannot both hold. This is the side that was chosen, and the notice is the
mechanism that keeps the trade visible instead of silent. No existing row is
rewritten, reclassified or deleted.

Measured

Identical synthetic JSONL stores, clean main (df4bf0546a, byte-identical to
this branch's merge-base on every file under test) against this branch.
Characters of rendered lessons. No token figure is claimed: the chars/4
estimate used elsewhere in this repo is not a tokenizer reading, and no
provider-reported usage was captured.

findings stored before after
5 2,234 2,383
60 13,729 2,698
190 41,079 2,712

Before grows with volume; after is flat. At 190 findings that is a ~15x
reduction. The small case is 149 characters larger after, from the second
block's header — reported rather than hidden. Zero embedding calls on both sides
(startup still spends no inference).

Not measured: no real-harness first-turn capture, so this PR makes no claim
about any specific reported context number, and does not claim to have
identified the cause of any particular session's usage. The benchmark also does
not exercise the window-scaled ceiling difference, because the JSONL path takes
the ceiling as its cap directly and both fixtures sat below it.

Tests

test/test_lesson_startup_budget.py, 34 cases: the authored vocabulary
(absent/unrecognized read as unstated, misspelling raises), budget arithmetic
(0 means unbounded and never wins a min, empty tier renders no block, a
too-small budget still reports), and both render paths (vector and JSONL) for
bounded growth, authored-before-unstated ordering, legacy rows served as rules,
announced overflow, and byte-identity for a caller naming no budget.

Mutation-verified, both ways:

  • reverting authored + unstated to unstated + authored fails
    test_an_authored_rule_outranks_newer_unstated_rows;
  • restoring the tier-blind updated[-_MAX_LESSONS_TOTAL:] fails
    test_saving_past_the_cap_through_the_store_keeps_authored_rules.

That second case exists because the direct-helper tests passed under the
reverted prune — they called _prune_to_total and left the store's own choice
of pruner unpinned. The route-level case is what actually guards the fix.

Two pre-existing tests changed, keeping every structural assertion and updating
only the overflow wording they pin; one (test_over_cap_preserves_complete_explicit_rules)
had its "all N rules appear" claim replaced with the invariant that still holds
and matters — trimming is by whole entry, so no partial rule is ever emitted.
Its presence check was also anchored on the rendered entry, because the fixture
rules are prefix-ambiguous (0-xxx… is a substring of 10-xxx…) and a bare
in reported rules as present that were never emitted.

Gates: black, isort, flake8, subprocess-encoding, docs-lint, comment-history all
clean. mypy reports 215 errors on this branch and 215 on a clean-main control —
zero introduced. 712 related backend tests pass.

Known gaps, deliberately not in this PR

  • Consolidation states no tier. learn_add, POST /api/lessons and the CLI
    now pass applies, but the consolidator does not: deciding a tier per extracted
    item is a judgement about the user's intent, not a mechanical passthrough, and it
    needs its own change to the extraction prompt. Its rows stay unstated, which is
    the safe rule-tier path.
  • The protected ceiling is still window-scaled. Making it window-independent
    would also bound preferences, a separate behaviour change to a protected
    document with its own truncation semantics.
  • The budgets are constants, not config. Each overflow notice names the limit
    it hit, so the constraint is visible; tuning is a follow-up.
  • Within the unstated class, user_explicit rows are not ordered ahead of
    extracted ones.
    Design suggested it and I think it is right — it is ordering
    bias, not the inference this PR rejects. Held back only to keep this diff to one
    reviewable change; say the word and it goes in.

Smoke test — scale sweep, relevance, and no-regression

Reproducible harness, run against clean main and this branch with the same
fixture
, so every cell is a before/after pair. Budgets are read from
context.py rather than restated, so the harness cannot drift from shipped values
(rules 7,458 chars, findings 1,650).
Characters and operation counts only — no token figure is derived, because the
chars/4 estimate in this repo is not a tokenizer reading.

Per scale the harness plants, in one store: 3 authored standing rules, 1 finding
naming a rare topic the query also names, and filler naming unrelated topics. The
relevant finding is planted early on purpose — an old-but-relevant row is
exactly what a newest-first budget drops. It then measures the startup render, the
explicit-reader render (what memory_recall / learn_list reach), and how many
embedding calls startup made.

Vector store (the primary tier)

lessons asked kept on disk chars before chars after rules kept relevant finding injected recallable embeds
0 0 0 0 0/0 → 0/0 n/a → n/a n/a → n/a 0
1 1 209 209 1/1 → 1/1 n/a → n/a n/a → n/a 0
5 5 423 579 3/3 → 3/3 yes → yes yes → yes 0
25 25 1,438 1,594 3/3 → 3/3 yes → yes yes → yes 0
100 100 5,413 1,904 3/3 → 3/3 yes → yes yes → yes 0
199 199 11,135 1,900 3/3 → 3/3 yes → yes yes → yes 0
500 500 28,593 1,900 3/3 → 3/3 yes → yes yes → yes 0
2000 2000 120,573 1,939 3/3 → 3/3 yes → yes yes → yes 0

JSONL fallback tier

lessons asked kept on disk chars before chars after rules kept relevant finding injected recallable embeds
0 0 0 0 0/0 → 0/0 n/a → n/a n/a → n/a 0
1 1 185 185 1/1 → 1/1 n/a → n/a n/a → n/a 0
5 5 399 548 3/3 → 3/3 yes → yes yes → yes 0
25 25 1,414 1,563 3/3 → 3/3 yes → yes yes → yes 0
100 100 5,389 1,873 3/3 → 3/3 yes → yes yes → yes 0
199 199 11,111 1,869 3/3 → 3/3 yes → yes yes → yes 0
500 200 11,744 1,907 0/3 → 3/3 no → no no → no 0
2000 200 12,744 1,891 0/3 → 3/3 no → no no → no 0

Verdict

Regressions: none, across all 16 scale × store combinations.

  • Bounded. 2,000 lessons costs 1,939 chars instead of 120,573 — a 62x
    reduction, and flat from 100 lessons upward. Before, the cost tracked the pile.
  • Standing rules always arrive, 3/3 at every scale on both tiers.
  • The task-relevant finding still arrives at every scale where it is on disk,
    and is recallable through the explicit reader where it is not injected.
  • Zero embedding calls at startup, every row, both tiers. The budget is not
    paid for with inference.
  • Small stores are barely affected — 5 lessons costs 149 chars more (the second
    block's header), reported rather than hidden.

Two things the sweep found that the earlier probes did not

1. A real regression this PR introduced, now fixed. At 100 and 199 lessons the
JSONL tier stopped injecting the relevant finding: that tier has no relevance
score, so newest-first dropped an old relevant row while newer irrelevant ones
fitted. The vector tier was unaffected (it ranks against the request already).
Fixed by ordering the FINDINGS tier by lexical overlap with the request
(order_findings_by_relevance — word tokens, no stemming, no embedding). Rules are
deliberately not reordered: a rule applies whatever the message is about, so
scoring one against the message is the inference this PR rejects. Pinned by three
tests and mutation-verified.

2. A pre-existing data-loss bug on main that this PR fixes. At 500 and 2,000
lessons, main keeps 0 of 3 standing rules — the tier-blind [-200:] prune
deletes them from disk, because they are the oldest rows. This branch keeps 3/3.
That is disk loss, not a prompt omission: on main those corrections are gone and
learn_list cannot show them.

Harness limitations, stated

The fixture needed two corrections worth recording, because both silently made an
earlier run measure nothing. Filler sharing a sentence frame cleared
write_lesson's 50% keyword-overlap dedup, so 25 requested lessons became 5 stored
and the vector sweep was vacuous; every filler token now carries its index. And the
harness initially omitted query_text on the JSONL path, which is not what
context.py passes — it reported the relevance regression as unfixed after the fix
had landed. There is no real-harness first-turn capture here, so this PR still makes
no claim about any specific reported context number.

Review round 2 — what changed

Three findings were legitimate defects my own diff introduced, so they were fixed
rather than rebutted (my standing override does not cover a defect the PR itself
exposes):

  • GPT F1, learn.py — at the 200-row cap a new on_topic row is the first
    eviction candidate, so the prune could drop the submission while save still
    returned "inserted". A write that did not land now returns "refused".
  • GPT F2, vector_memory.py — the dedup scan's three delete branches decide on
    text alone, so a longer overlapping on_topic write could tombstone an authored
    always rule. Added a tier guard on the delete direction only: an on_topic
    write may not retire an always row, an always write still may retire a
    finding (promoting guidance to a standing rule is the direction the user asked
    for), and a covered on_topic submission is still declined as
    substring_covered.
  • GPT finding 4, JSONL serialization — asdict emitted "applies": null on
    every row, rewriting legacy lines on the next save and making the two stores
    disagree about what absence means. _serializable now drops the key when unset.

Also fixed GPT finding 3: the findings-tier overflow notice now carries {limit}
and the same budget-not-judgement sentence the rule notice has, which is what the
spec promises for every overflow.

First Principles item 2 (zero consumers) is addressed: applies now has three
production writers — the learn_add MCP tool (enum in its schema, plus a
description telling the model to decide from what the user actually said rather
than from wording), POST /api/lessons, and the CLI reaching that route. The tier
is reachable in production, so the machinery is no longer inert.

First Principles item 1 (no RFC) is a maintainer decision and is not mine to
make.
It objects that this PR reverses a shipped default — keep every eligible
rule until the window-scaled ceiling — with no RFC covering it. That is a fair
reading: the base spec pinned that sentence and this diff rewrites it. Drafted
override reason for a maintainer who agrees the bounded budget is the right
default:

/ai-review override first-principles <head-sha>: the keep-every-rule default is a measured cost regression, not a product direction — startup injection scaled with both lesson count and model window (41,079 chars at 190 findings, 5x at a 1M window), the reversal is bounded and fully reported in-prompt, and no stored row is rewritten or deleted; an RFC would document a bug fix.

New tests: the prune-refusal path, the three tier-guard directions, and JSONL
serialization. All three fixes are mutation-verified (reverting each one fails its
own test). mypy is back at control parity (215 on both this branch and clean main).

@bolichen97
bolichen97 requested review from a team and chenmingwei23 September 21, 2026 02:50
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Sep 21, 2026
@bolichen97
bolichen97 force-pushed the perf/memory-startup-budget branch from 36247eb to b6589f1 Compare September 21, 2026 02:52
@github-actions

github-actions Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of 13dab93603a249fc4591e8162f9f8e88aac826af and found no blocking issues.

This comment is updated in place on each push.

Review details

FINDING -- docs/system-specs/modules/memory-skills-hooks.md:2779 -- "the CLI reaching that route" is false; CLI writes directly without applies, producing an unstated rule -> Fix: remove the CLI claim.
[GPT-REVIEWED] 13dab93

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 13dab93603a249fc4591e8162f9f8e88aac826af: <one-sentence reason>

@github-actions

github-actions Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — 🟡 CONCERNS

Design-level review of 13dab93603a249fc4591e8162f9f8e88aac826af — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

Spec-pinned retain-every-rule startup default is reversed; the diff cannot own that decision — only an explicit maintainer override can.

Watch

The old pin (test_over_cap_preserves_complete_explicit_rules asserted every rule appears below the ceiling) is rewritten to a bounded-budget-with-full-accounting invariant. The PR handles it correctly — treats the pin as a decision, keeps the two-stage contract for callers naming no budget, measures the cost (41,079 chars at 190 findings), and drafts the override — but the reversal is a product-default change this review cannot approve on the author's behalf. Consequence is reversible (additive field, no row rewritten, revert restores old behavior).
Clears when: a maintainer records the drafted /ai-review override first-principles 13dab936… (or equivalent explicit acceptance of the bounded default).

Suggestions

Take the deferred user_explicit-before-extracted ordering within the unstated class (offered in "Known gaps") as the immediate follow-up: until the consolidator states tiers, every store is effectively all-unstated and that ordering is the only lever protecting legacy explicit corrections inside an overflowing rule budget.

[DESIGN-REVIEWED] 13dab93

@github-actions

github-actions Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — ✅ human override accepted

@bolichen97 overrode this lane for 13dab93603a249fc4591e8162f9f8e88aac826af via /ai-review override (this commit only).

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 21, 2026
@github-actions

github-actions Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Opus 4.8 Review — ✅ no blocking findings

Reviewed 13dab93603a249fc4591e8162f9f8e88aac826af — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 13dab93

Verdict parsed from the review's SHA-scoped output markers for commit 13dab93603a249fc4591e8162f9f8e88aac826af.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 13dab93603a249fc4591e8162f9f8e88aac826af: <one-sentence reason>

@bolichen97
bolichen97 force-pushed the perf/memory-startup-budget branch from b6589f1 to 3162cce Compare September 21, 2026 03:11
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 21, 2026
@bolichen97
bolichen97 force-pushed the perf/memory-startup-budget branch from 3162cce to 89c7080 Compare September 21, 2026 03:21
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 21, 2026
@bolichen97
bolichen97 force-pushed the perf/memory-startup-budget branch from 89c7080 to 82861b6 Compare September 21, 2026 03:46
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 21, 2026
@bolichen97
bolichen97 force-pushed the perf/memory-startup-budget branch from 82861b6 to f65cd86 Compare September 21, 2026 04:13
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 21, 2026
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 21, 2026
@bolichen97

Copy link
Copy Markdown
Contributor Author

Answering the subtraction proposal on its merits, separately from the RFC blocker.

The premise is correct and I said the same thing myself: on a store whose rows all predate the field, the reduction comes from the budget alone. Wiring the allowance into the background branch does deliver the 41,079 -> 2,712 number without any of the tier apparatus.

It is not sufficient, and the reason is what the tier is for. A budget without it truncates rules and findings as ONE pool. Then:

  • The rows that fall out are chosen by recency or by overlap with the current message. A standing rule the user was most explicit about is exactly the row an accumulating pile of findings displaces, and on this repo's own store that is measurable: six authored directives all fell outside a 7,458-character budget behind forty newer untagged rows.
  • The 200-row disk prune stays tier-blind, so findings accumulating on disk evict the user's oldest authored rules permanently. A prompt omission is announced and readable through learn_list; a pruned row is gone.
  • write_lesson's dedup scan and the route's contradiction sweep both decide on text alone, so a finding can retire or decline a standing rule. Without a tier there is nothing to gate either on.
  • A greeting still spends the whole allowance, because "which of these is filler" is not answerable without knowing which rows are findings.

So the apparatus is not decoration on top of the budget; it is what makes the truncation safe rather than arbitrary. Shipping the budget alone would deliver the measured character reduction and, on the same stores, start silently choosing which of the user's explicit rules to drop. That is a worse default than the one being replaced, not a smaller one.

The honest counterweight, and it is real: on an all-legacy store the tier helps nothing today, because every row is unstated. The reduction there is the budget's, and classification only starts paying as new lessons are written with a stated value. That limit is in the spec rather than argued away, and no stored row is rewritten, reclassified or deleted to reach it.

Items 2, 3 and 4 are named as components of the same decision, which is accurate — they are the safety half of it, not separable features. Item 7 (refuse-at-capacity) is the one I would defend independently of the shape: reporting inserted for a row the prune removed is a false success on a memory write whatever the tier policy is.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 21, 2026
@bolichen97

Copy link
Copy Markdown
Contributor Author

/ai-review override first-principles 13d96af: keep-every-rule was a measured cost regression, not product direction. Bounded, reported in-prompt, no stored row rewritten or deleted.

@github-actions

Copy link
Copy Markdown
Contributor

Human judgment recorded

@bolichen97 marked the first-principles AI finding as false positive, not applicable, or explicitly accepted for 13d96af3b1ca04287108d4c4bfffd8bea2cc080a.

keep-every-rule was a measured cost regression, not product direction. Bounded, reported in-prompt, no stored row rewritten or deleted.

This decision applies only to this commit. A new push requires a new judgment.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 21, 2026
@bolichen97

Copy link
Copy Markdown
Contributor Author

Answering the Design review's two Watch items and its suggestion.

Watch 1 (maintainer acceptance of the bounded default) — applied on this head: the first-principles override is on record for 13d96af3b1ca04287108d4c4bfffd8bea2cc080a, authorized by the maintainer for exactly this already-adjudicated item.

Watch 2 (consolidation states no tier) — filed as #12495, with the reason it was not done here: deciding a tier per extracted item is a judgement about what the user meant, and the field's whole basis is that it is authored rather than derived. source does not answer it, category describes subject matter, and keyword shape appears in both classes. So the consolidator needs a deliberate way to STATE a tier, which is its own change with its own tests.

Suggestion (user_explicit before extracted, inside the unstated class) — declined here, and for a structural reason rather than cost. The JSONL Lesson carries no source field (ts, rule, category, negative, repo_scope, applies), so this ordering is only implementable on the vector path. Doing it on one store is exactly the cross-store divergence this PR's previous round existed to remove: the two stores had ordered their rule tiers by opposite policies and each documented its own as the rule, which cost an exact-topic match on a legacy JSONL store. Reintroducing that asymmetry to gain an ordering nudge is the wrong trade.

Doing it consistently means adding source to the JSONL schema and wiring it through every JSONL write path — a second schema field, wider than the ordering it enables, and squarely inside the shape question the first-principles lane is already weighing. #12495 is also the better lever on the same residual: it gives on_topic its main producer, which separates extracted advice from authored rules outright instead of ranking them against each other.

I agree the suggestion is the same KIND of move as authored-first, and that it points at a real residual. It belongs with #12495 rather than ahead of it.

@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Sep 21, 2026
A bare "hi" carried every stored lesson. The startup path computed the
lessons cap, passed it, and then ignored it: get_lessons_context's
background branch reads only hard_cap, which is the remaining protected
room. At a 1M window that room is about 500,000 characters, so startup
injection grew with both the lesson count and the model window, and an
empty query changed nothing because background ranking reorders without
dropping.

Split startup lessons into two blocks with two separate,
window-independent budgets, on an authored `applies` field: "always" is a
standing rule under the existing lessons budget, "on_topic" is a past
finding under a separate smaller one. A row with no value reads as
unstated and is served as a rule, because demoting a real standing rule
is the costlier mistake. Authored rules are ordered ahead of unstated
rows within the rule block; ranked together, a crowd of untagged rows
displaces exactly the rules the user was most explicit about. Every
overflow names its counts, names the budget, says the limit is a budget
rather than a judgement, and points at learn_list or memory_recall.

`applies` is authored and never derived. No stored property separates a
standing rule from a past finding: the consolidator can extract a real
safety correction, a human can type a note about an outage, category
describes subject matter, and "always" appears in both classes. So the
write path accepts only the two literals and raises on a misspelling,
while the read path treats anything unrecognized as unstated so one bad
row cannot abort prompt assembly.

Also make the JSONL prune tier-aware. A tier-blind [-200:] deletes the
user's oldest authored rules from disk as cheap findings accumulate,
which is worse than a prompt omission: an omission is announced and
readable through learn_list, a pruned row is gone.

Measured on identical synthetic JSONL stores, clean main against this
branch, characters of rendered lessons (no token figure is claimed):
5 findings 2,234 -> 2,383; 60 findings 13,729 -> 2,698; 190 findings
41,079 -> 2,712. Before grows with volume, after is flat. Zero embedding
calls on both sides.

Compatibility limit, stated rather than papered over: a store whose rows
all predate `applies` gets its reduction from the budget alone, and the
overflow notice is what keeps that trade visible. No existing row is
rewritten, reclassified or deleted.
@bolichen97

Copy link
Copy Markdown
Contributor Author

/ai-review override first-principles 13dab93: bounded startup lesson budgets replace the retain-every-rule default deliberately; the cost is measured (41,079 chars at 190 findings), every omission is announced with counts and a tool to read the rest, callers naming no budget keep the two-stage contract, and the change is additive and revertible.

@github-actions

Copy link
Copy Markdown
Contributor

Human judgment recorded

@bolichen97 marked the first-principles AI finding as false positive, not applicable, or explicitly accepted for 13dab93603a249fc4591e8162f9f8e88aac826af.

bounded startup lesson budgets replace the retain-every-rule default deliberately; the cost is measured (41,079 chars at 190 findings), every omission is announced with counts and a tool to read the rest, callers naming no budget keep the two-stage contract, and the change is additive and revertible.

This decision applies only to this commit. A new push requires a new judgment.

@buluoray buluoray left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified at head 13dab936. No code defect — the mechanism is careful and I could not break it. I am not approving, because the thing this PR decides is a default that Design Review already said only a maintainer can own, and I can put a number on what it costs. That number is at the end.

What I verified rather than took on trust

  • Tier assignment is authored, never derived. normalize_lesson_applies accepts only always/on_topic and raises on anything else rather than clamping; the read side never infers a tier from category, ts, source or wording. I specifically checked the consolidation cron, since that is where a derived tier would sneak in: _candidate_applies only reads the field, and api_lessons_create takes applies from its caller. Ordering within a tier is derived (relevance, newest-first as a stable tiebreak), which is the right split.
  • The omission is structurally un-silenceable. render_lesson_tier's contract is that the notice is never dropped to fit: when there is not room for one entry it still renders header + notice + footer, because "a silently absent block is indistinguishable from this user has no rules". That is the failure mode I went looking for, and it is the one thing the function is built to always be able to produce.
  • One huge lesson cannot starve the rest — the selection loop continues past an oversized entry instead of breaking, so smaller and more relevant rows still render.
  • No silent zero. The experience tier gets max(1, hard_cap - len(directive_block)) specifically so a directive block that ate the ceiling cannot hand it an unbounded budget. Worst case is a labelled "omitted all N" block.
  • Drop order is load-bearing, mutation-confirmed. Reversing authored + unclassified to unclassified + authored in a scratch copy reddens exactly the two precedence pins and nothing else. The truncation tests are not vacuous either — the 80-finding case asserts 0 < kept < 80 plus the notice, and the fixtures run well over budget.

The number

_LESSONS_CAP = _budget(0.226) = 7,458 chars, window-independent. On main the rule block was returned complete up to caps.protected_context, whose floor is _CONTEXT_BUDGET_BASE * 3 = 99,000 and which scales with the window above that. The four startup renderers now pass directive_budget=caps.lessons unconditionally, so the "complete below the ceiling" path survives only for a caller passing 0 — which the real path never does.

So the default for the standing-rule block goes from at least 99,000 chars to 7,458, a floor-to-floor reduction of more than 13x, and more on a large window.

And the mechanism that makes that cut safe is empty on day one. applies is new, so every existing lesson in every existing store is unstated. They therefore all land in the same tier, the tiering protects nothing yet, and the budget simply cuts. The authored tier only starts paying once rules are tagged — and cli.py's learn add has only rule, --category and --negative, with no --applies (GPT flagged the doc claiming otherwise; I confirmed the parser). The learn_add MCP tool does expose it. So a user who writes corrections from the CLI cannot put them in the protected tier at all.

To make this concrete rather than hypothetical: the lessons block injected into my own session right now already reports omitting 777 lessons at the current 99,000-char ceiling. After this change that surviving set is bounded at 7,458 chars — a small fraction of what the same store surfaces today, on the first boot after merge, with nothing tagged.

Why that is your call and not mine

Everything mitigating it is real and I verified each one: the loss is loud rather than silent, survivors are relevance-ordered (the JSONL path gains order_by_request_relevance here, which it did not have), no stored row is rewritten or deleted, applies is absent when unset, and it is a one-line revert. The PR body states the tradeoff openly and does not hide the measurement.

But this reverses a spec-pinned "retain every eligible in-scope rule" default, Design Review said in as many words that only a maintainer can own that, and the First Principles lane was cleared by the author's own override rather than by the lane. A reviewer approving on mechanism correctness would be deciding the default by omission — which is the specific thing I want to avoid doing to you here.

Two things that would change my answer, either one on its own: a floor that keeps the block complete below the old ceiling when nothing is tagged yet (so the cut arrives with the tiering rather than before it), or an explicit decision from you that 7,458 is the intended startup allowance and the omission notice plus memory_recall is the intended recovery path.

Unrelated and non-blocking: the body says the test file has 34 cases; it defines 48. Understated, harmless.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants