Repository navigation
docs(metrics): the occupancy pair comes from the fullest turn by ratio, not the largest used - #16674
Conversation
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing to check. [DESIGN-REVIEWED] d5e3401 Description read[DESCRIPTION-READ] def84cca4b12dbb6927610a3ac93852a77475e444deed179c937949b359bb571 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
1fbee86 to
4d47927
Compare
…o, not the largest used The spec described peak_context_used / context_window as coming from the turn/completed that reported the largest occupancy. context_trace picks the highest used / used_window ratio instead, so once a model switch moves the window the two name different turns: 90k of a 100k window is fuller than 200k of a 1M one. The code is the correct half -- SessionBreakdownTree divides the pair into a fill fraction, so the panel wants the fullest turn. Name the ratio in the spec, write out the model-switch case, and state the two properties the comparison already has: integer-exact cross-multiplication, and a reading with no stated window is not a candidate. Also resolve the overloaded word: occupancy means a ratio at the context block, so the unit statement now applies to the occupancy pair rather than to occupancy itself.
4d47927 to
d5e3401
Compare
iamwhatever
left a comment
There was a problem hiding this comment.
Reviewed: docs-only. Checked the new wording against the code at this head: dashboard/handlers/usage.py:869-886 picks the occupancy peak by the highest used/used_window ratio via cross-multiplication (used * peak_window >= peak_used * row_used_window), and a row with no stated used_window (row_used_window <= 0) is never a candidate. The spec now says exactly that, and the 90k/100k vs 200k/1M example shows why ranking by absolute used was wrong. CI green.
Problem / Motivation
Goal: The metrics spec describes the occupancy pair the Session Breakdown tree draws the way the code actually picks it, so a reader cannot "correct" working code into a wrong fill ratio.
docs/system-specs/modules/metrics.mdsays the occupancy pairpeak_context_used/context_windowcomes from theturn/completed"that reported the largest occupancy".
context_traceinsrc/kiro_crew/dashboard/handlers/usage.pypicks the turn with the highestused/used_windowratio, compared by cross-multiplication.While every turn in the window shares one context window the two readings name the
same turn. A model switch moves the window, and from that point they name different
turns: 90k used against a 100k window is 90% full, 200k used against a 1M window is
20% full. Read as the largest absolute
used, the spec's sentence crowns the secondturn; the code crowns the first.
The same paragraph also used the bare word occupancy for a token count, while line
856 of the same document defines it as "a per-session ratio". That is what made the
sentence readable two ways.
Why it matters
Nothing on the dashboard is wrong today, because the code's choice is the correct one.
The cost falls on whoever reads the spec next.
website/src/pages/SessionBreakdownTree.tsx:100divides the pair into a clamped 0..1fill fraction, so the Session Breakdown tree is asking for the fullest turn. A reader
who takes the spec at face value and "corrects" the ratio comparison to an absolute one
breaks that fill ratio: the panel would draw a 20%-full turn as the session's peak and
hide the turn that was actually about to compact. A spec sentence that reads as a defect
report against working code is an invitation to introduce the defect.
Not a goal
context_traceis correct and is not touched.(listed under Tests); a new one would assert what they assert.
contextblock's own "ordered by peak occupancy" wording. Thatsurface ranks by
peak_pct, a ratio, so it was already accurate.change, noted at the bottom.
What changed (motivation -> approach -> change)
Observed symptom: the spec and
context_tracename different turns as the peak once amodel switch moves the context window.
Root cause: the word occupancy carries two quantities in this one document. At line
856 it is "a per-session ratio"; in this paragraph it was a token count. The sentence
"the turn that reported the largest occupancy" is therefore true under one definition
and false under the other, and nothing in the sentence said which one applied.
The code is the correct half, so the change is in the document.
The sentence now says the pair comes from the FULLEST turn inside the window and names
the quantity: the highest
used/used_windowratio. It then says why it is the ratioand not the largest absolute
used, with the 90k/100k versus 200k/1M case written out,so the next reader meets the model-switch trap in the spec instead of discovering it in
the code. Two properties the implementation already has are stated alongside it: the
comparison is integer-exact by cross-multiplication, and a reading whose window the
provider never stated has no ratio and so is not a peak candidate at all.
The overloaded word is resolved in the same paragraph: "occupancy is tokens" becomes
"the occupancy pair is tokens". Occupancy keeps its ratio meaning throughout the
document, and the unit statement now applies to the named pair rather than to the
concept.
Backwards compatibility
Compatible: the diff is one prose paragraph in a specification document. No code, no
schema, no payload field and no test changes, so nothing that works today stops working.
Removes nothing: no path is hidden, no record or row deleted, no validator tightened, no
data migrated or pruned, and no key or public function removed or renamed. The sole
changed file is
docs/system-specs/modules/metrics.md.Tests
No new test. The behaviour the spec now describes is already locked in by
test/metrics/test_context_trace.py:test_peak_is_the_fullest_ratio_not_the_largest_used_across_a_switch-- two turns ondifferent windows, 90k/100k versus 200k/1M; requires the 90% turn's pair and a
rendered ratio of 0.9. This is the exact case the spec sentence got wrong.
test_the_occupancy_pair_comes_from_one_turn-- the fullest turn on the small window,a later emptier turn on a large one; requires the peak's own window to travel with it
rather than the newest configured size.
test_peak_keeps_the_later_turn_among_equally_full_readings-- the>=tie-breakkeeps the later of two equally-full turns across a switch.
test_a_reading_whose_window_is_unknown_is_not_a_peak_candidate-- a windowless 80kreading loses to the valid 40k/100k pair, so no peak is reported over a zero window.
Run locally from that file: 4 passed, 53 deselected.
Manual verification
N/A -- documentation-only diff with no runtime surface. Verified instead by reading the
implementation against the new wording and by the repository's own documentation gates,
all run locally on this diff:
scripts/docs_lint.py --testscripts/docs-lint.shscripts/check_comment_history.py --testscripts/check_comment_history.pyon this diff.github/scripts/take_away_check.pyon this diffflake8onhandlers/usage.pymypyonhandlers/usage.pyRelated Issues
Fixes #16671
Found by Opus review on #15367 (merged).
Pattern harvest
Rule candidate: review-prompt
Pattern: a specification sentence states a selection rule for a pair of values without
naming the quantity it ranks by, while the document defines that quantity two ways
elsewhere -- so the sentence is true under one reading and false under the other, and
the code is the only place the real rule is written down.
Worth considering separately, and not in this PR: the spec states the ratio rule in
prose and the code states it in a comment, with nothing tying the two together. A
docs-lint rule pinning a named invariant sentence to the function implementing it would
catch the next drift of this kind.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)