Repository navigation
fix: name the healthy signals on a mixed run, and drop the concessive still - #114
Merged
Merged
Conversation
code-quality-metrics-11ib (first of two problems): "It still flags" framed a healthy majority as a concession being walked back. Replaced with "It also flags", stating both facts plainly without the concessive. TDD: test "does not use the concessive word 'still' when a mixed run has more healthy signals than warnings" predicted RED against the literal "still flags" wording; confirmed red, then fixed with a one-word swap. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
code-quality-metrics-11ib (second, larger problem): the concern branch of renderTopSummary already names its worst finding by label, value and bar via ledByPhrase; the healthy side collapsed every good finding into the bare word "Most". A reader of a mixed report got one specific fact and one vague generality. Adds goodSignalMargin: how far a banded-good entry sits inside its own healthy band, as a fraction of the healthy boundary itself. This is the natural symmetry to how the concern side is already ranked, but scaled differently on purpose: every currently-calibrated band is two-band (fixed concern -1 regardless of value), so the concern formula itself carries no usable magnitude for a healthy value; scaling by the healthy boundary needs no critical bound and works for two-band and three-band entries alike. The healthy-dominant mixed branch now states the actual banded-good count and names its top 2 signals by that margin (describeStrongestHealthySignals), reusing describeGoodSignalPhrase so a healthy signal reads identically in both this branch and the all-healthy branch. This is a display-selection heuristic only; it does not touch which branch fires (that stays entirely bandedGoodCount/bandedConcerns.length, per the brief's constraint against inventing a new score to decide tone). TDD: test "names specific healthy signals with their own value and bar, not the word Most" predicted RED against the literal "Most metrics" wording; confirmed red, then implemented the margin/selection/join helpers and wired them into the headline. Also updates the pre-existing "never prints a number that does not trace to the catalog" fabrication-safety test to whitelist the new bandedGoodCount tally, the same way it already whitelists criticalCount/warningCount: it is the identical class of provably-correct catalog tally, now printed for the first time. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ming fix code-quality-metrics-11ib's brief requires explicit coverage of all four shapes: all healthy, mixed with one warning, mixed with several warnings, all concern. The first two are covered by the two prior commits' own tests; this adds the third (5-ish good, 2 warnings, mirroring the brief's measured daloopa shape). NOT a red-green cycle: this test passed immediately on arrival, with no production change, because the prior commit's fix was already written generally (any bandedGoodCount > bandedConcerns.length case), not scoped to exactly one warning. Recorded here rather than presented as a clean TDD cycle, per the working agreement. The two forced warnings (test_coverage_rate, p90_files_changed) are chosen so buildMetricCatalog's concern-descending sort's tie-break (both are two-band, so both carry the same fixed concern -1; ties preserve original catalog push order) deterministically picks test_coverage_rate as the named "led by" concern, matching the brief's own quoted wording. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
code-quality-metrics-11ib design decision: the all-healthy sentence (describeAllHealthy) named every banded good signal, one clause per metric. Honest, but unbounded -- the brief flagged this as the same scaling problem the mixed branch's own "Most"/"still" fix addressed, just not yet triggered at six clauses. Re-pointed at the same describeStrongestHealthySignals(..., 2) selection the mixed branch now uses (furthest inside its own band, by goodSignalMargin), so both branches read consistently: one selection rule, not one branch enumerating everything and the other naming a fixed few. The total count is still stated in the sentence, so a reader still knows how much evidence exists even though only the top 2 are named individually. TDD: rewrote the existing test (previously asserting all 6 labels present) to assert the new expected shape: total count "6 metrics" stated, top 2 by margin (Test/prod co-change, Sprawling commits) named, and the weakest signal (Large commits) explicitly absent, proving this selects rather than still enumerating everything by coincidence. Predicted and confirmed RED against the unmodified describeAllHealthy (missing "6 metrics"), then implemented the change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
PR AnalysisSize: small (based on production code) Strengths
Commit AnalysisTotal Commits: 4
Commit Details8c272ad Ken Judy (8/20/2026) 513fea5 Ken Judy (8/20/2026) bbc4f0e Ken Judy (8/20/2026) 1083c89 Ken Judy (8/20/2026) Test CoverageTest-to-Production Ratio: 1.24:1 (no calibrated target for this ratio; reported descriptively) DORA Capability AssessmentArchetype: harmonious-high-achiever
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes code-quality-metrics-11ib (P1).
PR #105 fixed the rarest case
It gave renderTopSummary an affirmative path for the all-healthy run only. Five of the six evaluation reports are mixed, so the fix missed nearly every report actually read.
Before, on a 6-good 1-warning run:
"Most metrics scored against a calibrated threshold in this run are healthy. It still flags 1 warning signal, led by Duplication density at 5.12 (warning)."
Six healthy results collapsed into the word "Most" while the single warning received a name, a value and a verdict. A reader got one specific fact and one vague generality.
After:
"5 metrics scored against a calibrated threshold in this run are healthy, led by Test/prod co-change at 55 (healthy at or above 23) and Sprawling commits at 8 (healthy at or below 18). It also flags 1 warning signal, led by Uncovered production at 15 (warning)."
And "still" is gone. A single concessive adverb framed the healthy majority as something being walked back; "also" states both facts without ranking one as an excuse for the other.
The selection rule, and why the existing one could not be reused
goodSignalMargin(entry) measures how far a good entry sits inside its own healthy band as a fraction of that boundary, so the top 2 are named. The obvious move would have been to reuse the concern field the warning side already ranks by, but that field is fixed at -1 for every good entry, because every calibrated band is currently two-band. It carries no magnitude at all for healthy entries. Scaling by the healthy boundary needs no critical bound and works uniformly across two-band and three-band entries.
Top 2 rather than all, and the same rule now applies to the all-healthy branch too, so the two read consistently instead of one enumerating everything and the other naming a fixed few. The all-healthy sentence previously listed all seven signals and ran long; it now names its strongest two and states the total count.
Sentences for all four cases, verified by running the tool rather than by reasoning
all healthy "All signals are positive, indicating this run commits sit within the healthy range of the reference benchmark on every calibrated metric measured (6 metrics): led by Test/prod co-change at 55 (healthy at or above 23) and Sprawling commits at 8 (healthy at or below 18)."
mixed, 1 warning "5 metrics scored against a calibrated threshold in this run are healthy, led by Test/prod co-change at 55 ... It also flags 1 warning signal, led by Uncovered production at 15 (warning)."
mixed, several "4 metrics scored against a calibrated threshold in this run are healthy, led by Sprawling commits at 8 ... It also flags 2 warning signals, led by Test/prod co-change at 12 (warning)."
all concern "This run flags 1 critical signal, led by Large commits at 40 (critical)." (pre-existing, unchanged)
Constraints held: no warning softened, hidden, reordered away or downgraded; a mixed run still reads as mixed and still names its worst finding with value and verdict; no new score decides the tone.
Fabrication guard extended rather than bypassed
A pre-existing test asserts the top summary never prints a number that does not trace to the catalog. The new bandedGoodCount tally had to be whitelisted, in the same class as the existing criticalCount and warningCount: a count of catalog entries rather than a derived figure.
Verification
714 tests pass across 41 suites, lint and typecheck clean. One of the four cycles is reported in its commit message as not a red-green cycle, because it passed on arrival once the previous fix was written generally, rather than being presented as clean TDD.
lib/narrative.js needed no change. Its prompt rule 1 already instructs symmetric specificity for positive findings; the asymmetry was specific to the deterministic template.
Known gap, left deliberately
The concern-dominant branch, used when concerns are greater than or equal to healthy entries, still names no healthy signals even when some exist. None of the six evaluation reports exercises that shape, since all have more healthy entries than warnings, so it was left rather than changed against an unmeasured case. Worth revisiting if a report ever lands there.
🤖 Generated with Claude Code