Skip to content

fix: deleted files should not pin the strict score - #751

Open
ThomFish wants to merge 1 commit into
peteromallet:mainfrom
ThomFish:fix/deleted-files-should-not-pin-strict-score
Open

ThomFish wants to merge 1 commit into
peteromallet:mainfrom
ThomFish:fix/deleted-files-should-not-pin-strict-score

Conversation

@ThomFish

Copy link
Copy Markdown

Problem

auto_resolved fails strict on purpose — a finding that merely stopped appearing may have been hidden rather than fixed, and strict is the mode that refuses to take that on trust.

But verify_disappeared auto-resolves for three different reasons and records them all as the same status:

if zone_map and file_path and should_skip_issue(zone_map, file_path, detector):
    previous["note"] = f"Auto-resolved: zone policy now skips {detector} for this file"
if file_deleted:
    previous["note"] = "Auto-resolved: source file no longer exists"
if detector in confirmed_detectors:
    previous["note"] = "Auto-resolved: absent from latest detector output"

The first two are not inferences. The file is checked against the filesystem; the zone is read from configuration. The tool establishes both for itself, and neither leaves anything a person could fix and attest — so the finding can never reach fixed, and never stops counting against strict.

The effect is that strict decays as a codebase deletes dead code, and no amount of work recovers it. Deleting a dead module is a good act that permanently lowers your score.

On the repository this was found on: 242 of 1553 strict failures (16%) are verified auto-resolutions. In Test health alone it is 130 of 631, which had held that dimension at 0.0 strict for eleven consecutive scans — 124 of them naming files that no longer exist. The dimension was unreachable by any amount of test-writing.

Fix

Record why an issue auto-resolved, and exempt from strict only the two kinds the tool verified itself.

  • resolution_kind ("file_deleted" / "zone_policy") is set at the two verified branches; the inferred branch explicitly sets None.
  • is_failure(issue, mode) replaces the bare status not in FAILURE_STATUSES_BY_MODE[mode] test at the two scoring call sites, since a status alone cannot carry the distinction.
  • Existing state is backfilled from the note each branch already wrote, so a repository gets the correction for its history rather than only for what it deletes next. No migration needed.

Deliberately unchanged

  • The third branch, "absent from latest detector output", still fails strict — that is the case the rule exists for.
  • verified_strict is untouched (auto_resolved already passes it).
  • An unrecognized resolution_kind is not a way out, so a hand-edited state file cannot invent an exemption.
  • Not a new status: that would have touched ~20 files across the lifecycle and UI for a scoring distinction.

Tests

5817 passed, 5 skipped. Seven new cases in TestVerifiedAutoResolution, covering both exempt kinds, the inferred case still failing, an unknown kind, other statuses, verified_strict invariance, and state written before the field existed.

Removing the exemption clause turns two of them red, so they fail on the unfixed code.

🤖 Generated with Claude Code

`auto_resolved` fails strict on purpose: a finding that merely stopped
appearing may have been hidden rather than fixed, and strict is the mode that
refuses to take that on trust.

But the merge auto-resolves for three different reasons and records them all as
the same status. Two of them are not inferences. `file_deleted` is checked
against the filesystem, and the zone-policy branch is read from configuration —
the tool establishes both for itself. Neither leaves anything a person could
fix and attest, so the finding can never reach `fixed` and never stops counting
against strict.

The effect is that strict decays as a codebase deletes dead code, and no amount
of work recovers it. On the repository this was found on, 242 of 1553 strict
failures (16%) are verified auto-resolutions; in the Test health dimension
alone it is 130 of 631, which had held that dimension at 0.0 strict for eleven
consecutive scans.

The fix records WHY an issue auto-resolved and exempts only the two kinds the
tool verified itself. `is_failure()` replaces the bare status-set test at the
two scoring call sites, since a status alone cannot carry that distinction.
Existing state is backfilled from the note each branch already wrote, so a
repository gets the correction for its history rather than only for what it
deletes next.

Deliberately unchanged: the third branch, "absent from latest detector output",
still fails strict — that is the case the rule exists for. `verified_strict` is
untouched, and an unrecognized `resolution_kind` is not a way out, so a
hand-edited state file cannot invent an exemption.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
github-actions Bot added a commit to citizenadam/desloppify that referenced this pull request Sep 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant