Skip to content

miner(purge): right-to-be-forgotten never deletes a policy-doc-cache entry #10341

Description

@JSONbored

⚠️ Definition of Done: this issue must be completed in full, in a single PR. Do not split this
work across multiple PRs, and do not defer any Deliverable below to a follow-up issue. A PR that
satisfies only some of the Deliverables, stubs a required test, or leaves a checkbox
partially-done does NOT resolve this issue and will be closed.

Context

packages/loopover-miner/lib/purge-cli.ts implements the operator-invoked, right-to-be-forgotten
repo purge. Its own doc comment states the acceptance criterion this feature exists to satisfy:
"Every purge is audit-observable by design ... so a purge can never be silent." Its
REAL_PURGE_TARGETS array (lines ~115-137) lists every store that gets purged by repo, and
separately, ATTEMPT_LOG_NOT_PURGEABLE_NOTE explicitly discloses the one known store
(attempt-log) that cannot be purged by repo (it has no repoFullName column) and surfaces that
fact directly in both the dry-run and real-purge CLI output.

packages/loopover-miner/lib/policy-doc-cache.ts (PolicyDocCacheStore, caching fetched
AI-USAGE.md/CONTRIBUTING.md content keyed by URL) exposes only get/put/close — no
purgeByRepo at all. It appears in NEITHER REAL_PURGE_TARGETS NOR alongside
ATTEMPT_LOG_NOT_PURGEABLE_NOTE — it is simply absent from purge entirely, with no disclosure.
store-maintenance.ts's own comment (around line 73) explains the original reasoning: "
policy-doc-cache.js stays out (keyed by URL, no repo column, exactly like attempt-log.js)" —
i.e. the intent was to treat it the same as attempt-log: purge-impossible, but disclosed.

That reasoning doesn't hold up on inspection. Unlike attempt-log's genuinely free-form payload,
policy-doc-cache's url key is a structured GitHub contents-API URL
(.../repos/{owner}/{repo}/contents/AI-USAGE.md, built in opportunity-fanout.ts) that
deterministically embeds owner/repo — the same kind of structured key
policy-verdict-cache.ts already successfully purges via a suffix-match approach. So a real
repo-scoped purge is actually feasible here, and even setting that aside, the "can't purge" framing
is never disclosed the way attempt-log's is — the cached content for a purged repo persists
indefinitely in policy-doc-cache.sqlite3 with zero visibility to the operator running the purge,
and neither test/unit/miner-purge-cli.test.ts nor the policy-doc-cache test file has any purge-
related test.

Requirements

  • Add a real repo-scoped purgeByRepo(repoFullName: string): number method to
    PolicyDocCacheStore in policy-doc-cache.ts, using the same URL-suffix-match approach
    policy-verdict-cache.ts's purgeByRepo already uses (or parsing owner/repo out of the stored
    URL directly — either technique is acceptable as long as it correctly matches only rows whose
    cached URL belongs to the target repo, not a substring false-positive on an unrelated repo whose
    name happens to be a substring of the target).
  • Register policy-doc-cache as a new entry in purge-cli.ts's REAL_PURGE_TARGETS, following
    the exact same shape (name, optionKey, opener, resolveDbPath, spec) as the other
    entries in that array, and add the corresponding PurgeCliOptions field / PurgeOpenerKey
    union member for dependency injection, matching the existing pattern for every other store in
    that file.
  • Add a LedgerPurgeSpec for policy-doc-cache in store-maintenance.ts, alongside the existing
    specs, following the same shape/documentation convention as POLICY_VERDICT_CACHE_PURGE_SPEC.
  • Update the comment in store-maintenance.ts (around line 73) that currently claims
    policy-doc-cache.js stays out — it must no longer describe this store as excluded once it is
    purgeable.

Deliverables

  • PolicyDocCacheStore has a working purgeByRepo(repoFullName) method that deletes only rows
    whose cached URL belongs to the given repo.
  • policy-doc-cache is a real entry in purge-cli.ts's REAL_PURGE_TARGETS, appearing in
    both dry-run and real-purge output the same way every other real purge target does.
  • The stale store-maintenance.ts comment claiming this store is excluded from purge is
    corrected.
  • A new regression test in the purge-cli test file asserting that purging a repo removes its
    policy-doc-cache rows and does NOT remove another repo's rows (guarding against a
    substring-match false positive).
  • A new regression test in the policy-doc-cache test file directly exercising purgeByRepo in
    isolation (inserting entries for two different repos, purging one, asserting only that repo's
    rows are gone).

All deliverables are required in this one PR — adding purgeByRepo to the store without wiring it
into purge-cli.ts's REAL_PURGE_TARGETS (or vice versa) does not resolve this issue.

Test Coverage Requirements

This repo's Codecov patch gate is 99%+ (branch-counted) on packages/loopover-miner/lib/**. Add
the new regression tests under test/unit/** (this package's tests live in the shared root
test/ directory, not packages/loopover-miner/test/**), extending the existing
test/unit/miner-purge-cli.test.ts and the existing policy-doc-cache test file. Both the new
purgeByRepo method and its wiring into purge-cli.ts must be exercised.

Expected Outcome

Purging a repo via loopover-miner purge --repo <owner/repo> removes that repo's cached
policy-doc-cache entries the same way it already removes every other repo-scoped store's data —
no cache silently outlives a purge, matching the purge feature's own "a purge can never be silent"
acceptance criterion.

Links & Resources

  • packages/loopover-miner/lib/policy-doc-cache.ts — the store to add purgeByRepo to.
  • packages/loopover-miner/lib/policy-verdict-cache.ts — its existing purgeByRepo (suffix-match
    approach) to mirror.
  • packages/loopover-miner/lib/purge-cli.tsREAL_PURGE_TARGETS (~lines 115-137),
    ATTEMPT_LOG_NOT_PURGEABLE_NOTE (~line 67), PurgeCliOptions/PurgeOpenerKey (~lines 78-102).
  • packages/loopover-miner/lib/store-maintenance.ts — the stale comment (~line 73) to correct, and
    the LedgerPurgeSpec shape to mirror for the new spec.

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:bugGittensor-scored bug fix — scores a 0.05x multiplier.help wantedExtra attention is needed

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions