Skip to content

Introduce the generated-oracle machinery #37

Closed

Description

@CaffeinatedCoder

Bring over the generated-oracle approach from CodoMetis.TypeKit, in the lighter form BLite uses: no covering arrays, no ledger, no replay footprints, no agent. The sweeps are ordinary MSTest classes, and docs/oracles.md describes the system.

The approach has four parts:

  • test/oracles/claims.json lists what the consumer docs promise, one claim per sentence, each quoted verbatim with the test that holds it. OracleCatalogueTests fails when a document no longer contains a quote.
  • ApiSurface enumerates every public member, and OracleSurfaceCoverageTests requires each one to be named by a claim or waived in surface-waivers.json.
  • oracles.json and seeds/ list the registered sweeps. An oracle only counts once a seed patch shows it failing; dotnet run tools/oracles.cs seeds proves them locally.
  • candidates.json registers disagreements against an issue, so the suite stays green without hiding them. An entry with no issue keeps failing, and an entry the fix made obsolete fails as stale.

State on 2026-10-17: in PR #54 (03b0445 on test/oracle-machinery), together with the probing round (#52). It lands before #38. The first finding is #53.

Done when the machinery is merged and the conventions tests hold it.

Activity

  1. added
    testsAdds or changes tests only (oracles, harnesses); no library code
    on Oct 17, 2026
  2. CaffeinatedCoder commented on Oct 17, 2026

    @CaffeinatedCoder
    OwnerAuthor

    Merge order: this lands before #38. The docs rewrite then re-anchors the claims.json quotes it rewords, and uses the catalogue to check that no promise was lost.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

testsAdds or changes tests only (oracles, harnesses); no library code

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions