Skip to content

t27b ledger: eval_hslm and ngram_hslm are unlisted, and two not-pass slots must be freed first #10858

Open

Description

@gHashTag

#10467 (b2a33e5) added specs/ml/eval_hslm.t27 and specs/ml/ngram_hslm.t27 to master with no rows in docs/reports/t27b_expectations.json. The reference passes both. t27b blocks both on ExprCall(undeclared fn): each does use ml::train_hslm; and calls into it, which is cross-module use-resolution (lane #9065).

t27b-native-ratchet therefore reports UNLISTED 2 for every PR built on master, even after #10837 lists the other eight drifted specs.

The two rows cannot just be added. The ledger sits at its cap, 139 not-pass rows (138 blocked + 1 codegen) against max_not_pass: 139, and the cap is not raised. Two blocked rows must start to pass first.

Boundary

  • docs/reports/t27b_expectations.json: add both rows in LC_ALL=C sort order, and keep the not-pass count at or below 139.
  • Free two not-pass slots by making t27b pass two currently blocked specs honestly, with new logic in a .t27 plan module and cli/t27b glue at net <= 0 hand-written lines. Candidates whose ledger blocker is unique:
  • Not in scope: raising max_not_pass, marking either spec n/a, deleting or weakening spec tests.

Scenarios

  1. Today: a PR built on master runs t27b-native-ratchet and gets UNLISTED 2 (eval_hslm, ngram_hslm), red.
  2. After the fix: the same PR gets UNLISTED 0, OVER CAP 0, UNEXPECTED PASS 0, green, with the cap unchanged.

Requirements

  • Two rows: {"path": "specs/ml/eval_hslm.t27", ...} and {"path": "specs/ml/ngram_hslm.t27", ...}, blocked or pass, sorted.
  • Two previously blocked rows flip to pass, each backed by a lab or CI corpus run.
  • Every new decision has test blocks in its .t27 plan, and a mutant of each new branch is killed.

Criteria

  • grep -o '"path": "[^"]*"' docs/reports/t27b_expectations.json | LC_ALL=C sort -c exits 0.
  • Not-pass rows (blocked + codegen) <= 139, and max_not_pass unchanged or lower.
  • t27b-native-ratchet is green on the PR; its verdict line shows UNLISTED 0.

Follows #10837, which freed one slot (@truncate in specs/tri/utils/utf8.t27) and listed hslm_tokenizer, hslm_corpus and six pass specs.

Success Criteria

  • grep -o '"path": "[^"]*"' docs/reports/t27b_expectations.json | LC_ALL=C sort -c exits 0
  • jq '[.[] | select(.status=="blocked" or .status=="codegen")] | length' docs/reports/t27b_expectations.json outputs a number ≤ 139
  • jq '.max_not_pass' docs/reports/t27b_expectations.json outputs a number ≤ 139
  • t27b-native-ratchet run on the PR prints a verdict line containing UNLISTED 0 and exits 0
  • Two specs previously listed as blocked in docs/reports/t27b_expectations.json now show status": "pass" in the same file
  • Each newly passing spec has at least one test block in its .t27 plan module and a mutant killed in CI (mutation score 100% for new branches)

Activity

  1. gHashTag commented on Oct 19, 2026

    @gHashTag
    OwnerAuthor

    State after #10853 merged and #10862 was rebuilt on master:

  2. added a commit that references this issue on Oct 19, 2026
  3. added
    shape-refusedClosed by the issue-shape gate (specs/policy/issue_shape.t27); a passing edit reopens it
    on Oct 19, 2026
  4. github-actions commented on Oct 19, 2026

    @github-actions
    Contributor

    Closed by the issue-shape gate (specs/policy/issue_shape.t27, #7498): an issue must have the template's sections. Edit the body; an edit that passes reopens it.

    • missing: ## Success Criteria -- an outcome a machine can settle: a number with a unit, a backticked file, exit 0, or a backticked command
  5. removed
    shape-refusedClosed by the issue-shape gate (specs/policy/issue_shape.t27); a passing edit reopens it
    on Oct 19, 2026
  6. gHashTag commented on Oct 19, 2026

    @gHashTag
    OwnerAuthor

    Reopened by the self-heal of #11007. The issue-shape gate closed this issue with a queued run that judged an older body from its event payload; the current body passes gen/c/policy/issue_shape.c, and the gate's own contract is that a passing body stays open. If the work is in fact finished or obsolete, close it as completed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions