Skip to content

miner(cross-repo-eval): acceptanceCriteriaPath fallback hardcodes the wrong filename, and a test locks it in #10331

Description

@JSONbored

⚠️ Definition of Done: this issue must be completed in full, in a single PR. Do not split this
work across multiple PRs, and do not defer any Deliverable below to a follow-up issue. A PR that
satisfies only some of the Deliverables, stubs a required test, or leaves a checkbox
partially-done does NOT resolve this issue and will be closed.

Context

packages/loopover-miner/lib/cross-repo-evaluation.ts imports the real acceptance-criteria
filename constant from the engine:

import { ACCEPTANCE_CRITERIA_FILENAME } from "@loopover/engine";

and uses it correctly at line 684 (rmSync(join(workspace.path, ACCEPTANCE_CRITERIA_FILENAME), { force: true })). ACCEPTANCE_CRITERIA_FILENAME is defined in
packages/loopover-engine/src/miner/acceptance-criteria.ts as
"acceptance-criteria.json".

But the fallback path built a few dozen lines later, around line 726, when
specResult.acceptanceCriteriaPath is nullish, hardcodes a different, wrong literal string
instead of reusing the same imported constant:

const task = {
  attemptId: `cross-repo-eval-${repoFullName.replace("/", "-")}`,
  workingDirectory: workspace.path,
  acceptanceCriteriaPath: specResult.acceptanceCriteriaPath ?? join(workspace.path, "ACCEPTANCE_CRITERIA.md"),
  ...

"ACCEPTANCE_CRITERIA.md" is not a file that buildCodingTaskSpec/writeAcceptanceCriteriaFile
ever actually writes — the real on-disk file is always named acceptance-criteria.json. If this
fallback branch is ever taken (i.e. specResult.acceptanceCriteriaPath is undefined/null), the
coding-agent driver's task.acceptanceCriteriaPath points at a file that does not exist in the
scratch workspace.

This wrong value is also locked in by an existing test: test/unit/miner-cross-repo-evaluation.test.ts
(around line 893) directly asserts expect(String(tasks[0]?.acceptanceCriteriaPath).endsWith("ACCEPTANCE_CRITERIA.md")).toBe(true); — i.e. the test currently requires the buggy filename, so
fixing the code without also fixing this assertion will fail the test suite.

Requirements

  • Replace the hardcoded "ACCEPTANCE_CRITERIA.md" literal in the fallback expression around line
    726 of packages/loopover-miner/lib/cross-repo-evaluation.ts with the already-imported
    ACCEPTANCE_CRITERIA_FILENAME constant, so the fallback path agrees with the real filename the
    spec builder writes and with the rmSync cleanup earlier in the same function.
  • Update the existing test at test/unit/miner-cross-repo-evaluation.test.ts (around line 893) so
    it asserts the fallback path ends with the real ACCEPTANCE_CRITERIA_FILENAME value
    (acceptance-criteria.json), not the old wrong literal. Do not weaken the assertion (e.g. do not
    change it to a loose "path exists" check) — it must still assert the exact expected filename.
  • Search packages/loopover-miner/lib/** and packages/loopover-miner/test/**-equivalent
    (test/unit/**) for any other place that hardcodes the literal string "ACCEPTANCE_CRITERIA.md"
    instead of importing ACCEPTANCE_CRITERIA_FILENAME, and fix every one found — this issue is not
    satisfied by fixing only the one call site quoted above if others exist.

Deliverables

  • cross-repo-evaluation.ts's fallback acceptanceCriteriaPath expression uses
    ACCEPTANCE_CRITERIA_FILENAME instead of the hardcoded wrong literal.
  • The existing test asserting the old wrong filename is corrected to assert the real filename,
    and still exercises the same fallback branch (i.e. still constructs a scenario where
    specResult.acceptanceCriteriaPath is nullish so the fallback executes).
  • A grep -rn '"ACCEPTANCE_CRITERIA.md"' (or equivalent) over
    packages/loopover-miner/lib/** and test/** run as part of the PR's own verification
    returns zero remaining matches of the wrong literal.

Both deliverables are required in one PR — a PR that fixes the source file but leaves the test
asserting the old wrong filename (or vice versa) does not resolve this issue.

Test Coverage Requirements

This repo's Codecov patch gate is 99%+ (branch-counted) on packages/loopover-miner/lib/**. The
corrected assertion in test/unit/miner-cross-repo-evaluation.test.ts already exercises the
changed line, so no new test file is required — but the existing test must be updated (not
skipped/deleted) so it continues to assert real behavior for this branch, and the patch diff on
the changed line in cross-repo-evaluation.ts must show as covered.

Expected Outcome

When specResult.acceptanceCriteriaPath is absent, the coding-agent driver is handed the path to
the file that buildCodingTaskSpec actually wrote (acceptance-criteria.json), not a filename
that was never created. The test suite's own assertion matches real on-disk behavior instead of
locking in the bug.

Links & Resources

  • packages/loopover-engine/src/miner/acceptance-criteria.ts — the real
    ACCEPTANCE_CRITERIA_FILENAME definition.
  • packages/loopover-miner/lib/cross-repo-evaluation.ts line 684 — the correct usage to mirror.
  • test/unit/miner-cross-repo-evaluation.test.ts around line 893 — the test that currently locks
    in the wrong filename.

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:bugGittensor-scored bug fix — scores a 0.05x multiplier.help wantedExtra attention is needed

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions