Skip to content

Update two stale test allowlists so the suite can be read again - #66

Merged
adamjohnwright merged 1 commit into
mainfrom
fix/and-logic-test-allowlist
Sep 10, 2026
Merged

Update two stale test allowlists so the suite can be read again#66
adamjohnwright merged 1 commit into
mainfrom
fix/and-logic-test-allowlist

Conversation

@adamjohnwright

Copy link
Copy Markdown
Contributor

Seven tests were failing locally, and had been for long enough that the failures were background noise. Five were a stale allowlist; two needed the generator fixes that just landed.

test_and_logic_consistency — stale allowlist

It allowed AND only on input / catalyst / positive regulator, so it failed on 149k edges across all 92 pathways. It would fail on every pathway in the catalog; only the first five are sampled.

Each addition was checked against what the solver actually does with and_or, not added to make the test green:

edge type count reaches the solver?
assembly 61,652 NoDS_ASSEMBLY_LIMITING routes assembly inputs to a limiting-reactant (min) rule before the AND/OR branch. AND is also correct: a Complex is the AND of its subunits.
depletion 5,064 Nocreate_reaction_from_edges never records is_and for depletion.
dissociation 82,394 Yes, and genuinely inconsistent — all 116k output edges are or, these are and.

I tested the third rather than assuming

Dissociation edges feed terminal readout sinks with exactly one producer, so the only question is whether a lone parent gets the hill_log tanh or passes through:

 fold_in   AND(hill_log)   OR(identity)
 0.85      0.85            0.85
 1.15      1.15            1.15
 2.0       1.9977          2.0
 100.0     74.07           100.0

Identity to four decimals at both classification cutoffs, within 0.1% out to fold 2, compressing only large folds. Flipping all dissociation edges to or and re-benchmarking moved nothing real — 11 changed predictions, all in one pathway, none with both arms converged: the same uuid-relabelling noise documented in specs/003.

Inconsistent but immaterial, so it's documented in the test rather than changed without evidence.

test_negative_regulators_marked_neg — stale allowlist

Asserted every negative edge is a regulator. depletion is negative by construction and postdates the test; the solver routes it separately and applies divide-form inhibition.

The last failure is real, and #64 + #65 already fixed it

test_main_edges_proportional_to_best_matches is an O(n²) detector doing its job. Regenerating on current main:

pathway v96 v97 pre-fix current main
Class I MHC 54.4 FAIL 174.5 FAIL 11.2 pass
NER 4.7 62.1 FAIL 7.8 pass

Total edges fall 178,198 → 14,627 and 136,603 → 1,739.

It still fails against the local output/ tree because that tree predates both fixes. output/ is gitignored and untracked, so refreshing it is a local-artifact operation with no repository effect — and CI, which has no output/, skips these tests entirely.

930 passing, 1 failing (that one), no new lint errors.

🤖 Generated with Claude Code

Seven tests were failing locally, and had been for long enough that the
failures were background noise. Five were a stale allowlist; two needed the
generator fixes that just landed.

test_and_logic_consistency allowed AND only on input / catalyst / positive
regulator, so it failed on 149k edges across all 92 pathways — it would fail on
every pathway in the catalog, and only the first five are sampled. Each edge
type added below was checked against what the solver actually does with
`and_or`, rather than added to make the test green:

- assembly (61,652 edges): AND is correct, a Complex IS the AND of its
  subunits. Also inert under the default config — DS_ASSEMBLY_LIMITING routes
  assembly inputs to a limiting-reactant (min) rule before the AND/OR branch.
- depletion (5,064): inert. create_reaction_from_edges pushes depletion edges
  to their own vector and never records is_and.
- dissociation (82,394): live, and genuinely inconsistent — all 116k `output`
  edges are `or`, these are `and`. They feed terminal readout sinks with one
  producer each, so the only question is whether a lone parent gets the
  hill_log tanh or passes through.

I tested that last one rather than assuming. hill_log is identity to four
decimal places at both classification cutoffs (0.85 and 1.15) and within 0.1%
out to fold 2, compressing only large folds (100 -> 74). Flipping all
dissociation edges to `or` and re-benchmarking moved nothing real: 11 changed
predictions, all in one pathway, none with both arms converged — the same
uuid-relabelling noise signature documented in specs/003. Inconsistent but
immaterial, so it is documented in the test rather than changed without
evidence.

test_negative_regulators_marked_neg asserted every negative edge is a
`regulator`. `depletion` is negative by construction and postdates the test;
the solver routes it separately and applies divide-form inhibition to it.

The last failure, test_main_edges_proportional_to_best_matches on Class I MHC,
is a real O(n^2) detector doing its job — and the memo fix (#64) plus the
cofactor guard (#65) resolve it. Measured by regenerating on current main:

  Class I MHC   ratio 54.4 (v96) -> 174.5 (v97 pre-fix) -> 11.2  pass
  NER           ratio  4.7       ->  62.1               ->  7.8  pass

with total edges falling 178,198 -> 14,627 and 136,603 -> 1,739. It still fails
against the local output/ tree because that tree predates both fixes;
regenerating it clears the last one. output/ is gitignored and untracked, so
this is a local-artifact refresh with no repository effect, and CI — which has
no output/ — skips these tests entirely.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@adamjohnwright
adamjohnwright merged commit d56e9f2 into main Sep 10, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant