Skip to content

Recover OBO idspaces so phenio node categories survive kgx contraction - #202

Merged
caufieldjh merged 1 commit into
masterfrom
fix/generic-obo-prefix-categories
Aug 13, 2026
Merged

caufieldjh merged 1 commit into
masterfrom
fix/generic-obo-prefix-categories

Conversation

@AO33

@AO33 AO33 commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Problem

Every monarch-kg release from 2026-06-03 on types DDPHENO (Dictyostelium phenotype) terms as biolink:NamedThing instead of biolink:PhenotypicFeature. It isn't just DDPHENO — 51,971 phenio nodes across eight ontologies lost their category between kg-phenio 20260508 and 20260603:

prefix was now n
DDPHENO biolink:PhenotypicFeature biolink:NamedThing 1,373
FBbt biolink:AnatomicalEntity biolink:NamedThing 28,766
EMAPA biolink:AnatomicalEntity biolink:NamedThing 8,853
WBbt biolink:AnatomicalEntity biolink:NamedThing 7,555
ZFA biolink:AnatomicalEntity biolink:NamedThing 3,231
XAO biolink:AnatomicalEntity biolink:NamedThing 1,830
CHR biolink:MolecularEntity biolink:NamedThing 309
ZFS biolink:LifeStage biolink:NamedThing 54

Downstream consumers that gate on category silently lose data. Anything selecting biolink:PhenotypicFeature neighbours drops Dictyostelium entirely, despite its 2,716 has_phenotype edges being present and intact.

Cause

kgx 2.7.0 made TsvSource.set_prefix_map additive rather than destructive, restoring kgx's 631-entry default prefix map for every source inheriting it — including ObographSource. That map carries a generic OBO -> http://purl.obolibrary.org/obo/ entry and has no per-ontology entry for the eight ontologies above, so the generic prefix now wins IRI contraction:

kgx 2.6.0:  http://purl.obolibrary.org/obo/DDPHENO_0000001  ->  DDPHENO:0000001
kgx 2.7.0:  http://purl.obolibrary.org/obo/DDPHENO_0000001  ->  OBO:DDPHENO_0000001

(FYPO, HP, MP, MONDO, CHEBI, UBERON and friends are unaffected — they have their own entries.)

phenio_node_sources.py looks the category up by CURIE prefix. NODE_SOURCES has no category under "OBO", so the generic OntologyClass -> NamedThing remap is never upgraded to the specific category. The pre-existing if node_curie_prefix == "OBO" branch recovered only XPO and HsapDv — the same bug class, patched by hand twice.

Fix

split_generic_obo_curie() splits the local part back into its idspace before the lookup, replacing the two special cases with the general rule. Case is preserved, so FBbt stays FBbt. Only well-formed <idspace>_<localid> local parts are split, so relation IRIs (OBO:fbbt#has_function_in), sub-namespaces (OBO:go/extensions/ro_0002092) and multi-underscore IDs (OBO:FBbt_root_00000000) keep the behaviour they already have.

Node IDs are deliberately left as-is. universalizer rewrites them during normalization, and they have to stay in step with the edge endpoints phenio_edge_sources.py writes — rewriting them here alone would leave node IDs at FBbt: while universalizer's obo_handle() still uppercased the edge endpoints to FBBT:, sending every FBbt/WBbt edge to the dangling pile.

Verification

Driven through the real koza transform (transform_source with phenio_node_sources.yaml) over a fixture built from real kgx 2.7.0 obojson output:

OBO:DDPHENO_0000001           biolink:PhenotypicFeature  infores:ddpheno
OBO:FBbt_00000001             biolink:AnatomicalEntity   infores:fbbt
OBO:WBbt_0005733              biolink:AnatomicalEntity   infores:wbbt
OBO:EMAPA_16040               biolink:AnatomicalEntity   infores:emapa
OBO:ZFA_0000001               biolink:AnatomicalEntity   infores:zfa
OBO:XAO_0000001               biolink:AnatomicalEntity   infores:xao
OBO:ZFS_0000001               biolink:LifeStage          infores:zfs
OBO:CHR_0000001               biolink:MolecularEntity    infores:chr
OBO:OBA_0000001               biolink:NamedThing         infores:oba   # no NODE_SOURCES category, unchanged
OBO:FBbt_root_00000000        biolink:NamedThing         infores:obo   # declined
OBO:fbbt#has_function_in      biolink:NamedThing         infores:obo   # declined
OBO:go/extensions/ro_0002092  biolink:NamedThing         infores:obo   # declined
FYPO / HP / MONDO / HGNC URI                                          # controls, unchanged

Same fixture on master vs this branch: the delta is exactly eight category changes and nine infores changes. Identical IDs, identical header, same row count in and out. Every recovered category matches what 20260508 shipped, so this restores the pre-regression state rather than introducing new typings.

The new tests fail without the fix (all 10 category cases and 3 split cases). Full suite passes, ruff check kg_phenio tests clean.

Downstream checks

  • BAD_PREFIXES is tested before the split, on the un-recovered prefix, so recovery can't newly exclude a node.
  • universalizer's update_categories overwrites only when the nodefile category is "" or biolink:OntologyClass, so the recovered specific categories survive normalization; namespace_cat_map is empty, so there's no ns_map path either.
  • Its RETAINED_CAT_LIST short-circuit (PhenotypicFeature subjects skip the biolink:category edge loop, including its remove_edges bookkeeping) is inert here: phenio_edge_sources.py already drops biolink:category predicates, so no such edge reaches the merged edge file.
  • In monarch-ingest, all four recovered categories are named thing descendants, so transform_phenio's valid_node_categories filter keeps them; nothing there branches on NamedThing, and qc_expect.yaml asserts only total phenio node/edge minimums, which don't change.
  • The deleted XPO and HsapDv special cases were already dead code — both prefixes have their own entries in kgx's map, so they contract directly and never reached that branch. Kept as test cases so the general rule can't silently lose them.

Out of scope

  • FBbt -> FBBT / WBbt -> WBBT uppercasing. universalizer's obo_handle() uppercases the prefix unconditionally, so any OBO:-collapsed mixed-case idspace comes out wrong. That is the actual source of the casing problem — normalize.py: use merged.monarch context (fixes FBbt/WBbt prefix uppercasing) #201 targeted the prefixmaps context and the 20260808 release still ships FBBT:. Fixing it means rewriting IDs in the node and edge transforms in lockstep; monarch-ingest currently normalizes the case on its side. Worth a follow-up.
  • phenio_edge_sources.py's own OBO handling ((subject.split("_"))[0][4:]). It only feeds the edge primary_knowledge_source, and switching it to the shared helper would change infores values for relation-IRI subjects. Left alone.
  • kgx itself. Preferring the most specific matching prefix over the generic OBO: one would fix this for every kgx consumer, but it shifts IDs broadly and belongs upstream on its own release cycle. Happy to open an issue there.

kgx's default prefix map carries a generic OBO -> http://purl.obolibrary.org/obo/
entry and no per-ontology entry for DDPHENO, EMAPA, XAO, ZFA, ZFS, CHR, FBbt or
WBbt, so their IRIs contract to OBO:DDPHENO_0000001. phenio_node_sources.py looks
the category up by CURIE prefix, and NODE_SOURCES has no category under "OBO", so
these nodes keep the generic biolink:NamedThing. 51,971 nodes -- including all
1,373 DDPHENO terms -- have been untyped since kg-phenio 20260603, when kgx 2.6.0
-> 2.7.0 restored the previously-wiped default prefix map for TsvSource and
everything inheriting it.

Split the local part back into its idspace before the lookup, replacing the XPO
and HsapDv special cases with the general rule. Node IDs are left as-is:
universalizer rewrites them during normalization and they have to stay in step
with the edge endpoints phenio_edge_sources.py writes.

Verified through the koza transform on real kgx 2.7.0 output -- the eight
ontologies regain the categories they had in 20260508, with identical IDs, row
count and columns.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@caufieldjh

Copy link
Copy Markdown
Contributor

Thanks @AO33 !

@caufieldjh
caufieldjh merged commit ddf0493 into master Aug 13, 2026
3 checks passed
@caufieldjh
caufieldjh deleted the fix/generic-obo-prefix-categories branch August 13, 2026 20:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants