Skip to content

Add an agent skill for NameRes, served at /llms.txt#259

Open
gaurav wants to merge 7 commits into
improve-documentationfrom
add-claude-skill
Open

Add an agent skill for NameRes, served at /llms.txt#259
gaurav wants to merge 7 commits into
improve-documentationfrom
add-claude-skill

Conversation

@gaurav

@gaurav gaurav commented Apr 17, 2026

Copy link
Copy Markdown
Collaborator

Gives NameRes an agent skill that a coding agent can load to use this API properly, and serves it from the API itself at GET /llms.txt.

Based on #262 (improve-documentation), so the skill can reference documentation/Babel.md and the rest of that work. Retarget to main once #262 merges.

Closes #290.

What's here

skills/nameres/SKILL.md — moved from a flat skills/nameres.md and given name/description frontmatter, because Claude Code discovers skills as <name>/SKILL.md directories and never loaded the flat file (#290). The description names trigger situations rather than describing the service, since that is what the model matches on.

GET /llms.txt — serves that same file with its frontmatter stripped. One copy of the text, and the instructions an instance hands out always match the API answering the queries. Linked from the OpenAPI info.description so an agent given only a base URL can find it.

LookupResult field descriptionsapi/server.py had 40 description= strings, every one on a request parameter, and none on the response model. An agent reading /openapi.json got a well-documented request and a completely unexplained response. The descriptions carry the traps, because that schema is what an agent reads without being told to.

documentation/LLMs.md — rewritten to install-and-maintain only. Its old install instructions could not work, and its "Example Workflows" section was a second copy of the API surface built from unverified values; it is deleted rather than fixed.

Corrections, all checked against a live instance

The skill's API-surface claims held up well — biolink_type vs biolink_types, POST-takes-query-params, the biolink: prefix asymmetry, taxon_specific inclusion were all correct. The example values did not:

Claim Reality
/lookup?string=diabetesMONDO:0005148 "diabetes mellitus" MONDO:0005148 is "type 2 diabetes mellitus" (21 identifiers, not 125). "diabetes mellitus" is MONDO:0005015.
CHEBI:46195 labelled "paracetamol", via DrugChemical conflation It is labelled "Acetaminophen", and the mechanism was wrong — acetaminophen/paracetamol are one compound by ordinary synonymy; DrugChemical joins drug products to active ingredients.
Scores around 42.5 / 38.1 / 51.2 Real scores run into the thousands (3770, 6872). Quoting any of them invites thresholding on a value with no fixed scale.
MONDO:0005044 "hypertension" Correct, as it happens.

The diabetes result is now the skill's worked example for not trusting the top hit, because the real answer teaches the lesson better than the invented one did: the winner is UMLS:C0011847 "Diabetes", a leftover-UMLS singleton with clique_identifier_count: 1, ahead of the concept anyone actually wants.

Also fixed the same MONDO:0005148 error where it appears in documentation/API.md.

Content that became a link

The skill was accumulating copies of things that live elsewhere. Now: the scoring boost table → Scoring.md (which already owned it, and the copy was a revision stale), the exhaustive parameter reference → API.md, why cliques look the way they do → Babel.md. The base URL was hardcoded 8×; it is stated once and every example uses a bare path, which is also what makes the document correct when served at /llms.txt from a non-RENCI instance.

The rule is written down in CLAUDE.md: the skill carries how to call the API, how to read the result, and the traps that silently produce a plausible wrong answer — anything a Babel rebuild could falsify gets linked instead.

⚠️ Verified against ITRB CI, not against code-matching data

This needs re-testing against NameRes Exp before shipping. No instance currently serves both recent data and a NameRes matching this repo:

Instance State
name-resolution-sri.renci.org (dev) 503 throughout this work
name-lookup.ci.transltr.io Used for verification. Babel 2025sep1, NameRes v1.5.2
name-lookup.transltr.io (prod) Solr index dated 2024-11-15; /status has no babel_version, nameres_version or biolink_model at all
name-resolution-exp.apps.renci.org (Exp) Corresponds to this repo's code — unreachable, RENCI is down

Two concrete divergences between ITRB CI and this repo that the examples could hide:

  1. /status response shape differs. CI (v1.5.2) nests the Solr fields under "solr" and adds a recent_queries block; this repo returns them flat. The skill therefore documents only babel_version, nameres_version and the conflations, and says older deployments may omit them — it does not document the shape.
  2. /synonyms on CI returns names_exactish. The current schema has that field stored="false", so this repo does not return it. Harmless, but it means a response captured from CI is not byte-identical to one from this code.

Before this ships, someone should re-run the skill's examples against Exp and confirm the diabetes disambiguation example still holds.

Out of scope

  • MCPAdd MCP endpoints #251. A skill is text with no runtime, deployment or auth, and works with any agent that can make an HTTP request; an MCP server would need all three to wrap an API that is already REST with an OpenAPI schema. Its tool descriptions would be written from this same content, so this is a prerequisite rather than a competitor.
  • A Python SDK — the existing Translator_sdk/name_resolver.py is fine; expanding it adds a second API surface to version and helps only Python callers.
  • Giving /synonyms a response modelHarmonize the /lookup and /synonyms response shapes #291, which has to settle the field names first.

Verification

  • 43 tests pass (up from 35 on Improve NameRes documentation: Translator Guide, Babel data guide, and doc drift fixes #262) against a local Solr 9.10.1 with the checked-in configset.
  • tests/test_llms_txt.py — the file exists, frontmatter is valid and matches its directory, /llms.txt is served as text/plain with frontmatter stripped, a missing file 404s with an actionable message, the links are absolute, and specific traps survive rewrites.
  • The Docker path was verified, not assumed. No Dockerfile change is needed here (COPY . /repo/NameResolution, and .dockerignore does not exclude skills/), but that is the one failure mode that passes locally and 404s in every deployment, so: docker build, assert SKILL_PATH.is_file() inside the image (resolves to /repo/NameResolution/skills/nameres/SKILL.md), then curl /llms.txt from the running container → 200 text/plain.
  • tests/test_docs_links.py's master-branch ban now also catches the raw.githubusercontent.com form the install instructions use.

Follow-up

If #231 (track-recent-times) merges after this, /status gains recent_queries and the skill's "Which data am I querying?" section should be revisited.

@gaurav
gaurav changed the base branch from main to improve-documentation July 27, 2026 08:39
@gaurav
gaurav force-pushed the add-claude-skill branch from 111f6e1 to 86894d4 Compare July 27, 2026 08:40
@gaurav gaurav changed the title Added LLM agent skill and documentation for using NameRes from AI agents Add an agent skill for NameRes, served at /llms.txt Jul 27, 2026
@gaurav
gaurav marked this pull request as ready for review July 27, 2026 09:06
gaurav and others added 6 commits July 27, 2026 05:50
…nts.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude Code discovers skills as <name>/SKILL.md with YAML frontmatter
supplying `name` and `description`. A flat skills/nameres.md is never loaded,
so /nameres did not exist and the skill never triggered. Move it and add the
frontmatter; the `description` names trigger situations rather than describing
the service, because that is what the model matches on to decide whether the
skill is relevant. Closes #290.

Corrections, all checked against a live instance rather than recalled:

- The flagship example was wrong. /lookup?string=diabetes does not return
  MONDO:0005148 "diabetes mellitus" -- that CURIE is "type 2 diabetes
  mellitus". The real top hit is UMLS:C0011847 "Diabetes", a leftover-UMLS
  singleton with clique_identifier_count 1, ahead of MONDO:0005015 "diabetes
  mellitus". That is a better example than the invented one, so it now leads
  the disambiguation section: it shows concretely why the top hit is the best
  textual match rather than the concept you want.
- Added `offset` and `debug`, which were missing from the parameter table.
  Without `offset` an agent cannot page past the 1000-result `limit` cap.
- `highlighting` was shown as `{}` with no explanation; that is what you get
  with the flag off. Documented the real shape.
- Added the always-present `explain` and `debug` response keys.

Content that was a second copy of something else is now a link: the scoring
boost table (Scoring.md owns it, and it was already a revision out of date),
the exhaustive parameter reference (API.md), and why cliques look the way they
do (Babel.md). Invented scores of 42.5/38.1/51.2 are gone -- real scores run
into the thousands, and quoting any of them invites thresholding on a value
that has no fixed scale.

The base URL was hardcoded eight times; it is stated once, with every example
using a bare path, so the document is correct when served from /llms.txt by an
instance that is not RENCI dev.

Adds what was missing entirely: /status and data provenance, a routing table
sending CURIE-to-CURIE work to NodeNorm instead, that autocomplete=false is
entity-linker mode, how to tell whether two names are the same concept, and
the /lookup-vs-/synonyms field divergences tracked in #291.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
/llms.txt returns skills/nameres/SKILL.md with its YAML frontmatter stripped,
read from disk per request. Serving the file rather than a second copy means
the instructions an instance hands out always match the API answering the
queries. The OpenAPI info.description points at it, so an agent given nothing
but a base URL can find it.

No Dockerfile change is needed -- the Dockerfile is `COPY . /repo/NameResolution`
and .dockerignore does not exclude skills/. Verified rather than assumed:
`docker build` then asserting SKILL_PATH.is_file() inside the image, and
curling /llms.txt from the running container. That is the one failure mode
that passes locally and 404s in every deployment, so the path constant carries
a comment saying so and a test asserts it resolves.

The OpenAPI gap this closes: server.py had 40 `description=` strings, every one
on a request parameter, while LookupResult's ten fields had none -- so an agent
reading /openapi.json got a well-documented request and an unexplained
response. The field descriptions carry the traps, because that schema is what
an agent reads without being told to: that `label` belongs to the clique
rather than to `curie`, that `types` is biolink:-prefixed here but not on
/synonyms, that `synonyms` is not quality-ordered, and that `score` is
unbounded and only comparable within one query.

Giving /synonyms a real response model is deliberately left to #291, which has
to settle the field names first.

test_apidocs.py asserts the exact path list, so it changes here rather than
breaking. test_docs_links.py's master-branch ban now also catches the
raw.githubusercontent.com form, which is what the install instructions use;
it is scoped to NCATSTranslator because server.py legitimately builds
biolink-model raw URLs and biolink-model really does default to master.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
documentation/LLMs.md's install instructions could not work: they saved a flat
.claude/skills/nameres.md, which Claude Code does not discover. Rewritten
around copying the directory, and around fetching /llms.txt from a running
instance when you want instructions that match the deployment.

Its "Example Workflows" section is deleted rather than fixed. It was a second
copy of the API surface, written from unverified values -- it is where the
claim that CHEBI:46195 is labelled "paracetamol" lived, which a live lookup
disproves (it is "Acetaminophen"), attributed to DrugChemical conflation, which
is the wrong mechanism: acetaminophen and paracetamol are one compound by
ordinary synonymy, while DrugChemical joins drug products to active
ingredients. The skill is now the only place that documents API usage.

CLAUDE.md gains the rule the skill is written to -- it carries how to call the
API, how to read the result, and the traps that silently produce a plausible
wrong answer, and links out for anything a rebuild could falsify -- plus the
requirement that its links be absolute, since a relative link resolves against
the API host once the file is served at /llms.txt.

Also corrects an example in API.md that the live checks caught: MONDO:0005148
is "type 2 diabetes mellitus", not "diabetes mellitus" (that is MONDO:0005015),
and its clique has 21 identifiers rather than the 125 shown. The /synonyms
sample also showed an empty `taxa`, which that endpoint omits entirely.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two tests for behaviour this branch added but did not cover: that every
LookupResult field carries a description (the gap this branch closed, which a
new undescribed field would silently reopen), and that info.description still
points at /llms.txt, which is the only way an agent given a bare base URL
finds the instructions.

No test for the frontmatter regex's \r? branch. I wrote one, then found it
passed against a \n-only regex too: read_text() opens in text mode, so a CRLF
checkout is already normalised before the regex runs. The comment claiming the
\r? was load-bearing is corrected rather than left to mislead; the regex keeps
it as belt-and-braces in case the read ever becomes a byte read.

CLAUDE.md records two things that cost time this session: `gh pr view`'s
commit and file counts are a cached summary that was stale by 37 commits, so
PR contents should be checked with the compare API; and a deployed NameRes is
not this code -- five instances disagreed about /status alone -- so a live
response is not ground truth for repo behaviour without checking
nameres_version first.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
/llms.txt reads skills/nameres/SKILL.md off disk, and that file reaches the
image only via the Dockerfile's blanket `COPY .` plus the absence of skills/
from .dockerignore. Nothing asserts either. Narrowing one of them -- a "slim
the image" change is the obvious way -- 404s the route in every deployment
while the entire suite still passes, because the tests run against the working
tree rather than the image.

This step builds the real image and asks it, so that failure mode turns CI red
instead of shipping. Verified in both directions: it passes today, and adding
/skills/ to .dockerignore makes it fail with the path in the message.

Uses TestClient inside the container rather than starting it and curling --
there is no port to bind, no readiness loop, and /llms.txt needs no Solr.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@gaurav
gaurav force-pushed the add-claude-skill branch from 454b4c4 to fc8105c Compare July 27, 2026 09:52
The link tests only ever looked at Markdown plus api/server.py and
openapi.yml. Four other files in this repo carry GitHub links and were
unchecked -- including documentation/NameResolution.ipynb, whose Colab badge
names a branch and which I had to fix by hand earlier this session, and
CITATION.cff, whose repository-code field is what Zenodo reads and which was
wrong in Babel for exactly that reason.

So the file set is now globbed by extension rather than listed. A hand-written
list of "where links live" is precisely what let those two rot unnoticed, and
it silently stops covering anything added later.

Widening it immediately found one more: data/name-lookup/values.yaml sets
babelVersionURL to a /blob/master/ URL, which becomes BABEL_VERSION_URL and is
what /status hands out -- ITRB CI is serving that master URL right now. That
file is a gitignored working copy of the translator-devops chart, so data/ is
excluded here and the real fix belongs in that repository.

CLAUDE.md records the trap this sweep kept running into: TranslatorSRI is not
uniformly stale. RENCI-Python-image, babel-validation and r3 genuinely still
live under that org, so the ban names the three moved repositories rather than
the org string.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

1 participant