Add an agent skill for NameRes, served at /llms.txt#259
Open
gaurav wants to merge 7 commits into
Open
Conversation
gaurav
marked this pull request as ready for review
July 27, 2026 09:06
…nts. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude Code discovers skills as <name>/SKILL.md with YAML frontmatter supplying `name` and `description`. A flat skills/nameres.md is never loaded, so /nameres did not exist and the skill never triggered. Move it and add the frontmatter; the `description` names trigger situations rather than describing the service, because that is what the model matches on to decide whether the skill is relevant. Closes #290. Corrections, all checked against a live instance rather than recalled: - The flagship example was wrong. /lookup?string=diabetes does not return MONDO:0005148 "diabetes mellitus" -- that CURIE is "type 2 diabetes mellitus". The real top hit is UMLS:C0011847 "Diabetes", a leftover-UMLS singleton with clique_identifier_count 1, ahead of MONDO:0005015 "diabetes mellitus". That is a better example than the invented one, so it now leads the disambiguation section: it shows concretely why the top hit is the best textual match rather than the concept you want. - Added `offset` and `debug`, which were missing from the parameter table. Without `offset` an agent cannot page past the 1000-result `limit` cap. - `highlighting` was shown as `{}` with no explanation; that is what you get with the flag off. Documented the real shape. - Added the always-present `explain` and `debug` response keys. Content that was a second copy of something else is now a link: the scoring boost table (Scoring.md owns it, and it was already a revision out of date), the exhaustive parameter reference (API.md), and why cliques look the way they do (Babel.md). Invented scores of 42.5/38.1/51.2 are gone -- real scores run into the thousands, and quoting any of them invites thresholding on a value that has no fixed scale. The base URL was hardcoded eight times; it is stated once, with every example using a bare path, so the document is correct when served from /llms.txt by an instance that is not RENCI dev. Adds what was missing entirely: /status and data provenance, a routing table sending CURIE-to-CURIE work to NodeNorm instead, that autocomplete=false is entity-linker mode, how to tell whether two names are the same concept, and the /lookup-vs-/synonyms field divergences tracked in #291. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
/llms.txt returns skills/nameres/SKILL.md with its YAML frontmatter stripped, read from disk per request. Serving the file rather than a second copy means the instructions an instance hands out always match the API answering the queries. The OpenAPI info.description points at it, so an agent given nothing but a base URL can find it. No Dockerfile change is needed -- the Dockerfile is `COPY . /repo/NameResolution` and .dockerignore does not exclude skills/. Verified rather than assumed: `docker build` then asserting SKILL_PATH.is_file() inside the image, and curling /llms.txt from the running container. That is the one failure mode that passes locally and 404s in every deployment, so the path constant carries a comment saying so and a test asserts it resolves. The OpenAPI gap this closes: server.py had 40 `description=` strings, every one on a request parameter, while LookupResult's ten fields had none -- so an agent reading /openapi.json got a well-documented request and an unexplained response. The field descriptions carry the traps, because that schema is what an agent reads without being told to: that `label` belongs to the clique rather than to `curie`, that `types` is biolink:-prefixed here but not on /synonyms, that `synonyms` is not quality-ordered, and that `score` is unbounded and only comparable within one query. Giving /synonyms a real response model is deliberately left to #291, which has to settle the field names first. test_apidocs.py asserts the exact path list, so it changes here rather than breaking. test_docs_links.py's master-branch ban now also catches the raw.githubusercontent.com form, which is what the install instructions use; it is scoped to NCATSTranslator because server.py legitimately builds biolink-model raw URLs and biolink-model really does default to master. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
documentation/LLMs.md's install instructions could not work: they saved a flat .claude/skills/nameres.md, which Claude Code does not discover. Rewritten around copying the directory, and around fetching /llms.txt from a running instance when you want instructions that match the deployment. Its "Example Workflows" section is deleted rather than fixed. It was a second copy of the API surface, written from unverified values -- it is where the claim that CHEBI:46195 is labelled "paracetamol" lived, which a live lookup disproves (it is "Acetaminophen"), attributed to DrugChemical conflation, which is the wrong mechanism: acetaminophen and paracetamol are one compound by ordinary synonymy, while DrugChemical joins drug products to active ingredients. The skill is now the only place that documents API usage. CLAUDE.md gains the rule the skill is written to -- it carries how to call the API, how to read the result, and the traps that silently produce a plausible wrong answer, and links out for anything a rebuild could falsify -- plus the requirement that its links be absolute, since a relative link resolves against the API host once the file is served at /llms.txt. Also corrects an example in API.md that the live checks caught: MONDO:0005148 is "type 2 diabetes mellitus", not "diabetes mellitus" (that is MONDO:0005015), and its clique has 21 identifiers rather than the 125 shown. The /synonyms sample also showed an empty `taxa`, which that endpoint omits entirely. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two tests for behaviour this branch added but did not cover: that every LookupResult field carries a description (the gap this branch closed, which a new undescribed field would silently reopen), and that info.description still points at /llms.txt, which is the only way an agent given a bare base URL finds the instructions. No test for the frontmatter regex's \r? branch. I wrote one, then found it passed against a \n-only regex too: read_text() opens in text mode, so a CRLF checkout is already normalised before the regex runs. The comment claiming the \r? was load-bearing is corrected rather than left to mislead; the regex keeps it as belt-and-braces in case the read ever becomes a byte read. CLAUDE.md records two things that cost time this session: `gh pr view`'s commit and file counts are a cached summary that was stale by 37 commits, so PR contents should be checked with the compare API; and a deployed NameRes is not this code -- five instances disagreed about /status alone -- so a live response is not ground truth for repo behaviour without checking nameres_version first. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
/llms.txt reads skills/nameres/SKILL.md off disk, and that file reaches the image only via the Dockerfile's blanket `COPY .` plus the absence of skills/ from .dockerignore. Nothing asserts either. Narrowing one of them -- a "slim the image" change is the obvious way -- 404s the route in every deployment while the entire suite still passes, because the tests run against the working tree rather than the image. This step builds the real image and asks it, so that failure mode turns CI red instead of shipping. Verified in both directions: it passes today, and adding /skills/ to .dockerignore makes it fail with the path in the message. Uses TestClient inside the container rather than starting it and curling -- there is no port to bind, no readiness loop, and /llms.txt needs no Solr. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The link tests only ever looked at Markdown plus api/server.py and openapi.yml. Four other files in this repo carry GitHub links and were unchecked -- including documentation/NameResolution.ipynb, whose Colab badge names a branch and which I had to fix by hand earlier this session, and CITATION.cff, whose repository-code field is what Zenodo reads and which was wrong in Babel for exactly that reason. So the file set is now globbed by extension rather than listed. A hand-written list of "where links live" is precisely what let those two rot unnoticed, and it silently stops covering anything added later. Widening it immediately found one more: data/name-lookup/values.yaml sets babelVersionURL to a /blob/master/ URL, which becomes BABEL_VERSION_URL and is what /status hands out -- ITRB CI is serving that master URL right now. That file is a gitignored working copy of the translator-devops chart, so data/ is excluded here and the real fix belongs in that repository. CLAUDE.md records the trap this sweep kept running into: TranslatorSRI is not uniformly stale. RENCI-Python-image, babel-validation and r3 genuinely still live under that org, so the ban names the three moved repositories rather than the org string. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Gives NameRes an agent skill that a coding agent can load to use this API properly, and serves it from the API itself at
GET /llms.txt.Based on #262 (
improve-documentation), so the skill can referencedocumentation/Babel.mdand the rest of that work. Retarget tomainonce #262 merges.Closes #290.
What's here
skills/nameres/SKILL.md— moved from a flatskills/nameres.mdand givenname/descriptionfrontmatter, because Claude Code discovers skills as<name>/SKILL.mddirectories and never loaded the flat file (#290). Thedescriptionnames trigger situations rather than describing the service, since that is what the model matches on.GET /llms.txt— serves that same file with its frontmatter stripped. One copy of the text, and the instructions an instance hands out always match the API answering the queries. Linked from the OpenAPIinfo.descriptionso an agent given only a base URL can find it.LookupResultfield descriptions —api/server.pyhad 40description=strings, every one on a request parameter, and none on the response model. An agent reading/openapi.jsongot a well-documented request and a completely unexplained response. The descriptions carry the traps, because that schema is what an agent reads without being told to.documentation/LLMs.md— rewritten to install-and-maintain only. Its old install instructions could not work, and its "Example Workflows" section was a second copy of the API surface built from unverified values; it is deleted rather than fixed.Corrections, all checked against a live instance
The skill's API-surface claims held up well —
biolink_typevsbiolink_types, POST-takes-query-params, thebiolink:prefix asymmetry,taxon_specificinclusion were all correct. The example values did not:/lookup?string=diabetes→MONDO:0005148"diabetes mellitus"MONDO:0005148is "type 2 diabetes mellitus" (21 identifiers, not 125). "diabetes mellitus" isMONDO:0005015.CHEBI:46195labelled "paracetamol", via DrugChemical conflationMONDO:0005044"hypertension"The
diabetesresult is now the skill's worked example for not trusting the top hit, because the real answer teaches the lesson better than the invented one did: the winner isUMLS:C0011847"Diabetes", a leftover-UMLS singleton withclique_identifier_count: 1, ahead of the concept anyone actually wants.Also fixed the same
MONDO:0005148error where it appears indocumentation/API.md.Content that became a link
The skill was accumulating copies of things that live elsewhere. Now: the scoring boost table →
Scoring.md(which already owned it, and the copy was a revision stale), the exhaustive parameter reference →API.md, why cliques look the way they do →Babel.md. The base URL was hardcoded 8×; it is stated once and every example uses a bare path, which is also what makes the document correct when served at/llms.txtfrom a non-RENCI instance.The rule is written down in
CLAUDE.md: the skill carries how to call the API, how to read the result, and the traps that silently produce a plausible wrong answer — anything a Babel rebuild could falsify gets linked instead.This needs re-testing against NameRes Exp before shipping. No instance currently serves both recent data and a NameRes matching this repo:
name-resolution-sri.renci.org(dev)name-lookup.ci.transltr.io2025sep1, NameResv1.5.2name-lookup.transltr.io(prod)/statushas nobabel_version,nameres_versionorbiolink_modelat allname-resolution-exp.apps.renci.org(Exp)Two concrete divergences between ITRB CI and this repo that the examples could hide:
/statusresponse shape differs. CI (v1.5.2) nests the Solr fields under"solr"and adds arecent_queriesblock; this repo returns them flat. The skill therefore documents onlybabel_version,nameres_versionand the conflations, and says older deployments may omit them — it does not document the shape./synonymson CI returnsnames_exactish. The current schema has that fieldstored="false", so this repo does not return it. Harmless, but it means a response captured from CI is not byte-identical to one from this code.Before this ships, someone should re-run the skill's examples against Exp and confirm the
diabetesdisambiguation example still holds.Out of scope
Translator_sdk/name_resolver.pyis fine; expanding it adds a second API surface to version and helps only Python callers./synonymsa response model — Harmonize the /lookup and /synonyms response shapes #291, which has to settle the field names first.Verification
tests/test_llms_txt.py— the file exists, frontmatter is valid and matches its directory,/llms.txtis served astext/plainwith frontmatter stripped, a missing file 404s with an actionable message, the links are absolute, and specific traps survive rewrites.COPY . /repo/NameResolution, and.dockerignoredoes not excludeskills/), but that is the one failure mode that passes locally and 404s in every deployment, so:docker build, assertSKILL_PATH.is_file()inside the image (resolves to/repo/NameResolution/skills/nameres/SKILL.md), thencurl /llms.txtfrom the running container →200 text/plain.tests/test_docs_links.py'smaster-branch ban now also catches theraw.githubusercontent.comform the install instructions use.Follow-up
If #231 (
track-recent-times) merges after this,/statusgainsrecent_queriesand the skill's "Which data am I querying?" section should be revisited.