Skip to content

Adopt graphify as a developer-only code + ADR knowledge graph - #677

Closed
IanMayo wants to merge 8 commits into
mainfrom
claude/graphify-tool-integration-vzn8pl
Closed

IanMayo wants to merge 8 commits into
mainfrom
claude/graphify-tool-integration-vzn8pl

Conversation

@IanMayo

@IanMayo IanMayo commented Aug 7, 2026

Copy link
Copy Markdown
Member

Why

Two things had outgrown grep.

Source. 1,581 TypeScript/Python files across ~20 workspace packages under two package managers, so "what actually consumes this?" was grep plus judgement.

The ADR citation network. 39 decisions live as ### ADR-NNN: headings inside a single ~19k-word decisions.md, cited 608 times across markdown, TypeScript, Python and LinkML YAML — ADR-030 alone 63 times. scripts/check-adr-refs.sh validated only the LinkML YAML subset, so roughly 90% of citations were unchecked, and the network was navigable only by grepping one very large file.

Scope is developer tooling only — not analyst-facing, not product code, nothing shipped changes.

What this adds

  • graphifyy==0.9.35 (Apache-2.0), invoked via uvx, never a workspace dependency so it cannot reach a shipped artefact (Article IX).
  • Code-only extraction — tree-sitter AST, zero LLM calls, no API key, no network. Verified byte-identical across consecutive builds; GRAPH_REPORT.md records Token cost: 0 input · 0 output. ~14,300 nodes / ~30,000 edges in ~45s.
  • scripts/extract-adr-graph.py — extracts the ADR citation graph deterministically (39 nodes, 501 cites edges).
  • scripts/check-adr-refs.sh widened from LinkML YAML to the whole repo, plus malformed-ID detection. ~1.5s, stays in task lint.
  • .mcp.json registers the first-party MCP server so Claude Code queries the graph instead of grepping.
  • task graph:build / refresh / report / pages, .graphifyignore for scoping.
  • graphify-publish.yml deploys the committed pages to gh-pages under code-graph/, plus an advisory freshness job.

Why we extract the ADR graph ourselves

Two graphify behaviours made its own extraction unusable here:

  1. Its generic docref extractor zero-pads IDs to four digits, so its nodes never join our three-digit convention.
  2. --code-only skips markdown entirely — where most citations live.

Edges anchor to graphify's recorded source_file values rather than re-derived node IDs, so they survive upstream changes to ID slugging. The in-place merge is idempotent. Graphify's own merge-graphs is unusable — it is built for cross-repository merges and namespaces every node (repo::…), severing exactly those anchors.

This is not a gate

knip, the custom rules in shared/eslint-rules/, pyright and tsc keep answering closed questions and failing the build. The graph answers open ones. Its clearest payoff is surfacing couplings that should become new shared/eslint-rules/ entries — that directory has grown whenever someone happened to notice a pattern; this makes it deliberate.

The publish workflow never runs on pull requests and cannot block a merge.

Known limits (recorded in ADR-041 and the guide)

  • The LinkML chain is invisible. shared/schemas LinkML → generated Pydantic/TS → consumers crosses a code generator no AST parser sees. The graph shows consumers of the generated types but cannot link them back to the schema. This is the chain Article II and ADR-033 care most about — do not read coverage into it.
  • The graph is an index, never a record. Even code-only extraction is 97% EXTRACTED / 3% INFERRED. It is not evidence and must never be cited as provenance.
  • No repo-wide force-directed page. Graphify refuses that rendering above 5,000 nodes; we have ~14,300. Published pages are a D3 collapsible tree plus the smaller interactive ADR graph.
  • Community labels are placeholders — naming needs an LLM backend we deliberately do not configure. god-nodes is unaffected and gives meaningful hubs (SelectionContext 237 edges, ContextType 144, create_catalog() 116).
  • specs/ excluded — 2.6M words, value banked on delivery.

Repository weight

graphify-out/ is committed (~17MB, of which graph.json is 15MB) so a fresh clone can query immediately and the published HTML is deployed rather than regenerated. cache/ and the incremental manifest stay ignored as machine state. The merge writes compact + sort_keys, ~3MB smaller than pretty-printed and byte-reproducible.

Ordering is load-bearing: cluster-only rewrites graph.json with its own serialiser, re-inflating it and collapsing same-endpoint edges (501 → 266). The ADR merge runs last in every task.

Verification

Check Result
Two consecutive builds byte-identical ✅ same SHA256
Build with no API key / no network ✅ Token cost: 0 input · 0 output
--merge idempotent over 3 runs ✅ stable +172 nodes / +501 edges
affected "ADR-030" --relation cites ✅ matches grep ground truth with file:line
explain "ADR-033" ✅ resolves title, degree 14
check-adr-refs.sh ✅ 608 citations / 39 ADRs; negative test fails correctly
ruff check . ✅ passes
Repo lint guards ✅ pass (check-eslint-drift-wiring.cjs fails on a clean tree too — pre-existing, unrelated)

Findings surfaced along the way


Generated by Claude Code

The source tree is 1,581 TS/Python files across ~20 workspace packages, so
"what consumes this?" was grep plus judgement. Separately, the ADR citation
network — 39 decisions living as `### ADR-NNN:` headings inside one ~19k-word
decisions.md, cited 608 times repo-wide — was navigable only by grepping that
file, and check-adr-refs.sh validated just the LinkML YAML subset (~10% of
citations).

Adds graphify (graphifyy==0.9.35, Apache-2.0) as developer tooling only. Not
analyst-facing, not product code, nothing shipped changes.

- Code-only extraction: tree-sitter AST, zero LLM calls, no API key, no
  network. Verified byte-identical across consecutive builds; GRAPH_REPORT.md
  records "Token cost: 0 input · 0 output".
- Invoked via `uvx --from graphifyy==0.9.35`, never a workspace dependency, so
  it cannot reach a shipped artefact (Article IX).
- scripts/extract-adr-graph.py extracts the ADR citation graph deterministically.
  Graphify's own docref extractor zero-pads IDs to four digits (so its nodes
  never join our three-digit convention) and --code-only skips markdown, where
  most citations live. Edges anchor to graphify's recorded source_file values
  rather than re-derived node IDs; the in-place merge is idempotent. Graphify's
  merge-graphs is unusable here — it namespaces node IDs (repo::…), severing
  those anchors.
- check-adr-refs.sh widened from LinkML YAML to the whole repo, plus
  malformed-ID detection. Runs in ~1.5s. ADR-031/032 allowlisted as
  reserved-but-unwritten per the numbering note in decisions.md.
- .mcp.json registers the first-party MCP server so Claude Code queries the
  graph instead of grepping.
- graphify-out/ is committed: the graph is what the MCP server reads on a fresh
  clone, and the published HTML is deployed rather than regenerated. The SHA256
  cache and incremental manifest stay ignored as machine state.
- graphify-publish.yml deploys the committed pages to gh-pages under
  code-graph/ beside spec-navigator, plus an advisory freshness job. It never
  runs on pull requests and cannot block a merge.

Not a gate: knip, shared/eslint-rules, pyright and tsc keep answering closed
questions. The graph answers open ones, and its clearest payoff is surfacing
couplings that should become new shared/eslint-rules entries.

Known limits, recorded in ADR-041 and the developer guide: the LinkML →
generated-types → consumers chain is invisible (no AST parser crosses a code
generator); specs/ is excluded (2.6M words, value banked on delivery); the
repo-wide force-directed page does not exist (graphify refuses above 5,000
nodes, we have ~14,300), so the published pages are a D3 tree plus the smaller
ADR graph.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012sWicQ3ssncgLmqYHw2VfA
@github-actions

github-actions Bot commented Aug 7, 2026 •

Copy link
Copy Markdown
Contributor

📋 Backlog Navigator

Review this PR's BACKLOG.md edits interactively:

👉 Open in Backlog Navigator

Updated for a6cde7e — the navigator reads BACKLOG.md live from this PR's head branch, so no rebuild is needed.

@IanMayo
IanMayo temporarily deployed to debrief-preview-pr-677 August 7, 2026 11:36 Inactive
Three fixes after the first CI run.

1. scripts/extract-adr-graph.py formatted with ruff format (the only genuine
   ruff finding attributable to this branch).

2. python-lint.yml now runs ruff from the lockfile (`uv run ruff`) instead of
   `uvx ruff`. The unpinned form silently drifts to whatever is latest at run
   time: a newer ruff enabled Markdown code-block formatting by default, which
   turned 104 untouched spec/evidence files red on the first PR to add a .py
   file (the workflow is path-filtered on **/*.py, so the drift stayed latent
   until then). CI scanned 3,322 files where local scanned 242 — the same
   command could never agree across the two. Pinning to uv.lock matches
   `task lint` and local development, and satisfies Article IX.2.

   This does not reformat the 104 pre-existing files; it stops an unpinned
   toolchain from deciding the repo's formatting rules. Worth a follow-up
   decision on whether to adopt Markdown code-block formatting deliberately.

3. Excluded graphify-out/ from the ADR extractor's own input. Now that the
   directory is committed it is git-tracked, so the graph's ADR node labels were
   being re-scanned as citations of themselves — edges climbed 501 -> 590 on
   the first rebuild and would never have converged. The exclusion sits before
   the file read so the 15MB graph.json is not loaded just to be discarded.
   Verified: three consecutive merges now report a stable +174 nodes / +510
   edges.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012sWicQ3ssncgLmqYHw2VfA
@IanMayo
IanMayo temporarily deployed to debrief-preview-pr-677 August 7, 2026 11:40 Inactive
Graphify refuses force-directed rendering above 5,000 nodes and the whole-repo
graph is ~14,300, so the interactive view was simply absent. Scoping is the only
lever — extraction is symbol-level with no granularity flag, and raising
GRAPHIFY_VIZ_NODE_LIMIT would produce an unreadable ~14MB hairball.

Measured both candidate splits. By top-level package everything fits but barely:
services 4,744 / shared 4,636 / apps 4,099, i.e. 5-18% headroom before pages
silently stop rendering. By workspace package the largest is shared/components at
3,154, with years of room.

Adds force-directed drill-downs for the five packages worth exploring —
shared/components (3,154), apps/vscode (1,784), services/calc (1,701),
services/io (1,208), apps/web-shell (863). The other ~18 workspace packages are
all under 600 nodes and read fine from the tree.

- scripts/build-graph-pages.sh assembles the whole published tree: landing index,
  whole-repo D3 tree, ADR graph, per-package drill-downs, GRAPH_REPORT.md. Node
  counts on the landing page are collected during the build rather than
  hard-coded, so they cannot rot. PACKAGES is the one list to edit when a package
  grows past ~800 nodes.
- task graph:pages now delegates to it; the publish workflow verifies the
  required pages exist rather than assembling anything itself.
- Fixed the freshness job, which as written would have reported "stale" on every
  run: a graph can never carry the SHA of the commit that contains it, so it
  always trails HEAD by one. It now compares the build commit against the last
  commit touching services/, shared/ or apps/, and reports how many graphed-source
  commits it is behind.

Committed page weight is 9.5MB, taking graphify-out/ to 26MB.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012sWicQ3ssncgLmqYHw2VfA
@IanMayo
IanMayo temporarily deployed to debrief-preview-pr-677 August 7, 2026 11:55 Inactive
The production publisher only fires on main, so a PR touching the graph had
nothing to click through — the interactive pages could only be reviewed after
merge, which is backwards.

Adds Preview Code Graph (branch), mirroring the existing spec-navigator-preview
pattern: workflow_dispatch with a branch picker, same slug derivation, same
refusal to publish main as a preview, deployed to
code-graph-preview/<slug>/ on gh-pages under keep_files: true so it cannot
clobber production or a sibling preview. Like the publisher it deploys committed
bytes rather than rebuilding. The job summary prints the URLs.

Also documents the zero-setup alternative — the pages are committed, so checking
out the branch and opening graphify-out/pages/index.html works with no server.

Records a caveat found while checking this: the pages are not self-contained.
The tree loads D3 from d3js.org and the force-directed views load vis-network
from unpkg.com, so they need network access to render. Fine for developer
tooling on a public site, but worth stating plainly given the product is
offline by default (Article III.4). Vendoring both would remove it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012sWicQ3ssncgLmqYHw2VfA
@IanMayo
IanMayo temporarily deployed to debrief-preview-pr-677 August 7, 2026 12:02 Inactive
A paths filter does not match on branch creation, so the first push of a new
preview/ branch silently produced no run — precisely the case the trigger
exists for. Pushing to preview/** is already an explicit request to deploy,
so there is nothing worth filtering.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012sWicQ3ssncgLmqYHw2VfA
@IanMayo
IanMayo temporarily deployed to debrief-preview-pr-677 August 7, 2026 12:03 Inactive
The published pages load D3 and vis-network from CDNs and so need internet
access. Confirmed as expected for development and maintenance tooling —
Article III.4's offline-by-default guarantee constrains the product, not the
toolchain used to build it. Documented as a settled decision rather than an
open caveat, with an explicit note not to vendor the libraries.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012sWicQ3ssncgLmqYHw2VfA
@IanMayo
IanMayo temporarily deployed to debrief-preview-pr-677 August 7, 2026 12:07 Inactive
The guide only described the workflow_dispatch route, which GitHub does not
expose until the workflow exists on the default branch — dispatch 404s until
this PR merges. Adds the push-to-preview/** route that works from any branch,
clarifies that the slug derives from the deployed branch name, and notes that
a 404 immediately after a green deploy is Pages propagation rather than a
failure.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012sWicQ3ssncgLmqYHw2VfA
@IanMayo
IanMayo temporarily deployed to debrief-preview-pr-677 August 7, 2026 12:08 Inactive
Previewing the graph still required knowing to push a preview/ branch or
dispatch a workflow, so a reviewer opening the PR had nothing to click.

Adds Code Graph PR Preview: on any PR touching graphify-out/pages/**, deploys
that PR's committed pages to pr-preview/pr-<n>/code-graph/ (the convention
already in use on gh-pages) and posts a sticky comment linking the landing
page, whole-repo tree, ADR graph and report. keep_files: true so it cannot
clobber production or a sibling preview.

Deploys committed bytes with no build step from the PR, but is still gated to
same-repository PRs: publishing fork-authored HTML to debrief.github.io would
let any contributor serve arbitrary content from our Pages domain. Fork PRs
get a comment explaining the checkout and preview/-branch routes instead.

The workflow file is listed in its own paths filter so that editing the
previewer re-runs it on the open PR — without that you cannot tell whether a
change works until after merge.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012sWicQ3ssncgLmqYHw2VfA
@IanMayo
IanMayo temporarily deployed to debrief-preview-pr-677 August 7, 2026 12:12 Inactive
@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

🕸️ Code Graph Preview

The interactive knowledge graph built from this PR:

👉 Open the code graph

Page Scope
Whole repository Collapsible tree, all nodes
ADR citation graph Which code and docs cite which decision
Graph report Hubs and communities

Updated for a6cde7e — deployed from the pages committed on this branch, not rebuilt.
GitHub Pages can take a minute to serve a fresh deploy.
The graph needs internet access (D3 and vis-network load from CDNs).

IanMayo commented Aug 7, 2026

Copy link
Copy Markdown
Member Author

Closing — the graph did not earn its place. Recording the evidence so this isn't re-litigated later.

What was measured

The graph was trialled against the 20 most recent open issues. Result: one clear win, two partial, seventeen no help.

The win (#675 — Layers panel not refreshing after a Chat-agent change) was real and verified: affected "LayersTreeProvider" returns seven dependents, and runToolTool.ts is not among them, while executeTool.ts is and calls addResultLayer at line 329. The Chat-agent path structurally cannot refresh the Layers panel. But grep -rln LayersTreeProvider yields the same file list — the graph's contribution was presentation, not capability.

Why it doesn't generalise

  • It is precise only when you already know the symbol. affected "<known symbol>" gives exact dependents with line numbers. query "what enables tools when a feature is selected" is keyword-seeded BFS with no semantic understanding — it seeded from Selected (a Storybook story), tools() (the CLI) and TOOLS (a model-routing probe), then walked outward from those wrong seeds. A developer picking up an unfamiliar ticket is in the second case by definition, so the tool is sharpest exactly where it is needed least.
  • It cannot cross Python↔TypeScript. That boundary is MCP/stdio at runtime and invisible to AST — the same root cause as the LinkML blind spot documented in the PR body. Issue Selecting feature in layers panel does not enable context-sensitive tools #674 spans exactly that boundary.
  • The ticket flow does not fit. Of the 20 issues, 14 are new-feature specs (7 of them the E09 epic) and 2 are TODO promotions that already cite exact file:line. Only 4 are "navigate existing code" problems.
  • The published pages were reviewed and judged not useful — 9.5MB of the 26MB, plus three workflows to maintain.

That does not justify 26MB of committed artefacts, four workflows, and ongoing rebuild discipline.

What survives

The widened scripts/check-adr-refs.sh is independent of graphify and stands on its own. It validates 610 ADR citations repo-wide instead of ~60 (previously only LinkML YAML — roughly 90% went unchecked), adds malformed-ID detection, and immediately found ADR-031 and ADR-032 cited from 12 files but never written. Around 130 lines of bash, no artefacts, no workflows.

That is being reopened as a small replacement PR.

Also worth keeping separately

python-lint.yml runs unpinned uvx ruff, so CI drifts to whatever is latest while task lint uses lockfile-pinned uv run ruff. The two can never agree — a newer ruff enabled Markdown code-block formatting and flagged 104 untouched files. That is a genuine pre-existing bug (Article IX.2) and is worth fixing on its own, independently of anything here.


Generated by Claude Code

@IanMayo IanMayo closed this Aug 7, 2026
IanMayo pushed a commit that referenced this pull request Aug 7, 2026
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

This branch was previously deployed

1 inactive deployment
debrief-preview-pr-677 — a6cde7ef Deployed Aug 7, 2026 by IanMayo
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants