Skip to content

feat(graphify): manage graphify as the 7th tool of the stack - #20

Merged
ousamabenyounes merged 2 commits into
mainfrom
feat/graphify-seventh-tool
Sep 4, 2026
Merged

ousamabenyounes merged 2 commits into
mainfrom
feat/graphify-seventh-tool

Conversation

@ousamabenyounes

Copy link
Copy Markdown
Collaborator

Why

graphify (Graphify-Labs/graphify) owns the one lane the other six can't reach: repo/doc discovery.

Without it, an agent answers "where is this wired" / "what breaks if I change X" with a burst of rg / find / sed / cat sweeps — routinely the single largest avoidable read in a session log. RTK can only compress what those commands already printed; it cannot stop them being run. graphify replaces the sweep with one bounded graphify query.

It was already in the repo — but only as an unmanaged TRY candidate lumped into Probe / Stacklit / Serena / Graphify in scan.sh: no state, no version, no upgrade path, no telemetry. This PR promotes it to a first-class managed tool on every surface.

What changed

Surface Change
status.sh 7th row + --json key. OK requires both halves — CLI on PATH and a registered skill. A CLI with no SKILL.md is installed-disabled: the graph can be built by hand but the assistant never reaches for it. Gates the exit code like the other six.
check-updates.sh installed from graphify --version, latest from the PyPI JSON API for graphifyy.
upgrade.sh routes to whichever installer owns the package (uv tool → pipx → pip), then re-runs graphify install.
toggle.sh refused like rtk/pxpipe (exit 3), printing its real on/off mechanism: graphify install / graphify uninstall.
tokenwar-statusline.sh [graphify <v>] badge, wired into the shared ⬆ update marker and the aggregate count.
gain.sh real telemetry from graphify benchmark on ~/.graphify/global-graph.json.
install.sh --with-graphify, folded into --all.
scan.sh the candidate row becomes a managed graphify row with a real state; Probe / Stacklit / Serena stays a candidate.
check.sh R4 also asserts the graphify CLI is present.

Three decisions worth reviewing

1. No pinned "latest" constant. pxpipe uses PXPIPE_NPM_VERSION="0.10.0". graphify deliberately does not: it ships weekly (0.9.49 → 0.9.53 in ten days), so a hardcoded number would go stale between tokenwar releases and report phantom up-to-date. The PyPI registry is queried with a 10s timeout, and every failure path — no curl, no network, malformed payload — degrades to unknown, never to a wrong verdict.

The payload is piped straight into node rather than staged in an env var. PyPI's project JSON lists every release file ever published and already exceeds the argv/env limit — the first implementation failed with Argument list too long and silently reported unknown. Caught locally; there is a test for it now.

2. pip -U is never run against a uv/pipx-owned install. Both keep the package in their own isolated venv. pip install -U graphifyy writes to an unrelated environment: the graphify on PATH stays on the old version while the upgrade reports success. upgrade.sh asks each manager whether it owns the package first. There is a dedicated regression test (upgrade never runs pip against a uv-owned graphify).

The upgrade also re-runs graphify install: skill files are copied at install time, so a version bump alone leaves the assistant reading the previous playbook.

3. graphify's number is real but is NOT summed into TOTAL. graphify benchmark is deterministic and offline, and reports:

  Corpus:          11,800 words → ~15,733 tokens (naive)
  Graph:           236 nodes, 284 edges
  Avg query cost:  ~347 tokens
  Reduction:       45.3x fewer tokens per query

That is a ratio per query, not a counter of tokens already saved. Summing it into TOTAL (tools) would credit the stack for every query nobody has made yet — precisely the fabrication this repo refuses elsewhere (caveman, ponytail). So the ratio goes in the note, the token column stays N/A, and TOTAL is unchanged. A test pins that: graphify's per-query ratio is NOT summed into TOTAL.

  tool            saved       note
  ─────────────────────────────────────────────────────────────
  RTK             45.9M       54008 commands (69.1%)
  claude-mem      4.9M        ~est: 98401 obs + 23338 summaries across 36 projects
  pxpipe          N/A         pxpipe events log not found
  graphify        N/A         236 nodes in the global graph, 45.3x fewer tokens per query …
  ─────────────────────────────────────────────────────────────
  TOTAL (tools)   50.8M       summed across tools with telemetry

Test verification (RED → GREEN)

RED — upstream main scripts, new tests applied (implementation stashed, tests/ kept):

not ok 1  statusline renders a green graphify badge with its version
not ok 4  check-updates reads graphify's latest version from the PyPI JSON API
not ok 5  check-updates degrades graphify to unknown when the registry is unreachable
not ok 7  upgrade uses uv tool when uv owns the package, then refreshes the skill
not ok 8  upgrade falls back to pipx when uv does not own the package
not ok 10 upgrade reports failure when no python installer is available
not ok 11 toggle refuses graphify and points at graphify install/uninstall
not ok 24 graphify N/A with an actionable note when no global graph exists
not ok 26 graphify surfaces the measured reduction ratio from its own benchmark
not ok 28 --json exposes graphify with zero saved_tokens and the ratio in its note
not ok 29 exit 0 when all 7 tools healthy
not ok 37 graphify absent → not-installed and exit 1
not ok 38 graphify CLI without a registered skill reports installed-disabled
not ok 40 scan recommends shell, context, memory, and code tools from local logs
…
21 not ok / 25 ok

GREEN — with the implementation:

46/46 ok on graphify.bats + gain.bats + status.bats + scan.bats
143/143 ok on the full suite (121 before this branch → 22 new tests)
shellcheck -S warning scripts/*.sh scripts/lib/*.sh install.sh uninstall.sh → clean

CI

New JSON contract smoke step. The text tables and the JSON contract are rendered by separate code paths, so a tool can be added to one and forgotten in the other — and tokenwar scan reads the JSON. The step asserts all 7 tools are keys of status.sh --json and all 6 measurable ones are keys of gain.sh --json. Exit codes are ignored on purpose: a bare runner has none of the tools installed, and the step checks the contract, not health.

shellcheck now also covers install.sh and uninstall.sh, which were previously unlinted.

Also fixed on the way

tests/status-fallback.bats asserted [[ "$output" != *"context-mode"*"installed-disabled"* ]]. A whole-output glob spans newlines, so any later row carrying installed-disabled satisfied it — the guard had stopped guarding. Scoped to the context-mode row.

Known drift, not addressed here

  • docs/tokenwar-stack.png still pictures six lanes. The README now says so explicitly rather than leaving the artwork silently wrong. Regenerating it is a design task.
  • index.html (the landing page) still says five tools — it was already two behind before this PR (pxpipe never landed there). Bringing it to seven is a separate change; folding a pxpipe catch-up into a graphify PR would make this diff unreviewable.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NXot81hzCH5ViVyXJDwX4W

ousamabenyounes and others added 2 commits September 4, 2026 18:13
graphify (Graphify-Labs/graphify) owned the one lane the other six could
not reach: repo/doc DISCOVERY. Without it an agent answers "where is this
wired" with a burst of rg/find/sed/cat sweeps, and RTK can only compress
what those commands already printed — it cannot stop them being run.
graphify replaces the sweep with one bounded `graphify query`.

It was already referenced by `tokenwar scan`, but only as an unmanaged
"TRY" candidate lumped in with Probe/Stacklit/Serena: no state, no
version, no upgrade path. This promotes it to a first-class managed tool
across every surface.

What lands:

  status.sh        7th row + --json key; OK requires BOTH halves (CLI on
                   PATH *and* a registered skill). A CLI with no SKILL.md
                   is installed-disabled — the graph exists but the agent
                   never reaches for it. Gates the exit code like the rest.
  check-updates.sh installed from `graphify --version`, latest from the
                   PyPI JSON API for `graphifyy`. Deliberately NOT a pinned
                   constant like pxpipe: graphify ships weekly, so a
                   hardcoded number reports phantom up-to-date. The payload
                   is piped into node — staged in an env var it exceeds the
                   argv limit ("Argument list too long") and silently
                   degrades to unknown.
  upgrade.sh       routes through whichever installer owns the package
                   (uv tool -> pipx -> pip), then re-runs `graphify install`
                   so the skill files match the new version. Running pip -U
                   against a uv/pipx install writes to an unrelated env: the
                   binary on PATH stays old while the upgrade reports success.
  toggle.sh        refused like rtk/pxpipe (exit 3) with its real on/off
                   mechanism: graphify install / graphify uninstall.
  statusline       [graphify <v>] badge, with the shared update arrow.
  gain.sh          real telemetry from `graphify benchmark` on the global
                   graph — but it measures a per-QUERY ratio, not cumulative
                   tokens already saved. The ratio goes in the note, the
                   token column stays N/A, and it is never summed into
                   TOTAL: crediting the stack for queries nobody has made
                   yet is exactly the fabrication this repo refuses.
  install.sh       --with-graphify (folded into --all), preferring an
                   isolated env because a shared pip install is what
                   produces upstream's ModuleNotFoundError.
  scan.sh          the candidate row becomes a managed `graphify` row with a
                   real state; `Probe / Stacklit / Serena` stays a candidate.
  check.sh         R4 also asserts the graphify CLI is present.

Docs updated in the same change: README (six -> seven, new lane section,
honest-accounting paragraph, install flags, statusline sample, scan
example), SKILL.md (tool table, activate/upgrade/test/telemetry sections),
docs/tokenwar-tools.md (graphify moves from the candidate table into the
core stack). The stack PNG still pictures six lanes and is called out as
such in the README rather than left silently wrong.

CI gains a JSON contract smoke: the text tables and the JSON contract are
rendered by separate code paths, so a tool can be added to one and
forgotten in the other — and `scan` reads the JSON.

Test verification (RED -> GREEN)

RED — upstream main scripts, new tests applied (impl stashed):
    21 failures across graphify.bats / gain.bats / status.bats / scan.bats
    not ok 1  statusline renders a green graphify badge with its version
    not ok 4  check-updates reads graphify's latest version from the PyPI JSON API
    not ok 7  upgrade uses uv tool when uv owns the package, then refreshes the skill
    not ok 26 graphify surfaces the measured reduction ratio from its own benchmark
    not ok 29 exit 0 when all 7 tools healthy
    not ok 38 graphify CLI without a registered skill reports installed-disabled
    (full log: 21 not ok, 25 ok)

GREEN — with the implementation:
    46/46 ok on those four files
    143/143 ok on the full suite (was 121 before this branch)
    shellcheck -S warning scripts/*.sh scripts/lib/*.sh install.sh uninstall.sh -> clean

Also fixes a latent test bug found on the way: status-fallback's
`[[ "$output" != *"context-mode"*"installed-disabled"* ]]` globbed across
newlines, so any later row carrying that state satisfied it. Scoped to the
context-mode row so the guard actually guards.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NXot81hzCH5ViVyXJDwX4W
…sent

CI caught a host leak the local run could not: `Verdict COMPLEMENTARY when
all PASS` stubs every non-plugin tool R4 checks, and graphify was added to
that list without a stub. On a dev box the real ~/.local/bin/graphify
satisfied the check; on a bare runner it does not, so R4 returned WARN and
the verdict became DEGRADED.

Stub graphify alongside rtk and pxpipe, and add the missing counterpart
test: R4 must WARN and name graphify when the CLI is absent. That one
narrows PATH so a developer's own graphify cannot hide the regression the
way it just did.

    RED (before): not ok 6 Verdict COMPLEMENTARY when all PASS
                  # `[[ "$output" == *"COMPLEMENTARY"* ]]' failed
    GREEN:        7/7 ok in check.bats; 144/144 in the full suite, also
                  under a CI-like PATH with ~/.local/bin stripped.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NXot81hzCH5ViVyXJDwX4W
@ousamabenyounes
ousamabenyounes merged commit 3751c9d into main Sep 4, 2026
1 check passed
@ousamabenyounes
ousamabenyounes deleted the feat/graphify-seventh-tool branch September 4, 2026 18:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant