Skip to content

Exercise the Claude Code deployed wiring end to end (comms-hsym) - #208

Merged
GraemeF merged 2 commits into
mainfrom
comms-hsym
Jul 31, 2026
Merged

GraemeF merged 2 commits into
mainfrom
comms-hsym

Conversation

@GraemeF

@GraemeF GraemeF commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Two commits, separable: a CI unblock, then the coverage this bead is about.

1. fix(ci) — main is currently red, for an unrelated reason

main's own CI (run 30475586392) fails on //#test:hermes, and has since hermes-agent 0.19.0 hit PyPI. The gate installs the host Hermes --no-deps on purpose, so the floating <1 upper bound means any dependency hermes-agent's import chain grows becomes a collection error here — 0.19.0 grew requests. Bounding at <0.19 resolves 0.18.2 and the suite is green (ruff clean, 91 passed).

This is not part of comms-hsym. It is in this PR because the pre-commit hook blocks all work until the gate passes, and the alternative was to evade the hook. comms-92b5 carries the follow-up question — whether to keep a bound at all, or add requests to the dev group and float again — which needs someone who knows which Hermes the pod image actually runs.

2. test: — the deployed wiring, executed

The gap. Every identity test asserted one half of the supply chain against a model of the other half:

  • tools-session.test.ts runs the real cache and adapter, but hand-writes session_id into callTool arguments. It asserts the seam works given the argument arrives.
  • hooks-manifest.test.ts and hooks.test.ts regex-scan and literal-pin the manifest, the hook and the tool schemas against each other. They execute nothing.

Each link was held to its own written-down spec; the composition was held to nothing.

What this adds. A rig whose premises are the shipped artefacts rather than its author's model of them:

  • the matcher and the hook command come from the real hooks/hooks.json;
  • the injection is performed by spawning the real inject-session-id.ts with the manifest's own interpreter, fed a genuine PreToolUse event on stdin;
  • the server is the real makeProgram, whose type signature forces the rig to supply exactly production's requirement set — there is no friendlier subset to reach for;
  • ephemeral mode is expressed the way deployment expresses it: by the absence of COMMY_BOT_NAME. server.ts picks the cache; the test never names a cache constructor.

The only substitution is the substrate, and the memory adapter reaches requireBound() on the same six verbs the Zulip adapter reaches boundHttp() on — so the bind decision observed is the deployed one.

The centrepiece is a table of all 21 shipped tools and what the deployed configuration does with each: 18 usable, and the three comms-tww6 verbs refusing for want of an identity because they sit outside the matcher. That table is the invariant the bead asks for — a tool that works on this fleet today must still work after the change — and a regression renders as a one-line diff naming the tool.

Teeth. Two negative controls sever a supply channel and require a refusal, so a green here cannot survive broken wiring. Verified by mutating the real artefacts and watching it go red:

mutation result
post dropped from the shipped matcher 7 of 8 fail
hook stops emitting updatedInput 7 of 8 fail
tools.ts stops threading the per-call session context 6 of 8 fail
react dropped from the matcher table diffs "react": "usable""refused-for-want-of-identity"

One honest caveat. I ran six mutations against the pre-existing suite looking for one it misses, and did not find one — the per-link pins are thorough. So the bead's original "green suite, broken production wiring" framing is weaker at HEAD than when it was filed, largely because #197 brought tools-session.test.ts. What remains, and what this closes, is that the pins are all agreement between written-down specs: if the shared premise about what Claude Code does is wrong, every pin agrees and every pin is wrong together. This rig is the only thing that runs the chain, so it is the only thing that can disagree with that premise.

Out of scope and untouched: the three comms-tww6 verbs are characterised as they are, not fixed; no session-id supply channel is touched, removed or narrowed.

GraemeF added 2 commits July 29, 2026 18:55
…annot satisfy (comms-92b5)

The hermes gate installs the host Hermes with --no-deps on purpose: the pod
provides its tree at runtime, so the test environment only needs what the
registration import path touches. That makes the upper bound load-bearing —
any dependency hermes-agent's import chain grows becomes a collection error
here, on a repo where nothing changed.

0.19.0 grew one. gateway -> agent.turn_context -> agent.conversation_compression
-> agent.model_metadata now imports requests, so every test module fails to
collect with ModuleNotFoundError. main's own CI has been red since, blocking
every PR.

Bounding at <0.19 resolves 0.18.2 and the suite is green (ruff clean, 91
passed). Whether to keep a bound at all, or add requests to the dev group and
float again, needs someone who knows which Hermes the pod image runs; comms-92b5
carries that question.

Claude-Session: https://claude.ai/code/session_013b6swgcUfN24THVVJwgzFT
Every identity test in this suite asserted one half of the supply chain
against a model of the other half. tools-session.test.ts runs the real cache
and adapter but hand-writes session_id into callTool arguments;
hooks-manifest.test.ts and hooks.test.ts pin the manifest, the hook and the
tool schemas against each other by reading source. Nothing executed the chain,
so each link was held to its own written-down spec and the composition was held
to nothing.

This adds a rig that runs it: the real hooks.json matcher, the real
inject-session-id.ts spawned with the manifest's own interpreter and fed a
genuine PreToolUse event, and the real makeProgram booted with no
COMMY_BOT_NAME and no CLAUDE_CODE_SESSION_ID — ephemeral mode expressed the way
deployment expresses it, so server.ts picks the cache rather than the test
naming one.

The centrepiece is a table of all 21 shipped tools and what the deployed
configuration does with each: 18 usable, and the three comms-tww6 verbs
refusing for want of an identity because they sit outside the matcher. That
table is the invariant — a tool that works on this fleet today must still work
after the change — and a regression renders as a one-line diff naming the tool.

Two negative controls sever a supply channel and require a refusal, so a green
here cannot survive broken wiring: the matcher stripped of post, and a host
that supplies no session id.

Behaviour is unchanged; no supply channel is touched and the tww6 verbs are
characterised as they are, not fixed.

Claude-Session: https://claude.ai/code/session_013b6swgcUfN24THVVJwgzFT
@GraemeF
GraemeF merged commit 8ff5223 into main Jul 31, 2026
2 checks passed
@GraemeF
GraemeF deleted the comms-hsym branch July 31, 2026 06:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant