Exercise the Claude Code deployed wiring end to end (comms-hsym) - #208
Merged
Merged
Conversation
…annot satisfy (comms-92b5) The hermes gate installs the host Hermes with --no-deps on purpose: the pod provides its tree at runtime, so the test environment only needs what the registration import path touches. That makes the upper bound load-bearing — any dependency hermes-agent's import chain grows becomes a collection error here, on a repo where nothing changed. 0.19.0 grew one. gateway -> agent.turn_context -> agent.conversation_compression -> agent.model_metadata now imports requests, so every test module fails to collect with ModuleNotFoundError. main's own CI has been red since, blocking every PR. Bounding at <0.19 resolves 0.18.2 and the suite is green (ruff clean, 91 passed). Whether to keep a bound at all, or add requests to the dev group and float again, needs someone who knows which Hermes the pod image runs; comms-92b5 carries that question. Claude-Session: https://claude.ai/code/session_013b6swgcUfN24THVVJwgzFT
Every identity test in this suite asserted one half of the supply chain against a model of the other half. tools-session.test.ts runs the real cache and adapter but hand-writes session_id into callTool arguments; hooks-manifest.test.ts and hooks.test.ts pin the manifest, the hook and the tool schemas against each other by reading source. Nothing executed the chain, so each link was held to its own written-down spec and the composition was held to nothing. This adds a rig that runs it: the real hooks.json matcher, the real inject-session-id.ts spawned with the manifest's own interpreter and fed a genuine PreToolUse event, and the real makeProgram booted with no COMMY_BOT_NAME and no CLAUDE_CODE_SESSION_ID — ephemeral mode expressed the way deployment expresses it, so server.ts picks the cache rather than the test naming one. The centrepiece is a table of all 21 shipped tools and what the deployed configuration does with each: 18 usable, and the three comms-tww6 verbs refusing for want of an identity because they sit outside the matcher. That table is the invariant — a tool that works on this fleet today must still work after the change — and a regression renders as a one-line diff naming the tool. Two negative controls sever a supply channel and require a refusal, so a green here cannot survive broken wiring: the matcher stripped of post, and a host that supplies no session id. Behaviour is unchanged; no supply channel is touched and the tww6 verbs are characterised as they are, not fixed. Claude-Session: https://claude.ai/code/session_013b6swgcUfN24THVVJwgzFT
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two commits, separable: a CI unblock, then the coverage this bead is about.
1.
fix(ci)— main is currently red, for an unrelated reasonmain's own CI (run 30475586392) fails on//#test:hermes, and has since hermes-agent 0.19.0 hit PyPI. The gate installs the host Hermes--no-depson purpose, so the floating<1upper bound means any dependency hermes-agent's import chain grows becomes a collection error here — 0.19.0 grewrequests. Bounding at<0.19resolves 0.18.2 and the suite is green (ruff clean, 91 passed).This is not part of comms-hsym. It is in this PR because the pre-commit hook blocks all work until the gate passes, and the alternative was to evade the hook.
comms-92b5carries the follow-up question — whether to keep a bound at all, or addrequeststo the dev group and float again — which needs someone who knows which Hermes the pod image actually runs.2.
test:— the deployed wiring, executedThe gap. Every identity test asserted one half of the supply chain against a model of the other half:
tools-session.test.tsruns the real cache and adapter, but hand-writessession_idintocallToolarguments. It asserts the seam works given the argument arrives.hooks-manifest.test.tsandhooks.test.tsregex-scan and literal-pin the manifest, the hook and the tool schemas against each other. They execute nothing.Each link was held to its own written-down spec; the composition was held to nothing.
What this adds. A rig whose premises are the shipped artefacts rather than its author's model of them:
hooks/hooks.json;inject-session-id.tswith the manifest's own interpreter, fed a genuine PreToolUse event on stdin;makeProgram, whose type signature forces the rig to supply exactly production's requirement set — there is no friendlier subset to reach for;COMMY_BOT_NAME.server.tspicks the cache; the test never names a cache constructor.The only substitution is the substrate, and the memory adapter reaches
requireBound()on the same six verbs the Zulip adapter reachesboundHttp()on — so the bind decision observed is the deployed one.The centrepiece is a table of all 21 shipped tools and what the deployed configuration does with each: 18 usable, and the three
comms-tww6verbs refusing for want of an identity because they sit outside the matcher. That table is the invariant the bead asks for — a tool that works on this fleet today must still work after the change — and a regression renders as a one-line diff naming the tool.Teeth. Two negative controls sever a supply channel and require a refusal, so a green here cannot survive broken wiring. Verified by mutating the real artefacts and watching it go red:
postdropped from the shipped matcherupdatedInputtools.tsstops threading the per-call session contextreactdropped from the matcher"react": "usable"→"refused-for-want-of-identity"One honest caveat. I ran six mutations against the pre-existing suite looking for one it misses, and did not find one — the per-link pins are thorough. So the bead's original "green suite, broken production wiring" framing is weaker at HEAD than when it was filed, largely because #197 brought
tools-session.test.ts. What remains, and what this closes, is that the pins are all agreement between written-down specs: if the shared premise about what Claude Code does is wrong, every pin agrees and every pin is wrong together. This rig is the only thing that runs the chain, so it is the only thing that can disagree with that premise.Out of scope and untouched: the three
comms-tww6verbs are characterised as they are, not fixed; no session-id supply channel is touched, removed or narrowed.