feat: decisions, jev, local model lifecycle and a model host by hawkeyexl · Pull Request #14 · hawkeyexl/inference

hawkeyexl · 2026-10-05T23:38:54Z

What changed

Decisions — a probability over a fixed set of answers rather than generated text — become an optional provider capability. That brings a local implementation, a hosted one, the helpers a consumer needs to prepare a local model, and a host that keeps local models loaded across processes. manni's proposal 0079 (hawkeyexl/manni#163) is the consumer: manni tracevals judges each Claude Code turn from a Stop hook, where every hook is a short-lived process.

ADR	Change
01013	`DecisionProvider`, `canDecide`, `decide` and `stateLimit` in Jev's request shape. `MockProvider` decides for offline tests. `InferenceProvider` is unchanged.
01014	`ensureModel` (downloads without loading, behind a cross-process lock), `modelState` and `fits`
01015	`jev`, a decide-only hosted provider that is never auto-detected and has no price entry
01016	llama-cpp `decide`, from the next-token probabilities of option letters, with the shared state evaluated once (erase or checkpoint, falling back to re-evaluation)
01017	A detached model host over a user-only named pipe or Unix socket. It has a token handshake, a first-in-first-out queue per model, session leases and `keepAlive`, plus `leaseModelHost`, `modelHostStatus`, `releaseModelHost` and `ModelHostBusyError`. `llamaCpp.host` defaults to `"off"`, so existing consumers see no change.

Also included: test(llama-cpp), which gives the every-backend-crash test room on a slow runner. It timed out at 5 seconds on two release runs.

Measured on one RTX 4090 with qwen3.5-4b:

Measurement	Result
Decide, CUDA, checkpoint reuse	About 160 ms per question
Decide, CPU	About 10.7 s per question
Host, first call (CUDA, includes start and load)	6.8 s
Host, a second client's call	1.1 s

Docs impact

The reference pages gained new sections and messages:

providers.mdx: Decisions, Mock decisions, Jev.
local-models.mdx: Decisions, The model host.
errors.mdx, warnings.mdx, errors-and-types.mdx: the new messages and types.
proposed-ia.md is updated.

check-docs-exports and check-error-coverage pass. The docs build, link checks and Doc Detective pass locally.

Verification

npm run typecheck, npm run build and npm test pass: 411 passed, 13 skipped (the live tests).
INFERENCE_LIVE_LLAMA=1 was run by hand for decide and for the host, on qwen3.5-4b and granite-4.1-3b-q2.
The prerelease 0.5.0-decide-and-host.3 is published from this branch, and manni builds against it.

🤖 Generated with Claude Code

A DecisionProvider answers a fixed set of options with a probability per option, in the request shape of TypeSafe's Jev System One API. canDecide detects it, so the base completeJSON contract is unchanged. One shared rule normalizes every provider's weights, and MockProvider decides offline from scripted choices or weights (mockDecisions, mockStateLimit). ADR 01013. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

# [0.5.0-decide-and-host.1](v0.4.0...v0.5.0-decide-and-host.1) (2026-10-05) ### Features * **decide:** add decisions as an optional provider capability ([4163e99](4163e99))

…cal model without loading it ensureModel downloads the weights and the runtime under a cross-process lock in the models directory. modelState reports ready, downloading or missing without waiting. fits compares the model size times the shared tierForBudget headroom with the provider's memory budget. The install lock is generalised into withDirLock and now reclaims a lock whose holder process is dead. ADR 01014. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…afe's Jev JevProvider posts a DecideRequest to /v1/systemone and normalizes the probabilities through the shared rule. It is a DecisionProvider only: completeJSON rejects, stateLimit is 32000, and detection never picks it. ProviderSpec.baseUrl and timeoutMs now also apply to jev. jev-1.13.0 is priced at $0.042 per million input tokens; the jev-latest alias is not. ADR 01015. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

jev carries no price. Cost reporting is not part of what the decision provider is for, so the pinned-model entry, its docs row and its test go. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Three backends crash in turn, each in a fresh worker, which takes about 5.5s on a GitHub runner against the 5s default. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

LlamaCppProvider is now a DecisionProvider. decide() letters each question's options A to Z after the state, opens the assistant turn with "Answer:" in the model's own chat template with thinking off, and reads the next-token probability of each letter, with and without a leading space, before the shared normalizeDecision rule. stateLimit() reports the room contextFor would allow for state plus the longest question. The worker evaluates the tokens every question shares once, then each question's tail, and erases back between questions. node-llama-cpp removes the tokens, restores a checkpoint on hybrid models such as Qwen3.5, or evaluates the state again; the token meter tells which, and the runtime reports it as reuse. A decide request crosses the worker boundary like a prompt, so a crashing backend still falls back. ADR 01016. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

# [0.5.0-decide-and-host.2](v0.5.0-decide-and-host.1...v0.5.0-decide-and-host.2) (2026-10-05) ### Bug Fixes * **jev:** drop the price table entry ([7486e9d](7486e9d)) ### Features * **jev:** add the jev provider, a hosted decision provider for TypeSafe's Jev ([cab2bcc](cab2bcc)) * **llama-cpp:** add ensureModel, modelState and fits to prepare a local model without loading it ([b7a46ee](b7a46ee))

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

# [0.5.0-decide-and-host.3](v0.5.0-decide-and-host.2...v0.5.0-decide-and-host.3) (2026-10-05) ### Features * **llama-cpp:** answer decisions from option-letter probabilities ([dcd9d59](dcd9d59))

… host A consumer that runs from hooks starts a fresh process per hook, so each paid a full model load and concurrent hooks each loaded their own copy. llamaCpp.host ("off" by default, "connect", "spawn") routes completeJSON, decide and stateLimit to a detached host (llama-hostd.js) that runs the same provider, worker and backend fallback, and keeps the weights loaded. The host listens on a per-user named pipe or 0700 Unix socket under the runtime directory, authenticates with a 0600 token file, and takes a start lock that a dead holder's successor reclaims. Calls queue FIFO per model, one in flight; a second model loads only if fits says it fits, else the idle one unloads first. hostWaitMs withdraws a call still queued with ModelHostBusyError. Models unload when no lease holds them and they are idle past keepAlive; the host exits when nothing is loaded and no client is connected. Adds leaseModelHost, modelHostStatus, releaseModelHost and ModelHostBusyError. ADR 01017. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

# [0.5.0-decide-and-host.4](v0.5.0-decide-and-host.3...v0.5.0-decide-and-host.4) (2026-10-05) ### Features * **llama-cpp:** keep local models loaded across processes in a model host ([fdb83e8](fdb83e8))

docmeta-release-authenticator · 2026-10-05T23:39:52Z

🎉 This PR is included in version 0.5.0-decide-and-host.4 🎉

The release is available on:

Your semantic-release bot 📦🚀

…ared prefix completeJSONShared(provider, { system, shared, items, schema }) returns one validated answer per item, asked as shared + item. On llama-cpp one session evaluates the shared prefix once and generates each answer under the schema's grammar, reusing decide's erase/checkpoint/re-evaluate machinery, in the worker and through the model host. Every other provider falls back to one completeValidatedJSON call per item with bounded concurrency. A failed item is { error } in its place, never dropped. ADR 01018. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

# [0.5.0-decide-and-host.5](v0.5.0-decide-and-host.4...v0.5.0-decide-and-host.5) (2026-10-06) ### Features * **llama-cpp:** answer many items with schema-valid JSON over one shared prefix ([4dc487e](4dc487e))

docmeta-release-authenticator · 2026-10-06T14:01:53Z

🎉 This PR is included in version 0.5.0-decide-and-host.5 🎉

The release is available on:

Your semantic-release bot 📦🚀

toStrictSchema made every property nullable, required ones included. A model could then answer null for a required field, stripNulls removed it, and the response failed the schema it was asked for. Haiku through OpenRouter did exactly that on every rule a turn did not touch. Only an optional property now becomes nullable, which is what strict mode needs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

# [0.5.0-decide-and-host.6](v0.5.0-decide-and-host.5...v0.5.0-decide-and-host.6) (2026-10-06) ### Bug Fixes * **openai:** keep a required property non-null in a strict schema ([6468f44](6468f44))

docmeta-release-authenticator · 2026-10-06T17:40:13Z

🎉 This PR is included in version 0.5.0-decide-and-host.6 🎉

The release is available on:

Your semantic-release bot 📦🚀

The provider sent no max_tokens, so a server that reserves credit for a model's whole output window refused it on a small budget. OpenRouter did that for Sonnet 5.5 at 65,536 tokens. Every request now carries openai.maxTokens, default 4096, and a response that stops at the cap throws as itself, as the anthropic provider already does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

# [0.5.0-decide-and-host.7](v0.5.0-decide-and-host.6...v0.5.0-decide-and-host.7) (2026-10-06) ### Bug Fixes * **openai:** cap every request with openai.maxTokens ([4b15e31](4b15e31))

docmeta-release-authenticator · 2026-10-06T17:48:46Z

🎉 This PR is included in version 0.5.0-decide-and-host.7 🎉

The release is available on:

Your semantic-release bot 📦🚀

…kens A decision probe built the next-token distribution over the whole vocabulary to read a few letters from it, about 58 of 82 ms a question on qwen3.5-4b. The probe now asks for the top 40 at temperature 1, with topP 1 so node-llama-cpp's default 0.95 does not cut them short. Letter ratios are unchanged, and normalizeDecision renormalizes as before. A letter outside the 40 weighs 0. A question with no letter among them is erased back to the shared start and read again over the whole vocabulary, so a decision is never empty. ADR 01019. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…pts at once Every call created a new context (165 to 220 ms), resolved the chat template again for its session (about 120 ms), and counted the system prompt and each prompt in a round trip of its own to the worker (about 30 ms for a 36-question decision). The worker now keeps one idle context per model. A session takes it, emptied with clearHistory(), when it holds at least the size asked for and at most twice it; otherwise a context of exactly that size replaces it, so ADR 01011's sizing still bounds memory. A crashed worker's replacement creates its own. The chat template is resolved once per model, and the parent sends every count asked for in one tick as one request. ADR 01019. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…n the grammar node-llama-cpp's JSON-schema grammar allows a newline and indentation before every property and a space after every colon and comma, and the model takes them up: a 36-item reasoning-first shared-prefix request spent 97% of its time decoding. node-llama-cpp 3.19 has no option for it and does not export the generator that does, so the worker rewrites the grammar text: whitespace rules become empty or a bare comma, and an inline [ ]? outside literals and classes is dropped. The stop sequence, trimming and maxLength bounds stay as they were. ADR 01019. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… ADR 01019 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

# [0.5.0](v0.4.0...v0.5.0) (2026-10-07) ### Features * decisions, jev, local model lifecycle and a model host ([#14](#14)) ([a6b5552](a6b5552))

docmeta-release-authenticator · 2026-10-07T02:13:45Z

🎉 This PR is included in version 0.5.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

The decide-and-host prerelease line merged as hawkeyexl/inference#14 and published 0.5.0, so manni no longer depends on a feature-branch build. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…a session to what it was asked (0079, 0080) (#163) * docs(proposals): 0078, family check and status, and the manni plugin Records the decision that `manni check` and `manni status` are the only two top-level family verbs, superseding 0034's "umbrella owns no verbs" rule and nothing else in it. A domain blocks only when the project set it up. One Claude Code plugin, plugin/manni, replaces plugin/tracevals. Adds CUJs D16 and M24, the family/ docs section to the IA, and the CLAUDE.md grammar and layout notes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(family): `manni check` and `manni status` run what a repo set up Two family verbs on the umbrella (proposal 0078). `status` says which domains are in play: a config section of their own, or content carrying their declarations. Built-in defaults never count. `check` runs every in-play check in process, over the collections or the given paths, and skips the rest with a reason. A Claude Code hook envelope on stdin is detected. PostToolUse narrows the run to the edited file and blocks on errors with exit 2. Stop narrows it to the working tree's changes plus the set-wide checks, and blocks once. SessionStart prints the status for the agent and exports MANNI_GENERATED_BY. meta validate gains a skip for files only the default schemas cover, and graph build can run in memory so check writes no file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(plugin): one manni plugin for Claude Code, with a marketplace plugin/manni replaces plugin/tracevals. SessionStart records the session manifest and runs `manni status`, and PostToolUse and Stop run `manni check`. Skills set up a repo, run the check, repair findings and author evals. The repository root is a marketplace, so install is `claude plugin marketplace add hawkeyexl/manni` and `claude plugin install manni@manni`. CI validates the plugin and the marketplace with `claude plugin validate --strict`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(family): status says nothing is set up when no collections are declared check reads only collection members, so with no collections every domain has nothing to check. status used to report cite, docevals and term as in play with a could-not-run reason. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(family): manni check, manni status and the Claude Code plugin A family/ section: check.mdx gates everything a repo set up (D16), claude-code.mdx installs the plugin and says what each hook does (M24), and reference/cli.mdx documents both verbs, their messages and the hook protocol. docs:check-cli verifies the reference against the umbrella's two family commands. The tracevals pages point at the plugin for session capture, and the machine-authorship page says the plugin exports MANNI_GENERATED_BY. Citations moved by the family commits are re-pinned with cite update. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(family): stamp created and provenance on the new pages Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(media): demo of manni check and manni status Script, captures, captions, transcript, thumbnail, GIF and the Remotion composition. The MP4 stays out of git. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(family): address review findings in check and status - A deletion counts as a change at a stop, so the whole-set checks still run when a session only removed a cited source. - A page that does not parse is left out of meta where only the default schemas would judge it, as a parsed page already was. - The paths scope checks documents only, so a member asset such as an .svg no longer errors meta. - A hook block carries the failing checks' reports, not the passing ones. - The session briefing names only the whole-set checks in play. - A context-window suffix such as [1m] is dropped from the exported model id. - check and status read the domain list from DOMAINS. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(family): address PR review, and put check on the docs landing page - DocevalsError carries a `nothing-resolved` code, and check tests it rather than matching message text. - A domain reporter whose JSON does not parse becomes an `error` entry, and the other checks' reports survive. - The plugin test asserts the PostToolUse matcher names exactly EDIT_TOOLS. - The session briefing lists three or more collections as `a, b or c`. - The docs landing page no longer says every command is domain-first. It introduces `manni status` and `manni check` and links the two journeys. The reference documents the JSON fallback. - The video script's Material note passes Vale. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: stamp provenance on the landing page and the family reference Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(family): status says unknown for a domain that cannot read its setup It used to say in play, which reads as set up when manni could not tell. check still runs the domain and reports what it finds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(family): stamp provenance on the reference Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(plugin): check the skill the session briefing names against the plugin Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(family): put agentLines' doc comment back on agentLines Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(proposals): 0079 tracevals checks each turn against the rules it read At Stop and SubagentStop, manni check judges the last turn against the agent-instruction files that governed it: CLAUDE.md, AGENTS.md and GEMINI.md (nested too), .claude/rules, .cursor/rules, Kiro steering, the Spec Kit constitution, imports, and skills with their reference files. A confident violation blocks once, inside 0078's repair pass. A fast model judges in the loop (Qwen3.5-4B or Jev by decision, Haiku by generation) and a stronger one extracts rules outside it. Nine gates keep the model unloaded when nothing applies, and a model host in @hawkeyexl/inference keeps local models loaded between hooks. Supersedes 0078 in part, for its tracevals in-play row. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(proposals): 0079 offers the Sonnet and Haiku split as an option Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(tracevals): extract rules from a rule source, and add command-matches The out-of-loop model reads one agent-instruction file and returns the directives a transcript can show followed or broken, each with an optional when. A gate drops a bad id, empty text, a repeat or an invalid when with a reason. Results cache by content hash, written atomically. command-matches arms a rule on a matching Bash command. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(meta): derive reads each page's git history eight at a time With provenance managed, derive runs `git blame` and a `git cat-file` per page, one page after another. On this repo's 239 pages that was 23s of git time, mostly process start-up. The pages are independent, so they now go through a bounded pool of min(8, cores); records keep input order. `meta validate` here drops from 35s to 19s and a bare `manni check` from 47s to 31s. The pool is fill's mapConcurrent, moved to src/shared/concurrency.ts with a test of its own. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(meta): re-pin the citations the derive pool moved 73 citations only moved and are re-pinned by cite update. Two are re-minted at new lines: the blame-only-with-provenance claim follows its block into the pool, and the --local claim, which on main pinned fill's worker loop, now pins the provider selection that keeps it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(tracevals): resolve the rule sources that governed a turn Every agent's instruction files count, each scoped by its own trigger: CLAUDE.md, AGENTS.md and GEMINI.md (nested too), .claude/rules with paths:, .cursor/rules with alwaysApply and globs, Kiro steering, the Spec Kit constitution, OpenSpec project.md, @ imports, and skills with the files read under their directory. Resolution is deterministic. Cursor's unquoted globs are read line by line, as Cursor reads them. A turn starts at the last prompt the user typed; reminders, meta records and hook feedback never start one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(meta): verify each commit's provenance stamp once, not once per line stampEvidence asked every blamed line to re-read its commit's page: parse the blob's frontmatter YAML, then hash each stamped range. A commit that touched many lines paid that for each one. Both stampEvidence and carriesStamp now read one cached answer per commit, the stamp entries that verify, kept in stamp order so the first covering entry still wins. On this repo `meta validate` drops from 19s to 9s and a bare `manni check` from 31s to 21s. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(meta): re-pin the citations the stamp cache moved Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * build(deps): move @hawkeyexl/inference to the decide-and-host prerelease It carries decisions, the jev provider and the model lifecycle helpers 0079 judges with. It moves to the stable 0.5.0 once that branch merges. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(family): brief the agent only when a domain is in play The SessionStart lines say manni check runs after each edit, which is untrue when collections are declared but nothing is set up. The table alone is printed then. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(family): stamp provenance on the reference Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(tracevals): judge a turn against the rules that governed it A provider that can decide gets one choice question per applicable rule, over the turn encoded once, with followed, violated, not_applicable and unclear as the options. Any other provider makes one generative call per run that lists only broken and unsure rules. A rule fails only at or above judge.zones.autoFail, unanimously, with no errored run. Raw answers cache by turn, rule set and model. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(graph): cache each page's body parse across builds graph build re-parsed every page with micromark on every run, and the manni check Stop gate builds the graph on every agent stop. Most pages do not change between runs. analyzeDoc now splits into parseBody, the corpus-free half (headings, raw link and image targets, code languages), and the link and image resolution that needs the corpus and routes. parseBody is a pure function of the bytes and the format, so buildGraph serves it from .manni/graph/analyze-cache/, one JSON file per page label, keyed on sha256 of the entry version, manni version, format, label and content. Frontmatter, resolution, the meta view and git history still run on every build, so the Turtle is byte-identical with or without the cache. A corrupt or malformed entry is a miss, and a failed write is dropped. On this repo (239 pages) graph build goes from 9.6s cold to 2.7s warm, and manni check from 25.7s to 16.7s (medians of 3). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(cli): load only the domain a command runs Every run imported every domain's module graph, Playwright, axe, Asciidoctor and the RDF stack included, so `manni --version` took about a second and a hook's `manni check <page>` paid the same before doing any work. The umbrella now imports each domain on demand and the bin mounts only the one argv names. `--version` mounts none, and help, a bare `manni` or an unknown command mount them all, so their output is unchanged. `manni check` imports each domain's core where it runs, `manni status` does the same, and lint requires Asciidoctor on the first .adoc file. The umbrella's buildProgram() is now async; its callers await it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(family): status counts cited pages without checking them manni status ran a whole cite check, with the sources off, only to count the pages that carry citations. That still read every claim end and parsed every marker, and it ran at the start of every Claude Code session. It now reads each page the way cite check does (its targets, its manifests, its entry rules) and counts the valid entries. readPage takes `markers: false` for that, so no marker is parsed. The count is the same: a marker never adds an entry. On this repo status goes from about 7.2s to 3.7s (median of 5). docevals and term keep their list paths. Both already resolve without running anything, and their cost is reading frontmatter that the count needs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(family): the stop gate runs term and graph only when a page changed At a Stop, manni check ran term check and the graph build and check over the whole set whenever the tree was dirty, about 9s here. Both read only collection documents, so a session that changed only code paid for checks that could not have moved. They now run at a Stop only when a collection document changed or was removed. The changed tree names deleted and renamed-away files apart for that, because a deleted term page can still break a reference. cite check still runs set-wide whenever the tree is dirty, since a source edit can drift any citation. A bare manni check is unchanged. The session briefing, the family CLI reference, the Claude Code page and proposal 0078's scope tables say so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(cite): index line starts once and read each source once per check lineAt and offsetOfLine scanned the page from its start on every call, so a page with hundreds of citations cost quadratic time in anchoredLines, paragraphAfter and parseStatements. They now look up a per-text index of LF offsets. `manni cite check` also re-read and re-split a source file for every citation that named it, and again for each quote check. A per-run cache, created in runCheck and passed through the page options, reads each tracked file and splits its lines once. Verbs that write files do not create one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(family): share one collection walk across the domains in a run Every domain manni check runs globs the same collections and spawns its own git check-ignore, about 150ms each on this repo. resolveTargetSet now consults a walk memo that withSharedWalks scopes to one check or status run with AsyncLocalStorage. Each caller gets a copy, a failed walk is not kept, and nothing is cached outside the run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(meta): read every page's blamed-commit blobs in one cat-file batch per root Provenance ran two git processes a page: blame, then a `cat-file --batch` for the page and manifest blobs at the commits blame named. Blames still run per page, a few at once, but the specs every page needs are now deduplicated and fetched in one batch for the whole root, and the `log --no-walk` for commits missing from a page's history is likewise one process, fed on stdin. On this repository a validate run drops from 484 git processes to 246. One batch now serves a whole root, so an answer past the read cap is asked for again in halves; only a single object past the cap is too large. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(meta): parse each manifest once per run in the orphan walk and in derive The stray-manifest walk parsed every `{page}` manifest on disk, including the ones the loader had just read for the pages the run loaded, and then threw those results away. `findPageManifests` now takes the set to skip, and the orphan check passes it the manifests it already read. Derive parsed a manifest's blob once for every page and every commit that read it, though it is mostly the same bytes each time. Within a run, each distinct manifest text is now parsed once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: re-pin the citations the speed-ups moved 207 citations only moved and are re-pinned by cite update. Ten pinned code the speed-ups rewrote; each claim still holds, and each is re-pinned at the lines that now carry it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: stamp provenance on the pages the speed-ups changed Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(tracevals): add tracevals prepare, check and release, and the conformance config tracevals.conformance puts the turn check in play. conformance.hook names the in-loop judge, falling back to tracevals.provider and model, which now also extract rules, run prepare and judge check by hand. providers.jev and providers.llama-cpp.keepAlive join the family's provider map. jev answers decisions only, so every generating tool leaves it out of its provider list and tracevals accepts it only as the hook's judge. Nine gates decide before any model loads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(family): say why RD removes two names, and what the walk key must cover Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: re-pin the citations the comments moved Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(proposals): 0079 records the details the implementation settled The verdict cache gate moves last, because its key needs the model's state limit. check's JSON may carry a null judge or extraction, and it gains an empty-turn closing line and an --offline refusal. prepare's hosted roles carry null bytes. A generating tool names jev as decisions only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(tracevals): keep the local judge loaded in the model host Under a hook a llama-cpp judge runs in the inference library's model host, leased by session with providers.llama-cpp.keepAlive as its idle timeout. prepare takes the lease at session start and loads the model, release returns it at session end, and a call queued for 120 seconds skips the turn. Runs by hand use a host that is already running and never start one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(proposals): 0079 drops the unreachable-host message The library reports a host that does not answer as no host, so release says No model host is running. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(family): say why an unparseable report is an error entry Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(family): manni check judges each turn when tracevals.conformance is set At Stop, manni check runs 0078's file checks and then tracevals' turn check, even with no collections and a clean tree. At SubagentStop it judges the subagent's own transcript. Both failures share one block, and the repair pass is shared too. A skip that needs saying is said once per session. status shows the two models and any model host. The plugin gains an async SessionStart prepare, SubagentStop and SessionEnd release. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(tracevals): catch a broken rule before the agent hands back (M25) A journey page for the turn check, the prepare, check and release reference, the conformance config, providers.jev and keepAlive, and command-matches. The family pages say tracevals judges each turn when tracevals.conformance is set. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(meta): meta fill refuses a decision-only provider The library now lists jev, which only decides, so the every-provider loop expects it refused rather than accepted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(media): demo of the turn check A rule in CLAUDE.md, a turn that broke it, manni tracevals check naming the rule with exit 1, and manni check blocking the stop. The MP4 stays out of git. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(family): the stop gate never builds the graph The graph build is the slowest whole-set check, and its findings span pages rather than the one an agent just wrote, which makes a poor fit for a one-repair-pass gate. A Stop now runs the per-file checks, every citation, and the glossary when a collection page changed. The graph stays in a bare manni check, which is what CI runs. The session briefing, the reference, the Claude Code page and proposal 0078 say so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(family): stamp provenance on the pages the stop-gate change edited Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(family): re-derive provenance for the pages the merge touched Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(tracevals): a file named in conformance.include applies to every turn A person names a file in include because it governs the agent. Waiting for the agent to read it let a turn that skipped it escape its rules, so it now applies always, as a CLAUDE.md does, and prepare warms it. The proposal also pins the verdict cache key, states that a generative block needs every run to name the same rule, says when MANNI_TRACEVALS_JUDGE is set, and cites the four-hop import depth. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(family): re-derive provenance for the pages the merge touched Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(tracevals): say a skip once under concurrency, and key extraction by temperature - sayOnce creates one marker per message exclusively, so of two parallel SubagentStop hooks only one says it. Session ids are hashed, so two never share a marker. - The turn check warns on stderr when it fails for a reason other than an operational error, rather than going silent on a bug. - The extraction cache key carries the temperature. - A backtick line inside a tilde fence no longer closes it, so an @ reference there is not read as an import. - The extraction provider builder separates mock's per-source provider from the one every other provider builds once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(tracevals): rename the shared regex memo and number gates as 0079 does compiledRe serves prompt-matches and command-matches alike. Comments now give the model-on-disk and memory gates as 7 and 8, and the verdict cache, inside judgeTurn, as 9. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: re-derive last-updated on the pages the 0078 squash merge dated Git dates their last body change to the squash commit, 2026-10-06. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(tracevals): read an SDK prompt as typed in both transcript formats isTypedPrompt accepted turnOrigin sdk but refused origin.kind sdk. Both fields now read against one set of origins, human and sdk. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(plugin): drop a stray brace the merge left in the hook helpers Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(family): re-derive provenance for the pages the merge touched Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(tracevals): keep an extracted rule whose when is empty qwen3.5-4b writes "when": {} on every rule it extracts, meaning no condition. The gate refused that as an invalid when, so the root CLAUDE.md yielded no rules at all. An empty when now reads as an absent one, and the rule applies to every turn. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(tracevals): score each rule as JSON instead of choosing a letter Each rule is its own question over the turn, answered as {followed, not-followed, not-applicable}, independent integers from 0 to 100, through the library's shared-prefix call, so a local model reads the turn once. The prompt drops the rule list and the nearer-file paragraph and keeps the user-override sentence. A rule fails at not-followed >= judge.zones.autoFail x 100 in every run. MANNI_TRACEVALS_REASONING=1 adds a reasoning field, last, for development. jev keeps its decisions, over the same three answers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(tracevals): reason before scoring, and remember earlier turns in a session ledger Each rule's answer now opens with one or two sentences of reasoning, then scores not-applicable, followed and not-followed, in that order. On the fixtures it was the only variant with no false failures that still caught every real one. MANNI_TRACEVALS_REASONING is gone. A rule about work the turn never did does not apply. The hook records each rule's outcome per turn in a session ledger, one per subagent, and later judgements see the last three outcomes that were not not-applicable. A check by hand reads the ledger and never writes it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * style(tracevals): space the type annotation on the judge result Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * build(deps): move @hawkeyexl/inference to 0.5.0-decide-and-host.7 It keeps required fields non-null in strict schemas, so Haiku through an OpenAI-compatible endpoint no longer fails every rule a turn did not touch, and it caps each openai request with openai.maxTokens. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(tracevals): cap the judge's reasoning, and skip the memory probe when nothing needs a model The reasoning is one sentence of at most 30 words, at most 240 characters, so a local model writes far fewer tokens per rule. A turn whose every verdict is cached, or whose model the host already holds, no longer forks a worker to probe memory. The state limit each model reports is remembered, so a cached turn is recognised without loading the model. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(tracevals): name the judged verdicts apart from the state-limit fetcher Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(tracevals): keep each turn's facts, pin early evidence, and report the session so far The ledger records what every turn did, even when no rule applied or none was in scope: commands run, files written, rule sources read, skills and subagents. Each judgement opens with those earlier facts, encoded once for every rule, so a rule about an earlier step sees that the step happened. A rule's first followed entry and every broken one survive the trim, and a run of turns out of scope is marked. tracevals check gains a per-rule session rollup, in pretty and JSON output. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(tracevals): let the judge reason in one or two sentences again A cap of one 30-word sentence, 240 characters, halved the reasoning tokens, but on the fixtures it added false failures for both Qwen and Haiku: a model cut off mid-reasoning commits to the wrong scores. The item asks for one or two sentences and the schema sets no length. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(tracevals): say an included rule file applies to every turn The conformance.include description said a file applies once the agent reads it. Proposal 0079 and the source resolver both hold every turn to an included file, whether or not the agent read it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(tracevals): hold a session to what it was asked and the procedures it ran (0080) (#169) * docs(proposals): 0080 tracevals holds a session to what it was asked Every rule is judged against the session so far, and blocks only on what the last turn did or claimed. The prompts the user typed, an approved plan and the specs a session touched become rule sources. A skill's procedure (order, gates, routes, conditionals, required and last steps) is read as rules, against an earlier-turns block now read from the transcript. 0079 is superseded in part. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(proposals): 0080 answers the first review and adds two known limits Rewords the exclude comment, since no glob matches typed prompts. States why followed reads the session and not-followed the last turn, with a stress test for a break after a long record. Names src/tracevals/CLAUDE.md in Touches, says the spec exclusion is reversed, and says why plans is not include. The benchmark of labeled turns adds two limits: the judge never sees tool results, and small models read overrides too freely. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(tracevals): keep a skill's rules in scope to the end of the session Under the hook and in tracevals check, a skill's or slash command's rules now apply from its first invocation to the end of the session (proposal 0080). The skills a procedure calls partway through no longer end its window. Batch run keeps ADR 01015's window. Extraction keeps a procedure's rules too: the order of steps, gates, routes, conditions, required steps and the last step. RULES_PROMPT_VERSION goes to 2, so each source is extracted again once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(tracevals): say how the per-turn check scopes a skill apart from run Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(tracevals): judge the last turn against the whole session Proposal 0080. The turn judge's wording now names the last turn and the session around it. A rule applies once the session does its kind of work, and only the last turn can break it. TURN_JUDGE_PROMPT_VERSION goes to 7, so no verdict from 0079's wording replays. "# Earlier in this session" is now read from the transcript instead of the ledger. Every earlier turn with a fact gets a line, newest last, with no cap of 15. Five kinds of fact join 0079's: a skill call, an agent spawn with the first line of its result, a question to the user, a task ticked in a task list, and a plan approved. Commands, paths and results are redacted before they are clipped. The block keeps a quarter of the turn's budget, drops its oldest lines first and says which went. The ledger still marks turns with no rules in scope. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(tracevals): hold a turn to the prompts, the approved plan and touched specs Proposal 0080, "What the session was asked". Six source formats join the turn's sources: prompt, plan, speckit-spec, kiro-spec, openspec-change and plans. Only a turn resolves them, never prepare. - prompt: every typed prompt up to the turn's end, as one numbered sequence. Display path `prompt`, trigger `typed prompts 1-<n>`. - plan: the plan of the last approved ExitPlanMode, from its input or the plan file written before it. Trigger `approved at turn <n>`. - Spec Kit, Kiro and live OpenSpec specs, and `conformance.plans` files, once the session touches them. Trigger `touched <path>`. A second extraction prompt (REQUIREMENTS_PROMPT_VERSION 1) reads them and keeps a spec's own ids, such as FR-001, T014 or 1.2. The rules cache key names the prompt, and rule files keep their existing keys. The mock provider reads request sources too, so the fixtures run offline. The new `conformance.plans` key takes globs, default []. The closing line counts sources rather than files, and the block reason says "from the files and requests that governed it". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(media): demo video for holding a turn to what the session was asked A 39 s captioned 1:1 demo of proposal 0080. Two prompts, the second a constraint; the agent edits src/legacy.ts anyway and says done. `manni tracevals check` names the broken request under `prompt`, exits 1, and lists every source it held the turn to: prompts, the approved plan and the touched Spec Kit, Kiro, OpenSpec and plans files. Real output from the committed requests fixture, judged by the mock provider as in CI. The MP4 stays out of git. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(tracevals): judge a turn against the session and what it was asked Proposal 0080 is implemented, so the M25 journey and the references say what the per-turn check does now. - The journey page judges the last turn against the session so far, and blocks only on what the last turn did or claimed. It adds what the session was asked (typed prompts, the approved plan, touched Spec Kit, Kiro and OpenSpec specs, and conformance.plans), ticks, and procedures as rules, with a skill's rules in scope to the session's end. A new Doc Detective test runs the requests fixture. - The configuration reference adds conformance.plans and says exclude removes plan and spec files too. - The report reference describes the earlier-turns block read from the transcript, with its new facts and the dropped-lines marker, and lists the six new source formats and their triggers. - The closing lines count sources, and the block reason says "from the files and requests that governed it". - M25's outcome covers requests, specs and procedures. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(proposals): 0080 matches the interface that shipped States that gate 4 now rarely stops a turn, since every turn has a typed prompt. Fixes the earlier-turns example's fact order, and says what the code does about spawns per line, MultiEdit ticks, untitled ticks, the singular marker, the plan file's location, rejection as an error result, sidechain prompts, the requests cache slot, and the skill window in tracevals check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(tracevals): find a spawn's call by ordinal, and never read a source with no file The earlier-turns block looked an agent spawn's call up by array position, which is wrong once the event array is not one event per ordinal. It now finds the event by its ordinal. A source with no file, such as the typed prompts, has path "", which resolved to the cwd and could be counted as read. turnFacts now skips it, for the ledger and the block alike. Tests cover both, a sidechain-only session, and a conformance.plans file under the hook. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(proposals): 0080 says when new sources are extracted, and what picks each closing line Also names the sources table's column for what it holds, and splits the sentences Vale flagged in the tracevals guidance and the demo's script and caption. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(tracevals): keep file-less sources out of the judge's reads and name them in prompts The judge now filters sources with no file before building the earlier-turns block, as turnFacts does. The requests prompt's header names the format when a caller passes no path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: t <t@example.invalid> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * fix(docs): re-derive last-updated after the 0080 squash, and note two intents in tracevals Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * build(deps): move @hawkeyexl/inference to stable 0.5.0 The decide-and-host prerelease line merged as hawkeyexl/inference#14 and published 0.5.0, so manni no longer depends on a feature-branch build. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: t <t@example.invalid> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

hawkeyexl and others added 12 commits October 5, 2026 15:40

chore(release): 0.5.0-decide-and-host.1 [skip ci]

0201bed

# [0.5.0-decide-and-host.1](v0.4.0...v0.5.0-decide-and-host.1) (2026-10-05) ### Features * **decide:** add decisions as an optional provider capability ([4163e99](4163e99))

fix(jev): drop the price table entry

7486e9d

jev carries no price. Cost reporting is not part of what the decision provider is for, so the pinned-model entry, its docs row and its test go. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

test(llama-cpp): give the every-backend-crash case room on a slow runner

fe34b6d

Three backends crash in turn, each in a fresh worker, which takes about 5.5s on a GitHub runner against the 5s default. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

chore: merge llama-cpp decide into feat/decide-and-host

44c5dd8

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

chore(release): 0.5.0-decide-and-host.3 [skip ci]

981fa42

# [0.5.0-decide-and-host.3](v0.5.0-decide-and-host.2...v0.5.0-decide-and-host.3) (2026-10-05) ### Features * **llama-cpp:** answer decisions from option-letter probabilities ([dcd9d59](dcd9d59))

chore(release): 0.5.0-decide-and-host.4 [skip ci]

e2cbc2b

# [0.5.0-decide-and-host.4](v0.5.0-decide-and-host.3...v0.5.0-decide-and-host.4) (2026-10-05) ### Features * **llama-cpp:** keep local models loaded across processes in a model host ([fdb83e8](fdb83e8))

docmeta-release-authenticator Bot added the released on @decide-and-host label Oct 5, 2026

hawkeyexl mentioned this pull request Oct 6, 2026

feat(tracevals): check each turn against the rules it read (0079) hawkeyexl/manni#163

Merged

7 tasks done

hawkeyexl and others added 2 commits October 6, 2026 06:59

chore(release): 0.5.0-decide-and-host.5 [skip ci]

dce5e36

# [0.5.0-decide-and-host.5](v0.5.0-decide-and-host.4...v0.5.0-decide-and-host.5) (2026-10-06) ### Features * **llama-cpp:** answer many items with schema-valid JSON over one shared prefix ([4dc487e](4dc487e))

hawkeyexl and others added 2 commits October 6, 2026 10:38

chore(release): 0.5.0-decide-and-host.6 [skip ci]

18ff41f

# [0.5.0-decide-and-host.6](v0.5.0-decide-and-host.5...v0.5.0-decide-and-host.6) (2026-10-06) ### Bug Fixes * **openai:** keep a required property non-null in a strict schema ([6468f44](6468f44))

hawkeyexl and others added 2 commits October 6, 2026 10:47

chore(release): 0.5.0-decide-and-host.7 [skip ci]

a4acd83

# [0.5.0-decide-and-host.7](v0.5.0-decide-and-host.6...v0.5.0-decide-and-host.7) (2026-10-06) ### Bug Fixes * **openai:** cap every request with openai.maxTokens ([4b15e31](4b15e31))

hawkeyexl and others added 4 commits October 6, 2026 11:18

docs(adrs): record the measured cost of a local call before and after…

e2f7953

… ADR 01019 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

hawkeyexl merged commit a6b5552 into main Oct 7, 2026
5 checks passed

hawkeyexl deleted the feat/decide-and-host branch October 7, 2026 02:05

docmeta-release-authenticator Bot pushed a commit that referenced this pull request Oct 7, 2026

chore(release): 0.5.0 [skip ci]

6c6fffb

# [0.5.0](v0.4.0...v0.5.0) (2026-10-07) ### Features * decisions, jev, local model lifecycle and a model host ([#14](#14)) ([a6b5552](a6b5552))

docmeta-release-authenticator Bot added the released label Oct 7, 2026

Navigation Menu

Repository navigation

feat: decisions, jev, local model lifecycle and a model host - #14

feat: decisions, jev, local model lifecycle and a model host#14
hawkeyexl merged 22 commits into
mainfrom
feat/decide-and-host

hawkeyexl commented Oct 5, 2026

Uh oh!

docmeta-release-authenticator Bot commented Oct 5, 2026

Uh oh!

docmeta-release-authenticator Bot commented Oct 6, 2026

Uh oh!

docmeta-release-authenticator Bot commented Oct 6, 2026

Uh oh!

docmeta-release-authenticator Bot commented Oct 6, 2026

Uh oh!

Uh oh!

docmeta-release-authenticator Bot commented Oct 7, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

hawkeyexl commented Oct 5, 2026

What changed

Docs impact

Verification

Uh oh!

docmeta-release-authenticator Bot commented Oct 5, 2026

Uh oh!

docmeta-release-authenticator Bot commented Oct 6, 2026

Uh oh!

docmeta-release-authenticator Bot commented Oct 6, 2026

Uh oh!

docmeta-release-authenticator Bot commented Oct 6, 2026

Uh oh!

Uh oh!

docmeta-release-authenticator Bot commented Oct 7, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants