perf: token-efficiency Phase 0 + Phase 1 (deterministic orchestrator default-ON)#3
Merged
Conversation
… state Move the volatile engagement context out of the single joined system block into a separate trailing volatileSystem block, so the large static prefix (base prompt + tool schemas + skills) stays a cacheable prefix across turns instead of being rewritten every turn. Minify the compact-state JSON and gate coordinator-only blocks (task graph, live subagents, command hints) off subagent turns. Measured on a real Juice Shop run: the ~26k-token static prefix is now served as a cache read every turn (~1.25M cache-read tokens, ~39% lower effective input).
Instruct the coordinator to write task prompts as compact objectives instead of full bash playbooks, and stop re-pasting findings/creds a subagent can read from shared state (new per-subagent context protocol). Trim the repeated background-task boilerplate to one line, and require status reports to be rendered from engagement state instead of hand-written prose tables. Cuts task-prompt size ~70% and transcript ~50% on a real Juice Shop run.
Adds a first-class, cross-agent record of attack vectors probed to a conclusion (attempted/confirmed/resolved/blocked). Both the coordinator and every subagent see a <resolved-vectors> block each turn and record into it when a vector is settled, so a dead end (e.g. a validated open-redirect) is never re-investigated and parallel agents do not duplicate each other's work. Fixes the observed loop where one vector was re-tested 7+ times across an engagement. - schema: ResolvedVector/VectorStatus, State.resolved_vectors (cap 300), toResolvedVectorsContext() render helper - store: addResolvedVector (dedup by target+vector, bumps attempts), getResolvedVectors - tools: state_update record_vector, state_query resolved_vectors - prompt: inject <resolved-vectors> for both roles; MANDATORY sections in pentest.txt + 7 subagent prompts - tests: schema round-trip, ordering, dedup/attempt-count rendering
Digests large raw bash outputs (nmap/nuclei/gobuster dumps) on the configured small model before they enter the expensive model's transcript, cutting both the summarization cost and the per-turn replay of 10-50 KB blobs. Reasoning is untouched: the coordinator and every subagent still run on the main model; the small model only compresses a tool dump. The raw output is kept retrievable by ref (truncation store), so the strategist can always pull ground truth if a digest is thin. Deliberately narrow and opt-in: - only the `bash` tool, only outputs >= 6 KB - runs only when a small model resolves (config small_model, or the provider's own small family); otherwise a graceful no-op, output stays raw. Auto-selection never crosses providers. - new session/small-model.ts (makeToolOutputSummarizer, thresholds, allow-list, digest-must-be-smaller guard, <think> stripping) - hook at the session/tools.ts result boundary; wired in prompt.ts - docs: customize-pentestcode.md explains small_model + the z.ai/GLM case - tests: allow-list gate, digest happy-path, size guard, empty/think
… flag-gated) Adds an opt-in deterministic orchestrator behind OPENCODE_EXPERIMENTAL_ORCHESTRATOR (default OFF — the manual task_graph/task path is untouched and cannot regress). The coordinator declares a task DAG once via `task_graph plan`; the harness then runs it as a continuous, concurrency-capped pipeline instead of the LLM hand- driving plan -> task -> dispatch per node (which was ~53% of coordinator calls). Measured on a Juice Shop A/B (same model, same prompt): coordinator tool calls 98 -> 16, task_graph 35 -> 3, manual task spawns 20 -> 0, ~50% fewer input tokens per unit coverage, and fewer provider rate-limit events. Core: - engagement/orchestrator.ts: pure, LLM-free scheduler. selectWave (cap-aware via REAL in-flight count, priority order, resolved-vectors ledger skip), waveStatus (quiescent signal), hasOutstandingWork. No isComplete/stop — the harness can never end an engagement; completion stays the coordinator's call. - engagement/store.ts: atomic modifyTaskGraph (Ref.modify) — parallel subagents settle concurrently and a get()+set() pair loses completions. - tool/task.ts: makeOrchestratedSpawner returns pump(ctx). pump fills free slots with dep-satisfied tasks, semaphore-serialized and staggered to pace provider load; each subagent on settle refills via pump (rolling — no straggler wait). DAG drained -> nudge coordinator to plan/conclude. TaskTool itself untouched. - tool/task-graph.ts: plan dispatches via pump under the flag. - effect/runtime-flags.ts: OPENCODE_ORCHESTRATOR_CONCURRENCY (cap, default 3), OPENCODE_ORCHESTRATOR_STAGGER_MS (default 800). - session/prompt.ts: flag-gated <orchestrator-dag> directive + wake-time advance nudge keyed off real subagent liveness. TUI: - routes/session/subagent-bar.tsx: derive the running-subagent bar from live child sessions (works for both the manual and orchestrated spawn paths). - /small-model slash command (component/dialog-small-model.tsx + app.tsx): picker like /models that writes small_model to global config to enable cheap-model offload; "None" disables it. Tests: orchestrator scheduler 17/17 (cap, ledger skip, wave cascade incl. the lost-completion race signature, rolling refill). typecheck green.
The summarizer swallowed every error via a bare catch, so a small_model that
was configured but not resolving/serving looked identical to "offload off" —
undiagnosable. Now:
- summarizer returns {digest} | {error} | undefined; on a failed small-model
call it captures + logs the real error (Effect.tapError -> logWarning) and
returns {error} instead of "".
- the tool boundary surfaces {error} inline: the raw output is kept with a
"[AR2 cheap-model offload UNAVAILABLE: <reason>]" note (metadata.ar2_error).
- prompt.ts logs once per session at startup whether offload is active
(model id) or DISABLED because small_model did not resolve.
Diagnostic for the observed "AR2 never fires": pinpoints resolve-failure vs
call-failure vs not-worthwhile. Tests updated (7/7).
Config.get() serves the per-directory merged config from an InstanceState ScopedCache populated once at boot. Config.invalidate() only dropped the cachedGlobal TTL cache, not that InstanceState cache — so a config change made from the UI (notably /small-model, via updateGlobal -> invalidate) was written to disk but never seen by the running session until a full restart. This is why AR2 cheap-model offload silently never fired: small_model was in the file and the model resolved in a fresh process (models CLI), but the long-running session's getSmallModel read a stale config with small_model undefined, so no summarizer was built. invalidate() now also invalidates the InstanceState cache, so the next get() rebuilds from the fresh file. Fixes all UI config updates, not just small_model.
Large scan dumps (30KB+) made the small-model offload call time out at 45s (and compete for provider concurrency), so it always fell back to raw. Now send only a head+tail sample (<=16KB) to the digester — the full raw output is still kept in the ref, so this is lossless where it matters — and raise the timeout to 90s.
The redaction test embedded a literal Google-API-key-shaped string, which GitHub
secret scanning flags as a leaked secret even though it's synthetic test data
(it exists only to verify the detector reports "Google API key"). Build it at
runtime from "AIza" + filler so no literal secret lives in source; the value
still matches the detector's /AIza[0-9A-Za-z_-]{20,}/ pattern.
A fan-out (coordinator + subagents + AR2 small-model calls + retries) hit z.ai with unbounded concurrent streams, producing 42-140 overload events per run and starving the AR2 offload calls (they timed out waiting in the provider queue). All LLM streams funnel through LLM.stream, so gate there with a per-provider concurrency limiter, holding one permit for each stream's lifetime (forked holder tied to the stream scope; released on scope close). TWO independent pools per provider — a big-model stream's NESTED small-model call (made mid-stream while the big stream still holds its permit) draws from the small pool, so a single shared pool cannot deadlock under nesting. Defaults: big (coordinator/subagents) = OPENCODE_LLM_MAX_CONCURRENCY ?? 4, small (offload/title) = floor(big/2). Applies regardless of the orchestrator flag, so the manual multi-subagent path benefits too.
The coordinator could spawn subagents but not stop them: task_graph abandon only de-tracked the node while the background subagent kept running and burning tokens (observed: an orphaned redundant agent the coordinator believed it couldn't kill). Each orchestrated job is tagged with metadata.taskNodeId, so abandon now finds its live background job and cancels both the session prompt and the background fiber (the same combo TaskTool uses on abort) — independent per job, so siblings are untouched. Prompt + tool description updated so the coordinator uses abandon to cut redundant/racing work instead of letting it run.
A live run showed the risk of conflating them: the coordinator "abandoned" a subagent it judged redundant (racing another) — but that agent actually SOLVED its task. Hard-killing on abandon (the previous behavior) would have destroyed useful work on a wrong judgment. Now two distinct verbs: - abandon: de-track only; if the subagent is still running it FINISHES and its result is still reported (use it or ignore it). For "looks redundant". - kill: de-track AND hard-stop the background job to reclaim tokens now. For "sure it's stuck/looping/wrong" — killing loses whatever it might produce. Prompt + tool description steer the coordinator to prefer abandon when unsure.
Mid-size raw bash outputs now stay verbatim; only genuinely large dumps get digested by the small model. Reduces the chance that a subtle-but-crucial detail on a hard challenge gets compressed away before the subagent reasons on it.
Validated on a real multi-host pivot run (PostExploitBench range-3): healthy rolling dispatch, no stalls, overload down ~87% vs the pre-limiter baseline, and authentic lateral movement (React RCE -> tunnel -> rooted an internal-only host). Coordinator orchestration calls per subagent dropped ~85% vs the flag-off baseline. Flip it on by default; keep a kill-switch (OPENCODE_DISABLE_ORCHESTRATOR=true) to fall back to the manual task_graph/task path if a regression surfaces.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Token-efficiency work across two phases, ending with the deterministic orchestrator enabled by default. Validated on a real multi-host PostExploitBench pivot range: healthy subagent dispatch, provider overload events −87% (≈140 → ≈24), and authentic lateral movement preserved. No regression in reasoning depth or multi-agent behavior — that was the hard constraint throughout.
14 commits, +1575 / −65 across 34 files.
Phase 0 — prompt & context economy
Phase 1 — orchestrator + cheap-model offload + dead-end ledger
AR1 — deterministic rolling-pipeline orchestrator
AR2 — cheap-model offload of large bash output
AR3 — resolved-vectors ledger
Supporting fixes
Default flip
Testing
Notes / follow-ups (out of scope for this PR)