A recursive agent architecture inspired by the Viable System Model — combining tool execution, coordination, resource control, independent review, and bounded online adaptation.
Architecture · Results · Quick start · Reproduce · 中文诊断
An agent should be able to change its behavior without confusing a suggestion with a proven improvement. VSM Evolving Agent makes that distinction explicit: actions operate inside contracts, memory retains its sources, and new strategies become available only after an episode boundary and a bounded eligibility review.
The current implementation uses one OpenAI-compatible model service and local SQLite memory. No embedding service or GPU is required for the tool benchmark.
| System | Responsibility | Implementation |
|---|---|---|
| S1 · Operations | Execute the current task; instantiate bounded child organizations | runtime/organization.py |
| S2 · Coordination | Serialize shared actions, track versions, deduplicate requests | systems/s2.py, tools/gateway.py |
| S3 · Control | Reserve and settle hierarchical token, call, tool, and time budgets | systems/s3.py |
| S3* · Review | Audit evidence, proposed actions, completion, and eligibility | systems/audit.py |
| S4 · Adaptation | Propose scoped changes grounded in observed episodes | systems/s4.py |
| S5 · Policy | Enforce identity, scope, allowed tools, and permitted changes | systems/s5.py |
Two timescales. Within an episode, S1 acts through coordination, resource accounting, and action checks. Between episodes, S4 proposes changes, S3*/S5 assess use eligibility, and the next episode receives a frozen strategy snapshot. Observed trace counterexamples can suspend an applicable strategy.
Three evidence boundaries. A strategy being retrieved does not prove it was followed. Following it does not prove task success. Task success does not prove the strategy caused an improvement. active in online-v2 means eligible for bounded use, not validated benefit. Advice without an executable trace retains unknown local outcomes.
See implementation details and the online-v2 protocol.
Local runs on the pinned public CrossEp-Tool release: 200 tasks, 50 per category, DeepSeek-V3.2, temperature 0, empty initial memory, matching within-category order, official offline scoring.
| Method | File system | Vehicle | Trading | Travel | Overall | Total tokens |
|---|---|---|---|---|---|---|
| No memory | 72% | 32% | 52% | 26% | 45.5% · 91/200 | 10.88M |
| BM25 | 76% | 80% | 56% | 50% | 65.5% · 131/200 | 20.63M |
| ACE-CrossEp | 66% | 68% | 74% | 44% | 63.0% · 126/200 | 21.96M |
| VSM online-v2 | 78% | 86% | 82% | 52% | 74.5% · 149/200 | 31.48M |
Against ACE-CrossEp, VSM wins 37 tasks exclusively and loses 14; both pass 112 and both fail 37. The net difference is +11.5 percentage points, with 43.4% more tokens including review and memory updates.
Read the result in scope. These are complete-system comparisons, not equal-compute memory ablations. ACE uses EvoMemBench's pinned ACE adapter, not a claim of full reproduction of the authors' system. The public tasks had development exposure, and these are single runs. The result does not establish causal gains from strategy evolution, statistical robustness, or architectural novelty.
The evolution telemetry is deliberately reported alongside the score: 47 candidates → 3 active strategies → 35 later tasks exposed. All three active strategies are generic failure hints without executable trace specifications; all 605 online observations remained unknown. Runtime behavior improved on several examples, but strategy-level benefit remains unproven.
Evidence: per-task scores · machine-readable summary · provenance and hashes · detailed diagnosis.
The initial ACE 0% was a decoder configuration error, corrected by offline rescoring of unchanged model outputs. Recovery note.
Requires Python 3.11+ and Git. The demo uses a local synthetic environment and makes no model calls.
git clone https://github.com/mojitote/vsm-evolving-agent.git
cd vsm-evolving-agent
python3 -m venv .venv
source .venv/bin/activate
pip install -e '.[test]'
cp .env.example .env
python -m vsm_agent check-config
python -m vsm_agent demo --output runs/demo-01
pytest -qFor live calls, edit .env locally. The key field is intentionally blank in the template; .env is excluded from Git.
# Fill in your provider API key and OpenAI-compatible endpoint below.
VSM_LLM_API_KEY=
VSM_LLM_BASE_URL=
VSM_LLM_MODEL=deepseek-v3.2
VSM_ALLOW_PAID=truecheck-config, the synthetic demo, and offline scoring do not call the model. Live execution requires both credentials and explicit paid-call enablement. Configure token, request, tool, and time limits before a full evaluation. See configuration and reproduction.
pip install -e '.[benchmark]'
python scripts/fetch_benchmark.py
# Real model calls: 2 tasks per category, 8 total; start with a fresh directory.
python scripts/run_online.py --live --mode full \
--limit-per-category 2 --phase development --output runs/dev-online-v2
# Scoring is isolated from generation.
python -m vsm_agent score runs/dev-online-v2/result.jsonl
python -m vsm_agent report runs/dev-online-v2The default .env.example limits are conservative and are not the limits used in the 200-task experiment. See the frozen protocol before launching a full run. Paid benchmarks are never part of CI.
vsm_agent/ Runtime, systems S1–S5, memory, evolution, benchmark adapters
scripts/ CLI helpers, benchmark fetcher, validation, figure generation
experiments/ Versioned experiment entry points and protocols
results/ Curated public scores, task order, summaries, provenance
reports/ Historical research notes and current behavior diagnosis
docs/ Architecture, protocols, setup, and editable SVG figures
tests/ Offline mechanics and regression checks
tools/ Local progress dashboard
Large raw runs, provider credentials, local databases, and fetched third-party source remain outside Git. Historical reports retain their original scope; the result bundle above is the current entry point. Published per-task scores allow aggregate checks; independently rescoring model behavior requires the raw trajectories, which are not bundled here.
- Implemented: recursive organizations; gateway constraints; budget accounting; evidence-linked memory; online-v2 eligibility and next-episode commit; trace-based local observation and suspension.
- Measured: 200-task system comparisons and paired outcomes above.
- Open: reviewer false rejections; premature or omitted actions; useful strategy generation; semantic verification of trace-free advice; same-executor causal ablations; evaluation on unexposed tasks.
The embodied adapter and independent validation utilities are experimental; their presence does not imply a completed embodied benchmark. See acceptance history.
Evaluation uses EvoMemBench, pinned by benchmark.lock.json, and its CrossEp-Tool / BFCL tooling and native memory adapters. Third-party source is fetched separately and retains its own license. See third-party notes.