diff --git a/.gitignore b/.gitignore
index 852c086..affc2ec 100644
--- a/.gitignore
+++ b/.gitignore
@@ -26,16 +26,7 @@ CybORG/Emulator/Logs/
CybORG/Emulator/logs/
CybORG/Emulator/Config/emulatorconfig.ini
docs/_build/
+site/
dist
-CybORG/Shared/Config/config.ini
-
-CybORG/Emulator/AWS/Config/awsconfig.ini
-
-*.swp
-*.pstats
-
-.ipynb_checkpoints/
-docs/
-
.ruff_cache/
\ No newline at end of file
diff --git a/DATASHEET.md b/DATASHEET.md
new file mode 100644
index 0000000..681a210
--- /dev/null
+++ b/DATASHEET.md
@@ -0,0 +1,82 @@
+# Datasheet & Responsible Use — NetForge RL
+
+Following *Datasheets for Datasets* (Gebru et al.) adapted for a simulated
+benchmark environment. This accompanies a NeurIPS Datasets & Benchmarks submission.
+
+## Motivation
+
+- **Purpose.** NetForge is a research gym and benchmark for autonomous cyber
+ defense: train, evaluate, break, and compare Red/Blue agents. The simulator
+ is the engine; NetForge Arena is the standardized evaluation contract
+ (scenarios, seeds, hidden tests, capability probes, agent cards).
+- **Gap addressed.** Existing cyber-RL environments are largely single-agent or
+ fixed-topology, and they rarely freeze a benchmark protocol. NetForge provides
+ procedural topologies, belief-state graph observations, a train/dev/hidden
+ split, scenario packs, risk-sensitive metrics, and a verified JAX throughput
+ backend.
+
+## Composition
+
+- **Instances.** Episodes are procedurally generated network topologies (100 hosts,
+ 3–5 subnets, optional OT subnet) with per-host OS/CVE/service/credential profiles.
+- **Generation.** `NetworkGenerator(seed)` is deterministic per seed. Training draws
+ from arbitrary seeds; evaluation (`evaluation_mode=True`) draws from a disjoint
+ held-out pool (seed offset 1000) never seen during training.
+- **No personal data.** All hosts, IPs, credentials, and logs are synthetic. CVE
+ identifiers (e.g. MS17-010, CVE-2019-0708) reference public catalog entries used
+ only as abstract vulnerability labels; no exploit code is included.
+
+## Collection / Construction
+
+- **Dynamics.** Two backends share one action taxonomy. The full-fidelity PettingZoo
+ env (`environment/parallel_env.py`) carries action durations, an event queue, SIEM
+ telemetry, and command-list deltas. The vectorized JAX kernel
+ (`backends/jax/vector_env.py`) implements a reduced transition core and is verified
+ equivalent to its NumPy reference (`backends/reference.py`) by a trajectory-level
+ parity test that gates CI (`tests/parity/test_jax_reference_equivalence.py`). The
+ JAX kernel is a scalable subset, not a bit-equivalent replica of the full env.
+- **Rewards.** Per-scenario decompositions; see `benchmarks/env_spec.py` for the full
+ table. Per-step rewards are small; the OT-safety scenarios (`ot_stuxnet`, and the
+ kinetic branch of `ransomware`) add a large ±10,000 terminal signal when a PLC is
+ physically destroyed, so catastrophic physical outcomes dominate the return.
+ `BaseScenario.normalized_reward` exposes a `tanh`-squashed value in `[-1, 1]` for
+ cross-scenario comparison.
+- **Determinism.** Deterministic under a seed. A per-episode RNG on `GlobalNetworkState`
+ drives stochastic actions (exploit rolls, phishing, PLC overload); the SIEM logger,
+ event templates, topology engine, physics, and green agent are all reseeded on
+ `reset(seed)`. See `tests/parity/test_golden_trajectory.py`.
+
+## Recommended Uses
+
+- Benchmarking cooperative/competitive MARL algorithms under partial observability.
+- Evaluating LLM agents as SOC operators (`benchmarks/llm_eval.py`).
+- Studying generalization via the train/held-out topology split
+ (`benchmarks/run_benchmark.py --gap`).
+
+## Limitations
+
+- **It is a simulation.** Dynamics are abstracted and have **not** been validated
+ against real network telemetry or live red-team engagements. Results do not
+ transfer to operational systems without further study.
+- **Abstracted exploits.** Exploit success is modeled via CVSS-weighted probabilities
+ and vulnerability flags, not real exploitation. CVE labels are not a fidelity claim.
+- **No real C2 integration.** Live-fire orchestration against real frameworks is
+ intentionally out of scope and gated behind an unimplemented isolation policy.
+
+## Responsible Use (dual-use statement)
+
+This environment trains both attacker and defender policies. It is released for
+**defensive research and benchmarking**. The simulator contains no operational
+exploit code, no malware, and no capability to act on real systems. Trained Red
+policies operate only inside the simulator's abstract action space and confer no
+real-world offensive capability. Users must not attempt to connect the environment
+to real or production infrastructure. We encourage use aligned with coordinated
+vulnerability disclosure norms.
+
+## Maintenance
+
+- **Versioning.** Environment changes that alter dynamics are reflected in the
+ parity reference and gated by CI; the golden-trajectory fingerprint test detects
+ unintended reward/termination drift.
+- **Reproducibility.** Pin the package version and seeds; `benchmarks/` scripts emit
+ timestamped JSON results with mean ± CI95.
diff --git a/README.md b/README.md
index 5200f16..e8824c1 100644
--- a/README.md
+++ b/README.md
@@ -1,137 +1,125 @@
-
-
-
-
-
+
+
+# NetForge
+
+**Cybersecurity gym for reinforcement learning**
+
+Red attacks a network. Blue contains it from SIEM, not from an oracle.
+PettingZoo (1 Red + 3 Blue) or Gymnasium (one Blue vs scripted Red).
+
+**Python 3.12+** · **PettingZoo** · **Gymnasium** · **Not on PyPI**
+
+[Docs](https://reforcemind.github.io/NetForge_RL/) ·
+[Start](https://reforcemind.github.io/NetForge_RL/getting-started.html) ·
+[Python](https://reforcemind.github.io/NetForge_RL/python-quickstart.html) ·
+[Cyber](https://reforcemind.github.io/NetForge_RL/cybersec/overview.html) ·
+[Contributing](CONTRIBUTING.md)
+
+
+
+
-
NetForge RL
-
-
Multi-agent cybersecurity environment for RL research. Red team vs Blue team on generated networks.
-
----
-
-## Why NetForge
-
-- **Standardized API** — PettingZoo `ParallelEnv`, Gymnasium spaces, conformance-tested.
- A `gymnasium.Env` single-agent facade (`NetForgeSingleAgentEnv`) drops straight into
- Stable-Baselines3 or CleanRL.
-- **Partial observability that means something** — Blue reads a filtered, optionally
- *delayed* SIEM feed; Red must recon a host before it can exploit it.
-- **Real telemetry** — actions emit real Windows/Sysmon event XML, encoded into
- observations by an NLP pipeline. Point an OpenAI, Anthropic, Google, or vLLM client at
- the raw logs and let it play SOC analyst.
-- **MITRE ATT&CK aligned** — actions carry ATT&CK technique IDs, and CVE identifiers are
- used as abstract vulnerability labels (no real exploit code); every episode reports which
- ATT&CK techniques Red actually exercised.
-- **Reproducible to the bit** — a fixed seed replays observations, SIEM embeddings, infos,
- and rewards identically. Test-guaranteed, not just claimed.
-- **Tunable difficulty** — named `easy` / `medium` / `hard` presets plus a frozen held-out
- evaluation split, so results are comparable across runs.
-- **Diagnostic probes + capability cards** — 6 targeted probes (memory, attention,
- temporal, precision, safety, generalization) score *what* a policy can and can't do,
- summarized as a per-policy radar chart.
-- **Deception as a mechanic** — decoys and honeytokens with a `deception_efficacy` metric
- quantifying how much of Red's effort was wasted on traps.
-- **Optional graph observations** — an experimental wrapper exposes hosts as nodes and
- reachability as edges (fog-of-war aware); the benchmarked default is the fixed-shape array.
-- **Self-play & Elo** — a population tournament rates every red and blue policy on one
- shared Elo ladder.
-- **Fast** — the JAX backend reaches **~2.5×10⁵ env-steps/s (~1.0M agent-steps/s)** at batch
- 4096 on CPU. It runs a reduced transition core with an in-kernel scalar alert (not the full
- SIEM text pipeline), so it is a throughput surrogate, not a replica of the Python engine.
-- **Trained, not just scripted, baselines** — a JIT-fused IPPO trainer whose entire
- rollout runs on-device; a committed run learns mean reward 0.06 → 0.71 on `ransomware`.
-
-## Install
+## 📰 News
-```bash
-pip install 'netforge_rl[jax] @ git+https://github.com/reforcemind/NetForge_RL'
-```
+**🗓️ September 2026**
+
+- 🛡️ **20** — Gym ids: PettingZoo `netforge/ransomware-v4` (1 Red + 3 Blue) and Gymnasium `NetForge/Blue-v4`. Blue trains on SIEM belief graphs, not the true map.
+- 📟 **20** — `netforge` CLI: `run`, `evaluate`, `questions`, HTML replay. Packs for hospital, cloud, APT, IoT, OT.
+- 🏗️ **20** — Arena train / dev / hidden splits. Typed `EnvConfig`. Sphinx docs, same kit as [FlowEdge](https://github.com/reforcemind/FlowEdge).
+
+🏷️ [Changelog](docs/changelog.md) · 📘 [Docs](https://reforcemind.github.io/NetForge_RL/)
+
+***
+
+## The problem
+
+A defender does not get the true compromise map. The SOC sees Sysmon-like logs
+that can be late, dropped, or noisy. Red starts blind and only sees hosts it has
+discovered. Exploits and isolates take ticks, not one step.
+
+A gym that hands Blue an oracle, pays for isolating the whole net, or scores
+mean reward against one scripted Red is training the wrong job.
-Extras: `jax` (vectorized backend), `render` (visualization), `finetune` (LLM PEFT),
-`rllib` (Ray multi-agent training). See the
-[install guide](https://reforcemind.github.io/NetForge_RL/quickstart/) for the full list.
+## What NetForge does
-## Quick start
+Each tick, agents pick `[action_type, host_index]`. Actions run, conflicts
+resolve, SIEM fires, Blue obs update from the buffer.
+
+**Red** — discover, exploit CVEs, escalate, exfil, or hit a PLC.
+**Blue** — isolate, ACL, restore, decoys (`169.254.x.x`). Quiet Red may never
+show up in logs.
+**Kinetic** — `OverloadPLC` can end the episode for Blue.
```python
-from netforge_rl.environment import make_env
+import netforge_rl
+import gymnasium as gym
+from pettingzoo import make
-env = make_env('medium', scenario_type='ransomware', seed=0)
+env = make('parallel', 'netforge/ransomware-v4', max_ticks=80)
obs, infos = env.reset(seed=0)
-while env.agents:
- actions = {a: env.action_space(a).sample() for a in env.agents}
- obs, rewards, term, trunc, infos = env.step(actions)
- if all(term.values()) or all(trunc.values()):
- break
+blue = gym.make('NetForge/Blue-v4', max_ticks=80)
```
-### JAX vectorized (4096 environments)
+
+
+
-```python
-import jax
-from netforge_rl.backends.jax import VectorEnvSpec
-from netforge_rl.bridges.jaxmarl import JaxMARLEnv, random_action_dict
-
-env = JaxMARLEnv(spec=VectorEnvSpec(n_hosts=100, n_red=1, n_blue=3), batch_size=4096)
-key = jax.random.PRNGKey(0)
-obs, state = env.reset(key)
-obs, state, reward, done, info = env.step(key, state, random_action_dict(env, key))
-```
+## Why this, not the usual stack
-### LLM as a Blue agent
+
Cybersecurity gym for reinforcement learning. Red compromises hosts. Blue contains them from SIEM, not from an oracle. Train with PettingZoo or Gymnasium.
+
+```{raw} html
+
+
Red vs Blue
+
SIEM telemetry
+
PettingZoo
+
Gymnasium
+
+
+
+
+ 1 Red · 3 Blue
+ DMZ, internal, restricted
+
+
+ SIEM only
+ Blue never sees the true map
+
+
+ durative actions
+ exploits and isolates take ticks
+
-# NetForge RL
-
-**A fast, reproducible multi-agent reinforcement learning environment for autonomous
-cyber-defense.** Red agents attack a procedurally generated enterprise/OT network
-(exploit, pivot, escalate, ransomware, kinetic sabotage); Blue agents defend it (monitor,
-analyze, isolate, deceive, restore). Built on the PettingZoo parallel API with a JAX
-vectorized backend for high-throughput training.
-
----
-
-## Why NetForge
-
-- **Standardized API** — PettingZoo `ParallelEnv`, Gymnasium spaces, conformance-tested.
-- **Partial observability that means something** — Blue sees only a filtered, optionally
- *delayed* SIEM feed; Red operates under fog of war and must recon before it can exploit.
-- **Realistic telemetry** — actions emit real Windows/Sysmon event XML (4624, 4688, Sysmon
- 1/3/10/22, …) which an NLP encoder turns into observations. Plug an LLM in as a SOC analyst.
-- **MITRE ATT&CK aligned** — exploits map to real CVEs (MS17-010, CVE-2019-0708, Log4Shell).
-- **Reproducible to the bit** — a fixed seed replays observations, SIEM embeddings, infos and
- rewards identically (test-guaranteed). See [Reproducibility](environment/reproducibility.md).
-- **Tunable difficulty** — named `easy`/`medium`/`hard` presets and a frozen held-out
- evaluation split. See [Difficulty & Splits](environment/difficulty.md).
-- **Diagnostic probes + capability cards** — a 6-capability suite (memory, attention, temporal,
- precision, safety, generalization) that isolates *what* a policy can and cannot do, summarised
- as a per-policy radar card. See [Diagnostics](diagnostics/overview.md).
-- **Graph-native observations** — hosts as nodes, reachability as edges, fog-of-war aware,
- one line to PyTorch Geometric / jraph. See [Graph Observations](environment/graph_observations.md).
-- **Self-play & Elo** — a population tournament that rates red and blue on one ladder. See
- [Self-Play](benchmarks/self_play.md).
-- **Deception as a mechanic** — decoys and honeytokens with a `deception_efficacy` metric
- measuring how much of Red's effort was wasted on traps.
-- **SOC export** — replay an episode as OCSF-style JSON for real SIEM tooling. See
- [SOC Export](interop/soc_export.md).
-- **Trained baselines, not just scripted** — a JIT-fused JAX IPPO trainer whose whole rollout
- runs on-device; a committed `ransomware` run learns mean reward 0.06 → 0.71. Checkpoints
- reload into self-play and capability cards. See [Baselines](benchmarks/baselines.md).
-- **Standard single-agent API** — `NetForgeSingleAgentEnv` is a `gymnasium.Env` (passes
- `check_env`) for Stable-Baselines3 / CleanRL, with the action mask in `info`.
-- **MITRE ATT&CK coverage** — episodes report which ATT&CK techniques Red exercised and the
- fraction of the taxonomy covered (`attack_coverage`).
-- **Fast** — JAX backend measured at **270,795 env-steps/s (1,083,181 agent-steps/s)** at
- batch 4096 on CPU, with an optional in-kernel numeric SIEM signal. See
- [Benchmarks](benchmarks/overview.md).
-- **JAX backend matches real action timing** — the vectorized backend enqueues and resolves
- actions against the same per-agent duration table as the Python engine: an agent can't act
- again until its pending action resolves, and isolating a host cancels an in-flight exploit
- against it. Verified against a NumPy reference with zero mismatches across all five
- scenarios under randomized rollouts.
-
-## Quick Start
+
+```
+
+## News
+
+**September 2026**
+
+- Gym ids: PettingZoo `netforge/ransomware-v4`, Gymnasium `NetForge/Blue-v4`.
+- `netforge` CLI: `run`, `evaluate`, `questions`, HTML replay.
+- Arena train / dev / hidden. Typed `EnvConfig`. {doc}`changelog`
+
+## The problem
+
+A defender does not get the true compromise map. The SOC sees Sysmon-like logs
+that can be late, dropped, or noisy. Red starts blind. Exploits and isolates
+take ticks. A gym that hands Blue an oracle, or scores mean reward against one
+scripted Red, is training the wrong job.
+
+## What NetForge does
+
+Red starts blind, discovers hosts, exploits CVEs, escalates, exfiltrates or
+hits a PLC. Blue isolates, restores, drops decoys, and reads Sysmon-like logs
+that can be late or noisy. Padding hosts (`169.254.x.x`) are decoys.
+
+```{image} _static/figures/purpose.svg
+:alt: Red, Blue, and the network
+:class: fe-fig
+```
+
+```{image} _static/figures/workflow.svg
+:alt: reset, observe SIEM, act, next tick
+:class: fe-fig
+```
+
+```{image} _static/figures/why-this.svg
+:alt: God-mode vs SIEM, instant logs vs delay, isolate-all vs SLA
+:class: fe-fig
+```
+
+## Why this, not the usual stack
+
+Blue never sees the true map. Logs can lag. Isolate-everything wrecks SLA.
+Eval uses a Red population. Trainers stay in your repo.
+
+## Install
```bash
-pip install 'netforge_rl[jax] @ git+https://github.com/reforcemind/NetForge_RL'
+python -m pip install 'netforge-rl @ git+https://github.com/reforcemind/NetForge_RL'
+netforge run hospital_ransomware --replay replay.html
```
```python
-from netforge_rl.environment import make_env
+import netforge_rl
+import gymnasium as gym
+from pettingzoo import make
-env = make_env('medium', scenario_type='ransomware', seed=0)
+env = make('parallel', 'netforge/ransomware-v4', max_ticks=80)
obs, infos = env.reset(seed=0)
-while env.agents:
- actions = {a: env.action_space(a).sample() for a in env.agents}
- obs, rewards, term, trunc, infos = env.step(actions)
- if all(term.values()) or all(trunc.values()):
- break
+blue = gym.make('NetForge/Blue-v4', max_ticks=80)
+```
+
+Not on PyPI. Python 3.12+. Trainers stay in your repo.
+
+MIT, with CybORG / DSTG notices in the LICENSE file.
+
+```{toctree}
+:hidden:
+:maxdepth: 1
+:caption: Start
+Overview
+getting-started
+python-quickstart
+arena
+questions
+submissions
+capabilities
+```
+
+```{toctree}
+:hidden:
+:maxdepth: 1
+:caption: Guides
+scenarios
+baselines
+datasets
+backends
+diagnostics/overview
+environment/difficulty
+environment/reproducibility
+environment/graph_observations
+training/curriculum
+training/rllib
+training/scenarios
+guides/verification
+interop/pettingzoo
+interop/soc_export
+```
+
+```{toctree}
+:hidden:
+:maxdepth: 1
+:caption: Benchmarks
+benchmarks/overview
+benchmarks/baselines
+benchmarks/self_play
+benchmarks/run
+```
+
+```{toctree}
+:hidden:
+:maxdepth: 1
+:caption: Architecture
+architecture/overview
+architecture/dynamic_topology
+architecture/ot_physics
+architecture/multimodal_obs
+architecture/zero_trust
+architecture/sim2real
+architecture/nlp_siem
+```
+
+```{toctree}
+:hidden:
+:maxdepth: 1
+:caption: Cyber
+cybersec/overview
+cybersec/threat_model
+cybersec/red_actions
+cybersec/blue_actions
+```
+
+```{toctree}
+:hidden:
+:maxdepth: 1
+:caption: API
+api/python
+api/environment
+api/siem
+api/nlp
+api/sim2real
+```
+
+```{toctree}
+:hidden:
+:maxdepth: 1
+:caption: Contribute
+contributing
+research
```
-See the [Quick Start Guide](quickstart.md) for the raw PettingZoo loop, the JAX vectorized
-loop, difficulty presets, baselines, and diagnostics.
-
-## What's inside
-
-| Layer | Package | Purpose |
-|---|---|---|
-| Environment | `environment/` | PettingZoo `NetForgeRLEnv`, difficulty presets, curriculum |
-| Scenarios | `scenarios/` | 5 reward/objective families (ransomware, APT, cloud, IoT, OT) |
-| Actions | `actions/red`, `actions/blue` | ATT&CK-aligned red/blue capabilities |
-| Telemetry | `siem/`, `nlp/` | Sysmon/Windows log synthesis + NLP encoding |
-| JAX backend | `backends/jax/` | Vectorized `vmap`/`jit` kernels + NumPy reference |
-| Baselines | `baselines/` | Random, heuristic, kill-chain red, JAX PPO |
-| Diagnostics | `diagnostics/` | Capability probes + oracle information-asymmetry |
-| Bridges | `bridges/` | RLlib, JaxMARL, CleanRL, DLPack adapters |
-| Semantic | `semantic/` | LLM SOC agents, grammars, fine-tuning recipes |
-
-## The five scenarios
-
-| Scenario | Red objective | Blue objective | Terminal condition |
-|---|---|---|---|
-| `ransomware` | Encrypt/compromise Corporate + Secure | Contain, restore, avoid downtime | all Corporate/Secure compromised, or PLC kinetic |
-| `apt_espionage` | Stealthy persistence + exfiltration | Detect and isolate every foothold | every infected host isolated |
-| `cloud_hybrid` | Breach the Secure enclave | Protect the Secure subnet SLA | every Secure host compromised |
-| `iot_grid` | Take the grid controllers | Keep controllers healthy | all controllers compromised |
-| `ot_stuxnet` | Drive a PLC to kinetic destruction | Prevent physical damage | any PLC kinetic destruction |
-
-## Citation
-
-```bibtex
-@misc{netforge_rl_2026,
- title = {NetForge RL: A Fast Multi-Agent Cybersecurity Benchmark for Reinforcement Learning},
- author = {ReforceMind},
- year = {2026},
- url = {https://github.com/reforcemind/NetForge_RL}
-}
+```{toctree}
+:hidden:
+:maxdepth: 1
+:caption: Project
+roadmap
+product-direction
+DATASHEET
+changelog
```
diff --git a/docs/interop/pettingzoo.md b/docs/interop/pettingzoo.md
new file mode 100644
index 0000000..fb80d58
--- /dev/null
+++ b/docs/interop/pettingzoo.md
@@ -0,0 +1,23 @@
+# PettingZoo and Gymnasium
+
+```python
+import netforge_rl
+import gymnasium as gym
+from pettingzoo import make
+
+env = make('parallel', 'netforge/ransomware-v4', max_ticks=200)
+single = gym.make('NetForge/Blue-v4', max_ticks=200)
+```
+
+| API | Id |
+|---|---|
+| PettingZoo parallel | `netforge/ransomware-v4`, `netforge/arena-v4` |
+| PettingZoo AEC | same ids |
+| Gymnasium | `NetForge/Blue-v4`, alias `NetForge-v4` |
+
+`gym.make("netforge_rl:NetForge/Blue-v4")` also works. Mask is flat `int8[132]`;
+`CyberActionSpace.sample(mask=…)` accepts it.
+
+Farama listing (docs PR to PettingZoo `third_party_envs`): Parallel + AEC
+`netforge/ransomware-v4`, Gymnasium `NetForge/Blue-v4`,
+https://reforcemind.github.io/NetForge_RL/
diff --git a/docs/interop/soc_export.md b/docs/interop/soc_export.md
new file mode 100644
index 0000000..72329cf
--- /dev/null
+++ b/docs/interop/soc_export.md
@@ -0,0 +1,24 @@
+# SOC export (OCSF)
+
+`record_siem=True` keeps the full stream. `export_ocsf` writes JSONL.
+
+```python
+from netforge_rl.environment.parallel_env import NetForgeRLEnv
+from netforge_rl.siem.export import export_ocsf, siem_to_ocsf
+
+env = NetForgeRLEnv(
+ {'scenario_type': 'ransomware', 'max_ticks': 100, 'record_siem': True}
+)
+env.reset(seed=0)
+export_ocsf(env, 'runs/episode.ocsf.jsonl')
+siem_to_ocsf(log_line, subnet='10.0.0.0/24', tick=42)
+```
+
+| EventID | OCSF |
+|---|---|
+| 4624 / 4625 / 4648 / 4768 / 4776 | Authentication (3002) |
+| 4688 / Sysmon 1 / 10 | Process (1007) |
+| Sysmon 3 | Network (4001) |
+| Sysmon 22 | DNS (4003) |
+
+Raw XML stays in `raw_event`. Honeytoken / incident → higher `severity_id`.
diff --git a/docs/product-direction.md b/docs/product-direction.md
new file mode 100644
index 0000000..3c1ab5a
--- /dev/null
+++ b/docs/product-direction.md
@@ -0,0 +1,19 @@
+# Scope
+
+A gym for training cyber-defense agents. Grow packs, metrics, and wrappers —
+not host-type count and not a bundled MAPPO.
+
+```{image} _static/figures/status.svg
+:alt: What ships in the gym
+:class: fe-fig
+```
+
+| In this repo | Bring your own |
+|---|---|
+| Network, SIEM, Red/Blue actions | MAPPO / QMIX / GNN trainers |
+| Belief graphs from logs | Privileged Blue graphs at train time |
+| YAML packs | Weekly ATT&CK dumps |
+| Heuristic / kill-chain baselines | Production SOC connectors |
+
+Sister docs kit: [FlowEdge](https://github.com/reforcemind/FlowEdge).
+{doc}`questions` · {doc}`arena`
diff --git a/docs/python-quickstart.md b/docs/python-quickstart.md
new file mode 100644
index 0000000..8164f36
--- /dev/null
+++ b/docs/python-quickstart.md
@@ -0,0 +1,45 @@
+# Python
+
+`import netforge_rl` registers the ids. CPython 3.12+. Not on PyPI.
+
+```bash
+python -m pip install 'netforge-rl @ git+https://github.com/reforcemind/NetForge_RL'
+# from a clone:
+python -m pip install -e ".[dev]"
+```
+
+```python
+import netforge_rl
+import gymnasium as gym
+from pettingzoo import make
+
+env = make('parallel', 'netforge/ransomware-v4', max_ticks=200)
+obs, infos = env.reset(seed=0)
+actions = {agent: env.action_space(agent).sample() for agent in env.agents}
+obs, rewards, term, trunc, infos = env.step(actions)
+
+blue = gym.make('NetForge/Blue-v4', max_ticks=200)
+```
+
+| API | Id | |
+|---|---|---|
+| PettingZoo parallel | `netforge/ransomware-v4` | 1 Red + 3 Blue |
+| PettingZoo parallel | `netforge/arena-v4` | same factory |
+| PettingZoo AEC | same ids | turn-based |
+| Gymnasium | `NetForge/Blue-v4` | one Blue vs scripted Red |
+| Gymnasium | `NetForge-v4` | alias |
+
+Action: `MultiDiscrete([32, 100])`, mask `int8[132]`.
+Blue graph: `GraphObservationWrapper`.
+
+| Extra | |
+|---|---|
+| *(none)* | env, CLI, heuristics |
+| `dev` | ruff, pytest |
+| `docs` | Sphinx |
+| `jax` | vectorized surrogate |
+| `rllib` | RLlib env wrapper |
+| `offline` | HDF5 export |
+| `nlp` / `llm_clients` | SIEM encoder / LLM agents |
+
+{doc}`interop/pettingzoo`
diff --git a/docs/questions.md b/docs/questions.md
new file mode 100644
index 0000000..5bad5c1
--- /dev/null
+++ b/docs/questions.md
@@ -0,0 +1,30 @@
+# Probes
+
+Small experiments the gym already runs. Heuristics are in-tree so they execute.
+
+```{image} _static/figures/belief.svg
+:alt: SIEM belief vs full state
+:class: fe-fig
+```
+
+```bash
+netforge questions
+netforge questions belief-vs-oracle --seeds 0 --max-ticks 40
+```
+
+| id | What it checks |
+|---|---|
+| `belief-vs-oracle` | How much compromise is missing from Blue’s SIEM graph |
+| `reward-design` | SLA-only / FP-penalized reward vs isolation |
+| `red-population` | One Blue vs several Reds |
+| `risk-sensitive` | Mean vs CVaR vs kinetic tails |
+| `time-mode` | Event-driven vs fixed-step |
+| `ood-topology` | More live hosts than training |
+| `adaptation` | Red changes strategy mid-episode |
+| `deception` | Honeypots / decoys |
+| `constrained-sla` | Defense under SLA / FP costs |
+| `gnn-belief` | Train a GNN on SIEM belief graphs (your trainer) |
+| `offline-to-online` | Pretrain on dumps, then online |
+| `league-psro` | Best response to a Red league |
+
+Add one in `netforge_rl/arena/questions/`. {doc}`submissions`
diff --git a/docs/quickstart.md b/docs/quickstart.md
deleted file mode 100644
index 5906642..0000000
--- a/docs/quickstart.md
+++ /dev/null
@@ -1,147 +0,0 @@
-# Quick Start Guide
-
-## Installation
-
-Requires Python 3.12+.
-
-```bash
-pip install 'netforge_rl[jax,render,finetune] @ git+https://github.com/reforcemind/NetForge_RL'
-```
-
-Extras: `jax` (vectorized backend), `render` (matplotlib + moviepy), `finetune`
-(TRL/PEFT for LLM agents), `rllib` (Ray RLlib bridge).
-
-## PettingZoo execution (single environment)
-
-The reference Python backend implements the PettingZoo parallel API. Each agent acts with a
-`MultiDiscrete([32, 100])` action — `[action_type_id, target_host_index]`.
-
-```python
-import numpy as np
-from netforge_rl.environment.parallel_env import NetForgeRLEnv
-
-env = NetForgeRLEnv({'scenario_type': 'ransomware', 'max_ticks': 50})
-obs, infos = env.reset(seed=0)
-
-while env.agents:
- actions = {a: env.action_space(a).sample() for a in env.agents}
- obs, rewards, term, trunc, infos = env.step(actions)
- if all(term.values()) or all(trunc.values()):
- break
-```
-
-Each `obs[agent]` is a dict with `obs` (256-d state), `action_mask` (132-d), `siem_embedding`
-(128-d), `adj_matrix`, and `delta_t`; Blue agents additionally get `blue_comm`.
-
-## Difficulty presets and the held-out split
-
-Instead of hand-writing config dicts, use named difficulty tiers and the frozen evaluation
-seed suite. See [Difficulty & Splits](environment/difficulty.md) for the full knob table.
-
-```python
-from netforge_rl.environment import make_env, EVAL_SEEDS
-
-# Train on any seeds at a chosen difficulty
-train_env = make_env('medium', scenario_type='apt_espionage', seed=0)
-
-# Report on held-out topologies never seen during training
-eval_env = make_env('hard', scenario_type='apt_espionage', evaluation=True, seed=EVAL_SEEDS[0])
-```
-
-## JAX vectorized execution
-
-Hardware-accelerated batched rollouts for high-throughput training.
-
-```python
-import jax
-from netforge_rl.backends.jax import VectorEnvSpec
-from netforge_rl.bridges.jaxmarl import JaxMARLEnv, random_action_dict
-
-env = JaxMARLEnv(spec=VectorEnvSpec(n_hosts=100, n_red=1, n_blue=3), batch_size=4096)
-key = jax.random.PRNGKey(0)
-
-obs, state = env.reset(key)
-obs, state, reward, done, info = env.step(key, state, random_action_dict(env, key))
-```
-
-## Running baselines
-
-```python
-from netforge_rl.environment.parallel_env import NetForgeRLEnv
-from netforge_rl.baselines.policies import KillChainRedPolicy, HeuristicBluePolicy
-
-env = NetForgeRLEnv({'scenario_type': 'ransomware', 'max_ticks': 150})
-env.reset(seed=0)
-red, blue = KillChainRedPolicy(seed=0), HeuristicBluePolicy(seed=0)
-
-while env.agents:
- actions = {a: (red.act(env, a) if 'red' in a else blue.act(env, a)) for a in env.agents}
- _, _, term, trunc, _ = env.step(actions)
- if all(term.values()) or all(trunc.values()):
- break
-```
-
-`KillChainRedPolicy` runs a recon → exploit → pivot kill-chain and actually compromises
-hosts; see [Baselines](benchmarks/baselines.md).
-
-## Single-agent training (Gymnasium / SB3 / CleanRL)
-
-For standard single-agent RL, `NetForgeSingleAgentEnv` is a `gymnasium.Env` that controls one
-agent against scripted opponents. It passes `gymnasium`'s `check_env` and exposes the action
-mask in `info` for maskable algorithms.
-
-```python
-from netforge_rl.environment import NetForgeSingleAgentEnv
-from netforge_rl.baselines.policies import KillChainRedPolicy
-
-env = NetForgeSingleAgentEnv(
- 'ransomware',
- controlled_agent='blue_dmz',
- opponents={'red_operator': KillChainRedPolicy(seed=0)},
-)
-obs, info = env.reset(seed=0) # obs: Box(384,), info['action_mask']: (132,)
-obs, reward, terminated, truncated, info = env.step(env.action_space.sample())
-```
-
-## Training IPPO on the JAX backend
-
-```python
-from netforge_rl.baselines.jax_ppo import ippo_train, PPOConfig, save_params
-
-out = ippo_train(PPOConfig(total_iters=40, num_steps=48, batch_size=128))
-save_params(out['params'], 'runs/ippo_blue.npz')
-print('reward:', out['reward_curve'][0], '->', out['reward_curve'][-1])
-```
-
-See [Baselines](benchmarks/baselines.md) for the committed learning curve.
-
-## Running the diagnostic probes
-
-```python
-from netforge_rl.diagnostics import all_diagnostics, run_diagnostic
-from netforge_rl.baselines.policies import HeuristicBluePolicy
-
-for probe in all_diagnostics():
- result = run_diagnostic(probe, HeuristicBluePolicy(seed=0), seed=0)
- print(f'{probe.capability:14} {probe.name:26} score={result.score:.2f}')
-```
-
-See [Diagnostics](diagnostics/overview.md) for what each capability measures.
-
-## LLM SOC agent (optional)
-
-`run_episode` takes a `clients` dict mapping each agent id to an LLM client; uncontrolled
-agents no-op, and unparseable replies are counted as `invalid_replies`.
-
-```python
-from netforge_rl.environment.parallel_env import NetForgeRLEnv
-from netforge_rl.semantic.clients.mock import MockLLMClient
-from netforge_rl.semantic.runner import run_episode
-
-env = NetForgeRLEnv({'scenario_type': 'ransomware', 'max_ticks': 50})
-result = run_episode(env, clients={'blue_dmz': MockLLMClient()}, seed=0)
-print(result.rewards, result.invalid_replies)
-```
-
-Swap `MockLLMClient` for an OpenAI/Anthropic/vLLM client to have a language model read raw
-SIEM logs and issue defensive actions.
diff --git a/docs/requirements.txt b/docs/requirements.txt
new file mode 100644
index 0000000..f37ea70
--- /dev/null
+++ b/docs/requirements.txt
@@ -0,0 +1,11 @@
+sphinx>=8.0
+furo>=2024.8.6
+myst-parser>=4.0
+sphinxcontrib-mermaid>=1.0.0
+numpy>=2.0
+gymnasium>=1.2
+pettingzoo>=1.26
+networkx>=3.4
+pyyaml>=6.0
+pillow>=10.0
+scikit-learn>=1.0
diff --git a/docs/research.md b/docs/research.md
new file mode 100644
index 0000000..f275b8f
--- /dev/null
+++ b/docs/research.md
@@ -0,0 +1,16 @@
+# Directions
+
+This tree is the gym: network, logs, actions, eval. Trainers and new exploit
+families stay out unless they are a wrapper, pack, or probe.
+
+- Belief vs oracle (SOC view vs true compromise)
+- Defense under SLA / false-positive cost
+- Reward that does not isolate the whole net
+- Offline logs → online fine-tune
+- Adaptive Red, leagues
+- Event time vs fixed ticks
+- LLM vs RL on the same env
+
+Catalog: `netforge questions`. Issues: [RESEARCH.md](https://github.com/reforcemind/NetForge_RL/blob/main/RESEARCH.md).
+
+{doc}`product-direction` · {doc}`roadmap`
diff --git a/docs/roadmap.md b/docs/roadmap.md
new file mode 100644
index 0000000..fc91e28
--- /dev/null
+++ b/docs/roadmap.md
@@ -0,0 +1,9 @@
+# Roadmap
+
+Backlog: [RESEARCH.md](https://github.com/reforcemind/NetForge_RL/blob/main/RESEARCH.md).
+Labels: `research` / `good first issue`.
+
+Shipped: PettingZoo / Gymnasium ids, SIEM belief graphs, eval CLI.
+
+Next: trainers **on** this env (out of tree), hidden-split harness, Farama listing,
+published checkpoints.
diff --git a/docs/scenarios.md b/docs/scenarios.md
new file mode 100644
index 0000000..47a6d67
--- /dev/null
+++ b/docs/scenarios.md
@@ -0,0 +1,18 @@
+# Scenarios
+
+Pick a family and a topology. No simulator patch.
+
+Families: `ransomware`, `apt_espionage`, `cloud_hybrid`, `iot_grid`, `ot_stuxnet`.
+
+Packs: `hospital_ransomware`, `cloud_iam`, `enterprise_apt`, `finance`,
+`iot_fleet`, `ot_plant`, `zero_trust`.
+
+```bash
+netforge run hospital_ransomware
+netforge run examples/my_scenario.yaml
+```
+
+Reward knobs (`RewardDesignWrapper`): `sla_only`, `security_only`,
+`fp_penalized`, `constrained_sla`, `safety_critical`, `sparse`.
+
+YAML lives in `netforge_rl/scenarios/packs/`.
diff --git a/docs/stylesheets/custom.css b/docs/stylesheets/custom.css
deleted file mode 100644
index eec8751..0000000
--- a/docs/stylesheets/custom.css
+++ /dev/null
@@ -1,107 +0,0 @@
-/* ReforceMind Target Site Emulation */
-
-@import url('https://fonts.googleapis.com/css2?family=JetBrains+Mono:wght@400;600&family=Libre+Franklin:wght@400;500;600;700&family=Newsreader:opsz,wght@6..72,400;6..72,500;6..72,600;6..72,700&display=swap');
-
-:root {
- /* Colors based on the reforcemind.github.io parchment/ink palette */
- --md-primary-fg-color: #1a4231; /* forest-intellect */
- --md-accent-fg-color: #8b2635; /* crimson-authority */
-
- --md-default-bg-color: #F8F5F0; /* parchment-base */
- --md-default-fg-color: #1A1D20; /* ink-black */
-
- --md-code-bg-color: #e9e5de;
- --md-code-fg-color: #1A1D20;
-}
-
-[data-md-color-scheme="slate"] {
- --md-default-bg-color: #0b0f19;
- --md-default-fg-color: #e2e8f0;
- --md-code-bg-color: #0f172a;
- --md-primary-fg-color: #2ab88a;
- --md-accent-fg-color: #ff6b6b;
-}
-
-/* Background overlay mimicking the target site gradients */
-body[data-md-color-scheme="default"] {
- background-image:
- radial-gradient(ellipse at top left, rgba(26, 66, 49, 0.1) 0%, transparent 50%),
- radial-gradient(ellipse at bottom right, rgba(139, 38, 53, 0.08) 0%, transparent 50%);
- background-attachment: fixed;
-}
-
-/* Typography Overrides */
-.md-typeset h1, .md-typeset h2, .md-typeset h3, .md-typeset h4, .md-typeset h5, .md-typeset h6 {
- font-family: 'Newsreader', serif;
- font-weight: 600;
- color: #1a4231;
-}
-
-[data-md-color-scheme="slate"] .md-typeset h1,
-[data-md-color-scheme="slate"] .md-typeset h2 {
- color: #2ab88a;
-}
-
-/* Header Overlay */
-.md-header {
- background: rgba(248, 245, 240, 0.8) !important;
- backdrop-filter: blur(10px);
- -webkit-backdrop-filter: blur(10px);
- border-bottom: 1px solid rgba(26, 66, 49, 0.1);
- color: #1a4231 !important;
-}
-
-[data-md-color-scheme="slate"] .md-header {
- background: rgba(11, 15, 25, 0.7) !important;
- border-bottom: 1px solid rgba(42, 184, 138, 0.2);
- color: #2ab88a !important;
-}
-
-/* Force header links to be visible on the light parchment background */
-.md-header__button.md-logo,
-.md-header__button.md-icon,
-.md-header__title,
-.md-search__input,
-.md-tabs__link {
- color: #1A1D20 !important;
-}
-
-[data-md-color-scheme="slate"] .md-header__button.md-logo,
-[data-md-color-scheme="slate"] .md-header__button.md-icon,
-[data-md-color-scheme="slate"] .md-header__title,
-[data-md-color-scheme="slate"] .md-tabs__link {
- color: #e2e8f0 !important;
-}
-
-/* Code Block Styling */
-.md-typeset pre > code {
- border-radius: 8px;
- border: 1px solid rgba(0, 0, 0, 0.05);
- box-shadow: inset 0 0 10px rgba(0, 0, 0, 0.02);
- font-family: 'JetBrains Mono', monospace;
-}
-
-/* Admonition Overrides */
-.md-typeset .admonition {
- border-radius: 8px;
- background: rgba(0, 0, 0, 0.02);
- border: 1px solid rgba(0, 0, 0, 0.05);
- border-left: 4px solid var(--md-admonition-fg-color);
- box-shadow: 0 4px 15px rgba(0, 0, 0, 0.03);
- transition: transform 0.2s ease, box-shadow 0.2s ease;
-}
-
-.md-typeset .admonition:hover {
- transform: translateY(-2px);
- box-shadow: 0 8px 25px rgba(0, 0, 0, 0.08);
-}
-
-/* Active Navigation Item */
-.md-nav__item--active > .md-nav__link {
- color: #8b2635 !important;
- font-weight: 600;
-}
-
-[data-md-color-scheme="slate"] .md-nav__item--active > .md-nav__link {
- color: #ff6b6b !important;
-}
diff --git a/docs/submissions.md b/docs/submissions.md
new file mode 100644
index 0000000..f1d7308
--- /dev/null
+++ b/docs/submissions.md
@@ -0,0 +1,24 @@
+# Your agent
+
+Drop in a Blue policy with `make_blue`. Eval reports mission, SLA, FPs, security,
+kinetic failures — not only return.
+
+```bash
+netforge evaluate examples/submissions/heuristic_blue.py --seeds 0 1 --max-ticks 40
+```
+
+```python
+NAME, TEAM, TECHNIQUE = 'my-blue', 'lab', 'ppo-belief'
+
+
+def wrap(env):
+ return env # belief graphs already on; do not attach oracle_graph
+
+
+def make_blue(seed=0):
+ return MyPolicy(seed=seed)
+```
+
+`act(env, agent_id) -> np.ndarray`. Optional `make_red`. `--card` adds probes.
+
+{doc}`arena` · `examples/submissions/heuristic_blue.py`
diff --git a/docs/training/curriculum.md b/docs/training/curriculum.md
new file mode 100644
index 0000000..7932681
--- /dev/null
+++ b/docs/training/curriculum.md
@@ -0,0 +1,11 @@
+# Curriculum
+
+`CurriculumWrapper` raises difficulty when a moving-average reward clears a threshold.
+
+| Phase | Hosts | Scenarios | Mechanics |
+|---|---|---|---|
+| Novice | 5 | ransomware | static, 3× reward |
+| Intermediate | 25 | + apt | DHCP every 80 ticks, 1.5× |
+| Expert | 100 | + iot_grid | DHCP 40, dynamic topology, 1× |
+
+{doc}`../architecture/dynamic_topology`
diff --git a/docs/training/rllib.md b/docs/training/rllib.md
index adef1ad..33cd69f 100644
--- a/docs/training/rllib.md
+++ b/docs/training/rllib.md
@@ -1,26 +1,13 @@
-# Training with Ray RLlib
-
-## Environment Wrapper
-
-NetForge implements the RLlib `MultiAgentEnv` interface via `NetForgeRLlibEnv`.
+# Ray RLlib
```python
from netforge_rl.bridges.rllib_bridge import NetForgeRLlibEnv
-env = NetForgeRLlibEnv({"scenario_type": "ransomware", "max_ticks": 100})
-```
-
-## RMAPPO Baseline
-
-The reference training script is [`benchmarks/rllib_rmappo.py`](https://github.com/reforcemind/NetForge_RL/blob/main/benchmarks/rllib_rmappo.py). It trains two shared LSTM policies (one red, one blue) with PPO on the legacy API stack.
-
-```bash
-python benchmarks/rllib_rmappo.py
+env = NetForgeRLlibEnv({'scenario_type': 'ransomware', 'max_ticks': 100})
```
-Key settings: `train_batch_size=1024`, `use_lstm=True`, legacy API stack (`enable_rl_module_and_learner=False`). Custom metrics from `NetForgeMetricsCallback` are logged per-agent each step.
-
-## PPO Configuration
+Reference: `python benchmarks/rllib_rmappo.py` — shared LSTM PPO, red + blue,
+`train_batch_size=1024`, legacy API stack.
```python
import ray
@@ -29,54 +16,26 @@ from ray.rllib.algorithms.ppo import PPOConfig
from netforge_rl.bridges.rllib_bridge import NetForgeRLlibEnv
ray.init()
-register_env("netforge-v0", lambda cfg: NetForgeRLlibEnv(cfg))
-
+register_env('netforge-v0', lambda cfg: NetForgeRLlibEnv(cfg))
config = (
PPOConfig()
- .environment("netforge-v0", env_config={"scenario_type": "ransomware"})
- .api_stack(enable_rl_module_and_learner=False, enable_env_runner_and_connector_v2=False)
+ .environment('netforge-v0', env_config={'scenario_type': 'ransomware'})
+ .api_stack(
+ enable_rl_module_and_learner=False,
+ enable_env_runner_and_connector_v2=False,
+ )
.multi_agent(
policies={
- "red_rmappo": (None, None, None, {"model": {"use_lstm": True}}),
- "blue_rmappo": (None, None, None, {"model": {"use_lstm": True}}),
+ 'red_rmappo': (None, None, None, {'model': {'use_lstm': True}}),
+ 'blue_rmappo': (None, None, None, {'model': {'use_lstm': True}}),
},
policy_mapping_fn=lambda agent_id, *a, **kw: (
- "red_rmappo" if "red" in agent_id else "blue_rmappo"
+ 'red_rmappo' if 'red' in agent_id else 'blue_rmappo'
),
)
.training(train_batch_size=1024)
)
algo = config.build()
-result = algo.train()
-```
-
-## Curriculum Learning
-
-`CurriculumWrapper` wraps `NetForgeRLEnv` and automatically progresses through three phases based on rolling mean episode reward.
-
-| Phase | Active hosts | Scenarios | DHCP | Dynamic topology | Reward scale | Graduate at |
-|---|---|---|---|---|---|---|
-| `novice` | 5 | ransomware | off | off | 3× | mean ≥ 60 over 10 eps |
-| `intermediate` | 25 | ransomware, apt | 80 ticks | off | 1.5× | mean ≥ 40 over 15 eps |
-| `expert` | 100 | ransomware, apt, iot | 40 ticks | churn + migration | 1× | final |
-
-```python
-from netforge_rl.environment.curriculum import CurriculumWrapper
-
-env = CurriculumWrapper(
- base_cfg={"max_ticks": 200},
- on_phase_advance=lambda idx, name: print(f"Phase {idx}: {name}"),
-)
-obs, _ = env.reset(seed=0)
```
-Each step's `info` dict contains `__curriculum__` with `phase`, `phase_index`, `mean_reward`, `window_fill`, and `phase_advanced`.
-
-Start from a specific phase or supply custom phases:
-
-```python
-from netforge_rl.environment.curriculum import CurriculumWrapper, PHASES
-
-env = CurriculumWrapper(phases=PHASES, start_phase=1) # start at intermediate
-```
-```
\ No newline at end of file
+Curriculum: {doc}`curriculum`.
diff --git a/docs/training/scenarios.md b/docs/training/scenarios.md
index 68c12f3..9047f55 100644
--- a/docs/training/scenarios.md
+++ b/docs/training/scenarios.md
@@ -1,123 +1,51 @@
-# Custom Scenarios
+# Custom scenarios
-Custom scenarios define the **win conditions** and **reward tensors** for the environment. Core resolution logic (e.g., executing an exploit) is handled independently by the `Functional Core`.
-
-## BaseScenario Interface
-
-Scenarios must inherit from `netforge_rl.scenarios.base_scenario.BaseScenario` and implement two abstract methods:
-
-1. `calculate_reward(self, agent_id, global_state, effect)`: Returns a scalar float representing the delta reward for the current step.
-2. `check_termination(self, global_state)`: Returns a dictionary mapping agent IDs to boolean terminal states.
-
-## Configuration: Ransomware
-
-In a Ransomware configuration, the Red agent seeks to compromise system integrity flags, while the Blue agent seeks to maximize healthy host ratios.
-
-### 1. Structure and Initialization
+Subclass `BaseScenario`: `calculate_reward(agent_id, global_state, effect)` and
+`check_termination(global_state)`. Exploits stay in the functional core.
```python
from netforge_rl.scenarios.base_scenario import BaseScenario
-class CustomRansomware(BaseScenario):
- def __init__(self, agents):
- self.agents = agents
-```
-
-### 2. Base Reward Routing
-A central `calculate_reward` method routes scalar logic to specific agent functions, applying global penalties (like action cost) universally.
-
-```python
+class CustomRansomware(BaseScenario):
def calculate_reward(self, agent_id, global_state, effect=None):
reward = 0.0
-
- # Base penalty for taking actions
if effect and getattr(effect, 'cost', 0) > 0:
reward -= effect.cost * 0.05
-
if 'red' in agent_id:
return self._red_reward(global_state, effect)
return self._blue_reward(global_state, effect)
-```
-### 3. Red Agent Logic
-
-Red agents are rewarded exclusively for successfully modifying the `GlobalNetworkState` access and integrity tensors.
-
-```python
def _red_reward(self, global_state, effect):
- reward = 0.0
- if effect and effect.success and effect.state_deltas:
- for key, val in effect.state_deltas.items():
- if 'privilege' in key and val == 'Root':
- reward += 5.0
- if 'system_integrity' in key and val == 'kinetic_destruction':
- reward += 10000.0
- return reward
-```
-
-### 4. Blue Agent Logic
+ if not (effect and effect.success and effect.state_deltas):
+ return 0.0
+ r = 0.0
+ for key, val in effect.state_deltas.items():
+ if 'privilege' in key and val == 'Root':
+ r += 5.0
+ if 'system_integrity' in key and val == 'kinetic_destruction':
+ r += 10000.0
+ return r
-Blue agents are rewarded proportionally to the number of nodes maintaining a baseline (healthy) state vector.
-
-```python
def _blue_reward(self, global_state, effect):
+ hosts = global_state.all_hosts.values()
healthy = sum(
- 1 for h in global_state.all_hosts.values()
- if h.compromised_by == 'None' and h.status != 'isolated'
+ 1 for h in hosts if h.compromised_by == 'None' and h.status != 'isolated'
)
return healthy / len(global_state.all_hosts)
-```
-
-### 5. Termination Logic
-
-The episode halts immediately if a critical Cyber-Physical node transitions to a destroyed state.
-```python
def check_termination(self, global_state):
- if any(h.system_integrity == 'kinetic_destruction' for h in global_state.all_hosts.values()):
- return {agent: True for agent in self.agents}
-
- return {agent: False for agent in self.agents}
-```
-
-## OT/SCADA Scenario: `ot_stuxnet`
-
-The `OTStuxnetScenario` rewards Red for achieving physical process destruction on PLC hosts, and Blue for keeping OT nodes clean. A 25% chance each episode generates an OT subnet (`10.0.99.0/24`) with `PLC_Firmware` hosts.
-
-### Physical state
-
-Each PLC host tracks three process variables governed by first-order lag ODEs (in `PLCPhysicsEngine`):
-
-| Variable | Nominal | Alarm | Critical | τ (ticks) |
-|---|---|---|---|---|
-| temperature (°C) | 40–60 | > 80 | > 120 | 20 |
-| pressure (bar) | 90–110 | > 130 or < 70 | > 180 or < 30 | 10 |
-| flow_rate (L/min) | 40–60 | > 90 or < 20 | > 150 or < 5 | 5 |
-
-Each tick: `x[t+1] = x[t] + (setpoint − x[t]) / τ + noise`
-
-### Attack flow
-
-`OverloadPLC` (red_operator action ID 20, requires Root on a PLC host) manipulates setpoints rather than directly spiking values:
-
-```python
-# setpoints jump to unsafe targets; physics engine drives process toward them
-temperature_setpoint += uniform(80, 150) # °C above normal
-pressure_setpoint *= uniform(1.5, 2.0) # 1.5–2× nominal
-flow_rate_setpoint = 3.0 # near-zero (choke)
+ dead = any(
+ h.system_integrity == 'kinetic_destruction'
+ for h in global_state.all_hosts.values()
+ )
+ return {agent: dead for agent in self.agents}
```
-Blue receives `SCADA_PHYSICAL_ALARM` SIEM alerts (severity 7) when variables enter alarm range, and `SCADA_KINETIC_BREACH` (severity 10) when critical thresholds are crossed. Blue has roughly 10 ticks between first alarm and physical destruction to isolate the PLC.
+`ot_stuxnet`: PLC ODE physics, `OverloadPLC` moves setpoints, Blue has ~10 ticks
+after `SCADA_PHYSICAL_ALARM` to isolate. Tables: {doc}`../architecture/ot_physics`.
```python
env = NetForgeRLEnv({'scenario_type': 'ot_stuxnet', 'max_ticks': 300})
-```
-
-## Registering Configurations
-
-Scenario classes are registered dynamically via the `parallel_env` instantiation dictionary:
-
-```python
env = NetForgeRLEnv({'scenario_type': 'custom_ransomware'})
-```
\ No newline at end of file
+```
diff --git a/examples/submissions/heuristic_blue.py b/examples/submissions/heuristic_blue.py
index a8ac9a9..8584b1a 100644
--- a/examples/submissions/heuristic_blue.py
+++ b/examples/submissions/heuristic_blue.py
@@ -1,6 +1,6 @@
"""CAGE-style ``make_blue`` example.
- netforge evaluate examples/submissions/heuristic_blue.py --seeds 0 --max-ticks 20
+netforge evaluate examples/submissions/heuristic_blue.py --seeds 0 --max-ticks 20
"""
NAME = 'heuristic-blue'
diff --git a/mkdocs.yml b/mkdocs.yml
deleted file mode 100644
index 926c1e7..0000000
--- a/mkdocs.yml
+++ /dev/null
@@ -1,128 +0,0 @@
-site_name: NetForge RL
-site_description: Multi-Agent Cybersecurity Reinforcement Learning Environment
-site_author: Igor Jankowski
-site_url: https://reforcemind.github.io/NetForge_RL/
-repo_url: https://github.com/reforcemind/NetForge_RL
-repo_name: reforcemind/NetForge_RL
-edit_uri: edit/main/docs/
-
-theme:
- name: material
- palette:
- - scheme: default
- primary: teal
- accent: deep orange
- toggle:
- icon: material/brightness-7
- name: Switch to dark mode
- - scheme: slate
- primary: teal
- accent: deep orange
- toggle:
- icon: material/brightness-4
- name: Switch to light mode
- font:
- text: Libre Franklin
- code: JetBrains Mono
- features:
- - navigation.tabs
- - navigation.tabs.sticky
- - navigation.sections
- - navigation.expand
- - navigation.top
- - navigation.footer
- - search.highlight
- - search.share
- - content.code.copy
- - content.code.annotate
- - content.tabs.link
- icon:
- repo: fontawesome/brands/github
- logo: material/shield-lock
-
-extra_css:
- - stylesheets/custom.css
-
-extra:
- social:
- - icon: fontawesome/brands/github
- link: https://github.com/xaiqo/NetForge_RL
-
-markdown_extensions:
- - admonition
- - pymdownx.details
- - pymdownx.superfences:
- custom_fences:
- - name: mermaid
- class: mermaid
- format: !!python/name:pymdownx.superfences.fence_code_format
- - pymdownx.highlight:
- anchor_linenums: true
- - pymdownx.inlinehilite
- - pymdownx.tabbed:
- alternate_style: true
- - pymdownx.emoji:
- emoji_index: !!python/name:material.extensions.emoji.twemoji
- emoji_generator: !!python/name:material.extensions.emoji.to_svg
- - pymdownx.tasklist:
- custom_checkbox: true
- - attr_list
- - md_in_html
- - tables
- - toc:
- permalink: true
-
-plugins:
- - search
- - mkdocstrings:
- handlers:
- python:
- paths: [.]
- options:
- docstring_style: google
- show_source: true
- show_root_heading: true
- show_root_full_path: false
- members_order: source
- separate_signature: true
- show_signature_annotations: true
-
-nav:
- - Home: index.md
- - Quick Start: quickstart.md
- - Datasheet: DATASHEET.md
- - Environment:
- - Difficulty & Splits: environment/difficulty.md
- - Reproducibility: environment/reproducibility.md
- - Graph Observations: environment/graph_observations.md
- - Diagnostics: diagnostics/overview.md
- - Interoperability:
- - SOC Export (OCSF): interop/soc_export.md
- - Architecture:
- - Overview: architecture/overview.md
- - Dynamic Topologies: architecture/dynamic_topology.md
- - OT/SCADA Physics: architecture/ot_physics.md
- - Multi-Modal Observations: architecture/multimodal_obs.md
- - Zero-Trust Identity: architecture/zero_trust.md
- - Sim2Real Bridge: architecture/sim2real.md
- - NLP-SIEM Pipeline: architecture/nlp_siem.md
- - Cybersecurity Reference:
- - Environment Overview: cybersec/overview.md
- - Threat Model & ATT&CK: cybersec/threat_model.md
- - Red Team Actions: cybersec/red_actions.md
- - Blue Team Actions: cybersec/blue_actions.md
- - API Reference:
- - Environment: api/environment.md
- - Sim2Real: api/sim2real.md
- - SIEM: api/siem.md
- - NLP Encoder: api/nlp.md
- - Training Guide:
- - Curriculum Learning: training/curriculum.md
- - RLlib + LSTM: training/rllib.md
- - Scenarios: training/scenarios.md
- - Benchmarks:
- - Overview: benchmarks/overview.md
- - Baselines: benchmarks/baselines.md
- - Self-Play & Elo: benchmarks/self_play.md
- - Run: benchmarks/run.md
- - Changelog: changelog.md
diff --git a/pyproject.toml b/pyproject.toml
index 09d5284..a22807c 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -122,6 +122,7 @@ docstring-code-line-length = "dynamic"
"netforge_rl/semantic/clients/*.py" = ["F401"]
"netforge_rl/render/replay_html.py" = ["E501"]
[tool.ruff.lint]
+select = ["E4", "E7", "E9", "F"]
ignore = ["E701", "E402", "E702"]
[tool.setuptools.packages.find]
diff --git a/tests/render/test_render_pipeline.py b/tests/render/test_render_pipeline.py
index 75bf41c..daf722d 100644
--- a/tests/render/test_render_pipeline.py
+++ b/tests/render/test_render_pipeline.py
@@ -68,6 +68,8 @@ def test_frame_recorder_append_and_len() -> None:
def test_frame_recorder_save_requires_frames(tmp_path) -> None:
pytest.importorskip('moviepy')
from netforge_rl.render.recorder import FrameRecorder
+
+ rec = FrameRecorder(fps=4)
with pytest.raises(ValueError):
rec.save(tmp_path / 'empty.gif')
- 
- 
- 
- 
+
+
+# NetForge
+
+**Cybersecurity gym for reinforcement learning**
+
+Red attacks a network. Blue contains it from SIEM, not from an oracle.
+PettingZoo (1 Red + 3 Blue) or Gymnasium (one Blue vs scripted Red).
+
+**Python 3.12+** · **PettingZoo** · **Gymnasium** · **Not on PyPI**
+
+[Docs](https://reforcemind.github.io/NetForge_RL/) ·
+[Start](https://reforcemind.github.io/NetForge_RL/getting-started.html) ·
+[Python](https://reforcemind.github.io/NetForge_RL/python-quickstart.html) ·
+[Cyber](https://reforcemind.github.io/NetForge_RL/cybersec/overview.html) ·
+[Contributing](CONTRIBUTING.md)
+
+
+
+
+