From 81e9c30b9894f7c91098681b9e801b082649d1f5 Mon Sep 17 00:00:00 2001 From: Chris Feijoo Date: Wed, 12 Aug 2026 02:42:01 +0200 Subject: [PATCH] Migrate the architecture prose into the docs bundle MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `petrinaut-core/docs/architecture/` was 3,100 lines of hand-written HTML plus a stylesheet, served by nothing and reachable only by opening files off disk. Its content is the half a generated page cannot produce: why a boundary sits where it does, what the alternatives were, which mistakes are easy to make. It moves to `content/` as MDX, where it merges into the generated tree instead of sitting beside it. A page names the layer it explains with `attachTo: core.simulation`, and lands under that layer's page with the generated reference for the same code. Links are written by name — `layer:core.simulation.engine`, `doc:simulation/memory-model` — and resolved at emit time, because a page's depth is only known once `attachTo` is applied. Unresolved targets fail the build. Five simulation deep-dives carry the detail the generated pages do not: the memory model and frame format, the fixed stage order of a step, the HIR pipeline and what the sandbox does not protect against, the worker protocol and its ack contract, and why only aggregates reach the UI from a Monte Carlo run. Four diagram components render the parts that are genuinely visual — byte layouts, lane diagrams, pipelines and message sequences. Migrating surfaced two claims that had gone stale: UUID support described as unimplemented, which is fully implemented, and a sandbox description thinner than what the code now does. These pages are hand-written and can go stale, which is why they avoid restating facts the generated pages own. The generated half is checked against the code; this half is not, and the docs say so. --- AGENTS.md | 4 + .../docs/architecture/architecture.css | 478 ----------- .../docs/architecture/authoring.html | 288 ------- .../docs/architecture/engine.html | 770 ------------------ .../docs/architecture/index.html | 439 ---------- .../docs/architecture/monte-carlo.html | 376 --------- .../docs/architecture/worker.html | 752 ----------------- .../content/components/byte-map.tsx | 88 ++ .../content/components/diagram.css | 257 ++++++ .../content/components/inline.tsx | 27 + .../content/components/lanes.tsx | 92 +++ .../content/components/pipeline.tsx | 67 ++ .../content/components/sequence.tsx | 94 +++ .../petrinaut-arch-docs/content/index.mdx | 78 ++ .../content/simulation/experiments.mdx | 126 +++ .../content/simulation/memory-model.mdx | 191 +++++ .../content/simulation/protocol.mdx | 168 ++++ .../content/simulation/stepping.mdx | 76 ++ .../content/simulation/user-code.mdx | 106 +++ .../content/two-execution-paths.mdx | 52 ++ .../content/working-on-the-architecture.mdx | 120 +++ 21 files changed, 1546 insertions(+), 3103 deletions(-) delete mode 100644 libs/@hashintel/petrinaut-core/docs/architecture/architecture.css delete mode 100644 libs/@hashintel/petrinaut-core/docs/architecture/authoring.html delete mode 100644 libs/@hashintel/petrinaut-core/docs/architecture/engine.html delete mode 100644 libs/@hashintel/petrinaut-core/docs/architecture/index.html delete mode 100644 libs/@hashintel/petrinaut-core/docs/architecture/monte-carlo.html delete mode 100644 libs/@hashintel/petrinaut-core/docs/architecture/worker.html create mode 100644 libs/@local/petrinaut-arch-docs/content/components/byte-map.tsx create mode 100644 libs/@local/petrinaut-arch-docs/content/components/diagram.css create mode 100644 libs/@local/petrinaut-arch-docs/content/components/inline.tsx create mode 100644 libs/@local/petrinaut-arch-docs/content/components/lanes.tsx create mode 100644 libs/@local/petrinaut-arch-docs/content/components/pipeline.tsx create mode 100644 libs/@local/petrinaut-arch-docs/content/components/sequence.tsx create mode 100644 libs/@local/petrinaut-arch-docs/content/index.mdx create mode 100644 libs/@local/petrinaut-arch-docs/content/simulation/experiments.mdx create mode 100644 libs/@local/petrinaut-arch-docs/content/simulation/memory-model.mdx create mode 100644 libs/@local/petrinaut-arch-docs/content/simulation/protocol.mdx create mode 100644 libs/@local/petrinaut-arch-docs/content/simulation/stepping.mdx create mode 100644 libs/@local/petrinaut-arch-docs/content/simulation/user-code.mdx create mode 100644 libs/@local/petrinaut-arch-docs/content/two-execution-paths.mdx create mode 100644 libs/@local/petrinaut-arch-docs/content/working-on-the-architecture.mdx diff --git a/AGENTS.md b/AGENTS.md index 900a948f282..a8221a79432 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -127,6 +127,10 @@ Verify with `yarn workspace @local/petrinaut-arch-docs lint:arch-docs`, which fa Full reference: `libs/@local/petrinaut-arch-docs/README.md`. +Hand-written MDX in `libs/@local/petrinaut-arch-docs/content/` carries the reasoning an import graph cannot express — why a boundary sits where it does, what the alternatives were. It is optional; the system works with that directory absent. + +Give such a page `attachTo: ` in its frontmatter and it nests inside the generated tree beneath that layer, rather than sitting in a separate section — that is how the simulation deep-dives are wired. Link between pages with `[text](layer:core.simulation.engine)` or `[text](doc:simulation/memory-model)`; relative paths break when a page's `attachTo` changes, and unresolved targets fail CI. + ## Contextual Rules CRITICAL: For the files referenced below, use your Read tool to load it on a need-to-know basis, ONLY when relevant to the SPECIFIC task at hand: diff --git a/libs/@hashintel/petrinaut-core/docs/architecture/architecture.css b/libs/@hashintel/petrinaut-core/docs/architecture/architecture.css deleted file mode 100644 index c9b6ef590c5..00000000000 --- a/libs/@hashintel/petrinaut-core/docs/architecture/architecture.css +++ /dev/null @@ -1,478 +0,0 @@ -/* Shared styles for the simulation architecture docs (architecture*.html). */ - -:root { - /* One colour per module, used consistently across all pages. */ - --engine: #2563eb; /* blue — engine (stepping, frames) */ - --engine-bg: #eff6ff; - --authoring: #9333ea; /* purple — compilation of user code */ - --authoring-bg: #faf5ff; - --worker: #059669; /* green — worker + transport protocol */ - --worker-bg: #ecfdf5; - --mc: #ea580c; /* orange — monte carlo / experiments */ - --mc-bg: #fff7ed; - --memory: #dc2626; /* red — buffers, where bytes live */ - --memory-bg: #fef2f2; - --ui: #0891b2; /* cyan — React / UI consumers */ - --ui-bg: #ecfeff; - --ink: #111111; - --muted: #555555; - --line: #d9d9d9; -} - -* { - box-sizing: border-box; -} - -body { - margin: 0 auto; - padding: 2rem 1.5rem 5rem; - max-width: 1000px; - background: #ffffff; - color: var(--ink); - font: - 15px/1.55 -apple-system, - BlinkMacSystemFont, - "Segoe UI", - Roboto, - Helvetica, - Arial, - sans-serif; -} - -h1 { - font-size: 1.7rem; - margin: 0.5rem 0 0.25rem; -} - -h2 { - font-size: 1.25rem; - margin: 2.5rem 0 0.75rem; - padding-bottom: 0.3rem; - border-bottom: 1px solid var(--line); -} - -h3 { - font-size: 1.02rem; - margin: 1.75rem 0 0.5rem; -} - -p { - max-width: 76ch; -} - -.subtitle { - color: var(--muted); - margin-top: 0; -} - -code, -pre { - font-family: - ui-monospace, "SF Mono", SFMono-Regular, Menlo, Consolas, monospace; - font-size: 0.86em; - background: #f5f5f5; - border-radius: 4px; -} - -code { - padding: 0.1em 0.35em; -} - -pre { - padding: 0.8rem 1rem; - overflow-x: auto; - border: 1px solid #ececec; -} - -pre code { - padding: 0; - background: none; -} - -a { - color: var(--engine); -} - -/* ---- Top navigation ------------------------------------------------- */ - -nav.docs { - display: flex; - flex-wrap: wrap; - gap: 0.4rem; - padding: 0.6rem 0; - margin-bottom: 1rem; - border-bottom: 2px solid var(--ink); - font-size: 0.9rem; -} - -nav.docs a { - text-decoration: none; - color: var(--ink); - padding: 0.25rem 0.7rem; - border: 1px solid var(--line); - border-radius: 999px; -} - -nav.docs a.current { - color: #fff; - background: var(--ink); - border-color: var(--ink); -} - -nav.docs a.tag-engine.current { - background: var(--engine); - border-color: var(--engine); -} -nav.docs a.tag-authoring.current { - background: var(--authoring); - border-color: var(--authoring); -} -nav.docs a.tag-worker.current { - background: var(--worker); - border-color: var(--worker); -} -nav.docs a.tag-mc.current { - background: var(--mc); - border-color: var(--mc); -} - -/* ---- Legend chips ---------------------------------------------------- */ - -.legend { - display: flex; - flex-wrap: wrap; - gap: 0.5rem 1rem; - margin: 0.75rem 0 1.25rem; - font-size: 0.82rem; -} - -.legend span::before { - content: ""; - display: inline-block; - width: 0.75em; - height: 0.75em; - margin-right: 0.4em; - border-radius: 3px; - vertical-align: -0.05em; -} - -.legend .l-engine::before { - background: var(--engine); -} -.legend .l-authoring::before { - background: var(--authoring); -} -.legend .l-worker::before { - background: var(--worker); -} -.legend .l-mc::before { - background: var(--mc); -} -.legend .l-memory::before { - background: var(--memory); -} -.legend .l-ui::before { - background: var(--ui); -} - -/* ---- Tables ----------------------------------------------------------- */ - -table { - border-collapse: collapse; - width: 100%; - margin: 0.75rem 0 1.25rem; - font-size: 0.88rem; -} - -th, -td { - border: 1px solid var(--line); - padding: 0.4rem 0.6rem; - text-align: left; - vertical-align: top; -} - -th { - background: #f6f6f6; - font-weight: 600; -} - -/* ---- Generic diagram primitives -------------------------------------- */ - -.box { - border: 1.5px solid var(--ink); - border-radius: 8px; - padding: 0.55rem 0.8rem; - background: #fff; -} - -.box strong { - display: block; - font-size: 0.9rem; -} - -.box small { - color: var(--muted); - font-size: 0.78rem; - line-height: 1.35; - display: block; -} - -.box.b-engine { - border-color: var(--engine); - background: var(--engine-bg); -} -.box.b-engine strong { - color: var(--engine); -} -.box.b-authoring { - border-color: var(--authoring); - background: var(--authoring-bg); -} -.box.b-authoring strong { - color: var(--authoring); -} -.box.b-worker { - border-color: var(--worker); - background: var(--worker-bg); -} -.box.b-worker strong { - color: var(--worker); -} -.box.b-mc { - border-color: var(--mc); - background: var(--mc-bg); -} -.box.b-mc strong { - color: var(--mc); -} -.box.b-memory { - border-color: var(--memory); - background: var(--memory-bg); -} -.box.b-memory strong { - color: var(--memory); -} -.box.b-ui { - border-color: var(--ui); - background: var(--ui-bg); -} -.box.b-ui strong { - color: var(--ui); -} - -/* Vertical pipeline: boxes separated by a downward arrow. */ -.pipeline { - display: flex; - flex-direction: column; - gap: 0; - margin: 1rem 0 1.5rem; - max-width: 620px; -} - -.pipeline > .step { - position: relative; -} - -.pipeline > .step + .step { - margin-top: 1.6rem; -} - -.pipeline > .step + .step::before { - content: "▼"; - position: absolute; - top: -1.45rem; - left: 2rem; - font-size: 0.8rem; - color: var(--muted); -} - -.pipeline > .step + .step[data-arrow]::after { - content: attr(data-arrow); - position: absolute; - top: -1.5rem; - left: 3.4rem; - font-size: 0.75rem; - color: var(--muted); - font-family: ui-monospace, Menlo, monospace; -} - -/* Horizontal flow: boxes with a → between them. */ -.hflow { - display: flex; - flex-wrap: wrap; - align-items: stretch; - gap: 0.35rem; - margin: 1rem 0 1.5rem; -} - -.hflow > .box { - flex: 1 1 0; - min-width: 118px; -} - -.hflow > .arr { - align-self: center; - color: var(--muted); - font-size: 1rem; - padding: 0 0.1rem; -} - -/* Two/three column thread lanes. */ -.lanes { - display: grid; - gap: 1rem; - margin: 1rem 0 1.5rem; -} - -.lanes.two { - grid-template-columns: 1fr 1fr; -} -.lanes.three { - grid-template-columns: 1fr 1fr 1fr; -} - -.lane { - border: 1px dashed #bbb; - border-radius: 10px; - padding: 0.75rem; - background: #fcfcfc; -} - -.lane > h4 { - margin: 0 0 0.6rem; - font-size: 0.8rem; - text-transform: uppercase; - letter-spacing: 0.06em; - color: var(--muted); -} - -.lane .box { - margin-bottom: 0.6rem; -} - -.lane .box:last-child { - margin-bottom: 0; -} - -/* ---- Byte-layout bars -------------------------------------------------- */ - -.bytes { - display: flex; - width: 100%; - margin: 0.75rem 0 0.25rem; - border: 1.5px solid var(--ink); - border-radius: 6px; - overflow: hidden; - font-family: ui-monospace, Menlo, monospace; - font-size: 0.72rem; - text-align: center; -} - -.bytes .seg { - padding: 0.45rem 0.2rem; - border-right: 1.5px solid var(--ink); - overflow: hidden; - white-space: nowrap; - text-overflow: ellipsis; -} - -.bytes .seg:last-child { - border-right: none; -} - -.bytes .seg em { - display: block; - font-style: normal; - color: var(--muted); - font-size: 0.66rem; -} - -.seg.s-header { - background: #ededed; -} -.seg.s-u32 { - background: #d3e5ff; -} -.seg.s-f64 { - background: #fecaca; -} -.seg.s-u8 { - background: #d1fae5; -} -.seg.s-tokens { - background: #fee2b8; -} - -.bytes-caption { - font-size: 0.78rem; - color: var(--muted); - margin: 0.15rem 0 1.25rem; -} - -/* ---- Callouts ----------------------------------------------------------- */ - -.note { - border-left: 4px solid #eab308; - background: #fefce8; - padding: 0.6rem 0.9rem; - margin: 1rem 0; - border-radius: 0 6px 6px 0; - font-size: 0.9rem; -} - -.note.refactor { - border-left-color: var(--memory); - background: var(--memory-bg); -} - -.note strong { - display: inline; -} - -/* ---- At-a-glance card ---------------------------------------------------- */ - -.glance { - display: grid; - grid-template-columns: max-content 1fr; - gap: 0.3rem 1.2rem; - border: 1px solid var(--line); - border-radius: 8px; - padding: 0.85rem 1rem; - margin: 1rem 0 1.5rem; - font-size: 0.88rem; - background: #fafafa; -} - -.glance dt { - font-weight: 600; - color: var(--muted); -} - -.glance dd { - margin: 0; -} - -/* ---- SVG diagrams --------------------------------------------------------- */ - -figure { - margin: 1rem 0 1.5rem; -} - -figure svg { - max-width: 100%; - height: auto; -} - -figcaption { - font-size: 0.78rem; - color: var(--muted); - margin-top: 0.25rem; -} - -@media (max-width: 720px) { - .lanes.two, - .lanes.three { - grid-template-columns: 1fr; - } -} diff --git a/libs/@hashintel/petrinaut-core/docs/architecture/authoring.html b/libs/@hashintel/petrinaut-core/docs/architecture/authoring.html deleted file mode 100644 index ae048fa3b0a..00000000000 --- a/libs/@hashintel/petrinaut-core/docs/architecture/authoring.html +++ /dev/null @@ -1,288 +0,0 @@ - - - - - - Simulation Architecture — Compilation (Authoring) - - - - - -

Compilation (Authoring)

-

- The HIR pipeline turns user-authored TypeScript into checked, - source-spanned buffer programs. Scenario code retains a separate - same-realm compiler hardened against accidents, not attackers. -

- -
-
Inputs
-
- Code strings stored on the SDCPN document (lambda, kernel, dynamics, - metric, scenario) -
-
Outputs
-
- Versioned HIR artifacts for runtime code; InitialMarking - for scenarios -
-
Errors
-
- Thrown at compile time with the offending item’s ID - (SDCPNItemError) → surfaced as protocol - error messages / result objects -
-
Runs on
-
- LSP worker (HIR compilation) · simulation workers (buffer programs) · - main thread (scenarios and timeline metric evaluation) -
-
- -

The compilation pipeline (module-style code)

-

- Dynamics, lambdas, kernels, and metrics all compile through the HIR. The - first three are authored as export default Wrapper(fn) - modules; metrics are authored as function bodies. -

- -
-
- User source - export default Lambda((input, parameters) => …) - (TypeScript allowed) -
- → -
- Lower to HIR - Parse the supported TypeScript subset with exact source spans -
- → -
- Check + analyze - Validate against places, token layouts, parameters, and output - shapes -
- → -
- Emit artifact - Generate buffer-native JS with offsets and strides baked in -
- → -
- Instantiate in runtime - Validate v4 fingerprint, then bind parameters, pool, and ABI - views -
-
- -
- The - LSP virtual files - (lsp/generate-virtual-files.ts) are the type-level twin of - this pipeline: they generate the Color_*, - Parameters, Dynamics, - TransitionKernel… declarations that Monaco checks against, so - editor diagnostics and runtime artifacts are produced from the same HIR - checks and emitters. -
- -

What gets compiled, where it runs

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
SurfaceAuthored asCompiled byExecuted inSignature at runtime
Transition lambdamodule, Lambda(fn)compileHirArtifacts in the LSP workerWorker, per enablement check - (f64, u64, u8, placeBases, indices) → boolean | rate -
Transition kernelmodule, TransitionKernel(fn)compileHirArtifacts in the LSP workerWorker, per firing - (views, placeBases, indices, outputViews, sink) → void -
Dynamicsmodule, Dynamics(fn) - compileHirArtifacts (colours with a real element) - Worker, per step per place - (placeBytes, tokenCount) → Float64Array -
Metricplain function body (no module)compileHirArtifacts in the LSP workerMain thread (timeline) · MC worker (experiments) - (f64, u64, u8, placeCounts, placeOffsets) → finite number - — NaN/∞ throw -
Scenarioper-place expressions or one code-mode bodyauthoring/scenario/compile-scenario.tsMain thread, before createSimulation→ InitialMarking (+ scenario parameter values)
Place visualizermodule, Visualization(fn) returning JSX - Host UI package (@hashintel/petrinaut, - ui/lib/compile-visualizer.ts — the one inherently - React-based surface, since visualizers author JSX) - Main thread render({tokens, parameters}) → ReactElement
- -

Scenario compilation → InitialMarking

- -
-
-

per_place mode

-
- Uncoloured place → expression string - Evaluated with parameters and scenario in - scope; result rounded and clamped >= 0 (a token count). -
-
- Coloured place → row arrays - Rows in colour element order; each row coerced through - coerceTokenRecord (typed defaults for missing columns, - extra columns throw). -
-
-
-

code mode

-
- One function body - Returns { PlaceName: count | TokenRecord[] } keyed by - place name (per_place uses place IDs — a known - asymmetry). Unknown names are silently dropped. -
-
- Result: InitialMarking (JSON) - Keyed by place ID. Fed to createSimulation, packed to - binary in buildSimulation. -
-
-
- -

Metric compilation and evaluation

-
-
- compileHirArtifacts - Lower, typecheck, and emit a buffer-native metric program. -
- → -
- createHirMetricEvaluator - Bind referenced place ordinals and the per-run string pool. -
- → -
- Per-frame evaluation - Quick sim: evaluated on the main thread against stored frames. - Experiments: evaluated inside the MC worker per run per frame - (metrics/specs.ts). -
-
- -

Scenario sandboxing (authoring/sandbox.ts)

- - -

Distributions (deferred sampling)

-

- HIR-emitted kernels construct small runtime values for - Distribution.Gaussian / Uniform / Lognormal. Kernels defer - sampling through the ABI sink; the engine samples each value once per - token with the seeded RNG (sample-distribution.ts), so - chained .map(fn) transforms share a single draw and runs stay - reproducible. When stochasticity is disabled, the runtime isn’t injected - and plain values are required. -

- -
- Compatibility seam. HIR artifacts contain a version and - fingerprint of their sanitized compilation input. Any schema, code, or - extension change requires recompilation before the engine will run them. -
- - diff --git a/libs/@hashintel/petrinaut-core/docs/architecture/engine.html b/libs/@hashintel/petrinaut-core/docs/architecture/engine.html deleted file mode 100644 index d135b93ba2c..00000000000 --- a/libs/@hashintel/petrinaut-core/docs/architecture/engine.html +++ /dev/null @@ -1,770 +0,0 @@ - - - - - - Simulation Architecture — Engine - - - - - -

Engine

-

- engine/ + frames/ — builds an SDCPN into a - runnable SimulationInstance and advances it one immutable - binary frame at a time. -

- -
-
Entry points
-
- buildSimulation(input) · - computeNextFrame(simulation) -
-
Inputs
-
- SDCPN snapshot, InitialMarking, parameter values, seed, dt, - maxTime (number | null; null = no limit), extensions -
-
Outputs
-
- New EngineFrame per step + - completionReason: "maxTime" | "deadlock" | null -
-
Events
-
- None — pure return values; the worker turns - them into protocol messages -
-
Runs on
-
- Worker thread (quick sim). The same engine functions are reused by - Monte Carlo with different frame - storage -
-
- -

Lifecycle

- -
-
- buildSimulation(input) - Flattens component instances, compiles user code (authoring): lambdas only where available, kernels only for coloured outputs, - dynamics only for colours with at least one real element. - Packs the initial marking into frame 0 (coercing token values by - element type). -
-
- SimulationInstance - frames: EngineFrame[] (history, frame 0 included) · - frameLayout · compiledTransitions · - differentialEquationFns · parameterValues · - dt · maxTime · currentTime · currentFrameNumber · rngState -
-
- computeNextFrame(simulation) - Returns a new instance value with one more frame appended — - the previous frames are never mutated. -
-
- -

One step, in order

- -
-
- 1 · maxTime? - If maxTime !== null and - currentTime >= maxTime, return - "maxTime" without computing. -
- → -
- 2 · Dynamics - Per place with a differential equation: slice its token region, - decode tokens, call the user Dynamics fn, apply Euler - x += dx·dt to real slots (discrete derivatives - are forced to 0). Builds an intermediate frame. -
- → -
- 3 · Transitions - executeTransitions: for each transition — enablement - (arc weights, inhibitor/read arcs), token-combination enumeration, - lambda (predicate / stochastic rate vs seeded RNG), kernel for - coloured outputs. Removals compact the buffer; additions - append. -
- → -
- 4 · Timers - If nothing fired, advance every transition’s - timeSinceLastFiringMs by dt. -
- → -
- 5 · Complete? - maxTime reached, or deadlock (nothing fired and no - transition is enabled). -
-
- -
- Every stage that changes state allocates a new - ArrayBuffer via createEngineFrame() — dynamics, - token removal, and token insertion each rebuild the frame. Immutability - makes history trivially correct at the cost of allocation churn per step. -
- -

EngineFrame — the binary format

-

- One ArrayBuffer, read through section-typed views. The frame - stores no IDs and no time — decoding requires the - EngineFrameLayout (place/transition order, per-place - strideBytes and TokenSlotLayout) derived from - the SDCPN, and time travels as payload metadata. -

-

- Source of truth: - libs/@hashintel/petrinaut-core/src/simulation/frames/internal-frame.ts - — createEngineFrame() computes every offset below and is the - only frame constructor; readEngineFrame() recreates the same - views for reading; the header constants live in the - HeaderOffset enum. -

- -
- - - - - - - - - - Memory map — example instance - - - byte offsets on the left · one typed-array view per section on the - right - - - - - - header — 64 B fixed - - magic "PFRM" · version · P · T · - - - section offsets · byteLength - - DataView, little-endian - - - - place token counts - u32 × P = 8 B - Uint32Array(buf, 64, P) - - tokens currently in each place - - - - - place value offsets - u32 × P = 8 B - Uint32Array(buf, 72, P) - - each place’s token run, relative to the token region start - - - - - transition elapsed - f64 × T = 16 B - Float64Array(buf, 80, T) - - ms since each transition last fired - - - - - transition firing counts - u32 × T = 8 B - Uint32Array(buf, 96, T) - - cumulative firings since frame 0 - - - - - - fired flags - - — u8 × T = 2 B - - - Uint8Array(buf, 104, T) - - - - padding — 6 B - - alignTo(…, 8) so f64 views stay aligned - - - - - - tokenBytes / tokenF64 views (b112…) - - - packed token structs — place-major, token-major; - - - this all-real example is pure f64 (booleans = u8) - - - - - - slot 0 · b112 · p1 · tok0 · x - - slot 1 · b120 · p1 · tok0 · y - - slot 2 · b128 · p1 · tok0 · z - - slot 3 · b136 · p1 · tok1 · x - - slot 4 · b144 · p1 · tok1 · y - - slot 5 · b152 · p1 · tok1 · z - - slot 6 · b160 · p2 · tok0 · a - - slot 7 · b168 · p2 · tok0 · b - - - - - p1 — byteOffset 0 - - 2 tokens × stride 24 B (x,y,z — all real) - - - p2 — byteOffset 48 - - 1 token × stride 16 B (a,b) → byte 112 + 48 - - - - - 0 - 64 - 72 - 80 - 96 - 104 - 106 - 112 - 176 - - - - byteLength = 176 (recorded in the header, checked on read) - - -
- Exact offsets for a concrete instance: P = 2 places (p1 - = 2 tokens × 24-byte stride, p2 = 1 token × 16-byte stride — all - dimensions real here, so every field is one f64; a boolean - dimension would appear as a single u8 byte inside its token’s stride), - T = 2 transitions → 176-byte frame. Sections are laid - out by createEngineFrame(); sections and strides are padded - to 8-byte boundaries by alignTo(). Block heights are not to - scale. -
-
- -

Header (offsets in bytes, little-endian, via DataView)

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
OffsetFieldTypeValue / meaning
0magicu320x5046524d (“PFRM”) — corrupt-frame guard
4versionu162 — the current (only) format; readers assert this on decode
6headerBytesu1664
8 / 12placeCount / transitionCountu32must match the layout (checked on read)
16tokenByteLengthu32token region length in bytes
20–40section offsetsu32 × 6byte offsets of each section above
44byteLengthu32whole-frame length (checked on read)
- -

Token region

-

- Each coloured place owns a contiguous run of - count × strideBytes bytes starting at its byte offset - (uncoloured places have stride 0 and only a count), giving O(1) access to - any place via the layout. Within a token, each element sits at its - layout-computed byte offset: real/integer as - f64, boolean as one u8 byte. Value coercion (integers - rounded, booleans 0/1) lives in engine/token-values.ts; byte - placement in engine/token-layout.ts. -

- -

Who reads and writes the buffer

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
PathDirectionView usedNotes
createEngineFrame(layout, snapshot)write (allocate) - DataView header + section views, bulk - Uint8Array.set - - Only constructor of frames; recomputes byte offsets from counts × - strides. -
- readEngineFrame(layout, frame) → - EngineFrameView - read - Uint32Array sections + tokenBytes (u8) / - tokenF64 views - - Zero-copy; toSnapshot() copies the token region into a - fresh Uint8Array. -
Dynamics (computePlaceNextState)read + writef64 view over a fresh byte copy - Euler touches only realFieldF64Offsets; discrete bytes - copied through untouched. -
- Transition firing (compute-possible-transition, - execute-transitions, remove-tokens…) - read + write - readTokenRecord / - encodeTokenValuesToBytes + byte-range copies - - Input tokens decoded to TokenRecords for user - lambdas/kernels; kernel outputs packed into per-token - Uint8Array blocks. -
SimulationFrameReader (main thread)readSame section views over the cloned buffer - getPlaceTokens() = decoded records via the place’s - TokenSlotLayout; getTransitionState() = - timers/flags. -
- -

Token attribute typing (discrete types)

- - -

Token memory layout — packed structs

-

- Since FRAME_VERSION = 2, tokens are stored as schema-driven - packed structs (array-of-structs; column layout was - considered and rejected because the workload — kernels, copies, UI reads — - is token-oriented). - engine/token-layout.ts (computeTokenSlotLayout) - is the single source of truth: -

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Logical typePhysical typeNotes
realf64 — 8 Bunchanged
integerf64 — 8 B, rounded on read/write - Exact only within ±2^53 (documented in the schema). - i32 rejected (silent wraparound at ±2^31); i64 rejected (bigint is - contagious into user code: fortune * 1.05 would throw). - If >2^53 is ever needed, add a separate opt-in - int64 element type instead of changing - integer. -
booleanu8 — 1 B - Not bit-packed (would break byte-granular copies for marginal - savings). -
uuid (128-bit, FE-1121)u64 × 2 — 16 B - Two little-endian lanes read via the shared - BigUint64Array view, combined to one - bigint at the boundary (~28 ns; lane-compare without - combining for equality, ~4× faster). Never routed through - number (NaN-payload hazard). Kernel outputs are - optional — omitted values auto-generate from the seeded RNG. -
string (FE-769)u64 pool reference — 8 B - The frame stores an ID into an append-only per-run string intern - pool (engine/string-pool.ts); the pool lives on - SimulationInstance, not on the frame, so frames stay - fixed-stride and byte-copyable. Equal strings share one ID; id 0 is - the pre-seeded "", so zeroed buffers decode cleanly. -
- - -

Determinism

-

- seeded-rng.ts provides a pure - nextRandom(state) → [value, nextState]. The RNG state lives - on SimulationInstance.rngState and is threaded through - stochastic lambda sampling and distribution draws, so a given - (SDCPN, marking, parameters, seed, dt) always reproduces the - same frame sequence. -

- -
- Refactoring seams. - (1) Per-step allocation: each stage rebuilds the whole buffer — a - double-buffer strategy (as Monte Carlo already does) would remove most - churn. (2) 128-bit types (UUID, FE-1121) slot into the layout as - u64×2 fields (align 8, read via BigUint64Array) - — the layout machinery is ready; the value plumbing (bigint - boundary, v5 coercion, seeded generation) is not. (3) The worker keeps the - full frames[] history although only the latest frame is - needed to advance — history retention belongs to the main-thread store. -
- - diff --git a/libs/@hashintel/petrinaut-core/docs/architecture/index.html b/libs/@hashintel/petrinaut-core/docs/architecture/index.html deleted file mode 100644 index 0df5a6d6e35..00000000000 --- a/libs/@hashintel/petrinaut-core/docs/architecture/index.html +++ /dev/null @@ -1,439 +0,0 @@ - - - - - - Petrinaut Simulation — Architecture Overview - - - - - -

Simulation Architecture — Overview

-

- How Petrinaut runs SDCPN simulations: who owns the memory, what crosses - thread boundaries, and how results are read back. Code lives in - petrinaut-core/src/simulation/ (engine, authoring, worker, - runtime, monte-carlo). These pages describe the headless core only — - consumers appear as "the host application" using the public API; the React - integration is documented separately in - @hashintel/petrinaut/ARCHITECTURE.md. Generated import maps spanning core, React, and UI modules are in the - dependency diagrams. -

- -
- Engine (stepping & frames) - Compilation (user code) - Worker & protocol - Monte Carlo - Memory / buffers - Host application -
- -

The two execution paths

-

- There are two independent ways to execute a net. - Quick Simulation runs one interactive simulation and - keeps every frame so the host can scrub. - Experiments run many Monte Carlo simulations with - bounded memory (two reusable buffers per run) and only ship - metric aggregates to the host — frame buffers never leave the worker. -

- -
-
-

Main thread — host application

-
- Run configuration - owns InitialMarking, parameter values, seed/dt/maxTime - (null = unbounded); calls createSimulation() -
-
- Playback driver - core playback module — picks the viewed frame - (getFrame(i)), drives ack/backpressure per play - mode -
-
- Experiment consumer - one MonteCarloExperiment handle per experiment, - subscribed to its stores -
-
- Rendering & metrics - read via SimulationFrameReader / metric frames -
-
-
-

Main thread — core runtime

-
- createSimulation() - runtime/simulation.ts — sanitizes SDCPN for extensions, flattens - component instances, owns lifecycle stores + events -
-
- SimulationFrameStore - runtime/frame-store.ts — retains every - SimulationFramePayload (ArrayBuffer + time) -
-
- SimulationFrameReader - frames/frame-reader.ts — typed-array views over a stored frame, - zero-copy reads -
-
- createMonteCarloExperiment() - monte-carlo/runtime/experiment.ts — status / progress / metrics - stores -
-
-
-

Worker threads

-
- simulation.worker - init → buildSimulation(); loop → - computeNextFrame(); streams frames under ack - backpressure -
-
- SimulationInstance.frames[] - full EngineFrame history — the compute source of truth -
-
- monte-carlo.worker - MonteCarloSimulator — round-robin - advanceAll(), metrics observed in-worker -
-
- 2 × ArrayBuffer per run - current / next, swapped each step; no history -
-
-
- -

Where the memory actually lives

-

- Everything the simulation computes is stored in raw - ArrayBuffers read through typed-array views. There is - no object graph of tokens — tokens are packed structs - (real/integer as f64 fields, - boolean as one u8 byte, stride rounded to 8 B), decoded to JS - objects only at the read boundary. -

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
MemoryStructureThreadLifetime / role
SDCPN documentPlain JS objects (editor state)Main - Snapshot is structured-cloned into the worker at init; - later edits don’t affect a running simulation. -
Initial marking - JSON (InitialMarking: count or - TokenRecord[] per place) - Main (host state) - Packed into frame 0 by - buildSimulation → packInitialPlaceMarking. -
EngineFrame (quick sim) - ArrayBuffer: 64-byte header + - Uint32/Float64/Uint8 sections - + packed token structs (byte-addressed places) - Worker - Immutable snapshot per step. The worker appends every frame to - SimulationInstance.frames[] — compute source of truth. -
SimulationFramePayload{ time, frame: ArrayBuffer }Worker → Main - Structured-clone copy per frame (no transfer list). - Retained forever by the in-memory frame store for scrubbing. -
SimulationFrameReader - Uint32Array/Float64Array views over the - stored buffer - Main - Zero-copy views. getPlaceTokens() materializes typed - TokenRecord objects via the place’s - TokenSlotLayout. -
Monte Carlo run buffers - 2 × ArrayBuffer per run (current / next), token-value - region has capacity + growth policy - MC worker - Swapped each step. Never sent to the main thread — only progress - counters and metric frames (small JSON) cross the boundary. -
Compiled user codeVersioned HIR buffer programs; scenarios remain plain JSWorkers (HIR programs) · Main (scenarios, timeline metrics) - Produced by authoring at init time. -
- -
- Refactoring seam — triple retention. Each quick-sim frame - currently exists three times: in the worker history - (SimulationInstance.frames[]), as a structured-clone copy in - the main-thread frame store, and transiently in the - postMessage queue. Frames are never transferred - (Transferable) and the worker history is only read by the - stepping loop’s latest entry. Retention policy is deliberately isolated - behind runtime/frame-store.ts so this can change without - touching consumers. -
- -

Quick Simulation — end-to-end data flow

- -
-
- SDCPN snapshot + InitialMarking + parameters + - seed/dt/maxTime - Assembled by the host (main thread). Scenario compilation may have - produced the marking and parameter overrides first. -
-
- simulation.worker — buildSimulation() - Sanitize by extensions → compile lambdas / kernels / dynamics → pack - frame 0. Replies frame(0) + ready. -
-
- computeNextFrame() × batch - Dynamics (Euler) → transitions (enablement, lambda, kernel) → new - immutable EngineFrame appended; time += dt. -
-
- SimulationFrameStore (main) - Appends every payload; publishes {count, latest} through - the frames store. -
-
- SimulationFrameReader - compileSimulationFrameReader(sdcpn) specialises the - layout once; readers are created per frame on demand (latest() - / getFrame(i)). -
-
- Host reads - The playback driver picks frameIndex; the host renders - from getTransitionState(), getPlaceTokens(), - and getPlaceTokenCount(); metric code reads the raw - packed frame through a HIR evaluator. -
-
- -

Protocols at a glance

-

- Both workers speak a small typed message protocol over - postMessage, wrapped in a - SimulationTransport (queues messages until the worker boots). - Full payloads and sequence diagrams are on the - Worker and - Monte Carlo pages. -

- - - - - - - - - - - - - - - - - - - - - - - - -
Host → WorkerWorker → HostFlow control
- Quick sim
worker/messages.ts - 		- init · start · pause · stop · setBackpressure · ack - - ready · frame · frames · paused · complete · error - - Ack-based backpressure: worker computes at most - maxFramesAhead past the last acked frame. -
- Monte Carlo
monte-carlo/worker/messages.ts - 		init · start · cancel - ready · progress · metricFrames · complete · cancelled · - error - - Fire-and-forget batches (advanceAll() × batchSize per - loop tick); cancellation checked between batches. -
- -

Module map

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
AreaPathOwnsDetails
Engineengine/, frames/Build + stepping, EngineFrame binary format, frame readersengine.html
Compilationauthoring/ - User code → JS functions (lambda, kernel, dynamics, metric, - scenario), sandbox hardening - authoring.html
- Worker + runtime - worker/, runtime/ - Transport protocol, backpressure, lifecycle stores, frame retention - worker.html
Monte Carlomonte-carlo/ - Batch runs, bounded buffers, metric pipeline, experiment handle - monte-carlo.html
- -

Design invariants worth keeping

- - - diff --git a/libs/@hashintel/petrinaut-core/docs/architecture/monte-carlo.html b/libs/@hashintel/petrinaut-core/docs/architecture/monte-carlo.html deleted file mode 100644 index 31a10a8eeba..00000000000 --- a/libs/@hashintel/petrinaut-core/docs/architecture/monte-carlo.html +++ /dev/null @@ -1,376 +0,0 @@ - - - - - - Simulation Architecture — Monte Carlo (Experiments) - - - - - -

Monte Carlo (Experiments)

-

- monte-carlo/ — runs many independent simulations of the same - net with bounded memory, aggregates metrics inside the worker, - and ships only small JSON metric frames to the UI. Frame buffers never - cross the thread boundary. -

- -
-
Core
-
- createMonteCarloSimulator(config) → - MonteCarloSimulator (synchronous, thread-agnostic) -
-
Worker
-
- monte-carlo.worker.ts + worker/messages.ts -
-
Host handle
-
- createMonteCarloExperiment() → stores: - status · progress · metrics + events; - start/cancel/dispose -
-
Inputs
-
- SDCPN, marking, parameters, seed, dt, - maxTime (required), runCount, metric specs -
-
Outputs
-
- MonteCarloWorkerProgress (run counters) + - MonteCarloUserDefinedMetricFrame[] -
-
- -

Layers

- -
-
- Host application - One experiment record per run of - createMonteCarloExperiment(); subscribes to the handle’s - status/progress/metrics stores and renders the metric frames. -
-
- Experiment handle (main) - runtime/experiment.ts — worker mode - (createWorker/transport) or - local mode (runs the simulator on the calling thread, used by - tests/embedding). -
-
- monte-carlo.worker - Builds the simulator + compiles metric specs, then loops: - advanceAll() × batchSize (default 4), post - progress + pending metricFrames, yield, - repeat. cancel is honoured between batches. -
-
- MonteCarloSimulator - Owns N × MonteCarloRun; deterministic round-robin - advanceAll() advances every active run one frame per - call, so long runs don’t starve short ones. -
-
- Engine functions, reused - Same enablement/lambda/kernel/dynamics code as quick sim (transition-effect.ts - adapts them to the MC buffers). -
-
- -

Run model

- - - - - - - - - - - - - - - - - - - - - - - - - -
Per runMeaning
- seed, parameterValues, - initialMarking - - Defaults derived from the experiment config; overridable per run - (runs[]), e.g. seed = base seed + index. -
status - ready → running → complete | error — errors are per - run, other runs continue. -
- frameNumber · currentTime · rngState · completionReason - - Progress; a run completes on its own deadlock or the shared - maxTime. -
- tokenByteCount · tokenByteCapacity · reallocations - - Buffer telemetry, exposed in MonteCarloRunSummary. -
- -

Memory — two buffers per run, swapped every step

- -
-
-

step k

-
- currentFrame (read) state at frame k -
-
- nextFrame (write) - dynamics + transitions write frame k+1 here -
-
-
-

after the step

-
- swap pointers - current ⇄ next — no allocation, no history. If the - next token count doesn’t fit, the target buffer alone reallocates: - nextCapacityBytes = max(requiredBytes, capacity × 2, 64). -
-
- metrics observe frame k+1 - then the data may be overwritten — readers are only valid during - observeFrame. -
-
-
- -

- The buffer layout (frame-buffer.ts) is a leaner sibling of - the quick-sim EngineFrame: same sections, - no 64-byte header, one extra - transitionElapsedFrames section, and a token region with - spare capacity: -

- -
-
- place countsu32 × P -
-
- place offsetsu32 × P -
-
- transition elapsedf64 × T -
-
- elapsed framesf64 × T -
-
- firing countsu32 × T -
-
- fired flagsu8 × T -
-
- token structsbytes, used ≤ capacity -
-
- sparecapacity -
-
-

- All views - (Uint32Array/Float64Array/Uint8Array) - are created once per buffer over a single ArrayBuffer; IDs - resolve to dense indices through the shared EngineFrameLayout - (layout.ts). -

- -

Metrics pipeline — computed where the data is

- -
-
- Specs - MonteCarloMetricSpec: expression (metric - code body) · placeTokenCountMean · - transitionFiringCount — serializable, sent in - init. -
- → -
- Compile before worker start - Expression metrics carry HIR artifacts; built-in specs create a - measure(run) function directly. -
- → -
- Sample per frame - observeFrame(ctx) visits every run’s current frame as a - SimulationFrameReader (forEachRunFrame); - sampleRuns filters active/completed/all. -
- → -
- Aggregate - Across runs: mean·sum·min·max·last → scalar, or keep the - run axis and bin it → distribution (exact or bin - width). Optionally aggregate over time. -
- → -
- Metric frames - MonteCarloUserDefinedMetricFrame — scalar - (value/frameValue/timeValue) or distribution (bins: [value, frequency][]) per frame. Small JSON. -
-
- -

Protocol (monte-carlo/worker/messages.ts)

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
DirectionTypePayload
Host → Workerinit - sdcpn, extensions?, initialMarking, parameterValues, seed, dt, - maxTime, runCount, batchSize?, metricSpecs? -
start—
cancel— (checked between batches)
Worker → Hostready—
progress - MonteCarloWorkerProgress = advance counters (advancedRuns, completedRuns, erroredRuns, activeRuns, - allFinished) + frameNumber, time, runCount -
metricFramesPending MonteCarloUserDefinedMetricFrame[] batch
complete / cancelledFinal (or last-known) progress
errormessage, itemId
- -
- Contrast with quick sim: no ack/backpressure — the worker - free-runs to completion in small batches; the only upstream control is - cancel. And the payloads are aggregates, not frames: the UI - never holds Monte Carlo simulation state. -
- -

What the host receives

- - -
- Refactoring seams. - (1) Runs are round-robin on one worker thread; the run-state model was - designed so runs can shard across multiple workers later. (2) User code - still receives decoded TokenRecord objects per firing — the - README’s planned “IR compilation” would let lambdas/kernels operate - directly on the numeric buffers. (3) Buffer growth is a naive ×2 doubling; - arc-weight static analysis could size buffers up front and eliminate - reallocations. -
- - diff --git a/libs/@hashintel/petrinaut-core/docs/architecture/worker.html b/libs/@hashintel/petrinaut-core/docs/architecture/worker.html deleted file mode 100644 index 0a898518bc3..00000000000 --- a/libs/@hashintel/petrinaut-core/docs/architecture/worker.html +++ /dev/null @@ -1,752 +0,0 @@ - - - - - - Simulation Architecture — Worker & Protocol - - - - - -

Worker & Protocol

-

- worker/ (protocol + worker entrypoint) and - runtime/ - (main-thread lifecycle, transport, frame retention). Everything between - the host application and the engine. -

- -
-
Protocol
-
- worker/messages.ts — discriminated unions over - postMessage -
-
Transport
-
- runtime/transport.ts — wraps a Worker factory; - queues messages until the worker boots -
-
Host handle
-
- createSimulation() → Simulation (stores: - status, frames; events; - run/pause/reset/ack/getFrame/dispose) -
-
Flow control
-
- Ack-based backpressure — the consumer’s pace bounds worker - memory/compute -
-
- -

Message protocol

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- Host → Worker (ToWorkerMessage) -
TypePayloadEffect in the worker
init - sdcpn, extensions?, initialMarking, parameterValues, seed, dt, - maxTime (number | null), maxFramesAhead?, batchSize? - - Runs buildSimulation() (compiles user code, packs frame - 0), replies frame(0) then ready. Resets - lastAckedFrame = -1. -
start— - Starts the async compute loop (no-op if running; refuses from - complete/error). -
pause—Stops the loop, keeps all state, replies paused.
stop—Discards the SimulationInstance entirely.
setBackpressuremaxFramesAhead?, batchSize?Live-updates loop tuning.
ackframeNumber - lastAckedFrame = max(lastAckedFrame, n) — unblocks - computation. -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- Worker → Host (ToMainMessage) -
TypePayloadEffect on the main thread
readyinitialFrameCount - Resolves the createSimulation promise; status → - Ready. -
frame / frames - SimulationFramePayload = - { time, frame: ArrayBuffer } (single / batch) - - Appended to the frame store; frames store publishes - {count, latest: reader}. -
pausedframeNumberStatus → Paused.
completereason: "deadlock" | "maxTime", frameNumberStatus → Complete; emitted on the event stream.
errormessage, itemId (offending SDCPN item if known) - Status → Error; rejects init if still initializing. -
- -

Sequence — one quick simulation

- -
- - - - - - - - - - - Host (playback driver) - - - - createSimulation (main) - - - - simulation.worker - - - - - - - - - - createSimulation(config) - - - - init {sdcpn, initialMarking, seed, dt…} - - - buildSimulation() - - - compile + frame 0 - - - - - frame {t:0, ArrayBuffer²} - - ready {initialFrameCount:1} - - Promise resolves · status "Ready" - - - - simulation.run() - - start - - - - - loop blocked: lastAckedFrame = -1 - - - → nothing computes until first ack - - - - - - - loop — while frames remain to compute - - - - - - ack(frameNumber) — playback mode decides when - - - - - - computeNextFrame() - - - × batchSize while - - - n - acked < ahead - - - - - frames [{time, ArrayBuffer²}, …] - - frames store → {count, latest reader} - - - - complete {reason: deadlock | maxTime} - - status "Complete" + event emitted - -
- ² = the frame ArrayBuffer is - structured-clone copied - across the boundary (no transfer list); the worker keeps its own copy in - SimulationInstance.frames[]. -
-
- -

Backpressure — the ack contract

- - 
// worker compute loop (simplified from simulation.worker.ts)
-while (isRunning) {
-  if (lastAckedFrame < 0 || currentFrame - lastAckedFrame >= maxFramesAhead) {
-    await delay(10); continue;          // wait for the consumer
-  }
-  batch = compute up to batchSize frames (stop on complete/error)
-  post "frame" | "frames"
-  await delay(0);                        // let pause/stop/ack messages in
-}
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Play mode (core playback module)maxFramesAheadbatchSizeAck behaviour
viewOnlyn/a - No core backpressure profile — the React playback provider pauses - the worker and never acks, so nothing new is computed. -
computeBuffer4010 - Acks when playback is within ~0.5 s of the last computed frame — - keeps a small rolling buffer ahead of the playhead. -
computeMax10000500Acks every arrival → compute as fast as possible.
Worker defaults10001000Used when init omits the settings.
- -

Lifecycle

- -
- - - - - - - - - Initializing - - Ready - - Running - - Paused - - Complete - - Error - - - ready msg - - run() - - pause() - - run() - - deadlock / maxTime - - error msg - - - reset(): stop + clear store → Ready (from any state) - - -
- Main-thread SimulationState. The worker mirrors a smaller - internal status (ready / running / complete / error); - complete and error are terminal for the worker — - reset() sends stop and requires a fresh - init to run again. -
-
- -

- Main-thread runtime responsibilities (createSimulation) -

- - -
- Refactoring seams. - (1) Frames could be posted with a transfer list to halve copies — - but not as-is: computeNextFrame() re-reads the latest stored - frame to compute the next one, so transferring (detaching) the posted - buffer first requires stepping from a retained working copy (e.g. - latest-only retention or a double buffer). (2) reset() keeps - the main-thread status Ready but the worker has discarded its - simulation — a subsequent run() sends start to a - worker with nothing to run; a full re-init is what actually happens in the - UI flow. (3) The 10 ms poll while waiting for acks is a busy-wait; a - promise-per-ack would be exact. -
- - diff --git a/libs/@local/petrinaut-arch-docs/content/components/byte-map.tsx b/libs/@local/petrinaut-arch-docs/content/components/byte-map.tsx new file mode 100644 index 00000000000..b8541da8f06 --- /dev/null +++ b/libs/@local/petrinaut-arch-docs/content/components/byte-map.tsx @@ -0,0 +1,88 @@ +/** + * A byte-addressed memory map: offset gutter on the left, one row per section. + * + * This is the diagram the hand-written HTML pages did best — reading the frame + * format as a table loses the sense of a single contiguous buffer, which is the + * whole point of the format. Offsets are supplied rather than computed, because + * the map illustrates one worked example rather than a live layout. + */ + +import "./diagram.css"; +import { Inline } from "./inline"; + +export interface SegmentSpec { + /** Byte offset where this section starts. */ + offset: number; + name: string; + /** Element type and count, e.g. `u32 × P`. */ + type?: string; + /** The typed-array view used to read it. */ + view?: string; + accent?: "core" | "worker" | "none"; +} + +export interface ByteMapProps { + title?: string; + segments: SegmentSpec[]; + /** Total length, rendered as the closing offset. */ + byteLength?: number; + caption?: string; +} + +const accentVariable = (accent: SegmentSpec["accent"]): string | undefined => + accent === undefined || accent === "none" + ? undefined + : `var(--pnd-${accent})`; + +export const ByteMap = ({ + title, + segments, + byteLength, + caption, +}: ByteMapProps) => ( +
+ {title === undefined ? null : ( +
{title}
+ )} +
+ {segments.map((segment) => ( + + ))} + {byteLength === undefined ? null : ( + <> +
{byteLength}
+
+ + )} +
+ {caption === undefined ? null : ( +
+ +
+ )} +
+); + +const Segment = ({ segment }: { segment: SegmentSpec }) => ( + <> +
{segment.offset}
+
+ + + + {segment.type === undefined ? null : ( + + + + )} + {segment.view === undefined ? null : ( + + + + )} +
+ +); diff --git a/libs/@local/petrinaut-arch-docs/content/components/diagram.css b/libs/@local/petrinaut-arch-docs/content/components/diagram.css new file mode 100644 index 00000000000..2f2eabf60f0 --- /dev/null +++ b/libs/@local/petrinaut-arch-docs/content/components/diagram.css @@ -0,0 +1,257 @@ +/* + * Shared styling for the architecture diagram components. + * + * Deliberately self-contained: plain CSS with no preprocessor, no design-system + * import, and no framework assumptions, so the same file works in the Starlight + * site and in any host that renders the bundle. Colours derive from the host's + * text colour where possible, and everything falls back sensibly if the host + * defines none of the custom properties. + */ + +.pnd { + --pnd-line: color-mix(in srgb, currentColor 25%, transparent); + --pnd-muted: color-mix(in srgb, currentColor 65%, transparent); + --pnd-surface: color-mix(in srgb, currentColor 5%, transparent); + + --pnd-core: #3676b8; + --pnd-react: #7051b5; + --pnd-ui: #3d8055; + --pnd-worker: #b5762f; + + margin: 1.5rem 0; + font-size: 0.85rem; + line-height: 1.45; +} + +.pnd-title { + margin-bottom: 0.5rem; + font-size: 0.8rem; + font-weight: 600; + color: var(--pnd-muted); +} + +.pnd-lanes { + display: grid; + gap: 0.75rem; + grid-template-columns: repeat(var(--pnd-columns, 1), minmax(0, 1fr)); +} + +.pnd-lane { + display: flex; + flex-direction: column; + gap: 0.5rem; + padding: 0.75rem; + border: 1px solid var(--pnd-line); + border-radius: 0.5rem; + background: var(--pnd-surface); +} + +.pnd-lane-title { + font-size: 0.75rem; + font-weight: 600; + letter-spacing: 0.02em; + text-transform: uppercase; + color: var(--pnd-muted); +} + +.pnd-box { + padding: 0.5rem 0.6rem; + border: 1px solid var(--pnd-line); + border-left: 3px solid var(--pnd-accent, var(--pnd-line)); + border-radius: 0.35rem; + background: var(--pnd-surface); +} + +.pnd-box-label { + font-weight: 600; +} + +.pnd-box-note { + margin-top: 0.15rem; + color: var(--pnd-muted); +} + +.pnd-box code, +.pnd-lane code, +.pnd-step code { + font-size: 0.9em; +} + +/* Pipeline ------------------------------------------------------------- */ + +.pnd-pipeline { + display: flex; + flex-wrap: wrap; + align-items: stretch; + gap: 0.4rem; +} + +/* + * One flex item per step, arrow included, so a wrap never separates an arrow + * from the box it points at. + */ +.pnd-step-group { + display: flex; + align-items: stretch; + gap: 0.4rem; + flex: 1 1 8rem; + min-width: 8rem; +} + +.pnd-step { + flex: 1 1 auto; + padding: 0.5rem 0.6rem; + border: 1px solid var(--pnd-line); + border-radius: 0.35rem; + background: var(--pnd-surface); +} + +.pnd-step-index { + font-size: 0.7rem; + font-weight: 700; + color: var(--pnd-muted); +} + +.pnd-arrow { + align-self: center; + color: var(--pnd-muted); +} + +/* Visible to assistive technology, not on screen. */ +.pnd-sr-only { + position: absolute; + width: 1px; + height: 1px; + padding: 0; + margin: -1px; + overflow: hidden; + clip-path: inset(50%); + white-space: nowrap; + border: 0; +} + +/* Byte map ------------------------------------------------------------- */ + +.pnd-bytes { + display: grid; + grid-template-columns: auto 1fr; + gap: 0 0.6rem; + font-family: ui-monospace, SFMono-Regular, Menlo, monospace; +} + +.pnd-offset { + padding-top: 0.45rem; + text-align: right; + color: var(--pnd-muted); + font-size: 0.75rem; + white-space: nowrap; +} + +.pnd-seg { + display: flex; + flex-wrap: wrap; + gap: 0.25rem 0.6rem; + align-items: baseline; + padding: 0.4rem 0.6rem; + margin-bottom: 0.25rem; + border: 1px solid var(--pnd-line); + border-left: 3px solid var(--pnd-accent, var(--pnd-line)); + border-radius: 0.3rem; + background: var(--pnd-surface); +} + +.pnd-seg-name { + font-weight: 600; +} + +.pnd-seg-type, +.pnd-seg-view { + color: var(--pnd-muted); + font-size: 0.78rem; +} + +.pnd-seg-view { + margin-left: auto; +} + +/* Sequence ------------------------------------------------------------- */ + +.pnd-sequence { + display: grid; + gap: 0.35rem; +} + +.pnd-msg { + display: grid; + grid-template-columns: minmax(0, 1fr) auto minmax(0, 1fr); + gap: 0.5rem; + align-items: center; +} + +.pnd-msg-label { + padding: 0.35rem 0.55rem; + border: 1px solid var(--pnd-line); + border-radius: 0.3rem; + background: var(--pnd-surface); +} + +.pnd-msg-dir { + color: var(--pnd-muted); + font-size: 1rem; +} + +.pnd-msg-note { + color: var(--pnd-muted); +} + +.pnd-msg-spacer { + min-width: 0; +} + +.pnd-actors { + display: grid; + grid-template-columns: minmax(0, 1fr) auto minmax(0, 1fr); + gap: 0.5rem; + margin-bottom: 0.5rem; + font-size: 0.75rem; + font-weight: 600; + letter-spacing: 0.02em; + text-transform: uppercase; + color: var(--pnd-muted); +} + +.pnd-actors span:last-child { + text-align: right; +} + +/* Legend --------------------------------------------------------------- */ + +.pnd-legend { + display: flex; + flex-wrap: wrap; + gap: 0.75rem; + margin-top: 0.6rem; + font-size: 0.75rem; + color: var(--pnd-muted); +} + +.pnd-legend span::before { + content: ""; + display: inline-block; + width: 0.6rem; + height: 0.6rem; + margin-right: 0.3rem; + border-radius: 0.15rem; + background: var(--pnd-accent, currentColor); +} + +@media (max-width: 640px) { + .pnd-lanes { + grid-template-columns: minmax(0, 1fr); + } + + .pnd-msg, + .pnd-actors { + grid-template-columns: minmax(0, 1fr); + } +} diff --git a/libs/@local/petrinaut-arch-docs/content/components/inline.tsx b/libs/@local/petrinaut-arch-docs/content/components/inline.tsx new file mode 100644 index 00000000000..0777ef723e4 --- /dev/null +++ b/libs/@local/petrinaut-arch-docs/content/components/inline.tsx @@ -0,0 +1,27 @@ +/** + * Renders a plain string, turning `backticked` spans into ``. + * + * Diagram components take strings rather than nodes on purpose. JSX written + * inside MDX is compiled by the host's MDX renderer — Astro's, here — and + * handing that to a React component fails at render time with "Objects are not + * valid as a React child". Strings avoid the interop entirely, which also means + * a host embedding the bundle needs no special MDX/JSX configuration. + * + * Backticks are the one piece of formatting these diagrams actually need, so + * they are supported directly rather than by accepting arbitrary markup. + */ + +import type { ReactElement } from "react"; + +export const Inline = ({ text }: { text: string }): ReactElement => ( + <> + {text.split("`").map((part, index) => + // Odd indices are the spans that sat between a pair of backticks. + index % 2 === 1 ? ( + {part} + ) : ( + {part} + ), + )} + +); diff --git a/libs/@local/petrinaut-arch-docs/content/components/lanes.tsx b/libs/@local/petrinaut-arch-docs/content/components/lanes.tsx new file mode 100644 index 00000000000..c447ddb95b0 --- /dev/null +++ b/libs/@local/petrinaut-arch-docs/content/components/lanes.tsx @@ -0,0 +1,92 @@ +/** + * Lane-and-box diagrams: parallel columns of boxes, one column per thread or + * mode. + * + * Data-driven rather than hand-marked-up, so a page describes the content and + * the component owns the presentation. + */ + +import "./diagram.css"; +import { Inline } from "./inline"; + +/** Accent colours, keyed to the layer families the diagrams talk about. */ +export type Accent = "core" | "react" | "ui" | "worker" | "none"; + +const accentVariable = (accent: Accent | undefined): string | undefined => + accent === undefined || accent === "none" + ? undefined + : `var(--pnd-${accent})`; + +export interface BoxSpec { + label: string; + note?: string; + accent?: Accent; +} + +export interface LaneSpec { + title: string; + accent?: Accent; + boxes: BoxSpec[]; +} + +export const Box = ({ label, note, accent }: BoxSpec) => ( +
+
+ +
+ {note === undefined ? null : ( +
+ +
+ )} +
+); + +export interface LanesProps { + title?: string; + lanes: LaneSpec[]; + /** Colour key rendered beneath the lanes. */ + legend?: { label: string; accent: Accent }[]; +} + +export const Lanes = ({ title, lanes, legend }: LanesProps) => ( +
+ {title === undefined ? null : ( +
{title}
+ )} +
+ {lanes.map((lane) => ( +
+
{lane.title}
+ {lane.boxes.map((box, index) => ( + + ))} +
+ ))} +
+ {legend === undefined ? null : ( +
+ {legend.map((entry) => ( + + {entry.label} + + ))} +
+ )} +
+); diff --git a/libs/@local/petrinaut-arch-docs/content/components/pipeline.tsx b/libs/@local/petrinaut-arch-docs/content/components/pipeline.tsx new file mode 100644 index 00000000000..575999809cf --- /dev/null +++ b/libs/@local/petrinaut-arch-docs/content/components/pipeline.tsx @@ -0,0 +1,67 @@ +/** + * A numbered left-to-right chain of steps, wrapping on narrow screens. + * + * Used where the old HTML pages drew an arrow chain: the compilation pipeline, + * the fixed order of a simulation step, the metrics pipeline. + */ + +import "./diagram.css"; +import { Inline } from "./inline"; + +export interface StepSpec { + label: string; + note?: string; +} + +export interface PipelineProps { + title?: string; + steps: StepSpec[]; + /** Numbers the steps. Off for pipelines where order is not a sequence. */ + numbered?: boolean; +} + +export const Pipeline = ({ title, steps, numbered = false }: PipelineProps) => ( +
+ {title === undefined ? null : ( +
{title}
+ )} +
+ {steps.map((step, index) => ( + // A fragment per step so the separator sits between, not inside, boxes. + + ))} +
+
+); + +const Step = ({ + index, + numbered, + step, +}: { + index: number; + numbered: boolean; + step: StepSpec; +}) => ( + // The arrow is grouped with the step that follows it rather than emitted as a + // sibling. As separate flex items they wrapped independently, which left an + // arrow stranded at the end of a row pointing at a box on the next one. +
+ {index === 0 ? null : ( + + )} +
+ {numbered ?
{index + 1}
: null} +
+ +
+ {step.note === undefined ? null : ( +
+ +
+ )} +
+
+); diff --git a/libs/@local/petrinaut-arch-docs/content/components/sequence.tsx b/libs/@local/petrinaut-arch-docs/content/components/sequence.tsx new file mode 100644 index 00000000000..6d761804877 --- /dev/null +++ b/libs/@local/petrinaut-arch-docs/content/components/sequence.tsx @@ -0,0 +1,94 @@ +/** + * A two-actor message sequence: who sends what, in order, with side notes. + * + * Kept to two actors deliberately. The hand-written HTML had three columns + * (host, main-thread runtime, worker), which read as three lifelines but made + * every message ambiguous about which hop it described. Two actors and an + * explicit direction says the same thing without the ambiguity. + */ + +import "./diagram.css"; +import { Inline } from "./inline"; + +export interface MessageSpec { + /** `right` is left-actor → right-actor. */ + direction: "right" | "left"; + label: string; + note?: string; +} + +/** A phase separator, rendered as a full-width note rather than a message. */ +export interface PhaseSpec { + phase: string; +} + +export type SequenceRow = MessageSpec | PhaseSpec; + +export interface SequenceProps { + title?: string; + actors: [string, string]; + rows: SequenceRow[]; +} + +const isPhase = (row: SequenceRow): row is PhaseSpec => "phase" in row; + +export const Sequence = ({ title, actors, rows }: SequenceProps) => ( +
+ {title === undefined ? null : ( +
{title}
+ )} +
+ {actors[0]} +
+
+ {rows.map((row, index) => + isPhase(row) ? ( +
+ +
+ ) : ( + /* + * The direction arrow is decorative, so a screen reader would + * otherwise get the label with no indication of who sent it. The + * visually-hidden sentence carries that, and is the only place + * direction is stated in text. + */ +
+ + {row.direction === "right" + ? `${actors[0]} to ${actors[1]}: ` + : `${actors[1]} to ${actors[0]}: `} + + {row.direction === "right" ? ( + <> +
+ +
+ +
+ {row.note === undefined ? null : } +
+ + ) : ( + <> +
+ {row.note === undefined ? null : } +
+ +
+ +
+ + )} +
+ ), + )} +
+
+); diff --git a/libs/@local/petrinaut-arch-docs/content/index.mdx b/libs/@local/petrinaut-arch-docs/content/index.mdx new file mode 100644 index 00000000000..490a2666763 --- /dev/null +++ b/libs/@local/petrinaut-arch-docs/content/index.mdx @@ -0,0 +1,78 @@ +--- +title: Petrinaut architecture +description: How the Petrinaut packages fit together, and where to start reading. +sidebar_order: 0 +--- + +Petrinaut is a visual editor and simulation runtime for Stochastic Dynamic +Coloured Petri Nets (SDCPN). This documentation is in two halves that are +maintained differently, and it is worth knowing which you are reading. + +## The diagrams + +Every box is a layer and every arrow is real imports, counted from the +dependency graph rather than drawn by hand. There are three kinds, each showing +a different slice: + +- The [Architecture](architecture) page opens with the **top-level layers** — + the whole system at a glance, with everything below the first level folded + into its parent. +- Every layer page opens with its **neighbourhood**: what that layer depends on, + and what depends on it. This is usually the one you want. +- A layer that has sub-layers also gets a **drill-down** of its direct children. + +Where a layer has more neighbours than fit legibly, the remainder is drawn as a +single "further layers" node rather than omitted, and the full list is always in +the tables on the page. + +**Generated pages** — everything under [Architecture](architecture) — are +extracted from annotations in the source on every build, and CI fails if they +drift. Layer sizes and dependency counts all come from the code. If a generated page disagrees with the code, that is a bug in the +annotations, not in the page. + +**Authored pages** — this one, and its siblings — carry the reasoning that no +import graph can supply: why a boundary sits where it does, what the alternatives +were, which mistakes are easy to make. They are hand-written and can go stale, so +they avoid restating facts the generated pages already own. + +## Where to start + +- [Two execution paths](two-execution-paths) — the single most important thing to + understand before changing anything in the simulation layers. +- [Architecture](architecture) — the generated map: layers, dependencies, + boundaries. +- [Working on the architecture](working-on-the-architecture) — how to declare a + layer, and what CI will hold you to. + +## Simulation internals + +Detail that the generated pages do not carry — binary formats, message +protocols, and the costs behind each design choice. + +- [Memory model](doc:simulation/memory-model) — which buffer lives on which thread, + and the frame format. +- [How a step is computed](doc:simulation/stepping) — the fixed stage order, and + determinism. +- [Compiling user code](doc:simulation/user-code) — the HIR pipeline, scenarios, and + what the sandbox does not protect against. +- [Worker protocol and backpressure](doc:simulation/protocol) — messages, and + the ack contract. +- [Monte Carlo experiments](doc:simulation/experiments) — bounded memory, and why + only aggregates reach the UI. + +## The packages at a glance + +`@hashintel/petrinaut-core` is the headless half: the document model, the HIR +compiler for user-authored code, the simulation runtimes, and the language +server. It has no React, no DOM and no Monaco, which is what lets it run +unchanged in Node and inside workers. + +`@hashintel/petrinaut` is the editor built on top. It splits into a React layer +that owns state and a UI layer that renders it — and the dependency only goes one +way, which is enforced rather than merely intended. + +Three further packages are not yet covered by the generated model: +`@hashintel/petrinaut-cli`, `@apps/petrinaut-website`, and `apps/petrinaut-opt` +(the Python Optuna service). +Adding them is a matter of declaring layers in their source; see +[Working on the architecture](working-on-the-architecture). diff --git a/libs/@local/petrinaut-arch-docs/content/simulation/experiments.mdx b/libs/@local/petrinaut-arch-docs/content/simulation/experiments.mdx new file mode 100644 index 00000000000..deb62aa9622 --- /dev/null +++ b/libs/@local/petrinaut-arch-docs/content/simulation/experiments.mdx @@ -0,0 +1,126 @@ +--- +title: Monte Carlo experiments +description: How many runs execute in bounded memory, and why only aggregates reach the UI. +sidebar_order: 10 +attachTo: core.simulation.monte-carlo +--- + +import { Lanes } from "@diagrams/lanes"; + +An experiment runs many independent simulations of the same net with bounded +memory, aggregates metrics inside the worker, and ships only small JSON metric +frames to the host. **Frame buffers never cross the thread boundary.** + +The engine itself is shared with the interactive path — the same enablement, +lambda, kernel and dynamics code — adapted to different frame storage. + +## The layers + +The host holds one experiment handle per experiment, subscribed to its status, +progress and metric stores. The handle runs either in worker mode or in local +mode, where the simulator runs on the calling thread; local mode is what tests +and embedders use. + +Inside the worker, the loop is: advance every run one frame, post progress and +any pending metric frames, yield, repeat. Cancellation is checked between +batches, and the default batch is 4. + +The simulator owns N runs and advances them **round-robin**, one frame each per +call. That is deliberate: it stops long runs from starving short ones, so +progress reporting stays meaningful and a single pathological run cannot stall +the batch. + +## Per run + +Each run carries its own seed, parameter values and initial marking — defaults +come from the experiment config and are overridable per run, typically as +`base seed + index`. + +Stepping errors are **per run**: once the experiment is running, one run throwing +does not stop the others, which is what makes a large sweep survive a parameter +combination that fails mid-flight. A run completes on its own deadlock or on the +shared max time. + +Construction is not isolated. Every run state is built up front in the simulator +constructor, so an invalid marking or parameter set in any single run throws +before the experiment starts and none of the runs execute. + +Each run also reports buffer telemetry — bytes used, capacity, and reallocation +count — which is the first thing to look at when an experiment is slower than +expected. + +## Two buffers, swapped every step + + + +The consequence to internalise: **frame readers handed to metric code are only +valid during observation.** They are views over a buffer that the next step will +overwrite. Anything a metric needs later must be extracted into the value it +returns. + +If the next token count does not fit, only the target buffer reallocates, taking +the larger of what is required and double the current capacity. + +The Monte Carlo buffer layout is a leaner sibling of the interactive frame: the +same sections, **no 64-byte header**, one extra section for elapsed frames, and a +token region with spare capacity beyond what is used. Views are created once per +buffer rather than per read. + +## Metrics are computed where the data is + +```text +specs → compile before start → sample per frame → aggregate → metric frames +``` + +Metric specs are serializable and sent at init: either an expression (carrying +HIR artifacts) or a built-in such as mean place-token count or transition firing +count. + +Sampling visits every run's current frame through the same reader interface the +interactive path uses. Aggregation happens across runs — mean, sum, min, max, +last for a scalar — or keeps the run axis and bins it into a distribution. + +What reaches the host is therefore small JSON: a scalar frame carries a value +per frame and per time, and a distribution frame carries bins as value/frequency +pairs. A host can paint a bins-by-frames heatmap, or derive medians and +percentiles from the bins, **without ever holding simulation state**. + +## Contrast with the interactive path + +No acks, no backpressure: the worker free-runs to completion in small batches, +and the only upstream control is cancel. That is affordable precisely because the +payloads are aggregates rather than frames — there is no consumer pace to respect +because there is no unbounded data to hold back. + +## Known costs + +Runs are round-robin on one worker thread, though the run-state model was +designed so runs can shard across workers later. Buffer growth is naive doubling; +static analysis of arc weights could size buffers up front and remove +reallocations entirely. diff --git a/libs/@local/petrinaut-arch-docs/content/simulation/memory-model.mdx b/libs/@local/petrinaut-arch-docs/content/simulation/memory-model.mdx new file mode 100644 index 00000000000..c236ed3e511 --- /dev/null +++ b/libs/@local/petrinaut-arch-docs/content/simulation/memory-model.mdx @@ -0,0 +1,191 @@ +--- +title: Memory model +description: Where simulation state actually lives, and the binary frame format it lives in. +sidebar_order: 10 +attachTo: core.simulation +--- + +import { ByteMap } from "@diagrams/byte-map"; +import { Lanes } from "@diagrams/lanes"; + +Everything the simulation computes is stored in raw `ArrayBuffer`s read through +typed-array views. There is no object graph of tokens: tokens are packed structs, +decoded to JavaScript objects only at the read boundary. + +Knowing which buffer lives on which thread is the thing that makes the rest of +the simulation layers make sense. + +## Where each piece lives + + + +| Memory | Structure | Thread | Lifetime | +| ------------------- | -------------------------------------------------------------------- | ------------- | ------------------------------------------------------------------------------------------ | +| SDCPN document | Plain JS objects (editor state) | Main | Structured-cloned into the worker at init. Later edits do not affect a running simulation. | +| Initial marking | JSON — count or `TokenRecord[]` per place | Main | Packed into frame 0 by `buildSimulation`. | +| `EngineFrame` | `ArrayBuffer` — 64-byte header plus typed sections and packed tokens | Worker | Immutable snapshot per step, appended to the instance's frame history. | +| Frame payload | `{ time, frame: ArrayBuffer }` | Worker → Main | Structured-clone copy per frame. Retained by the frame store for scrubbing. | +| Frame reader | Typed-array views over the stored buffer | Main | Zero-copy. Materialises `TokenRecord`s only when asked. | +| Monte Carlo buffers | 2 × `ArrayBuffer` per run | MC worker | Swapped each step. Never cross to the main thread. | +| Compiled user code | Versioned HIR buffer programs | Workers | Produced once at init. | + +The asymmetry between rows 4 and 6 is the whole difference between the two +execution paths — see [Two execution paths](doc:two-execution-paths). + +## The frame format + +One `ArrayBuffer` per frame, read through section-typed views. The frame stores +**no IDs and no time**: decoding requires the layout derived from the SDCPN +(place and transition order, per-place stride and slot layout), and time travels +alongside as payload metadata. + +`createEngineFrame()` computes every offset and is the only frame constructor; +`readEngineFrame()` recreates the same views for reading. + + + +Header fields worth knowing: `magic` (`0x5046524d`, a corrupt-frame guard), +`version` (currently `2`, asserted on decode), `placeCount`/`transitionCount` +(checked against the layout on read), and `byteLength` (checked on read). + +### Token packing + +Each coloured place owns a contiguous run of `count × strideBytes` starting at +its byte offset, giving O(1) access to any place. Uncoloured places have stride +0 and only a count. + +| Element type | Physical | Notes | +| ------------ | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `real` | `f64` — 8 B | | +| `integer` | `f64` — 8 B, rounded | Exact within ±2^53. `i32` was rejected for silent wraparound; `i64` because bigint is contagious into user code. | +| `boolean` | `u8` — 1 B | Not bit-packed: that would break byte-granular copies for marginal savings. | +| `uuid` | `u64 × 2` — 16 B | Two little-endian lanes via a shared `BigUint64Array`, combined to one `bigint` at the boundary. Never routed through `number`. | +| `string` | `u64` pool reference — 8 B | The frame stores an ID into an append-only per-run intern pool. The pool lives on the instance, not the frame, so frames stay fixed-stride and byte-copyable. | + +Fields are ordered per colour by decreasing alignment, and the stride is rounded +up to 8 bytes. One module owns all token-byte indexing; every whole-token move is +a byte-range copy. + +Because the string pool never crosses the worker boundary with the frames, each +frame payload ships an append-only delta of new strings that the main-thread +store accumulates and hands to the reader for decoding. + +## Known cost: triple retention + +Each quick-simulation frame currently exists three times — in the worker's frame +history, as a structured-clone copy in the main-thread store, and transiently in +the `postMessage` queue. Frames are never transferred, and the worker's history +is only ever read at its latest entry. + +Retention is deliberately isolated behind the frame store, so this can change +without touching consumers. Transferring buffers instead of copying is not a drop-in +fix: computing the next frame re-reads the latest stored frame, so detaching the +posted buffer first requires stepping from a retained working copy. diff --git a/libs/@local/petrinaut-arch-docs/content/simulation/protocol.mdx b/libs/@local/petrinaut-arch-docs/content/simulation/protocol.mdx new file mode 100644 index 00000000000..56a342e4ff7 --- /dev/null +++ b/libs/@local/petrinaut-arch-docs/content/simulation/protocol.mdx @@ -0,0 +1,168 @@ +--- +title: Worker protocol and backpressure +description: The messages between host and simulation worker, and the ack contract that bounds memory. +sidebar_order: 10 +attachTo: core.simulation.worker +--- + +import { Sequence } from "@diagrams/sequence"; + +Both simulation workers speak a small typed message protocol over `postMessage`, +wrapped in a transport that queues messages until the worker boots. This page +covers the quick-simulation worker; the experiment worker is on +[Monte Carlo](doc:simulation/experiments). + +## Messages + +**Host → worker** + +| Type | Payload | Effect | +| ----------------- | ---------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- | +| `init` | net, marking, parameters, seed, `dt`, `maxTime`, backpressure settings | Builds the simulation, packs frame 0, replies `frame(0)` then `ready`. Resets the ack cursor. | +| `start` | — | Starts the compute loop. No-op if running; refused after completion or error. | +| `pause` | — | Stops the loop, keeps all state, replies `paused`. | +| `stop` | — | Discards the simulation entirely. | +| `setBackpressure` | `maxFramesAhead`, `batchSize` | Live-updates loop tuning. | +| `ack` | `frameNumber` | Raises the ack cursor, unblocking computation. | + +**Worker → host** + +| Type | Payload | Effect | +| ------------------ | ----------------------------------------- | ---------------------------------------------------------------- | +| `ready` | initial frame count | Resolves the creation promise; status becomes Ready. | +| `frame` / `frames` | `{ time, frame }`, single or batch | Appended to the frame store, which republishes count and latest. | +| `paused` | frame number | Status becomes Paused. | +| `complete` | `"deadlock"` or `"maxTime"`, frame number | Status becomes Complete, emitted on the event stream. | +| `error` | message, offending item id if known | Status becomes Error; rejects creation if still initialising. | + +Errors carry the SDCPN item id where known, which is what lets the editor +highlight the transition whose lambda threw rather than showing a bare message. + +## One run, end to end + + + +## The ack contract + +**Nothing is computed until the consumer asks for it.** The worker's loop blocks +while the ack cursor is behind, so the consumer's pace bounds the worker's memory +and CPU: + +```js +while (isRunning) { + if (lastAckedFrame < 0 || currentFrame - lastAckedFrame >= maxFramesAhead) { + await delay(10); // wait for the consumer + continue; + } + // compute up to batchSize frames, stopping on complete or error + post("frames", batch); + await delay(0); // let pause / stop / ack messages in +} +``` + +Note the first condition: after `init`, the cursor starts below zero, so a fresh +worker computes **nothing** until the first ack. Starting a run is therefore two +steps, `start` then acks, and a host that starts but never acks looks hung while +behaving exactly as designed. + +Two details the sketch above smooths over, both worth knowing before relying on +the ordering: + +- The ahead limit is checked once per batch, not once per frame, so a batch can + run up to `batchSize - 1` frames past `maxFramesAhead`. It bounds memory in + aggregate rather than capping the cursor exactly. +- On the final iteration the worker posts `complete` before the batch holding the + last frames. A host that stores frames on `frames` and tears down on `complete` + will drop that final batch unless it drains after completing. + +The per-mode profiles the core exposes: + +| Play mode | `maxFramesAhead` | `batchSize` | Behaviour | +| --------------- | ---------------- | ----------- | ----------------------------------------------------------------------------------------- | +| view only | — | — | No profile: the React layer pauses the worker and never acks, so nothing new is computed. | +| compute buffer | 40 | 10 | Acks when playback is within ~0.5 s of the last computed frame. | +| compute max | 10000 | 500 | Acks on every arrival — compute as fast as possible. | +| worker defaults | 1000 | 1000 | Used when `init` omits the settings. | + +## Lifecycle + +```text +Initializing ──ready──▶ Ready ──run()──▶ Running ──pause()──▶ Paused + │ │ + └──── run() ◀────────┘ + Running ──deadlock | maxTime──▶ Complete + any ──error msg───────────▶ Error +``` + +That is the **main-thread** state. The worker mirrors a smaller internal status, +in which complete and error are terminal. + +A sharp edge: `reset()` returns the main-thread status to Ready, but the worker +has discarded its simulation, so a subsequent `run()` sends `start` to a worker +with nothing to run. What actually happens in the UI flow is a full re-init. +Treat reset as "throw this away", not "rewind". + +## Main-thread responsibilities + +The host-side runtime does four things worth knowing about: + +- **Snapshot preparation** — sanitizes the net for enabled extensions and + flattens component instances, so the main-thread layout matches what the worker + produces. A place-count mismatch throws on read rather than decoding garbage. +- **Transport** — queues outbound messages until the worker resolves; tests can + inject a pre-built transport instead of a real worker. +- **Retention** — the in-memory store keeps every frame so playback can scrub. + Alternative stores (latest-only, sliding window, persisted) can be swapped in + without changing consumers. +- **Publication** — status and frames as readable stores, plus an event stream. + +## Known costs + +The 10 ms poll while waiting for acks is a busy-wait; a promise per ack would be +exact. And frames could be posted with a transfer list to halve copying, but only +once stepping no longer re-reads the latest stored frame — see +[Memory model](doc:simulation/memory-model). diff --git a/libs/@local/petrinaut-arch-docs/content/simulation/stepping.mdx b/libs/@local/petrinaut-arch-docs/content/simulation/stepping.mdx new file mode 100644 index 00000000000..39debb4ecd7 --- /dev/null +++ b/libs/@local/petrinaut-arch-docs/content/simulation/stepping.mdx @@ -0,0 +1,76 @@ +--- +title: How a step is computed +description: The fixed order of one simulation step, and what makes a run reproducible. +sidebar_order: 10 +attachTo: core.simulation.engine +--- + +import { Pipeline } from "@diagrams/pipeline"; + +`buildSimulation(input)` turns an SDCPN into a runnable instance; +`computeNextFrame(simulation)` advances it by one frame. Each frame it returns is +immutable, and neither function emits events: the worker is what turns their +return values into protocol messages. + +Neither is pure. Stepping mutates the instance it is given, and the transition +path reuses kernel staging buffers between calls, so a `SimulationInstance` must +not be stepped concurrently or reused across runs. + +Building an instance flattens component instances, compiles user code, and packs +the initial marking into frame 0. Compilation is selective: lambdas only where +available, kernels only for coloured outputs, dynamics only for colours with at +least one real element. + +## One step, in order + +The order is fixed, and several surprising behaviours follow from it. + + + +Discrete derivatives are forced to zero in step 2; removals compact the buffer +and additions append in step 3. + +Dynamics run before transitions, so a transition sees token values already +advanced by this step's integration. Timers only advance on a step where nothing +fired, so they measure idle time rather than wall time. + +## Immutability, and what it costs + +Every stage that changes state allocates a new buffer: dynamics, token removal, +and token insertion each rebuild the frame. That makes history trivially correct +— previous frames are never mutated — at the cost of allocation churn per step. + +This is the single clearest optimisation opportunity in the engine. Monte Carlo already +demonstrates the alternative: two buffers swapped per step, no history. Bringing +that to the interactive path means moving history retention wholly onto the main +thread, which the frame store already isolates. + +## Determinism + +One seeded RNG is threaded through stochastic lambda sampling and distribution +draws, as a pure `nextRandom(state) → [value, nextState]` whose state lives on +the instance. + +The guarantee: the same `(SDCPN, marking, parameters, seed, dt)` always +reproduces the same frame sequence. This is what makes experiments comparable +across runs, and it is why the RNG state is threaded rather than global — a +global would couple runs sharing a worker. diff --git a/libs/@local/petrinaut-arch-docs/content/simulation/user-code.mdx b/libs/@local/petrinaut-arch-docs/content/simulation/user-code.mdx new file mode 100644 index 00000000000..284bea7fd1e --- /dev/null +++ b/libs/@local/petrinaut-arch-docs/content/simulation/user-code.mdx @@ -0,0 +1,106 @@ +--- +title: Compiling user code +description: How authored TypeScript becomes buffer programs, and what the sandbox does and does not protect. +sidebar_order: 10 +attachTo: core.simulation.authoring +--- + +import { Pipeline } from "@diagrams/pipeline"; + +Dynamics, lambdas, kernels and metrics all compile through the HIR: user +TypeScript is lowered to a source-spanned IR, checked against the net, and +emitted as buffer-native JavaScript with offsets and strides baked in. Scenario +code takes a separate, simpler path. + + + +The editor's type checking is the same pipeline seen from the other end: the LSP +generates the `Color_*`, `Parameters`, `Dynamics` and kernel declarations Monaco +checks against, so editor diagnostics and runtime artifacts come from the same +HIR checks and emitters. + +## What compiles where + +| Surface | Authored as | Executed in | +| ----------------- | -------------------------------------------- | -------------------------------------------------- | +| Transition lambda | module, `Lambda(fn)` | Worker, per enablement check | +| Transition kernel | module, `TransitionKernel(fn)` | Worker, per firing | +| Dynamics | module, `Dynamics(fn)` | Worker, per step per place | +| Metric | plain function body | Main thread (timeline) and MC worker (experiments) | +| Scenario | per-place expressions, or one code-mode body | Main thread, before the run starts | +| Place visualizer | module, `Visualization(fn)` returning JSX | Main thread, at render | + +The visualizer is the one inherently React-based surface — it authors JSX — which +is why it compiles in the UI package rather than in the headless core. + +Metrics must return a finite number; NaN and infinity throw rather than +propagating into a chart. + +## Scenarios produce a marking, not a run + +Scenario compilation is separate from the HIR pipeline and produces an initial +marking (plus parameter overrides) before the simulation is created. + +Two modes, with one asymmetry worth remembering: **per-place mode is keyed by +place ID, code mode is keyed by place name.** In code mode, unknown names are +silently dropped. + +In per-place mode an uncoloured place takes an expression evaluated with +parameters in scope, rounded and clamped at zero; a coloured place takes row +arrays in colour-element order, each coerced with typed defaults for missing +columns, and extra columns throw. + +## What the sandbox is for + +Scenario expressions run with a list of globals shadowed — `window`, `document`, +`globalThis`, `fetch`, `importScripts`, `Function`, timers and similar — so those +name lookups resolve to `undefined`. Because shadowing only stops identifier +lookup, the sandbox additionally blocks the constructor-chain escape +(`({}).constructor.constructor` reaching `Function`) for the duration of the +call, by swapping the `.constructor` descriptors on the built-in prototypes and +restoring them afterwards. + +**Treat this as robustness hardening, not isolation.** User code executes with +`new Function` in the same realm as the host. It protects against typos and +accidental API use, not against an attacker. + +## Distributions are sampled once + +Kernels can construct distribution values rather than numbers. They defer +sampling through the ABI sink, and the engine samples each value once per token +with the seeded RNG. That is what lets chained `.map(fn)` transforms share a +single draw and keeps runs reproducible. + +When stochasticity is disabled the runtime is not injected at all, and plain +values are required. + +## Compatibility + +HIR artifacts carry a version and a fingerprint of their sanitized compilation +input. Any schema, code, or extension change requires recompilation before the +engine will run them — a stale artifact is refused rather than silently +mismatched against a changed net. diff --git a/libs/@local/petrinaut-arch-docs/content/two-execution-paths.mdx b/libs/@local/petrinaut-arch-docs/content/two-execution-paths.mdx new file mode 100644 index 00000000000..455b491dd71 --- /dev/null +++ b/libs/@local/petrinaut-arch-docs/content/two-execution-paths.mdx @@ -0,0 +1,52 @@ +--- +title: Two execution paths +description: Why Quick Simulation and Experiments share an engine but not a memory model. +sidebar_order: 1 +--- + +There are two independent ways to execute a net, and almost every surprising +thing about the simulation layers follows from the difference between them. + +**Quick Simulation** runs one interactive simulation and keeps _every frame_, so +the user can scrub backwards and forwards through the run. Frames accumulate in +a store on the main thread. + +**Experiments** run many Monte Carlo simulations with _bounded memory_ — two +reusable buffers per run — and ship only metric aggregates back to the host. +Frame buffers never leave the worker. + +## Why not one path + +The obvious simplification is to make experiments reuse the interactive +simulator and throw away frames afterwards. It does not work: an experiment is +hundreds or thousands of runs, and retaining frames even transiently makes peak +memory scale with run length times concurrency. Bounding memory is not an +optimisation on the experiment path, it is the reason that path exists. + +The consequence to keep in mind: **anything you add to the frame format costs +memory on the interactive path and nothing on the experiment path, while +anything you add to metric aggregation costs on both.** A per-frame field is +cheap to add and expensive to keep; an aggregate is the reverse. + +## What this means when changing code + +The engine is shared, so a change to stepping or to the frame layout affects both +paths. The _controllers_ are not shared — the interactive path goes through the +simulation controller with its frame store, the experiment path through the Monte +Carlo runtime — so a change to run lifecycle usually needs doing twice, and +forgetting one is the classic bug here. + +If you are touching either, read the generated pages for +[the engine](layer:core.simulation.engine), +[the simulation controller](layer:core.simulation.runtime) and +[the Monte Carlo runtime](layer:core.simulation.monte-carlo) — their +declared invariants say which properties you must not break. + +## Actual mode is a third source, but not a third path + +Actual mode renders an execution that came from somewhere else entirely — a live +external system rather than a simulation. It does not run the engine at all. It +reaches the same canvas and timeline because both read frames through the +[execution-frame interface](layer:react.execution-frame), which abstracts +over "where frames come from". That indirection is why adding Actual mode did not require +touching the canvas. diff --git a/libs/@local/petrinaut-arch-docs/content/working-on-the-architecture.mdx b/libs/@local/petrinaut-arch-docs/content/working-on-the-architecture.mdx new file mode 100644 index 00000000000..ef295fa9507 --- /dev/null +++ b/libs/@local/petrinaut-arch-docs/content/working-on-the-architecture.mdx @@ -0,0 +1,120 @@ +--- +title: Working on the architecture +description: How to declare a layer, refine one, and what CI holds you to. +sidebar_order: 2 +--- + +The generated pages come from annotations in the Petrinaut source. This page is +the short version of how to change them; the full reference lives in +[`libs/@local/petrinaut-arch-docs/README.md`](https://github.com/hashintel/hash/blob/main/libs/@local/petrinaut-arch-docs/README.md). + +## Declaring a layer + +Add frontmatter to the folder's `README.md`: + +```yaml +--- +layer: core.simulation.monte-carlo +role: Runs many simulations with bounded frame memory +--- +``` + +The prose below the frontmatter becomes the layer's page body, so an existing +folder README needs only the frontmatter added. + +For a folder with a barrel entry file and no README, annotate the entry file +instead: + +```ts +/** + * @layerRoot core.simulation.monte-carlo + * @role Runs many simulations with bounded frame memory + */ +``` + +Use one mechanism or the other, never both on the same folder. `layer` and +`role` are the only frontmatter keys, and `@layerRoot` and `@role` the only +tags; anything else is either an error or ignored. + +## You do not need to annotate every file + +A file with no annotation belongs to the nearest ancestor folder that declares a +layer. Thirty-seven declarations cover four hundred and thirteen files. Declare a layer where +the architecture genuinely changes — a new boundary, a different responsibility — +not on every directory that exists. + +## Moving or renaming a folder + +Nothing to update, provided the folder keeps its declaration with it. That is the +main practical gain over the old path-matching script, where a move silently +re-bucketed files into a fallback group. + +Renaming a _layer id_ is a real change: it moves the layer's page and invalidates +inbound links, and every descendant id has to move with it. + +## Adding a new package + +Add it to `packages` in +[`architecture.config.ts`](https://github.com/hashintel/hash/blob/main/libs/@local/petrinaut-arch-docs/architecture.config.ts), +declare a root layer in its source, and rebuild. Build configuration outside the +package's source directory is deliberately excluded — it configures the build, it +is not part of the design. + +## What CI will hold you to + +Run the check before pushing: + +```bash +yarn workspace @local/petrinaut-arch-docs lint:arch-docs +``` + +It fails when a source file no layer covers appears, when a layer id implies an +ancestor nothing declares, when a dependency violates a declared rule, and when +the import graph reaches a source file no layer claims. + +The bundle itself is build output and is git-ignored, so there is nothing to +regenerate before pushing. Only the annotations are versioned. To read the docs +locally: + +```bash +yarn workspace @local/petrinaut-arch-docs doc:architecture +``` + +## Adding a hand-written guide + +Authored pages live in `libs/@local/petrinaut-arch-docs/content/` and are +optional. To have one sit _inside_ the architecture tree rather than beside it, +name the layer it explains: + +```yaml +--- +title: Memory model +attachTo: core.simulation +sidebar_order: 10 +--- +``` + +The page then appears beneath that layer, and the layer's page links to it under +"Guides". + +Because `attachTo` decides where the page ends up, link to other pages by name +rather than by path — `[text](layer:core.simulation.engine)` for a generated +page, `[text](doc:simulation/memory-model)` for another authored one. A target +that does not resolve fails the check. + +## Declaring a rule + +Rules live in `architecture.config.ts` and are checked against the real import +graph: + +```ts +{ + from: "react", + to: "ui", + reason: "state providers must not depend on the components that render them", +} +``` + +Add one only when the code already satisfies it — a rule is a statement of fact +that CI keeps true, not an aspiration. Check first with a build; if there are +existing violations, either fix them in the same change or leave the rule out.