Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .changeset/domain-formats.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
"@statelyai/graph": minor
---

Add explicit GraphSON 3 typed TinkerGraph, DGML, CX2 and RDF 1.1 dataset/syntax adapters with strict native profiles and source-preserving document converters. RDF syntax uses optional N3; pure JSON/dataset modules remain dependency-free.

Fix wrong-entity document edits across 13 legacy converters and new DGML. Ambiguous source identity changes and unsupported mappings now reject with diagnostics rather than targeting another entity or replaying stale source. Expand pinned independent native validators, package subpath contracts and measured coverage gates.
7 changes: 7 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,10 @@ tests/native/fixtures/graphviz/schema/LICENSE.txt -text -whitespace
# Decoded draw.io payload snapshots intentionally retain source whitespace and EOF.
tests/native/fixtures/drawio/page-first.mxgraph.xml -text -whitespace
tests/native/fixtures/drawio/page-second.mxgraph.xml -text -whitespace

# Official DGML schema retains upstream bytes.
tests/native/fixtures/dgml/*.xsd -text -whitespace

# Authored RDF/GraphSON fixture hashes require stable LF bytes.
tests/native/fixtures/rdf/scoped.trig -text
tests/native/fixtures/graphson/modern.json -text
2 changes: 2 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -113,5 +113,7 @@ jobs:
java-version: '21'
- name: Install Pinned PlantUML Oracle
run: pnpm setup:native:plantuml
- name: Install Pinned GraphSON Oracle
run: pnpm setup:native:graphson
- name: Validate Native Formats and Converter Coverage
run: pnpm test:coverage
16 changes: 14 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ A graph is just `{ nodes, edges }` data, and every operation is a standalone, tr
- **One model for real diagrams.** Directed and undirected edges (even mixed), nested nodes, named ports, and positions and sizes: what node editors, statecharts, and architecture diagrams need, and most graph libraries leave out.
- **Fast.** Fastest in most of our [cross-library benchmarks](./docs/benchmarks.md) against graphology, ngraph, graphlib, and cytoscape. For example, it builds a 100k-node graph 9–15× faster than graphology.
- **Algorithms you can trust.** Shortest paths, centrality, communities, flow, matching, isomorphism, and more. Every query and algorithm is tested against edge cases such as self-loops, parallel edges, and unknown ids, and algorithms are iterative, so deep graphs won't overflow the stack.
- **Works with your tools.** Convert to and from 14 formats, including Graphviz DOT, Mermaid, GraphML, D2, React Flow, and Cytoscape. Lay out with 8 engines, including ELK, dagre, and Graphviz. Turn XState machines into graphs. Each adapter is an optional subpath import.
- **Works with your tools.** Convert to and from graph formats including Graphviz DOT, Mermaid, GraphML, D2, React Flow, and Cytoscape. Lay out with 8 engines, including ELK, dagre, and Graphviz. Turn XState machines into graphs. Each adapter is an optional subpath import.
- **Typed end to end.** Generic data types for nodes, edges, ports, and the graph itself.

## Installation
Expand Down Expand Up @@ -67,6 +67,7 @@ supported mappings; they do not establish conformance to an entire format.
| --- | --- | --- | --- |
| Adjacency list, Cytoscape, D3, DOT, ELK, GEXF, GML, GraphML, JGF, TGF, xyflow | Partial | Partial | None |
| CSV/TSV, Graphology export JSON, Graphlib JSON, NetworkX node-link JSON, adjacency matrix, Matrix Market, Graphviz JSON, draw.io, Pajek, scoped PlantUML | Partial | Partial | None |
| GraphSON3 typed TinkerGraph objects, DGML, CX2, RDF1.1 dataset/syntax profiles | Partial | Partial | Partial scoped sources |
| Edge list | Partial | Full for endpoint tuples | None |
| D2 | Partial | Partial | Partial comments |
| Mermaid architecture, block, C4, class, ER, flowchart, GitGraph, ishikawa, mindmap, requirement, Sankey, sequence, state, UseCase | Partial | Partial | Partial annotations |
Expand All @@ -80,7 +81,9 @@ without making them native features. Use a GraphDocument when those details matt
Additional adapters: [adjacency matrix](src/formats/adjacency-matrix/README.md),
[Matrix Market](src/formats/matrix-market/README.md),
[Graphviz JSON](src/formats/graphviz-json/README.md), [draw.io](src/formats/drawio/README.md),
[Pajek](src/formats/pajek/README.md), [PlantUML](src/formats/plantuml/README.md).
[Pajek](src/formats/pajek/README.md), [PlantUML](src/formats/plantuml/README.md),
[GraphSON](src/formats/graphson/README.md), [DGML](src/formats/dgml/README.md),
[CX2](src/formats/cx2/README.md), [RDF](src/formats/rdf/README.md).

```ts
import { getFormatSupportEntry } from '@statelyai/graph/format-support';
Expand Down Expand Up @@ -121,13 +124,22 @@ const exactSource = graphMLDocumentConverter.to(persisted);

CSV/TSV use explicit node and edge tables with a serializable schema; default optional cells encode JSON, including visible labels. Graphology, Graphlib and NetworkX subpaths convert their native JSON profiles without runtime library dependencies. Mapping limits: [CSV/TSV](src/formats/csv/README.md), [Graphology](src/formats/graphology/README.md), [Graphlib](src/formats/graphlib/README.md), [NetworkX](src/formats/networkx/README.md).

GraphSON uses an explicit typed TinkerGraph object profile; DGML and CX2 preserve
scoped native declarations. RDF maps subject/object terms to nodes and statements
to directed edges, retaining predicate and graph terms. The dependency-free
`/rdf` dataset API is separate from optional N3-backed `/rdf/syntax`. These
domain mappings reject incompatible Graph state rather than guessing.

Every native converter subpath offers a `*DocumentConverter` registration. The
`@statelyai/graph/document` factory also accepts explicit custom codecs, including
VisualGraph codecs. Documents retain exact original text/native JSON values on
no-op export, including comments, extensions and unsupported constructs, after
JSON persistence. Projection can be partial or unavailable and carries explicit
diagnostics. This guarantees source retention, not full semantic interpretation.

Ambiguous source identity edits reject when generated IDs cannot establish correspondence;
matching private lineage does not override live native identity.

Supported Graph edits patch native source and must pass reparse proof. Unsupported
or conflicting edits throw instead of replaying stale source. Native source edits
remain authoritative when Graph is unchanged. Regeneration is an explicit,
Expand Down
34 changes: 33 additions & 1 deletion docs/native-validation.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@ Global floors remain 95% lines, 91% statements, 93% functions and 84% branches.
[The committed configuration](../vitest.config.ts) also sets line/branch floors
for selected D2, Mermaid, XML, ELK and primary CSV/JSON adapters, plus adjacency
matrix/core, Matrix Market, Graphviz JSON, draw.io, Pajek, PlantUML and all six
expanded Mermaid profiles with their shared parser helpers. Coverage output is
expanded Mermaid profiles with their shared parser helpers, plus scoped GraphSON,
DGML, CX2 and RDF core/syntax modules. Coverage output is
ignored. Native semantic assertions complement coverage; neither proves full
language coverage.

Expand All @@ -40,6 +41,9 @@ language coverage.
| NetworkX (Python) | 3.7 | Native node-link JSON and GML readers/writers |
| SciPy / NumPy (Python) | 1.18.1 / 2.5.4 | Matrix Market numeric/math interoperability |
| igraph (Python) | 1.0.0 | Pajek reader vertices/sections/attributes |
| Apache TinkerPop Java / Gremlin JavaScript | 3.8.2 | GraphSON3 typed TinkerGraph object/element/property semantics |
| NDEx ndex2 (Python) | 3.12.0 | CX2 native declarations/defaults/topology |
| RDFLib (Python) / N3 (JavaScript) | 7.6.0 / 2.13.12 | RDF1.1 selected syntax/dataset semantics |
| PlantUML | 1.2026.8 | Compiler-produced SVG and class XMI, scoped component/class dialect |

Python oracles use an isolated task environment, with exact versions in
Expand Down Expand Up @@ -140,3 +144,31 @@ label emission/projection/patches reject them, while astral characters and safe
aliases remain supported. D2's compiler retains NUL label fields but replaces lone
surrogates; its adapters reject only that proven surrogate loss. These tests
inspect native compiled/display fields rather than asserting file-byte encoding.

<!-- domain profile tooling/proof from scripts/native/graphson-tools.json, tests/native/{graphson,dgml,cx2,rdf}.test.ts -->

GraphSON native setup uses `pnpm setup:native:graphson`: twelve Maven Central
JARs pinned individually by SHA256, derived from the verified official Apache
TinkerPop 3.8.2 distribution. The manifest records URLs, versions, distribution
provenance and licensing. Tools stay in ignored `.tools-native/` and never enter
runtime imports or package contents. Java subprocesses have a 20-second bound;
`GRAPH_NATIVE_JAVA` selects the task Java 21 executable. Actual Java mapper tests
use `TinkerIoRegistryV3` plus extended value types, preserve unsafe magnitudes in
raw serialized strings, and assert observed native topology/property semantics.
No GraphSON adjacency/writeGraph interoperability is claimed.

DGML uses the official Microsoft 2009 XSD with xmllint, including negative schema
controls for dynamic property attributes; this does not establish Windows renderer
behavior. CX2 uses the maintained NDEx Python reader with observed default/null
limitations. RDF uses N3 strict grammars and independent RDFLib parser/store
observations and dataset isomorphism; empty-graph and language-case distinctions
are checked before the native store normalizes them. Per-format READMEs and
fixture provenance describe each bounded interpretation claim. Exact document
source retention is separate from these native semantic assertions.

<!-- combined domain verification evidence from plans/011-domain-formats-verification.md -->

Final combined evidence belongs in the
[domain formats verification report](../plans/011-domain-formats-verification.md).
Isolated measurements do not establish a combined release gate; existing global
coverage thresholds remain unchanged.
17 changes: 15 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,11 @@
"./adjacency-matrix": "./dist/formats/adjacency-matrix/index.mjs",
"./converter": "./dist/formats/converter/index.mjs",
"./csv": "./dist/formats/csv/index.mjs",
"./cx2": "./dist/formats/cx2/index.mjs",
"./cytoscape": "./dist/formats/cytoscape/index.mjs",
"./d2": "./dist/formats/d2/index.mjs",
"./d3": "./dist/formats/d3/index.mjs",
"./dgml": "./dist/formats/dgml/index.mjs",
"./document": "./dist/formats/document/index.mjs",
"./dot": "./dist/formats/dot/index.mjs",
"./drawio": "./dist/formats/drawio/index.mjs",
Expand All @@ -44,13 +46,16 @@
"./graphlib": "./dist/formats/graphlib/index.mjs",
"./graphml": "./dist/formats/graphml/index.mjs",
"./graphology": "./dist/formats/graphology/index.mjs",
"./graphson": "./dist/formats/graphson/index.mjs",
"./graphviz-json": "./dist/formats/graphviz-json/index.mjs",
"./jgf": "./dist/formats/jgf/index.mjs",
"./matrix-market": "./dist/formats/matrix-market/index.mjs",
"./mermaid": "./dist/formats/mermaid/index.mjs",
"./networkx": "./dist/formats/networkx/index.mjs",
"./pajek": "./dist/formats/pajek/index.mjs",
"./plantuml": "./dist/formats/plantuml/index.mjs",
"./rdf": "./dist/formats/rdf/index.mjs",
"./rdf/syntax": "./dist/formats/rdf/syntax.mjs",
"./tgf": "./dist/formats/tgf/index.mjs",
"./xyflow": "./dist/formats/xyflow/index.mjs",
"./kernel": "./dist/kernel.mjs",
Expand Down Expand Up @@ -97,7 +102,8 @@
"bench:report": "tsx bench/compare/report.ts",
"test:native": "vitest run --config vitest.native.config.ts",
"test:coverage": "vitest run --config vitest.coverage.config.ts --coverage",
"setup:native:plantuml": "node scripts/native/setup-plantuml.mjs"
"setup:native:plantuml": "node scripts/native/setup-plantuml.mjs",
"setup:native:graphson": "node scripts/native/setup-graphson.mjs"
},
"devDependencies": {
"@changesets/changelog-github": "^0.5.2",
Expand All @@ -111,6 +117,7 @@
"@types/d3-force": "^3.0.10",
"@types/d3-hierarchy": "^3.1.7",
"@types/jsdom": "30.0.0",
"@types/n3": "1.26.4",
"@types/node": "^25.0.3",
"@vitest/coverage-v8": "^4.0.18",
"@xyflow/system": "^0.0.75",
Expand All @@ -132,8 +139,10 @@
"graphology-metrics": "^2.4.0",
"graphology-shortest-path": "^2.1.0",
"graphology-types": "^0.24.8",
"gremlin": "3.8.2",
"jsdom": "30.1.2",
"mermaid": "12.1.0",
"n3": "2.13.12",
"ngraph.graph": "^20.1.2",
"ngraph.path": "^1.6.1",
"publint": "^0.3.15",
Expand All @@ -160,7 +169,8 @@
"graphology-layout-forceatlas2": "^0.10.0",
"webcola": "^3.4.0",
"xstate": "^5.19.0",
"zod": "^4.0.0"
"zod": "^4.0.0",
"n3": "^2.13.12"
},
"peerDependenciesMeta": {
"@dagrejs/dagre": {
Expand Down Expand Up @@ -204,6 +214,9 @@
},
"fflate": {
"optional": true
},
"n3": {
"optional": true
}
}
}
26 changes: 26 additions & 0 deletions plans/006-domain-exchange-formats.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Plan 006: Add DGML and CX2 semantic profiles

Planned at `8d1b139`, 2026-10-12. Priority P2; effort L; risk MED; direction. Depends on DONE plans 001–004. Worktree `/Users/davidkpiano/.codex/worktrees/graph/domain-formats`, branch `feature/domain-formats`, fetched main base. Never modify shared checkout.

## Why and current state

The merged verification report identifies DGML and CX2 as missing domain formats. Graph uses plain JSON nodes/edges; converters are prefixed standalone functions. `src/formats/graphlib/index.ts` exemplifies `createFormatConverter`, native carrier storage, and `createGraphDocumentConverter({format,codec,project,patch})`. The document API keeps original source independent of its partial projection, reparses edits, and rejects unsupported changes. `src/formats/structured-json.ts` and `preservation.ts` contain reusable validation/identity helpers. Follow these contracts; do not change Graph or document core.

## Scope and ownership

Executor owns `src/formats/dgml/**`, `src/formats/cx2/**`, `tests/dgml*.test.ts`, `tests/cx2*.test.ts`, matching `tests/native/*` tests/fixtures/helpers, and format READMEs. Another executor owns shared barrels, package/build/coverage/CI/contracts and root docs. Coordinate required dependencies with it. Do not edit plans or other converters; never revert others' edits. No commits until reviewer approval.

## Steps

1. Drift check: `git diff --stat 8d1b139..HEAD -- src/formats/document src/formats/preservation.ts src/formats/structured-json.ts` must be empty. Read local AGENTS and exemplars. Ask integration owner to install exact lockfile dependencies first.
2. Research authoritative DGML Microsoft reference and NDEx CX2 specification/current maintained reader. Pin source/version provenance. Define explicit profiles in README before coding. DGML maps Id/Label and directed links, category/property definitions, Contains membership, groups; reject multiple/cyclic containment rather than guessing. Native DGML implicitly creates undeclared link endpoint nodes: support with declaration provenance, exact no-op source, bounded explicit declaration edits only with reparse proof. CX2 maps aspect-oriented numeric IDs/topology and declared typed attributes/defaults/aliases; retain opaque visual/custom aspects. Native identities must remain distinct and exact; numeric unsafe IDs reject. Duplicate entities and CX2 dangling endpoints reject. Self-loops/parallel links supported when native permits.
3. Implement import/export with plain serializable typed native data; native carrier retains unknown fields/aspects. Live mapped fields authoritative over stale carrier. Do not silently change declared data types. Fresh Graph export either represent fields explicitly or reject unsupported semantics. Do not advertise full interpretation.
4. Add document registrations: exact source no-op after JSON persistence, unsupported source retains source with null projection; bounded label/property edits only when semantic proof available, otherwise strict rejection. Preserve comments/unknown XML or aspect records on edits. No broad regex that confuses nested elements/namespaces. Use shared lexical XML helpers where appropriate.
5. Regression tests: IDs, aliases/defaults/types, group/category distinction, missing vs null vs empty, unknown fields, opaque constructs, parallel/self-loop topology, invalid/cyclic native inputs, stale metadata precedence, exact source round-trip, supported edits and rejection/conflict controls, 10k topology performance guard. Native evidence must use official schema or maintained independent reader; self-roundtrip is insufficient. Record any unavailable oracle honestly, no fake native label.
6. Match readme skill `/Users/davidkpiano/.agents/skills/readme/SKILL.md`. Run focused tests and types; report shared wiring to integration owner.

## Commands and done criteria

`pnpm install --frozen-lockfile` once by integration owner. `pnpm exec vitest run tests/dgml*.test.ts tests/cx2*.test.ts`; `pnpm typecheck`; `pnpm typecheck:repo`; native tests via actual Node24 `pnpm dlx node@24.15.0 node_modules/vitest/vitest.mjs run --config vitest.native.config.ts` with focused files. Reviewer runs `pnpm verify` and full combined coverage. All exit0; meaningful independent semantic assertions and preservation tests; only owned files changed. Never claim Windows Visual Studio rendering without proof.

STOP and report if native specification contradicts a proposed mapping, existing API must break, domain behavior remains ambiguous, or dependencies require changing core runtime. Reviewer maintains index and approves publication. Future versions must preserve explicit profiles and evidence limitations.
Loading
Loading