Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 4 additions & 18 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,21 +28,7 @@ jobs:
# registry comparison runs in publish.yml, where npm's state is the point.
- run: bun run check:release -- --pack

# There was a `downstream` job here that checked out refract-org/refract-ui and
# ran its test suite on every pull request and push. It could not detect a
# regression in this repository, because refract-ui does not consume this one:
# its package.json declares `dependencies: {}`, nothing under its src/ imports
# an @refract-org package, and its AGENTS.md records the arrangement on purpose
# — "Types mirror @refract-org/evidence-graph schemas (defined locally in
# src/types.ts)". So the job installed a second repository and ran its single
# test against its own local types, with this PR's diff nowhere in the picture.
#
# Measured on run 34146471011: gate 14s, downstream 8s, and because downstream
# declared `needs: gate` it started only after gate finished — 29s of wall clock
# for 17s of checking.
#
# A real downstream check is still worth having, and would have to compare
# refract-ui's mirrored types against this repo's evidence-graph schemas, since
# drift between them is the actual failure mode the mirror creates. Running
# refract-ui's own suite cannot see that drift: those tests assert refract-ui
# against the very copy that drifted.
# The viewer (apps/ui) and the docs site (apps/docs) have their own path-filtered
# workflows, viewer.yml and docs-site.yml, so neither can hold up this gate. The
# viewer's runs whenever evidence-graph or the captured examples change: it
# type-checks against the engine's EventType and imports the engine's output.
43 changes: 43 additions & 0 deletions .github/workflows/docs-site.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
name: Docs site

# apps/docs, the public documentation site. Separate from the CI gate so a site
# change cannot block an engine release, and the engine gate does not wait on
# the site build.
on:
pull_request:
branches: [main]
paths:
- apps/docs/**
- package.json
- bun.lock
- .github/workflows/docs-site.yml
push:
branches: [main]
paths:
- apps/docs/**
- package.json
- bun.lock
- .github/workflows/docs-site.yml

permissions:
contents: read

jobs:
docs:
runs-on: ubuntu-latest
defaults:
run:
working-directory: apps/docs
steps:
- uses: actions/checkout@v7
- uses: oven-sh/setup-bun@v2
with:
bun-version: "1.3.14"
- uses: actions/setup-node@v6
with:
node-version: 22
- run: bun install --frozen-lockfile
working-directory: .
- run: bun run lint
- run: bun run test
- run: bun run build
48 changes: 48 additions & 0 deletions .github/workflows/viewer.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
name: Viewer

# apps/ui, the browser-only file viewer. It imports the engine's event types
# from packages/evidence-graph (types only, nothing bundled) and its contract
# test reads the engine output captured under docs/examples, so a change to
# either runs this too: that is where the viewer and the engine can drift.
on:
pull_request:
branches: [main]
paths:
- apps/ui/**
- packages/evidence-graph/**
- docs/examples/**
- package.json
- bun.lock
- .github/workflows/viewer.yml
push:
branches: [main]
paths:
- apps/ui/**
- packages/evidence-graph/**
- docs/examples/**
- package.json
- bun.lock
- .github/workflows/viewer.yml

permissions:
contents: read

jobs:
viewer:
runs-on: ubuntu-latest
defaults:
run:
working-directory: apps/ui
steps:
- uses: actions/checkout@v7
- uses: oven-sh/setup-bun@v2
with:
bun-version: "1.3.14"
- run: bun install --frozen-lockfile
working-directory: .
# The viewer type-checks against evidence-graph's emitted declarations.
- run: bun x tsc -b packages/evidence-graph
working-directory: .
- run: bun run lint
- run: bun run test
- run: bun run build
4 changes: 4 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,10 @@ bun run lint # biome lint packages/
- **packages/cli**: CLI tool (`refract` command)
- **packages/persistence**: SQLite storage (bun:sqlite)
- **packages/eval**: Evaluation harness
- **apps/docs**: Public documentation site (CC-BY-4.0). Private workspace; `bun run lint|test|build` from its directory; CI is `docs-site.yml`.
- **apps/ui**: Browser-only file viewer. Private workspace that imports evidence-graph's event types as types only; CI is `viewer.yml`, which also runs on evidence-graph and `docs/examples` changes.

The root `bun run test` and `bun run lint` cover `packages/` only; each app has its own `AGENTS.md`.

## Semantic Enrichment (v0.5.0+)

Expand Down
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,13 @@
- `refract review`: source watchlists, saved fetched/reviewed revision cursors, resumable history fetching, and explicit acknowledgment of an inspected report.
- Markdown and JSON review reports with source identity, changed text, revision links, mechanical filters, saved inputs, and coverage. `review --sample` demonstrates the complete workflow offline.
- Optional revision parent and source-text availability metadata. Coverage reports missing inputs and avoids text comparisons across known gaps.
- **The docs site and the viewer are checked in CI.** Since #37 imported them into `apps/docs` and `apps/ui`, nothing ran their lint, tests or builds: their workflows sat under `apps/*/.github`, which GitHub does not read, and the root suite excludes `apps/`. `docs-site.yml` and `viewer.yml` now run each app's checks, path-filtered and separate from the engine gate. The viewer's also runs when `packages/evidence-graph` or `docs/examples` change.

### Changed
- **The apps use the monorepo's tooling.** Their own lockfiles (two `bun.lock`, one `package-lock.json`), `.husky` hooks, `mise.toml` and `prepare: husky` scripts are removed; the root lockfile, hook and toolchain cover them, and each app's `.lintstagedrc.json` still applies to its files. Their `package.json` repository and issue links point at this repository.

### Fixed
- **The viewer's event types had drifted from the engine's.** `apps/ui` kept its own copy, which still listed `claim_first_seen`, `claim_softened` and the other `claim_*` names the engine renamed `sentence_*`, and lacked `edit_cluster_detected` and `talk_activity_spike`; timeline colours were keyed to the old names, so removed and modified sentences got none. It now imports `EventType` and `EvidenceEvent` from `@refract-org/evidence-graph` as types only — nothing from the engine is bundled — and its list of event types is a `Record<EventType, true>`, so the build fails when the engine adds or renames a type. Softened and strengthened wording is coloured from the event's `directionSignal`. A contract test imports, lists and re-exports the engine output in `docs/examples/` and an event of every type.
- Latest passage disappearance now reports absence from the latest fetched revision. Partial matches expose inspectable candidate text and uncertainty; unavailable latest text returns `unknown` through the CLI and MCP.
- Bounded analysis no longer mixes unrelated cached revisions into the selected range. Interactive analysis serves its captured inputs; the explorer accepts explicit bounds and shows coverage.
- **`refract verify --html` marked every event's proof "Merkle Root Match"**, including in a bundle that had failed verification. Each row now shows the verifier's result for that event's hash and for its proof, the page lists the checks that failed, and it states what `verify` checks and what it does not. The receipt and the command's output no longer say "cryptographically verified" or "integrity verified": nothing in a bundle is signed. Revision IDs read from the bundle are HTML-escaped like the other fields.
Expand Down
20 changes: 10 additions & 10 deletions COMPATIBILITY.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Cross-Repo Compatibility Matrix
# Compatibility Matrix

When updating `@refract-org/*` packages, downstream repos must be checked for compatibility. This document defines the dependency chain and what breaks where.
When updating `@refract-org/*` packages, downstream consumers — the apps in this repository and the projects outside it — must be checked for compatibility. This document defines the dependency chain and what breaks where.

## Dependency Order

Expand All @@ -25,7 +25,7 @@ as this tree. `publish.yml` publishes in the order the script prints.

| Package | In this tree | On npm (2026-09-29) | Downstream consumers |
|---|---|---|---|
| `@refract-org/evidence-graph` | 0.5.3 | 0.5.2 | every package; NextConsensus; refract-ui (mirrored types) |
| `@refract-org/evidence-graph` | 0.5.3 | 0.5.2 | every package; NextConsensus; `apps/ui` (types only, from this tree) |
| `@refract-org/ingestion` | 0.3.3 | 0.3.3 | mcp, cli; NextConsensus |
| `@refract-org/analyzers` | 0.5.2 | 0.5.2 | mcp, cli; NextConsensus |
| `@refract-org/mcp` | 0.1.0 | 0.1.0 | cli |
Expand All @@ -44,29 +44,29 @@ that only analyzers 0.5.0 has.
|---|---|---|---|
| NextConsensus (`nextconsensus`, `nextconsensus-app`) | `@refract-org/analyzers@^0.5.0`, `@refract-org/evidence-graph@^0.5.0`, `@refract-org/ingestion@^0.3.1`, each through one adapter file per repo | 0.5.0 / 0.5.0 / 0.3.1 | any library release; see [docs/refract-and-nextconsensus.md](./docs/refract-and-nextconsensus.md) |
| `demo-data/` (monorepo) | JSONL format matching evidence-graph types | — | EvidenceEvent schema change |
| refract-ui | Types mirrored locally from evidence-graph | — | EventType or EvidenceEvent schema change |
| `apps/ui` (monorepo) | `EventType` and `EvidenceEvent` imported from `packages/evidence-graph` as types only | this tree | checked on every evidence-graph change by `viewer.yml` |
| refract-py | Wraps `@refract-org/cli` via subprocess | — | CLI flag additions/removals |
| refract-docs | References CLI examples and schema | — | CLI flag changes or schema version bumps |
| `apps/docs` (monorepo) | References CLI examples and schema | — | CLI flag changes or schema version bumps |

## What Breaks When

### evidence-graph EventType changes

**If you add a new event type:**
- refract-ui: add to local type definitions in `src/types.ts`
- refract-docs: update schema.md and events.md
- `apps/ui`: the build fails until the type is added to `EVENT_TYPE_RECORD` in `src/types.ts`
- `apps/docs`: update schema.md and events.md
- refract-py: no change (dataclasses are generic)
- `demo-data/`: update JSONL files if they should include the new type

**If you rename an event type:**
- BREAKING for all consumers. Create a migration guide.
- refract-ui type mirrors must be updated
- refract-docs schema reference must be updated
- `apps/ui`: the build fails until `EVENT_TYPE_RECORD` and any CSS keyed to the old name are updated
- `apps/docs` schema reference must be updated

### CLI flag changes

- refract-py `_run()` method calls CLI with flags — any flag rename breaks the SDK
- refract-docs CLI reference page must be updated
- `apps/docs` CLI reference page must be updated

### AnalyzerConfig changes

Expand Down
27 changes: 18 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,12 @@ refract/
│ ├── mcp/ # Model Context Protocol server implementation
│ ├── persistence/ # SQLite local caching and revision store
│ └── eval/ # Evaluation harness & benchmark verification
└── docs/ # Architecture specs, recipes & boundary definitions
├── apps/
│ ├── docs/ # Public documentation site (CC-BY-4.0)
│ └── ui/ # Browser-only file viewer
├── docs/ # Architecture specs, recipes & boundary definitions
│ └── examples/ # Captured revision inputs and engine output
└── demo-data/ # Local datasets
```

## Development and testing
Expand All @@ -180,6 +185,8 @@ bun run test
bun run lint
```

The apps are workspaces with their own checks, run from their directories: `bun run lint`, `bun run test` and `bun run build` in `apps/docs` or `apps/ui`. CI runs them in `docs-site.yml` and `viewer.yml`, separately from the engine gate.

## Packages

| Package | Role |
Expand Down Expand Up @@ -216,18 +223,20 @@ integration without AGPL obligations. See [nextconsensus.com](https://nextconsen

## Ecosystem

These repos extend the core engine:
These build on the core engine:

| Repo | Purpose |
|------|---------|
| [refract-docs](https://github.com/refract-org/refract-docs) | Public documentation site — [integrations](https://refract-org.github.io/refract-docs/integrations/), quickstart, CLI, SDK, tutorials |
| [refract-ui](https://github.com/refract-org/refract-ui) | Standalone visualization — load JSONL, render timelines, diffs, citations |
| Project | Purpose |
|---------|---------|
| [Docs site](./apps/docs) | Public documentation site — [integrations](https://refract-org.github.io/refract-docs/integrations/), quickstart, CLI, SDK, tutorials |
| [Viewer](./apps/ui) | Browser-only file viewer — load JSONL, render timelines, diffs, citations |
| [Demo data](./demo-data) | Local datasets; the [offline walkthrough](./docs/recipes.md#try-the-offline-walkthrough) uses a fictional library history |
| [refract-py](https://github.com/refract-org/refract-py) | Python adapter — calls the CLI and returns dataclasses or DataFrames |

The core owns event contracts, CLI behavior, and reproducible examples. The other
repos publish guides or consume those interfaces. See the [repository map and
consolidation proposal](./docs/repositories.md) for ownership and migration steps.
The core owns event contracts, CLI behavior, and reproducible examples. The docs
site and the viewer live under `apps/` with their own licenses; the viewer imports
the engine's event types and is checked against its captured output. The Python
adapter is still a separate repository. See the [repository map](./docs/repositories.md)
for ownership and the remaining migration steps.

### Position in the revisable-delegation loop

Expand Down
1 change: 0 additions & 1 deletion apps/docs/.github/CODEOWNERS

This file was deleted.

16 changes: 0 additions & 16 deletions apps/docs/.github/dependabot.yml

This file was deleted.

21 changes: 0 additions & 21 deletions apps/docs/.github/workflows/ci.yml

This file was deleted.

33 changes: 0 additions & 33 deletions apps/docs/.github/workflows/deploy.yml

This file was deleted.

1 change: 0 additions & 1 deletion apps/docs/.husky/pre-commit

This file was deleted.

24 changes: 12 additions & 12 deletions apps/docs/AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,31 +1,31 @@
# Agent instructions for refract-docs

Refract-docs is the public documentation site for the Refract project — an open-source deterministic observation engine for public revision histories.
Refract-docs is the public documentation site for the Refract project — an open-source deterministic observation engine for public revision histories. Its source is `apps/docs` in the Refract monorepo.

## Quick Start

```bash
npm install # Install dependencies
npm run build # Build static HTML — docs/ → dist/
npm run lint # Biome check
npm run test # Vitest (verifies docs compile)
bun install # at the repository root; installs every workspace
bun run build # Build static HTML — docs/ → dist/
bun run lint # Biome check
bun run test # Vitest (verifies docs compile)
```

## Tooling

- **Package manager:** npm
- **Package manager:** Bun, through the repository root's workspace and lockfile
- **Linter/Formatter:** Biome
- **Tests:** Vitest
- **Pre-commit:** Husky runs Biome on staged files
- **CI:** ci.yml (main) + deploy.yml (GitHub Pages)
- **Pre-commit:** the repository root's Husky hook runs lint-staged, which uses this directory's `.lintstagedrc.json`
- **CI:** `.github/workflows/docs-site.yml` at the repository root
- **Commit style:** Conventional Commits

## Verification

```bash
npm run lint # Biome check
npm run test # Build verification
npm run build # Production build
bun run lint # Biome check
bun run test # Build verification
bun run build # Production build
```

## Repository structure
Expand Down Expand Up @@ -69,5 +69,5 @@ docs/

```bash
node build.mjs # builds static HTML from docs/ → dist/
npm test # runs build (verifies docs compile without errors)
bun run test # runs build (verifies docs compile without errors)
```
Loading
Loading