NCB, the National Capability Benchmark

NCB is a prototype benchmark of a country's capacity to anticipate change, coordinate action, learn, adapt and build under uncertainty.

The benchmark covers the countries in the registry and nine dimensions. Each dimension is scored from 0 to 100 with equal weights. There is no composite score or headline ranking. Every raw indicator remains visible with its source and year. Confidence is published beside the score and is never folded into it.

Brazil is the primary case. The same dimensions, indicators, transforms and normalization frame apply to every country in the set. Read docs/WHY.md for the claim under test and what would disprove it.

Quick start

pnpm install
pnpm bench all
pnpm dev

pnpm bench all fetches World Bank observations from 1990, scores the countries, runs diagnostics, writes the report, and regenerates the capability agendas. To refresh the source-backed Trust and Coordination adapter rows as well, run pnpm bench evs fetch and pnpm bench vdem fetch before the scoring step.

The viewer runs at https://ncb.localhost through portless on port 3888. To run without the proxy, use either:

PORTLESS=0 pnpm dev
pnpm --filter @ncb/web dev:app

Both commands serve http://localhost:3888.

Run the offline Delphi pipeline with:

pnpm bench delphi --mock
pnpm bench score
pnpm bench report

Mock output is deterministic and is not evidence about a country.

The green gate is:

pnpm build
pnpm typecheck

Both commands run the release-notes gate. Run it directly with pnpm changelog:check when changing either release version or CHANGELOG.md.

Run pnpm bench validate after changing data/delphi or data/evidence.

Commands

The CLI lives in packages/core and is available through pnpm bench.

Command	Purpose
pnpm bench ingest [--from 1990] [--snapshot]	Fetch World Bank history into data/observations and record restatements in revisions.json.
pnpm bench evs fetch	Fetch and parse the pinned Joint EVS/WVS results tables (A165 trust, A173 perceived control, A080_01 charitable membership, and the E069_17 court confidence check) into the shared observation store. trust fetch is an alias.
pnpm bench vdem fetch	Fetch and parse the pinned V-Dem v15 civil-society series into the shared observation store.
pnpm bench score	Normalize and score the current observations, writing the published output.
pnpm bench diagnose	Run correlations, redundancy checks and the GDP-sensitivity test.
pnpm bench report	Write data/out/report.md.
pnpm bench agenda [BRA ...] [--lang pt-BR]	Write one capability agenda per requested country and language.
pnpm bench delphi [--mock]	Run the model panel or the deterministic offline stand-in.
pnpm bench cost	Measure panel prompts and estimate the cost before a real run.
pnpm bench prompt ...	Print exact panel prompts or write chat-ready bundles.
pnpm bench merge ...	Merge pasted chat replies into a Delphi run file.
pnpm bench probe --search	Find candidate World Bank series by name and database.
pnpm bench probe --series a,b[@db]	Test candidate series across the country set before wiring them.
pnpm bench research inventory	Build the deterministic country × gap research queue and corpus balance report.
pnpm bench research scout	Ask the AI research scout for bounded, unpublished leads; add --mock or --prompt-only for offline work.
pnpm bench research critique --in FILE	Red-team a scout run; it cannot approve publication.
pnpm bench validate [--fetch]	Schema-check Delphi, evidence and institutional data. --fetch checks evidence URLs too.
pnpm test	Run the core pipeline and adapter regression tests.
pnpm bench all	Run ingest, score, diagnose, report and agenda in order.
pnpm changelog:check	Verify the newest App and Dataset release notes match their source versions.

The nine dimensions

The benchmark measures:

1. Anticipation
2. Agency
3. Coordination
4. Trust
5. Learning
6. Experimentation
7. Adaptability
8. Building
9. Shared Purpose

The indicator registry in packages/core/src/model/indicators.ts is the single source of truth. It contains scored indicators, declared gaps and retired indicators. Gaps remain visible, lower confidence and form the data-collection agenda. Retired indicators stay visible with the reason they were rejected.

Missing values are dropped from the mean. Nothing is imputed. A dimension with fewer than two observed indicators publishes no score: score is null and belowCoverageFloor is true.

How scoring works

For each scored indicator, the pipeline:

1. takes the most recent comparable value for each country, with source and year;
2. applies its declared transform, such as per-million, logarithmic or none;
3. winsorizes extreme values with Tukey fences at three interquartile ranges;
4. normalizes to 0 through 100 against the current country set, reversing lower-is-better indicators;
5. averages the available indicators inside each dimension with equal weights;
6. computes confidence = coverage × recency × source_quality separately.

Every country in the registry sets the normalization frame. The frame stays fixed within a dataset version. Adding a country changes the frame, restates published scores and requires a major dataset version bump. The dataset and app release versions are both defined in packages/core/src/model/version.ts.

Ingestion keeps the full World Bank history from 1990 while scoring uses the latest value. Every ingest compares itself with the previous observation file and appends changes to data/observations/revisions.json. Snapshots are written only when --snapshot is passed.

Trends and agendas

The default momentum spans are ten and twenty years. A trend uses only indicators observed at both ends of the span, so its basket can be smaller than the current dimension score. Momentum entries carry the span, change, basket size and any historical clamp count. Indicator series retain raw, normalized and source-tier values without interpolation or carry-forward.

The agenda pipeline turns each country into three lists:

• raise for low scores with usable evidence;
• measure for dimensions whose confidence is below the usable band;
• hold for the remaining dimensions.

The language-neutral agenda JSON and rendered English or Brazilian Portuguese documents are written to data/out/agenda. The agenda is computed from the score and registry; it does not invent a new number. Where a country has an institutional network, each dimension-level agenda item also carries the ids of the related institution nodes for investigation. Those links are navigation, not evidence or score inputs.

Two assessment tracks

The benchmark keeps measurement and interpretation in separate tracks.

The evidence expansion workflow is AI-first but source-first. pnpm bench research inventory builds deterministic uncovered country-gap slots; research scout uses a structured model call to propose leads and source search targets; research critique red-teams them. These runs are stored under data/research/runs and never enter data/evidence automatically. A researcher must still open and verify the source, write the limits paragraph and pass pnpm bench validate before a delivery appears on the agenda. Read docs/RESEARCH-AI.md for the full loop.

Source-backed measurement. The registry defines each indicator and its source, units, transform and direction. World Bank data and reproducible source adapters both emit the same observation shape, then enter the shared scoring pipeline alongside the small set of manually authored observations. The first adapter imports the official Joint EVS/WVS A165 generalized interpersonal-trust table; it carries publisher-weighted values into data/observations and does not ask an AI model to invent or score them. Other named sources remain explicit gaps until their adapters and coverage are ready. The pipeline turns all accepted observations into the canonical score, confidence and indicator rows. Documented deliveries in data/evidence explain gaps but never change a score or its confidence. Read docs/EVIDENCE.md before adding one.

Delphi panel. The panel pairs language models with fixed analytical stances. It reviews thin or questionable dimensions using the source-backed evidence brief. It does not create observations or silently replace measured values. It uses two rounds: in round two, each panelist sees anonymized round-one scores and rationales. The run stores medians, interquartile ranges, model identity, country scope, dataset version, prompt version and provenance. Panel estimates remain in delphiScore and delphiIqr. The explicit blendedScore falls back to Delphi only when no indicator is observed, and blendedFrom says which track supplied it.

Real runs use the Vercel AI Gateway:

export AI_GATEWAY_API_KEY=...
export NCB_PANEL=anthropic/claude-opus-5,openai/gpt-5,google/gemini-2.5-pro
pnpm bench cost --max-coverage 1
pnpm bench delphi --rounds 2 --max-coverage 1 --activate

A country-restricted or coverage-restricted run is saved as an archive and does not replace data/delphi/latest.json unless --activate is passed. Use a restricted run, including --max-coverage 0.5, as a preflight. Use --max-coverage 1 for the full nine-dimension panel. A published run after a country-set change must cover the rebased set. A run with provenance "mock" is an offline stand-in and is not evidence. A run with fewer than three panelists is a session estimate, not a panel. Read docs/PANEL.md for the full contract.

Institutional data. Bodies no country owns, such as the UN or the World Bank, are recorded once in a global ledger and reached from a country map by id. Country-specific institutional networks are explanatory data. They show where capability is held and how it moves. They never enter a score or confidence. The current data and authoring rules are in docs/INSTITUTIONS.md.

Versioning and changelog

CHANGELOG.md is the single human-curated release history for the dataset and viewer. App releases describe the whole product; Dataset releases describe the scored data contract. The /changelog page, footer, Atom feed, sitemap and /llms.txt all read or link to that source. The build checks that entries are dated, unique, newest first within each stream and aligned with APP_VERSION or DATASET_VERSION; it never writes release notes or commits generated changes.

The complete release workflow and the major/minor/patch rules are in docs/VERSIONING.md.

Published output

data/out is generated output. pnpm bench score also writes the Frictionless Data Package descriptor and JSON Schemas.

File	Contents
data/out/index.json	Slim list of every country with nine dimension results.
data/out/countries/{ISO3}.json	One country in full, including indicator rows and yearly series.
data/out/indicators/{id}.json	One indicator across the country set.
data/out/table.csv	Flat country by dimension table.
data/out/diagnostics.json	Correlations, redundancy and GDP-sensitivity diagnostics, plus the income each is read against.
data/out/report.md	Human-readable findings report.
CHANGELOG.md	Human-readable release history, rendered at /changelog.
data/delphi/{runId}.json	Immutable record of one Delphi run and its provenance.
data/delphi/latest.json	Active Delphi run copied from one archived run.
data/out/agenda/{ISO3}.json	Language-neutral computed agenda.
data/out/agenda/{ISO3}.{lang}.md	Rendered agenda for a language lexicon.
data/out/datapackage.json	Dataset metadata, sources, license, resources and schemas.
data/out/schema/*.json	JSON Schemas for the published data shapes.

Use index.json for cross-country lists. Use a country file for one detailed profile and an indicator file for one measure across countries. Do not load all country files to build a list.

Viewer and deployment

The Next.js viewer is in apps/web. It provides country profiles, capability comparisons, agendas, indicators, sources, diagnostics, the Delphi panel, patterns, limits, decisions, the glossary and a challenge page. All of it is English, because the comparison is the subject. Brazil also has a country layer at /brasil: one country read in its own language, with its institution map and its subnational spread beside the national shape.

Vercel deploys the project national-capability-benchmark under the ev-io scope with apps/web as the Root Directory. The build command is:

pnpm --filter @ncb/core build && next build

Data is read from data/out at request time, so refreshed output must be committed before deploying.

Before quoting a number

Read docs/KNOWN-ARTEFACTS.md before using a score. The main risks are:

• Experimentation is still partly inferred from formal invention data.
• Coordination and Trust remain difficult to separate from wealth-correlated evidence.
• A country score is a coarse national proxy for capability below the national level.
• The current country frame is a comparison set, not the world.

Keep the score beside its confidence, observed-indicator count and coverage-floor status. When quoting a trend, include its span and matched basket size. When quoting a Delphi estimate, include its run ID, provenance, panel size and dataset version. Record the generation date too.

Repository map

packages/core   registry, ingestion, normalization, scoring, diagnostics, Delphi and CLI
apps/web        Next.js viewer
data/           observations, Delphi runs, evidence, institution maps and generated output
docs/           rationale, decisions, limits, panel and authoring guides

Start with:

• CONTRIBUTING.md for gap fills, evidence, languages and country changes;
• docs/RESEARCH-ROADMAP.md for the agent-ready research queue, Trust work package and source promotion gates;
• docs/WHY.md for the claim and scope;
• docs/DECISIONS.md for methodological choices;
• docs/KNOWN-ARTEFACTS.md for known measurement failures;
• AGENTS.md for repository invariants and World Bank API traps.

License

The code is MIT. World Bank data is published under CC BY 4.0. The typefaces use the SIL Open Font License 1.1. The Envisioning brand is not covered by the code license. See NOTICE.md before reusing the repository.