Skip to content

INDEX.md

Latest commit

 

History

History
133 lines (93 loc) · 6.25 KB

INDEX.md

File metadata and controls

133 lines (93 loc) · 6.25 KB
Raw

AR.IO Gateway Documentation

Welcome to the AR.IO Gateway documentation. This index provides an overview of all available documentation.

Getting Started

Document Description
Linux Setup Installation and setup guide for Linux
Windows Setup Installation and setup guide for Windows
Environment Variables Complete reference for all configuration options

Core Features

CDB64 Root Transaction Index

Fast, offline lookups for data item to root transaction mappings.

Document Description
CDB64 Root-TX Index How the gateway uses CDB64 indexes: lookup order, /ar-io/offsets answers, configuration, sources, watching, troubleshooting
CDB64 Tools Reference CLI tools for creating indexes
CDB64 Format Specification Technical file format details

Index Sharing

Gateways publish signed index bands and subscribe to each other's.

Document Description
Index Sharing: the index-swarm sidecar Publishing and subscribing over HTTP and BitTorrent: setup and status scripts (--l1 to fill L1 history from bands), the tracker through the gateway's HTTPS, RSS feeds and WebSeeds, lookup order, running behind nginx, the torrent engine, metrics
Index export Building root-TX and L1 bands with the index-export service: checks, running it, L1 layouts and lookup files, adoption, alerting
Index Sharing: the publication protocol Discovering, verifying and reading bands without the sidecar: the signed document, byte routes, following an index in a BitTorrent client through its RSS feed, L1 bands, lookup files, querying over HTTP, a reference client

Rate Limiting & Payments

Document Description
x402 and Rate Limiting Rate limiter configuration and x402 payment protocol

Filters

Document Description
Filters Transaction and bundle filter syntax

Chunk Ingest Cache

Document Description
Optimistic Chunk Ingest Cache Validating and caching chunks posted to the gateway, with confirmation-driven cleanup

Data Cache Cleanup

Document Description
Contiguous Data Cache Cleanup The two disk-pressure reclaimers (filesystem-walk worker vs. index evictor) and their shared watermark semantics; plus the index evictor's LRU ordering, tier promotion, and backfill

Data Export

Document Description
Parquet and ClickHouse Exporting data to Parquet format
ClickHouse Pipeline Architecture SQLite → Parquet → ClickHouse pipeline and GraphQL routing
ClickHouse Schema and Query Optimizations Table layout, indexes, projections, and GraphQL query shape

Deployment

Document Description
Deployment Topologies Proxy edge, shared ClickHouse, partitioning, and app-split topologies
The ar-io-node CLI The gateway's command-line tool: building, checking and exporting index bands (index-band-build, index-band-verify, index-band-export), filling an L1 index from published bands, while the gateway runs (--fill) or into a new database, and checking it against the chain (index-l1-import, index-l1-verify, index-l1-audit), and every ar.io CLI command, run in the core image through tools/ar-io-node

Reference

Document Description
Glossary Definitions of terms and concepts
OpenAPI Specification REST API specification

Arweave Internals

Technical details about Arweave data structures.

Document Description
Merkle Tree Structure How Arweave merkle trees work
Transaction and Chunk Offsets Offset calculations for data retrieval

Architecture Decision Records

Document Description
001 - ClickHouse GQL GraphQL with ClickHouse backend
002 - ArNS Cache Timing ArNS resolution caching strategy
003 - ArNS Undername Limits Undername resolution limits
004 - Optimistic L1 Transaction Indexing Index a signed L1 tx before it mines + the never-serve-as-permanent guard
005 - Chunk Data Cache Indexed Eviction Per-dataRoot SQLite eviction index for the chunk cache, with a derived ingest-confirmation age floor
006 - qBittorrent as the Torrent Engine Why Index Sharing drives qBittorrent rather than Transmission or rqbit

Testing

Document Description
Auto-Verify Cross-source indexing verification (SQLite / Parquet / bundle-parser / ClickHouse)

Developer Tools

Document Description
Tools README CLI tools for development, debugging, and testing (fetch-with-hint, queue-missing-bundles, test-clickhouse-graphql, release/worktree tooling, etc.)

Processes

Document Description
Release Process How to create and publish releases

Database Schemas

SQLite schema documentation is in the sqlite/ directory.

Diagrams

Architecture diagrams, rendered from the PlantUML sources in diagrams/src/:

Diagram Shows
Gateway The gateway's services and stores, including Index Sharing
Chain API Chain requests through Envoy to Arweave
Data API Data retrieval and caching
GraphQL API GraphQL query routing