parquetable

A lightweight Apache Parquet reader and writer for Rust and TypeScript/WebAssembly — small enough to ship to the browser, with full support for nested types.

• 🦀 A Rust crate (cargo) for native and wasm32 targets.
• 📦 An npm package (wasm-bindgen) with a columnar typed-array API and a ~50 KB .wasm.
• 🪆 Nested types — list, struct, map, and their arbitrary nesting.
• 🔌 No Arrow dependency, no C toolchain on the default path — just pure Rust.

Every type, encoding, and codec is round-trip verified against Apache Arrow / PyArrow.

Why?

Reading and writing Parquet in the browser usually means one of two things: pull in the Rust arrow + parquet stack compiled to WebAssembly (powerful, but several megabytes), or use a pure-JavaScript reader (tiny, but read-only at its core and JS-speed for decode).

parquetable aims for the middle: a single small WebAssembly module that both reads and writes Parquet — including nested types — with a plain columnar typed-array API and no Arrow dependency. It's a fresh Rust port of the MIT-licensed carquet C library, organized around minimal dependencies so the default build stays compact. The same core is an idiomatic Rust crate, so you get one implementation for native, wasm32, and npm.

How it compares

	parquetable	parquet-wasm	hyparquet
Approach	Rust → WASM	Rust (arrow+parquet) → WASM	Pure JavaScript
Read / Write	✅ / ✅	✅ / ✅	✅ / ✅ (separate writer pkg)
Nested (list / struct / map)	✅	✅	✅
Bundle (compressed)	~52 KB br (snappy) ~108 KB br (all codecs)	~456 KB – 1.2 MB br	~10 KB gz (read core)
Compression	snappy default; gzip / lz4 / zstd opt-in	all built in	uncompressed + snappy; more via add-on
Output shape	columnar typed arrays	Arrow (zero-copy to Arrow JS)	row / column JS objects
Read one row group (from memory)	✅	✅	✅
Async / streaming I/O (range reads)	❌	✅	✅ (partial reads)
Predicate / column pushdown	❌	✅	✅

_{Sizes are compressed WebAssembly. parquetable figures are measured (make wasm-pkg); others are from their project docs. brotli (br) vs gzip (gz) as each project reports.}

When to reach for which: use parquet-wasm if you live in the Arrow ecosystem and want zero-copy Arrow with async + pushdown; use hyparquet for the absolute smallest read-only footprint with no WASM; use parquetable when you want a small WASM that reads and writes (nested types included) with a plain typed-array API and no Arrow dependency — or you need the same engine as a Rust crate.

Install

Rust

[dependencies]
parquetable = "0.1"
# codecs beyond snappy are opt-in: features = ["gzip", "lz4", "zstd"]

npm (the wasm binding)

npm install parquetable

Quick start

Rust

use parquetable::{read_table, ColumnData, PhysicalType, Repetition, SchemaBuilder, WriterBuilder};

let schema = SchemaBuilder::new("root")
    .column("id", PhysicalType::Int64, Repetition::Required)
    .string("name", Repetition::Optional)
    .build()?;

let mut writer = WriterBuilder::new(schema).build();
writer.write_batch(&[
    ColumnData::Int64(vec![1, 2, 3]),
    ColumnData::ByteArray(vec![b"a".to_vec(), b"b".to_vec(), b"c".to_vec()]),
])?;
let bytes = writer.finish()?;

let table = read_table(&bytes)?;
assert_eq!(table.num_rows(), 3);

Nested columns use a recursive Column with a Values enum (Leaf / List / Struct); see docs/nested_types_plan.md and crates/parquetable/tests/nested.rs.

TypeScript / JavaScript

readParquet hands back columns as typed arrays; a column is nullable iff it carries a validity mask (1 = present, 0 = null). writeParquet takes the same shape back.

import { readParquet, writeParquet } from "parquetable";

const bytes = writeParquet({
  columns: [
    { name: "id", kind: "leaf", type: "int64", validity: null, values: BigInt64Array.from([1n, 2n, 3n]) },
    { name: "name", kind: "leaf", type: "string", validity: Uint8Array.from([1, 0, 1]), values: ["a", "", "c"] },
    {
      name: "tags", kind: "list", logical: { name: "list" }, validity: null,
      offsets: Int32Array.from([0, 2, 2, 3]),
      child: { name: "element", kind: "leaf", type: "string", validity: null, values: ["x", "y", "z"] },
    },
  ],
}, { compression: "snappy" });

const table = readParquet(bytes);            // { numRows, columns }
for (const col of table.columns) {
  if (col.kind === "leaf") console.log(col.name, col.type, col.values);
}

Nested types nest arbitrarily; a map is a list with logical: { name: "map" } whose child is a { key, value } struct. The read and write shapes are symmetric, so writeParquet(readParquet(bytes)) round-trips. Full types live in crates/parquetable-wasm/js/parquetable.d.ts.

Reading one row group at a time

When the whole file already lives in memory — say a monthly file cached in IndexedDB whose row groups partition the data by day — ParquetFile copies the bytes into WASM once and decodes a single row group on demand. The footer is parsed up front; each readRowGroup(i) touches only that group's bytes.

import { ParquetFile } from "parquetable";

const file = new ParquetFile(cachedBytes);   // copied in + footer validated once
console.log(file.numRowGroups, file.numRows);

const day = file.readRowGroup(3);            // decode just the 4th row group
// ... or a contiguous span: file.readRowGroupRange(0, 5)
file.free();                                 // release the WASM-side buffer

The Rust API mirrors it: parse once with Reader::new(&bytes), then reader.read_row_group(i) / reader.read_row_group_range(start, end).

Note: this is in-memory partial decode, not streaming I/O — the full file must be present. For files up to a few MB (already downloaded and cached) that's exactly the fast path; you skip decoding every other row group without needing range requests.

What's supported

Area	Coverage
Physical types	BOOLEAN, INT32, INT64, INT96, FLOAT, DOUBLE, BYTE_ARRAY, FIXED_LEN_BYTE_ARRAY
Logical types	STRING, DATE, TIME, TIMESTAMP, DECIMAL, INTEGER, UUID, JSON, BSON, ENUM, FLOAT16, LIST, MAP
Nested	LIST, STRUCT, MAP and arbitrary nesting (list<list>, list<struct>, struct of list, …)
Encodings	Reads all: PLAIN, RLE/bit-pack, RLE_DICTIONARY, the DELTA_* family, BYTE_STREAM_SPLIT. Writes PLAIN, dictionary, and BYTE_STREAM_SPLIT
Compression	Reads/writes uncompressed + snappy (default); gzip / lz4 / lz4_raw / zstd behind cargo features (all pure-Rust; zstd encode via an optional C crate)
Integrity & metadata	Per-page CRC32 checksums, column statistics (min/max/null-count), multiple row groups
Partial reads	Decode a single row group (or a contiguous range) from an in-memory file — footer parsed once, only that group's bytes touched. Ideal for a cached file whose row groups partition the data (e.g. one per day)

Size (measured)

Default build (snappy, read + write + nested), via make wasm-pkg:

Build	raw	gzip	brotli
default (snappy)	~150 KB	~63 KB	~52 KB
all codecs (snappy + gzip + lz4 + zstd)	~314 KB	~126 KB	~108 KB

How it works

• Pay-for-what-you-use nesting. Flat/linear columns keep a dense, zero-copy-friendly path — no offsets, no level streams, no tree walk. Nested columns add exactly one integer pass over Dremel repetition/definition levels to build Arrow-style offsets + per-level validity (or the inverse on write). The shred/assemble is a general recursion over the schema tree, so list, struct, map, and their nesting all fall out of the same code.
• Small by default. The default build pulls in only Snappy (pure Rust). Each additional codec is a cargo feature gating a pure-Rust crate, so a WASM build only pays for what it enables. Release/WASM profiles use opt-level="z", LTO, panic="abort", and wasm-opt -Oz.
• One engine, two front ends. The reader/writer are plain Rust; the npm package is a thin wasm-bindgen layer (js-sys, no serde) that surfaces leaf columns as real typed arrays.

Not yet (and honest limitations)

parquetable is young and deliberately scoped. It currently does not do:

• Async / streaming I/O — range reads that fetch only part of a file over the network. The whole file must be in memory first. (Once it is, you can decode a single row group at a time — see below.)
• Predicate / column-projection pushdown (a read pulls every column of the requested row groups).
• Bloom filters, or the page/offset index.
• DataPage V2 (reading a V2 page errors; the writer emits V1).
• Write-side DELTA_* encodings (they are read-only for now).
• Zero-copy interop with Arrow JS (you get typed arrays; convert if you need Arrow).

If you need those today, parquet-wasm is the mature, Arrow-native choice.

Development

make test          # Rust unit + integration tests
make check         # everything: tests, wasm build, and all PyArrow / wasm interop suites
make wasm-node     # build the nodejs wasm package and run the JS round-trip test
make wasm-pkg      # build the publishable bundler package (crates/parquetable-wasm/pkg) + size report
make venv          # create .venv with pyarrow for the interop checks

Cross-compatibility is enforced by a set of PyArrow interop suites (we read files it writes, and it reads ours) covering encodings, codecs, logical types, nested types, and — through the JS typed-array boundary — the wasm binding itself. See CLAUDE.md for the codebase map.

License

Licensed under either of Apache License, Version 2.0 or MIT license at your option. parquetable is a Rust port of the MIT-licensed carquet C library; ports and attributions (carquet, Google Snappy) are recorded in NOTICE.