Skip to content

Backup and restore of collections through the API: ChromaDotNet.Client.Backup and the command chromadotnet - #166

Merged
Laurianti merged 12 commits into
mainfrom
backup
Nov 15, 2026
Merged

Laurianti merged 12 commits into
mainfrom
backup

Conversation

@Laurianti

@Laurianti Laurianti commented Nov 15, 2026 •

Copy link
Copy Markdown
Member

Two new packages save collections of a database into a zip file through the API of Chroma, and write them back into the same server or another one.

ChromaDotNet.Client.Backup (netstandard2.0, net462, net10.0)

  • ChromaBackup.BackupAsync writes a zip file: manifest.json (format and version, server, times, counts and the SHA-256 of the records of each collection), and for each collection collection.json (name, metadata, configuration_json and schema as the server sent them, dimension) and records.jsonl, a record a line with id, metadata, embedding, text_chunk and uri. The first four are the names of the lines of Chroma Data Pipes.
  • Embeddings are written as the numbers that read back as the same floats; metadata keeps the JSON of Chroma (1, 1.0, lists, sparse vectors).
  • Records come with gets of limit and offset, up to 5 at a time, written in their order, until a page is not full. A request is tried again on 429, 408, a 5xx other than 501, or no answer, from 0.5 s doubled to 60 s or as its Retry-After says.
  • A server that takes writes during the backup can leave records of different moments in the file. The counts at the start and at the end, an id given twice, or another number of records set ChangedDuringBackup in the result and in the manifest.
  • ChromaBackup.VerifyAsync compares a file with the server: documents and URIs the same, embeddings within the tolerances of np.allclose, metadata exactly, the schema without the keys that the records brought in, the dimension and the metadata of the collection.
  • ChromaBackup.RestoreAsync compares every collection of the file that exists before it writes anything, as the copy command of the CLI of VecturaDB does. A collection that is not the same follows ChromaRestoreExisting: Fail (the default), Resume, Force or Skip, for all of them, by name, or from ChooseExisting. Force deletes and adds a different record instead of an upsert, because of KD-49. A new collection gets the schema of the file as the server sent it.

ChromaDotNet.Tool, the command chromadotnet (.NET 10, with System.CommandLine 3.0.0)

  • backup, restore and verify, with --url, --token or CHROMA_API_KEY, --tenant, --database and --distributed.
  • A restore asks for each collection that exists and is not the same; without a terminal and without --yes it leaves it as it is and exits with 1. A backup of a server that took writes, or a verification that finds a difference, exits with 2.
  • With Chroma 0.x it warns and goes on. Every Chroma 1.x answers 1.0.0 as its version, so the 1.x versions cannot be told apart.
  • Also as a file that runs without .NET, built with NativeAOT, for linux-x64 and linux-arm64 (glibc 2.34 or later), linux-musl-x64, osx-arm64, osx-x64 and win-x64: attached to each release with SHA256SUMS and their Sigstore signatures.

Additions to the public API of ChromaDotNet.Client

  • ChromaCollectionDefinition.SchemaJson (JsonElement?): the schema of a new collection as JSON, sent as it is, in place of Schema and Configuration.
  • ChromaException.RetryAfter (TimeSpan?): the delay of the Retry-After of the answer; null when the answer has none.

CI: the job native publishes the command for the six platforms, every warning an error, and on Linux backs up, restores and verifies a collection of Chroma 1.5.9 with the file. release-assets.yml attaches the files, and compares the two new packages with nuget.org too. The tests of the backup run on Chroma 1.5.9 and later and on VecturaDB, and are skipped on the older versions of the matrix. The job of VecturaDB starts Chroma 1.5.9 as a second server, so that a backup of each one is restored into the other.

Documentation: docs/backup.md, linked from the README.

Tested with the SDK 11.0.100, as the CI builds: every test passed against Chroma 1.5.9, also in a tenant and database of their own and with the netstandard2.0 builds, and against VecturaDB 1.0.0-preview.9. By hand against Chroma Cloud: the tests of the backup, the restore and the command, and a backup of Chroma Cloud restored into VecturaDB and Chroma 1.5.9, and back. Distributed Chroma installed by yourself: not tested.

The native command: linux-x64, linux-arm64 and linux-musl-x64 back up, restore and verify a collection of Chroma 1.5.9 in the job native; win-x64, the file of that job, did the same by hand from VecturaDB 1.0.0-preview.9 into Chroma 1.5.9 and back; osx-arm64 and osx-x64 are only built and answer --version: not tested against a server.

…r sent it

A definition can carry the schema of a collection as JSON, the SchemaJson that ChromaCollection reads from a server, and
the create or get-or-create request sends it as it is, with the fields the client does not know: a restore creates the
collection with the indexes and settings of the one it saved. It is the whole schema, so Schema and Configuration stay
null, and the embedding function that a generator declares is not added to it. On Chroma Cloud its keys go through the
same quota of bytes as the keys of a ChromaCollectionSchema.

Tested against Chroma 1.5.9 and VecturaDB 1.0.0-preview.9: a collection created with the schema of another reads back
the same schema.
…tion

The Retry-After of a failed answer was internal, for the write retries of the client. Public next to StatusCode, it lets
a caller that retries on its own wait as long as the server asks, as VecturaDB does when it refuses a write to a SPANN
collection with 429.
…s comparison with the server

ChromaBackup.BackupAsync saves collections of the database of a client through the API into a zip file:
- manifest.json: the format and its version, the server, the times, and for each collection its counts and the SHA-256
  of its records;
- collections/0001/collection.json: name, metadata, configuration_json and schema as the server sent them, and the
  dimension;
- collections/0001/records.jsonl: a record a line, with the names of the lines of Chroma Data Pipes (id, metadata,
  embedding, text_chunk) and the uri. The embeddings are the numbers that read back as the same floats, and the metadata
  keeps the JSON of Chroma: an integer stays 1, a float 1.0, a list a list, a sparse vector
  {"#type":"sparse_vector",...}.

The records come with gets of limit and offset, which every Chroma has; the search of the copy command of the CLI of
VecturaDB is not in Chroma 1.5.9. As the copy command: up to 5 gets at a time, written in their order; a request goes
again while waiting can fix the error (429, 408, a 5xx but 501, no answer), from 0.5 s doubled to 60 s or as its
Retry-After says; the count again at the end. The pages go on until one is not full, so the records added meanwhile come
in too, and an id given again by a later page goes once. A count that changed, an id given again or another number of
records says that the server took writes during the backup: the result and the manifest tell it. The file is written
next to its path and renamed at the end, and an existing file stays.

ChromaBackup.VerifyAsync compares the collections of a file with the server, as the copy command compares a source with
its target: documents and URIs the same, embeddings within the tolerances of np.allclose, metadata exactly the same, the
schema without the keys that the records brought in, the dimension and the metadata of the collection. A file whose
records do not match the digest of its manifest throws ChromaBackupFormatException.

Tested against Chroma 1.5.9, also with the netstandard2.0 builds, and VecturaDB 1.0.0-preview.9, sparse vectors
included.
…ce into its target

Every collection of the file that exists on the server is compared first, and nothing is written before. One that is the
same stays as it is. One that is not follows ChromaRestoreExisting, for all of them or by name:
- Fail, the default, throws before any write and says what differs;
- Resume adds the missing records, and leaves the different ones, the ones only on the server, the schema and the
  metadata;
- Force makes the collection the same: with another schema or dimension it is deleted and created again; otherwise the
  records only on the server go, the different ones are deleted and added again, the missing ones added, and the
  metadata set. The copy command upserts a different record; on Chroma 1.0.21 to 1.5.9 an upsert of a record that
  exists, with its embedding, can lose a record from the index (KD-49), so the restore deletes and adds.

A new collection is created with the schema of the file as it is (SchemaJson) and its metadata; as in the copy command,
the old settings hnsw:* that Chroma refuses next to a schema come at the end with the metadata, and a 409 to the
creation reads the collection. A record without an embedding, which the server sent as null, cannot go back, and a
different one stays as the server has it. A server that refuses sparse vectors, as Chroma 1.5.9 with "Sparse vector
indexing is not enabled in local", gets an error that names the keys, and a new collection that got no record is
deleted. At the end the count must be the one written.

Tested against Chroma 1.5.9, also with the netstandard2.0 builds, and VecturaDB 1.0.0-preview.9, and from each one into
the other with CHROMA_TEST_SECOND_IMAGE, sparse vectors from VecturaDB into Chroma 1.5.9 included.
…arison, and Skip

ChromaRestoreOptions.ChooseExisting gets what the comparison found for each collection that exists and is not the same,
after every collection is compared and before any write, as the copy command of the CLI asks its questions; it chooses
for the collections that ExistingByCollection does not name. ChromaRestoreExisting.Skip leaves a collection as it is
while the others are restored, as the copy command leaves a collection it was not told what to do with; the result says
Skipped.
…e and verify

A dotnet tool on .NET 10, rolling forward to later versions, with System.CommandLine 3.0.0:
- chromadotnet backup --url <server> (--all | --collections <name> ...) --out <file>, with --batch and --concurrent;
- chromadotnet restore --url <server> --in <file> [--collections <name> ...] [--resume | --force] [--yes];
- chromadotnet verify --url <server> --in <file> [--collections <name> ...].
--token, --tenant and --database, or CHROMA_API_KEY, CHROMA_TENANT and CHROMA_DATABASE as Chroma names them, so that the
key stays out of the history of the shell; on Chroma Cloud without a tenant, the one of the API key; --distributed for
distributed Chroma not on Chroma Cloud.

As the copy command, a restore asks for each collection that exists and is not the same, after comparing all of them:
resume, force or leave; with --resume or --force it asks again unless --yes; without a terminal and without --yes it
leaves the collection as it is, says how to run it again, and exits with 1. A backup of a server that took writes
meanwhile exits with 2, and so does a verification that finds a difference. With a server the tests do not try, other
than Chroma 1.5.9, VecturaDB 1.0.0-preview.9 and Chroma Cloud, the command warns and goes on: distributed Chroma
installed by yourself too.

Tested against Chroma 1.5.9, and installed from its package: backup, restore and verify of a collection of VecturaDB
1.0.0-preview.9.
…ed across the two

The tests of the backup, the restore and the command chromadotnet against a server run on Chroma 1.5.9 and later and on
a server already running (VecturaDB, distributed Chroma, Chroma Cloud), the servers the backup is tested with; on the
older ones of the matrix they are skipped with that reason. The job of VecturaDB starts Chroma 1.5.9 as a second server
(CHROMA_TEST_SECOND_IMAGE), so that a backup of each one is restored into the other.
…rites holds

docs/backup.md, linked from the README: a server that takes writes during the backup, which can leave records of
different moments in the file, and how the result and the command tell it; the command chromadotnet and the library;
what the file holds and what a backup cannot save through the API; a restore into a collection that exists. Tested with
Chroma 1.5.9, VecturaDB and Chroma Cloud, which runs on distributed Chroma; distributed Chroma installed by yourself:
not tested.
@Laurianti
Laurianti requested a review from a team as a code owner November 15, 2026 09:24
…do not cover

On Chroma 0.4.10 to 1.0.0 of the matrix the tests of the backup, the restore and the command failed instead of being
skipped: an Assume in a SetUp of NUnit fails the test, and the TearDown then deleted a path that the SetUp had not set.
The SetUp now calls Assert.Ignore, and the TearDown deletes only a path that was set. With the condition forced to false,
the 17 tests of the four fixtures are skipped and none fails.
Chroma 1.5.9 answers 1.0.0 to /version, as every Chroma 1.x does, so the command warned that the server it is tested
with was not tested. It now warns only for Chroma 0.x, and says that the versions 1.x cannot be told apart.
…Net.Tool too

The job compares the packages of the CI with the ones on nuget.org by their ids, which were only the client and the
package of dependency injection: with the two new packages it would fail before the signatures and the assets.
… that runs without .NET

-p:NativeCommand=true publishes the command with NativeAOT. The job native of the CI publishes it for linux-x64,
linux-arm64, linux-musl-x64 (in the AOT image of the SDK on Alpine, as the actions of GitHub do not run there),
osx-arm64, osx-x64 and win-x64, every warning an error; on Linux the file backs up, restores and verifies a collection
of Chroma 1.5.9. release-assets.yml attaches the six files to the release, with SHA256SUMS and their Sigstore
signatures. docs/backup.md says how to download and check them, and that the files for glibc need 2.34 or later.

Tested on linux-x64: 10 MB, no warning, and backup, restore and verify in debian:stable-slim without .NET, from
VecturaDB 1.0.0-preview.9 into Chroma 1.5.9 and back.
@Laurianti
Laurianti merged commit 5c79aff into main Nov 15, 2026
23 checks passed
@Laurianti
Laurianti deleted the backup branch November 15, 2026 10:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants