[arthurmccray]

Follows #13 (tooling, correctness fixes, typing/pathlib). Base: c966b14.

Breaking. The dataset YAML schema and the public attribute surface both
change. Needs a minor version bump and release notes.

---

1. metadata is a dataclass

json-schema.json closes the dataset entry with additionalProperties: false,
so an entry is a fixed set of fields. It is now a frozen dataclass,
DatasetMetadata, with Author nested, built only through
DatasetMetadata.from_spec().

Before, the constructor named some fields and left the rest in a metadata
dict, so ds.detector worked and ds.technique raised. Now everything the YAML
declares is on ds.metadata.

before	after
ds.description, ds.license, ds.doi, ds.detector, ds.detector_manufacturer	ds.metadata.<field>
ds.technique raised	ds.metadata.technique
ds.source, ds.file, ds.checksum	unchanged, now read-only properties

The four fields the download machinery needs — source, file, checksum,
size_bytes — stay reachable off the dataset, plus size for display. They are
read-only; override one through the constructor
(data.CuZnHAADF(checksum=...)).

Validation runs when the class is built, not when it is instantiated. A
malformed YAML fails the import that reads it, naming the file and the offending
key, rather than making the dataset disappear from the browser.

folder was declared in the schema and used by none of the 20 files. quality
was popped by the constructor, absent from the schema and from every file. Both
dropped.

__repr__ is one line (<DatasetMetadata smallPtychography.hspy · 4D-STEM · 12.5 MB>); the generated dataclass repr ran to about 1000 characters, nearly all
description. print(ds.metadata) gives the full record, wrapped, unset fields
omitted.

2. data_size becomes size_bytes

"1.4 GB" cannot be sorted, summed or compared. All 20 files now carry an
integer, taken from the Content-Length of each source.

• ds.size_bytes -> int
• ds.size -> "1.4 GB", driving the repr and the widgets
• json-schema.json: size_bytes, integer, minimum 0

test_source_url_resolves asserts the header still matches the declared value,
which also catches a source file being replaced — invisible today until someone's
checksum fails.

One correction fell out: AlNanocrystals was declared 1.4 GB and is
1381831542 bytes, so it now displays as 1.38 GB. The other 19 reproduce their
old strings exactly.

3. The YAML is validated

Nothing checked the shipped files against the shipped schema — not CI, not
import, not the issue action.

• test_yaml_matches_schema, parametrized over every file
• schema/dataclass parity tests for DatasetMetadata and Author, so the two
  descriptions of one record cannot drift
• the same validation in on_new_issue.yml, before the PR is opened
• jsonschema is a test/dev dependency only; the files are validated in CI, so
  validating again at import would cost every user startup time for nothing

jsonschema was added to the tests extra as well as dev — it was only in
dev, so the test matrix would have skipped validation entirely.

4. Vendor lists

Two lists, hand-duplicated in two files, with nothing checking they agreed, and a
third entry path ignoring both.

where	was
json-schema.json microscope_vendor	real enum, enforced
json-schema.json detector_manufacturer	enum nested under a stray options key, inert
_build_docs.py:860	_MANUFACTURERS / _VENDORS tuples

Two of the five detector manufacturers in use, NORDIF and Allied Vision,
were already outside the list and passed only because that enum was inert.

Both enums are dropped. One canonical list in datasets/vendors.yaml feeds the
docs form (as a datalist, suggesting without constraining) and a
difflib.get_close_matches check that fails a near-miss and warns on a
genuinely new vendor. A hard enum fails the legitimate new vendor and blocks a
contributor who has done nothing wrong; this catches the failure that actually
happens.

Short names are the check's weak point — JOEL for JEOL scores too low to be
called a typo and comes back as a warning. Documented on the function.

5. A query API

The only way to filter the catalogue was the widget's search box. From Python
you reached into a private-looking module.

em_database.list_datasets()
em_database.search("jeol eels")
em_database.filter(technique="4D-STEM", tags="Strain")
em_database.filter(microscope_vendor=["JEOL", "Hitachi"])
em_database.filter(downloaded=True, location="shared")

All three return dataset objects, so a result can be downloaded directly.
search() matches the same lowercased blob catalogue.entry() builds for the
widget, by the same rule — every whitespace-separated term must appear, not
necessarily in the same field. The widget keeps matching in the browser, because
a round trip to the kernel on every keystroke would make typing lag; what is
shared is the blob and the rule.

filter() compares strings exactly but case-insensitively, tests membership for
tags and authors, treats a list as any-of, and raises on an unknown field —
a typo that quietly returned the whole index would be worse than an error.

It is list_datasets(), not datasets(), because em_database/datasets/ is a
subpackage: importing it binds the module onto the parent package and replaces a
function of the same name. Sphinx's autosummary walks every submodule, so this
is not hypothetical. query.py is named that way rather than search.py for the
same reason. A test walks every submodule and asserts no exported name was
replaced.

Not in this PR

• sphinx_gallery executes every example on every docs build, and two of them
  download a 12.5 MB dataset. Same problem as the HEAD requests above; the fix
  costs those two examples their rendered output.
• _build_docs.py is 1255 lines shipping inside the installed package, imported
  only by doc/source/conf.py. It belongs in doc/.