[conradbzura]

Description

Every tiles_v2 tileset takes a filesystem path and opens it itself, so a server fronting remote data must download each file and keep it on disk for the tileset's lifetime. Add Source, a shared abstraction over "a path, or a factory that opens a fresh binary handle", and adopt it in the two h5py-backed tilesets.

This is the first of four backend-scoped slices under #13. h5py goes first because CoolerTileset and MultivecTileset have no index file, no sibling detection and no scan ceiling — the smallest honest proof of the abstraction. The remaining slices are #15 (pybigtools), #16 (oxbow) and #17 (apsw).

A factory, not an open handle, is the accepted shape: oxbow may reopen its source between queries so it needs a callable, while h5py and pybigtools want a live object. Calling a factory once serves both; a handle cannot be un-consumed. An already-open file is refused at construction, because the repair a caller reaches for — wrapping it in lambda: handle — returns the same exhausted handle every time and would serve records from wherever the last query left the cursor. The handle a factory returns must be fresh, seekable and binary, and all three are checked where the factory is called rather than left to fail inside a reader — h5py reports a missing seek as "expected str, bytes or os.PathLike object", which sends the caller looking for a path bug.

A path is still handed to h5py as a path. That is load-bearing rather than cosmetic: routing a path through an opened handle selects HDF5's fileobj driver instead of the native sec2, and sec2 is what lets HDF5 recognise a file already open read-only in the process and share one file struct. Fifty tilesets over one file cost one descriptor rather than fifty. Source.for_reader() returns the path string it was given, so the path branch is the same call it always was.

clodius does not depend on fsspec — the contract is a stdlib Callable[[], IO[bytes]], which fsspec, smart_open, boto3 and a bare lambda all satisfy. fsspec is a dev dependency so the tests can show a real remote-capable filesystem meeting it.

Fixes #14. Part of #13.

Proposed changes

clodius/core/source.py (new) — Source, stdlib-only and public, exported from clodius.core. Two entry points: coerce takes a caller's untyped argument and decides which shape it is; the constructor takes the shape already decided and holds every invariant coerce would — a path is decoded with os.fsdecode and must be non-empty, a factory must be callable, exactly one of the two is given. Keyword-only, so a factory cannot land in the path slot. path/for_reader/open/open_text serve the readers; size/sibling/has_sibling back the scan ceiling and index detection. sibling returns nothing for a factory rather than guessing a filename, which is what forces an index to be passed explicitly.

clodius/tiles_v2/_h5.py (new) — H5Backed, holding the open-and-release protocol both HDF5 tilesets share, with a _validate(file) hook each overrides for its own schema check. It lives in tiles_v2 rather than beside Source because clodius.core must not depend on h5py. The property opens lazily under a double-checked lock, so concurrent first access calls the caller's factory once rather than N times and leaves nothing close() cannot reach; close() releases the handle in a finally, because a failing HDF5 close must not strand it. A path-backed source produces no handle at all.

cooler.py, multivec.py — take their source through H5Backed and declare neither file nor close. The first positional parameter is source: SourceLike, documented in both class docstrings along with every other parameter. Each constructor releases what it opened if the work after the open raises, since a constructor that raises returns no object for anyone to close.

core/tileset.py — _src: Source declared as an annotated, unassigned attribute beside ndim, and a __repr__ reading it; a tileset was previously anonymous in a stack trace. The ownership rule is stated on close(): a tileset releases what it opened and nothing else.

bed.py, bedpe.py, gxf.py, variant.py — sibling-index detection routed through Source.has_sibling, deleting three byte-identical copies of the probe. The copies tested os.path.exists, which a directory named <path>.tbi satisfies, so a tileset declared itself indexed and failed on first open — and that flag decides whether a tile range-queries or scans. alignment.py keeps its own probe until its slice.

cooler.py (separate fix, first commit) — a cooler with no resolutions group was refused exactly once, because the check ran after the opened file was cached; every later access raised a bare KeyError. The refusal is now repeatable and raises TilesetUnavailable, so a server renders it as a refusal rather than a 500. This is breaking for anything catching ValueError from that path.

Test cases

#	Test suite	Given	When	Then	Coverage target
1	test_source	A path as a string, bytes, or PathLike	It is coerced	A path source is produced, decoded to text	Path normalization
2	test_source	A zero-argument callable, or a handle class	It is coerced	A factory source is produced	Factory acceptance
3	test_source	An already-open file	It is coerced	TypeError naming the callable form	The refusal
4	test_source	An empty path, through either entry point	A source is constructed	ValueError	The guard both doors hold
5	test_source	A path or factory passed positionally	A source is constructed	TypeError	The slot a factory used to land in
6	test_source	A non-callable factory, or both shapes, or neither	A source is constructed	TypeError or ValueError	The remaining constructor invariants
7	test_source	A path-backed source	Its reader argument is taken	The identical string object	The unchanged-call claim
8	test_source	The same file as both shapes, and an fsspec-backed factory	Each is opened and read	The file's actual bytes	Read fidelity, no adapter needed
9	test_source	A text handle, a non-seekable handle, or a non-handle	The source is opened	TypeError naming the missing capability, handle released	The published contract
10	test_source	A factory-backed source opened as text	The wrapper closes	The binary handle closes too	Text-wrapper ownership
11	test_source	A file, a directory, a missing path, a raising factory, an unseekable handle	Each is measured	The byte count, or nothing	Ceiling measurement
12	test_source	Two candidate index suffixes, one, none, a directory, a factory	Siblings are searched	The earlier wins; a directory and a factory find none	Index detection
13	test_cooler, test_multivec	The same file by path and by factory	Every tile in the ladder is requested, balanced and not	Payloads match tile for tile and none is an error	Source-shape equivalence
14	test_cooler, test_multivec	A tileset built from a path, and one from a factory	Its HDF5 file is opened	The native sec2 driver, and fileobj respectively	The path branch is the same call
15	test_cooler	Twenty tilesets over one path	Each opens its file	The process holds one descriptor, not twenty	HDF5 handle sharing
16	test_cooler	A cold factory-backed tileset and eight threads released together	They all touch the file first	The factory is called once and every handle closes	Concurrent first access
17	test_cooler	An open tileset whose h5py.File.close raises	It is closed	The handle is released anyway	Release on a failing close
18	test_cooler, test_multivec	A file that is valid HDF5 but not the expected schema	Construction fails	Every handle the factory opened is closed	Ownership on the failing path
19	test_cooler	A cooler with no resolutions group	Its info is requested three times	The same TilesetUnavailable each time	The refusal does not decay
20	test_cooler, test_multivec	A factory-backed tileset that was closed	A tile is requested again	It is served from a second handle	Reopen after close
21	test_cooler, test_multivec	A factory-backed tileset, opened	It is closed	Both the HDF5 file and the handle are released	Paired close
22	test_bed	A plain BED with a directory named <path>.tbi beside it	The tileset is constructed	It does not report itself indexed	The reconciled sibling probe
23	test_tileset	A tileset with no source, a path, or an empty path	It is rendered	The class alone, or the class and path	Both repr branches
24	test_tileset	The base class	Its annotations are read	_src is declared and unassigned	The published attribute name