SQLite-based single-file store for zarr v3 datasets, in Python.
SQLiteStore provides a single-file storage backend for Zarr. Key advantages over alternative single-file formats (e.g., ZipStore):
- Support for key deletion, overwriting, and partial value writes.
- Full ACID guarantees provided by SQLite.
- High availability of SQLite implementations across programming languages and environments.
Example usage:
import zarr
from zarr_sqlite import SQLiteStore
with SQLiteStore("my_zarr_file.zarrdb") as store:
root = zarr.create_group(store=store)
foo = root.create_group('foo')
bar = foo.create_group('bar')
z1 = bar.create_array(name='baz', shape=(10000, 10000), chunks=(1000, 1000), dtype='int32')
z1[:] = 42SQLiteStore otherwise behaves identically to other stores used with zarr, see
the zarr user guide
for more information.
The store format is described in the document SPEC.md. This document should allow the implementation of SQLiteStore for other programming languages or Zarr libraries.
The version of this library was incremented to v1.0 to reflect the fact that it complies with v1 of the SQLiteStore Specification. However, this is a relatively new library that has not seen a lot of real-world use yet, therefore users should expect the occasional bug, and possibly breaking API changes in future versions, if absolutely necessary.
The database format however, defined by the Specification, can be expected to be stable.