{"repo":"xqlsystems/duckdb-zarr","free":true,"listed":false,"github":"https://github.com/xqlsystems/duckdb-zarr","clone":"git clone https://github.com/xqlsystems/duckdb-zarr.git","description":"Query Zarr stores with SQL directly from DuckDB","language":"Rust","stars":41,"topics":["duckdb","sql","xarray","zarr"],"license":null,"category":"databases-storage","readme_excerpt":"duckdb-zarr A Rust DuckDB extension that lets you query Zarr stores with SQL — in the same spirit as xarray-sql and zarr-datafusion, but as a first-class DuckDB extension with no external query engine. See docs/design.md for the full design and docs/ome-zarr.md for a small bioimage example. Remote stores Read directly from a URL — HTTP/HTTPS, S3, GCS, or Azure. Object stores can't list directories, so a remote store must carry consolidated metadata : a Zarr v2 .zmetadata object or a Zarr v3 consolidated metadata block. Most published datasets already do (e.g. anything written by xarray with consolidated=True ), so they read straight from their URL as shown above. S3/GCS/Azure credentials come from DuckDB's secrets manager ( CREATE SECRET ... TYPE S3 ). Status Active development. Phases 1–3 are implemented: - Phase 1 — read zarr , read zarr metadata , read zarr groups table functions; Zarr v3; CF conventions (fill values, scale/offset, bounds variables, aux coords) - Phase 2 — Zarr v2, Blosc/LZ4, replacement scan for local .zarr paths, projection pushdown - Phase 3 — HTTP/HTTPS stores, dims= and array path= selection, recursive array discovery See the phased plan for what's next. Building Requires: Rust toolchain, Python 3.11+, make, git. Testing SQLLogicTest cases live in test/sql/ . The HTTP suite ( test/test http integration .py ) reads stores over HTTP: synthetic v2 .zmetadata and v3 consolidated metadata fixtures via a loopback server, plus the live public GPCP store (net","default_branch":null,"files":null,"tree":[],"storefront":"/r/xqlsystems","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/xqlsystems/duckdb-zarr/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}