{"repo":"quackscience/rawduck","free":true,"listed":false,"github":"https://github.com/quackscience/rawduck","clone":"git clone https://github.com/quackscience/rawduck.git","description":"Experimental RawMergeTree-like Extension for DuckDB","language":"C++","stars":30,"topics":["duckdb","json","duckdb-json","faster-json","rawmergetree","rawtree","duckdb-extension","unstructured-data"],"license":"MIT","category":"analytics","readme_excerpt":"RawDuck Schema-less JSON analytics for DuckDB, RawMergeTree style RawDuck brings the RawMergeTree \"ingest first, schema later\" model to DuckDB: point raw JSON, NDJSON files, or OTLP telemetry at tables that don't exist yet — RawDuck creates them, types them, flattens nested objects into real columns, transforms and evolves the schema as the data changes. ⚡ Benefits No CREATE TABLE , no schema declarations, no json extract at query time. Because data lands shredded into native typed columns instead of opaque JSON strings, analytical queries run 27–63× faster on telemetry queries, 6× smaller on disk — see benchmark. ⚙️ Under the hood RawDuck delivers a complete engine rather than a parser: ingestion is transactional, pipelined, and multi-threaded through DuckDB's own catalog and storage APIs ( BEGIN / ROLLBACK ) and the optimizer observes the workload and adapts — physically re-sorting tables by the columns queries actually filter on (incrementally, MergeTree-parts style) and answering recurring aggregations from projections. Install Usage Attach a store and INSERT raw JSON — tables, typed columns, and schema all emerge from the data: The ingest schema accepts any SQL source through a fully parallel zero-copy sink Ingest a different shape and the table follows the data: new keys become columns, conflicting types widen, missing keys read as NULL — nothing is ever dropped. And RawMergeTree tables stay regular DuckDB tables, so every statement and tool works at native speed: For i","default_branch":null,"files":null,"tree":[],"storefront":"/r/quackscience","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/quackscience/rawduck/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}