{"repo":"ilum-cloud/duck_lineage","free":true,"listed":false,"github":"https://github.com/ilum-cloud/duck_lineage","clone":"git clone https://github.com/ilum-cloud/duck_lineage.git","description":"A extension for DuckDB, which captures lineage events for executed queries","language":"Python","stars":78,"topics":["data-lineage","data-management","duckdb","duckdb-extension"],"license":"MIT","category":"analytics","readme_excerpt":"DuckLineage Extension for DuckDB This is an extension for DuckDB that automatically captures and emits OpenLineage events for every query executed. This enables automated data lineage, governance, and observability for DuckDB workloads. DuckLineage is developed by ILUM, the free data lakehouse platform for a cloud native world. Supported Features This extension currently implements the following OpenLineage capabilities: - Automatic events: Emits START, COMPLETE and FAIL events for each query execution. - Dataset detection: Extracts input and output datasets from query plans (tables, CREATE/INSERT/UPDATE/DELETE/COPY, basic file table functions). - Schema capture: Records dataset schema (column names and types) as a dataset facet. - Column-level lineage: Emits the OpenLineage columnLineage dataset facet on output datasets, tracking which input columns flow into each output column and whether the transformation is direct or indirect. Supports column refs, expressions, aliases, CAST, star expansion, JOINs, aggregation, UNION/INTERSECT/EXCEPT, window functions, CTAS, INSERT INTO SELECT, file scans (CSV/Parquet), PIVOT, and UNNEST. - Facets: Job sql facet, run parent facet (via OPENLINEAGE PARENT\\ env vars), processing engine , dataSource and catalog facets, lifecycle change facets (CREATE/DROP/ALTER/OVERWRITE/RENAME/TRUNCATE), and basic outputStatistics (row count). - DuckLake support: Works with DuckLake catalogs. For DuckLake with external storage (S3, local path), the dataset ","default_branch":null,"files":null,"tree":[],"storefront":"/r/ilum-cloud","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/ilum-cloud/duck_lineage/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}