{"repo":"definite-app/crabwalk","free":true,"listed":false,"github":"https://github.com/definite-app/crabwalk","clone":"git clone https://github.com/definite-app/crabwalk.git","description":"SQL transformation tool for DuckDB written in Rust","language":"Rust","stars":76,"topics":["dbt","duckdb","rust"],"license":null,"category":"data-pipelines","readme_excerpt":"Crabwalk: An Easy SQL Orchestration Tool Crabwalk is a lightweight SQL orchestrator built on top of DuckDB. It processes SQL files in a folder, determines dependencies, and runs them in the correct order. Demo Video Features - SQL Orchestration : Automatically determine the execution order of SQL queries based on dependencies - Flexible Output Types : Configure outputs as tables, views, or files (Parquet, CSV, JSON) - Model-level Configuration : Set output types and other options at the model level using SQL comments - Schema Generation : Generate detailed XML database schema including tables, columns, and relationships - Column-level Lineage : Track data lineage at the column level to understand data flow - Schema Visualization : Create interactive HTML visualizations of database schemas and dependencies - S3 Integration : Backup and restore your DuckDB database to/from S3 (optional) - Lightweight : Minimal dependencies, fast execution - Environment Variables : Support for environment variables in SQL queries Installation From Source Usage Getting Started with the Example The project includes a simple example you can run to see Crabwalk in action: The example processes these files: - examples/simple/staging/stg customers.sql - A simple customers table - examples/simple/staging/stg orders.sql - A simple orders table - examples/simple/marts/customer orders.sql - Joins customers and orders with model-level config: @config: {output: {type: \"view\"}} - examples/simple/marts/order ","default_branch":null,"files":null,"tree":[],"storefront":"/r/definite-app","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/definite-app/crabwalk/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}