{"repo":"lisancao/lakehouse-at-home","free":true,"listed":false,"github":"https://github.com/lisancao/lakehouse-at-home","clone":"git clone https://github.com/lisancao/lakehouse-at-home.git","description":"A fully open-source, self-hostable data lakehouse for local development and testing of modern data workflows","language":"Python","stars":107,"topics":["big-data","lakehouse","open-source","self-hosted"],"license":null,"category":"self-hosted-apps","readme_excerpt":"Lakehouse at Home A fully open-source, self-hostable data lakehouse for local development and testing of modern data workflows. Run production-grade infrastructure on your laptop with Apache Spark, Iceberg, and Kafka - no cloud account required. Includes a realistic data generation framework to test batch and streaming pipelines. Why Lakehouse at Home? - Learn data engineering with real tools, not toy examples - Develop and test Spark jobs locally before deploying to production - Experiment with Iceberg table formats, streaming pipelines, and medallion architecture - Deploy (optional) to your cloud provider when ready using included Terraform templates Stack Component Version Purpose ----------- --------- --------- Apache Spark 4.0 / 4.1 Distributed compute Apache Iceberg 1.10 ACID table format Apache Kafka 3.6 Event streaming Apache Airflow 3.1 Workflow orchestration PostgreSQL 16 Catalog metadata SeaweedFS - S3-compatible storage Unity Catalog 0.3.1 REST catalog (optional) Requirements Resource Minimum Recommended ---------- --------- ------------- RAM 8 GB 16 GB Disk 20 GB 50 GB CPU 4 cores 8 cores Software : Docker, Java 17+ (21 for Spark 4.1), Python 3.10+, Poetry Getting Started AI-Assisted Setup Using Claude Code, Cursor, or another AI coding assistant? Point it at this repo: The agent will use CLAUDE.md for context and ./lakehouse setup to validate prerequisites. Manual Setup See Installation Guide for detailed OS-specific setup. CLI See CLI Reference for all commands","default_branch":null,"files":null,"tree":[],"storefront":"/r/lisancao","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/lisancao/lakehouse-at-home/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}