{"repo":"treeverse/lakeFS","free":true,"listed":false,"github":"https://github.com/treeverse/lakeFS","clone":"git clone https://github.com/treeverse/lakeFS.git","description":"lakeFS - Data version control for your data lake | Git for data","language":"Go","stars":5487,"topics":["data-engineering","data-versioning","go","object-storage","data-lake","aws-s3","data-quality","azure-blob-storage","google-cloud-storage","golang"],"license":"Apache-2.0","category":"databases-storage","readme_excerpt":"lakeFS is Data Version Control (Git for Data) lakeFS is an open-source tool that transforms your object storage into a Git-like repository. It enables you to manage your data lake the way you manage your code. With lakeFS you can build repeatable, atomic, and versioned data lake operations - from complex ETL jobs to data science and analytics. lakeFS supports AWS S3, Azure Blob Storage, and Google Cloud Storage as its underlying storage service. It is API compatible with S3 and works seamlessly with all modern data frameworks such as Spark, Hive, AWS Athena, DuckDB, and Presto. For more information, see the documentation. Getting Started You can spin up a standalone sandbox instance of lakeFS: Once you've got lakeFS running, open http://127.0.0.1:8000/ in your web browser. Getting Started with Docker Alternatively, you can run lakeFS a server using Docker: Quickstart 👉🏻 For a hands-on walk through of the core functionality in lakeFS head over to the quickstart to jump right in! Make sure to also have a look at the lakeFS samples. These are a rich resource of examples of end-to-end applications that you can build with lakeFS. Why Do I Need lakeFS? ETL Testing with Isolated Dev/Test Environment When working with a data lake, it’s useful to have replicas of your production environment. These replicas allow you to test these ETLs and understand changes to your data without impacting downstream data consumers. Running ETL and transformation jobs directly in production without pr","default_branch":null,"files":null,"tree":[],"storefront":"/r/treeverse","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/treeverse/lakeFS/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}