{"repo":"newfront/hitchhikers_guide_to_deltalake_streaming","free":true,"listed":false,"github":"https://github.com/newfront/hitchhikers_guide_to_deltalake_streaming","clone":"git clone https://github.com/newfront/hitchhikers_guide_to_deltalake_streaming.git","description":"Don't Panic. This guide will help you when it feels like the end of the world.","language":"Jupyter Notebook","stars":32,"topics":["apache","apache-spark","deltalake","hitchhikers-guide"],"license":"Apache-2.0","category":"data-pipelines","readme_excerpt":"The Hitchhiker's Guide to Delta Lake Streaming Data+AI Summit Session DON'T PANIC This is a collection (growing hopefully as time goes on) providing tips and tricks to ensure your experience building and maintaining Streaming Delta Lake applications (and the tables that power them) is absolutely joyful, even when the shit hits the fan :). Remember, when all else fails to take a deep breath, count to 5, and dig into the following content. Using the Guide Take a look at the outline provided in /hitchhikers guide/README.md to learn how to use the Guide in your Delta Lake streaming adventures. Getting up and Running Note: For x86 64 (linux/amd64) or (linux/arm64) use the common docker-compose.yaml Docker Image: This is using the newfrontdocker/delta-docker:4.0.0 image. This will be replaced with the official delta-docker image after https://github.com/delta-io/delta-docker/pull/17 is merged and the image is pushed. cd hitchhikers guide && docker compose up Note: The docker-compose.yaml includes settings to clamp the amount of local resources available for data processing (upper limit: 4 cpu cores, 16GB ram). If you want to increase the number of cores to reflect your laptop,desktop,server then remember to save at least 1 cpu core and at least 1GB ram for the root OS. Then head into the Jupyter Lab environment and start exploring. The main content will be located in http://127.0.0.1:8888/lab/tree/hitchhikers guide Datasets ECommerce Data - the dataset used in the hitchhikers guide","default_branch":null,"files":null,"tree":[],"storefront":"/r/newfront","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/newfront/hitchhikers_guide_to_deltalake_streaming/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}