{"repo":"shauryashaurya/learn-data-munging","free":true,"listed":false,"github":"https://github.com/shauryashaurya/learn-data-munging","clone":"git clone https://github.com/shauryashaurya/learn-data-munging.git","description":"Notes on Data Engineering with Pandas, PySpark, Dask, Ray, Arrow DataFusion, Polars etc.","language":"Jupyter Notebook","stars":53,"topics":["dask","data-engineering","jupyter","pandas","pyspark","ray","spark","arrow","dask-distributed","datafusion"],"license":"MIT","category":"data-pipelines","readme_excerpt":"Data Munging Using \\ X\\ in Python, Rust & Julia Data Engineering Workshops on some of the more popular libraries, frameworks and tech circa 2023-2024. Data Engineers working with Python, Rust and Julia :P --- Notebooks 00 Python Collections This set of notebooks works through examples of how some pretty sophisticated data engineering can be done using Python Collections, Itertools and Functools. It uses the small MovieLens dataset. Basic Collections and the Module: Notebook also 01 Numpy NumPy vs Python Collections Notebook also 02 Pandas Wrangling MovieLens with Pandas - Part 1: Getting Started, Load the MovieLens dataset: Notebook also Wrangling MovieLens with Pandas - Part 2: Playing with the Movies and Ratings data: Notebook also 03 Spark 01 - Toy introduction to the basics 01 - Setting up Spark locally (on Windows): Notebook also 02 - How to run Apache Spark based notebooks in Google Colab: Notebook%20Google Colab setup Spark download data.ipynb) also %20Google Colab setup Spark download data.ipynb) 02 - A set of notebooks exploring data wrangling in depth using the MovieLens dataset Part 01: Overview, Starting Spark and Loading the data: Notebook or Part 02: Data Analysis basics using tags.csv from the MovieLens dataset: Notebook or 03 - Harder to solve problems - trying to capture a variety of \"problem types\" at appear hard to solve in the wild 01: Just the questions: Notebook.ipynb) or .ipynb) 02: Added hints, patterns etc. to the questions, read through if you are no","default_branch":null,"files":null,"tree":[],"storefront":"/r/shauryashaurya","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/shauryashaurya/learn-data-munging/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}