{"repo":"sanyassyed/DataEngineering_SFO_Eviction_Data_ETL_Pipeline","free":true,"listed":false,"github":"https://github.com/sanyassyed/DataEngineering_SFO_Eviction_Data_ETL_Pipeline","clone":"git clone https://github.com/sanyassyed/DataEngineering_SFO_Eviction_Data_ETL_Pipeline.git","description":"Data Engineering & Analysis Project- San Francisco Eviction Data ETL Pipeline An end-to-end batch data pipeline for performing ETL on San Francisco City Eviction Data from DataSF. The project aims to analyze eviction trends and patterns from historical to current data.","language":"Python","stars":10,"topics":["bigquery","data-engineering","data-visualization","etl","gcp","pipeline","pyspark","terraform","data-analysis"],"license":null,"category":"data-pipelines","readme_excerpt":"San Francisco Eviction Data ELT - A Data Engineering Zoomcamp Capstone Project --- The goal of this project is to build an end-to-end batch data pipeline to perform ELT (Extract Load Transform) on San Francisco City Eviction Data available at DataSF. And perform ELT monthly in order to analyse the eviction patterns from historical data to till date. Table of contents - Problem statement - Dataset - Proposed Solution - Data Pipeline Overview - Technologies - Architecture - ELT Steps - The Dashboard - Key Findings - Reproduction - Conclusion - Future Work - References & Resources Problem statement --- ### Dataset : The Dataset selected for this project is the Eviction Notice Dataset of San Francisco obtained from DataSF. This data includes eviction notices filed with the San Francisco Rent Board per San Francisco Administrative Code 37.9(c). Notices are published since January 1, 1997. The publishing/update frequency of the dataset is monthly . The Data is extracted via Socrata Open Data API (SODA). The columns in the Dataset and their description is available here ### Proposed Solution : This project aims at extracting this data from the source via API and building a BATCH ELT which will be scheduled to run monthly and update the connected Dashbord for monthly Analytics & Reporting. Data Pipeline Overview --- This is a Batch Pipeline which will perform ELT on the 1st of every month at 00:05 am (as source data is updated monthly at the end of each month). The ELT steps include:","default_branch":null,"files":null,"tree":[],"storefront":"/r/sanyassyed","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/sanyassyed/DataEngineering_SFO_Eviction_Data_ETL_Pipeline/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}