{"repo":"ccao-data/data-architecture","free":true,"listed":false,"github":"https://github.com/ccao-data/data-architecture","clone":"git clone https://github.com/ccao-data/data-architecture.git","description":"Codebase for CCAO data infrastructure construction and management","language":"R","stars":12,"topics":["aws","aws-athena","aws-s3","data-architecture","data-engineering"],"license":null,"category":"data-pipelines","readme_excerpt":"CCAO Data Infrastructure This repository stores the code for the CCAO Data Department's ETL pipelines and data lakehouse. This infrastructure supports the Data Team's modeling, reporting, and data integrity work. Quick Links - :file folder: dbt Data Catalog - Documentation for all CCAO data lakehouse tables and views - :nut and bolt: dbt README - How to develop CCAO data infrastructure using dbt - :test tube: dbt Tests and QC Reports - How to add and run data tests, unit tests, and QC reports using dbt - :pencil: dbt Generic Test Documentation - Definitions for CCAO generic dbt tests, which are functions that we use to define our QC tests Repository Structure - ./dbt contains the models and tests that build our Athena data lakehouse; dbt mainly acts as a transformation and documentation layer on top of our raw data - ./docs contains design documents and other supplemental documentation - ./etl contains ETL scripts used to load raw and slightly cleaned up data into the lakehouse as dbt sources - ./socrata contains column transformations for the CCAO's Open Data Portal assets","default_branch":null,"files":null,"tree":[],"storefront":"/r/ccao-data","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/ccao-data/data-architecture/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}