{"repo":"KaterynaD/dbt_scd2_plus","free":true,"listed":false,"github":"https://github.com/KaterynaD/dbt_scd2_plus","clone":"git clone https://github.com/KaterynaD/dbt_scd2_plus.git","description":"Slowly Changing Dimension Type 2 (scd2) custom materialization","language":"PowerShell","stars":11,"topics":["dbt","scd2"],"license":"Apache-2.0","category":"data-pipelines","readme_excerpt":"Slowly Changing Dimension Type 2 (scd2) Custom Materialization dbt Package What does this dbt package do? This DBT package provides a materialization that builds advanced version of slowly changing dimension type 2 (SCD2): - A new record is added if there is a change in check cols column list like it's done in the original Check dbt snapshot strategy. - It uses updated at column like in the original Timestamp dbt snapshot strategy to define the time limits when a record is valid (valid from - valid to columns). - You can load data in a batch like one-time, historical, initial load. The batch may contain several versions of the same entity or even duplicates (the same unique key and updated at ). There is a deduplication embedded in the logic. - If there is not a complete duplicate record (the same unique key and updated at but different check cols ), the logic can use loaded at (data loaded in a staging area timestamp) to update check cols with most recent known values. - The dimension is loaded incrementally. It means, if a target table does not exist, it's created from the first data batch/row. Otherwise, new records are inserted and existing updated if needed. - The load process supports out of order transactions. It means, if there is already an entity version in the dimension for a specific time period and later, you receive in your staging area, a new version of the same entity for the part of the existing time period (change in a past, already loaded data), the existin","default_branch":null,"files":null,"tree":[],"storefront":"/r/KaterynaD","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/KaterynaD/dbt_scd2_plus/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}