{"repo":"vmware/versatile-data-kit","free":true,"listed":false,"github":"https://github.com/vmware/versatile-data-kit","clone":"git clone https://github.com/vmware/versatile-data-kit.git","description":"One framework to develop, deploy and operate data workflows with Python and SQL.","language":"Python","stars":489,"topics":["data-science","data-engineering","sql","trino","data-lineage","etl","elt","data-pipelines","data-engineer","data-warehouse"],"license":"Apache-2.0","category":"data-pipelines","readme_excerpt":"--- One framework to🧑‍💻 Develop ▶️ Deploy and 📊 Operate data workflows with Python and SQL --- 🎯 Write shorter, more readable code. 🔄 Ready-to-use data ETL/ELT patterns. 🧩 Lego-like extensibility. 🚀 Single click deployment. 🛠 Operate and monitor. ️ --- Introduction to the VDK SDK Framework to simplify data ingestion and data processing. Write any code using Python or SQL. A toolset enabling you to run data jobs. Get started with VDK SDK: ➡ Install Quickstart VDK . Only requirement is Python 3.7+. pip install quickstart-vdk vdk --help ➡ Develop your First Data Job if you are impatient to start quickly. Data Ingestion Extract data from various sources (HTTP APIs, Databases, CSV, etc.). Ensure data fidelity with minimal transformations. Load data to your preferred destination (database, cloud storage). Ingestion examples: ➡ Ingesting data from REST API into Database ➡ Ingesting data from DB into Database ➡ Ingesting local CSV file into Database ➡ Incremental ingestion using Job Properties Data Transformation SQL and Python parameterized transformations. Extensible templates for data modeling. Creates a dataset or table as a product. Get started with transforming data: ➡ Data Modeling: Treating Data as a Product ➡ Processing data using SQL and local database ➡ Processing data using Kimball warehousing templates Data Job Deployment (build, deploy, release) VDK Control Service provides REST API for users to crea","default_branch":null,"files":null,"tree":[],"storefront":"/r/vmware","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/vmware/versatile-data-kit/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}