{"repo":"apicrafter/datacrafter","free":true,"listed":false,"github":"https://github.com/apicrafter/datacrafter","clone":"git clone https://github.com/apicrafter/datacrafter.git","description":"NoSQL extract, transform, load (ETL) toolkit with Python","language":"Python","stars":16,"topics":["data-pipelines","etl","data-engineering"],"license":"Apache-2.0","category":"data-pipelines","readme_excerpt":"Datacrafter - NoSQL ETL Tool Datacrafter is an open-source NoSQL ETL (Extract, Transform, Load) tool designed for data extraction, transformation, and loading with a focus on NoSQL data formats. It provides a command-line interface for building data pipelines that extract data from various sources, process it, and load it into different destinations. Note: This project is in alpha stage. Code migration from a closed repository is in progress, and documentation is being continuously improved. Features - NoSQL-first : JSON Lines and BSON are the native intermediate formats - CLI-first YAML projects : declare extract → process → load in datacrafter.yml - File and URL extraction : CSV, JSON, JSONL, XML, XLS/XLSX, ZIP+XML, patterned HTML indexes, RSS/Atom, DCAT catalogs, APIBackuper, trusted Python collect() scripts - Record transforms : keymap , typemap , custom Python process(record) , plus optional autotype (sample-based type inference) and autoid (stable id ) - Inspect : datacrafter schema and datacrafter metrics read JSONL in output/ (or current/ ) - Dry-run : datacrafter run --dry-run validates config and prints a plan without downloading or writing - Destinations : JSONL, BSON, CSV, Parquet (optional pyarrow ), MongoDB, ArangoDB, CouchDB, Meilisearch - Open-data packaging : datapackage.json beside file output; ${MONGO URI} / ${VAR:-default} in YAML - Multiple extractors : extractors: list sharing one processor and destination Reserved in CLI but not implemented yet : builds","default_branch":null,"files":null,"tree":[],"storefront":"/r/apicrafter","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/apicrafter/datacrafter/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}