{"repo":"whythawk/whyqd","free":true,"listed":false,"github":"https://github.com/whythawk/whyqd","clone":"git clone https://github.com/whythawk/whyqd.git","description":"data wrangling simplicity, complete audit transparency, and at speed","language":"Python","stars":35,"topics":["data-science","data-analysis","data-wrangling","munging","data-management","open-data","open-science","python","pandas","crosswalks"],"license":"BSD-3-Clause","category":"databases-storage","readme_excerpt":"whyqd: simplicity, transparency, speed What is it? More research, less wrangling whyqd (/wɪkɪd/) is a curatorial toolkit intended to produce well-structured and predictable data for research analysis. It provides an intuitive method for creating schema-to-schema crosswalks for restructuring messy data to conform to a standardised metadata schema. It supports rapid and continuous transformation of messy data using a simple series of steps. Once complete, you can import wrangled data into more complex analytical or database systems. whyqd plays well with your existing Python-based data-analytical tools. It uses Ray and Modin as a drop-in replacement for Pandas to support processing of large datasets, and Pydantic for data models. Each definition is saved as JSON Schema-compliant file. This permits others to read and scrutinise your approach, validate your methodology, or even use your crosswalks to import and transform data in production. Once complete, a transform file can be shared, along with your input data, and anyone can import and validate your crosswalk to verify that your output data is the product of these inputs. Why use it? whyqd allows you to get to work without requiring you to achieve buy-in from anyone or change your existing code. If you don't want to spend days or weeks slogging through data when all you want to do is test whether your source data are even useful. If you already have a workflow and established software which includes Python and pandas, and don","default_branch":null,"files":null,"tree":[],"storefront":"/r/whythawk","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/whythawk/whyqd/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}