{"repo":"capitalone/DataProfiler","free":true,"listed":false,"github":"https://github.com/capitalone/DataProfiler","clone":"git clone https://github.com/capitalone/DataProfiler.git","description":"What's in your data? Extract schema, statistics and entities from datasets","language":"Python","stars":1574,"topics":["python","privacy","pii","npi","nlp","data-science","gdpr","data-analysis","data-labels","avro"],"license":"Apache-2.0","category":"machine-learning","readme_excerpt":"Data Profiler What's in your data? The DataProfiler is a Python library designed to make data analysis, monitoring, and sensitive data detection easy. Loading Data with a single command, the library automatically formats & loads files into a DataFrame. Profiling the Data, the library identifies the schema, statistics, entities (PII / NPI) and more. Data Profiles can then be used in downstream applications or reports. Getting started only takes a few lines of code (example csv): Note: The Data Profiler comes with a pre-trained deep learning model, used to efficiently identify sensitive data (PII / NPI). If desired, it's easy to add new entities to the existing pre-trained model or insert an entire new pipeline for entity recognition. For API documentation, visit the documentation page. If you have suggestions or find a bug, please open an issue. If you want to contribute, visit the contributing page. ------------------ Install To install the full package from pypi : pip install DataProfiler[full] If you want to install the ml dependencies without generating reports use DataProfiler[ml] If the ML requirements are too strict (say, you don't want to install tensorflow), you can install a slimmer package with DataProfiler[reports] . The slimmer package disables the default sensitive data detection / entity recognition (labler) Install from pypi: pip install DataProfiler ------------------ What is a Data Profile? In the case of this library, a data profile is a dictionary containin","default_branch":null,"files":null,"tree":[],"storefront":"/r/capitalone","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/capitalone/DataProfiler/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}