{"repo":"wbsg-uni-mannheim/PyDI","free":true,"listed":false,"github":"https://github.com/wbsg-uni-mannheim/PyDI","clone":"git clone https://github.com/wbsg-uni-mannheim/PyDI.git","description":"The PyDI framework provides methods for end-to-end data integration. The framework covers all steps of the integration process, including schema matching, data translation, entity matching, and data fusion. The framework offers traditional string-based methods as well as modern LLM- and embedding-based techniques for these tasks.","language":"HTML","stars":20,"topics":["data-fusion","data-integration","data-normalization","entity-matching","information-extraction","data-translation","schema-matching","entity-resolution"],"license":"Apache-2.0","category":"data-pipelines","readme_excerpt":"PyDI - Python Data Integration Framework The PyDI framework provides methods for end-to-end data integration. The framework covers all steps of the integration process, including schema matching, data translation, entity matching, and data fusion. The framework offers both traditional string-based methods as well as modern LLM- and embedding-based techniques for these tasks. PyDI is designed as a set of independent, composable modules that operate on pandas DataFrames as the underlying data structure, ensuring interoperability with third-party packages that rely on pandas. This page provides an overview of the PyDI framework. Further details about the functionality of the framework are found in the Wiki. In order to learn how to use the framework, please read the Tutorials or have a look at the Use Cases which illustrate how PyDI is used for end-to-end data integration. Installing PyDI You can install PyDI via pip: Functionality The PyDI framework covers all steps of the data integration process, including data loading, schema matching, data translation, entity matching, and data fusion. This section gives an overview of the functionality and the alternative methods that are provided for each of these steps. Schema Matching : Schema matching identifies attributes in multiple schemata that have the same meaning. PyDI provides four schema matching methods which either rely on attribute labels or data values, or exploit an existing mapping of records in order to find attribute c","default_branch":null,"files":null,"tree":[],"storefront":"/r/wbsg-uni-mannheim","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/wbsg-uni-mannheim/PyDI/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}