{"repo":"abcsys/libem","free":true,"listed":false,"github":"https://github.com/abcsys/libem","clone":"git clone https://github.com/abcsys/libem.git","description":"Compound AI toolchain for fast and accurate entity matching, powered by LLMs.","language":"Python","stars":28,"topics":["entity-matching","entity-resolution","llm","python","compound-ai-systems","data-matching","large-language-models","data-integration","openai","ai"],"license":"Apache-2.0","category":"ai-agents","readme_excerpt":"Scalable entity matching with human-level accuracy, powered by LLMs and tooling. [Jul'24] Libem Arena: Online Evaluation and Leaderboard for EM [Jun'24] Liberal Entity Matching as a Compound AI Toolchain [Jun'24] Presented at Compound AI Systems Workshop - Data + AI Summit 2024 Libem is an open-source, compound AI toolchain designed to perform and streamline entity matching (EM). EM involves identifying whether two descriptions refer to the same entity, a task crucial in data management and integration. Traditional EM methods have evolved from rule-based to LLM-based systems, yet they fall short due to their reliance on static knowledge and rigid, predefined prompts. Libem addresses these limitations by adopting a modular, tool-oriented approach. It supports dynamic tool use, self-refinement, and optimization, allowing it to adapt and refine its processes based on the dataset and performance metrics. Unlike existing LLM-based EM systems, which are usually in the form of Python notebooks, Libem offers a composable and reusable toolchain that can be easily incorporated into applications or used as a service via APIs. Specifically, Libem can be used as a library or a CLI tool and can be configured to use different models, tools, and parameters. Libem supports a variety of models, including GPT-4o, GPT-4o-mini, GPT-4, GPT-3.5-turbo, and Llama3 as well as tools to facilitate entity matching, such as browsing and data preparation. --- Table of Contents - Installation - Libem Usage ","default_branch":null,"files":null,"tree":[],"storefront":"/r/abcsys","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/abcsys/libem/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}