{"repo":"VIDA-NYU/magneto-matcher","free":true,"listed":false,"github":"https://github.com/VIDA-NYU/magneto-matcher","clone":"git clone https://github.com/VIDA-NYU/magneto-matcher.git","description":"[VLDB '25] Magneto combines small and large language models to provide cost-effective schema matching.","language":"Jupyter Notebook","stars":19,"topics":["bdf-toolbox","data-integration","schema-matching"],"license":"Apache-2.0","category":"data-pipelines","readme_excerpt":"Magneto: Combining Small and Large Language Models for Schema Matching Welcome to Magneto 🧲 This repository contains the codebase of our paper: Magneto: Combining Small and Large Language Models for Schema Matching (VLDB '25) Magneto is an innovative framework designed to enhance schema matching (SM) by intelligently combining small, pre-trained language models (SLMs) with large language models (LLMs). Our approach is structured to be both cost-effective and broadly applicable. The framework operates in two distinct phases: - Candidate Retrieval : This phase involves using SLMs to quickly identify a manageable subset of potential matches from a vast pool of possibilities. Optional LLM-powered fine-tuning can be performed. - Match Reranking : In this phase, LLMs take over to assess and reorder the candidates, simplifying the process for users to review and select the most suitable matches. Contents Environment Setup Code Structure Example Usage Citations Environment Setup Clone the Repository Install Dependencies Data Preparation The data folder contains the datasets used for data integration tasks. Download the data folder from this Google Drive link and place it in the data directory. Contents include: - gdc : GDC benchmark from the paper. Contains ten tumor analysis study datasets to be matched to Genomics Data Commons (GDC) standards (also available on Zenodo: DOI 10.5281/zenodo.14963587). - Valentine-datasets : Schema matching benchmark from Valentine paper (also availab","default_branch":null,"files":null,"tree":[],"storefront":"/r/VIDA-NYU","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/VIDA-NYU/magneto-matcher/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}