{"repo":"athrael-soju/Snappy","free":true,"listed":false,"github":"https://github.com/athrael-soju/Snappy","clone":"git clone https://github.com/athrael-soju/Snappy.git","description":"🐊 Snappy's unique approach unifies vision-language late interaction with structured OCR for region-level knowledge retrieval. Like the project? Drop a star! ⭐","language":"Python","stars":90,"topics":["computer-vision","multimodal-ai","multivector-search","python","rag","typescript","vector-database","vector-search","vision-ai","document-retrieval"],"license":"MIT","category":"machine-learning","readme_excerpt":"Snappy - Spatially-Grounded Document Retrieval via Patch-to-Region Relevance Propagation Read the Full Paper on arxiv.org : Spatially-Grounded Document Retrieval via Patch-to-Region Relevance Propagation Snappy implements region-level document retrieval by unifying vision-language models with OCR through spatial coordinate mapping. Unlike traditional systems that return entire pages (VLMs) or lack semantic grounding (OCR-only), Snappy uses ColPali's patch-level similarity scores as spatial relevance filters over OCR-extracted regions; operating entirely at inference time without additional training. Motivation Vision-language models like ColPali achieve state-of-the-art document retrieval by embedding pages as images with fine-grained patch representations. However, they return entire pages as retrieval units, introducing irrelevant content into RAG context windows. Conversely, OCR systems extract structured text with bounding boxes but cannot assess which regions are relevant to a query. Snappy bridges these paradigms through patch-to-region relevance propagation . The approach formalizes coordinate mapping between vision transformer patch grids (32×32) and OCR bounding boxes, repurposing ColPali's late interaction mechanism to generate interpretability maps. Patch similarity scores propagate to OCR regions via IoU-weighted intersection, enabling two-stage retrieval: efficient candidate retrieval using mean-pooled embeddings, followed by full-resolution region reranking. Thi","default_branch":null,"files":null,"tree":[],"storefront":"/r/athrael-soju","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/athrael-soju/Snappy/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}