{"repo":"upstash/wikipedia-semantic-search","free":true,"listed":false,"github":"https://github.com/upstash/wikipedia-semantic-search","clone":"git clone https://github.com/upstash/wikipedia-semantic-search.git","description":"Semantic Search on Wikipedia with Upstash Vector","language":"TypeScript","stars":471,"topics":["ai","search","semantic","vector","vector-database"],"license":"MIT","category":"databases-storage","readme_excerpt":"Indexing Millions of Wikipedia Articles With Upstash Vector This repository contains the code and documentation for our project on indexing millions of Wikipedia articles using Upstash Vector, as described in our blog post. Project Overview We've created a semantic search engine and Upstash RAG Chat SDK using Wikipedia data to demonstrate the capabilities of Upstash Vector and RAG Chat SDK. The project involves: 1. Preparing and embedding Wikipedia articles 2. Indexing the vectors using Upstash Vector 3. Building a Wikipedia semantic search engine 4. Implementing a RAG chatbot Key Features - Indexed over 144 million vectors from Wikipedia articles in 11 languages - Used BGE-M3 embedding model for multilingual support - Implemented semantic search with cross-lingual capabilities - Created a RAG chatbot using Upstash RAG Chat SDK Technologies Used - Upstash Vector: For storing and querying vector embeddings - Upstash Redis: For storing chat sessions - Upstash RAG Chat SDK: For building the RAG Chat application - SentenceTransformers: For generating embeddings - Meta-Llama-3-8B-Instruct: As the LLM provider through QStash LLM APIs Development To run the project locally, follow these steps: 1. Go to Upstash Console to manage your databases: - Create a new Vector database with embedding model support. You can choose the BGE-M3 model for multilingual support. - Create a new Redis database for storing chat sessions. - Copy the credentials for both Redis and Vector. Also copy the QSt","default_branch":null,"files":null,"tree":[],"storefront":"/r/upstash","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/upstash/wikipedia-semantic-search/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}