{"repo":"outpoot/vyntr","free":true,"listed":false,"github":"https://github.com/outpoot/vyntr","clone":"git clone https://github.com/outpoot/vyntr.git","description":"Independent search engine. Includes web crawling, search indexing, dictionary API, and more. https://vyntr.com","language":"TypeScript","stars":364,"topics":["crawler","duckduckgo","engine","google","python","rust","search","tantivy","web"],"license":null,"category":"scrapers-browser-automation","readme_excerpt":"Vyntr.com - the independent search engine. Privacy Policy Terms of Service License YouTube video Vyntr is a search engine project with multiple components: Components - Genesis - Web crawler and content analyzer - Pulse - Search indexing system using Tantivy - Lexicon - WordNet-based dictionary lookup service - Website - Frontend interface at vyntr.com Setup 1. Create a .env file in the root directory: 2. Set up the database: 3. Set up individual components: - Genesis crawler: Follow genesis setup - Lexicon service: Follow lexicon setup - Website: Follow website setup Pipeline 1. Genesis crawler collects and analyzes web pages 2. Data is stored in partitioned JSONL files in S3 3. Content is cleaned through dataset . 3. Content is processed through embedding tools (vector), or Pulse (full-text). 4. Website frontend provides search interface. Requirements - Python with uv package manager - Node.js - PostgreSQL with pgvector - Docker - Bun runtime (for Lexicon service) - Rust toolchain Dataset The Vyntr dataset is not publicly available. For licensing inquiries, please contact contact@outpoot.com. You may also use the official API provided at https://vyntr.com/api. License This project is licensed under the Creative Commons Attribution-NonCommercial 4.0 International License ( CC BY-NC 4.0) . See the LICENSE file for details. Individual components may have additional licensing requirements. See their respective directories for specific licensing information. WordNet data used in","default_branch":null,"files":null,"tree":[],"storefront":"/r/outpoot","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/outpoot/vyntr/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}