{"repo":"biolds/sosse","free":true,"listed":false,"github":"https://github.com/biolds/sosse","clone":"git clone https://github.com/biolds/sosse.git","description":"Selenium Open Source Search Engine & crawler","language":"Python","stars":409,"topics":["crawler","search","search-engine","automation","python","scrapping"],"license":"AGPL-3.0","category":"scrapers-browser-automation","readme_excerpt":"Sosse 🦦 Discover Sosse — the Selenium Open Source Search Engine built for powerful web archiving, crawling, and search. Explore all its features and capabilities on the official website. Whether you're a developer, researcher, or data enthusiast, Sosse is ready to support your projects. Join the community on GitHub or GitLab to submit feature requests, report bugs, contribute code, or start a discussion. Key Features - 🌍 Web Page Search : Search the content of web pages, including dynamically rendered ones, with advanced queries. (doc) - 🕑 Recurring Crawling : Crawl pages at fixed intervals or adapt the rate based on content changes. (doc) - 🔖 Web Page Archiving : Archive HTML content, adjust links for local use, download required assets, and support dynamic content. (doc) - 🏷️ Tags : Organize and filter crawled or archived pages using tags for better search and management. (doc) - 📂 File Downloads : Batch download binary files from web pages. (doc) - 📡 Webhooks : Integrate with external services using highly flexible webhooks. Connect to proprietary AI platforms (doc) or locally hosted solutions (doc) to enable advanced data extraction, summarization, auto-tagging, notifications, and more. - 🔔 Atom Feeds : Generate content feeds for websites that don’t have them, or receive updates when a new page containing a keyword is published. (doc) - 🔒 Authentication : The crawler can authenticate to access private pages and retrieve content. (doc) - 👥 Permissions : Admins ca","default_branch":null,"files":null,"tree":[],"storefront":"/r/biolds","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/biolds/sosse/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}