{"repo":"wzdnzd/harvester","free":true,"listed":false,"github":"https://github.com/wzdnzd/harvester","clone":"git clone https://github.com/wzdnzd/harvester.git","description":"Intelligent data acquisition framework for GitHub and web sources","language":"Python","stars":603,"topics":["ai","anthropic","deepseek","gemini","openai","qwen"],"license":null,"category":"ai-agents","readme_excerpt":"Harvester - Universal Data Acquisition Framework 📖 中文文档 English 🔗 More Tools A universal, adaptive data acquisition framework designed for comprehensive information acquisition from multiple sources including GitHub, network mapping platforms (FOFA, Shodan), and arbitrary web endpoints. While the current implementation focuses on AI service provider key discovery as a practical example, the framework is architected for extensibility to support diverse data acquisition scenarios. --- ⭐⭐⭐ If this project helps you, please give it a star! Your support motivates us to keep improving and adding new features. --- Table of Contents - Key Features - Quick Start - Architecture - Directory Structure - Troubleshooting - Contributing Project Goals The system aims to build a universal data acquisition framework primarily targeting: - GitHub : Code repositories, issues, commits, and API endpoints - Network Mapping Platforms : - FOFA - Cyberspace mapping and asset discovery - Shodan - Internet-connected device search engine - Arbitrary Web Endpoints : Custom APIs, web services, and data sources - Extensible Architecture : Plugin-based system for easy integration of new data sources Current Data Source Support Data Source Status Description ----------- ------------- --------------------------------------- GitHub API ✅ Implemented Full API integration with rate limiting GitHub Web ✅ Implemented Web scraping with intelligent parsing FOFA 🚧 Planned Cyberspace asset discovery integration Shod","default_branch":null,"files":null,"tree":[],"storefront":"/r/wzdnzd","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/wzdnzd/harvester/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}