{"repo":"coder-hxl/x-crawl","free":true,"listed":false,"github":"https://github.com/coder-hxl/x-crawl","clone":"git clone https://github.com/coder-hxl/x-crawl.git","description":"Flexible Node.js AI-assisted crawler library","language":"TypeScript","stars":1877,"topics":["crawl","crawler","nodejs","typescript","spider","flexible","puppeteer","javascript","multifunction","chromium"],"license":"MIT","category":"scrapers-browser-automation","readme_excerpt":"x-crawl · English 简体中文 x-crawl is a flexible Node.js AI-assisted crawler library. Flexible usage and powerful AI assistance functions make crawler work more efficient, intelligent and convenient. It consists of two parts: - Crawler: It consists of a crawler API and various functions that can work normally even without relying on AI. - AI: Integrate ollama and openai, AI simplifies many tedious operations. If you find x-crawl helpful, or you like x-crawl, you can give x-crawl repository a like on GitHub A star. Your support is the driving force for our continuous improvement! thank you for your support! Sponsors IPWO × x-crawl 当项目需要采集数据时，稳定的网络环境是刚需。IPWO 提供全球住宅代理资源，帮助开发者提升数据采集的稳定性与灵活性。 提供动态、静态双类型 IP，支持http/https/socks5多协议， 免费测试入口 。折扣码“IPWO”。 Features - 🤖 AI Assistance - Integrate ollama and openai, powerful AI assistance function makes crawler work more efficient, intelligent and convenient. - 🖋️ Flexible writing - A single crawling API is suitable for multiple configurations, and each configuration method has its own advantages. - ⚙️Multiple uses - Supports crawling dynamic pages, static pages, interface data and file data. - ⚒️ Control page - Crawling dynamic pages supports automated operations, keyboard input, event operations, etc. - 👀 Device Fingerprinting - Zero configuration or custom configuration to avoid fingerprint recognition to identify and track us from different locations. - 🔥 Asynchronous Sync - Asynchronous or synchronous crawling mode without switching cra","default_branch":null,"files":null,"tree":[],"storefront":"/r/coder-hxl","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/coder-hxl/x-crawl/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}