{"repo":"janreges/siteone-crawler","free":true,"listed":false,"github":"https://github.com/janreges/siteone-crawler","clone":"git clone https://github.com/janreges/siteone-crawler.git","description":"SiteOne Crawler is a cross-platform website crawler and analyzer for SEO, security, accessibility, and performance optimization—ideal for developers, DevOps, QA engineers, and consultants. Supports Windows, macOS, and Linux (x64 and arm64).","language":"Rust","stars":860,"topics":["analyzer","crawler","crawling","seo","seotools","stress-testing","swoole","website","performance","testing"],"license":"MIT","category":"scrapers-browser-automation","readme_excerpt":"SiteOne Crawler SiteOne Crawler is a powerful and easy-to-use website analyzer, cloner, and converter designed for developers seeking security and performance insights, SEO specialists identifying optimization opportunities, and website owners needing reliable backups and offline versions. Now rewritten in Rust for maximum performance, minimal resource usage, and zero runtime dependencies. The transition from PHP+Swoole to Rust resulted in 25% faster execution and 30% lower memory consumption while producing identical output. Discover the SiteOne Crawler advantage: Run Anywhere: Single native binary for 🪟 Windows , 🍎 macOS , and 🐧 Linux (x64 & arm64). No runtime dependencies. Work Your Way: Launch the binary without arguments for an interactive wizard 🧙 with 10 preset modes, use the extensive command-line interface 📟 (releases, ▶️ video) for automation and power, or enjoy the intuitive desktop GUI application 💻 (GUI app, ▶️ video) for visual control. Rich Output Formats: Interactive HTML audit report 📊 with sortable tables and quality scoring (0.0-10.0) (see nextjs.org sample), detailed JSON for programmatic consumption, and human-readable text for terminal. Send HTML reports directly to your inbox via built-in SMTP mailer 📧. CI/CD Integration: Built-in quality gate ( --ci ) with configurable thresholds — exit code 10 on failure enables automated deployment blocking. Also useful for cache warming — crawling the entire site after deployment populates your reverse proxy","default_branch":null,"files":null,"tree":[],"storefront":"/r/janreges","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/janreges/siteone-crawler/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}