{"repo":"0xMassi/webclaw","free":true,"listed":false,"github":"https://github.com/0xMassi/webclaw","clone":"git clone https://github.com/0xMassi/webclaw.git","description":"Fast, local-first web content extraction for LLMs. Scrape, crawl, extract structured data — all from Rust. CLI, REST API, and MCP server.","language":"Rust","stars":2276,"topics":["ai-agents","cli","llm","markdown","mcp","rust","self-hosted","web-extraction","web-scraping","web-crawler"],"license":"AGPL-3.0","category":"scrapers-browser-automation","readme_excerpt":"English 简体中文 webclaw Turn websites into clean markdown, JSON, and LLM-ready context. CLI, MCP server, REST API, and SDKs for AI agents and RAG pipelines. --- Most web scraping tools give your agent one of two bad outputs: - a blocked page, login wall, or empty app shell - raw HTML full of nav, scripts, styling, ads, and duplicated boilerplate webclaw.io is the hosted web extraction API for webclaw. This repo contains the open-source CLI, MCP server, extraction engine, and self-hostable server. webclaw turns a URL into clean content your tools can actually use. Use it from the terminal, wire it into Claude/Cursor through MCP, call the hosted API from your app, or self-host the OSS server. --- Install Agent setup The fastest way to connect webclaw to Claude Code, Claude Desktop, Cursor, Windsurf, OpenCode, Codex CLI, and other MCP-compatible tools: The installer detects supported clients and configures the MCP server for you. Homebrew Prebuilt binaries Download macOS, Linux, and Windows binaries from GitHub Releases. Docker Cargo If building from source fails because native build tools are missing, install the platform prerequisites: OS Command --- --- Debian / Ubuntu sudo apt install -y pkg-config libssl-dev cmake clang git build-essential Fedora / RHEL sudo dnf install -y pkg-config openssl-devel cmake clang git make gcc Arch sudo pacman -S pkg-config openssl cmake clang git base-devel macOS xcode-select --install --- Quick Start Scrape one page Return LLM-optimized text Keep","default_branch":null,"files":null,"tree":[],"storefront":"/r/0xMassi","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/0xMassi/webclaw/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}