{"repo":"brettdavies/crawl4ai-skill","free":true,"listed":false,"github":"https://github.com/brettdavies/crawl4ai-skill","clone":"git clone https://github.com/brettdavies/crawl4ai-skill.git","description":"Scrape JavaScript-heavy sites and extract structured data via reusable CSS schemas. Portable agent skill wrapping the Crawl4AI CLI and Python SDK.","language":"Python","stars":46,"topics":["ai-agents","ai-skills","anthropic","claude","claude-code","claude-skills","crawl4ai","data-extraction","web-crawling","web-scraping"],"license":null,"category":"scrapers-browser-automation","readme_excerpt":"Crawl4AI Agent Skill Scrape JavaScript-heavy sites and extract structured data via reusable CSS schemas. A portable agent skill that wraps the Crawl4AI CLI and Python SDK, written in the Anthropic SKILL.md format and consumable by any agent host that loads SKILL.md-format bundles (Claude Code, Codex, Cursor, OpenCode, Cline, and others). Verified against Crawl4AI library version 0.8.9 (pinned in VERSION ). Features - JS-aware crawling : full headless-browser rendering with wait until=networkidle defaults - Schema-based extraction : derive a CSS selector schema once via LLM, apply it forever with no further LLM cost - LLM extraction : per-request structured extraction when a schema is not worth deriving - Content filtering : BM25 relevance filter and quality-based pruning, plain markdown or markdown-fit output - Concurrent batch crawling : multi-URL processing with per-job concurrency caps - Session management : persistent sessions for authenticated, multi-step flows - CLI and SDK : both the crwl command-line tool and the crawl4ai Python SDK Installation Clone the repo into the skills directory your agent host loads from: For other agent hosts (Codex, Cursor, OpenCode, Cline, custom agents), clone into whichever directory your host scans for SKILL.md-format bundles. Refer to your host's documentation for the skills directory location. The bundle root contains SKILL.md , so the skill registers automatically once the directory is on the host's skills search path. Prerequisites T","default_branch":null,"files":null,"tree":[],"storefront":"/r/brettdavies","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/brettdavies/crawl4ai-skill/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}