{"repo":"D4Vinci/Scrapling","free":true,"listed":false,"github":"https://github.com/D4Vinci/Scrapling","clone":"git clone https://github.com/D4Vinci/Scrapling.git","description":"🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!","language":"Python","stars":74528,"topics":["ai","ai-scraping","automation","crawler","crawling","crawling-python","data","data-extraction","mcp","mcp-server","playwright","python","scraping","selectors","stealth","web-scraper","web-scraping","web-scraping-python","webscraping","xpath"],"license":"BSD-3-Clause","category":"data_extraction","readme_excerpt":"\n\n

\n \n \n \n \"Scrapling\n \n \n
\n Effortless Web Scraping for the Modern Web\n

\n\n

\n \"D4Vinci%2FScrapling\n
\n العربيه | Español | Português (Brasil) | Français | Deutsch | 简体中文 | 日本語 | Русский | 한국어\n
\n \n \"Tests\"\n \n \"PyPI\n \n \"Docker\n \"PyPI\n \n \"Static\n \n \"OpenClaw\n
\n \n \"Discord\"\n \n \n \"X\n \n
\n \n \"Supported\n

\n\n

\n Selection methods\n ·\n Fetchers\n ·\n Spiders\n ·\n Proxy Rotation\n ·\n CLI\n ·\n MCP\n

\n\nScrapling is an adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl.\n\nIts parser learns from website changes and automatically relocates your elements when pages update. Its fetchers bypass anti-bot systems like Cloudflare Turnstile out of the box. And its spider framework lets you scale up to concurrent, multi-session crawls with pause/resume and automatic proxy rotation - all in a few lines of Python. One library, zero compromises.\n\nBlazing fast crawls with real-time stats and streaming. Built by Web Scrapers for Web Scrapers and regular users, there's something for everyone.\n\n```python\nfrom scrapling.fetchers import Fetcher, AsyncFetcher, StealthyFetcher, DynamicFetcher\nStealthyFetcher.adaptive = True\np = StealthyFetcher.fetch('https://example.com', headless=True, network_idle=True) # Fetch website under the radar!\nproducts = p.css('.product', auto_save=True) # Scrape data that survives website design changes!\nproducts = p.css('.product', adaptive=True) # Later, if the website structure changes, pass `adaptive=True` to find them!\n```\nOr scale up to full crawls\n```python\nfrom scrapling.spiders import Spider, Response\n\nclass MySpider(Spider):\n name = \"demo\"\n start_urls = [\"https://example.com/\"]\n\n async def parse(self, response: Response):\n for item in response.css('.product'):\n yield {\"title\": item.css('h2::text').get()}\n\nMySpider().start()\n```\n\n

\n \n \"At\n \n

\n\n# Platinum Sponsors\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
\n \n \n \n \n NodeMaven - reliable proxy provider with the highest quality IP on the market. Use promo code SCRAPLING35 for 35% discount on proxies.\n
\n \n \n \n Proxidize provides mobile and residential proxies for scraping, browser automation, SEO monitoring, AI agents, and data collection. Use code scrapling20 for 20% off.\n
\n \n \n \n ColdProxy provides residential and datacenter proxies for stable web scraping, public data collection, and geo-targeted testing across 195+ countries.\n
\n \n \n \n Scrapling handles Cloudflare Turnstile. For enterprise-grade protection, \n Hyper Solutions\n provides API endpoints that generate valid antibot tokens for Akamai, DataDome, Kasada, and Incapsula. Simple API calls, no browser automation required.
\n \n \n \n Hey, we built \n BirdProxies\n because proxies shouldn't be complicated or overpriced. Fast residential and ISP proxies in 195+ locations, fair pricing, and real support.
\n Try our FlappyBird game on the landing page for free data!\n
\n