{"repo":"Praeviso/crawl4weibo","free":true,"listed":false,"github":"https://github.com/Praeviso/crawl4weibo","clone":"git clone https://github.com/Praeviso/crawl4weibo.git","description":"An out-of-the-box Weibo scraper Python library, based on a successfully tested solution, usable without cookies.","language":"Python","stars":43,"topics":["crawler","weibo","python"],"license":"MIT","category":"scrapers-browser-automation","readme_excerpt":"Crawl4Weibo 中文文档 English --- Crawl4Weibo is a ready-to-use Weibo (微博) web scraper Python library that simulates mobile requests, handles common anti-scraping strategies, and returns structured data models—ideal for data collection, analysis, and monitoring scenarios. ✨ Features - No Cookie Required : Runs without cookies, automatically initializes session with mobile User-Agent - Browser-Based Cookie Fetching : Uses Playwright to simulate real browsers for enhanced anti-scraping bypass - Optional Logged-In Cookies : Interactive login and persisted storage state for more complete data - Built-in 432 Protection : Handles anti-scraping protection with exponential backoff retry mechanism - Unified Proxy Pool Management : Supports both dynamic and static IP proxy pools with configurable TTL, polling strategies, and automatic cleanup - Standardized Data Models : Clean User , Post , and Comment data models with recursive access to reposted content - Long Text Expansion : Supports expanding truncated long posts, keyword search, user list fetching, and batch pagination - Comment Scraping : Fetch post comments with automatic pagination and support for nested replies - Image Download Utilities : Download images from single posts, batches, or entire pages with duplicate file detection - Video Download Utilities : Download videos with multi-quality selection (720p/SD/HD), streaming download, retry and proxy support - Unified Logging & Error Types : Quickly locate network, parsing, or auth","default_branch":null,"files":null,"tree":[],"storefront":"/r/Praeviso","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/Praeviso/crawl4weibo/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}