{"repo":"gosom/scrapemate","free":true,"listed":false,"github":"https://github.com/gosom/scrapemate","clone":"git clone https://github.com/gosom/scrapemate.git","description":"Golang Crawling and scraping framework","language":"Go","stars":207,"topics":["go-framework","golang","web-crawler","web-scraping","crawler","go","scraper","spider"],"license":"MIT","category":"scrapers-browser-automation","readme_excerpt":"scrapemate Scrapemate is a web crawling and scraping framework written in Golang. It is designed to be simple and easy to use, yet powerful enough to handle complex scraping tasks. Features - Low level API & Easy High Level API - Customizable retry and error handling - Javascript Rendering with ability to control the browser - Screenshots support (when JS rendering is enabled) - Capability to write your own result exporter - Capability to write results in multiple sinks - Default CSV writer - Caching (File/LevelDB/Custom) - Custom job providers (memory provider included) - Headless and Headful support when using JS rendering - Automatic cookie and session handling - Rotating HTTP/HTTPS/SOCKS5 proxy support JavaScript Rendering Scrapemate uses Playwright for JavaScript rendering. It requires the Playwright browsers to be installed. Build and run without any special tags: Example Usage The books-to-scrape-simple example demonstrates JavaScript rendering with Playwright: Installation Quickstart (hit CTRL-C to exit) Migrating from v0.9.x to v1.0.0 Version 1.0.0 introduces a BrowserPage interface abstraction for browser automation. This is a breaking change for users who use JavaScript rendering with BrowserActions . Update BrowserActions signature Accessing the underlying browser page If you need browser-specific features, use Unwrap() : See CHANGELOG.md for the full list of changes. Documentation You can find more documentation here For the High Level API see this example. Read ","default_branch":null,"files":null,"tree":[],"storefront":"/r/gosom","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/gosom/scrapemate/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}