{"repo":"my8100/scrapydweb","free":true,"listed":false,"github":"https://github.com/my8100/scrapydweb","clone":"git clone https://github.com/my8100/scrapydweb.git","description":"Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI. Docs 文档 :point_right:","language":"Python","stars":3412,"topics":["scrapy","scrapyd","scrapyd-ui","scrapyd-api","scrapyd-admin","scrapyd-manage","log-parsing","log-analysis","scrapyd-monitor","scrapyd-keeper"],"license":"GPL-3.0","category":"scrapers-browser-automation","readme_excerpt":":abc: English :mahjong: 简体中文 ScrapydWeb: Web app for Scrapyd cluster management, with support for Scrapy log analysis & visualization. Scrapyd :x: ScrapydWeb :x: LogParser :book: Recommended Reading :link: How to efficiently manage your distributed web scraping projects :link: How to set up Scrapyd cluster on Heroku :eyes: Demo :link: scrapydweb.herokuapp.com :rocket: Sponsor This project is supported by IPRoyal. You can get premium quality proxies at unbeatable prices with a discount using this referral link! :stars: Features View contents - :diamond shape with a dot inside: Scrapyd Cluster Management - :100: All Scrapyd JSON API Supported - :ballot box with check: Group, filter and select any number of nodes - :computer mouse: Execute command on multinodes with just a few clicks - :mag: Scrapy Log Analysis - :bar chart: Stats collection - :chart with upwards trend: Progress visualization - :bookmark tabs: Logs categorization - :battery: Enhancements - :package: Auto packaging - :male detective: Integrated with :link: LogParser - :alarm clock: Timer tasks - :e-mail: Monitor & Alert - :iphone: Mobile UI - :closed lock with key: Basic auth for web UI :computer: Getting Started View contents :warning: Prerequisites :heavy exclamation mark: Make sure that :link: Scrapyd has been installed and started on all of your hosts. :bangbang: Note that for remote access, you have to manually set 'bind address = 0.0.0.0' in :link: the configuration file of Scrapyd and restart Scrapyd to ma","default_branch":null,"files":null,"tree":[],"storefront":"/r/my8100","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/my8100/scrapydweb/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}