{"repo":"saeeddhqan/evine","free":true,"listed":false,"github":"https://github.com/saeeddhqan/evine","clone":"git clone https://github.com/saeeddhqan/evine.git","description":"Interactive CLI Web Crawler","language":"Go","stars":181,"topics":["crawler","web-crawler","scraper","golang","cli","data-mining","go","fuzzing","osint"],"license":"GPL-3.0","category":"scrapers-browser-automation","readme_excerpt":"Evine Interactive CLI Web Crawler. Evine is a simple, fast, and interactive web crawler and web scraper written in Golang. Evine is useful for a wide range of purposes such as metadata and data extraction, data mining, reconnaissance and testing. If you like the project, give it a star. It forces me to develop the project! Install From Binary Pre-build binary releases are also available(Suggested). From source From GitHub Note: golang 1.13.x required. Commands & Usage Keybinding Description ---------------------------------------- --------------------------------------- Enter Run crawler (from URL view) Enter Display response (from Keys and Regex views) Tab Next view Ctrl+Space Run crawler Ctrl+S Save response Ctrl+Z Quit Ctrl+R Restore to default values (from Options and Headers views) Ctrl+Q Close response save view (from Save view) It will display help for the tool: flag Description Example ------ ------------- --------- -url URL to crawl for evine -url toscrape.com -url-exclude string Exclude URLs maching with this regex (default \". \") evine -url-exclude ?id= -domain-exclude string Exclude in-scope domains to crawl. Separate with comma. default=root domain evine -domain-exclude host1.tld,host2.tld -code-exclude string Exclude HTTP status code with these codes. Separate whit '\\ ' (default \". \") evine -code-exclude 200,201 -delay int Sleep between each request(Millisecond) evine -delay 300 -depth Scraper depth search level (default 1) evine -depth 2 -thread int The number o","default_branch":null,"files":null,"tree":[],"storefront":"/r/saeeddhqan","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/saeeddhqan/evine/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}