{"repo":"hunterhug/marmot","free":true,"listed":false,"github":"https://github.com/hunterhug/marmot","clone":"git clone https://github.com/hunterhug/marmot.git","description":"💐Marmot A Golang HTTP Download","language":"Go","stars":27,"topics":["spider","crawler","gospider","marmot","gohttp"],"license":"Apache-2.0","category":"scrapers-browser-automation","readme_excerpt":"Project: Marmot HTTP Download 中文介绍 HTTP Download Helper, Supports Many Features such as Cookie Persistence, HTTP(S) and SOCKS5 Proxy.... 1. Introduction World-Wide-Web robot, also known as spiders and crawlers. The principle is to falsify network data by constructing appointed HTTP protocol data packet, then request resource to the specified host, goal is to access the data returned. There are a large number of web information, human's hand movement such as copy-paste data from web page is time-consuming and laborious , thus inspired the data acquisition industry. Batch access to public network data does not break the law, but because there is no difference, no control, very violent means will lead to other services is not stable, therefore, most of the resources provider will filtering some data packets(falsify), in this context, batch small data acquisition has become a problem. Integrated with various requirements, such as various API development, automated software testing(all this have similar technical principle). So this project come into the world(very simple). The Marmot is very easy to understand, just like Python's library requests (Not yet Smile -- ). By enhancing native Golang HTTP library, help you deal with some trivial logic (such as collecting information, checking parameters), and add some fault-tolerant mechanisms (such as add lock, close time flow, ensure the high concurrent run without accident). It provides a human friendly API interface, you can reuse i","default_branch":null,"files":null,"tree":[],"storefront":"/r/hunterhug","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/hunterhug/marmot/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}