{"repo":"xtuhcy/gecco","free":true,"listed":false,"github":"https://github.com/xtuhcy/gecco","clone":"git clone https://github.com/xtuhcy/gecco.git","description":"Easy to use lightweight web crawler（易用的轻量化网络爬虫）","language":"Java","stars":2511,"topics":["gecco","crawler","jsoup","fastjson","java","dynamic"],"license":"MIT","category":"scrapers-browser-automation","readme_excerpt":"What is Gecco Gecco is a easy to use lightweight web crawler developed with java language.Gecco integriert jsoup, httpclient, fastjson, spring, htmlunit, redission ausgezeichneten framework,Let you only need to configure a number of jQuery style selector can be very quick to write a crawler.Gecco framework has excellent scalability, the framework based on the principle of open and close design, to modify the closure, the expansion of open.At the same time Gecco is based on a very open MIT open source protocol, whether you are a user or want to jointly improve the Gecco developer, welcome to request pull.If you like the crawler framework,please star or fork! - 中文说明 Main features - [x] Easy to use, use jQuery style selector to extract elements - [x] Support for asynchronous Ajax requests in the page - [x] Support page JavaScript variable extraction - [x] Using Redis to realize distributed crawling,reference gecco-redis - [x] Support the development of business logic with Spring,reference gecco-spring - [x] Support htmlunit extension,reference gecco-htmlunit - [x] Support extension mechanism - [x] Support download UserAgent random selection - [x] Support the download proxy server randomly selected Framework overview Download Download via Maven Dependent project httpclient，jsoup，fastjson，reflections，cglib，rhino，log4j，jmxutils，commons-lang3 Quick start DynamicGecco The purpose of DynamicGecco is to implement the runtime configuration of the crawl rule without defining the SpiderBe","default_branch":null,"files":null,"tree":[],"storefront":"/r/xtuhcy","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/xtuhcy/gecco/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}