{"repo":"jonathanhefner/grubby","free":true,"listed":false,"github":"https://github.com/jonathanhefner/grubby","clone":"git clone https://github.com/jonathanhefner/grubby.git","description":"Fail-fast web scraping","language":"Ruby","stars":13,"topics":["ruby","web-scraping","mechanize"],"license":"MIT","category":"scrapers-browser-automation","readme_excerpt":"grubby [Fail-fast][] web scraping. grubby adds a layer of utility and error-checking atop the marvelous [Mechanize gem][]. See API listing below, or browse the [full documentation][]. [Fail-fast]: https://en.wikipedia.org/wiki/Fail-fast [Mechanize gem]: https://rubygems.org/gems/mechanize [full documentation]: https://www.rubydoc.info/gems/grubby/ Examples The following code scrapes stories from the Hacker News front page: Hacker News also offers a JSON API, which may be more robust for scraping purposes. grubby can scrape JSON just as well: Core API - Grubby - #fulfill - #get mirrored - #ok? - #time between requests - Scraper - .each - .scrape - .scrapes - [ #[] ](https://www.rubydoc.info/gems/grubby/Grubby/Scraper:[]) - #to h - PageScraper - .scrape file - #page - JsonScraper - .scrape file - #json - Mechanize::File - #save to - #save to! - Mechanize::Page - #at! - #search! - Mechanize::Page::Link - #to absolute uri - URI - #basename - #query param Auxiliary API grubby loads a few gems that extend Ruby objects with utility methods. Some of those methods are listed below. See each gem's documentation for a complete API listing. - gorge (docs) - Pathname#file crc32 - Pathname#file md5 - Pathname#file sha1 - ryoba (docs) - Nokogiri::XML::Node#matches! - Nokogiri::XML::Node#text! - Nokogiri::XML::Node#uri - Nokogiri::XML::Searchable#ancestor! - Nokogiri::XML::Searchable#ancestors! - Nokogiri::XML::Searchable#at! - Nokogiri::XML::Searchable#search! Recommended Gems The following","default_branch":null,"files":null,"tree":[],"storefront":"/r/jonathanhefner","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/jonathanhefner/grubby/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}