{"repo":"davidesantangelo/webinspector","free":true,"listed":false,"github":"https://github.com/davidesantangelo/webinspector","clone":"git clone https://github.com/davidesantangelo/webinspector.git","description":"Ruby gem to inspect completely a web page. It scrapes a given URL, and returns you its meta, links, images more.","language":"Ruby","stars":307,"topics":["ruby","inspector","web-inspector","scraper","rubygem"],"license":"MIT","category":"scrapers-browser-automation","readme_excerpt":"WebInspector Ruby gem to inspect web pages. It scrapes a given URL and returns its title, description, meta tags, links, images, and more. Installation Add this line to your application's Gemfile: And then execute: $ bundle Or install it yourself as: $ gem install webinspector Usage Initialize a WebInspector instance With options Accessing response status and headers Accessing page data Working with meta tags Filtering links and images by domain Searching for words JavaScript and Stylesheets Language Detection Structured Data Security Information Performance Metrics Content Type Technology Detection HTML Tag Statistics RSS/Atom Feeds Social Media Links Robots.txt and Sitemap CMS Detection Accessibility Score Mobile-Friendly Check Export all data to JSON Changelog Version 1.2.0 New Features: - RSS/Atom feed detection with feeds method - Social media profile extraction with social links method - CMS detection and information with cms info method (WordPress, Drupal, Joomla, Shopify, Wix, Squarespace) - Accessibility scoring with accessibility score method - Mobile-friendly detection with mobile friendly? method - Robots.txt and sitemap URL detection with robots txt url and sitemap url methods Improvements: - Enhanced Request module with valid? and ssl? methods for better URL validation - Improved Meta module with author and publisher extraction - Better error handling across all modules - Performance improvements with internal caching Contributors - Steven Shelby (@stevenshelby)","default_branch":null,"files":null,"tree":[],"storefront":"/r/davidesantangelo","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/davidesantangelo/webinspector/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}