{"repo":"alaz/legitbot","free":true,"listed":false,"github":"https://github.com/alaz/legitbot","clone":"git clone https://github.com/alaz/legitbot.git","description":"🤔 Is this Web request from a real search engine🕷 or from an impersonating agent 🕵️‍♀️?","language":"Ruby","stars":29,"topics":["search-engine","googlebot","bot","ruby","security","protection","impersonation","detect-crawlers","fake","ruby-gem"],"license":null,"category":"security-tools","readme_excerpt":"Legitbot Ruby gem to make sure that an IP really belongs to a bot, typically a search engine. Usage Suppose you have a Web request and you would like to check it is not diguised: bot will be nil if no bot signature was found in the User-Agent . Otherwise, it will be an object with methods Sometimes you already know which search engine to expect. For example, you might be using rack-attack: Or if you do not like all those ghoulish crawlers stealing your content, evaluating it and getting ready to invade your site with spammers, then block them all: Versioning Semantic versioning with the following clarifications: - MINOR version is incremented when support for new bots is added. - PATCH version is incremented when validation logic for a bot changes (IP list updated, for example). Supported - Ahrefs - AmazonAdBot - AmazonBot - Applebot - Baidu spider - Bingbot - BLEXBot (WebMeUp) - DataForSEO - DuckAssistBot - DuckDuckBot - Google crawlers - IAS - OpenAI GPTBot - Oracle Data Cloud Crawler - Marginalia - Meta / Facebook Web crawlers - Perplexity - Petal search engine - Pinterest - Twitterbot, the list of IPs is in the Troubleshooting page - Yandex robots License Apache 2.0 Other projects - Play Framework variant in Scala: play-legitbot - Article When (Fake) Googlebots Attack Your Rails App - Voight-Kampff is a Ruby gem that detects bots by User-Agent - crawler detect is a Ruby gem and Rack middleware to detect crawlers by few different request headers, including User-Agent - Pro","default_branch":null,"files":null,"tree":[],"storefront":"/r/alaz","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/alaz/legitbot/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}