{"repo":"JayBizzle/Crawler-Detect","free":true,"listed":false,"github":"https://github.com/JayBizzle/Crawler-Detect","clone":"git clone https://github.com/JayBizzle/Crawler-Detect.git","description":"🕷 CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent","language":"PHP","stars":2398,"topics":["php","user-agent","crawler","spider","bots","detect","hacktoberfest"],"license":"MIT","category":"scrapers-browser-automation","readme_excerpt":"crawlerdetect.io About CrawlerDetect is a PHP library for detecting bots, crawlers and spiders via the User-Agent and HTTP FROM headers. It currently recognises thousands of user agents and is updated regularly. Installation Usage Passing headers from a request object With no arguments, CrawlerDetect reads $ SERVER . If your headers come from somewhere else — a PSR-7 request, Symfony's HeaderBag , Swoole, or a Lambda event — pass them in directly. Both real header names ( User-Agent ) and PHP's SAPI names ( HTTP USER AGENT ) are understood, and values may be strings or arrays of strings. Prefer this over isCrawler($request- getHeaderLine('User-Agent')) . Some crawlers — Googlebot in particular — send a genuine browser User-Agent and identify themselves in another header such as From or Sec-CH-UA . Passing the full set lets CrawlerDetect check all of them; passing a single string can only ever check one. Contributing If you find a bot, spider or crawler that CrawlerDetect fails to detect, please open a pull request that: - adds the regex pattern to the $data array in src/Fixtures/Crawlers.php - adds the failing user agent string to tests/data/user agent/crawlers.txt The raw/Crawlers.json and raw/Crawlers.txt files are regenerated automatically by export.php after merge — no need to touch them. If you're not able to submit a PR, open an issue with the user agent string and we'll take it from there. Ports & Integrations CrawlerDetect has been ported to a number of other language","default_branch":null,"files":null,"tree":[],"storefront":"/r/JayBizzle","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/JayBizzle/Crawler-Detect/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}