{"repo":"moskrc/crawlerdetect","free":true,"listed":false,"github":"https://github.com/moskrc/crawlerdetect","clone":"git clone https://github.com/moskrc/crawlerdetect.git","description":"🕷CrawlerDetect is a Python library designed to identify bots, crawlers, and spiders by analyzing their user agents.","language":"Python","stars":45,"topics":["python","user-agent","crawler","spider","bot","detect"],"license":"MIT","category":"scrapers-browser-automation","readme_excerpt":"About CrawlerDetect This is a Python wrapper for CrawlerDetect a web crawler detection library. It helps identify bots, crawlers, and spiders using the user agent and other HTTP headers. Currently synced with JayBizzle/Crawler-Detect v1.4.1 - 1462 crawler patterns. How to install How to use Method Reference camelCase snake case Description ----------- ------------ ----------------------------------- isCrawler() is crawler() Check if user agent is a crawler getMatches() get matches() Get the name of detected crawlers Variant 1 Variant 2 Variant 3 Output the name of the bot that matched (if any) Get version of the library Contributing The patterns and testcases are synced from the PHP repo. If you find a bot/spider/crawler user agent that crawlerdetect fails to detect, please submit a pull request with the regex pattern and a testcase to the upstream PHP repo. Failing that, just create an issue with the user agent you have found, and we'll take it from there :) Development This project uses uv for dependency management and packaging. Setup Running tests Update crawlers from upstream PHP repo Bump version Bumping updates the version in pyproject.toml and crawlerdetect/ init .py , commits the change, and creates a matching vX.Y.Z git tag. Pass the part to increment: Or set an explicit version instead of incrementing a part: Preview the change first with --dry-run -v (nothing is written or committed): Once bumped, push the commit and tag, then publish to PyPI: How to publish a new","default_branch":null,"files":null,"tree":[],"storefront":"/r/moskrc","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/moskrc/crawlerdetect/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}