{"repo":"crawlbase/crawlbase-python","free":true,"listed":false,"github":"https://github.com/crawlbase/crawlbase-python","clone":"git clone https://github.com/crawlbase/crawlbase-python.git","description":"Fast python library for the Crawlbase API","language":"Python","stars":26,"topics":["crawler","crawling","scraper","scraping","scraping-api","scraping-python","scraping-websites"],"license":"Apache-2.0","category":"scrapers-browser-automation","readme_excerpt":"Crawlbase API Python class A lightweight, dependency free Python class that acts as wrapper for Crawlbase API. Installing Choose a way of installing: - Download the python class from Github. - Or use PyPi Python package manager. pip install crawlbase Then import the CrawlingAPI, ScraperAPI, etc as needed. Crawling API First initialize the CrawlingAPI class. GET requests Pass the url that you want to scrape plus any options from the ones available in the API documentation. Example: You can pass any options from Crawlbase API. Example: POST requests Pass the url that you want to scrape, the data that you want to send which can be either a json or a string, plus any options from the ones available in the API documentation. Example: You can send the data as application/json instead of x-www-form-urlencoded by setting option post content type as json. Javascript requests If you need to scrape any website built with Javascript like React, Angular, Vue, etc. You just need to pass your javascript token and use the same calls. Note that only .get is available for javascript and not .post . Same way you can pass javascript additional options. Original status You can always get the original status and Crawlbase status from the response. Read the Crawlbase documentation to learn more about those status. cb status is the preferred Crawlbase status field. The library reads cb status from the API when present, and falls back to pc status otherwise. Both keys are set on response['headers'] t","default_branch":null,"files":null,"tree":[],"storefront":"/r/crawlbase","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/crawlbase/crawlbase-python/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}