{"repo":"oxylabs/automated-web-scraper-autoscraper","free":true,"listed":false,"github":"https://github.com/oxylabs/automated-web-scraper-autoscraper","clone":"git clone https://github.com/oxylabs/automated-web-scraper-autoscraper.git","description":"This tutorial shows how to automate your web scraping processes using AutoScaper – one of Python web scraping libraries available.","language":null,"stars":121,"topics":["autoscraper","python","scraping","web-scraping"],"license":null,"category":"scrapers-browser-automation","readme_excerpt":"Automated Web Scraper With Python AutoScraper This tutorial will show you how to automate your web scraping processes using AutoScaper – one of the several Python web scraping libraries available. Check out a more detailed tutorial on our blog. Methods to install AutoScraper Scraping products with AutoScraper + Scraping product category URLs + Scraping product information from a single webpage + Scraping all the products on a specific category How to use AutoScraper with proxies Saving and loading an AutoScraper model Methods to install AutoScraper First things first, let’s install the AutoScraper library. There are actually several ways to install and use this library, but for this tutorial, we’re going to use the Python package index (PyPI) repository using the following pip command: Scraping products with AutoScraper This section showcases an example to auto scrape public data with the AutoScraper module in Python using the Oxylabs Scraping Sandbox website as a target. The target website has three thousand products in different categories. Scraping product category URLs Now, if you want to scrape the links to the category pages, you can do it with the following trivial code: Note that the Oxylabs Sandbox uses JavaScript to load some elements dynamically, such as the category buttons. Since AutoScraper doesn’t support JavaScript rendering, you won’t be able to scrape all category links. For instance, you can access the “Xbox platform” category but not the subcategories insi","default_branch":null,"files":null,"tree":[],"storefront":"/r/oxylabs","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/oxylabs/automated-web-scraper-autoscraper/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}