{"repo":"ZoranPandovski/ProdirectScraper","free":true,"listed":false,"github":"https://github.com/ZoranPandovski/ProdirectScraper","clone":"git clone https://github.com/ZoranPandovski/ProdirectScraper.git","description":":necktie: Web scraper for http://www.prodirectselect.com/ :shoe:","language":"Python","stars":18,"topics":["scrapy-spider","scrapy-crawler","python","webscraping","scrapy","scraper","spider","webscraper"],"license":"MIT","category":"scrapers-browser-automation","readme_excerpt":"ProdirectScraper Installation Installing Scrapy inside a virtual environment on all platforms. Python packages can be installed either globally (a.k.a system wide), or in user-space. We do not recommend installing scrapy system wide. Instead, we recommend that you install scrapy within a so-called “virtual environment” (virtualenv). Virtualenvs allow you to not conflict with already-installed Python system packages (which could break some of your system tools and scripts), and still install packages normally with pip (without sudo and the likes). To install it globally (having it globally installed actually helps here), it should be a matter of running: Inside virtual env install ProdirectScraper dependencies: Config Settings These are the basic options: After that edit the configuration specific to the category of product you would like to scrape. For trainers: For men's clothing: For women's clothing: Running the Spiders To put our spider to work, go to the project’s top level directory and run: where \"SCRAPER\" must be one of the following: - trainers - mensclothing - womensclothing This command runs the spider with name trainers, that will crawl http://www.prodirectselect.com/ website and send mail with lowest prices, model description and link to trainers, which size is specified in configuration.ini","default_branch":null,"files":null,"tree":[],"storefront":"/r/ZoranPandovski","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/ZoranPandovski/ProdirectScraper/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}