{"repo":"OmarEinea/GoodReadsScraper","free":true,"listed":false,"github":"https://github.com/OmarEinea/GoodReadsScraper","clone":"git clone https://github.com/OmarEinea/GoodReadsScraper.git","description":":books: A GoodReads.com Scraper script to get books reviews including text and rating.","language":"Python","stars":44,"topics":["goodreads","python","beautifulsoup","selenium","webscraping"],"license":"GPL-3.0","category":"scrapers-browser-automation","readme_excerpt":"GoodReads Reviews Scraper This is a python 3 web scraping script to get books reviews from goodreads.com, using the web browser automation tool Selenium, and BeautifulSoup for pulling data out of HTML. I used it to scrape around 700k Arabic reviews in 2018 (Arabic reviews are fewer than English ones). Papers We experimented on the collected data and published details about it in two research papers: - BRAD 1.0: Book reviews in Arabic dataset (IEEE). - An Annotated Huge Dataset for Standard and Colloquial Arabic Reviews for Subjective Sentiment Analysis (Elsevier). Contents - Analyzer.py: short script to display some statistics about scraped books reviews - Books.py: class to scrape books ids from goodreads list or lists search or shelf - Browser.py: subclass of Chrome WebDriver class that's specialized for GoodrReads browsing - Reviews.py: class to scrape reviews from goodreads books using books ids - Sample.py: sample script showing a complete use of the scraper with error handling - Tools.py: set of function tools that are used in other scripts - Writer.py: class to write scraped reviews to files - requirements.txt: list of Required Python modules to be installed Requirements To install requirements listed in requirements.txt, you'll need to run this (depends on your os): Also, as this is using Selenium to control the Chrome Browser, so you'll need to download its driver for your specific os from here. Documentation - A Books object (from Books.py) represents the books that","default_branch":null,"files":null,"tree":[],"storefront":"/r/OmarEinea","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/OmarEinea/GoodReadsScraper/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}