{"repo":"till-tietz/parsel","free":true,"listed":false,"github":"https://github.com/till-tietz/parsel","clone":"git clone https://github.com/till-tietz/parsel.git","description":"parallel execution of RSelenium","language":"R","stars":14,"topics":["cran","parallel","r","rselenium","web-scraping"],"license":null,"category":"scrapers-browser-automation","readme_excerpt":"parsel parsel is a framework for parallelized dynamic web-scraping using RSelenium . Leveraging parallel processing, it allows you to run any RSelenium web-scraping routine on multiple browser instances simultaneously, thus greatly increasing the efficiency of your scraping. parsel utilizes chunked input processing as well as error catching and logging, to ensure seamless execution of your scraping routine and minimal data loss, even in the presence of unforeseen RSelenium errors. parsel additionally provides convenient wrapper functions around RSelenium methods, that allow you to quickly generate safe scraping code with minimal coding on your end. You can find the parsel website here. Installation Usage Parallel Scraping The following example will hopefully serve to illustrate the functionality and ideas behind how parsel operates. We’ll set up the following scraping job: 1. navigate to a random Wikipedia article 2. retrieve its title 3. navigate to the first linked page on the article 4. retrieve the linked page’s title and first section and parallelize it with parsel . parsel requires two things: 1. a scraping function defining the actions to be executed in each RSelenium instance. Actions to be executed in each browser instance should be written in the conventional RSelenium syntax with remDr$ specifying the remote driver. 2. some input x to those actions (e.g. search terms to be entered in search boxes or links to navigate to etc.) Now that we have our scrape function an","default_branch":null,"files":null,"tree":[],"storefront":"/r/till-tietz","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/till-tietz/parsel/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}