{"repo":"leaky-forms/leaky-forms-crawler","free":true,"listed":false,"github":"https://github.com/leaky-forms/leaky-forms-crawler","clone":"git clone https://github.com/leaky-forms/leaky-forms-crawler.git","description":"Puppeteer based crawler to measure email and password exfiltration","language":"JavaScript","stars":27,"topics":["email-tracking","web-crawler","tracker-radar-collector"],"license":null,"category":"scrapers-browser-automation","readme_excerpt":"Crawler In order to detect email and password exfiltration, we extended Tracker Radar Collector(TRC) by adding EmailPasswordFieldsCollector that finds and fills email and password fields. In order to investigate the effect of users’ consent preferences, we integrated Consent-O-Matic into our crawler. Lastly, we used the Fathom-based email field detector model used in Firefox Relay add-on. email-field-collector branch contains the version that we used in our study. The main branch contains the rebased code. This repository contains code for re-running web crawler. Installation - Clone this project locally ( git clone git@github.com:asumansenol/leaky-forms.git ) - Install all dependencies ( npm i ) - Run the command line tool: Command line parameters Below we give a description of the parameters that are passed to the crawler. - -o, --output - (required) output folder where output files will be created - -u, --url - single URL to crawl - -i, --input-list - path to a text file with list of URLs to crawl (each in a separate line) - -d, --data-collectors - comma separated list (e.g -d 'requests,cookies' ) of data collectors that should be used (all by default) - -c, --crawlers - override the default number of concurrent crawlers (default number is picked based on the number of CPU cores) - --reporters - comma separated list (e.g. --reporters 'cli,file,html' ) of reporters to be used ('cli' by default) - -v, --verbose - instructs reporters to log additional information (e.g. for \"c","default_branch":null,"files":null,"tree":[],"storefront":"/r/leaky-forms","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/leaky-forms/leaky-forms-crawler/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}