{"repo":"website-scraper/node-website-scraper","free":true,"listed":false,"github":"https://github.com/website-scraper/node-website-scraper","clone":"git clone https://github.com/website-scraper/node-website-scraper.git","description":"Download website to local directory (including all css, images, js, etc.)","language":"JavaScript","stars":1749,"topics":["javascript","website-scraper","scraper","nodejs","hacktoberfest"],"license":"MIT","category":"scrapers-browser-automation","readme_excerpt":"website-scraper Options Plugins Log and debug Frequently Asked Questions Contributing Code of Conduct Download the website to the local directory (including all css, images, js, etc.) Note: by default dynamic websites (where content is loaded by js) may not be saved correctly because website-scraper doesn't execute js, it only parses http responses for html and css files. If you need to download a dynamic website take a look at website-scraper-puppeteer. Sponsors Maintenance of this project is made possible by all the contributors and sponsors. If you'd like to sponsor this project and have your avatar or company logo appear below click here. 💖 Requirements nodejs version = 20.18.1 website-scraper since v5 is pure ESM (it doesn't work with CommonJS), read more in release v5.0.0 docs Installation Usage options urls - urls to download, required directory - path to save files, required sources - selects which resources should be downloaded recursive - follow hyperlinks in html files maxRecursiveDepth - maximum depth for hyperlinks maxDepth - maximum depth for all dependencies request - custom options for http module got subdirectories - subdirectories for file extensions defaultFilename - filename for index page prettifyUrls - prettify urls ignoreErrors - whether to ignore errors on resource downloading urlFilter - skip some urls filenameGenerator - generate filename for downloaded resource requestConcurrency - set maximum concurrent requests plugins - plugins, allow to customi","default_branch":null,"files":null,"tree":[],"storefront":"/r/website-scraper","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/website-scraper/node-website-scraper/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}