{"repo":"lightfeed/scrapedown","free":true,"listed":false,"github":"https://github.com/lightfeed/scrapedown","clone":"git clone https://github.com/lightfeed/scrapedown.git","description":"HTML to Markdown with CSS selector and XPath annotations","language":"TypeScript","stars":13,"topics":["crawler","data-pipeline","html-parser","html-to-markdown","llm","llm-extraction","llm-scraper","markdown","nlp","web-data-extraction"],"license":"MIT","category":"scrapers-browser-automation","readme_excerpt":"Scrapedown HTML to Markdown with CSS selector and XPath annotations — built for LLM-powered web scraping. Why? HTML-to-Markdown converters produce clean, readable content for both humans and LLMs — but the DOM structure is lost along the way. You can always feed Markdown to an LLM to extract structured information, but that costs tokens on every page, every time. What if the LLM could also see where each piece of content lives in the DOM? Then it can generate robust scraping code — stable selectors and XPaths that run without any LLM in the loop, saving tokens and improving accuracy on long or repetitive pages. Scrapedown does exactly this: it converts HTML to Markdown and annotates each element with its CSS selector and/or XPath, so an LLM can produce precise, reusable scraper code in one shot. Install Quick start Output: An LLM receiving this output can immediately generate scraper code: CLI Running the CLI Via npx (no install needed): Global install: Local development (before publishing): CLI options Real-world example: Hacker News Output (trimmed): An LLM reading this can immediately produce a working scraper: API scrapedown(html, options?) Convert HTML to annotated Markdown in one call. new Scrapedown(options?) Create a reusable converter instance (avoids re-creating Turndown rules on every call). Options Option Type Default Description --- --- --- --- annotationPlacement 'inline' \\ 'footnote' 'inline' inline emits HTML comments next to each element. footnote appends [^s","default_branch":null,"files":null,"tree":[],"storefront":"/r/lightfeed","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/lightfeed/scrapedown/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}