{"repo":"VAllens/CrawlerSamples","free":true,"listed":false,"github":"https://github.com/VAllens/CrawlerSamples","clone":"git clone https://github.com/VAllens/CrawlerSamples.git","description":"A small, practical C# sample that crawls a modern JavaScript-rendered page with Puppeteer Sharp, then parses and extracts structured data with AngleSharp.","language":"C#","stars":42,"topics":["dotnetcore","anglesharp","puppeteer","chsarp","crawler","headless-chrome","headless-chromium","headless-browsers","headless"],"license":"MIT","category":"scrapers-browser-automation","readme_excerpt":"Crawler Samples A small, practical C# sample that crawls a modern JavaScript-rendered page with Puppeteer Sharp, then parses and extracts structured data with AngleSharp. The demo targets the public .NET organization repositories page on GitHub and prints the extracted repository list as JSON. Why this sample exists Many real-world pages are no longer static HTML. This repository shows a maintainable pattern for: 1. Rendering dynamic pages in a headless Chromium browser ( PuppeteerSharp ) 2. Parsing the resulting DOM with a standards-based HTML parser ( AngleSharp ) 3. Mapping DOM nodes into typed models and serializing them with System.Text.Json It is intended as a learning / reference project for .NET developers building crawlers, scrapers, or HTML extraction tools. Requirements - .NET 10 SDK - Network access on first run (downloads a Chromium revision used by Puppeteer Sharp) Quick start Or open CrawlerSamples.sln in Visual Studio / VS Code / Rider and run the console app. Publish (optional) You can also use the included publish profiles under src/CrawlerSamples.ConsoleApp/Properties/PublishProfiles/ . Sample output The app prints a JSON array of repository models, for example: Screenshot: Project layout Notes - On first run, Puppeteer Sharp downloads a Chromium package into a local cache. This can take a while depending on your network. - If the Chromium download fails, the process exits with an exception. - GitHub's UI markup can change. If selectors stop matching, updat","default_branch":null,"files":null,"tree":[],"storefront":"/r/VAllens","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/VAllens/CrawlerSamples/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}