{"repo":"tylergibbs1/feedstock","free":true,"listed":false,"github":"https://github.com/tylergibbs1/feedstock","clone":"git clone https://github.com/tylergibbs1/feedstock.git","description":"High-performance web crawler and scraper for TypeScript, powered by Bun and Playwright","language":"TypeScript","stars":11,"topics":["bun","crawler","playwright","typescript","web-crawler","web-scraping"],"license":"Apache-2.0","category":"scrapers-browser-automation","readme_excerpt":"Feedstock High-performance web crawler and scraper for TypeScript, powered by Bun and Playwright . Features - Single & multi-page crawling with concurrent execution - Deep crawling — BFS, DFS, BestFirst, Bandit (UCB1 online learning), and Focused (RL/Q-learning) strategies - Content extraction — CSS selectors, regex, XPath, table, accessibility tree, and composite multi-strategy extraction - Markdown generation with citation support - Smart caching with ETag/Last-Modified validation via bun:sqlite and multi-signal freshness evaluation (sitemap, HTTP headers, content hash, time decay) - URL filtering — pattern, domain, and content-type filters - URL scoring — keyword relevance, path depth, freshness, domain authority, neural quality estimation (online-learning), and UCB1 bandit scoring - Rate limiting — per-domain with exponential backoff - Robots.txt parsing and compliance - Built-in stealth mode — one flag enables random user-agents, navigator.webdriver override, plugin/language spoofing, human-like mouse/scroll simulation. Enhanced mode generates spatially-consistent fingerprint profiles (UA, platform, WebGL, screen, canvas all agree) - Anti-bot detection with auto-retry on blocked pages - Multiple browser backends — Playwright (Chromium/Firefox/WebKit), generic CDP (Browserbase, Browserless, etc.), or Lightpanda (local/cloud) - Proxy rotation — round-robin strategy with health tracking - URL seeding — discover URLs from sitemaps - Accessibility snapshots — compact semantic","default_branch":null,"files":null,"tree":[],"storefront":"/r/tylergibbs1","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/tylergibbs1/feedstock/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}