{"repo":"teng-lin/agent-fetch","free":true,"listed":false,"github":"https://github.com/teng-lin/agent-fetch","clone":"git clone https://github.com/teng-lin/agent-fetch.git","description":"Full-content web fetcher for AI agents — Chrome TLS fingerprinting, browser impersonation, and multi-strategy article extraction","language":"TypeScript","stars":303,"topics":["ai-agents","anti-bot-detection","content-extraction","fetcher","html-to-markdown","tls-fingerprint","web-scraping","nodejs","readability","typescript"],"license":"MIT","category":"ai-agents","readme_excerpt":"agent-fetch Full-content web fetcher for AI agents and content workflows. Standard HTTP tools (curl, wget, or an agent's built-in web fetch) are often served truncated or different responses because servers inspect the client's network fingerprint. agent-fetch uses browser impersonation so servers respond as they would to a real browser, then runs multiple extraction strategies to pull the complete article — every paragraph, heading, and link. Also supports multi-page crawling, persistent cookies, and custom CSS selectors. Runs locally with no API keys or cloud dependencies. Also useful for: - NotebookLM can't add a URL as a source — extract the content and paste it as text - RAG pipelines need clean markdown from web pages, not HTML soup or truncated summaries - LLM conversations where you need the full article in context, not a 3-paragraph summary Built-in agent fetch Cloud extraction APIs agent-fetch ------------------------- --------------------- --------------------- ---------------------------------------------------------------------------------- Content Summary or truncation Full (usually) Full article text Structure Plain text blob Markdown (varies) Markdown with headings, links, lists Runs locally Yes No Yes API key required No Yes No Extraction strategies 1 (basic parse) 1–2 Multiple (Readability, JSON-LD, Next.js, RSC, WP API, text-density, CSS selectors) Open source N/A Partial Yes Install Or run without installing: AI Agents (Claude Code, Codex, Cursor, Copilot)","default_branch":null,"files":null,"tree":[],"storefront":"/r/teng-lin","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/teng-lin/agent-fetch/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}