{"repo":"Riddhish1/CogniScrape","free":true,"listed":false,"github":"https://github.com/Riddhish1/CogniScrape","clone":"git clone https://github.com/Riddhish1/CogniScrape.git","description":"Intelligent Web Scraping Library with LLMs","language":"TypeScript","stars":65,"topics":["ai","library","llm","scraper"],"license":"MIT","category":"scrapers-browser-automation","readme_excerpt":"🕷️ CogniScrape Intelligent Web Scraping with LLMs - A TypeScript library that combines traditional web scraping with Large Language Models for intelligent, structured data extraction. ✨ Features - 🤖 Dual LLM Support : Ollama (free/local) + Google Gemini (cloud) - 📊 Graph-Based Architecture : Composable, reusable node pipelines - 🚀 Production-Ready : Built-in caching, retries, rate limiting, and proxy rotation - 🎯 Smart Parsing : Automatic HTML→Markdown conversion and intelligent chunking - ✅ Schema Validation : Zod integration for type-safe outputs - 📝 Multiple Formats : JSON, CSV, XML, PDF support - 🌐 Browser Automation : Playwright for dynamic content - 🧠 RAG Integration : Retrieval-Augmented Generation for better accuracy 📦 Installation 🚀 Quick Start Basic Web Scraping with Gemini Using Ollama (100% Free & Local) 🎯 Available Graphs Graph Purpose Use Case ------- --------- ---------- SmartScraperGraph Basic scraping Extract data from single URL SmartScraperMultiGraph Multi-URL scraping Scrape multiple sources (parallel/sequential) SearchGraph Internet search + scrape Search engines + content extraction DepthSearchGraph Deep analysis Search + reasoning + comprehensive analysis CSVScraperGraph CSV export Scrape data → export to CSV JSONScraperGraph JSON export Schema-validated JSON output 📚 Examples Multi-URL Scraping CSV Export with Schema Validation Internet Search Graph ⚙️ Configuration Options 🔧 Production Features Proxy Rotation Retry with Exponential Backof","default_branch":null,"files":null,"tree":[],"storefront":"/r/Riddhish1","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/Riddhish1/CogniScrape/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}