{"repo":"carlosplanchon/spidercreator","free":true,"listed":false,"github":"https://github.com/carlosplanchon/spidercreator","clone":"git clone https://github.com/carlosplanchon/spidercreator.git","description":"Automated web scraping spider generation using Browser Use and LLMs. Streamline the creation of Playwright-based spiders with minimal manual coding. Ideal for large enterprises with recurring data extraction needs.","language":"Python","stars":223,"topics":["ai","automation","browser-use","crawling","llm","python","rpa","scraping","spider","vibe-coding"],"license":"AGPL-3.0","category":"scrapers-browser-automation","readme_excerpt":"Generate Playwright Spiders with AI. Automated web scraping spider generation using Browser Use and LLMs. Generate Playwright spiders with minimal technical expertise. THIS LIBRARY IS HIGHLY EXPERIMENTAL Rationale Extracting data with LLMs is expensive. Spider Creator offers an alternative where LLMs are only used for the spider creation process, and the spiders themselves can then run using traditional methods, which are very cheap. This tradeoff makes it ideal for users seeking affordable, recurring scraping tasks. Costs: We don't have benchmarks yet. However, in internal tests using GPT, initial figures show a cost of around $2.50 per page. If you need to create a spider that takes into account two types of pages (for example, product listings and product details), this would cost around $5. DeepWiki Docs: https://deepwiki.com/carlosplanchon/spidercreator 🚀 Quick Start We recommend using Python 3.13. ⚠️ This package is not on PyPI yet. To get started, clone the repo and run your code from its main folder: Install Playwright: Export your OpenAI API KEY: Generate your Playwright spider: Result: Examples For more working examples, check the examples folder ⚙️ How It Works Main Workflow The user provides a prompt describing the desired task or data. 1. The system, leveraging Browser Use opens a browser and performs the task based on the prompt. The browser activity is recorded. 2. The Spider Creator module generates a web scraper (spider) from the recorded actions. Spider Cre","default_branch":null,"files":null,"tree":[],"storefront":"/r/carlosplanchon","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/carlosplanchon/spidercreator/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}