{"repo":"cameronking4/nextjs-firecrawl-starter","free":true,"listed":false,"github":"https://github.com/cameronking4/nextjs-firecrawl-starter","clone":"git clone https://github.com/cameronking4/nextjs-firecrawl-starter.git","description":"Nextjs 15 Firecrawl app to scrape doc links for an LLM. Use it as a starter kit to build your Firecrawl app. Turn any developer documentation into a GPT knowledge base. Pre-made Github Action to crawl and commit markdown responses to repo.","language":"TypeScript","stars":28,"topics":["docs","firecrawl","github-actions","llm","nextjs","rombo","shadcn-ui","web-scraping"],"license":"MIT","category":"scrapers-browser-automation","readme_excerpt":"Next.js Firecrawl Starter This Nextjs app aims to provide a modern web interface for crawling documentation and processing it for LLM use. Use the output markdown , xml , or zip files to build knowledge files to copy over to a vector database, a ChatGPT GPT, an OpenAI Assistant, Claude Artifacts, Vapi.ai, Aimdoc, or any other LLM tool. The Next app generates a .md file, .xml file, or .zip of markdown files ready for LLM consumption, inspired by the devdocs-to-llm Jupyter notebook by Alex Fazio. Features - 🌐 Serverless architecture using Firecrawl API v1 - ⚡ Real-time crawl status updates - 🎨 Modern UI with dark/light mode support - 📂 Crawl History using Local Storage - 💥 Github Action defined to manually run crawl function and commit to /knowledge bases folder Github Action Use the Github Action template to define automations. Leverage Github Actions cron to schedule crawls for a given site and commit markdown file directly to repo. Manual Trigger https://github.com/user-attachments/assets/fdc0f1a3-1632-44fc-b382-332377d73ed6 Scheduled Crawl (available on Github Marketplace) Add this to any Github Repo to start crawling on a schedule. It will commit the output results automatically after crawling to a specified folder. Default is to crawl Hacker News everyday at midnight and store results in the /knowledge bases folder. OpenAPI Spec & Custom GPT Actions You can use this project to serve endpoints for your LLM tools. In ChatGPT, you can click Create a GPT and then Create A","default_branch":null,"files":null,"tree":[],"storefront":"/r/cameronking4","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/cameronking4/nextjs-firecrawl-starter/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}