{"repo":"groktopus/groktocrawl","free":true,"listed":false,"github":"https://github.com/groktopus/groktocrawl","clone":"git clone https://github.com/groktopus/groktocrawl.git","description":"Self-hosted, API-compatible Firecrawl alternative with Agent endpoint. MIT license. One docker compose up.","language":"Python","stars":72,"topics":[],"license":"MIT","category":"self-hosted-apps","readme_excerpt":"GroktoCrawl GroktoCrawl is a self-hosted, MIT-licensed web data platform compatible with the Firecrawl v2 API surface. It combines scraping, crawl and map jobs, search, structured extraction, browser automation, monitors, semantic retrieval, an autonomous research agent, and an MCP server in one Docker deployment. Start here Choose your path. Both run the same Docker stack defined in docker-compose.yml ; they differ in which services and credentials you configure. Local demo (fixture profile) The fastest end-to-end smoke test, with no external credentials. The fixture profile starts a local LLM fixture ( llm-svc ) and two fixture test sites ( test-site , tier3-fixture ). Config: copy .env.sample to .env . No credentials are required; the optional direct SlopSearX MCP companion ( slopsearx-mcp ) is opt-in via the mcp Compose profile and only then needs a non-empty SLOPSEARX MCP AUTH TOKEN . Expected success output: curl http://localhost:8080/health returns {\"status\":\"ok\", \"checks\":{...}} once all dependencies are up, and ./groktocrawl scrape https://example.com prints example.com rendered as markdown. Production minimum (no fixture profile) The same stack pointed at a real LLM provider, an open-web search backend, and a hardened API — without the fixture-only services. Config (in .env ): an OpenAI-compatible LLM provider ( LLM BASE URL , LLM API KEY , LLM MODEL ), BRAVE API KEY for web search, and API KEY for API authentication. llm-svc , test-site , and tier3-fixture are fixt","default_branch":null,"files":null,"tree":[],"storefront":"/r/groktopus","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/groktopus/groktocrawl/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}