{"repo":"alizdavoodi/MCPDocSearch","free":true,"listed":false,"github":"https://github.com/alizdavoodi/MCPDocSearch","clone":"git clone https://github.com/alizdavoodi/MCPDocSearch.git","description":"This project provides a toolset to crawl websites wikis, tool/library documentions and generate Markdown documentation, and make that documentation searchable via a Model Context Protocol (MCP) server, designed for integration with tools like Cursor.","language":"Python","stars":42,"topics":["crawler","mcp","mcp-server"],"license":"MIT","category":"mcp-servers","readme_excerpt":"Documentation Crawler & MCP Server This project provides a toolset to crawl websites, generate Markdown documentation, and make that documentation searchable via a Model Context Protocol (MCP) server, designed for integration with tools like Cursor. Features - Web Crawler ( crawler cli ) : - Crawls websites starting from a given URL using crawl4ai . - Configurable crawl depth, URL patterns (include/exclude), content types, etc. - Optional cleaning of HTML before Markdown conversion (removes nav links, headers, footers). - Generates a single, consolidated Markdown file from crawled content. - Saves output to ./storage/ by default. - MCP Server ( mcp server ) : - Loads Markdown files from the ./storage/ directory. - Parses Markdown into semantic chunks based on headings. - Generates vector embeddings for each chunk using sentence-transformers ( multi-qa-mpnet-base-dot-v1 ). - Caching: Utilizes a cache file ( storage/document chunks cache.pkl ) to store processed chunks and embeddings. - First Run: The initial server startup after crawling new documents may take some time as it needs to parse, chunk, and generate embeddings for all content. - Subsequent Runs: If the cache file exists and the modification times of the source .md files in ./storage/ haven't changed, the server loads directly from the cache, resulting in much faster startup times. - Cache Invalidation: The cache is automatically invalidated and regenerated if any .md file in ./storage/ is modified, added, or remove","default_branch":null,"files":null,"tree":[],"storefront":"/r/alizdavoodi","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/alizdavoodi/MCPDocSearch/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}