{"repo":"VectifyAI/PageIndex","free":true,"listed":false,"github":"https://github.com/VectifyAI/PageIndex","clone":"git clone https://github.com/VectifyAI/PageIndex.git","description":"📑 PageIndex: Document Index for Vectorless, Reasoning-based RAG","language":"Python","stars":35206,"topics":["agentic-ai","agents","ai","ai-agents","context-engineering","information-retrieval","llm","rag","reasoning","retrieval","retrieval-augmented-generation","vector-database"],"license":"MIT","category":"rag_framework","readme_excerpt":"
\n\n- 🔥 [**Agentic Vectorless RAG**](https://github.com/VectifyAI/PageIndex/blob/main/examples/agentic_vectorless_rag_demo.py) — A simple agentic, vectorless RAG [example](#-agentic-vectorless-rag-an-example) with *self-hosted PageIndex*, using OpenAI Agents SDK.\n- [**Scale PageIndex to Millions of Documents**](https://pageindex.ai/blog/pageindex-filesystem) — *PageIndex File System* is a file-level tree indexing layer that lets PageIndex reason over an entire corpus, not just a single document, enabling massive-scale document search.\n- [PageIndex Chat](https://chat.pageindex.ai) — Human-like document analysis agent [platform](https://chat.pageindex.ai) for professional long documents. Also available via [MCP](https://pageindex.ai/developer) or [API](https://pageindex.ai/developer).\n- [PageIndex Framework](https://pageindex.ai/blog/pageindex-intro) — Deep dive into PageIndex: an *agentic, in-context tree index* that enables LLMs to perform *reasoning-based, context-aware retrieval* over long documents.\n\n \n\n\n\n---\n\n# 📑 Introduction to PageIndex\n\nAre you frustrated with vector database retrieval accuracy for long professional documents? Traditional vector-based RAG relies on semantic *similarity* rather than true *relevance*. But **similarity ≠ relevance** — what we truly need in retrieval is **relevance**, and that requires **reasoning**. When working with professional documents that demand *contextual understanding*, domain expertise, and multi-step reasoning, similarity search often falls short — missing what's relevant but not similar, and returning what's similar yet not relevant.\n\nInspired by AlphaGo, we propose **[PageIndex](https://vectify.ai/pageindex)** — a **vectorless**, **reasoning-based RAG** system that builds a **hierarchical tree index** from long documents, and uses LLMs to **reason** *over that index* for **agentic, context-aware retrieval**. The retrieval is *traceable* and *explainable*, with no vector DBs or chunking.\nPageIndex simulates how *human experts* navigate and extract knowledge from complex documents through *tree search*, enabling LLMs to *think* and *reason* their way to the most relevant document sections. It performs retrieval in two steps:\n\n1. Generate a “Table-of-Contents” **tree structure index** of documents\n2. Perform (agentic) reasoning-based retrieval through **tree search**\n\n
\n\n### 🎯 Core Features\n\n> PageIndex is a vectorless, reasoning-based RAG engine that mirrors how humans read, delivering traceable, explainable, and context-aware retrieval, without vector databases or chunking.\n\nCompared to traditional vector-based RAG, **PageIndex** features:\n- **No Vector DB**: Uses document structure and LLM reasoning for retrieval, instead of vector similarity search.\n- **No Chunking**: Documents are organized into natural sections, not artificial chunks.\n- **Better Traceability & Explainability**: Retrieval is reasoning-driven and grounded in explicit page and section references, making every result traceable and interpretable — no more “vibe retrieval” with opaque, approximate vector search.\n- **Context-Aware Retrieval**: Retrieval depends on your full context (e.g., conversation history and domain knowledge), and easily incorporates new context.\n- **Human-like Retrieval**: Mirrors how human experts navigate and extract knowledge from complex documents.\n\nPageIndex achieved **state-of-the-art** [98.7% accuracy](https://github.com/VectifyAI/Mafin2.5-FinanceBench) on FinanceBench (financial document QA benchmark), vastly outperforming vector RAG solutions on professional document analysis ([blog post](https://vectify.ai/blog/Mafin2.5)).\n\n### 📍 Explore PageIndex\n\nTo learn more, please see a detailed introduction to the [PageIndex framework](https://pageindex.ai/blog/pageindex-intro). Check out [our GitHub](https://docs.pageindex.ai/open-source) for open-source code, and the [cookbooks](https://docs.pageindex.ai/cookbook), [tutorials](https://docs.pageindex.ai/tutorials), and [blog](https://pageindex.ai/blog) for more usage guides and examples.\n\nThe PageIndex service is available as a ChatGPT-style [chat platform](https://chat.pageindex.ai), or can be integrated via [MCP](https://pageindex.ai/developer) or [API](https://pageindex.ai/developer), with [enterprise](https://pageindex.ai/enterprise) deployment available.\n\n### 🛠️ Deployment Options\n- **Self-host** — run locally with this open-source repo (using standard PDF parsing).\n- **Cloud Service** — production-grade pipeline with enhanced OCR, tree building, and retrieval for best results. Try instantly on our [Chat Platform](https://chat.pageindex.ai/), or integrate via [MCP](https://pageindex.ai/developer) or [API](https://pageindex.ai/developer).\n- **Enterprise** — dedicated or private deployment (VPC, on-prem). [Contact us](https://ii2abc2jejf.typeform.com/to/gVv7qkaN) or [book a demo](https://calendly.com/pageindex/meet) to learn more.\n\n### 🧪 Quick Hands-on\n\n- ⚡ [**PageIndex Flash**](pageindex/flash) *(preview)* — ultra fast PageIndex tree structure generation from PDFs.\n- 🔥 [**Agentic Vectorless RAG**](examples/agentic_vectorless_rag_demo.py) *(latest)* — a simple but complete **agentic vectorless RAG** [example](#-agentic-vectorless-rag-an-example) with *self-hosted* PageIndex, using OpenAI Agents SDK.\n- Try the [Vectorless RAG](https://github.com/VectifyAI/PageIndex/blob/main/cookbook/pageindex_RAG_simple.ipynb) notebook — a *minimal*, hands-on example of reasoning-based RAG using PageIndex.\n- Check out [Vision-based Vectorless RAG](https://github.com/VectifyAI/PageIndex/blob/main/cookbook/vision_RAG_pageindex.ipynb) — no OCR; a minimal, vision-based & reasoning-native RAG pipeline that works directly over page images.\n \n
\n\n---\n\n# 🌲 PageIndex Tree Structure\n\nPageIndex can transform lengthy PDF documents into a semantic **tree structure**, similar to a _“table of contents”_ but optimized for use with LLMs and AI agents. It's ideal for: financial reports, legal documents, regulatory filings, technical manuals, medical literature, academic textbooks, and any long, complex professional documents.\n\nBelow is an example PageIndex tree structure. Also see more example [documents](https://github.com/VectifyAI/PageIndex/tree/main/examples/documents) and generated [tree structures](https://github.com/VectifyAI/PageIndex/tree/main/examples/documents/results).\n\n```jsonc\n...\n{\n \"title\": \"Financial Stability\",\n \"node_id\": \"0006\",\n \"start_index\": 21,\n \"end_index\": 22,\n \"summary\": \"The Federal Reserve ...\",\n \"nodes\": [\n {\n \"title\": \"Monitoring Financial Vulnerabilities\",\n \"node_id\": \"0007\",\n \"start_index\": 22,\n \"end_index\": 28,\n \"summary\": \"The Federal Reserve's monitoring ...\"\n },\n {\n \"title\": \"Domestic and International Cooperation and Coordination\",\n \"node_id\": \"0008\",\n \"start_index\": 28,\n \"end_index\": 31,\n \"summary\": \"In 2023, the Federal Reserve collaborated ...\"\n }\n ]\n}\n...\n```\n\nYou can generate PageIndex tree structures with this open-source repo. Or use our [API](https://pageindex.ai/developer) for higher-quality results powered by our enhanced OCR and tree building","default_branch":"main","files":156,"tree":[".claude/commands/dedupe.md",".gitattributes",".github/dependabot.yml",".github/scripts/autoclose-labeled-issues.js",".github/scripts/comment-on-duplicates.sh",".github/workflows/autoclose-labeled-issues.yml",".github/workflows/backfill-dedupe.yml",".github/workflows/codeql.yml",".github/workflows/dependency-review.yml",".github/workflows/issue-dedupe.yml",".github/workflows/publish.yml",".github/workflows/remove-autoclose-label.yml",".github/workflows/tests.yml",".gitignore","LICENSE","README.md","cookbook/README.md","cookbook/agentic_retrieval.ipynb","cookbook/pageIndex_chat_quickstart.ipynb","cookbook/pageindex_RAG_simple.ipynb","cookbook/vision_RAG_pageindex.ipynb","examples/agentic_vectorless_rag_demo.py","examples/documents/2023-annual-report-truncated.pdf","examples/documents/2023-annual-report.pdf","examples/documents/PRML.pdf","examples/documents/Regulation Best Interest_Interpretive release.pdf","examples/documents/Regulation Best Interest_proposed rule.pdf","examples/documents/attention-residuals.pdf","examples/documents/earthmover.pdf","examples/documents/four-lectures.pdf","examples/documents/q1-fy25-earnings.pdf","examples/documents/results/2023-annual-report-truncated_structure.json","examples/documents/results/2023-annual-report_structure.json","examples/documents/results/PRML_structure.json","examples/documents/results/Regulation Best Interest_Interpretive release_structure.json","examples/documents/results/Regulation Best Interest_proposed rule_structure.json","examples/documents/results/earthmover_structure.json","examples/documents/results/four-lectures_structure.json","examples/documents/results/q1-fy25-earnings_structure.json","examples/tutorials/doc-search/README.md","examples/tutorials/doc-search/description.md","examples/tutorials/doc-search/metadata.md","examples/tutorials/doc-search/semantics.md","examples/tutorials/tree-search/README.md","pageindex/__init__.py","pageindex/_version.py","pageindex/agent_tools.py","pageindex/client.py","pageindex/cloud_api.py","pageindex/config.yaml","pageindex/errors.py","pageindex/flash/README.md","pageindex/flash/__init__.py","pageindex/flash/api.py","pageindex/flash/assets/time_vs_pages.png","pageindex/flash/blocks/__init__.py","pageindex/flash/blocks/build.py","pageindex/flash/blocks/join_rules.py","pageindex/flash/classification/__init__.py","pageindex/flash/classification/body_text.py","pageindex/flash/classification/header_footer.py","pageindex/flash/classification/keyword_tables.py","pageindex/flash/classification/toc_boilerplate.py","pageindex/flash/clustering/__init__.py","pageindex/flash/clustering/build.py","pageindex/flash/clustering/merge_rules.py","pageindex/flash/columns/__init__.py","pageindex/flash/columns/gutters.py","pageindex/flash/columns/splitting.py","pageindex/flash/data/boilerplate_phrases.json","pageindex/flash/data/dictionaries.json","pageindex/flash/data/glyph_name_table.json","pageindex/flash/data/normalized_unicodes.json","pageindex/flash/data/script_bucket_table.json","pageindex/flash/embedded_toc.py","pageindex/flash/heading_detection/__init__.py","pageindex/flash/heading_detection/candidates.py","pageindex/flash/heading_detection/detectors.py","pageindex/flash/heading_detection/keyword_tables.py","pageindex/flash/heading_detection/neighbors.py","pageindex/flash/heading_detection/page_scan.py","pageindex/flash/heading_detection/style_detectors.py","pageindex/flash/heading_detection/text_checks.py","pageindex/flash/labels/__init__.py","pageindex/flash/labels/caption_regions.py","pageindex/flash/labels/caption_text.py","pageindex/flash/main.py","pageindex/flash/model/__init__.py","pageindex/flash/model/block.py","pageindex/flash/model/char_stats.py","pageindex/flash/model/numbering.py","pageindex/flash/model/rects.py","pageindex/flash/model/span_line.py","pageindex/flash/outline/__init__.py","pageindex/flash/outline/filtering.py","pageindex/flash/outline/tree.py","pageindex/flash/outline_assembly/__init__.py","pageindex/flash/outline_assembly/assembly.py","pageindex/flash/outline_assembly/candidates.py","pageindex/flash/outline_assembly/cliques.py","pageindex/flash/outline_assembly/selection.py","pageindex/flash/outline_assembly/style_context.py","pageindex/flash/parser_pdfium_charlevel/__init__.py","pageindex/flash/parser_pdfium_charlevel/char_extract.py","pageindex/flash/parser_pdfium_charlevel/cmap_parse.py","pageindex/flash/parser_pdfium_charlevel/code_walk.py","pageindex/flash/parser_pdfium_charlevel/content_stream.py","pageindex/flash/parser_pdfium_charlevel/font_unicode.py","pageindex/flash/parser_pdfium_charlevel/geometry.py","pageindex/flash/parser_pdfium_charlevel/glyph_tables.py","pageindex/flash/parser_pdfium_charlevel/merge.py","pageindex/flash/parser_pdfium_charlevel/pdf_objects.py","pageindex/flash/parser_pdfium_charlevel/pipeline.py","pageindex/flash/parser_pdfium_charlevel/remerge.py","pageindex/flash/parser_pdfium_charlevel/text_normalize.py","pageindex/flash/parser_pdfium_charlevel/unicode_apply.py","pageindex/flash/parser_pdfium_parallel.py","pageindex/flash/phases/__init__.py","pageindex/flash/phases/line_numbers.py","pageindex/flash/phases/page_view.py","pageindex/flash/stats/__init__.py","pageindex/flash/stats/aggregates.py","pageindex/flash/stats/scripts.py","pageindex/flash/title/__init__.py","pageindex/flash/title/detect.py","pageindex/flash/title/dicts.py","pageindex/flash/title/scoring.py","pageindex/flash/tokens/__init__.py","pageindex/flash/tokens/hashing.py","pageindex/flash/tokens/token_types.py","pageindex/flash/tokens/tokenizer.py","pageindex/flash/tokens/tries.py","pageindex/integrations/__init__.py","pageindex/integrations/anthropic_sdk.py","pageindex/integrations/claude_agent_sdk.py","pageindex/integrations/openai_agents.py","pageindex/local_api.py","pageindex/local_chat.py","pageindex/local_store.py","pageindex/mcp_bridge.py","pageindex/page_index_classic.py","pageindex/page_index_md.py","pageindex/tree_optimize.py","pageindex/utils.py","pyproject.toml","requirements.txt","run_pageindex.py","tests/conftest.py","tests/data/cloud_mcp_contract.json","tests/test_agent_tools.py","tests/test_client.py","tests/test_issue_163.py","tests/test_local_chat.py","tests/test_package_surface.py","tests/test_page_index.py","tests/test_page_index_md.py"],"storefront":"/r/VectifyAI","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/VectifyAI/PageIndex/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}
![](\"https://docs.pageindex.ai/images/cookbook/vectorless-rag.png\")
\n \n![\"View](\"https://img.shields.io/badge/View_on_GitHub-Agentic_Vectorless_RAG-blue?style=for-the-badge&logo=github\"){kind=icon}
\n ![\"Open](\"https://img.shields.io/badge/Open_In_Colab-Vectorless_RAG-orange?style=for-the-badge&logo=googlecolab\"){kind=icon}
\n ![\"Open](\"https://img.shields.io/badge/Open_In_Colab-Vision_RAG-orange?style=for-the-badge&logo=googlecolab\"){kind=icon}
\n