{"repo":"Omkar-ratzar/Gnosis","free":true,"listed":false,"github":"https://github.com/Omkar-ratzar/Gnosis","clone":"git clone https://github.com/Omkar-ratzar/Gnosis.git","description":"A lightweight semantic search engine","language":"Python","stars":11,"topics":["embedding","ollama","pipeline","python","semantic","semantic-search","vector","vector-search","airflow","apache-airflow"],"license":"MIT","category":"data-pipelines","readme_excerpt":"Gnosis A Next-Generation Multimodal Semantic Search Pipeline Orchestrated with Apache Airflow. Vision The goal is to make unstructured data easily searchable without manual tagging or strict organization. Real-world storage systems become messy over time. Gnosis automatically understands and indexes content—documents and images—turning raw data into a searchable knowledge base. Complete Architecture Revamp Gnosis has transitioned from a basic file-system watchdog model to a robust, Apache Airflow-based orchestration system. This iteration guarantees reliable pipelines, robust retry mechanisms, scalable processing, and transparent visual monitoring. The core processing logic is cleanly decoupled and served seamlessly through an interactive API and frontend web UI. How It Works Gnosis operates seamlessly using dedicated processing pipelines and robust backing services: 1. Robust File Ingestion - Files are securely uploaded through the FastAPI-powered backend ( /api/upload ). - Artifacts securely store in app/data/ prior to pipeline ingestion. 2. Airflow Orchestration and Pipelines - Replaced legacy watchdog.py tracking by stringing together ingestion and computation tasks into reliable DAGs ( airflow/dags ). - Orchestrates explicit processing pathways via Document ( run docs ) and Image ( run images ) Pipelines. 3. Image Preprocessing & Understanding - Generates structured descriptions (via MiniCPM-V and Ollama). - Aggregates and stores EXIF metadata. - Connects non-textual ima","default_branch":null,"files":null,"tree":[],"storefront":"/r/Omkar-ratzar","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/Omkar-ratzar/Gnosis/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}