{"repo":"TrustifAI/trustifai","free":true,"listed":false,"github":"https://github.com/TrustifAI/trustifai","clone":"git clone https://github.com/TrustifAI/trustifai.git","description":"TrustifAI: A Comprehensive Framework for AI Trustworthiness","language":"Python","stars":12,"topics":["agentic-ai","ai","generative-ai","llm","metrics","metrics-visualization","trust","trustworthy-ai","trustifai"],"license":"MIT","category":"ai-agents","readme_excerpt":"TrustifAI 🛡️Quantify, Visualize, and Explain Trust in AI. TrustifAI is a Python-based framework designed to evaluate the trustworthiness of LLM responses and Retrieval-Augmented Generation (RAG) systems. Unlike simple evaluation frameworks that rely on a single \"correctness\" score, TrustifAI computes a multi-dimensional Trust Score based on grounding, consistency, alignment, and diversity. It also includes visualizations to help showcase why a model output was deemed unreliable. 📊 Key Metrics TrustifAI evaluates trustworthiness using four orthogonal vectors or trust signals. The final Trust Score is a weighted aggregation of these components. Offline Metrics (For already generated RAG response) Metric Definition Purpose ------ ------------ --------- Evidence Coverage Segment-level entailment check. The answer is tokenized into sentences and each sentence is verified against retrieved documents using an NLI (Natural Language Inference) approach. Detects hallucinations. Ensures every claim is supported by the provided context. Epistemic Consistency Measures semantic stability ($1 - \\sigma$) across $k$ stochastic generations. Samples $k$ responses at high temperature and computes the mean cosine similarity against the original answer. Detects model inconsistency. Hallucinated answers tend to vary significantly between runs. Semantic Drift Sentence-level maximum similarity between the Answer/Query embedding and embeddings of sentences from the retrieved documents. Also returns ","default_branch":null,"files":null,"tree":[],"storefront":"/r/TrustifAI","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/TrustifAI/trustifai/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}