{"repo":"benitomartin/llm-observability-opik","free":true,"listed":false,"github":"https://github.com/benitomartin/llm-observability-opik","clone":"git clone https://github.com/benitomartin/llm-observability-opik.git","description":"LLM Evaluation and Observability System for Football Content","language":"Python","stars":31,"topics":["bertscore","comet-ml","cosine-similarity","evaluation-metrics","hallucination","huggingface-transformers","mongodb","openai","pre-commit","python"],"license":"MIT","category":"ai-agents","readme_excerpt":"Football Teams AI Evaluation and Observability A modular pipeline for evaluating and observing LLM performance on football teams' content --- A complete, observable LLM pipeline for evaluating and observing the performance of large language models (LLMs) applied to football teams' content. This project uses Wikipedia data about football teams to generate summaries, create QA datasets , and evaluate model responses using state-of-the-art tools: - ZenML for pipeline orchestration and experiment tracking - MongoDB for structured storage and vector-based retrieval - Opik for LLM evaluation and observability Designed for research , benchmarking , and experimentation , the system is fully configurable, supports semantic search , and enables fine-grained analysis of LLM behavior across a range of evaluation metrics. Overview - ETL Pipeline : Crawl, parse, and ingest Wikipedia articles into MongoDB. - Summarization Pipeline : Generate summaries for each article using LLMs. - Evaluation Pipelines : Score summaries and QA datasets using BERTScore, cosine similarity, answer relevancy, and hallucinations. - Experiment Tracking : Integrated with ZenML and Opik for experiment and metric visualization. - Configurable : Customize settings via YAML and environment variables. Project Structure Getting Started Prerequisites - Python 3.12+ - uv - ZenML - OpenAI - Opik - MongoDB Installation 1. Clone the repository 1. Install dependencies 1. Create MongoDB Account Create an account at MongoDB and","default_branch":null,"files":null,"tree":[],"storefront":"/r/benitomartin","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/benitomartin/llm-observability-opik/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}