{"repo":"ufal/factgenie","free":true,"listed":false,"github":"https://github.com/ufal/factgenie","clone":"git clone https://github.com/ufal/factgenie.git","description":"Lightweight self-hosted span annotation tool","language":"Python","stars":44,"topics":["annotations","data-annotation","llm","visualization","annotation","annotation-tool","web-interface","token-classification","word-classification","span-labeling"],"license":"MIT","category":"ai-agents","readme_excerpt":"factgenie Annotate LLM outputs with a lightweight, self-hosted web application 🌈 📢 News - 2026-01-15 - Version 1.2.1 is released, see the 👉️ changelog. - 2025-03-06 - Release 1.1.0 is out and comes with many significant improvements! See the 👉️ changelog. - 2024-11-13 - We released version 1.0.1: our first official release! 🎉 👉️ How can factgenie help you? Outputs from large language models (LLMs) may contain errors: semantic, factual, and lexical. With factgenie, you can have the error spans annotated: - From LLMs through an API. - From humans through a crowdsourcing service. Factgenie can provide you: 1. A user-friendly website for collecting annotations from human crowdworkers. 2. API calls for collecting equivalent annotations from LLM-based evaluators. 3. A visualization interface for visualizing the data and inspecting the annotated outputs. --- What does factgenie not help with is collecting the data (we assume that you already have these), starting the crowdsourcing campaign (for that, you need to use a service such as Prolific.com) or running the LLM evaluators (for that, you need a local framework such as Ollama or a proprietary API). 🏃 Quickstart Make sure you have Python =3.9 installed. If you want to quickly try out factgenie, you can install the package from PyPI: However, the recommended approach for using factgenie is using an editable package: This approach will allow you to manually modify configuration files and write your own data classes. After ins","default_branch":null,"files":null,"tree":[],"storefront":"/r/ufal","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/ufal/factgenie/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}