{"repo":"jorge-martinez-gil/proof-of-deliberation","free":true,"listed":false,"github":"https://github.com/jorge-martinez-gil/proof-of-deliberation","clone":"git clone https://github.com/jorge-martinez-gil/proof-of-deliberation.git","description":"Verifying Human Annotation Reliability from Interaction Traces","language":"Python","stars":40,"topics":["active-learning","data-engineering","human-factors","human-in-the-loop"],"license":"MIT","category":"data-pipelines","readme_excerpt":"🧠 Proof-of-Deliberation (PoD) Process-based supervision certification for streaming Human-in-the-Loop active learning --- PoD attaches a credibility signal to every human-generated label using observable interaction traces , deliberation time and its coupling with task difficulty, rather than the label content itself, and gates model updates accordingly. This repository is the official, reproducible companion to the paper. It packages the protocol as an installable Python library ( pod ), exposes console-script entry points, and ships frozen configurations, unit tests, and per-dataset reproduction recipes. --- 📋 Contents - ⚡ Quickstart - 🗂️ Repository layout - 🔬 The PoD protocol at a glance - 📊 Reproducing the paper - ✈️ Hick-Hyman validation on ATC logs - 👥 Real-user labeling study - 🛠️ Development - 🐳 Docker - 🧾 Artifact evaluation - 📡 Real-time PoD dashboard - 📖 Citation - 📜 License --- ⚡ Quickstart Step 1 — Clone and install Step 2 — Reproduce the F1 trajectory figures from Section 6 Step 3 (optional) — Hick-Hyman validation (Section 6.5) For developers (tests, lint, type-check): --- 🗂️ Repository layout Generated experiment outputs ( out / ) and downloaded datasets ( data cache uci224/ , etc.) are produced on demand and intentionally not tracked. --- 🔬 The PoD protocol at a glance PoD checks three independent signals for each queried label. A label is incorporated into the learner only when all three checks agree : Layer Symbol Implementation --- --- --- De","default_branch":null,"files":null,"tree":[],"storefront":"/r/jorge-martinez-gil","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/jorge-martinez-gil/proof-of-deliberation/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}