{"repo":"bhope/hedge","free":true,"listed":false,"github":"https://github.com/bhope/hedge","clone":"git clone https://github.com/bhope/hedge.git","description":"Adaptive hedged requests for Go. Cut your p99 latency with zero configuration. Based on Google's \"The Tail at Scale\" paper.","language":"Go","stars":285,"topics":["concurrency","go","golang","golang-library","grpc","hedging","microservices","performance","tail-latency"],"license":"MIT","category":"networking-infra","readme_excerpt":"hedge In a fan-out architecture with 100 backends, 63% of your requests hit at least one straggler (1 - 0.99^100). hedge learns per-host latency distributions using DDSketch, fires a backup request when the primary exceeds its estimated p90, and caps hedge rate with a token bucket to prevent load amplification during outages. Result: p99 drops from 64ms to 17ms - a 74% reduction - with 9% overhead. Zero configuration. Supports streaming workloads including LLM inference (measures time-to-first-token, not just headers). Based on Dean & Barroso, \"The Tail at Scale\" (CACM 2013) and Masson et al., \"DDSketch\" (VLDB 2019). Quick Start Table of Contents - Quick Start - Evaluation - How it works - Why not a static threshold? - Streaming / LLM inference - Benchmark results - End-to-end latency - gRPC support - Configuration - Observability - References - Contributing - License Evaluation 50,000 requests against a simulated backend with lognormal base latency (mean=5ms, stddev=2ms) and 5% straggler probability (10x multiplier). Configuration p50 p90 p95 p99 p999 Overhead ------------------ ------- ------- -------- -------- --------- ---------- No hedging 5.0ms 8.9ms 17.1ms 64.3ms 104.5ms 0.0% Static 10ms 5.0ms 8.9ms 13.1ms 17.4ms 46.5ms 7.4% Static 50ms 5.0ms 8.9ms 18.9ms 54.7ms 60.2ms 2.1% Adaptive (hedge) 5.0ms 8.9ms 12.3ms 17.0ms 59.4ms 8.9% Adaptive matches the best hand-tuned static threshold at p99 (17.0ms vs 17.4ms) with no manual configuration. Static 50ms is too conservative a","default_branch":null,"files":null,"tree":[],"storefront":"/r/bhope","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/bhope/hedge/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}