{"repo":"axiomhq/hyperloglog","free":true,"listed":false,"github":"https://github.com/axiomhq/hyperloglog","clone":"git clone https://github.com/axiomhq/hyperloglog.git","description":"HyperLogLog with lots of sugar (Sparse, LogLog-Beta bias correction and TailCut space reduction) brought to you by Axiom","language":"Go","stars":1046,"topics":["axiom","data-structures","go","golang","hyperloglog"],"license":"MIT","category":"dev-tools","readme_excerpt":"HyperLogLog - an algorithm for approximating the number of distinct elements An improved version of HyperLogLog for the count-distinct problem, approximating the number of distinct elements in a multiset. This implementation offers enhanced performance, flexibility, and simplicity while maintaining accuracy. Note on Implementation History The initial version of this work (tagged as v0.1.0) was based on \"Better with fewer bits: Improving the performance of cardinality estimation of large data streams - Qingjun Xiao, You Zhou, Shigang Chen\". However, the current implementation has evolved significantly from this original basis, notably moving away from the tailcut method. Current Implementation The current implementation is based on the LogLog-Beta algorithm, as described in: \"LogLog-Beta and More: A New Algorithm for Cardinality Estimation Based on LogLog Counting\" by Jason Qin, Denys Kim, and Yumei Tung (2016). Key features of the current implementation: Metro hash used instead of xxhash Sparse representation for lower cardinalities (like HyperLogLog++) LogLog-Beta for dynamic bias correction across all cardinalities 8-bit registers for convenience and simplified implementation Order-independent insertions and merging for consistent results regardless of data input order Removal of tailcut method for a more straightforward approach Flexible precision allowing for 2^4 to 2^18 registers This implementation is now more straightforward, efficient, and flexible, while remaining ba","default_branch":null,"files":null,"tree":[],"storefront":"/r/axiomhq","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/axiomhq/hyperloglog/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}