{"repo":"Jonathan-Pearce/calibration-toolbox","free":true,"listed":false,"github":"https://github.com/Jonathan-Pearce/calibration-toolbox","clone":"git clone https://github.com/Jonathan-Pearce/calibration-toolbox.git","description":"NumPy library for calibration metrics","language":"Python","stars":77,"topics":["calibration","deep-learning","visualization","metrics","recalibration"],"license":"MIT","category":"analytics","readme_excerpt":"Calibration Toolbox Calibration Toolbox is a Python library for evaluating machine learning model calibration using binning-based metrics. The package provides a comprehensive collection of calibration metrics and visualization tools for research in deep learning and uncertainty quantification. Features - Comprehensive Metrics : ECE, MCE, RMSCE, ACE, SCE, and more - General Calibration Error Framework : Flexible GCE function for custom metrics - Framework-Agnostic : Works with NumPy arrays - no PyTorch or TensorFlow required - Visualization Tools : Reliability diagrams, confidence histograms, and class-wise calibration curves - Well-Tested : Extensive test coverage with edge case handling - Research-Oriented : Implementations based on recent research papers Installation From PyPI (once published) From Source Development Installation Quick Start Metrics General Calibration Error (GCE) The GCE is a flexible framework that can compute various calibration metrics through parameter configuration: Where: - $ acc(b,k) $ and $ conf(b,k) $ are the accuracy and confidence of bin $ b $ for class $ k $ - $ n {bk} $ is the number of predictions in bin $ b $ for class $ k $ - $ N $ is the total number of data points - $ K $ is the number of classes - $ p $ is the norm parameter (1, 2, or ∞) Usage: Expected Calibration Error (ECE) Reference: Naeini et al. (2015). \"Obtaining Well Calibrated Probabilities Using Bayesian Binning.\" AAAI. ECE measures the difference between model confidence and ","default_branch":null,"files":null,"tree":[],"storefront":"/r/Jonathan-Pearce","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/Jonathan-Pearce/calibration-toolbox/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}