{"repo":"Labbeti/aac-metrics","free":true,"listed":false,"github":"https://github.com/Labbeti/aac-metrics","clone":"git clone https://github.com/Labbeti/aac-metrics.git","description":"Metrics for evaluating Automated Audio Captioning systems, designed for PyTorch.","language":"Python","stars":75,"topics":["audio","audio-captioning","captioning","metrics","text"],"license":"MIT","category":"analytics","readme_excerpt":"Audio Captioning metrics (aac-metrics) Metrics for evaluating Automated Audio Captioning systems, designed for PyTorch. Why using this package? - Easy to install and download - Produces same results than caption-evaluation-tools, fense and mace repositories - Provides 15 different metrics: - BLEU [[1]](#bleu) - ROUGE-L [[2]](#rouge-l) - METEOR [[3]](#meteor) - CIDEr-D [[4]](#cider) - SPICE [[5]](#spice) - SPIDEr [[6]](#spider) - BERTScore [[7]](#bertscore) - SPIDEr-max [[8]](#spider-max) - SBERT-sim [[9]](#fense) - FER [[9]](#fense) - FENSE [[9]](#fense) - SPIDEr-FL [[10]](#spider-fl) - CLAP-sim [[11]](#clap-sim) :new: - MACE [[12]](#mace) :new: - Vocab (unique word vocabulary absolute or relative to references) Installation Install the pip package: If you want to check if the package has been installed and the version, you can use this command: Download the external code and models needed for METEOR, SPICE, SPIDEr, SPIDEr-max, PTBTokenizer, SBERTSim, FER, FENSE and SPIDEr-FL: Notes: - The external code for SPICE, METEOR and PTBTokenizer is stored in /.cache/aac-metrics . - The weights of the FENSE fluency error detector and the the SBERT model are respectively stored by default in /.cache/torch/hub/fense data and /.cache/torch/sentence transformers . Usage Evaluate default metrics The full evaluation pipeline to compute AAC metrics can be done with aac metrics.evaluate function. Evaluate DCASE2024 metrics To compute metrics for the DCASE2023 challenge, just set the argument ","default_branch":null,"files":null,"tree":[],"storefront":"/r/Labbeti","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/Labbeti/aac-metrics/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}