{"repo":"linto-ai/whisper-timestamped","free":true,"listed":false,"github":"https://github.com/linto-ai/whisper-timestamped","clone":"git clone https://github.com/linto-ai/whisper-timestamped.git","description":"Multilingual Automatic Speech Recognition with word-level timestamps and confidence","language":"Python","stars":2840,"topics":["deep-learning","speech","speech-recognition","speech-to-text","asr","machine-learning","python","python3","pytorch","attention-is-all-you-need"],"license":"AGPL-3.0","category":"machine-learning","readme_excerpt":"whisper-timestamped Multilingual Automatic Speech Recognition with word-level timestamps and confidence. Description Notes on other approaches Installation First installation Additional packages that might be needed Docker Light installation for CPU Upgrade to the latest version Usage Python Command line Utility Functions Plot of word alignment Example output API Reference Main Transcription Function Utility Functions File Writing Functions Options that may improve results Accurate Whisper transcription Running Voice Activity Detection (VAD) before sending to Whisper Detecting disfluencies Acknowledgments/Support Citations Description Whisper is a set of multi-lingual, robust speech recognition models trained by OpenAI that achieve state-of-the-art results in many languages. Whisper models were trained to predict approximate timestamps on speech segments (most of the time with 1-second accuracy), but they cannot originally predict word timestamps. This repository proposes an implementation to predict word timestamps and provide a more accurate estimation of speech segments when transcribing with Whisper models . Besides, a confidence score is assigned to each word and each segment. The approach is based on Dynamic Time Warping (DTW) applied to cross-attention weights, as demonstrated by this notebook by Jong Wook Kim. There are some additions to this notebook: The start/end estimation is more accurate. Confidence scores are assigned to each word. If possible (without beam sea","default_branch":null,"files":null,"tree":[],"storefront":"/r/linto-ai","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/linto-ai/whisper-timestamped/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}