{"repo":"bytedance/SALMONN","free":true,"listed":false,"github":"https://github.com/bytedance/SALMONN","clone":"git clone https://github.com/bytedance/SALMONN.git","description":"SALMONN family: A suite of advanced multi-modal LLMs","language":null,"stars":1506,"topics":["audio","audio-processing","large-language-models","multi-modal","speech","speech-recognition","bytedance","tsinghua-university","music","iclr2024"],"license":"Apache-2.0","category":"media-processing","readme_excerpt":"SALMONN family: A suite of advanced multi-modal LLMs 🚀🚀 Welcome to the repo of SALMONN !! The SALMONN model family consists of a series of advanced multi-modal large language models. For more details, please refer to the corresponding branches. - SALMONN 2 - [[ICLR 2026] ELLSA](https://github.com/bytedance/SALMONN/tree/ELLSA) - video-SALMONN 2 - [[ICML 2025] F-16](https://github.com/bytedance/F-16) - [[ICML 2025] video-SALMONN-o1](https://github.com/bytedance/SALMONN/tree/video-salmonn-o1) - [[ICASSP 2025 & ACL 2025] SALMONN for speech quality assessment](https://github.com/bytedance/SALMONN/tree/speech quality assessment) - [[ICML 2024] video-SALMONN](https://github.com/bytedance/SALMONN/tree/videosalmonn) - [[ICLR 2024] SALMONN](https://github.com/bytedance/SALMONN/tree/salmonn) 🔥 News - [2026-04-20] We have released the model and inference code for ELLSA ! See here! ELLSA is the first end-to-end model that unifies vision, speech, text and action in a streaming full-duplex framework, enabling joint multimodal perception and concurrent generation. - [2025-07-08] We have opensourced video-SALMONN 2 ! video-SALMONN 2 is a powerful audio-visual LLM that generates high-quality audio-visual video captions and achieves competitive performance on general video QA benchmarks. - [2025-06-01] We have opensourced QualiSpeech dataset - A speech quality assessment dataset with natural language reasoning. You can use QualiSpeech to develop your own audio LLM for speech quality assessme","default_branch":null,"files":null,"tree":[],"storefront":"/r/bytedance","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/bytedance/SALMONN/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}