{"repo":"zhenye234/xcodec","free":true,"listed":false,"github":"https://github.com/zhenye234/xcodec","clone":"git clone https://github.com/zhenye234/xcodec.git","description":"AAAI 2025: Codec Does Matter: Exploring the Semantic Shortcoming of Codec for Audio Language Model","language":"Python","stars":309,"topics":["audio","audio-codec","codec","music","sound","speech","text-to-music","text-to-sound","text-to-speech","tokenizer"],"license":"MIT","category":"media-processing","readme_excerpt":"X-Codec Unified Semantic and Acoustic Codec for Audio Language Model. X-Codec-2.0 released! Paper Title : Codec Does Matter: Exploring the Semantic Shortcoming of Codec for Audio Language Model (AAAI 2025) Authors : Zhen Ye, Peiwen Sun, Jiahe Lei, Hongzhan Lin, Xu Tan, Zheqi Dai, Qiuqiang Kong, Jianyi Chen, Jiahao Pan, Qifeng Liu, Yike Guo , Wei Xue Experiments on VALL-E Highlight You can easily apply our approach to enhance any existing acoustic codec: For example For more details, please refer to our code. Available models X-Codec is part of Hugging Face's Transformers library (see model documentation). Below are the Transformers-compatible checkpoints on the Hugging Face Hub 🤗 Model checkpoint Semantic Model Domain Training Data -------------------------------------------- ----------------------------------------------------------------------- --------------- ------------------------------- xcodec-hubert-librispeech facebook/hubert-base-ls960 Speech Librispeech xcodec-wavlm-mls (not mentioned in paper) microsoft/wavlm-base-plus Speech MLS English xcodec-wavlm-more-data (not mentioned in paper) microsoft/wavlm-base-plus Speech MLS English + Internal data xcodec-hubert-general ZhenYe234/hubert base general audio General audio 200k hours internal data xcodec-hubert-general-balanced (not mentioned in paper) ZhenYe234/hubert base general audio General audio More balanced data Below are the original checkpoints. Model name Hugging Face Config Semantic Model Domain Training Data","default_branch":null,"files":null,"tree":[],"storefront":"/r/zhenye234","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/zhenye234/xcodec/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}