{"repo":"modal-labs/quillman","free":true,"listed":false,"github":"https://github.com/modal-labs/quillman","clone":"git clone https://github.com/modal-labs/quillman.git","description":"A voice chat app","language":"Python","stars":1212,"topics":["language-model","serverless","speech-recognition","speech-to-text","ai","python"],"license":"MIT","category":"chat-messaging","readme_excerpt":"QuiLLMan: Voice Chat with Moshi A complete voice chat app powered by a speech-to-speech language model and bidirectional streaming. On the backend is Kyutai Lab's Moshi model, which will continuously listen, plan, and respond to the user. It uses the Mimi streaming encoder/decoder model to maintain an unbroken stream of audio in and out, and a speech-text foundation model to determine when and how to respond. Thanks to bidirectional websocket streaming and use of the Opus audio codec for compressing audio across the network, response times on good internet can be nearly instantaneous, closely matching the cadence of human speech. You can find the demo live here. This repo is meant to serve as a starting point for your own language model-based apps, as well as a playground for experimentation. Contributions are welcome and encouraged! [Note: this code is provided for illustration only; please remember to check the license before using any model for commercial purposes.] File structure 1. React frontend ( src/frontend/ ), served by src/app.py 2. Moshi websocket server ( src/moshi.py ) Developing locally Requirements - modal installed in your current Python virtual environment ( pip install modal ) - A Modal account ( modal setup ) - A Modal token set up in your environment ( modal token new ) Developing the inference module The Moshi server is a Modal class module to load the models and maintain streaming state, with a FastAPI http server to expose a websocket interface over th","default_branch":null,"files":null,"tree":[],"storefront":"/r/modal-labs","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/modal-labs/quillman/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}