{"repo":"tracel-ai/burn-lm","free":true,"listed":false,"github":"https://github.com/tracel-ai/burn-lm","clone":"git clone https://github.com/tracel-ai/burn-lm.git","description":"Democratizing large model inference and training on any device.","language":"Rust","stars":267,"topics":["burn","large-model-inference","open-webui","openai-api","rust","tui"],"license":null,"category":"cli-tools","readme_excerpt":"Burn LM --- Burn-LM aims at democratizing large model inference and training on any device. Quick Start Launch a Burn LM shell with: Type help to get a list of commands. Available Models The list of models is very small at the moment since we're focused on performance optimization. Still, we're accepting high quality contributions to port open-source models to Burn-LM. Here's the current list of supported models: Model Size --------- ------ Llama 3 8B Llama 3.1 8B Llama 3.2 1B, 3B TinyLlama 1.1B Adding a New Model Models can be easily integrated with Burn LM by implementing the InferenceServer trait to create a pluggable server that can be added to the Burn LM registry. To bootstrap a new model server you can use the dedicated command new : This will create a new crate named burn-lm-inference-my-model and automatically register it in burn-lm-registry . The bootstraped server is a model-less server that just repeat the prompt it is given. You can also get inspiration from the other crate with the crate burn-lm-llama .","default_branch":null,"files":null,"tree":[],"storefront":"/r/tracel-ai","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/tracel-ai/burn-lm/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}