{"repo":"KolosalAI/Kolosal","free":true,"listed":false,"github":"https://github.com/KolosalAI/Kolosal","clone":"git clone https://github.com/KolosalAI/Kolosal.git","description":"Kolosal AI is an OpenSource and Lightweight alternative to LM Studio to run LLMs 100% offline on your device.","language":"C++","stars":455,"topics":["llm","c","cpp","deepseek","gemma","gemma2","gemma3","gpt","llama","llama2"],"license":"Apache-2.0","category":"ai-agents","readme_excerpt":"Kolosal AI https://github.com/user-attachments/assets/589cfb48-f806-493d-842b-3b6953b64e79 Kolosal AI is an open-source desktop application designed to simplify the training and inference of large language models on your own device. It supports any CPU with AVX2 instructions and also works with AMD and NVIDIA GPUs. Built to be lightweight (only 20 MB compiled), Kolosal AI runs smoothly on most edge devices, enabling on-premise or on-edge AI solutions without heavy cloud dependencies. - License: Apache 2.0 - Developer: Genta Technology - Community: Join our Discord Key Features 1. Universal Hardware Support - AVX2-enabled CPUs - AMD and NVIDIA GPUs 2. Lightweight & Portable - Compiled size 20 MB - Ideal for edge devices like Raspberry Pi or low-power machines 3. Wide Model Compatibility - Supports popular models like Mistral , LLaMA , Qwen , and many more - Powered by the Genta Personal Engine built on top of Llama.cpp, you can see the source code at https://github.com/genta-technology/inference-personal 4. Easy Dataset Generation & Training - Build custom datasets with minimal overhead - Train models using UnsLOTH or other frameworks - Deploy locally or as a server in just a few steps 5. On-Premise & On-Edge Focus - Keeps data private on your own infrastructure - Lowers costs by avoiding expensive cloud-based solutions Use Cases - Local AI Inference: Quickly run LLMs on your personal laptop or desktop for offline or on-premise scenarios. - Edge Deployment: Bring large languag","default_branch":null,"files":null,"tree":[],"storefront":"/r/KolosalAI","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/KolosalAI/Kolosal/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}