\n
![](\"./core/http/static/logo.png\"){kind=logo}
\n
\n
\n\n
\nDeutsch |\nEspañol |\nfrançais |\n日本語 |\n한국어 |\nPortuguês |\nРусский |\n中文\n
\n\n**LocalAI** is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.\n\n**A small core, not a bundle.** Each backend wraps a best-in-class engine (llama.cpp, vLLM, whisper.cpp, stable-diffusion, MLX...) in its own image, pulled only when a model needs it. You install nothing you don't use.\n\n- **Composable by design**: backends are separate and pulled on demand, so you install only what your model needs\n- **Open and extensible**: load any model, or build your own backend in any language against an open interface\n- **Drop-in API compatibility**: OpenAI, Anthropic, and ElevenLabs APIs across every backend\n- **Any model, any modality**: LLMs, vision, voice, image, and video behind one API\n- **Any hardware**: NVIDIA, AMD, Intel, Apple Silicon, Vulkan, or CPU-only\n- **Multi-user ready**: API key auth, user quotas, role-based access\n- **Built-in AI agents**: autonomous agents with tool use, RAG, MCP, and skills\n- **Privacy-first**: your data never leaves your infrastructure\n\n\n\nCreated by [Ettore Di Giacinto](https://github.com/mudler) and maintained by the [LocalAI team](#team).\n\n> [:book: Documentation](https://localai.io/) | [:speech_balloon: Discord](https://discord.gg/uJAeKSAGDy) | [💻 Quickstart](https://localai.io/basics/getting_started/) | [🖼️ Models](https://models.localai.io/) | [❓FAQ](https://localai.io/faq/)\n\n## Guided tour\n\nhttps://github.com/user-attachments/assets/08cbb692-57da-48f7-963d-2e7b43883c18\n\n![\"Download](\"https://img.shields.io/badge/Download-macOS-blue?style=for-the-badge&logo=apple&logoColor=white\"){kind=icon}
\n\n\n> **Note:** The DMG is not signed by Apple. After installing, run: `sudo xattr -d com.apple.quarantine /Applications/LocalAI.app`. See [#6268](https://github.com/mudler/LocalAI/issues/6268) for details.\n\n### Containers (Docker, podman, ...)\n\n> Already ran LocalAI before? Use `docker start -i local-ai` to restart an existing container.\n\n#### CPU only:\n\n```bash\ndocker run -ti --name local-ai -p 8080:8080 localai/localai:latest\n```\n\n#### NVIDIA GPU:\n\n```bash\n# CUDA 13\ndocker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-13\n\n# CUDA 12\ndocker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12\n\n# NVIDIA Jetson ARM64 (CUDA 12, for AGX Orin and similar)\ndocker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-nvidia-l4t-arm64\n\n# NVIDIA Jetson ARM64 (CUDA 13, for DGX Spark)\ndocker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-nvidia-l4t-arm64-cuda-13\n```\n\n#### AMD GPU (ROCm):\n\n```bash\ndocker run -ti --name local-ai -p 8080:8080 --device=/dev/kfd --device=/dev/dri --group-add=video localai/localai:latest-gpu-hipblas\n```\n\n#### Intel GPU (oneAPI):\n\n```bash\ndocker run -ti --name local-ai -p 8080:8080 --device=/dev/dri/card1 --device=/dev/dri/renderD128 localai/localai:latest-gpu-intel\n```\n\n#### Vulkan GPU:\n\n```bash\ndocker run -ti --name local-ai -p 8080:8080 localai/localai:latest-gpu-vulkan\n```\n\n### Loading models\n\n```bash\n# From the model gallery (see available models with `local-ai models list` or at https://models.localai.io)\nlocal-ai run llama-3.2-1b-instruct:q4_k_m\n# From Huggingface\nlocal-ai run huggingface://TheBloke/phi-2-GGUF/phi-2.Q8_0.gguf\n# From the Ollama OCI registry\nlocal-ai run ollama://gemma:2b\n# From a YAML config\nlocal-ai run https://gist.githubusercontent.com/.../phi-2.yaml\n# From a standard OCI registry (e.g., Docker Hub)\nlocal-ai run oci://localai/phi-2:latest\n```\n\nTo work with a running LocalAI server from the terminal, start the built-in agent from another shell. It answers questions, reads your files and runs commands on your machine, asking you to approve anything that changes state. Inside a session, `/models` lists installed models and `/model 
\n![\"LocalAI](\"https://img.shields.io/badge/License-MIT-yellow.svg?style=for-the-badge\"){kind=icon}
\n![\"Follow](\"https://img.shields.io/badge/X-%23000000.svg?style=for-the-badge&logo=X&logoColor=white&label=LocalAI_API\"){kind=icon}
\n![\"Join](\"https://img.shields.io/badge/dynamic/json?color=blue&label=Discord&style=for-the-badge&query=approximate_member_count&url=https%3A%2F%2Fdiscordapp.com%2Fapi%2Finvites%2FuJAeKSAGDy%3Fwith_counts%3Dtrue&logo=discord\"){kind=icon}
\n