{"repo":"wsmlby/homl","free":true,"listed":false,"github":"https://github.com/wsmlby/homl","clone":"git clone https://github.com/wsmlby/homl.git","description":"The easiest & fastest way to run LLMs in your home lab","language":"Python","stars":90,"topics":["homelab","llm","localllm","openai"],"license":"Apache-2.0","category":"ai-agents","readme_excerpt":"HoML: The best of Ollama and vLLM Deprecated, Docker Model Runner is the preferred way to run llms with vLLM. HoML aims to combine the ease of use of Ollama with the high-performance inference capabilities of vLLM. This project provides a simple and intuitive command-line interface (CLI) to run and manage large language models, powered by the speed and compatibility of vLLM. Project Status Beta: This project is in beta. All planned features are implemented, but there may still be some bugs. We welcome feedback and contributions! Why HoML? Ollama has set a new standard for ease of use in running local LLMs. Its simple CLI and user-friendly approach have made it incredibly popular. On the other hand, vLLM is a state-of-the-art inference engine known for its high throughput and broad compatibility with models from the Hugging Face Hub. HoML brings the best of both worlds together, offering: An Ollama-like experience: A simple, intuitive CLI that just works. High-performance inference: Powered by vLLM for maximum speed. Broad model compatibility: Access to a vast range of models from the Hugging Face Hub. Features One-Line Installation: A simple, one-line script for easy installation and upgrades across a wide range of machines. Simple CLI: An intuitive command-line interface for managing and running models. Easy Model Management: A pull command to download models from the Hugging Face Hub. Automatic GPU Memory Management: HoML intelligently manages your GPU memory. Models are au","default_branch":null,"files":null,"tree":[],"storefront":"/r/wsmlby","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/wsmlby/homl/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}