{"repo":"hybridgroup/yzma","free":true,"listed":false,"github":"https://github.com/hybridgroup/yzma","clone":"git clone https://github.com/hybridgroup/yzma.git","description":"Go with your own intelligence - Go applications that directly integrate llama.cpp for local inference using hardware acceleration.","language":"Go","stars":559,"topics":["golang","llama","llamacpp","llm","purego","vlm","cuda","metal","vulkan","rocm"],"license":null,"category":"ai-agents","readme_excerpt":"yzma - Go with your own intelligence yzma lets you write Go applications that directly integrate llama.cpp for fully local inference using hardware acceleration. - Run the latest Vision Language Models (VLM) and Large/Small/Tiny Language Models (LLM) on Linux, macOS, or Windows. - Use any available hardware acceleration such as CUDA, Metal), or Vulkan for maximum performance. - yzma uses the purego and ffi packages so CGo is not needed. - Works with the newest llama.cpp releases so you can use the latest features, performance improvements, and bugfixes. This example uses the SmolLM2-135M-GGUF model: Install yzma , then download the model using the yzma command line tool: And run the Go program: Installation You can use the convenient yzma command line tool to download the llama.cpp prebuilt libraries for your platform. You can also have your application self-download them automatically at installation time, including auto-detection for CUDA and ROCm. See INSTALL.md for installation instructions for macOS, Linux, and Windows. We also have specific information on running yzma on Raspberry Pi, NVIDIA Jetson Orin, and the Arduino UNO Q. Examples We have several examples of how you can use yzma in our examples directory. Vision Language Model (VLM) Multimodal Example This example uses the Qwen2.5-VL-3B-Instruct-Q8 0 VLM model to process both a text prompt and an image, then displays the result. See the code here. Small Language Model (SLM) Interactive Chat Example You can use yzma","default_branch":null,"files":null,"tree":[],"storefront":"/r/hybridgroup","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/hybridgroup/yzma/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}