{"repo":"swellweb/reame","free":true,"listed":false,"github":"https://github.com/swellweb/reame","clone":"git clone https://github.com/swellweb/reame.git","description":"CPU-first LLM inference server on llama.cpp. Runs useful models on free-tier ARM boxes; rewriting the input made it ~6x faster and more accurate than tuning the engine. MIT, benchmarks and failures included.","language":"C++","stars":107,"topics":["cpu","gguf","inference","kv-cache","llama-cpp","llm","openai-api","speculative-decoding","local-llm","self-hosted"],"license":"MIT","category":"ai-agents","readme_excerpt":"▶ Try the live demo — a real Reame instance serving an LLM from a free Oracle ARM box, in your browser. Reame is an LLM server for ordinary CPUs — the office PC, a €5 VPS, a free-tier cloud box. Your data stays on your machine, and the server gets faster the longer it works for you. Your local LLM re-reads the same text a thousand times. Reame reads it once. No GPU. No per-token bill. One OpenAI-compatible API that any existing client can talk to, with two engines underneath: nucleo , our own engine written from scratch for small models — where it beats llama.cpp — and llama.cpp for the big ones. --- What can I do with it? Put a concierge on your website The flagship use. A visitor writes \"how much would that smooth floor cost for my living room?\" — your price list calls it \"self-levelling epoxy resin for residential interiors\", and the visitor neither knows nor cares. Reame understands that this is what they are asking, and answers \"€35.00 per m²\" in seconds, from a free-tier box. That is what a keyword bot cannot do: bots match rules, and die on the first question phrased in the visitor's own words. Underneath Reame there are real language models — a small one living on the free box, a big one for the hard questions — and no rules to write: feed it your data, and questions arrive the way people actually type them. On top of that, one guarantee a bare LLM cannot give: when the answer is your data (a price, a phone number, an opening time), Reame does not let the model genera","default_branch":null,"files":null,"tree":[],"storefront":"/r/swellweb","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/swellweb/reame/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}