{"repo":"loomax-labs/InferrLM","free":true,"listed":false,"github":"https://github.com/loomax-labs/InferrLM","clone":"git clone https://github.com/loomax-labs/InferrLM.git","description":"InferrLM - On-device AI for iOS & Android","language":"TypeScript","stars":105,"topics":["anthropic","document-processing","edge-ai","embeddings","gemini","gguf","http-server","llama-cpp","llamacpp","local-inference"],"license":"AGPL-3.0","category":"networking-infra","readme_excerpt":"InferrLM (Previously Inferra) InferrLM is a mobile application that brings LLMs & SLMs directly to your Android & iOS device and lets your device act as a local server. Cloud-based models like Claude, Gemini and ChatGPT are also supported. File attachments with RAG are also well-supported for local models. If you want to support me and the development of this project, you can donate to me through Ko-fi. Demo Demos of using the Apple Foundation Model and downloading MLX models from HuggingFace and running them on-device. Features Core Inference - Core local inference through llama.cpp with support for GGUF models on both Android and iOS. - Local MLX inference for Apple Silicon devices. - Seamless integration with cloud-based models from OpenAI, Gemini, and Anthropic. You need your own API keys and an InferrLM registered account for remote models. Using remote models is optional. - Customizable base URLs for OpenAI-compatible providers like OpenRouter, Groq, Ollama, LM Studio, Together AI. This allows you to access alternative API endpoints within the app. - Apple Foundation model support for Apple Intelligence supported devices. Vision and Multimodal - Vision support through multimodal models with their corresponding projector (mmproj) files. You can read more about them here. - Built-in camera lets you capture pictures directly within the app and send them to models. Document Processing and RAG - RAG (Retrieval-Augmented Generation) support for enhanced document understanding","default_branch":null,"files":null,"tree":[],"storefront":"/r/loomax-labs","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/loomax-labs/InferrLM/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}