{"repo":"redbco/infermesh","free":true,"listed":false,"github":"https://github.com/redbco/infermesh","clone":"git clone https://github.com/redbco/infermesh.git","description":"GPU-aware inference mesh for large-scale AI serving","language":"Rust","stars":35,"topics":["ai-inference","ai-infrastructure","distributed-systems","fault-tolerance","gpu-inference","high-availability","inference-engine","ml-infrastructure","model-serving","observability"],"license":"AGPL-3.0","category":"analytics","readme_excerpt":"InferMesh InferMesh is a GPU- and network-aware mesh for large-scale AI serving and training . It provides a distributed control plane, topology- and GPU-aware routing, and standardized observability across heterogeneous environments. --- 💡 Why InferMesh is different? InferMesh treats inference and training as first-class citizens on a single mesh fabric: a topology-aware underlay (gossip, Raft, secure networking, link coordinates) with a workload overlay (routers, schedulers, adapters) and a pluggable strategy host. Unlike point tools (serving frameworks, trainers, or generic service meshes), it makes GPU, runtime, and network signals actionable in real time for both request routing and collective topology, and lets you swap in user-defined strategies (Rust or WASM) that run under strict latency budgets with built-in A/B and observability. It scales across clusters and regions with cell summaries, supports edge-to-cloud paths, and optimizes for SLOs, utilization, and cost—all through one consistent API and label model. --- ✨ What is InferMesh? Modern AI at scale faces three hard problems: 1. Observability – understanding GPU health, utilization, queue depths, collective timings, and tail latency across a large fleet. 2. Placement & Routing – sending the right work to the right resources (inference requests and training collectives) while accounting for network cost and GPU pressure. 3. Cost & Reliability – minimizing cost per token/step and reducing failure blast-radius wit","default_branch":null,"files":null,"tree":[],"storefront":"/r/redbco","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/redbco/infermesh/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}