{"repo":"bentoml/BentoML","free":true,"listed":false,"github":"https://github.com/bentoml/BentoML","clone":"git clone https://github.com/bentoml/BentoML.git","description":"The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!","language":"Python","stars":8791,"topics":["model-serving","mlops","llmops","generative-ai","llm-inference","model-inference-service","inference-platform","deep-learning","llm-serving","machine-learning"],"license":"Apache-2.0","category":"machine-learning","readme_excerpt":"Unified Model Serving Framework 🍱 Build model inference APIs and multi-model serving systems with any open-source or custom AI models. 👉 Join our forum! What is BentoML? BentoML is a Python library for building online serving systems optimized for AI apps and model inference. - 🍱 Easily build APIs for Any AI/ML Model. Turn any model inference script into a REST API server with just a few lines of code and standard Python type hints. - 🐳 Docker Containers made simple. No more dependency hell! Manage your environments, dependencies and model versions with a simple config file. BentoML automatically generates Docker images, ensures reproducibility, and simplifies how you deploy to different environments. - 🧭 Maximize CPU/GPU utilization. Build high performance inference APIs leveraging built-in serving optimization features like dynamic batching, model parallelism, multi-stage pipeline and multi-model inference-graph orchestration. - 👩‍💻 Fully customizable. Easily implement your own APIs or task queues, with custom business logic, model inference and multi-model composition. Supports any ML framework, modality, and inference runtime. - 🚀 Ready for Production. Develop, run and debug locally. Seamlessly deploy to production with Docker containers or BentoCloud. Getting started Install BentoML: Define APIs in a service.py file. 💻 Run locally Install PyTorch and Transformers packages to your Python virtual environment. Run the service code locally (serving at http://localho","default_branch":null,"files":null,"tree":[],"storefront":"/r/bentoml","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/bentoml/BentoML/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}