{"repo":"jina-ai/serve","free":true,"listed":false,"github":"https://github.com/jina-ai/serve","clone":"git clone https://github.com/jina-ai/serve.git","description":"☁️ Build multimodal AI applications with cloud-native stack","language":"Python","stars":21861,"topics":["neural-search","cloud-native","deep-learning","machine-learning","framework","grpc","kubernetes","multimodal","mlops","pipeline"],"license":"Apache-2.0","category":"machine-learning","readme_excerpt":"Jina-Serve Jina-serve is a framework for building and deploying AI services that communicate via gRPC, HTTP and WebSockets. Scale your services from local development to production while focusing on your core logic. Key Features - Native support for all major ML frameworks and data types - High-performance service design with scaling, streaming, and dynamic batching - LLM serving with streaming output - Built-in Docker integration and Executor Hub - One-click deployment to Jina AI Cloud - Enterprise-ready with Kubernetes and Docker Compose support Comparison with FastAPI Key advantages over FastAPI: - DocArray-based data handling with native gRPC support - Built-in containerization and service orchestration - Seamless scaling of microservices - One-command cloud deployment Install See guides for Apple Silicon and Windows. Core Concepts Three main layers: - Data : BaseDoc and DocList for input/output - Serving : Executors process Documents, Gateway connects services - Orchestration : Deployments serve Executors, Flows create pipelines Build AI Services Let's create a gRPC-based AI service using StableLM: Deploy with Python or YAML: Use the client: Build Pipelines Chain services into a Flow: Scaling and Deployment Local Scaling Boost throughput with built-in features: - Replicas for parallel processing - Shards for data partitioning - Dynamic batching for efficient model inference Example scaling a Stable Diffusion deployment: Cloud Deployment Containerize Services 1. Structure","default_branch":null,"files":null,"tree":[],"storefront":"/r/jina-ai","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/jina-ai/serve/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}