{"repo":"anastasiosyal/phi4-multimodal-instruct-server","free":true,"listed":false,"github":"https://github.com/anastasiosyal/phi4-multimodal-instruct-server","clone":"git clone https://github.com/anastasiosyal/phi4-multimodal-instruct-server.git","description":"Phi4 Multimodal Instruct - OpenAI endpoint and Docker Image for self-hosting","language":"Python","stars":40,"topics":[],"license":"MIT","category":"self-hosted-apps","readme_excerpt":"Phi-4 Multimodal Instruct Server Self-Hosted OpenAI-Compatible Chat Completion Endpoint This project enables you to self-host an OpenAI-compatible chat completion API endpoint powered by the microsoft/Phi-4-multimodal-instruct model. Deployed via a FastAPI server within a Docker container, this solution supports multimodal inputs—text, images, and audio—and leverages NVIDIA GPU acceleration for efficient inference. Tested on AWS on G6 instance Features - Multimodal Capabilities : Process text, image, and audio inputs seamlessly. - OpenAI Compatibility : Integrate easily with the OpenAI Python client or any HTTP client. - GPU Support : Utilize NVIDIA GPUs for enhanced performance. - Persistent Caching : Store model weights in a Docker volume to avoid repeated downloads. - Scalable Design : Handle multiple requests concurrently with Uvicorn workers. Prerequisites Before you begin, ensure the following are installed on your system: - Docker : For containerization. - Docker Compose : For managing multi-container setups. - NVIDIA Container Toolkit : For GPU support within Docker. - A Compatible NVIDIA GPU : Required for accelerated inference. Setup Instructions Follow these steps to build and run the self-hosted endpoint. 1. Clone the Repository 2. Build the Docker Image The provided Dockerfile sets up the environment with CUDA, Python, PyTorch, and required libraries. Build the image with: This command creates an image named phi4-server with: - CUDA 12.4.1 and Ubuntu 22.04 as the","default_branch":null,"files":null,"tree":[],"storefront":"/r/anastasiosyal","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/anastasiosyal/phi4-multimodal-instruct-server/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}