{"repo":"NillionNetwork/nilAI","free":true,"listed":false,"github":"https://github.com/NillionNetwork/nilAI","clone":"git clone https://github.com/NillionNetwork/nilAI.git","description":"nilAI is the core offering of Nillion SecretLLM a TEE-based AI inference service.","language":"Python","stars":31,"topics":["ai","privacy","tee"],"license":"MIT","category":"machine-learning","readme_excerpt":"nilAI Overview nilAI is a platform designed to run on Confidential VMs with Trusted Execution Environments (TEEs). It ensures secure deployment and management of multiple AI models across different environments, providing a unified API interface for accessing various AI models with robust user management and model lifecycle handling. Prerequisites - Docker - Docker Compose - Hugging Face API Token (for accessing certain models) Configuration 1. Environment Setup - Copy the .env.sample file to .env - Update the environment variables in .env : - HUGGINGFACE API TOKEN : Your Hugging Face API token - Obtain token by requesting access on the specific model's Hugging Face page. For example, to request access for the Llama 1B model, you can ask here. Note that for the Llama-8B model, you need to make a separate request. Deployment Options 1. Docker Compose Deployment (Recommended) Development Environment Then, to deploy: Production Environment To deploy: Note : Remove lines for models you do not wish to deploy. Testing Without GPU To deploy: 2. Using the Docker Compose Helper Script For easier management of multiple compose files and image substitutions, use the docker-composer.py script: Basic Usage Image Substitution Replace default images with custom ones (useful for production deployments with specific image versions): Script Options - --dev : Include development-specific configurations - --prod : Include production-specific configurations - -f, --file : Include additional compo","default_branch":null,"files":null,"tree":[],"storefront":"/r/NillionNetwork","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/NillionNetwork/nilAI/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}