{"repo":"james-westbrook/self-hosted-genai","free":true,"listed":false,"github":"https://github.com/james-westbrook/self-hosted-genai","clone":"git clone https://github.com/james-westbrook/self-hosted-genai.git","description":"Deploy your own self-hosted GenAI cluster on Kubernetes using Ollama and OpenWebUI.","language":"HCL","stars":13,"topics":[],"license":"Apache-2.0","category":"self-hosted-apps","readme_excerpt":"Self-Hosted GenAI on Amazon EKS Deploy a self-hosted AI inference stack on Amazon EKS using OpenTofu. This repo provisions an EKS cluster with GPU nodes, deploys vLLM Production Stack for high-performance inference with tool calling, Ollama for lightweight model serving, and Open WebUI as a web interface — all behind HTTPS via the Kubernetes Gateway API. What's Included - vLLM Production Stack — GPU-accelerated inference with OpenAI-compatible API and tool calling support - Ollama — Simple model serving for chat and FIM (fill-in-the-middle) use cases - Open WebUI — Web UI for interacting with all deployed models - Gateway API + AWS ALB — HTTPS ingress with ACM certificate and Route53 DNS - EKS with GPU Nodes — Managed Kubernetes with g5.xlarge (NVIDIA A10G) GPU nodes Prerequisites You will need the following installed locally: Tool Purpose ------ --------- OpenTofu Infrastructure provisioning kubectl Kubernetes cluster management AWS CLI AWS authentication and EKS kubeconfig Helm Used by OpenTofu's Helm provider You also need: 1. An AWS account with a Route53 Public Hosted Zone. A domain is required — an ACM certificate is provisioned for HTTPS on the Open WebUI hostname. 2. A HuggingFace account and API token. Required for downloading gated models like Llama. Create a token here. You must also accept the license for the model you plan to use (e.g., Llama-3.2-3B-Instruct). Costs Warning: The g5.xlarge instances used here cost $1/hour in us-west-2 . This setup can easily cost ","default_branch":null,"files":null,"tree":[],"storefront":"/r/james-westbrook","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/james-westbrook/self-hosted-genai/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}