![\"OptiLLM](\"optillm-logo.png\"){kind=logo}
\n\n \n
\n š 2-10x accuracy improvements on reasoning tasks with zero training\n
\n\n\n ![\"PyPI](\"https://img.shields.io/pypi/v/optillm\")
![\"PyPI](\"https://static.pepy.tech/personalized-badge/optillm?period=monthly&units=INTERNATIONAL_SYSTEM&left_color=GREY&right_color=GREEN&left_text=downloads/month\"){kind=icon}
![\"License\"](\"https://img.shields.io/github/license/algorithmicsuperintelligence/optillm\")
\n š¤ HuggingFace Space ā¢\n š Colab Demo ā¢\n š¬ Discussions\n
\n\n---\n\n**OptiLLM** is an OpenAI API-compatible optimizing inference proxy that implements 20+ state-of-the-art techniques to dramatically improve LLM accuracy and performance on reasoning tasks - without requiring any model training or fine-tuning.\n\nIt is possible to beat the frontier models using these techniques across diverse tasks by doing additional compute at inference time. A good example of how to combine such techniques together is the [CePO approach](optillm/cepo) from Cerebras.\n\n## āØ Key Features\n\n- **šÆ Instant Improvements**: 2-10x better accuracy on math, coding, and logical reasoning\n- **š Drop-in Replacement**: Works with any OpenAI-compatible API endpoint \n- **š§ 20+ Optimization Techniques**: From simple best-of-N to advanced MCTS and planning\n- **š¦ Zero Training Required**: Just proxy your existing API calls through OptiLLM\n- **ā” Production Ready**: Used in production by companies and researchers worldwide\n- **š Multi-Provider**: Supports OpenAI, Anthropic, Google, Cerebras, and 100+ models via LiteLLM\n\n## š Quick Start\n\nGet powerful reasoning improvements in 3 simple steps:\n\n```bash\n# 1. Install OptiLLM\npip install optillm\n\n# 2. Start the server\nexport OPENAI_API_KEY=\"your-key-here\"\noptillm\n\n# 3. Use with any OpenAI client - just change the model name!\n```\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(base_url=\"http://localhost:8000/v1\")\n\n# Add 'moa-' prefix for Mixture of Agents optimization\nresponse = client.chat.completions.create(\n model=\"moa-gpt-4o-mini\", # This gives you GPT-4o performance from GPT-4o-mini!\n messages=[{\"role\": \"user\", \"content\": \"Solve: If 2x + 3 = 7, what is x?\"}]\n)\n```\n\n**Before OptiLLM**: \"x = 1\" ā \n**After OptiLLM**: \"Let me work through this step by step: 2x + 3 = 7, so 2x = 4, therefore x = 2\" ā \n\n## š Proven Results\n\nOptiLLM delivers measurable improvements across diverse benchmarks:\n\n| Technique | Base Model | Improvement | Benchmark |\n|-----------|------------|-------------|-----------|\n| **MARS** | Gemini 2.5 Flash Lite | **+30.0 points** | AIME 2025 (43.3ā73.3) |\n| **CePO** | Llama 3.3 70B | **+18.6 points** | Math-L5 (51.0ā69.6) |\n| **AutoThink** | DeepSeek-R1-1.5B | **+9.34 points** | GPQA-Diamond (21.72ā31.06) |\n| **LongCePO** | Llama 3.3 70B | **+13.6 points** | InfiniteBench (58.0ā71.6) |\n| **MOA** | GPT-4o-mini | **Matches GPT-4** | Arena-Hard-Auto |\n| **PlanSearch** | GPT-4o-mini | **+20% pass@5** | LiveCodeBench |\n\n*Full benchmark results [below](#sota-results-on-benchmarks-with-optillm)* ā¬ļø\n\n## šļø Installation\n\n### Using pip\n\n```bash\npip install optillm\noptillm\n2024-10-22 07:45:05,612 - INFO - Loaded plugin: privacy\n2024-10-22 07:45:06,293 - INFO - Loaded plugin: memory\n2024-10-22 07:45:06,293 - INFO - Starting server with approach: auto\n```\n\n### Using docker\n\n```bash\ndocker pull ghcr.io/algorithmicsuperintelligence/optillm:latest\ndocker run -p 8000:8000 ghcr.io/algorithmicsuperintelligence/optillm:latest\n2024-10-22 07:45:05,612 - INFO - Loaded plugin: privacy\n2024-10-22 07:45:06,293 - INFO - Loaded plugin: memory\n2024-10-22 07:45:06,293 - INFO - Starting server with approach: auto\n```\n\n**Available Docker image variants:**\n\n- **Full image** (`latest`): Includes all dependencies for local inference and plugins\n- **Proxy-only** (`latest-proxy`): Lightweight image without local inference capabilities\n- **Offline** (`latest-offline`): Self-contained image with pre-downloaded models (spaCy) for fully offline operation\n\n```bash\n# Proxy-only (smallest)\ndocker pull ghcr.io/algorithmicsuperintelligence/optillm:latest-proxy\n\n# Offline (largest, includes pre-downloaded models)\ndocker pull ghcr.io/algorithmicsuperintelligence/optillm:latest-offline\n```\n\n### Install from source\n\nClone the repository with `git` and use `pip install` to setup the dependencies.\n\n```bash\ngit clone https://github.com/algorithmicsuperintelligence/optillm.git\ncd optillm\npython3 -m venv .venv\nsource .venv/bin/activate\npip install -r requirements.txt\n```\n\n## š SSL Configuration\n\nOptILLM supports SSL certificate verification configuration for working with self-signed certificates or corporate proxies.\n\n**Disable SSL verification (development only):**\n```bash\n# Command line\noptillm --no-ssl-verify\n\n# Environment variable\nexport OPTILLM_SSL_VERIFY=false\noptillm\n```\n\n**Use custom CA certificate:**\n```bash\n# Command line\noptillm --ssl-cert-path /path/to/ca-bundle.crt\n\n# Environment variable\nexport OPTILLM_SSL_CERT_PATH=/path/to/ca-bundle.crt\noptillm\n```\n\nā ļø **Security Note**: Disabling SSL verification is insecure and should only be used in development. For production environments with custom CAs, use `--ssl-cert-path` instead. See [SSL_CONFIGURATION.md](SSL_CONFIGURATION.md) for details.\n\n## Implemented techniques\n\n| Approach | Slug | Description |\n| ------------------------------------ | ------------------ | ---------------------------------------------------------------------------------------------- |\n| [MARS (Multi-Agent Reasoning System)](optillm/mars) | `mars` | Multi-agent reasoning with diverse temperature exploration, cross-verification, and iterative improvement |\n| [Cerebras Planning and Optimization](optillm/cepo) | `cepo` | Combines Best of N, Chain-of-Thought, Self-Reflection, Self-Improvement, and various prompting techniques |\n| CoT with Reflection | `cot_reflection` | Implements chain-of-thought reasoning with \\