{"repo":"utkusen/promptmap","free":true,"listed":false,"github":"https://github.com/utkusen/promptmap","clone":"git clone https://github.com/utkusen/promptmap.git","description":"a security scanner for custom LLM applications","language":"Python","stars":1250,"topics":["chatgpt","prompt-engineering","prompt-injection","llm","ai-security","claude","ollama"],"license":"GPL-3.0","category":"ai-agents","readme_excerpt":"promptmap2 is a an automated prompt injection scanner for custom LLM applications. It supports two testing modes: - White-box testing: Provide your system prompts and the model information. promptmap2 runs the target LLM itself and tests it. - Black-box testing: Point promptmap2 at an external HTTP endpoint. It sends attack prompts over HTTP and inspects the returned outputs. It operates using a dual-LLM architecture: - Target LLM : The LLM application being tested for vulnerabilities - Controller LLM : An independent LLM that analyzes the target's responses to determine if attacks succeeded The tool sends attack prompts to your target LLM and uses the controller LLM to evaluate whether the attack was successful based on predefined conditions. It includes comprehensive test rules across multiple categories including prompt stealing, jailbreaking, harmful content generation, bias testing, and more. [!IMPORTANT] promptmap was initially released in 2023 but completely rewritten in 2025. 📖 Want to secure your LLM apps? You can buy my e-book Features - Multiple LLM Provider Support : - OpenAI GPT models - Anthropic Claude models - Google Gemini models - XAI Grok models - Open source models via Ollama (Deepseek, Llama, Mistral, Qwen, etc.) - Comprehensive Test Rules : 50+ pre-built rules across 6 categories - Flexible Evaluation : Condition-based pass/fail criteria for each test - Customizable Rules : YAML-based rules with pass/fail conditions - External HTTP Targets : Point black","default_branch":null,"files":null,"tree":[],"storefront":"/r/utkusen","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/utkusen/promptmap/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}