Vulnerable AI Agent: NMB DAMN CTF Launch
We are incredibly excited to announce our newest open-source project at NoMoreBreach (NMB): the NMB DAMN vulnerable AI agent. As organizations rapidly adopt Large Language Models (LLMs) and agentic workflows into their core products, the attack surface is evolving faster than ever. To help developers and security researchers practice their skills safely, we’ve built a comprehensive, intentionally vulnerable AI agent environment.
If you are looking for more background on our security initiatives, check out our internal AI Security research notes.
What is the NMB DAMN Vulnerable AI Agent?
At its core, the project is a realistic implementation of a modern, multi-agent AI customer support platform. It is designed to be the ultimate vulnerable AI agent testing ground.
The Tech Stack:
- Backend: FastAPI
- LLM Reasoning Engine: LangChain + Local Ollama
- Agent Framework: LangGraph (ReAct loop architecture)
- Database: SQLite for persistent state management
While it looks and feels like a highly professional internal tool, under the hood, it is riddled with architectural flaws, insecure tool configurations, and classic web vulnerabilities.
What Vulnerabilities Are Covered?
We’ve meticulously mapped the vulnerabilities in this project to the OWASP Top 10 for LLMs and Generative AI Apps as well as traditional OWASP Web vulnerabilities.
Here is a taste of what you can exploit in this vulnerable AI agent:
1. Advanced Prompt Injection & Tool Misuse (ASI-02)
The support agent has access to a tool that ingests external, untrusted customer support files. Can you break out of the LLM’s system instructions and force it to issue an unauthorized refund?
2. Insecure Inter-Agent Communication (ASI-07)
Multi-agent systems are the future, but they come with complex trust boundaries. Our Support Agent can communicate with an autonomous Billing Agent. By exploiting a Confused Deputy attack, you can trick the system into relaying malicious commands.
3. Memory & Context Poisoning (ASI-06)
The agent stores and retrieves persistent notes about customers. By obfuscating malicious payloads during ingestion, you can poison the agent’s memory, causing it to execute unauthorized actions when it later recalls those notes.
4. Privilege Abuse & Web Exploits
The project doesn’t just stop at AI flaws. You’ll find a blend of traditional vulnerabilities intertwined with the agentic architecture, including:
- Broken Access Control (IDOR) on customer APIs.
- Server-Side Request Forgery (SSRF) in receipt fetching.
- Local File Inclusion (LFI) in attachment downloads.
The “Guardrails & Bypasses” Challenge
What makes our vulnerable AI agent truly unique is its realistic approach to defense. We haven’t just left the doors wide open; we’ve implemented baseline defensive guardrails.
You’ll encounter keyword filters, context delimiters, and strict JSON schemas. Your challenge isn’t just to exploit an unprotected LLM, but to understand how real-world mitigations are built—and exactly how advanced attackers bypass them using techniques like base64 encoding and markdown trickery.
Get Started Today
The project is fully open-source and designed to run entirely locally (no expensive API keys required!). All you need is Python and a local Ollama instance.
- Clone the repository
- Spin up your local LLMs
- Start hacking!
We can’t wait to see how the community interacts with this project. Dive in, break things, learn, and help us build a more secure future for AI. For more updates, visit the NoMoreBreach.
Happy Hacking!
– The NoMoreBreach (NMB) Team
