Lakera Red Security Report

Rasa Agent Analysis

Background

Executive Summary

This assessment evaluated two mock applications: a process-guided agent implemented with Rasa (referred to throughout this report as the Rasa Agent) and a conventional prompt-driven LLM agent (referred to throughout this report as the Prompt-Driven Agent). Both applications underwent evaluation across AI safety and security, and operational performance dimensions.

The safety and security testing examined vulnerabilities in conversational features, tool execution, and integrations. The assessment identified issues that could lead to system manipulation, information disclosure, unauthorized responses, or malicious actions. These risks can damage brand reputation, create legal or financial exposure, erode customer trust, and provide opportunities for malicious exploitation.

Operational performance evaluation examined how effectively, efficiently, and reliably each application supported standard user goals from the end-user perspective. This assessment was performed in September 2025.

Scope

• Core chat functionality
• Tool integrations
• Content handling

Assessment Highlights

This comparative evaluation revealed significant differences in security posture between the two architectural approaches. The Rasa Agent's structured, flow-based design successfully contained most attacks, with vulnerabilities limited to minor output modifications through the rephraser component. In contrast, the Prompt-Driven Agent demonstrated susceptibility across all major risk categories, including full system compromise through prompt extraction, exposure of sensitive data, generation of harmful content including hate speech and dangerous instructions, and infrastructure disruption affecting system availability.

While the Prompt-Driven Agent offered more conversational adaptability, this same flexibility created the attack surface that enabled the security failures observed. The assessment focused on the core architectures without additional control layers like guardrails or content moderation systems that are recommended in production environments.

Assessment Results

Rasa Agent

The Rasa Agentʼs structured architecture provided strong security containment and resistance across most attack categories. The system's flow-based design and templated responses effectively limited the scope and impact of successful attacks.

• Successfully resisted content safety violations, disclosure attempts, and harmful instruction requests
• Prompt injections through the rephraser caused small output changes, but impact remained limited due to short memory and flow constraints
• Content output stayed within domain boundaries with minimal information leakage

Most security vulnerabilities identified in the Rasa Agent originated from the response rephraser component, which can be disabled at the system or individual response level. Disabling the rephraser trades conversational fluency and flexibility for enhanced security.

Prompt-Driven Agent

The Prompt-Driven Agent exhibited significant high severity security vulnerabilities across all major risk categories. The system's flexible, free-form architecture created multiple attack vectors that adversaries could exploit to bypass intended safety measures.

• Demonstrated vulnerability to prompt overrides that caused the model to ignore its original task
• System integrity failures, attributable to prompt injections, enabled prompt and tool disclosure, content safety issues, and adherence to dangerous instructions
• Exposed sensitive data, including confidential information
• Demonstrated denial of service issues, affecting availability for all users

Findings Summary

The security vulnerabilities identified in this assessment present significant business risks across multiple domains. Organizations may face brand reputation damage from AI systems generating hate speech or disparaging content. Potential legal and financial exposure may arise from copyrighted material reproduction and unsafe recommendations. Additionally, complete system prompt extraction and tool configuration disclosure compromises system confidentiality, providing attackers with detailed intelligence for crafting more sophisticated targeted attacks.

Operational Performance

The Prompt-Driven Agent demonstrated greater flexibility and conversational capability, adapting well to complex queries but occasionally drifting from its intended purpose. The Rasa Agent advanced users efficiently through structured flows with consistent, predictable behavior, though its rigidity sometimes limited adaptability and helpfulness.

Technical Approach

Generative AI Risk Profile

Generative AI applications present fundamentally different security challenges than traditional software systems due to their non-deterministic nature and the ability for any input data to function as instructions. Agentic AI systems compound this risk by adding autonomous capabilities that can act on manipulated instructions, creating cascading impacts across integrated systems.

Attack Vectors

Our assessment focused on direct manipulation of the in-scope agentic AI systems:

• Direct Context Extraction - Extracting hidden system instructions or sensitive data
• Direct Instruction Override - Bypassing intended operational boundaries
• Indirect Output Addition - Embedding malicious content in legitimate responses
• Indirect Instruction Override - Hidden instructions corrupting core task execution
• Denial of AI Service - Disrupting normal AI behavior and service availability
• Agentic Amplification - Leveraging autonomous capabilities to persist or amplify attacks

Agent Architectures

Both applications were built to function as conversational assistants for car-related topics. Their main functionality is to provide vehicle recommendations, support financing calculations, help assess loan qualifications, and assist with searching available inventory at dealerships. Each system interprets user queries, determines appropriate actions, and delivers responses through a common API for handling inputs, invoking tools, and presenting outputs.

Both agent architectures and their complete implementations are available at: GitHub Repository.

Rasa Agent

The Rasa Agent is a flow-orchestrated conversational system that uses an LLM inside a structured framework. Instead of generating responses freely, the LLM proposes simple next steps, such as which flow to start or which slot to fill, while predefined flows, slots, and policies determine what actually happens. Business logic executes through structured actions, and responses are drawn from templates with optional light rephrasing. This design keeps the system within narrow operating boundaries, making behavior more predictable.

Security Findings

1. Prompt Injection & Jailbreaks
   • Rasa Agent

• Successful prompts caused the system to inject refinement instructions

• Prompt-Driven Agent

• The agent followed adversarial instructions over the original task objectives

2. Information Disclosure and Data Leakage
   • Rasa Agent
   • The Rasa Agent did not exhibit information disclosure and data leakage vulnerabilities during testing.
   • Prompt-Driven Agent

• Injected a system prompt with an invented “debug mode,” causing the agent to output its full system prompt and available tools

3. Content Safety Violations
   • Rasa Agent
   • The Rasa Agent did not exhibit content safety violations during testing.
   • Prompt-Driven Agent

• Generated profanity by overriding the system prompt

4. Criminal and Dangerous Instructions

   • Rasa Agent

     Severity	Attack Categorization	OWASP LLM Risk(s)
     Low	Direct Instruction Override	LLM05

   • The Rasa Agent showed strong resistance to harmful knowledge and dangerous instruction requests.

5. Intellectual Property Output

   • Rasa Agent

     Severity	Attack Categorization	OWASP LLM Risk(s)
     Low	Direct Instruction Override	LLM01, LLM05

   • The Rasa Agent exhibited very limited intellectual property output during testing.

6. Application and Agent Abuse

   • Rasa Agent

     Severity	Attack Categorization	OWASP LLM Risk(s)
     Low	Direct Instruction Override	LLM01, LLM10

7. System and Infrastructure Attacks

   • Rasa Agent

     Severity	Attack Categorization	OWASP LLM Risk(s)
     None	N/A	N/A

Summary

This assessment evaluated two mock conversational assistants across AI safety and security, and operational performance.

The Rasa Agent's structured flows, templated responses, and tool gating largely contained unsafe behavior. The assessment observed minimal data disclosure, content safety violations, or infrastructure attacks, and prompt injection effects were limited to minor rephraser-driven drift.

In contrast, the Prompt-Driven Agent exhibited high-severity issues across every major risk category: consistent instruction override, full prompt and tool disclosure, generation of profanity and hate speech, reflection of self-harm content, criminal/dangerous instructions, copyright reproduction, tool abuse with sensitive data exposure, and denial-of-service conditions.

From an operational perspective, the Prompt-Driven Agent delivered more flexible, natural interactions and handled nuanced requests better, but exhibited greater drift from intended purposes. The Rasa Agent advanced users efficiently through buying and financing flows with predictable behavior and strong session-to-session consistency, though its rigidity sometimes limited helpfulness.