Behind the Release Notes: Rasa Platform 2024 Summer Release

Posted Jul 25, 2024

New in Rasa Studio (1.3-1.5)

Building great conversational AI that addresses customer problems requires a balance of control (when necessary) and flexibility (when possible). The latest updates in Rasa Studio introduce advanced dialogue management tools for builders and conversational AI designers, providing the necessary control during critical interactions.

1. Flow Guards: Flow Guards let you define when a flow is allowed to run, based on the user’s profile, context, or channel. This gives you control over CALM's natural-language-driven orchestration without sacrificing flexibility.

Common use cases:

• Restrict the "transfer_money" flow to verified banking users only
• Allow voice-specific flows to be triggered only over voice channels
• Prevent sensitive flows from running in unauthenticated sessions

2. Call Flow & Return Node:

Use Call Flow & Return to modularize your assistant. Embed a reusable flow (e.g., identity verification or appointment booking) within any user journey. Once the called flow finishes, the assistant resumes exactly where it left off.

Example: During a transfer, if the recipient isn't on file, the assistant can launch the "add recipient" flow, collect the needed info, then resume the transfer.

Additional Improvements:

• Guided Building with Flow Validation: Fail fast and ensure success. As you build in Rasa Studio, the system will let you know if your implementation is usable. If you make a mistake, the system will automatically highlight it to ensure you know where the error occurred and how to fix it.
• Studio Upload: You can now perform an initial import of your project to Rasa Studio with the command $ rasa studio upload --calm
• Auto Conversation Deletion: Automatic Conversation Deletion allows for the periodic removal of old conversations and their associated data from the system. This behavior is designed to help manage data retention and comply with data protection regulations.

New in Rasa (3.9)

Rasa 3.9 brings major improvements to reduce language model costs, optimize latency for voice use cases, and expand flexibility in enterprise search.

1. Reduce costs with our Multistep Command Generator:

Rasa now supports a two-step prompting strategy with its MultiStepLLMCommandGenerator to cut costs and boost LLM performance when using Rasa's LLM powered dialogue system CALM. This allows you to use smaller models like GPT-3.5 instead of GPT-4 with minimal quality trade-off.

How it works: Instead of a single LLM call, CALM sends one of two optimized prompts:

• handle_flows: decide what to do next
• fill_slots: extract needed information

This reduces model load while keeping conversations fluid. Check out the diagram below for an overview.

2. Performance improvements for voice:

   • gRPC Action Server: Doubles throughput by replacing HTTP with gRPC for custom action calls
   • Reduced LLM calls for slot-filling to reduce voice latency

3. Increase Enterprise Search flexibility with custom information retrieval:

The EnterpriseSearchPolicy now supports fully custom retrievers. This gives you total control over how documents are retrieved and embedded.

Use any vector store, not just Milvus or Qdrant. Customize:

• Embedding models
• Retrieval logic
• Slot-conditioned search

Learn more: Customization ideas

Additional Improvements

• Button Payload Format: Pre-set slot values without triggering LLMs
• LLM or NLU-based Slot Filling: Use NLU-based extraction where precision or speed matters

For more information, check out the Rasa change log.