Generative Search | Rasa Documentation

Build your first agent in just a few minutes with Rasa Copilot.

On this page

If Generative Search is enabled, the Enterprise Search Policy uses an LLM to generate a relevant, context-aware response. The response is generated based on the conversation transcript, relevant document snippets retrieved from the knowledge base, and the slot values of the conversation.

Generative Search Configuration

Generative Search is enabled by default in EnterpriseSearchPolicy. You can explicitly enable it by setting the use_generative_llm parameter to true in the config.yml file:

policies:
...  
- name: EnterpriseSearchPolicy
  use_generative_llm: true

LLM

• Rasa Pro <=3.7.x
• Rasa Pro >=3.8.x
• Rasa Pro >=3.11.x

You can choose the OpenAI model that is used for the LLM by adding the llm.model parameter to the config.yml file.

policies:
# - ...
  - name: rasa_plus.ml.EnterpriseSearchPolicy
    llm:
      model: "gpt-4.1-mini-2025-04-14"
# - ...

You can choose which LLM to use for the answer generation by adding the llm.model_group parameter to the config.yml file.

policies:
# - ...
  - name: EnterpriseSearchPolicy
    llm:
      model_group: "openai-gpt-direct"
# - ...

endpoints.yml

   model_groups:
     - id: openai-gpt-direct
       models:
         - model: "gpt-5-mini-2025-08-07"
           provider: "openai"

The default LLM used for answer generation on current Rasa Pro is gpt-5-mini-2025-08-07. For more details on how to configure different LLMs, see the LLM Configuration documentation.

Prompt

You can change the prompt template used to generate a response based on retrieved documents by setting the prompt_template property in the config.yml:

policies:
# - ...
  - name: rasa_plus.ml.EnterpriseSearchPolicy
    prompt: prompts/enterprise-search-policy-template.jinja2

The prompt is a Jinja2 template that can be used to customize the prompt. The following variables are available in the prompt:

• docs: The list of documents retrieved from the document search.
• slots: The list of slots currently available in the conversation.
• current_conversation: The current conversation with the user. Number of messages in the conversation can be configured by the policy parameter max_history

Answering the Question

Based on the above sections, please formulate an answer to the question or request in the user's last message. It is important that you ensure the answer is grounded in the provided documents and conversation context. Avoid speculating or making assumptions beyond the given information and keep your answers short, 2 to 3 sentences at most.

Security Considerations

The component uses, by default, an LLM to generate rephrased responses.

The following threat vectors should be considered:

• Privacy: Most LLMs are run as remote services. The component sends your assistant's conversations to remote servers for prediction.
• Hallucination: When generating answers, it is possible that the LLM changes your document content in a way that the meaning is no longer exactly the same. The temperature parameter allows you to control this trade-off. A low temperature will only allow for minor variations.
• Prompt Injection: Messages sent by your end users to your assistant will become part of the LLM prompt. That means a malicious user can potentially override the instructions in your prompt.