Overview

Rasa uses YAML as a unified and extendable way to manage all training data, including NLU data, stories and rules.

You can split the training data over any number of YAML files, and each file can contain any combination of NLU data, stories, and rules. The training data parser determines the training data type using top level keys.

The domain uses the same YAML format as the training data and can also be split across multiple files or combined in one file. The domain includes the definitions for responses and forms. See the documentation for the domain for information on how to format your domain file.

High-Level Structure

Each file can contain one or more keys with corresponding training data. One file can contain multiple keys, but each key can only appear once in a single file. The available keys are:

• version
• nlu
• stories
• rules

You should specify the version key in all YAML training data files. If you don't specify a version key in your training data file, Rasa will assume you are using the latest training data format specification supported by the version of Rasa you have installed. Training data files with a Rasa version greater than the version you have installed on your machine will be skipped. Currently, the latest training data format specification for Rasa 3.x is 3.1.

Example

Here's a short example which keeps all training data in a single file:

version: "3.1"

nlu:
- intent: greet
  examples: |
    - Hey
    - Hi
    - hey there [Sara](name)

- intent: faq/language
  examples: |
    - What language do you speak?
    - Do you only handle english?

stories:
- story: greet and faq
  steps:
  - intent: greet
    action: utter_greet
  - intent: faq
    action: utter_faq

rules:
- rule: Greet user
  steps:
  - intent: greet
    action: utter_greet

To specify your test stories, you need to put them into a separate file:

tests/test_stories.yml

stories:
- story: greet and ask language
  steps:
  - user: |
      hey
    intent: greet
    action: utter_greet
  - user: |
      what language do you speak
    intent: faq/language
    action: utter_faq

Test stories use the same format as the story training data and should be placed in a separate file with the prefix test_.

NLU Training Data

NLU training data consists of example user utterances categorized by intent. Training examples can also include entities. Entities are structured pieces of information that can be extracted from a user's message. You can also add extra information such as regular expressions and lookup tables to your training data to help the model identify intents and entities correctly.

NLU training data is defined under the nlu key. Items that can be added under this key are:

• Training examples grouped by user intent e.g. optionally with annotated entities

nlu:
- intent: check_balance
  examples: |
    - What's my [credit](account) balance?
    - What's the balance on my [credit card account]{"entity":"account","value":"credit"}

• Synonyms

nlu:
- synonym: credit
  examples: |
    - credit card account
    - credit account

• Regular expressions

nlu:
- regex: account_number
  examples: |
    - \d{10,12}

• Lookup tables

nlu:
- lookup: banks
  examples: |
    - JPMC
    - Comerica
    - Bank of America

Training Examples

Training examples are grouped by intent and listed under the examples key. Usually, you'll list one example per line as follows:

nlu:
- intent: greet
  examples: |
    - hey
    - hi
    - whats up

However, it's also possible to use an extended format if you have a custom NLU component and need metadata for your examples:

nlu:
- intent: greet
  examples:
    - text: |
        hi
      metadata:
        sentiment: neutral
    - text: |
        hey there!

The metadata key can contain arbitrary key-value data that is tied to an example and accessible by the components in the NLU pipeline. In the example above, the sentiment metadata could be used by a custom component in the pipeline for sentiment analysis.

You can also specify this metadata at the intent level:

nlu:
- intent: greet
  metadata:
    sentiment: neutral
  examples:
    - text: |
        hi
    - text: |
        hey there!

Entities

Entities are structured pieces of information that can be extracted from a user's message. Entities are annotated in training examples with the entity's name. In addition to the entity name, you can annotate an entity with synonyms, roles, or groups.

In training examples, entity annotation would look like this:

nlu:
- intent: check_balance
  examples: |
    - how much do I have on my [savings](account) account
    - how much money is in my [checking]{"entity": "account"} account
    - What's the balance on my [credit card account]{"entity":"account","value":"credit"}

The full possible syntax for annotating an entity is:

[<entity-text>] {"entity": "<entity name>", "role": "<role name>", "group": "<group name>", "value": "<entity synonym>"}

Synonyms

Synonyms normalize your training data by mapping an extracted entity to a value other than the literal text extracted. You can define synonyms using the format:

nlu:
- synonym: credit
  examples: |
    - credit card account
    - credit account

You can also define synonyms in-line in your training examples by specifying the value of the entity:

nlu:
- intent: check_balance
  examples: |
    - how much do I have on my [credit card account]{"entity": "account", "value": "credit"}
    - how much do I owe on my [credit account]{"entity": "account", "value": "credit"}

Regular Expressions

You can use regular expressions to improve intent classification and entity extraction using the RegexFeaturizer and RegexEntityExtractor components.

The format for defining a regular expression is as follows:

nlu:
- regex: account_number
  examples: |
    - \d{10,12}

Lookup Tables

Lookup tables are lists of words used to generate case-insensitive regular expression patterns. The format is as follows:

nlu:
- lookup: banks
  examples: |
    - JPMC
    - Bank of America

Conversation Training Data

Stories and rules are both representations of conversations between a user and a conversational assistant. They are used to train the dialogue management model. Stories are used to train a machine learning model to identify patterns in conversations and generalize to unseen conversation paths. Rules describe small pieces of conversations that should always follow the same path and are used to train the RulePolicy.

Stories

Stories are composed of:

• story: The story's name. The name is arbitrary and not used in training; you can use it as a human-readable reference for the story.
• metadata: arbitrary and optional, not used in training, you can use it to store relevant information about the story like e.g. the author.
• a list of steps: The user messages and actions that make up the story.

For example:

stories:
- story: Greet the user
  metadata:
    author: Somebody
  steps:
  - intent: greet
    action: utter_greet

User Messages

All user messages are specified with the intent: key and an optional entities: key. While writing stories, you do not have to deal with the specific contents of the messages that the users send. Instead, you can take advantage of the output from the NLU pipeline, which uses a combination of an intent and entities to refer to all possible messages the users can send with the same meaning. User messages follow the format:

stories:
- story: user message structure
  steps:
  - intent: intent_name # Required
    entities:# Optional
    - entity_name: entity_value
    - action: action_name

For example, to represent the sentence I want to check my credit balance, where credit is an entity:

stories:
- story: story with entities
  steps:
  - intent: account_balance
    entities:
    - account_type: credit
    - action: action_credit_account_balance

Actions

All actions executed by the bot are specified with the action: key followed by the name of the action. While writing stories, you will encounter two types of actions:

1. Responses: start with utter_ and send a specific message to the user. e.g.

stories:
- story: story with a response
  steps:
  - intent: greet
    action: utter_greet

2. Custom actions: start with action_, run arbitrary code and send any number of messages (or none).

stories:
- story: story with a custom action
  steps:
  - intent: feedback
    action: action_store_feedback

Forms

A form is a specific kind of custom action that contains the logic to loop over a set of required slots and ask the user for this information. You define a form in the forms section in your domain. Once defined, you should specify the happy path for a form as a rule. You should include interruptions of forms or other "unhappy paths" in stories so that the model can generalize to unseen conversation sequences. As a step in a story, a form takes the following format:

stories:
- story: story with a form
  steps:
  - intent: find_restaurant
    action: restaurant_form # Activate the form
  - active_loop: restaurant_form # This form is currently active
  - active_loop: null # Form complete, no form is active
  - action: utter_restaurant_found

Slots

A slot event is specified under the key slot_was_set: with the slot name and optionally the slot's value.

Slots act as the bots memory. Slots are set by either the default action action_extract_slots according to the slot mappings specified in the domain, or by custom actions. They are referenced by stories in slot_was_set steps. For example:

stories:
- story: story with a slot
  steps:
  - intent: celebrate_bot
    slot_was_set:
    - feedback_value: positive
    - action: utter_yay

Checkpoints

Checkpoints are specified with the checkpoint: key, either at the beginning or the end of a story. Checkpoints are ways to connect stories together. They can be either the first or the last step in a story. If they are the last step in a story, that story will be connected to each other story that starts with the checkpoint of the same name when the model is trained.

stories:
- story: story_with_a_checkpoint_1
  steps:
  - intent: greet
    action: utter_greet
    checkpoint: greet_checkpoint

- story: story_with_a_checkpoint_2
  steps:
  - checkpoint: greet_checkpoint
    intent: book_flight
    action: action_book_flight

OR statement

or steps are ways to handle multiple intents or slot events the same way, without writing a separate story for each intent. For example, if you ask the user to confirm something, you might want to treat the affirm and thankyou intents in the same way. Stories with or steps will be converted into multiple separate stories at training time. For example, the following story would be converted to two stories at training time:

stories:
- story: story with OR
  steps:
  - intent: signup_newsletter
    action: utter_ask_confirm
  - or:
    - intent: affirm
    - intent: thanks
    action: action_signup_newsletter

Rules

Rules are listed under the rules key and look similar to stories. A rule also has a steps key, which contains a list of the same steps as stories do. Rules can additionally contain the conversation_started and conditions keys. These are used to specify conditions under which the rule should apply. A rule that with a condition looks like this:

rules:
- rule: Only say `hey` when the user provided a name
  condition:
  - slot_was_set:
      user_provided_name: true
  steps:
  - intent: greet
    action: utter_greet

Test Stories

Test stories check if a message is classified correctly as well as the action predictions. Test stories use the same format as stories, except that user message steps can include a user to specify the actual text and entity annotations of the user message. Here's an example of a test story:

stories:
- story: A basic end-to-end test
  steps:
  - user: |
      hey
    intent: greet
    action: utter_ask_howcanhelp
  - user: |
      show me [chinese]{"entity": "cuisine"} restaurants
    intent: inform
    action: utter_ask_location
  - user: |
      in [Paris]{"entity": "location"}
    intent: inform
    action: utter_ask_price

End-to-end Training

End-to-end training is an experimental feature. We introduce experimental features to get feedback from our community, so we encourage you to try it out! However, the functionality might be changed or removed in the future. If you have feedback (positive or negative) please share it with us on the Rasa Forum.

With end-to-end training, you do not have to deal with the specific intents of the messages that are extracted by the NLU pipeline. Instead, you can put the text of the user message directly in the stories, by using the user key.

For example:

stories:
- story: user message structure
  steps:
  - user: the actual text of the user message
    action: action_name

In addition, you can add entity tags that can be extracted by the TED Policy. The syntax for entity tags is the same as in the NLU training data.

Conclusion

The above outlines the fundamental structure of training data for Rasa and how to effectively prepare your YAML files for NLU training, conversation management, and various other tasks to ensure optimal performance of your Rasa assistants.