Training Data Format

These docs are for version 1.x of Rasa Open Source.

Data Formats

You can provide training data as Markdown or as JSON, as a single file or as a directory containing multiple files. Note that Markdown is usually easier to work with.

Markdown Format

Markdown is the easiest Rasa NLU format for humans to read and write. Examples are listed using the unordered list syntax, e.g. minus -, asterisk *, or plus +. Examples are grouped by intent, and entities are annotated as Markdown links, e.g. [<entity text>](<entity name>), or by using the following syntax [<entity-text>]{"entity": "<entity name>"}. Using the latter syntax, you can also assign synonyms, roles, or groups to an entity, e.g. [<entity-text>]{"entity": "<entity name>", "role": "<role name>", "group": "<group name>", "value": "<entity synonym>"}. The keywords role, group, and value are optional in this notation.

## intent:check_balance
- what is my balance <!-- no entity -->
- how much do I have on my [savings](source_account) <!-- entity "source_account" has value "savings" -->
- how much do I have on my [savings account]{"entity": "source_account", "value": "savings"} <!-- synonyms, method 1-->
- Could I pay in [yen](currency)?  <!-- entity matched by lookup table -->

## intent:greet
- hey
- hello

## synonym:savings   <!-- synonyms, method 2 -->
- pink pig

## regex:zipcode
- [0-9]{5}

## lookup:additional_currencies  <!-- specify lookup tables in an external file -->
path/to/currencies.txt

JSON Format

The JSON format consists of a top-level object called rasa_nlu_data, with the keys common_examples, entity_synonyms and regex_features. The most important one is common_examples.

{
    "rasa_nlu_data": {
        "common_examples": [],
        "regex_features" : [],
        "lookup_tables"  : [],
        "entity_synonyms": []
    }
}

Improving Intent Classification and Entity Recognition

Common Examples

Common examples have three components: text, intent and entities. The first two are strings while the last one is an array.

• The text is the user message [required]
• The intent is the intent that should be associated with the text [optional]
• The entities are specific parts of the text which need to be identified [optional]

Entities are specified with a start and an end value, which together make a range to apply to the string.

## intent:restaurant_search
- show me [chinese](cuisine) restaurants

Regular Expression Features

Regular expressions can be used to support the intent classification and entity extraction. For example:

## regex:zipcode
- [0-9]{5}

## regex:greet
- hey[^\s]*

Lookup Tables

Lookup tables provide a convenient way to supply a list of entity examples. The supplied lookup table files must be in a newline-delimited format. For example:

## lookup:plates
data/test/lookup_tables/plates.txt

Normalizing Data

Entity Synonyms

If you define entities as having the same value they will be treated as synonyms. Here is an example of that:

## intent:search
- in the center of [NYC]{"entity": "city", "value": "New York City")
- in the centre of [New York City](city)

To use the synonyms defined in your training data, you need to make sure the pipeline contains the EntitySynonymMapper component.

Note

Please note that adding synonyms using the above format does not improve the model’s classification of those entities. Entities must be properly classified before they can be replaced with the synonym value.