Training Data Format

The training data for Rasa NLU is structured into different parts:

• Common Examples: The only part that is mandatory, including various intents and examples.
• Synonyms: Maps extracted entities to the same name, e.g., mapping "my savings account" to simply "savings".
• Regex Features: Can define patterns for deterministic entities, aiding the intent classification.
• Lookup Tables: Supply a list of entity examples, helping in identifying them in user inputs.

Data Formats

You can provide training data as Markdown or as JSON, as a single file or as a directory containing multiple files.

Markdown Format

Markdown is the easiest format for humans to read and write. Examples are listed using the unordered list syntax. The training examples are grouped by intent, and entities can be annotated in various ways.

## intent:check_balance
- what is my balance <!-- no entity -->
- how much do I have on my [savings](source_account) <!-- entity "source_account" has value "savings" -->
- how much do I have on my [savings account]{"entity": "source_account", "value": "savings"} <!-- synonyms, method 1-->
## intent:greet
- hey
- hello

JSON Format

The JSON format consists of a top-level object called rasa_nlu_data, including keys such as common_examples, entity_synonyms, and regex_features.

{
    "rasa_nlu_data": {
        "common_examples": [],
        "regex_features" : [],
        "lookup_tables"  : [],
        "entity_synonyms": []
    }
}

Improving Intent Classification and Entity Recognition

Common examples have three components: text, intent, and entities. The first two are strings while the last one is an array.

Common Examples

Entities are specified with a start and end value, defining a range for matching the text. For instance:

## intent:restaurant_search
- show me [chinese](cuisine) restaurants

Regular Expression Features

Regex can be used to detect entities. Here’s an example:

## regex:zipcode
- [0-9]{5}

Lookup Tables

You can load the contents of a lookup table file, which must be newline-delimited, allowing for easy matching in training data.

Normalizing Data

Entity Synonyms

If multiple entities share the same value, define them as synonyms which can enhance the model's understanding.

## synonym:New York City
- NYC
- nyc
- the big apple