Tuning Your NLU Model

How to Choose a Pipeline

In Rasa, incoming messages are processed by a sequence of components. These components are executed one after another in a so-called processing pipeline defined in your config.yml. Choosing an NLU pipeline allows you to customize your model and finetune it on your dataset.

To get started, you can let the Suggested Config feature choose a default pipeline for you. Just provide your bot's language in the config.yml file and leave the pipeline key out or empty.

language: fr  # your 2-letter language code

pipeline:

# intentionally left empty

Sensible Starting Pipelines

If you're starting from scratch, it's often helpful to start with pretrained word embeddings. Pre-trained word embeddings are helpful as they already encode some kind of linguistic knowledge. For example, if you have a sentence like “I want to buy apples” in your training data, and Rasa is asked to predict the intent for “get pears”, your model already knows that the words “apples” and “pears” are very similar. This is especially useful if you don't have enough training data.

If you are getting started with one of spaCy's supported languages, we recommend the following pipeline:

pipeline:
  - name: SpacyNLP
  - name: SpacyTokenizer
  - name: SpacyFeaturizer
  - name: RegexFeaturizer
  - name: LexicalSyntacticFeaturizer
  - name: CountVectorsFeaturizer
      analyzer: "char_wb"
      min_ngram: 1
      max_ngram: 4
  - name: DIETClassifier
    epochs: 100
  - name: EntitySynonymMapper
  - name: ResponseSelector
    epochs: 100

This pipeline uses the SpacyFeaturizer, which provides pre-trained word embeddings.

If you don't use any pre-trained word embeddings inside your pipeline, you are not bound to a specific language and can train your model to be more domain-specific. If there are no word embeddings for your language or you have very domain-specific terminology, we recommend using the following pipeline:

pipeline:
  - name: WhitespaceTokenizer
  - name: RegexFeaturizer
  - name: LexicalSyntacticFeaturizer
  - name: CountVectorsFeaturizer
      analyzer: "char_wb"
      min_ngram: 1
      max_ngram: 4
  - name: DIETClassifier
    epochs: 100
  - name: EntitySynonymMapper
  - name: ResponseSelector
    epochs: 100

Component Lifecycle

Each component processes an input and/or creates an output. The order of the components is determined by the order they are listed in the config.yml; the output of a component can be used by any other component that comes after it in the pipeline. Some components only produce information used by other components in the pipeline. Other components produce output attributes that are returned after the processing has finished.

For example, for the sentence "I am looking for Chinese food", the output is:

{
  "text":"I am looking for Chinese food",
  "entities":[
    {
      "start":8,
      "end":15,
      "value":"chinese",
      "entity":"cuisine",
      "extractor":"DIETClassifier",
      "confidence":0.864
    }
  ],
  "intent":{"confidence":0.6485910906220309,"name":"restaurant_search"},
  "intent_ranking":[
    {"confidence":0.6485910906220309,"name":"restaurant_search"},
    {"confidence":0.1416153159565678,"name":"affirm"}
  ]
}

This is created as a combination of the results of the different components in the following pipeline:

pipeline:
  - name: WhitespaceTokenizer
  - name: RegexFeaturizer
  - name: LexicalSyntacticFeaturizer
  - name: CountVectorsFeaturizer
      analyzer: "char_wb"
      min_ngram: 1
      max_ngram: 4
  - name: DIETClassifier
  - name: EntitySynonymMapper
  - name: ResponseSelector

For example, the entities attribute here is created by the DIETClassifier component.

Every component can implement several methods from the Component base class; in a pipeline these different methods will be called in a specific order. Assuming we added the following pipeline to our config.yml:

pipeline:
  - name:"Component A"
  - name:"Component B"
  - name:"Last Component"

The image below shows the call order during the training of this pipeline:

Component Lifecycle

Before the first component is created using the create function, a so-called context is created (which is nothing more than a python dict). This context is used to pass information between the components. For example, one component can calculate feature vectors for the training data, store that within the context, and another component can retrieve these feature vectors from the context and do intent classification.

Initially, the context is filled with all configuration values. The arrows in the image show the call order and visualize the path of the passed context. After all components are trained and persisted, the final context dictionary is used to persist the model's metadata.

Doing Multi-Intent Classification

You can use multi-intent classification to predict multiple intents (e.g. check_balances+transfer_money), or to model hierarchical intent structure (e.g. feedback+positive being more similar to feedback+negative than chitchat).

To do multi-intent classification, you need to use the DIETClassifier in your pipeline. You'll also need to define these flags in whichever tokenizer you are using:

intent_tokenization_flag: Set it to True, so that intent labels are tokenized.
intent_split_symbol: Set it to the delimiter string that splits the intent labels. In this case +, default _.

Here's an example configuration:

language:"en"

pipeline:
  - name:"WhitespaceTokenizer"
    intent_tokenization_flag: True
    intent_split_symbol: "+"
  - name:"CountVectorsFeaturizer"
  - name:"DIETClassifier"

When to Use Multi-Intents

Let's say you have a financial services bot and you have examples for intents check_balances and transfer_money:

nlu:
  - intent: check_balances
    examples: |
      - How much money do I have?
      - what's my account balance?
  - intent: transfer_money
    examples: |
      - I want to transfer money to my savings account
      - transfer money

However, your bot receives incoming messages like this one, which combine both intents:

User: How much money do I have? I want to transfer some to savings.

User wants to know balance in order to transfer money

If you see enough of these examples, you can create a new intent multi-intent check_balances+transfer_money and add the incoming examples to it, for example:

nlu:
  - intent: check_balances+transfer_money
    examples: |
      - How much money do I have? I want to transfer some to savings.
      - What's the balance on my account? I need to transfer some so I want to know how much I have

How to Use Multi-Intents for Dialogue Management

Multi-intent classification is intended to help with the downstream task of action prediction after a multi-intent. There are two complementary ways to use multi-intents in dialogue training data:

Add regular stories or rules for the multi-intent. For example, given the following two rules for each individual intent:

rules:
  - rule: check account balance
    steps:
      - intent: check_balances
      - action: action_check_balances
  - rule: transfer money
    steps:
      - intent: transfer_money
      - action: action_transfer_money

You could add another rule for the multi-intent that specifies a sequence of actions to address both intents:

rules:
  - rule: check balances and transfer money
    steps:
      - intent: check_balances+transfer_money
      - action: action_check_balances
      - action: action_transfer_money

Allow a machine-learning policy to generalize to the multi-intent scenario from single-intent stories.

When using a multi-intent, the intent is featurized for machine learning policies using multi-hot encoding. That means the featurization of check_balances+transfer_money will overlap with the featurization of each individual intent. Machine learning policies (like TEDPolicy) can then make a prediction based on the multi-intent even if it does not explicitly appear in any stories. It will typically act as if only one of the individual intents was present, however, so it is always a good idea to write a specific story or rule that deals with the multi-intent case.

Comparing Pipelines

Rasa gives you the tools to compare the performance of multiple pipelines on your data directly. See Comparing NLU Pipelines for more information.

Choosing the Right Components

There are components for entity extraction, for intent classification, response selection, pre-processing, and others. If you want to add your own component, for example to run a spell-check or to do sentiment analysis, check out Custom NLU Components.

A pipeline usually consists of three main parts:

Tokenization

You can process whitespace-tokenized (i.e. words are separated by spaces) languages with the WhitespaceTokenizer. If your language is not whitespace-tokenized, you should use a different tokenizer. We support a number of different tokenizers, or you can create your own custom tokenizer.

Featurization

You need to decide whether to use components that provide pre-trained word embeddings or not. We recommend in cases of small amounts of training data to start with pre-trained word embeddings. Once you have a larger amount of data and ensure that most relevant words will be in your data and therefore will have a word embedding, supervised embeddings, which learn word meanings directly from your training data, can make your model more specific to your domain. If you can't find a pre-trained model for your language, you should use supervised embeddings.

Intent Classification / Response Selectors

Depending on your data you may want to only perform intent classification, entity recognition or response selection. Or you might want to combine multiple of those tasks. We support several components for each of the tasks. We recommend using DIETClassifier for intent classification and entity recognition and ResponseSelector for response selection.

By default all of these components consume all available features produced in the pipeline. However, sometimes it makes sense to restrict the features that are used by a specific component.