Tuning Your NLU Model

You are viewing documentation for our open source project which is maintained by the community. If you want to get started building assistants with Rasa please check out our latest documentation here.

How to Choose a Pipeline

In Rasa, incoming messages are processed by a sequence of components. These components are executed one after another in a so-called processing pipeline defined in your config.yml. Choosing an NLU pipeline allows you to customize your model and finetune it on your dataset.

To get started, you can let the Suggested Config feature choose a default pipeline for you. Just provide your bot's language in the config.yml file and leave the pipeline key out or empty.

Sensible Starting Pipelines

If you're starting from scratch, it's often helpful to start with pretrained word embeddings. Pre-trained word embeddings are helpful as they already encode some kind of linguistic knowledge. For example, if you have a sentence like “I want to buy apples” in your training data, and Rasa is asked to predict the intent for “get pears”, your model already knows that the words “apples” and “pears” are very similar. This is especially useful if you don't have enough training data.

If you are getting started with one of spaCy's supported languages, we recommend the following pipeline:

id: default_spacy_bot
language: "fr" # your two-letter language code
pipeline:
  - name: SpacyNLP
  - name: SpacyTokenizer
  - name: SpacyFeaturizer
  - name: RegexFeaturizer
  - name: LexicalSyntacticFeaturizer
  - name: CountVectorsFeaturizer
    analyzer: "char_wb"
    min_ngram: 1
    max_ngram: 4
  - name: DIETClassifier
    epochs: 100
  - name: EntitySynonymMapper
  - name: ResponseSelector
    epochs: 100

This pipeline uses the SpacyFeaturizer, which provides pre-trained word embeddings (see Language Models).

If you don't use any pre-trained word embeddings inside your pipeline, you are not bound to a specific language and can train your model to be more domain-specific. If there are no word embeddings for your language or you have very domain-specific terminology, we recommend using the following pipeline:

id: default_config_bot
language: "fr" # your two-letter language code
pipeline:
  - name: WhitespaceTokenizer
  - name: RegexFeaturizer
  - name: LexicalSyntacticFeaturizer
  - name: CountVectorsFeaturizer
    analyzer: "char_wb"
    min_ngram: 1
    max_ngram: 4
  - name: DIETClassifier
    epochs: 100
  - name: EntitySynonymMapper
  - name: ResponseSelector
    epochs: 100

This pipeline uses the CountVectorsFeaturizer to train on only the training data you provide. This pipeline can handle any language in which words are separated by spaces. If this is not the case for your language, check out alternatives to the WhitespaceTokenizer.

Component Lifecycle

Each component processes an input and/or creates an output. The order of the components is determined by the order they are listed in the config.yml; the output of a component can be used by any other component that comes after it in the pipeline. Some components only produce information used by other components in the pipeline. Other components produce output attributes that are returned after the processing has finished.

For example, for the sentence "I am looking for Chinese food", the output is:

{
  "text": "I am looking for Chinese food",
  "entities": [
    {
      "start": 8,
      "end": 15,
      "value": "chinese",
      "entity": "cuisine",
      "extractor": "DIETClassifier",
      "confidence": 0.864
    }
  ],
  "intent": {"confidence": 0.6485910906220309, "name": "restaurant_search"},
  "intent_ranking": [
    {"confidence": 0.6485910906220309, "name": "restaurant_search"},
    {"confidence": 0.1416153159565678, "name": "affirm"}
  ]
}

This is created as a combination of the results of the different components in the following pipeline:

pipeline:
  - name: WhitespaceTokenizer
  - name: RegexFeaturizer
  - name: LexicalSyntacticFeaturizer
  - name: CountVectorsFeaturizer
    analyzer: "char_wb"
    min_ngram: 1
    max_ngram: 4
  - name: DIETClassifier
  - name: EntitySynonymMapper
  - name: ResponseSelector

For example, the entities attribute here is created by the DIETClassifier component.

Doing Multi-Intent Classification

You can use multi-intent classification to predict multiple intents (e.g. check_balances+transfer_money), or to model hierarchical intent structure (e.g. feedback+positive being more similar to feedback+negative than chitchat).

To do multi-intent classification, you need to use the DIETClassifier in your pipeline. You'll also need to define these flags in whichever tokenizer you are using:

intent_tokenization_flag: Set it to True, so that intent labels are tokenized.
intent_split_symbol: Set it to the delimiter string that splits the intent labels. In this case +, default _.

Here's an example configuration:

language: "en"
pipeline:
  - name: "WhitespaceTokenizer"
    intent_tokenization_flag: true
    intent_split_symbol: "+"
  - name: "CountVectorsFeaturizer"
  - name: "DIETClassifier"

How to Use Multi-Intents for Dialogue Management

Multi-intent classification is intended to help with the downstream task of action prediction after a multi-intent. There are two complementary ways to use multi-intents in dialogue training data:

Add regular stories or rules for the multi-intent. For example, given the following two rules for each individual intent:

    rules:
      - rule: check account balance
        steps:
          - intent: check_balances
          - action: action_check_balances
      - rule: transfer money
        steps:
          - intent: transfer_money
          - action: action_transfer_money
    ```

You could add another rule for the multi-intent that specifies a sequence of actions to address both intents:

```yaml
    rules:
      - rule: check balances and transfer money
        steps:
          - intent: check_balances+transfer_money
          - action: action_check_balances
          - action: action_transfer_money
    ```

2. Allow a machine-learning policy to generalize to the multi-intent scenario from single-intent stories.

When using a multi-intent, the intent is featurized for machine learning policies using multi-hot encoding. That means the featurization of `check_balances+transfer_money` will overlap with the featurization of each individual intent. Machine learning policies (like [TEDPolicy](https://legacy-docs-oss.rasa.com/docs/rasa/policies#ted-policy)) can then make a prediction based on the multi-intent even if it does not explicitly appear in any stories. It will typically act as if only one of the individual intents was present, however, so it is always a good idea to write a specific story or rule that deals with the multi-intent case.

### Comparing Pipelines

Rasa gives you the tools to compare the performance of multiple pipelines on your data directly. See [Comparing NLU Pipelines](https://legacy-docs-oss.rasa.com/docs/rasa/testing-your-assistant#comparing-nlu-pipelines) for more information.

## Choosing the Right Components

There are components for entity extraction, for intent classification, response selection, pre-processing, and others. If you want to add your own component, for example to run a spell-check or to do sentiment analysis, check out [Custom NLU Components](https://legacy-docs-oss.rasa.com/docs/rasa/components).

### Tokenization

You can process whitespace-tokenized (i.e. words are separated by spaces) languages with the [WhitespaceTokenizer](https://legacy-docs-oss.rasa.com/docs/rasa/components#whitespacetokenizer). If your language is not whitespace-tokenized, you should use a different tokenizer. We support a number of different [tokenizers](https://legacy-docs-oss.rasa.com/docs/rasa/components), or you can create your own [custom tokenizer](https://legacy-docs-oss.rasa.com/docs/rasa/components).

### Featurization

You need to decide whether to use components that provide pre-trained word embeddings or not. We recommend in cases of small amounts of training data to start with pre-trained word embeddings. Once you have a larger amount of data and ensure that most relevant words will be in your data and therefore will have a word embedding, supervised embeddings, which learn word meanings directly from your training data, can make your model more specific to your domain. If you can't find a pre-trained model for your language, you should use supervised embeddings.

#### Pre-trained Embeddings

The advantage of using pre-trained word embeddings in your pipeline is that if you have a training example like: “I want to buy apples”, and Rasa is asked to predict the intent for “get pears”, your model already knows that the words “apples” and “pears” are very similar. This is especially useful if you don't have enough training data. We support a few components that provide pre-trained word embeddings:

1. [MitieFeaturizer](https://legacy-docs-oss.rasa.com/docs/rasa/components#mitiefeaturizer)
2. [SpacyFeaturizer](https://legacy-docs-oss.rasa.com/docs/rasa/components#spacyfeaturizer)
3. [ConveRTFeaturizer](https://legacy-docs-oss.rasa.com/docs/rasa/components#convertfeaturizer)
4. [LanguageModelFeaturizer](https://legacy-docs-oss.rasa.com/docs/rasa/components#languagemodelfeaturizer)

If your training data is in English, we recommend using the [ConveRTFeaturizer](https://legacy-docs-oss.rasa.com/docs/rasa/components#convertfeaturizer). The advantage of the [ConveRTFeaturizer](https://legacy-docs-oss.rasa.com/docs/rasa/components#convertfeaturizer) is that it doesn't treat each word of the user message independently, but creates a contextual vector representation for the complete sentence. For example, if you have a training example, like: “Can I book a car?”, and Rasa is asked to predict the intent for “I need a ride from my place”, since the contextual vector representation for both examples are already very similar, the intent classified for both is highly likely to be the same. This is also useful if you don't have enough training data.

An alternative to [ConveRTFeaturizer](https://legacy-docs-oss.rasa.com/docs/rasa/components#convertfeaturizer) is the [LanguageModelFeaturizer](https://legacy-docs-oss.rasa.com/docs/rasa/components#languagemodelfeaturizer) which uses pre-trained language models such as BERT, GPT-2, etc. to extract similar contextual vector representations for the complete sentence.

### Entity Extraction

Entity extraction involves parsing user messages for required pieces of information. Rasa provides entity extractors for custom entities as well as pre-trained ones like dates and locations. Here is a summary of the available extractors and what they are best used for:

| Component | Requires | Model | Notes |
| --- | --- | --- | --- |
| `DIETClassifier` | N/A | conditional random field on top of a transformer | good for training custom entities |
| `CRFEntityExtractor` | sklearn-crfsuite | conditional random field | good for training custom entities |
| `SpacyEntityExtractor` | spaCy | averaged perceptron | provides pre-trained entities |
| `DucklingEntityExtractor` | running duckling | context-free grammar | provides pre-trained entities |
| `MitieEntityExtractor` | MITIE | structured SVM | good for training custom entities |
| `EntitySynonymMapper` | existing entities | N/A | maps known synonyms |

### Improving Performance

#### Handling Class Imbalance

To mitigate the problem of class imbalance in classification, you can use a `balanced` batching strategy. This algorithm ensures that all classes are represented in every batch. Balanced batching is used by default. In order to turn it off include `batch_strategy: sequence` in your config file.

```yaml
language: "en"
pipeline:
  - name: "DIETClassifier"
    batch_strategy: sequence

Accessing Diagnostic Data

To gain a better understanding of what your models do, you can access intermediate results of the prediction process. To do this, you need to access the diagnostic_data field of the Message and Prediction objects.

Configuring Tensorflow

TensorFlow allows configuring options in the runtime environment via TF Config submodule. Rasa supports a smaller subset of these configuration options. All configuration options are specified using environment variables as shown in subsequent sections.

Optimizing CPU Performance

Set TF_INTRA_OP_PARALLELISM_THREADS as an environment variable to specify the maximum number of threads that can be used to parallelize the execution of one operation. The default value is 0 which means TensorFlow would allocate one thread per CPU core.

Set TF_INTER_OP_PARALLELISM_THREADS as an environment variable to specify the maximum number of threads that can be used to parallelize the execution of multiple non-blocking operations. The default value is 0 which means TensorFlow would allocate one thread per CPU core.

Optimizing GPU Performance

Set the environment variable TF_FORCE_GPU_ALLOW_GROWTH to True to prevent Rasa from blocking all of the available GPU memory.

Set the environment variable TF_GPU_MEMORY_ALLOC to limit the absolute amount of GPU memory that can be used by a Rasa process (e.g., "0:1024, 1:2048").