## How to Choose a Pipeline In Rasa Open Source, incoming messages are processed by a sequence of components. These components are executed one after another in a so-called processing `pipeline` defined in your `config.yml`. Choosing an NLU pipeline allows you to customize your model and finetune it on your dataset. To get started, you can let the [Suggested Config](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/model-configuration#suggested-config) feature choose a default pipeline for you. Just provide your bot's `language` in the `config.yml` file and leave the `pipeline` key out or empty. ```yaml language: fr # your 2-letter language code pipeline: # intentionally left empty ``` ### Sensible Starting Pipelines If you're starting from scratch, it's often helpful to start with pretrained word embeddings. Pre-trained word embeddings are helpful as they already encode some kind of linguistic knowledge. For example, if you have a sentence like “I want to buy apples” in your training data, and Rasa is asked to predict the intent for “get pears”, your model already knows that the words “apples” and “pears” are very similar. This is especially useful if you don't have enough training data. If you are getting started with one of [spaCy's supported languages](https://spacy.io/usage/models#languages), we recommend the following pipeline: ```yaml language: "fr" # your two-letter language code pipeline: - name: SpacyNLP - name: SpacyTokenizer - name: SpacyFeaturizer - name: RegexFeaturizer - name: LexicalSyntacticFeaturizer - name: CountVectorsFeaturizer analyzer: "char_wb" min_ngram: 1 max_ngram: 4 - name: DIETClassifier epochs: 100 - name: EntitySynonymMapper - name: ResponseSelector epochs: 100 ``` This pipeline uses the [SpacyFeaturizer](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/components#spacyfeaturizer), which provides pre-trained word embeddings. If you don't use any pre-trained word embeddings inside your pipeline, you are not bound to a specific language and can train your model to be more domain specific. If there are no word embeddings for your language or you have very domain specific terminology, we recommend using the following pipeline: ```yaml language: "fr" # your two-letter language code pipeline: - name: WhitespaceTokenizer - name: RegexFeaturizer - name: LexicalSyntacticFeaturizer - name: CountVectorsFeaturizer analyzer: "char_wb" min_ngram: 1 max_ngram: 4 - name: DIETClassifier epochs: 100 - name: EntitySynonymMapper - name: ResponseSelector epochs: 100 ``` This pipeline uses the [CountVectorsFeaturizer](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/components#countvectorsfeaturizer) to train on only the training data you provide. This pipeline can handle any language in which words are separated by spaces. If this is not the case for your language, check out [alternatives to the WhitespaceTokenizer](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/components#tokenizers). ##### note If you want to use custom components in your pipeline, see [Custom NLU Components](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/components). ### Component Lifecycle Each component processes an input and/or creates an output. The order of the components is determined by the order they are listed in the `config.yml`; the output of a component can be used by any other component that comes after it in the pipeline. Some components only produce information used by other components in the pipeline. Other components produce `output` attributes that are returned after the processing has finished. For example, for the sentence "I am looking for Chinese food", the output is: ```json { "text": "I am looking for Chinese food", "entities": [ { "start": 8, "end": 15, "value": "chinese", "entity": "cuisine", "extractor": "DIETClassifier", "confidence": 0.864 } ], "intent": { "confidence": 0.6485910906220309, "name": "restaurant_search" }, "intent_ranking": [ { "confidence": 0.6485910906220309, "name": "restaurant_search" }, { "confidence": 0.1416153159565678, "name": "affirm" } ] } ``` This is created as a combination of the results of the different components in the following pipeline: ```yaml pipeline: - name: WhitespaceTokenizer - name: RegexFeaturizer - name: LexicalSyntacticFeaturizer - name: CountVectorsFeaturizer analyzer: "char_wb" min_ngram: 1 max_ngram: 4 - name: DIETClassifier - name: EntitySynonymMapper - name: ResponseSelector ``` For example, the `entities` attribute here is created by the `DIETClassifier` component. Every component can implement several methods from the `Component` base class; in a pipeline these different methods will be called in a specific order. Assuming we added the following pipeline to our `config.yml`: ```yaml pipeline: - name: "Component A" - name: "Component B" - name: "Last Component" ``` The image below shows the call order during the training of this pipeline: Component Lifecycle Before the first component is created using the `create` function, a so called `context` is created (which is nothing more than a python dict). This context is used to pass information between the components. For example, one component can calculate feature vectors for the training data, store that within the context and another component can retrieve these feature vectors from the context and do intent classification. Initially the context is filled with all configuration values. The arrows in the image show the call order and visualize the path of the passed context. After all components are trained and persisted, the final context dictionary is used to persist the model's metadata. ### Doing Multi-Intent Classification You can use multi-intent classification to predict multiple intents (e.g. `check_balances+transfer_money`), or to model hierarchical intent structure (e.g. `feedback+positive` being more similar to `feedback+negative` than `chitchat`). To do multi-intent classification, you need to use the [DIETClassifier](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/components#dietclassifier) in your pipeline. You'll also need to define these flags in whichever tokenizer you are using: - `intent_tokenization_flag`: Set it to `True`, so that intent labels are tokenized. - `intent_split_symbol`: Set it to the delimiter string that splits the intent labels. In this case `+`, default `_`. Here's an example configuration: ```yaml language: "en" pipeline: - name: "WhitespaceTokenizer" intent_tokenization_flag: True intent_split_symbol: "+" - name: "CountVectorsFeaturizer" - name: "DIETClassifier" ``` #### When to Use Multi-Intents Let's say you have a financial services bot and you have examples for intents `check_balances` and `transfer_money`: ```yaml nlu: - intent: check_balances examples: | - How much money do I have? - what's my account balance? - intent: transfer_money examples: | - I want to transfer money to my savings account - transfer money ``` However, your bot receives incoming messages like this one, which combine both intents: User: How much money do I have? I want to transfer some to savings. User wants to know balance in order to transfer money If you see enough of these examples, you can create a new intent multi-intent `check_balances+transfer_money` and add the incoming examples to it, for example: ```yaml nlu: - intent: check_balances+transfer_money examples: | - How much money do I have? I want to transfer some to savings. - What's the balance on my account? I need to transfer some so I want to know how much I have ``` ##### note The model will not predict any combination of intents for which examples are not explicitly given in training data. As accounting for every possible intent combination would result in a combinatorial explosion of the number of intents, you should only add those combinations of intents for which you see enough examples coming in from real users. #### How to Use Multi-Intents for Dialogue Management Multi-intent classification is intended to help with the downstream task of action prediction _after_ a multi-intent. There are two complementary ways to use multi intents in dialogue training data: 1) Add regular stories or rules for the multi-intent. For example, given the following two rules for each individual intent: ```yaml rules: - rule: check account balance steps: - intent: check_balances action: action_check_balances - rule: transfer money steps: - intent: transfer_money action: action_transfer_money ``` You could add another rule for the multi-intent that specifies a sequence of actions to address both intents: ```yaml rules: - rule: check balances and transfer money steps: - intent: check_balances+transfer_money - action: action_check_balances - action: action_transfer_money ``` 2) Allow a machine-learning policy to generalize to the multi-intent scenario from single-intent stories. When using a multi-intent, the intent is featurized for machine learning policies using multi-hot encoding. That means the featurization of `check_balances+transfer_money` will overlap with the featurization of each individual intent. Machine learning policies (like [TEDPolicy](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/policies#ted-policy)) can then make a prediction based on the multi-intent even if it does not explicitly appear in any stories. It will typically act as if only one of the individual intents was present, however, so it is always a good idea to write a specific story or rule that deals with the multi-intent case. ### Comparing Pipelines Rasa gives you the tools to compare the performance of multiple pipelines on your data directly. See [Comparing NLU Pipelines](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/testing-your-assistant#comparing-nlu-pipelines) for more information. ## Choosing the Right Components There are components for entity extraction, for intent classification, response selection, pre-processing, and others. If you want to add your own component, for example to run a spell-check or to do sentiment analysis, check out [Custom NLU Components](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/components). A pipeline usually consists of three main parts: ### Tokenization You can process whitespace-tokenized (i.e. words are separated by spaces) languages with the [WhitespaceTokenizer](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/components#whitespacetokenizer). If your language is not whitespace-tokenized, you should use a different tokenizer. We support a number of different [tokenizers](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/components), or you can create your own [custom tokenizer](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/components). ##### note Some components further down the pipeline may require a specific tokenizer. You can find those requirements on the individual components' `requires` parameter. If a required component is missing inside the pipeline, an error will be thrown. ### Featurization You need to decide whether to use components that provide pre-trained word embeddings or not. We recommend in cases of small amounts of training data to start with pre-trained word embeddings. Once you have a larger amount of data and ensure that most relevant words will be in your data and therefore will have a word embedding, supervised embeddings, which learn word meanings directly from your training data, can make your model more specific to your domain. If you can't find a pre-trained model for your language, you should use supervised embeddings. #### Pre-trained Embeddings The advantage of using pre-trained word embeddings in your pipeline is that if you have a training example like: “I want to buy apples”, and Rasa is asked to predict the intent for “get pears”, your model already knows that the words “apples” and “pears” are very similar. This is especially useful if you don't have enough training data. We support a few components that provide pre-trained word embeddings: 1. [MitieFeaturizer](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/components#mitiefeaturizer) 2. [SpacyFeaturizer](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/components#spacyfeaturizer) 3. [ConveRTFeaturizer](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/components#convertfeaturizer) 4. [LanguageModelFeaturizer](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/components#languagemodelfeaturizer) If your training data is in English, we recommend using the [ConveRTFeaturizer](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/components#convertfeaturizer). The advantage of the [ConveRTFeaturizer](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/components#convertfeaturizer) is that it doesn't treat each word of the user message independently, but creates a contextual vector representation for the complete sentence. For example, if you have a training example, like: “Can I book a car?”, and Rasa is asked to predict the intent for “I need a ride from my place”, since the contextual vector representation for both examples are already very similar, the intent classified for both is highly likely to be the same. This is also useful if you don't have enough training data. An alternative to [ConveRTFeaturizer](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/components#convertfeaturizer) is the [LanguageModelFeaturizer](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/components#languagemodelfeaturizer) which uses pre-trained language models such as BERT, GPT-2, etc. to extract similar contextual vector representations for the complete sentence. See [HFTransformersNLP](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/components#hftransformersnlp) for a full list of supported language models. If your training data is not in English you can also use a different variant of a language model which is pre-trained in the language specific to your training data. For example, there are chinese (`bert-base-chinese`) and japanese (`bert-base-japanese`) variants of the BERT model. A full list of different variants of these language models is available in the [official documentation of the Transformers library](https://huggingface.co/transformers/pretrained_models.html). [spacynlp](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/components#spacyfeaturizer) also provides word embeddings in many different languages, so you can use this as another alternative, depending on the language of your training data. #### Supervised Embeddings If you don't use any pre-trained word embeddings inside your pipeline, you are not bound to a specific language and can train your model to be more domain specific. For example, in general English, the word “balance” is closely related to “symmetry”, but very different to the word “cash”. In a banking domain, “balance” and “cash” are closely related and you'd like your model to capture that. You should only use featurizers from the category [sparse featurizers](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/components#featurizers), such as [CountVectorsFeaturizer](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/components#countvectorsfeaturizer), [RegexFeaturizer](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/components#regexfeaturizer) or [LexicalSyntacticFeaturizer](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/components#lexicalsyntacticfeaturizer), if you don't want to use pre-trained word embeddings. ### Intent Classification / Response Selectors Depending on your data you may want to only perform intent classification, entity recognition or response selection. Or you might want to combine multiple of those tasks. We support several components for each of the tasks. We recommend using [DIETClassifier](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/components#dietclassifier) for intent classification and entity recognition and [ResponseSelector](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/components#responseselector) for response selection. By default all of these components consume all available features produced in the pipeline. However, sometimes it makes sense to restrict the features that are used by a specific component. For example, [ResponseSelector](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/components#responseselector) is likely to perform better if no features from the [RegexFeaturizer](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/components#regexfeaturizer) or [LexicalSyntacticFeaturizer](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/components#lexicalsyntacticfeaturizer) are used. To achieve that, you can do the following: Set an alias for every featurizer in your pipeline via the option `alias`. By default the alias is set to the full featurizer class name, for example, `RegexFeaturizer`. You can then specify, for example, on the [ResponseSelector](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/components#responseselector) via the option `featurizers` what features from which featurizers should be used. If you don't set the option `featurizers` all available features will be used. Here is an example configuration file where the `DIETClassifier` is using all available features and the `ResponseSelector` is just using the features from the `ConveRTFeaturizer` and the `CountVectorsFeaturizer`. ```yaml language: "en" pipeline: - name: ConveRTTokenizer - name: ConveRTFeaturizer alias: "convert" - name: RegexFeaturizer alias: "regex" - name: LexicalSyntacticFeaturizer alias: "lexical-syntactic" - name: CountVectorsFeaturizer alias: "cvf-word" - name: CountVectorsFeaturizer alias: "cvf-char" analyzer: "char_wb" min_ngram: 1 max_ngram: 4 - name: DIETClassifier epochs: 100 - name: EntitySynonymMapper - name: ResponseSelector featurizers: ["convert", "cvf-word"] epochs: 100 ``` ### Entity Extraction Entity extraction involves parsing user messages for required pieces of information. Rasa Open Source provides entity extractors for custom entities as well as pre-trained ones like dates and locations. Here is a summary of the available extractors and what they are best used for: | Component | Requires | Model | Notes | | --- | --- | --- | --- | | `DIETClassifier` | N/A | conditional random field on top of a transformer | good for training custom entities | | `CRFEntityExtractor` | sklearn-crfsuite | conditional random field | good for training custom entities | | `SpacyEntityExtractor` | spaCy | averaged perceptron | provides pre-trained entities | | `DucklingEntityExtractor` | running duckling | context-free grammar | provides pre-trained entities | | `MitieEntityExtractor` | MITIE | structured SVM | good for training custom entities | | `EntitySynonymMapper` | existing entities | N/A | maps known synonyms | ## Improving Performance ### Handling Class Imbalance Classification algorithms often do not perform well if there is a large class imbalance, for example if you have a lot of training data for some intents and very little training data for others. To mitigate this problem, you can use a `balanced` batching strategy. This algorithm ensures that all classes are represented in every batch, or at least in as many subsequent batches as possible, still mimicking the fact that some classes are more frequent than others. Balanced batching is used by default. In order to turn it off and use a classic batching strategy include `batch_strategy: sequence` in your config file. ```yaml language: "en" pipeline: # - ... other components - name: "DIETClassifier" batch_strategy: sequence ``` ### Accessing Diagnostic Data ##### New in 2.3 Diagnostic data was made available. To gain a better understanding of what your models do, you can access intermediate results of the prediction process. To do this, you need to access the `diagnostic_data` field of the [Message](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/reference/rasa/shared/nlu/training_data/message) and [Prediction](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/reference/rasa/core/policies/policy#predict_action_probabilities) objects, which contain information about attention weights and other intermediate results of the inference computation. You can use this information for debugging and fine-tuning, e.g. with [RasaLit](https://github.com/RasaHQ/rasalit). After you've [trained a model](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/command-line-interface#rasa-train), you can access diagnostic data for DIET, given a processed message, like this: ```python nlu_diagnostic_data = message.as_dict()[DIAGNOSTIC_DATA] for component_name, diagnostic_data in nlu_diagnostic_data.items(): attention_weights = diagnostic_data["attention_weights"] print(f"attention_weights for {component_name}:") print(attention_weights) text_transformed = diagnostic_data["text_transformed"] print(f"\ntext_transformed for {component_name}:") print(text_transformed) ``` And you can access diagnostic data for TED like this: ```python prediction = policy.predict_action_probabilities( GREET_RULE, domain, RegexInterpreter() ) print(f"{prediction.diagnostic_data.get('attention_weights')}") ``` ## Configuring Tensorflow TensorFlow allows configuring options in the runtime environment via [TF Config submodule](https://www.tensorflow.org/api_docs/python/tf/config). Rasa Open Source supports a smaller subset of these configuration options and makes appropriate calls to the `tf.config` submodule. This smaller subset comprises of configurations that developers frequently use with Rasa Open Source. All configuration options are specified using environment variables as shown in subsequent sections.