You are viewing documentation for our open source project which is maintained by the community. If you want to get started building assistants with Rasa please check out our latest [documentation here](/content/docs/index.html).

You can customize the policies your assistant uses by specifying the `policies` key in your project's `config.yml`. There are different policies to choose from, and you can include multiple policies in a single configuration. Here's an example of what a list of policies might look like:

```yaml
policies:
  - name: MemoizationPolicy
  - name: TEDPolicy
    max_history: 5
    epochs: 200
  - name: RulePolicy
```

### Starting from scratch?
If you don't know which policies to choose, leave out the `policies` key from your `config.yml` completely. If you do, the [Suggested Config](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/model-configuration#suggested-config) feature will provide default policies for you.

## Action Selection
At every turn, each policy defined in your configuration will predict a next action with a certain confidence level. For more information about how each policy makes its decision, read into the policy's description below. The policy that predicts with the highest confidence decides the assistant's next action.

### Maximum number of predictions
By default, your assistant can predict a maximum of 10 next actions after each user message. To update this value, you can set the environment variable `MAX_NUMBER_OF_PREDICTIONS` to the desired number of maximum predictions.

### Policy Priority
In the case that two policies predict with equal confidence (for example, the Memoization and Rule Policies might both predict with confidence 1), the priority of the policies is considered. Rasa Open Source policies have default priorities that are set to ensure the expected outcome in the case of a tie. They look like this, where higher numbers have higher priority:

- 6 - `RulePolicy`
- 3 - `MemoizationPolicy` or `AugmentedMemoizationPolicy`
- 2 - `UnexpecTEDIntentPolicy`
- 1 - `TEDPolicy`

In general, it is not recommended to have more than one policy per priority level in your configuration. If you have 2 policies with the same priority and they predict with the same confidence, the resulting action will be chosen randomly.

If you create your own policy, use these priorities as a guide for figuring out the priority of your policy. If your policy is a machine learning policy, it should most likely have priority 1, the same as the `TEDPolicy`.

### Overriding policy priorities
All policy priorities are configurable via the `priority` parameter in the policy's configuration, but we **do not recommend** changing them outside of specific cases such as custom policies. Doing so can lead to unexpected and undesired bot behavior.

## Machine Learning Policies
### TED Policy
The Transformer Embedding Dialogue (TED) Policy is a multi-task architecture for next action prediction and entity recognition. The architecture consists of several transformer encoders that are shared for both tasks. A sequence of entity labels is predicted through a Conditional Random Field (CRF) tagging layer on top of the user sequence transformer encoder output corresponding to the input sequence of tokens. For the next action prediction, the dialogue transformer encoder output and the system action labels are embedded into a single semantic vector space. We use the dot-product loss to maximize the similarity with the target label and minimize similarities with negative samples.

If you want to learn more about the model, check out [our paper](https://arxiv.org/abs/1910.00486) and on our [youtube channel](https://www.youtube.com/watch?v=j90NvurJI4I&list=PL75e0qA87dlG-za8eLI6t0_Pbxafk-cxb&index=14&ab_channel=Rasa) where we explain the model architecture in detail.

TED Policy architecture comprises the following steps:

1. Concatenate features for:
   - user input (user intent and entities) or user text processed through a user sequence transformer encoder,
   - previous system actions or bot utterances processed through a bot sequence transformer encoder,
   - slots and active forms
   for each time step into an input vector to the embedding layer that precedes the dialogue transformer.
2. Feed the embedding of the input vector into the dialogue transformer encoder.
3. Apply a dense layer to the output of the dialogue transformer to get embeddings of the dialogue for each time step.
4. Apply a dense layer to create embeddings for system actions for each time step.
5. Calculate the similarity between the dialogue embedding and embedded system actions. This step is based on the [StarSpace](https://arxiv.org/abs/1709.03856) idea.
6. Concatenate the token-level output of the user sequence transformer encoder with the output of the dialogue transformer encoder for each time step.
7. Apply CRF algorithm to predict contextual entities for each user text input.

### Configuration:
You can pass configuration parameters to the `TEDPolicy` using the `config.yml` file. If you want to fine-tune your model, start by modifying the following parameters:
- `epochs`:
  This parameter sets the number of times the algorithm will see the training data (default: `1`). One `epoch` is equals to one forward pass and one backward pass of all the training examples. Sometimes the model needs more epochs to properly learn.
  Sometimes more epochs don't influence the performance. The lower the number of epochs the faster the model is trained.

- `max_history`:
  This parameter controls how much dialogue history the model looks at to decide which action to take next. Default `max_history` for this policy is `None`, which means that the complete dialogue history since session restart is taken into account. If you want to limit the model to only see a certain number of previous dialogue turns, you can set `max_history` to a finite value. Please note that you should pick `max_history` carefully, so that the model has enough previous dialogue turns to create a correct prediction. See [Featurizers](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/policies/#featurizers) for more details.

Here is how the config would look like:
```yaml
policies:
  - name: TEDPolicy
    epochs: 200
    max_history: 8
```

- `number_of_transformer_layers`:
  This parameter sets the number of sequence transformer encoder layers to use for sequential transformer encoders for user, action and action label texts and for dialogue transformer encoder.
  (defaults: `text: 1, action_text: 1, label_action_text: 1, dialogue: 1`). The number of sequence transformer encoder layers corresponds to the transformer blocks to use for the model.

- `transformer_size`:
  This parameter sets the number of units in the sequence transformer encoder layers to use for sequential transformer encoders for user, action and action label texts and for dialogue transformer encoder.
  (defaults: `text: 128, action_text: 128, label_action_text: 128, dialogue: 128`). The vectors coming out of the transformer encoders will have the given `transformer_size`.

- `weight_sparsity`:
  This parameter defines the fraction of kernel weights that are set to 0 for all feed forward layers in the model (default: `0.8`). The value should be a number between 0 and 1. If you set `weight_sparsity` to 0, no kernel weights will be set to 0, the layer acts as a standard feed forward layer. You should not set `weight_sparsity` to 1 as this would result in all kernel weights being 0, i.e. the model is not able to learn.

- `split_entities_by_comma`:
  This parameter defines whether adjacent entities separated by a comma should be treated as one, or split. For example, entities with the type `ingredients`, like "apple, banana" can be split into "apple" and "banana". An entity with type `address`, like "Schönhauser Allee 175, 10119 Berlin" should be treated as one.
  Can either be `True`/`False` globally:

```yaml
  policies:
    - name: TEDPolicy
      split_entities_by_comma: True
  ```
  or set per entity type, such as:

```yaml
  policies:
    - name: TEDPolicy
      split_entities_by_comma:
        address: False
        ingredients: True
  ```

- `constrain_similarities`:
  This parameter when set to `True` applies a sigmoid cross entropy loss over all similarity terms.
  This helps in keeping similarities between input and negative labels to smaller values. This should help in better generalization of the model to real world test sets.

- `model_confidence`:
  This parameter allows the user to configure how confidences are computed during inference. It can take two values:
  - `softmax`: Confidences are in the range `[0, 1]` (old behavior and current default). Computed similarities are normalized with the `softmax` activation function.
  - `linear_norm`: Confidences are in the range `[0, 1]`. Computed dot product similarities are normalized with a linear function.

- `use_gpu`:
  This parameter defines whether a GPU (if available) will be used training. By default, `TEDPolicy` will be trained on GPU if a GPU is available (i.e. `use_gpu` is `True`). To enforce that `TEDPolicy` uses only the CPU for training, set `use_gpu` to `False`.

The above configuration parameters are the ones you should configure to fit your model to your data. However, additional parameters exist that can be adapted.

### UnexpecTED Intent Policy
This feature is experimental. We introduce experimental features to get feedback from our community, so we encourage you to try it out! However, the functionality might be changed or removed in the future.

`UnexpecTEDIntentPolicy` helps you review conversations and also allows your bot to react to unlikely user turns. It is an auxiliary policy that should only be used in conjunction with at least one other policy, as the only action that it can trigger is the special [`action_unlikely_intent`](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/default-actions#action_unlikely_intent) action.

`UnexpecTEDIntentPolicy` has the same model architecture as [`TEDPolicy`](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/policies#ted-policy). The difference is at a task level. Instead of learning the best action to be triggered next, `UnexpecTEDIntentPolicy` learns the set of intents that are most likely to be expressed by the user given the conversation context from training stories. It uses the learned information at inference time by checking if the predicted intent by NLU is the most likely intent. If the intent predicted by NLU is indeed likely to occur given the conversation context, `UnexpecTEDIntentPolicy` does not trigger any action. Otherwise, it triggers an [`action_unlikely_intent`](https://legacy-docs-oss.rasa.com/docs/rasa/2.x/default-actions#action_unlikely_intent) with a confidence of `1.00`.

### Configuration:
You can pass configuration parameters to the `UnexpecTEDIntentPolicy` using the `config.yml` file.
If you want to fine-tune model's performance, start by modifying the following parameters:
- `epochs`:
  This parameter sets the number of times the algorithm will see the training data (default: `1`). One `epoch` is equals to one forward pass and one backward pass of all the training examples. 
  Sometimes the model needs more epochs to learn properly. Sometimes more epochs don't influence the performance. The lower the number of epochs the faster the model is trained.

- `max_history`:
  This parameter controls how much dialogue history the model looks at before making an inference.
  Default `max_history` for this policy is `None`, which means that the complete dialogue history since session (re)start is taken into account. 
  If you want to limit the model to only see a certain number of previous dialogue turns, you can set `max_history` to a finite value.
  Please note that you should pick `max_history` carefully, so that the model has enough previous dialogue turns to create a correct prediction.

Here is how the config would look like:
```yaml
policies:
  - name: UnexpecTEDIntentPolicy
    max_history: 8
    epochs: 200
```

- `ignore_intents_list`:
  This parameter lets you configure `UnexpecTEDIntentPolicy` to not predict `action_unlikely_intent` for a subset of intents.
  You might want to do this if you come across a certain list of intents for which there are too many false warnings generated.

- `tolerance`:
  The `tolerance` parameter is a number that ranges from `0.0` to `1.0` (inclusive).
  It helps to adjust the threshold score used during prediction of `action_unlikely_intent` at inference time.
  As you increase the value of `tolerance`, the number of false warnings should decrease.

The above configuration parameters are the ones you should try tweaking according to your use case and training data. However, additional parameters exist that you could adapt.

### Memoization Policy
The `MemoizationPolicy` remembers the stories from your training data. It checks if the current conversation matches the stories in your `stories.yml` file. If so, it will predict the next action from the matching stories of your training data with a confidence of `1.0`. If no matching conversation is found, the policy predicts `None` with confidence `0.0`.

When looking for a match in your training data, the policy will take the last `max_history` number of turns of the conversation into account.

### Configuring Policies
One important hyperparameter for Rasa Open Source policies is the `max_history`. This controls how much dialogue history the model looks at to decide which action to take next.

Here is how the config would look like:
```yaml
policies:
  - name: TEDPolicy
    max_history: 5
    epochs: 200
    batch_size: 50
    max_training_samples: 300
```

### Data Augmentation
When you train a model, Rasa Open Source will create longer stories by randomly combining the ones in your stories files. The `--augmentation` flag allows you to set the `augmentation_factor`. The `augmentation_factor` determines how many augmented stories are subsampled during training.