Policies

Configuring Policies

The rasa.core.policies.Policy class decides which action to take at every step in the conversation.

There are different policies to choose from, and you can include multiple policies in a single rasa.core.agent.Agent.

Note
Per default a maximum of 10 next actions can be predicted by the agent after every user message. To update this value you can set the environment variable MAX_NUMBER_OF_PREDICTIONS to the desired number of maximum predictions.

Your project’s config.yml file takes a policies key which you can use to customize the policies your assistant uses. In the example below, the last two lines show how to use a custom policy class and pass arguments to it.

policies:
  - name: "KerasPolicy"
    featurizer:
    - name: MaxHistoryTrackerFeaturizer
      max_history: 5
      state_featurizer:
        - name: BinarySingleStateFeaturizer
  - name: "MemoizationPolicy"
    max_history: 5
  - name: "FallbackPolicy"
    nlu_threshold: 0.4
    core_threshold: 0.3
    fallback_action_name: "my_fallback_action"
  - name: "path.to.your.policy.class"
    arg1: "..."

Max History

One important hyperparameter for Rasa Core policies is the max_history. This controls how much dialogue history the model looks at to decide which action to take next.

You can set the max_history by passing it to your policy’s Featurizer in the policy configuration yaml file.

Note
Only the MaxHistoryTrackerFeaturizer uses a max history, whereas the FullDialogueTrackerFeaturizer always looks at the full conversation history. See Featurization of Conversations for details.

Data Augmentation

When you train a model, by default Rasa Core will create longer stories by randomly gluing together the ones in your stories files. This is because if you have stories like:

# thanks
* thankyou
   - utter_youarewelcome

# bye
* goodbye
   - utter_goodbye

you actually want to teach your policy to ignore the dialogue history when it isn’t relevant and just respond with the same action no matter what happened before.

You can alter this behavior with the --augmentation flag. Which allows you to set the augmentation_factor. The augmentation_factor determines how many augmented stories are subsampled during training. The augmented stories are subsampled before training since their number can quickly become very large, and we want to limit it.

The number of sampled stories is augmentation_factor x10. By default augmentation is set to 20, resulting in a maximum of 200 augmented stories.

--augmentation 0 disables all augmentation behavior. The memoization based policies are not affected by augmentation (independent of the augmentation_factor) and will automatically ignore all augmented stories.

Action Selection

At every turn, each policy defined in your configuration will predict a next action with a certain confidence level. For more information about how each policy makes its decision, read into the policy’s description below. The bot’s next action is then decided by the policy that predicts with the highest confidence.

In the case that two policies predict with equal confidence (for example, the Memoization and Mapping Policies always predict with confidence of either 0 or 1), the priority of the policies is considered. Rasa policies have default priorities that are set to ensure the expected outcome in the case of a tie. They look like this, where higher numbers have higher priority:

5. FormPolicy
6. FallbackPolicy and TwoStageFallbackPolicy
7. MemoizationPolicy and AugmentedMemoizationPolicy
8. MappingPolicy
9. TEDPolicy, EmbeddingPolicy, KerasPolicy, and SklearnPolicy

This priority hierarchy ensures that, for example, if there is an intent with a mapped action, but the NLU confidence is not above the nlu_threshold, the bot will still fall back. In general, it is not recommended to have more than one policy per priority level, and some policies on the same priority level, such as the two fallback policies, strictly cannot be used in tandem.

Warning
All policy priorities are configurable via the priority: parameter in the configuration, but we do not recommend changing them outside of specific cases such as custom policies. Doing so can lead to unexpected and undesired bot behavior.

Keras Policy

The KerasPolicy uses a neural network implemented in Keras to select the next action. The default architecture is based on an LSTM, but you can override the KerasPolicy.model_architecture method to implement your own architecture.

def model_architecture(
    self, input_shape: Tuple[int, int], output_shape: Tuple[int, Optional[int]]
) -> tf.keras.models.Sequential:
    """Build a keras model and return a compiled model."""

from tensorflow.keras.models import Sequential
    from tensorflow.keras.layers import (
        Masking,
        LSTM,
        Dense,
        TimeDistributed,
        Activation,
    )

# Build Model
    model = Sequential()

# the shape of the y vector of the labels,
    # determines which output from rnn will be used
    # to calculate the loss
    if len(output_shape) == 1:
        # y is (num examples, num features) so
        # only the last output from the rnn is used to
        # calculate the loss
        model.add(Masking(mask_value=-1, input_shape=input_shape))
        model.add(LSTM(self.rnn_size, dropout=0.2))
        model.add(Dense(input_dim=self.rnn_size, units=output_shape[-1]))
    elif len(output_shape) == 2:
        # y is (num examples, max_dialogue_len, num features) so
        # all the outputs from the rnn are used to
        # calculate the loss, therefore a sequence is returned and
        # time distributed layer is used
        
        # the first value in input_shape is max dialogue_len,
        # it is set to None, to allow dynamic_rnn creation
        # during prediction
        model.add(Masking(mask_value=-1, input_shape=(None, input_shape[1])))
        model.add(LSTM(self.rnn_size, return_sequences=True, dropout=0.2))
        model.add(TimeDistributed(Dense(units=output_shape[-1])))
    else:
        raise ValueError(
            "Cannot construct the model because"
            "length of output_shape = {} "
            "should be 1 or 2."
            "".format(len(output_shape))
        )

model.add(Activation("softmax"))

model.compile(
        loss="categorical_crossentropy", optimizer="rmsprop", metrics=["accuracy"]
    )

if common_utils.obtain_verbosity() > 0:
        model.summary()

return model

and the training is run here:

def train(
    self,
    training_trackers: List[DialogueStateTracker],
    domain: Domain,
    **kwargs: Any,
) -> None:

np.random.seed(self.random_seed)
    tf.random.set_seed(self.random_seed)

training_data = self.featurize_for_training(training_trackers, domain, **kwargs)
    # noinspection PyPep8Naming
    shuffled_X, shuffled_y = training_data.shuffled_X_y()

if self.model is None:
        self.model = self.model_architecture(
            shuffled_X.shape[1:], shuffled_y.shape[1:]
        )

logger.debug(
        f"Fitting model with {training_data.num_examples()} total samples and a "
        f"validation split of {self.validation_split}."
    )

# filter out kwargs that cannot be passed to fit
    self._train_params = self._get_valid_params(
        self.model.fit, **self._train_params
    )

self.model.fit(
        shuffled_X,
        shuffled_y,
        epochs=self.epochs,
        batch_size=self.batch_size,
        shuffle=False,
        verbose=common_utils.obtain_verbosity(),
        **self._train_params,
    )
    self.current_epoch = self.epochs

logger.debug("Done fitting Keras Policy model.")

You can implement the model of your choice by overriding these methods, or initialize KerasPolicy with pre-defined keras model.

In order to get reproducible training results for the same inputs you can set the random_seed attribute of the KerasPolicy to any integer.

Mapping Policy

The MappingPolicy can be used to directly map intents to actions. The mappings are assigned by giving an intent the property triggers, e.g.:

intents:
 - ask_is_bot:
     triggers: action_is_bot

An intent can only be mapped to at most one action. The bot will run the mapped action once it receives a message of the triggering intent. Afterwards, it will listen for the next message. With the next user message, normal prediction will resume.

If you do not want your intent-action mapping to affect the dialogue history, the mapped action must return a UserUtteranceReverted() event. This will delete the user’s latest message, along with any events that happened after it, from the dialogue history. This means you should not include the intent-action interaction in your stories.

Note
If you use the MappingPolicy to predict bot utterance actions directly (e.g. triggers: utter_{}), these interactions must go in your stories, as in this case there is no UserUtteranceReverted() and the intent and the mapped response action will appear in the dialogue history.

Memoization Policy

The MemoizationPolicy just memorizes the conversations in your training data. It predicts the next action with confidence 1.0 if this exact conversation exists in the training data, otherwise it predicts None with confidence 0.0.

Fallback Policy

The FallbackPolicy invokes a fallback action if at least one of the following occurs:

1. The intent recognition has a confidence below nlu_threshold.
2. The highest ranked intent differs in confidence with the second highest ranked intent by less than ambiguity_threshold.
3. None of the dialogue policies predict an action with confidence higher than core_threshold.

Configuration:

policies:
  - name: "FallbackPolicy"
    nlu_threshold: 0.3
    ambiguity_threshold: 0.1
    core_threshold: 0.3
    fallback_action_name: 'action_default_fallback'

You can also configure the FallbackPolicy in your python code:

from rasa.core.policies.fallback import FallbackPolicy
from rasa.core.policies.keras_policy import KerasPolicy
from rasa.core.agent import Agent

fallback = FallbackPolicy(fallback_action_name="action_default_fallback",
                           core_threshold=0.3,
                           nlu_threshold=0.3,
                           ambiguity_threshold=0.1)

agent = Agent("domain.yml", policies=[KerasPolicy(), fallback])

Note
You can include either the FallbackPolicy or the TwoStageFallbackPolicy in your configuration, but not both.

Two-Stage Fallback Policy

The TwoStageFallbackPolicy handles low NLU confidence in multiple stages by trying to disambiguate the user input.

1. If an NLU prediction has a low confidence score or is not significantly higher than the second highest ranked prediction, the user is asked to affirm the classification of the intent.
2. If they affirm, the story continues as if the intent was classified with high confidence from the beginning.
3. If they deny, the user is asked to rephrase their message.
4. If the classification of the rephrased intent was confident, the story continues as if the user had this intent from the beginning.
5. If the rephrased intent was not classified with high confidence, the user is asked to affirm the classified intent.
6. If the user affirms the intent, the story continues as if the user had this intent from the beginning.
7. If the user denies, the original intent is classified as the specified deny_suggestion_intent_name, and an ultimate fallback action is triggered (e.g., a handoff to a human).

Configuration:

policies:
  - name: TwoStageFallbackPolicy
    nlu_threshold: 0.3
    ambiguity_threshold: 0.1
    core_threshold: 0.3
    fallback_core_action_name: "action_default_fallback"
    fallback_nlu_action_name: "action_default_fallback"
    deny_suggestion_intent_name: "out_of_scope"

Note
You can include either the FallbackPolicy or the TwoStageFallbackPolicy in your configuration, but not both.

Form Policy

The FormPolicy is an extension of the MemoizationPolicy which handles the filling of forms. Once a FormAction is called, the FormPolicy will continually predict the FormAction until all required slots in the form are filled.