Failing Gracefully with the TwoStageFallbackPolicy

post by Tobias_Wochinger on Jan 24, 2019

Check out our blog post on the TwoStageFallbackPolicy which came with Rasa Core 0.13. It provides an intuitive and user-friendly conversation flow in case of NLU messages with low classification confidence. Have you tried it already? Share your experiences with us in the thread below

post by huberrom on Feb 6, 2019

Hey @Tobias_Wochinger, we are going to replace the new fallback policy in our project, and looking at the code it seems that the default configuration is only saying something like “Do you mean [name_intent]”, but in your blog, you are proposing a choice between (I think) the two highest confident intent.

Could you share the code for this custom action ?

I would like to personalise it with something like propose two intent if the difference in confidence between then is lower than 10, or only propose the highest confident one.

But it’s a really nice feature, everyone was looking forward to have it in our bot

post by Tobias_Wochinger on Feb 7, 2019

@huberrom Great to your that you happy with the new policy

This is our current implementation which we are using in our bot on the documentation: rasa-demo/actions.py at 096568e64df4c25902f91fcf7b409fa2fcd05b16 · RasaHQ/rasa-demo · GitHub .

We are currently also working on an improved version of it, which is currently a pr include entities in button titles by wochinge · Pull Request #277 · RasaHQ/rasa-demo · GitHub (might be merged till you read this ).

The basic idea of the implementation is that we read the button titles from a csv file. We saw that it is important to add potentially recognised entities to the payload of the button and to the button message title. We first put the entities in brackets after the intent description, but now we want to actually use different button titles depending on the intent and the extracted entities.

Let me know if you have any more questions

post by Silentvoice976 on Mar 23, 2019

Hey i am having an issue when i tried to use the csv file to read the meaning of intent just like in the demo bot…it shows up like did you mean user_ask_bot_name? It shows intent name instead of the meaning in csv file why is this?

post by huberrom on Mar 26, 2019

Can you share your csv file to be sure it has the right format ?

It seems to work for me

post by huberrom on Mar 26, 2019

Hey @Tobias_Wochinger, finally have the time to try the policy, look really great so far but I have a small problem :

I override both action_default_ask_rephrase and action_default_ask_affirmation because I want to change the text from english to french, so it shouldn’t change the general behaviour, but I have the following problem :

(The second picture if following the first one, after asking to rephrase, I’m just saying “Vitale” again, and the end of the second picture is the bot asking to rephrase again)

I’m mind, I thought that after the second “Something else”, the bot would answer “Sorry I don’t provide this skill”, but it’s only running “action_default_ask_rephrase”

Any idea on this ?

post by Silentvoice976 on Mar 26, 2019

@huberrom hey thanks for the reply, i used the same format and code in the demo bot and its not executing the right intent when i press the button i have raised the issue on the github page #3133. intent_description_mapping.csv (4.1 KB)

post by huberrom on Mar 27, 2019

Yes sounds good to me.

Don’t you have a warning saying that it can’t find the file ?

Maybe it’ll help, I override both action_default_ask_rephrase and action_default_ask_affirmation with the code I found here : rasa-demo/actions.py at 096568e64df4c25902f91fcf7b409fa2fcd05b16 · RasaHQ/rasa-demo · GitHub and here rasa_core/action.py at 431d4c89c5025bec2be51c065add2ab272089bdf · RasaHQ/rasa_core · GitHub .

Add some logs to see if the csv is correctly read. Don’t forget to add both of them in your domain.yml so that they are called instead of the default one

post by Silentvoice976 on Apr 1, 2019

When i addded action_default_ask_affirmation to the domain.yml i start getting this error

Failed to run custom action ‘action_default_ask_affirmation’. Action server responded with a non 200 status code of 500. Make sure your action server properly runs actions and returns a 200 once the action is executed. Error: 500 Server Error: INTERNAL SERVER ERROR for url: http://localhost:5055/webhook

post by huberrom on Apr 1, 2019

Well, that means that either your action server is not started (or correctly set up) or you didn’t define the action_default_ask_affirmation

post by mohamedHassanKa on Apr 1, 2019

class ActionDefaultAskAffirmation(Action):
    def name(self):
        return "action_default_ask_affirmation"
    def __init__(self):
        import pandas as pd
        self.intent_mappings = {}
        # read the mapping from a csv and store it in a dictionary
        with open('data\intent_description_mapping.csv', newline='', encoding='utf-8') as file:
            csv_reader = csv.reader(file)
            for row in csv_reader:
                self.intent_mappings[row[0]] = row[1]
    # the rest of the code

post by huberrom on Apr 1, 2019

Do you have the logs of the action server when the action is called ?

post by mohamedHassanKa on Apr 1, 2019

2019-04-01 10:36:30 DEBUG    rasa_core.processor  - Received user message 'how can we ge tthis all' with intent '{'name': 'ask_whatspossible', 'confidence': 0.2748849016662266}' and entities '[]'
2019-04-01 10:36:30 DEBUG    rasa_core.processor  - Logged UserUtterance - tracker now has 32 events
2019-04-01 10:36:30 DEBUG    rasa_core.processor  - Current slot values:
        age: None
        category: None
        month: None
        nationality: None
        requested_slot: nationality
        ssc: None
        ssc_level: None
        year: None
2019-04-01 10:36:30 DEBUG    rasa_core.policies.memoization  - Current tracker state [{'intent_inform_about_internship_nationality': 1.0, 'prev_utter_start_scholarship': 1.0, 'active_form_scholarship_form': 1.0},
{'prev_scholarship_form': 1.0, 'intent_inform_about_internship_nationality': 1.0, 'active_form_scholarship_form': 1.0}, {'prev_action_listen': 1.0, 'intent_inform_about_internship_nationality': 1.0, 'active_form_scholarship_form': 1.0}, {'intent_inform_about_internship_nationality': 1.0, 'active_form_scholarship_form': 1.0, 'prev_utter_noworries': 1.0}]

post by mohamedHassanKa on Apr 1, 2019

In case the affirmations by the user didn’t help to resolve the ambiguous intent, it will show buttons to the user. We are asking if they want to restart the conversation and if they have feedback for the bot (thumbs up, thumps down buttons).

post by Tobias_Wochinger on Jan 22, 2020

Yes, it’s exactly doing that. It maps the plain intent names (e.g. inform, greet) to nicer prompts (e.g. say hello). Does that make sense?