After Fallback policy is started Stories stops working - Rasa Open Source - Rasa Community Forum
After Fallback policy is started Stories stops working
post by NikhilBansal21 on Jan 10, 2019
Hi,
I have written stories using the interactive learning tool. I am training my bot using 3 main policies i.e - Memorization, Keras and have a custom fallback policy with core threshold = 0.3 and nlu threshold = 0.2.
I have at least 40-50 stories which makes my bot work properly. I have been tracking user state when the bot goes into a dialogue conversation and also a custom action triggers every time a user skips the expected output.
i.e
Happy path
U - I want to order something
B - what do you want to order?
U - Good morning!
B - Good Morning
B - What do you want to order?
This is a dummy flow for explaining a little bit about my flow!
Now the problem is coming whenever a user goes into fallback.
Bot handles it like I wanted, but it does not follow the multi-step utterance or the story I wanted it to handle i.e
Not so happy path
U - I want to order something
B - what do you want to order?
U - I want you to drive the car (fallback)
B - Sorry I cannot order that!
On debugging the HTTP server, I am finding that the bot is switching from Memorization policy to Fallback policy, and then it goes to Keras policy and hence it stops following the story. Therefore I did not get the response structure I mentioned in the first story.
Secondly - on removing the Fallback policy this issue gets resolved but whenever I enter some gibberish input like -
Unhappy path
U - I want to order something
B - what do you want to order?
U - bshbehbgbbevbtb
Here the bot predicts a wrong intent and thus action as well!
How can I control the policies so that I can prevent the bot from hindering stories as well as use the fallback policy for gibberish content?
post by NikhilBansal21 on Jan 14, 2019
hey @Juste / @Ghostvv can you guys help me out with this please?
As I have gone through the fallback policy docs it states that as soon as fallback is encountered the bot utters fallback action and reverts back to the previous current state. But this is not happening in my case. The bot starts predicting with the fallback policy for all the utterances added by the user after the fallback event occurred. Sorry for bothering you guys with a tag but this problem is not getting solved by me and I really need to finish this flow.
post by Ghostvv on Jan 14, 2019
It looks like you have a custom fallback action action_fallback. Please read a note in the docs about fallback: Fallback Actions
post by NikhilBansal21 on Jan 14, 2019
I have read that and did that way also but the same issue is coming.
post by Ghostvv on Jan 14, 2019
Could you please put your fallback code here?
post by NikhilBansal21 on Jan 14, 2019
class ActionFallback(Action):
def name(self):
return 'action_fallback'
def run(self, dispatcher, tracker, domain):
response = '{intentName:Fallback1 , Response_user: Sorry! I was not able to find an answer for that}'
dispatcher.utter_message(response)
post by Ghostvv on Jan 14, 2019
I don’t see where, it returns UserUtteranceReverted.
post by NikhilBansal21 on Jan 14, 2019
But I am not using this action for fallback my fallback action is being used from the domain file.
post by Ghostvv on Jan 14, 2019
You set your fallback action to be an utter template action, which doesn’t rewind the previous user utterance.
post by NikhilBansal21 on Jan 14, 2019
Ok Ill follow this and will see if it works!
post by NikhilBansal21 on Jan 15, 2019
Is there any way to prevent that? CZ I tried with stories and it does not pick up stories at the time of fallback.
post by Ghostvv on Jan 15, 2019
If you don’t return UserUtteranceReverted, it will keep everything, but then you need to put the fallback action into the stories.
post by NikhilBansal21 on Jan 15, 2019
Is their any way to prevent that? CZ I tried with stories and it does not pick up stories at the time of fallback.
post by Ghostvv on Jan 15, 2019
If you use custom fallback without UserUtteranceReverted, from the logs above it seems to me the policy cannot recall what to do after utter_fallback which could be due to the fact that there is no similar training story, or because the intent was classified as None.
post by vidhya on Jul 3, 2019
Hi All, I created a Rasa init project and immediately added the configuration for fallback, so that any irrelevant texts (say I want a coffee) will go to fallback.
I added the utter_default as well. But after adding the fallback, all the utterances go for fallback action. For example, Hi hello etc., whereas before fallback it was going for an appropriate intent.
post by Ghostvv on Jul 4, 2019
Rasa core doesn’t see actual utterances, it sees the result of Rasa NLU classification, so it depends on intents you have for these phrases and your stories.