👋 Introducing the Rasa Playground

Fallback issues and policy advice

1.5k views

post by travelmail26 on Jun 28, 2021

I recently started having an issue where the default fallback intent will be triggered and the bot just repeats the default utterance, even when I use statements that are clearly in the NLU. The fallback seems to be more easily triggered, too.

I’ve pasted my Config.yml below. I’ve read the fallback policy documents and options for different AugmentedMemoizationPolicy, but I’m still pretty confused.

My bot should be designed for long conversations and go between different story types. The most common intent for my bot is inform.

I’m open to using a two-stage fallback policy, but confess that I don’t understand how this could help my particular situation very well.

# Configuration for Rasa NLU.
# https://rasa.com/docs/rasa/nlu/components/
language: en
pipeline:
   - name: WhitespaceTokenizer
   - name: RegexFeaturizer
   - name: LexicalSyntacticFeaturizer
   - name: CountVectorsFeaturizer
   - name: CountVectorsFeaturizer
     analyzer: char_wb
     min_ngram: 1
     max_ngram: 4
   - name: DIETClassifier
     epochs: 10
     constrain_similarities: true
   - name: EntitySynonymMapper
   - name: ResponseSelector
     epochs: 10
     constrain_similarities: true
   - name: FallbackClassifier
     threshold: 0.05
     ambiguity_threshold: 0.05
policies:
   - name: MemoizationPolicy
   - name: TEDPolicy
     max_history: 5
     epochs: 10
     constrain_similarities: true
   - name: RulePolicy
     core_fallback_threshold: 0.05
     core_fallback_action_name: "action_default_fallback"
     enable_fallback_prediction: True

post by nik202 on Jul 2, 2021

@travelmail26 Heya! Can you tell why threshold is 0.05?

Question: Above policy is working fine with the bot/user conversation? But sometime when you update the code, train it. and run it; it start triggering you default fallback? Right.

@travelmail26 ? Yes or No?

post by travelmail26 on Jul 2, 2021

I wanted the fallback to have a low threshold and figured maybe this would fix the issue, but maybe I’m wrong.

post by nik202 on Jul 2, 2021

@travelmail26 This is fine for your use:

# Configuration for Rasa NLU.
# https://rasa.com/docs/rasa/nlu/components/
language: en
pipeline:
  - name: WhitespaceTokenizer
  - name: RegexFeaturizer
  - name: LexicalSyntacticFeaturizer
  - name: CountVectorsFeaturizer
  - name: CountVectorsFeaturizer
    analyzer: "char_wb"
    min_ngram: 1
    max_ngram: 4
  - name: DIETClassifier
    epochs: 100
    ranking_length: 5
    constrain_similarities: true
    model_confidence: linear_norm
  - name: EntitySynonymMapper
  - name: ResponseSelector
    epochs: 100
    constrain_similarities: true
    model_confidence: linear_norm

# Configuration for Rasa Core.
# https://rasa.com/docs/rasa/core/policies/
policies:
  - name: MemoizationPolicy
  - name: TEDPolicy
    max_history: 5
    epochs: 100
  - name: RulePolicy
    nlu_threshold: 0.4
    core_threshold: 0.4
    fallback_action_name: "action_default_fallback"

@travelmail26 good luck with your error.