RASA NLU training with custom entities in depth - Rasa Open Source - Rasa Community Forum

RASA NLU training with custom entities in depth

post by sharath04929 on Sep 19, 2019

I am using RASA NLU for intent identification and entities are custom and having few doubts on training pattern. Please find the details below.

  1. If your question consists of 6 words, suppose if we have mapped all words to a single custom entity or the majority of words are mapped to an entity, will NLU consider even entity words to identify intents, or will it ignore them if we mapped them as an entity?
  2. And also generally, NLU prefers only some kind of lengthy statements or can we add like below example.

RAM Failure as an Intent and the statement is very directly RAM down. Do you think, is there anything wrong here or it’s absolutely fine? Is it compulsory that we need to make the statement lengthy like My system is not working and may be RAM problem etc.? 3. Is it correct that if we have a good number of examples, we can use n Number of intents and do not need to limit? Can we extend like 10 or 100 intents also?

  1. How to train the custom entities, example if we have mapped a few words as entities and in the future some nearby words might not be recognized, how do we resolve them smartly? If we do everything as a dataset, it would be like one-to-one pattern, where is the learning scope in NLU?

Thanks and let me know if you need any more information. Also, I’m not sure whether any questions are irrelevant, but these came with curiosity.

Regards, Sharath
Email: id:sharath0492@gmail.com

post by erohmensing on Sep 19, 2019

  1. Intent classification and entity recognition are completely separate – all words including entity words are used in intent classification!

  2. It doesn’t prefer short or lengthy statements – you should include both if you think it’s probable your users could use either one.

  3. Yes, if you have a good number of examples, you can increase your intents – but the goal shouldn’t be to have a bunch of intents. Usually, if you have lots of intents, this signals that some of your intents should be merged, and entities used to really define the difference in follow-up action after input.

  4. Not sure what you mean by this; I think the machine learning model will take care of that for you! For example, if you are picking up names, you don’t need to include every possible name as an example for your bot to realize that the blank in "my name is ___" to be classified as a name.

post by sharath04929 on Oct 1, 2019

Thanks for the detailed answers; a few are clear, and I have some more doubts on the above answers.

  1. We are facing one problem: Entities Are Not Generalizing. I assume this is because of a lack of training data for entities and found below point: Provide enough examples (> 20) per entity so that the conditional random field can generalize and pick up the data.
    Does this mean we should train more than 20 ways of each entity?

  2. What are the tools used in the kitchen?
    Example: Tools is one custom entity, and do we need to train them differently like automation process, fastest way and auto tools, etc., representing the tools entity?

  3. And also, if possible, can you share with me how custom entities are recognized in detail, like article or tutorial or any code snippet?

post by erohmensing on Oct 1, 2019

Yes, for any custom entity, we should train more than 20 examples.
Yes, if there is enough training data, though names are notably a hard task for entity extraction because there are so many that don’t follow a particular pattern.

post by sharath04929 on Oct 1, 2019

Coming to Intents design, as you mentioned, we can increase if we have a good number of examples, but the goal shouldn’t be a bunch of intents. In ideal cases, how many intents will be better?
From NLU point of view, we need to include in the pipeline like “batch_strategy: sequence” or “balanced batching”?

post by erohmensing on Oct 1, 2019

The number of intents depends entirely on the domain, but if you find two intents are confused often, they likely should be merged. Balanced batching (default on the latest Rasa) is a good solution for unequal intents. It will require more memory to train; if your dataset is relatively small, this shouldn’t be an issue.

post by sharath04929 on Oct 1, 2019

I am very much interested to see the entity extraction mechanism. Can you share some forum links or tutorials or code snippets for entity extraction and how exactly it happens in Rasa?

tip for improving confidence score

New intents are very different, and I observed only one difference: count-wise, new intents have more statements because they are generic, while old intents are domain-specific.