What is the best method for entity extraction for names? - Rasa Open Source - Rasa Community Forum
What is the best method for entity extraction for names?
post by adrianhumphrey111 on Jan 6, 2019
I would like to get the name from a user. For example, “Hello my name is Adrian.” or just “Adrian”. What is the best way to extract this entity?
For example, should I just add a lot more training examples, or is there a custom pipeline that helps extract names?
post by srikar_1996 on Jan 7, 2019
Spacy has PERSON which is a built-in name entity. It works well for common names.
post by Akshit on Jan 7, 2019
What if I have some unusual names then? I made a .txt file and added lot of names in that and linked a lookup table in nlu data. Seemed to be working but is this a good way of doing it? I also want to add respective Id with the names. What kind of format can we add in the lookup table file?
Can we link other types of files in the lookup table like excel or maybe something else?
post by srikar_1996 on Jan 7, 2019
Yes, lookup table is another option. It works well if you have unusual names. Names are difficult to extract in general so a lookup table is a good way to do that.
Look up tables can only be in form of list in the code or new line separated .txt files. I don’t think adding the ID here would work.
post by adrianhumphrey111 on Jan 7, 2019
Could I have a quick example of using the PERSON entity with spacy?
post by srikar_1996 on Jan 7, 2019
Your nlu.md should contain the PERSON as the entity and you should also have spacy in your config file. For example
intent:greet_user
Hello my name is [Adrian](PERSON)I am [John](PERSON)Hi, I am [Arya](PERSON)
post by adrianhumphrey111 on Jan 7, 2019
AWESOME Thank you!
post by Akshit on Jan 7, 2019
So I have to use an associated ID for the name as when a user says “How is John doing” I need to call multiple APIs to get information on John.
So when NLU detects John as a named entity then I need to pass a user_id to my database server where I get a JSON response of data on John! So how can I map an ID to a name? Do you have any idea for this?
post by srikar_1996 on Jan 7, 2019
Maybe you could store the names and corresponding IDs in a database and once you extract the name, use it to get the ID and in-turn use that for your API calls?
post by Akshit on Jan 7, 2019
Okay like send the name as a parameter and get back the ID in another API?
post by srikar_1996 on Jan 7, 2019
Yes, you can do that.
post by lgrinberg on Feb 6, 2019
Best to use the following (worked for me):
- Use a lookup table
- Use the approach described here Providing conversation context to the NLU using microservices
For our NLU model for the name microservice we used NER_Spacy and NER_CRF and the combination worked for all the cases with pretty basic NLU training data.
post by luofanghao on Jun 18, 2019
I also believe that an “input box” widget (like you input your name on web pages) can help.
post by prasanth55555 on Aug 14, 2019
@srikar_1996 can you share the example of how to use the pipeline example including PERSON tag?
post by sameertikoo on Aug 14, 2019
In case the name of the person is not in the NLU and we still want to process it, what would be the best method to get that?
post by srikar_1996 on Aug 19, 2019
language: “en” pipeline:
- name: “nlp_spacy”
- name: “tokenizer_spacy”
- name: “intent_featurizer_spacy”
- name: “ner_spacy”
- name: “intent_classifier_sklearn”
Add these to your pipeline, it should work.
post by huseyinyilmaz01 on Sep 27, 2020
Do we have to also define targeted entities that Spacy is able to extract as a regular entity? E.g.
- Please open a ticket for [James](PERSON) about [printer](device)
Or, should we leave it to Spacy as below?
- Please open a ticket for James about [printer](device)
post by dhruv_bhimani on Dec 25, 2022
You don’t have to provide tagged NLU data for entities that are being extracted through SpaCy.