Entity Extraction

Entity extraction involves parsing user messages for required pieces of information. Rasa Open Source provides entity extractors for custom entities as well as pre-trained ones like dates and locations. Here is a summary of the available extractors and what they are used for:

Component	Requires	Model	Notes
`CRFEntityExtractor`	sklearn-crfsuite	conditional random field	good for training custom entities
`SpacyEntityExtractor`	spaCy	averaged perceptron	provides pre-trained entities
`DucklingHTTPExtractor`	running duckling	context-free grammar	provides pre-trained entities
`MitieEntityExtractor`	MITIE	structured SVM	good for training custom entities
`EntitySynonymMapper`	existing entities	N/A	maps known synonyms
`DIETClassifier`		conditional random field on top of a transformer	good for training custom entities

The “entity” Object

After parsing, an entity is returned as a dictionary. There are two fields that show information about how the pipeline impacted the entities returned: the extractor field of an entity tells you which entity extractor found this particular entity, and the processors field contains the name of components that altered this specific entity.

The use of synonyms can cause the value field not match the text exactly. Instead it will return the trained synonym.

{
  "text": "show me chinese restaurants",
  "intent": "restaurant_search",
  "entities": [
    {
      "start": 8,
      "end": 15,
      "value": "chinese",
      "entity": "cuisine",
      "extractor": "CRFEntityExtractor",
      "confidence": 0.854,
      "processors": []
    }
  ]
}

Note: The confidence will be set by the CRFEntityExtractor component. The DucklingHTTPExtractor will always return 1. The SpacyEntityExtractor extractor and DIETClassifier do not provide this information and return null.

Some extractors, like duckling, may include additional information. For example:

{
  "additional_info":{
    "grain":"day",
    "type":"value",
    "value":"2018-06-21T00:00:00.000-07:00",
    "values":[
      {
        "grain":"day",
        "type":"value",
        "value":"2018-06-21T00:00:00.000-07:00"
      }
    ]
  },
  "confidence":1.0,
  "end":5,
  "entity":"time",
  "extractor":"DucklingHTTPExtractor",
  "start":0,
  "text":"today",
  "value":"2018-06-21T00:00:00.000-07:00"
}

Custom Entities

Almost every chatbot and voice app will have some custom entities. A restaurant assistant should understand chinese as a cuisine, but to a language-learning assistant it would mean something very different. The CRFEntityExtractor and the DIETClassifier component can learn custom entities in any language, given some training data. See Training Data Format for details on how to include entities in your training data.

Entities Roles and Groups

Warning: This feature is experimental. We introduce experimental features to get feedback from our community, so we encourage you to try it out! However, the functionality might be changed or removed in the future. If you have feedback (positive or negative) please share it with us on the forum.

Assigning custom entity labels to words allows you to define certain concepts in the data. For example, we can define what a city is:

I want to fly from [Berlin](city) to [San Francisco](city).

However, sometimes you want to specify entities even further. Let’s assume we want to build an assistant that should book a flight for us. The assistant needs to know which of the two cities in the example above is the departure city and which is the destination city. Berlin and San Francisco are still cities, but they play a different role in our example. To distinguish between the different roles, you can assign a role label in addition to the entity label.

- I want to fly from [Berlin]{