These docs are for version 1.x of Rasa Open Source. # User Guide - [Installation](https://legacy-docs-v1.rasa.com/1.10.25/user-guide/installation/) - [Tutorial: Rasa Basics](https://legacy-docs-v1.rasa.com/1.10.25/user-guide/rasa-tutorial/) - [Tutorial: Building Assistants](https://legacy-docs-v1.rasa.com/1.10.25/user-guide/building-assistants/) - [Command Line Interface](https://legacy-docs-v1.rasa.com/1.10.25/user-guide/command-line-interface/) - [Architecture](https://legacy-docs-v1.rasa.com/1.10.25/user-guide/architecture/) - [Messaging and Voice Channels](https://legacy-docs-v1.rasa.com/1.10.25/user-guide/messaging-and-voice-channels/) - [Testing Your Assistant](https://legacy-docs-v1.rasa.com/1.10.25/user-guide/testing-your-assistant/) - [Setting up CI/CD](https://legacy-docs-v1.rasa.com/1.10.25/user-guide/setting-up-ci-cd/) - [Validate Data](https://legacy-docs-v1.rasa.com/1.10.25/user-guide/validate-files/) - [Configuring the HTTP API](https://legacy-docs-v1.rasa.com/1.10.25/user-guide/configuring-http-api/) - [Deploying Your Rasa Assistant](https://legacy-docs-v1.rasa.com/1.10.25/user-guide/how-to-deploy/) - [Cloud Storage](https://legacy-docs-v1.rasa.com/1.10.25/user-guide/cloud-storage/) # NLU - [About](https://legacy-docs-v1.rasa.com/1.10.25/nlu/about/) - [Using NLU Only](https://legacy-docs-v1.rasa.com/1.10.25/nlu/using-nlu-only/) - [Training Data Format](https://legacy-docs-v1.rasa.com/1.10.25/nlu/training-data-format/) - [Language Support](https://legacy-docs-v1.rasa.com/1.10.25/nlu/language-support/) - [Choosing a Pipeline](https://legacy-docs-v1.rasa.com/1.10.25/nlu/choosing-a-pipeline/) - [Components](https://legacy-docs-v1.rasa.com/1.10.25/nlu/components/) - [Entity Extraction](https://legacy-docs-v1.rasa.com/1.10.25/nlu/entity-extraction/#) # Core - [About](https://legacy-docs-v1.rasa.com/1.10.25/core/about/) - [Stories](https://legacy-docs-v1.rasa.com/1.10.25/core/stories/) - [Domains](https://legacy-docs-v1.rasa.com/1.10.25/core/domains/) - [Responses](https://legacy-docs-v1.rasa.com/1.10.25/core/responses/) - [Actions](https://legacy-docs-v1.rasa.com/1.10.25/core/actions/) - [Reminders and External Events](https://legacy-docs-v1.rasa.com/1.10.25/core/reminders-and-external-events/) - [Policies](https://legacy-docs-v1.rasa.com/1.10.25/core/policies/) - [Slots](https://legacy-docs-v1.rasa.com/1.10.25/core/slots/) - [Forms](https://legacy-docs-v1.rasa.com/1.10.25/core/forms/) - [Retrieval Actions](https://legacy-docs-v1.rasa.com/1.10.25/core/retrieval-actions/) - [Interactive Learning](https://legacy-docs-v1.rasa.com/1.10.25/core/interactive-learning/) - [Fallback Actions](https://legacy-docs-v1.rasa.com/1.10.25/core/fallback-actions/) - [Knowledge Base Actions](https://legacy-docs-v1.rasa.com/1.10.25/core/knowledge-bases/) # Conversation Design - [Dialogue Elements](https://legacy-docs-v1.rasa.com/1.10.25/dialogue-elements/dialogue-elements/) - [Small Talk](https://legacy-docs-v1.rasa.com/1.10.25/dialogue-elements/small-talk/) - [Completing Tasks](https://legacy-docs-v1.rasa.com/1.10.25/dialogue-elements/completing-tasks/) - [Guiding Users](https://legacy-docs-v1.rasa.com/1.10.25/dialogue-elements/guiding-users/) # API Reference - [Action Server](https://legacy-docs-v1.rasa.com/1.10.25/api/action-server/) - [HTTP API](https://legacy-docs-v1.rasa.com/1.10.25/api/http-api/) - [Jupyter Notebooks](https://legacy-docs-v1.rasa.com/1.10.25/api/jupyter-notebooks/) - [Agent](https://legacy-docs-v1.rasa.com/1.10.25/api/agent/) - [Custom NLU Components](https://legacy-docs-v1.rasa.com/1.10.25/api/custom-nlu-components/) - [Rasa SDK](https://legacy-docs-v1.rasa.com/1.10.25/api/rasa-sdk/) - [Events](https://legacy-docs-v1.rasa.com/1.10.25/api/events/) - [Tracker](https://legacy-docs-v1.rasa.com/1.10.25/api/tracker/) - [Tracker Stores](https://legacy-docs-v1.rasa.com/1.10.25/api/tracker-stores/) - [Event Brokers](https://legacy-docs-v1.rasa.com/1.10.25/api/event-brokers/) - [Lock Stores](https://legacy-docs-v1.rasa.com/1.10.25/api/lock-stores/) - [Training Data Importers](https://legacy-docs-v1.rasa.com/1.10.25/api/training-data-importers/) - [Featurization of Conversations](https://legacy-docs-v1.rasa.com/1.10.25/api/core-featurization/) - [TensorFlow Configuration](https://legacy-docs-v1.rasa.com/1.10.25/api/tensorflow_usage/) - [Migration Guide](https://legacy-docs-v1.rasa.com/1.10.25/migration-guide/) - [Rasa Open Source Change Log](https://legacy-docs-v1.rasa.com/1.10.25/changelog/) # Migrate from (beta) - [Dialogflow](https://legacy-docs-v1.rasa.com/1.10.25/migrate-from/google-dialogflow-to-rasa/) - [Wit.ai](https://legacy-docs-v1.rasa.com/1.10.25/migrate-from/facebook-wit-ai-to-rasa/) - [LUIS](https://legacy-docs-v1.rasa.com/1.10.25/migrate-from/microsoft-luis-to-rasa/) - [IBM Watson](https://legacy-docs-v1.rasa.com/1.10.25/migrate-from/ibm-watson-to-rasa/) # Reference - [Glossary](https://legacy-docs-v1.rasa.com/1.10.25/glossary/) # Versions viewing: 1.10.25 # Entity Extraction Entity extraction involves parsing user messages for required pieces of information. Rasa Open Source provides entity extractors for custom entities as well as pre-trained ones like dates and locations. Here is a summary of the available extractors and what they are used for: | Component | Requires | Model | Notes | | --- | --- | --- | --- | | `CRFEntityExtractor` | sklearn-crfsuite | conditional random field | good for training custom entities | | `SpacyEntityExtractor` | spaCy | averaged perceptron | provides pre-trained entities | | `DucklingHTTPExtractor` | running duckling | context-free grammar | provides pre-trained entities | | `MitieEntityExtractor` | MITIE | structured SVM | good for training custom entities | | `EntitySynonymMapper` | existing entities | N/A | maps known synonyms | | `DIETClassifier` | | conditional random field
on top of a transformer | good for training custom entities | ## The “entity” Object After parsing, an entity is returned as a dictionary. There are two fields that show information about how the pipeline impacted the entities returned: the `extractor` field of an entity tells you which entity extractor found this particular entity, and the `processors` field contains the name of components that altered this specific entity. The use of synonyms can cause the `value` field not match the `text` exactly. Instead it will return the trained synonym. ```json { "text": "show me chinese restaurants", "intent": "restaurant_search", "entities": [ { "start": 8, "end": 15, "value": "chinese", "entity": "cuisine", "extractor": "CRFEntityExtractor", "confidence": 0.854, "processors": [] } ] } ``` Note the `confidence` will be set by the `CRFEntityExtractor` component. The `DucklingHTTPExtractor` will always return `1`. The `SpacyEntityExtractor` extractor and `DIETClassifier` do not provide this information and return `null`. Some extractors, like `duckling`, may include additional information. ```json { "additional_info":{ "grain":"day", "type":"value", "value":"2018-06-21T00:00:00.000-07:00", "values":[ { "grain":"day", "type":"value", "value":"2018-06-21T00:00:00.000-07:00" } ] }, "confidence":1.0, "end":5, "entity":"time", "extractor":"DucklingHTTPExtractor", "start":0, "text":"today", "value":"2018-06-21T00:00:00.000-07:00" } ``` ## Custom Entities Almost every chatbot and voice app will have some custom entities. A restaurant assistant should understand `chinese` as a cuisine, but to a language-learning assistant it would mean something very different. The `CRFEntityExtractor` and the `DIETClassifier` component can learn custom entities in any language, given some training data. See [Training Data Format](https://legacy-docs-v1.rasa.com/1.10.25/nlu/training-data-format/#training-data-format) for details on how to include entities in your training data. ## Entities Roles and Groups Assigning custom entity labels to words allows you to define certain concepts in the data. For example, we can define what a city is: ```plaintext I want to fly from [Berlin](city) to [San Francisco](city). ``` However, sometimes you want to specify entities even further. Let’s assume we want to build an assistant that should book a flight for us. The assistant needs to know which of the two cities in the example above is the departure city and which is the destination city. `Berlin` and `San Francisco` are still cities, but they play a different role in our example. To distinguish between the different roles, you can assign a role label in addition to the entity label. ```plaintext - I want to fly from [Berlin]{"entity": "city", "role": "departure"} to [San Francisco]{"entity": "city", "role": "destination"}. ``` You can also group different entities by specifying a group label next to the entity label. The group label can, for example, be used to define different orders. ```plaintext Give me a [small]{"entity": "size", "group": "1"} pizza with [mushrooms]{"entity": "topping", "group": "1"} and a [large]{"entity": "size", "group": "2"} [pepperoni]{"entity": "topping", "group": "2"} ``` See [Training Data Format](https://legacy-docs-v1.rasa.com/1.10.25/nlu/training-data-format/#training-data-format) for details on how to define entities with roles and groups in your training data. The entity object returned by the extractor will include the detected role/group label. ```json { "text": "Book a flight from Berlin to SF", "intent": "book_flight", "entities": [ { "start": 19, "end": 25, "value": "Berlin", "entity": "city", "role": "departure", "extractor": "DIETClassifier" }, { "start": 29, "end": 31, "value": "San Francisco", "entity": "city", "role": "destination", "extractor": "DIETClassifier" } ] } ``` ## Extracting Places, Dates, People, Organizations spaCy has excellent pre-trained named-entity recognizers for a few different languages. You can test them out in this [interactive demo](https://demos.explosion.ai/displacy-ent/). ## Dates, Amounts of Money, Durations, Distances, Ordinals The [duckling](https://duckling.wit.ai/) library does a great job of turning expressions like “next Thursday at 8pm” into actual datetime objects. For example: ```plaintext "next Thursday at 8pm" => {"value":"2018-05-31T20:00:00.000+01:00"} ``` ## Regular Expressions (regex) You can use regular expressions to help the CRF model learn to recognize entities. ## Passing Custom Features to `CRFEntityExtractor` If you want to pass custom features, such as pre-trained word embeddings, to `CRFEntityExtractor`, you can add any dense featurizer to the pipeline before the `CRFEntityExtractor`. `CRFEntityExtractor` automatically finds the additional dense features.