These docs are for version 1.x of Rasa Open Source. ## User Guide - [Installation](https://legacy-docs-v1.rasa.com/1.7.0/user-guide/installation/) - [Tutorial: Rasa Basics](https://legacy-docs-v1.rasa.com/1.7.0/user-guide/rasa-tutorial/) - [Tutorial: Building Assistants](https://legacy-docs-v1.rasa.com/1.7.0/user-guide/building-assistants/) - [Command Line Interface](https://legacy-docs-v1.rasa.com/1.7.0/user-guide/command-line-interface/) - [Architecture](https://legacy-docs-v1.rasa.com/1.7.0/user-guide/architecture/) - [Messaging and Voice Channels](https://legacy-docs-v1.rasa.com/1.7.0/user-guide/messaging-and-voice-channels/) - [Evaluating Models](https://legacy-docs-v1.rasa.com/1.7.0/user-guide/evaluating-models/) - [Validate Data](https://legacy-docs-v1.rasa.com/1.7.0/user-guide/validate-files/) - [Running the Server](https://legacy-docs-v1.rasa.com/1.7.0/user-guide/running-the-server/) - [Running Rasa with Docker](https://legacy-docs-v1.rasa.com/1.7.0/user-guide/running-rasa-with-docker/) - [Cloud Storage](https://legacy-docs-v1.rasa.com/1.7.0/user-guide/cloud-storage/) ## NLU - [About](https://legacy-docs-v1.rasa.com/1.7.0/nlu/about/) - [Using NLU Only](https://legacy-docs-v1.rasa.com/1.7.0/nlu/using-nlu-only/) - [Training Data Format](https://legacy-docs-v1.rasa.com/1.7.0/nlu/training-data-format/) - [Choosing a Pipeline](https://legacy-docs-v1.rasa.com/1.7.0/nlu/choosing-a-pipeline/) - [Language Support](https://legacy-docs-v1.rasa.com/1.7.0/nlu/language-support/) - [Entity Extraction](https://legacy-docs-v1.rasa.com/1.7.0/nlu/entity-extraction/#) - [Components](https://legacy-docs-v1.rasa.com/1.7.0/nlu/components/) ## Core - [About](https://legacy-docs-v1.rasa.com/1.7.0/core/about/) - [Stories](https://legacy-docs-v1.rasa.com/1.7.0/core/stories/) - [Domains](https://legacy-docs-v1.rasa.com/1.7.0/core/domains/) - [Responses](https://legacy-docs-v1.rasa.com/1.7.0/core/responses/) - [Actions](https://legacy-docs-v1.rasa.com/1.7.0/core/actions/) - [Policies](https://legacy-docs-v1.rasa.com/1.7.0/core/policies/) - [Slots](https://legacy-docs-v1.rasa.com/1.7.0/core/slots/) - [Forms](https://legacy-docs-v1.rasa.com/1.7.0/core/forms/) - [Retrieval Actions](https://legacy-docs-v1.rasa.com/1.7.0/core/retrieval-actions/) - [Interactive Learning](https://legacy-docs-v1.rasa.com/1.7.0/core/interactive-learning/) - [Fallback Actions](https://legacy-docs-v1.rasa.com/1.7.0/core/fallback-actions/) - [Knowledge Base Actions](https://legacy-docs-v1.rasa.com/1.7.0/core/knowledge-bases/) ## Conversation Design - [Dialogue Elements](https://legacy-docs-v1.rasa.com/1.7.0/dialogue-elements/dialogue-elements/) - [Small Talk](https://legacy-docs-v1.rasa.com/1.7.0/dialogue-elements/small-talk/) - [Completing Tasks](https://legacy-docs-v1.rasa.com/1.7.0/dialogue-elements/completing-tasks/) - [Guiding Users](https://legacy-docs-v1.rasa.com/1.7.0/dialogue-elements/guiding-users/) ## API Reference - [Action Server](https://legacy-docs-v1.rasa.com/1.7.0/api/action-server/) - [HTTP API](https://legacy-docs-v1.rasa.com/1.7.0/api/http-api/) - [Jupyter Notebooks](https://legacy-docs-v1.rasa.com/1.7.0/api/jupyter-notebooks/) - [Agent](https://legacy-docs-v1.rasa.com/1.7.0/api/agent/) - [Custom NLU Components](https://legacy-docs-v1.rasa.com/1.7.0/api/custom-nlu-components/) - [Rasa SDK](https://legacy-docs-v1.rasa.com/1.7.0/api/rasa-sdk/) - [Events](https://legacy-docs-v1.rasa.com/1.7.0/api/events/) - [Tracker](https://legacy-docs-v1.rasa.com/1.7.0/api/tracker/) - [Tracker Stores](https://legacy-docs-v1.rasa.com/1.7.0/api/tracker-stores/) - [Event Brokers](https://legacy-docs-v1.rasa.com/1.7.0/api/event-brokers/) - [Lock Stores](https://legacy-docs-v1.rasa.com/1.7.0/api/lock-stores/) - [Training Data Importers](https://legacy-docs-v1.rasa.com/1.7.0/api/training-data-importers/) - [Featurization of Conversations](https://legacy-docs-v1.rasa.com/1.7.0/api/core-featurization/) - [Migration Guide](https://legacy-docs-v1.rasa.com/1.7.0/migration-guide/) - [Rasa OSS Change Log](https://legacy-docs-v1.rasa.com/1.7.0/changelog/) ## Migrate from (beta) - [Dialogflow](https://legacy-docs-v1.rasa.com/1.7.0/migrate-from/google-dialogflow-to-rasa/) - [Wit.ai](https://legacy-docs-v1.rasa.com/1.7.0/migrate-from/facebook-wit-ai-to-rasa/) - [LUIS](https://legacy-docs-v1.rasa.com/1.7.0/migrate-from/microsoft-luis-to-rasa/) - [IBM Watson](https://legacy-docs-v1.rasa.com/1.7.0/migrate-from/ibm-watson-to-rasa/) ## Reference - [Glossary](https://legacy-docs-v1.rasa.com/1.7.0/glossary/) # Entity Extraction ## Introduction Here is a summary of the available extractors and what they are used for: | Component | Requires | Model | Notes | | --- | --- | --- | --- | | `CRFEntityExtractor` | sklearn-crfsuite | conditional random field | good for training custom entities | | `SpacyEntityExtractor` | spaCy | averaged perceptron | provides pre-trained entities | | `DucklingHTTPExtractor` | running duckling | context-free grammar | provides pre-trained entities | | `MitieEntityExtractor` | MITIE | structured SVM | good for training custom entities | | `EntitySynonymMapper` | existing entities | N/A | maps known synonyms | If your pipeline includes one or more of the components above, the output of your trained model will include the extracted entities as well as some metadata about which component extracted them. The `processors` field contains the names of components that altered each entity. Note: The `value` field can be different from what appears in the text. If you use synonyms, an extracted entity like `chinees` will be mapped to a standard value, e.g. `chinese`. Example response: ```json { "text": "show me chinese restaurants", "intent": "restaurant_search", "entities": [ { "start": 8, "end": 15, "value": "chinese", "entity": "cuisine", "extractor": "CRFEntityExtractor", "confidence": 0.854, "processors": [] } ] } ``` Some extractors, like `duckling`, may include additional information. For example: ```json { "additional_info":{ "grain":"day", "type":"value", "value":"2018-06-21T00:00:00.000-07:00", "values":[ { "grain":"day", "type":"value", "value":"2018-06-21T00:00:00.000-07:00" } ] }, "confidence":1.0, "end":5, "entity":"time", "extractor":"DucklingHTTPExtractor", "start":0, "text":"today", "value":"2018-06-21T00:00:00.000-07:00" } ``` Note: The confidence will be set by the CRF entity extractor. The duckling entity extractor will always return 1. The SpacyEntityExtractor extractor does not provide this information and returns null. ## Custom Entities Almost every chatbot and voice app will have some custom entities. A restaurant assistant should understand `chinese` as a cuisine, but to a language-learning assistant it would mean something very different. The `CRFEntityExtractor` component can learn custom entities in any language, given some training data. See [Training Data Format](https://legacy-docs-v1.rasa.com/1.7.0/nlu/training-data-format/#training-data-format) for details on how to include entities in your training data. ## Extracting Places, Dates, People, Organisations spaCy has excellent pre-trained named-entity recognisers for a few different languages. You can test them out in this [interactive demo](https://demos.explosion.ai/displacy-ent/). We don’t recommend that you try to train your own NER using spaCy unless you have a lot of data and know what you are doing. Note that some spaCy models are highly case-sensitive. ## Dates, Amounts of Money, Durations, Distances, Ordinals The [duckling](https://duckling.wit.ai/) library does a great job of turning expressions like “next Thursday at 8pm” into actual datetime objects that you can use. Example: ```json "next Thursday at 8pm" => {"value":"2018-05-31T20:00:00.000+01:00"} ``` The list of supported languages can be found [here](https://github.com/facebook/duckling/tree/master/Duckling/Dimensions). Duckling can also handle durations like “two hours”, amounts of money, distances, and ordinals. ## Regular Expressions (regex) You can use regular expressions to help the CRF model learn to recognize entities. In your training data, you can provide a list of regular expressions, each of which provides the `CRFEntityExtractor` with an extra binary feature, which says if the regex was found (1) or not (0). ## Passing Custom Features to `CRFEntityExtractor` If you want to pass custom features to `CRFEntityExtractor`, you can add any dense featurizer to the pipeline before the `CRFEntityExtractor`. `CRFEntityExtractor` automatically finds the additional dense features and checks if the dense features are an iterable of `len(tokens)`, where each entry is a vector.