Components

Note

For clarity, we have renamed the pre-defined pipelines to reflect what they do rather than which libraries they use as of Rasa NLU 0.15. The tensorflow_embedding pipeline is now called supervised_embeddings, and spacy_sklearn is now known as pretrained_embeddings_spacy. Please update your code if you are using these.

This is a reference of the configuration options for every built-in component in Rasa NLU. If you want to build a custom component, check out Custom NLU Components.

Word Vector Sources

MitieNLP

Short MITIE initializer

Outputs nothing

Requires nothing

Description Initializes mitie structures. Every mitie component relies on this, hence this should be put at the beginning of every pipeline that uses any mitie components.

Configuration

pipeline:
- name: "MitieNLP"
  # language model to load
  model: "data/total_word_feature_extractor.dat"

For more information where to get that file from, head over to installing MITIE.

SpacyNLP

Short spacy language initializer

Outputs nothing

Requires nothing

Description Initializes spacy structures. Every spacy component relies on this, hence this should be put at the beginning of every pipeline that uses any spacy components.

Configuration

pipeline:
- name: "SpacyNLP"
  # language model to load
  model: "en_core_web_md"

# when retrieving word vectors, this will decide if the casing
  # of the word is relevant. E.g. `hello` and `Hello` will
  # retrieve the same vector, if set to `false`. For some
  # applications and models it makes sense to differentiate
  # between these two words, therefore setting this to `true`.
  case_sensitive: false

Text Featurizers

MitieFeaturizer

Short MITIE intent featurizer

Outputs nothing, used as an input to intent classifiers that need intent features (e.g. SklearnIntentClassifier)

Requires MitieNLP

Type Dense featurizer

Description Creates features for intent classification using the MITIE featurizer.

Note NOT used by the MitieIntentClassifier component. Currently, only SklearnIntentClassifier is able to use precomputed features.

Configuration

pipeline:
- name: "MitieFeaturizer"

SpacyFeaturizer

Short spacy intent featurizer

Outputs nothing, used as an input to intent classifiers that need intent features (e.g. SklearnIntentClassifier)

Requires SpacyNLP

Type Dense featurizer

Description Creates features for intent classification using the spacy featurizer.

Configuration

pipeline:
- name: "SpacyFeaturizer"

ConveRTFeaturizer

Short Creates a vector representation of user message and response (if specified) using ConveRT model.

Outputs nothing, used as an input to intent classifiers and response selectors that need intent features and response features respectively (e.g. EmbeddingIntentClassifier and ResponseSelector)

Requires nothing

Type Dense featurizer

Description Creates features for intent classification and response selection. Uses the default signature to compute vector representations of input text.

Warning Since ConveRT model is trained only on an english corpus of conversations, this featurizer should only be used if your training data is in english language.

Configuration

pipeline:
- name: "ConveRTFeaturizer"

Intent Classifiers

MitieIntentClassifier

Short MITIE intent classifier (using a text categorizer)

Outputs intent

Requires A tokenizer and a featurizer

Output-Example

{
    "intent": {"name": "greet", "confidence": 0.98343}
}

Description This classifier uses MITIE to perform intent classification. The underlying classifier is using a multi-class linear SVM with a sparse linear kernel (see MITIE trainer code).

Configuration

pipeline:
- name: "MitieIntentClassifier"

SklearnIntentClassifier

Short sklearn intent classifier

Outputs intent and intent_ranking

Requires A featurizer

Output-Example

{
    "intent": {"name": "greet", "confidence": 0.78343},
    "intent_ranking": [
        {
            "confidence": 0.1485910906220309,
            "name": "goodbye"
        },
        {
            "confidence": 0.08161531595656784,
            "name": "restaurant_search"
        }
    ]
}

Description The sklearn intent classifier trains a linear SVM which gets optimized using a grid search. In addition to other classifiers it also provides rankings of the labels that did not “win”. The spacy intent classifier needs to be preceded by a featurizer in the pipeline. This featurizer creates the features used for the classification.

Configuration

pipeline:
- name: "SklearnIntentClassifier"
  # Specifies the list of regularization values to
  # cross-validate over for C-SVM.
  # This is used with the ``kernel`` hyperparameter in GridSearchCV.
  C: [1, 2, 5, 10, 20, 100]
  # Specifies the kernel to use with C-SVM.
  # This is used with the ``C`` hyperparameter in GridSearchCV.
  kernels: ["linear"]

EmbeddingIntentClassifier

Short Embedding intent classifier

Outputs intent and intent_ranking

Requires A featurizer

Output-Example

{
    "intent": {"name": "greet", "confidence": 0.8343},
    "intent_ranking": [
        {
            "confidence": 0.385910906220309,
            "name": "goodbye"
        },
        {
            "confidence": 0.28161531595656784,
            "name": "restaurant_search"
        }
    ]
}

Description The embedding intent classifier embeds user inputs and intent labels into the same space. Supervised embeddings are trained by maximizing similarity between them. This algorithm is based on StarSpace. However, in this implementation the loss function is slightly different and additional hidden layers are added together with dropout. This algorithm also provides similarity rankings of the labels that did not “win”. The embedding intent classifier needs to be preceded by a featurizer in the pipeline. This featurizer creates the features used for the embeddings.

Configuration

pipeline:
- name: "EmbeddingIntentClassifier"

Entity Extractors

MitieEntityExtractor

Short MITIE entity extraction (using a MITIE NER trainer)

Outputs appends entities

Requires MitieNLP

Output-Example

{
    "entities": [{"value": "New York City",
                  "start": 20,
                  "end": 33,
                  "confidence": null,
                  "entity": "city",
                  "extractor": "MitieEntityExtractor"}]
}

Description This uses the MITIE entity extraction to find entities in a message. The underlying classifier is using a multi class linear SVM with a sparse linear kernel and custom features. The MITIE component does not provide entity confidence values.

Configuration

pipeline:
- name: "MitieEntityExtractor"

SpacyEntityExtractor

Short spaCy entity extraction

Outputs appends entities

Requires SpacyNLP

Output-Example

{
    "entities": [{"value": "New York City",
                  "start": 20,
                  "end": 33,
                  "entity": "city",
                  "confidence": null,
                  "extractor": "SpacyEntityExtractor"}]
}

Description Using spaCy this component predicts the entities of a message. spaCy uses a statistical BILOU transition model. As of now, this component can only use the spaCy builtin entity extraction models and can not be retrained. This extractor does not provide any confidence scores.

Configuration

pipeline:
- name: "SpacyEntityExtractor"
  # dimensions to extract
  dimensions: ["PERSON", "LOC", "ORG", "PRODUCT"]

EntitySynonymMapper

Short Maps synonymous entity values to the same value.

Outputs modifies existing entities that previous entity extraction components found

Requires nothing

Description If the training data contains defined synonyms (by using the value attribute on the entity examples). this component will make sure that detected entity values will be mapped to the same value.

CRFEntityExtractor

Short conditional random field entity extraction

Outputs appends entities

Requires A tokenizer

Output-Example

{
    "entities": [{"value":"New York City",
                  "start": 20,
                  "end": 33,
                  "entity": "city",
                  "confidence": 0.874,
                  "extractor": "CRFEntityExtractor"}]
}

Description This component implements conditional random fields to do named entity recognition. CRFs can be thought of as an undirected Markov chain where the time steps are words and the states are entity classes. If POS features are used (pos or pos2), spaCy has to be installed.

Configuration

pipeline:
- name: "CRFEntityExtractor"
  # The features are a ``[before, word, after]`` array with
  # before, word, after holding keys about which
  # features to use for each word, for example, ``"title"``
  features: [["low", "title"], ["bias", "suffix3"], ["upper", "pos", "pos2"]]

# The flag determines whether to use BILOU tagging or not.
  # BILOU tagging is more rigorous however
  # requires more examples per entity. Rule of thumb: use only
  # if more than 100 examples per entity.
  BILOU_flag: true

# This is the value given to sklearn_crfcuite.CRF tagger before training.
  max_iterations: 50

# This is the value given to sklearn_crfcuite.CRF tagger before training.
  # Specifies the L1 regularization coefficient.
  L1_c: 0.1

# This is the value given to sklearn_crfcuite.CRF tagger before training.
  # Specifies the L2 regularization coefficient.
  L2_c: 0.1

DucklingHTTPExtractor

Short Duckling lets you extract common entities like dates, amounts of money, distances, and others in a number of languages.

Outputs appends entities

Requires nothing

Output-Example

{
    "entities": [{"end": 53,
                  "entity": "time",
                  "start": 48,
                  "value": "2017-04-10T00:00:00.000+02:00",
                  "confidence": 1.0,
                  "extractor": "DucklingHTTPExtractor"}]
}

Description To use this component you need to run a duckling server.