# Choosing a Pipeline

In Rasa Open Source, incoming messages are processed by a sequence of components. These components are executed one after another in a so-called processing `pipeline` defined in your `config.yml`. Choosing an NLU pipeline allows you to customize your model and finetune it on your dataset.

## How to Choose a Pipeline

### The Short Answer

If your training data is in English, a good starting point is the following pipeline:

```yaml
language: "en"

pipeline:
  - name: ConveRTTokenizer
  - name: ConveRTFeaturizer
  - name: RegexFeaturizer
  - name: LexicalSyntacticFeaturizer
  - name: CountVectorsFeaturizer
  - name: CountVectorsFeaturizer
    analyzer: "char_wb"
    min_ngram: 1
    max_ngram: 4
  - name: DIETClassifier
    epochs: 100
  - name: EntitySynonymMapper
  - name: ResponseSelector
    epochs: 100
```

If your training data is not in English, start with the following pipeline:

```yaml
language: "fr"  # your two-letter language code

pipeline:
  - name: WhitespaceTokenizer
  - name: RegexFeaturizer
  - name: LexicalSyntacticFeaturizer
  - name: CountVectorsFeaturizer
  - name: CountVectorsFeaturizer
    analyzer: "char_wb"
    min_ngram: 1
    max_ngram: 4
  - name: DIETClassifier
    epochs: 100
  - name: EntitySynonymMapper
  - name: ResponseSelector
    epochs: 100
```

### Choosing the Right Components

There are components for entity extraction, for intent classification, response selection, pre-processing, and others. You can learn more about any specific component on the [Components](https://legacy-docs-v1.rasa.com/1.10.26/nlu/components/#components) page.

A pipeline usually consists of three main parts:

- [Tokenization](https://legacy-docs-v1.rasa.com/1.10.26/nlu/choosing-a-pipeline/#tokenization)
- [Featurization](https://legacy-docs-v1.rasa.com/1.10.26/nlu/choosing-a-pipeline/#featurization)
- [Entity Recognition / Intent Classification / Response Selectors](https://legacy-docs-v1.rasa.com/1.10.26/nlu/choosing-a-pipeline/#entity-recognition-intent-classification-response-selectors)

#### Tokenization

For tokenization of English input, we recommend the [ConveRTTokenizer](https://legacy-docs-v1.rasa.com/1.10.26/nlu/components/#converttokenizer). You can process other whitespace-tokenized (words are separated by spaces) languages with the [WhitespaceTokenizer](https://legacy-docs-v1.rasa.com/1.10.26/nlu/components/#whitespacetokenizer).

#### Featurization

You need to decide whether to use components that provide pre-trained word embeddings or not.

##### Pre-trained Embeddings

The advantage of using pre-trained word embeddings in your pipeline is that if you have a training example like: "I want to buy apples", and Rasa is asked to predict the intent for "get pears", your model already knows that the words "apples" and "pears" are very similar. We support a few components that provide pre-trained word embeddings:

1. [MitieFeaturizer](https://legacy-docs-v1.rasa.com/1.10.26/nlu/components/#mitiefeaturizer)
2. [SpacyFeaturizer](https://legacy-docs-v1.rasa.com/1.10.26/nlu/components/#spacyfeaturizer)
3. [ConveRTFeaturizer](https://legacy-docs-v1.rasa.com/1.10.26/nlu/components/#convertfeaturizer)
4. [LanguageModelFeaturizer](https://legacy-docs-v1.rasa.com/1.10.26/nlu/components/#languagemodelfeaturizer)

#### Entity Recognition / Intent Classification / Response Selectors

Depending on your data you may want to only perform intent classification, entity recognition, or response selection. We recommend using [DIETClassifier](https://legacy-docs-v1.rasa.com/1.10.26/nlu/components/#diet-classifier) for intent classification and entity recognition and [ResponseSelector](https://legacy-docs-v1.rasa.com/1.10.26/nlu/components/#response-selector) for response selection.

### Handling Class Imbalance

Classification algorithms often do not perform well if there is a large class imbalance. To mitigate this problem, you can use a `balanced` batching strategy.

```yaml
language: "en"

pipeline:
  - name: "DIETClassifier"
    batch_strategy: sequence
```

### Component Lifecycle

Each component processes an input and/or creates an output. The order of the components is determined by the order they are listed in the `config.yml`.