# Choosing a Pipeline

In Rasa Open Source, incoming messages are processed by a sequence of components. These components are executed one after another in a so-called processing `pipeline` defined in your `config.yml`. Choosing an NLU pipeline allows you to customize your model and finetune it on your dataset.

### How to Choose a Pipeline

#### The Short Answer

If your training data is in English, a good starting point is the following pipeline:

```yaml
language: "en"

pipeline:
  - name: ConveRTTokenizer
  - name: ConveRTFeaturizer
  - name: RegexFeaturizer
  - name: LexicalSyntacticFeaturizer
  - name: CountVectorsFeaturizer
  - name: CountVectorsFeaturizer
    analyzer: "char_wb"
    min_ngram: 1
    max_ngram: 4
  - name: DIETClassifier
    epochs: 100
  - name: EntitySynonymMapper
  - name: ResponseSelector
    epochs: 100
```

If your training data is not in English, start with the following pipeline:

```yaml
language: "fr"  # your two-letter language code

pipeline:
  - name: WhitespaceTokenizer
  - name: RegexFeaturizer
  - name: LexicalSyntacticFeaturizer
  - name: CountVectorsFeaturizer
  - name: CountVectorsFeaturizer
    analyzer: "char_wb"
    min_ngram: 1
    max_ngram: 4
  - name: DIETClassifier
    epochs: 100
  - name: EntitySynonymMapper
  - name: ResponseSelector
    epochs: 100
```

### A Longer Answer

We recommend using the following pipeline, if your training data is in English:

```yaml
language: "en"

The pipeline contains the [ConveRTFeaturizer](https://legacy-docs-v1.rasa.com/1.10.12/nlu/components/#convertfeaturizer) that provides pre-trained word embeddings of the user utterance.

For example, if you have a sentence like “I want to buy apples” in your training data, and Rasa is asked to predict the intent for “get pears,” your model already knows that the words “apples” and “pears” are very similar. This is especially useful if you don’t have enough training data.

If your training data is not in English, but you still want to use pre-trained word embeddings, we recommend using the following pipeline:

```yaml
language: "fr"  # your two-letter language code

pipeline:
  - name: SpacyNLP
  - name: SpacyTokenizer
  - name: SpacyFeaturizer
  - name: RegexFeaturizer
  - name: LexicalSyntacticFeaturizer
  - name: CountVectorsFeaturizer
  - name: CountVectorsFeaturizer
    analyzer: "char_wb"
    min_ngram: 1
    max_ngram: 4
  - name: DIETClassifier
    epochs: 100
  - name: EntitySynonymMapper
  - name: ResponseSelector
    epochs: 100
```

### Choosing the Right Components

There are components for entity extraction, for intent classification, response selection, pre-processing, and others. You can learn more about any specific component on the [Components](https://legacy-docs-v1.rasa.com/1.10.12/nlu/components/#components) page. If you want to add your own component, for example, to run a spell-check or to do sentiment analysis, check out [Custom NLU Components](https://legacy-docs-v1.rasa.com/1.10.12/api/custom-nlu-components/#custom-nlu-components).

A pipeline usually consists of three main parts:

- [Tokenization](https://legacy-docs-v1.rasa.com/1.10.12/nlu/choosing-a-pipeline/#tokenization)
- [Featurization](https://legacy-docs-v1.rasa.com/1.10.12/nlu/choosing-a-pipeline/#featurization)
- [Entity Recognition / Intent Classification / Response Selectors](https://legacy-docs-v1.rasa.com/1.10.12/nlu/choosing-a-pipeline/#entity-recognition-intent-classification-response-selectors)

## Comparing Pipelines

Rasa gives you the tools to compare the performance of multiple pipelines on your data directly. See [Comparing NLU Pipelines](https://legacy-docs-v1.rasa.com/1.10.12/user-guide/testing-your-assistant/#comparing-nlu-pipelines) for more information.

## Handling Class Imbalance

Classification algorithms often do not perform well if there is a large class imbalance, for example if you have a lot of training data for some intents and very little training data for others. To mitigate this problem, you can use a `balanced` batching strategy. This algorithm ensures that all classes are represented in every batch, or at least in as many subsequent batches as possible, still mimicking the fact that some classes are more frequent than others. Balanced batching is used by default. In order to turn it off and use a classic batching strategy include `batch_strategy: sequence` in your config file.

```yaml
language: "en"

pipeline:
  - name: "DIETClassifier"
    batch_strategy: sequence
```

### Component Lifecycle

Each component processes an input and/or creates an output. The order of the components is determined by the order they are listed in the `config.yml`; the output of a component can be used by any other component that comes after it in the pipeline.

For example, for the sentence _"I am looking for Chinese food"_, the output is:

```json
{
    "text": "I am looking for Chinese food",
    "entities": [
        {
            "start": 8,
            "end": 15,
            "value": "chinese",
            "entity": "cuisine",
            "extractor": "DIETClassifier",
            "confidence": 0.864
        }
    ],
    "intent": {"confidence": 0.6485910906220309, "name": "restaurant_search"},
    "intent_ranking": [
        {"confidence": 0.6485910906220309, "name": "restaurant_search"},
        {"confidence": 0.1416153159565678, "name": "affirm"}
    ]
}
```

---

👋 I can help you get started with Rasa and answer your technical questions.