These docs are for version 1.x of Rasa Open Source.

## User Guide

- [Installation](https://legacy-docs-v1.rasa.com/1.9.5/user-guide/installation/)
- [Tutorial: Rasa Basics](https://legacy-docs-v1.rasa.com/1.9.5/user-guide/rasa-tutorial/)
- [Tutorial: Building Assistants](https://legacy-docs-v1.rasa.com/1.9.5/user-guide/building-assistants/)
- [Command Line Interface](https://legacy-docs-v1.rasa.com/1.9.5/user-guide/command-line-interface/)
- [Architecture](https://legacy-docs-v1.rasa.com/1.9.5/user-guide/architecture/)
- [Messaging and Voice Channels](https://legacy-docs-v1.rasa.com/1.9.5/user-guide/messaging-and-voice-channels/)
- [Testing Your Assistant](https://legacy-docs-v1.rasa.com/1.9.5/user-guide/testing-your-assistant/#)
- [Validate Data](https://legacy-docs-v1.rasa.com/1.9.5/user-guide/validate-files/)
- [Configuring the HTTP API](https://legacy-docs-v1.rasa.com/1.9.5/user-guide/configuring-http-api/)
- [Deploying Your Rasa Assistant](https://legacy-docs-v1.rasa.com/1.9.5/user-guide/how-to-deploy/)
- [Cloud Storage](https://legacy-docs-v1.rasa.com/1.9.5/user-guide/cloud-storage/)

## NLU

- [About](https://legacy-docs-v1.rasa.com/1.9.5/nlu/about/)
- [Using NLU Only](https://legacy-docs-v1.rasa.com/1.9.5/nlu/using-nlu-only/)
- [Training Data Format](https://legacy-docs-v1.rasa.com/1.9.5/nlu/training-data-format/)
- [Language Support](https://legacy-docs-v1.rasa.com/1.9.5/nlu/language-support/)
- [Choosing a Pipeline](https://legacy-docs-v1.rasa.com/1.9.5/nlu/choosing-a-pipeline/)
- [Components](https://legacy-docs-v1.rasa.com/1.9.5/nlu/components/)
- [Entity Extraction](https://legacy-docs-v1.rasa.com/1.9.5/nlu/entity-extraction/)

## Core

- [About](https://legacy-docs-v1.rasa.com/1.9.5/core/about/)
- [Stories](https://legacy-docs-v1.rasa.com/1.9.5/core/stories/)
- [Domains](https://legacy-docs-v1.rasa.com/1.9.5/core/domains/)
- [Responses](https://legacy-docs-v1.rasa.com/1.9.5/core/responses/)
- [Actions](https://legacy-docs-v1.rasa.com/1.9.5/core/actions/)
- [Reminders and External Events](https://legacy-docs-v1.rasa.com/1.9.5/core/reminders-and-external-events/)
- [Policies](https://legacy-docs-v1.rasa.com/1.9.5/core/policies/)
- [Slots](https://legacy-docs-v1.rasa.com/1.9.5/core/slots/)
- [Forms](https://legacy-docs-v1.rasa.com/1.9.5/core/forms/)
- [Retrieval Actions](https://legacy-docs-v1.rasa.com/1.9.5/core/retrieval-actions/)
- [Interactive Learning](https://legacy-docs-v1.rasa.com/1.9.5/core/interactive-learning/)
- [Fallback Actions](https://legacy-docs-v1.rasa.com/1.9.5/core/fallback-actions/)
- [Knowledge Base Actions](https://legacy-docs-v1.rasa.com/1.9.5/core/knowledge-bases/)

## Conversation Design

- [Dialogue Elements](https://legacy-docs-v1.rasa.com/1.9.5/dialogue-elements/dialogue-elements/)
- [Small Talk](https://legacy-docs-v1.rasa.com/1.9.5/dialogue-elements/small-talk/)
- [Completing Tasks](https://legacy-docs-v1.rasa.com/1.9.5/dialogue-elements/completing-tasks/)
- [Guiding Users](https://legacy-docs-v1.rasa.com/1.9.5/dialogue-elements/guiding-users/)

## API Reference

- [Action Server](https://legacy-docs-v1.rasa.com/1.9.5/api/action-server/)
- [HTTP API](https://legacy-docs-v1.rasa.com/1.9.5/api/http-api/)
- [Jupyter Notebooks](https://legacy-docs-v1.rasa.com/1.9.5/api/jupyter-notebooks/)
- [Agent](https://legacy-docs-v1.rasa.com/1.9.5/api/agent/)
- [Custom NLU Components](https://legacy-docs-v1.rasa.com/1.9.5/api/custom-nlu-components/)
- [Rasa SDK](https://legacy-docs-v1.rasa.com/1.9.5/api/rasa-sdk/)
- [Events](https://legacy-docs-v1.rasa.com/1.9.5/api/events/)
- [Tracker](https://legacy-docs-v1.rasa.com/1.9.5/api/tracker/)
- [Tracker Stores](https://legacy-docs-v1.rasa.com/1.9.5/api/tracker-stores/)
- [Event Brokers](https://legacy-docs-v1.rasa.com/1.9.5/api/event-brokers/)
- [Lock Stores](https://legacy-docs-v1.rasa.com/1.9.5/api/lock-stores/)
- [Training Data Importers](https://legacy-docs-v1.rasa.com/1.9.5/api/training-data-importers/)
- [Featurization of Conversations](https://legacy-docs-v1.rasa.com/1.9.5/api/core-featurization/)
- [TensorFlow Configuration](https://legacy-docs-v1.rasa.com/1.9.5/api/tensorflow_usage/)
- [Migration Guide](https://legacy-docs-v1.rasa.com/1.9.5/migration-guide/)
- [Rasa Open Source Change Log](https://legacy-docs-v1.rasa.com/1.9.5/changelog/)

## Migrate from (beta)

- [Dialogflow](https://legacy-docs-v1.rasa.com/1.9.5/migrate-from/google-dialogflow-to-rasa/)
- [Wit.ai](https://legacy-docs-v1.rasa.com/1.9.5/migrate-from/facebook-wit-ai-to-rasa/)
- [LUIS](https://legacy-docs-v1.rasa.com/1.9.5/migrate-from/microsoft-luis-to-rasa/)
- [IBM Watson](https://legacy-docs-v1.rasa.com/1.9.5/migrate-from/ibm-watson-to-rasa/)

## Reference

- [Glossary](https://legacy-docs-v1.rasa.com/1.9.5/glossary/)

## Versions

viewing: 1.9.5

### Warning

This document is for an old version of Rasa. The latest version is 1.10.26.

# Testing Your Assistant

## End-to-End Testing

Rasa Open Source lets you test dialogues end-to-end by running through test conversations and making sure that both NLU and Core make correct predictions.

To do this, you need some stories in the end-to-end format, which includes both the NLU output and the original text.
Here are some examples:

By default, Rasa Open Source saves conversation tests to `tests/conversation_tests.md`. You can test your assistant against them by running:

```
$ rasa test
```

## Evaluating an NLU Model

A standard technique in machine learning is to keep some data separate as a _test set_. You can [split your NLU training data](https://legacy-docs-v1.rasa.com/1.9.5/user-guide/command-line-interface/#train-test-split) into train and test sets using:

```
rasa data split nlu
```

If you’ve done this, you can see how well your NLU model predicts the test cases using this command:

```
rasa test nlu -u train_test_split/test_data.md --model models/nlu-20180323-145833.tar.gz
```

## Comparing NLU Pipelines

By passing multiple pipeline configurations (or a folder containing them) to the CLI, Rasa will run a comparative examination between the pipelines.

```
$ rasa test nlu --config pretrained_embeddings_spacy.yml supervised_embeddings.yml --nlu data/nlu.md --runs 3 --percentages 0 25 50 70 90
```

## Intent Classification

The evaluation script will produce a report, confusion matrix, and confidence histogram for your model.

The report logs precision, recall and f1 measure for each intent and entity, as well as providing an overall average.
You can save these reports as JSON files using the `--report` argument.

## Response Selection

The evaluation script will produce a combined report for all response selector models in your pipeline.

The report logs precision, recall and f1 measure for each response, as well as providing an overall average. You can save these reports as JSON files using the `--report` argument.

## Entity Extraction

To evaluate entity extraction we apply a simple tag-based approach.

| extracted | Simple tags (score) | BILOU tags (score) |
| --- | --- | --- |
| \[near Alexanderplatz\](loc) \[tonight\](time) | loc loc time (3) | B-loc L-loc U-time (3) |
| \[near\](loc) \[Alexanderplatz\](loc) \[tonight\](time) | loc loc time (3) | U-loc U-loc U-time (1) |

## Evaluating a Core Model

You can evaluate your trained model on a set of test stories by using the evaluate script:

```
rasa test core --stories test_stories.md --out results
```

### Comparing Core Configurations

To choose a configuration for your core model, or to choose hyperparameters for a specific policy, you want to measure how well Rasa Core will generalise to conversations which it hasn’t seen before.