# Testing Your Assistant

## Overview
These docs are for version 1.x of Rasa Open Source.

[Docs for the new version 2.0 can be found here.](/content/docs/rasa/index.html)

## User Guide
- [Installation](https://legacy-docs-v1.rasa.com/1.9.4/user-guide/installation/)
- [Tutorial: Rasa Basics](https://legacy-docs-v1.rasa.com/1.9.4/user-guide/rasa-tutorial/)
- [Tutorial: Building Assistants](https://legacy-docs-v1.rasa.com/1.9.4/user-guide/building-assistants/)
- [Command Line Interface](https://legacy-docs-v1.rasa.com/1.9.4/user-guide/command-line-interface/)
- [Architecture](https://legacy-docs-v1.rasa.com/1.9.4/user-guide/architecture/)
- [Messaging and Voice Channels](https://legacy-docs-v1.rasa.com/1.9.4/user-guide/messaging-and-voice-channels/)
- [Testing Your Assistant](https://legacy-docs-v1.rasa.com/1.9.4/user-guide/testing-your-assistant/#)
- [Validate Data](https://legacy-docs-v1.rasa.com/1.9.4/user-guide/validate-files/)
- [Configuring the HTTP API](https://legacy-docs-v1.rasa.com/1.9.4/user-guide/configuring-http-api/)
- [Deploying Your Rasa Assistant](https://legacy-docs-v1.rasa.com/1.9.4/user-guide/how-to-deploy/)
- [Cloud Storage](https://legacy-docs-v1.rasa.com/1.9.4/user-guide/cloud-storage/)

## NLU
- [About](https://legacy-docs-v1.rasa.com/1.9.4/nlu/about/)
- [Using NLU Only](https://legacy-docs-v1.rasa.com/1.9.4/nlu/using-nlu-only/)
- [Training Data Format](https://legacy-docs-v1.rasa.com/1.9.4/nlu/training-data-format/)
- [Language Support](https://legacy-docs-v1.rasa.com/1.9.4/nlu/language-support/)
- [Choosing a Pipeline](https://legacy-docs-v1.rasa.com/1.9.4/nlu/choosing-a-pipeline/)
- [Components](https://legacy-docs-v1.rasa.com/1.9.4/nlu/components/)
- [Entity Extraction](https://legacy-docs-v1.rasa.com/1.9.4/nlu/entity-extraction/)

## Core
- [About](https://legacy-docs-v1.rasa.com/1.9.4/core/about/)
- [Stories](https://legacy-docs-v1.rasa.com/1.9.4/core/stories/)
- [Domains](https://legacy-docs-v1.rasa.com/1.9.4/core/domains/)
- [Responses](https://legacy-docs-v1.rasa.com/1.9.4/core/responses/)
- [Actions](https://legacy-docs-v1.rasa.com/1.9.4/core/actions/)
- [Reminders and External Events](https://legacy-docs-v1.rasa.com/1.9.4/core/reminders-and-external-events/)
- [Policies](https://legacy-docs-v1.rasa.com/1.9.4/core/policies/)
- [Slots](https://legacy-docs-v1.rasa.com/1.9.4/core/slots/)
- [Forms](https://legacy-docs-v1.rasa.com/1.9.4/core/forms/)
- [Retrieval Actions](https://legacy-docs-v1.rasa.com/1.9.4/core/retrieval-actions/)
- [Interactive Learning](https://legacy-docs-v1.rasa.com/1.9.4/core/interactive-learning/)
- [Fallback Actions](https://legacy-docs-v1.rasa.com/1.9.4/core/fallback-actions/)
- [Knowledge Base Actions](https://legacy-docs-v1.rasa.com/1.9.4/core/knowledge-bases/)

## Conversation Design
- [Dialogue Elements](https://legacy-docs-v1.rasa.com/1.9.4/dialogue-elements/dialogue-elements/)
- [Small Talk](https://legacy-docs-v1.rasa.com/1.9.4/dialogue-elements/small-talk/)
- [Completing Tasks](https://legacy-docs-v1.rasa.com/1.9.4/dialogue-elements/completing-tasks/)
- [Guiding Users](https://legacy-docs-v1.rasa.com/1.9.4/dialogue-elements/guiding-users/)

## API Reference
- [Action Server](https://legacy-docs-v1.rasa.com/1.9.4/api/action-server/)
- [HTTP API](https://legacy-docs-v1.rasa.com/1.9.4/api/http-api/)
- [Jupyter Notebooks](https://legacy-docs-v1.rasa.com/1.9.4/api/jupyter-notebooks/)
- [Agent](https://legacy-docs-v1.rasa.com/1.9.4/api/agent/)
- [Custom NLU Components](https://legacy-docs-v1.rasa.com/1.9.4/api/custom-nlu-components/)
- [Rasa SDK](https://legacy-docs-v1.rasa.com/1.9.4/api/rasa-sdk/)
- [Events](https://legacy-docs-v1.rasa.com/1.9.4/api/events/)
- [Tracker](https://legacy-docs-v1.rasa.com/1.9.4/api/tracker/)
- [Tracker Stores](https://legacy-docs-v1.rasa.com/1.9.4/api/tracker-stores/)
- [Event Brokers](https://legacy-docs-v1.rasa.com/1.9.4/api/event-brokers/)
- [Lock Stores](https://legacy-docs-v1.rasa.com/1.9.4/api/lock-stores/)
- [Training Data Importers](https://legacy-docs-v1.rasa.com/1.9.4/api/training-data-importers/)
- [Featurization of Conversations](https://legacy-docs-v1.rasa.com/1.9.4/api/core-featurization/)
- [TensorFlow Configuration](https://legacy-docs-v1.rasa.com/1.9.4/api/tensorflow_usage/)
- [Migration Guide](https://legacy-docs-v1.rasa.com/1.9.4/migration-guide/)
- [Rasa Open Source Change Log](https://legacy-docs-v1.rasa.com/1.9.4/changelog/)

## Migrate from (beta)
- [Dialogflow](https://legacy-docs-v1.rasa.com/1.9.4/migrate-from/google-dialogflow-to-rasa/)
- [Wit.ai](https://legacy-docs-v1.rasa.com/1.9.4/migrate-from/facebook-wit-ai-to-rasa/)
- [LUIS](https://legacy-docs-v1.rasa.com/1.9.4/migrate-from/microsoft-luis-to-rasa/)
- [IBM Watson](https://legacy-docs-v1.rasa.com/1.9.4/migrate-from/ibm-watson-to-rasa/)

## Reference
- [Glossary](https://legacy-docs-v1.rasa.com/1.9.4/glossary/)

## Versions
viewing: 1.9.4

## Warning
**This document is for an old version of Rasa. The latest version is 1.10.26.**

## End-to-End Testing
Rasa Open Source lets you test dialogues end-to-end by running through test conversations and making sure that both NLU and Core make correct predictions. To do this, you need some stories in the end-to-end format, which includes both the NLU output and the original text. Here are some examples:

By default Rasa Open Source saves conversation tests to `tests/conversation_tests.md`. You can test your assistant against them by running:
```sh
$ rasa test
```

## Evaluating an NLU Model
A standard technique in machine learning is to keep some data separate as a _test set_. You can 
[split your NLU training data](https://legacy-docs-v1.rasa.com/1.9.4/user-guide/command-line-interface/#train-test-split) into train and test sets using:
```sh
rasa data split nlu
```

If you’ve done this, you can see how well your NLU model predicts the test cases using this command:
```sh
rasa test nlu -u train_test_split/test_data.md --model models/nlu-20180323-145833.tar.gz
```

## Comparing NLU Pipelines
By passing multiple pipeline configurations (or a folder containing them) to the CLI, Rasa will run a comparative examination between the pipelines.
```sh
$ rasa test nlu --config pretrained_embeddings_spacy.yml supervised_embeddings.yml --nlu data/nlu.md --runs 3 --percentages 0 25 50 70 90
```

## Intent Classification
The evaluation script will produce a report, confusion matrix, and confidence histogram for your model. The report logs precision, recall, and f1 measure for each intent and entity, as well as providing an overall average.

## Response Selection
The evaluation script will produce a combined report for all response selector models in your pipeline.

## Entity Extraction
The `CRFEntityExtractor` is the only entity extractor which you train using your own data, and so is the only one that will be evaluated.

## Evaluating a Core Model
You can evaluate your trained model on a set of test stories by using the evaluate script:
```sh
rasa test core --stories test_stories.md --out results
```

## Comparing Core Configurations
To choose a configuration for your core model, or to choose hyperparameters for a specific policy, you want to measure how well Rasa Core will generalise to conversations which it hasn’t seen before. Rasa Core has some scripts to help you choose and fine-tune your policy configuration.