Testing Your Assistant
These docs are for version 1.x of Rasa Open Source.
User Guide
- Installation
- Tutorial: Rasa Basics
- Tutorial: Building Assistants
- Command Line Interface
- Architecture
- Messaging and Voice Channels
- Testing Your Assistant
- Setting up CI/CD
- Validate Data
- Configuring the HTTP API
- Deploying Your Rasa Assistant
- Cloud Storage
NLU
- About
- Using NLU Only
- Training Data Format
- Language Support
- Choosing a Pipeline
- Components
- Entity Extraction
Core
- About
- Stories
- Domains
- Responses
- Actions
- Reminders and External Events
- Policies
- Slots
- Forms
- Retrieval Actions
- Interactive Learning
- Fallback Actions
- Knowledge Base Actions
Conversation Design
API Reference
- Action Server
- HTTP API
- Jupyter Notebooks
- Agent
- Custom NLU Components
- Rasa SDK
- Events
- Tracker
- Tracker Stores
- Event Brokers
- Lock Stores
- Training Data Importers
- Featurization of Conversations
- TensorFlow Configuration
- Migration Guide
- Rasa Open Source Change Log
Migrate from (beta)
Reference
Versions
viewing: 1.10.5
Testing Your Assistant
End-to-End Testing
Rasa Open Source lets you test dialogues end-to-end by running through test conversations and making sure that both NLU and Core make correct predictions.
To do this, you need some stories in the end-to-end format, which includes both the NLU output and the original text. Here are some examples:
By default Rasa Open Source saves conversation tests to tests/conversation_tests.md.
You can test your assistant against them by running:
$ rasa test
Evaluating an NLU Model
A standard technique in machine learning is to keep some data separate as a test set. You can split your NLU training data
Comparing NLU Pipelines
By passing multiple pipeline configurations (or a folder containing them) to the CLI, Rasa will run a comparative examination between the pipelines.
Intent Classification
The evaluation script will produce a report, confusion matrix, and confidence histogram for your model.
Entity Extraction
The CRFEntityExtractor is the only entity extractor which you train using your own data...
Evaluating a Core Model
You can evaluate your trained model on a set of test stories by using the evaluate script:
rasa test core --stories test_stories.md --out results
Comparing Core Configurations
To choose a configuration for your core model, or to choose hyperparameters for a specific policy...
👋 I can help you get started with Rasa and answer your technical questions.