Testing Your Assistant
These docs are for version 1.x of Rasa Open Source.
User Guide
- Installation
- Tutorial: Rasa Basics
- Tutorial: Building Assistants
- Command Line Interface
- Architecture
- Messaging and Voice Channels
- Testing Your Assistant
- Setting up CI/CD
- Validate Data
- Configuring the HTTP API
- Deploying Your Rasa Assistant
- Cloud Storage
NLU
- About
- Using NLU Only
- Training Data Format
- Language Support
- Choosing a Pipeline
- Components
- Entity Extraction
Core
- About
- Stories
- Domains
- Responses
- Actions
- Reminders and External Events
- Policies
- Slots
- Forms
- Retrieval Actions
- Interactive Learning
- Fallback Actions
- Knowledge Base Actions
Conversation Design
API Reference
- Action Server
- HTTP API
- Jupyter Notebooks
- Agent
- Custom NLU Components
- Rasa SDK
- Events
- Tracker
- Tracker Stores
- Event Brokers
- Lock Stores
- Training Data Importers
- Featurization of Conversations
- TensorFlow Configuration
- Migration Guide
- Rasa Open Source Change Log
Migrate from (beta)
Reference
Versions
viewing: 1.10.21
Testing Your Assistant
End-to-End Testing
Rasa Open Source lets you test dialogues end-to-end by running through test conversations and making sure that both NLU and Core make correct predictions.
Evaluating an NLU Model
A standard technique in machine learning is to keep some data separate as a test set.
Comparing NLU Pipelines
By passing multiple pipeline configurations to the CLI, Rasa will run a comparative examination between the pipelines.
Intent Classification
The evaluation script will produce a report, confusion matrix, and confidence histogram for your model.
Response Selection
The evaluation script will produce a combined report for all response selector models in your pipeline.
Entity Extraction
The CRFEntityExtractor is the only entity extractor which you train using your own data.
Entity Scoring
To evaluate entity extraction we apply a simple tag-based approach.
Evaluating a Core Model
You can evaluate your trained model on a set of test stories by using the evaluate script.
Comparing Core Configurations
To choose a configuration for your core model, you want to measure how well Rasa Core will generalise to conversations which it hasn’t seen before.