Training Data Importers

Open Source Project Documentation

You are viewing documentation for our open source project which is maintained by the community. If you want to get started building assistants with Rasa please check out our latest documentation here.

Using the `--data` command line argument

You can specify where Rasa should look for training data on your disk. Rasa then loads any potential training files and uses them to train your assistant.

If needed, you can also customize how Rasa imports training data. Potential use cases for this might be:

using a custom parser to load training data in other formats
using different approaches to collect training data (e.g. loading them from different resources)

You can write a custom importer and instruct Rasa to use it by adding the section importers to your configuration file and specifying the importer with its full class path:

importers:
  - name: "module.CustomImporter"
    parameter1: "value"
    parameter2: "value2"
  - name: "RasaFileImporter"

The name key is used to determine which importer should be loaded. Any extra parameters are passed as constructor arguments to the loaded importer.

Note

TrainingDataImporter and its subclasses contain no async methods since Rasa 3.0. In order to migrate your custom importers and make them work with Rasa 3.0, you also need to replace your async methods with synchronized ones. Please see the migration guide for more information.

Tip

You can specify multiple importers. Rasa will automatically merge their results.

RasaFileImporter (default)

By default Rasa uses the importer RasaFileImporter. If you want to use it on its own, you don't have to specify anything in your configuration file. If you want to use it together with other importers, add it to your configuration file:

importers:
  - name: "module.CustomImporter"
    parameter1: "value"
    parameter2: "value2"
  - name: "RasaFileImporter"

MultiProjectImporter (experimental)

New in 1.3

This feature is currently experimental and might change or be removed in the future. Share your feedback on it in the forum to help us making this feature ready for production.

With this importer you can train a model by combining multiple reusable Rasa projects. You might, for example, handle chitchat with one project and greet your users with another. These projects can be developed in isolation, and then combined when you train your assistant.

For example, consider the following directory structure:

.
├── config.yml
└── projects
    ├── GreetBot
    │   ├── data
    │   │   ├── nlu.yml
    │   │   └── stories.yml
    │   └── domain.yml
    └── ChitchatBot
        ├── config.yml
        ├── data
        │   ├── nlu.yml
        │   └── stories.yml
        └── domain.yml

Here the contextual AI assistant imports the ChitchatBot project which in turn imports the GreetBot project. Project imports are defined in the configuration files of each project.

To instruct Rasa to use the MultiProjectImporter module, you need add it to the importers list in your root config.yml.

importers:
  - name: "MultiProjectImporter"

Then, in the same file, specify which projects you want to import by adding them to the imports list.

imports:
  - projects/ChitchatBot

The configuration file of the ChitchatBot needs to reference GreetBot:

imports:
  - ../GreetBot

Since the GreetBot project does not specify further project to import, it doesn't need a config.yml.

Rasa uses paths relative from the configuration file to import projects. These can be anywhere on your filesystem where file access is permitted.

During the training process Rasa will import all required training files, combine them, and train a unified AI assistant. The training data is merged at runtime, so no additional training data files are created.

Policies and NLU Pipelines

Rasa will use the policy and NLU pipeline configuration of the root project directory during training. Policy and NLU configurations of imported projects will be ignored.

Watch out for merging

Equal intents, entities, slots, responses, actions and forms will be merged, e.g. if two projects have training data for an intent greet, their training data will be combined.

Writing a Custom Importer

If you are writing a custom importer, this importer has to implement the interface of TrainingDataImporter:

from typing import Optional, Text, Dict, List, Union

import rasa

from rasa.shared.core.domain import Domain
from rasa.shared.nlu.interpreter import RegexInterpreter
from rasa.shared.core.training_data.structures import StoryGraph
from rasa.shared.importers.importer import TrainingDataImporter
from rasa.shared.nlu.training_data.training_data import TrainingData

class MyImporter(TrainingDataImporter):
    """Example implementation of a custom importer component."""
    def __init__(self,
                 config_file: Optional[Text] = None,
                 domain_path: Optional[Text] = None,
                 training_data_paths: Optional[Union[List[Text], Text]] = None,
                 **kwargs: Dict):
        """Constructor of your custom file importer."
        pass
    def get_domain(self) -> Domain:
        path_to_domain_file = self._custom_get_domain_file()
        return Domain.load(path_to_domain_file)
    def _custom_get_domain_file(self) -> Text:
        pass
    def get_stories(self, interpreter: "NaturalLanguageInterpreter" = RegexInterpreter(),
                    exclusion_percentage: Optional[int] = None) -> StoryGraph:
        from rasa.shared.core.training_data.story_reader.yaml_story_reader import YAMLStoryReader
        path_to_stories = self._custom_get_story_file()
        return YAMLStoryReader.read_from_file(path_to_stories, self.get_domain())
    def _custom_get_story_file(self) -> Text:
        pass
    def get_config(self) -> Dict:
        path_to_config = self._custom_get_config_file()
        return rasa.utils.io.read_config_file(path_to_config)
    def _custom_get_config_file(self) -> Text:
        pass
    def get_nlu_data(self, language: Optional[Text] = "en") -> TrainingData:
        from rasa.shared.nlu.training_data import loading
        path_to_nlu_file = self._custom_get_nlu_file()
        return loading.load_data(path_to_nlu_file)
    def _custom_get_nlu_file(self) -> Text:
        pass