Telemetry Event Reference

You are viewing documentation for our open source project which is maintained by the community. If you want to get started building assistants with Rasa please check out our latest documentation here.

Telemetry events are only reported if telemetry is enabled. A detailed explanation on the reasoning behind collecting optional telemetry events can be found in our telemetry documentation.

Model Training

Training Started

A training of a Rasa machine learning model got started. The event provides information on aggregated training data statistics. Event properties:

• language (string): Language model is trained with, e.g. 'en'.
• training_id (string): Generated unique identifier for this training.
• type (string): Type of model trained, either 'nlu', 'core' or 'rasa'.
• pipeline (undefined): List of the pipeline configurations used for training.
• policies (undefined): List of the policy configurations used for training.
• train_schema (undefined): Training graph schema for graph recipe
• predict_schema (undefined): Predict graph schema for graph recipe
• num_intent_examples (integer): Number of NLU examples.
• num_entity_examples (integer): Number of entity examples.
• num_actions (integer): Number of actions defined in the domain.
• num_templates (integer): Number of templates or responses defined in the domain.
• num_conditional_response_variations (integer): Number of conditional response variations defined in the domain.
• num_slot_mappings (integer): Number of total slot mappings defined in the domain.
• num_custom_slot_mappings (integer): Number of custom slot mappings defined in the domain.
• num_conditional_slot_mappings (integer): Number of slot mappings with conditions attached defined in the domain.
• num_slots (integer): Number of slots defined in the domain.
• num_forms (integer): Number of forms defined in the domain.
• num_intents (integer): Number of intents defined in the domain.
• num_entities (integer): Number of entities defined in the domain.
• num_story_steps (integer): Number of story steps available.
• num_lookup_tables (integer): Number of different lookup tables.
• num_synonyms (integer): Total number of entity synonyms defined.
• num_regexes (integer): Total number of regexes defined.
• is_finetuning (boolean): True if a model is trained by finetuning an existing model.
• recipe (string): Recipe used in training the model, either 'default.v1' or 'graph.v1'.

Training Completed

The training of a Rasa machine learning model finished. The event provides information about the resulting model. Event properties:

• training_id (string): Generated unique identifier for this training. Can be used to join with 'Training Started'.
• type (string): Type of model trained, either 'nlu', 'core' or 'rasa'.
• runtime (integer): The time in seconds it took to train the model.

Model Testing

Model Core Tested

Triggered when a Core model is getting tested. Event properties:

• project (string,null): Fingerprint of the project the tested model got trained in.
• num_story_steps (integer): Number of story steps used for testing
• end_to_end (boolean): Indicates if tests are running in end-to-end mode, testing message handling and dialogue handling at the same time.

Model NLU Tested

Triggered when an NLU model is getting tested. Event properties:

• num_intent_examples (integer): Number of NLU examples.
• num_entity_examples (integer): Number of entity examples.
• num_lookup_tables (integer): Number of different lookup tables.
• num_synonyms (integer): Total number of entity synonyms defined.
• num_regexes (integer): Total number of regexes defined.

End-to-End Testing

End-to-End Testing Started

Triggered when end-to-end testing has been started. Event properties:

• number_of_test_cases (integer): Number of test cases to be run.
• number_of_fixtures (integer): Number of fixtures defined globally.
• uses_fixtures (boolean): Indicates if any fixtures have been defined globally.

Model Serving

Interactive Learning Started

Triggered when an interactive learning session got started. Event properties:

• skip_visualization (boolean): Whether the visualization of stories should be shown during the interactive learning session.
• save_in_e2e (boolean): Whether the data should be stored in end-to-end format.

Server Started

Triggered when a Rasa Open Source server gets started. Event properties:

• input_channels (array): Names of the used input channels.
• api_enabled (boolean): Indicator if the API is enabled or if only the input channel is running.
• number_of_workers (integer): Amount of Sanic workers started as part of the server.
• endpoints_nlg (string,null): Type of the used NLG endpoint.
• endpoints_nlu (string,null): Type of the used NLU endpoint.
• endpoints_action_server (string,null): Type of the used action server.
• endpoints_model_server (string,null): Type of the used model server.
• endpoints_tracker_store (string,null): Type of the used tracker store.
• endpoints_lock_store (string,null): Type of the used lock store.
• endpoints_event_broker (string,null): Type of the used event broker.
• project (string,null): Hash of the deployed model the server is started with.

Shell Started

Triggered when a shell session is started to talk to a trained bot. Event properties:

• type (string): Type of the model, either 'nlu', 'core' or 'rasa'.

Rasa X Local Started

Triggered when a Rasa X is started in local mode.

Markers Extraction

Markers Extraction Initiated

Triggered when marker extraction has been initiated. Event properties:

• strategy (string): Strategy to use when selecting trackers to extract from.
• only_extract (boolean): Indicates if path to write out statistics hasn't been specified.
• seed (boolean): The seed to initialise the random number generator for use with the 'sample' strategy.
• count (integer,null): Number of trackers to extract from (for any strategy except 'all').

Markers Extracted

Triggered when markers have been extracted. Event properties:

• trackers_count (integer): Number of processed trackers.

Markers Parsed

Triggered when markers have been successfully parsed. Event properties:

• marker_count (integer): Number of parsed markers.
• max_depth (integer): Maximum depth of the parsed markers.
• branching_factor (integer): Maximum number of children of any of the parsed markers.

Markers Statistics Computed

Triggered when marker statistics have been computed. Event properties:

• trackers_count (integer): Number of processed trackers.

Data Handling

Training Data Split

Triggered when training data gets split. Event properties:

• fraction (number): Percentage of the data which goes into training data (the rest goes into the test set).
• type (string): Type of data, either 'nlu', 'core' or 'rasa'.

Training Data Validated

Triggered when training data gets validated. Event properties:

• validation_success (boolean): Whether the validation was successful.

Training Data Converted

Triggered when training data gets converted. Event properties:

• output_format (string): Target format of the converter.
• type (string): Type of data, either 'nlu', 'core', 'config' or 'nlg'.

Tracker Exported

Triggered when conversations get exported from a tracker store through an event broker. Event properties:

• event_broker (string): Name of the used event broker.
• tracker_store (string): Name of the used tracker store.
• number_of_exported_events (integer): Number of events exported through the event broker.

Story Visualization Started

Triggered when stories are getting visualized.

Miscellaneous

Telemetry Disabled

Triggered when telemetry reporting gets disabled. Last event sent before disabling telemetry. This event is not sent, if the user never enabled telemetry reporting before deactivating it.

Project Created

Triggered when a project is created using rasa init. Event properties:

• init_directory (string): Hash of the directory path the project is created in.