## Why testing your assistant? For more information E2E testing, see the [E2E testing product documentation](/content/docs/pro/testing/evaluating-assistant/index.html). To write test cases, you need to create a YAML file inside the `tests` directory of your project. The name of the file should be `e2e_test_cases.yml`. You can also create a subdirectory inside the `tests` directory and place your test case YAML files there. These files will be automatically discovered and run by Rasa, however, you need to provide the path to the subdirectory as a positional argument to the `rasa test e2e` command. Each input file must contain the `test_cases` key. The value of this key is a list of test cases. Each test case must include a name given to the `test_case` key and a list of test steps given to the `steps` key. A step can be either one of the following: - `user`: refers to a user's message. - `bot`: denotes a textual response generated by the bot. - `utter`: refers to a bot response name as defined in the domain file. - `slot_was_set`: indicates the successful setting of a slot, and depending on how it's defined: - if a slot name is provided, e.g. `my_slot`: checks that a slot with the given name is set. - if a key-value pair is provided, e.g. `my_slot: value`: checks that a slot with the given name is set with the expected value. - `slot_was_not_set`: specifies the failure to set a slot, and depending on the definition: - if a slot name is provided, e.g. `my_slot`: checks that a slot with the given name is not set. - if a key-value pair is provided, e.g. `my_slot: value`: checks that a slot with the given name is not set or that its value differs from the expected value. The following example illustrates how `user`, `bot`, `utter`, and `slot_was_set` steps can be used in a test case: ```yaml tests/e2e_test_cases.yml test_cases: - test_case: user books a restaurant # name of the test case must be provided and be unique steps: - user: I want to book a table for 4 at Xaigon for 8pm tonight - slot_was_set: - book_restaurant_name_of_restaurant: Xaigon - book_restaurant_number_of_people: "4" - book_restaurant_date: tonight - book_restaurant_time: 8pm - utter: utter_restaurant_available - utter: utter_ask_book_restaurant_reservation_name - user: Emil - slot_was_set: - book_restaurant_reservation_name: Emil - utter: utter_ask_book_restaurant_confirmation - user: yes - bot: Thank you for booking the table at Xaigon for 4 people at 8pm tonight. ``` In each of the steps after the `user` step, you can specify multiple expected events. Any additional events that are found are ignored and do not cause the test to fail. ## Slots Slots can be specified as a list of either string values (representing the slot name) or of slot name and slot value pairs. If the slot is specified as a key-value pair, the values are also compared. If the slot step contains only the slot name, it is asserted that the slot was either set (when using `slot_was_set` step) or not (when using `slot_was_not_set` step). ```yaml tests/e2e_test_cases.yml - slot_was_set: - book_restaurant_name_of_restaurant: Xaigon - book_restaurant_number_of_people: "4" - book_restaurant_time - slot_was_not_set: - book_restaurant_reservation_name - book_restaurant_reservation_date: "2023-11-11" ``` It is not required to specify all the slots events in the `slot_was_set` or `slot_was_not_set` steps, you can indicate only a subset of slots that you want to check. You can think of `slot_was_not_set` as an inverse of `slot_was_set` and when specifying a `slot_was_not_set` step it looks for the absence of the `SlotSet` event in the tracker store but only for that particular `user` step, not globally. ## Fixtures for Pre-Filled Slots Using fixtures is an optional feature that enables the pre-filling of slots, ensuring specific context before individual test cases are run. You can define fixtures in either: - **Test case files** — using the `fixtures` key at the top level of a test case YAML file (e.g. `e2e_test_cases.yml`). - **Conftest files** — using the same `fixtures` key in a `conftest.yml` or `conftest.yaml` file at the root of your test tree or in any subfolder. Within a single file, each fixture name must be unique. Across files, the same fixture name can be used to override a fixture from a parent scope. Fixture names correspond to sets of slot key-value pairs. When a particular test case needs predefined slot values, you can reference the fixture name within the test case definition by adding it to the `fixtures` key. Consider the following example, which includes a test case file with fixtures and two test cases that leverage these fixtures: ```yaml tests/fixture-tests.yml fixtures: - premium: - membership_type: premium - logged_in: True - standard: - logged_in: False - membership_type: standard test_cases: - test_case: "test_premium_booking" fixtures: - premium steps: - user: "Hi!" - bot: "Welcome back! How can I help you?" - user: "I want to book a trip." - utter: utter_ask_location - user: "I would like to travel to Lisbon." - slot_was_set: - location: "Lisbon" - utter: utter_ask_date - user: "I would like to travel on 22nd of June." - slot_was_set: - travel_date: "2023-06-22" - bot: "Great! I will book your trip to Lisbon on 22nd of June." - bot: "You saved 20% by being a premium member." - test_case: "test_anonymous_booking" fixtures: - standard steps: - user: "Hi!" - bot: "Hey! How can I help you?" - user: "I want to book a trip." - utter: utter_ask_location - user: "I would like to travel to Paris." - slot_was_set: - location: "Paris" - utter: utter_ask_date - user: "I would like to travel on 2nd of April." - slot_was_set: - travel_date: "2023-04-02" - bot: "Great! I will book your trip to Paris on 2nd of April." - bot: "You can also choose to save 20% by becoming a premium member." ``` These slots in fixtures are set after the `action_session_start` action and before the first step is executed by the test runner. ### Conftest-style fixture hierarchy You can define fixtures in **conftest** files (`conftest.yml` or `conftest.yaml`) so that all e2e tests under a given path see them, without repeating definitions in every test file. Fixtures defined in `tests/conftest.yml` are visible to all test files under `tests/`.