Help me make Documentation for Checkpoints better

The current documentation for Checkpoints does not define what checkpoints actually are. It mentions that you can “use them to modularize and simplify your training data” (sounds promising…), but also that "[they] should be used with caution, if at all”… hmmm… OK, so what are they?

I’ll read further…:

Checkpoints can be useful, but do not overuse them. Using lots of checkpoints can quickly make your example stories hard to understand, and will slow down training. Here is an example of stories that contain checkpoints:

Then it goes on with an example that does indeed use checkpoints. Looking at the example,

- checkpoint: check_asked_question

Reading the example, I can infer that maybe a checkpoint is a way to label a sequence of steps so they can be reused in multiple stories. But I would have to play around and test to figure out, and so would anyone else who was trying to learn about them. Questions:

1. How do I create a checkpoint?
2. How do I reference that checkpoint? (guessing that’s just using a step “checkpoint: abc”)
3. Are they officially deprecated in favor of Rules and Response Selectors?

If someone can tell me enough to answer these questions, I’m happy to update the documentation, to make the Rasa development experience a bit better for the next person.

Thanks!!

@TomV My thoughts on that is not to use much of the checkpoints, as it basic nature is to connect two stories together. It can be either the first or the last step in a story. Further, using a lots of checkpoints can quickly make your stories hard to understand. It makes sense to use them if a sequence of steps is repeated often in different stories, on the other hand stories without checkpoints are easier to read and write. Even, using lots of checkpoints can quickly make your example stories hard to understand, and even slow down your training.

I’d suggest not to use checkpoints, but If you want to strict your stories, you can go ahead. But, user can break the flow by giving an intent that is not expected. How you will deal that scenario? If the user says “Heya” two times then the flow breaks and the bot gets unpredictable and confused.

I hope you have seen this doc for creating checkpoints: Stories

Linking two stories: second story run after first story

- story: handle user affirm
  steps:
  - checkpoint: check_asked_question
  - intent: affirm
  - action: action_handle_affirmation
  - checkpoint: check_flow_finished

- story: handle user deny
  steps:
  - checkpoint: check_asked_question
  - intent: deny
  - action: action_handle_denial
  - checkpoint: check_flow_finished

@TomV I agree checkpoints are still this nebulous nugget that isn’t well defined in the docs. I learn more by looking at working code than by reading a lot of documentation. Now that I think about it, I haven’t noticed the use of checkpoints in any of Rasa’s examples.

I utilize checkpoints like the old GOTO statements in Basic. If I build a new story that utilizes the same steps I’ll break that out into a checkpoint I don’t feel their hard to follow, no harder than reading python so long as you name them to make sense.

To my knowledge there is no return back from a checkpoint so you can’t have one in the middle of a story. BUT you could have a story branch to call say two different checkpoints and the end of those 2 checkpoints call a new common checkpoint.

It would be a fun experiment to build 3 or 4 stories that all have a set of the same steps, then break out checkpoints to see the difference in training time and or actual accuracy when chatting.

@nik202 Thanks for your reply and the link to the docs. Unfortunately, it was those exact docs that are missing a clear definition of checkpoint, and more specifically how to create one.

I’m currently guessing (since there is no example or documentation on it) that to create a checkpoint is identical to creating a story (e.g., replace “story” with “checkpoint”, and you now have set of steps you can “include” in other stories.

My point is anyone reading the docs should not have to guess, it should be there in the docs.

@nik202 – I especially appreciate your example of why a checkpoint, because it is not flexible would break easily when given an unexpected intent (which seems to be the most likely intent)

Thanks!!

@jonathanpwheat Yes, you can pretty much turn any story (exact same format) into a checkpoint just by adding a - checkpoint: some_name as the first entry under steps: Mind you, you have to reference that checkpoint from somewhere or else I think you get an error, or at least a warning when training or validating.

Here’s an example from my general incident stories. These don’t all connect to each other in a straight line here (hopefully that’s not confusing), but they show how to structure calls to and from checkpoints.

version: "2.0"
stories:
  - story: General issue incident submission - Get More Information
    steps:
      - intent: triage_submit_incident_request_more_information
      - action: utter_tell_inform_thank_you
      - checkpoint: get_general_information

Here is what I call a “story fragment” (my term not Rasa’s) - because it starts with the checkpoint for the above story AND it calls a checkpoint at the end.

  - story: General Collect information and create an incident
    steps:
      - checkpoint: get_general_information
      - action: account_activation_followup_questions_form
      - slot_was_set:
        - specific_application: Outlook
      - active_loop: account_activation_followup_questions_form
      - active_loop: null
      - checkpoint: support_triage_ask_handoff_to_agent

Here are 2 more stories that link up to the ask about a hand-off or ticket

  - story: Account Activation lapse 30 to 90 days
    steps:
      - intent: account_lapse_90
      - action: utter_tell_account_lapse_30to90
      - action: utter_tell_account_reactivate_cant_resolve
      - checkpoint: support_triage_ask_handoff_to_agent

- story: Account Activation lapse greater than 90 days
    steps:
      - intent: account_lapse_90plus
      - action: utter_tell_account_lapse_greater_than_90
      - checkpoint: support_triage_ask_handoff_to_agent

These two checkpoints are where I “drain” a story and will integrate handing off to an agent or simply submitting a ticket

  - story: Support Triage Ask and Choose Handoff to support agent
    steps:
      - checkpoint: support_triage_ask_handoff_to_agent
      - action: utter_ask_triage_hand_off_to_agent
      - intent: triage_handoff_to_agent
      - action: utter_tell_triage_handoff_to_agent_connecting
      - action: utter_tell_triage_end_of_integration

- story: Support Triage Ask and Choose Incident Report
    steps:
      - checkpoint: support_triage_handoff_to_agent
      - action: utter_ask_triage_hand_off_to_agent
      - intent: triage_submit_incident_request
      - action: utter_tell_triage_submit_incident_request
      - action: utter_tell_triage_end_of_integration

So those examples cover all the different ways to use a checkpoint that I know of anyway.

Hope that helps!