👋 Introducing the Rasa Playground

Modify pattern_clarification

post by Elliot94 on Sep 6, 2024

I would like to use the pattern_classification, but modify it to instead use an LLM call to paraphrase the user utterance. I can get this part done using a custom action, but what I would like to do is to actually use the paraphrased sentence to continue the command generation.

To make it more clear, I want to make it something like this:

user: Jimmy said he wanted some cash. send him a hundred
rasa: pattern_clarification
llm: Transfer hundred dollars to Jimmy
rasa: transfer_money_flow

post by jtrasa on Sep 6, 2024

This is an interesting idea, can you elaborate on the underlying problem you’re trying to solve? It sounds like you’re running into unexpected calls to pattern_clarification, is that correct? If so, your first step should be improving your flow descriptions.

post by Elliot94 on Sep 6, 2024

I have like super detailed flow descriptions that are a couple of lines themselves. I even added examples. My main issue is that I want this to be working with smaller LLMs. When using GPT-4o I got no issues. But something like phi-3 or gemma2 and it really struggles when the user input is a bit ambiguous. But when I tested with a bunch of examples I could think up, it always ended in pattern_classification. So my idea was to have a detailed set of instructions for the LLM to rephrase the user input, and then use that to identify which flow.

I could see how to implement this when using APIs, but any solution that can be used with inspect and shell would be much appreciated.