Streaming response for custom actions - Rasa CALM - Rasa Community Forum
👋 Introducing the Rasa Playground
Streaming response for custom actions
post by avatar on Mar 19, 2024
I was thinking of using openai api within a custom action and stream the streaming response coming from openai to my chatbot. Openai is giving me streaming response and i am doing “dispatcher.utter_message(openai_res_chunk)” from actions, but the response is not getting streamed in chunks to my bot and is available only after last chunk from openai. I am also passing “stream”: True while sending request to rasa server, but nothing is working. How to achieve this?
post by jtrasa on Mar 25, 2024
Out of the box, the SDK CollectingDispatcher does not support streaming (see source code here).
Given that Rasa has an open source offering, you could extend the SDK to add streaming, however I don’t have any guidance for you on how to do so.
post by saimadib on Jan 14, 2025
Is there any way to stream the response?