👋 Introducing the Rasa Playground

Playing audio clips as RASA response

post by athenasaurav on Aug 10, 2020

Hello Everyone,

I want to play an audio file as a response to rasa chatbot used in an audio chatbot connected to a phone bot. Is it possible to autoplay an audio or video file. I know how to play a video link in rasa chatbot using embed component but it is not autoplaying.

post by rctatman on Aug 12, 2020

Hmm, that sounds like something that would vary quite a bit by front end. The SocketIOutput method in the custom connector from this blog post on using Mozilla DeepSpeech & text to speech might be a good starting point.

post by athenasaurav on Aug 14, 2020

Hello @rctatman,

Thank You for your response. I have already seen that and implemented it. For AI voice clips I have thousands of recording to play the voice note. My issue is mentioned here.

On my windows system I used this to make utterance worked for autoplay :

  utter_recording:
  - custom:
    attachment:
      type: video
      payload:
        src: fullpath/filename.mp4
      autoplay: true

It worked perfectly fine. But when I tried doing the same on GCP like this :

  utter_recording:
  - custom:
    attachment:
      type: video
      payload:
        src: /etc/rasa/files/video.mp4
      autoplay: true

I get this error in webchat widget :

Any luck finding solution either autoplay videos via link or local files on GCP will be very helpful.

post by rctatman on Aug 14, 2020

Hm, gotcha. Since it’s working locally it looks like it might be a problem with the file path, or possibly a permissions thing? I might ping someone at GCP and see if they can’t provide some more insight.

post by athenasaurav on Aug 15, 2020

That will be great.

post by athenasaurav on Aug 17, 2020

Hello @rctatman. Any updates regarding my Question?

post by rctatman on Aug 17, 2020

Ah, sorry, I realize what I said was a bit ambiguous: I’d recommend that you ask someone from GCP about this. If you tag a StackOverflow question they’re usually pretty responsive.

post by Fares on Aug 26, 2021

Hi @rctatman , please I’m looking for how can rasa understand a voice-message sent with Facebook or telegram?

post by athenasaurav on Aug 27, 2021

Hello @Fares ,

Are you looking for Voice response from the bot or the user send voice response to RASA bot.

In case you are looking for the later one i.e sending voice message from Facebook or telegram to the RASA bot, you need to take it as a payload and send it to some Speech to text (STT) API (for ex Google or IBM) and then feed the response of STT API to rasa Bot NLU to get a response from RASA.

post by Fares on Aug 27, 2021

Thanks for your response. Yes, I’m looking for the user send voice response to RASA bot. In fact how can I send the voice message from Facebook to STT API? and then get the text from these API to send it to RASA BOT? I’m really so confused

post by athenasaurav on Aug 28, 2021

I think you need to write a custom action to create text to be passed to rasa and then use the converted text.

Here this link will help you on how to send speech file to google to get converted into text.

post by najaf on Feb 22, 2024

Custom json:

{
  "attachment": {
    "type": "audio",
    "payload": {
      "src": "audio/Hi.mp3"
    },
    "autoplay": true
  }
}

that what I am getting

post by anitatukci on Feb 26

Rasa itself only sends the response payload — autoplay depends on the client (web widget or phone integration). Browsers usually block autoplay with sound unless the user has interacted first. For phone bots (e.g., Twilio), you should return audio using the proper telephony markup so it plays automatically. For web chat, you’ll need custom frontend JS to trigger playback. If your source is a video, you could also extract content first using tools like get youtube transcript