👋 Introducing the Rasa Playground

Rasa UI / Rasa Platform

post by Spen on Mar 25, 2019

Intents, entities, synonyms, stories and domain get unmanageable really fast. Directly editing the respective files leads most definitely to mistakes that mostly stay unnoticed. We currently see no other way than to fully abstract all Rasa NLU and Rasa Core related stuff with a single UI. The only two components in the development are then the Rasa UI and the action-server.

We were looking for such a Rasa UI.

What the Rasa UI should provide:

• Web-based UI
• View to edit intents, entities, synonyms, stories and domain (train and test data)
• Example chatbot interface to test bot
• UI for interactive training (the console version is hardly usable and very error prone)
• Automatic version management (sync training and testing data with Git repository)
• Launch retraining via UI
• Track user conversations (filterable for low confidence)
• Easy way to directly annotate and add training data from user conversations
• Show analytics about real live usage (count usage of intents, actions, …)
• Show historical real live performance (count of fallbacks, …)
• Show historical model performance on test data (nlu, core, e2e) tagged by versions
• Optional: User management

What UI components we found (updated):

• paschmann - Rasa UI: Looked promising but was very disappointing when we tried it. A lot of functionality that seemed to be there doesn’t actually work.
• rasa-nlu-trainer: Way too limited to be usable.
• Articulate: Also looked very promising at first but revealed to be a standalone solution that only uses Rasa NLU in the background rather than being an UI frontend for Rasa. Tries to be a standalone solution that competes with Rasa (their solution deviates in main parts from Rasa and they plan to release their own “core” solution competing with Rasa Core).
• Botgen: Only Rasa NLU and no on-premise offering.
• Rasa Platform: The Rasa Platform provides most of the functionality we need. Missing are analytics for failed conversations and statistics on test performance over versions. Also the UI is currently bundled with a rather expensive service subscription.

post by huberrom on Mar 25, 2019

Hey, I’m working with and on rasa ui, and I know it’s not perfect, but which functionality doesn’t work for you ? In your list :

What the Rasa UI should provide:

• Web-based UI -> Yes
• View to edit intents, entities, synonyms, stories and domain (train and test data) -> Yes for the NLU part (doesn’t handle core)
• Example chatbot interface to test bot -> Yes
• UI for interactive training (the console version is hardly usable and very error prone) -> No but doesn’t say anywhere it does. Would be great tho
• Automatic version management (sync training and testing data with Git repository) -> No
• Launch retraining via UI -> Yes
• Track user conversations (filterable for low confidence) -> Yes, and the “filter” is coming this week (working on it)
• Easy way to directly annotate and add training data from user conversations -> Yes
• Show analytics about real live usage (count usage of intents, actions, …) -> Yes (for the intents part)
• Show historical real live performance (count of fallbacks, …) -> Show no matches
• Show historical model performance on test data (nlu, core, e2e) tagged by versions -> No (would be great too)
• Optional: User management -> No

post by Spen on Mar 25, 2019

Hi Romain, to clarify, what rasa ui are you referring to?

post by huberrom on Mar 25, 2019

Sorry my bad, Rasa UI from paschmann.

post by Spen on Mar 25, 2019

Thanks for the clarification.
No. On a surface level it seems paschmann rasa ui provides many of these functionalities but actually it is just a facade. On GitHub I just asked how I can use the ui for an existing agent and got the following reply (shortened):

currently rasa-ui supports importing an agent only for nlu data in json format […]

That small sentence already contains so many limitations that it makes the UI useless for us.

• Editing data via imports and exports is unacceptable - files should directly be changed.
• What is the point in the UI if the core data is not included? For us none.
• Our agent uses .md files.

The main README of the project promises many functionalities that the actual UI just doesn’t have. The UI even has fake / placeholder fields that just don’t provide any functionality (for example stories and actions). I directly stumbled upon many bugs. This ui is a real mess and more fake than that it actually provides anything useful. We will definitely stay away from something like that.

Also the Yes to many of the above listed functionalities is far-fetched. Don’t get me wrong, I don’t want to badmouth paschmann rasa ui. Just everything I saw after actually testing it was devastating. I really much disliked that there are functionalities promised that are just not there. This will keep me from looking at this project in the future.

I see that paschmann rasa ui is regularly promoted on this forum. I wish to actually discuss the official Rasa Platform UI here and to get some clarification from the Rasa Team about the state of their UI. The official information on the website is way too sparse and I really wish they extensively show (with images) on their website what the UI can do.

post by huberrom on Mar 25, 2019

Yes, of course if you need to handle the core, Rasa UI is not enough for you.

We faced the same problem as you did regarding the lack of information about the Rasa platform.

We contacted the sales team and we had a call with a quick demo of the platform, I don’t think you’ll have more information on the forum. Maybe the demo is the best for you if you want to know everything about the platform.

post by wrathagom on Apr 23, 2019

Sorry to dredge up an old post here, but we’ve gotten several questions referencing this post. I am the principal designer behind Articulate and wanted to respond here.

Sebastian actually created an issue on Articulate’s repo and I have been lax in responding there as well . But I will make it a priority/copy some of this answer there as well.

For some background here is the post from when we launched Articulate. We’ve been around the Rasa community for quite some time now and though we’re not as active now we were at one time.

We have chosen, as of yet, not to implement Core inside of Articulate, but rather rely on a more traditional rule based system. Our rule based system manages context, slots, lifespans, etc much like Dialogflow does. (Our initial chatbots were built on API.ai). We’ve been watching Core closely waiting for the time to be right to bring it into Articulate.

We’ve had specific questions on a few pieces of your statement:

• “closed solution” - There is nothing about Articulate that is closed source. The entire thing is on Github and is Apache 2.0 licensed.
• “competes with Rasa” - We can’t compete with Rasa on new science and we won’t try to. What they are doing is amazing and cutting edge. We do hope that we can develop Articulate to the point where it can allow more people (non developers/CLI) to build with Rasa NLU. And in the end we build/use/support it because we need it for our own development tasks.
• “release their own “core” solution” - On our roadmap is the development of a system, which may be core based, that allows rules and models to work together:

In my response to your issue I wrote, “We have some ambitions to implement a core like approach in the future”. which was confusing. At the moment what we would like to achieve is not replacing our dialogue engine with Core, but rather use our rules to bootstrap stories, which in turn enable Core. The marketing person in me wants to call this “ML handoff” where stories are generated from real conversations. With the hope that at some point in time Core is in full control and the rules fade away.

post by maelezzabdi on May 3, 2019

Hi Stephane,

I am Maël, co-founder of Botgen, a NLP platform based on top of RASA NLU. We have just released the V1 of our platform. We might not check all your needs but I would be very interested to have your feedback:

• Website: https://botgenplatform.com/
• Demo video (we have added some features in between): - YouTube
• Documentation: https://botgenplatform.com/documentation-eng/

You can join me at: mael.ezzabdi@botgenplatform.com

Do not hesitate
Best

post by Spen on May 6, 2019

To us this is by no means an old post. For us, a proper Rasa UI that fits our criteria is still non-existent. Until there is some solution that fits most of our above mentioned criteria, this thread is here to update and discuss other / new or updated UI tools for the Rasa Stack.

Regarding your questions:

“closed solution” - There is nothing about Articulate that is closed source.

I wasn’t referring to the licensing of your solution. I meant to point out that Articulate is a standalone solution. We are looking for a UI that can accompany an existing Rasa Stack and only acts as a UI and not a standalone full solution.

“competes with Rasa” - We can’t compete with Rasa on new science and we won’t try to.

We might have a different perspective on this but I respect that this is how you want your product to be marketed.

As I pointed out in the aforementioned GitHub issue, Articulate uses very different terminology compared to Rasa. That definitely adds to the perception that Articulate is a standalone solution that only uses Rasa NLU in the background rather than an UI for working with the Rasa Stack.

post by Spen on May 6, 2019

Hi Maël,

good to see new solutions.

Ignoring the missing functionality compared to our criteria, I could not find an on-premise offering of your solution.

post by Spen on May 6, 2019

Today @Dominik presented me the Rasa Platform. I hope it is fine if I share some impressions about what I saw.

While being different from what we envisioned the Rasa Platform UI ticks most of the boxes. The UI provides a view where all conversations are listed, also including a chat window to test the bot. From these windows, it is possible to flag messages for other developers for further investigation. It is also possible to directly edit intents, annotated sentences and save the stories of the current conversation. Thereby this also covers the interactive training. It also makes it very simple to add new training data from real conversations (all within the UI).

The UI also has views to edit training data, stories, utter messages, and the model configuration. In addition, there is a view that lists all user input that is not within the training data.

The model can be retrained with a single button press within the UI. Retraining a model will lead to new model versions shown in an extra view. It is possible to set which of these versions should run in production and development. Versions can be archived as well and contain the training data of that version.

An analytics view shows some stats on the current usage (sessions, users, messages, …) but has no stats on the usage of intent nor actions. Stats on real live performance in terms of failed conversations (fallbacks or similar) are currently missing. Also stats on test and training performance are currently not represented in the UI.

User management exists.

Dominik promised that some features are in the works but I’m not sure what I’m allowed to share so I will leave it with that. I hope I got everything right and that this impression might help somebody else who is looking for information on the Rasa Platform UI.

post by luofanghao on Sep 20, 2019

found another one called botfront: GitHub - botfront/botfront: An open source chatbot platform built with Rasa

post by Beherasaptami on Apr 28, 2020

hey @huberrom any updates on paschman UI i tried that and i’m facing some issues like i am unable to initiate chat inside UI but if i am trying to run with custom widget then it’s working fine and any suggestions like how can i load multiple models at the same time in other words is there any possible way with pashman repository where multibots can work.

post by Johnny_Am on Jun 14, 2020

Hello all, I’m still new to the Rasa community but I’m interested in the topic of this thread, I’m actually trying to implement a UI mainly for the stories.md in order to introduce new scenarios in a better way. I read about something called Rasa Platform UI? Is this the best among the mentioned? What would you recommend as a basis for my project? The stories.md will be written using nodejs. So the UI would have to edit that file. Thanks for the help