In an earlier blog post, we described the pilot with a chatbot on the NPO Radio 5 website. On all programme pages of the website, there was a chat icon available that invited people to ask questions about programmes, events, music, vote on a poll, or just start a chat. In this blog post, you can find an overview on our learnings and insights.
Setup and initial insights
Total amount of users during the pilot phase
The pilot took place between the 26th of June and the 4th of August, 2019. In total, more than 700 unique visitors used the chatbot and almost 1900 user messages were sent. This means that on average, one user sent 2 to 3 messages. Questions about presenters, such as 'Tell me more about Jeroen van Inkel?', programmes, such as 'When is Arbeidsvitaminen on air?', and songs, such as 'Which song was played?', were generally answered quite well. We added an automatic connection with the editorial programme, where information about shows, playlists and DJs is managed by an editorial team.
Examples of succesful conversations
Very specific questions about a guest in a show or a location of a festival were not understood by the chatbot. In order to still help those users, we introduced a temporary functionality halfway. When the chatbot did not understand a question, it would ask the user if they were willing to leave their email address behind, so that the editor could answer the question by email in a later stage. The entire conversation would then be sent to the editorial team, so that they were still able to help the user. About 30 users actually did this.
What was striking is that a lot of users did not have a specific question, but just wanted to leave a comment or remark. We did not train the chatbot well enough however, to respond on these in a proper manner.
We also included a poll in the chatbot, with which users could vote on their favorite song. In total, 58 different polls were created and around 1600 votes were counted. We also asked people to create an account, which 27 users did. This is not a lot however, which could be the result of two reasons:
The questions were tucked away pretty deep in the conversation
It was not clear what the gain of an account was (what's in it for me?)
We also offered an online survey at the end of conversations, but unfortunately only 9 users filled this out, so this is not really representative. One of the main outcomes was that users felt unsatisfied if a question was not understood or answered well.
Main learnings & tips for implementing a chatbot
Start small with a simple conversation and elaborate step-by-step.
It is labor-intensive work to prepare the chatbot, and monitor and adjust it at the same time, so you need enough editorial capacity.
It is quite a steep learning curve from how to create the chatbot, so you need a professional or a specialised company, which was in our case consortium partner Faktion.
You need to make it very clear to users what they can expect, and guide them in using the chatbot in a right manner.
Focus on frequenly asked questions.
If you have a connection with an editorial system, then make sure this is up-to-date and as complete as possible (with programme information, DJ bio's, etc).
The chatbot seems to be very useful for deployment at events, in order to answer frequenly asked questions (when is the event, how can I get there, etc).
Compared to the regular amount of user questions via email, there were quite a lot of chat conversations. It looks like users find it quite interesting to use, or an easy way to make contact.
Interested in also implementing a chatbot in your radio station - with free support from MARCONI? Then join the open pilots and contact us on contact@projectmarconi.eu
