Video and Slides
Outline: Pitfalls and potholes of content moderation for chatbots
Elayne Ruane, PhD researcher in QA of Conversational AI at LERO Centre
- Chatbots can provide a fast and convenient experience to customers who need to solve a problem, complete a task, or get some information.
- The promise of the speed and availability of a machine combined with the conversation and accessibility of a human is an attractive solution.
- But what about when chatbots fail to live up to those expectations, the user gets frustrated, and the conversation gets heated?
- Abusive messages from users towards chatbots are not uncommon and moderation efforts are fraught with unintended consequences.
- This presentation will discuss approaches to content moderation for chatbots, common pitfalls, and some recommendations for handling abusive messages.
You can ask Elayne any questions about this presentation in the comments section of this weblog, or contact Elayne directly with the info at the end of the presentation.
Moderation is more than a social media chat room problem. Its an issue for any brand using a chatbot to engage with its customers and prospects. Google has a team dedicated to it within their AI Ethics.
Chat can be voice or text, as voice will be converted to text. Though with ASR (Automatic Speech Recognition) care needs to be taken as an innocent regional pronunciation could be converted into a profanity.
Moderation is highly application specific, as some words may be permissible in some context e.g. in healthcare or education, that would not be expected in general customer service.
Elayne works on bots, especially in customer service; abusive messages come with the territory, they are unavoidable. Even abuse directed at the people creating the bot. Its born out of customer frustration. A critical point Elayne makes is the decisions a brand makes around moderation have a significant impact of how customers perceive the brand and in protecting the user.
What is offensive is much more than covered by profanity filters. Its ideas, and is subject to regional and cultural variations. Elayne provides excellent examples of the impact of unsupervised training, and how some simple filters have unintended consequences. The critical point Elayne states is do not create the bot in a vacuum, use a diverse team to help define what is offensive / abusive language for your specific application and brand.
This is just leading up to identifying abuse. Next comes how to deal with abuse. Is it a non-response, deflection, no response (silence), informing, reporting, escalating to a live agent. Interestingly, bots in general and bots with feminine voices receive substantially more abuse than live agents. Elayne provides lots of interesting examples of unintended consequences.
The cost of a false positive can be high for both the brand and the customer/user. Even if you choose to constrain the bot space to only respond to specific intents with a limited vocabulary, the training data generated still needs work. And remember those chatbot interactions are public, they can and will be recorded, and potentially make their way onto social media.
Elayne provides excellent advice on managing moderation:
- Thoughtful design is important (legally, morally, commercially…)
- Protecting the user is #1 – give users recourse and the benefit of the doubt
- Your chatbot is just ones and zeros but your team are people!
Thank you Elayne for an insightful presentation on Chatbot Moderation. I hope you’ve raised the TADSummit community’s awareness of this critically important and under-discussed topic.