Crowd-Powered Dialog
•Download as PPTX, PDF•
0 likes•250 views
Panel Presentation from SIGDIAL 2016 on the Future of Intelligent Personal Assistants.
Recommended
More Related Content
Recently uploaded
Recently uploaded (20)
Featured
Featured (20)
Crowd-Powered Dialog
- 7. talkingtothecrowd.org So what? • Learning what we don’t know? • Data-driven improvement • Hybrid Workflows
- 8. talkingtothecrowd.org Hybrid Workflows Guardian: A Crowd-Powered Spoken Dialog System for Web APIs. HCOMP 2015. InstructableCrowd: Creating IF-THEN Rules via Conversations with the Crowd. In Submission.
- 9. talkingtothecrowd.org What is new? Facebook[m] “Call Center” courtesy of BBC.
Editor's Notes
- Hi everyone, my name is Jeff Bigham, and I’m from Carnegie Mellon University, where I’m in the human-computer interaction institute and the language technologies institutes. I’m here because my group and I have been creating and deploying crowd-powered dialog systems over the past few years. Over the past year or two, we’ve been working to gradually automate these systems. I call this a “top down” approach, start with the working crowd-powered system and then gradually automate it. It’s kind of a reverse of what we often do in computer science.
- I was asked to address this question of where we are, current state of the world. People are interacting with technology via speech, specifically they’re interacting with their phones and other devices using speech to perform specific functions that they’ve learned to work well on their devices. I was browsing my local library a while back, and I found this book that I think summarizes this state of affairs reasonably well:
- The book is called “Talking to SIRI” and it’s a comprehensive look at what one could say to Siri about a year ago. It’s a reasonably large book, about ¾ of an inch thick. It’s out of date now. Maybe it would need to be an inch now. But, regardless, we are at a stage where we’re teaching people what are systems can do, and how they can ask those systems to do things for them in ways that those systems can understand them.
- In response to this, my group and I created a “crowd-powered system” for conversational assistance. The idea is that you ask it whatever you’d like to know, and a group of workers decides how to respond within a minute or two, often faster. Workers propose responses, others vote. Workers stay around as long as they can, and a memory space helps keep the system robust over multiple turns, even as individual crowd workers leave. It’s not perfect, but it has human intelligence behind it so it’s pretty good, very robust.
- You too can try it, go to talkingwiththecrowd.org, and sign up, and you can chat to it from any Google Hangouts enabled device. Even though you know it’s people, you’ll likely be surprised at its breadth and robustness.
- So, you might be thinking, so what… people can talk to people, that’s the most obvious thing in the world. And, you’re mostly right. There are some challenges when we introduce an imperfect crowd that’s only working on our task for a short time before leaving, if you’ve worked with Mecahnical Turk before you might be moderately impressed, but for the most part that’s right.
- Hybrid workflows afford us a different path toward automation, which is automating the deployed crowd-powered system over time. We’ve been creating system that have the crowd working with automation. Guardian is a system where mostly non-expert workers from Mechanical Turk turn the APIs on programmable web.com into little dialog systems, which they then helped to run. The real-time crowd does slot filling, transitions states, and formulates responses. InstructableCrowd has the crowd help formulate IF-THEN rules for the user’s phone to operate. For instance, if it snows overnight, wake me a little earlier so I’ll have time to sweep off my car, … or remind me of my next meeting, but only if it’s in a different place. The crowd can help turn my natural language description into actionable rules that can run on my phone. We’ve thus been able to release conversational systems powered by the crowd that allow data collection, and the goal of this is to support learning at a lower level that is likely to be easier to incorporate into automated systems.
- The idea of using people along with your automated assistant is not new. Most large software companies and many startups have efforts in this space. Sometimes it’s very explicit, such as in the case of Facebook[m]. Other times, human input comes from a team of engineers who, for instance, decide which new templates to incorporate into Siri. If Apple can get ahead by engineering more and more templates, and Alexa can crowdsource more and more skills from its growing community of developers, is that any different than having the crowd do it? I think the difference is that in our research we’ve embraced human input. We make it explicit, we attempt to scale it, and we run deployed systems using it. As automation continues its difficult journey building systems capable of rich and robust dialog from the ground up, we believe that we might be able to speed things up by building systems capable of rich and robust dialog from the top down.