Recommended
Recommended
More Related Content
Similar to Train to Tame: Improve Communications Apps with TensorFlow
Similar to Train to Tame: Improve Communications Apps with TensorFlow (20)
More from Elizabeth (Lizzie) Siegle
More from Elizabeth (Lizzie) Siegle (20)
Recently uploaded
Recently uploaded (20)
Train to Tame: Improve Communications Apps with TensorFlow
- 1. Train to Tame Toxicity: Improve Communications Apps with TensorFlow Project Voice Lizzie Siegle
- 3. • Developer evangelist @ Twilio • ML enthusiast @lizziepika
- 6. @lizziepika
- 8. @lizziepika
- 10. @lizziepika
- 12. @lizziepika
- 13. @lizziepika
- 14. @lizziepika
- 15. @lizziepika Never fear, solutions are here! Solutions
- 18. @lizziepika • OS library • Image classification, NLP, generate text/music/etc! • Python, JS, Swift, C, etc.
- 19. Speech Command Recognizer Warden, P. (2018) "Speech commands: A dataset for limited-vocabulary speech recognition" https://arxiv.org/pdf/1804.03209.pdf
- 20. • Flexible • Ecosystem • Support Why TensorFlow? @lizziepika
- 21. TensorFlow + Toxicity • Toxicity model to detect toxic language • Node.js module @lizziepika
- 22. @lizziepika
Editor's Notes
- For anyone and everyone (click)
- As Alanis Morisette almost said, isn’t it ironic that I work at a communications company and still think communication is hard? I develop and teach web, iOS, and ML solutions for Twilio, a communications company that enables developers to send/receive messages, phone calls, emails, and more with code. So what’s a developer-centric solution to the communication Q? (click)
- How about machine learning? Let’s go over 5 ways ML can make communication easier
- #1: IVRs. You could have a DTMF keypad IVR (as shown on left) but that’s limiting. (click) This conversational IVR (on the right) made with Twilio Autopilot, a bot-building platform where you build once and then deploy across multiple channels like phone calls, SMS, Messenger, Alexa, and more. (click)
- #2: Personal assistants are one use case you may use daily or often: ML-based personal assistants like Siri, Alexa assist with info/answering questions like providing directions or turning on lights helping make life easier and more entertaining. Many chatbots now use ML to better serve people in a more seamless experience, often using voice recognition/search which leads us to another ML-in-communication use case
- #3: Recognition: image recognition, video recognition, voice/audio recognition. These are used by Alexa and Siri, but also businesses. You can search images for faces or objects, like this image recognition via MMS hack that is detecting pizza here. On the other hand, video chat is more for real-time, like if you want facial recognition quickly, tennis shot detection, etc.
- #4: Who hasn’t gotten an email from a Nigerian prince? Help solve that problem and save $ and time by detecting spam, fraud/low-quality content in emails, messages, phone calls, and more.
- #5: Lastly for this list of ML use cases for communication (which is by no means a complete list): Text analysis. Can analyze sentiment (ie. Is something +, -, or neutral), entities (parse text for nouns, phrases, important key words), and more, all falling under Natural Language Processing (NLP.) Text summarization, real-time translation, language detection in voice apps as well as text-based app. This could also similarly be applied to recommendations.
- Here we have an example of performing sentiment analysis and entity analysis on a real-time transcription of a phone call. Great for summarizing phone calls and seeing how it is going in real-time. Is the customer happy? Sad? What is the general gist of it? See how it parses out “DNA” and “DNA Test” in Truth Hurts by Lizzo from “I just took a DNA Test”
- The primary topic this talk will be tackling…toxicity, using a ready-made, already-trained TensorFlow model that is open-sourced online! You can see the code right there!
- Has anyone here had a toxic experience on the internet? (click)
- *raises hand* I know I have. From Hacker News to sharing some technical blog posts on some subReddits…phew!
- How can we make the internet less toxic? More inclusive? (click) We can’t censor speech and kill comments necessarily. (click) Have we given up? Have the internet trolls won?
- Never fear, lots of solutions for addressing internet toxicity are here!
- Bot with an algorithm to find negative messages (click) Research on deep learning methods to detect abusive comments (click) Deep learning, text classification, and NLP! All things we’ve talked about! (click) Chrome extension Tune filtering out bad comments
- TensorFlow is one popular vehicle for pursuing or accomplishing some of these solutions. (click) Though a recruiter may think it is a Pokemon, I assure you it is not *smile*
- Started out as library developed by group of researchers + engineers on Google Brain within Google AI org. OS => more support from community, contributors outside existing core team Companies such as NVIDIA, Twitter, Snapchat, and more using TensorFlow in prod Something for everyone: beginners to researchers/PhDs TF Lite for IoT devices/mobile+embedded ML,TensorBoard for visual debugging,TFHub for reusable ML, TF.js for the web It’s now not a library, it’s an ecosystem
- TensorFlow for voice: ready-made, pre-trained model whose underlying deep neural network trained using the TensorFlow Speech Commands Dataset. Online and offline streaming recognition
- Very flexible, easy to customize More libraries+tools built on top of it, broadening TF ecosystem ie Keras, high-level neural network library giving you the tools+primitives needed to implement complicated or not-complicated architectures across languages+platforms TF has better docs, more tutorials, huge community, lots of resources (Google $), marketing
- Model trained on civil comments dataset, containing ~2 million comments labeled for toxicity. 7 possible labels: identity_attack (identity-based hate), insult, obscene, sexual_explicit, threat, toxicity, and more
- This is what we’ll be creating: a chat app that detects toxic/not-nice language and prevents it from being posted to the channel If live-coding demo fails, you know it worked at one point
- Already done some setup: I have this Twilio chat demo app Clone it on the command line
- You will need these credentials which you can find in the README of the chat demo app.
- Now you, too, can call yourself a Machine Learning researcher.