Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Smarter & Cuter Bots

259 views

Published on

Presented at Web Unleashed 2017
More info at www.fitc.ca/webu

Rachel White
Microsoft
Overview

What if we made bots cuter and more intelligent? Instead of using text as fodder for this bot, Rachel will show you how to use Twitter’s API, ImageMagick, and Microsoft’s Face API to manipulate selfies with cute overlays in a simple and easy to use node twitter bot.

Objective

Learn how to use machine learning apis to build a twitter bot

Target Audience

Anyone interested in using Node.js to build a twitter bot

Assumed Audience Knowledge

Node.js

Four Things Audience Members Will Learn

REST APIs
Twitter bots
Cognitive services
Twitter APIs

Published in: Internet
  • Be the first to comment

  • Be the first to like this

Smarter & Cuter Bots

  1. 1. Smarter & Cuter Bots
  2. 2. About Me! Rachel White @ohhoe Technical Evangelist @ Microsoft I love cats I love cute things I love twitter bots! Smarter & Cuter Bots -
  3. 3. About Me! RoboKitty – Automated Cat Feeder https://github.com/rachelnicole/robokitty Smarter & Cuter Bots -
  4. 4. About Me! Glitch Art! Smarter & Cuter Bots -
  5. 5. About Me! Artist Residency @ Pioneer Works Smarter & Cuter Bots -
  6. 6. About Me! Video Games Smarter & Cuter Bots -
  7. 7. About Me! Cute Twitter Bots Smarter & Cuter Bots -
  8. 8. Notable Bot Types Machine Learning Markov Chain Corpus Fed Generative Image Bots Just Plain ~ Weird ~ Smarter & Cuter Bots -
  9. 9. Machine Learning What is it? Smarter & Cuter Bots - “Machine learning is the subfield of computer science that gives computers the ability to learn without being explicitly programmed .” Arthur Samuel, 1959
  10. 10. Machine Learning What is it? Smarter & Cuter Bots -
  11. 11. Machine Learning Smarter & Cuter Bots - Within the field of data analytics, machine learning is a method used to devise complex models and algorithms that lend themselves to prediction; in commercial use, this is known as predictive analytics. Source: https://en.wikipedia.org/wiki/Machine_learning
  12. 12. Machine Learning Smarter & Cuter Bots - These analytical models allow researchers, data scientists, engineers, and analysts to "produce reliable, repeatable decisions and results" and uncover "hidden insights" through learning from historical relationships and trends in the data. Source: https://en.wikipedia.org/wiki/Machine_learning
  13. 13. Source: http://freakonometrics.hypotheses.org/20274
  14. 14. Machine Learning Smarter & Cuter Bots - Computer Vision, Predictive Text, Emotional Analysis, Facial Detection, Deep Learning, Neural Networks, etc etc etc etc and so on
  15. 15. Machine Learning Smarter & Cuter Bots - There’s a lot. But they’re still trained models, they still require some sort of input from a human being.
  16. 16. Natural Language Processing Smarter & Cuter Bots - Modern NLP algorithms are based on machine learning, especially statistical machine learning. The paradigm of machine learning is different from that of most prior attempts at language processing. Prior implementations of language-processing tasks typically involved the direct hand coding of large sets of rules. Source:
  17. 17. Natural Language Processing Smarter & Cuter Bots - The machine-learning paradigm calls instead for using general learning algorithms — often, although not always, grounded in statistical inference — to automatically learn such rules through the analysis of large corpora of typical real-world examples. A corpus (plural, "corpora") is a set of documents (or sometimes, individual sentences) that have been hand-annotated with the correct values to be learned. Source: https://en.wikipedia.org/wiki/Natural_language_processing#Using_machine_learning
  18. 18. Natural Language Processing Smarter & Cuter Bots - Types of NLP: Linguistic Analysis Language Understanding Text Analytics Analyzing Tone
  19. 19. Markov Chain What is it? Smarter & Cuter Bots - “a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event.”
  20. 20. Markov Chain Oh. Smarter & Cuter Bots -Source: http://setosa.io/ev/markov-chains/ With two states (A and B) in our state space, there are 4 possible transitions (not 2, because a state can transition back into itself). If we're at 'A' we could transition to 'B' or stay at 'A'. If we're at 'B' we could transition to 'A' or stay at 'B'.
  21. 21. Markov Chain Smarter & Cuter Bots -Source: http://setosa.io/ev/markov-chains/
  22. 22. Corpus Fed What is it? Smarter & Cuter Bots - “In linguistics, a corpus (plural corpora) or text corpus is a large and structured set of texts (nowadays usually electronically stored and processed). They are used to do statistical analysis and hypothesis testing, checking occurrences or validating linguistic rules within a specific language territory.” Source: https://en.wikipedia.org/wiki/Text_corpus
  23. 23. Corpus Fed What is it? Smarter & Cuter Bots - Often used in Natural Language Processing, but more often used with generic twitter bots that combine or utilize text snippets in a more generalized way.
  24. 24. Corpus Fed Smarter & Cuter Bots - Source: https://github.com/dariusk/corpora
  25. 25. Corpus Fed Smarter & Cuter Bots - Source: https://github.com/rachelnicole/isReallyCute-bot
  26. 26. Generative Images What is it? Smarter & Cuter Bots - SVG generative through tracery Compiled through Image Magick / Graphics Magick Altering raw image binaries
  27. 27. Generative Images Smarter & Cuter Bots - Source: @amandaglosson made @tinyneighbor
  28. 28. Generative Images Smarter & Cuter Bots - Source: @v21 made @softlandscapes
  29. 29. Smarter & Cuter Bots - Source: http://cheapbotsdonequick.com/source/softlandscapes
  30. 30. Weird What is it? Smarter & Cuter Bots - Idk. Just weird stuff Seriously though, a lot of corpus fed text, random images, and general nonsensical markov chains for the purpose of being weird and fun.
  31. 31. Smarter & Cuter Bots - Source: @deathmtn made @cleromancer
  32. 32. Weird Smarter & Cuter Bots - Source: @ckolderup made @OminousZoom
  33. 33. So What Are We Making? Smarter & Cuter Bots - Twitter Bot! ( obviously ) Utilizing Twitter Streaming APIs GraphicsMagick Microsoft’s Cognitive Services – specifically the Face API
  34. 34. Twit Smarter & Cuter Bots - Make sure you have node & npm installed https://github.com/ttezel/twit Sign up for twitter app for auth tokens
  35. 35. Graphics Magick Smarter & Cuter Bots - GraphicsMagick is a tool to create, edit, and compose / composite images. It’s super versatile, you can access it from the cli or C, C++, Lua, Perl, PHP, Python, Tcl, Ruby, Windows .NET, or Windows COM programming interfaces. There’s a node port of it you can use via npm install gm. But It’s kind of a pain in the ass Source: http://www.graphicsmagick.org/
  36. 36. Graphics Magick Smarter & Cuter Bots - Down / Up Sampling of images are kind of poor quality It requires some hacky workarounds In node, you’ve got to use a ton of callbacks to do multiple compositing. To host it on a server you have to be able to install some binaries L Source: http://www.graphicsmagick.org/
  37. 37. Microsoft Cognitive Services Smarter & Cuter Bots - It’s rad. Whole suite of Machine Learning REST APIs Computer Vision, Content Moderator, Emotion, Face, Video, Bing Speech, Custom Recognition, Speaker Recognition, Bing Spell Check, Language Understanding, Linguistic Analysis, Text Analytics, Translator, WebLM, Academic, Entity Linking, Knowledge Exploration, Recommendations, Lots of Bing things. Source: https://www.microsoft.com/cognitive-services/
  38. 38. Microsoft Cognitive Services Smarter & Cuter Bots - It’s super easy to use If you know how to hit an endpoint, you can use everything. We’re going to use the Face API Source: https://www.microsoft.com/cognitive-services/
  39. 39. Microsoft Face API Smarter & Cuter Bots - Detect one or more human faces in an image and get back face rectangles for where in the image the faces are, along with face attributes which contain machine learning-based predictions of facial features. The face attribute features available are: Age, Gender, Pose, Smile, and Facial Hair along with 27 landmarks for each face in the image. Source: https://www.microsoft.com/cognitive-services/en-us/face-api
  40. 40. Microsoft Face API Smarter & Cuter Bots - Let’s try it out. Source: https://www.microsoft.com/cognitive-services/en-us/face-api
  41. 41. Okay, but Rachel…. Smarter & Cuter Bots - What exactly are we making?
  42. 42. Magical and Cute Image Bot!!! Smarter & Cuter Bots -
  43. 43. Smarter & Cuter Bots -
  44. 44. ~ Code Demo Time ~ Smarter & Cuter Bots -
  45. 45. Magical and Cute Image Bot Smarter & Cuter Bots -
  46. 46. I Launched it in November! Smarter & Cuter Bots - People immediately tried to break it! Sometimes it glitched out
  47. 47. I’m Launching it Right NOW!!!!!!!! Smarter & Cuter Bots -
  48. 48. Thank You! Smarter & Cuter Bots - Follow me on twitter: @ohhoe http://rachelisaweso.me http://imcool.online Github Source: https://github.com/rachelnicole/magicalncute http://aka.ms/smarterandcuter

×