Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The Unreasonable Benefits of Deep Learning


Published on

Dan Kuster led a talk at Sentiment Analysis Symposium discussing why businesses should consider adopting deep learning solutions. Key takeaways include simplicity, accuracy, flexibility, and some hacks for working with the tech.

About the Session:
Machine learning is becoming the tool of choice for analyzing text and image data. While traditional text processing solutions rely on the ability of experts to encode domain knowledge, machine learning models learn this directly from the data. Deep learning is a branch of machine learning that like the human brain quickly learns hierarchical representations of concepts, and it has been key to unlocking state-of-the-art results on a range of text and image classification tasks such as sentiment analysis and beyond.

In this session, we will show the impact of a deep learning based approach over NLP and traditional machine learning based methods for text analysis across key dimensions such as accuracy, flexibility, and the amount of required training data. Specifically, we will discuss how deep learning models are now setting the records for state-of-the-art accuracy in sentiment analysis. We will also demonstrate the flexibility of this approach by showing how the features learned by one model can be easily reused in different domains (e.g., handling additional languages, or predicting new categories) to drastically reduce the time to deployment. Finally, we will touch on the ability of this method to handle additional types of data beyond text, e.g, images, for maximum insight.

Published in: Data & Analytics
  • Be the first to comment

The Unreasonable Benefits of Deep Learning

  1. 1. The Unreasonable Benefits of Deep Learning Daniel  Kuster,  Ph.D.   @djkust @indicodata
  2. 2. “All good researchers will tell you that the most promising direction is the one they are currently pursuing. If they thought something else was more promising, they would be doing that instead.” — G. Hinton
  3. 3. What is deep learning? …a method for applying simple mathematical functions to data.
  4. 4. web: search, facial recognition smartphones: speech -> text email: smart reply mail: handwriting -> digits cars: pedestrian detection art & design: artistic style transfer
  5. 5. Wait, why now? ~1960’s (visual cortex is a deep neural network) Simple neurons ⇾ hierarchical features ⇾ complex ~1990’s (computational models) Neural networks ⇾ simple functions, applied piecewise ~ 2000’s (the internet + cheap storage) Lots of data ~2012 (2 GPUs beat Google’s 16,000 CPU cluster) Very fast and cheap parallel computing power Deep neural networks ≈ mathematics + data
  6. 6. How deep learning works: (1-minute theoretical explanation)
  7. 7. Cat detector = eyes + fur + nose + … …but how do we discover the features? and where are they?
  8. 8. input simple math functions pooling simple math functions classifier But how do we know what to detect? learn from the data…
  9. 9. How to use deep learning: (10-second practical explanation)
  10. 10. data (content) model classifier data (predictions) How to use deep learning
  11. 11. Maybe you are skeptical that deep learning will have a lasting impact…
  12. 12. Maybe you are skeptical that mathematics
 will have a lasting impact?
  13. 13. “The enormous usefulness of mathematics in the natural sciences is something bordering the mysterious and there is no rational explanation for it.” —Eugene Wigner (1960) 
 “The Unreasonable Effectiveness of 
 Mathematics in the Natural Sciences” Unreasonable Benefits of Deep Learning
  14. 14. “…mathematical formulation…leads in an uncanny number of cases to an amazingly accurate description of a large class of phenomena.” “…the concepts of mathematics are not chosen for their conceptual simplicity…but for their amenability to clever manipulations and to striking, brilliant arguments.” —Eugene Wigner (1960) 
 “The Unreasonable Effectiveness of 
 Mathematics in the Natural Sciences” Unreasonable Benefits of Deep Learning
  15. 15. simplicity Unreasonable Benefits of Deep Learning accuracy flexibility hacks
  16. 16. simplicity Unreasonable Benefits of Deep Learning accuracy flexibility hacks
  17. 17. Simple? Compared to what? • Expert systems
 Domain expertise ⇾ think a lot ⇾ codify rules (e.g., 1700 pages of English grammar)
 More data, more pain.
 Previous wave of “A.I.” (good rules can seem magical). • Traditional machine learning
 Data ⇾ domain expertise ⇾ feature extraction ⇾ learned weights
 Learn everything from scratch. 
 Manual feature engineering, biased and tedious.
 More data helps! • Deep neural networks
 Data ⇾ model ⇾ learned weights
 End-to-end learning, directly from examples. Like we (humans) do.
 Can learn transferable features.
 More data really helps!
  18. 18. Example: the simplest text analysis task: sentiment!
  19. 19. text (reviews) model classifier Sentiment: compress text to one bit of info 1 0
  20. 20. text (reviews) model classifier Sentiment: compress text to one bit of info
  21. 21. Sentiment over time: the shapes of stories
  22. 22. story + vis + code
  23. 23. text model classifier Emotion: compress text to __ bits of info 😀 😉 😡 😈 😭 😍 😎 😞
  24. 24. Emotion import indicoio indicoio.emotion(“What?!?!? I had no idea this sort of thing existed!”) (~two lines of code)
  25. 25. text model classifier Personality, topics, political lean, language, … compress text to __ bits of info
  26. 26. image model classifier image filtering: compress images to one bit of info mature
 content safe (for work)
  27. 27. Content filtering (especially for user-generated content) You have a brand Your brand has an identity (Disney vs. Calvin Klein) Your audience might have different sensibilities than you do, about what is appropriate for your brand Filter out the inappropriate content at your own custom threshold
  28. 28. simplicity Unreasonable Benefits of Deep Learning accuracy flexibility hacks
  29. 29. Most accurate sentiment API: (93.8% on IMDB)
  30. 30. OK, accuracy is great… …but how does this help me solve 
 problems faster/cheaper/better?
  31. 31. simplicity Unreasonable Benefits of Deep Learning accuracy flexibility hacks
  32. 32. Access features directly …feed into a new classifier data (content) model classifier data (predictions)
  33. 33. Get the code (free):
  34. 34. Labeling 100k+ examples…sucks! …labeling a few hundred is just
 a couple hours at the coffeeshop.
  35. 35. data (content) model classifier data (predictions) What’s happening in the the middle? Let’s look at some features
  36. 36. RNN for sentiment prediction t-SNE of recurrent features: pos/neg words
  37. 37. Features learned some rules of English from binary sentiment labels
  38. 38. “The big payoff of deep learning is to allow learning higher levels of abstraction” — Y. Bengio
  39. 39. simplicity Unreasonable Benefits of Deep Learning accuracy flexibility hacks!
  40. 40. Features are compressed knowledge Who says we can’t combine them?
  41. 41. Experiment:
 image features + text features A man standing in a field holding 
 a small parachute image encoder text encoder similarity(image, text)
  42. 42. “in the sky” ⇾ most similar images
  43. 43. photo with wine glass ⇾ intent ⇽ campaign image-in-image search
  44. 44. Q: What problems can be solved with a deep neural network? A: If a human mind can do it in 1/10th of a second, a deep neural network can probably do it well enough… assuming you have data!
  45. 45. “Many scientists (myself included) take a sadistic pleasure in proving other people wrong. — Y. LeCun
  46. 46. The Unreasonable Benefits of Deep Learning: simplicity, accuracy, flexibility, hacks Questions? Daniel  Kuster,  Ph.D.   @djkust @indicodata
  47. 47. Image credits: Unsplash (backgrounds) Google search, Facebook AI Research (DeepFace), nVidia, Gatys et al. (arXiv: 1508.06576) Brigitewear International (Borat swimsuit) Imgur (skeptical dogs) Jack the cat …and the team at indico!
  48. 48. “A machine learning researcher, a crypto-currency expert, and an Erlang programmer walk into a bar. Facebook buys the bar for $27 billion.”