Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

915 keynote stern_using our laptop

130 views

Published on

#EMSNYCDAY1

Published in: Marketing
  • Be the first to comment

  • Be the first to like this

915 keynote stern_using our laptop

  1. 1. Jim Sterne eMetrics Summit Digital Analytics Association Artificial Intelligence for Marketing Getting Started Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
  2. 2. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Genetic Algorithm Covariance Kernels Gaussian Processes Hyperparameter Tuning Convex Gradient Methods Gradient Boosted Method Particle Swarm Intelligence Convolutional Neural Architecture Heterogeneous Configuration Models Spatio-Temporal Hierarchical Bayesian Optimization The Language Barrier
  3. 3. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics As Seen on TV “Strong AI” – thinks and acts human Sentience “Weak AI” – task specific Functional AI: Anything computers can’t SciFi: Anything AI can’t
  4. 4. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Why Machine Learning Now? 50 years of study Huge amount of data Specialize chipsets
  5. 5. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Software Grows Up Specific Logic Mathematical Model Do this, then this, then this Describe numerical relationships If this happens, do that Calculate alternatives If confused, report error Human compares results & iterates Statistical Model Artificial Intelligence Calculate probabilities Uses examples to figure it out Project likelihoods and changes it's mind Human compares & iterates
  6. 6. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Artificial Intelligence Natural Language Processing Computer Vision Conversation Bots Robots Machine Learning
  7. 7. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Artificial Intelligence Speech to text This means that Repeated correction Taught over time Contractions Accents Patois Wreck a nice beach Recognize speech Natural Language Processing
  8. 8. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Natural Language Processing Can I help you? Yes, I have a problem Oh? – one of the keys is broken Current Customer with my keyboard Hardware 1. Take it to a local store 2. Send it in for repair 3. Send it in for replacement FAQ How long? Loaner? Data back up? Warranty? Incoming: 805-403-4075 Customer service
  9. 9. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Natural Language Processing
  10. 10. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Artificial Intelligence Natural Language Speech to text This means that Repeated correction Taught over time Conversation Bots Text to meaning Concept & emotion imitation Repeated correction Taught over time
  11. 11. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
  12. 12. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Artificial Intelligence Natural Language Speech to text This means that Repeated correction Taught over time Conversation Bots Text to meaning Concept & emotion imitation Repeated correction Taught over time
  13. 13. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Artificial Intelligence Natural language Speech to text This means that Repeated correction Taught over time Conversation Bots Text to meaning Concept & emotion imitation Repeated correction Taught over time Vision Pattern discovery This means that Repeated correction Taught over time
  14. 14. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
  15. 15. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
  16. 16. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
  17. 17. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
  18. 18. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Facial Recognition
  19. 19. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Rules based This means that Repeated correction Is taught over time Pattern discovery This means that Repeated correction Learns over time Complex concept imitation Emotional intelligence imitation Repeated correction Is taught and learns over time Robots
  20. 20. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics In-Store Robots
  21. 21. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Artificial Intelligence Natural Language Processing - Call center Conversation Bots - Customer service Computer Vision - Social media Robots - In store Machine Learning - Everything else
  22. 22. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Detect Discover the most predictive attributes for a given outcome 01 Decide Infer rules from the data, weigh the attributes, and suggest a course of action 02 Revise Alter opinions about attributes and their weightings 03 Detect Discover the most predictive attributes for a given outcome 01 Decide Infer rules from the data, weigh the attributes, and suggest a course of action 02 Revise Alter opinions about attributes and their weightings 03 3 Needs for 3 Deeds of Machine Learning
  23. 23. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Detect Discover the most predictive attributes for a given outcome 01 Decide Infer rules from the data, weigh the attributes, and suggest a course of action 02 Develop Alter opinions about attributes and their weightings 03 Detect Discover the most predictive attributes for a given outcome 01 Decide Infer rules from the data, weigh the attributes, and suggest a course of action 02 Develop Alter opinions about attributes and their weightings 03Wind speed 3 Needs for 3 Deeds of Machine Learning Barometric pressure Temperature Hours of daylight Sunrise, sunset Rain? UV Index
  24. 24. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Detect Discover the most predictive attributes for a given outcome 01 Decide Infer rules from the data, weigh the attributes, and suggest a course of action 02 Develop Alter opinions about attributes and their weightings 03 Detect Discover the most predictive attributes for a given outcome 01 Decide Infer rules from the data, weigh the attributes, and suggest a course of action 02 Develop Alter opinions about attributes and their weightings 03 Wind speed 3 Needs for 3 Deeds of Machine Learning Barometric pressure Hours of daylight Sunrise, sunset Rain: UV Index Temperature
  25. 25. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Detect Discover the most predictive attributes for a given outcome 01 Decide Infer rules from the data, weigh the attributes, and suggest a course of action 02 Develop Alter opinions about attributes and their weightings 03 Detect Discover the most predictive attributes for a given outcome 01 Decide Infer rules from the data, weigh the attributes, and suggest a course of action 02 Develop Alter opinions about attributes and their weightings 03Day Part Gender Age Income Location Education Behavior Weather 3 Needs for 3 Deeds of Machine Learning
  26. 26. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Detect Discover the most predictive attributes for a given outcome 01 Decide Infer rules from the data, weigh the attributes, and suggest a course of action 02 Develop Alter opinions about attributes and their weightings 03 Detect Discover the most predictive attributes for a given outcome 01 Decide Infer rules from the data, weigh the attributes, and suggest a course of action 02 Develop Alter opinions about attributes and their weightings 03Day Part Gender Age Income Location Education Behavior Weather 3 Needs for 3 Deeds of Machine Learning
  27. 27. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Detect Discover the most predictive attributes for a given outcome 01 Decide Infer rules from the data, weigh the attributes, and suggest a course of action 02 Develop Alter opinions about attributes and their weightings 03 Detect Discover the most predictive attributes for a given outcome 01 Decide Infer rules from the data, weigh the attributes, and suggest a course of action 02 Develop Alter opinions about attributes and their weightings 03 GOAL: Conversion DATA: previous purchase search term pageviews time-on-item email opt in post code Y >1.5 * >5 Y Then send 15% off email 3 Needs for 3 Deeds of Machine Learning
  28. 28. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Detect Discover the most predictive attributes for a given outcome 01 Decide Infer rules from the data, weigh the attributes, and suggest a course of action 02 Revise Alter opinions about attributes and their weightings 03 Detect Discover the most predictive attributes for a given outcome 01 Decide Infer rules from the data, weigh the attributes, and suggest a course of action 02 Revise Alter opinions about attributes and their weightings 03 GOAL: Conversion DATA: previous purchase search term pageviews time-on-item email opt in post code Y >1.5 * >5 Y Then send 15% off email >2 >4 3 Needs for 3 Deeds of Machine Learning Control:
  29. 29. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Detect Discover the most predictive attributes for a given outcome 01 Decide Infer rules from the data, weigh the attributes, and suggest a course of action 02 Revise Alter opinions about attributes and their weightings 03 Detect Discover the most predictive attributes for a given outcome 01 Decide Infer rules from the data, weigh the attributes, and suggest a course of action 02 Revise Alter opinions about attributes and their weightings 03 Data Goal Control 3 Needs for 3 Deeds of Machine Learning
  30. 30. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics More data than a human can wrangle More attributes than a human can manage More permutations than a human can comprehend
  31. 31. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Classification Clustering Segmentation Gender M F Age A Age B Age C Man and Machinevs.
  32. 32. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Classification Clustering Segmentation Motorcycle Insurance Targeting Man and Machinevs.
  33. 33. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Testing A. Buy one get one free B. Two for the price of one A2.1.1 A2.1.2 15% Lift A2.1 A2.2 A B A1 A2 B1 B2 Man and Machinevs.
  34. 34. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Is there a pattern? Is there an anomaly? What can be omitted? What if we did it backwards? What if we changed the time scale? What if we look at it sideways? What additional data would be revealing? What if it had wheels? What would Chuck Norris do? What if this is the wrong problem? Man and Machine
  35. 35. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Testing A. Buy one get one free B. Two for the price of one A2.1.1 A2.1.2 15% Lift A2.1 A2.2 A B A1 A2 B1 B2 Man and Machine
  36. 36. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Learning Machine Learning What's it good at? How is it classified? How does it work?
  37. 37. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Learning Machine Learning High Dimensionality High Cardinality What's it good at?
  38. 38. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Dimensionality = Elements per Object dog bird grumpy keyboardLOL can haz cheeseburger? fish cool hip crazy aloofsnootytrying to kill you tail fur claws Low Customers: 10 columns name, address, phone, DoB, interests, orders, CLTV, etc. Medium Web analytics: 100 columns High Language: > 1,000 dimensions cat
  39. 39. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Cardinality = Options per Element High: Phone # 7.442 billion Medium: ZIP Code 43,000 Low: Alive or Dead 2.5 Age 122
  40. 40. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Learning Machine Learning High Dimensionality High Cardinality What's it good at?
  41. 41. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Learning Machine Learning How is it classified? Supervised Unsupervised Reinforcement
  42. 42. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Supervised: You know the right answer Correcting Autocorrect Dog: Yes Cat: No Tag a friend?
  43. 43. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Unsupervised Segment my customers Find look-alike prospects Create customer personas Good for unlabeled data Tell me something I don't know
  44. 44. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
  45. 45. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Reinforcement Learning Given: data goal action feedback Respond to the environment
  46. 46. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Learning Machine Learning How does it work? Decision Trees / Random Forest Support Vector Machines Neural Nets / Deep Learning Ensemble
  47. 47. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Decision Trees  Random Forest Message A Message B
  48. 48. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Random data samples Random variables Decision Trees  Random Forest
  49. 49. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Random data samples Random variables Decision Trees  Random Forest
  50. 50. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Random data samples Random variables Solution Decision Trees  Random Forest
  51. 51. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Support Vector Machines
  52. 52. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Support Vector Machines
  53. 53. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Support Vector Machines
  54. 54. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Support Vector Machines
  55. 55. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Neural Network  Deep Learning Go to the movies?
  56. 56. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Neural Network  Deep Learning Go to the movies?
  57. 57. Wisdom of Machines 57 Validation Each model’s predictive accuracy is tested on the hold out data set. Wisdom of the crowd The combination of models that delivers the best accuracy is selected and deployed.
  58. 58. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Learning Machine Learning Language High Dimensionality Lots of Elements per Object High Cardinality Lots of Options per Element Supervised / Unsupervised Examples vs. Exploration Decision Trees / Random Forest Random data & variables Support Vector Machines Looking at it from a different angle Neural Nets / Deep Learning Sort of how we think the mind works Ensemble Diversity Rules
  59. 59. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Bringing AI Into Your Organization Look what followed me home! Can we keep him?
  60. 60. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics AI Onboarding Tips Clearly identified goals
  61. 61. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics AI Onboarding Tips Clearly identified goals Know Your Data Valid Credible Reliable Consistent Clean Unbiased Defined Relevant Correlate-able Understandable Complete Timely
  62. 62. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics AI Onboarding Tips Clearly identified goals Know Your Data Would I advise my uncle? Would I stake my reputation? Would I risk my own money? Would I bet my job?
  63. 63. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics AI Onboarding Tips Clearly identified goals Know Your Data Start with repetitive, taxing tasks machines can do better Ranking Sorting big data Finding patterns Finding look-alikes Counting, measuring Finding a needle in a haystack One of these things is not like the other
  64. 64. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics What Can ML Do Better? Testing Lead scoring Meeting scheduling Personalizing content Inbound e-mail sorting Social media monitoring Programmatic advertising Creating social media messages & ad copy
  65. 65. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics AI Onboarding Tips Clearly identified goals Know Your Data Start with repetitive, taxing tasks machines can do better Buy vs. Build?
  66. 66. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics
  67. 67. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics AI Onboarding Tips Clearly identified goals Know Your Data Start with repetitive, taxing tasks machines can do better Buy vs. Build? Buy!
  68. 68. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics AI Onboarding Tips Clearly identified goals Know Your Data Start with repetitive, taxing tasks machines can do better Buy vs. Build Determine which data sets are useful
  69. 69. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics AI Onboarding Tips Clearly identified goals Know Your Data Start with repetitive, taxing tasks machines can do better Buy vs. Build Determine which data sets are useful Become proficient at the Smell Test
  70. 70. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics The Smell Test
  71. 71. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics AI Onboarding Tips Clearly identified goals Know Your Data Start with repetitive, taxing tasks machines can do better Buy vs Build Determine which data sets are useful Become proficient at the Smell Test Be the change you want to see
  72. 72. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Find other enthusiasts (meet-ups) Find internal enthusiasts (host a meet-up) Lunch and Learn (buy them lunch) Combine resources to make every decision lead to creating an AI Center of Excellence Be The Change You Want to See
  73. 73. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics -1 Ignorance or ennui 0 Aware and Learning 1 Ad-Hoc Experimentation 2 Organized Experimentation 3 Goal Setting 4 System Training 5 System Testing 6 System Deployed 7 Continuous Learning Everything Maturity Model
  74. 74. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics AI Onboarding Tips Clearly identified goals Start with repetitive, taxing tasks Buy vs. build Know your data Determine which data sets are useful Be the change you want to see Become proficient at the Smell Test Hone your domain knowledge
  75. 75. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Man and Machine
  76. 76. Jim Sterne – jsterne@targeting.com – emetrics.org – @jimsterne – #eMetrics Jim Sterne eMetrics Summit Digital Analytics Association Artificial Intelligence for Marketing Getting Started

×