Big Data Berlin – Automating Decisions is the Next Frontier for Big Data

3,702 views

Published on

Just collecting, storing and analyzing data is not enough. In order to benefit from it, you have to overcome organizational and human inertia and establish automated processes that use insights gained from your data.

This presentation has been presented at http://dataconomy.com/28-august-2014-big-data-berlin/

Published in: Software
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,702
On SlideShare
0
From Embeds
0
Number of Embeds
2,074
Actions
Shares
0
Downloads
25
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Big Data Berlin – Automating Decisions is the Next Frontier for Big Data

  1. 1. LARS TRIELOFF – BLUE YONDER – I’M @TRIELOFF ON TWITTER AUTOMATING DECISIONS – THE NEXT FRONTIER FOR BIG DATA
  2. 2. WHAT NOBODY TELLS YOU ABOUT THE FRONTIER
  3. 3. WHO IS USING BIG DATA TODAY
  4. 4. WHERE BIG DATA IS USED Effective Use Marketing Finance Everyone Else SOURCE: OWN, POTENTIALLY BIASED OBSERVATIONS
  5. 5. WHERE BIG DATA IS USED Potential Use Marketing Finance Everyone Else SOURCE: OWN, POTENTIALLY BIASED OBSERVATIONS
  6. 6. Better Decisions Faster Data More Data THREE APPROACHES
  7. 7. DIGITAL MARKETING’S APPROACH: MORE DATA
  8. 8. FINANCE’S APPROACH FASTER DATA
  9. 9. BUT WHAT ABOUT BETTER DECISIONS?
  10. 10. Self-Actualization Love/Belonging Safety Physiological Esteem
  11. 11. Storage Collection Prediction Analysis Decision
  12. 12. Storage Collection Prediction Analysis Decision Self-Actualization Love/Belonging Safety Physiological Esteem
  13. 13. “… all others bring data.” ! — W. EDWARDS DEMING
  14. 14. HOW DATA-DRIVEN DECISIONS SHOULD WORK COMPUTER COLLECTS COMPUTER STORES HUMAN ANALYZES HUMAN PREDICTS HUMAN DECIDES
  15. 15. HOW DATA-DRIVEN DECISIONS REALLY WORK COMPUTER COLLECTS COMPUTER STORES HUMAN ANALYZES COMMUNICATION BREAKDOWN HUMAN DECIDES
  16. 16. COMMUNICATION BREAKDOWN Communication Breakdown, It's always the same, I'm having a nervous breakdown, Drive me insane! — LED ZEPPELIN • Drill-down analysis … misunderstood or distorted • Metrics dashboards … contradictory and confusing • Monthly reports … ignored after two iterations • In-house analyst teams … overworked and powerless
  17. 17. HOW DATA-DRIVEN DECISIONS REALLY WORK HTTP://DILBERT.COM/STRIPS/COMIC/2007-05-16/
  18. 18. HOW DECISIONS REALLY SHOULD WORK COMPUTER COLLECTS COMPUTER STORES COMPUTER ANALYZES COMPUTER PREDICTS COMPUTER DECIDES
  19. 19. 99.9% of all business decisions can be automated
  20. 20. HOW DECISIONS ARE MADE Human Decisions Business Rules No Decision, Decision-by-default
  21. 21. BUSINESS RULES = PROGRAMMING • Business rules are like programs – written by non-programmers • Business rules can be contradictory, incomplete, and complex beyond comprehension • Business rules have no built-in feedback mechanism: “It is the rule, because it is the rule”
  22. 22. HUMAN DECISION MAKING • • System 1: Fast, automatic, frequent, emotional, stereotypic, subconscious • • System 2: Slow, effortful, infrequent, logical, calculating, conscious DANIEL KAHNEMANN, THINKING FAST AND SLOW
  23. 23. HOW TO DECIDE FAST FREQUENT DECISION MAKING MEANS FAST DECISION MAKING, MEANS USING HEURISTICS OR COGNITIVE BIASES Anchoring effect IKEA effect Over-justification effect Bandwagon effect Confirmation bias Substitution Availability heuristic Texas Sharpshooter Fallacy Gambler’s fallacy Illusory correlation Rhyme as reason effect Hindsight bias Zero-risk bias Framing effect Sunk cost fallacy Overconfidence Outcome bias Inattentional Blindness Benjamin Franklin effect Anecdotal evidence Negativity bias Loss aversion Backfire effect
  24. 24. HOW COMPUTERS DECIDE FAST MACHINE LEARNING OFFERS AN ALTERNATIVE TO HUMAN COGNITIVE BIASES AND CAN BE MADE FAST THROUGH BIG DATA K-Means Clustering Markov Chain Monte Carlo Support Vector Machines Naive Bayes Affinity Propagation Decision Trees Nearest Neighbors Least Angle Regression Logistic Regression Spectral clustering Restricted Bolzmann Machines
  25. 25. Better decisions through predictive applications
  26. 26. HOW PREDICTIVE APPLICATIONS WORK COLLECT & STORE ANALYZE CORRELATIONS BUILD DECISION MODEL DECIDE & TEST OPTIMIZE
  27. 27. WHY TEST? “Correlation doesn’t imply causation, but it does waggle its eyebrows suggestively and gesture furtively while mouthing ‘look over there’” –RANDALL MUNROE
  28. 28. STORY TIME (Not safe for vegetarians)
  29. 29. THE GROUND BEEF DILEMMA
  30. 30. THE GROUND BEEF DILEMMA Yesterday Today Tomorrow Next Delivery Next Day In Stock Demand
  31. 31. THE GROUND BEEF DILEMMA • Order too much and you will have to throw meat away when it goes bad. You lose money and cows die in vain • Order too little and you won’t serve all your potential customers. You lose money and customers stay hungry.
  32. 32. CHALLENGE #1 ACCURATELY PREDICT DEMAND
  33. 33. CHALLENGE #2 AUTOMATE REPLENISHMENT COLLECT STOCK AND SALES PREDICT DEMAND TRADE OFF COSTS CREATE ORDERS IN ERP SYSTEM OPTIMIZE & REPEAT
  34. 34. @BYANALYTICS_EN @TRIELOFF

×