Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
calculation | consulting
data science leadership
(TM)
c|c
(TM)
charles@calculationconsulting.com
calculation|consulting
Data Science Leadership
(TM)
charles@caclulationconsulting.com
calculation | consulting data science leadership
Who Are We?
c|c
(TM)
Dr. Charles H. Martin, PhD
University of Chicago, Ch...
BackStory: in 2011, Search Changed. Forever.
• first $1B IPO since Google
• Machine Learning based SEO algorithms
• Measure...
BackStory: in 2011, Search Changed. Forever.
• Google adapted (Panda)
• Lack of diversification
• Lack of adaptation
• Stoc...
• first $1B collapse due to Panda ?
• CPC revenues down
• premium online publishers died
collapse
?
stock price 2011-2012
c...
c|c
(TM)
Panda-Induced ‘Market Crash’
Google CPC dropped just after Panda
calculation | consulting data science leadership...
Data Science is Different
c|c
(TM)
Davenport
calculation | consulting data science leadership
Generating sustainable reven...
Data Science is Different
c|c
(TM)
Davenport
calculation | consulting data science leadership
Generating sustainable reven...
Problem: Data Scientists are Different
c|c
(TM)
Davenport
calculation | consulting data science leadership(TM)
10
not all ...
Problem: Data Scientists are Different
c|c
(TM)
Davenport
calculation | consulting data science leadership
theoretical phy...
Problem: Data Scientists are Different
c|c
(TM)
Davenport
calculation | consulting data science leadership(TM)
12
not all ...
Managing: Data Science Process
• Acquire Domain Knowledge
• Formulate Hypothesis
• Generate Model(s) from the Data
• Predi...
Managing: Data Science Process
c|c
(TM)
(TM)
calculation | consulting data science leadership
14
c|c
(TM)
• Systems Thinking: leveraging the inter-relationships
between data, marketing, and the customer
• Knowledge Tran...
c|c
(TM)
• Cross-functional engineering, product, marketing, finance
• Autonomous: separate from the traditional engineerin...
Solution: Collecting and Organizing Data
(TM)
c|c
(TM)
• Most companies are struggling organizing their data
• Data needs ...
Solutions: Hadoop and Big Data
(TM)
c|c
(TM)
• Hadoop is an internal data ecosystem
• Hadoop appears to have won the adopt...
Solutions: Cloud
(TM)
c|c
(TM)
• Startups don’t need infrastructure
• long term Data Storage is virtually free
• Amazon Re...
Solutions: Spark
(TM)
c|c
(TM)
• Next Gen Platform for Machine Learning
• Sits on Hadoop or the Cloud
• Still very high to...
Problem: Measurements
(TM)
c|c
(TM)
good experiments are amazing
calculation | consulting data science leadership
21
“If y...
Data Science’s Measurement Problem
(TM)
c|c
(TM)
good experiments are hard to design
calculation | consulting data science...
Data Science’s Measurement Problem
(TM)
c|c
(TM)
good experiments are hard to design
calculation | consulting data science...
Data Science’s Measurement Problem
(TM)
c|c
(TM)
good experiments are hard to design
calculation | consulting data science...
c|c
(TM)
(TM)
Problem: The Cult of the Algorithm
calculation | consulting data science leadership
25
what can algos actual...
c|c
(TM)
(TM)
Problem: What can Machine Learning Do?
calculation | consulting data science leadership
26
what can algos ac...
Demand Algos: Gas Station Analogy
Problem: where to open a gas station ?
Need: good traffic, weak competition
c|c
(TM)
less...
SAAS Machine Learning Algos
c|c
(TM)
calculation | consulting data science leadership
(TM)
28
$100,000 • 167 teams
Diabeti...
SAAS Machine Learning Algos
c|c
(TM)
calculation | consulting data science leadership
(TM)
29
machine learning apis
c|c
(TM)
(TM)
Problem: What can Deep Learning Do?
calculation | consulting data science leadership
30
what can algos actua...
c|c
(TM)
(TM)
Problem: Externalities
calculation | consulting data science leadership
31
external factors can change
c|c
(TM)
(TM)
Problem: Externalities
calculation | consulting data science leadership
32
“Zynga is our best company ever!”...
c|c
(TM)
(TM)
Solution: Algorithmic Accountability
calculation | consulting data science leadership
An asset is an economi...
c|c
(TM)
(TM)
Algorithmic Accountability
calculation | consulting data science leadership
34
does revenue depends on hidde...
c|c
(TM)
(TM)
Algorithmic Accountability
calculation | consulting data science leadership
35
do decisions depend on hidden...
c|c
(TM)
(TM)
Solution: Algorithmic Transparency
calculation | consulting data science leadership
36
can you be transparen...
c|c
(TM)
(TM)
Algorithmic Accountability
calculation | consulting data science leadership
Do you depend on some else’s mar...
(TM)
c|c
(TM)
c | c
charles@calculationconsulting.com
Upcoming SlideShare
Loading in …5
×

Cc hass b school talk 2105

694 views

Published on

Haas Business School Talk 2015

Published in: Business
  • Be the first to comment

Cc hass b school talk 2105

  1. 1. calculation | consulting data science leadership (TM) c|c (TM) charles@calculationconsulting.com
  2. 2. calculation|consulting Data Science Leadership (TM) charles@caclulationconsulting.com
  3. 3. calculation | consulting data science leadership Who Are We? c|c (TM) Dr. Charles H. Martin, PhD University of Chicago, Chemical Physics NSF Fellow in Theoretical Chemistry Over 10 years experience in applied Machine Learning Developed ML algos for Demand Media; the first $1B IPO since Google Lean Start Ups: Aardvark (acquired by Google), eHow, Mode Wall Street: BlackRock, GLG Fortune 500: Big Pharma, Telecom, eBay www.calculationconsulting.com charles@calculationconsulting.com (TM) 3
  4. 4. BackStory: in 2011, Search Changed. Forever. • first $1B IPO since Google • Machine Learning based SEO algorithms • Measure the demand for search, and fulfill it data science algorithms created a billion $ company c|c (TM) (TM) Demand Media calculation | consulting data science leadership(TM) 4 eHow.com
  5. 5. BackStory: in 2011, Search Changed. Forever. • Google adapted (Panda) • Lack of diversification • Lack of adaptation • Stock price never recovered algorithmic accountability: DMD or Google? c|c (TM) IPO Panda stock price 2011-2012 (TM) calculation | consulting data science leadership DMD (TM) 5
  6. 6. • first $1B collapse due to Panda ? • CPC revenues down • premium online publishers died collapse ? stock price 2011-2012 c|c (TM) $1B in ad revenue was repriced and reallocated Problem: Cornering the market on search induced a market crash calculation | consulting data science leadership(TM) 6
  7. 7. c|c (TM) Panda-Induced ‘Market Crash’ Google CPC dropped just after Panda calculation | consulting data science leadership(TM) 7
  8. 8. Data Science is Different c|c (TM) Davenport calculation | consulting data science leadership Generating sustainable revenue requires Data Science Leadership and Execution (TM) 8 “Companies need a Spock in the boardroom”
  9. 9. Data Science is Different c|c (TM) Davenport calculation | consulting data science leadership Generating sustainable revenue requires Data Science Leadership and Execution (TM) 9 http://www.theonion.com/articles/national-science-foundation-science-hard,1405/
  10. 10. Problem: Data Scientists are Different c|c (TM) Davenport calculation | consulting data science leadership(TM) 10 not all techies are the same
  11. 11. Problem: Data Scientists are Different c|c (TM) Davenport calculation | consulting data science leadership theoretical physics machine learning specialist (TM) 11 experimental physics data scientist engineer software, browser tech, dev ops, … not all techies are the same
  12. 12. Problem: Data Scientists are Different c|c (TM) Davenport calculation | consulting data science leadership(TM) 12 not all techies are the same
  13. 13. Managing: Data Science Process • Acquire Domain Knowledge • Formulate Hypothesis • Generate Model(s) from the Data • Predict Revenue Gains • Backtest Predictions on your Data • A/B Test in Production • Attribute Gains to Model(s) c|c (TM) (TM) acting solving framing calculation | consulting data science leadership 13
  14. 14. Managing: Data Science Process c|c (TM) (TM) calculation | consulting data science leadership 14
  15. 15. c|c (TM) • Systems Thinking: leveraging the inter-relationships between data, marketing, and the customer • Knowledge Transfer: mentoring — not training — to develop both personal mastery and team learning • Mental Models: create a base of small-scale models for thinking about how to use your data • Knowledge Sharing: foster collaboration between research, engineering, and product to drive revenue Managing: Learning from Data calculation | consulting data science leadership(TM) 15
  16. 16. c|c (TM) • Cross-functional engineering, product, marketing, finance • Autonomous: separate from the traditional engineering product lifecycle. self-organizing and self-managing • Experimental: form hypothesis, analyze data, make predictions, run backtests, A/B testing • Self-sustaining: not a cost center; generates revenue (TM) Data Science is Different calculation | consulting data science leadership 16
  17. 17. Solution: Collecting and Organizing Data (TM) c|c (TM) • Most companies are struggling organizing their data • Data needs to be examined • Don’t assume data is correct or useful • More is More: simple algos work • More is Less: noise is noise Data not examined is not collected calculation | consulting data science leadership 17
  18. 18. Solutions: Hadoop and Big Data (TM) c|c (TM) • Hadoop is an internal data ecosystem • Hadoop appears to have won the adoption wars ? • Hadoop : 90% deployments internal • Hadoop is a cost center • ROI needs cut across business divisions Algorithms, not data, generate revenue calculation | consulting data science leadership 18
  19. 19. Solutions: Cloud (TM) c|c (TM) • Startups don’t need infrastructure • long term Data Storage is virtually free • Amazon Redshift • Google Big Query • Cloud is the future ? Algorithms, not data, generate revenue calculation | consulting data science leadership 19
  20. 20. Solutions: Spark (TM) c|c (TM) • Next Gen Platform for Machine Learning • Sits on Hadoop or the Cloud • Still very high touch • Limited algos Algorithms, not data, generate revenue calculation | consulting data science leadership 20
  21. 21. Problem: Measurements (TM) c|c (TM) good experiments are amazing calculation | consulting data science leadership 21 “If you can’t measure it, you can’t fix it.” DJ Patil,White House Chief Data Scientist
  22. 22. Data Science’s Measurement Problem (TM) c|c (TM) good experiments are hard to design calculation | consulting data science leadership 22 http://www.forbes.com/sites/lizryan/2014/02/10/if-you-cant-measure-it-you-cant-manage-it-is-bs/
  23. 23. Data Science’s Measurement Problem (TM) c|c (TM) good experiments are hard to design calculation | consulting data science leadership 23 “Data science has a measurement problem. Simple metrics may not address complex situations. But complex metrics present myriad problems.” “As we strive for better algorithms, we often fail to think critically about what it means for predictions to be ‘good’” http://www.kdnuggets.com/2015/03/data-science-measurement-problem-accuracy-auroc-f1.html
  24. 24. Data Science’s Measurement Problem (TM) c|c (TM) good experiments are hard to design calculation | consulting data science leadership 24 “Buffett found it 'extraordinary' that academics studied such things. They studied what was measurable, rather than what was meaningful.‘ … to a man with a hammer, everything looks like a nail.” ― Roger Lowenstein, Buffett: The Making of an American Capitalist
  25. 25. c|c (TM) (TM) Problem: The Cult of the Algorithm calculation | consulting data science leadership 25 what can algos actually do ? “We have a new machine learning algo that anticipate your needs over time and behave accordingly”
  26. 26. c|c (TM) (TM) Problem: What can Machine Learning Do? calculation | consulting data science leadership 26 what can algos actually do ?
  27. 27. Demand Algos: Gas Station Analogy Problem: where to open a gas station ? Need: good traffic, weak competition c|c (TM) less competitors no traffic sweet spot great traffic too many competitors calculation | consulting data science leadership all businesses balance supply and demand (TM) 27
  28. 28. SAAS Machine Learning Algos c|c (TM) calculation | consulting data science leadership (TM) 28 $100,000 • 167 teams Diabetic Retinopathy Detection $15,000 • 341 teams March Machine Learning Mania 2015 machine learning contests
  29. 29. SAAS Machine Learning Algos c|c (TM) calculation | consulting data science leadership (TM) 29 machine learning apis
  30. 30. c|c (TM) (TM) Problem: What can Deep Learning Do? calculation | consulting data science leadership 30 what can algos actually do ?
  31. 31. c|c (TM) (TM) Problem: Externalities calculation | consulting data science leadership 31 external factors can change
  32. 32. c|c (TM) (TM) Problem: Externalities calculation | consulting data science leadership 32 “Zynga is our best company ever!” (2010) John Doerr, Google Investor, LegendaryVC http://venturebeat.com/2010/11/16/google-investor-john-doerr-zynga-is-our-best-company-ever/ one marketplace | big risks
  33. 33. c|c (TM) (TM) Solution: Algorithmic Accountability calculation | consulting data science leadership An asset is an economic resource. Anything tangible or intangible that is capable of being owned or controlled to produce value and that is held to have positive economic value is considered an asset. algorithms can be valuable assets 33
  34. 34. c|c (TM) (TM) Algorithmic Accountability calculation | consulting data science leadership 34 does revenue depends on hidden algos ? • WebMD Google SEO • Amazon Product Listing Algo • Pinterest Relevance Algo • Twitter Spam filter • Apple App Store Rankings
  35. 35. c|c (TM) (TM) Algorithmic Accountability calculation | consulting data science leadership 35 do decisions depend on hidden factors ? A 'Crisis' in Online Ads: One-Third of Traffic Is Bogus http://www.wsj.com/articles/SB10001424052702304026304579453253860786362 Now Algorithms Are DecidingWhomTo Hire… http://www.npr.org/blogs/alltechconsidered/2015/03/23/394827451/now-algorithms-are-deciding-whom-to-hire-based-on-voice What you don’t know about Internet algorithms is hurting you… http://www.washingtonpost.com/news/the-intersect/wp/2015/03/23/what-you-dont-know-about-internet-algorithms-is-hurting-you-and-you-probably-dont-know-very-much/
  36. 36. c|c (TM) (TM) Solution: Algorithmic Transparency calculation | consulting data science leadership 36 can you be transparent and not be gamed ? http://fortune.com/2015/03/18/how-do-you-govern-a-hidden-fluid-and-amoral-algorithm/ 83% of the participants in the study changed their behavior once they knew about the algorithm How do you govern a (hidden, fluid and amoral) algorithm? participants mistakenly believed that their friends intentionally chose not to show them stories
  37. 37. c|c (TM) (TM) Algorithmic Accountability calculation | consulting data science leadership Do you depend on some else’s marketplace? How does your revenue depend on algos? Do you need an internal algo ? Who will manage it? build it? maintain it? algorithms have unforeseen liabilities 37
  38. 38. (TM) c|c (TM) c | c charles@calculationconsulting.com

×