Working with Data Workshop

1. Working with data define - disassemble - evaluate - act workshop [comp_name] [date]

2. jankyri.com | mail@jankyri.com You know nothing John Snow? source: John Snow - the original uploader was Rsabbatini at English Wikipedia CC BY 4.0, via Wikimedia Commons, https://commons.wikimedia.org/wiki/File:John_Snow.jpg Water pump by Justinc, CC BY 2.0, https://en.wikipedia.org/wiki/File:John_Snow_memorial_and_pub.jpg - Physician - London - ~1810 - 1860 - Has a memorial plus a pub named after him!

3. jankyri.com | mail@jankyri.com Cholera epidemic London 1854 Cause: unknown extremely deadly: 600 people in ~15 days Theory 1: “foul” air Solution: washing Theory 2: contaminated water Solution: Use clean water Cholera: two competing hypotheses

4. jankyri.com | mail@jankyri.com Cholera: two competing hypotheses source: https://en.wikipedia.org/wiki/File:Snow-cholera-map-1.jpg Theory 1: “foul” air Solution: washing Theory 2: germs in contaminated water Solution: Use clean water

5. jankyri.com | mail@jankyri.com What is this? ● a workshop ● 60% theory, 40% practice ● learn to ● define ● disassemble ○ data ● evaluate ○ facts ○ context ● act on ○ insights

6. jankyri.com | mail@jankyri.com Why? Goal: ● data literacy ○ equip you with a mental model for data, facts and insights ○ get a feeling for ● assumptions ● forecasts ● act on insights ○ understand your work better ○ find ways to get clarity and improve Nobody is born a master analyst. All of you can learn this.

7. jankyri.com | mail@jankyri.com 1. Define 1. define

8. jankyri.com | mail@jankyri.com What is data? 1. define

9. jankyri.com | mail@jankyri.com What is data? Data is the new oil! Clive Humby source: http://ana.blogs.com/maestros/2006/11/data_is_the_new.html 1. define

10. jankyri.com | mail@jankyri.com What is data? Data is just like crude. It’s valuable, but if unrefined it cannot really be used. It has to be changed into gas, plastic, chemicals, etc to create a valuable entity that drives profitable activity; so must data be broken down, analyzed for it to have value. First, we must understand that data is not insight. Clive Humby 1. define

11. jankyri.com | mail@jankyri.com What is data? data qualitative quantitative labels ● good ● average ● bad ● green ● cold ● fluid continuous ● 900.54 € ● 1.5 s ● 33.56 % discreet ● 147 signups ● 80 clicks ● 5 conversions 1. define

12. jankyri.com | mail@jankyri.com What is data? insight explicit: fact data implicit: context action 1. define

13. jankyri.com | mail@jankyri.com 2. Disassemble 2. disassemble

14. jankyri.com | mail@jankyri.com 2. Disassemble 2. disassemble source: https://en.wikipedia.org/wiki/File:Hieroglyphs_from_the_tomb_of_Seti_I.jpg

15. jankyri.com | mail@jankyri.com 2. Disassemble Average customer rating: “good” 2. disassemble

16. jankyri.com | mail@jankyri.com 2. Disassemble Revenue: 160,465.89 € Clicks: 8,562 Lines of code: 1,000 2. disassemble

17. jankyri.com | mail@jankyri.com 2. Context matters Revenue: 160,465.89 € Clicks: 8,562 Lines of code: 1,000Average customer rating: “good” qualitative quantitative 2. disassemble

18. jankyri.com | mail@jankyri.com 2. Context matters Source: https://www.sheffield.ac.uk/faculty/social-sciences/news/self-employed-happier-more-engaged-at-work- 1.770210 19 March 2018 Self-employed people happier and more engaged at work, study finds Self-employed people are happier and more engaged in their work than those in any other profession, according to a new study of 5,000 workers. The self-employed workers who took part in the research worked in a range of industries, including management consultancy, financial services, retail, education, insurance and real estate. 2. disassemble

19. jankyri.com | mail@jankyri.com 2. Known vs. unknown entities Things we know don’t know know don’t know know don’t know facts that may be wrong and should be checked against data. questions we can answer by reporting, which we should baseline and automate. intuition, which we should quantify and teach to improve effectiveness and efficiency. exploration, which is where unfair advantages and epiphanies live. Source: Donald Rumsfeld press briefing 2002, adapted from: https://en.wikipedia.org/wiki/Johari_window Croll, Alistair and Yoskovitz, Benjamin, Lean Analytics: Use Data to Build a Better Startup Faster, O’Reilly, 2013 2. disassemble

20. jankyri.com | mail@jankyri.com 3. Evaluate 3. evaluate

21. jankyri.com | mail@jankyri.com 3. Evaluation Introduction Things we know don’t know know don’t know know don’t know facts questions intuition exploration easy medium hard ease to evaluate medium large low NA impact if wrong medium 3. evaluate

22. jankyri.com | mail@jankyri.com 3. Introduction: Wrong intuition and bias is human "[Apple's iPhone] is the most expensive phone in the world and it doesn't appeal to business customers because it doesn't have a keyboard …" Steve Ballmer, 2007 "There's just not that many videos I want to watch." Steve Chen, CTO and co-founder of YouTube expressing concerns about his company’s long term viability, 2005 "There is no reason anyone would want a computer in their home." Ken Olson, Digital Equipment Corp., 1977 3. evaluate

23. jankyri.com | mail@jankyri.com 3. Introduction: Wrong intuition and bias is human But: Predictions are very important subjective → objective reality Why are predictions tricky? 3. evaluate

24. jankyri.com | mail@jankyri.com 3. Human Psychology: Kahneman System 1: “Thinking fast” gut reaction System 2: “Thinking slow” analytical thinking % of activity 3. evaluate

25. jankyri.com | mail@jankyri.com 3. Human Psychology in Evaluation: WYSIATI WYSIATI = what you see is all there is Law of small numbers generalizing small # of observations to population Confirmation bias focus on limited evidence confirming your mental story, fail to seek contrary facts Overconfidence ignoring what you don’t know, focus on what easily comes to mind Hindsight bias focus on past outcomes, not on quality of process Over-optimism Plan with best-case scenarios instead of weighing costs, risks and potential blocks 3. evaluate Source: https://icons8.com/icon/set/brain/doodle

26. jankyri.com | mail@jankyri.com 3. Overcoming WYSIATI with science “Doing data analysis without explicitly defining your problem or goal is like heading out on a road trip without having decided on a destination.” Source: Head First Data Analysis, Milton, Michael, O’Reilly, 2009 3.1 Have a hypothesis X X 3. evaluate

27. jankyri.com | mail@jankyri.com 3. Overcoming WYSIATI with science ● Your world view determines your analytical view ○ Thinking slow → Thinking fast ● Predictions make your assumptions and uncertainty explicit ● groups ○ can balance uncertainty ○ increase accuracy 3.2 Make your predictions 3. evaluate

28. jankyri.com | mail@jankyri.com ● compare facts ○ raw data (outliers) ○ interval comparison (week, month, year) ■ beware of small # of data points! ● Seek contrary evidence, ○ talk to people → adapt the outside view 3. Overcoming WYSIATI with science 3.3 Test and falsify your predictions 3. evaluate

29. jankyri.com | mail@jankyri.com 3. Evaluation: best practices scientific method ● come up with a hypothesis ● make a prediction ● do ○ Ask colleagues for prediction ○ Conduct research to falsify your prediction ○ Conduct an experiment to test the prediction 3. evaluate

30. jankyri.com | mail@jankyri.com 3. Game time: Flash forecasting Forecast intervals with a confidence level 1) Get pen & paper 2) Write down confidence interval, lower bound, upper bound, draw distribution Example: 80% confidence level = you are right 4 / 5 guesses If you don't know anything about the subject at all and think 1 %, 50 % and 99 % are equally likely to be the true answer, your bounds should be something like 10-90 %. 3. evaluate Source: https://www.aforeseeablefuture.com

31. jankyri.com | mail@jankyri.com 3. Game time: Flash forecasting Ready? 3. evaluate

32. jankyri.com | mail@jankyri.com Game time What is the share of working persons in Germany in 2018? Example: ~ 80 million inhabitants (December 2017) 80 % confidence level: 8 - 72 million 3. evaluate

33. jankyri.com | mail@jankyri.com Game time Age in Germany: nominal and relative distribution Source: http://www.gbe-bund.de, Indikator 1 der ECHI shortlist: Bevölkerung nach Geschlecht und Alter Source: https://service.destatis.de/bevoelkerungspyramide/#!y=2017 3. evaluate

34. jankyri.com | mail@jankyri.com Game time Source: https://www.aforeseeablefuture.com How old was Martin Luther King Jr. when he was assassinated? 3. evaluate

35. jankyri.com | mail@jankyri.com Game time How many percent of the world population have access to the internet? 3. evaluate

36. jankyri.com | mail@jankyri.com Game time More questions at aforeseeablefuture.com or build your company’s question set! 3. evaluate

37. jankyri.com | mail@jankyri.com 3. Game time: Flash forecasting Be conscious about System 1 thinking and actively engage System 2! ● Are your ranges ○ wide? → signals your assumed distribution ○ small? → possibly you are overconfident ● What is the distance of the actual value to your bounds? ○ does your mental image of the distribution of data reflect the real world? ● What is your accuracy over time? ○ are you prone to a certain bias? ○ is there an area you can learn more about to increase your accuracy? Source: https://hbr.org/2016/05/superforecasting-how-to-upgrade-your-companys-judgment 3. evaluate

38. jankyri.com | mail@jankyri.com 3. Game time: Flash forecasting Be conscious about System 1 thinking and actively engage System 2! ● individuals’ forecasting ability improves already after 1 hour ● use teams boosts accuracy ● track prediction performance and provide rapid feedback ○ use past data for training ○ possibly monthly challenges → develop a “sanity compass” for facts 3. evaluate

39. jankyri.com | mail@jankyri.com 3. Forecasting: winning as a group Process: Avoid anchors! 1) diverging phase, in which the issue, assumptions, and approaches to finding an answer are explored from multiple angles 2) evaluating phase, which includes time for productive disagreement 3) converging phase, when the team settles on a prediction Trust among members of a team is key for good outcomes (speak your mind)! 3. evaluate

40. jankyri.com | mail@jankyri.com 3. Game time: Flash forecasting Actively engage System 2! ● Are your ranges ○ wide? → signals your assumed distribution ○ small? → possibly you are overconfident ● Does your mental image of the distribution of data reflect the real world? ● What is your accuracy over time? ○ are you prone to a certain bias? ○ is there an area you can learn more about to increase your accuracy? Source: https://hbr.org/2016/05/superforecasting-how-to-upgrade-your-companys-judgment 3. evaluate

41. jankyri.com | mail@jankyri.com 3. Game time: Flash forecasting ● individuals’ forecasting ability improves already after 1 hour ● use teams boosts accuracy ● track prediction performance and provide rapid feedback ○ use past data for training ○ possibly monthly challenges → develop a “sanity compass” for facts 3. evaluate

42. jankyri.com | mail@jankyri.com 4. Act 4. act

43. jankyri.com | mail@jankyri.com 4. Action: Being data-driven What is “being data-driven”? 4. act

44. jankyri.com | mail@jankyri.com 4. Action: What is “being data-driven”? 4. act scientific method 1. hypothesis 2. prediction 3. test or falsify Kill the HiPPO X high data quality ● relevant ● trustworthy access to data ● you need ● when you need it Source: https://www.iconspng.com/image/122092/hippo-line-art

45. jankyri.com | mail@jankyri.com 4. act consume numbers (stand up, dashboard) make argument with data explore and use data for decision 4. Action: What is “being data-driven”?

46. jankyri.com | mail@jankyri.com 4. Action: Old school data-driven 4. act Meteorologists & Intelligence Services ● they are sometimes wrong ● everybody has access to data ● people talk about data ○ standardized metrics ○ similar or common understanding of core data concepts

47. jankyri.com | mail@jankyri.com 4. Step 1: Become data literate 4. act data literacy the ability to understand, use and communicate data effectively

48. jankyri.com | mail@jankyri.com 4. Step 1: Become data literate: vanity vs. reality 4. act Metric Definition Relevant Context Actionable Distance km maintenance no Time h maintenance no Speed km/h travel yes A B Source: https://www.iconfinder.com/icons/285810/auto_automobile_car_sedan_vehicle_icon#size=128

49. jankyri.com | mail@jankyri.com 4. Step 1: Become data literate: OMTM 4. act Metric Definition Relevant Context Actionable Speed km/h travel yes A B focus on One Metric That Matters

50. jankyri.com | mail@jankyri.com 4. Step 1: Become data literate: Metric vs. KPI 4. act metric ● relevant number ● not always actionable ● meta information KPI ● measures core task ● has a target (even if only “line in the sand”)

51. jankyri.com | mail@jankyri.com 4. Step 1: Become data literate: Ratios 4. act A good metric or KPI is a ratio ● actionable ○ accelerate, brake ● inherently comparative ○ WoW, MoM, YoY: sudden spike or trend? ● allows for comparing “opposing” facts ○ speeding tickets/km 150 km/h 2018-04-13 average speed

52. jankyri.com | mail@jankyri.com 4. Step 2: Act on data 4. act A good metric or KPI changes the way you work

53. jankyri.com | mail@jankyri.com 4. Step 2: Act on data: daily or weekly checklist 4. act 1. What are my most important tasks? 2. What am I and my team working to achieve (OKRs)? 3. What is my One Metric that Matters? 4. What is my KPI and what is the target? 5. What is my hypothesis? action target feedbackunderstand

54. jankyri.com | mail@jankyri.com Recap & Wrap Up 1. Define a. data vs. insights 2. Disassemble a. quantitative vs. qualitative b. known vs. unknown c. context matters 3. Evaluate a. What You See Is All There Is b. have hypothesis c. make and falsify your own prediction 4. Act a. One Metric That Matters b. KPI = metric + target c. ratios are your friend

55. jankyri.com | mail@jankyri.com Thank you!

56. jankyri.com | mail@jankyri.com Questions?

Working with Data Workshop

Jan Kyri

Working with Data Workshop