This document summarizes a lecture on interpreting data like a professional. It outlines that the lecture will teach how data fuels product vision mapping, review the main data types and their uses in product development, and explain how to use frameworks and key performance indicators to achieve a product vision. The lecture also discusses defining and collecting structured versus unstructured data, using customer journey maps, personas, and customer lifetime value as frameworks to focus data interpretation, and selecting metrics that matter like net promoter score and customer retention rate.

1. Dawn of the Data Age Lecture Series
Interpreting Data Like a Pro

2. Hi. I’m Luciano Pesci…
Co-Founder & CEO, EMPERITAS
● Team of economists and data scientists delivering bi-weekly Customer Lifetime Value intelligence so
our clients can beat their competitors for the most profitable customers.
Founder & Director, Utah Community Research Group, Univ. of Utah
● Teach microeconomics, data science, applied research, & American economic history.
2

3. Today’s Lecture Outline
● Teach you why data is fuel for product vision mapping.
● Show you the main data types & their uses in product dev.
● Explain using Frameworks & KPIs to achieve your Product Vision.
3

4. 4Data & Product Vision Mapping

5. Data is Fuel
● Data = Information
○ It’s an efficient way to express information.
○ It has no mystical (or mythical) qualities.
● People who are good at predicting the future
(Superforecasters*) are flexible in their beliefs &
assess new info given what they already know.
5
*Superforecasting: a.co/1aXPu8x

6. What’s “Analytics?”
● Analytics is best done as an agile process.
○ It’s very different from traditional statistics, and
requires a Data Detective type of data scientist.*
● It isn’t about blindly following data, it’s
about using all available information plus
your intuition to understand something.
6
*Types of Data Scientists (00:17:39): youtu.be/KMMvChAYV2g

7. Dashboard Dependence
● Beware of dashboard dependence!
● Individual data points are of limited use
because there are no silver-bullet metrics.
○ Some metrics deserve to be readily visible (KPIs).
○ Avoid “vanity metrics” that aren’t actionable.
7

8. Quantify Your Roadmap
● Make a visualization (milestones, tasks, pivots).
○ Think dimensionally, use color & shape to show multiple
points of information (like DRIs and/or data type) along
with the sequential flow path.
● Use the SMART Goals* system to identify the data
you’ll get for/at each part of your roadmap.
8
*SMART Goals (00:01:15): youtu.be/VqMCK7Whyd4

9. Hypothesis Testing
● Your tests need to be formalized “a priori.”
○ Should be well organized and trackable.
○ Your system must survive you being hit by a truck.
● Test should be done multiple times to
ensure results are from sampling error.
9

10. A Warning on Testing
● Two errors are possible when testing (law example):
○ Type 1 Error (false positive) - innocent person is found guilty
○ Type 2 Error (false negative) - guilty person is found innocent
● “Confidence” requires more evidence for each test.
○ Higher confidence levels more more evidence.
○ Using your results & intuition, pivot if necessary.
10

11. 11Data Types & Uses for Product

12. Differing Definitions of Data*
12
*Data Types (00:02:34): youtu.be/SirK0SSBeZg
● There are many ways to define data, each
requires a different approach when utilizing it:
○ Origin - How it was created.
○ Totality - If it’s a sample or a census.
○ Scope - Whether it’s been captured over time.
○ Measurement - How it was quantified.

13. Structured vs Unstructured
● PMs have to decide how to collect data:
○ Structured Data is usually pre-coded with a value or label.
○ Unstructured Data is often text and needs coding.
● Usually means scale-based feedback versus
direct comments from users/customers.
○ App store reviews have both (star rating & text-entry).
13

14. Project Manager Use Cases for Data
● Feature selection (new & backlog).
● Segmenting user/customers.
● Understanding sensitivities.
● KPI tracking (seasonalized).
● Predictive usage modeling.
14

15. Misinterpretation
● Even with the cleanest most complete data you
can mess things up through misinterpretation.
● The only solution is to involve multiple people.
○ Everyone will see things from different perspectives, so
you need to be able to work constructively together.
■ Superforecasters proved this was possible & powerful.
15

16. 16Frameworks & KPIs With Impact

17. Metrics That Matter
● Top Metrics For Project Managers*
○ Net Promoter Score
○ Customer Retention Rate
○ Customer Lifetime Value
● If you can break the departmental
silos and combine different data
you’ll get better metrics & KPIs.
17
*Becoming A Data-Driven Product Manager: goo.gl/j6v3yk

18. Using Frameworks
● You have to decide on how to organize and
interpret your data or you’ll get lost in it.
● Frameworks like the Customer Journey, Personas,
and Customer Lifetime Value force you to focus
all your data on things with high impact.
18

19. Customer Journey Map
● The journey a customer takes from initial awareness
through conversion, product usage to churn.*
● What you do as PMs is being influenced by
the marketing & sales efforts earlier in this journey.
19
*Hack Your Customer Journey: youtu.be/DKBr4PTANDA

20. Personas
● Not all users/customers are equal.
● Personalization is expected, use your data
to move beyond averages to groups.
○ Ultimately you should understand every individual.
20

21. ● Total value of a customer from first purchase to
churn. Requires historical data & future prediction
(to get the present discounted value till churn).
○ Includes monetary & non-monetary components.
● Pareto Principle** means there’s a pareto persona.
Customer Lifetime Value*
21
*Calculating Your CLV: youtu.be/iCX-afWhmZ4
**Pareto Principle (00:04:16): youtu.be/pyNrxUB-tBc

22. 22Summing Up

23. What We Covered Today...
● Why data is fuel for product vision mapping.
● The main data types & their uses in product dev.
● Using Frameworks & KPIs to achieve your Product Vision.
23

24. JOIN US FOR THE NEXT LECTURE
Customer Research for Product Managers, Thursday March 1st, 2018
emperitas.com/lecture