Unfortunately, I've seen many "comprehensive data literacy guides" that fail to cover the breadth of that topic. Therefore rather than try to cover it in a single guide, I'm going to try to cover it from a number of angles, over successive articles. This first part of the slide deck shows Lesson 1, which is about avoiding some common statistical biases and pitfalls (aka banana peels) when discussing simple ratios, KPIs, P&Ls, budgets, etc.

1. Avoiding costly pitfalls in
interpreting business
data
COMMON SENSE Data Insights
Subscribe to this Linkedin Newsletter: COMMON SENSE Data Insights

2. Course Outline
Lesson 1.
Simple KPIs,
ratios and
subtotals
Lesson 3.
Probability
and risk
Lesson 4.
Association,
correlation
and
regression
Lesson 5.
Data
Gathering
Lesson 2.
Data
visualisation
Lesson 6.
Machine
learning and
AI
Subscribe to this Linkedin Newsletter: COMMON SENSE Data Insights

3. 1. Simple KPIs, ratios
and subtotals
We will cover these potential pitfalls:
• Aggregation without context or drilldown
• Starting with the data instead of starting with
the high value actionable insights
• Inconsistent units of analysis
• KPIs that that don’t drive the right behaviour
Subscribe to this Linkedin Newsletter: COMMON SENSE Data Insights

4. Aggregation without context or drilldown
Explanation / Examples
Aggregation is something that we do
every time we produce a P&L or report
any ratio or metric whatsoever. It has
both advantages and pitfalls:
Advantages:
◦ Allows us to quickly digest, compare
and draw conclusions about a complex
reality.
Pitfalls:
◦ Metrics based on totals, ratios and
averages can be distorted by abnormal
distributions and outliers, and that’s
where the real insights might be
hidden.
◦ Show me some trends and benchmarks
so I can understand context
◦ Show me the performance of the teams
in the top and bottom quartiles.
◦ Show me the performance of the
individuals in the top and bottom 1% ,
regardless of team
◦ What causes this underperformance
and overperformance?
◦ Can we slice & dice the performance by
categories other than team (customer,
supplier, location etc?)
Techniques that preparers
should use
Explore the data using the following
visuals:
◦ trend line,
◦ histogram,
◦ box plot,
◦ principal components analysis (PCA)
◦ scatter-diagram,
◦ pareto charts,
◦ Interactive dashboards with drilldowns,
and slice & dice features
Use the above analysis for your own
insights gathering. Then share only the
actionable insights with your executive
audience.
Examples: Profit, churn, NPS. Essentially any subtotals, ratios, and indices - if presented without context or drilldown.
Subscribe to this Linkedin Newsletter: COMMON SENSE Data Insights
Questions a data literate
audience should ask

5. Starting with the data instead of the
actionable insights
Explanation / Examples
◦ Trying to understand customers by
looking at CRM data only, and not
understanding the customer journey,
segmentation, gain creators, pain
creators, persona and value
proposition.
◦ Building a data warehouse before
identifying KPIs.
Questions a data literate
audience should ask
◦ What insights are we trying to see?
◦ Give me some examples of the
decisions and actions we will take
based on these insights?
◦ What amount of effort is required to
gather, clean, structure and report on
this data?
◦ How much data is going into our data
pipeline without a defined use-case?
Techniques that preparers
should use
◦ Focus first on the insights that have the
highest strategic value.
◦ Understand what data we have but
don’t need, and what data we need but
don’t have.
◦ Build the data pipeline in a manner
that delivers the required insights fast,
but allows for future scaling.
Subscribe to this Linkedin Newsletter: COMMON SENSE Data Insights

6. Inconsistent unit of analysis
Explanation / Examples
◦ Comparing apples with oranges eg.
departments vs locations
◦ Comparing financial results in different
currencies (yes I’ve actually seen people load
different currencies into a data warehouse and not
convert them to a common currency).
◦ Showing long term monetary trends
without adjusting for inflation.
Questions a data literate
audience should ask
◦ Are we comparing locations or
departments?
◦ How did you handle the fluctuations in
exchange rate and inflation between
the beginning and end of the period?
Techniques that preparers
should use
◦ Prepare a data dictionary, with
definitions and names that executives
understand.
◦ Exchange rate fluctuations are best
handled in an ERP system and exported
to a data pipeline as a single,
consolidated currency.
◦ If there is significant inflation during
the period analysed, monetary values
should be converted from nominal to
real values.
Subscribe to this Linkedin Newsletter: COMMON SENSE Data Insights

7. KPIs that don’t drive the right behaviour
Explanation / Examples
◦ Incentivising the manager of a start-up
business division on profit instead of
growth. The result is that the manager
may maximise profit at the expense of
growth. Over the medium term, the
organisation would have made more
profit if they had focused the manager
on growth in the early years.
Questions a data literate
audience should ask
◦ Are there ways for the people being
measured to improve their KPIs without
improving the organisation's
performance?
◦ If a team is about to fall 5% short of
their annual target, what behaviours
are we likely to see and are they
aligned with our strategy and values?
Techniques that preparers
should use
◦ Have empathetic conversations with
the people who will be measured and
try to understand the implicit
incentives hidden in the measures.
◦ Use value driver analysis and
multivariate regression analysis to find
potential drivers of performance. Then
use A/B testing to validate whether
these are the true drivers.
◦ Use a balanced scorecard to ensure
that one organisational objective is not
sacrificed to maximise another one.
◦ Rather than using top down targets,
gamify performance improvement by
running league tables between teams.
Subscribe to this Linkedin Newsletter: COMMON SENSE Data Insights

8. Course Progress
Lesson 1
Simple KPIs,
ratios and
subtotals
Lesson 2
Data
visualisation
Lesson 3
Probability
and Risk
Lesson 4
Association,
correlation,
and
regression
Lesson 5
Data
gathering
Lesson 6
Machine
Learning
and AI
Subscribe to this Linkedin Newsletter: COMMON SENSE Data Insights

9. That wasLesson 1.Subscribe to this
Newsletter for Lessons 2-6.And please
comment to tell me what topics you
would like more detailon
Subscribe to this Linkedin Newsletter: COMMON SENSE Data Insights