Quantitative data essentials for charities - Learning Lab

© 2021 Copyright ISC Ltd.
Quantitative Data Essentials
Ian J Seath
Improvement Skills Consulting Ltd.
“Without data, you’re just another person with an opinion”
W. Edwards Deming

Session outline (09:30-11:00)
Understanding what quantitative data is and when to use it
Data sources and data collection methods
Basic analysis techniques to help describe your data
Pictorial description of data – essential charts
Potential pitfalls – how much data is “enough”?
Using data to draw conclusions and support decision-making
Workshop Notes:

My third sector experience
◼ Trustee:
❑ Dachshund
Health UK
◼ Non-Exec
Director:
❑ The Kennel
Club
◼ Steering
Group:
❑ Pro Bono OR

Session outline
Understanding what quantitative data is and when to use it
Data sources and data collection methods
Basic analysis techniques to help describe your data
Pictorial description of data – essential charts
Potential pitfalls – how much data is “enough”?
Using data to draw conclusions and support decision-making

3 reasons you might want to use data
To describe what is happening
currently or historically
These are descriptive statistics
To forecast what might happen
in the future, based on past
data
These are predictive analytic models
To make decisions in the face
of uncertainty
These are models to evaluate
alternative options

3 reasons you might want to use data
To describe what is happening
currently or historically
To forecast what might happen
in the future, based on past
data
To make decisions in the face
of uncertainty

ULTIMATELY, ALL DATA ARE
QUANTITATIVE!
Quant. vs. Qual.

What’s your biggest quantitative data challenge?
◼ Poll:
❑ We don’t know what data we should collect
❑ We don’t know how best to analyse our available data
❑ We don’t know how best to present our data analyses
❑ We don’t have the right tools or IT to make best use of data
❑ We don’t know how to use data to improve our performance
❑ Not enough people are interested in using data to improve our
performance

Data Orchard’s Data Maturity Framework
https://www.dataorchard.org.uk/resources/data-maturity-framework

5 uses of data
User data: Information on the characteristics of the people you are
reaching.
Engagement data: Information on how service users are using
your service, and the extent to which they use it.
Feedback data: Information on what people think about the
service.
Outcomes data: Information on the short term changes, benefits
or assets people have got from the service.
Impact data: Information on the long-term difference that has
resulted from the service.
More info: https://www.inspiringimpact.org/learn-to-measure/plan/decide-what-data-to-collect/

The Golden Rules of Measurement
➢ No measurement without recording
➢ No recording without analysis
➢ No analysis without action

Data sources
• CRM systems (e.g. Salesforce, Zoho)
• Mailing lists (e.g. Mailchimp)
User data
• Booking systems (e.g. Eventbrite)
• Finance systems (e.g. Xero)
Engagement data
• Survey systems (e.g. SurveyMonkey, Google/Microsoft Forms)
• Presentation tools (e.g. Mentimeter, Poll Everywhere)
Feedback data
• Survey systems
• Focus Groups, Interviews
Outcomes data
• Published statistics (e.g. ONS)
• Research projects/Impact Analysis
Impact data

Don’t forget: simple data collection tools
Checksheets/Tallysheets
Concentration Diagram
(John Snow Broad St. pump cholera 1854)
Feedback Emojis
Dot voting

◼ An aeroplane flies round the four 100-mile sides of
a square
◼ It flies at 100 mph on side 1, 200 mph on side 2,
300 mph on side 3 and 400 mph on side 4.
◼ What is its average speed?
100 m.p.h.
300 m.p.h.
200 m.p.h.
400 m.p.h.
Answer in
Chat
100 miles
100
miles

THE “MISLEADING” AVERAGE
“Lies, damned lies and statistics.”

Do you know what “average” means?
◼ The length of time (in days) taken to process 10 grant
applications was recorded
◼ What was the average time it took
(from application received to completion)?
Grant
1
Grant
2
Grant
3
Grant
4
Grant
5
Grant
6
Grant
7
Grant
8
Grant
9
Grant
10
6 6.5 7 7 7 7.5 8 8 10 13

Mean, Median and Mode
◼ Arithmetic Mean - the sum of values divided by the number of values, often called “the
average” (8.0 in our example)
◼ Median - the middle value when all the values are arranged in order [or the mean of the two
middle values if there is an even number in the list] (7.25 in our example)
◼ Mode - the most frequently occurring value (7 in our example)
If the Mean = the Median, the data is distributed symmetrically
The Median and Mode are not affected by extreme values in a set of data, unlike the Mean
Grant
1
Grant
2
Grant
3
Grant
4
Grant
5
Grant
6
Grant
7
Grant
8
Grant
9
Grant
10
6 6.5 7 7 7 7.5 8 8 10 13

Which “average” would you use & why?
0
2
4
6
8
10
12
1 2 3 4 5 6 7 8 9 10
No.
of
cases
Time to repond (Days)
Time to respond to Grant Application (Days)
N = 33
0
2
4
6
8
10
12
1 2 3 4 5 6 7 8 9 10
No.
of
cases
N = 33
A B

Some more questions…
◼ Which one would you want to be held accountable for managing?
◼ Where would you set a Service Level Agreement?
0
2
4
6
8
10
12
1 2 3 4 5 6 7 8 9 10
No.
of
cases
N = 33
0
2
4
6
8
10
12
1 2 3 4 5 6 7 8 9 10
No.
of
cases
N = 33
Mean Median Mean Median
3.6 3 4.7 5

Excel makes this easy…
https://www.excel-easy.com/examples/histogram.html

It’s helpful to understand Variation
Bell-shaped Skewed
Plateau
Bi-modal

What a Histogram might tell you
◼ Bell-shaped - a symmetrically shaped distribution which typically represents
data randomly distributed, but clustered around a central value
◼ Positive or negative skews - where the average value of the whole set of data
is to the left (-) or right (+) of the central value. Look out for specification
limits at the boundaries of the distribution which might be causing data to be
dropped from the population. More extreme shapes are also known as
“precipices”
◼ Bimodal - where there are two peaks. Usually indicates two sets of data (e.g.
two teams or locations), with different Means have been mixed
◼ Plateau - occurs where several sets of data have been mixed (e.g. from a
number of customers/locations/groups)

GRAPHS AND CHARTS
“A picture paints a thousand words.”

Graphs and Charts

When to use Graphs for data
◼ Use graphs when you have more than ten data points, or if
you want to show people “the big picture”, not detailed data
◼ Use graphs when you need to show trends, over time
◼ Don’t clutter a graph with too many different sets of data; it’s
usually better to split the data into separate graphs

Pie Charts
◼ The data points in a Pie
Chart are displayed as a
percentage of the whole
pie
◼ Good for: showing
proportions, at a glance
◼ Not good for: showing
trends or comparisons
over time
Avoid using 3D Pie
Charts

Bar Charts
◼ In Bar Charts, categories are
typically organised along the
horizontal axis and values up
the vertical axis
◼ Bar Charts illustrate
comparisons among individual
items and may be “stacked”
or “100% stacked”
◼ Good for: showing quantities of
responses in different
categories; often best when
sorted into biggest to smallest
◼ Not good for: showing trends
over time (use a Line Graph)

Histograms
◼ In Histograms, a variable (e.g.
Time) is displayed along the
horizontal axis and frequency up
the vertical axis
◼ Good for: showing the variation
in a set of data and to help
decide if the Mean or Median
are the best choice of average
to quote
◼ Not good for: showing variations
over time
◼ N.B. Excel also calls these “Bar
Charts” unless you use the Data
Analysis Add-in Pack

PARETO ANALYSIS
“Separate the vital few from the trivial many.”

Pareto Analysis
20%
80%
◼ 80% of problems or errors are often due to only 20% of the
causes (The “Vital Few”)
◼ The remaining 80% of causes account for only 20% of the
problems or errors (The “Trivial Many”)
Causes
Problem
Occurrences
Also known as the 80:20 rule
20%
80% The “Vital
Few”
Causes
The “Trivial
Many”
Causes
Most of
the
problems

Pareto Diagram
◼ A Pareto Diagram is a
particular type of Bar Chart
◼ Category data is presented in
decreasing size, from left to
right and a Cumulative % line
is also drawn
◼ Good for: showing the 80:20
Rule – highlighting the few
categories that account for the
majority of performance or
issues
◼ Not good for: showing data
over time (but sometimes worth
showing “before” and “after”)

IS PERFORMANCE IMPROVING?
“Two data points do not indicate a trend.”

Line Graphs
◼ In a Line Graph, time data is
distributed evenly along the
horizontal axis, and all value
data is distributed up the vertical
axis
◼ Good for: showing how results
have changed over time (trends)
◼ Not good for: comparing lots of
different sets of results (too
many lines make it hard to see
what's going on)
◼ N.B. Excel enables you to
overlay a statistically derived
trend line

Poll
◼ Which of the charts should you be making more use of?
❑ Pie
❑ Bar
❑ Histogram
❑ Pareto
❑ Line

HOW MUCH DATA IS ENOUGH?
“Anecdotes are not statistics.”

Sampling
◼ In many cases, we obtain data through sampling; often
because it is simply not possible to measure every single
item, or to log every activity, transaction or contact
◼ The purpose of sampling is to collect an unbiased subset
which will give you a manageable amount of data
◼ When you take samples, they should be representative
(statistically valid and reliable) and economic to collect
(quick and cost-effective)

Population vs. Sample
Beneficiary Satisfaction
Unhappy Happy
If we surveyed every single
beneficiary over a year to find
out how happy they were with
our support, this is what we
might find.

Contact Centre’s sample of 10 people
Customer Satisfaction
Unhappy Happy
How happy are our
beneficiaries according
to our staff?

Volunteers’ sample of 10 people
Unhappy Happy
How happy are our
beneficiaries according
to our volunteers?

Validation: depends on sample size
Unhappy Happy
Your ability to validate
beneficiary satisfaction data
depends on sample size.
If you pick too small a sample
you could, purely by chance,
find very different results and
draw the wrong conclusions.

http://www.surveysystem.com/sscalc.htm

If you have 6000 beneficiaries per year
+ or - 3
We serve 500 beneficiaries/month
We should sample 75 beneficiaries/month
You might, therefore, say if 83% of
beneficiaries are ‘Happy’:
“We are 95% confident that between
80 and 86% of beneficiaries are Happy”
You can also work out
the CI for a known
sample size

Terms you need to understand
◼ Confidence Interval (Margin of Error)
❑ The plus-or-minus figure usually reported in
newspaper or television opinion poll results
❑ If you pick a CI of 5 and 83% of your sample picks
‘Happy’, you can be “sure” that the 78-88% of the
entire population would have picked ‘Happy’
◼ Confidence Level
❑ Tells you how “sure” you can be that the population
would pick an answer within the Confidence Interval
❑ A 95% CL is most commonly used and means, for the
example above, you can be 95% sure that the true
population is between 78 and 88%

Sampling guidelines
◼ With static populations (e.g. customers, staff), use random sampling; for
example using Random Number Tables to decide what (and when) to sample
❑ Random sampling means that every unit in a population will have an equal probability of
being chosen in the sample
◼ With time-based data, collect data in sub-groups of 5 values, equally spaced
in time (e.g. services are delivered, or transactions are carried out
continuously over a period of time – call handling in a contact centre)
❑ If it is not feasible to take sub-groups, take individual values at regular intervals; e.g. every
10th or 100th

“If you wait for the perfect set of data, you’ll be waiting a
very long time”

Concentration Diagram – what does the data say?
◼ This is a well-known example
from WW2
◼ In order to decide where to
add extra protection to
bombers, the RAF recorded
where bullets had penetrated
returning aircraft
◼ Where would you add extra
protection?

Is weekly caseload increasing, decreasing or not changing?
What can you conclude
from this chart and its
source data?
Week
New
Cases Week
New
Cases
1 35 11 45
2 79 12 37
3 125 13 102
4 85 14 47
5 60 15 52
6 3 16 16
7 138 17 9
8 120 18 86
9 116 19 60
10 40 20 66

4 Week Moving Average
Quick & dirty:
Weeks 1-10 average = 80
Weeks 11-20 average = 52
https://www.excel-easy.com/examples/moving-average.html
Week
New
Cases Week
New
Cases
1 35 11 45
2 79 12 37
3 125 13 102
4 85 14 47
5 60 15 52
6 3 16 16
7 138 17 9
8 120 18 86
9 116 19 60
10 40 20 66

Forecasting can be hard, especially for the future!

Pro Bono OR
◼ The Pro Bono OR scheme connects volunteer analysts with good causes. Analysts donate their time
and skills to help charities or other voluntary organisations facing difficult decisions or looking for
improvement.
◼ The OR Society wants more organisations to benefit from operational research (OR) and recognises
the great opportunity to bring together third sector organisations who would find OR hugely beneficial,
with analysts who are looking for ways of contributing to society over and above their day-job.
◼ Since Pro Bono OR was founded in 2011, volunteer analysts have undertaken over 100 projects
helping third sector organisations achieve better insight, better decisions or better ways of working.
◼ We are able to provide a wide range of support in the following areas:
• Strategic planning/review
• Data analysis and insight
• Options appraisal
• Decision-making
• Process improvement
• Impact measurement
• Contact: amy.hughes@theorsociety.com

ian.seath@improvement-skills.co.uk
07850 728506
@ianjseath
uk.linkedin.com/in/ianjseath
Prepared for Measuring the Good and Coalition for Efficiency by
Ian J Seath
Improvement Skills Consulting Ltd.
www.improvement-skills.co.uk

Quantitative data essentials for charities - Learning Lab

Recommended

Recommended

More Related Content

What's hot

What's hot (12)

Similar to Quantitative data essentials for charities - Learning Lab

Similar to Quantitative data essentials for charities - Learning Lab (20)

More from Improvement Skills Consulting Ltd.

More from Improvement Skills Consulting Ltd. (20)

Recently uploaded

Recently uploaded (20)

Quantitative data essentials for charities - Learning Lab