Examining why correlations are useful tools to analyse results between two samples, such as expected and actual results.

1. Analysis 101: Correlation v
Causation
Correlations & Dependence

2. Well Known…
“Correlation doesn’t equal Causation”

3. WHY?

4. Definition: Causation
1kg 2kg
 Two or More Things Happen at Once
 Known, Observable Chain of Events/Links
 Maths Best Tool for Job (Iconic Models [Fairly] Simple)
 Doesn’t Need Tonnes of Data
This Goes Down
This Goes Up

5. Definition: Correlation
1kg 2kg
 Two or More Things Happen at Once
 Non-Observable Chain of Events (What’s in the black-box?)
 May or May Not be Causal
 Statistics Best Tool for Job - Strength of Correlation can be Considered the ‘Chance’ they’re Causal
 …But go find out for sure!
This Goes Down
This Goes Up

6. Because for All You Know…
This Goes Down
This Goes Up
1kg 2kg
2kg
 In Complex/Black Box Systems, Never Assume
 Makes an Ass out of U and Me!

7. Calculating Correlation Coefficients
• Many Equations/Algorithms
• Most Famous, Pearson Product Moment
Coefficient
• BEWARE: Pearson spots Linear Correlations!

8. Calculating Correlation Coefficients
• Calculations for Data Samples
• Bars denote Mean-Average for Datasets X & Y
• Xi, Yi Denote i-th Sample of Dataset X and Y
• Commonly Available
– Excel Correl() Function = Sample Correlation
– Cor() in R

9. Correlations Need DATA!
• …Because it’s Statistical
• 25 – 30 Data Points Minimum for Each of X & Y
• < 25 Causes Greater Uncertainty
• Sign-is Important!
– Positive Correlation = Increase in Y when X increases
– Negative Correlation = Decrease in Y when X increases
• Magnitude = Correlation Strength.
Value of the Correlation
Coefficient
Strength of Correlation
1 Perfect
0.7 - 0.99 Strong
0.4 - 0.69 Moderate
0.1 - 0.39 Weak
0 - 0.09 Zero

10. EXAMPLE: Maths & Writing Tests
Maths Data Writing Data
Data Point Math Scores Data Point Math Scores
1 44.5 14 41.2
2 44.7 15 66.4
3 70.5 16 51.0
4 54.7 17 46.9
5 38.4 18 53.0
6 61.4 19 52.3
7 56.3 20 59.6
8 46.3 21 59.3
9 54.4 22 50.3
10 38.3 23 52.2
11 58.8 24 41.8
12 45.1 25 46.4
13 53.9 26 49.9
Data Point Writing Scores Data Point Writing Scores
1 64.5 14 51.5
2 43.7 15 65.1
3 56.7 16 59.3
4 56.7 17 56.7
5 46.3 18 54.1
6 64.5 19 43.0
7 39.1 20 56.7
8 39.1 21 54.1
9 51.5 22 47.6
10 64.5 23 48.9
11 43.7 24 48.9
12 41.1 25 54.1
13 59.3 26 64.5
Correlation = 0.215601457 (Not very strong)
CONCLUSION: Can’t use maths test scores as any sort of expectation on
written tests

11. Visualising Correlations
Graphs
e.g. Change in one variable
presents closely matched by a
change another.
Correlation Matrices
Dataset X
Dataset Y
e.g. Quantitative Surveys.
1. Count respondent scores per
question
2. Plot questions against each other

12. Great For
• A/B-testing Hypotheses
• Effect of Retrospective
Changes on Stories
– Multiple Items = Multiple
Data Points
• Guerrilla Testing Factors
– Post-experiments
• Experimental Verification
• Empirically Verifying Claims
from Politicians ;)
Not So Good For
• Where System Statics and
Dynamics are Known
– Unless identifying reasons for
error
– Simply Generates Waste
Otherwise
• Qualitative Results
• Retrospective Changes Where
Only a Handful of Results Are
available
– e.g. team changes or sickness
(unless you have enough data)

13. Advanced Concepts
• Using Multiple, Linked Correlations Increases
Certainty
– Identify Factors or Behaviours…
– …Potentially using Other, Strongly Correlated
Variables
• Correlation Matrices First Step in Factor
Analysis
– Identifying Influential Factors Above the Noise

14. Thanks for Viewing
Further Reading
Correlation (Math is Fun, Advanced)
http://www.mathsisfun.com/data/correlation.html
“Pearson Product-moment Correlation Coefficient” Wikipedia
http://en.wikipedia.org/wiki/Pearson_product-moment_correlation_coefficient
“Correlation & Dependence” Wikipedia
http://en.wikipedia.org/wiki/Correlation_and_dependence
Factor Analysis
http://en.wikipedia.org/wiki/Factor_analysis
Ethar Alali @EtharUK @Dynacognetics
Managing Director & Chief Architect
Polymath-MathMo. Programming since 9 years old. TOGAF 9 Certified, Classic
and Agile-EA, change agent.
Blog: GoadingtheITGeek.blogspot.co.uk
About Us
Specialist ICT Strategists & Advisors.
Member of HiveMind Network for some of
the biggest household and corporate multi-nationals.
Accredited Growth Voucher Advisors
certified to deliver IT & Web Growth
Consultancy as part of the UK government’s
Growth Voucher Scheme.
Accreditations & Associations