This document summarizes a statistics lecture about the research process and why statistics are needed in optometry and vision science. It discusses the steps of evidence-based practice including asking questions, acquiring evidence, appraising evidence, and applying evidence. It also covers generating and testing theories, levels of measurement, measurement error, validity, reliability, types of research such as correlational and experimental research, and methods of data collection and analysis. The goal is to explain the research process and why statistics are an essential tool for evidence-based practice in optometry.

1. STATISTICS LECTURE NUMBER 2: THE
RESEARCH PROCESS AND WHY DO
OPTOMS AND VISION SCIENTISTS NEED
STATS?
Given by: Dr Kirsten Challinor1

2. COMMONWEALTH OF AUSTRALIA
Copyright Regulations 1969
WARNING
This material has been reproduced and communicated to you by or on behalf of the
University of New South Wales pursuant to Part VB of the Copyright Act 1968 (the Act).
The material in this communication may be subject to copyright under the Act. Any further
reproduction or communication of this material by you may be the subject of copyright
protection under the Act.
Do not remove this notice.

3. IN THE LAST LECTURE…
3

4. EVIDENCE BASED PRACTICE
EBP
(Hoffman, 2010)
4
EBP is the combination of the best available evidence from research, the patient’s
preferences/circumstances, the clinical environment and the practitioner’s expertise.

5. IN SUMMARY-PROCESS OF EBP
ASK formulating
answerable
questions
ACQUIRE
searching for the
best evidence
APPRAISE
critically assess
the evidence
APPLY
the appraised evidence
to patient / practice
AUDIT
evaluating outcome of EBP
process
5
(Dawes, 2005)
Covered in your tutorial homework
https://www.eboptometry.com/

6. 6
http://www.students4bestevidence.net

7. TODAY’S OUTLINE
 Explanation of each of the steps in the research
process flowchart
 Types of data
 Generating and testing theories
 Measurement error
 Validity
 Reliability
7

8. WHY DO WE NEED STATISTICS?
8

9. http://xkcd.com/1161/
9

10. APPRAISE
Evaluating the relevant research evidence, to find the highest
quality (most reliable, or valid) evidence available relevant to
your question.
Critical appraisal is the process of assessing and interpreting
evidence by systematically considering its validity and its
relevance to the question.
 Internal validity: the extent to which the research is reliable.
 External validity: is an indication of the generalisability of the
findings.
10

11. SCIENCE AND WHY WE NEED STATISTICS
 Provide knowledge and understanding of how the
world works by providing EVIDENCE
 Development of technology
 Philosophy
 No other species of animals is capable of this!
 A formal process of investigation of cause and
effect: - for that we need numbers!
11

12. APPRAISE
12
Questions Yes No
Were subjects randomized? The study is not likely to be biased by
subject grouping.
Subject allocation may cause bias.
Was there a control? Is the control group
within this study, or historical?
There is unlikely to be a placebo effect in
the treatment group. We can be less sure
of this, though, if the control group data
are taken from a previous study.
Subjects were in therapy, but there is no
comparison with those not in therapy, so
we cannot know to what extent any
treatment effect is due to the treatment.
Is the population clinically relevant for my
application?
Findings may be population-specific. The findings may apply to one population
but not to the population in which the
therapy is to be applied.
Is attrition (reduction in numbers)
described?
If attrition rate is low, the findings are not
confounded by this factor.
We do not know the results in subjects
who withdraw from the study.
Were experimenters and subjects “blind”
in this trial?
The findings are not biased by expectation
of outcomes.
The experimenters and the subjects may
have unintentionally or otherwise
affected the outcome.
Are the subject groups comparable? The subject groups were equal at
baseline, so are likely to have been
similarly affected.
Outcomes in the groups may differ due to
factors other than the treatment.
Was subject treatment equal across
groups, apart from the therapy?
The subject groups were equal in all
respects apart from the therapy.
Outcomes in the groups may differ due to
factors other than the treatment.
Are the results both clinically and
statistically significant?
The results are clinically relevant. Results may be statistically significant, but
have no clinical significance. They may not
be statistically significant, in which case
there is no effect.
http://www.eboptometry.com
EBP in action
Step 3: Appraise

13. 13
http://stats.stackexchange.com/questions/423/what-is-your-favorite-
data-analysis-cartoon

14. THE RESEARCH PROCESS
14

15. THE RESEARCH PROCESS
15
This flowchart is from your chapter 1 in your Andy Field textbook (see last slide).

16. INITIAL OBSERVATION
 Find something that needs
explaining
 Observe the real world
 Read other research
 Investigation
 Test the concept: collect data
 Collect data to see whether your
hunch is correct
 To do this you need to define
variables
 Anything that can be measured and
can differ across entities or time.
 What is being measured? What is
being manipulated? 16

17. THE RESEARCH PROCESS
17
This flowchart is from your chapter 1 in your Andy Field textbook (see last slide).

18. GENERATING AND TESTING THEORIES
• Theories
– An idea about a general principle or set of principles that
explain known findings about a topic and from which
new hypotheses can be generated.
• Hypothesis
– A prediction from a theory.
– E.g. the number of people turning up for a Big Brother
audition that have narcissistic personality disorder will
be higher than the general level (1%) in the population.
• Falsification
– The act of disproving a theory or hypothesis.
18

19. RESEARCH HYPOTHESIS
Hypothesis = A proposition for reasoning
= A suggestion as to why
something might be as it is
= A prediction from a theory.
A testable statement of the state of the world.
Question to class. Is this a testable statement?
“The Beatles were the most
influential band ever”.
Good theories produce hypotheses that are scientific statements.
Scientific statements are ones that can be verified with reference to empirical
evidence.

20. ACTIVITY
Turing a research question into a testable hypothesis
i) Identify if each statement in the list below is a scientific
statement or not?
Remember that scientific statements be can be proved/
are testable.
ii) For the statements that are not testable, can you
change their wording to make them scientific?
List of statements
• Chocolate is the best food.
• Watching television makes you happy.
• Cricket is the world’s most popular sport to watch.
• Coke is the worst drink.
20

21. THE RESEARCH PROCESS
21

22. DATA COLLECTION : WHAT TO MEASURE?
Hypothesis:
– Ice cream makes you happy
(produces endorphins).
Independent Variable
– The proposed cause
– A predictor variable
– A manipulated variable (in experiments)
– Ice cream in the hypothesis above
Dependent Variable
– The proposed effect
– An outcome variable
– Measured not manipulated (in experiments)
– Endorphins in the hypothesis above
22

23. LEVELS OF MEASUREMENT – NOT ALL DATA
ARE THE SAME
Categorical (entities are divided into distinct categories):
– Binary variable: There are only two categories
• e.g. dead or alive.
– Nominal variable: There are more than two categories
• e.g. whether someone is an omnivore, vegetarian, vegan, or
fruitarian.
– Ordinal variable: The same as a nominal variable but the
categories have a logical order
• e.g. whether people got a fail, a pass, a merit or a distinction in
their exam.
23

24. Continuous (entities get a distinct score and can take
on any value along a scale):
– Interval variable: Equal intervals on the variable
represent equal differences in the property being
measured
• e.g. the difference between 6 and 8 is equivalent to the
difference between 13 and 15.
– Ratio variable: The same as an interval variable, but the
ratios of scores on the scale must also make sense
• e.g. a score of 16 on an anxiety scale means that the person
is, in reality, twice as anxious as someone scoring 8.
24

25. MEASUREMENT ERROR
 Measurement is imperfect!
 Measurement error
 The discrepancy between the actual value we’re trying
to measure, and the number we use to represent that
value.
 Example:
 You (in reality) weigh 80 kg.
 You stand on your bathroom scales and they say 83 kg.
 The measurement error is 3 kg.
25
www.paduiblog.com

26. 26

27. VALIDITY
• Whether an instrument measures what it set out to
measure.
• Content validity
– Evidence that the content of a test corresponds to the
content of the construct it was designed to cover
• Ecological validity
– Evidence that the results of a study, experiment or test
can be applied, and allow inferences, to real-world
conditions.
27

28. RELIABILITY
 Reliability
 The ability of the measure to produce the same results
under the same conditions.
 Test-Retest Reliability
 The ability of a measure to produce consistent results
when the same entities are tested at two different points
in time.
28

29. VALID? RELIABLE?
29

30. TYPES OF RESEARCH
 Correlational research:
– Observing what naturally goes on in
the world without directly interfering
with it.
 Cross-sectional research:
– This term implies that data come from
people at different age points with
different people representing each age
point.
 Experimental research:
– One or more variable is systematically
manipulated to see their effect (alone
or in combination) on an outcome
variable.
– Statements can be made about cause
and effect.

31. THE POWER OF THE PLACEBO
BBC HORIZON
31
8-15min
Full documentary here:
http://www.dailymotion.com/video/x1moo91_horizon-2013-2014-8-the-power-of-
the-placebo_lifestyle

32. EXPERIMENTAL RESEARCH METHODS
Cause and Effect (Hume, 1748)
 Cause and effect must occur close together in time
(contiguity);
 The cause must occur before an effect does;
 The effect should never occur without the presence of the
cause.
Confounding variables: the ‘Tertium Quid’
 A variable (that we may or may not have measured) other
than the predictor variables that potentially affects an
outcome variable.
 E.g. The relationship between breast implants and suicide is
confounded by self esteem.
Ruling out confounds (Mill, 1865)
 An effect should be present when the cause is present and
that when the cause is absent the effect should be absent
also.
 Control conditions: the cause is absent.
32

33. METHODS OF DATA COLLECTION
• Between-group/Between-subject/independent
– Different entities in experimental conditions
• Repeated measures (within-subject)
– The same entities take part in all experimental
conditions.
– Economical
– Practice effects
– Fatigue
33

34. TYPES OF VARIATION
• Systematic Variation
– Differences in performance created by a specific
experimental manipulation.
• Unsystematic Variation
– Differences in performance created by unknown factors.
• Age, Gender, IQ, Time of day, Measurement error etc.
• Randomization
– Minimizes unsystematic variation.
34

35. THE RESEARCH PROCESS
35

36. 36
http://stats.stackexchange.com/questions/423/what-is-your-favorite-data-
analysis-cartoon

37. TYPES OF DATA ANALYSIS
37
 Quantitative Methods
 Testing theories using numbers
 Measurement
 Cause and effect – direct manipulations
http://stats.stackexchange.com/questions/423/what-is-your-favorite-data-analysis-
cartoon

38. TYPES OF DATA ANALYSIS
 Qualitative Methods
 Testing theories using language
 Magazine articles/Interviews
 Conversations
 Newspapers
 Media broadcasts
38

39. RESEARCH PROCESS SUMMARY

40. FURTHER INFO/ADDITIONAL MATERIAL
1) Spend some time looking at the textbook website. Set up
mobile study if you like (10mins)
http://www.uk.sagepub.com/field4e/study/default.htm
2) Hook up to EBP social media (10 mins)
Twitter examples
 @EBPoptometry
 @EvidenceMatters
 @EBMOnline
 @cochranecollab
Online examples
 http://www.badscience.net/about-dr-ben-goldacre/
 http://www.facebook.com/evidencebasedoptometry
 http://www.cochrane.org/about-us
 http://evidencebasedmedicine.com.au/ 40

41. FURTHER INFO/ADDITIONAL MATERIAL CONT
3) Watch this TED talk (14 mins):
http://www.ted.com/talks/ben_goldacre_battling_bad_
science.html
4) In the talk above, Goldacre talks about Placebos.
What is the placebo effect? Find a good definition
and write a few sentences for yourself about what a
placebo is. (10 mins)
http://www.plosone.org/article/info%3Adoi%2F10.137
1%2Fjournal.pone.0058247
5) Read the Sicily statement (Dawes et al, 2005). (25
mins). http://www.biomedcentral.com/1472-6920/5/1 41

42. ALWAYS REMEMBER:
“THE PLURAL OF ANECDOTE IS NOT
DATA”
https://sites.google.com/site/skepticalmedicine/the-
plural-of-anecdote-is-not-data
42
Anecdote - a short account of a particular incident or event,
especially of an interesting or amusing nature.
Data - a series of observations, measurements, or facts;
information

43. EBP FOR OPTOMETRY SITE
 http://www.eboptometry.com/content/optometry/step
-1-ask/practitioners-students-teachers/step-1-ask
43

44. COLDPLAY PARAODY
 https://www.youtube.com/watch?v=QUW0Q8tXVUc

44

45. http://www.uk.sagepub.com/field4e/main.htm
45
http://www.uk.sagepub.com/field4e/study/default.htm