In qualitative research we are primarily concerned with trustworthiness and credibility Quantitative research is concerned with reliability (can the results be replicated), and validity (do the instruments measure what they are supposed to)

1. Data Quality in
Quantitative
Research
Dr Ryan Thomas Williams

2. Reliability and validity
• In qualitative research we are primarily concerned with
trustworthiness and credibility
• Quantitative research is concerned with reliability (can the results be
replicated), and validity (do the instruments measure what they are
supposed to)

3. Validity
• Winter (2000, p. 1) defines validity as ‘a demonstration that a particular
instrument or fact measures what it intends, purports or claims to measure, that
an account accurately represents those features that is intended to describe,
explain or theorise’.
• Researchers must be able to evidence that instruments used for understanding a
research phenomenon are connected in this way.
• This is particularly important in abstract research or theoretical constructs such
as, intelligence, anxiety, motivation, and empathy as there are no uniformly
agreed measures or units
• Foci of validity may include the following items: credibility, transferability,
dependability, and confirmability

4. Main types of validity
• Construct validity- is the researcher measuring what they think they
are?
• Statistical validity- does the independent variable change because of
chance? Statistically significance
• Internal validity- was the research carried out correctly? Design and
outcomes
• External validity- can the results be generalised?

5. Bases of validity in quantitative research Bases of validity in qualitative research
Controllability Natural
Isolation, control, and manipulation of variables Thick description and high detail on required or
important aspects
Replicability Uniqueness
Predictability Emergence, unpredictability
Generalizability Uniqueness
Context-freedom Context-boundedness
Fragmentation and atomization of research Holism
Randomization of samples Purposive sample/no sampling
Neutrality Value-ladenness of observations
Objectivity Confirmability
Observability Observability and non-observable meanings and
intentions
Inference Description, inference, and explanation
Etic research Emic research
Internal validity Credibility
External validity Transferability
Reliability Dependability
Observations Meanings

6. Reliability
• Reliability is often described as precision and accuracy within a
research project and explores how consistent and replicable the
results are over time
• Guba and Lincoln (1994) argue that the concept of reliability is largely
positivist meaning that terms such as ‘consistency’ in the context of
testing is quantitative
• Pilots
• Requires a ‘degree of control and manipulation of the phenomena

7. How can we ensure reliability?
• Test and retest- the sample for high correlation between each test.
• Inter-observer- lack of consistency in judgment by observer.
• Parallel reliability- are the indicators consistent or is there too much
overlap in resultant answers?
• Internal consistency- such as Cronbach’s alpha. Would normally be
used to test the internal reliability of the questions asked.

8. Cronbach’s Alpha
• A group of respondents are presented with a set of
statements designed to measure optimistic and pessimistic mindsets.
They must rate their agreement with each statement on a scale from
1 to 5. If the test is internally consistent, an optimistic respondent
should generally give high ratings to optimism indicators and low
ratings to pessimism indicators. The correlation is calculated between
all the responses to the “optimistic” statements, but the correlation is
very weak. This suggests that the test has low internal consistency

9. Table 22. Konting et al’s (2009)
Cronbach’s alpha scale.
Cronbach’s alpha Reliability level
0.91-1.00 Excellent
0.81-0.90 Good
0.71-0.80 Acceptable
0.61-0.70 Questionable
0.51-0.60 Poor
< 0.5 Unacceptable