Social science, quantitative and qualitative approaches

1. Measurement in social science:
quantitative and qualitative approaches
Federica Russo
Philosophy | Humanities | Amsterdam
russofederica.wordpress.com | @federicarusso

2. Overview
Measurement in social science
Conceptual issues
Methodological precepts
What do we actually measure?
Two examples: “socio-economic status” and “age”
Combining qualitative and quantitative methods
Observe on small scale before you measure
Measure on large scale based on observation
2

3. MEASUREMENT IN SOCIAL SCIENCE
3

4. A theory of measurement
A thriving debate in phil sci, see e.g.
https://philpapers.org/s/measurement
https://plato.stanford.edu/entries/measurement-
science/
Traces back to at least Suppes 1998:
Two problems for measurement:
The problem of representation
Attach a number to an ‘object’
The problem of determining the procedure
The choice of the scale also depends on theory
4

5. Social science procedures
Zeller and Carmines (1980) follow Blalock:
Measurement is the process of linking abstract concepts to
empirical indicators
The possibility of answering research questions
depends on the robustness of our measurement
procedures
Measurement procedures above theorising
5

6. Measurement and reality
Cartwright and Chang (2008)
Practitioner’s problem: whether measurements are correct
Philosopher’s problem: whether we measure what we want to measure
Bohrnstedt (2010)
In social science there are some clear and tangible measures
E.g. age, birth, number of children, marital status …
For more blurred concepts, observe the covariation between
indicators, and infer their reality
6

7. A mantra:
The more you measure, the better
Is it always the case?

8. Measurement itself, especially if carried out using sophisticated
instruments or analysed using complex methodology, is seen to have the
attributes of ‘science’, and often taken effectively as a justification for
believing the results that are presented as if they have a meaningful
relation to whatever social process they are claimed to
measure. […]
New technologies such as powerful dynamic computer graphics do have the
potential to convey findings and patterns in powerful ways, but whether
they are used to inform rather than merely impress, remains an
open question.
[…] a better understanding is needed of the difference between data
that ‘confirms’ a theory by providing a good model fit, and
data that allows us to explain observed data patterns using as
much potentially falsifiable information as possible.
Harvey Goldstein (2012)
8

9. WHAT
DO WE ACTUALLY MEASURE?
Measuring “socio-economic status” and “age”
9

10. At the extremes of measurement
Age
Very easy to measure
Does it just represent a
definite biological status?
Does it have any explanatory
import?
SES
Very controversial how we
should measure it
How can it represent one’s
status?
What is its import in
explanation of social or social /
health outcomes?
10

11. Measuring socio-economic status
Theoretical approaches
Weberian, Marxist, Colemanian, …
Identification of different indicators, different types of variables
Procedure: class stratification
E.g., Goldthorpe Class Schema
Grouping of types of workers
Why measuring SES?
E.g., correlation with health outcomes, or other economic
variables, …
12

12. What does SES do?
Categorise?
A classificatory variable
What part of the populations are more exposed, have higher
prevalence …
Explain?
Active part in the explanation of diseases
Mixed aetiology!
What are the active causal pathways from exposure to
outcome?
Social practices / norms / habits to explain (and to prevent) exposure
13

13. Measuring age
Easy to measure
Accessibility of data, straightforward question, …
How to measure it
Categorically, Continuously
Using Age
Control:
Adjust results of statistical analyses (control for age)
Predict:
Age structure helps predict results
Categorise:
Grouping and collapsing multiple categories into fewer categories; Care with loss
of information, residual confounding
14

14. What age stands for
Biological age
A typical health status, for that age
Social age
Social practices that are typical of that age
…
Any explanatory import in using age?
15

15. COMBINING QUANTITATIVE AND
QUALITATIVE APPROACHES
16

16. Modelling data, interpreting outcomes
Quantitative studies
• Large samples and data sets
• Analysis of correlations
• Validation via robustness
tests, etc.
Qualitative studies
• Small samples and groups
• Description of practices
• Difficult to generalise
17

17. The mantra, again:
The sample: the bigger, the better
Measurement: the more precise, the better
Is it really the case?
18

18. The extra information that
statistics does not give us
Description of
Practices
Interactions
Influences
Background
Norms
…
go SMALL first!
19

19. The information that
statistics does give us
Categorise the ‘practices, interactions,
backgrounds, …’ into measurable variables
Is it generalisable?
An empirical question. Not a priori, determined by size
NOW GO big!
20

20. TO SUM UP AND CONCLUDE
21

21. Quantitative measurement mantras: a social science trend,
also to meet requests from policy-making
What is evidence for decisions?
What methods we can trust?
…
Reflecting on measuring SES and Age we may conclude that
Quantitative measurement is not necessarily panacea
Qualitative measurement can help a great deal
Next item on the research agenda
How to combine quali- and quantitative data?
To what extent do QCA, mixed/multi-methods succeed?
22

22. SELECTED REFERENCES
23

23. Blalock H.M. All of his work!
Bohrnstedt G. (2010), An Overview of Measurement in the Social Sciences.
Burt R. 1991 Measuring age as a structural concept. Social Networks 13
Cartwright N. and Chang H. 2008 Measurement, in The Routledge Companion to
Philosophy of Science, pp. 367-375.
Cartwright N. and Bradburn N., A theory of measurement
Goldstein H. 2012. Francis Galton, measurement, psychometrics and social progress.
Assessment in Education: Principles, Policy & PracticeVol. 19, No. 2
Marks G. The measurement of socioeconomic status and social class in the LSAY project.
Technical Paper
Reijneveld S A 1998 Age in epidemiological analysis, J Epidemiol Community Health
2003;57
Suppes P. 1998 Theory of Measurement. E. Craig (Ed.), Routledge Encyclopedia of
Philosophy. pp. 243-249.
Zeller and Carmines 1980. Measurement in the social sciences. The link between theory
and practice. CUP
24