Quote:Measurement implies three requirements: 1) We have to have a characterization of the quantity or category, that is we have to be able to identify its boundaries and know what belongs to it and what does not (characterization).; 2) we have to have a metrical system that appropriately represents the quantity or category (representation); and 3) we have to have rules for applying the metrical system to produce measurement results (procedures). How to represent Ballung concepts“An overall measure of quality was then constructed by computing a weighted average of the indicators using weights derived from the faculty survey. “
Is better measurement the solution?
The case of ‘SES’ and ‘age’
Center Leo Apostel, VrijeUniversiteitBrussel
Centre for Reasoning, University of Kent
Measurement in social science
Some classic and more recent discussions
A common theme: better measurement is better
Measuring ‘SES’ and ‘Age’
Challenges in measurement and interpretation
Challenges to the theme: better measurement is not always the
Integrating qualitative and quantitative methods
Observe on small scale before you measure
Measure on large scale based on observation
A theory of measurement
Two problems for measurement:
The problem of representation
Attach a number to an ‘object’
Look at the structure of the theory, and yet …
The problem of determining the procedure
The choice of the scale also depends on theory
The focus on procedural aspects
Zeller and Carmines 1980
measurement is the process of linking abstract concepts to
The possibility to answer research questions depends on
robustness of our measurement procedures
Measurement procedures above theorising
Measurement and realism
Cartwright and Chang 2008
Practitioner’s problem: whether measurements are correct
Philosopher’s problem: whether we measure what we want to measure
Nominalism conventionalism OR operationalism
Naïve realism problem of justification and nomic measurement
In social science
Suppes’ measurement theory solves representation problem and leaves
open procedure problem
howto measure a concept within a theory
Variability and contingency of concepts to measure
Non value-free measurements
Realism and indicators
In social science there are some clear and tangible measures
E.g. age, birth, number of children, marital status …
For more blurred concepts
Observe the covariation between indicators, and infer their reality
Establishing a trend
What do we measure? Is it real?
It must be real, somehow
The better we measure the better we represent ‘real’ objects
Is it always the case?
Is realism the problem?
And is better measurement the solution?
Measurement itself, especially if carried out using
sophisticated instruments or analysed using complex
methodology, is seen to have the attributes of
‘science’, and often taken effectively as a
justification for believing the resultsthat
are presented as if they have a meaningful
relation to whatever social process they
are claimed to measure.
Harvey Goldstein 2012
At the extremes of measurement,
a common problem
• Very easy to measure
• What does it represent?
• Does it have any
• Very controversial how we
should measure it
• What does it represent?
• What is its import in
explanation of social or
social / health outcomes?
Measuring ‘Ballung’ concepts
Cartwright and Bradburn
Characterisation; Representation; Procedure
Refer to a single quantity
Have unclear boundaries and relations (Ballung)
They hinder a development of social science into ‘proper’ science
How to represent Ballung concepts
“One is to represent them with a table or vector of features laying out the
dimensions along which the family resemblances in question lie […] The other
is to shed much of the original meaning and zero in on some more precisely
definable feature from the congestion that constitutes the concept.”
Then, go ahead with chosen procedure
Weberian, Marxist, Colemanian
Identification of different indicators,
different types of variables
Goldthorpe Class Schema
Grouping of types of workers
What do we need SES for?
Consider social epidemiology
SES is highly correlated with health outcomes
Asbestos related deaths in Barking
Cancer related deaths in Eternit workers
Cancer incidence in Taranto
What does SES do?
A classificatory variable
What part of the populations are more exposed, have higher
Active part in the explanation of diseases
What are the active causal pathways from exposure to
Social practices / norms / habits to explain (and to prevent) exposure
Which one to choose?
Measurement – categorisation – explanation
Measurement, alone, does not explain
Measurement, alone, only categorises
Include SES to explain a phenomenon
Easy to measure
Accessibility of data, straightforward question, …
Choose to measure
Easy data to get – use it!
Typical uses of ‘age’
Adjust results of statistical analyses (control for age)
Age structure helps predict results
grouping and collapsing multiple categories
into fewer categories
Care with loss of information, residual confounding
What age stands for
A typical health status, for that age
Social practices that are typical of that age
Any explanatory import?
Traditional problem of measurement in social science
The trend: justify naïve realism by better measurement
Question the trend through two examples
SES and Age
One step back
Where do we get information
Focus on explanation rather than realism
We may need to describe before measuring
Better measurement is not necessarily panacea
To measure better we need to describe better
Difficulty: not just a social science trend
Oppose the trend in requests from policymakers
What is evidence
What information we can trust
What methods we can trust
What / why do we measure?
In the area of data collection and presentation at the present
time, likewise, there seems little ground for optimism. Even in those
societies, such as parts of Australia, where crude league tables used
to be eschewed, increasing political and commercial pressures
seem to be gaining the upper hand. New technologies such as
powerful dynamic computer graphics do have the potential to
convey findings and patterns in powerful ways, but whether
they are used to inform rather than merely
impress, remains an open question.
Perhaps the most that one can hope for is that we could reflect
more on Galton and his legacy. In particular, a better understanding
is needed of the difference between data that ‘confirms’ a
theory by providing a good model fit, and data that
allows us to explain observed data patternsusing
as much potentially falsifiable information as possible.
Harvey Goldstein 2012
George W. Bohrnstedt, An Overview of Measurement in the Social Sciences.
Burt R. 1991 Measuring age as a structural concept. Social Networks 13
Cartwright N. and Chang H. 2008 Measurement, in The Routledge Companion to
Philosophy of Science, pp. 367-375.
Cartwright N. and Bradburn N., A theory of
Goldstein H. 2012. Francis Galton, measurement, psychometrics and social progress.
Assessment in Education: Principles, Policy &PracticeVol. 19, No. 2
Marks G. The measurment of socioeconomic status and social class in the LSAY project.
Technical Paper http://www.acer.edu.au/documents/LSAY_techrep14.pdf
Reijneveld S A 1998 Age in epidemiological analysis, J Epidemiol Community Health
Suppes P. 1998 Theory of Measurement. E. Craig (Ed.), Routledge Encyclopedia of
Philosophy. pp. 243-249.
Zeller and Carmines 1980. Measurement in the social sciences. The link between theory
and practice. CUP 34