Is better measurement the solution?
The case of ‘SES’ and ‘age’
Federica Russo
Center Leo Apostel, VrijeUniversiteitBrusse...
Overview
Measurement in social science
Some classic and more recent discussions
A common theme: better measurement is bett...
MEASUREMENT IN SOCIAL SCIENCE
3
A theory of measurement
Suppes 1998
Two problems for measurement:
The problem of representation
Attach a number to an ‘obj...
The focus on procedural aspects
Zeller and Carmines 1980
Follow Blalock:
measurement is the process of linking abstract co...
Measurement and realism
Cartwright and Chang 2008
Practitioner’s problem: whether measurements are correct
Philosopher’s p...
Realism and indicators
Bohrnstedt
In social science there are some clear and tangible measures
E.g. age, birth, number of ...
Establishing a trend
The worry
What do we measure? Is it real?
The solution
It must be real, somehow
The better we measure...
Is it always the case?
Is realism the problem?
And is better measurement the solution?
Measurement itself, especially if carried out using
sophisticated instruments or analysed using complex
methodology, is se...
MEASURING
SOCIO-ECONOMIC STATUS AND AGE
11
At the extremes of measurement,
a common problem
Age
• Very easy to measure
• What does it represent?
• Does it have any
e...
Measuring ‘Ballung’ concepts
Cartwright and Bradburn
Measurement requirements:
Characterisation; Representation; Procedure...
Measuring SES
Theoretical approaches
Weberian, Marxist, Colemanian
Identification of different indicators,
different types...
What do we need SES for?
Consider social epidemiology
SES is highly correlated with health outcomes
Asbestos related death...
What does SES do?
Categorise?
A classificatory variable
What part of the populations are more exposed, have higher
prevale...
Which one to choose?
Measurement – categorisation – explanation
Measurement, alone, does not explain
Measurement, alone, o...
Measuring age
Easy to measure
Accessibility of data, straightforward question, …
Choose to measure
Categorically
Continuou...
Typical uses of ‘age’
Control
Adjust results of statistical analyses (control for age)
Predict
Age structure helps predict...
What age stands for
Biological age
A typical health status, for that age
Social age
Social practices that are typical of t...
SOURCES OF INFORMATION
21
Where do we get
the information from?
Quantitative studies
Large samples, large data sets
Correlations to be validated
The...
Where do we get
the information from?
Qualitative studies
Small samples, small numbers
Detailed description of practices
S...
Establishing a trend
Sample:
The bigger the better
Measurement
The more precise the better
24
Should we alwaysfollowthis trend?
The ‘extra’ information
that statistics does not give us
Description of
Practices
Interactions
Influences
Background
Norms...
The information that
statistics does give us
Categorise the ‘practices, interactions,
backgrounds, …’ into measurable vari...
TO SUM UP
28
Traditional problem of measurement in social science
The trend: justify naïve realism by better measurement
Question the t...
TO CONCLUDE
30
Better measurement is not necessarily panacea
To measure better we need to describe better
Difficulty: not just a social s...
What / why do we measure?
In the area of data collection and presentation at the present
time, likewise, there seems littl...
REFERENCES
33
George W. Bohrnstedt, An Overview of Measurement in the Social Sciences.
http://www7.nationalacademies.org/dbasse/Measurem...
Upcoming SlideShare
Loading in...5
×

Russo measurment rovaniemi

146
-1

Published on

Is better measurement always the solution? The case of 'age' and 'SES'.

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
146
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Quote:Measurement implies three requirements: 1) We have to have a characterization of the quantity or category, that is we have to be able to identify its boundaries and know what belongs to it and what does not (characterization).; 2) we have to have a metrical system that appropriately represents the quantity or category (representation); and 3) we have to have rules for applying the metrical system to produce measurement results (procedures). How to represent Ballung concepts“An overall measure of quality was then constructed by computing a weighted average of the indicators using weights derived from the faculty survey. “
  • Russo measurment rovaniemi

    1. 1. Is better measurement the solution? The case of ‘SES’ and ‘age’ Federica Russo Center Leo Apostel, VrijeUniversiteitBrussel Centre for Reasoning, University of Kent
    2. 2. Overview Measurement in social science Some classic and more recent discussions A common theme: better measurement is better Measuring ‘SES’ and ‘Age’ Challenges in measurement and interpretation Challenges to the theme: better measurement is not always the solution Integrating qualitative and quantitative methods Observe on small scale before you measure Measure on large scale based on observation 2
    3. 3. MEASUREMENT IN SOCIAL SCIENCE 3
    4. 4. A theory of measurement Suppes 1998 Two problems for measurement: The problem of representation Attach a number to an ‘object’ Look at the structure of the theory, and yet … The problem of determining the procedure The choice of the scale also depends on theory 4
    5. 5. The focus on procedural aspects Zeller and Carmines 1980 Follow Blalock: measurement is the process of linking abstract concepts to empirical indicators The possibility to answer research questions depends on robustness of our measurement procedures Measurement procedures above theorising 5
    6. 6. Measurement and realism Cartwright and Chang 2008 Practitioner’s problem: whether measurements are correct Philosopher’s problem: whether we measure what we want to measure Nominalism conventionalism OR operationalism Naïve realism  problem of justification and nomic measurement In social science Suppes’ measurement theory solves representation problem and leaves open procedure problem howto measure a concept within a theory Variability and contingency of concepts to measure Non value-free measurements 6
    7. 7. Realism and indicators Bohrnstedt In social science there are some clear and tangible measures E.g. age, birth, number of children, marital status … For more blurred concepts Observe the covariation between indicators, and infer their reality 7
    8. 8. Establishing a trend The worry What do we measure? Is it real? The solution It must be real, somehow The better we measure the better we represent ‘real’ objects 8
    9. 9. Is it always the case? Is realism the problem? And is better measurement the solution?
    10. 10. Measurement itself, especially if carried out using sophisticated instruments or analysed using complex methodology, is seen to have the attributes of ‘science’, and often taken effectively as a justification for believing the resultsthat are presented as if they have a meaningful relation to whatever social process they are claimed to measure. Harvey Goldstein 2012 10
    11. 11. MEASURING SOCIO-ECONOMIC STATUS AND AGE 11
    12. 12. At the extremes of measurement, a common problem Age • Very easy to measure • What does it represent? • Does it have any explanatory import? SES • Very controversial how we should measure it • What does it represent? • What is its import in explanation of social or social / health outcomes? 12
    13. 13. Measuring ‘Ballung’ concepts Cartwright and Bradburn Measurement requirements: Characterisation; Representation; Procedure Concepts Refer to a single quantity Have unclear boundaries and relations (Ballung) They hinder a development of social science into ‘proper’ science How to represent Ballung concepts “One is to represent them with a table or vector of features laying out the dimensions along which the family resemblances in question lie […] The other is to shed much of the original meaning and zero in on some more precisely definable feature from the congestion that constitutes the concept.” Then, go ahead with chosen procedure 13
    14. 14. Measuring SES Theoretical approaches Weberian, Marxist, Colemanian Identification of different indicators, different types of variables Class stratification Goldthorpe Class Schema Grouping of types of workers 14
    15. 15. What do we need SES for? Consider social epidemiology SES is highly correlated with health outcomes Asbestos related deaths in Barking Cancer related deaths in Eternit workers Cancer incidence in Taranto … 15
    16. 16. What does SES do? Categorise? A classificatory variable What part of the populations are more exposed, have higher prevalence … Explain? Active part in the explanation of diseases Mixed aetiology! What are the active causal pathways from exposure to outcome? Social practices / norms / habits to explain (and to prevent) exposure 16
    17. 17. Which one to choose? Measurement – categorisation – explanation Measurement, alone, does not explain Measurement, alone, only categorises Include SES to explain a phenomenon 17
    18. 18. Measuring age Easy to measure Accessibility of data, straightforward question, … Choose to measure Categorically Continuously Easy data to get – use it! 18
    19. 19. Typical uses of ‘age’ Control Adjust results of statistical analyses (control for age) Predict Age structure helps predict results Categorise grouping and collapsing multiple categories into fewer categories Care with loss of information, residual confounding 19
    20. 20. What age stands for Biological age A typical health status, for that age Social age Social practices that are typical of that age … Any explanatory import? 20
    21. 21. SOURCES OF INFORMATION 21
    22. 22. Where do we get the information from? Quantitative studies Large samples, large data sets Correlations to be validated The bigger the better, the more precise the better 22
    23. 23. Where do we get the information from? Qualitative studies Small samples, small numbers Detailed description of practices Small does not allow generalisation 23
    24. 24. Establishing a trend Sample: The bigger the better Measurement The more precise the better 24
    25. 25. Should we alwaysfollowthis trend?
    26. 26. The ‘extra’ information that statistics does not give us Description of Practices Interactions Influences Background Norms … GO small FIRST! 26
    27. 27. The information that statistics does give us Categorise the ‘practices, interactions, backgrounds, …’ into measurable variables Is it generalisable? An empirical question! Now go BIG! 27
    28. 28. TO SUM UP 28
    29. 29. Traditional problem of measurement in social science The trend: justify naïve realism by better measurement Question the trend through two examples SES and Age One step back Where do we get information Focus on explanation rather than realism We may need to describe before measuring 29
    30. 30. TO CONCLUDE 30
    31. 31. Better measurement is not necessarily panacea To measure better we need to describe better Difficulty: not just a social science trend Oppose the trend in requests from policymakers What is evidence What information we can trust What methods we can trust 31
    32. 32. What / why do we measure? In the area of data collection and presentation at the present time, likewise, there seems little ground for optimism. Even in those societies, such as parts of Australia, where crude league tables used to be eschewed, increasing political and commercial pressures seem to be gaining the upper hand. New technologies such as powerful dynamic computer graphics do have the potential to convey findings and patterns in powerful ways, but whether they are used to inform rather than merely impress, remains an open question. Perhaps the most that one can hope for is that we could reflect more on Galton and his legacy. In particular, a better understanding is needed of the difference between data that ‘confirms’ a theory by providing a good model fit, and data that allows us to explain observed data patternsusing as much potentially falsifiable information as possible. Harvey Goldstein 2012 32
    33. 33. REFERENCES 33
    34. 34. George W. Bohrnstedt, An Overview of Measurement in the Social Sciences. http://www7.nationalacademies.org/dbasse/Measurement_in_Social_Sciences.pdf Burt R. 1991 Measuring age as a structural concept. Social Networks 13 Cartwright N. and Chang H. 2008 Measurement, in The Routledge Companion to Philosophy of Science, pp. 367-375. Cartwright N. and Bradburn N., A theory of measurement.http://www7.nationalacademies.org/dbasse/Common%20Metrics_Me asurement_for_Science_and_Policy.pdf Goldstein H. 2012. Francis Galton, measurement, psychometrics and social progress. Assessment in Education: Principles, Policy &PracticeVol. 19, No. 2 Marks G. The measurment of socioeconomic status and social class in the LSAY project. Technical Paper http://www.acer.edu.au/documents/LSAY_techrep14.pdf Reijneveld S A 1998 Age in epidemiological analysis, J Epidemiol Community Health 2003;57 Suppes P. 1998 Theory of Measurement. E. Craig (Ed.), Routledge Encyclopedia of Philosophy. pp. 243-249. Zeller and Carmines 1980. Measurement in the social sciences. The link between theory and practice. CUP 34
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×