Correlational data, causal hypotheses and validity


Published on

Published in: Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Correlational data, causal hypotheses and validity

  1. 1. Correlational data,causal hypotheses, and validity<br />Federica Russo<br />Philosophy, Kent<br />
  2. 2. Overview<br />A shared problem across the sciences<br />Making causal sense of correlational data<br />The structural strategy<br />Looking for, modelling and testing structures<br />Establishing the validity of a causal model<br />The interventionist strategy<br />Invariance under intervention<br />Interventions and experiments<br />‘Weak invariance’ and observational data<br />2<br />
  3. 3. Making sense of correlational data<br />A shared problem<br />Correlational data coming from observations and/or from experiments<br />Are those correlations causal or not?<br />An old philosophical question, indeed<br />What is the extra-content making correlations causal?<br />An under-discussed issue<br />How to make sense of correlations coming from observations?<br />A reformulated question<br />When is a statistical model built to make causal sense<br />of correlational data valid?<br />3<br />
  4. 4. Preliminary remarks<br />Focus on methodology/epistemology<br />rather than metaphysics of causality<br />Narrow down the scope to quantitative social science<br />A tradition of scientific inquiry back to Quetelet and Durkheim,<br />Blalock and Duncan, …<br />The hard task is with observational data<br />Why reformulating the ‘old’ question?<br />Is it a condition that grants causal results?<br />4<br />
  5. 5. Structural strategy<br />5<br />
  6. 6. The structural strategy<br />A general methodological framework <br />Embraces various statistical methods<br />Provides the general principles of the<br />test ‘set-up’ for causal hypotheses<br />6<br />
  7. 7. Structural modelling vsstructural equation modelling<br />Not co-extensive terms!<br />Pearl:<br />Ambiguous use of the two; not explicit meaning of structural<br />Woodward<br />Says it deals with SM, but does in fact with SEM<br />Hoover<br />Does not make explicit the meaning of structural<br />Cartwright<br />Restricted discussion to SEM in economics and role of ideal experiments<br />7<br />
  8. 8. What makes a structural model structural?<br />A question left by and large unanswered by current accounts<br />The question I am going to tackle next<br />In a nutshell:<br />Structural modelling is the modelling of mechanisms <br />8<br />
  9. 9. Structural models model mechanisms<br />To model mechanisms means to formulate<br />suitable causal hypotheses<br />to put forward for empirical testing<br />To decide about the results of tests<br />is to decide whether the model is valid<br />9<br />
  10. 10. Looking for structures<br />Beyond descriptive knowledge<br />Unveil the mechanism that supposedly explains correlations<br />Not just assuming a ‘data generating process’<br />10<br />
  11. 11. What mechanism?<br />‘Modelling mechanisms’ does not depend<br />on a metaphysical account of mechanism<br />No ontological commitment to the (degree of)<br />physical existence of (social) mechanisms<br />Mechanisms are epistemic: they carry explanatory power<br />‘Mechanism schemata’ give the description of the behaviour<br />Do track something real: making sense of what actually happens<br />In line with MDC or Bechtel & Abrahmsen<br />At variance with Woodwardian account<br />11<br />
  12. 12. Building structures<br />Formulate causal hypotheses<br />Build the statistical model<br />Test the model<br />Conclude to the validity/invalidity of the model<br />The role of background knowledge<br />Yes … that’s hypothetico-deductivism…<br />But …<br />12<br />
  13. 13. Causal hypotheses and mechanisms: how they are linked<br />Causal hypotheses are not of the form X causes Y<br />There is a whole set of hypotheses and assumptions that<br />altogether can be interpreted as hypothesising the mechanism<br />This whole set is formally translated into the so-called<br />recursive decomposition<br />13<br />
  14. 14. What are the causes of self-rated health in the Baltic countries<br />in the ‘90s?<br />XY<br />Take the joint probability distribution + Make assumptions<br />P(Education, Locus of Control, Physical Health, …, Self-Rated Health)<br />P(X1, X1, X3, …Y)<br />perform a recursive decomposition of the type<br />P(Y)= P(X1) P(X3) P(X2|X3) … P(Y|X2, X3)<br />Read as:<br /> Self-Rated Health depends on Education; on Locus of Control through Psychological distress; on Alcohol Consumption which also depends on Physical Health; …<br />14<br />
  15. 15. 15<br />
  16. 16. Testing structures<br />Goodness of fit, significance of parameters …<br />Invariance under intervention<br />… better, under changes of the environment<br />That is:<br />whether the relation actually remains stable across<br />different portions of the data set, <br />different data sets with observations from different populations<br />different time periods<br />16<br />
  17. 17. Validity<br />Decide whether the story about the mechanism<br />meant to make sense of correlational data<br />provides a plausible explanation<br />about what is really going on in the world<br />Cook and Campbell on internal validity<br />Representativeness of the sample and<br />Replicability of the study<br />To be complemented with<br />Background knowledge<br />Explanatory power<br />17<br />
  18. 18. Take home message<br />To decide whether correlations are causal or not<br />we have to decide about<br />the validity of the whole model.<br />That is,<br />whether the mechanism provides<br />a good enough explanation of the correlations.<br />18<br />
  19. 19. The interventionist strategy<br />19<br />
  20. 20. Invariance under intervention<br /> “A generalization G is invariant if G would continue to hold under some intervention that changes the value of X in such a way that, according to G, the value of Y would change — ’continue to hold’ in the sense that G correctly describes how the value of Y would change under this intervention.” (Woodward and Hitchcock 2003)<br />Invoked because it provides<br />a definition of causality,<br />or it bestows empirical generalisations explanatory power, <br />or some combination of the two<br />It is counterfactually defined<br />Under intervention, the generalisationwould continue to hold<br />Relevant for explanation<br />Invariant generalisations allow answering withbd-questions<br />20<br />What if things had been different<br />
  21. 21. How does the interventionist modeller make causal sense of correlational data?<br />Invariance under intervention!<br />
  22. 22. The role of experiments<br />Does interventionism presuppose experimentalism,<br />or does it not?<br />If it does:<br />we have no methodological story for observational contexts<br />If it does not:<br />we have to modify the requirement of invariance<br />22<br />
  23. 23. […] The kind of counterfactuals that are relevant to understanding causation are connected to experiments — either actual or hypothetical. […] Counterfactuals are understood as claims about what would happen if a certain sort of experiment were to be performed […]<br />(Woodward 2002, emphasis in the original)<br />23<br />
  24. 24. Interventionismis essentially conceptual<br />The project<br />Investigates the nature of causation<br />Aims to give identity conditions for causal relations<br />Structural modellers can live with conceptual interventionism<br />Experiments are not necessarily key<br /><ul><li>However, a methodological story about testing is needed too</li></ul>Arguably test conditions in experimental and<br />non-experimental contexts significantly differ<br />Causal assessment requires the validity of the whole model,<br />not simply one condition to be satisfied<br />24<br />
  25. 25. Interventionismis not entirely conceptual<br />The project<br />Tells how a phenomenon would change after certain interventions<br />Ties a knot between how- and why-questions<br />Invariance under intervention<br />Distinguish accidental and causal generalisations<br />Gives generalisations explanatory power (through withbd-questions)<br />Experiments are here key<br /><ul><li>But then the projects is unsuitable to observational contexts</li></ul>25<br />
  26. 26. Some remarks<br />There are many tests to perform, not just invariance<br />And we need to assess the validity of the whole model<br />Ideal manipulations won’t do<br />Indeed it depends on the meaning of ideal …<br />26<br />
  27. 27. How much ideal is ‘ideal’?<br />You don’t have to actually intervene:<br />idealmanipulationswill do<br />Not quite:<br />1. Some ideal interventions don’t make physical sense<br />2. Some ideal interventions cannot be tested<br />Were we to intervene: a conceptualanalysis<br />27<br />
  28. 28. Observational data andweak invariance<br />‘Possible-cause generalisations’:<br />Those established by structural models<br />in the special sciences<br />[my case!]<br />They are weakly invariant,<br />i.e. stable across subpopulations or partitions of the data set<br />28<br />
  29. 29. For example, the authors note that some association appears between smoking and lung cancer in every well-designed study on sufficiently large and representative populations with which they are familiar. There is evidence of a higher frequency of lung cancer among smokers than among nonsmokers, when potentially confounding variables are controlled for, among both men and women, among people of different genetic backgrounds, across different diets, different environments, and different socioeconomic conditions […]. The precise level and quantitative details of the association do vary, for example, the incidence of lung cancer among smokers is higher in lower socioeconomic groups, but the fact that there is some association or other is stable or robust across a wide variety or different groups and background circumstances. […] Thus, although Cornfield et al. do not exhibit a precise deterministic or probabilistic generalization that is invariant across different circumstances [meaning: across interventions] the cumulative impact of their evidence is to show that the relationship between smoking and lung cancer is relatively invariant in the weak sense described above.<br />(Woodward 2003, p.312, emphasis and brackets added) <br />29<br />
  30. 30. Therefore …<br />… charitably interpreted:<br />interventionists don’t disagree with me that much<br />Yet …<br />Misplacing ‘interventions’ overshadows the ‘validity of the whole model’<br />‘Weak invariance’ still is a condition to test<br /><ul><li>‘Strong’ and ‘weak’ interventionism can be subsumed</li></ul> under the structural modelling <br />30<br />
  31. 31. To sum up<br />Structural strategy<br />Looking for structures,<br />i.e. mechanisms<br />Building a statistical model<br />Represent the mechanism<br />Perform statistical tests and<br />confront with background knowledge<br /><ul><li>To make causal sense of correlations:</li></ul> Evaluate the validity of the model<br />Interventionist strategy<br />Invariance under intervention:<br />As a conceptual thesis:<br />lacks methodological counterpart<br />As methodological thesis:<br />doesn’t make sense in observational contexts<br />Weak invariance:<br />tested in observational contexts<br /><ul><li>To make causal sense of correlations:</li></ul> Test a condition<br />31<br />
  32. 32. To conclude<br />Consider an overarching view of causal modelling<br />Switch focus<br />from establishing a causal claim by testing acondition<br />to evaluatingthevalidity of the model<br />Do justice to those scientific domains where causal relations are established in absence of interventions strictusensu<br />Discuss how metaphysical and methodological accounts<br />can possibly live together<br />32<br />