DELIVERING EFFECTIVEQUALITATIVE RESEARCHFOR POLICY MAKINGGIUSEPPE A. VELTRI, INSTITUTE FOR PROSPECTIVETECHNOLOGICAL STUDIE...
QUALITATIVE RESEARCHAND POLICY MAKERS• Although there is an increasing recognition of the usefulness of  qualitative resea...
POLICY MAKINGENVIRONMENT• The policy making environment does a large use of  quantitative study to have ‘evidence-based po...
QUALITATIVE ANDQUANTITATIVE RESEARCHMay 27, 2011
QUALITATIVE RESEARCHMAIN FEATURESMay 27, 2011
LIMITS OF QUALITATIVERESEARCH(from the perspective of a policy maker, perceived as such)• Not generalizability (and in gen...
QUALITATIVE RESEARCHAND POLICY MAKINGMay 27, 2011
RISKS OFQUANTIFICATION• Overconfidence in mathematical models       •      This uncritical acceptance amounts to the assum...
RISKS OFQUANTIFICATION /2•    Putting a situation into numbers enforces the belief that     things are linear and events a...
RISKS OFQUANTIFICATION /3• "The more any quantitative social indicator is used for social  decision-making, the more subje...
WHY MIXED METHODS?• As much as they are powerful, quantitative methodologies are  not immune from shortcomings and they em...
TRIANGULATIONS• Data triangulation  • Data triangulation involves the use of different sources of    data/information• Inv...
TRIANGULATIONS /2• Methodological triangulation  • It involves the use of multiple qualitative and/or quantitative    meth...
COSTS• Qualitative studies are on average cheaper than  quantitative studies.• Policy makers are less concerned about a co...
HOT SPOTS FORQUALITATIVE RESEARCH• Qualitative research has two main functions: it can be part of  the so called ‘Evidence...
HOT SPOTS FOR QUALITATIVERESEARCH /2• Ex-ante assessment   • Ex-ante assessment involves undertaking an evaluation of     ...
THANK YOU!Giuseppe.Veltri@ec.europa.euMay 27, 2011
Presented at the 2nd European conference on Qualitative Research for Policy Making, 26 -27 May 2011, Belfast              ...
Upcoming SlideShare
Loading in …5
×

Delivering effective qualitative research for policy making

1,511 views

Published on

Presented at the 2nd European Conference on Qualitative Research for Policy Making, 26-27 May 2011, Belfast, UK

Organised by Merlien Institute

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,511
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
34
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Delivering effective qualitative research for policy making

  1. 1. DELIVERING EFFECTIVEQUALITATIVE RESEARCHFOR POLICY MAKINGGIUSEPPE A. VELTRI, INSTITUTE FOR PROSPECTIVETECHNOLOGICAL STUDIES (IPTS)QUALITATIVE RESEARCH FOR POLICY MAKING 2011, BELFAST 26-27 OF MAY.
  2. 2. QUALITATIVE RESEARCHAND POLICY MAKERS• Although there is an increasing recognition of the usefulness of qualitative research in policy making environments, the general understanding does not see it as a stand alone research endeavor.• Based on my experience, qualitative research is rather well received in a context of mixed methodologies, but it is not perceived as funds-worthy if not connected to a quantitative study.• Oppositional views of qualitative vs. quantitative research are fruitless.May 27, 2011
  3. 3. POLICY MAKINGENVIRONMENT• The policy making environment does a large use of quantitative study to have ‘evidence-based policies’.• There has been an increase of the use of qualitative study because of the failure of quantitative indicators or to support the adoption of particularly controversial quantitative measurements.• Policy makers are responsive to the argument of having a more balanced share of quantitative and qualitative studies.May 27, 2011
  4. 4. QUALITATIVE ANDQUANTITATIVE RESEARCHMay 27, 2011
  5. 5. QUALITATIVE RESEARCHMAIN FEATURESMay 27, 2011
  6. 6. LIMITS OF QUALITATIVERESEARCH(from the perspective of a policy maker, perceived as such)• Not generalizability (and in general misunderstanding of qualitative research epistemological position)• Lack of methodological rigor• Lack of transparency• Time consuming• Complexity expansion rather than reduction• Low rhetorical and argumentative value in conflict of claimsMay 27, 2011
  7. 7. QUALITATIVE RESEARCHAND POLICY MAKINGMay 27, 2011
  8. 8. RISKS OFQUANTIFICATION• Overconfidence in mathematical models • This uncritical acceptance amounts to the assumption that reality is identical to our rational reconstruction of reality• Because they are logically consistent, mathematical models screen out ambiguity • Ambiguity is real, but quantitative models have little to no room for it. Ambiguity arises whenever there are two (or more) courses of action that are equally important yet conflict with one anotherMay 27, 2011
  9. 9. RISKS OFQUANTIFICATION /2• Putting a situation into numbers enforces the belief that things are linear and events are necessarily comparable.. • ..since for any two numbers one is larger and one is smaller. Suppose that you modeled teaching on a scale that goes from 1 to 10. You would conclude that a teacher with a score of 8.3 was better than one with 6.8, but this would inevitably ignore all kinds of qualitative factors• Any system that involves human behavior, like economics or finance, is inherently self-referential. • We are all participants in, not just observers of, these systems. Self-referential systems are notoriously difficult to control and predict because they are often chaotic, not deterministic. Most mathematical models do not take this element of self-reference seriously and anyhow chaotic systems, by definition, cannot be predictedMay 27, 2011
  10. 10. RISKS OFQUANTIFICATION /3• "The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”• Campbells Law was published in 1976 by Donald T. Campbell, an experimental social science researcher and the author of many works on research methodology.• Very relevant for policy.May 27, 2011
  11. 11. WHY MIXED METHODS?• As much as they are powerful, quantitative methodologies are not immune from shortcomings and they embed the risks of quantification that are not always fully understood by users of quantitative data.• Those risks of quantification are very relevant for policy makers• Triangulation is the methodological principle to adopt and argue in support of.May 27, 2011
  12. 12. TRIANGULATIONS• Data triangulation • Data triangulation involves the use of different sources of data/information• Investigator triangulation • It involves using several different investigators/evaluators in an evaluation project.• Theory triangulation • It involves the use of multiple professional perspectives to interpret a single set of data/informationMay 27, 2011
  13. 13. TRIANGULATIONS /2• Methodological triangulation • It involves the use of multiple qualitative and/or quantitative methods to study the issue. If the conclusions from each methods are the same, then validity is established.• Environmental triangulation • This type of triangulation involves the use of different locations, settings and other key factors related to the environment in which the study took place such as time of the day, day of the week or season of the year.May 27, 2011
  14. 14. COSTS• Qualitative studies are on average cheaper than quantitative studies.• Policy makers are less concerned about a cost and more about how to justify that cost.• There is nothing more expensive than ill-designed policyMay 27, 2011
  15. 15. HOT SPOTS FORQUALITATIVE RESEARCH• Qualitative research has two main functions: it can be part of the so called ‘Evidence based policies’; It can be a valuable tool in Evaluation Research• The focus in qualitative research on exploring meanings and behavior in depth, identifying diverse perspectives, capturing processes and social contexts, and using flexible methods, means that qualitative research is seen as being capable of making a distinctive contribution to policy evaluation.May 27, 2011
  16. 16. HOT SPOTS FOR QUALITATIVERESEARCH /2• Ex-ante assessment • Ex-ante assessment involves undertaking an evaluation of the conditions for the launch of a program or institution.• Cross cultural studies • In international organizations where the diversity of social, political and institutional contexts is high (e.g. EU)May 27, 2011
  17. 17. THANK YOU!Giuseppe.Veltri@ec.europa.euMay 27, 2011
  18. 18. Presented at the 2nd European conference on Qualitative Research for Policy Making, 26 -27 May 2011, Belfast For more information Please visit: http://www.merlien.orgMay 27, 2011

×