Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Internal and external evaluation of scientific research


Published on

In this presentation the impact of science is analyzed. The central concept is that of solving problems. The tensions of science debate are pointed out: academic freedom vs. demands of society.

Published in: Education
  • Be the first to comment

Internal and external evaluation of scientific research

  1. 1. Internal and external evaluation of scientific research Presentation at Hanken, Helsinki 22.4.2016 PhD Antti Hautamäki Research professor (emeritus), University of Jyväskylä Adjunct professor, University of Helsinki Consulting Sustainable Innovation
  2. 2. Basic research and its impact • Science is systematic research which produces new knowledge (results) • To be effective science must follow its own interests and methods • The way from research results to practical applications is long and indirect • A major problem is how to stimulate impact effectively and maintain a consistent high quality of the research
  3. 3. Tensions in science debate • Truth vs. practice • Pure science vs. usefulness • Basic science vs. applied science • Quality vs. relevance • Autonomy vs. external steering • Collegiality vs. managerialism • Budgetary funding vs. competed funding • Science logic vs. market logic
  4. 4. Scientific method (ideal, Ch. Peirce) • The scientific method • Is fallible • Purposely tests itself • Criticizes itself • Corrects itself • Improves itself • Other “methods” are based on tenacity, authority, or a priori assumptions
  5. 5. Basic elements of scientific method • PROBLEMS: identification and classification of problems • DATA: systematic observation, measurement, and experiment • HYPOTHESES: formulation, testing, and modification of hypotheses • EVALUATION: iteration, replication, peer review, open data
  6. 6. Qualitative research • Differs from standard hypothetic-deductive method, where hypotheses are tested by their deductive consequences and predictions • Target is more to understand phenomena (meanings) than to explain them • A variety of methods: action theory, grounded theory, ethnography, hermeneutics, narrative analysis, observation, phenomenology etc. • Challenges: generalization, objectivity, sampling
  7. 7. Progress of science (L. Laudan: Progress and Its Problems, 1977) • Progressiveness of a theory or an approach is related to its problem solving effectiveness • Empirical problems are often wh-questions related to the world around us: what, why, when, how,.. • Conceptual problems are a) internal related to theoretical difficulties (concepts, methods) or b) external rising from conflicts between different theories • Anomalous problems for a theory are empirical problems which a competitor has solved
  8. 8. Measure of progress • The aim of science is to maximize the scope of solved empirical problems, while minimizing the scope of anomalous and conceptual problems • Theories are produced in research traditions (schools), which are sets of general assumptions about the entities and processes in a domain of study • The central cognitive test for any theory is whether it provides satisfactory solutions to important problems
  9. 9. Instrumentalism in science • According to instrumentalism science is first of all providing workable solutions to practical problems (vs. realism = uncovering the reality) • Solutions might be formulated as “technical norms”: • If you perform action A, then you will get result B • Technical norms are “value-neutral” • But values are involved in selecting which problems are worth to solve!
  10. 10. Humboldtian university • Humboldt University in Berlin (est. 1810) was the model of modern large-scale universities • W. von Humboldt's model was based on two ideas of the Enlightenment: the individual and the world citizen • Principles: • Academic freedom and autonomy of universities • The pursuit of knowledge as a base for culture, civilization and education (German “Bildung”) • The unity of teaching and research
  11. 11. Neo-humboldtian university (A.Hautamäki & P. Ståhle, Ristiriitainen tiedepolitiikkamme, 2012) • Bias of Humboldtian university: the pursuit of truth without paying attention to societal and economic issues and challenges • In Neo-humboldtian university wicked problems are taken to be an organizing principle for research, education and problem solving • Wicked problems are the grand challenges of contemporary world such as climate change, health and food issues, polarization, and security • They are complex and difficult to solve
  12. 12. Impact analysis Human and financial resources Activities Processes Goods and services produced Initial impact Longer-term impact
  13. 13. Impact of science • Scientific impact • Cumulation and growth of knowledge • Citation impact quantifies the citation usage of scholarly works; Impact factor measures the citation impact of an academic journal • Societal impact • Our society is increasingly focused on requiring justification for the investments made in scientific research: need for societal impact of science
  14. 14. Scientific impact • Academic publications in journals with high impact factors • Quality as measured by citation • Peer review is central in journals and funding decisions • Bias: support “normal science”, not open to radical openings, innovative approaches or interdisciplinary research • Societal impact is neglected
  15. 15. Societal impact • Some dimensions of impact: • Development of skills and knowledge for employment • Business–university collaboration in research and innovation • Graduate recruitment: the interface between students, universities and employers • Universities in their local communities: enabling economic growth • How to measure impact, what are good indicators? • Case Finland (next two slides)
  16. 16. Universities core funding 2013-2016 in Finland (and changes for 2017- 2020) + 2% -2% -1% 1 % = 16 m€ +1%
  17. 17. Strategic Research Council Academy of Finland • New funding for strategic research (55 m€ /y) • “The SRC funds high-quality research that has great societal impact. The research should seek to find concrete solutions to grand challenges that require multidisciplinary approaches. An important element of such research is active collaboration between those who produce new knowledge and those who use it.” • “The Government determines the research needs and decides the final themes, which the SRC then formulates into research programmes and funding calls.” • SRC programmes run for 3–6 years
  18. 18. Readings • Berman R.P. (2012). Creating the Market University, How Academic Science Became an Economic Engine. Princeton & Oxford: Yale University Press. • Geiger R.L. (2004) Knowledge and Money, Research Universities and the Paradox of Marketplace. Stanford: Stanford University Press. • Hautamäki A. (2010). Sustainable innovation. A New Age of Innovation and Finland’s Innovation Policy. Sitra Reports 87. Helsinki. • Hautamäki Antti ja Ståhle Pirjo: Ristiriitainen tiedepolitiikkamme, Suuntana innovaatiot vai sivistys? Helsinki: Gaudeamus 2012 • Hautamäki Antti, Ståhle Pirjo, Oksanen Kaisa ja Tukiainen Taina (2016). Vaikuttavaa tutkimusta – Kokeiluehdotuksia tutkimuksen vaikuttavuuden ja kaupallistamisen edistämiseksi. Työ- ja elinkeinoministeriön julkaisuja 2/2016. • Oksanen Kaisa ja Hautamäki Antti (2014): “Uushumboldtilainen yliopisto – suomalainen ratkaisu yliopistojen kehittämiseen.” Tiedepolitiikka 1/2014, 46-52.