Indicator-Based Monitoring of anInterdisciplinary Field of ScienceThe Example of Educational ResearchDr. Andreas Oskar Kem...
Contents1. Project Overview2. Project Context and Objective „Empirical Turn“ in Educational Research New Governance Mode...
1. Project Overview Title:Development and Changing Dynamics of a HeterogeneousSocial Science Field. Using the Example of ...
2. Project Context and Objective„Empirical Turn“ in Educational Research DFG Research Units / Priority Programmes for Emp...
Project ContextNew Governance Model in Research FundingBasic and third party funding in higher education institutions in G...
Project Objective Analysis of the structure and development of and within educationalresearch Analysis of communication ...
Data SourcesProject Data Sources:- SOFIS (GESIS)- BMBF Framework Programme Coverage: 9139 projects Period: 1995-2009 (e...
3. Types of Research IndicatorsHinze/Glänzel (2012)Cologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitorin...
4.1 Analysis of Project DataCologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educational Research ...
Cologne, May 28 – 31 May │
Research Activity: Core Disciplines Over Time (2/2)Decline in psychological projectsCologne, May 28 – 31 May │ Kempf, Sond...
Cologne, May 28 – 31 May │
Funding Bodies Over Time (2/2)ResultsThere, indeed, is an increase in the share of funding byfoundations and the EU. Diffe...
Research Cooperation Over Time (1/2)ResultsNetworking among research institutions increasesCologne, May 28 – 31 May │ Kemp...
Research Cooperation Over Time (2/2)ResultsThe share of cooperations is higher in third partyfunded than in self-financed ...
Empirical Research MethodsResultsThere is an increase in the use of empiricalmethods between the last two time blocks.Colo...
4.2 Analysis of Publication DataCologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educational Resea...
4.2 Analysis of Publication Data – First ResultsHinze/Glänzel (2012)Cologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST...
Publication ActivityCologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educational Research PageResu...
Discipline-Specific Publication Types (1/3)Cologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educat...
Publication Types (2/3)Peer-Review Status of Journal ArticlesCologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │...
Discipline-Specific Publication Types (3/3)Cologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educat...
Multi-AuthorshipCologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educational Research PageResultsO...
International Character of PublicationsCologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educationa...
5. SummaryResearch projects in educational research are characterized by:- a growing importance in the Social Sciences,- a...
6. Outlook- Further analysis on the interrelationbetween research projects and publications,- Thematic analysis of project...
AcknowledgementsAlexander BotteUte SondergeldJürgen WiesenhütterDr. Gabriel SchuiKatja SingletonVeronika Kuhberg-LassonDr....
Thank youfor your attention!Contact:Dr. Andreas Oskar Kempf, M.A.(LIS)Leibniz-Institute for the Social Sciences (GESIS)and...
BibliographyDeutsche Forschungsgemeinschaft (2012) Förderatlas 2012. Kennzahlen zuröffentlich finanzierten Forschung in De...
Upcoming SlideShare
Loading in...5
×

Kempf, Sondergeld: Indicator-Based Monitoring of an Interdisciplinary Field of Science

181

Published on

Dr. Andreas Oskar Kempf, Ute Sondergeld: Indicator-Based Monitoring of an Interdisciplinary Field of Science. The Example of Educational Research - Presentation at IASSIST 2013

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
181
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Kempf, Sondergeld: Indicator-Based Monitoring of an Interdisciplinary Field of Science

  1. 1. Indicator-Based Monitoring of anInterdisciplinary Field of ScienceThe Example of Educational ResearchDr. Andreas Oskar Kempf, M.A.(LIS)Leibniz-Institute for the Social Sciences (GESIS)Ute Sondergeld, M.A., Wiss. Dok.German Institute for International Educational Research (DIPF)Cologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educational Research
  2. 2. Contents1. Project Overview2. Project Context and Objective „Empirical Turn“ in Educational Research New Governance Model in Research Funding Data Sources3. Research Indicators4. Findings4.1 Analysis of Project Data4.2 Analysis of Publication Data5. Summary6. OutlookCologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educational Research Page
  3. 3. 1. Project Overview Title:Development and Changing Dynamics of a HeterogeneousSocial Science Field. Using the Example of Educational Research Duration:05/2011 - 04/2014 Funding:Leibniz Association (SAW funding measure) Project Lead:German Institute for International Educational Research (DIPF) Project cooperation between:Cologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educational Research Page 3
  4. 4. 2. Project Context and Objective„Empirical Turn“ in Educational Research DFG Research Units / Priority Programmes for EmpiricalEducation Research (2000/2002) Introduction of National Educational Standards (2003) Agreement on a National Report on Education (2004) Framework Programme for the Promotion of EmpiricalEducational Research (BMBF) (2007)Cologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educational Research Page
  5. 5. Project ContextNew Governance Model in Research FundingBasic and third party funding in higher education institutions in Germany over timeCologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educational Research Page 5Income(inbn€)Rateofthirdpartyfunding(%)
  6. 6. Project Objective Analysis of the structure and development of and within educationalresearch Analysis of communication processes within educational researchCologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educational Research Page 6EducationalResearchResearchprojectsIndicatorsPublicationoutputCitations
  7. 7. Data SourcesProject Data Sources:- SOFIS (GESIS)- BMBF Framework Programme Coverage: 9139 projects Period: 1995-2009 (end of project)Publications Random sample of 270 projects ofthe core disciplines: educationalsciences, psychology, sociology;periods: 1995-97, 2006-08 Data collection of research output Coverage: about 2.000 publicationsAnalysis of Reception Web of Science, Scopus, GoogleScholarCologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educational Research Page 7
  8. 8. 3. Types of Research IndicatorsHinze/Glänzel (2012)Cologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educational Research PageThird party fundingHinze/Glänzel (2012)based upon Hornbostel1999: p. 59.
  9. 9. 4.1 Analysis of Project DataCologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educational Research Page
  10. 10. Cologne, May 28 – 31 May │
  11. 11. Research Activity: Core Disciplines Over Time (2/2)Decline in psychological projectsCologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educational Research Page61,5%66.4%12.6% 9.7%8.2% 10.0%0%10%20%30%40%50%60%70%80%1995-1997 1998-2000 2001-2003 2004-2006 2007-2009SharePeriodEducational sciencesPsychologySociologyn = 9139
  12. 12. Cologne, May 28 – 31 May │
  13. 13. Funding Bodies Over Time (2/2)ResultsThere, indeed, is an increase in the share of funding byfoundations and the EU. Differences in funding sourcesdiminish.Cologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educational Research Page 130%5%10%15%20%25%1995-1997 1998-2000 2001-2003 2004-2006 2007-2009SharePeriodFed. statesFed. gov.DFGFoundationsEUThird Party Funding (n = 5915)HypothesisIncrease in the share of funding by foundations and the EU.
  14. 14. Research Cooperation Over Time (1/2)ResultsNetworking among research institutions increasesCologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educational Research Page 147.20%9.40%13.80%17.80%18.60%0%5%10%15%20%1995-1997 1998-2000 2001-2003 2004-2006 2007-2009SharePeriodPercentage of Cooperation Projects over Time (n = 7260)HypothesisA growth in national and international cooperations
  15. 15. Research Cooperation Over Time (2/2)ResultsThe share of cooperations is higher in third partyfunded than in self-financed projects.Cologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educational Research Page9.90%17.40%0.00%2.00%4.00%6.00%8.00%10.00%12.00%14.00%16.00%18.00%20.00%Self-Financed Projects Third Party Funded ProjectsShareFunding Type of ProjectsFunding Types in Cooperation Projects (n = 7260)HypothesisA higher proportion of cooperation projects in third partyfunded than in in-house projects
  16. 16. Empirical Research MethodsResultsThere is an increase in the use of empiricalmethods between the last two time blocks.Cologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educational Research Page 1674.5%71.5% 69.4%64.3%67.1%25.5% 28.5% 30.6% 35.7%32.9%0%10%20%30%40%50%60%70%80%1995-1997 1998-2000 2001-2003 2004-2006 2007-2009SharePeriodempiricalnon-empiricalUse of Research Methods over Time (n = 9139)HypothesisIncrease in the use of empirical research methods
  17. 17. 4.2 Analysis of Publication DataCologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educational Research Page
  18. 18. 4.2 Analysis of Publication Data – First ResultsHinze/Glänzel (2012)Cologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educational Research PageThird party fundingHinze/Glänzel (2012)based upon Hornbostel1999: p. 59.
  19. 19. Publication ActivityCologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educational Research PageResultsThe number of publications in total declines.Likewise, discipline-specific differences seem to decline.HypothesisAn increase in the number of publications.41227832532726831510059200200400600800100012001995-1997 2006-2008CountPeriodSociologyEducational SciencesPsychologyIn TotalNumber of Publications (n = 1925)
  20. 20. Discipline-Specific Publication Types (1/3)Cologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educational Research PageHypothesisA greater number of journal articles in psychology than in sociology orin educational sciences.ResultsIn psychology there is a preference for publishing in journalarticles. Over time, a significant increase in the number of journalarticles is measurable in educational sciences. In sociology thenumber of journal articles decreases significantly.0.330.220.230.350.34 0.3900.050.10.150.20.250.30.350.40.451995-1997 2006-2008EMMPeriodSociologyEducational SciencesPsychologyEstimated Marginal Means (EMM) of Journal Articlesfor each Discipline (n = 535)
  21. 21. Publication Types (2/3)Peer-Review Status of Journal ArticlesCologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educational Research Page 21HypothesisA greater number of publications with peer-review status in psychologyResultsIn general psychological projects publish more often in peer-reviewed journals than the other disciplines do.Over time we measured a significant decrease in sociological anda significant increase in educational projects.45.20%20.50%12.30%31.10%42.60%48.30%0.00%10.00%20.00%30.00%40.00%50.00%60.00%1995-1997 2006-2008SharePeriodSociologyEducational SciencesPsychologyPercentage of Journal Articles with Peer-Review Status (n = 306)
  22. 22. Discipline-Specific Publication Types (3/3)Cologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educational Research Page 220.280.330.480.260.30.200.10.20.30.40.50.61995-1997 2006-2008EMMPeriodSociologyEducationalSciencesPsychologyEstimated Marginal Means (EMM) of Monographs for each Discipline (n=558)HypothesisA greater number of monographs in sociology and educationalsciences than in psychologyResultsMonographs are more often published in educational sciences andin sociology than in psychology. In psychology and in educationalsciences publishing in monographs decreases significantly. Ineducational sciences this decrease is most pronounced.
  23. 23. Multi-AuthorshipCologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educational Research PageResultsOver time, the number of authors in total per publicationgrows significantly due to a significant increase inpsychology and in educational sciences.HypothesisAn increase in the number of authors per publication1.82.11.71.71.522.22.700.511.522.531995-1997 2006-2008CountsPeriodNumber ofAuthors in TotalSociologyEducationalSciencesPsychologyEstimated Marginal Means (EMM) of Authorsper Publication (n = 1925)
  24. 24. International Character of PublicationsCologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educational Research Page9.90%14.20%0.00%2.00%4.00%6.00%8.00%10.00%12.00%14.00%16.00%1995-1997 2006-2008SharePeriodPercentage of Publications Written in Englishin Total (n = 1925)HypothesisNumber of publications written in English increases.ResultsThe number of publications written in English increases significantly.Comparing the three core-disciplines, psychological publications havethe highest share of publications written in English.10.00%6.40%20.40%0.00%5.00%10.00%15.00%20.00%25.00%Sociology Educational Sciences PsychologyShareDisciplineSociologyEducationalSciencesPsychologyDiscipline-Specific Differences in English Publication(1995-07 + 2006-08) (n = 1925)
  25. 25. 5. SummaryResearch projects in educational research are characterized by:- a growing importance in the Social Sciences,- an increase in third-party funding,- a growing diversity in funding sources,- an increase in networking activities among research institutes,- a slow increase in the use of empirical research methods since the year2005.Publishing in educational research is characterized by:- an increase in networking of authors,- an internationalization effect over time,- disciplinary differences; over time publishing in educational researchseems to align to psychological publication behaviour.Cologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educational Research Page 26
  26. 26. 6. Outlook- Further analysis on the interrelationbetween research projects and publications,- Thematic analysis of projects,- Further analysis of publication data,- Reception analysis of publications,- Indicator specification.Cologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educational Research Page 27
  27. 27. AcknowledgementsAlexander BotteUte SondergeldJürgen WiesenhütterDr. Gabriel SchuiKatja SingletonVeronika Kuhberg-LassonDr. Sybille HinzeValerie AmanMarion SchmidtPeter MutschkeDr. Philipp MayrKarima Haddou ou MoussaDr. Andreas Oskar KempfCologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educational Research Page 28
  28. 28. Thank youfor your attention!Contact:Dr. Andreas Oskar Kempf, M.A.(LIS)Leibniz-Institute for the Social Sciences (GESIS)andreas.kempf@gesis.orgUte Sondergeld, M.A., Wiss. Dok.German Institute for International Educational Research (DIPF)sondergeld@dipf.deCologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educational Research Page 29
  29. 29. BibliographyDeutsche Forschungsgemeinschaft (2012) Förderatlas 2012. Kennzahlen zuröffentlich finanzierten Forschung in Deutschland. Weinheim.URL: http://www.dfg.de/download/pdf/dfg_im_profil/evaluation_statistik/foerderatlas/dfg-foerderatlas_2012.pdf (20.05.2013)Hinze, Sybille/Glänzel, Wolfgang (2012) Scientometric Indicators in Use: AnOverview. Präsentation im Rahmen der European Summer School forScientometrics (ESSS), Leven, Juli 2012.URL: http://www.scientometrics-school.eu/images/2012Hinze.pdf (25.05.2013)Hornbostel, Stefan (1999): Welche Indikatoren zu welchem Zweck: Input,Throughput, Output. In: Röbbecke, Martina/Simon, Dagmar (Hrsg.):Qualitätsförderung durch Evaluation? Ziele, Aufgaben und Verfahren vonForschungsbewertungen im Wandel. Dokumentation des Workshops vom 20.und 21. Mai 1999. Wissenschaftszentrum Berlin für Sozialforschung.URL: http://skylla.wzb.eu/pdf/1999/p99-003.pdf (23.05.2013)Cologne, May 28 – 31 May │ Kempf, Sondergeld │ IASSIST 2013 │ Monitoring Educational Research Page 30
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×