“Open Research Data: Implications for Science and Society”, Warsaw, Poland, May 28–29, 2015, conference organized by the Open Science Platform — an initiative of the Interdisciplinary Centre for Mathematical and Computational Modelling at the University of Warsaw. pon.edu.pl @OpenSciPlatform #ORD2015
This presentation was provided by Jonathan Adams of Clarivate Analytics during the NISO event, "Is This Still Working? Incentives to Publish, Metrics, and New Reward Systems," held on February 20, 2019.
This presentation was provided by Todd Carpenter of NISO and Nicky Agate of Columbia University during the NISO event, "Is This Still Working? Incentives to Publish, Metrics, and New Reward Systems," held on February 20, 2019.
Understanding the Depth of Google Scholar and its Implication for Webometrics...Idowu Adegbilero-Iwari
A presentation on Google Scholar, webometrics ranking of higher institutions and Open Access to research publications. The presentation details the parameters Google scholar uses for indexing research publications and the implication of that for the visibility of scholars, their institutions and their webometrics rank.
Bibliometrics, Webometrics, Altmetrics, Alternative metrics.Andrea Scharnhorst
A. Scharnhorst (2016) Bibliometrics, Webometrics, Altmetrics, Alternative metrics. Presentation given at the COST Action TD1210 Knowescape Workshop “Alternative metrics or tailored metrics: Science dynamics for science policy”, November 9-10, 2016 Warsaw
“Open Research Data: Implications for Science and Society”, Warsaw, Poland, May 28–29, 2015, conference organized by the Open Science Platform — an initiative of the Interdisciplinary Centre for Mathematical and Computational Modelling at the University of Warsaw. pon.edu.pl @OpenSciPlatform #ORD2015
This presentation was provided by Jonathan Adams of Clarivate Analytics during the NISO event, "Is This Still Working? Incentives to Publish, Metrics, and New Reward Systems," held on February 20, 2019.
This presentation was provided by Todd Carpenter of NISO and Nicky Agate of Columbia University during the NISO event, "Is This Still Working? Incentives to Publish, Metrics, and New Reward Systems," held on February 20, 2019.
Understanding the Depth of Google Scholar and its Implication for Webometrics...Idowu Adegbilero-Iwari
A presentation on Google Scholar, webometrics ranking of higher institutions and Open Access to research publications. The presentation details the parameters Google scholar uses for indexing research publications and the implication of that for the visibility of scholars, their institutions and their webometrics rank.
Bibliometrics, Webometrics, Altmetrics, Alternative metrics.Andrea Scharnhorst
A. Scharnhorst (2016) Bibliometrics, Webometrics, Altmetrics, Alternative metrics. Presentation given at the COST Action TD1210 Knowescape Workshop “Alternative metrics or tailored metrics: Science dynamics for science policy”, November 9-10, 2016 Warsaw
There are many online and in-person courses available for librarians to learn about research data management, data analysis, and visualization, but after you have taken a course, how do you go about applying what you have learned? While it is possible to just start offering classes and consultations, your service will have a better chance of becoming relevant if you consider stakeholders and review your institutional environment. This lecture will give you some ideas to get started with data services at your institution.
The United Nations Sustainable Development Goals (SDGs) are a powerful tool to track progress towards a sustainable society that meets the needs of the present without compromising the ability of future generations to meet their own needs. The research community, including librarians as information professionals and publishers, have a key role in delivering high quality information to researchers, organizations and policy makers in order to act on the world that we all live in. In the presentation, the speaker will:
give an overview of some of broader industry initiatives around the SDGs, but in particular introduce analytics and collaborations around the SDG research landscape;
address questions such as north-south collaboration and gender aspects;
introduce key finding from a recent report Pathways to Net Zero: The Impact of Clean Energy Research
All these hope to serve to illustrate the title of the talk – The Power of Data to Advance the SDGs.
Open science and the individual researcherBram Zandbelt
Slides for the Feb 8, 2017 lab meeting of Roshan Cools' Motivation & Cognitive Control group (Donders Institute), discussing the following paper:
McKiernan, E. C., Bourne, P. E., Brown, C. T., Buck, S., Kenall, A., Lin, J., … Yarkoni, T. (2016). How open science helps researchers succeed. eLife, 5, e16800. https://doi.org/10.7554/eLife.16800.
De Groote _2024_How to write a great paper v3.pptxssuserb44912
This presentation proposes an effective way to increase publication productivity for research in agricultural economics and rural development, based on regularly scheduled writing and a systematic method. It first explains how to schedule 25% of your working time for writing. It then describes the method, breaking down the writing task into five practical steps: 1) develop a story line to describe new and important research; 2) mold the story into a short outline, according to the basic structure of a scientific paper; 3) assemble the evidence for a convincing case in relevant and well-organized tables and figures; 4) write the first draft of the story; and, finally, 5) rewrite and edit the draft for style and language. With careful targeting of good journals, and some luck in the reviewing process, the proposed method should lead to several publications each year.
Journal Club - Best Practices for Scientific ComputingBram Zandbelt
Journal Club presentation for Cools lab at Donders Institute, Radboud University, Nijmegen, the Netherlands
Date: October 28, 2015
Paper:
Wilson, G., Aruliah, D. A., Brown, C. T., Hong, N. P. C., Davis, M., Guy, R. T., ... & Wilson, P. (2014). Best practices for scientific computing. PLoS Biology, 12(1), e1001745.
Research Skills Session 4: Evaluate a paper qualityNader Ale Ebrahim
Assessing the quality of a paper is a challenging issue. So, there is a requirement to evaluate a paper based on some other metrics which cover many aspects of publication quality. The quality of the article can be estimated by many aspects, such as, the number of citations, the journal IF, the author h-index, the Altmetric score, number of views, and the paper content itself. The workshop concentrates on how to evaluate and measure a paper quality by introducing some indicators.
MARKET SEGMENTATION IN SOCIAL NETWORKS THROUGH SOCIAL STRUCTURE ANALYSIS (Dep...Knut Linke
MARKET SEGMENTATION IN SOCIAL NETWORKS THROUGH SOCIAL STRUCTURE ANALYSIS
Topicality:
necessity to conduct an analysis of German social network users determined by the increasing complexity and market penetration of the Internet. These factors require modernising and extending existing market segmentation and customer group differentiation models.
Aim:
to perform an analysis of data randomly collected within the framework of the study to obtain a better understanding of Internet users and provide a more comprehensive basis for product development and research approaches. To divide social network users into groups and apply existing and established models of lifeworlds and milieus used in the study of societies. This should ensure a qualitative extension of the main model and available sub-models. The results of the study reveal significant differences among both the examined lifeworlds and social network users.
Reward, reproducibility and recognition in research - the case for going OpenDanny Kingsley
The is a keynote presentation for the Eleventh Annual Munin Conference on Scholarly Publishing http://site.uit.no/muninconf/
21 November 2016
The advent of the internet has meant that scholarly communication has changed immeasurably over the past two decades but in some ways it has hardly changed at all. The coin in the realm of any research remains the publication of novel results in a high impact journal – despite known issues with the Journal Impact Factor. This elusive goal has led to many problems in the research process: from hyperauthorship to high levels of retractions, reproducibility problems and 'cherry picking' of results. The veracity of the academic record is increasingly being brought into question. An additional problem is this static reward systems binds us to the current publishing regime, preventing any real progress in terms of widespread open access or even adoption of novel publishing opportunities. But there is a possible solution. Increased calls to open research up and provide a greater level of transparency have started to yield practical real solutions. This talk will cover the problems we currently face and describe some of the innovations that might offer a way forward.
Haustein, S. (2017). The evolution of scholarly communication and the reward ...Stefanie Haustein
Haustein, S. (2017, February). The evolution of scholarly communication and the reward system of science. Fourth Annual KnoweScape Conference 2017, 22–24 February 2017, Sofia (Bulgaria). keynote
http://knowescape.org/knowescape2017/
Reflection AssignmentThis week there will be no formal discu.docxringrid1
Reflection Assignment
This week there will be no formal discussion for our class. However, there is a reading assignment. Based on the reading assignment there is a reflection requirement. This is basically a written paragraph of about five to six sentences about what you have read. Your reflection should be posted on the discussion board (remember it is just a paragraph of five to six sentences) regarding your readings for this week. No discussions are required this week.
Reading Assignment
Our reading assignment for our class this week will involve:
Chapter 16 – Internet, Secondary Analysis and Historical Research
Chapter 17 - Intervention
Your class participation is the basis for grading of this requirement. Please note that I am actively going through everyone’s phrase three written assignments. Thank you for your continued diligence in our course.
Under chapter 16 this week, we will explore topics such as incorporating the internet for your research, revisiting participant testing as well as interviewing. Ethical concerns, historical research, and its appraisal.
In review of chapter 17, intervention in research will be explained. As per our text, not all research involves an intervention. Frequently, interventions are seen within improvement projects frequently completed in DNP programs. At this phase of research, the principle investigator interacts with their research team. Documentation stems from the methodology section.
Investigating the internet in research, please know and understand the following.
Internet-based research method
refers to any research method that uses the Internet to collect data. Most commonly, the Web has been used as the means for conducting the study, but e-mail has been used as well. The use of e-mail to collect data dates back to the 1980s while the first uses of the Web to collect data started in the mid-1990s. Whereas e-mail is principally limited to survey and questionnaire methodology, the Web, with its ability to use media, has the ability to execute full experiments and implement a wide variety of research methods. The use of the Internet offers new opportunities for access to participants allowing for larger and more diverse samples.
Reference
Salkind, N. J. (2010).
Encyclopedia of research design
(Vols. 1-0). Thousand Oaks, CA: SAGE Publications, Inc. doi: 10.4135/9781412961288
Secondary analysis
is the re-analysis of either qualitative or quantitative data already collected in a previous study, by a different researcher normally wishing to address a new research question.
Reference
Tate, J. A., Happ, M. B. (2018). Qualitative secondary analysis: A case exemplar.
Journal of Pediatric Health Care
. Volume 32, Issue 3, p. 308-312.
Historical inquiry
proceeds with the formulation of a problem or set of questions worth pursuing. In the most direct approach, students might be encouraged to analyze a document, record, or site itself. Who produced it, when, how, and why? What is the e.
There are many online and in-person courses available for librarians to learn about research data management, data analysis, and visualization, but after you have taken a course, how do you go about applying what you have learned? While it is possible to just start offering classes and consultations, your service will have a better chance of becoming relevant if you consider stakeholders and review your institutional environment. This lecture will give you some ideas to get started with data services at your institution.
The United Nations Sustainable Development Goals (SDGs) are a powerful tool to track progress towards a sustainable society that meets the needs of the present without compromising the ability of future generations to meet their own needs. The research community, including librarians as information professionals and publishers, have a key role in delivering high quality information to researchers, organizations and policy makers in order to act on the world that we all live in. In the presentation, the speaker will:
give an overview of some of broader industry initiatives around the SDGs, but in particular introduce analytics and collaborations around the SDG research landscape;
address questions such as north-south collaboration and gender aspects;
introduce key finding from a recent report Pathways to Net Zero: The Impact of Clean Energy Research
All these hope to serve to illustrate the title of the talk – The Power of Data to Advance the SDGs.
Open science and the individual researcherBram Zandbelt
Slides for the Feb 8, 2017 lab meeting of Roshan Cools' Motivation & Cognitive Control group (Donders Institute), discussing the following paper:
McKiernan, E. C., Bourne, P. E., Brown, C. T., Buck, S., Kenall, A., Lin, J., … Yarkoni, T. (2016). How open science helps researchers succeed. eLife, 5, e16800. https://doi.org/10.7554/eLife.16800.
De Groote _2024_How to write a great paper v3.pptxssuserb44912
This presentation proposes an effective way to increase publication productivity for research in agricultural economics and rural development, based on regularly scheduled writing and a systematic method. It first explains how to schedule 25% of your working time for writing. It then describes the method, breaking down the writing task into five practical steps: 1) develop a story line to describe new and important research; 2) mold the story into a short outline, according to the basic structure of a scientific paper; 3) assemble the evidence for a convincing case in relevant and well-organized tables and figures; 4) write the first draft of the story; and, finally, 5) rewrite and edit the draft for style and language. With careful targeting of good journals, and some luck in the reviewing process, the proposed method should lead to several publications each year.
Journal Club - Best Practices for Scientific ComputingBram Zandbelt
Journal Club presentation for Cools lab at Donders Institute, Radboud University, Nijmegen, the Netherlands
Date: October 28, 2015
Paper:
Wilson, G., Aruliah, D. A., Brown, C. T., Hong, N. P. C., Davis, M., Guy, R. T., ... & Wilson, P. (2014). Best practices for scientific computing. PLoS Biology, 12(1), e1001745.
Research Skills Session 4: Evaluate a paper qualityNader Ale Ebrahim
Assessing the quality of a paper is a challenging issue. So, there is a requirement to evaluate a paper based on some other metrics which cover many aspects of publication quality. The quality of the article can be estimated by many aspects, such as, the number of citations, the journal IF, the author h-index, the Altmetric score, number of views, and the paper content itself. The workshop concentrates on how to evaluate and measure a paper quality by introducing some indicators.
MARKET SEGMENTATION IN SOCIAL NETWORKS THROUGH SOCIAL STRUCTURE ANALYSIS (Dep...Knut Linke
MARKET SEGMENTATION IN SOCIAL NETWORKS THROUGH SOCIAL STRUCTURE ANALYSIS
Topicality:
necessity to conduct an analysis of German social network users determined by the increasing complexity and market penetration of the Internet. These factors require modernising and extending existing market segmentation and customer group differentiation models.
Aim:
to perform an analysis of data randomly collected within the framework of the study to obtain a better understanding of Internet users and provide a more comprehensive basis for product development and research approaches. To divide social network users into groups and apply existing and established models of lifeworlds and milieus used in the study of societies. This should ensure a qualitative extension of the main model and available sub-models. The results of the study reveal significant differences among both the examined lifeworlds and social network users.
Reward, reproducibility and recognition in research - the case for going OpenDanny Kingsley
The is a keynote presentation for the Eleventh Annual Munin Conference on Scholarly Publishing http://site.uit.no/muninconf/
21 November 2016
The advent of the internet has meant that scholarly communication has changed immeasurably over the past two decades but in some ways it has hardly changed at all. The coin in the realm of any research remains the publication of novel results in a high impact journal – despite known issues with the Journal Impact Factor. This elusive goal has led to many problems in the research process: from hyperauthorship to high levels of retractions, reproducibility problems and 'cherry picking' of results. The veracity of the academic record is increasingly being brought into question. An additional problem is this static reward systems binds us to the current publishing regime, preventing any real progress in terms of widespread open access or even adoption of novel publishing opportunities. But there is a possible solution. Increased calls to open research up and provide a greater level of transparency have started to yield practical real solutions. This talk will cover the problems we currently face and describe some of the innovations that might offer a way forward.
Haustein, S. (2017). The evolution of scholarly communication and the reward ...Stefanie Haustein
Haustein, S. (2017, February). The evolution of scholarly communication and the reward system of science. Fourth Annual KnoweScape Conference 2017, 22–24 February 2017, Sofia (Bulgaria). keynote
http://knowescape.org/knowescape2017/
Reflection AssignmentThis week there will be no formal discu.docxringrid1
Reflection Assignment
This week there will be no formal discussion for our class. However, there is a reading assignment. Based on the reading assignment there is a reflection requirement. This is basically a written paragraph of about five to six sentences about what you have read. Your reflection should be posted on the discussion board (remember it is just a paragraph of five to six sentences) regarding your readings for this week. No discussions are required this week.
Reading Assignment
Our reading assignment for our class this week will involve:
Chapter 16 – Internet, Secondary Analysis and Historical Research
Chapter 17 - Intervention
Your class participation is the basis for grading of this requirement. Please note that I am actively going through everyone’s phrase three written assignments. Thank you for your continued diligence in our course.
Under chapter 16 this week, we will explore topics such as incorporating the internet for your research, revisiting participant testing as well as interviewing. Ethical concerns, historical research, and its appraisal.
In review of chapter 17, intervention in research will be explained. As per our text, not all research involves an intervention. Frequently, interventions are seen within improvement projects frequently completed in DNP programs. At this phase of research, the principle investigator interacts with their research team. Documentation stems from the methodology section.
Investigating the internet in research, please know and understand the following.
Internet-based research method
refers to any research method that uses the Internet to collect data. Most commonly, the Web has been used as the means for conducting the study, but e-mail has been used as well. The use of e-mail to collect data dates back to the 1980s while the first uses of the Web to collect data started in the mid-1990s. Whereas e-mail is principally limited to survey and questionnaire methodology, the Web, with its ability to use media, has the ability to execute full experiments and implement a wide variety of research methods. The use of the Internet offers new opportunities for access to participants allowing for larger and more diverse samples.
Reference
Salkind, N. J. (2010).
Encyclopedia of research design
(Vols. 1-0). Thousand Oaks, CA: SAGE Publications, Inc. doi: 10.4135/9781412961288
Secondary analysis
is the re-analysis of either qualitative or quantitative data already collected in a previous study, by a different researcher normally wishing to address a new research question.
Reference
Tate, J. A., Happ, M. B. (2018). Qualitative secondary analysis: A case exemplar.
Journal of Pediatric Health Care
. Volume 32, Issue 3, p. 308-312.
Historical inquiry
proceeds with the formulation of a problem or set of questions worth pursuing. In the most direct approach, students might be encouraged to analyze a document, record, or site itself. Who produced it, when, how, and why? What is the e.
Research Evaluation in an Open Science contextHilda Muchando
The Knowledge Exchange has published the report ‘𝙊𝙥𝙚𝙣𝙣𝙚𝙨𝙨 𝙋𝙧𝙤𝙛𝙞𝙡𝙚: 𝙈𝙤𝙙𝙚𝙡𝙡𝙞𝙣𝙜 𝙧𝙚𝙨𝙚𝙖𝙧𝙘𝙝 𝙚𝙫𝙖𝙡𝙪𝙖𝙩𝙞𝙤𝙣 𝙛𝙤𝙧 𝙤𝙥𝙚𝙣 𝙨𝙘𝙝𝙤𝙡𝙖𝙧𝙨𝙝𝙞𝙥’.
The report presents how the Openness Profile can help address existing gaps in the assessment of Open Science in relation to issues such as:
• The need to accelerate the transition to Open - operationalising and normalising open scholarship practices has proven challenging.
• Conflicting ambitions combined with strong network effects that punish those who deviate from sector norms around research assessment and practice.
• The economic nature of challenges, either financial or relating to actors’ incentives, associated with the transition to open scholarship.
• Distortion of researcher behaviour due to over-reliance on traditional metrics.
• Underfunded and underdeveloped funder grant information systems. Poor adoption of PIDs and little to no interoperability with downstream stakeholders.
• Key contributors to the academic knowledge ecosystem being under-recognised
• Research being organised with ‘well defined’ rules that do not include ‘open’-related criteria.
The potential to improve open research evaluation practice as well as the requirements to implement the Openness Profile are addressed, including recommendations for stakeholders.
Dissemination of ResearchThe goal of the scientific process is n.docxstelzriedemarla
Dissemination of Research
The goal of the scientific process is not only the knowledge gained from research, but the dissemination of that knowledge. For science to advance, those lessons must be shared, scrutinized, understood, and appreciated by others. This week, you will be introduced to methods of displaying and presenting data, summarizing and communicating the fruits of one's research labors in an effective, clear, and compelling manner. You will also identify areas for disseminating research, including journals and conferences.
Students will:
Prepare a research proposal outline for presentation
To complete
this Assignment:
Design a 9- to 12-slide presentation of your research proposal including the following information:
Title of your research :
Childhood Immunization
The research problem :
Does educating parents in communities about childhood immunization reduce diseases rate or enhance vaccine rate among children?
The purpose of your research
The research design
The population and sample
Data collection strategies
Analysis procedure
Possible areas for dissemination
References
Resources Needed for this assigment listed below and also added as attachement
Neutens, J. J., & Rubinson, L. (2014).
Research techniques for the health sciences
(5th ed.). San Francisco: Benjamin Cummings.
Chapter 13, "Techniques for Data Presentation"
Chapter 13 provides techniques for data presentations, including table presentations and figure presentations. These techniques can be used to enhance text, increase productivity, and provide a variety of interactive activities.
Chapter 14, "Communicating your Research"
Once research has been conducted, the information and data is written in a report format for dissemination. Chapter 14 presents steps for writing the introduction, review of literature, methodology, presentation and analysis of data, and summary and conclusion.
https://www.jyi.org/2018-august/2018/7/30/now-is-the-time-to-reassess-stance-on-hpv-vaccine
Your written assignments must follow APA guidelines. Be sure to support your work with specific citations from this week's Learning Resources and additional scholarly sources as appropriate. Refer to the
Pocket Guide to APA Style
to ensure your in-text citations and reference list are correct.
...
Open science curriculum for students, June 2019Dag Endresen
Living Norway seminar on Open Science in Trondheim 12th June 2019.
https://livingnorway.no/2019/04/26/living-norway-seminar-2019/
https://www.gbif.no/events/2019/living-norway-seminar.html
Introduction to “Research Tools”: Tools for Collecting, Writing, Publishing, ...Nader Ale Ebrahim
“Research Tools” enable researchers to collect, organize, analyze, visualize and publicized research outputs. I have collected over 700 tools that enable researchers to follow the correct path in research and to ultimately produce high-quality research outputs with more accuracy and efficiency. “Research Tools” consists of a hierarchical set of nodes. It has four main nodes: (1) Searching the literature, (2) Writing a paper, (3) Targeting suitable journals, and (4) Enhancing visibility and impact of the research. This presentation will provide an overview to the most important tools from searching literature to disseminating researcher outputs. The e-skills learned from the workshop are useful across various research disciplines and research institutions.
Similar to Incentivize Replication in Economics: Can Data Journals Help? (20)
0x01 - Newton's Third Law: Static vs. Dynamic AbusersOWASP Beja
f you offer a service on the web, odds are that someone will abuse it. Be it an API, a SaaS, a PaaS, or even a static website, someone somewhere will try to figure out a way to use it to their own needs. In this talk we'll compare measures that are effective against static attackers and how to battle a dynamic attacker who adapts to your counter-measures.
About the Speaker
===============
Diogo Sousa, Engineering Manager @ Canonical
An opinionated individual with an interest in cryptography and its intersection with secure software development.
This presentation, created by Syed Faiz ul Hassan, explores the profound influence of media on public perception and behavior. It delves into the evolution of media from oral traditions to modern digital and social media platforms. Key topics include the role of media in information propagation, socialization, crisis awareness, globalization, and education. The presentation also examines media influence through agenda setting, propaganda, and manipulative techniques used by advertisers and marketers. Furthermore, it highlights the impact of surveillance enabled by media technologies on personal behavior and preferences. Through this comprehensive overview, the presentation aims to shed light on how media shapes collective consciousness and public opinion.
Sharpen existing tools or get a new toolbox? Contemporary cluster initiatives...Orkestra
UIIN Conference, Madrid, 27-29 May 2024
James Wilson, Orkestra and Deusto Business School
Emily Wise, Lund University
Madeline Smith, The Glasgow School of Art
This presentation by Morris Kleiner (University of Minnesota), was made during the discussion “Competition and Regulation in Professions and Occupations” held at the Working Party No. 2 on Competition and Regulation on 10 June 2024. More papers and presentations on the topic can be found out at oe.cd/crps.
This presentation was uploaded with the author’s consent.
Have you ever wondered how search works while visiting an e-commerce site, internal website, or searching through other types of online resources? Look no further than this informative session on the ways that taxonomies help end-users navigate the internet! Hear from taxonomists and other information professionals who have first-hand experience creating and working with taxonomies that aid in navigation, search, and discovery across a range of disciplines.
Acorn Recovery: Restore IT infra within minutesIP ServerOne
Introducing Acorn Recovery as a Service, a simple, fast, and secure managed disaster recovery (DRaaS) by IP ServerOne. A DR solution that helps restore your IT infra within minutes.
Incentivize Replication in Economics: Can Data Journals Help?
1. ZBW is Member of the Leibniz Association.
Incentivize Replication in Economics
– Can Data Journals Help?
IASSIST 2016
Bergen, May 31 – June 3, 2016
Ralf Toepfer
ZBW Leibniz Information Centre for Economics (Kiel / Hamburg)
Photo: Lukas Roth
Photo: Sönke Wurr, Münchow-Industrie-Fotos
2. 2
Agenda
• Replication(s) (in economics)
− The Replication Crisis
− „Replication“ – a definition
− Opinions on data sharing and replication
− Barriers to replication
• Publication opportunities for replication works
− Scholarly Journals
− Data Journals
• Summary & Conclusion
5. Seite 5
Replication in Economics Research
• Dewald et al. (1986) attempted to replicate 54
papers published in the Journal of Money, Credit
and Banking and could replicate only two (2/54)
• McCullogh et al. (2006) tried to replicate 69
articles published in the same journal and could
only replicate 14 (14/69)
• McCullogh et al. (2008) attempted to replicate 117
articles published in the Federal Reserve Bank of
St. Louis Review and could only replicate 9
(9/117)
• Chang et al. (2015) successfully replicate 29 of 59
papers (49%) from several journals even with
help from the authors
Chang, Andrew C., and Phillip Li (2015). „Is Economics Research Replicable? Sixty Published Papers from
Thirteen Journals Say „Usually Not“, „Finance and Economics Discussion Series 2015-083. Washington: Board
of Governors of the Federal Reserve System, http://dx.doi.org/10.17016/FEDS.2015.083
6. What is replication?
Page 6
• Replication in economics could be using:
a) same data and same methods
b) same methods but different data
c) same data but different methods
d) new data and new methods
• Distinction
a. „pure“ replication
b. „scientific“ replication
Daniel S. Hamermesh: Replication in Economics. NBER Working Paper 13026. April 2007.
http://www.nber.org/papers/w13026
Maren Duvendack, Richard W. Palmer-Jones and Robert Reed: Replications in Economics: A Progress
Report. Econ Journal Watch 12(2), May 2015: 164-191
7. Definition matters!
Page 7
„Thus if a replication test gives discrepant
results, under current usage of the term,
this could mean a wide spectrum of
things – from signaling a legitimate
disagreement over the best method
(science), to signaling incompetence and
fraud (pseudoscience).“ (Clemens
2015:1)
Michael A. Clemens: The Meaning of Failed Replications: A
Review and Proposal. IZA DP No. 9000.
http://ftp.iza.org/dp9000.pdf
Definition matters!
8. 8
Data Sharing
„The status quo in empirical research in economics and management
is not to share data.“ (Andreoli-Versbach, Mueller-Langer 2014, p.11)
(N=488) Do not share
data
Sporadically
share data
Share data
regulary
Responses 394 82 12
Percent 80.74% 16.8% 2.46%
Andreoli-Versbach, P., Mueller-Langer, F., Open access to data: An ideal professed but not practised. Res.
Policy (2014), http://dx.doi.org/j.respol.2014.04.008
9. 9
Opinions on Data Sharing
On a scale from 1 to 5;
1=„Strongly disagree“ –
5=„Agree completely“
Strongly
Disagree
2 3 4 Agree
Completely
Researchers should
generally publish their data
(N=1491)
1,95% 5,9% 16,57% 31,32% 44,27%
Freely available research
data is a great contribution
to scientific progress
(N=1449)
1,73% 3,8% 11,32% 25,05% 58,11%
It is common in my
discipline / research
community to share data
(N=1436)
13,86% 23,33% 27,37% 24,44% 11%
Benedikt Fecher, Sascha Friesike, Marcel Hebing, Stephanie Linek, Armin Sauermann: A Reputation Economy: Results from
an Empirical Survey on Academic Data Sharing, Berlin and Kiel, February 2015, RatSWD Working Paper 246
10. 10
Discipline-specific differences
„It is common in my discipline/research community to share data“
Discipline (Observations) Strongly
Disagree
2 3 4 Agree
Completely
Natural science (472) 6,36% 18,64% 26,69% 30,72% 17,58%
Social Science (425) 19,76% 27,29% 26,59% 19,76% 6,59%
Human science (168) 10,71% 17,26% 29,76% 30,95% 11,31
Engineering (119) 10,92% 22,69% 40,34% 18,49% 7,56%
Humanities (168) 26,19% 29,76% 20,83% 16,07% 7,14%
Agricultural science (64) 7,81% 28,13% 28,13% 28,13% 7,81%
Total (1416) 13,70% 23,16% 27,54% 25,58% 11,02%
Benedikt Fecher, Sascha Friesike, Marcel Hebing, Stephanie Linek, Armin Sauermann: A Reputation
Economy: Results from an Empirical Survey on Academic Data Sharing, Berlin and Kiel, February 2015,
RatSWD Working Paper 246
11. Replication studies
Page 11
Researcher‘s opinions about replication
studies:
• 84% agree that replications are necessary
for improving scientific output
• 71% disagree with the statement that
replications are not worthwhile
• 58% of the researchers never attempted
any replication study
Benedikt Fecher, Mathis Fräßdorf and Gert G. Wagner: Perceptions
and Practices of Replication by Social and Behavioral Scientists –
Making Replications a Mandatory Element of Curricula Would Be
Useful. DIW Berlin. Discussion Paper 1572, April 2016
12. An ideal professed but not to be practiced
„Economists treat
replications the way
teenagers treat chastity –
as an ideal professed but
not to be practiced.“
Page 12
Daniel S. Hamermesh: Replication in Economics. NBER Working Paper 13026. April 2007.
http://www.nber.org/papers/w13026
13. Barriers to replication
• Low data sharing rates
• Issues with data documentation
Limited attractiveness to conduct replication studies
Page 13
„The most important reason why researchers might be reluctant to conduct a replication
study is ‚because it is difficult to successfully accomplish and it carries more risk than
potential reward for both the replicator and the originator of the research‘“
(Park 2004, quote according to Fecher, Fräßdorf & Wagner 2016:4)
Park CL. What is the value of replicating other studies? Res Eval. 2004;13:189-195
doi:10.3152/147154404781776400
14. What can be done to foster replication?
I. Scholarly Journals
II. Data Journals
Page 14
„By itself, access to data and code might be inadequate to incentivize
replication studies: Researchers also need outlets to publish the results of
replication efforts.“ (Duvendack et al. 2015:164)
https://replicationnetwork.com/replication-studies/
http://replication.uni-goettingen.de/wiki/index.php/Main_Page
15. Publication opportunities I: Scholarly Journals
Page 15
Maren Duvendack, Richard W. Palmer-Jones and Robert Reed:
Replications in Economics: A Progress Report. Econ Journal Watch
12(2), May 2015: 164-191
16. Publication opportunities I: Scholarly Journals
Page 16
Maren Duvendack, Richard W. Palmer-Jones and Robert Reed:
Replications in Economics: A Progress Report. Econ Journal Watch
12(2), May 2015: 164-191
• Journal of Applied Econometrics accounts for
about one-fifth (19.1%) of all replication
studies
• Six journals accounts for almost 60% (58.7%)
• Only ten journals have ever published more
than three replication studies
„There is a structural lack of
interest in replication“ (Hamermesh
2007:9)
17. Publication opportunites II: Data Journals
• Data Journal
− focuses on the dataset rather than the research article
− data creators get credit for their work
− dataset must be citable and peer reviewed
• Data Paper
− provide a citable journal publication
− describe data in a structured human-readable form
− bring the existence of the data to the attention of the scholary
community
Page 17
Chavan and Penev: The data paper: a mechanism to incentivize data publishing in biodiversity science.
BMC Bioinformatics 2011 12(Suppl 15):S2. doi: 10.1186/1471-2105-12-S15-S2
18. Publication opportunities II: Data Journals
Page 18
• DFG-funded project „International
Journal of Economic Micro Data
(IJEMD)
• Open Access, e-only „Data Journal“
• Three types of articles
− descriptions (data papers)
− replication studies
− research articles
• Program code and descriptions will
be published in a data archive
• Authors could be honored in due
form of citations and reputation
http://gepris.dfg.de/gepris/projekt/278201669
Enhanced concept of data
journals
19. Summary & Conclusion
• There is a need for more replications in economics to regain trust and
credibility.
• In the current system neither data sharing nor replicating other people‘s result
progresses researchers career.
• Though established scholarly journals have adopted replication policies in
recent years, replication activities only slightly increased.
• Therefore, dedicated publication platforms for replication studies that offer
the possibility to reward authors in form of citations and reputation are
needed.
• Data Journals should enhance their scope and publish not only data papers
but also replication studies.
Page 19
20. Literature
Andreoli-Versbach, P., Mueller-Langer, F., Open access to data: An ideal professed but not practised. Res. Policy (2014),
http://dx.doi.org/j.respol.2014.04.008
Benedikt Fecher, Sascha Friesike, Marcel Hebing, Stephanie Linek, Armin Sauermann: A Reputation Economy: Results from an Empirical
Survey on Academic Data Sharing, Berlin and Kiel, February 2015, RatSWD Working Paper 246.
http://www.ratswd.de/dl/RatSWD_WP_246.pdf
McCullough, B.D. / McGeary, Kerry Anne / Harrison, Teresa D.: Lessons from the JMCB Archive. Journal of Money, Credit, and Banking, vol.
38, No.4(2006),pp. 1093-1107
Michael A. Clemens: The Meaning of Failed Replications: A Review and Proposal. IZA DP No. 9000. http://ftp.iza.org/dp9000.pdf
Maren Duvendack, Richard W. Palmer-Jones and Robert Reed: Replications in Economics: A Progress Report. Econ Journal Watch 12(2), May
2015: 164-191
Benedikt Fecher, Mathis Fräßdorf and Gert G. Wagner: Perceptions and Practices of Replication by Social and Behavioral Scientists – Making
Replications a Mandatory Element of Curricula Would Be Useful. DIW Berlin. Discussion Paper 1572, April 2016
Chang, Andrew C., and Phillip Li (2015). „Is Economics Research Replicable? Sixty Published Papers from Thirteen Journals Say „Usually
Not“, „Finance and Economics Discussion Series 2015-083. Washington: Board of Governors of the Federal Reserve System,
http://dx.doi.org/10.17016/FEDS.2015.083
Christian Zimmermann: On the Need for a Replication Journal. Federal Reserve Bank of St. Louis Working Paper Series. Working Paper 2015-
016A http://research.stlouisfed.org/wp/2016/2015-016.pdf
Daniel S. Hamermesh: Replication in Economics. NBER Working Paper 13026. April 2007. http://www.nber.org/papers/w13026
Dewald, W.G., Thursby, J.G., Anderson, R.G., (1986). Replication in empirical economics: the Journal of Money, Credit and Banking project.
American Economic Review 76 (4), 587-603
McCullogh, B.D., McGeary, K.A., Harrison, T.D. (2008): Do economics journal archives promote replicable research? Canadian Journal of
Economics 41 (4), 1406-1420
Park CL. What is the value of replicating other studies? Res Eval. 2004;13:189-195 doi:10.3152/147154404781776400
Chavan and Penev: The data paper: a mechanism to incentivize data publishing in biodiversity science. BMC Bioinformatics 2011 12(Suppl
15):S2 doi:10.1186/1471-2105-12-S15-S2
20
21. 21
Thanks for your attention!
Contact:
Ralf Toepfer
ZBW – Leibniz Information Centre for
Economics
Neuer Jungfernstieg 21, 20354
Hamburg, Germany
E: r.toepfer@zbw.eu
T: +49(0)40.42834-449
Twitter: @thiaru