SlideShare a Scribd company logo
Academic incentives and science
Gerry Carter
DO ACADEMIC
INCENTIVES
HURT SCIENCE ?
YES.
1.YES
2. Maybe- what’s the argument?
3. Huh, what are you talking about?
4. NO
What is science?
• Wikipedia: systematic enterprise that builds and
organizes knowledge in the form of testable explanations
and predictions
• Dictionary: systematic knowledge of the physical world
gained through observation and experimentation
What is science?
• Wikipedia: systematic enterprise that builds and organizes
knowledge in the form of testable explanations and predictions
• Dictionary: systematic knowledge of the physical world gained
through observation and experimentation
• the most rigorous way of gaining new knowledge about
nature (whatever that may be), and the sum of that
knowledge
• When you are adhering to the highest standards of logic and
evidence, you are thinking scientifically.(And when you’re not,
you’re not.)*
*Author: Sam Harris
science
• Rigor
• Falsifiability
• Repeatability
• Measures of uncertainty
What is academia?
• Wikipedia: the internationally recognized establishment
of professional scholars and students, usually centered
around colleges and universities, who are engaged in
higher education and research.
• the environment or community concerned with the pursuit
of research, education, and scholarship.
• the “job” of being a scholar at a university and the
community of those scholars
What is academia?
• Wikipedia: the internationally recognized establishment
of professional scholars and students, usually centered
around colleges and universities, who are engaged in
higher education and research.
• the environment or community concerned with the pursuit
of research, education, and scholarship.
• the “job” of being a scholar at a university
• By “academia”, I don’t mean teaching (vs research).
• And I don’t mean the humanities
• I mean the aspects that affect scientific research
How is scientific success different from
academic success?
What is success?
Science
= being correct, falsifiable, and repeatable
Academia
= having influence and impact, being hired, getting funding
My argument
1. Incentive structures of science are philosophical.
Examples: double-blind design, replication, and peer-review.
They are typically designed to aid scientific goals.
2. Incentives structures in academia are economic.
Examples: tenure, h-index, and journal impact factors. They
are not designed to meet scientific goals. They emerge as
byproducts of human goals and economic competition within
and among social groups.
(Economic competition exists between publishing companies, between
individuals, between labs, between departments, between universities, and
academia vs other state-funded institutions).
3. Therefore, many academic incentives  bad science.
4. Question: How can we design better academic incentive
structures so that they serve the interests of science?
My argument
1. Incentive structures of science are philosophical.
Examples: double-blind design, replication, and peer-review.
They are typically designed to aid scientific goals.
2. Incentives structures in academia are economic.
Examples: tenure, h-index, and journal impact factors. They
are not designed to meet scientific goals. They emerge as
byproducts of human goals and economic competition within
and among social groups.
(Economic competition exists between publishing companies, between
individuals, between labs, between departments, between universities, and
academia vs other state-funded institutions).
3. Therefore, many academic incentives  bad science.
4. Question: How can we design better academic incentive
structures so that they serve the interests of science?
My argument
1. Incentive structures of science are philosophical.
Examples: double-blind design, replication, and peer-review.
They are typically designed to aid scientific goals.
2. Incentives structures in academia are economic.
Examples: tenure, h-index, and journal impact factors. They
are not designed to meet scientific goals. They emerge as
byproducts of human goals and economic competition within
and among social groups.
(Economic competition exists between publishing companies, between
individuals, between labs, between departments, between universities,
and academia vs other state-funded institutions).
3. Therefore, many academic incentives  bad science.
4. Question: How can we design better academic incentive
structures so that they serve the interests of science?
My argument
1. Incentive structures of science are philosophical.
Examples: double-blind design, replication, and peer-review.
They are typically designed to aid scientific goals.
2. Incentives structures in academia are economic.
Examples: tenure, h-index, and journal impact factors. They
are not designed to meet scientific goals. They emerge as
byproducts of human goals and economic competition within
and among social groups.
(Economic competition exists between publishing companies, between
individuals, between labs, between departments, between universities, and
academia vs other state-funded institutions).
3. Therefore, many academic incentives  bad science.
4. Question: How can we design better academic incentive
structures so that they serve the interests of science?
My argument
1. Incentive structures of science are philosophical.
Examples: double-blind design, replication, and peer-review.
They are typically designed to aid scientific goals.
2. Incentives structures in academia are economic.
Examples: tenure, h-index, and journal impact factors. They
are not designed to meet scientific goals. They emerge as
byproducts of human goals and economic competition within
and among social groups.
(Economic competition exists between publishing companies, between
individuals, between labs, between departments, between universities, and
academia vs other state-funded institutions).
3. Therefore, many academic incentives  bad science.
4. Question: How can we design better academic incentive
structures so that they serve the interests of science?
Science and academia
have many shared incentives
To be a good scientist and a good academic…
• Do good research
• Make important discoveries
• Publish good papers
• Produce public knowledge
• Teach science to students well
As a reviewer, when do you reject a paper
for not enough significance?
As a reviewer, when do you reject a paper
for not enough significance?
What makes a paper good enough?
Science: good enough to be useful evidence for readers
Academia: good enough to be an accolade for the writer
The academic filter in biology
• 16,000 new US Biology PhD students/ year
• Of those recruits, 63% stay until they graduate
• Average time to degree = 7 years
• Of those graduates, 70% get a postdoc
• Of those postdocs, 30% do 1+ more postdoc
• Average postdoc time = ? but more than 4 years
• Within 6 years, 15% of postdocs get a tenure-track job
• We have a 6-7% chance of getting the job we were trained for.
Source: ascb.org “Where will a biology PhD take you”
Mentors: old vs. young
BUT…
…”Not exactly the sort of approach likely to reap
accolades today.”
“I doubt that I would fare particularly well in today’s
academic environment.”
Divergent interests (1 of 2)
To be a good scientist… To be a good academic…
Be skeptical of your results “Sell” your results
Divergent interests (1 of 2)
To be a good scientist… To be a good academic…
Be skeptical of your results “Sell” your results
Interpret conclusions carefully Highlight/exaggerate importance
Divergent interests (1 of 2)
To be a good scientist… To be a good academic…
Be skeptical of your results “Sell” your results
Interpret conclusions carefully Highlight/exaggerate importance
Publish negative results Publish “strategically”
Divergent interests (1 of 2)
To be a good scientist… To be a good academic…
Be skeptical of your results “Sell” your results
Interpret conclusions carefully Highlight/exaggerate importance
Publish negative results Publish “strategically”
Ignore social prestige Use impact factors to make writing
decisions
Divergent interests (1 of 2)
To be a good scientist… To be a good academic…
Be skeptical of your results “Sell” your results
Interpret conclusions carefully Highlight/exaggerate importance
Publish negative results Publish “strategically”
Ignore social prestige Use impact factors to make writing
decisions
Challenge authority Cite authority. Make friends.
Divergent interests (1 of 2)
To be a good scientist… To be a good academic…
Be skeptical of your results “Sell” your results
Interpret conclusions carefully Highlight/exaggerate importance
Publish negative results Publish “strategically”
Ignore social prestige Use impact factors to make writing
decisions
Challenge authority Cite authority. Make friends.
Replicate. Replicate. Replicate… if you must
Divergent interests (1 of 2)
To be a good scientist… To be a good academic…
Be skeptical of your results “Sell” your results
Interpret conclusions carefully Highlight/exaggerate importance
Publish negative results Publish “strategically”
Ignore social prestige Use impact factors to make writing
decisions
Challenge authority Cite authority. Make friends.
Replicate. Replicate. Replicate… if you must
Novel exciting results are less likely to be
true. Double-check them.
Publish novel exciting results before you
get scooped.
Divergent interests (2 of 2)
To be a good scientist… To be a good academic…
P=0.06 P=0.04
Masicampoa & Lalande
2012 Quart J Exp Psych
Divergent interests (2 of 2)
To be a good scientist… To be a good academic…
P=0.06 P=0.04
Help others replicate and find flaws in
your work
Have a “territory”
Divergent interests (2 of 2)
To be a good scientist… To be a good academic…
P=0.06 P=0.04
Help others replicate and find flaws in
your work
Have a “territory”
Be clear Sound smart
Divergent interests (2 of 2)
To be a good scientist… To be a good academic…
P=0.06 P=0.04
Help others replicate and find flaws in
your work
Have a “territory”
Be clear Sound smart
“Science is a marathon, not a sprint” Funded work must be “transformative”
Divergent interests (2 of 2)
To be a good scientist… To be a good academic…
P=0.06 P=0.04
Help others replicate and find flaws in
your work
Have a “territory”
Be clear Sound smart
“Science is a marathon, not a sprint” Funded work must be “transformative”
Natural history is crucial for biological
understanding
Natural history is “low impact”
Divergent interests (2 of 2)
To be a good scientist… To be a good academic…
P=0.06 P=0.04
Help others replicate and find flaws in
your work
Have a “territory”
Be clear Sound smart
“Science is a marathon, not a sprint” Funded work must be “transformative”
Natural history is crucial for biological
understanding
Natural history is “low impact”
Long-term goals for important truths Short-term results with high impact
Divergent interests (2 of 2)
To be a good scientist… To be a good academic…
P=0.06 P=0.04
Help others replicate and find flaws in
your work
Have a “territory”
Be clear Sound smart
“Science is a marathon, not a sprint” Funded work must be “transformative”
Natural history is crucial for biological
understanding
Natural history is “low impact”
Long-term goals for important truths Short-term results with high impact
Build a consensus. Hot controversial topics are higher impact
What is academic impact?
• Influence on your field
• Usually involves counting citations, regardless of whether
you are being cited for being right or being wrong.
• Impact is measured at the level of journals (impact factor)
and authors (h-index)
• But actual scientific progress made by articles, methods,
hypotheses, theories, and often teams of researchers
What is academic impact?
• A fundamental problem:
• Over time, scientific validity and importance of an article
both increases and decreases.
• Measures of total impact can only increase.
• Papers that are influentially wrong have higher impact
metrics than papers that are correct.
A survey
From Daniel Dennett:
• (A) You solve the major problem of your choice so
conclusively that there is nothing left to say (thanks
to you, part of the field closes down forever, and you
get a footnote in history).
• (B) You write an article or book of such tantalizing
perplexity and controversy that it stays on the
required reading list for centuries to come.
Which would you choose?
Philosophers and Scientists
From Daniel Dennett:
• (A) You solve the major problem of your choice so
conclusively that there is nothing left to say (thanks
to you, part of the field closes down forever, and you
get a footnote in history).
• (B) You write an article or book of such tantalizing
perplexity and controversy that it stays on the
required reading list for centuries to come.
Scientists choose A. Philosophers often choose B.
Some of the most influential scholars
said little that was correct
Jacques Derrida
“Brilliant and controversial, Jacques Derrida is one
of the most important thinkers of our time, his
considerable body of work having ineradicably
altered the landscape of thought in the 20th and
21st centuries.”
• Wikipedia: best known for developing a form of semiotic analysis known as
deconstruction
• one of the major figures associated with post-structuralism and postmodern
philosophy
• published more than 40 books, hundreds of essays
• significant influence upon the humanities and social sciences, including—in
addition to philosophy and literature—law, anthropology, historiography, linguistics,
sociolinguistics, psychoanalysis, political theory, feminism, and gay and lesbian
studies…. also influenced architecture (in the form of deconstructivism), music, art,
and art critics. Derrida was said to "leave behind a legacy of himself as the
'originator' of deconstruction.”
Deconstruction denies the possibility of a "pure presence": "the present or presence of
sense to a full and primordial intuition".[13][14] It thus denies the possibility of essential
or intrinsic and stable meaning and the unmediated access to "reality”.
Dictionary: a method of critical analysis of philosophical and literary language that
emphasizes the internal workings of language and conceptual systems, the relational
quality of meaning, and the assumptions implicit in forms of expression.
“Pluralistic ignorance”
• no one actually believes, but everyone believes that
everyone else believes
The Emperor’s
New Clothes
The anti-academic culture of science
Richard Feynman on his Nobel Prize:
http://www.liveleak.com/view?i=e5a_1185599458
How should we change academia?
Towards a better academia…
1. Make publishing trivial. Make
peer-review rigorous.
2. Encourage replication.
3. Decentralize funding, rather than
centralized, trickle-down funding
Publishing
• Make publishing trivial. Make peer-review rigorous.
Article-level metrics.
• Get rid of impact factors, and then journals.
Journals make money from nothing
Journals make money from nothing
Journals are brands
Journals are brands
In theory, the whole point of a brand is quality control. But in practice,
brands manipulate consumers into seeing high quality regardless of
actual quality.
Journals are brands
In theory, the whole point of a brand is quality control. But in practice,
brands manipulate consumers into seeing high quality regardless of
actual quality.
Journal brands serve as third-party assessment, so that readers “can
judge an article by its cover” and can outsource their evaluation of papers
to 1 editor and 2+ reviewers. But this is better accomplished by a rating
system, like we use for judging movies and books.
Public can read
Publishing
Data
Conclusions
Paper Overhyped
bullshitPicked up
by media
This is how it should work.
“Experimental
treatments affected
gut biota (p=0.01)
and were correlated
with longevity
(p=0.03)”
“Gut microbiota is
linked to longevity
in male mice”
“Ten reasons why
your microbiome is
the key to health
and long life”
Public can
read
Public can read
Publishing
Data
Conclusions
Paper Overhyped
bullshitPicked up
by media
This is how it should work.
But this is how it often works…
SPIN
Data Journal.
University
access only.
Overhyped
bullshit
SPIN
Slightly
hyped
paperWritten for
high
impact
SPIN
Journal demands
brevity and certainty
Publishing
• Make publishing trivial. Make peer-review rigorous.
Article-level metrics.
• Get rid of journals, impact factors, and academic publishing as a
business.
• Central repository of open-access, peer-reviewed articles (and
their reviews).
• Magazines can then draw stories from those articles with
varying degrees of rigor (Nature, New Scientist, Fox News).
This is already basically how ArXive and PLOS ONE work.
Publishing
• Make publishing trivial. Make peer-review rigorous.
Article-level metrics.
• Get rid of journals, impact factors, and academic publishing as a
business.
• Central repository of open-access, peer-reviewed articles (and their
reviews). Magazines can then draw stories from those articles with
varying degrees of rigor (Nature, New Scientist, Fox News).
• Publish negative results and datasets.
Publishing
• Make publishing trivial. Make peer-review rigorous.
Article-level metrics.
• Get rid of journals, impact factors, and academic publishing as a
business.
• Central repository of open-access, peer-reviewed articles (and their
reviews). Magazines can then draw stories from those articles with
varying degrees of rigor (Nature, New Scientist, Fox News).
• Publish negative results and datasets.
• Published peer reviews. Make them objective and open access.
Focus on soundness of results, not subjective importance (which
can be assessed over time). Have standardized peer review
checklists.
Replication
• Encourage replication.
• Open data.
• Metrics to identify what is worth replicating.*
• Publish replications.
A replication crisis
Social psychology
Psychology
Biomedicine
Funding
• Decentralize funding
• Rather than centralized, trickle-down funding
• One idea: most labs receive baseline funding. Some percentage of
all funds must be invested in another lab.
Bollen et al. 2014. From funding agencies to scientific agency:
collective allocation of science funding as an alternative to peer
review. EMBO Reports.
Scientists spend about 40% of their time
chasing grants (2007 U.S. Govt study)
experimental physicist at Columbia University did the calculations:
some grants have a net negative value: they do not pay for the time
that applicants and peer reviewers spend on them.
(Dr. No Money: The Broken Science Funding System. 2011. Scientific American.)
Funding
• Decentralize funding
• Rather than centralized, trickle-down funding
• Use peer-review-like “voting” for funding.
• One idea: most labs receive baseline funding. 30% of all funds
must be invested in another lab justified by merit or promise.
.
Bollen et al. 2014. From funding agencies to scientific agency: collective
allocation of science funding as an alternative to peer review. EMBO Reports
In summary
• Academic incentives can erode good scientific practices.
• We can change academic practices. However, there
needs to be a cooperative consensus, because this is a
public goods dilemma. Like a boycott, it’s only cost-
effective, if everyone does it.
• It would be useful to enact policy changes that offset the
poor incentive structures of academia.
What do you think?
Thanks for the discussion.

More Related Content

What's hot

Deep learning health care
Deep learning health care  Deep learning health care
Deep learning health care
Meenakshi Sood
 
Factor Analysis in Research
Factor Analysis in ResearchFactor Analysis in Research
Factor Analysis in Research
Qasim Raza
 
Hypothesis testing
Hypothesis testingHypothesis testing
Hypothesis testing
Afra Fathima
 
Overview of the systematic review process
Overview of the systematic review processOverview of the systematic review process
Overview of the systematic review process
Rizwan S A
 
How to calculate power in statistics
How to calculate power in statisticsHow to calculate power in statistics
How to calculate power in statistics
Stat Analytica
 
Using NVivo QSR Theory and Practice for Qualitative Data Analysis in a PhD
Using NVivo QSR Theory and Practice for Qualitative Data Analysis in a PhDUsing NVivo QSR Theory and Practice for Qualitative Data Analysis in a PhD
Using NVivo QSR Theory and Practice for Qualitative Data Analysis in a PhD
KEDGE Business School
 
Prediction of cardiovascular disease with machine learning
Prediction of cardiovascular disease with machine learningPrediction of cardiovascular disease with machine learning
Prediction of cardiovascular disease with machine learning
Pravinkumar Landge
 
Health Prediction System - an Artificial Intelligence Project 2015
Health Prediction System - an Artificial Intelligence Project 2015Health Prediction System - an Artificial Intelligence Project 2015
Health Prediction System - an Artificial Intelligence Project 2015
Maruf Abdullah (Rion)
 
Regression shrinkage: better answers to causal questions
Regression shrinkage: better answers to causal questionsRegression shrinkage: better answers to causal questions
Regression shrinkage: better answers to causal questions
Maarten van Smeden
 
Qualitative Data Analysis (Steps)
Qualitative Data Analysis (Steps)Qualitative Data Analysis (Steps)
Qualitative Data Analysis (Steps)
guest7f1ad678
 
SMART HEALTH PREDICTION USING DATA MINING by Dr.Mahboob Khan Phd
SMART HEALTH PREDICTION USING DATA MINING by Dr.Mahboob Khan PhdSMART HEALTH PREDICTION USING DATA MINING by Dr.Mahboob Khan Phd
SMART HEALTH PREDICTION USING DATA MINING by Dr.Mahboob Khan Phd
Healthcare consultant
 
Introduction and Tools of Research
Introduction and Tools of ResearchIntroduction and Tools of Research
Introduction and Tools of Research
Myke Evans
 
Data analysis
Data analysisData analysis
Data analysis
amlbinder
 
Clinical Decision Support Systems
Clinical Decision Support SystemsClinical Decision Support Systems
Clinical Decision Support Systems
pradhasrini
 
Fixed-effect and random-effects models in meta-analysis
Fixed-effect and random-effects models in meta-analysisFixed-effect and random-effects models in meta-analysis
Fixed-effect and random-effects models in meta-analysis
Rizwan S A
 
Key informant interview
Key informant interviewKey informant interview
Population health management
Population health management Population health management
Importance of Data Analytics
 Importance of Data Analytics Importance of Data Analytics
Importance of Data Analytics
Product School
 
Meta analysis
Meta analysisMeta analysis
Meta analysis
Bhaswat Chakraborty
 
4. Formulating research problems
4. Formulating research problems4. Formulating research problems
4. Formulating research problems
Razif Shahril
 

What's hot (20)

Deep learning health care
Deep learning health care  Deep learning health care
Deep learning health care
 
Factor Analysis in Research
Factor Analysis in ResearchFactor Analysis in Research
Factor Analysis in Research
 
Hypothesis testing
Hypothesis testingHypothesis testing
Hypothesis testing
 
Overview of the systematic review process
Overview of the systematic review processOverview of the systematic review process
Overview of the systematic review process
 
How to calculate power in statistics
How to calculate power in statisticsHow to calculate power in statistics
How to calculate power in statistics
 
Using NVivo QSR Theory and Practice for Qualitative Data Analysis in a PhD
Using NVivo QSR Theory and Practice for Qualitative Data Analysis in a PhDUsing NVivo QSR Theory and Practice for Qualitative Data Analysis in a PhD
Using NVivo QSR Theory and Practice for Qualitative Data Analysis in a PhD
 
Prediction of cardiovascular disease with machine learning
Prediction of cardiovascular disease with machine learningPrediction of cardiovascular disease with machine learning
Prediction of cardiovascular disease with machine learning
 
Health Prediction System - an Artificial Intelligence Project 2015
Health Prediction System - an Artificial Intelligence Project 2015Health Prediction System - an Artificial Intelligence Project 2015
Health Prediction System - an Artificial Intelligence Project 2015
 
Regression shrinkage: better answers to causal questions
Regression shrinkage: better answers to causal questionsRegression shrinkage: better answers to causal questions
Regression shrinkage: better answers to causal questions
 
Qualitative Data Analysis (Steps)
Qualitative Data Analysis (Steps)Qualitative Data Analysis (Steps)
Qualitative Data Analysis (Steps)
 
SMART HEALTH PREDICTION USING DATA MINING by Dr.Mahboob Khan Phd
SMART HEALTH PREDICTION USING DATA MINING by Dr.Mahboob Khan PhdSMART HEALTH PREDICTION USING DATA MINING by Dr.Mahboob Khan Phd
SMART HEALTH PREDICTION USING DATA MINING by Dr.Mahboob Khan Phd
 
Introduction and Tools of Research
Introduction and Tools of ResearchIntroduction and Tools of Research
Introduction and Tools of Research
 
Data analysis
Data analysisData analysis
Data analysis
 
Clinical Decision Support Systems
Clinical Decision Support SystemsClinical Decision Support Systems
Clinical Decision Support Systems
 
Fixed-effect and random-effects models in meta-analysis
Fixed-effect and random-effects models in meta-analysisFixed-effect and random-effects models in meta-analysis
Fixed-effect and random-effects models in meta-analysis
 
Key informant interview
Key informant interviewKey informant interview
Key informant interview
 
Population health management
Population health management Population health management
Population health management
 
Importance of Data Analytics
 Importance of Data Analytics Importance of Data Analytics
Importance of Data Analytics
 
Meta analysis
Meta analysisMeta analysis
Meta analysis
 
4. Formulating research problems
4. Formulating research problems4. Formulating research problems
4. Formulating research problems
 

Similar to Academia

Academic Research Essay
Academic Research EssayAcademic Research Essay
Academic Research Essay
Buy Cheap Essay Omaha
 
Writing for publication… Some tentative ideas
Writing for publication… Some tentative ideasWriting for publication… Some tentative ideas
Writing for publication… Some tentative ideas
University of South Africa (Unisa)
 
AGU15 & Impact.tech talk
AGU15 & Impact.tech talkAGU15 & Impact.tech talk
AGU15 & Impact.tech talk
microryza
 
Due Jan 6Week 8Using Figure 1.2 in Ch. 1 of Exploring Resear.docx
Due Jan 6Week 8Using Figure 1.2 in Ch. 1 of Exploring Resear.docxDue Jan 6Week 8Using Figure 1.2 in Ch. 1 of Exploring Resear.docx
Due Jan 6Week 8Using Figure 1.2 in Ch. 1 of Exploring Resear.docx
jacksnathalie
 
NIH Neuroethics Meeting
NIH Neuroethics MeetingNIH Neuroethics Meeting
NIH Neuroethics Meeting
John C. Besley
 
Covering Scientific Research #SciCommLSU
Covering Scientific Research #SciCommLSUCovering Scientific Research #SciCommLSU
Covering Scientific Research #SciCommLSU
Paige Jarreau
 
Dismissive Reviews, Citation Cartels, and the Replication Crisis.pptx
Dismissive Reviews, Citation Cartels, and the Replication Crisis.pptxDismissive Reviews, Citation Cartels, and the Replication Crisis.pptx
Dismissive Reviews, Citation Cartels, and the Replication Crisis.pptx
Richard P Phelps
 
Critical-Thinking-Learn-Higher-1.pptx
Critical-Thinking-Learn-Higher-1.pptxCritical-Thinking-Learn-Higher-1.pptx
Critical-Thinking-Learn-Higher-1.pptx
MZIKRULLAH
 
Pedagogical research: getting published
Pedagogical research: getting publishedPedagogical research: getting published
Pedagogical research: getting published
Chris Willmott
 
Select an issue or event in wellness and explore it.pdf
Select an issue or event in wellness and explore it.pdfSelect an issue or event in wellness and explore it.pdf
Select an issue or event in wellness and explore it.pdf
study help
 
Select an issue or event in wellness and explore it.pdf
Select an issue or event in wellness and explore it.pdfSelect an issue or event in wellness and explore it.pdf
Select an issue or event in wellness and explore it.pdf
study help
 
Purpose, theory and policy for higher education
Purpose, theory and policy for higher educationPurpose, theory and policy for higher education
Purpose, theory and policy for higher education
George Roberts
 
Essay Topics On Religion.pdf
Essay Topics On Religion.pdfEssay Topics On Religion.pdf
Essay Topics On Religion.pdf
Rhonda Ramirez
 
Overview In this you will select an issue or.docx
Overview In this you will select an issue or.docxOverview In this you will select an issue or.docx
Overview In this you will select an issue or.docx
bkbk37
 
2022 - Fostering Strategic Science Communication related to Trust
2022 - Fostering Strategic Science Communication related to Trust2022 - Fostering Strategic Science Communication related to Trust
2022 - Fostering Strategic Science Communication related to Trust
John C. Besley
 
Lesson-1-3.pptx
Lesson-1-3.pptxLesson-1-3.pptx
Lesson-1-3.pptx
PerkzNilo
 
How to develop student creativity by robert j. sternberg, wendy melissa willi...
How to develop student creativity by robert j. sternberg, wendy melissa willi...How to develop student creativity by robert j. sternberg, wendy melissa willi...
How to develop student creativity by robert j. sternberg, wendy melissa willi...
TESDARegion5NATCAC
 
Letters to a young(er) scholar: On (alternatives in) publishing
Letters to a young(er) scholar:  On (alternatives in) publishingLetters to a young(er) scholar:  On (alternatives in) publishing
Letters to a young(er) scholar: On (alternatives in) publishing
University of South Africa (Unisa)
 
Narrative Literature Reviews
Narrative Literature ReviewsNarrative Literature Reviews
Narrative Literature Reviews
Melissa Dudas
 
High level management theory 陳茂盛--1031113
High level management theory   陳茂盛--1031113High level management theory   陳茂盛--1031113
High level management theory 陳茂盛--1031113
茂盛 陳
 

Similar to Academia (20)

Academic Research Essay
Academic Research EssayAcademic Research Essay
Academic Research Essay
 
Writing for publication… Some tentative ideas
Writing for publication… Some tentative ideasWriting for publication… Some tentative ideas
Writing for publication… Some tentative ideas
 
AGU15 & Impact.tech talk
AGU15 & Impact.tech talkAGU15 & Impact.tech talk
AGU15 & Impact.tech talk
 
Due Jan 6Week 8Using Figure 1.2 in Ch. 1 of Exploring Resear.docx
Due Jan 6Week 8Using Figure 1.2 in Ch. 1 of Exploring Resear.docxDue Jan 6Week 8Using Figure 1.2 in Ch. 1 of Exploring Resear.docx
Due Jan 6Week 8Using Figure 1.2 in Ch. 1 of Exploring Resear.docx
 
NIH Neuroethics Meeting
NIH Neuroethics MeetingNIH Neuroethics Meeting
NIH Neuroethics Meeting
 
Covering Scientific Research #SciCommLSU
Covering Scientific Research #SciCommLSUCovering Scientific Research #SciCommLSU
Covering Scientific Research #SciCommLSU
 
Dismissive Reviews, Citation Cartels, and the Replication Crisis.pptx
Dismissive Reviews, Citation Cartels, and the Replication Crisis.pptxDismissive Reviews, Citation Cartels, and the Replication Crisis.pptx
Dismissive Reviews, Citation Cartels, and the Replication Crisis.pptx
 
Critical-Thinking-Learn-Higher-1.pptx
Critical-Thinking-Learn-Higher-1.pptxCritical-Thinking-Learn-Higher-1.pptx
Critical-Thinking-Learn-Higher-1.pptx
 
Pedagogical research: getting published
Pedagogical research: getting publishedPedagogical research: getting published
Pedagogical research: getting published
 
Select an issue or event in wellness and explore it.pdf
Select an issue or event in wellness and explore it.pdfSelect an issue or event in wellness and explore it.pdf
Select an issue or event in wellness and explore it.pdf
 
Select an issue or event in wellness and explore it.pdf
Select an issue or event in wellness and explore it.pdfSelect an issue or event in wellness and explore it.pdf
Select an issue or event in wellness and explore it.pdf
 
Purpose, theory and policy for higher education
Purpose, theory and policy for higher educationPurpose, theory and policy for higher education
Purpose, theory and policy for higher education
 
Essay Topics On Religion.pdf
Essay Topics On Religion.pdfEssay Topics On Religion.pdf
Essay Topics On Religion.pdf
 
Overview In this you will select an issue or.docx
Overview In this you will select an issue or.docxOverview In this you will select an issue or.docx
Overview In this you will select an issue or.docx
 
2022 - Fostering Strategic Science Communication related to Trust
2022 - Fostering Strategic Science Communication related to Trust2022 - Fostering Strategic Science Communication related to Trust
2022 - Fostering Strategic Science Communication related to Trust
 
Lesson-1-3.pptx
Lesson-1-3.pptxLesson-1-3.pptx
Lesson-1-3.pptx
 
How to develop student creativity by robert j. sternberg, wendy melissa willi...
How to develop student creativity by robert j. sternberg, wendy melissa willi...How to develop student creativity by robert j. sternberg, wendy melissa willi...
How to develop student creativity by robert j. sternberg, wendy melissa willi...
 
Letters to a young(er) scholar: On (alternatives in) publishing
Letters to a young(er) scholar:  On (alternatives in) publishingLetters to a young(er) scholar:  On (alternatives in) publishing
Letters to a young(er) scholar: On (alternatives in) publishing
 
Narrative Literature Reviews
Narrative Literature ReviewsNarrative Literature Reviews
Narrative Literature Reviews
 
High level management theory 陳茂盛--1031113
High level management theory   陳茂盛--1031113High level management theory   陳茂盛--1031113
High level management theory 陳茂盛--1031113
 

Recently uploaded

C# Interview Questions PDF By ScholarHat.pdf
C# Interview Questions PDF By ScholarHat.pdfC# Interview Questions PDF By ScholarHat.pdf
C# Interview Questions PDF By ScholarHat.pdf
Scholarhat
 
Bài tập bộ trợ anh 7 I learn smart world kì 1 năm học 2022 2023 unit 1.doc
Bài tập bộ trợ anh 7 I learn smart world kì 1 năm học 2022 2023 unit 1.docBài tập bộ trợ anh 7 I learn smart world kì 1 năm học 2022 2023 unit 1.doc
Bài tập bộ trợ anh 7 I learn smart world kì 1 năm học 2022 2023 unit 1.doc
PhngThLmHnh
 
Edukasyong Pantahanan at Pangkabuhayan 1: Personal Hygiene
Edukasyong Pantahanan at  Pangkabuhayan 1: Personal HygieneEdukasyong Pantahanan at  Pangkabuhayan 1: Personal Hygiene
Edukasyong Pantahanan at Pangkabuhayan 1: Personal Hygiene
MJDuyan
 
SEQUNCES Lecture_Notes_Unit4_chapter11_sequence
SEQUNCES  Lecture_Notes_Unit4_chapter11_sequenceSEQUNCES  Lecture_Notes_Unit4_chapter11_sequence
SEQUNCES Lecture_Notes_Unit4_chapter11_sequence
Murugan Solaiyappan
 
How to Manage Shipping Connectors & Shipping Methods in Odoo 17
How to Manage Shipping Connectors & Shipping Methods in Odoo 17How to Manage Shipping Connectors & Shipping Methods in Odoo 17
How to Manage Shipping Connectors & Shipping Methods in Odoo 17
Celine George
 
formative Evaluation By Dr.Kshirsagar R.V
formative Evaluation By Dr.Kshirsagar R.Vformative Evaluation By Dr.Kshirsagar R.V
formative Evaluation By Dr.Kshirsagar R.V
DrRavindrakshirsagar1
 
CTD Punjab Police Past Papers MCQs PPSC PDF
CTD Punjab Police Past Papers MCQs PPSC PDFCTD Punjab Police Past Papers MCQs PPSC PDF
CTD Punjab Police Past Papers MCQs PPSC PDF
hammadmughal76316
 
How to Create a New Article in Knowledge App in Odoo 17
How to Create a New Article in Knowledge App in Odoo 17How to Create a New Article in Knowledge App in Odoo 17
How to Create a New Article in Knowledge App in Odoo 17
Celine George
 
How to Handle the Separate Discount Account on Invoice in Odoo 17
How to Handle the Separate Discount Account on Invoice in Odoo 17How to Handle the Separate Discount Account on Invoice in Odoo 17
How to Handle the Separate Discount Account on Invoice in Odoo 17
Celine George
 
How to Manage Early Receipt Printing in Odoo 17 POS
How to Manage Early Receipt Printing in Odoo 17 POSHow to Manage Early Receipt Printing in Odoo 17 POS
How to Manage Early Receipt Printing in Odoo 17 POS
Celine George
 
Imagination in Computer Science Research
Imagination in Computer Science ResearchImagination in Computer Science Research
Imagination in Computer Science Research
Abhik Roychoudhury
 
How to Create Sequence Numbers in Odoo 17
How to Create Sequence Numbers in Odoo 17How to Create Sequence Numbers in Odoo 17
How to Create Sequence Numbers in Odoo 17
Celine George
 
What is Rescue Session in Odoo 17 POS - Odoo 17 Slides
What is Rescue Session in Odoo 17 POS - Odoo 17 SlidesWhat is Rescue Session in Odoo 17 POS - Odoo 17 Slides
What is Rescue Session in Odoo 17 POS - Odoo 17 Slides
Celine George
 
Howe Writing Center - Orientation Summer 2024
Howe Writing Center - Orientation Summer 2024Howe Writing Center - Orientation Summer 2024
Howe Writing Center - Orientation Summer 2024
Elizabeth Walsh
 
Bedok NEWater Photostory - COM322 Assessment (Story 2)
Bedok NEWater Photostory - COM322 Assessment (Story 2)Bedok NEWater Photostory - COM322 Assessment (Story 2)
Bedok NEWater Photostory - COM322 Assessment (Story 2)
Liyana Rozaini
 
Kesadaran_Berbangsa_dan_Bernegara_Nasion.pptx
Kesadaran_Berbangsa_dan_Bernegara_Nasion.pptxKesadaran_Berbangsa_dan_Bernegara_Nasion.pptx
Kesadaran_Berbangsa_dan_Bernegara_Nasion.pptx
artenzmartenkai
 
modul ajar kelas x bahasa inggris 2024-2025
modul ajar kelas x bahasa inggris 2024-2025modul ajar kelas x bahasa inggris 2024-2025
modul ajar kelas x bahasa inggris 2024-2025
NurFitriah45
 
Configuring Single Sign-On (SSO) via Identity Management | MuleSoft Mysore Me...
Configuring Single Sign-On (SSO) via Identity Management | MuleSoft Mysore Me...Configuring Single Sign-On (SSO) via Identity Management | MuleSoft Mysore Me...
Configuring Single Sign-On (SSO) via Identity Management | MuleSoft Mysore Me...
MysoreMuleSoftMeetup
 
How to Add a Filter in the Odoo 17 - Odoo 17 Slides
How to Add a Filter in the Odoo 17 - Odoo 17 SlidesHow to Add a Filter in the Odoo 17 - Odoo 17 Slides
How to Add a Filter in the Odoo 17 - Odoo 17 Slides
Celine George
 
NC Public Schools Involved in NCDPI, Zipline Partnership
NC Public Schools Involved in NCDPI, Zipline PartnershipNC Public Schools Involved in NCDPI, Zipline Partnership
NC Public Schools Involved in NCDPI, Zipline Partnership
Mebane Rash
 

Recently uploaded (20)

C# Interview Questions PDF By ScholarHat.pdf
C# Interview Questions PDF By ScholarHat.pdfC# Interview Questions PDF By ScholarHat.pdf
C# Interview Questions PDF By ScholarHat.pdf
 
Bài tập bộ trợ anh 7 I learn smart world kì 1 năm học 2022 2023 unit 1.doc
Bài tập bộ trợ anh 7 I learn smart world kì 1 năm học 2022 2023 unit 1.docBài tập bộ trợ anh 7 I learn smart world kì 1 năm học 2022 2023 unit 1.doc
Bài tập bộ trợ anh 7 I learn smart world kì 1 năm học 2022 2023 unit 1.doc
 
Edukasyong Pantahanan at Pangkabuhayan 1: Personal Hygiene
Edukasyong Pantahanan at  Pangkabuhayan 1: Personal HygieneEdukasyong Pantahanan at  Pangkabuhayan 1: Personal Hygiene
Edukasyong Pantahanan at Pangkabuhayan 1: Personal Hygiene
 
SEQUNCES Lecture_Notes_Unit4_chapter11_sequence
SEQUNCES  Lecture_Notes_Unit4_chapter11_sequenceSEQUNCES  Lecture_Notes_Unit4_chapter11_sequence
SEQUNCES Lecture_Notes_Unit4_chapter11_sequence
 
How to Manage Shipping Connectors & Shipping Methods in Odoo 17
How to Manage Shipping Connectors & Shipping Methods in Odoo 17How to Manage Shipping Connectors & Shipping Methods in Odoo 17
How to Manage Shipping Connectors & Shipping Methods in Odoo 17
 
formative Evaluation By Dr.Kshirsagar R.V
formative Evaluation By Dr.Kshirsagar R.Vformative Evaluation By Dr.Kshirsagar R.V
formative Evaluation By Dr.Kshirsagar R.V
 
CTD Punjab Police Past Papers MCQs PPSC PDF
CTD Punjab Police Past Papers MCQs PPSC PDFCTD Punjab Police Past Papers MCQs PPSC PDF
CTD Punjab Police Past Papers MCQs PPSC PDF
 
How to Create a New Article in Knowledge App in Odoo 17
How to Create a New Article in Knowledge App in Odoo 17How to Create a New Article in Knowledge App in Odoo 17
How to Create a New Article in Knowledge App in Odoo 17
 
How to Handle the Separate Discount Account on Invoice in Odoo 17
How to Handle the Separate Discount Account on Invoice in Odoo 17How to Handle the Separate Discount Account on Invoice in Odoo 17
How to Handle the Separate Discount Account on Invoice in Odoo 17
 
How to Manage Early Receipt Printing in Odoo 17 POS
How to Manage Early Receipt Printing in Odoo 17 POSHow to Manage Early Receipt Printing in Odoo 17 POS
How to Manage Early Receipt Printing in Odoo 17 POS
 
Imagination in Computer Science Research
Imagination in Computer Science ResearchImagination in Computer Science Research
Imagination in Computer Science Research
 
How to Create Sequence Numbers in Odoo 17
How to Create Sequence Numbers in Odoo 17How to Create Sequence Numbers in Odoo 17
How to Create Sequence Numbers in Odoo 17
 
What is Rescue Session in Odoo 17 POS - Odoo 17 Slides
What is Rescue Session in Odoo 17 POS - Odoo 17 SlidesWhat is Rescue Session in Odoo 17 POS - Odoo 17 Slides
What is Rescue Session in Odoo 17 POS - Odoo 17 Slides
 
Howe Writing Center - Orientation Summer 2024
Howe Writing Center - Orientation Summer 2024Howe Writing Center - Orientation Summer 2024
Howe Writing Center - Orientation Summer 2024
 
Bedok NEWater Photostory - COM322 Assessment (Story 2)
Bedok NEWater Photostory - COM322 Assessment (Story 2)Bedok NEWater Photostory - COM322 Assessment (Story 2)
Bedok NEWater Photostory - COM322 Assessment (Story 2)
 
Kesadaran_Berbangsa_dan_Bernegara_Nasion.pptx
Kesadaran_Berbangsa_dan_Bernegara_Nasion.pptxKesadaran_Berbangsa_dan_Bernegara_Nasion.pptx
Kesadaran_Berbangsa_dan_Bernegara_Nasion.pptx
 
modul ajar kelas x bahasa inggris 2024-2025
modul ajar kelas x bahasa inggris 2024-2025modul ajar kelas x bahasa inggris 2024-2025
modul ajar kelas x bahasa inggris 2024-2025
 
Configuring Single Sign-On (SSO) via Identity Management | MuleSoft Mysore Me...
Configuring Single Sign-On (SSO) via Identity Management | MuleSoft Mysore Me...Configuring Single Sign-On (SSO) via Identity Management | MuleSoft Mysore Me...
Configuring Single Sign-On (SSO) via Identity Management | MuleSoft Mysore Me...
 
How to Add a Filter in the Odoo 17 - Odoo 17 Slides
How to Add a Filter in the Odoo 17 - Odoo 17 SlidesHow to Add a Filter in the Odoo 17 - Odoo 17 Slides
How to Add a Filter in the Odoo 17 - Odoo 17 Slides
 
NC Public Schools Involved in NCDPI, Zipline Partnership
NC Public Schools Involved in NCDPI, Zipline PartnershipNC Public Schools Involved in NCDPI, Zipline Partnership
NC Public Schools Involved in NCDPI, Zipline Partnership
 

Academia

  • 1. Academic incentives and science Gerry Carter
  • 4. 1.YES 2. Maybe- what’s the argument? 3. Huh, what are you talking about? 4. NO
  • 5. What is science? • Wikipedia: systematic enterprise that builds and organizes knowledge in the form of testable explanations and predictions • Dictionary: systematic knowledge of the physical world gained through observation and experimentation
  • 6. What is science? • Wikipedia: systematic enterprise that builds and organizes knowledge in the form of testable explanations and predictions • Dictionary: systematic knowledge of the physical world gained through observation and experimentation • the most rigorous way of gaining new knowledge about nature (whatever that may be), and the sum of that knowledge • When you are adhering to the highest standards of logic and evidence, you are thinking scientifically.(And when you’re not, you’re not.)* *Author: Sam Harris
  • 7. science • Rigor • Falsifiability • Repeatability • Measures of uncertainty
  • 8. What is academia? • Wikipedia: the internationally recognized establishment of professional scholars and students, usually centered around colleges and universities, who are engaged in higher education and research. • the environment or community concerned with the pursuit of research, education, and scholarship. • the “job” of being a scholar at a university and the community of those scholars
  • 9. What is academia? • Wikipedia: the internationally recognized establishment of professional scholars and students, usually centered around colleges and universities, who are engaged in higher education and research. • the environment or community concerned with the pursuit of research, education, and scholarship. • the “job” of being a scholar at a university • By “academia”, I don’t mean teaching (vs research). • And I don’t mean the humanities • I mean the aspects that affect scientific research
  • 10. How is scientific success different from academic success?
  • 11. What is success? Science = being correct, falsifiable, and repeatable Academia = having influence and impact, being hired, getting funding
  • 12. My argument 1. Incentive structures of science are philosophical. Examples: double-blind design, replication, and peer-review. They are typically designed to aid scientific goals. 2. Incentives structures in academia are economic. Examples: tenure, h-index, and journal impact factors. They are not designed to meet scientific goals. They emerge as byproducts of human goals and economic competition within and among social groups. (Economic competition exists between publishing companies, between individuals, between labs, between departments, between universities, and academia vs other state-funded institutions). 3. Therefore, many academic incentives  bad science. 4. Question: How can we design better academic incentive structures so that they serve the interests of science?
  • 13. My argument 1. Incentive structures of science are philosophical. Examples: double-blind design, replication, and peer-review. They are typically designed to aid scientific goals. 2. Incentives structures in academia are economic. Examples: tenure, h-index, and journal impact factors. They are not designed to meet scientific goals. They emerge as byproducts of human goals and economic competition within and among social groups. (Economic competition exists between publishing companies, between individuals, between labs, between departments, between universities, and academia vs other state-funded institutions). 3. Therefore, many academic incentives  bad science. 4. Question: How can we design better academic incentive structures so that they serve the interests of science?
  • 14. My argument 1. Incentive structures of science are philosophical. Examples: double-blind design, replication, and peer-review. They are typically designed to aid scientific goals. 2. Incentives structures in academia are economic. Examples: tenure, h-index, and journal impact factors. They are not designed to meet scientific goals. They emerge as byproducts of human goals and economic competition within and among social groups. (Economic competition exists between publishing companies, between individuals, between labs, between departments, between universities, and academia vs other state-funded institutions). 3. Therefore, many academic incentives  bad science. 4. Question: How can we design better academic incentive structures so that they serve the interests of science?
  • 15. My argument 1. Incentive structures of science are philosophical. Examples: double-blind design, replication, and peer-review. They are typically designed to aid scientific goals. 2. Incentives structures in academia are economic. Examples: tenure, h-index, and journal impact factors. They are not designed to meet scientific goals. They emerge as byproducts of human goals and economic competition within and among social groups. (Economic competition exists between publishing companies, between individuals, between labs, between departments, between universities, and academia vs other state-funded institutions). 3. Therefore, many academic incentives  bad science. 4. Question: How can we design better academic incentive structures so that they serve the interests of science?
  • 16. My argument 1. Incentive structures of science are philosophical. Examples: double-blind design, replication, and peer-review. They are typically designed to aid scientific goals. 2. Incentives structures in academia are economic. Examples: tenure, h-index, and journal impact factors. They are not designed to meet scientific goals. They emerge as byproducts of human goals and economic competition within and among social groups. (Economic competition exists between publishing companies, between individuals, between labs, between departments, between universities, and academia vs other state-funded institutions). 3. Therefore, many academic incentives  bad science. 4. Question: How can we design better academic incentive structures so that they serve the interests of science?
  • 17. Science and academia have many shared incentives To be a good scientist and a good academic… • Do good research • Make important discoveries • Publish good papers • Produce public knowledge • Teach science to students well
  • 18. As a reviewer, when do you reject a paper for not enough significance?
  • 19. As a reviewer, when do you reject a paper for not enough significance? What makes a paper good enough? Science: good enough to be useful evidence for readers Academia: good enough to be an accolade for the writer
  • 20. The academic filter in biology • 16,000 new US Biology PhD students/ year • Of those recruits, 63% stay until they graduate • Average time to degree = 7 years • Of those graduates, 70% get a postdoc • Of those postdocs, 30% do 1+ more postdoc • Average postdoc time = ? but more than 4 years • Within 6 years, 15% of postdocs get a tenure-track job • We have a 6-7% chance of getting the job we were trained for. Source: ascb.org “Where will a biology PhD take you”
  • 21. Mentors: old vs. young BUT… …”Not exactly the sort of approach likely to reap accolades today.” “I doubt that I would fare particularly well in today’s academic environment.”
  • 22. Divergent interests (1 of 2) To be a good scientist… To be a good academic… Be skeptical of your results “Sell” your results
  • 23. Divergent interests (1 of 2) To be a good scientist… To be a good academic… Be skeptical of your results “Sell” your results Interpret conclusions carefully Highlight/exaggerate importance
  • 24. Divergent interests (1 of 2) To be a good scientist… To be a good academic… Be skeptical of your results “Sell” your results Interpret conclusions carefully Highlight/exaggerate importance Publish negative results Publish “strategically”
  • 25. Divergent interests (1 of 2) To be a good scientist… To be a good academic… Be skeptical of your results “Sell” your results Interpret conclusions carefully Highlight/exaggerate importance Publish negative results Publish “strategically” Ignore social prestige Use impact factors to make writing decisions
  • 26. Divergent interests (1 of 2) To be a good scientist… To be a good academic… Be skeptical of your results “Sell” your results Interpret conclusions carefully Highlight/exaggerate importance Publish negative results Publish “strategically” Ignore social prestige Use impact factors to make writing decisions Challenge authority Cite authority. Make friends.
  • 27. Divergent interests (1 of 2) To be a good scientist… To be a good academic… Be skeptical of your results “Sell” your results Interpret conclusions carefully Highlight/exaggerate importance Publish negative results Publish “strategically” Ignore social prestige Use impact factors to make writing decisions Challenge authority Cite authority. Make friends. Replicate. Replicate. Replicate… if you must
  • 28. Divergent interests (1 of 2) To be a good scientist… To be a good academic… Be skeptical of your results “Sell” your results Interpret conclusions carefully Highlight/exaggerate importance Publish negative results Publish “strategically” Ignore social prestige Use impact factors to make writing decisions Challenge authority Cite authority. Make friends. Replicate. Replicate. Replicate… if you must Novel exciting results are less likely to be true. Double-check them. Publish novel exciting results before you get scooped.
  • 29. Divergent interests (2 of 2) To be a good scientist… To be a good academic… P=0.06 P=0.04 Masicampoa & Lalande 2012 Quart J Exp Psych
  • 30. Divergent interests (2 of 2) To be a good scientist… To be a good academic… P=0.06 P=0.04 Help others replicate and find flaws in your work Have a “territory”
  • 31. Divergent interests (2 of 2) To be a good scientist… To be a good academic… P=0.06 P=0.04 Help others replicate and find flaws in your work Have a “territory” Be clear Sound smart
  • 32. Divergent interests (2 of 2) To be a good scientist… To be a good academic… P=0.06 P=0.04 Help others replicate and find flaws in your work Have a “territory” Be clear Sound smart “Science is a marathon, not a sprint” Funded work must be “transformative”
  • 33. Divergent interests (2 of 2) To be a good scientist… To be a good academic… P=0.06 P=0.04 Help others replicate and find flaws in your work Have a “territory” Be clear Sound smart “Science is a marathon, not a sprint” Funded work must be “transformative” Natural history is crucial for biological understanding Natural history is “low impact”
  • 34. Divergent interests (2 of 2) To be a good scientist… To be a good academic… P=0.06 P=0.04 Help others replicate and find flaws in your work Have a “territory” Be clear Sound smart “Science is a marathon, not a sprint” Funded work must be “transformative” Natural history is crucial for biological understanding Natural history is “low impact” Long-term goals for important truths Short-term results with high impact
  • 35. Divergent interests (2 of 2) To be a good scientist… To be a good academic… P=0.06 P=0.04 Help others replicate and find flaws in your work Have a “territory” Be clear Sound smart “Science is a marathon, not a sprint” Funded work must be “transformative” Natural history is crucial for biological understanding Natural history is “low impact” Long-term goals for important truths Short-term results with high impact Build a consensus. Hot controversial topics are higher impact
  • 36. What is academic impact? • Influence on your field • Usually involves counting citations, regardless of whether you are being cited for being right or being wrong. • Impact is measured at the level of journals (impact factor) and authors (h-index) • But actual scientific progress made by articles, methods, hypotheses, theories, and often teams of researchers
  • 37. What is academic impact? • A fundamental problem: • Over time, scientific validity and importance of an article both increases and decreases. • Measures of total impact can only increase. • Papers that are influentially wrong have higher impact metrics than papers that are correct.
  • 38. A survey From Daniel Dennett: • (A) You solve the major problem of your choice so conclusively that there is nothing left to say (thanks to you, part of the field closes down forever, and you get a footnote in history). • (B) You write an article or book of such tantalizing perplexity and controversy that it stays on the required reading list for centuries to come. Which would you choose?
  • 39. Philosophers and Scientists From Daniel Dennett: • (A) You solve the major problem of your choice so conclusively that there is nothing left to say (thanks to you, part of the field closes down forever, and you get a footnote in history). • (B) You write an article or book of such tantalizing perplexity and controversy that it stays on the required reading list for centuries to come. Scientists choose A. Philosophers often choose B.
  • 40. Some of the most influential scholars said little that was correct
  • 41. Jacques Derrida “Brilliant and controversial, Jacques Derrida is one of the most important thinkers of our time, his considerable body of work having ineradicably altered the landscape of thought in the 20th and 21st centuries.” • Wikipedia: best known for developing a form of semiotic analysis known as deconstruction • one of the major figures associated with post-structuralism and postmodern philosophy • published more than 40 books, hundreds of essays • significant influence upon the humanities and social sciences, including—in addition to philosophy and literature—law, anthropology, historiography, linguistics, sociolinguistics, psychoanalysis, political theory, feminism, and gay and lesbian studies…. also influenced architecture (in the form of deconstructivism), music, art, and art critics. Derrida was said to "leave behind a legacy of himself as the 'originator' of deconstruction.”
  • 42. Deconstruction denies the possibility of a "pure presence": "the present or presence of sense to a full and primordial intuition".[13][14] It thus denies the possibility of essential or intrinsic and stable meaning and the unmediated access to "reality”. Dictionary: a method of critical analysis of philosophical and literary language that emphasizes the internal workings of language and conceptual systems, the relational quality of meaning, and the assumptions implicit in forms of expression.
  • 43. “Pluralistic ignorance” • no one actually believes, but everyone believes that everyone else believes The Emperor’s New Clothes
  • 44. The anti-academic culture of science Richard Feynman on his Nobel Prize: http://www.liveleak.com/view?i=e5a_1185599458
  • 45. How should we change academia?
  • 46. Towards a better academia… 1. Make publishing trivial. Make peer-review rigorous. 2. Encourage replication. 3. Decentralize funding, rather than centralized, trickle-down funding
  • 47. Publishing • Make publishing trivial. Make peer-review rigorous. Article-level metrics. • Get rid of impact factors, and then journals.
  • 48. Journals make money from nothing
  • 49. Journals make money from nothing
  • 51. Journals are brands In theory, the whole point of a brand is quality control. But in practice, brands manipulate consumers into seeing high quality regardless of actual quality.
  • 52. Journals are brands In theory, the whole point of a brand is quality control. But in practice, brands manipulate consumers into seeing high quality regardless of actual quality. Journal brands serve as third-party assessment, so that readers “can judge an article by its cover” and can outsource their evaluation of papers to 1 editor and 2+ reviewers. But this is better accomplished by a rating system, like we use for judging movies and books.
  • 53. Public can read Publishing Data Conclusions Paper Overhyped bullshitPicked up by media This is how it should work. “Experimental treatments affected gut biota (p=0.01) and were correlated with longevity (p=0.03)” “Gut microbiota is linked to longevity in male mice” “Ten reasons why your microbiome is the key to health and long life”
  • 54. Public can read Public can read Publishing Data Conclusions Paper Overhyped bullshitPicked up by media This is how it should work. But this is how it often works… SPIN Data Journal. University access only. Overhyped bullshit SPIN Slightly hyped paperWritten for high impact SPIN Journal demands brevity and certainty
  • 55. Publishing • Make publishing trivial. Make peer-review rigorous. Article-level metrics. • Get rid of journals, impact factors, and academic publishing as a business. • Central repository of open-access, peer-reviewed articles (and their reviews). • Magazines can then draw stories from those articles with varying degrees of rigor (Nature, New Scientist, Fox News). This is already basically how ArXive and PLOS ONE work.
  • 56. Publishing • Make publishing trivial. Make peer-review rigorous. Article-level metrics. • Get rid of journals, impact factors, and academic publishing as a business. • Central repository of open-access, peer-reviewed articles (and their reviews). Magazines can then draw stories from those articles with varying degrees of rigor (Nature, New Scientist, Fox News). • Publish negative results and datasets.
  • 57. Publishing • Make publishing trivial. Make peer-review rigorous. Article-level metrics. • Get rid of journals, impact factors, and academic publishing as a business. • Central repository of open-access, peer-reviewed articles (and their reviews). Magazines can then draw stories from those articles with varying degrees of rigor (Nature, New Scientist, Fox News). • Publish negative results and datasets. • Published peer reviews. Make them objective and open access. Focus on soundness of results, not subjective importance (which can be assessed over time). Have standardized peer review checklists.
  • 58. Replication • Encourage replication. • Open data. • Metrics to identify what is worth replicating.* • Publish replications.
  • 59. A replication crisis Social psychology Psychology Biomedicine
  • 60. Funding • Decentralize funding • Rather than centralized, trickle-down funding • One idea: most labs receive baseline funding. Some percentage of all funds must be invested in another lab. Bollen et al. 2014. From funding agencies to scientific agency: collective allocation of science funding as an alternative to peer review. EMBO Reports.
  • 61. Scientists spend about 40% of their time chasing grants (2007 U.S. Govt study) experimental physicist at Columbia University did the calculations: some grants have a net negative value: they do not pay for the time that applicants and peer reviewers spend on them. (Dr. No Money: The Broken Science Funding System. 2011. Scientific American.)
  • 62. Funding • Decentralize funding • Rather than centralized, trickle-down funding • Use peer-review-like “voting” for funding. • One idea: most labs receive baseline funding. 30% of all funds must be invested in another lab justified by merit or promise. . Bollen et al. 2014. From funding agencies to scientific agency: collective allocation of science funding as an alternative to peer review. EMBO Reports
  • 63. In summary • Academic incentives can erode good scientific practices. • We can change academic practices. However, there needs to be a cooperative consensus, because this is a public goods dilemma. Like a boycott, it’s only cost- effective, if everyone does it. • It would be useful to enact policy changes that offset the poor incentive structures of academia.
  • 64. What do you think? Thanks for the discussion.