presentation from MBEES 2014

1. Author's gender aects rating of academic articles
Imagine a scientist...

2. Author's gender aects rating of academic articles
Imagine a scientist...
source: Steinke et al. (2007),Assessing media inuences on middle schoolaged children's perceptions
of women in science using the draw-a-scientist test (DAST), Science Communication 29 (1).

3. Author's gender aects rating of academic articles
Imagine a scientist...
source: Steinke et al. (2007),Assessing media inuences on middle schoolaged children's perceptions
of women in science using the draw-a-scientist test (DAST), Science Communication 29 (1).

4. Author's gender aects rating of academic articles
Imagine a scientist...
`Male scientist' stereotype

5. Author's gender aects rating of academic articles
Imagine a scientist...
`Male better scientist' stereotype

6. Author's gender aects rating of academic articles
Author's gender aects rating of academic articles
Evidence from an incentivized, deception-free laboratory experiment
Magdalena Smyk
Michaª Krawczyk
GendEQU project
GRAPE UW
University of Warsaw
June 2, 2014

7. Author's gender aects rating of academic articles
Experiment design and procedures - how?
The experiment

8. Author's gender aects rating of academic articles
Experiment design and procedures - how?
Treatments
First step: articles
10 papers
each pair - written by mixed-gender couple
each pair - one paper published in top economic journal and one working
paper
erased names
two by two design:
a female economist (in Polish: ekonomistka)
a male economist (in Polish: ekonomista)
a young female economist
a young male economist

9. Author's gender aects rating of academic articles
Experiment design and procedures - how?
What was the experiment about?
Task:
Read the entire article and answer on a scale 0-7 how you evaluate:
author's competence
language quality
methodology
literature review
scientic signicance
scientic quality
intelligibility
In your opinion was this paper published in one of the top economic
journals or was not published at all?
Reward
80 PLN (20EUR 25USD)
additional 10 PLN - if answer to the question about publication was correct.

10. Author's gender aects rating of academic articles
Experiment design and procedures - how?
What we expected?
Hypothesis 1
Female-authored paper will less often be judged as published and receive lower
ratings than male-authored.
Hypothesis 2
Papers written by young economists will less often be judged as published.
Intuition
The gender bias will be greater for the question of whether the paper was
published (incentivized and indirect question).

11. Author's gender aects rating of academic articles
Experiment design and procedures - how?
Design
Sessions
10 sessions conducted at the Laboratory of Experimental Economics,
University of Warsaw (Poland)
pen-and-paper
45 minutes per session
193 subjects

12. Author's gender aects rating of academic articles
Experiment design and procedures - how?
Design
Sessions
10 sessions conducted at the Laboratory of Experimental Economics,
University of Warsaw (Poland)
pen-and-paper
45 minutes per session
193 subjects
Subjects
familiar with economics and procient in English
60% - women
2/3 students (majority: economics masters)

13. Author's gender aects rating of academic articles
Results
Results

14. Author's gender aects rating of academic articles
Results
Female-authored papers - less often judged as published.

15. Author's gender aects rating of academic articles
Results
Authors' young age - insignicant.

16. Author's gender aects rating of academic articles
Results
No gender bias for ratings...
male female Two-sided MWW
author author test
average average (p-value)
author's competence 5.15 4.99 0.17
language quality 4.98 4.92 0.79
proper methodology used 4.83 4.77 0.62
comprehensiveness of literature review 4.76 4.66 0.73
scientic signicance of results 4.08 4.17 0.76
overall scientic quality 4.70 4.56 0.29

17. Author's gender aects rating of academic articles
Results
No gender bias for ratings...
male female Two-sided MWW
author author test
average average (p-value)
author's competence 5.15 4.99 0.17
language quality 4.98 4.92 0.79
proper methodology used 4.83 4.77 0.62
comprehensiveness of literature review 4.76 4.66 0.73
scientic signicance of results 4.08 4.17 0.76
overall scientic quality 4.70 4.56 0.29
...yet strong positive correlation between ratings and judged published
author's language methodology literature scientic scientic
competence quality review signicance quality
judged 0.4767*** 0.3691*** 0.4152*** 0.3968*** 0.4121*** 0.4975***
published

18. Author's gender aects rating of academic articles
Results
Probit model: subjects characteristics
Table : Paper judged as published: probit regressions
(1) (2) (3) (4)
female author - 0.353* - 0.350* - 0.370* - 0.462*
young author 0.146 0.147 0.133 0.143
female subject - 0.561*** - 0.428
year of study 0.00341 - 0.000533
not a student - 0.290 - 0.315
female author*male subject 0.255
paper eects included NO YES YES YES
observations 193 193 173 173
pseudo R2 0.017 0.039 0.0759 0.0774
*** p0.01, ** p0.05, * p0.1

19. Author's gender aects rating of academic articles
Conclusions and discussion
Conclusions
The gender of the author matters when it comes to the evaluation of a
paper (at least in the eld of economics and between Polish students)...

20. Author's gender aects rating of academic articles
Conclusions and discussion
Conclusions
The gender of the author matters when it comes to the evaluation of a
paper (at least in the eld of economics and between Polish students)...
but seemingly not author's age.

21. Author's gender aects rating of academic articles
Conclusions and discussion
Conclusions
The gender of the author matters when it comes to the evaluation of a
paper (at least in the eld of economics and between Polish students)...
but seemingly not author's age.
Why those results are important? - because of our subjects
the time when important early career decisions are made
inexperienced researchers may easily be discouraged.
young women fail to see successful female economists that could serve as
role models.

22. Author's gender aects rating of academic articles
Conclusions and discussion
Thank you for your attention!
Magdalena Smyk
msmyk@wne.uw.edu.pl