The document analyzes the appeal and effectiveness of donation badges in influencing developer responses to Eclipse bug reports. Despite a significant percentage of developers showing support for open-source initiatives, the implementation of badges did not significantly improve response times and was ultimately deemed ineffective, leading to their removal in 2016. The study highlights the complex interaction between monetary factors and software development processes, using statistical methods to assess causal relationships.

1. Are Donation Badges Appealing?:
A Case Study of Developer Responses
to Eclipse Bug Reports
Keitaro Nakasai, Hideaki Hata, and Kenichi Matsumoto
IEEE Software ( Volume: 36 , Issue: 3 , May-June 2019 ), ESEC/FSE 2019 Journal First

3. Now you can
donate to Eclipse
Dec 2007
special mirror site
‘Friend of Eclipse’ logo
Bug 282088 -
Expand Friends of
Eclipse Program
Jul 2009
Linux Foundation offered
various benefits
New Friend of Eclipse
Program Launched
Oct 2014
T-shirt, discount on
EclipseCons, O'Reilly
discount, etc.
Bug 514954 - Make
"Friends of Eclipse"
more appealing
Apr 2017
Bug 434249 - Add
decorator for
Friends of Eclipse
Nov 2014
Badges implemented
Bug 496899 - Remove
Friends mirror
Jun 2016
The friends mirror is no longer
needed

4. Bug 514954 - Make
"Friends of Eclipse"
more appealing
Apr 2017
I want to give back since Eclipse technology is free 84.44%
I want to support open source 71.85%
I want a t-shirt/other benefits 5.93%
I want to help improve Eclipse technology 72.59%
I feel a sense of closeness to a community or group 9.63%
Giving people more value as a Friend has proved not to make a
difference over the past 3 years and our various campaigns. That is
why we removed it in September 2016.
We ran a survey in June 2016 and over 84% of our respondents we
non committers and most of them (80%) have been using Eclipse for
over 5 years. What they said was:
We no longer promote the Friend status, but kept it at 35$ in the
background and FAQs.

6. A friend of Eclipse badge
(Donation badge)

7. Research Question
Are Donation Badges
Appealing?
Do donation badges affect developer
response time?

8. Difference in Differences
One of the core methods in today’s
econometrics toolkit to address causal
questions.
Regression analysis estimating the effect of
sharp changes in the economic environment
or changes in government policy.

9. DID Model
Trend in the control group
Trend in the treatment group
Counterfactual trend
in the treatment group
Treatment effect
Quantile DID

10. Bug 434249 - Add
decorator for
Friends of Eclipse
Nov 2014
Badges implemented
Nov 13, 2014
Before (2 years) After (2 years)

11. Nov 13, 2014
Before (2 years) After (2 years)
At least one bug report in both periods
957
31

12. Nov 13, 2014
Before (2 years) After (2 years)
# of months worked in Bugzilla
# of bug report submissions in the periods of before and after
# of commits in Git repositories
957
31
31
Propensity Score Matching

13. Nov 13, 2014
Before (2 years) After (2 years)
31
31
617
453 245
507
Metrics for bug reports: Severity, OS, Component, Community, Social relationship

14. Metric Description Coeffs P-value
Donor Reporter is a donor or not 2.259 0.007
Period Submitted time is before or after -1.598 0.193
Badge It has a donation badge or not -2.219 0.037
Enhancement Severity is enhancement or not 0.668 0.412
Windows Issue is related to Windows or not 1.119 0.106
Linux Issue is related to Linux or not 0.675 0.477
MacOS Issue is related to MacOS or not 0.793 0.382
Component Response days in median for the belonging components 0.317 0.109
Community # of contributors in the belonging components 0.000 0.917
Time A numerical order of time in months 0.152 0.193
Relationship # of reports in which the reporter and the first responder have worked together 0.005 0.371

15. Metric Description Coeffs P-value
Donor Reporter is a donor or not 2.259 0.007
Period Submitted time is before or after -1.598 0.193
Badge It has a donation badge or not -2.219 0.037
Enhancement Severity is enhancement or not 0.668 0.412
Windows Issue is related to Windows or not 1.119 0.106
Linux Issue is related to Linux or not 0.675 0.477
MacOS Issue is related to MacOS or not 0.793 0.382
Component Response days in median for the belonging components 0.317 0.109
Community # of contributors in the belonging components 0.000 0.917
Time A numerical order of time in months 0.152 0.193
Relationship # of reports in which the reporter and the first responder have worked together 0.005 0.371
Donation badges decrease developer response time 2 hours (median).
The median of developer response time in 1,822 studied bug reports is 3.5 hours.

16. Implications
Donation badges can work as
signals, similar to badges in GitHub
[Trockman et al., ICSE 2018].
Monetary factors can affect software
development processes.
Statistical causal inference can be a
useful toolkit for supporting decision
making.

17. Limitations
Threats to internal validity: there can
be unknown factors.
The badge effect is not eternal.
Designing and maintaining the
system is needed.
DID cannot work in other conditions.
Other methods: [Zhao et al., ASE
2017], [Murphy-Hill et al., ICSE 2019].