This document summarizes research into understanding the heterogeneity of contributors in bug bounty programs. Through quantitative analysis of 2,504 contributors across 82 programs and qualitative analysis via surveys, the researchers identified two main types of active contributors: project-specific contributors who work on specific programs for days finding bugs due to interest in the organizations/products, and non-specific contributors who spend at most 6 hours on a project.

1. Understanding the Heterogeneity of
Contributors in Bug Bounty Programs
Hideaki Hata (NAIST)
Mingus Guo, M. Ali Babar (University of Adelaide)
ESEM 2017

2. Bug is money
2

3. ● Checks or check-like
certificates awarded by
Donald Knuth for finding
mathematical errors or
making suggestions for his publication
● MIT Technology Review describes as
“among computerdom’s most
prized trophies”
Knuth Reward Check
Google Image Search “Knuth reward check”
3

4. Black marketGray market
Trade between sellers and
government agencies or other non-
criminal clients
The NSA devoted $25.1 million in
2013 to purchase software
vulnerabilities
M. Fidler, Regulating the zero-day trade: A preliminary analysis,
I/S: A J. of Law and Policy for the Inf. Society, vol. 11, December 2015.
Johannes Bader, hacker_two, CC BY 2.0
4

5. A rewards program offered by
an organization to external
parties, authorizing them to
perform security
assessments on the
organization’s assets.
Economically efficient
compared to the cost of hiring
full-time security researchers
[*].
Bug Bounty Programs (White Markets)
Esben Friis-Jensen, The History of Bug Bounty Programs, Cobalt, 2014.
5
[*] M. Finifter, D. Akhawe, and D. Wagner, An empirical study of vulnerability rewards programs,
USENIX Conf. on Security, 2013.

6. Research Question
What kind of contributors
are there in bug bounty
programs?
6

7. Approach
Quantitative analysis
Collect 2,504 distinct
contributors from 82 programs,
then apply archetypal analysis
Qualitative analysis
Obtain 7 answers from a survey
7

8. Quantitative Analysis
8
82 programs 2,504 contributors and activities

9. An unsupervised learning
method similar to clustering
Individual data points are
described based on the
distance from extreme points
(archetypes)
Archetypal Analysis
Example
Manuel J. A. Eugster, Archetypal Analysis - Mining of the Extreme, HIIT seminar, 2012.
9

10. Less active Project-specific Non-specific
10

11. Qualitative Analysis: A Survey
Less active
Project-specific
Non-specific
4 answers
3 answers
11

12. 12
Project-specific
Non-specific

13. Other: A few hours of my time, but days or weeks of computer time
13
Non-specific
Project-specific

14. Project-specific contributors work
on programs because of the
organizations and products, and
tend to spend several days finding
and reporting a bug, although non-
specific contributors spend at
most 6 hours on a project.
14

15. Summary
We found two different types of
active contributors in bug bounty
programs.
15

16. Beyond Onion Models
Bug bounty program contributors
are almost outside of traditional
software development onion models
[+]. However, talented contributors
are highly required and have a
significant impact on software
development.
Like_the_Grand_Canyon, Red onion, CC BY 2.0
16
[+] . Nakakoji et al., Evolution patterns of open-source
software systems and communities, IWPSE, 2002.