Hunting Bugs - Running a Bug Bounty Program Slide deck put together for https://c0r0n4con.com/ (April/2020) Abstract Security best practices can't guarantee that a system or app is 100% free of bugs, so we have to detect bugs and vulnerabilities before the bad guys were able to explore them. Join us to hear the experience (takeaways and challenges) running a Bug Bounty program in a financial institution, bringing together the hacker community, AppSec and CSIRT teams. Outline We have been heard a lot about Bug Bounty (BB) programs and how security researchers has been making a lot of money by reporting bugs. Too much has been said from the researchers’ perspective, and too little from the companies running a BB program. An evolution of the endless Responsible Disclosure discussions and Vulnerability Report Programs, Bug Bounty programs have become a new trend in the information security industry, providing a valid communication channel for external entities to report the existence of bugs and vulnerabilities in a company’s platform and services. In this talk we will discuss the experience of a Brazilian Bank on implementing a bug bounty program as part of its application security strategy. From the point of view of an organization running a bug bounty program, we will present the challenges and benefits (expected and unexpected ones). BB program also brings the researcher community closer to the industry, representing a relevant initiative to strengthen relationship with the information security community. Running a BB program also demands strong commitment on having the issues fixed and also improves the information security visibility across the organization. Since a vulnerability can represent severe losses, detection and response are critical for the business. In order to have a proper response time for reported vulnerabilities , the CSIRT and AppSec teams play a key role in the BB program, by leading the remediation efforts whenever necessary.

1. Hunting bugs ?!
How to run a Bug Bounty Program and not being fired!
Anchises Moraes | Cyber Evangelist
@anchisesbr
@C6Bank @GaroaHC @BSidesSP
@CSAbr @LWomcy

2. AGENDA
The value of a Bug Bounty
program
Challenges and
Lessons learned @ C6 Bank

3. Hunting bugs ?!

4. It is not just about the money !
Hunting bugs ?!

5. WTF IS BUG BOUNTY ?

6. www
Public
Know
Researchers
Research Verification Triage
Correction
plan
Bounty
Renediation
Validate report:
scope, severity
Test and validate
the bug
Review
severity,
correction effort
Notification Portal
Reports &
researchers
management (profile,
reputarion)
How works
Hunting bugs ?!

7. Different approaches to help finding bugs
Code
Review
Code
Security
Tests
Vulnerability
analysis
Pentest
Red
Team
Threat
Hunting
Vulnerability
Disclosure
BUG
BOUNTY
ProductionDEV, QA
Hunting bugs ?!

8. Notification channel
BENEFITS TO
ORGNIZATIONS
Source: www.justice.gov
Hunting bugs ?!

9. Notification channel
Diversity
BENEFITS TO
ORGNIZATIONS
• Study: “An Empirical Study of
Vulnerability Rewards
Programs” (Berkeley)
• An increase in the number of
researchers looking for vulnerabilities
yields an increase in the diversity of
vulnerabilities discovered.
Hunting bugs ?!

10. Notification channel
Diversity
Cost x Benefit
BENEFITS TO
ORGNIZATIONS
• Study: “An Empirical Study of Vulnerability
Rewards Programs” (Berkeley)
• A Vulnerability Research Program (VRP)
can be a cost-effective mechanism for
finding security vulnerabilities.
• The cost of (…) VRPs is comparable to
the cost of just one member of the
browser security team.
• Each of these VRPs finds many more
vulnerabilities than any one researcher
is likely to be able to find.
https://mfinifter.github.io/papers/vrps-usenix2013.pdf
Hunting bugs ?!

11. CHALLENGES

12. “PEOPLE /
CRIMINALS WILL
TEST THE SECURITY
OF OUR
APPLICATION”
THEY ALREADY DO
THAT !
USUAL OBJECTIONS (Before BB)
Hunting bugs ?!

13. “WHAT IF THEY FIND
BUGS?”
WE WILL HAVE TO
FIX THEM (ASAP)
BEFORE BAD GUYS
ALSO FIND THEM.
USUAL OBJECTIONS (Before BB)
Hunting bugs ?!

14. USUAL OBJECTIONS (Before BB)
“HOW TO PREDICT
THE IMPACT (#
BUGS) AND COST OF
THE BB PROGRAM?”
I DON’T KNOW
L
Hunting bugs ?!

15. Classificação:
Interna
SOME CHALLENGES
(Running BB)
Hunting bugs ?!

16. Classificação:
Interna
volatility
Total reports
Valid reports
Hunting bugs ?!

17. Classificação:
Interna
false positives
Total reports
Valid reports
Hunting bugs ?!

18. TO BUILD AND TO
FOLLOW A
RESPONSE PROCESS
Manage response times
(SLA)
• Response to researcher &
reward
• Internal proccess for
vulnerability fix
Hunting bugs ?!

19. GEOGRAPHY
CHALLENGE
Local business
X
Global BB program
Hunting bugs ?!

20. LESSONS LEARNED

21. https://medium.com/@C6bank/c6-bank-lan%C3%A7a-programa-de-bug-bounty-1f419ebec9f2
LESSONS LEARNED
BB Program launched in
May 2019
Private program
(invite only)
C6 Bank to launch “bug bounty” program
Hunting bugs ?!

22. RED TEAM GOT
UPSET
BB program exposing
bugs they couldn’t find
before!
Hunting bugs ?!

23. “Testing only proves
the presence of bugs,
not the absence of
them.”
Louis Cremen
https://hackernoon.com/introducing-the-infosec-colour-wheel-blending-developers-with-red-and-blue-security-teams-6437c1a07700
Hunting bugs ?!

24. NO ONE LIKES
DUPLICATED REPORTS
Too bad for researchers
(frustration) and for the
company (noise)
• Study: “Diversity or Concentration?
Hackers’ Strategy for Working Across
Multiple Bug Bounty Programs” (MIT)
• Sorting the few new valid issues from
the deafening noise of submitted trivial
bugs, non-issues, duplicates is time
consuming and can drain resources.
• (…) invalid and duplicate submissions
greatly out-number valid submissions
for open programs.
Hunting bugs ?!

25. INTERNAL METRICS TO
MEASURE THE BUG IMPACT
& COST
Bounty Table / Rewards (Best case) (US$)
Company
Crítical
High
Medium
Low
Robinhood
$ 50.000
many
many
$ 100
Coinbase
$ 50.000
$ 15.000
$ 2.000
$ 200
Paypal
$ 20.000
$ 10.000
$ 1.000
$ 100
Goldman Sachs
$ 15.000
many
many
$ 250
QIWI
$ 5.000
$ 5.000
$ 500
$ 200
Credit Karma
$ 5.000
$ 2.250
$ 700
$ 250
Savedroid
$ 2.500
$ 1.500
$ 500
$ 250
Plaid
$ 2.500
$ 1.000
$ 500
$ 250
RecargaPay
$ 2.000
$ 750
$ 300
$ 150
Omise
$ 800
$ 400
$ 200
$ 100
Liberapay
$ 500
$ 300
$ 50
$ -
Source:
Hackerone
Hunting bugs ?!

26. MY FAVORITES <3
BB improves the capability
to detect vulnerabilities
Represents a strong
security posture
Help building a good
relationship with infosec
community
FIXING A BUG IN PRODUCTION
Hunting bugs ?!

27. THANK YOU, GRACIAS !

28. Anchises Moraes | Cyber Evangelist
@anchisesbr @c6bank
https://www.c6bank.com.br

29. Classificação:
Interna
REFERENCES AND SUGGESTED READING
M. Finifter, D. Akhawe and D. Wagner, “An Empirical Study of Vulnerability Rewards
Programs” (Berkeley), 2013, https://mfinifter.github.io/papers/vrps-usenix2013.pdf.
K. Huang, M. Siegel, S. Madnick, X. LI and Z. Feng, “Diversity or Concentration?
Hackers’ Strategy for Working Across Multiple Bug Bounty Programs” (MIT), Dec. 2016.
R. Ellis, K. Huang, M. Siegel, K. Moussouris and J. Houghton, “Fixing a Hole: The Labor
Market for Bugs” (MIT), 2017.
Hunting bugs ?!