The document discusses the advantages of replacing traditional penetration testing with bug bounty programs, highlighting key differences in testers, coverage, and the payment model. Bug bounty programs involve a larger and more diverse pool of independent researchers for ongoing security assessments, while penetration tests are typically time-limited and fixed-cost. A case study from Canvas illustrates the effectiveness of bug bounties in identifying critical vulnerabilities and the growing trend towards crowdsourced security.

1. 3 REASONS TO SWAP YOUR
NEXT PEN TEST WITH A BUG
BOUNTY PROGRAM

2. Jason Haddix, Head of Trust and Security Wade Billings, VP of Technology Services
2
YOUR SPEAKERS

3. AGENDA
• Key differences between bug bounties and penetration testing
• Definitions
• Testers
• Coverage
• Model
• Canvas by Instructure Case Study
• Q&A
3
DOWNLOAD OUR REPORT ‘HEAD TO HEAD:
BUG BOUNTIES VS. PENETRATION TESTING”
https://bugcrowd.com/penetration-testing

4. WHAT IS PENETRATION TESTING?
A penetration is…
• A time-boxed, fixed-cost assessment
• External consultants try to find as many vulnerabilities and config issues as possible and exploiting
those vulnerabilities to determine the risks
A penetration is NOT…
• A red team assessment
4

5. | CONFINDENTIAL INFORMATION
WHAT IS A BUG BOUNTY?
3/14/175
Independent security
researchers from all
over the world are
recruited
Vulnerabilities
are found and
reported
Rewards are
exchanged for reporting
vulnerabilities in
company applications

6. PENETRATION TESTING VS. BUG BOUNTIES: KEY DIFFERENCES
6

7. TESTERS:
MANY VS. FEW
Not only is the testing pool much
larger, but it is also more diverse,
providing organizations with a broad
set of skills and expertise.
7

8. COVERAGE:
ONGOING VS.
POINT-IN-TIME
Security assessment should be
continuous, especially as
development processes become
more agile. Penetration testing
can’t offer that coverage. Bug
bounties can.
8

9. MANY WAYS TO USE BUG BOUNTY PROGRAMS
9
Start with invite only
private program to
gain experience
Deliver ongoing security assurance with
continuous private and/or public program
Project or app specific On-Demand
Start with invite only
private program to
gain experience
Expand scope to increase
value & researcher
engagement

10. MODEL:
PAY-FOR-RESULTS
VS. CONTRACT-
BASED
Bug bounties utilize a pay-for-results
model that encourages deeper and
more focused testing. Higher severity
bugs carry a bigger incentive.
10

11. 11
CASE STUDY

12. SECURITY AT CANVAS
• Published security notices
• Extensive security testing
• Open security audits since 2011
• Working with independent researchers

13. RESULTS: SIX YEARS OF PUBLIC SECURITY AUDITS
13
0
10
20
30
40
Average pen test findings 2011 - 2013 Average bug bounty findings 2014 - 2016
Non-critical vulnerabilities High-critical vulnerabilities

14. KEY LEARNINGS: MORE THAN JUST THE RESULTS
14

15. FUTURE OF BUG BOUNTIES…

16. | CONFINDENTIAL INFORMATION
WIDE ADOPTION OF CROWDSOURCED SECURITY
3/14/1716
FINANCIAL SERVICES CONSUMER TECH RETAIL & ECOMMERE AUTOMOTIVE
INFRASTRUCTURE TECH SECURITY TECHNOLOGY OTHER
2/3rd of
Programs are
Private

17. Q&A