By this point, most organizations have acquired at least one code or application scanning technology to incorporate into their software security program. Unfortunately, for many organizations the scanner represents the entirety of that so-called “program” and often the scanners are not used correctly or on a consistent basis. This presentation looks at the components of a comprehensive software security program, the role that automation plays in these programs and tools and techniques that can be used to help increase the value an organization receives from its application scanning activities. It starts by examining common traps organizations fall into where they fail to address coverage concerns – either breadth of scanning coverage across the application portfolio or depth of coverage issues where application scans do not provide sufficient insight into the security state of target applications. After discussing approaches to address these coverage issues, the presentation walks through metrics organizations can use to keep tabs on their scanning progress to better understand what is being scanned, how frequently and at what depth. The presentation also contains a demonstration of how freely available tools such as the open source ThreadFix application vulnerability management platform and the OWASP Zed Attack Proxy (ZAP) scanner can be combined to create a baseline scanning program for an organization and how this approach can be generalized to use any scanning technology.

1. Do
You
Have
a
Scanner
or
a
Scanning
Program?

2. About
Me
• Dan
Cornell
• Founder
and
CTO
of
Denim
Group
• So@ware
developer
by
background
(Java,
.NET,
etc)
• OWASP
San
Antonio
• 15
years
experience
in
so@ware
architecture,
development
and
security

3. • StaQc
or
Dynamic?
(Or
Both?)
• Desktop,
Enterprise
or
Cloud
– (Or
All
the
Above?)
3
Who
Has
Purchased
an
Automated
Scanner?

4. Who
Here
Is
Happy
With
Their
Scanner?
• Yes
• No
• Kind
Of
• Not
Sure
4

5. Why
or
Why
Not?
Why
or
Why
Not?
5

6. Successful
So@ware
Security
Programs
• Common
Goal
– Reduce
Risk
by…
• Reliably
CreaQng
Acceptably
Secure
So@ware
• Obligatory
“People,
Process,
Technology”
Reference
– Anybody
got
a
good
Sun
Tzu
quote?
– I’d
se^le
for
a
von
Clausewitz…
– Or
perhaps
we
need
to
look
at
Dalai
Lama
quotes
(topic
for
a
different
day)
• Common
AcQviQes
– ImplementaQon
must
be
Qed
to
the
specific
organizaQon
6

7. What
Part
Does
Scanning
Play?
• OpenSAMM
-­‐
Automated
scanning
is
part
of
both
the
“Security
TesQng”
and
“Code
Review”
Security
PracQces
within
the
VerificaQon
Business
FuncQon
– Dynamic
scanning
and
staQc
scanning,
respecQvely
• Common
starQng
point
for
many
organizaQons
embarking
on
so@ware
security
programs
– There
are
lots
of
commercial
and
freely
available
products
that
can
be
used
in
support
of
this
acQvity
RED
FLAG:
Q:
What
are
you
doing
for
so:ware
security?
A:
We
bought
[Vendor
Scanner
XYZ]
***
BEWARE
FOSTERING
A
CHECKBOX
CULTURE
***
7

8. Scanning
Program:
AnQ-­‐
Pa^erns
• “Dude
With
a
Scanner”
approach
– Can
also
be
implemented
as
the
“lady
with
a
scanner”
approach
• “SaaS
and
Forget”
approach
8

9. Scanner
Program
Metrics
• Breadth
• Depth
• Frequency

10. Is
Your
Scanner
Missing
Something?
• Breadth
“Misses”
– Inadequate
applicaQon
porholio
– ApplicaQons
not
being
scanned
• Depth
“Misses”
– IneffecQve
crawling
ignores
applicaQon
a^ack
surface
– False
negaQves
resulQng
in
ignorance
of
legiQmate
vulnerabiliQes
– Excessive
false
posiQves
causing
results
to
be
ignored
• Frequency
“Misses”
– ApplicaQons
not
being
scanned
o@en
enough
10

11. Security
TesQng:
Be^er
Pa^erns
• Breadth-­‐First
Scanning
– You
want
a
scanning
program,
not
a
scanner
• Deep
Assessment
of
CriQcal
ApplicaQons
– Automated
scanning,
manual
scan
review
and
assessment
• Understand
that
scanning
is
a
means
to
an
end
– Not
an
end
in
and
of
itself
– Start
of
vulnerability
management
11

12. What
Goes
Into
a
Good
Scanning
Program?
• Solid
Understanding
of
A^ack
Surface
• RealisQc
Concept
of
Scanner
EffecQveness
• Disciplined
History
of
Scanning
• PrioriQzed
TesQng
Efforts
12

13. What
Is
Your
So@ware
A^ack
Surface?
13
So@ware
You
Currently
Know
About
Why?
• Lots
of
value
flows
through
it
• Auditors
hassle
you
about
it
• Formal
SLAs
with
customers
menQon
it
• Bad
guys
found
it
and
caused
an
incident
(oops)
What?
• CriQcal
legacy
systems
• Notable
web
applicaQons

14. What
Is
Your
So@ware
A^ack
Surface?
14
Add
In
the
Rest
of
the
Web
ApplicaQons
You
Actually
Develop
and
Maintain
Why
Did
You
Miss
Them?
• Forgot
it
was
there
• Line
of
business
procured
through
non-­‐
standard
channels
• Picked
it
up
through
a
merger
/
acquisiQon
What?
• Line
of
business
applicaQons
• Event-­‐specific
applicaQons

15. What
Is
Your
So@ware
A^ack
Surface?
15
Add
In
the
So@ware
You
Bought
from
Somewhere
Why
Did
You
Miss
Them?
• Most
scanner
only
really
work
on
web
applicaQons
so
no
vendors
pester
you
about
your
non-­‐web
applicaQons
• Assume
the
applicaQon
vendor
is
handling
security
What?
• More
line
of
business
applicaQons
• Support
applicaQons
• Infrastructure
applicaQons

16. What
Is
Your
So@ware
A^ack
Surface?
16
MOBILE!
THE
CLOUD!
Why
Did
You
Miss
Them?
• Any
jerk
with
a
credit
card
and
the
ability
to
submit
an
expense
report
is
now
runs
their
own
private
procurement
office
What?
• Support
for
line
of
business
funcQons
• MarkeQng
and
promoQon

17. A^ack
Surface:
The
Security
Officer’s
Journey
• Two
Dimensions:
– PercepQon
of
So@ware
A^ack
Surface
– Insight
into
Exposed
Assets
17
PercepQon
Insight

18. • As
percepQon
of
the
problem
of
a^ack
surface
widens
the
scope
of
the
problem
increases
A^ack
Surface:
The
Security
Officer’s
Journey
18
PercepQon
Insight
Web
Applications

19. • As
percepQon
of
the
problem
of
a^ack
surface
widens
the
scope
of
the
problem
increases
A^ack
Surface:
The
Security
Officer’s
Journey
19
PercepQon
Insight
Web
Applications
Client-Server
Applications

20. • As
percepQon
of
the
problem
of
a^ack
surface
widens
the
scope
of
the
problem
increases
A^ack
Surface:
The
Security
Officer’s
Journey
20
PercepQon
Insight
Web
Applications
Client-Server
Applications
Desktop
Applications

21. • As
percepQon
of
the
problem
of
a^ack
surface
widens
the
scope
of
the
problem
increases
A^ack
Surface:
The
Security
Officer’s
Journey
21
PercepQon
Insight
Web
Applications
Client-Server
Applications
Desktop
Applications
Cloud
Applications
and Services

22. • As
percepQon
of
the
problem
of
a^ack
surface
widens
the
scope
of
the
problem
increases
A^ack
Surface:
The
Security
Officer’s
Journey
22
PercepQon
Insight
Web
Applications
Client-Server
Applications
Desktop
Applications
Cloud
Applications
and Services
Mobile
Applications

23. • Discovery
acQviQes
increase
insight
A^ack
Surface:
The
Security
Officer’s
Journey
23
PercepQon
Insight
Web
Applications

24. • Discovery
acQviQes
increase
insight
A^ack
Surface:
The
Security
Officer’s
Journey
24
PercepQon
Insight
Web
Applications

25. • Discovery
acQviQes
increase
insight
A^ack
Surface:
The
Security
Officer’s
Journey
25
PercepQon
Insight
Web
Applications

26. • Over
Qme
you
end
up
with
a
progression
A^ack
Surface:
The
Security
Officer’s
Journey
26
PercepQon
Insight
Web
Applications

27. • Over
Qme
you
end
up
with
a
progression
A^ack
Surface:
The
Security
Officer’s
Journey
27
PercepQon
Insight
Web
Applications
Client-Server
Applications

28. Desktop
Applications
Client-Server
Applications
• Over
Qme
you
end
up
with
a
progression
A^ack
Surface:
The
Security
Officer’s
Journey
28
PercepQon
Insight
Web
Applications

29. Desktop
Applications
Client-Server
Applications
• Over
Qme
you
end
up
with
a
progression
A^ack
Surface:
The
Security
Officer’s
Journey
29
PercepQon
Insight
Web
Applications
Cloud
Applications
and Services

30. Desktop
Applications
Client-Server
Applications
• Over
Qme
you
end
up
with
a
progression
A^ack
Surface:
The
Security
Officer’s
Journey
30
PercepQon
Insight
Web
Applications
Cloud
Applications
and Services
Mobile
Applications

31. • When
you
reach
this
point
it
is
called
“enlightenment”
• You
won’t
reach
this
point
A^ack
Surface:
The
Security
Officer’s
Journey
31
PercepQon
Insight
Web
Applications
Client-Server
Applications
Desktop
Applications
Cloud
Applications
and Services
Mobile
Applications

32. An Application
Test
What
Goes
Into
An
ApplicaQon
Test?
32

33. Dynamic
Analysis
What
Goes
Into
An
ApplicaQon
Test?
33
Static
Analysis

34. Automated
Application
Scanning
What
Goes
Into
An
ApplicaQon
Test?
34
Static
Analysis
Manual
Application
Testing

35. Automated
Application
Scanning
What
Goes
Into
An
ApplicaQon
Test?
35
Automated
Static
Analysis
Manual
Application
Testing
Manual
Static
Analysis

36. Unauthenticated
AutomatedScan
What
Goes
Into
An
ApplicaQon
Test?
36
Automated
Static
Analysis
Blind
Penetration
Testing
Manual
Static
Analysis
Authenticated
AutomatedScan
Informed
ManualTesting

37. Unauthenticated
AutomatedScan
What
Goes
Into
An
ApplicaQon
Test?
37
Automated
SourceCode
Scanning
Blind
Penetration
Testing
ManualSource
CodeReview
Authenticated
AutomatedScan
Informed
ManualTesting
Automated
BinaryAnalysis
ManualBinary
Analysis

38. Value
and
Risk
Are
Not
Equally
Distributed
• Some
ApplicaQons
Ma^er
More
Than
Others
– Value
and
character
of
data
being
managed
– Value
of
the
transacQons
being
processed
– Cost
of
downQme
and
breaches
• Therefore
All
ApplicaQons
Should
Not
Be
Treated
the
Same
– Allocate
different
levels
of
resources
to
assurance
– Select
different
assurance
acQviQes
– Also
must
o@en
address
compliance
and
regulatory
requirements
38

39. Do
Not
Treat
All
ApplicaQons
the
Same
• Allocate
Different
Levels
of
Resources
to
Assurance
• Select
Different
Assurance
AcQviQes
• Also
Must
O@en
Address
Compliance
and
Regulatory
Requirements
39

40. • Free
/
Open
Source
vulnerability
management
and
aggregaUon
plaVorm:
– Allows
so@ware
security
teams
to
reduce
the
Qme
to
remediate
so@ware
vulnerabiliQes
– Enables
managers to speak intelligently about the status / trends of software security within their
organization.
• Features/Benefits:
– Imports
dynamic,
staQc
and
manual
tesQng
results
into
a
centralized
plahorm
– Removes
duplicate
findings
across
tesQng
plahorms
to
provide
a
prioriQzed
list
of
security
faults
– Eases
communicaQon
across
development,
security
and
QA
teams
– Exports
prioriQzed
list
into
defect
tracker
of
choice
to
streamline
so@ware
remediaQon
efforts
– Auto
generates
web
applicaQon
firewall
rules
to
protect
data
during
vulnerability
remediaQon
– Empowers
managers
with
vulnerability
trending
reports
to
pinpoint
team
issues
and
illustrate
applicaQon
security
progress
– Benchmark
security
pracQce
improvement
against
industry
standards
• Freely
available
under
the
Mozilla
Public
License
(MPL)
2.0
• Download
available
at:
www.denimgroup.com/threadfix
• Code
available
at:
h^ps://code.google.com/p/threadfix/
40
The
ThreadFix
Approach

41. ThreadFix
DemonstraQon
• Building
Your
ApplicaQon
Porholio
• Storing
Scanning
Results
Over
Time
• ReporQng
– Trending
– Vulnerability
RemediaQon
Progress
– Scanner
Benchmarking
– Porholio
Status
41

42. • Build
Your
ApplicaQon
Porholio
• Characterize
the
EffecQveness
of
Efforts
Made
to
Date
• Build
a
Plan
for
Coverage
• Monitor
Progress
42
Steps
for
Improvement

43. 43
Dan
Cornell
Principal
and
CTO
dan@denimgroup.com
Twi^er
@danielcornell
+1
(210)
572-­‐4400
www.denimgroup.com
blog.denimgroup.com
QuesQons?