Security Testing for Testing Professionals

TE
AM Tutorial
4/30/13 8:30AM

Security Testing for Testing
Professionals
Presented by:
Jeff Payne
Coveros, Inc.

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

Jeff Payne
Jeff Payne is CEO and founder of Coveros, Inc., a software company that builds secure software
applications using agile methods. Since its inception in 2008, Coveros has become a market leader in
secure agile principles and has been recognized by Inc. magazine as one of the fastest growing private
US companies. Prior to founding Coveros, Jeff was chairman of the board, CEO, and cofounder of Cigital,
Inc., a market leader in software security consulting. Jeff has published more than thirty papers on
software development and testing, and testified before Congress on issues of national importance,
including intellectual property rights, cyberterrorism, and software quality.

9/5/2012

Security Testing for Test
Professionals

© Copyright 2011 Coveros, Inc.. All rights reserved.

1

About Coveros

Coveros helps organizations accelerate the delivery of secure, reliable
software
Our consulting services:
–
–
–
–

Agile software development
Application security
Software quality assurance
Software process improvement

Corporate Partners

Our key markets:
–
–
–
–

Financial services
Healthcare
Defense
Critical Infrastructure


2

1

9/5/2012

Agenda
Introduction to Security Testing

Security Testing Framework
Appropriate Security Testing Tools
Wrap up


3

Trainer

Jeffery Payne
Jeffery Payne is CEO and founder of Coveros, Inc., a software company that
helps organizations accelerate the delivery of secure, reliable software. Coveros
uses agile development methods and a proven software assurance framework to
build security and quality into software from the ground up. Prior to founding
Coveros, Jeffery was Chairman of the Board, CEO, and co-founder of Cigital, Inc.
Under his direction, Cigital became a leader in software security and software
quality solutions, helping clients mitigate the risk of software failure. Jeffery is a
recognized software expert and popular speaker at both business and technology
conferences on a variety of software quality, security, and agile development
topics. He has also testified before Congress on issues of national importance,
including intellectual property rights, cyber-terrorism, Software research funding,
and software quality.


4

2

9/5/2012

Expectations
What are your expectations for this tutorial?
What do you wish to learn?
What questions do you want answered?


5

Introduction to Security Testing


6

3

9/5/2012

What is Information Security?

When you hear the term “Information Security” and
“Security Testing”:
What do you think they mean?
What comes to mine?


7

What is Information Security?
Definition of Information Security
Information Security means protecting information and
information systems from unauthorized access, use,
disclosure, disruption, modification, perusal, inspection,
recording or destruction.
The key concepts of Information Security include:
–
–
–
–
–

Confidentiality
Integrity
Availability
Authenticity
Non-Repudiation


8

4

9/5/2012

The Software Security Problem
Our IT systems are not castles any longer!


9

Understanding Risk
How to Define Security Risk in Software
Common Security Nomenclature
– Risk: a possible future event which, if it occurs, will lead to an
undesirable outcome
– Threat: A potential cause of an undesirable outcome
– Vulnerability: Any weakness, administrative process, or act of
physical exposure that makes an information asset susceptible to
exploit by a threat.
– An exploit is a piece of software, a chunk of data, or sequence of
commands that takes advantage of a vulnerability in order to cause
unintended or unanticipated behavior to occur on computer
software, hardware, or something electronic.


10

5

9/5/2012

Security Testing
What? How?
Security Testing is testing used to determine whether an
information system protects its data from its threats.
Security Testing is not a silver bullet for your enterprise
security. Security Testing doesn’t fix your security, it only
makes you aware of it. Security must be built into your
software
A sound Security Testing process performs testing
activities:
–
–
–
–
–

Before development begins
During requirements definition and software design
During implementation
During deployment
During maintenance and operations


11

Security Testing Framework


12

6

9/5/2012

Security testing before development begins
Overview
“Testing” before development begins is really a QA function
to assess the readiness of the organization to build secure
software applications.
Always remember that security testing evaluates the
security posture of your applications, it does not build
security in.
Irrespective of your findings, do not become the “quality
police”.


13

Review Security Policies and Standards
Understand the policies and standards that have been
adopted by the organization and their relationship to
software security
Examples:
–
–
–
–

Privacy policies regarding your customer data
Service level agreements with clients
IT security standards you must adhere to
PCI compliance activities for credit card transactions

Your goal is to understand these policies and standards to
the level that will allow you to validate security requirements
and effectively test the end product against them

14

7

9/5/2012

Review Secure Software Development Lifecycle
If the security of your software is an enterprise concern, the
development team should be adhering to a defined secure
software development lifecycle model.
– Defines development activities that builds security in
– Defines security testing activities performed by appropriate parties
(development, testing, security org, operations, etc.)

Common secure software development models
– Microsoft’s Secure Development Lifecycle (SDL)
– Coveros SecureAgile process
– There are others as well

Secure software standards
– Secure coding standard

15

Security testing during definition and design
Overview
Testing activities during requirements definition and
software design focus on assuring that security has been
effectively integrated into software requirements and the
overall architecture and design of the product
Typical activities include:
–
–
–
–

Security requirements development/validation
Architecture and design reviews
Threat modeling
Test strategy and planning


16

8

9/5/2012

What is a Security Requirement?
Security Requirements describe functional and nonfunctional requirements that need to be satisfied in order to
achieve the security attributes of an IT system or
application.

What does that mean?
Security Requirements are formulated at different levels of
abstraction and provide how the system should act, and
what should not happen.
The major difference you may notice is the use of
“negative requirements”. We stop thinking about what
the application should do and start thinking about how
we can get it to do something it was never intended to.


17

Functional Requirements
Your Standard Definition
Functional Requirements: These are statements of
services the system should provide, how the system should
react to particular inputs and how the system should behave
in particular situations. In some cases, the functional
requirements may also have explicitly state what the system
should not do.

Where does the Security fit in?
Each Functional Security Requirement utilizes uses case
and misuse cases.
These requirements reflect potential threats to the system.


18

9

9/5/2012

Exercise
Functional Security Requirements
Break into teams of 2-3 people.
Each team will identify potential misuse cases with the
following security requirement, if any exist.
If a misuse case is identified, write a replacement or
additional functional requirement(s).
– It would be best to make sure no misuse cases can be derived from
your new requirement(s).

Exercise Time Limit: 15 Minutes


19

Exercise
Functional Security Requirement
SecureChat Authentication Requirements
– When a user attempts to authenticate with a valid username and an
invalid password, the application shall not authenticate the user and
return them to the authentication page.
– The system must alert the user that their attempt to authenticate has
failed due to an incorrect password (“Invalid Password”) utilizing the
standard error text formatting.
– When a user attempts to authenticate with a invalid username, the
application shall not authenticate the user and return them to the
authentication page.
– The system must alert the user that their attempt to authenticate has
failed due to an incorrect username (“Invalid Username”) utilizing
the standard error text formatting.
– What a user attempts to authenticate using a username and a valid
password, the application shall authenticate the user and redirect
them to the homepage.


20

10

9/5/2012

Exercise
Functional Security Requirements Discussion
How could an attacker attempt to thwart the system?
What are the core information security concepts we should
be concerned with?
What issues exist with the current requirements?
How would you fix the current requirements?


21

Exercise
Formal Authentication Use/Misuse Case Artifact
Enter username
and password

Threatens

SecureChat
User

User
authentication

Brute Force
Attack

Mitigates
Show generic
error message

Guess User
Accounts
Mitigates

Lock account
after N failed
login attempts

SecureChat
Server

Hacker

Mitigates
Dictionary
Attacks
Mitigates

Validate
password
minimum
length and
complexity

22

11

9/5/2012

Non-Functional Requirements
Your Standard Definition
Non-Functional Requirements: These are constraints on
the services or functions offered by the system. They
include timing constraints, constraints on the development
process and standards. Non-functional requirements often
apply to the system as a whole. They do not usually just
apply to individual system features or services.

Where does the Security fit in?
These are security related architectural requirements, like
"robustness" or "minimal performance and scalability". This
requirement type is typically derived from architectural
principals and good practice standards.
They could also be required activities during development
like “coding guidelines”, “data classification” and “test
methodology”.

23

Non-Functional Requirements
Examples
SecureChat shall ensure that data is protected from
unauthorized access at all times.
SecureChat shall have an availability of 99.9%.
SecureChat shall process a minimum of 8 transactions per
second.
Each SecureChat build must undergo static-analysis prior to
release.
All external communications between the application and
the SecureChat central servers must be encrypted.

24

12

9/5/2012

Architectural and Design Reviews
Architectural and design reviews focus on determining
whether the stated architecture / design enforces the
appropriate level of security as defined in the requirements.
Typically performed by security architects and/or other
software leads within the organization.
Examines these artifacts for flaws such as:
– Violation of trust boundaries
– Distributed control of authorization
– Custom algorithms for cryptography / random number generation


25

Assessing your risk – Answers the ‘so what?’ question
Identifying threats and flaws in your design only result in
better security if the flaws are mitigated to minimize the
threat.
But at what cost to the organization?
What benefit?
How do you convince management to fund mitigation
efforts?

26

13

9/5/2012

Threat modeling / risk assessment
Threat modeling – a process by which any threats to a
piece of software are identified and mitigated
A variety of approaches exist for doing threat modeling
Microsoft STRIDE model
–
–
–
–

Diagram your system – high level dataflow diagrams
Identify threats – each type of entity/interaction has enemies
Mitigate threats – determine security controls
Validate mitigations – test effectiveness of these controls

The basis for comprehensive security testing

27

Risk Assessments
Information on architectural flaws and known threats from
our threat model are often combined together to estimate
the likelihood and consequence of a flaw resulting in
significant business impact
Highly Likely

Likely

Unlikely

Business-critical

High priority

Priority

Priority

Business concern

High priority

Priority

Not a Priority

Minor or cosmetic

Not a Priority

Not a Priority

Not a Priority


28

14

9/5/2012

Exercise
Risk Assessment
Your company, SecureTelco, has developed an instant
messaging program to be used for private use in customers
homes and for companies and government agencies.
SecureChat requires users to sign up with an account prior
to using the system. After authenticating with a username
and password, each user can message other users and
expect their conversations to be private.
Users have the ability to add/remove friends from their
contact list, search for friends based on their email, block
users from IMing them, become “invisible” to all users on
demand.
Messages archives and activities logs document user
behavior and can be retrieved by the user or a SecreTelco
Administrator through the application or by the
administrative console, respectively.

29

Exercise
Risk Assessment
Highly Likely

Likely

Unlikely

Business-critical

High priority

Priority

Priority

Business concern

High priority

Priority

Not a Priority

Minor or cosmetic

Not a Priority

Not a Priority

Not a Priority


30

15

9/5/2012

Exercise
Risk Assessment Discussion
What threats exist for this application?
What features, if implemented incorrectly, provide a threat
with an opportunity and how big would it be?
What would be your business justification for correcting
these issues?


31

Test strategy and planning
Overview
A security test strategy should include the inputs to the
individual test plans and test plans that include:
– A specification of what types of tests should be performed,
– A specification of which what point in the delivery process should
they be performed,
– What the expected output of each test plan is.

The extent of your strategy obviously depends on the scale
of the project you're running, but much of this content is
reusable in smaller more focused projects or in more agile
security engineering programs.


32

16

9/5/2012

Developing a security test plan
What should be included?
Describe and detail your process and procedures for
security testing
–
–
–
–

When should testing begin?
How are test results reported?
Who validates and verifies findings/results?
When are vulnerabilities addressed?

Types of tests you should include in your test plan:
–
–
–
–

Security Feature Testing
Risk Based Testing of functional and non-functional requirements
Internal Penetration Tests
External (Independent) Penetration Tests


33

Integrating security requirements in test plans
Know your Security Requirements
It is important that each tester understand the security
requirements for your application and what they imply.
Often Security requirements may come in conflict with another type
of requirement. If there are conflicts, it is important that you identify
those concerns and the requirements are clarified by a Business
Analyst.

In most organizations, security requirements are not well
defined if it all.
A general rule of thumb: Make sure your core information security
concepts are all covered. If not, request that they are.

Understand which security requirements are functional and
which are non-functional, this will have an impact how you
plan to test them.

34

17

9/5/2012

Integrating security requirements in test plans
Testing Security Requirements
Feature testing covers positive security requirements. This
typically ensures the software behaves according to
customer expectations.
Example – If security requirements state that the length of any user
input must be validated, then a feature test suite should be created
to exercise the application inputs and verify that this requirement is
implemented correctly.

Testers should also cover negative security testing or RiskDriven testing. Each test is intended to probe for a specific
risk or vulnerability. These risk may have been identified
during your risk assessment.
Example – Cross Site Scripting and SQL Injection;
These vulnerabilities are not obviously features of the
application, therefore the fall under the negative security
requirements umbrella.

35

Security testing during implementation
Overview
Testing activities during implementation focus on assuring
that the software is implemented properly according to its
requirements and design
Key activities during implementation include:
– Secure code review – identifying security vulnerabilities in source
code
– Testing individual components/features for security
– Testing requirements at the appropriate level


36

18

9/5/2012

Secure code review
Secure code review identifies vulnerabilities within source
code that potentially impact system security.
Examples
– Buffer overflows
– Race conditions

Secure code review is a combination of manual and
automated analysis
– Tool providers: HP Fortify, IBM Ounce, Coverity

Secure code review is typically done by developers or a
dedicated security team

37

Testing components and features
The testing of components and individual features will
identify code that improperly implements functionality
against its requirements.
While some feature testing has historically been done at the
system level, more and more of this type of testing today is
done on individual units / stories by either a developer or
code savvy test engineer.
Review of tests performed at this level should look for
common gaps that lead to security issues:
– Inadequate testing of error handling routines
– Insufficient protection during system reboot
– Forgetting to test administrative capabilities


38

19

9/5/2012

Testing common security controls
Due to the security-critical nature of many of our
applications, it is common to see the following security
controls implemented within our software.
Each must be validated in order to work!
Authentication & Access Control
Input Validation & Encoding
Encryption
User and Session Management
Error and Exception Handling
Auditing and Logging


39

Common Approaches to Authentication
All About Authentication
When we refer to authentication in computer security, we
refer to the process of attempting to verify the digital identity
of the sender of a communication.
– A common example of such a process it the login process.
– Authentication always depends upon using one or more
authentication factors.

Testing authentication schemas means understanding how
the process works and using that information to circumvent
the authentication mechanism.


40

20

9/5/2012

Authentication Testing Schemas
Credentials transport over an encrypted channel
– The tester must try to understand if the data inputted by the user is
transmitted using secure protocols that protect them from an
attacker or not.

Testing for user enumeration
– The tester must verify if it is possible to collect a set of valid users
by interacting with the authentication mechanism of the application.
This will become useful for brute force testing.

Testing for guessable (dictionary) user accounts
– The tester must validate that there are no default user accounts or
guessable username/password combinations

Brute force testing
– When dictionary attacks don’t succeed, the tester can attempt brute
force methods to gain access. This is not often easy to accomplish
because of time constraints.

41

Authentication Testing Schemas (cont.)
Testing for bypassing authentication schema
– The tester must validate that other application resources are
adequately protected, and can’t be used to bypass authentication
using those other resources.

Testing for vulnerable remember password and password
reset features
– The tester must analyze how the application manages the process
of “password resets”. The tester must all check whether the
application allows the user to store passwords in the browser.

Testing for logout and browser cache management
– The tester must check that the logout and caching functions are
properly implemented.


42

21

9/5/2012

Testing for CAPTCHA
– Used by many applications to ensure the response is not generated
by a computer, CAPTCHA (“Completely Automated Public Trust test
to tell Computers and Humans Apart”) implementations are often
vulnerable to various kinds of attacks.

Testing multiple factor authentication
– The tester must test the following scenarios:
One Time Password Generator Tokens
Crypto devices like USB tokens or smart cards
X.509 Certificates
Random OTP sent via SMS

Testing for race conditions
– The tester must ensure that an unexpected result on a multithread
application doesn’t create an authentication flaw. By their nature,
Race Conditions are difficult to test for


43

Testing for session management schema
– The tester must test the security of a session tokens issues to the
client browser:
How to reverse engineer a cookie
How to manipulate cookies to hijack a session

Testing for cookie attributes
– The tester must check if an application can take the necessary
precautions when assigning cookies and test the cookie attributes.

Testing for session fixation
– The tester must validate that an application renews the cookie after
a successful user authentication, so that an attacker could not utilize
a session fixation vulnerability.


44

22

9/5/2012

Testing for exposed session variables
– The tester must validate that it is not possible to create a replay
session attack utilizing exposed session information.

Testing for CSRF (Cross Site Request Forgery)
– The tester must ensure that there is not a way to force an
unknowing user to execute unwanted actions on a web application
they are authenticated on.


45

Risk-based Testing
Risk-based Testing focuses on testing that the risks
identified during threat modeling and design reviews were
properly mitigated in the code
Define negative tests that validate these issues have been
mitigated.
Perform these tests at whatever level is appropriate to
identify any remaining vulnerabilities.


46

23

9/5/2012

Integration and Systems Testing
Testing non-functional security requirements that span
features within the system
Includes Web Application Security testing of any web-based
interfaces
Often includes Penetration Testing type activities to “test
like a hacker”
–
–
–
–

Fuzzing
Password crackers
Network port scanners
Dynamic input strings

47

Security testing during deployment
Overview
Testing during the deployment process focuses on those
tests that cannot be adequately completed within a
development/QA environment plus any third party IV&V
– Penetration testing
– Load and performance testing (for availability)
– Configuration testing

Penetration testing is typically done by a team of security
experts


48

24

9/5/2012

Security testing during maintenance / support
Overview
Testing during maintenance and support focuses on:
– Assuring that any identified vulnerabilities within the application,
supporting software, or network configuration are patched and
revalidated


49


50

Supporting tools

25

9/5/2012

Tools to Support Security Testing
Secure Code Scanners
Where to use?
– Analyze source code for inherent vulnerabilities and/or compliance
with secure coding standards

Free Tools
–
–
–
–

FxCop
FindBugs
RATS
clang

Paid Tools
– HP Fortify
– IBM Ounce
– Coverity


51

Web Application Scanners
Where to use?
– Looking for XSS, Injection and input validation vulnerabilities; some
tools will attempt to actively exploit vulnerabilities.

Free Tools
–
–
–
–
–

Nikto
W3af
Paros
Skipfish
wfuzz

Paid Tools
– Netsparker
– WebSecurify


52

26

9/5/2012

Password Crackers/Brute Force Tools
Where to use?
– When you want to break the default credentials or test your
authentication mechanisms against common security tools.

Free Tools
– THC Hydra
– Cain and Abel
– Wfuzz

Paid Tools
– John the Ripper


53

Network Security Tools
Where to use?
– Scanning for mis-configurations
– Testing for OS, application and network vulnerabilities

Free Tools
– OpenVAS

Paid Tools
– Nessus
– Core Impact


54

27

9/5/2012

Questions?
Contact Information:
Jeffery Payne
Jeff.payne@coveros.com
703.431.2920

55

28

Security Testing for Testing Professionals

More Related Content

What's hot

Viewers also liked

Similar to Security Testing for Testing Professionals

More from TechWell

Recently uploaded

Security Testing for Testing Professionals