This document discusses security testing approaches for the OWASP Top 10 vulnerabilities. It argues that superficial security tests that only examine the user interface provide an illusion of knowledge and security. To fully understand risks, tests need to examine the entire application stack, including the backend, and consider vulnerabilities like security misconfiguration, missing access controls, and use of vulnerable components. Examples are given showing how vulnerabilities can remain undetected if the full scope of the application is not tested, including the code, configurations, dependencies and infrastructure. A holistic approach to security testing that incorporates reverse engineering is advocated to have a realistic understanding of risks.

1. Security Testing & The Depth
Behind OWASP Top 10
Yaniv Simsolo, CISSP
Image: Hubble Telescope: The cat’s eye nebula

2. OWASP Top 10 2013
OWASP Top 10 – 2013 has evolved:
• 2013-A1 – Injection
• 2013-A2 – Broken Authentication and Session
Management
• 2013-A3 – Cross Site Scripting (XSS)
• 2013-A4 – Insecure Direct Object References
• 2013-A5 – Security Misconfiguration
• 2013-A6 – Sensitive Data Exposure
• 2013-A7 – Missing Function Level Access Control
• 2013-A8 – Cross-Site Request Forgery (CSRF)
• 2013-A9 – Using Known Vulnerable Components (NEW)
• 2013-A10 – Unvalidated Redirects and Forwards

3. OWASP Top 10 2013
OWASP Top 10 – 2013 Resources:
• https://www.owasp.org/index.php/Top_10_2013-
Top_10
• OWASP Top 10 2013 presentation by Dave Wichers,
on the OWASP web site

4. Mapping Top 10: From 2010 to 2013
Source: OWASP Top 10 2013 presentation by Dave Wichers

5. Assumptions
• In Information Security – several top 10 exist
– OWASP Top 10 is dominant
• “Top 3”: we all know about XSS’s Injections,
CSRF’s etc.
• Most organizations are well aware of these
issues

6. Assumptions
• OK. What now?
• “Top 6” = (“Top 3”) + (“we test what we can”):
– Broken authentication and session management
– Unvalidated redirects and forwards
– Insecure direct object references
• Most organizations are aware of these issues
• OK, What now?

7. What did we miss?
• Security misconfiguration – A5.
• Missing Function Level access control – A7.
• Using known vulnerable components – A9
• A6 – sensitive data exposure now includes a
merge of:
– Insufficient transport layer protection (2010 – A9)
– Insecure cryptographic storage (2010-A7)

8. What did we miss?
• Security misconfiguration – A5.
– (almost) not Web Application but:
Application/system
• Missing Function Level access control – A7.
– Partial Web Application, Partial
Application/system
• Using known vulnerable components – A9
– (almost) not Web Application but:
Application/syste

9. What did we miss?
• A6 – sensitive data exposure now includes a
merge of:
– Insufficient transport layer protection (2010 – A9)
– Insecure cryptographic storage (2010-A7)
• Is this just Web Application?
• Is the problem more severe once we look
below the Web Layer?

10. What did we miss? Example
Security misconfiguration – A5
+
Using known vulnerable components – A9
=
Perimeter is not working

11. The Problem
Image: Hubble Telescope: The cat’s eye nebula

12. Over Complexity
• Too much data
• Endless attack possibilities
• Too many security solutions, vendors,
products
• No homogenous approach

13. The Attack Vectors
– Any system
– Any infrastructure
– Any communication
– Any language
– Any architecture
– Any component
– Any information, any
data
– Any physical layer
– Any logical layer
– Any storage device /
facility
– Any (communication)
channel
– Any interface
– Any encryption
– Any environment
– Any site (including DR)
– Any transaction
– Any log and audit trail
– Any archive
– Any process (operations,
ongoing, development)

14. The Attack Types
Takeover
Data theft
– Any system
– Any infrastructure
– Any communication
– Any language
– Any architecture
– Any component
– Any information, any
data
– Any physical layer
– Any logical layer
– Any storage device /
facility
– Any (communication)
channel
– Any interface
– Any encryption
– Any environment
– Any site (including DR)
– Any transaction
– Any log and audit trail
– Any archive
– Any process (operations,
ongoing, development)
Data tampering
System integrity disruption
Business Logic manipulation
Eavesdropping
Backdoors – built in by design
Backdoors – creation by attackers
Unintentional attacks
Intentional by authorized entities
Attacks by non-human entities
Denial of Service
De Facto Denial of Service
Authorization bypass
Access bypass
Smuggling, Splitting and evasion-type attacks

15. The Problem
Even the simplified security areas present a
demanding challenge. For example - XSS:
• Very difficult to detect all variants in modern
systems
• Almost impossible to retain high security level
once achieved

16. Common Solutions
• Superficial security tests.
– Many “good reasons”:
• Budget
• Time constraints
• Lack of understanding
• Over complexity

17. Common Solutions
• Impacts of superficial security tests in the
long run?
– Partial to no security
– Poor security practices
– These organizations effect the security
market, pulling downwards!
– Loss or partial integrity of security
professionals
–Worse still: false sense of security

18. Where Did That Got Us?
• Ludicrous security warnings:
– January 2013: Department of Homeland Security:
Do not use Java. Remove the JRE.
– April 2014: Department of Homeland Security:
Versions 6 – 11 of IE are not to be used.
– April 2014: OpenSSL is insecure

19. Where Did That Got Us?
• Poor security in design and architecture
• (Almost) no security in Agile/Continuous
Delivery developed code

20. Modern Systems Common Pitfall
• Modern systems are more secured. ???
20

21. Where Did That Got Us?
• Challenging security presentations:
– In-Depth Security is dead (RSA conference 2011)
– Security is dead (Rugged coding - RSA conference
2012)
• Ignorance is bliss….

22. Security Testing
Image: Hubble Telescope: The cat’s eye nebula

23. How to Test?
• This is messy. VERY messy.
• There are shortcuts

24. How to Test?
• Actually – most is quiet easy to test.
• Go back to theory.
• Forget about the payloads.

25. The Fallback Common Option
• Test the GUI
• Black Box testing methodology
• Exclude the difficult stuff from scope
• This is a “good” solution: it fits organizations
and security professionals

26. The Fallback Common Option
• “The greatest enemy of knowledge is not
ignorance, it is the illusion of knowledge.”
― Stephen Hawking
• Testing just the GUI illusion of knowledge
• Testing just the FE illusion of security
• Increasingly often we are requested to test
much less than the actual scope.
• Consider carefully prior to testing – what
should be the actual testing scope

27. How to test?
• “Supreme excellence consists in breaking the enemy's
resistance without fighting.” Sun Tzu
• Common Mobile WCF architecture
– Where is the presentation layer?
– Which entities are granted access to business logic?

28. How to test?
• OWASP top 10 – mobile:
Source: OWASP Top 10 Mobile project

29. The Oracle Exadata Example
• Oracle Exadata simplified:
– Data Warehouse platform
– Consolidation/Grid platform
– Storage platform
• Exadata security best practices consist of:
– The “regular stuff”
– Database standard security
– Data Warehouse specialized security
– Consolidation/Grid specialized security

30. The Oracle Exadata Example
• Oracle Exadata (as a database platform) Security
Testing Benchmark:
– Organization A tested:
• The databases
• The environments
• The Data Warehouse specialized security
• The Exadata itself
– Organization B tested:
• Just some deployed databases
• Partial security testing for each database
• Worse still: Exadata not to be tested as a policy
• Who said: 2013-A5 Security Misconfiguration?

31. Testing A5, A7, A9
• “If you know the enemy and know yourself
you need not fear the results of a hundred
battles”, Sun Tzu
• Do we really know ourselves?
• Where are A5, A7 and A9 implemented?
• Not testing the BE  illusion of knowing

32. The Windows XP Example
• Organization C, defines and enforces strict
development and deployment security
standards towards all its suppliers/customers.
• Over 60 pages of procedures and instructions.
• Insisting on supporting Windows XP based
systems.
• Who said: 2013-A9 Using Known Vulnerable
Components?

33. 2013-A9 Using known Vulnerable
Components
• A vendor offers DBAAS
– Excellent: beat the market offering *AAS
something...
• How can the organization trust the security of
DBAAS?
– Will separation be enforced?
– Will compartmentalization be enforced?
• Did we really tested and can trust the Cloud
on which the DBAAS is based?

34. Declarative Security
• What?
• One of the foundations of modern languages
run-time security.
• Mostly ignored or bypassed.
• Who said: Security misconfiguration – A5,
Missing Function Level access control – A7?

35. Declarative Security
• “Deployment descriptors must provide certain
structural information for each component if this
information has not been provided in annotations or
is not to be defaulted.” (Oracle docs.)

36. Declarative Security
• “Engage people with what they expect; it is
what they are able to discern and confirms
their projections. It settles them into
predictable patterns of response, occupying
their minds while you wait for the
extraordinary moment — that which they
cannot anticipate.” Sun Tzu
• Lack or weak declarative security: Once code
access achieved – the extraordinary will be
feasible.

37. Declarative Security
• Poor design due to no design
• Cancelling off declarative security or ignoring
declarative security  revoking language
security fundamentals.
• Common real life deployment descriptors:
// Do what you will. Totally permissive policy file.
grant {
permission java.security.AllPermission;
};
•  Killing my own code!

38. Reverse Engineering (A5, A6, A9)
• What for?
• Why for Mobile security testing ONLY?
• From Wikipedia:
– Reverse engineering is the process of discovering
the technological principles of a device, object, or
system through analysis of its structure, function,
and operation.

39. Testing A2, A5, A6
• 2013 A6 – Sensitive data exposure
• 2013 A5 – Security misconfiguration
• 2013 A2 – Broken authentication
• Too much use of “third singulars”
– The actual minute details of the tested object
dissolve

40. 2013-A5 Security Misconfiguration
• There is no external access!
• The intended users will only
perform intended actions…
• Virtualization  Separation
40

41. 2013-A5 Security Misconfiguration
• How do organizations secure legacy unsecured
systems?
• Install terminals (e.g. Citrix) as the presentation
layer / access control layer.
• Challenge: manage multiple users across multiple
systems.
• Result: the terminals are partially secure.
– Too many terminals to manage over long periods
– Some insecure
– The insecure terminals are the attacker entry points.

42. Critical Thinking
Takeover
Data theft
– Any system
– Any infrastructure
– Any communication
– Any language
– Any architecture
– Any component
– Any information, any
data
– Any physical layer
– Any logical layer
– Any storage device /
facility
– Any (communication)
channel
– Any interface
– Any encryption
– Any environment
– Any site (including DR)
– Any transaction
– Any log and audit trail
– Any archive
– Any process (operations,
ongoing, development)
Data tampering
System integrity disruption
Business Logic manipulation
Eavesdropping
Backdoors – built in by design
Backdoors – creation by attackers
Unintentional attacks
Intentional by authorized entities
Attacks by non-human entities
Denial of Service
De Facto Denial of Service
Authorization bypass
Access bypass
Smuggling, Splitting and evasion-type attacks

43. Critical Thinking
• Critical thinking is the ability to think clearly
and rationally. This requires reflective and
independent thinking. (Philosophy field)
• For organization security is too difficult: over
complexity, too much to orchestrate, etc.
• Increasingly often we are requested to test
much less than the actual scope.
• Some organizations will not be educated.
• Push the industry back up with those
organizations that can be educated.

44. Critical Thinking
• For the security professionals, security is a
challenge. Hence, always employ critical
thinking and review the process of testing
itself.
– Flexibility under varying technologies
– Use automated testing tools to the max AND be
always aware of their limitations
– Scoping accurately is mandatory

45. Qustions?
Yaniv Simsolo, CISSP
Image: Hubble Telescope: The cat’s eye nebula