CSCE 548  Secure Software Development Risk-Based Security Testing
Reading <ul><li>This lecture:  </li></ul><ul><ul><li>Risk-Based Security Testing, McGraw: Chapter 7 </li></ul></ul><ul><li...
Application of Touchpoints Requirement and  Use cases Architecture  and Design Test Plans Code Tests and Test Results Feed...
Software Testing <ul><li>Running a program or system with the intent of finding errors  </li></ul><ul><li>evaluating capab...
Quality Assurance <ul><li>External quality: correctness,  reliability, usability, integrity </li></ul><ul><li>Interior (en...
Correctness Testing <ul><li>Black box:  </li></ul><ul><ul><li>Test data are derived from the specified functional requirem...
Correctness Testing <ul><li>White box: </li></ul><ul><ul><li>Software under test are visible to the tester  </li></ul></ul...
Performance Testing <ul><li>Goal: bottleneck identification, performance comparison and evaluation, etc. </li></ul><ul><li...
Reliability Testing <ul><li>Probability of failure-free operation of a system  </li></ul><ul><li>Dependable software: it d...
Security Testing <ul><li>Test: finding flaws in software can be exploited by attackers </li></ul><ul><li>Quality, reliabil...
Risk-Based Testing <ul><li>Identify risks </li></ul><ul><li>Create tests to address identified risks </li></ul><ul><li>Sec...
Penetration Testing <ul><li>Performed after the software is completed </li></ul><ul><ul><li>Evaluate operational environme...
Security Testing <ul><li>Can be applied before the product is completed  </li></ul><ul><li>Different levels of testing (e....
Risk Analysis  <ul><li>Design phase analysis: </li></ul><ul><ul><li>Identify and rank risks </li></ul></ul><ul><ul><li>Dis...
System-Level Testing <ul><li>Focus on the properties of the integrated software system </li></ul><ul><li>Penetration testi...
Behavior in the Presence of Malicious Attack <ul><li>What happens when the software fails? </li></ul><ul><ul><li>Safety cr...
Vulnerabilities <ul><li>Design-level </li></ul><ul><ul><li>Hardest to detect </li></ul></ul><ul><ul><li>Prevalent and crit...
Security Testing <ul><li>Functional security testing: testing security mechanisms for functional capabilities </li></ul><u...
Who Should Perform the Test? <ul><li>Standard testing organizations </li></ul><ul><ul><li>Functional testing </li></ul></u...
How to Test? <ul><li>White box analysis </li></ul><ul><ul><li>Understanding and analyzing source code and design </li></ul...
Malicious Input <ul><li>Software: takes input </li></ul><ul><li>Trust input? </li></ul><ul><ul><li>Malformed or malicious ...
What Else? <ul><li>Testing for malicious input: necessary but NOT sufficient </li></ul><ul><li>Risk-based security testing...
Next Class <ul><li>Security Operations </li></ul>
Upcoming SlideShare
Loading in …5
×

Lecture notes

1,029 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,029
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
16
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Lecture notes

  1. 1. CSCE 548 Secure Software Development Risk-Based Security Testing
  2. 2. Reading <ul><li>This lecture: </li></ul><ul><ul><li>Risk-Based Security Testing, McGraw: Chapter 7 </li></ul></ul><ul><li>Next lecture: </li></ul><ul><ul><li>Security Operations, McGraw: Chapter 9 </li></ul></ul>
  3. 3. Application of Touchpoints Requirement and Use cases Architecture and Design Test Plans Code Tests and Test Results Feedback from the Field 5. Abuse cases 6. Security Requirements 2. Risk Analysis External Review 4. Risk-Based Security Tests 1. Code Review (Tools) 2. Risk Analysis 3. Penetration Testing 7. Security Operations
  4. 4. Software Testing <ul><li>Running a program or system with the intent of finding errors </li></ul><ul><li>evaluating capability of the system and determining that its requirements </li></ul><ul><li>Physical processes vs. Software </li></ul><ul><li>Testing purposes </li></ul><ul><ul><li>To improve quality </li></ul></ul><ul><ul><li>For Verification & Validation (V&V) </li></ul></ul><ul><ul><li>For reliability estimation </li></ul></ul>
  5. 5. Quality Assurance <ul><li>External quality: correctness, reliability, usability, integrity </li></ul><ul><li>Interior (engineering) quality: efficiency, testability, documentation, structure </li></ul><ul><li>Future (adaptability) quality: flexibility, reusability, maintainability </li></ul>
  6. 6. Correctness Testing <ul><li>Black box: </li></ul><ul><ul><li>Test data are derived from the specified functional requirements without regard to the final program structure </li></ul></ul><ul><ul><li>Data-driven, input/output driven, or requirements-based </li></ul></ul><ul><ul><li>Functional testing </li></ul></ul><ul><ul><li>No implementation details of the code are considered </li></ul></ul>
  7. 7. Correctness Testing <ul><li>White box: </li></ul><ul><ul><li>Software under test are visible to the tester </li></ul></ul><ul><ul><li>Testing plans: based on the details of the software implementation </li></ul></ul><ul><ul><li>Test cases: derived from the program structure </li></ul></ul><ul><ul><li>glass-box testing, logic-driven testing, or design-based testing </li></ul></ul>
  8. 8. Performance Testing <ul><li>Goal: bottleneck identification, performance comparison and evaluation, etc. </li></ul><ul><li>Explicit or implicit requirements </li></ul><ul><li>&quot;Performance bugs&quot; – design problems </li></ul><ul><li>Test: usage, throughput, stimulus-response time, queue lengths, etc. </li></ul><ul><li>Resources to be tested: network bandwidth requirements, CPU cycles, disk space, disk access operations, memory usage, etc. </li></ul>
  9. 9. Reliability Testing <ul><li>Probability of failure-free operation of a system </li></ul><ul><li>Dependable software: it does not fail in unexpected or catastrophic ways </li></ul><ul><li>Difficult to test </li></ul><ul><li>(see later lecture on software reliability) </li></ul>
  10. 10. Security Testing <ul><li>Test: finding flaws in software can be exploited by attackers </li></ul><ul><li>Quality, reliability and security are tightly coupled </li></ul><ul><li>Software behavior testing </li></ul><ul><ul><li>Need: risk-based approach using system architecture information and attacker’s model </li></ul></ul>
  11. 11. Risk-Based Testing <ul><li>Identify risks </li></ul><ul><li>Create tests to address identified risks </li></ul><ul><li>Security testing vs. penetration testing </li></ul><ul><ul><li>Level of approach </li></ul></ul><ul><ul><li>Timing of testing </li></ul></ul>
  12. 12. Penetration Testing <ul><li>Performed after the software is completed </li></ul><ul><ul><li>Evaluate operational environment </li></ul></ul><ul><ul><li>Dynamic behavior </li></ul></ul><ul><li>Outside  in activity – defending perimeters </li></ul><ul><li>Cursory </li></ul>
  13. 13. Security Testing <ul><li>Can be applied before the product is completed </li></ul><ul><li>Different levels of testing (e.g., component/unit level vs. system level) </li></ul><ul><li>Testing environment </li></ul><ul><li>Detailed </li></ul>
  14. 14. Risk Analysis <ul><li>Design phase analysis: </li></ul><ul><ul><li>Identify and rank risks </li></ul></ul><ul><ul><li>Discusses inter-component assumptions </li></ul></ul><ul><li>Component/unit testing </li></ul><ul><ul><li>Test for: </li></ul></ul><ul><ul><ul><li>Unauthorized misuse of and access to the target assets </li></ul></ul></ul><ul><ul><ul><li>Violations of assumptions </li></ul></ul></ul><ul><ul><li>Breaking system into a number of discrete parts </li></ul></ul><ul><ul><li>Risk can be mitigated within the bounds of contextual assumptions </li></ul></ul>
  15. 15. System-Level Testing <ul><li>Focus on the properties of the integrated software system </li></ul><ul><li>Penetration testing = Security testing </li></ul><ul><li>Using data flow diagrams, models, and inter-component documentations, identify </li></ul><ul><ul><li>Inter-component failures </li></ul></ul><ul><ul><li>Design level security risks </li></ul></ul><ul><li>Use misuse cases to enhance test plan </li></ul>
  16. 16. Behavior in the Presence of Malicious Attack <ul><li>What happens when the software fails? </li></ul><ul><ul><li>Safety critical systems </li></ul></ul><ul><li>Track risk over time </li></ul><ul><li>Security relative to </li></ul><ul><ul><li>Information and services protected </li></ul></ul><ul><ul><li>Skills and resources of adversaries </li></ul></ul><ul><ul><li>Cost of protection </li></ul></ul><ul><li>System vulnerabilities </li></ul>
  17. 17. Vulnerabilities <ul><li>Design-level </li></ul><ul><ul><li>Hardest to detect </li></ul></ul><ul><ul><li>Prevalent and critical </li></ul></ul><ul><ul><li>Requires great expertise to detect – hard to automate </li></ul></ul><ul><li>Implementation specific </li></ul><ul><ul><li>Critical </li></ul></ul><ul><ul><li>Easier to detect – some automation </li></ul></ul>
  18. 18. Security Testing <ul><li>Functional security testing: testing security mechanisms for functional capabilities </li></ul><ul><li>Adversarial security testing: risk-based security testing </li></ul><ul><ul><li>Understanding and simulating the attacker’s approach </li></ul></ul><ul><li>Both approaches must be used </li></ul><ul><li>Security attacks may ignore the security mechanism to exploits of the software defects </li></ul>
  19. 19. Who Should Perform the Test? <ul><li>Standard testing organizations </li></ul><ul><ul><li>Functional testing </li></ul></ul><ul><li>Software security professionals </li></ul><ul><ul><li>Risk-based security testing </li></ul></ul><ul><ul><li>Important: expertise and experience </li></ul></ul>
  20. 20. How to Test? <ul><li>White box analysis </li></ul><ul><ul><li>Understanding and analyzing source code and design </li></ul></ul><ul><ul><li>Very effective finding programming errors </li></ul></ul><ul><ul><li>Can be supported by automated static analyzer </li></ul></ul><ul><ul><li>Disadvantage: high rate of false positives </li></ul></ul><ul><li>Black box analysis </li></ul><ul><ul><li>Analyze a running program </li></ul></ul><ul><ul><li>Probe the program with various input (malicious input) </li></ul></ul><ul><ul><li>No need for any code – can be tested remotely </li></ul></ul>
  21. 21. Malicious Input <ul><li>Software: takes input </li></ul><ul><li>Trust input? </li></ul><ul><ul><li>Malformed or malicious input may lead to security compromise </li></ul></ul><ul><ul><li>What is the input? </li></ul></ul><ul><ul><ul><li>Data vs. control </li></ul></ul></ul><ul><li>Attacker toolkit </li></ul>
  22. 22. What Else? <ul><li>Testing for malicious input: necessary but NOT sufficient </li></ul><ul><li>Risk-based security testing </li></ul><ul><ul><li>Planning tests (use forest-level view) </li></ul></ul><ul><ul><li>Need operational aspects </li></ul></ul><ul><li>System state vs. applications used </li></ul><ul><li>Multithread system – time-based attacks </li></ul>
  23. 23. Next Class <ul><li>Security Operations </li></ul>

×