The document presents an overview of fault models and fuzz testing techniques aimed at evaluating software robustness, dependability, and security. It discusses various approaches, examples of faults, and common pitfalls associated with these testing methods. Additionally, it highlights the importance of using specific tools for fault injection and fuzz testing to uncover hard-to-detect bugs.

1. Fault Models and Fuzz TechniquesShmuel GershonSTAREAST 2011Copyright © CC:BY-NC-SA 2007-11, Shmuel Gershon.

2. About...Shmuel Gershon

3. Testing Engineer

4. http://testing.gershon.info

5. Creator of Rapid Reporter

6. Twitter: @sgershon, Skype: sgershon, shmuel@gershon.infoDisclaimer:Names and Brands referenced herein may be claimed as property of third parties

7. Views expressed in this presentation are solely my own, and do not in any manner represent the views of my employer

8. Information in this presentation is provided 'as is' without any warranties or representations of any kindCopyright © CC:BY-NC-SA 2007-11

9. Fault Models and FuzzTalk ObjectivesUnderstand the principles of the techniquesMeet tools available for both techniquesLearn to apply the methods (in different app types)Perceive drawbacks and difficultiesLay down a foundation for further researchHave at least one new idea for your tests :)Testing is questioning a product in order to evaluate its value to a person that mattersAdapted from Jerry Weinberg + James Bach + Michael BoltonCopyright © CC:BY-NC-SA 2007-11

10. Putting Concepts into ContextAvailabilityRobustnessDependabilitySecurityReliabilityStabilityCopyright © CC:BY-NC-SA 2007-11

11. Fault Models and FuzzTalk OutlineFault ModelsOverviewExamples + DemosPitfalls and TricksFuzzingOverviewExamples + DemosPitfalls and TricksSummary and QuestionsCopyright © CC:BY-NC-SA 2007-11

12. Fault Models OverviewConceptual DiagramOS – Disk StorageInjectionRuntimeFaultLayerApplication Under TestOS – User I/OOS - MemoryOS - NetworkCopyright © CC:BY-NC-SA 2007-11

13. Fault Models Overview – DefinitionDefinitions:Introducing faults in order to test (error handling) code paths, that might otherwise rarely be followedAssess the robustness of software by checking it's reaction to adverse eventsPurposeValidation of: Robustness, Dependability (Availability) and SecurityAlso known as: Recovery Code test, Fault Injection, Negative test, Error Handling, Stress test...Copyright © CC:BY-NC-SA 2007-11

14. Fault Models Overview – ApproachesRandom Runtime Fault Injections:A tool control the type, time and location to ‘attack’. May modify the coverage of a set of tests (issues found may be harder to reproduce or debug).Initiated Runtime Fault Injection:Specific tests in which the faults are controlled at specific point, aiming a clear error handling flow.Bugs found in this approach may be easier to reproduce, accept and fix.(This is the approach we'll focus on).Copyright © CC:BY-NC-SA 2007-11

15. Fault Models Overview – ReactionsReactions against Runtime Fault BugsIt will never happen in real lifeA user will not do thatIt should fail in such conditionsIt is an unsupported scenarioWe can’t fix thisIt is a third-party problemHow would you reply?Whose decision it ultimately is?Copyright © CC:BY-NC-SA 2007-11

16. Fault Injection – Example #1Memory Starvation:Refuses to allocate a memory upon request (Insufficient Memory).Pinball:Silently skips the action,no harm done.WordPad:Disappears with your most valuable workCopyright © CC:BY-NC-SA 2007-11

17. Fault Injection – Example #2Network Errors:Simulates an error responses for common network requests and resources.Browser Wars:One browser survives