Approaches to Software Testing

10,565 views
10,252 views

Published on

A presentation that provides an overview of software testing approaches including "schools" of software testing and a variety of testing techniques and practices.

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
10,565
On SlideShare
0
From Embeds
0
Number of Embeds
16
Actions
Shares
0
Downloads
116
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Approaches to Software Testing

  1. 1. Approaches to Software Testing: An Introduction Customized and presented to: Time Warner, New York, NY, 6/2006 Scott Barber Chief Technologist PerfTestPlus, Inc. www.PerfTestPlus.com Approaches to Software Testing: Page 1© 2006 PerfTestPlus All rights reserved. An I ntroduction
  2. 2. Who am I?- Chief Technologist of PerfTestPlus- Executive Director of the Association of Software Testing- Co-Founder of the Workshop on Performance and Reliability- An international speaker and contributer to various publications- My specialties include - Testing and analyzing performance for complex systems - Developing customized testing methodologies - Embedded systems testing - Testing biometric identification and security systems- I am a member of - IEEE - American MENSA - the Context-Driven School of Software Testing -and a signatory to the Manifesto for Agile Software Development www.PerfTestPlus.com Approaches to Software Testing: Page 2© 2006 PerfTestPlus All rights reserved. An I ntroduction
  3. 3. Who am I? My name is Scott and Im a... Test-o-holic. www.PerfTestPlus.com Approaches to Software Testing: Page 3© 2006 PerfTestPlus All rights reserved. An I ntroduction
  4. 4. Welcome & OverviewLogistics/Facilities?Who are each of you? - Name - Business Card Title - What you actually do - What you hope to take away from this seminar - What is distracting you www.PerfTestPlus.com Approaches to Software Testing: Page 4© 2006 PerfTestPlus All rights reserved. An I ntroduction
  5. 5. Welcome & OverviewWhat are we going to be talking about? - How and where testing fits in - The purpose of testing - The value of testingHow will this help me do my job better? - Who is responsible for what & when - How to work together to improve the quality of the product - Avoiding “turf-wars” - Improving the fit between process and purpose www.PerfTestPlus.com Approaches to Software Testing: Page 5© 2006 PerfTestPlus All rights reserved. An I ntroduction
  6. 6. Seminar ObjectivesAt the conclusion of this seminar, attendees will befamiliar with and have a basic understanding ofthe following items, the contexts where they aregenerally practiced, their strengths and weaknessesin context and where to go for more information: - Popular system test life-cycles and their relationship to popular software development life-cycles - Industry common system testing approaches - Industry common system testing practices www.PerfTestPlus.com Approaches to Software Testing: Page 6© 2006 PerfTestPlus All rights reserved. An I ntroduction
  7. 7. Agenda Self-Categorization - Testing “School” - Testing Life Cycle - Testing Techniques/Practices Break Approaches/Schools Life Cycles Break Techniques/Practices Putting it all Together Wrap-up www.PerfTestPlus.com Approaches to Software Testing: Page 7© 2006 PerfTestPlus All rights reserved. An I ntroduction
  8. 8. Self-CategorizationOn the walls of the room are several flip-chart sheets withdescriptions of certain common classifications related tosoftware testing.I have provided each of you with “voting stickers” that youwill use to indicate how well your project fits into thedescribed category.On each chart is a horizontal scale. You can place one stickerper sheet anywhere along the scale indicating if your projectfits each description: - Extremely well - Somewhat well - Not at all wellIf you are completely uncertain where your project fits, placeyour sticker in the box labeled www.PerfTestPlus.com Approaches to Software Testing: Page 8© 2006 PerfTestPlus All rights reserved. An I ntroduction
  9. 9. Ready... Set.... Go! www.PerfTestPlus.com Approaches to Software Testing: Page 9© 2006 PerfTestPlus All rights reserved. An I ntroduction
  10. 10. Descriptions RevealedThe Four “Schools” of Software Testing ● Analytical/Mathematical ● Factory/Process ● Quality Control ● Context-Driven www.PerfTestPlus.com Approaches to Software Testing: Page 10© 2006 PerfTestPlus All rights reserved. An I ntroduction
  11. 11. Descriptions RevealedSoftware Testing Life Cycles ● Waterfall/Big Bang ● Iterative/Spiral ● Test First/TDD ● Agile www.PerfTestPlus.com Approaches to Software Testing: Page 11© 2006 PerfTestPlus All rights reserved. An I ntroduction
  12. 12. Descriptions RevealedSoftware Testing Techniques/Practices ● Scripted Manual/Automated ● Black Box/UAT ● Exploratory ● Unit/Developer/White Box www.PerfTestPlus.com Approaches to Software Testing: Page 12© 2006 PerfTestPlus All rights reserved. An I ntroduction
  13. 13. Questions www.PerfTestPlus.com Approaches to Software Testing: Page 13© 2006 PerfTestPlus All rights reserved. An I ntroduction
  14. 14. 10 Min Break www.PerfTestPlus.com Approaches to Software Testing: Page 14© 2006 PerfTestPlus All rights reserved. An I ntroduction
  15. 15. Testing “Schools”What are they?Why make the distinction?How do they relate to your project?*Note* The slides in this segment heavily reference the original Four Schools of Software Testing as coined and presented by Brett Pettichord which can be found at www.pettichord.com. www.PerfTestPlus.com Approaches to Software Testing: Page 15© 2006 PerfTestPlus All rights reserved. An I ntroduction
  16. 16. Four Views of Software TestingAnalytic School - sees testing as rigorous and technical with many proponents in academiaFactory School - sees testing as a way to measure progress with emphasis on cost and repeatable standardsQuality School - emphasizes process, policing developers and acting as the gatekeeperContext-Driven School - emphasizes people, setting out to find the bugs that will be most important to stakeholders www.PerfTestPlus.com Approaches to Software Testing: Page 16© 2006 PerfTestPlus All rights reserved. An I ntroduction
  17. 17. Why Classify Testing into Schools?Understand why testing experts disagree- Not simply a matter of personality or experience- There are often underlying reasons for disagreementImprove the basis for debate- Differences in values may explain why we favor different policies- Explain how my school differs from the othersBut it can lead to oversimplification www.PerfTestPlus.com Approaches to Software Testing: Page 17© 2006 PerfTestPlus All rights reserved. An I ntroduction
  18. 18. What is a School?A school is not a technique.A school is defined by:- Standards of criticism- Exemplar techniques- Hierarchies of values www.PerfTestPlus.com Approaches to Software Testing: Page 18© 2006 PerfTestPlus All rights reserved. An I ntroduction
  19. 19. Analytic School: Core BeliefsSoftware is a logical artifactTesting is a branch of CS/Mathematics (i.e. Objective, rigorous and comprehensive)Testing techniques must have a logico-mathematical form (i.e. “one right answer”)Testing is technicalKey Question:Which techniques should we use? www.PerfTestPlus.com Approaches to Software Testing: Page 19© 2006 PerfTestPlus All rights reserved. An I ntroduction
  20. 20. Analytic School ExemplarCode Coverage- aka“Structural” testing- Dozens of code-coverage metrics have been designed and compared- Provides an objective measure of testing www.PerfTestPlus.com Approaches to Software Testing: Page 20© 2006 PerfTestPlus All rights reserved. An I ntroduction
  21. 21. Analytic SchoolImplications- Require precise and detailed specifications- Testers verify whether the software conforms to its specification- Anything else isn’t testingMost prevalent- Academia- Telecom- Safety-Critical www.PerfTestPlus.com Approaches to Software Testing: Page 21© 2006 PerfTestPlus All rights reserved. An I ntroduction
  22. 22. Factory School: Core BeliefsTesting must be managed (i.e. Predictable, repeatable, planned)Testing must be cost-effective (i.e. Low-skilled workers require direction)Testing validates the productTesting measures development progressKey Questions:- How can we measure whether we’re makingprogress?- When will we be done? www.PerfTestPlus.com Approaches to Software Testing: Page 22© 2006 PerfTestPlus All rights reserved. An I ntroduction
  23. 23. Factory School ExemplarTraceability Matrix- Make sure that every requirement has been tested www.PerfTestPlus.com Approaches to Software Testing: Page 23© 2006 PerfTestPlus All rights reserved. An I ntroduction
  24. 24. Factory SchoolImplications- Require clear boundaries between testing and other activities (start/stop criteria)- Resist changing plans (complicates progress tracking)- Software testing assembly line (V-model)- Accept management assumptions about testing- Encourage standards, “best practices,” and certificationMost prevalent- Enterprise IT- Government www.PerfTestPlus.com Approaches to Software Testing: Page 24© 2006 PerfTestPlus All rights reserved. An I ntroduction
  25. 25. Quality School: Core BeliefsSoftware quality requires disciplineTesting determines whether development processes are being followed.Testers may need to police developers to follow the rules.Testers have to protect users from bad software.Key Question:Are we following a good process? www.PerfTestPlus.com Approaches to Software Testing: Page 25© 2006 PerfTestPlus All rights reserved. An I ntroduction
  26. 26. Quality School ExemplarThe Gatekeeper- The software isn’t ready until QA says it’s ready www.PerfTestPlus.com Approaches to Software Testing: Page 26© 2006 PerfTestPlus All rights reserved. An I ntroduction
  27. 27. Quality SchoolImplications- Prefer “Quality Assurance” over “Testing”- Testing is a stepping stone to “process improvement”- May alienate developersMost prevalent- Large bureaucracies- Organizations under stress www.PerfTestPlus.com Approaches to Software Testing: Page 27© 2006 PerfTestPlus All rights reserved. An I ntroduction
  28. 28. Context-Driven School: Core BeliefsSoftware is created by people. People set the context.Testing finds bugs. A bug is anything that could bug a stakeholder.Testing provides information to the projectTesting is a skilled, mental activityTesting is multidisciplinaryKey Question:What testing would be most valuable right now? www.PerfTestPlus.com Approaches to Software Testing: Page 28© 2006 PerfTestPlus All rights reserved. An I ntroduction
  29. 29. Context-Driven School ExemplarExploratory Testing- Concurrent test design and test execution- Rapid learning www.PerfTestPlus.com Approaches to Software Testing: Page 29© 2006 PerfTestPlus All rights reserved. An I ntroduction
  30. 30. Context-Driven SchoolImplications- Expect changes. Adapt testing plans based on test results.- Effectiveness of test strategies can only be determined with field research- Testing research requires empirical and psychological study- Focus on skill over practiceMost prevalent- Commercial, Market-driven Software www.PerfTestPlus.com Approaches to Software Testing: Page 30© 2006 PerfTestPlus All rights reserved. An I ntroduction
  31. 31. What is Testing?Analytic School says - A branch of computer science and mathematicsFactory School says - A managed processQuality School says - A branch of software quality assuranceContext-Driven School says - A branch of development www.PerfTestPlus.com Approaches to Software Testing: Page 31© 2006 PerfTestPlus All rights reserved. An I ntroduction
  32. 32. What school is right for you?Analytic?Factory?Quality?Context-Driven? www.PerfTestPlus.com Approaches to Software Testing: Page 32© 2006 PerfTestPlus All rights reserved. An I ntroduction
  33. 33. Questions www.PerfTestPlus.com Approaches to Software Testing: Page 33© 2006 PerfTestPlus All rights reserved. An I ntroduction
  34. 34. Software Life CyclesWaterfall – Sequential, minimal feedback loopsV-Model – Parallel, decoupled, minimal feedback loopsIterative – Similar to lots of little waterfallsSpiral – Like iterative, but presumes iterations get smallerTDD/TFD – Test first, developers do much testingAgile/XP – Adaptive, incremental, just-in-time, just enough www.PerfTestPlus.com Approaches to Software Testing: Page 34© 2006 PerfTestPlus All rights reserved. An I ntroduction
  35. 35. Waterfall www.PerfTestPlus.com Approaches to Software Testing: Page 35© 2006 PerfTestPlus All rights reserved. An I ntroduction
  36. 36. V-model www.PerfTestPlus.com Approaches to Software Testing: Page 36© 2006 PerfTestPlus All rights reserved. An I ntroduction
  37. 37. Iterative www.PerfTestPlus.com Approaches to Software Testing: Page 37© 2006 PerfTestPlus All rights reserved. An I ntroduction
  38. 38. Spiral www.PerfTestPlus.com Approaches to Software Testing: Page 38© 2006 PerfTestPlus All rights reserved. An I ntroduction
  39. 39. Test Driven/Test First Development www.PerfTestPlus.com Approaches to Software Testing: Page 39© 2006 PerfTestPlus All rights reserved. An I ntroduction
  40. 40. Agile / XP www.PerfTestPlus.com Approaches to Software Testing: Page 40© 2006 PerfTestPlus All rights reserved. An I ntroduction
  41. 41. Life Cycle SummaryWhich Life Cycle does your project use?What are the pros/cons of that Life Cycle Model?Which Testing Schools seem to fit with which Life Cycles?How about Hybrids?Are there others? www.PerfTestPlus.com Approaches to Software Testing: Page 41© 2006 PerfTestPlus All rights reserved. An I ntroduction
  42. 42. Questions www.PerfTestPlus.com Approaches to Software Testing: Page 42© 2006 PerfTestPlus All rights reserved. An I ntroduction
  43. 43. 10 Min Break www.PerfTestPlus.com Approaches to Software Testing: Page 43© 2006 PerfTestPlus All rights reserved. An I ntroduction
  44. 44. Testing Techniques/PracticesScripted – Follow the well-defined predetermined stepsRegression – Find changes from previous releaseAd Hoc – One time test to answer a specific questionAutomated – Computer conducts tests and reports resultsExploratory – Simultaneous learning, test design & test execution to detect defects of interest to a stakeholderUnit/Developer/White Box – Tests not using the UIBlack Box/UAT – Expected usage and error modesShotgun – “If you use a shotgun, you dont have to aim that carefully” www.PerfTestPlus.com Approaches to Software Testing: Page 44© 2006 PerfTestPlus All rights reserved. An I ntroduction
  45. 45. Testing Techniques/PracticesWhat testing techniques have you seen work...- Particularly well- Particularly poorly- Well in specific circumstancesWhat other techniques/practices...- Do you know of- Would you recommend- Would you like to know more about or try www.PerfTestPlus.com Approaches to Software Testing: Page 45© 2006 PerfTestPlus All rights reserved. An I ntroduction
  46. 46. Questions www.PerfTestPlus.com Approaches to Software Testing: Page 46© 2006 PerfTestPlus All rights reserved. An I ntroduction
  47. 47. Examples of what works well togetherWaterfall + Unit Tests + Scripted + UAT = FactoryV-Model + Unit Tests + Scripted = AnalyticalIterative + Regression + Scripted + UAT = QualityTDD/TFP + Unit Tests + Exploratory = Context-Driven www.PerfTestPlus.com Approaches to Software Testing: Page 47© 2006 PerfTestPlus All rights reserved. An I ntroduction
  48. 48. Examples of what doesnt work together Waterfall + Regression Agile + Scripted Analytical + Exploratory Context-Driven + Anything done without adding value www.PerfTestPlus.com Approaches to Software Testing: Page 48© 2006 PerfTestPlus All rights reserved. An I ntroduction
  49. 49. Moving ForwardCompared to where you placed your stickers earlier, where would you like those stickers to be in the future and why? www.PerfTestPlus.com Approaches to Software Testing: Page 49© 2006 PerfTestPlus All rights reserved. An I ntroduction
  50. 50. Questions www.PerfTestPlus.com Approaches to Software Testing: Page 50© 2006 PerfTestPlus All rights reserved. An I ntroduction
  51. 51. Contact Info Scott Barber Chief Technologist PerfTestPlus, IncE-mail: Web Site:sbarber@perftestplus.com www.PerfTestPlus.com www.PerfTestPlus.com Approaches to Software Testing: Page 51© 2006 PerfTestPlus All rights reserved. An I ntroduction

×