EuroSTAR Software Testing Conference 2010 presentation on Passion For Testing, By Examples by Peter Zimmerer. See more at: http://conference.eurostarsoftwaretesting.com/past-presentations/

1. Copyright © Siemens AG 2010. All rights reserved.
Corporate Technology
EUROSTAR 2010
Copenhagen, Denmark
December 2, 2010
Peter Zimmerer
Principal Engineer
Siemens AG
Corporate Technology
81739 Munich, Germany
peter.zimmerer@siemens.com
http://www.siemens.com/corporate-technology/
Passion For Testing –By Examples

2. Contents
Motivation
From theory to practicePractical, metaphoric analogy examples
Risk-based testing strategy
Testing of non-functional requirements (TDD)
Integration Integration Testing ArchitectureSummary

3. Motivation
Putting“theoretical” testing stuff into practiceoften fails … If you want to become a good tester, or even better a great tester with passion you will have to learn, train, and practice a lot of things. Sometimes people get lost in all the learning and education stuff, and especially putting “theoretical” concepts and topics into practice fails quite often. To make a difference in this session I would like to share with you three practical, metaphoric analogy examples from different areas that will help you to better understand, implement, and remember the corresponding testing concepts and practices until the end of your life.

4. Risk-based testing strategyBase the testing strategy on business goals and priorities Risk-based testing strategy (RBT) Risk identification
Risk = P ×D
P Probability of failure
Frequency of use
Chance of failure: criticality & complexity at implementation & usage, lack of quality
D Damage (consequence & cost for business & test & usage) Risk analysis –Product risk analysis workshopRisk response –Test objectives, test levels, test design methods …

5. Risk-based testing strategy
Example

6. Risk-based testing strategy –Summary
As a tester think outside the box to identifyrisks
As a tester be a master in risk identification, communication, and negotiation
As a tester actively participate in a product risk analysis workshopas one (or rather the) important stakeholder

7. Testing of non-functional requirements (TDD)
Focus on usingrequirements not only on perceivingrequirements
Reviews are often too passive –requirements are only augmented but not questioned
Quality is a result of usage
Describing / Specifying a test (even better: more tests …) for a non-functional requirement will help youto really understand the requirement
Preventive testing is built upon the observation that
one of the most effective ways of specifying something is
to describe (in detail) how you would accept (test) it
if someone gave it to you.David Gelperin, Bill Hetzel
Preventive testing is the basic idea of
any kind of test-driven development (TDD) approach
is a precondition

8. 8 s
0
100
200
300400500
600
700
concurrent users
6 s
response times
4 s
2 s
What’s the difference? Performance (testing) vs. Scalability (testing)
Performance (Testing)
Scalability (Testing)

9. Performance (testing) vs. Scalability (testing) –Reality
Example
Experiment

10. Performance & Performance testingPerformance
The degree to which a system or component accomplishes its designated functions within given constraints, such as speed, accuracy, or memory usage(IEEE 1990).
The speed at which a computer operates … during a benchmark test. The benchmark test usually involves some combination of work that attempts to imitate the kinds of work the computer does during actual use. The total effectiveness of a computer system, including throughput, individual response time, and availability(http://www.whatis.com/). Performance testing
Testing conducted to evaluate the compliance of a system or component with specified performance requirements (IEEE 1990).

11. ScalabilityScalability
The ease with which a system or component can be modifiedto fit the problem area(CMU SEI Glossary at http://www.sei.cmu.edu/str/indexes/glossary/). Scalability
It is the ability of a computer application or product (hardware or software) to continue to function well when it (or its context) is changedin size or volume in order to meet a user need. Typically, the rescaling is to a larger size or volume. It is the ability not only to function well in the rescaled situation, but to actually take full advantageof it in terms of performance (user response time and so forth) and the larger number of users that could be handled (http://www.whatis.com/).
Requires a balanced partnership between hardware and software.

12. Performance testing vs. Scalability testing
Performance testing ≠ Scalability testing
Closed systems ≠ Open systems
0
0,5
1
1,5
2
2,5
3
3,5
4
50 100 150 200
Users
Response Time
Scalability testing

13. Performance testing vs. Scalability testing –Summary
Understand the difference betweenperformance and scalabilityby understanding the difference betweenperformance testingand scalability testing
Describing / Specifying a test (even better: more tests …) for a non-functional requirement will help youto really understand the requirement
Focus on usingrequirements not only on perceivingrequirements
is a precondition

14. Integration  Integration Testing  Architecture
The goal of integration testing is to test in a grey-box manner
 The interaction of components and subsystems
 The interaction and embedding with the environment and system
configuration
 The dynamic behavior and communication of the system
 Control flow and data flow
 The architecture and design as specified in the
Software Architecture Description document
During integration test execution the
internal behavior of the system under test
is monitored by using tracing facilities
to provide the required information
Integration testing
Integration
Architecture

15. Integration Integration Testing
Integration
Constructive: Integration of components / subsystems to get a running system (or parts)
Quick-checkby performing a smoketestonly for most importantinterfaces and functions
Periodicaldelivery of integrated system (or parts) corresponding to the integration plan
Successful integration is a precondition for handoverto next step / level (integration testing)
Integration testing
Quality assuring: Test of the system to detect bugs in the interworking of its parts, to gain confidence, to mitigate risks, etc.
Detailed checkof the system by performing a systematictest of allinterfaces and functions corresponding to the test concept
Progressivetest as a separate testing level corresponding to test concept with defined begin and end dates
Successful integration testing is a precondition for handoverto next step / level (system testing)

16. Integration Integration Testing Architecture
Example

17. Integration Integration Testing Architecture –Summary
Integration ≠ Integration testingActively involve architects in integration testing
Select an appropriate integration strategy that supports and drives the goals and benefits of integration testing
Address and follow integration testing needsin the architecture (including testability) Reconsider and balanceusage of stubs & mocksagainstevidence & informative valueof integration testing
Integration testingIntegrationArchitecture

18. Summary
3 Practical, metaphoric analogy examples
that will help you to become
even more passionate about testing in the future
Be free to share these examples with your colleagues and stakeholders (testers as well as non-testers) to show excellent testing practices as one result of your passion for testing
Use examples, metaphors, and visualizations to improve effectiveness and efficiency in testing