Software Testing (Part I)A Brief Presentation By Disney E. Lai
Outline● Fundamentals of Testing● Testing in the Software Life Cycle● Discussion
Fundamentals of TestingWHY IS TESTING NECESSARY?
Fundamentals of TestingThe Threats of Bugs● Damaged Reputation ● Pollution/Waste● High/Unpredictable ● Lost jobs maintenance cost ● Lost lives● Unexpected delays ● Lost rights● Lack of confidence ● Lost missions● Lawsuits ● Lost Wars and many more ....
Fundamentals of TestingARIANE 5 Flight 501 (1996)● Pride of the European Space Agency● Exploded 37 seconds after launched● Cause of failure – Inertial Reference System software failed to compute 64-bit float number. Software reused from Ariane 4 only uses 16-bit.● Cost of Failure - $8 billion
Fundamentals of TestingPatriot Missile Bug (1991)● Patriot Missile System● Designed to intercept and destroy incoming missile● Early version failed to respond to incoming SCUD missiles● Cause of failure – software rounding error incorrectly calculate time● Cost of Failure – 28 lives lost and many injured.
Fundamentals of TestingMars Climate Orbiter (1998)● Designed to stay on Mars low orbit to study Mars climate.● Failed to stabilize in orbit. Destroyed by Mars atmospheric pressure● Cause of failure – software system use imperial units rather than metric● Cost of Failure – ~$330 million, lost mission
Fundamentals of TestingSoyuz TMA-10 (2007)● Return module carrying 3 person including Malaysian Angkasawan● Computer glitch caused a steeper ballistic descent● Extreme G-force● Cost of Failure – ~ 200KM off target, fried Angkasawan (almost)
Fundamentals of TestingUniversiti Sains Malaysia (2009)● Online Application System● Misinformed all 8,173 prequalified applicants of successfull entry rather than only 3,599 applicants.● Caused frustration and anger.● Cost of Failure – ~ lost of reputation?
Fundamentals of TestingCity Bus Terminal First Day (2006)● RM4.5 Million City Bus Terminal Inanam● Computerised ticketing system failed to function due to computer glitch – Manua alternative● Caused frustration and anger.● Cost of Failure – ~ lost of reputation? Lost service
Fundamentals of TestingKD Tunku Abdul Rahman (2010)
Fundamentals of TestingWhere Do Bugs Come From?● People Errors Bugs (defects) Failures● Requirements and design specifications● Code (Business Logic and UI)● Documentation (electronic and hard copy)● Failures People Dissatisfaction with quality
Fundamentals of TestingWhere Do Bugs Come From?● Bugs Occur Due to.... ● Programmer, Analyst, and other individuals contributing (including tester!) ● Complexity of code, infrastructure or problem ● Changing and Meshing of Technologies ● Time Pressure ● Many system interactions
Fundamentals of TestingWhere Do Bugs Come From?● Failures Occur Due to bugs and.... ● Environmental conditions – dust, humidity... ● Misuse (deliberate and accidental)
Fundamentals of TestingWhat does QUALITY means to you?● Fitness for use vs conformance to requirements.● TESTING AND QUALITY● Important Quality characteristics? OR● Testing, Quality Assurance and Quality Improvement
Fundamentals of TestingTesting Objectives● Find bugs – information programmers need to fix important bugs● Increase Confidence Level – Quality● Prevent defects – early detection● Help management understand system quality● And so on....
Fundamentals of Testing TESTING OBJECTIVES PLANS ? ACTIONS
Fundamentals of TestingTest Phases● Unit/Component Test● Integration Test● System Test● Acceptance Test/Pilot Test● Maintenance Test● Operational Test
Fundamentals of TestingTesting Vs. Debugging● Testing - Find failures that caused by bugs● Debugging – Fix defect ● Identifies root cause of a bug ● Repairs the code ● Checks that defects is fixed correctly● Different responsibilities: ● Testers Test ● Programmers Debug
Fundamentals of TestingTesting Reveals Presence of Bugs Testing can reveal the presence of bugs, but cannot prove their absence
Fundamentals of TestingMission Impossible: Exhaustive TestingExhaustive testing (all combinations of inputs andpreconditions) is NOT possibleExhaustive testing as a way to prove the software works is amisexpectationBad Expectation create problem for test pro and teams ● Unachievable high demands on test group ● Perception of incompetence
Fundamentals of TestingBenefit of Early QA and Testing● Cost of bug tends to increase as the project progresses....Defect Clustering● Bugs tend to appear in clustersPesticide Paradox● One technique alone is not effective to find defects● Use new and different techniques to find defect
Fundamentals of TestingAbsence-of-Errors Fallacy● Finding and fixing many bug does not guarantee satisfaction● Successful projects balance – features, schedule, budget and quality
Fundamentals of TestingFUNDAMENTAL TEST PROCESS
Fundamentals of TestingTesting Processes I ON EXECUTION NTAT G S CLOSURE NNIN LYSI IGN EME DES ANA PLA EVALUATING EXIT CRITERIA IMPL REPORTING TEST RESULT CONTROL
Fundamentals of TestingPlanning and Control ● Determine Test Scope, Risks, Objectives, Strategies, Resource ● Scheduling ● Implement test strategies ● Determine test exit criteria ● Measure and analyze results ING ● Monitor and document N ● Initiative corrective actions PLAN ● Make decisions CONTROL
Fundamentals of TestingAnalysis and Design● Review the test basis● Identify and prioritize● Evaluate testability● Design and prioritize combinations of test data● Identify test data needed for test condition● Design test environment● Identify infrastructure tools EXECUTION ION G S NTAT CLOSURE NNIN LYSI N IG EM E DES EVALUATING ANA PLA EXIT CRITERIA IMPL REPORTING TEST RESULT CONTROL
Fundamentals of TestingImplementation and Execution● Develop, implement and prioritize test cases● Create test harnesses, scripts● Organize test suites, and sequences of test procedures● Verify the test environment● Execute test cases (manual or automated)● Log test results● Compare actual and expected EXECUTION● Report and analyze incidents ION G● Repeat corrected and/or updated S NTAT CLOSURE NNIN LYSI N IG EM E DES EVALUATING ANA PLA test EXIT CRITERIA IMPL REPORTING● Run confirmation and/or TEST RESULT regression test CONTROL
Fundamentals of TestingExit Criteria, Reporting, and Closure● Check test logs against exit criteria● Assess if more tests are required● Write test summarry report● Confirm test deliverables● Finalize and archive testware, test environment● Deliver testware to maintenance org.● Perform restrospective to capture EXECUTION improvements for future releases ION G S NTAT CLOSURE NNIN LYSI N IG EM E DES EVALUATING ANA PLA EXIT CRITERIA IMPL REPORTING TEST RESULT CONTROL
Testing In The SW Life Cycle Testing Static Dynamic Static Review White box Black boxAnalysis Non Functional Functional
Testing In The SW Life CycleOther Test scopes● Functionality● Security● Interoperability● Performance and Reliability● Stress, Capacity and Volume● Maintenance and Maintainability● Usability and UI● Configuration and Portability● ...
Testing In The SW Life CycleISO 9126 Quality Standard for Software