• Save
Smart Testing-Test Less, Cover More, Benefit Manifold- STeP-IN Summit 2014 Speaker Session Presentation
Upcoming SlideShare
Loading in...5
×
 

Smart Testing-Test Less, Cover More, Benefit Manifold- STeP-IN Summit 2014 Speaker Session Presentation

on

  • 157 views

Recently, Harbinger presented a session at the Pune round of STeP-IN Summit 2014, an International software testing conference. STeP-IN unlike many conferences dwelled right into the practical ...

Recently, Harbinger presented a session at the Pune round of STeP-IN Summit 2014, an International software testing conference. STeP-IN unlike many conferences dwelled right into the practical approaches towards improving the quality of software. Many interesting software testing strategies and methods like mind mapping, outcome based testing were discussed along with testing in mobile and social media era.
In our 2 hours speaker session ‘Smart Testing – Test less, Cover more, Benefit manifold’ conducted by Avinash Magdum, General Manager-Quality, Harbinger Systems, we talked about the root cause of defect leakages and their solutions to enable quality delivery with increased test coverage and reduced testing efforts.

Statistics

Views

Total Views
157
Views on SlideShare
85
Embed Views
72

Actions

Likes
0
Downloads
0
Comments
0

1 Embed 72

http://blog.harbinger-systems.com 72

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Smart Testing-Test Less, Cover More, Benefit Manifold- STeP-IN Summit 2014 Speaker Session Presentation Smart Testing-Test Less, Cover More, Benefit Manifold- STeP-IN Summit 2014 Speaker Session Presentation Presentation Transcript

  • © Harbinger Systems | www.harbinger-systems.com Smart Testing – Test Less, Cover More, Benefit Manifold Avinash Magdum STeP-IN SUMMIT 2014
  • © Harbinger Systems | www.harbinger-systems.com Agenda
  • © Harbinger Systems | www.harbinger-systems.com Vicious Cycle of Quality Challenges Shorter Release Cycle Inadequate Time for Testing Insufficient Test Coverage High Number of Defect Leakages Too Many Maintenance Releases Adoption of Agile Methodology Increasing Testing Scope
  • © Harbinger Systems | www.harbinger-systems.com Root Cause of Defect Leakages Insufficient Test Cases • No Traceability • No clarity on requirements • Misinterpretation of requirements Poor Test Planning • No Impact analysis • Incorrect Prioritization Inadequate Time to Execute • Increasing OS – Browser support • Variety of devices to be supported
  • © Harbinger Systems | www.harbinger-systems.com The Solution…
  • © Harbinger Systems | www.harbinger-systems.com Code Coverage Analysis Definition: Its measure of the degree to which the source code of an application is tested by it’s test suite. Categories Functional Coverage Conditional Coverage Branch Coverage Statement Coverage
  • © Harbinger Systems | www.harbinger-systems.com Functional Coverage Definition: It is the ratio of number of functions (methods) covered by the test suite over number of functions (methods) in the application. For example, if an application has 4500 methods, and if 2700 of them get called after executing entire test suite then, Functional coverage = 2700 / 4500 = 60%
  • © Harbinger Systems | www.harbinger-systems.com • Definition: It is the ratio of number of code paths (branches) covered by the test suite over number of code paths in the application. Branch Coverage In this code snippet, there are 5 code paths within switch block. So for this block, if our test cases can visit only 3 cases, then the branch coverage for this block is 60%. Same could be extended to entire application
  • © Harbinger Systems | www.harbinger-systems.com • Definition: It is the ratio of number of decisions covered by the test suite over total number of decisions in the application. Conditional / Decision Coverage In this code snippet, there are 4 decisions within the if block. There are 4 decisions – both a and b negative, only a negative, only b negative and both a and b positive If we test only for both negative and both positive values, conditional coverage is 50%
  • © Harbinger Systems | www.harbinger-systems.com • Definition: It is the ratio of number of statements executed by the test suite over total number of statements in the application. For example, if an application has 600,000 lines of code, and if only 240,000 of them get executed by the entire test suite then, Statement coverage = 240,000 / 600,000 = 40% Statement Coverage
  • © Harbinger Systems | www.harbinger-systems.com Code Coverage Tools Java • Cobertura (open source) • Jcover (commercial) • EMMA (open source) .NET • dotCover (commercial) • Ncover (commercial) C / C++ / VC++ • BullsEye (commercial) • Gcov (open source)
  • © Harbinger Systems | www.harbinger-systems.com Objective 1: Identifying Missing Test Cases
  • © Harbinger Systems | www.harbinger-systems.com Execute entire test suite Select / Unselect assemblies / modules for coverage analysis Verify the Coverage browser Instrument the application Select right code coverage tool Identifying Missing Test Cases - I
  • © Harbinger Systems | www.harbinger-systems.com Map uncovered methods (functional code) on UI actions Group Uncovered Methods Dead Code Error Handling Code Functional Code Identify Uncovered Methods Analyze Coverage Report Identifying Missing Test Cases - II
  • © Harbinger Systems | www.harbinger-systems.com Identify Uncovered Branches Analyze coverage report – Increased coverage Execute new test cases Create new test cases to include above UI actions Identifying Missing Test Cases - III
  • © Harbinger Systems | www.harbinger-systems.com Analyze coverage report – Increased coverage Execute new test cases Create new test cases to include above UI actions Map Uncovered Branches on UI actions Identifying Missing Test Cases - IV
  • © Harbinger Systems | www.harbinger-systems.com Objective 2: Identifying Redundant Test Cases
  • © Harbinger Systems | www.harbinger-systems.com Identifying Redundant Test Cases - I Compare every two successive Statement Coverage counters After every execution of a test case, check Statement Coverage Execute Test Cases sequentially, for a Feature Setup Application For Code Coverage Analysis
  • © Harbinger Systems | www.harbinger-systems.com Identifying Redundant Test Cases - II Remove all the Redundant Test Cases from the Test Suite Repeat this Process for all the Features Repeat above steps for all the Test Cases of the Feature If Statement Coverage Counters are identical, mark the later test case as Redundant
  • © Harbinger Systems | www.harbinger-systems.com Objective 3: Identifying Optimal set of Test Cases (Test Plan) for any Maintenance Releases
  • © Harbinger Systems | www.harbinger-systems.com It’s a tool to identify optimal subset of Test Suite for any maintenance release It is created using Code Coverage Analysis Gives list of test cases impacted by any code change Reduces testing efforts drastically Arrests the bug leakages Impact Analyzer
  • © Harbinger Systems | www.harbinger-systems.com Creating Impact Analyzer - I Execute Test Cases sequentially, one at a time Reset all the Coverage counters Create a database with 3 tables Method Master Test Case Master Test Case Methods Map Setup Application For Code Coverage Analysis
  • © Harbinger Systems | www.harbinger-systems.com Creating Impact Analyzer - II Create an UI to query modified Method and get list of all associated Test Cases Repeat the Process for all the Test Cases Create a row in the Map table specifying Test Case ID, and IDs of all the called Methods List all the Method names called by the test case Get Method Coverage after execution of test case
  • © Harbinger Systems | www.harbinger-systems.com Creating Test Plan using I. A. Add all the test cases impacted by each of the method and include these in the test plan Repeat the Process for all the Methods For the method execute a query in Impact Analyzer to get a list of test cases calling that method Identify names of each method modified Get a list of Defects fixed for the release
  • © Harbinger Systems | www.harbinger-systems.com Takeaways • Facilitates to identify and add missing test cases – Reduces Defect Leakages drastically • Facilitates to identify redundant test cases – Reduces testing efforts • Using Impact Analysis, provides optimum set of test cases for maintenance releases – Reduces testing efforts with highest coverage Smart Testing – Less efforts, More Coverage, Higher Quality, Less Cost
  • © Harbinger Systems | www.harbinger-systems.com Q&A Visit us at: www.harbinger-systems.com Write to us at: hsplinfo@harbingergroup.com
  • © Harbinger Systems | www.harbinger-systems.com Thank You! Visit us at: www.harbinger-systems.com Write to us at: hsplinfo@harbingergroup.com Blog: blog.harbinger-systems.com Twitter: twitter.com/HarbingerSys (@HarbingerSys) Slideshare: slideshare.net/hsplmkting Facebook: facebook.com/HarbingerSys LinkedIn: linkedin.com/company/harbinger-systems 18