© Harbinger Systems | www.harbinger-systems.com
Smart Testing –
Test Less,
Cover More,
Benefit Manifold
Avinash Magdum
STe...
© Harbinger Systems | www.harbinger-systems.com
Agenda
© Harbinger Systems | www.harbinger-systems.com
Vicious Cycle of Quality Challenges
Shorter Release
Cycle
Inadequate Time
...
© Harbinger Systems | www.harbinger-systems.com
Root Cause of Defect Leakages
Insufficient Test Cases
• No Traceability
• ...
© Harbinger Systems | www.harbinger-systems.com
The Solution…
© Harbinger Systems | www.harbinger-systems.com
Code Coverage Analysis
Definition: Its measure of the degree to which
the ...
© Harbinger Systems | www.harbinger-systems.com
Functional Coverage
Definition: It is the ratio of number of functions
(me...
© Harbinger Systems | www.harbinger-systems.com
• Definition: It is the ratio of number of code
paths (branches) covered b...
© Harbinger Systems | www.harbinger-systems.com
• Definition: It is the ratio of number of decisions
covered by the test s...
© Harbinger Systems | www.harbinger-systems.com
• Definition: It is the ratio of number of
statements executed by the test...
© Harbinger Systems | www.harbinger-systems.com
Code Coverage Tools
Java
• Cobertura (open source)
• Jcover (commercial)
•...
© Harbinger Systems | www.harbinger-systems.com
Objective 1:
Identifying Missing Test Cases
© Harbinger Systems | www.harbinger-systems.com
Execute entire test suite
Select / Unselect assemblies / modules for
cover...
© Harbinger Systems | www.harbinger-systems.com
Map uncovered methods (functional code) on
UI actions
Group Uncovered Meth...
© Harbinger Systems | www.harbinger-systems.com
Identify Uncovered Branches
Analyze coverage report – Increased coverage
E...
© Harbinger Systems | www.harbinger-systems.com
Analyze coverage report – Increased coverage
Execute new test cases
Create...
© Harbinger Systems | www.harbinger-systems.com
Objective 2:
Identifying Redundant Test Cases
© Harbinger Systems | www.harbinger-systems.com
Identifying Redundant Test Cases - I
Compare every two successive Statemen...
© Harbinger Systems | www.harbinger-systems.com
Identifying Redundant Test Cases - II
Remove all the Redundant Test Cases ...
© Harbinger Systems | www.harbinger-systems.com
Objective 3:
Identifying Optimal set of Test Cases (Test Plan)
for any Mai...
© Harbinger Systems | www.harbinger-systems.com
It’s a tool to identify optimal subset of Test Suite
for any maintenance r...
© Harbinger Systems | www.harbinger-systems.com
Creating Impact Analyzer - I
Execute Test Cases sequentially, one at a tim...
© Harbinger Systems | www.harbinger-systems.com
Creating Impact Analyzer - II
Create an UI to query modified Method and ge...
© Harbinger Systems | www.harbinger-systems.com
Creating Test Plan using I. A.
Add all the test cases impacted by each of ...
© Harbinger Systems | www.harbinger-systems.com
Takeaways
• Facilitates to identify and add missing test cases
– Reduces D...
© Harbinger Systems | www.harbinger-systems.com
Q&A
Visit us at: www.harbinger-systems.com
Write to us at: hsplinfo@harbin...
© Harbinger Systems | www.harbinger-systems.com
Thank You!
Visit us at: www.harbinger-systems.com
Write to us at: hsplinfo...
Upcoming SlideShare
Loading in …5
×

Smart Testing-Test Less, Cover More, Benefit Manifold- STeP-IN Summit 2014 Speaker Session Presentation

632 views

Published on

Recently, Harbinger presented a session at the Pune round of STeP-IN Summit 2014, an International software testing conference. STeP-IN unlike many conferences dwelled right into the practical approaches towards improving the quality of software. Many interesting software testing strategies and methods like mind mapping, outcome based testing were discussed along with testing in mobile and social media era.
In our 2 hours speaker session ‘Smart Testing – Test less, Cover more, Benefit manifold’ conducted by Avinash Magdum, General Manager-Quality, Harbinger Systems, we talked about the root cause of defect leakages and their solutions to enable quality delivery with increased test coverage and reduced testing efforts.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
632
On SlideShare
0
From Embeds
0
Number of Embeds
174
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Smart Testing-Test Less, Cover More, Benefit Manifold- STeP-IN Summit 2014 Speaker Session Presentation

  1. 1. © Harbinger Systems | www.harbinger-systems.com Smart Testing – Test Less, Cover More, Benefit Manifold Avinash Magdum STeP-IN SUMMIT 2014
  2. 2. © Harbinger Systems | www.harbinger-systems.com Agenda
  3. 3. © Harbinger Systems | www.harbinger-systems.com Vicious Cycle of Quality Challenges Shorter Release Cycle Inadequate Time for Testing Insufficient Test Coverage High Number of Defect Leakages Too Many Maintenance Releases Adoption of Agile Methodology Increasing Testing Scope
  4. 4. © Harbinger Systems | www.harbinger-systems.com Root Cause of Defect Leakages Insufficient Test Cases • No Traceability • No clarity on requirements • Misinterpretation of requirements Poor Test Planning • No Impact analysis • Incorrect Prioritization Inadequate Time to Execute • Increasing OS – Browser support • Variety of devices to be supported
  5. 5. © Harbinger Systems | www.harbinger-systems.com The Solution…
  6. 6. © Harbinger Systems | www.harbinger-systems.com Code Coverage Analysis Definition: Its measure of the degree to which the source code of an application is tested by it’s test suite. Categories Functional Coverage Conditional Coverage Branch Coverage Statement Coverage
  7. 7. © Harbinger Systems | www.harbinger-systems.com Functional Coverage Definition: It is the ratio of number of functions (methods) covered by the test suite over number of functions (methods) in the application. For example, if an application has 4500 methods, and if 2700 of them get called after executing entire test suite then, Functional coverage = 2700 / 4500 = 60%
  8. 8. © Harbinger Systems | www.harbinger-systems.com • Definition: It is the ratio of number of code paths (branches) covered by the test suite over number of code paths in the application. Branch Coverage In this code snippet, there are 5 code paths within switch block. So for this block, if our test cases can visit only 3 cases, then the branch coverage for this block is 60%. Same could be extended to entire application
  9. 9. © Harbinger Systems | www.harbinger-systems.com • Definition: It is the ratio of number of decisions covered by the test suite over total number of decisions in the application. Conditional / Decision Coverage In this code snippet, there are 4 decisions within the if block. There are 4 decisions – both a and b negative, only a negative, only b negative and both a and b positive If we test only for both negative and both positive values, conditional coverage is 50%
  10. 10. © Harbinger Systems | www.harbinger-systems.com • Definition: It is the ratio of number of statements executed by the test suite over total number of statements in the application. For example, if an application has 600,000 lines of code, and if only 240,000 of them get executed by the entire test suite then, Statement coverage = 240,000 / 600,000 = 40% Statement Coverage
  11. 11. © Harbinger Systems | www.harbinger-systems.com Code Coverage Tools Java • Cobertura (open source) • Jcover (commercial) • EMMA (open source) .NET • dotCover (commercial) • Ncover (commercial) C / C++ / VC++ • BullsEye (commercial) • Gcov (open source)
  12. 12. © Harbinger Systems | www.harbinger-systems.com Objective 1: Identifying Missing Test Cases
  13. 13. © Harbinger Systems | www.harbinger-systems.com Execute entire test suite Select / Unselect assemblies / modules for coverage analysis Verify the Coverage browser Instrument the application Select right code coverage tool Identifying Missing Test Cases - I
  14. 14. © Harbinger Systems | www.harbinger-systems.com Map uncovered methods (functional code) on UI actions Group Uncovered Methods Dead Code Error Handling Code Functional Code Identify Uncovered Methods Analyze Coverage Report Identifying Missing Test Cases - II
  15. 15. © Harbinger Systems | www.harbinger-systems.com Identify Uncovered Branches Analyze coverage report – Increased coverage Execute new test cases Create new test cases to include above UI actions Identifying Missing Test Cases - III
  16. 16. © Harbinger Systems | www.harbinger-systems.com Analyze coverage report – Increased coverage Execute new test cases Create new test cases to include above UI actions Map Uncovered Branches on UI actions Identifying Missing Test Cases - IV
  17. 17. © Harbinger Systems | www.harbinger-systems.com Objective 2: Identifying Redundant Test Cases
  18. 18. © Harbinger Systems | www.harbinger-systems.com Identifying Redundant Test Cases - I Compare every two successive Statement Coverage counters After every execution of a test case, check Statement Coverage Execute Test Cases sequentially, for a Feature Setup Application For Code Coverage Analysis
  19. 19. © Harbinger Systems | www.harbinger-systems.com Identifying Redundant Test Cases - II Remove all the Redundant Test Cases from the Test Suite Repeat this Process for all the Features Repeat above steps for all the Test Cases of the Feature If Statement Coverage Counters are identical, mark the later test case as Redundant
  20. 20. © Harbinger Systems | www.harbinger-systems.com Objective 3: Identifying Optimal set of Test Cases (Test Plan) for any Maintenance Releases
  21. 21. © Harbinger Systems | www.harbinger-systems.com It’s a tool to identify optimal subset of Test Suite for any maintenance release It is created using Code Coverage Analysis Gives list of test cases impacted by any code change Reduces testing efforts drastically Arrests the bug leakages Impact Analyzer
  22. 22. © Harbinger Systems | www.harbinger-systems.com Creating Impact Analyzer - I Execute Test Cases sequentially, one at a time Reset all the Coverage counters Create a database with 3 tables Method Master Test Case Master Test Case Methods Map Setup Application For Code Coverage Analysis
  23. 23. © Harbinger Systems | www.harbinger-systems.com Creating Impact Analyzer - II Create an UI to query modified Method and get list of all associated Test Cases Repeat the Process for all the Test Cases Create a row in the Map table specifying Test Case ID, and IDs of all the called Methods List all the Method names called by the test case Get Method Coverage after execution of test case
  24. 24. © Harbinger Systems | www.harbinger-systems.com Creating Test Plan using I. A. Add all the test cases impacted by each of the method and include these in the test plan Repeat the Process for all the Methods For the method execute a query in Impact Analyzer to get a list of test cases calling that method Identify names of each method modified Get a list of Defects fixed for the release
  25. 25. © Harbinger Systems | www.harbinger-systems.com Takeaways • Facilitates to identify and add missing test cases – Reduces Defect Leakages drastically • Facilitates to identify redundant test cases – Reduces testing efforts • Using Impact Analysis, provides optimum set of test cases for maintenance releases – Reduces testing efforts with highest coverage Smart Testing – Less efforts, More Coverage, Higher Quality, Less Cost
  26. 26. © Harbinger Systems | www.harbinger-systems.com Q&A Visit us at: www.harbinger-systems.com Write to us at: hsplinfo@harbingergroup.com
  27. 27. © Harbinger Systems | www.harbinger-systems.com Thank You! Visit us at: www.harbinger-systems.com Write to us at: hsplinfo@harbingergroup.com Blog: blog.harbinger-systems.com Twitter: twitter.com/HarbingerSys (@HarbingerSys) Slideshare: slideshare.net/hsplmkting Facebook: facebook.com/HarbingerSys LinkedIn: linkedin.com/company/harbinger-systems 18

×