Testing and Debugging            Testing, Bug Fixing and Debugging the CodeYordan DimitrovTeam Leader, TeamPulse,Telerik C...
Testing and Debugging             Testing, Bug Fixing and Debugging the CodeYordan DimitrovTelerik Corporationwww.telerik....
Table of Contents What Is Testing? Seven Testing Principles Developer Testing   Developer vs. QA Testing   Debugging ...
What Is Testing?
Main Test Activities Testing is not just running   tests, but also:   Planning and control   Choosing test conditions  ...
Main Objectives in Testing Testing pursues several   objectives:   Finding defects   Gaining confidence about the level...
Different Viewpoints / Different                             Objectives Objectives of testing differ according to the poi...
Different Viewpoints / Different                           Objectives (2) Objectives of testing differ according to the p...
Seven Testing Principles
Seven Testing Principles (1)1.Testing shows presence of defects   Testing can show that defects are present   Cannot pro...
Seven Testing Principles (2)2. Exhaustive testing is impossible   All combinations of inputs and preconditions    are usu...
Seven Testing Principles (3)3. Early testing   Testing activities shall be started as early as    possible     And shall...
Seven Testing Principles (4)4.Defect clustering   Testing effort shall be focused proportionally    To the expected and ...
Seven Testing Principles (5)5.Pesticide paradox   Same tests repeated over and over again tend    to loose their effectiv...
Seven Testing Principles (6)6. Testing is context dependent   Testing is done differently in different contexts    Examp...
Seven Testing Principles (7)7.Absence-of-errors fallacy   Finding and fixing defects itself does not help in    these cas...
Developer TestingTesting as a Priority of the Developer
Developer vs. QA Testing Software is tested in numerous ways  Some are typically performed by developers  Some are more...
Developer Testing Developer testing refers to testing by the developer Usually the following   tests are priority of dev...
Testing as a Priority of the QA                                  Engineer Numerous additionalkinds of testing are perform...
Debugging vs. Testing Testing   A means of initial detection of errors Debugging   A means of diagnosing and correctin...
Black-box vs. White-box                                   Testing Testing is usually   broken into two broad categories: ...
Black-box Techniques Black-box techniques are  a way to derive and select test conditions, test cases, or test data  Bas...
White-box Techniques White-box techniques  Also called structural or glass-box techniques  Based on an analysis of the ...
Role of Developer Testing in     Software Quality
Role of Developer Testing in                         Software Quality Individual           testing steps typically find l...
Why is Testing a Hard Activity? Testings goal runs                  counter to the goals of other development activities...
How Much Testing is Enough? How much testing    should be done is a matter of risk:  Too much testing can delay the prod...
How Much Time Should Be              Spent in Developer Testing ?Developer testing should probably take 8 to 25 percentof ...
Recommended Approach to    Developer Testing           Ground Rules and Tips  for Effective Development and Testing
Recommended Approach to                     Developer Testing Test for each relevant requirement   Make sure that the re...
Recommended Approach to                    Developer Testing (2) Use "basistesting" to add detailed test cases to those t...
Test First or Test Last? Effort is the same Detect defects earlier Forces you to think at least a little bit Exposes r...
Limitations of Developer                                   Testing Developer tests tend to be "clean tests“ Developer te...
Bag of Testing Tricks Incomplete Testing Structured Basis Testing Data-Flow Testing Equivalence Partitioning Error Gu...
Structured Basis Testing Test each statement in a program        at least once Compute the minimum number of test cases:...
SampleStatement1;       <-- 1Statement2;if ( x < 10 ) {       <-- 2   Statement3;}Statement4;(1)Count "1" for the routine ...
Sample Test CasesCase       Minimum memory use                     Most readable output 1     Nominal case                ...
Data-Flow Testing The normal combination of data states  A variable is defined, used one or more times,   and perhaps ki...
Data-Flow Testing (2) The key to writing data-flow test cases is to exercise all possible defined-used paths:   All defi...
Sample Test CasesCase               Test Description         if ( Condition 1 )                                           ...
Sample Test CasesCase                          Test Description 7       Define companyRetirement in line 12, and use it fi...
Boundary AnalysisCase                             Test Description 1     Case 1 is defined so that the true condition for ...
Compound Boundaries Minimum and Maximum allowable                  values Case                       Test Description  10...
Classes of Bad Data   Too little data (Case 2-11)       The wrong size of data   Too much data                     Uni...
Classes of Good Data   Nominal cases—middle-of-the-road, expected values   Minimum normal configuration   Maximum norma...
Typical Errors Which Classes Contain      the Most Errors? Errors by Classification   The scope of most errors is fairl...
Typical Errors (2) Errors by Classification   Misunderstanding the design is a recurring    theme in studies of programm...
Improving Your Testing Planning to Test Retesting (Regression Testing) Automated Testing                               ...
Finding a Defect1.Stabilize the error2.Locate the source of the error  a) Gather the data  b) Analyze the data and form hy...
Finding a Defect      Demo
Tips for Finding Defects (1) Use all available   data Refine the test cases Check unit tests Use available   tools Re...
Tips for Finding Defects (2) Narrow the suspicious   region of the code Be suspicious              of classes and routin...
Fixing a Defect Understand the problem before you fix it Understand the program,      not just the problem Confirm the ...
Psychological Considerations Your ego tells you that your code is                                   good and doesnt have ...
Psychological Considerations (2)                                   58
Psychological Distance How "Psychological        Distance" can help                      Second           Psychological F...
Testing and Quality Testing can give confidence in the quality      of the software if it finds few or no defects If def...
Testing and Debugging    курсове и уроци по програмиране, уеб дизайн – безплатно     BG Coder - онлайн състезателна систем...
Free Trainings @ Telerik Academy “High-Quality   Programming Code"    course @ Telerik Academy       codecourse.telerik....
Upcoming SlideShare
Loading in...5
×

12. Testing and Debugging - Testing, Bug Fixing and Debugging the Code

1,247

Published on

High-Quality Code @ Telerik Academy
Telerik Software Academy: http://codecourse.telerik.com/
The website and all video materials are in Bulgarian
Content:
What Is Testing?
Seven Testing Principles
Developer Testing
Developer vs. QA Testing
Debugging vs. Testing
Black-box vs. White-box Testing
Role of Developer Testing in Software Quality
Recommended Approach to Developer Testing

Published in: Education, Technology
1 Comment
2 Likes
Statistics
Notes
No Downloads
Views
Total Views
1,247
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
136
Comments
1
Likes
2
Embeds 0
No embeds

No notes for slide
  • Testing is the most popular quality-improvement activity—a practice supported by a wealth of industrial and academic research and by commercial experience.e
  • Unit testing is the execution of a complete class, routine, or small program that has been written by a single programmer or team of programmers, which is tested in isolation from the more complete system.Component testing is the execution of a class, package, small program, or other program element that involves the work of multiple programmers or programming teams, which is tested in isolation from the more complete system.Integration testing is the combined execution of two or more classes, packages, components, or subsystems that have been created by multiple programmers or programming teams. This kind of testing typically starts as soon as there are two classes to test and continues until the entire system is complete.Regression testing is the repetition of previously executed test cases for the purpose of finding defects in software that previously passed the same set of tests.System testing is the execution of the software in its final configuration, including integration with other software and hardware systems. It tests for security, performance, resource loss, timing problems, and other issues that can&apos;t be tested at lower levels of integration.
  • Some programmers use the terms “testing” and “debugging” interchangeably, but careful programmers distinguish between the two activities. Testing is a means of detecting errors. Debugging is a means of diagnosing and correcting the root causes of errors that have already been detected.
  • Testing is usually broken into two broad categories: black box testing and whitebox (or glass box) testing. “Black box testing” refers to tests in which the tester cannot see the inner workings of the item being tested. This obviously does not apply when you test code that you have written! “White box testing” refers to tests in which the tester is aware of the inner workings of the item being tested. This is the kind of testing that you as a developer use to test your own code. Both black box and white box testing have strengths and weaknesses; white box testing … is the kind of testing that developers perform.
  • Incomplete TestingSince exhaustive testing is impossible, practically speaking, the art of testing is that of picking the test cases most likely to find errors. Of the 1066 possible test cases, only a few are likely to disclose errors that the others don&apos;t. You need to concentrate on picking a few that tell you different things rather than a set that tells you the same thing over and over.When you&apos;re planning tests, eliminate those that don&apos;t tell you anything new—that is, tests on new data that probably won&apos;t produce an error if other, similar data didn&apos;t produce an error. Various people have proposed various methods of covering the bases efficiently, and several of these methods are discussed in the following sections.Structured Basis TestingIn spite of the hairy name, structured basis testing is a fairly simple concept. The idea is that you need to test each statement in a program at least once. If the statement is a logical statement—an if or a while, for example—you need to vary the testing according to how complicated the expression inside the if or while is to make sure that the statement is fully tested. The easiest way to make sure that you&apos;ve gotten all the bases covered is to calculate the number of paths through the program and then develop the minimum number of test cases that will exercise every path through the program.
  • Now that you&apos;ve created six test cases for the routine and satisfied the demands of structured basis testing, can you consider the routine to be fully tested? Probably not.This kind of testing assures you only that all of the code will be executed. It does not account for variations in data.Data-flow testing is based on the idea that data usage is at least as error-prone as control flow. Boris Beizer claims that at least half of all code consists of data declarations and initializations (Beizer 1990).
  • 1. One study found that 85 percent of errors could be corrected without modifying more than one routine (Endres 1975).2. The three most common sources of errors were thin application-domain knowledge, fluctuating and conflicting requirements, and communication and coordination breakdown.3.pair of studies performed many years ago found that, of total errors reported, roughly 95% are caused by programmers4. One study found that 36% of all construction errors were clerical misakes5.Another study found that 19% of the errors resulted from misunderstood design6.Another study found that 19% of the errors resulted from misunderstood design
  • 1. One study found that 85 percent of errors could be corrected without modifying more than one routine (Endres 1975).2. The three most common sources of errors were thin application-domain knowledge, fluctuating and conflicting requirements, and communication and coordination breakdown.3.pair of studies performed many years ago found that, of total errors reported, roughly 95% are caused by programmers4. One study found that 36% of all construction errors were clerical misakes5.Another study found that 19% of the errors resulted from misunderstood design6.Another study found that 19% of the errors resulted from misunderstood design
  • The program is supposed to print a list of employees and their income-tax withholdings in alphabetical order.
  • 12. Testing and Debugging - Testing, Bug Fixing and Debugging the Code

    1. 1. Testing and Debugging Testing, Bug Fixing and Debugging the CodeYordan DimitrovTeam Leader, TeamPulse,Telerik Corporation,Blog:http://blogs.telerik.com/jordandimitrov/Telerik Corporation www.telerik.com
    2. 2. Testing and Debugging Testing, Bug Fixing and Debugging the CodeYordan DimitrovTelerik Corporationwww.telerik.com
    3. 3. Table of Contents What Is Testing? Seven Testing Principles Developer Testing  Developer vs. QA Testing  Debugging vs. Testing  Black-box vs. White-box Testing Role of Developer Testing in Software Quality Recommended Approach to Developer Testing 3
    4. 4. What Is Testing?
    5. 5. Main Test Activities Testing is not just running tests, but also:  Planning and control  Choosing test conditions  Designing and executing test cases  Checking results  Evaluating exit criteria  Reporting on the testing process and system under test  Finalizing or completing closure activities 6
    6. 6. Main Objectives in Testing Testing pursues several objectives:  Finding defects  Gaining confidence about the level of quality  Providing information for decision-making  Preventing defects 7
    7. 7. Different Viewpoints / Different Objectives Objectives of testing differ according to the point of view:  Development testing – cause as many failures as possible and fix them  Acceptance testing – confirm that the system works as expected  Assessment – assess the quality of the software and its readiness for release 8
    8. 8. Different Viewpoints / Different Objectives (2) Objectives of testing differ according to the point of view:  Maintenance testing – check for new defects, introduced during development  Operational testing – assess system characteristics such as reliability or availability 9
    9. 9. Seven Testing Principles
    10. 10. Seven Testing Principles (1)1.Testing shows presence of defects  Testing can show that defects are present  Cannot prove that there are no defects  Appropriate testing reduces the probability for defects 11
    11. 11. Seven Testing Principles (2)2. Exhaustive testing is impossible  All combinations of inputs and preconditions are usually almost infinite number  Testing everything is not feasible  Except for trivial cases  Risk analysis and priorities should be used to focus testing efforts 12
    12. 12. Seven Testing Principles (3)3. Early testing  Testing activities shall be started as early as possible  And shall be focused on defined objectives  The later a bug is found – the more it costs! 13
    13. 13. Seven Testing Principles (4)4.Defect clustering  Testing effort shall be focused proportionally  To the expected and later observed defect density of modules  A small number of modules usually contains most of the defects discovered  Responsible for most of the operational failures 14
    14. 14. Seven Testing Principles (5)5.Pesticide paradox  Same tests repeated over and over again tend to loose their effectiveness  Previously undetected defects remain undiscovered  New and modified test cases should be developed 15
    15. 15. Seven Testing Principles (6)6. Testing is context dependent  Testing is done differently in different contexts  Example: safety-critical software is tested differently from an e-commerce site 16
    16. 16. Seven Testing Principles (7)7.Absence-of-errors fallacy  Finding and fixing defects itself does not help in these cases:  The system built is unusable  Does not fulfill the users’ needs and expectations 17
    17. 17. Developer TestingTesting as a Priority of the Developer
    18. 18. Developer vs. QA Testing Software is tested in numerous ways  Some are typically performed by developers  Some are more commonly performed by specialized test personnel  QA Engineers 19
    19. 19. Developer Testing Developer testing refers to testing by the developer Usually the following tests are priority of developer testing:  Unit tests  Component tests  Integration tests  Sometimes regression tests and system tests are also included 20
    20. 20. Testing as a Priority of the QA Engineer Numerous additionalkinds of testing are performed by QA Engineers and rarely performed by developers:  Beta tests  Platform tests  Customer-acceptance  Stress tests tests  Usability tests  Performance tests  Etc.  Configuration tests 21
    21. 21. Debugging vs. Testing Testing  A means of initial detection of errors Debugging  A means of diagnosing and correcting the root causes of errors that have already been detected 22
    22. 22. Black-box vs. White-box Testing Testing is usually broken into two broad categories:  Black-box testing  White-box testing 24
    23. 23. Black-box Techniques Black-box techniques are a way to derive and select test conditions, test cases, or test data  Based on an analysis of the test basis documentation  Also called specification-based or behavioral techniques  Tests are based on the way the system is supposed to work 25
    24. 24. White-box Techniques White-box techniques  Also called structural or glass-box techniques  Based on an analysis of the structure of the component or system  Information about how the software is constructed?  E.g., code and detailed design information  Usually a priority of developers 26
    25. 25. Role of Developer Testing in Software Quality
    26. 26. Role of Developer Testing in Software Quality Individual testing steps typically find less than 50 percent of the errors present each  (Unit test, component test, and integration test) The combination of testing steps often finds less than 60 percent of the errors present (Jones 1998) 28
    27. 27. Why is Testing a Hard Activity? Testings goal runs counter to the goals of other development activities Testing can never completely prove the absence of errors Testing by itself does not improve software quality Testing requires you to assume that youll find errors in your code 29
    28. 28. How Much Testing is Enough? How much testing should be done is a matter of risk:  Too much testing can delay the product release and increase the product price  Insufficient testing hides risks of errors in the final product 30
    29. 29. How Much Time Should Be Spent in Developer Testing ?Developer testing should probably take 8 to 25 percentof the total project time 31
    30. 30. Recommended Approach to Developer Testing Ground Rules and Tips for Effective Development and Testing
    31. 31. Recommended Approach to Developer Testing Test for each relevant requirement  Make sure that the requirements have been implemented Test for each relevant design concern  Make sure that the design has been implemented 33
    32. 32. Recommended Approach to Developer Testing (2) Use "basistesting" to add detailed test cases to those that test the requirements and the design Use a checklist of the kinds of errors youve made on the project to date or have made on previous projects 34
    33. 33. Test First or Test Last? Effort is the same Detect defects earlier Forces you to think at least a little bit Exposes requirements problems sooner Run it when you want Source: flickr 35
    34. 34. Limitations of Developer Testing Developer tests tend to be "clean tests“ Developer testing tends to have an optimistic view of test coverage Developer testing tends to skip more sophisticated kinds of test coverage 36
    35. 35. Bag of Testing Tricks Incomplete Testing Structured Basis Testing Data-Flow Testing Equivalence Partitioning Error Guessing Boundary Analysis Classes of Bad Data Classes of Good Data Use Test Cases That Make Hand-Checks Convenient 37
    36. 36. Structured Basis Testing Test each statement in a program at least once Compute the minimum number of test cases:  Start with 1 for the straight path through the routine  Add 1 for each of the following keywords, or their equivalents: if, while, repeat, for, and, and or  Add 1 for each case in a case statement  If the case statement doesnt have a default case, add 1 more 38
    37. 37. SampleStatement1; <-- 1Statement2;if ( x < 10 ) { <-- 2 Statement3;}Statement4;(1)Count "1" for the routine itself.(2)Count "2" for the if.
    38. 38. Sample Test CasesCase Minimum memory use Most readable output 1 Nominal case All boolean conditions are true 2 The initial for condition is false numEmployees < 1 3 m_employee[ id The first if is false ].governmentRetirementWith-held >=MAX_GOVT_RETIREMENT 4 The second if is false because not m_employee[ id the first part of the and is false ].WantsRetirement 5 The second if is false because not EligibleForRetirement( the second part of the and is m_employee[id] ) false 6 not EligibleForPersonalRetirement( The third if is false m_employee[ id ] ) 40
    39. 39. Data-Flow Testing The normal combination of data states  A variable is defined, used one or more times, and perhaps killed Source: http://blog.radvision.com 41
    40. 40. Data-Flow Testing (2) The key to writing data-flow test cases is to exercise all possible defined-used paths:  All definitions  Test every definition of every variable  I.e., every place at which any variable receives a value  All defined-used combinations  Test every combination of defining a variable in one place and using it in another 42
    41. 41. Sample Test CasesCase Test Description if ( Condition 1 ) { 1 Condition 1 = true, Condition 2 = x = a; true } else 2 { Condition 1 = false, Condition 2 = x = b; false } ? if ( Condition 2 ) x = a; y = x – 1; { y = x + 1; } ? else x = b; y = x + 1; { y = x - 1; }
    42. 42. Sample Test CasesCase Test Description 7 Define companyRetirement in line 12, and use it first in line 26 This isnt necessarily covered by any of the previous test cases 8 Define companyRetirement in line 12, and use it first in line 31 This isnt necessarily covered by any of the previous test cases
    43. 43. Boundary AnalysisCase Test Description 1 Case 1 is defined so that the true condition for m_employee[ ID ]. governmentRetirementWithheld < MAX_GOVT_RETIREMENT is the first case on the true side of the boundary 3 Case 3 is defined so that the false condition for m_employee[ ID ]. governmentRetirementWithheld < MAX_GOVT_RETIREMENT is on the false side of the boundary 9 An additional test case is added for the case directly on the boundary in which m_employee [ ID ].governmentRetirementWithheld = MAX_GOVT_RETIREMENT 45
    44. 44. Compound Boundaries Minimum and Maximum allowable values Case Test Description 10 A large group of employees, each of whom has a large salary (what constitutes "large" depends on the specific system being developed)—for the sake of example, well say 1000 employees, each with a salary of $250,000, none of whom have had any social security tax withheld and all of whom want retirement withholding. 11 A group of 10 employees, each of whom has a salary of $0.00. 46
    45. 45. Classes of Bad Data Too little data (Case 2-11)  The wrong size of data Too much data  Uninitialized data The wrong kind of data Case Test Description 12 An array of 100,000,000 employees. Tests for too much data. 13 A negative salary. Wrong kind of data. 14 A negative number of employees. Wrong kind of data. 47
    46. 46. Classes of Good Data Nominal cases—middle-of-the-road, expected values Minimum normal configuration Maximum normal configuration Compatibility with old data Case Test Description 16 A group of one employee. To test the minimum normal configuration. 17 A group of 500 employees. To test the maximum normal configuration. 48
    47. 47. Typical Errors Which Classes Contain the Most Errors? Errors by Classification  The scope of most errors is fairly limited  Many errors are outside the domain of construction  Most construction errors are the programmers fault  Clerical errors (typos) are a surprisingly common source of problems 49
    48. 48. Typical Errors (2) Errors by Classification  Misunderstanding the design is a recurring theme in studies of programmer errors  Most errors are easy to fix  Its a good idea to measure your own organizations experiences with errors Source: http://kathrynvercillo.com 50
    49. 49. Improving Your Testing Planning to Test Retesting (Regression Testing) Automated Testing 51
    50. 50. Finding a Defect1.Stabilize the error2.Locate the source of the error a) Gather the data b) Analyze the data and form hypothesis c) Determine how to prove or disprove the hypothesis d) Prove or disprove the hypothesis by 2c3.Fix the defect4.Test the fix5.Look for similar errors 52
    51. 51. Finding a Defect Demo
    52. 52. Tips for Finding Defects (1) Use all available data Refine the test cases Check unit tests Use available tools Reproduce the error several different ways Generate more data to generate more hypotheses Use the results of negative tests Brainstorm for possible hypotheses 54
    53. 53. Tips for Finding Defects (2) Narrow the suspicious region of the code Be suspicious of classes and routines that have had defects before Check code that’s changed recently Expand the suspicious region of the code Integrate incrementally Check for common defects Talk to someone else about the problem Take a break from the problem 55
    54. 54. Fixing a Defect Understand the problem before you fix it Understand the program, not just the problem Confirm the defect diagnosis Relax Save the original source code Fix the problem not the symptom Make one change at a time Add a unit test that expose the defect Look for similar defects Source: http://www.movingseniorsbc.com 56
    55. 55. Psychological Considerations Your ego tells you that your code is good and doesnt have a defect even when youve seen that it has one. How "Psychological Set" Contributes to Debugging Blindness 57
    56. 56. Psychological Considerations (2) 58
    57. 57. Psychological Distance How "Psychological Distance" can help Second Psychological First Variable Variable Distancestoppt stcppt Almost invisibleshiftrn shiftrm Almost nonedcount bcount Smallclaims1 claims2 Smallproduct sum Large 59
    58. 58. Testing and Quality Testing can give confidence in the quality of the software if it finds few or no defects If defects are found, the quality increases when those defects are fixed Lessons learnt from previous mistakes improve future performance 62
    59. 59. Testing and Debugging курсове и уроци по програмиране, уеб дизайн – безплатно BG Coder - онлайн състезателна система - online judge курсове и уроци по програмиране – Телерик академия форум програмиране, форум уеб дизайн уроци по програмиране и уеб дизайн за ученици ASP.NET курс - уеб програмиране, бази данни, C#, .NET, ASP.NET http://academy.telerik.com програмиране за деца – безплатни курсове и уроци ASP.NET MVC курс – HTML, SQL, C#, .NET, ASP.NET MVC безплатен SEO курс - оптимизация за търсачки алго академия – състезателно програмиране, състезаниякурсове и уроци по програмиране, книги – безплатно от Наков курс мобилни приложения с iPhone, Android, WP7, PhoneGap уроци по уеб дизайн, HTML, CSS, JavaScript, Photoshop Дончо Минков - сайт за програмиране free C# book, безплатна книга C#, книга Java, книга C# Николай Костов - блог за програмиране безплатен курс "Качествен програмен код" безплатен курс "Разработка на софтуер в cloud среда" C# курс, програмиране, безплатно
    60. 60. Free Trainings @ Telerik Academy “High-Quality Programming Code" course @ Telerik Academy  codecourse.telerik.com Telerik Software Academy  academy.telerik.com Telerik Academy @ Facebook  facebook.com/TelerikAcademy Telerik Software Academy Forums  forums.academy.telerik.com
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×