3. The Reality … Most errors are found after development… SOURCE: NIST Report,2002 Errors Found Post-Development Errors Found Pre-Development Time in Actual Development Time Looking for Errors Although developers spend a majority of their time looking for errors…
4. Defect Analysis and Reproduction in Different Phases Coding Production Test System Test Integration Test Unit Test
20. Team-wide Deployment and Workflow Scheduled Test Server Developer Machines Architect / Technical lead SCM Source Control code & tests Team Server Team Server Test Results Professional Editions Server Edition (Batch Mode) Architect Edition Team Coding Standards Nightly Results
21.
22.
Editor's Notes
So, why do I need to use Parasoft technologies? To improve quality … so where are the defects? 85% of defects are introduced at the beginning of developer But most of them are not found until later Where is more expensive <click> These later defects are complex to diagnose and are more symptoms than bugs themselves <click> Where as the problems earlier in the cycle are easier to identify and often represent catalysts to bugs <click to next slide>
Parasoft ’ s methodology is called ADP. ADP is the idea of preventing errors in order to help achieve better quality and more reliable software. What ADP consists of: - identifying errors as soon as they are introduced - learn from past mistakes so that we can 1) prevent the error from reoccurring in the future, 2) to improve the development process - With error prevention comes defect prevention, which less bugs in the software and better quality code - From fixing bugs to improving software lifecycle …
Parasoft helps to achieve Error Prevention by providing the solutions and infrastructure to help organizations achieve automated defect prevention. There are 2 main components that is needed for ADP: Automation Without automation, it is very difficult to implement any testing practice effectively. - Testing is very tedious and time consuming. - If done manually, first of all developers can make mistakes. Secondly, there is no way to ensure that testing is actually being done, and at the same time done correctly, and continuously. Parasoft provides toolsets to automate key testing practices such as … to help identify/prevent bugs early in the development process. 2) Team Infrastructure For testing to be effective, it needs to be incorporated within the team ’ s workflow, and be conditioned as part of a team behavior. The benefits of testing will be compromised even if only 1 person within a team is not following the required standards, or performing testing correctly. Parasoft solutions actually provides this team infrastructure to ensure that these test practices are ingrained into existing development process, in a way that does not disrupt the normal workflow.
Now let’s take a look at C++test, and how this solution can help organizations achieve Error Prevention in their C/C++ development C++test is a tool for testing C/C++ code. It helps companies automate key error prevention/detection practices at early stages of the development process. C++test performs 2 main functionalities: automatic static analysis and unit testing. It helps to automate the following tasks: - Automated static analysis. There are 2 types of static analysis, which is used for different purposes: pattern matching (enforcing best coding practices) and flow analysis (for detecting runtime errors) - Automation of code review - Automate unit testing - For embedded development, C++test allows test cases to be run at the host or directly on the target device. - code coverage analysis to help gauge the effectiveness of your testing. Now we will take a look at each of these technologies in turn.
The purpose of this slide is to illustrate how C++test and Insure++ can be used for similar validation strategies at different stages of the SDLC, but also from different perspectives: Starting from initial code development, we can apply preventative practices for a whole range of typical software defects, and use the auto-generated unit testing to verify unit robustness and correctly functional implementation. We also provide the auto-stubbing feature to help isolate and test software units early in the SDLC When developers start to integrate software modules (or if they are working on a legacy project) the ability to significantly reduce debugging efforts, and protect the code from the introduction of new bugs and defects are provided by Flow Analysis, Regression Testing and Code Review practices; automated by the C++test product Finally when the executable application/software is ready, the code is most easily tested under typical use-case scenarios using Insure++, to help quickly identify both symptoms and root causes of any remaining memory related runtime defects still existing in the code. This combination of carefully building our software product with industry best practices, and also focusing on a user ’ s perspective to test the product, produces a highly reliably software product.
Covers all main functionalities of C++test Now I am going to go over how C++Test fits within a team environment, and it can be utilized to drive ADP. First of all for C++test we offer different editions to suit different development roles/needs.
When C++test is deployed in a team environment, this diagram shows the team infrastructure and workflow which drives ADP. 1)First, the team architect we utilize the architect edition to configure team requirements and test settings for the entire group. Architect has full control to specify the desired test settings and help to standardize across the entire team. 2) After the architect will upload the test settings to our module the Team Server. This module centrally administers and shares these test settings/assets across the team, Ensure that all test practices are consistently and uniformly applied across the team. This way you can max the effectiveness of your testing procedures. 3) & 4) Your developers will then download the test settings from the TCM to perform static analysis and unit testing with C++Test Professional running on their local desktops. When the developer is sure that the code is error free he will add it to source control. In this way developers test as soon as they finish coding, catch/fix errors early on we ensure the integrity of the code which enters the source code repository. 5) At the build level, C++Test Server Edition can be configured to run the team’s entire test suite against your whole project automatically each night. - These test settings can be retrieved from the TCM. It can be the same as the developer’s configuration, or extra tests can be added. - At this stage, you will also run BD on the whole source code to help identify runtime bugs within your application act as a second line of defense against errors. Guarantee that the required standards and tests are being followed 6) After the test are complete C++Test will send all test data to Parasoft Concerto, which collects and analyzes information collected from multiple sources to provide detailed development metrics and reports. Parasoft Concerto helps teams to gain visibility and control of their development process, Teams can utilize the info gathered via the Concerto to make better decisions, identify potential problems in their process and improve upon. 7) Finally,the Server Edition will be able to generate 2 types of reports: 1- Summary reports for the managers to help them monitor 2- Individualized reports for the developers which contains the tasks and violations that each developer is responsible for. The reports can be retrieved from the Team Server Module, or automatically emailed to each developers. This infrastructure provides: 1- Automation via Parasoft C++test 2- Fully ingrains testing practices into the developer ’ s daily workflow to ensure that these practices are non intrusively introduced, and effectively implemented across the whole team 3- Provides a means to be able to monitor your development process, identify potential issues, and in turn be able to learn from past mistakes and improve your process. Essential elements of ADP.