Introduction to Parasoft C++TEST


Published on

Static Code Analysis, Dynamic Application Monitoring, Code Coverage and Unit testing as a combined method in Software Error Prevention process.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • So, why do I need to use Parasoft technologies? To improve quality … so where are the defects? 85% of defects are introduced at the beginning of developer But most of them are not found until later Where is more expensive <click> These later defects are complex to diagnose and are more symptoms than bugs themselves <click> Where as the problems earlier in the cycle are easier to identify and often represent catalysts to bugs <click to next slide>
  • Parasoft ’ s methodology is called ADP. ADP is the idea of preventing errors in order to help achieve better quality and more reliable software. What ADP consists of: - identifying errors as soon as they are introduced - learn from past mistakes so that we can 1) prevent the error from reoccurring in the future, 2) to improve the development process - With error prevention comes defect prevention, which less bugs in the software and better quality code - From fixing bugs to improving software lifecycle …
  • Parasoft helps to achieve Error Prevention by providing the solutions and infrastructure to help organizations achieve automated defect prevention. There are 2 main components that is needed for ADP: Automation Without automation, it is very difficult to implement any testing practice effectively. - Testing is very tedious and time consuming. - If done manually, first of all developers can make mistakes. Secondly, there is no way to ensure that testing is actually being done, and at the same time done correctly, and continuously. Parasoft provides toolsets to automate key testing practices such as … to help identify/prevent bugs early in the development process. 2) Team Infrastructure For testing to be effective, it needs to be incorporated within the team ’ s workflow, and be conditioned as part of a team behavior. The benefits of testing will be compromised even if only 1 person within a team is not following the required standards, or performing testing correctly. Parasoft solutions actually provides this team infrastructure to ensure that these test practices are ingrained into existing development process, in a way that does not disrupt the normal workflow.
  • Now let’s take a look at C++test, and how this solution can help organizations achieve Error Prevention in their C/C++ development C++test is a tool for testing C/C++ code. It helps companies automate key error prevention/detection practices at early stages of the development process. C++test performs 2 main functionalities: automatic static analysis and unit testing. It helps to automate the following tasks: - Automated static analysis. There are 2 types of static analysis, which is used for different purposes: pattern matching (enforcing best coding practices) and flow analysis (for detecting runtime errors) - Automation of code review - Automate unit testing - For embedded development, C++test allows test cases to be run at the host or directly on the target device. - code coverage analysis to help gauge the effectiveness of your testing. Now we will take a look at each of these technologies in turn.
  • The purpose of this slide is to illustrate how C++test and Insure++ can be used for similar validation strategies at different stages of the SDLC, but also from different perspectives: Starting from initial code development, we can apply preventative practices for a whole range of typical software defects, and use the auto-generated unit testing to verify unit robustness and correctly functional implementation. We also provide the auto-stubbing feature to help isolate and test software units early in the SDLC When developers start to integrate software modules (or if they are working on a legacy project) the ability to significantly reduce debugging efforts, and protect the code from the introduction of new bugs and defects are provided by Flow Analysis, Regression Testing and Code Review practices; automated by the C++test product Finally when the executable application/software is ready, the code is most easily tested under typical use-case scenarios using Insure++, to help quickly identify both symptoms and root causes of any remaining memory related runtime defects still existing in the code. This combination of carefully building our software product with industry best practices, and also focusing on a user ’ s perspective to test the product, produces a highly reliably software product.
  • Covers all main functionalities of C++test Now I am going to go over how C++Test fits within a team environment, and it can be utilized to drive ADP. First of all for C++test we offer different editions to suit different development roles/needs.
  • When C++test is deployed in a team environment, this diagram shows the team infrastructure and workflow which drives ADP. 1)First, the team architect we utilize the architect edition to configure team requirements and test settings for the entire group.  Architect has full control to specify the desired test settings and help to standardize across the entire team.   2) After the architect will upload the test settings to our module the Team Server. This module centrally administers and shares these test settings/assets across the team,  Ensure that all test practices are consistently and uniformly applied across the team.  This way you can max the effectiveness of your testing procedures.   3) & 4) Your developers will then download the test settings from the TCM to perform static analysis and unit testing with C++Test Professional running on their local desktops. When the developer is sure that the code is error free he will add it to source control.  In this way developers test as soon as they finish coding, catch/fix errors early on  we ensure the integrity of the code which enters the source code repository.   5) At the build level, C++Test Server Edition can be configured to run the team’s entire test suite against your whole project automatically each night. - These test settings can be retrieved from the TCM. It can be the same as the developer’s configuration, or extra tests can be added. - At this stage, you will also run BD on the whole source code to help identify runtime bugs within your application  act as a second line of defense against errors.  Guarantee that the required standards and tests are being followed 6) After the test are complete C++Test will send all test data to Parasoft Concerto, which collects and analyzes information collected from multiple sources to provide detailed development metrics and reports. Parasoft Concerto helps teams to gain visibility and control of their development process, Teams can utilize the info gathered via the Concerto to make better decisions, identify potential problems in their process and improve upon. 7) Finally,the Server Edition will be able to generate 2 types of reports: 1- Summary reports for the managers to help them monitor 2- Individualized reports for the developers which contains the tasks and violations that each developer is responsible for. The reports can be retrieved from the Team Server Module, or automatically emailed to each developers. This infrastructure provides: 1- Automation via Parasoft C++test 2- Fully ingrains testing practices into the developer ’ s daily workflow to ensure that these practices are non intrusively introduced, and effectively implemented across the whole team 3- Provides a means to be able to monitor your development process, identify potential issues, and in turn be able to learn from past mistakes and improve your process.  Essential elements of ADP.
  • Project Manager, Developer, QA Manager, Project Architect
  • Introduction to Parasoft C++TEST

    1. 1. Parasoft ADP and C++test Ellen Chiang
    2. 2. Parasoft Corporate Overview <ul><li>Founded in 1987, privately held </li></ul><ul><li>Headquarters in Monrovia, CA </li></ul><ul><li>300+ employees worldwide </li></ul><ul><li>16 locations worldwide </li></ul><ul><li>Technical innovator </li></ul><ul><ul><li>15 US patents for software technology </li></ul></ul><ul><li>10,000+ customers worldwide, over 58% of Fortune 500 clients </li></ul><ul><li>Gartner Magic Quadrant: Visionary </li></ul>
    3. 3. The Reality … Most errors are found after development… SOURCE: NIST Report,2002 Errors Found Post-Development Errors Found Pre-Development Time in Actual Development Time Looking for Errors Although developers spend a majority of their time looking for errors…
    4. 4. Defect Analysis and Reproduction in Different Phases Coding Production Test System Test Integration Test Unit Test
    5. 5. Improving Quality: Typical Approach
    6. 6. Automated Defect Prevention <ul><li>Methodology and solution for detecting an error once and implementing practices to prevent the class of error from ever happening again. </li></ul><ul><ul><li>Isolation and identification of errors early in the process </li></ul></ul><ul><ul><li>Uses information gained to prevent reoccurring software bugs and to progressively improve the development process </li></ul></ul><ul><ul><li>Refocuses teams from testing for and fixing bugs to improving the software lifecycle and delivering higher quality software. </li></ul></ul><ul><ul><li>Promoting team behavior and integrating into the development process </li></ul></ul>
    7. 7. Parasoft Delivers Automated Error Prevention <ul><li>Automation </li></ul><ul><ul><li>Automatically perform key testing practices </li></ul></ul><ul><ul><ul><li>Coding Standards enforcement </li></ul></ul></ul><ul><ul><ul><li>Unit Testing (White/Black box, regression testing) </li></ul></ul></ul><ul><ul><ul><li>Run Time Error Detection </li></ul></ul></ul><ul><li>Team Infrastructure </li></ul><ul><ul><li>Address key error prevention roles </li></ul></ul><ul><ul><li>Automated tools configured for group behavior </li></ul></ul><ul><ul><li>Integrate with existing technologies </li></ul></ul><ul><ul><li>Error prevention infrastructure to capture and process information </li></ul></ul><ul><ul><li>Use feedback to improve software lifecycle and prevent errors </li></ul></ul>
    8. 8. What is C++Test? <ul><li>C++Test improves C/C++ developer and team productivity by automating a number of high-impact techniques for error prevention and detection at early stages of development process . </li></ul>
    9. 9. What Is C++Test? <ul><li>C++Test is a comprehensive tool for automated static and dynamic analysis of C and C++ code: </li></ul><ul><ul><li>Coding policy enforcement </li></ul></ul><ul><ul><li>Data flow analysis </li></ul></ul><ul><ul><li>Automated code review </li></ul></ul><ul><ul><li>Automated unit testing </li></ul></ul><ul><ul><li>Execution of unit tests on host and target </li></ul></ul><ul><ul><li>Automated Regression testing </li></ul></ul><ul><ul><li>Test/code coverage </li></ul></ul><ul><ul><li>Runtime Error Detection </li></ul></ul><ul><ul><li>Team deployment/process integration </li></ul></ul><ul><ul><li>ADP Quality Framework </li></ul></ul>Static Analysis Dynamic Analysis
    10. 10. C++Test Core Features and Benefits <ul><li>Coding Standards Static Analysis </li></ul><ul><li>Automatic Bug Detection </li></ul><ul><li>Automated Code Review </li></ul><ul><li>Automated Unit Testing </li></ul><ul><li>Automated Regression Testing </li></ul><ul><li>Embedded Testing </li></ul><ul><li>Runtime Error Detection </li></ul>
    11. 11. C++Test Core Features and Benefits <ul><ul><li>Reads source code and checks code against select coding standard rules. </li></ul></ul><ul><ul><li>Automatically enforces 1400+ industry best practices </li></ul></ul><ul><ul><li>Finds complex code through Metrics rules </li></ul></ul><ul><ul><li>“ RuleWizard ” feature for creating custom rules without scripting </li></ul></ul><ul><ul><li>Early identification of coding errors </li></ul></ul><ul><ul><li>Supports the group, project, division, as well as corporate standards </li></ul></ul><ul><ul><li>Check for instances of known bugs to prevent reoccurring issues </li></ul></ul><ul><li>Coding Standards Static Analysis </li></ul><ul><li>Automatic Bug Detection </li></ul><ul><li>Automated Code Review </li></ul><ul><li>Automated Unit Testing </li></ul><ul><li>Automated Regression Testing </li></ul><ul><li>Embedded Testing </li></ul><ul><li>Runtime Error Detection </li></ul>
    12. 12. C++Test Core Features and Benefits <ul><li>Coding Standards Adherence </li></ul><ul><li>Automatic Bug Detection </li></ul><ul><li>Automated Code Review </li></ul><ul><li>Automated Unit Testing </li></ul><ul><li>Automated Regression Testing </li></ul><ul><li>Embedded Testing </li></ul><ul><li>Runtime Error Detection </li></ul><ul><ul><li>“ BugDetective ” feature: unique static technology based on flow analysis </li></ul></ul><ul><ul><li>Automatically finds runtime bugs without writing, executing or reviewing test cases </li></ul></ul><ul><ul><li>Enables early and effortless detection of critical runtime errors. </li></ul></ul><ul><ul><li>Pinpoints bugs in complex execution paths that span multiple methods, class files and packages </li></ul></ul>
    13. 13. C++Test Core Features and Benefits <ul><li>Coding Standards Adherence </li></ul><ul><li>Automatic Bug Detection </li></ul><ul><li>Automated Code Review </li></ul><ul><li>Automated Unit Testing </li></ul><ul><li>Automated Regression Testing </li></ul><ul><li>Embedded Testing </li></ul><ul><li>Runtime Error Detection </li></ul><ul><ul><li>“ Code Review” module for automation of code review process </li></ul></ul><ul><ul><li>Facilitates identification of high-level problems in code </li></ul></ul><ul><ul><li>Saves time: conduct code reviews without face-to-face meetings </li></ul></ul><ul><ul><li>Promotes flexible scheduling </li></ul></ul><ul><ul><li>Supports distributed development </li></ul></ul><ul><ul><li>Ensures all new code gets reviewed and all identified issues are resolved </li></ul></ul>
    14. 14. C++Test Core Features and Benefits <ul><li>Coding Standards Adherence </li></ul><ul><li>Automatic Bug Detection </li></ul><ul><li>Automated Code Review </li></ul><ul><li>Automated Unit Testing </li></ul><ul><li>Automated Regression Testing </li></ul><ul><li>Embedded Testing </li></ul><ul><li>Runtime Error Detection </li></ul><ul><ul><li>Auto generates test cases to exercise code. </li></ul></ul><ul><ul><li>Auto generates stubs for external calls </li></ul></ul><ul><ul><li>White box testing: Automatically tests robustness of code construction </li></ul></ul><ul><ul><li>Black box testing: Verifies functionality of the code </li></ul></ul><ul><ul><li>Reports 6 metrics of code coverage </li></ul></ul><ul><ul><li>Easily extend test cases to increase code coverage </li></ul></ul>
    15. 15. C++Test Core Features and Benefits <ul><li>Coding Standards Adherence </li></ul><ul><li>Automatic Bug Detection </li></ul><ul><li>Automated Code Review </li></ul><ul><li>Automated Unit Testing </li></ul><ul><li>Automated Regression Testing </li></ul><ul><li>Embedded Testing </li></ul><ul><li>Runtime Error Detection </li></ul><ul><ul><li>Prior test run results kept with project </li></ul></ul><ul><ul><li>Click of button runs regression tests </li></ul></ul><ul><ul><li>Easily configured for nightly builds </li></ul></ul><ul><ul><li>Guarantees quality of code as it evolves </li></ul></ul>
    16. 16. C++Test Core Features and Benefits <ul><li>Coding Standards Adherence </li></ul><ul><li>Automatic Bug Detection </li></ul><ul><li>Automated Code Review </li></ul><ul><li>Automated Unit Testing </li></ul><ul><li>Automated Regression Testing </li></ul><ul><li>Embedded Testing </li></ul><ul><li>Runtime Error Detection </li></ul><ul><ul><li>Supports host-based testing: perform static analysis and unit testing as the code is developed </li></ul></ul><ul><ul><li>Early testing exposes majority of the problems with the application logic before deployment on the target. </li></ul></ul><ul><ul><li>Supports testing on the target: Export test harness and cross-compile for execution of test cases directly on the target. </li></ul></ul><ul><ul><li>Expose issues that may only occur on the target. </li></ul></ul><ul><ul><li>Develop/maintain one set of tests which can be reused for execution on multiple targets. </li></ul></ul>
    17. 17. C++Test Core Features and Benefits <ul><li>Coding Standards Adherence </li></ul><ul><li>Automatic Bug Detection </li></ul><ul><li>Automated Code Review </li></ul><ul><li>Automated Unit Testing </li></ul><ul><li>Automated Regression Testing </li></ul><ul><li>Embedded Testing </li></ul><ul><li>Runtime Error Detection </li></ul><ul><ul><li>Automatically exposes defects that occur as the application is exercised-including memory leaks, null pointers, uninitialized memory, buffer overflows, and more. </li></ul></ul><ul><ul><li>The instrumentation used to perform this runtime error detection is lightweight and suitable for running on the target board, simulator, or host for embedded testing. </li></ul></ul><ul><ul><li>C++test provides complete information indicating the root causes of the errors along with a detailed stack trace to facilitate issue remediation . </li></ul></ul>
    18. 18. Parasoft Automated Technologies: Testing throughout the SDLC Enforce Correct Coding and Security Policy Auto Generate Unit Tests … with stubs Flow Analysis Tests Regular Regression Testing Application Testing Code Review Code Review <ul><ul><li>Runtime Error Detection </li></ul></ul>Coding Production Test System Test Integration Test Unit Test
    19. 19. C++Test Configuration <ul><li>Professional Edition </li></ul><ul><ul><li>Developer desktop usage </li></ul></ul><ul><li>Architect Edition </li></ul><ul><ul><li>RuleWizard for custom rule creation </li></ul></ul><ul><li>Server Edition </li></ul><ul><ul><li>Command line for integration with nightly build systems </li></ul></ul><ul><ul><li>BugDetective for detection of runtime issues </li></ul></ul><ul><ul><li>Provides reports to development team </li></ul></ul><ul><ul><li>Includes Team Server Module </li></ul></ul><ul><li>Team Server </li></ul><ul><ul><li>Stores and distributes rules, test configurations, and reports </li></ul></ul><ul><ul><li>Standardizes testing methods across a team </li></ul></ul>
    20. 20. Team-wide Deployment and Workflow Scheduled Test Server Developer Machines Architect / Technical lead SCM Source Control code & tests Team Server Team Server Test Results Professional Editions Server Edition (Batch Mode) Architect Edition Team Coding Standards Nightly Results
    21. 21. C++Test benefits <ul><li>Automatically find software defects using static analysis of code </li></ul><ul><li>Implement, configure, and automatically enforce team-or company-specific coding standards and guidelines </li></ul><ul><li>Ensure that code is reviewed for higher level issues by facilitating and automating the code review process </li></ul><ul><li>Automatically generate and validate baseline unit and module tests </li></ul><ul><li>Ensure code integrity via automated regression testing </li></ul><ul><li>Assess effectiveness of existing tests by providing detailed coverage information </li></ul><ul><li>Improve efficiency and test effectiveness by promoting “team behavior” and integrating key test practices into the development process </li></ul>
    22. 22. Summary <ul><li>Reduce cost by finding errors early in the development process </li></ul><ul><li>Improve efficiency </li></ul><ul><ul><li>Reduce testing time </li></ul></ul><ul><ul><li>Reduce testing effort </li></ul></ul><ul><ul><li>Reduce number of bugs introduced into QA by catching errors early in the development process </li></ul></ul><ul><li>Increase stability and confidence </li></ul><ul><ul><li>Increase testing completeness </li></ul></ul><ul><ul><li>Improve visibility into the development process </li></ul></ul><ul><ul><li>Improve efficiency and effectiveness of development process </li></ul></ul>