[Europe   merge world tour] Coverity Development Testing
Upcoming SlideShare
Loading in...5
×
 

[Europe merge world tour] Coverity Development Testing

on

  • 153 views

Presentation from Coverity at the European Merge World Tour - Coverity Development Testing

Presentation from Coverity at the European Merge World Tour - Coverity Development Testing

Statistics

Views

Total Views
153
Views on SlideShare
153
Embed Views
0

Actions

Likes
0
Downloads
2
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

[Europe   merge world tour] Coverity Development Testing [Europe merge world tour] Coverity Development Testing Presentation Transcript

  • Coverity Development Testing Accelerating Risk Mitigation through Continuous Integration and Development Testing
  • “Software is Eating the World” Software - Marc Andreessen Health Financial Communications SCM / Logistics Enterprise Mobile 81% of business leaders believe technology is a fundamental element of their business model Over 60 million tablets and 175 million smart phones will be in the workplace by the end of 2012 By 2016, open source software will be included in mission- critical applications within 99% of Global 2000 enterprises Automotive
  • Development Testing … … is transforming software development by: Reducing operational costs Accelerating development and time to market Protecting brands from catastrophic failure
  • Why All the Risk? Software Complexity and Speed have Outpaced Legacy Testing Methods Development Testing Software Complexity Time to Market Testing MethodsSecurity Testing Functional Testing Performance Testing Manual Testing
  • Fewer defects escape development Design Development Quality & Security Assurance Product Release & Management Development Testing Transform software testing, from reactive to proactive
  • Transformation Maturity Model Level 1 Automatic Defect Detection No new defects introduced. Level 3 Developer Workflow Optimization Feeding all components into the developer workflow Level 4 Code Governance Establish source code acceptance criteria Complete Enterprise Code Assurance All critical code and code impacted by change is tested IntegrationintoSDLCHigh Development Testing Adoption High Level 2 Identification of Residual Risk Ensure critical code is prioritized & tested
  • How Coverity Static Analysis Works Mimicks the behavior of dozens of compilers Integrates with existing build systems Statically tests all execution paths Finds defects and inconsistent coding patterns AnalyzeBuild Explains the location and root cause of defects Manage and share triage of defects across teams Present & Manage
  • Meaningful, real results Focus on finding real defects, not style violations or superficial issues. Over 12 years of experience analyzing open source and commercial code. Industry-leading low false positive/negative rate False positive rates typically below 15% False positives waste time, hinder adoption, and reduce trust in the results. Broadest Checker Library + Deepest Algorithms Optimal balance of breadth, depth, and scalability to large code bases. High Quality Results
  • Sample Project: PostgreSQL Defects Fixed in 2012 per Category Category # Defects Impact Memory  –  corrup,ons   20   High   Memory  -­‐  illegal  accesses   10   High   Resource  leaks   43   High   Unini,alized  variables   10   High   API  usage  errors   1   Medium   Control  flow  issues   4   Medium   Error  handling  issues   14   Medium   Incorrect  expression   3   Medium   Insecure  data  handling   24   Medium   Integer  handling  issues   8   Medium   Null  pointer  dereferences   43   Medium   Code  maintainability  issues   58   Low   Security  best  prac,ces  viola,ons   15   Low   Grand  Total   253   •  ~20 Developers •  Weekly Build •  680k LOC •  False Positive Rate: 11.1% •  Defect Density: 0,273
  • We Find Critical Defects •  Tomcat Webserver 5.5.17 •  Among several hundred defects, we found a “reverse lock bug” that can lead to deadlock of the entire server
  • Focus testing time where it matters … don’t waste time writing tests you don’t need Test Advisor Improving Unit Testing Effectiveness and Efficiency High Risk Code High Risk Code
  • Risk Mitigation Architecture Test Advice Actionable work items to address risk due to inadequate testing Test Policy Evaluation •  Critical code analysis •  Change impact analysis •  Test execution analysis Test Monitoring Code Ownership and Change History Static Code Analysis Customized Test Policy
  • Move Quality into the Inner Loop of Development Code Build Test Nightly Build Continuous Integration Finding and Fixing Quality Defects
  • QA Development Testing Workflow Code Check In Development Security Audit Static Analysis Results Nightly/Continuous Build Regression Test •  Built into development process •  Retesting minimized •  Immediately actionable by developers •  Reduces burden on auditing team Developer QA Security
  • Issue Responsibility Is Critical
  • Ingredients for Success Code Build Test Nightly Build Continuous Integration High-Fidelity Code Compilation High- Performance Analysis Low False Positive Rate Detecting Critical Defects Easy Defect Navigation and Comprehension Comprehensive Triage and Remediation Management Visibility and Governance Team Collaboration
  • Governance with Metrics Automated high-fidelity analysis on daily basis 18 Fast and educated triage of results to categorize and prioritize issuesAccurate Data Precise actions based on comprehensive data analysis Trusted Data
  • Policy Definition and Monitoring Definition of organizational-wide policies for code quality Aggregated sanity view of code by component, team, supplier
  • Supplier SLA Enforcement Supplier self-certification based upon policies
  • Transformation Maturity Model Level 1 Automatic Defect Detection No new defects introduced. Level 3 Developer Workflow Optimization Feeding all components into the developer workflow Level 4 Code Governance Establish source code acceptance criteria Complete Enterprise Code Assurance All critical code and code impacted by change is tested IntegrationintoSDLCHigh Development Testing Adoption High Level 2 Identification of Residual Risk Ensure critical code is prioritized & tested
  • Coverity Development Testing Platform Security Advisor Test Advisor Analysis Packs Coverity SAVE™ Static Analysis Verification Engine SDLC Integrations Policy Manager Quality Advisor Architecture Analysis Dynamic Analysis FindBugs™ Analysis Analysis Integration Toolkit Coverity Connect Test Execution Third Party Metrics Build/ Continuous Integration HP ALM IDE Code Coverage Defect Tracking SCM
  • ü  Proven significant operational cost reductions ü  Metric visibility of code estate onshore and offshore ü  Proven history of finding crash causing or unexpected behavior causing defects ü  Process Improvement of the Application Lifecycle Management Coverity Summary
  • Questions ?!?! ! !