Diab Compiler Quality Overview

  • 425 views
Uploaded on

Diab Compiler is the best in the Market, find out why...

Diab Compiler is the best in the Market, find out why...

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
425
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
6
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Wind River Diab Compiler –Testing and Quality
  • 2. Introduction  The compiler is a key tool for any safety critical applications  Compiler code generation issues can be very costly for customers with fielded products  Diab Compiler team strives to provide maximum visibility of defects found in the compiler – Access to information is key to assisting customers investigate and uncover product impacts before they happen in the field – This openness risks negatively impacting perception of customers  Wind River has built up a significant test harness to validate each new compiler release – Always room for further improvements in testing and new investments – Recently Wind River has been receiving more reported defects from our Automotive customers – Wind River is investigating why this is happening even when we have been increasing investment in compiler testing2 | © 2011 Wind River. All Rights Reserved.
  • 3. Challenges of Compiler Testing  Compiler testing is complex  Virtually infinite number of option combinations and code that can be run through the compiler  Testing must be done across 10 major processor architectures and almost 100 sub-architectures  Wide range of customer markets and applications  Determining a representative set of tests is challenging3 | © 2011 Wind River. All Rights Reserved.
  • 4. Diab Compiler Testing  Wind River is running millions of compiler tests – Standards test suites: e.g Plum Hall, Modena, Paranoia  Added ACE SuperTest C/C++ compiler test and validation suite – NEW investment in 2011 – World’s largest compiler test suite with over 2 Million tests – Benchmarks: EEMBC, Nullstone, PowerStone, customer supplied benchmarks  Added D-Bench in 2011 – Regression tests (6500+ test cases)  Verification tests for fixed defects are continuously be added to the regression test suite  Standard compiler tests may not always be representative of Automotive customers use cases4 | © 2011 Wind River. All Rights Reserved.
  • 5. Getting Closer to the Customer  Customer specific tests are always better than off-the shelf application tests – Actual customer use cases of the compiler  Testing the compiler using customer supplied tests/benchmarks leads to improved test coverage of the compiler for customers – Code from key Automotive customers is already used in Diab testing  Wind River would be happy to incorporate more tests from customers into the test harness – However, customers aren’t always willing to share their source code and WR can’t always verify correct execution of customer applications  Wind River can test the compiler using customer option sets used in various customer projects – There is certainly an opportunity to review the current test process and ensure we are aligned with customer option sets5 | © 2011 Wind River. All Rights Reserved.
  • 6. In Quality Improvement Plan Development  Investigate root cause of recent customer reported defects – Review defect reports from Automotive customers from last two years – Look for patterns that indicate where updates are needed to Diab test harness – Review tested option sets to ensure they continue to be in-line with customer use cases  Acquire code coverage data from key customers – Compare current compiler test suite coverage to help guide new test investments  Look for additional compiler test suites to add value to current test harness – Assuming low impact based on generic nature of available test suites and current coverage from existing suites being used for Diab  Acquire more “Automotive” source code from customers and partners – Need help from customers and partners in the Automotive industry – Agree with stakeholders on representative code for compiler testing – Currently discussing with customers and topic for upcoming German “road trip”6 | © 2011 Wind River. All Rights Reserved.
  • 7. Sharing Information with Customers  Wind River’s primary customer support and communication tool is the Wind River On-line Support web portal (OLS)  OLS provides information on Diab compiler defects for customers  The defect tool has some limitations such as advanced sorting – Some of these limitations are due to OLS – Some of these limitations are due to the limits of WR’s underlying ClearQuest defect tracking database  Improvements are being investigated – Better export to Excel is currently being tested – How to better characterize and sort by arch and versions is being studied7 | © 2011 Wind River. All Rights Reserved.