Your SlideShare is downloading. ×
0
Test What Matters Most
Test What Matters Most
Test What Matters Most
Test What Matters Most
Test What Matters Most
Test What Matters Most
Test What Matters Most
Test What Matters Most
Test What Matters Most
Test What Matters Most
Test What Matters Most
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Test What Matters Most

261

Published on

Presentation that giv

Presentation that giv

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
261
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Test What Matters Most Develop Testing: Totaalvoetbal for Software Development 12th September, Amsterdam Johnny Willemsen, CTO Remedy IT jwillemsen@remedy.nl
  • 2. Remedy IT Company of specialists focused on middleware and component technologies Dedicated to open standards and preferable open source Actively involved in the development of ACE/TAO/CIAO/DAnCE/OpenDDS open source software suite Copyright © 20132
  • 3. What is ACE? ADAPTIVE Communication Environment Open-source object-oriented (OO) framework that implements many core patterns for concurrent distributed communication software Strong adoption in the telecom, medical, aerospace, defense and financial services industries TAO, CIAO, DAnCE, and OpenDDS are products that are created on top of ACE and form together the ATCDO open source suite. 3 Copyright © 2013
  • 4. ACE and Coverity Scan What is Coverity Scan? • Free static analysis for open source projects • Find and fix defects in C/C++ or Java • Tests every line of code and potential execution path • Explains root cause of each defect making it easy to fix bugs ACE and Coverity Scan • Joined in April 2011 • Fixed 3263 defects as of September 2013 • 1795 defects outstanding • Defect density of 0.24 Copyright © 20134
  • 5. ACE testing challenges Goal: reduce risk through actionable, efficient testing Large, decentralized project with distributed development teams: • 7.8 million lines of C++ code • 20 primary developers, 100 developers committing code to the repository, 2375 developers contributed Testing a priority, but not enforceable, so most code remains untested • Current overall code coverage is 27% (57% for critical components) • Over 275,000 functions not fully tested including examples and other code that does not require testing Copyright © 20135
  • 6. Creating an ENFORCEABLE test policy Many concerns before the project began • Where do I start, and how do I prioritize, with so much untested code? • How much real risk is there that I need to address ASAP? • How do I hold developers accountable for testing their own code? • How do I ensure code is tested as new features are developed? Copyright © 20136
  • 7. Creating an ENFORCEABLE test policy ACE developed a basic Coverity Test Advisor policy to address these concerns • Filtering rules to determine which code is important to test • Analyzing untested code against those rules to identify missing tests and prioritize them Copyright © 20137
  • 8. 4 Step policy with a big impact 8 Step Remaining Violations Testing Policy 0 275,313 Simple line coverage All insufficiently- or un-tested functions 1 12,833 Focus on core components Premise: certain components are more important to test than others 2 2,967 Only called code (ignore uncalled) Premise: we only write tests for important functions 3 2,757 Exclude debug, logging code Premise: diagnostic messages are not important to test 4 2,588 Since 6.1: 605 Exclude error cases, prioritize violations by release Premise: testing error cases provides minimal value and it is often difficult to trigger this code Note: These numbers are for the exact same code Copyright © 2013
  • 9. Using Coverity ACE performs a weekly build dedicated to Coverity scan • Takes 27 hours to compile all source code with gcc • Takes 19 hours to analyze using Coverity scan Coverity scan analyzes the code and publishes the Scan results New issues are emailed to all developers The ACE build czar will enforce that any new issue gets resolved by the original developer Supports our goal that quality increases weekly and never decreases 9 Copyright © 2013
  • 10. Addressing the issues Test violations in the core code are most important Try to resolve high priority violations with funding through maintenance and support When resolving bugs, test violations are used to focus the new unit tests on what is really needed When adding a feature to ACE, Test Advisor is used to determine that the feature is well tested After a decrease in number of test violations, the basic Test Advisor policy can be relaxed 10 Copyright © 2013
  • 11. Want to know more? Remedy IT Office: +31-10-5220139 Website: www.remedy.nl/www.theaceorb.nl Slideshare: www.slideshare.net/RemedyIT Twitter: @RemedyIT Join Coverity Scan scan.coverity.com Copyright © 201311

×