Successfully reported this slideshow.

20080622 bupt qa_opensource


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

20080622 bupt qa_opensource

  1. 1. Quality Assurance at Novell for Linux Products Oliver Ries Director OPS Quality Assurance
  2. 2. Agenda • How Open Source QA is different • Aspects of OpenSource QA at Novell – Test Approach – Test Environments – Test Automation – Maintenance Testing • openSUSE & QA – future plans 2 © Novell Inc. All rights reserved
  3. 3. Software Testing as in the Books Acceptance Testing Product Regression Testing Stress Testing System Testing Component Testing Integration Testing Unit Testing Code 3 © Novell Inc. All rights reserved
  4. 4. Multi Phase Test Approach UNIT TEST -Package dependencies -Correct libraries - Integrated tests and sanity checks Code Autobuild COMPONENT - Functional tests for all packages TEST - Test Cases from PDB - Priority Based (CORE->ADDON) Package TIDB/Cerberus Harness FEATURE TEST - Manual test of all features included in the feature document Feature Doc TIDB SYSTEM - Stress and reliability tests TEST - Industry test suites - Certification tests Installed Systems Cerberus REGRESSION - Regression of all automated tests TESTS - Validation of fixed bugs - Regression of failed feature tests TIDB/Cerberus automated manual/interactive 4 © Novell Inc. All rights reserved
  5. 5. Multi Phase Test Approach II Unit Test Component Feature Test Test Packages Beta 1 Component Test Beta 2 Special Focus: SystemTest Beta 3 - Kernel - Network Stress - SysMgmt - Reliability - GUI - Stability - Scalability Regression Beta 4 Test System test RC 1 Cert Tests RC 2 5 © Novell Inc. All rights reserved
  6. 6. Test Environment
  7. 7. Overview of the Test Environment Engineer Test Machines PDB i386 x86-64 KOTD Feature (kernel of the day) Document test machines TIDB / i386 x86-64 IA64 PPC s390 TESTLINK QA LAB Network Autobuild QA Kernel Lab and ISV Test (Oracle, Veritas,...) Reference Hosts SLES9 SLES8 NLD SLOX 7 © Novell Inc. All rights reserved
  8. 8. Test Automation
  9. 9. Automation Architecture TXT, HTML, CSV, SQL, XML Create reports create reports start check and update CTCS SQL db "FAILED" "PASSED" Test Control Files ... Apache test Samba test Bind test Mount test ... test script script script script script Start Results Httpd Smbd named Mount /dev/nnn App 1 Nmbd mount -t nfs App 2 Startssl Smbclient App 3 Application Layer Smbmount 9 © Novell Inc. All rights reserved
  10. 10. Maintenance Update Testing
  11. 11. MainCoord initiates SWAMP QA checks on incl. Prio, etc. Patch is build and pkg. available needed arch and products SLES 8 SLES 8 SLES 9 OES ... SLES N i386 s390 x86_64 i386 <ARCH> Update Test Update Test Update Test Update Test Update Test Bug Fix Test Bug Fix Test Bug Fix Test Bug Fix Test Bug Fix Test Feature Test Feature Test Feature Test Feature Test Feature Test Feature Test for Type A/B by OES team for ISV Tests ISV Tests ISV Tests Type C ISV Tests ? Reject Collect Results NO Tests passed ? Alert TPMs, Sec Team, YES Support (NTS), TAMs, Approve Send report to ISVs 11 © Novell Inc. All rights reserved
  12. 12. openSUSE & QA
  13. 13. How can the community contribute? • bugzilla / testopia – main interface and used by Novell internally as well • Novell is opensourcing test tools / infrastructure / documentation – QA code will be available in the Build Service – Build Service will be able to run tests after a build job is done – easier integration for Software maintainers • building a QA community – awards & prizes for the best QA contributors (reports, test cases, ...) • better cooperation between the community and Novell – QA community manager 13 © Novell Inc. All rights reserved
  14. 14. Testopia (off-line) Demo
  15. 15. 15 © Novell Inc. All rights reserved
  16. 16. 16 © Novell Inc. All rights reserved
  17. 17. 17 © Novell Inc. All rights reserved
  18. 18. 18 © Novell Inc. All rights reserved
  19. 19. 19 © Novell Inc. All rights reserved
  20. 20. 20 © Novell Inc. All rights reserved
  21. 21. 21 © Novell Inc. All rights reserved
  22. 22. 22 © Novell Inc. All rights reserved
  23. 23. 23 © Novell Inc. All rights reserved
  24. 24. 24 © Novell Inc. All rights reserved
  25. 25. Questions, Feedback, Discussions
  26. 26. Thank you for your attention 26 © Novell Inc. All rights reserved
  27. 27. Unpublished Work of Novell, Inc. All Rights Reserved. This work is an unpublished work and contains confidential, proprietary, and trade secret information of Novell, Inc. Access to this work is restricted to Novell employees who have a need to know to perform tasks within the scope of their assignments. No part of this work may be practiced, performed, copied, distributed, revised, modified, translated, abridged, condensed, expanded, collected, or adapted without the prior written consent of Novell, Inc. Any use or exploitation of this work without authorization could subject the perpetrator to criminal and civil liability. General Disclaimer This document is not to be construed as a promise by any participating company to develop, deliver, or market a product. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. Novell, Inc. makes no representations or warranties with respect to the contents of this document, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. The development, release, and timing of features or functionality described for Novell products remains at the sole discretion of Novell. Further, Novell, Inc. reserves the right to revise this document and to make changes to its content, at any time, without obligation to notify any person or entity of such revisions or changes. All Novell marks referenced in this presentation are trademarks or registered trademarks of Novell, Inc. in the United States and other countries. All third-party trademarks are the property of their respective owners.