20080622 bupt qa_opensource


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

20080622 bupt qa_opensource

  1. 1. Quality Assurance at Novellfor Linux ProductsOliver Riesories@novell.comDirector OPS Quality Assurance
  2. 2. Agenda • How Open Source QA is different • Aspects of OpenSource QA at Novell – Test Approach – Test Environments – Test Automation – Maintenance Testing • openSUSE & QA – future plans2 © Novell Inc. All rights reserved
  3. 3. Software Testing as in the Books Acceptance Testing Product Regression Testing Stress Testing System Testing Component Testing Integration Testing Unit Testing Code3 © Novell Inc. All rights reserved
  4. 4. Multi Phase Test ApproachUNIT TEST -Package dependencies -Correct libraries - Integrated tests and sanity checks Code AutobuildCOMPONENT - Functional tests for all packagesTEST - Test Cases from PDB - Priority Based (CORE->ADDON) Package TIDB/Cerberus HarnessFEATURETEST - Manual test of all features included in the feature document Feature Doc TIDBSYSTEM - Stress and reliability testsTEST - Industry test suites - Certification tests Installed Systems CerberusREGRESSION - Regression of all automated testsTESTS - Validation of fixed bugs - Regression of failed feature tests TIDB/Cerberus automated manual/interactive 4 © Novell Inc. All rights reserved
  5. 5. Multi Phase Test Approach II Unit Test Component Feature Test Test PackagesBeta 1 Component TestBeta 2 Special Focus: SystemTestBeta 3 - Kernel - Network Stress - SysMgmt - Reliability - GUI - Stability - Scalability RegressionBeta 4 Test System testRC 1 Cert TestsRC 25 © Novell Inc. All rights reserved
  6. 6. Test Environment
  7. 7. Overview of the Test Environment Engineer Test Machines PDB i386 x86-64 KOTD Feature (kernel of the day) Document test machines TIDB / i386 x86-64 IA64 PPC s390 TESTLINK QA LAB Network Autobuild QA Kernel Lab and ISV Test (Oracle, Veritas,...) Reference Hosts SLES9 SLES8 NLD SLOX7 © Novell Inc. All rights reserved
  8. 8. Test Automation
  9. 9. Automation Architecture TXT, HTML, CSV, SQL, XML Create reports create reports start check and update CTCS SQL db "FAILED" "PASSED"Test Control Files ... Apache test Samba test Bind test Mount test ... test script script script script script Start Results Httpd Smbd named Mount /dev/nnn App 1 Nmbd mount -t nfs App 2 Startssl Smbclient App 3Application Layer Smbmount9 © Novell Inc. All rights reserved
  10. 10. Maintenance Update Testing
  11. 11. MainCoord initiates SWAMP QA checks onincl. Prio, etc. http://w2d.suse.de/abuildstat/patch-statusPatch is build and pkg. available needed arch and productsSLES 8 SLES 8 SLES 9 OES ... SLES Ni386 s390 x86_64 i386 <ARCH>Update Test Update Test Update Test Update Test Update TestBug Fix Test Bug Fix Test Bug Fix Test Bug Fix Test Bug Fix TestFeature Test Feature Test Feature Test Feature Test Feature Test Feature Test for Type A/B by OES team forISV Tests ISV Tests ISV Tests Type C ISV Tests ? Reject Collect Results NO Tests passed ? Alert TPMs, Sec Team, YES Support (NTS), TAMs, Approve Send report to ISVs qa@suse.de11 © Novell Inc. All rights reserved
  12. 12. openSUSE & QA
  13. 13. How can the community contribute? • bugzilla / testopia – main interface and used by Novell internally as well • Novell is opensourcing test tools / infrastructure / documentation – QA code will be available in the Build Service – Build Service will be able to run tests after a build job is done – easier integration for Software maintainers • building a QA community – awards & prizes for the best QA contributors (reports, test cases, ...) • better cooperation between the community and Novell – QA community manager13 © Novell Inc. All rights reserved
  14. 14. Testopia (off-line) Demo
  15. 15. 15 © Novell Inc. All rights reserved
  16. 16. 16 © Novell Inc. All rights reserved
  17. 17. 17 © Novell Inc. All rights reserved
  18. 18. 18 © Novell Inc. All rights reserved
  19. 19. 19 © Novell Inc. All rights reserved
  20. 20. 20 © Novell Inc. All rights reserved
  21. 21. 21 © Novell Inc. All rights reserved
  22. 22. 22 © Novell Inc. All rights reserved
  23. 23. 23 © Novell Inc. All rights reserved
  24. 24. 24 © Novell Inc. All rights reserved
  25. 25. Questions, Feedback, Discussions
  26. 26. Thank you for your attention26 © Novell Inc. All rights reserved
  27. 27. Unpublished Work of Novell, Inc. All Rights Reserved.This work is an unpublished work and contains confidential, proprietary, and trade secret information of Novell, Inc.Access to this work is restricted to Novell employees who have a need to know to perform tasks within the scopeof their assignments. No part of this work may be practiced, performed, copied, distributed, revised, modified,translated, abridged, condensed, expanded, collected, or adapted without the prior written consent of Novell, Inc.Any use or exploitation of this work without authorization could subject the perpetrator to criminal and civil liability.General DisclaimerThis document is not to be construed as a promise by any participating company to develop, deliver, or market aproduct. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon inmaking purchasing decisions. Novell, Inc. makes no representations or warranties with respect to the contentsof this document, and specifically disclaims any express or implied warranties of merchantability or fitness for anyparticular purpose. The development, release, and timing of features or functionality described for Novell productsremains at the sole discretion of Novell. Further, Novell, Inc. reserves the right to revise this document and tomake changes to its content, at any time, without obligation to notify any person or entity of such revisions orchanges. All Novell marks referenced in this presentation are trademarks or registered trademarks of Novell, Inc.in the United States and other countries. All third-party trademarks are the property of their respective owners.