20080622 bupt qa_opensource
Upcoming SlideShare
Loading in...5

20080622 bupt qa_opensource






Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

20080622 bupt qa_opensource 20080622 bupt qa_opensource Presentation Transcript

  • Quality Assurance at Novellfor Linux ProductsOliver Riesories@novell.comDirector OPS Quality Assurance
  • Agenda • How Open Source QA is different • Aspects of OpenSource QA at Novell – Test Approach – Test Environments – Test Automation – Maintenance Testing • openSUSE & QA – future plans2 © Novell Inc. All rights reserved
  • Software Testing as in the Books Acceptance Testing Product Regression Testing Stress Testing System Testing Component Testing Integration Testing Unit Testing Code3 © Novell Inc. All rights reserved
  • Multi Phase Test ApproachUNIT TEST -Package dependencies -Correct libraries - Integrated tests and sanity checks Code AutobuildCOMPONENT - Functional tests for all packagesTEST - Test Cases from PDB - Priority Based (CORE->ADDON) Package TIDB/Cerberus HarnessFEATURETEST - Manual test of all features included in the feature document Feature Doc TIDBSYSTEM - Stress and reliability testsTEST - Industry test suites - Certification tests Installed Systems CerberusREGRESSION - Regression of all automated testsTESTS - Validation of fixed bugs - Regression of failed feature tests TIDB/Cerberus automated manual/interactive 4 © Novell Inc. All rights reserved
  • Multi Phase Test Approach II Unit Test Component Feature Test Test PackagesBeta 1 Component TestBeta 2 Special Focus: SystemTestBeta 3 - Kernel - Network Stress - SysMgmt - Reliability - GUI - Stability - Scalability RegressionBeta 4 Test System testRC 1 Cert TestsRC 25 © Novell Inc. All rights reserved
  • Test Environment
  • Overview of the Test Environment Engineer Test Machines PDB i386 x86-64 KOTD Feature (kernel of the day) Document test machines TIDB / i386 x86-64 IA64 PPC s390 TESTLINK QA LAB Network Autobuild QA Kernel Lab and ISV Test (Oracle, Veritas,...) Reference Hosts SLES9 SLES8 NLD SLOX7 © Novell Inc. All rights reserved
  • Test Automation
  • Automation Architecture TXT, HTML, CSV, SQL, XML Create reports create reports start check and update CTCS SQL db "FAILED" "PASSED"Test Control Files ... Apache test Samba test Bind test Mount test ... test script script script script script Start Results Httpd Smbd named Mount /dev/nnn App 1 Nmbd mount -t nfs App 2 Startssl Smbclient App 3Application Layer Smbmount9 © Novell Inc. All rights reserved
  • Maintenance Update Testing
  • MainCoord initiates SWAMP QA checks onincl. Prio, etc. http://w2d.suse.de/abuildstat/patch-statusPatch is build and pkg. available needed arch and productsSLES 8 SLES 8 SLES 9 OES ... SLES Ni386 s390 x86_64 i386 <ARCH>Update Test Update Test Update Test Update Test Update TestBug Fix Test Bug Fix Test Bug Fix Test Bug Fix Test Bug Fix TestFeature Test Feature Test Feature Test Feature Test Feature Test Feature Test for Type A/B by OES team forISV Tests ISV Tests ISV Tests Type C ISV Tests ? Reject Collect Results NO Tests passed ? Alert TPMs, Sec Team, YES Support (NTS), TAMs, Approve Send report to ISVs qa@suse.de11 © Novell Inc. All rights reserved
  • openSUSE & QA
  • How can the community contribute? • bugzilla / testopia – main interface and used by Novell internally as well • Novell is opensourcing test tools / infrastructure / documentation – QA code will be available in the Build Service – Build Service will be able to run tests after a build job is done – easier integration for Software maintainers • building a QA community – awards & prizes for the best QA contributors (reports, test cases, ...) • better cooperation between the community and Novell – QA community manager13 © Novell Inc. All rights reserved
  • Testopia (off-line) Demo
  • 15 © Novell Inc. All rights reserved
  • 16 © Novell Inc. All rights reserved
  • 17 © Novell Inc. All rights reserved
  • 18 © Novell Inc. All rights reserved
  • 19 © Novell Inc. All rights reserved
  • 20 © Novell Inc. All rights reserved
  • 21 © Novell Inc. All rights reserved
  • 22 © Novell Inc. All rights reserved
  • 23 © Novell Inc. All rights reserved
  • 24 © Novell Inc. All rights reserved
  • Questions, Feedback, Discussions
  • Thank you for your attention26 © Novell Inc. All rights reserved
  • Unpublished Work of Novell, Inc. All Rights Reserved.This work is an unpublished work and contains confidential, proprietary, and trade secret information of Novell, Inc.Access to this work is restricted to Novell employees who have a need to know to perform tasks within the scopeof their assignments. No part of this work may be practiced, performed, copied, distributed, revised, modified,translated, abridged, condensed, expanded, collected, or adapted without the prior written consent of Novell, Inc.Any use or exploitation of this work without authorization could subject the perpetrator to criminal and civil liability.General DisclaimerThis document is not to be construed as a promise by any participating company to develop, deliver, or market aproduct. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon inmaking purchasing decisions. Novell, Inc. makes no representations or warranties with respect to the contentsof this document, and specifically disclaims any express or implied warranties of merchantability or fitness for anyparticular purpose. The development, release, and timing of features or functionality described for Novell productsremains at the sole discretion of Novell. Further, Novell, Inc. reserves the right to revise this document and tomake changes to its content, at any time, without obligation to notify any person or entity of such revisions orchanges. All Novell marks referenced in this presentation are trademarks or registered trademarks of Novell, Inc.in the United States and other countries. All third-party trademarks are the property of their respective owners.