Manuel Gallas
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
803
On Slideshare
803
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
1
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. LCG-SPI: SW-Testing LCG Applications Area GridPP 7 th Collaboration Meeting Manuel. Gallas @ cern . ch LCG/SPI LCG Software Process & Infrastructure LCG Software Process & Infrastructure
  • 2. Outline and Context SPI SW-Testing
    • Overview
    • Test FrameWorks
    • User support
    • Test policies
    • Test documents
    LCG Infrastructure SPI project http:// spi . cern . ch / LHC grid software applications (LHC experiments, projects, etc) LCG Application Area POOL PI SIMU SEAL SW-Testing
  • 3. Overview
    • Software testing will be an integral part of the software development process.
    • All level of software testing should be run as part of an automatic process.
    GOAL Software testing Integration test System test Acceptance test Unit test Sw-testing team Integration Tests System Tests Examples LHC experiments Use in Exp. Test FrameWorks SW-testing doc SW- testing support SPI SW-Testing SW-testing polices Code Documentation Software developer CODE Work Package Test Automated testing Nightly Building System SW-Testing
  • 4. Test FrameWorks
    • Requirements:
    • Work with different languages (C++, Python, …)
    • Allow two ways of testing: “test the output of the code” and “test inside the testing-code”.
    • Easy way to integrate existing tests.
    • Allow the creation of dependencies among tests
    • Organize the tests by components or packages.
    • Regression testing
    • Easy integration with the Nightly Building System.
    • Provide also a graphical interface.
    Aim: to help developers:
    • to produce code for testing
    • to run tests in automatic way
    • Our inputs:
    • Contacts within HEP-community.
    • What is available as free open source code.
    • Our constrains:
    • Avoid commercial software and
    • licensing problems.
    • Avoid “do it yourself solutions”
    • Try to adopt commonly used
    • open-source software .
    SW-Testing
  • 5. Test FrameWorks: X-Unit family
    • Scope: Unit –testing
    • The same “assertion style” in different languages.
    • Provides :- The name of the test case that failed.
    • -The name of the source file that contains the test.
    • -The line number where the failure occurred
    • Different platforms/compilers: (Linux/Solaris/Windows)
    X-Unit family JUnit QtUnit CppUnit PyUnit
        • A simple test :
        • S ubclass the TestCase CppUnit or PyUnit class
        • O verride the method runTest().
        • When you want to check a value, call the (CppUnit or PyUnit) ASSERT (bool) and pass in an expresion that is true if the test succeeds
    SW-Testing
  • 6. Test FrameWorks: Oval
    • Can be use for Unit-test to Validation tests.
    • Compare the output log file with a given reference file (Smart comparison of those lines which start with “[Oval]”)
    • It is possible to set different run environments.
    • Can run external scripts and external binaries.
    • Oval:
    • validation and regression
    • used in CMS
    Ovalfile (configuration) Executable Test Source Code Reference file Log file Oval run Oval diff Oval build SW-Testing
  • 7. Test FrameWorks: QMTest
    • Uses a graphical interface for creating and running tests
    • Runs tests in parallel
    • Organizes tests hierarchically
    • Supports execution of a single test or many at once
    • Records dependencies between tests
    • Can be run in batch mode -> easy integration with the Nightly-Building systems
    QMTest SW-Testing
  • 8. Test FrameWorks: Global picture Test Scripts Valgrind Aim: to help developers:
    • to produce code for testing
    • to run tests in automatic way
    OvalFile Oval L og file Ref file ?
    • Automated tests
    • Regression test
    CppUnit PyUnit X-Unit family JUnit QtUnit QMTest
    • Uses a graphical interface for
    • creating and running tests
    • Runs tests in parallel
    • Organizes tests hierarchically
    • Supports execution of a single
    • test or many at once
    • Records dependencies
    • between tests
    • In LCG AppArea runs what is in
    • the OvalFile and uses oval.
    Oval SW-Testing
  • 9. SPI SW-Testing
    • Overview
    • Test FrameWorks
    • User support
    • Test policies
    • Test documents
    SW-Testing
  • 10. User support: Web information http ://spi.cern.ch/ Test FrameWorks HowTo Test doc Policies SW-Testing
  • 11. User support: Tools supported
    • Different platforms:
    • (Linux/Solaris/Windows)
    • Different compilers
    • Integrated with SCRAM
    • and ready to be used
    • Installed in AFS/DFS
    • Instructions to install
    • them in a local machine
    How to use them ? CppUnit PyUnit Provided as free software by the SPI External Software Service Oval QMTest SW-Testing
  • 12. User support: How-To 1 2 3 Follow sw-testing policies SW-Testing
  • 13. SPI SW-Testing
    • Overview
    • Test FrameWorks
    • User support
    • Test policies
    • Test documents
    SW-Testing
  • 14. Sw-Testing policies http ://spi.cern.ch/software_development.html
    • Sw-testing policies are
    • needed to ensure:
    • Software is created together
    • with the tests.
    • Test directory structure and
    • naming should be homogeneous
    • and this will imply easy life to
    • have automatic tools.
    • Common and well integrated
    • tools
    For the projects For the developers For the testing responsible SW-Testing
  • 15. SPI SW-Testing
    • Overview
    • Test FrameWorks
    • User support
    • Test policies
    • Test documents
    SW-Testing
  • 16. Test Documents
    • For all project is needed a Test Plan in order to have a clear idea of the type of tests, schedule, environment and responsibilities.
    • We use a template
    • Test Case template collects information about the existing tests and how they cover the expected functionality of the software.
    SW-Testing
  • 17.
    • Thanks to:
    • PPARC
    • LHC experiments
    • LCG
    Feedback and interaction are always welcome!! Conclusions: Software testing is an important task and we have tools in place to do it. SPI SW-Testing
    • Overview
    • Test FrameWorks
    • User support
    • Test policies
    • Test documents
    SW-Testing