Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. LCG-SPI: SW-Testing LCG Applications Area GridPP 7 th Collaboration Meeting Manuel. Gallas @ cern . ch LCG/SPI LCG Software Process & Infrastructure LCG Software Process & Infrastructure
  2. 2. Outline and Context SPI SW-Testing <ul><li>Overview </li></ul><ul><li>Test FrameWorks </li></ul><ul><li>User support </li></ul><ul><li>Test policies </li></ul><ul><li>Test documents </li></ul>LCG Infrastructure SPI project http:// spi . cern . ch / LHC grid software applications (LHC experiments, projects, etc) LCG Application Area POOL PI SIMU SEAL SW-Testing
  3. 3. Overview <ul><li>Software testing will be an integral part of the software development process. </li></ul><ul><li>All level of software testing should be run as part of an automatic process. </li></ul>GOAL Software testing Integration test System test Acceptance test Unit test Sw-testing team Integration Tests System Tests Examples LHC experiments Use in Exp. Test FrameWorks SW-testing doc SW- testing support SPI SW-Testing SW-testing polices Code Documentation Software developer CODE Work Package Test Automated testing Nightly Building System SW-Testing
  4. 4. Test FrameWorks <ul><li>Requirements: </li></ul><ul><li>Work with different languages (C++, Python, …) </li></ul><ul><li>Allow two ways of testing: “test the output of the code” and “test inside the testing-code”. </li></ul><ul><li>Easy way to integrate existing tests. </li></ul><ul><li>Allow the creation of dependencies among tests </li></ul><ul><li>Organize the tests by components or packages. </li></ul><ul><li>Regression testing </li></ul><ul><li>Easy integration with the Nightly Building System. </li></ul><ul><li>Provide also a graphical interface. </li></ul>Aim: to help developers: <ul><li>to produce code for testing </li></ul><ul><li>to run tests in automatic way </li></ul><ul><li>Our inputs: </li></ul><ul><li>Contacts within HEP-community. </li></ul><ul><li>What is available as free open source code. </li></ul><ul><li>Our constrains: </li></ul><ul><li>Avoid commercial software and </li></ul><ul><li>licensing problems. </li></ul><ul><li>Avoid “do it yourself solutions” </li></ul><ul><li>Try to adopt commonly used </li></ul><ul><li>open-source software . </li></ul>SW-Testing
  5. 5. Test FrameWorks: X-Unit family <ul><li>Scope: Unit –testing </li></ul><ul><li>The same “assertion style” in different languages. </li></ul><ul><li>Provides :- The name of the test case that failed. </li></ul><ul><li>-The name of the source file that contains the test. </li></ul><ul><li>-The line number where the failure occurred </li></ul><ul><li>Different platforms/compilers: (Linux/Solaris/Windows) </li></ul>X-Unit family JUnit QtUnit CppUnit PyUnit <ul><ul><ul><li>A simple test : </li></ul></ul></ul><ul><ul><ul><li>S ubclass the TestCase CppUnit or PyUnit class </li></ul></ul></ul><ul><ul><ul><li>O verride the method runTest(). </li></ul></ul></ul><ul><ul><ul><li>When you want to check a value, call the (CppUnit or PyUnit) ASSERT (bool) and pass in an expresion that is true if the test succeeds </li></ul></ul></ul>SW-Testing
  6. 6. Test FrameWorks: Oval <ul><li>Can be use for Unit-test to Validation tests. </li></ul><ul><li>Compare the output log file with a given reference file (Smart comparison of those lines which start with “[Oval]”) </li></ul><ul><li>It is possible to set different run environments. </li></ul><ul><li>Can run external scripts and external binaries. </li></ul><ul><li>Oval: </li></ul><ul><li>validation and regression </li></ul><ul><li>used in CMS </li></ul>Ovalfile (configuration) Executable Test Source Code Reference file Log file Oval run Oval diff Oval build SW-Testing
  7. 7. Test FrameWorks: QMTest <ul><li>Uses a graphical interface for creating and running tests </li></ul><ul><li>Runs tests in parallel </li></ul><ul><li>Organizes tests hierarchically </li></ul><ul><li>Supports execution of a single test or many at once </li></ul><ul><li>Records dependencies between tests </li></ul><ul><li>Can be run in batch mode -> easy integration with the Nightly-Building systems </li></ul>QMTest SW-Testing
  8. 8. Test FrameWorks: Global picture Test Scripts Valgrind Aim: to help developers: <ul><li>to produce code for testing </li></ul><ul><li>to run tests in automatic way </li></ul>OvalFile Oval L og file Ref file ? <ul><li>Automated tests </li></ul><ul><li>Regression test </li></ul>CppUnit PyUnit X-Unit family JUnit QtUnit QMTest <ul><li>Uses a graphical interface for </li></ul><ul><li>creating and running tests </li></ul><ul><li>Runs tests in parallel </li></ul><ul><li>Organizes tests hierarchically </li></ul><ul><li>Supports execution of a single </li></ul><ul><li>test or many at once </li></ul><ul><li>Records dependencies </li></ul><ul><li>between tests </li></ul><ul><li>In LCG AppArea runs what is in </li></ul><ul><li>the OvalFile and uses oval. </li></ul>Oval SW-Testing
  9. 9. SPI SW-Testing <ul><li>Overview </li></ul><ul><li>Test FrameWorks </li></ul><ul><li>User support </li></ul><ul><li>Test policies </li></ul><ul><li>Test documents </li></ul>SW-Testing
  10. 10. User support: Web information http ://spi.cern.ch/ Test FrameWorks HowTo Test doc Policies SW-Testing
  11. 11. User support: Tools supported <ul><li>Different platforms: </li></ul><ul><li>(Linux/Solaris/Windows) </li></ul><ul><li>Different compilers </li></ul><ul><li>Integrated with SCRAM </li></ul><ul><li>and ready to be used </li></ul><ul><li>Installed in AFS/DFS </li></ul><ul><li>Instructions to install </li></ul><ul><li>them in a local machine </li></ul>How to use them ? CppUnit PyUnit Provided as free software by the SPI External Software Service Oval QMTest SW-Testing
  12. 12. User support: How-To 1 2 3 Follow sw-testing policies SW-Testing
  13. 13. SPI SW-Testing <ul><li>Overview </li></ul><ul><li>Test FrameWorks </li></ul><ul><li>User support </li></ul><ul><li>Test policies </li></ul><ul><li>Test documents </li></ul>SW-Testing
  14. 14. Sw-Testing policies http ://spi.cern.ch/software_development.html <ul><li>Sw-testing policies are </li></ul><ul><li>needed to ensure: </li></ul><ul><li>Software is created together </li></ul><ul><li>with the tests. </li></ul><ul><li>Test directory structure and </li></ul><ul><li>naming should be homogeneous </li></ul><ul><li>and this will imply easy life to </li></ul><ul><li>have automatic tools. </li></ul><ul><li>Common and well integrated </li></ul><ul><li>tools </li></ul>For the projects For the developers For the testing responsible SW-Testing
  15. 15. SPI SW-Testing <ul><li>Overview </li></ul><ul><li>Test FrameWorks </li></ul><ul><li>User support </li></ul><ul><li>Test policies </li></ul><ul><li>Test documents </li></ul>SW-Testing
  16. 16. Test Documents <ul><li>For all project is needed a Test Plan in order to have a clear idea of the type of tests, schedule, environment and responsibilities. </li></ul><ul><li>We use a template </li></ul><ul><li>Test Case template collects information about the existing tests and how they cover the expected functionality of the software. </li></ul>SW-Testing
  17. 17. <ul><li>Thanks to: </li></ul><ul><li>PPARC </li></ul><ul><li>LHC experiments </li></ul><ul><li>LCG </li></ul>Feedback and interaction are always welcome!! Conclusions: Software testing is an important task and we have tools in place to do it. SPI SW-Testing <ul><li>Overview </li></ul><ul><li>Test FrameWorks </li></ul><ul><li>User support </li></ul><ul><li>Test policies </li></ul><ul><li>Test documents </li></ul>SW-Testing
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.