Automatic test automation What we do, how far we got, and where do we want to go
Agenda Our original target SAP Tools or non SAP tools for testautomation Fully automatic test of business processes How fa...
Agenda Our original target SAP Tools or non SAP tools for testautomation Fully automatic test of business processes How fa...
Targets: Testautomation <ul><li>Testautomation of the essential business processes in order to confirm the correct functio...
Agenda Our original target SAP Tools or non SAP tools for testautomation Fully automatic test of business processes How fa...
How far did we manage <ul><li>Testautomation of the essential business processes in order to confirm the correct functiona...
How far did we manage - the essentials <ul><li>We had to find solutions to these problems: </li></ul><ul><ul><li>Business ...
How far did we manage input methods  Examples of test processes with various input methods Dialogue – „normal“ online user...
How far did we manage Dynamic supply of test data <ul><li>Some „trivial“ test data problems </li></ul><ul><ul><li>Which da...
How far did we manage test result reliability <ul><li>How to reach test reliability: </li></ul><ul><ul><li>You need to kno...
How far did we manage complex business processes <ul><ul><li>The means of referenced eCatt scripts does not offer a config...
How far did we manage Requirements - summary <ul><ul><li>If you want to control your test process, you must create and pro...
How far did we manage Solution – head lines <ul><ul><li>Our test processes generate and, where required, process their tes...
How far did we manage Solution – head lines <ul><ul><li>We developed additional SAP functions in order to embed the SAP eC...
Agenda Our original target SAP Tools or non SAP tools for testautomation Fully automatic test of business processes How fa...
Fully automatic test of business processes Basic example <ul><li>One testprocess contains several test steps </li></ul><ul...
Fully automatic test of business processes Testprocesses – Monitoring overview
Fully automatic test of business processes testprocess – test steps
Fully automatic test of business processes testprocess – test steps
Fully automatic test of business processes testprocess – test parameters and values
Fully   automatic test of business processes The parameters are picked up by suceeding steps
Fully automatic test of business processes Some examples – screen shots <ul><li>To minimize maintenance efforts, it is ver...
Fully automatic test of business processes Creation of a school as a business partner
Fully automatic test of business processes Creation of a school as a business partner
Fully automatic test of business processes Creation of a school as a business partner
Fully automatic test of business processes Creation of a school as a business partner
Fully automatic test of business processes Creation of a school as a business partner
Fully automatic test of business processes Creation of an employer as a business partner
Fully automatic test of business processes Creation of an employer as a business partner
Fully automatic test of business processes Creation of an employer as a business partner
Fully automatic test of business processes Creation of an employer as a business partner
Fully automatic test of business processes Creation of an employer as a business partner
Fully automatic test of business processes Creation of an employer as a business partner
Fully automatic test of business processes Creation of an employer as a business partner
Fully automatic test of business processes Creation of an employer as a business partner
Fully automatic test of business processes Creation of an employer as a business partner
Fully automatic test of business processes Creation of an employer as a business partner
Fully automatic test of business processes Some examples – screen shots <ul><li>To cover all different requirements with o...
Fully automatic test of business processes Some examples – screen shots <ul><li>Below all entries for the ‚Partnerinfo‘-ta...
Fully automatic test of business processes Some examples – screen shots <ul><li>Here the protocol entry, that was originat...
Agenda Our original target SAP Tools or non SAP tools for testautomation Fully automatic test of business processes How fa...
Fully automatic test of output management  What was the problem and how it was solved <ul><li>A precise and reliable test ...
Fully automatic test of output management  Two kinds of field inspection <ul><li>Mainly in order to avoid loosing control ...
Appearance checks RDI <ul><li>Target: </li></ul><ul><ul><li>Do all required fields occurr?  </li></ul></ul><ul><ul><li>Did...
Appearance checks Examples <ul><li>Comparison with the  OPM documentation  in order to check that no unexpected fields occ...
Content Checks How it works <ul><li>The content check contains two components </li></ul><ul><li>Extraction of the expected...
Content Check OPM checks Businessprocess Print part
Content Checks <ul><li>Extraction of expected values </li></ul>
Content chekcs <ul><li>Comparison – expected - actual </li></ul>
Agenda Our original target SAP Tools or non SAP tools for testautomation Fully automatic test of business processes How fa...
Perspectives Where do we go from here
Agenda Our original target SAP Tools or non SAP tools for testautomation Fully automatic test of business processes How fa...
<ul><li>SAP-Ecatt </li></ul><ul><ul><ul><li>SAP native free to use tool </li></ul></ul></ul><ul><ul><ul><li>Very integrate...
<ul><li>No non SAP product provides a professional SAP integration </li></ul><ul><li>With these products you won‘t be able...
eCATT or different SAP tools for testautomation? <ul><ul><li>Supported protocols  : </li></ul></ul><ul><ul><ul><li>SAP GUI...
Upcoming SlideShare
Loading in …5
×

SAP Test automation - fully automatic test of complex business processes including output management XSF and RDI

7,129 views

Published on

We take the wording "automation" very seriously. Our version of test automation is indeed fully automatic. No one has to modify test data or any other input parameter in order to conduct regression tests. We support different kinds of input interfaces - Dialogues, IDOC-interface, Batch processing, files are generated and submitted all of it automatically. Even the checks in order to evaluate if the test was successful are performed automatically.
A recent feature of out automatic test system is the controlled generation of XSF and RDI output files, and the automatic evaluation of the correctness of the file content. Our setup consists of large number of isolated units which can be linked exchanging parameters. The isolated units are linked together through table entries. We have been up and running for some years
We are looking forward to displaying our setup and discuss different aspects of automatic testing with you. In case you don't believe us - we aren't surprised, but please give us the chance to convince you

Published in: Technology, Business
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
7,129
On SlideShare
0
From Embeds
0
Number of Embeds
22
Actions
Shares
0
Downloads
208
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

SAP Test automation - fully automatic test of complex business processes including output management XSF and RDI

  1. 1. Automatic test automation What we do, how far we got, and where do we want to go
  2. 2. Agenda Our original target SAP Tools or non SAP tools for testautomation Fully automatic test of business processes How far did we manage Fully automatic test of output management Perspectives
  3. 3. Agenda Our original target SAP Tools or non SAP tools for testautomation Fully automatic test of business processes How far did we manage Fully automatic test of output management Perspectives
  4. 4. Targets: Testautomation <ul><li>Testautomation of the essential business processes in order to confirm the correct functionality of the software </li></ul><ul><ul><li>to simulate the essential business processes covered by the software </li></ul></ul><ul><ul><li>to apply fully automatic tests of very complex business processes </li></ul></ul><ul><ul><li>to significantly reduce staff required for the manual test </li></ul></ul><ul><ul><li>Shorter lead times, optimizing the use of available testing ressources </li></ul></ul><ul><ul><li>Precise repeatability of test scenarios across multiple releases </li></ul></ul><ul><li>Testresult reliabilty </li></ul><ul><ul><li>Increased product stability and quality </li></ul></ul><ul><ul><li>Increase the reliabilty of the test performance </li></ul></ul><ul><ul><li>Increasing the test coverage level </li></ul></ul><ul><li>Documentation of test results </li></ul><ul><ul><li>Automatic generation of test results documentation </li></ul></ul>
  5. 5. Agenda Our original target SAP Tools or non SAP tools for testautomation Fully automatic test of business processes How far did we manage Fully automatic test of output management Perspectives
  6. 6. How far did we manage <ul><li>Testautomation of the essential business processes in order to confirm the correct functionality of the software </li></ul><ul><li> to simulate the essential business processes covered by the software </li></ul><ul><li> to apply fully automatic tests of very complex business processes </li></ul><ul><li> to significantly reduce staff required for the manual test  Shorter lead times, optimizing the use of available testing ressources </li></ul><ul><li> Precise repeatability of test scenarios across multiple releases </li></ul><ul><li>Testresult reliabilty </li></ul><ul><li> Increased product stability and quality  Increase the reliabilty of the test performance  Increasing the test coverage level </li></ul><ul><li>Documentation of test results </li></ul><ul><li> Automatic generation of test results documentation </li></ul>
  7. 7. How far did we manage - the essentials <ul><li>We had to find solutions to these problems: </li></ul><ul><ul><li>Business processes do not pay attention to the input methods of the software </li></ul></ul><ul><ul><ul><li>Some of these input methods are not supported by eCatt ( IDOC and batch processing) </li></ul></ul></ul><ul><ul><li>In order to achieve fully automatic regression tests we must find a way to produce test data dynamically without human interference </li></ul></ul><ul><ul><li>In order to achieve absolute test result reliability, we had to find a way to remove factors which would reduce the sharpness of the test processes </li></ul></ul><ul><ul><li>The complicated and complex business processes consist of a large number of steps which need to be performed in a controlled sequence and to exchange parameters in a controlled and secure manner </li></ul></ul>
  8. 8. How far did we manage input methods Examples of test processes with various input methods Dialogue – „normal“ online user transaction – easily supported by eCatt Batchprocess – Dunning ( occasionally with a user interface to type in the information to be processed – but the processing itself is batch and asynchronous Creation and processing of input files Creation and processing of IDOC files Automatic database inspection checks
  9. 9. How far did we manage Dynamic supply of test data <ul><li>Some „trivial“ test data problems </li></ul><ul><ul><li>Which date do I need for this dunning test proces? </li></ul></ul><ul><ul><li>Which date is it in two months? </li></ul></ul><ul><ul><li>How do I know if this person has already been „inserted“ into the system </li></ul></ul><ul><ul><li>How do I know, which date it is in 20 days – also if it needs to be a bank day </li></ul></ul>
  10. 10. How far did we manage test result reliability <ul><li>How to reach test reliability: </li></ul><ul><ul><li>You need to know everything about the test data </li></ul></ul><ul><ul><li>How old it is </li></ul></ul><ul><ul><li>Which attributes it has </li></ul></ul><ul><ul><li>Each and every attribute </li></ul></ul><ul><ul><li>Each and every connection or link to other data elements </li></ul></ul>
  11. 11. How far did we manage complex business processes <ul><ul><li>The means of referenced eCatt scripts does not offer a configurative flexibility </li></ul></ul><ul><ul><li>The referenced eCatt scripts would not be the right means for the input methods which can not be processed with eCatt </li></ul></ul>
  12. 12. How far did we manage Requirements - summary <ul><ul><li>If you want to control your test process, you must create and process all required data yourself. You cannot rely on the quality of data already in the system. </li></ul></ul><ul><ul><li>You need to be able to produce valid test data automatically and supply this test data also automatically to your test process. </li></ul></ul><ul><ul><li>You need to be able to service different kind of input methods in order to be able to offer test of entire business processes . </li></ul></ul><ul><ul><li>If you want to provide a fully automatic test service you must focus on relatively low maintenance. </li></ul></ul>
  13. 13. How far did we manage Solution – head lines <ul><ul><li>Our test processes generate and, where required, process their test data themselves. </li></ul></ul><ul><ul><li>Our test processes support different input methods. </li></ul></ul><ul><ul><li>The steps of our test processes are linked together and exchange parameters through configurations (rather simple table entries). </li></ul></ul><ul><ul><li>Due to the configuration ability we concentrate our maintenance on changes in the software products which we test. </li></ul></ul>
  14. 14. How far did we manage Solution – head lines <ul><ul><li>We developed additional SAP functions in order to embed the SAP eCatt standard functions into a solution which would fit our requirements. </li></ul></ul><ul><ul><ul><li>The addtitionals provide several test configuration functions </li></ul></ul></ul><ul><ul><ul><ul><li>Test process definition </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Test steps parameter exchange </li></ul></ul></ul></ul><ul><ul><ul><ul><li>IDOC configuration </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Setup of test files </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Etc </li></ul></ul></ul></ul><ul><ul><li>Test performing functions </li></ul></ul><ul><ul><ul><li>Start up and scheduling of test processes (not available in the eCatt environment) </li></ul></ul></ul><ul><ul><ul><li>Utilities to start up the test steps in the correct sequence </li></ul></ul></ul><ul><ul><li>Test result displaying functions </li></ul></ul><ul><ul><ul><li>Various views on the results of the performed tests </li></ul></ul></ul>
  15. 15. Agenda Our original target SAP Tools or non SAP tools for testautomation Fully automatic test of business processes How far did we manage Fully automatic test of output management Perspectives
  16. 16. Fully automatic test of business processes Basic example <ul><li>One testprocess contains several test steps </li></ul><ul><li>The steps of the testprocess are linked together through table entries </li></ul><ul><li>Each test step produces parameters </li></ul><ul><li>These parameters can be picked and processed up by succeeding steps </li></ul><ul><li>The test steps are also supplied with test data from the standard eCatt test data container </li></ul>
  17. 17. Fully automatic test of business processes Testprocesses – Monitoring overview
  18. 18. Fully automatic test of business processes testprocess – test steps
  19. 19. Fully automatic test of business processes testprocess – test steps
  20. 20. Fully automatic test of business processes testprocess – test parameters and values
  21. 21. Fully automatic test of business processes The parameters are picked up by suceeding steps
  22. 22. Fully automatic test of business processes Some examples – screen shots <ul><li>To minimize maintenance efforts, it is very important, to create the test scripts as modular as possible. For example, there is the business partner dialogue; in oscare ®, there are 39 different roles for bp’s and according to most of them, the dialogue is different, mainly concerning the number of tab strips, which ones are to process and their order. </li></ul><ul><li>Hereinafter two examples are shown by screenshots, at first a school and then an employer are created. </li></ul>
  23. 23. Fully automatic test of business processes Creation of a school as a business partner
  24. 24. Fully automatic test of business processes Creation of a school as a business partner
  25. 25. Fully automatic test of business processes Creation of a school as a business partner
  26. 26. Fully automatic test of business processes Creation of a school as a business partner
  27. 27. Fully automatic test of business processes Creation of a school as a business partner
  28. 28. Fully automatic test of business processes Creation of an employer as a business partner
  29. 29. Fully automatic test of business processes Creation of an employer as a business partner
  30. 30. Fully automatic test of business processes Creation of an employer as a business partner
  31. 31. Fully automatic test of business processes Creation of an employer as a business partner
  32. 32. Fully automatic test of business processes Creation of an employer as a business partner
  33. 33. Fully automatic test of business processes Creation of an employer as a business partner
  34. 34. Fully automatic test of business processes Creation of an employer as a business partner
  35. 35. Fully automatic test of business processes Creation of an employer as a business partner
  36. 36. Fully automatic test of business processes Creation of an employer as a business partner
  37. 37. Fully automatic test of business processes Creation of an employer as a business partner
  38. 38. Fully automatic test of business processes Some examples – screen shots <ul><li>To cover all different requirements with one script, it is neccessary, to encapsulate every tab strip within one single script. All of those scripts are referred within a so called ‚masterscript‘. The testdatacontainer is used, to manage the process by different flags, like for example ‚ I_DO_PARTNERINFO ‘. If the flag is set, the relevant tab strip is processed, otherwise not. </li></ul><ul><li>Moreover, the same tab strips can have different positions, depending on the role. To handle that fact, a table had to be developed, where the information is stored, which tab strip can be found at what place (according to the role), what is it‘s name and what is it‘s ID. </li></ul>
  39. 39. Fully automatic test of business processes Some examples – screen shots <ul><li>Below all entries for the ‚Partnerinfo‘-tab are shown: </li></ul>
  40. 40. Fully automatic test of business processes Some examples – screen shots <ul><li>Here the protocol entry, that was originated while choosing the ‚Partnerinfo‘-tab strip during runtime, when name and id were determined: </li></ul>
  41. 41. Agenda Our original target SAP Tools or non SAP tools for testautomation Fully automatic test of business processes How far did we manage Fully automatic test of output management Perspectives
  42. 42. Fully automatic test of output management What was the problem and how it was solved <ul><li>A precise and reliable test of the correct generation and formation of RDI and XSF files is rather problematic. </li></ul><ul><ul><li>The requirements describing how these fields supposedly should be populated were defined and stored in testautomation. </li></ul></ul><ul><ul><li>Functions to examine the correct population of the fields of these output files were developed. </li></ul></ul><ul><ul><li>Most of the required business processes had already been test automated. </li></ul></ul><ul><ul><li>Eventually we just had to add the output function and connect them to the field inspection functions. </li></ul></ul>
  43. 43. Fully automatic test of output management Two kinds of field inspection <ul><li>Mainly in order to avoid loosing control of the automated output inspection checks we split the checks up in two categories. </li></ul><ul><ul><li>Appearance checks – does the output field actually occur in the output stream? </li></ul></ul><ul><ul><li>Content checks – does the output field hold exactly the expected value? </li></ul></ul>
  44. 44. Appearance checks RDI <ul><li>Target: </li></ul><ul><ul><li>Do all required fields occurr? </li></ul></ul><ul><ul><li>Did unexpected fields occurr? </li></ul></ul><ul><li>How it works: </li></ul><ul><ul><li>We test automate the business process </li></ul></ul><ul><ul><li>The steps of the business process populate the data fields, which should eventually show up in the output streams. </li></ul></ul><ul><ul><li>The OPM field documentation of the data streams is transferred to a simple R/3 table </li></ul></ul><ul><ul><li>The appearance checks are appended to the business and print part process </li></ul></ul>OPM checks Business process Printing part
  45. 45. Appearance checks Examples <ul><li>Comparison with the OPM documentation in order to check that no unexpected fields occurred </li></ul>Selective check in order to inspect if these three fields occurred.
  46. 46. Content Checks How it works <ul><li>The content check contains two components </li></ul><ul><li>Extraction of the expected field content </li></ul><ul><ul><li>Configuration table entries set up exactly from which teststeps, exactly which parameter the expected content of this output field holds. </li></ul></ul><ul><ul><li>The function that extracts the expected field content reads these configuration entries and picks out the parameter values. </li></ul></ul><ul><li>Comparison - expected with actual output stream fields </li></ul><ul><ul><li>This function compares the expected field content with the actual stream fields. </li></ul></ul>
  47. 47. Content Check OPM checks Businessprocess Print part
  48. 48. Content Checks <ul><li>Extraction of expected values </li></ul>
  49. 49. Content chekcs <ul><li>Comparison – expected - actual </li></ul>
  50. 50. Agenda Our original target SAP Tools or non SAP tools for testautomation Fully automatic test of business processes How far did we manage Fully automatic test of output management Perspectives
  51. 51. Perspectives Where do we go from here
  52. 52. Agenda Our original target SAP Tools or non SAP tools for testautomation Fully automatic test of business processes How far did we manage Fully automatic test of output management Perspectives
  53. 53. <ul><li>SAP-Ecatt </li></ul><ul><ul><ul><li>SAP native free to use tool </li></ul></ul></ul><ul><ul><ul><li>Very integrated automation capabilities and interfaces </li></ul></ul></ul><ul><ul><ul><li>No serious support of non SAP GUI Dialogues </li></ul></ul></ul><ul><li>Tosca </li></ul><ul><ul><ul><li>Acquisition and / or licensing costs </li></ul></ul></ul><ul><ul><ul><li>„ Videocam“-Technology </li></ul></ul></ul><ul><ul><ul><li>Films everything also WEB dialogues (understands nothing) </li></ul></ul></ul><ul><ul><ul><li>No SAP-Integration </li></ul></ul></ul><ul><ul><ul><li>No SAP Interfaces </li></ul></ul></ul><ul><ul><ul><li>Very limited (if at all available) test result checks </li></ul></ul></ul><ul><li>Compuware und Mercury </li></ul><ul><ul><ul><li>High priced and licensing costs </li></ul></ul></ul><ul><ul><ul><li>Films everything also WEB dialogues (understands nothing ) </li></ul></ul></ul><ul><ul><ul><li>No SAP-Integration </li></ul></ul></ul><ul><ul><ul><li>No SAP Interfaces </li></ul></ul></ul><ul><ul><ul><li>Very limited (if at all available) test result checks </li></ul></ul></ul> SAP or non SAP tools for testautomation
  54. 54. <ul><li>No non SAP product provides a professional SAP integration </li></ul><ul><li>With these products you won‘t be able to conduct </li></ul><ul><ul><li>Regression tests without manual processing of test data - (this is already the first knock-out) </li></ul></ul><ul><ul><li>A detailed query or inspection of the SAP Database content, in order to evaluate the result of yout test operation. (this is how it got the second knock-out) </li></ul></ul><ul><ul><li>supply an exact repeatability without sacrifizing the test reliability </li></ul></ul><ul><ul><li>If you want to try it anyway, it would require extreme ressources, furthermore the question: &quot;Why don‘t we do the whole test manually anyway&quot; would have to be answered </li></ul></ul> SAP or non SAP tools for testautomation
  55. 55. eCATT or different SAP tools for testautomation? <ul><ul><li>Supported protocols : </li></ul></ul><ul><ul><ul><li>SAP GUI </li></ul></ul></ul><ul><ul><ul><li>WebDynpro </li></ul></ul></ul><ul><ul><ul><li>Java </li></ul></ul></ul><ul><ul><ul><li>Web, SAP-Web (Portal) </li></ul></ul></ul><ul><ul><ul><li>Win32 </li></ul></ul></ul><ul><ul><ul><li>.NET </li></ul></ul></ul><ul><ul><ul><li>Terminal (3270, 5250) </li></ul></ul></ul><ul><ul><ul><li>PowerBuilder </li></ul></ul></ul><ul><ul><ul><li>Oracle Forms, Oracle Applications, Siebel, PeopleSoft </li></ul></ul></ul><ul><ul><ul><li>MS Office, VB, Visual Age </li></ul></ul></ul><ul><ul><li>License costs : depends </li></ul></ul>SAP Quick Test Professional by HP <ul><ul><li>Supported protocols: </li></ul></ul><ul><ul><ul><li>SAP GUI </li></ul></ul></ul><ul><ul><ul><li>WebDynpro Java </li></ul></ul></ul><ul><ul><ul><li>WebDynpro ABAP (only selected objects) </li></ul></ul></ul><ul><ul><ul><li>SAP Web Services </li></ul></ul></ul><ul><ul><li>License costs: none </li></ul></ul>SAP eCATT

×