The document discusses the progress and future goals of fully automated test automation for business processes, focusing on the use of SAP tools and non-SAP tools. It addresses the challenges encountered in producing reliable test data and ensuring test result reliability across complex business processes, emphasizing the need for dynamic test data generation and automated test processes. The document outlines the development of solutions to enhance SAP ECATT functionality and improve maintenance efficiency while ensuring accurate outcome monitoring of automated testing.

1. Automatic test automation What we do, how far we got, and where do we want to go

2. Agenda Our original target SAP Tools or non SAP tools for testautomation Fully automatic test of business processes How far did we manage Fully automatic test of output management Perspectives

3. Agenda Our original target SAP Tools or non SAP tools for testautomation Fully automatic test of business processes How far did we manage Fully automatic test of output management Perspectives

4. Targets: Testautomation Testautomation of the essential business processes in order to confirm the correct functionality of the software to simulate the essential business processes covered by the software to apply fully automatic tests of very complex business processes to significantly reduce staff required for the manual test Shorter lead times, optimizing the use of available testing ressources Precise repeatability of test scenarios across multiple releases Testresult reliabilty Increased product stability and quality Increase the reliabilty of the test performance Increasing the test coverage level Documentation of test results Automatic generation of test results documentation

5. Agenda Our original target SAP Tools or non SAP tools for testautomation Fully automatic test of business processes How far did we manage Fully automatic test of output management Perspectives

6. How far did we manage Testautomation of the essential business processes in order to confirm the correct functionality of the software  to simulate the essential business processes covered by the software  to apply fully automatic tests of very complex business processes  to significantly reduce staff required for the manual test  Shorter lead times, optimizing the use of available testing ressources  Precise repeatability of test scenarios across multiple releases Testresult reliabilty  Increased product stability and quality  Increase the reliabilty of the test performance  Increasing the test coverage level Documentation of test results  Automatic generation of test results documentation

7. How far did we manage - the essentials We had to find solutions to these problems: Business processes do not pay attention to the input methods of the software Some of these input methods are not supported by eCatt ( IDOC and batch processing) In order to achieve fully automatic regression tests we must find a way to produce test data dynamically without human interference In order to achieve absolute test result reliability, we had to find a way to remove factors which would reduce the sharpness of the test processes The complicated and complex business processes consist of a large number of steps which need to be performed in a controlled sequence and to exchange parameters in a controlled and secure manner

8. How far did we manage input methods Examples of test processes with various input methods Dialogue – „normal“ online user transaction – easily supported by eCatt Batchprocess – Dunning ( occasionally with a user interface to type in the information to be processed – but the processing itself is batch and asynchronous Creation and processing of input files Creation and processing of IDOC files Automatic database inspection checks

9. How far did we manage Dynamic supply of test data Some „trivial“ test data problems Which date do I need for this dunning test proces? Which date is it in two months? How do I know if this person has already been „inserted“ into the system How do I know, which date it is in 20 days – also if it needs to be a bank day

10. How far did we manage test result reliability How to reach test reliability: You need to know everything about the test data How old it is Which attributes it has Each and every attribute Each and every connection or link to other data elements

11. How far did we manage complex business processes The means of referenced eCatt scripts does not offer a configurative flexibility The referenced eCatt scripts would not be the right means for the input methods which can not be processed with eCatt

12. How far did we manage Requirements - summary If you want to control your test process, you must create and process all required data yourself. You cannot rely on the quality of data already in the system. You need to be able to produce valid test data automatically and supply this test data also automatically to your test process. You need to be able to service different kind of input methods in order to be able to offer test of entire business processes . If you want to provide a fully automatic test service you must focus on relatively low maintenance.

13. How far did we manage Solution – head lines Our test processes generate and, where required, process their test data themselves. Our test processes support different input methods. The steps of our test processes are linked together and exchange parameters through configurations (rather simple table entries). Due to the configuration ability we concentrate our maintenance on changes in the software products which we test.

14. How far did we manage Solution – head lines We developed additional SAP functions in order to embed the SAP eCatt standard functions into a solution which would fit our requirements. The addtitionals provide several test configuration functions Test process definition Test steps parameter exchange IDOC configuration Setup of test files Etc Test performing functions Start up and scheduling of test processes (not available in the eCatt environment) Utilities to start up the test steps in the correct sequence Test result displaying functions Various views on the results of the performed tests

15. Agenda Our original target SAP Tools or non SAP tools for testautomation Fully automatic test of business processes How far did we manage Fully automatic test of output management Perspectives

16. Fully automatic test of business processes Basic example One testprocess contains several test steps The steps of the testprocess are linked together through table entries Each test step produces parameters These parameters can be picked and processed up by succeeding steps The test steps are also supplied with test data from the standard eCatt test data container

17. Fully automatic test of business processes Testprocesses – Monitoring overview

18. Fully automatic test of business processes testprocess – test steps

19. Fully automatic test of business processes testprocess – test steps

20. Fully automatic test of business processes testprocess – test parameters and values

21. Fully automatic test of business processes The parameters are picked up by suceeding steps

22. Fully automatic test of business processes Some examples – screen shots To minimize maintenance efforts, it is very important, to create the test scripts as modular as possible. For example, there is the business partner dialogue; in oscare ®, there are 39 different roles for bp’s and according to most of them, the dialogue is different, mainly concerning the number of tab strips, which ones are to process and their order. Hereinafter two examples are shown by screenshots, at first a school and then an employer are created.

23. Fully automatic test of business processes Creation of a school as a business partner

24. Fully automatic test of business processes Creation of a school as a business partner

25. Fully automatic test of business processes Creation of a school as a business partner

26. Fully automatic test of business processes Creation of a school as a business partner

27. Fully automatic test of business processes Creation of a school as a business partner

28. Fully automatic test of business processes Creation of an employer as a business partner

29. Fully automatic test of business processes Creation of an employer as a business partner

30. Fully automatic test of business processes Creation of an employer as a business partner

31. Fully automatic test of business processes Creation of an employer as a business partner

32. Fully automatic test of business processes Creation of an employer as a business partner

33. Fully automatic test of business processes Creation of an employer as a business partner

34. Fully automatic test of business processes Creation of an employer as a business partner

35. Fully automatic test of business processes Creation of an employer as a business partner

36. Fully automatic test of business processes Creation of an employer as a business partner

37. Fully automatic test of business processes Creation of an employer as a business partner

38. Fully automatic test of business processes Some examples – screen shots To cover all different requirements with one script, it is neccessary, to encapsulate every tab strip within one single script. All of those scripts are referred within a so called ‚masterscript‘. The testdatacontainer is used, to manage the process by different flags, like for example ‚ I_DO_PARTNERINFO ‘. If the flag is set, the relevant tab strip is processed, otherwise not. Moreover, the same tab strips can have different positions, depending on the role. To handle that fact, a table had to be developed, where the information is stored, which tab strip can be found at what place (according to the role), what is it‘s name and what is it‘s ID.

39. Fully automatic test of business processes Some examples – screen shots Below all entries for the ‚Partnerinfo‘-tab are shown:

40. Fully automatic test of business processes Some examples – screen shots Here the protocol entry, that was originated while choosing the ‚Partnerinfo‘-tab strip during runtime, when name and id were determined:

41. Agenda Our original target SAP Tools or non SAP tools for testautomation Fully automatic test of business processes How far did we manage Fully automatic test of output management Perspectives

42. Fully automatic test of output management What was the problem and how it was solved A precise and reliable test of the correct generation and formation of RDI and XSF files is rather problematic. The requirements describing how these fields supposedly should be populated were defined and stored in testautomation. Functions to examine the correct population of the fields of these output files were developed. Most of the required business processes had already been test automated. Eventually we just had to add the output function and connect them to the field inspection functions.

43. Fully automatic test of output management Two kinds of field inspection Mainly in order to avoid loosing control of the automated output inspection checks we split the checks up in two categories. Appearance checks – does the output field actually occur in the output stream? Content checks – does the output field hold exactly the expected value?

44. Appearance checks RDI Target: Do all required fields occurr? Did unexpected fields occurr? How it works: We test automate the business process The steps of the business process populate the data fields, which should eventually show up in the output streams. The OPM field documentation of the data streams is transferred to a simple R/3 table The appearance checks are appended to the business and print part process OPM checks Business process Printing part

45. Appearance checks Examples Comparison with the OPM documentation in order to check that no unexpected fields occurred Selective check in order to inspect if these three fields occurred.

46. Content Checks How it works The content check contains two components Extraction of the expected field content Configuration table entries set up exactly from which teststeps, exactly which parameter the expected content of this output field holds. The function that extracts the expected field content reads these configuration entries and picks out the parameter values. Comparison - expected with actual output stream fields This function compares the expected field content with the actual stream fields.

47. Content Check OPM checks Businessprocess Print part

48. Content Checks Extraction of expected values

49. Content chekcs Comparison – expected - actual

50. Agenda Our original target SAP Tools or non SAP tools for testautomation Fully automatic test of business processes How far did we manage Fully automatic test of output management Perspectives

51. Perspectives Where do we go from here

52. Agenda Our original target SAP Tools or non SAP tools for testautomation Fully automatic test of business processes How far did we manage Fully automatic test of output management Perspectives

53. SAP-Ecatt SAP native free to use tool Very integrated automation capabilities and interfaces No serious support of non SAP GUI Dialogues Tosca Acquisition and / or licensing costs „ Videocam“-Technology Films everything also WEB dialogues (understands nothing) No SAP-Integration No SAP Interfaces Very limited (if at all available) test result checks Compuware und Mercury High priced and licensing costs Films everything also WEB dialogues (understands nothing ) No SAP-Integration No SAP Interfaces Very limited (if at all available) test result checks SAP or non SAP tools for testautomation

54. No non SAP product provides a professional SAP integration With these products you won‘t be able to conduct Regression tests without manual processing of test data - (this is already the first knock-out) A detailed query or inspection of the SAP Database content, in order to evaluate the result of yout test operation. (this is how it got the second knock-out) supply an exact repeatability without sacrifizing the test reliability If you want to try it anyway, it would require extreme ressources, furthermore the question: "Why don‘t we do the whole test manually anyway" would have to be answered SAP or non SAP tools for testautomation

55. eCATT or different SAP tools for testautomation? Supported protocols : SAP GUI WebDynpro Java Web, SAP-Web (Portal) Win32 .NET Terminal (3270, 5250) PowerBuilder Oracle Forms, Oracle Applications, Siebel, PeopleSoft MS Office, VB, Visual Age License costs : depends SAP Quick Test Professional by HP Supported protocols: SAP GUI WebDynpro Java WebDynpro ABAP (only selected objects) SAP Web Services License costs: none SAP eCATT