Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Test Automation in Business and Enterprise Digital Strategies

Originally presented by Krista Männistö of Bilot and Ari Varjonen of Qualmari at a recent Worksoft event at the Hotel Arthur in Helsinki

  • Login to see the comments

  • Be the first to like this

Test Automation in Business and Enterprise Digital Strategies

  1. 1. Automation in Digital Strategy 19/05/2016 www.bilot.fi
  2. 2. Worksoft customer conference Helsinki 18th of May 2016 Krista Männistö, Bilot Ari Varjonen, Qalmari 19/05/2016 www.bilot.fi 1
  3. 3. BILOT PARTNERSHIPS 19/05/2016 www.bilot.fi 2 PREFERRED SOLMAN DELIVERY AND DEVELOPMENT PARTNER CERTIFIED RESELLER, DEPLOYMENT AND DEVELOPMENT PARTNER NORDICS
  4. 4. Agenda 19/05/2016 www.bilot.fi 3 • Automation in Digital Strategy • Test Automation for Business • Role of SAP Solution Manager in Test Automation • Metso business case
  5. 5. Automation in Digital Strategy 19/05/2016 www.bilot.fi 4
  6. 6. What, How and When to Automate? • WHAT: • Separate the code testing and the E2E process testing. For process tests use process knowledge first and automation knowledge next. Let the process owners design what needs to be tested. • HOW: • Find a correct set of tools to Automate. Make the Manual testing work together with Automation because full-automation is not achievable. • WHEN: • Do you have any Standards to follow, are you already doing structured testing, what is your current maturity of testing? Process understanding from the perspective of change and E2E testing? 19/05/2016 www.bilot.fi 5
  7. 7. What to ask more? • What is our Automation plan and what is the framework behind it? Who has the ownership of these? • Do we run our test scenarios in hybrid landscape? • Cost-savings from time and resources are expected ROIs, but what else is there? • How much should we focus on re-usability when designing our Test cases? • Is the E2E scenarios our main goal when automating? • Are we doing testing continuously, do we need lights-out testing? • How Agile our development wants to be? • Does our Change and Test Management work hand-in-hand? 19/05/2016 www.bilot.fi 6
  8. 8. From IT-oriented way of working… 19/05/2016 www.bilot.fi 7 Business IT Business Requirement Design – process owners, project/change managers, developers Build – process owners, project/change managers, developers Test – key/end users (testers), test managers Deploy – project/change managers, technical team Approval Maintenance – process, owners, key users, support team
  9. 9. …to more collaborative way of work 19/05/2016 www.bilot.fi 8 Business IT Business Requirement Design – process owners, project/change managers, developers Build – process owners, project/change managers, developers Test – key/end users (testers), test managers Deploy – project/change managers, technical team Approval Maintenance – process, owners, key users, support team
  10. 10. Test Automation for Business 19/05/2016 www.bilot.fi 9
  11. 11. Business Benefits of Test Automation Testing costs and resources Downtime of test clients Development freeze during updates and main releases Testing cycles Time-to-market Technology risk Amount of defects when systems change 19/05/2016 www.bilot.fi 10 Reliability of testing Quality of testing Operational efficiency Time for other productive work Understanding of real end-to- end business processes Documentation of test cases Documentation of E2E business processes
  12. 12. Impacts of Test Automation to Release schedules 19/05/2016 www.bilot.fi 11 Scope & Design Build User Acceptance Testing Regression testing Final fixes and Technical Go- Live Scope & Design Build User Acceptance Testing Regression testing Final fixes and Technical Go- Live Manual testing Test automation scenario 1 Test automation scenario 2 Scope & Design Build Intensive support period Intensive support User Acceptance Testing Regression testing Final fixes and Technical Go- Live Intensive support
  13. 13. Thrive with Reusable Building blocks 19/05/2016 www.bilot.fi 12 E2E process 3 E2E process 2 E2E process 1 Transactional process step 1 ... Transactional process step 3 Transactional process step 2 Transactional process step 2 ... Transactional process step 4 Transactional process step 3 Transactional process step 1 ... Transactional process step 5 Transactional process step 5 Data variables Data layouts and recordsets
  14. 14. Benefits beyond process testing There is a lot more that you can do with Worksoft Certify besides regular testing activities. Tool could be used in addition of automated unit, integration or regression testing also for: • Master data maintenance and fixing • Mass uploads either to development, test or production clients • Actual system configuration steps • Process documentation and modelling • Documenting test cases for individual change requests for business users • Training purposes • System recovery testing after unplanned service breaks • Process performance testing • Testing of outputs 19/05/2016 www.bilot.fi 13
  15. 15. Role of SAP Solution Manager in Test Automation 19/05/2016 www.bilot.fi 14
  16. 16. Full SAP Change Cycle in Control 19/05/2016 www.bilot.fi 15 Hybrid Landscape Project/Release/ Maintenance Blueprint Manual/ Auto Test Case Req for Change BPCA RBPD Manual & Automated Test Cycles Test OK Automated/ Manual Deployment STMS,CTS+,cCTS… Daily, weekly, monthly Business Requirements Change Doc Transport Request(s) TBOM Worksoft Analyze Worksoft Certify Worksoft Certify SAP UPL Test Scope DEV Build Change OK
  17. 17. Automated Change Management with SAP SolMan & Worksoft Certify Integration 19/05/2016 www.bilot.fi 16 SolMan Test Mgt Worksoft Certify System Landscape Initiate testing from test plan Launch and run test script Perform actual process steps Return test results Return test logs Scope and create test plan Test management reporting SolMan Change Mgt SAP Solution Manager Build change Validate Business Requirement Approve & deploy change to PRD
  18. 18. SAP SolMan – Worksoft Certify 19/05/2016 www.bilot.fi 17 SAP Solution Manager Worksoft Certify Solution documentation Change management Change analysis Test scope Test plan Test package Tester assignments Test configuration Test data container Test scheduling Test defect management Manual testing Test reporting Test scenario generation Test execution Test execution logs Test execution report Test scenario error correction
  19. 19. Metso Business case 19/05/2016 www.bilot.fi 18
  20. 20. Metso as a corporation – Products and usage of SAP Metso is the world’s leading industrial company in the mining and aggregates industries and in the flow control business. - Employs around 14,000 professionals in 50 countries. - SAP is used as ERP currently in 27 different countries, mainly in Mining and Construction sites. - Metso has monthly SAP releases and uses Worksoft Certify test automation via SAP Solution Manager for example for monthly regression testing. - SAP was upgraded to EhP7 05/2015 and just transferred to SAP HANA Enterprise Cloud (HEC). 19/05/2016 www.bilot.fi 19
  21. 21. Test automation project at Metso – From beginning to present • Test automation project was launched at Metso in February 2014. Target of the project was to automate 80-90% of manual SAP related testing during year 2014.  This target was achieved! • At the moment Metso has documented and automated 135 end-to-end processes and are running automated tests via SAP Solution Manager with lights-out scheduling. 19/05/2016 www.bilot.fi 20 02/14 06/14 08/14 01/15 05/16 Project Kickoff First workshops on-site SAP Solman integration & 45 E2E automated test cases 135 E2E scenarios created & test automation was also used for example in HEC project testing Automation level of ~90% with 2 virtual machines and test automation used in monthly regression testing
  22. 22. Current level of test automation – Amount of test scripts • Currently Metso has automated 135 E2E test scripts. Also some utilities, transactional unit test scripts and mass upload/data creations have been created in Worksoft Certify side. These E2E scenarios include: • 570 Transactional processes • 8600 Process steps • Test automation used to test processes from: • All used SAP modules (like MM, SD, WM, FI, CO, PP etc...) • ISA/IPC portals • WM portal activities 19/05/2016 www.bilot.fi 21
  23. 23. Achieved benefits of test automation – Key learnings of the project “Key learning of the project has been to understand how important testing truly is in order to deliver changes to production environment in a safely manner. We have also only now came to realize how unreliable and unrepeatable manual testing results have earlier been. This has caused a huge risk in our way of operating with change management and also delayed the end to end lead time of change requests.” 19/05/2016 www.bilot.fi 22
  24. 24. Achieved benefits of test automation – Key learnings of the project • Test automation has made internal processes more flexible and improved productivity by releasing time and resources for other more productive work than manual testing. Testing could be performed more frequently and whenever needed. • Cost savings have been achieved from hours saved in testing and quality of testing.  Impact on release schedules as regression testing can be performed now in only 2 days instead of 6 days before including ~300 manual testing hours by offshore team. Testing can be also performed now mainly in night time. 19/05/2016 www.bilot.fi 23
  25. 25. Achieved benefits of test automation – Example how test automation was used in HANA project • E2E automated test scenarios were run during SAP HANA Enterprise Cloud (HEC) project test phases between March – May 2016 • Sandbox testing phases • 150 test scenarios after sandbox installation • 130 regression test scenarios again after compliance changes were transported to sandbox • Quality system testing phases • 100 chosen test scenarios run in quality system before releasing system to business UVT • In addition 130 regression test scenarios in quality system before actual go-live • Test automation found for example non working integrations that Metso was then able to fix before business UVT phase • Automated testing took all together under 60 hours compared to possible thousands of hours of pure manual testing 19/05/2016 www.bilot.fi 24
  26. 26. 19/05/2016 www.bilot.fi 25

×