The document discusses continuous integration and continuous testing for SAP applications. It defines continuous integration as automatically building and testing code changes when developers commit changes to version control. Continuous testing involves running end-to-end tests on a separate schedule from the first build. The document recommends Worksoft's Certify and Execution Manager tools to help enterprises scale testing across their continuous integration and delivery processes by automating end-to-end test execution from CI servers and across environments.

1. Welcome!
Continuous Integration
Testing for SAP
Shift-Left Testing with Continuous Integration
Solutions from Worksoft
©2018 Worksoft, Inc. All trademarks and product names mentioned herein are the
property of their respective owners in the United States, Canada and/or other countries.
The information contained herein is subject to change without notice. All rights
reserved.
November 2018

2. Agenda
• It’s a Continuous World
• CI Key Considerations
• Running Certify Tests from CI
Servers
• Scaling Testing with Worksoft
Execution Manager
• Q and A
3
Chris Kraus
Worksoft Product Manager
ckraus@Worksoft.com

3. Business Developer
PRE-PROD PROD
Tester Operations
Continuous Integration
Continuous Improvement
Continuous Deployment
Continuous Delivery
Continuous Testing
Continuous Quality
Plan Prepare/Explore Realize: Build Test Deploy Run
It’s a Continuous World…

4. Continuous Integration 101
• Developer
• Writes code
• Commits the code into the source
code control system
• Source Control System
• Triggers the CI Server
• CI Server
• Triggers the automated build and
tests
• Reports results back to the
developer
5
CI Server
“CI is a must-do in the DevOps space, right
after version control becomes ubiquitous.”
- State of DevOps Report 2018

5. Continuous Integration 101
• Developer
• Writes code
• Commits the code into the source
Code Control System
• Source Control System
• Triggers the CI Server
• CI Server
• Triggers the automated build and
tests
• Reports results back to the developer
• Build Kept Under 10 minutes
• Second stage build needed for full
end-to-end testing
6
CI Server
“The second stage build runs a different suite
of tests that do hit the real database and involve
more end-to-end behavior.”
- Martin Fowler, Continuous Integration
First Build Triggered by
Source Control Change
2nd Stage Build
Batch Triggered
from Test
Management
System of Record

6. Continuous Integration Ecosystem –SAP View
7
Requirements Management
Tracking and Reporting
System
Defect Tracking
Source Code
Control System
Test Management System
Test Automation Library
and Execution API
CI Server

7. Core Principles of Continuous Integration
8
Sources:
SAP: https://developers.sap.com/tutorials/ci-best-practices-ci-cd.html
Martin Fowler: https://www.martinfowler.com/articles/continuousIntegration.html
Keep Everything Under
Version Control
• Requirements
• Changes Lists
Automate the Build
• Centralized and reproducible
Run Unit Tests in the
First Build
• Identify regressions as soon as
possible
• Test as much as possible
• Orchestrate tests to maximize
resource utilization
Commit Early and Often
• Integrate in small steps
Build Each Change
• Test each change
Fix Errors Immediately
• Do not accumulate errors
Keep the Build Fast
• Institute a “second” build to run
end-to-end tests
• Schedule slow tests during off-
peak hours
Test in a Copy of the
Production Environment
• Test how the user will use the
system

8. • No easy way to map from requirements >
source control > change lists > test
management system
• Multiple sprints working in parallel but not on
the same board
• Understanding of a process lies with the
business
• A single process can include 100s of steps
9
“We struggle with knowing exactly what to test and exactly are business processes
are. With the adoption of the Fiori interface and SuccessFactors, we see things only
becoming more complicated.”
- SAP Functional Lead
Key Considerations: Source Control
System and Change Lists

9. Key Considerations: What Tests Run When and Where
10
Application Owners / DevTest Pre-Prod Prod
Sprint Team
End-to-End Testing Considerations
• Ownership of test creation and
maintenance
• Test reuse/scalability – ability to
feed CI/CT cycles
• UI version changes
Continuous
Integration
1st Build:
Unit Tests
Continuous
Testing
2nd Build:
EtoE Tests
Operational Change
Management Team

10. Solution: A Two-Tiered
Approach
11
Example: Scheduled Continuous Testing
• If it takes 4 hours to to do end-to-end
• Run it every 6 hours to reduce the number
of changes
• This still makes it faster to do root cause
analysis (AKA Check in often)
• But for testing (MTTR/MTTI)
Source: Forrester Research

11. Continuous Integration and Continuous Testing: Better Together
12
Realize: Build Test Deploy
Sprint 1 - 3
Sprint 4 - 6
Unit Tests
Unit Tests
Functional
Test
Functional
Test
Functional
Integration Test
Sprint 7 - 10
Sprint 11 - 14
Unit Tests
Unit Tests
Functional
Test
Functional
Test
Functional
Integration Test

12. Cross Platform End-to-End Test Automation
Bidirectional Communication with CI Servers and ALMs
Worksoft Certify
Execution Manager
Worksoft CI Solution for Enterprise Applications

13. Certify: Test Creation and Management
• In DevOps developers write unit tests and “drop”
them into a folder to be run
• In Worksoft standard naming conventions tell you if
your folder is production or sandbox
• Create a folder with your on-demand, daily, weekly
tests
• New features
• Create a bookmark of your folders
• Run your bookmarks easily from API’s
14

14. DevOps Tool Chain
• What technologies will you
need
• Who will be using
• What needs to work together
UI Driven
• Logging in and out of system
• Differences in UIs and version
updates
• Orchestration and execution of
tests on demand at scale
15
“We need to extend ERP testing thru CI/CD so we can release changes at the pace we need
to.. Sr. management wants us to be more agile and ensure changes don’t break things
upstream or downstream from where that change took place.”
- Sr. Mgr. Technical Services
Second Build: Scaling and Testing in Pre-Prod

15. Solution: Execution Manager (EM) Working with CI Server
System Access
• Enable unattended UI
testing by auto-logging into
machines
System Security
• EM not only handles logging
in to the machine but also
logging out of the machine
after the test has been run
Test Orchestration
• EM orchestrates test across
a variety of OSs, devices
and applications
Parallel Test Execution
• EM can schedule and run
tests in remote or cloud-
based test labs when the CI
server and test lab are the
same, or via REST calls
ALM System Support
• Certify and EM integrations
with ALM systems can be
used to execute test plans
from the ALM and post
results
16

16. Worksoft Execution Manager
Supports Continuous Integration and Continuous Testing
• Schedule tests to run nights
and weekends
• Execute locally and in the
cloud
• Hardware or virtual machines
• Add intelligence to your
schedule:
• Run daily, hourly, weekly
• From CI/CD
• Run tests based on outcome
of prior tests
• Reports delivered via e-mail

17. Worksoft CI/CT Process Flow with Jenkins
Jenkins builds
and creates build
number
Build Number
Call Execution
Manager
REST
Run Certify Tests
Get Status
CI/CD Server
On premise Test Lab 2
EM
Agent
Execution
Manager
Remote Test Lab 1
EM
Agent
EM
Agent
EM
Agent
EM
Agent
EM
Agent

18. Demo Time!

19. Summary: Continuous Integration is Not Enough!
20 Source: SAP

20. Related Resources
• Related Blogs:
• Is Continuous Integration and Continuous Testing a One Size Fits All? No …
and Here’s Why.
• Top 5 Questions Asked About Worksoft in Agile-Plus-DevOps Environments
• Demos:
• Continuous Integration with Jenkins Demo
• Daily Continuous Testing with Worksoft and Jenkins
• End-to-End Agile Testing for SAP with Worksoft Certify
• ALM Integration Support
• Micro Focus (HP) ALM: https://www.worksoft.com/micro-focus-hp-alm-
integrations-support
• SAP Solution Manager: https://www.worksoft.com/testing-framework-sap-
solution-manager
• Panaya Dynamix: https://youtu.be/giPqj6JLapQ
21

21. Thank YOU!
22