Containerized end-to-end testing provides advantages like isolation of environments, reproducible application environments, and optimized parallel execution. Sakuli is an open source tool that allows defining end-to-end tests using JavaScript or Java and executing them within Docker containers. It supports testing graphical user interfaces using technologies like X11 forwarding or VNC. Sakuli provides functions for web testing and interacting with applications and defines a structure for test cases with initialization, test steps, and exception handling. It can integrate with monitoring and CI tools to report test results.

1. Containerized End-2-End Testing
+
,Tobias Schneck ConSol Software GmbH

2. Characteristics of End-2-End Testing
Different types of testing:
Regression tests
Functional approval tests
Parallel tests with GUIs are complex
Stateful tests: user logins, sessions, history
Setup and cleanup of test data
Manual effort > effort for test automation

3. Advantages of Container Technology
Isolation of environments
Repository for versioning and distribution
Reproducible application environment
Dockerfile, docker-compose.yml
Optimized for parallel execution and cloud systems
Less memory and CPU overhead (shared Linux kernel)
Starting containers on-the-fly
Unique command line interface (orchestration tools)

4. Containerized GUIs
### start the docker container via x-forwarding
docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix:rw rasch/inkscape
### start the docker container with VNC interface
# connect via URL: http://localhost:6911/vnc_auto.html?password=vncpassword
docker run -it -p 5911:5901 -p 6911:6901 consol/ubuntu-xfce-vnc
docker run -it -p 5912:5901 -p 6912:6901 consol/centos-xfce-vnc
docker run -it -p 5913:5901 -p 6913:6901 consol/ubuntu-icewm-vnc:dev
docker run -it -p 5913:5901 -p 6914:6901 consol/centos-icewm-vnc:dev

5. What's provided by ?
Category
Web tests through HTML selectors
Restricted to browser content
Open Source & Java API
Headless execution
Test writing assistance
(recorder, tag finder)
Automatable / Test result reporting
(CI, DB, monitoring environment)

6. Monitoring Integration
Nagios OMD Incinga Check_MK

7. Test Definition (Java Script)

8. Test Case Structure
// testcase.js
/*************************************
* Initialization
*************************************/
_dynamicInclude($includeFolder);
var testCase = new TestCase( 60, 70);
var env = new Environment();
var appNotepad = new Application( "gedit");
var region = new Region();
/******************************
* Description of the test case
******************************/
try {
//...
/************************************************
* Exception handling and shutdown of test case
**********************************************/
} catch (e) {
testCase.handleException(e);
} finally {
testCase.saveResult();
}

9. Call all Sahi Functions for Web Testing
// testacase.js
/************************
* Call Sahi Functions
***********************/
_navigateTo( "http://sahi.example.com/_s_/dyn/Driver_initialized" );
_highlight(_link( "SSL Manager" ));
_isVisible(_link( "SSL Manager" ));
_highlight(_link( "Logs"))
_click(_link( "Logs"))
testCase.endOfStep( "Test Sahi landing page" , 5);

10. Fluent API for UI Testing
// testacase.js
/*** open calculator app ***/
var calculatorApp = new Application( "galculator" ).open();
testCase.endOfStep( "Open Calculator" , 3);
/*** calculate 525 + 100 ***/
var calculatorRegion = calculatorApp.getRegion();
calculatorRegion.type( "525")
.find( "plus.png" )
.click()
.type( "100");
calcRegion.find( "result.png" ).click();
calcRegion.waitForImage( "625", 5);
testCase.endOfStep( "calculate 525 +100" , 20);

11. Custom Functions
// e.g. excluded into a separate common.js
/**********
* Combine click and highlight
*********/
function clickHighlight ($selector) {
_highlight($selector);
_click($selector);
}
/***************
* Open PDF in native viewer
**************/
var PDF_EDITOR_NAME = "masterpdfeditor3" ;
function openPdfFile (pdfFileLocation ) {
return new Application(PDF_EDITOR_NAME + ' "' + pdfFileLocation + '"').open();
}

12. Test Definition (Java)
public class FirstExampleTest extends AbstractSakuliTest {
private static final String CITRUS_URL = "http://www.citrusframework.org/" ;
private Environment env;
private Region screen;
@BeforeClass
@Override
public void initTC() throws Exception {
super.initTC();
env = new Environment();
screen = new Region();
browser.open();
}
@Override
protected TestCaseInitParameter getTestCaseInitParameter () throws Exception {
return new TestCaseInitParameter( "test_citrus" );
}
@Test
public void testMyApp() throws Exception {
// ... testcode
}
}
For the Maven dependencies, take a look at the .Java DSL Documentaion

13. Test Definition (Java)
public class FirstExampleTest extends AbstractSakuliTest {
// ... initializing code
@Test
public void testCitrusHtmlContent () throws Exception {
browser.navigateTo(CITRUS_URL);
ElementStub heading1 = browser.paragraph( "Citrus Integration Testing" );
heading1.highlight();
assertTrue(heading1.isVisible());
ElementStub download = browser.link( "/Download v.*/" );
download.highlight();
assertTrue(download.isVisible());
download.click();
ElementStub downloadLink = browser.cell( "2.6.1");
downloadLink.highlight();
assertTrue(downloadLink.isVisible());
}
@Test
public void testCitrusPictures () throws Exception {
browser.navigateTo(CITRUS_URL);
env.setSimilarity( 0.8);
screen.find( "citrus_logo.png" ).highlight();
env.type(Key.END);
screen.find( "consol_logo.png" ).highlight();
}
}

14. Sakuli End-2-End Testing Container

15. Demo - Sakuli Container
# start the docker container
docker run -it -p 5911:5901 -p 6911:6901 consol/sakuli-ubuntu-xfce
docker run -it -p 5912:5901 -p 6912:6901 consol/sakuli-centos-xfce
docker run -it -p 5913:5901 -p 6913:6901 consol/sakuli-ubuntu-xfce-java
docker run -it -p 5914:5901 -p 6914:6901 consol/sakuli-centos-xfce-java
# start in parallel via docker-compose
# use docker-compos.yml from https://github.com/ConSol/sakuli/tree/master/docker
docker-compose up

16. Bakery Demo Setup

17. Bakery Demo Setup

18. Bakery Demo
git clone https://github.com/toschneck/sakuli-example-bakery-testing.git
# start jenkins
jenkins/deploy_jenkins.sh
# start OMD montioring
omd-nagios/deploy_omd.sh
# start the build of the application images
bakery-app/app-deployment-docker-compose/deploy_app.sh
#start tests
sakuli-tests/execute_all.sh
#start tests for monitoring
sakuli-tests/execute_all_4_monitoring.sh

19. What's next?
Headless execution - Linux: VNC & Docker Windows: ?
Cloud-ready container forOpenShift (security, source-2-image)
Video recording of the test execution (error documentation)
Web UI to handle Sakuli test suites
Connect 3rd-party test management tools(HP QC, TestRail, ...)
Improve test result presentation in CI tools
Selenium as an alternative to Sahi
Implement Junit 5 test runner

20. Links
ConSol/sakuli
ConSol/sakuli-examples
toschneck/sakuli-example-bakery-testing
sakuli@consol.de @sakuli_e2e

21. Thank you!
Tobias Schneck
ConSol Software GmbH
Franziskanerstraße 38
D-81669 München
Tel: +49-89-45841-100
Fax: +49-89-45841-111
tobias.schneck@consol.
toschneck
info@consol.de
www.consol.de
ConSol