See: http://www.oop-konferenz.de/oop2017/konferenz/konferenzprogramm/sessiondetails/action/detail/session/do-54/title/containerized-end-2-end-testing-automate-it.html Setting up unit- and integration tests are tasks which developers have learned to deal. But testing UIs from the perspective of an end user is mostly still a challenge. The key question usually to answer is: How it is possible to make these end-2-end tests stable, scalable and reproducible? The upcoming container technologies bring the hope of managing and automating your UI tests as easily as your container deployment. A live demo will show how it is possible to test and monitor a web- or a rich-client application in a containerized Linux UI. Target Audience: Architects, Developers, Tester, DevOps, Monitoring Teams Prerequisites: none Level: Practicing Extended Abstract The end-2-end test example will be demonstrated by the open source tools "Docker" and "Sakuli". The demo shows how to do a blackbox UI test of a complex application, which is build on purpose of the current microservice pattern. The results will be interpreted and visualized in the Jenkins CI build pipeline and we will take a look into the UI activity of the testing container during the test execution. In an alternative scenario the previous build end-2-end tests will be executed with the objective to forward there results to a monitoring system like OMD Nagios. This use case makes it possible to continuously measure the runtime of different test execution steps and create alerts on broken thresholds. This approach enables a new perspective of monitoring. It is now possible to check constantly if the functionality of an application, like the user login or the search engine is up and running - not just the sever healthy state.

1. Containerized End-2-End Testing
+
,Tobias Schneck ConSol Software GmbH

2. Test Pyramid
Robust
Independent
Isolated
Fast

3. Characteristics of End-2-End Testing
Different types of testing:
Regression tests
Functional approval tests
Parallel tests with GUIs are complex
Stateful tests: user logins, sessions, history
Setup and cleanup of test data
Manual effort > effort for test automation

4. Advantages of Container Technology
Isolation of environments
Repository for versioning and distribution
Reproducible application environment
Dockerfile, docker-compose.yml
Optimized for parallel execution and cloud systems
Less memory and CPU overhead (shared Linux kernel)
Starting containers on-the-fly
Unique command line interface (orchestration tools)

5. Virtual Machine vs. Container
(e.g. Docker)

6. Containerized GUIs
### start the docker container via x-forwarding
docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix:rw rasch/inkscape
### start the docker container with VNC interface
# connect via URL: http://localhost:6911/vnc_auto.html?password=vncpassword
docker run -it -p 5911:5901 -p 6911:6901 consol/ubuntu-xfce-vnc
docker run -it -p 5912:5901 -p 6912:6901 consol/centos-xfce-vnc
docker run -it -p 5913:5901 -p 6913:6901 consol/ubuntu-icewm-vnc:dev
docker run -it -p 5913:5901 -p 6914:6901 consol/centos-icewm-vnc:dev

7. What's provided by ?
Category
Web tests through HTML selectors
Restricted to browser content
Open Source & Java API
Headless execution
Test writing assistance
(recorder, tag finder)
Automatable / Test result reporting
(CI, DB, monitoring environment)

8. Monitoring Integration
Nagios OMD Icinga Check_MK

9. Test Definition (Java Script)

10. Test Case Structure
// testcase.js
/*************************************
* Initialization
*************************************/
_dynamicInclude($includeFolder);
var testCase = new TestCase( 60, 70);
var env = new Environment();
var appNotepad = new Application( "gedit");
var region = new Region();
/******************************
* Description of the test case
******************************/
try {
//...
/************************************************
* Exception handling and shutdown of test case
**********************************************/
} catch (e) {
testCase.handleException(e);
} finally {
testCase.saveResult();
}

11. Call all Sahi Functions for Web Testing
// testacase.js
/************************
* Call Sahi Functions
***********************/
_navigateTo( "http://sahi.example.com/_s_/dyn/Driver_initialized" );
_highlight(_link( "SSL Manager" ));
_isVisible(_link( "SSL Manager" ));
_highlight(_link( "Logs"))
_click(_link( "Logs"))
testCase.endOfStep( "Test Sahi landing page" , 5);

12. Fluent API for UI Testing
// testacase.js
/*** open calculator app ***/
var calculatorApp = new Application( "galculator" ).open();
testCase.endOfStep( "Open Calculator" , 3);
/*** calculate 525 + 100 ***/
var calculatorRegion = calculatorApp.getRegion();
calculatorRegion.type( "525")
.find( "plus.png" )
.click()
.type( "100");
calcRegion.find( "result.png" ).click();
calcRegion.waitForImage( "625", 5);
testCase.endOfStep( "calculate 525 +100" , 20);

13. Custom Functions
// e.g. excluded into a separate common.js
/**********
* Combine click and highlight
*********/
function clickHighlight ($selector) {
_highlight($selector);
_click($selector);
}
/***************
* Open PDF in native viewer
**************/
var PDF_EDITOR_NAME = "masterpdfeditor3" ;
function openPdfFile (pdfFileLocation ) {
return new Application(PDF_EDITOR_NAME + ' "' + pdfFileLocation + '"').open();
}

14. Test Definition (Java)
public class FirstExampleTest extends AbstractSakuliTest {
private static final String CITRUS_URL = "http://www.citrusframework.org/" ;
private Environment env;
private Region screen;
@BeforeClass
@Override
public void initTC() throws Exception {
super.initTC();
env = new Environment();
screen = new Region();
browser.open();
}
@Override
protected TestCaseInitParameter getTestCaseInitParameter () throws Exception {
return new TestCaseInitParameter( "test_citrus" );
}
@Test
public void testMyApp() throws Exception {
// ... testcode
}
}
For the Maven dependencies, take a look at the .Java DSL Documentaion

15. Test Definition (Java)
public class FirstExampleTest extends AbstractSakuliTest {
// ... initializing code
@Test
public void testCitrusHtmlContent () throws Exception {
browser.navigateTo(CITRUS_URL);
ElementStub heading1 = browser.paragraph( "Citrus Integration Testing" );
heading1.highlight();
assertTrue(heading1.isVisible());
ElementStub download = browser.link( "/Download v.*/" );
download.highlight();
assertTrue(download.isVisible());
download.click();
ElementStub downloadLink = browser.cell( "2.6.1");
downloadLink.highlight();
assertTrue(downloadLink.isVisible());
}
@Test
public void testCitrusPictures () throws Exception {
browser.navigateTo(CITRUS_URL);
env.setSimilarity( 0.8);
screen.find( "citrus_logo.png" ).highlight();
env.type(Key.END);
screen.find( "consol_logo.png" ).highlight();
}
}

16. Sakuli End-2-End Testing Container

17. Demo - Sakuli Container
# start the docker container
docker run -it -p 5911:5901 -p 6911:6901 consol/sakuli-ubuntu-xfce
docker run -it -p 5912:5901 -p 6912:6901 consol/sakuli-centos-xfce
docker run -it -p 5913:5901 -p 6913:6901 consol/sakuli-ubuntu-xfce-java
docker run -it -p 5914:5901 -p 6914:6901 consol/sakuli-centos-xfce-java
# start in parallel via docker-compose
# use docker-compos.yml from https://github.com/ConSol/sakuli/tree/master/docker
docker-compose up

18. Bakery Demo Setup

19. Bakery Demo Setup

20. Bakery Demo
git clone https://github.com/toschneck/sakuli-example-bakery-testing.git
# start jenkins
jenkins/deploy_jenkins.sh
# start OMD montioring
omd-nagios/deploy_omd.sh
# start the build of the application images
bakery-app/app-deployment-docker-compose/deploy_app.sh
#start tests
sakuli-tests/execute_all.sh
#start tests for monitoring
sakuli-tests/execute_all_4_monitoring.sh

21. What's next?
Cloud-ready Containers!
In development for release 02/2017:
Security: user namespaces
Source2Image
Build and run configurations

22. What's next?
Headless execution - Linux: VNC & Docker Windows: ?
Video recording of the test execution (error documentation)
Web UI to handle Sakuli test suites
Connect 3rd-party test management tools
(HP QC, TestRail, ...)
Selenium as an alternative to Sahi
Implement Junit 5 test runner

23. Links
ConSol/sakuli
ConSol/sakuli-examples
toschneck/sakuli-example-bakery-testing
sakuli@consol.de @sakuli_e2e

24. Thank you Munich!
Tobias Schneck
ConSol Software GmbH
Franziskanerstraße 38
D-81669 München
Tel: +49-89-45841-100
Fax: +49-89-45841-111
tobias.schneck@consol.
toschneck
info@consol.de
www.consol.de
ConSol