Containerized End-2-End-Testing - ContainerDays 2016 in Hamburg Example: Sakuli Example Bakery Testing https://github.com/toschneck/sakuli-example-bakery-testing Setting up unit- and integration tests are tasks which developers have learned to deal. But testing UIs from the perspective of an end user is mostly still a challenge. The key question usually to answer is: How it is possible to make these end-2-end tests stable, scalable and reproducible? The upcoming container technologies bring the hope of managing and automating your UI tests as easily as your container deployment. After a short introduction to the concepts, a live demo will show how it is possible to test a web- or a rich-client application in a containerized Linux UI. The example will be demonstrated by the open source tools "Docker" and "Sakuli". The execution and reporting of the tests will be managed by the continuous integration platform "Jenkins", which brings the ability to integrate everything into your build pipeline.

1. Containerized End­2­End Testing
+
,
Tobias Schneck ConSol Software GmbH

2. Test Pyramid
Robust
Independent
Isolated
Fast

3. Characteristics of End­2­End Testing
Different types of testing:
Regression tests
Functional approval tests
Parallel tests with GUIs are complex
Stateful tests: user logins, sessions, history
Setup and cleanup of test data

4. Manual effort > effort for test automation
Advantages of Container Technology
Isolation of environments
Repository for versioning and distribution
Reproducible application environment
Dockerfile, docker­compose.yml
Starting containers on­the­fly
Optimized for parallel execution
Less memory overhead
Shared Linux kernel

5. Containerized GUIs
### start the docker container via x­forwarding
docker run ­it ­e DISPLAY=$DISPLAY ­v /tmp/.X11­unix:/tmp/.X11­unix:rw rasch/inkscape
### start the docker container with VNC interface
# connect via URL: http://localhost:6911/vnc_auto.html?password=vncpassword
docker run ­it ­p 5911:5901 ­p 6911:6901 consol/centos­xfce­vnc
docker run ­it ­p 5912:5901 ­p 6912:6901 consol/ubuntu­xfce­vnc

6. What's provided by ?
Category
Web tests through HTML selectors
Restricted to browser content
Open Source & Java API
Headless execution
Test writing assistance
(recorder, screenshot finder)
Automatable / Test result reporting
(CI, DB, monitoring environment)

7. Nagios OMD Incinga
Monitoring Integration

8. Sakuli End­2­End Testing Container

9. Demo ­ Sakuli Container
# start the docker container
docker run ­it ­p 5911:5901 ­p 6911:6901 consol/sakuli­centos­xfce
docker run ­it ­p 5912:5901 ­p 6912:6901 consol/sakuli­ubuntu­xfce
# start in parallel via docker­compose
# use docker­compos.yml from https://github.com/ConSol/sakuli/tree/master/docker
docker­compose up

10. Bakery Demo Setup

11. Bakery Demo Setup

12. Bakery Demo
git clone https://github.com/toschneck/sakuli­example­bakery­testing.git
# start jenkins
jenkins/deploy_jenkins.sh
# start OMD montioring
omd­nagios/deploy_omd.sh
# start the build of the application images
bakery­app/app­deployment­docker­compose/deploy_app.sh
#start tests
sakuli­tests/execute_all.sh
#start tests for monitoring
sakuli­tests/execute_all_4_monitoring.sh

13. What's next?
Headless execution ­ Linux: VNC & Docker Windows: ?
Video recording of the test execution (error documentation)
Web UI to handle Sakuli test suites
Connect 3rd­party test management tools (HP QC, TestRail, ...)
Improve test result presentation in CI tools
Selenium as an alternative to Sahi

14. Links
https://github.com/ConSol/sakuli
https://github.com/toschneck/sakuli­
example­bakery­testing
https://twitter.com/sakuli_e2e
https://twitter.com/consol_de

15. Thank you Hamburg!
Tobias Schneck
ConSol Software GmbH
Franziskanerstraße 38
D­81669 München
Tel: +49­89­45841­100
Fax: +49­89­45841­111
tobias.schneck@consol.de
info@consol.de
www.consol.de