Slides of our Webinar, held on March 20th 2018, where the main features of ElasTest, at the time being, were presented. This included our ability to launch docker and docker-compose SuTs, and tests, gathering logs and metrics, and inspecting resulst with out Log Analyzer. All these features, including ElasTest capabilities for manual testing, were showcased at the end in a demo conducted by out Project Coodinator. The webinar was recorded and it's available at: https://youtu.be/OL4yktND6Uo.

1. http://elastest.io
ElasTest
Funded by the
European Union
Happy testing!!
20/03/18 | ElasTest Webinar

2. Why?

3. Why ElasTest?
●
Modern systems are complex
Distributed Software / Cloud Native / Microservices

4. Why ElasTest?
●
We need
tests to
assure
their
quality

5. Why ElasTest?
What tools do exist for E2E testing in
complex cloud native apps?

6. Why ElasTest?
●
The IDE (to develop tests)

7. Why ElasTest?
●
The CI Server (to run tests after changes)

8. Why ElasTest?
●
These tools are focused in managing test
execution

9. Why ElasTest?
●
These tools are focused in managing test
execution

10. Why ElasTest?
●
Current testing tools have limitations
when facing E2E tests of complex
distributed software
– Log analysis
– Web testing
– SuT instrumentation

11. Limitations of testing tools
●
Log analysis

12. Limitations of testing tools
●
Log analysis
Where in the
source code is the
log of the failing
test?

13. Limitations of testing tools
●
Log analysis
– How to query logs of the System Under
Test?
Elasticsearch Kibana Logstash

14. Limitations of testing tools

15. Limitations of testing tools
Test logs are in the
IDE / CI, not here

16. Limitations of testing tools
Which log entries are
related to failure?
Time filter?

17. Limitations of testing tools
Is that the best tool to
know why a test has
failed?

18. Limitations of testing tools
OtrosLogViewer is a powerful log analysis tool

19. Limitations of testing tools
●
What if a test wants to know if there are
WARN messages in SuT logs?

20. Limitations of testing tools
●
It would be very useful to be able to compare the
log of a failed test with previous passing
executions

21. Why ElasTest?
●
Current testing tools have limitations
with E2E tests of complex distributed
software
– Log analysis
– Web testing
– SuT instrumentation

22. ●
Web Testing
– Selenium is used to control
browsers
– There are problems when
developing tests
●
Browsers are opened during
execution grabbing the focus
●
You can’t work while tests are
running in your laptop
Limitations of testing tools

23. ●
Web Testing
– There are problems when
executing tests in CI
●
You have to configure Xvfb for
headless testing
●
You have to configure custom tools
to record test executions
●
Video files are not synchronized with
logs (you have to jump manually)
Limitations of testing tools

24. ●
Web Testing
– Docker is helping here
●
Official selenium
images
– However
●
A lot of manual
configuration needed
●
No solution for Safari
or Edge
Limitations of testing tools

25. ●
Web Testing
– There are commercial cloud services
that solve these problems
Limitations of testing tools

26. Limitations of testing tools

27. Why ElasTest?
●
Current testing tools have limitations
with E2E tests of complex distributed
software
– Log analysis
– Web testing
– SuT instrumentation

28. ●
SuT instrumentation
– SuT Monitoring (logs, metrics analysis)
– Fault tolerance / Resilience testing
– Testing with different network conditions
(bandwidth, latency, jitter...)
Limitations of testing tools

29. Limitations of testing tools

30. ●
SuT instrumentation
– No integration with CI systems
– Hard to manage from tests
– Hard to compare test executions with
different SuT configurations
Limitations of testing tools

31. What is?

32. What is ElasTest?
●
An open source platform
●
Designed to ease E2E testing of complex
distributed systems
●
You can install it in your laptop and in your
CI server
Apache License 2

33. What is ElasTest?
●
Main features (currently)
– End to end test execution
– SuT execution
– Log and metrics visualization and recording
– Log Analysis
– Web browsers management (Chrome and
Firefox)
– Jenkins integration
– TestLink integration

34. What is ElasTest?
●
ElasTest is used...
– through a web interface
– from tests in your IDE
– with CI servers plugins

35. What is ElasTest?

36. Features
●
When using ElasTest, how do I develop
my e2e tests?
– With your favorite language
– With your favorite testing framework
– With your usual tools like selenium
– Just run them inside a docker container

37. Features
●
How do you execute the SuT?
– Deployed inside ElasTest
●
As a docker container
●
As a docker-compose project
– Deployed elsewhere
●
No restrictions
●
It can be monitored using standard agents
(more on that later)

38. Logs and metrics

39. Logs and metrics

40. Logs and metrics

41. Log Analyzer

42. Log Analyzer

43. Web Browsers

44. Jenkins integration

45. Jenkins integration

46. Jenkins integration

47. Jenkins integration

48. Jenkins integration

49. Jenkins integration

50. TestLink integration
+

51. TestLink integration

52. TestLink integration

53. Technologies

54. Architecture
●
ElasTest is a microservice application
– Made up of several components
– Talking to each other via
●
REST APIs
●
Message Queues
●
Websockets
– Platform independent

55. Architecture
55

56. Architecture
56

57. Architecture
57

58. Architecture
58

59. Architecture
59

60. Architecture
60

61. Technologies
●
ElasTest Manager technologies:

62. Technologies
●
Persistence and monitoring processing

63. Technologies
●
Infrastructure management (elasticity)
* Q2/Q3 2018

64. Technologies
●
ElasTest is an application with different
services deployed in docker containers
– ElasTest Manager
– Tests and SuT execution
– Web browsers
– Persistence services

65. Technologies
●
It is very easy to download and run in any
operating system
> docker run … elastest/platform start

66. Technologies
●
ElasTest platform container:
– ElasTest is composed by several docker
containers
– Docker-compose is “static”
– We didn’t want users to have docker-compose
installed (it is an additional installation step)
– We got inspiration on Eclipse-che

67. Technologies
●
ElasTest platform container:
– To start ElasTest, the “platform” container is
executed
– Contains docker compose tool and use the host
docker daemon
●
-v /var/run/docker.sock:/var/run/docker.sock
– It has several commands: start, wait, update…
– It shows user friendly / ElasTest related messages
in console

68. Technologies
●
ElasTest platform container:
– Start several ElasTest components using its docker-
compose.yml file depending on parameters
– Every component is developed in its own GitHub
repository to facilitate collaboration between teams
– We use git submodules to “include” component
repositories into platform repository
– It manipulates docker-compose.yml files (remove
binded ports, remove container names, inject
environment variables...)

69. Plugins

70. Plugins
●
ElasTest can be extended through plugins
●
This enables new testing tools to be added
to the platform
●
Within the project several plugins are being
developed
●
Browser service is one of those plugins

71. Plugins
●
ElasTest will develop plugins providing
– Web browsers
– Security services
– Monitoring
– Emulation of IoT sensors and actuators
– Big data services

72. Plugins
●
ElasTest monitoring
– ElasticSearch to store/search metrics and logs
– Logstash to “process” metrics and logs
– Capture
●
Logs are retrieved using syslog driver using
TCP
●
Metrics are retrieved using dockbeat
https://hub.docker.com/r/ingensi/dockbeat/

73. Plugins
●
ElasTest monitoring
– Log Analyzer
●
Implemented in Angular 2+ (as the overall GUI)
●
Uses ElasticSearch http API to query log entries
●
Uses ag-grid table component (open source
version)

74. Plugins
●
Web Browsers service
– Based on selenium official docker images
– A webdriver protocol proxy that intercepts
webdriver commands
– Create browsers on demand (instead of using
registered ones)
– Issues commands automatically
●
Capture browser console
and send it to ElasTest’s Logstash

75. Plugins
●
Web Browsers service
– Browser docker images provide VNC server
– We use noVNC technolgy to see and interact
with an VNC server from the web browser
– It also allows to record a video with the
session
https://github.com/bonigarcia/novnc

76. Plugins
●
Web Browsers service
– We are working on doing the same with
windows VMs to manage Internet Explorer
and Edge
– Based on modern.ie free Windows VM images
– Selenium is installed automatically using
ansible and vncdotool
https://github.com/sibson/vncdotool

77. In action!!

78. Roadmap

79. Roadmap
●
1.0: Q1 2018
– Test parameterization and comparison
– Tests can react on events monitored in SuT
– IoT devices emulation
– Security testing
– Plugin system
– WebRTC services
– Test management

80. Roadmap
●
2.0: Q3/Q4 2018
– Infrastructure managed by ElasTest (elasticity)
– Mobile testing
– Performance testing
– Resilience testing
– Test cost estimation
– Recommendation service

81. European project

82. European Project
●
ElasTest is a project funded by the
European Commission
http://elastest.eu
ElasTest: an elastic platform for testing
complex distributed large software systems
Duration: From 2017-01-01 to 2019-12-31
Total cost: EUR 5 031 187,50
EU contribution: EUR 4 270 187
Coordinated in: Spain
Topic(s): ICT-10-2016 - Software Technologies
Call for proposal: H2020-ICT-2016-1
Funded by the European Union

83. European Project
●
Coordination
●
Participants

84. How you can support the project?
●
Follow us on twitter!
●
Give us stars in GitHub
●
Subscribe on users mailing list / forum
@ElasTestIO
https://github.com/elastest/elastest
https://groups.google.com/forum/#!forum/elastest-users

85. Stay tuned
http://elastest.io
https://github.com/elastest
@elastestio
Thanks!