ElasTest is an open source platform to execute e2e tests for web applications and complex distributed cloud native systems. It is based on docker and offers several services to ease testing implementation. This slides were used when presenting ElasTest in Panel Sistemas company in Madrid.

1. http://elastest.io
ElasTest
Funded by the
European Union
Happy testing!!
04/12/17 | Madrid

2. Who we are?
@micael_gallego
micael.gallego@urjc.es
@micaelgallego
@fgortazar
francisco.gortazar@urjc.es
@fgortazar
Consultancy / Training
Cloud Computing
Web Technologies
Extreme Programming
Testing / Git / Jenkins
Software Architecture
Concurrent Programming
http://codeurjc.es
Micael Gallego
Patxi Gortázar

3. Why?

4. Why ElasTest?
●
Modern systems are complex
Distributed Software / Cloud Native / Microservices

5. Why ElasTest?
●
Experience in testing complex
distributed systems

6. A media server for real time video
communication
The software under test
Why ElasTest?

7. A soup of technologies

8. Some numbers
• 30 code repositories
• ~400 Jenkins jobs
• +1,000 tests
• +20 different environments to test
• +80 artifacts to be deployed at
release time

9. Testing with multiple containers

10. Challenges
• Gathering data
– Logs from several containers
– Metrics (CPU, memory, network…)
– Files generated (browser recording)
• Integrating data with test execution
– Comparing some data across several
executions
– Data integration

11. Challenges
• Reproducing real-world conditions
– Failing nodes
– Failing networks
– Network bandwidth (4G, 5G, DSL)
– Network topologies (NAT traversal,
firewalls, …)
– High loads (bursting CPUs)

12. Why ElasTest?
●
We need
tests to
assure
their
quality

13. Why ElasTest?
What tools do exist for E2E testing in
complex cloud native apps?

14. Why ElasTest?
●
The IDE (to develop tests)

15. Why ElasTest?
●
The CI Server (to run tests after changes)

16. Why ElasTest?
●
These tools are focused in managing test
execution

17. Why ElasTest?
●
These tools are focused in managing test
execution

18. Why ElasTest?
●
Current testing tools have limitations
when facing E2E tests of complex
distributed software
– Log analysis
– Web testing
– SuT instrumentation

19. Limitations of testing tools
●
Log analysis

20. Limitations of testing tools
●
Log analysis
Where in the
source code is the
log of the failing
test?

21. Limitations of testing tools
●
Log analysis
– How to query logs of the System Under
Test?
Elasticsearch Kibana Logstash

22. Limitations of testing tools

23. Limitations of testing tools
Test logs are in the
IDE / CI, not here

24. Limitations of testing tools
Which log entries are
related to failure?
Time filter?

25. Limitations of testing tools
Is that the best tool to
know why a test has
failed?

26. Limitations of testing tools
OtrosLogViewer is a powerful log analysis tool

27. Limitations of testing tools
●
What if a test wants to know if there are
WARN messages in SuT logs?

28. Limitations of testing tools
●
It would be very useful to be able to compare the
log of a failed test with previous passing
executions

29. Why ElasTest?
●
Current testing tools have limitations
with E2E tests of complex distributed
software
– Log analysis
– Web testing
– SuT instrumentation

30. ●
Web Testing
– Selenium is used to control
browsers
– There are problems when
developing tests
●
Browsers are opened during
execution grabbing the focus
●
You can’t work while tests are
running in your laptop
Limitations of testing tools

31. ●
Web Testing
– There are problems when
executing tests in CI
●
You have to configure Xvfb for
headless testing
●
You have to configure custom tools
to record test executions
●
Video files are not synchronized with
logs (you have to jump manually)
Limitations of testing tools

32. ●
Web Testing
– Docker is helping here
●
Official selenium
images
– However
●
A lot of manual
configuration needed
●
No solution for Safari
or Edge
Limitations of testing tools

33. ●
Web Testing
– There are commercial cloud services
that solve these problems
Limitations of testing tools

34. Limitations of testing tools

35. Why ElasTest?
●
Current testing tools have limitations
with E2E tests of complex distributed
software
– Log analysis
– Web testing
– SuT instrumentation

36. ●
SuT instrumentation
– SuT Monitoring (logs, metrics analysis)
– Fault tolerance / Resilience testing
– Testing with different network conditions
(bandwidth, latency, jitter...)
Limitations of testing tools

37. Limitations of testing tools

38. ●
SuT instrumentation
– No integration with CI systems
– Hard to manage from tests
– Hard to compare test executions with
different SuT configurations
Limitations of testing tools

39. What is?

40. What is ElasTest?
●
An open source platform
●
Designed to ease E2E testing of complex
distributed systems
●
You can install it in your laptop and in your
CI server
Apache License 2

41. What is ElasTest?
●
Main features (currently)
– End to end test execution
– SuT execution
– Log and metrics visualization and recording
– Log Analysis
– Web browsers management (Chrome and
Firefox)
– Jenkins integration

42. What is ElasTest?
●
ElasTest is used...
– through a web interface
– from tests in your IDE
– with CI servers plugins

43. What is ElasTest?

44. Features
●
When using ElasTest, how do I develop
my e2e tests?
– With your favorite language
– With your favorite testing framework
– With your usual tools like selenium
– Just run them inside a docker container

45. Features
●
How do you execute the SuT?
– Deployed inside ElasTest
●
As a docker container
●
As a docker-compose project
– Deployed elsewhere
●
No restrictions
●
It can be monitored using standard agents
(more on that later)

46. Logs and metrics

47. Logs and metrics

48. Logs and metrics

49. Log Analyzer

50. Log Analyzer

51. Web Browsers

52. Jenkins integration

53. Jenkins integration

54. Jenkins integration

55. Jenkins integration

56. Jenkins integration

57. Jenkins integration

58. In action!!

59. Roadmap

60. Roadmap
●
First beta: Q4 2017
●
1.0: Q1 2018
– Test parameterization and comparison
– Tests can react on events monitored in SuT
– IoT devices emulation
– Security testing
– Plugin system

61. Roadmap
●
2.0: Q2/Q3 2018
– Infrastructure managed by ElasTest (elasticity)
– Mobile testing
– WebRTC services
– Test management
– Performance testing
– Resilience testing
– Test cost estimation
– Recommendation service

62. Technologies

63. Technologies
●
It is very easy to download and run in any
operating system
> docker run … elastest/platform start
* Coming soon

64. Technologies
●
ElasTest is an application with different
services deployed in docker containers
– ElasTest Manager
– Tests and SuT execution
– Web browsers
– Persistence services

65. Technologies
●
ElasTest Manager technologies:

66. Technologies
●
Persistence and monitoring processing

67. Technologies
●
Infrastructure management (elasticity)
* Q2/Q3 2018

68. European project

69. European Project
●
ElasTest is a project funded by the
European Commission
http://elastest.eu
ElasTest: an elastic platform for testing
complex distributed large software systems
Duration: From 2017-01-01 to 2019-12-31
Total cost: EUR 5 031 187,50
EU contribution: EUR 4 270 187
Coordinated in: Spain
Topic(s): ICT-10-2016 - Software Technologies
Call for proposal: H2020-ICT-2016-1
Funded by the European Union

70. European Project
●
Coordination
●
Participants

71. How you can support the project?
●
Follow us on twitter!
●
Give us stars in GitHub
●
Subscribe on users mailing list / forum
@ElasTestIO
https://github.com/elastest/elastest
https://groups.google.com/forum/#!forum/elastest-users

72. Stay tuned
http://elastest.io
https://github.com/elastest
@elastestio
Thanks!