Testing large distributed heterogeneous systems in cloud environments is a complex task. This situation becomes especially difficult when carrying out end-to-end tests, in which the whole system is exercised, typically through its graphical user interface (GUI) with impersonated users. These tests are typically expensive to write and time consuming to run. This paper contributes to the solution of this problem by proposing an open source framework called ElasTest, which can be seen as an elastic platform to carry out end-to-end testing for different types of applications, including web and mobile. In particular, this piece or research puts the accent on the capability to impersonate final users, presenting a real case study in which end-to-end tests have been carried out to assess the correctness of real-time communications among browsers using WebRTC.

1. http://elastest.io
User Impersonation as
a Service in End-to-End
Testing
Funded by the
European Union
Boni García
boni.garcia@urjc.es
MODELSWARD Special Sessions (AMARETTO 2018)
January 23rd 2018 | Funchal, Portugal

2. Table of contents
1. Introduction
2. Background
3. ElasTest: platform for end-to-end testing
4. User Impersonation as a Service
5. Case study: testing WebRTC applications
6. Conclusions and future work
2

3. 1. Introduction
• Large distributed heterogenous systems are more
and more common (e.g. microservices
architectures, cloud native apps, etc.)
• Testing this kind of software is complex, especially
to verify the system as a whole
3
End-to-end testing
happens at system
level with users
impersonated

4. Table of contents
1. Introduction
2. Background
3. ElasTest: platform for end-to-end testing
4. User Impersonation as a Service
5. Case study: testing WebRTC applications
6. Conclusions and future work
4

5. 2. Background
• Recent surveys confirm
the existence of gap
between the current
and the desired status
of test automation for
distributed
heterogeneous
systems, prioritizing the
relevance of test
automation features for
these systems
(Lima, 2016)
5
25%
52%
7%
16%
WHAT IS THE LEVEL OF TEST AUTOMATION FOR
DISTRIBUTED AND HETEROGENEOUS SYSTEMS?
Only manual testing
Automatic test execution (with manual test scripting/coding)
Automatic test generation (with manual execution)
Automatic test generation and execution

6. 2. Background
• The main mechanism used in the current state-of-
the-art for the functional testing of web and mobile
applications consists on impersonating a user
through some kind of GUI automation
• Selenium is the most popular solution:
6

7. Table of contents
1. Introduction
2. Background
3. ElasTest: platform for end-to-end testing
4. User Impersonation as a Service
5. Case study: testing WebRTC applications
6. Conclusions and future work
7

8. 3. ElasTest: platform for end-to-end testing
• ElasTest is an open source platform aimed to
ease the end-to-end testing activities for different
types of distributed applications and services
• ElasTest manages the full testing lifecycle,
deploying and monitoring the SUT, executing the
end-to-end tests and exposing the results to
software engineers and testers
8
http://elastest.io/

9. 3. ElasTest: platform for end-to-end testing
• ElasTest architecture:
9

10. Table of contents
1. Introduction
2. Background
3. ElasTest: platform for end-to-end testing
4. User Impersonation as a Service
5. Case study: testing WebRTC applications
6. Conclusions and future work
10

11. 4. User Impersonation as a Service
• ElasTest implements an user impersonation as a
service capability that provides Software as a
Service (SaaS) extending the W3C WebDriver
with advanced capabilities:
1. Evaluation of the perceived Quality of
Experience (QoE) of users on real-time
multimedia applications
2. Equivalent automation capability for sensors
and smart devices for Internet of Things (IoT)
11

12. 4. User Impersonation as a Service
• Extension to W3C WebDriver recommendation by
ElasTest User Impersonation Service:
12
Method Path Description
POST /session/{sessionId}/element/{elementId}/event Subscribe to a given event
within an element
GET /session/{sessionId}/event/{subscriptionId} Read the value of event for a
given subscription
DELETE /session/{sessionId}/event/{subscriptionId} Remove a subscription
GET /session/{sessionId}/vnc Get remote session
DELETE /session/{sessionId}/vnc Delete remote session
POST /session/{sessionId}/usermedia Set user media for WebRTC
GET /session/{sessionId}/stats Read the WebRTC stats
POST /session/{sessionId}/element/{elementId}/latency Measure end-to-end latency of
a WebRTC session
POST /session/{sessionId}/element/{elementId}/quality Measure quality of a WebRTC
session

13. Table of contents
1. Introduction
2. Background
3. ElasTest: platform for end-to-end testing
4. User Impersonation as a Service
5. Case study: testing WebRTC applications
6. Conclusions and future work
13

14. 5. Case study: testing WebRTC applications
• WebRTC is the umbrella term for a number of
technologies aimed to bring Real Time
Communications to the Web
• W3C (JavaScript APIs): getUserMedia,
PeerConnection, DataChannels
• IETF (protocol stack): ICE, SDP, TURN, STUN, …
14

15. 5. Case study: testing WebRTC applications
• Our case study is based on
OpenVidu, an open source
videoconferencing WebRTC
framework
• Question driving this study:
“Is the ElasTest user
impersonation service
capable of improving the
end-to-end testing process
within the OpenVidu
project?”
15
http://openvidu.io/

16. 5. Case study: testing WebRTC applications
• Demo
16

17. Table of contents
1. Introduction
2. Background
3. ElasTest: platform for end-to-end testing
4. User Impersonation as a Service
5. Case study: testing WebRTC applications
6. Conclusions and future work
17

18. 6. Conclusions and future work
• ElasTest is an open source platform aimed to
ease end-to-end tests for heterogenous large
distributed systems
• ElasTest implements a User Impersonation as a
Service (UIaaS) extending the W3C WebDriver
recommendation
• Existing test codebases with Selenium and
Appium are completely compatible with ElasTest
• Some features are still under development, such
as measurement of the end-users’ perceived QoE
or support for IoT devices
18