Today’s cyber-physical systems (CPS) span IoT and cloud-based datacenter infrastructures, which are highly heterogeneous with various types of uncertainty. Thus, testing uncertainties in these CPS is a challenging and multidisciplinary activity. We need several tools for modeling, deployment, control, and analytics to test and evaluate uncertainties for different configurations of the same CPS. In this paper, we explain why using state-of-the art model-driven engineering (MDE) and model-based testing (MBT) tools is not adequate for testing uncertainties of CPS in IoT Cloud infrastruc- tures. We discus how to combine them with techniques for elastic execution to dynamically provision both CPS under test and testing utilities to perform tests in various IoT Cloud infrastructures.

1. Testing Uncertainty of Cyber-Physical
Systems in IoT Cloud Infrastructures:
Combining Model-Driven Engineering
and Elastic Execution
Hong-Linh Truong, Luca Berardinelli
Distributed Systems Group, TU Wien
truong@dsg.tuwien.ac.at
http://rdsea.github.io
TECPS'17, 13 July, 2017, Santa Barbara, USA 1

2. Outline
 Our focused CPS
 IoT Cloud Systems, Elastic Execution, Uncertainty
 Top-down and bottom up approaches
 MDE/MBT versus Elastic Execution
 Combining methods and tool pipelines
 Early results
 Summary
TECPS'17, 13 July, 2017, Santa Barbara, USA 2

3. Our view on IoT Cloud Systems
3
TECPS'17, 13 July, 2017, Santa Barbara, USA
Source: Duc-Hung Le, Nanjangud C. Narendra, Hong Linh Truong:
HINC - Harmonizing Diverse Resource Information across IoT, Network Functions, and Clouds. FiCloud 2016: 317-324

4. Cyber-physical systems in IoT
Cloud Infrastructures
 Heavily based on virtual resources
 Loosely couple and tightly couple interactions
 Different degrees of data and control interactions
TECPS'17, 13 July, 2017, Santa Barbara, USA 4
(Virtual) Cyber-physical systems

5. IoT Cloud Systems - Elastic
Execution
5TECPS'17, 13 July, 2017, Santa Barbara, USA
http://tuwiendsg.github.io/iCOMOT/
Examples:
Elastic execution is a fundamental aspect in IoT Cloud Systems,
strongly changing methods for design and execution of CPS/IoT

6. Testing uncertainty in CPS
 Uncertainty:
 due to lack of knowledge, especially due to the complexity and
diversity of resources and interactions in IoT and Cloud systems
 Supporting testing uncertainties and uncertainties analytics
 Emerging novel aspects: data uncertainties (data/data-centric
CPS), elasticity of CPS resources (w.r.t function and composition),
and Governance (related to business/trustworthiness)
 How to discover them and then deal with them?
 Uncertainty analytics through testing (H2020 U-Test objectives)
 Also adaptation of resources considering uncertainties
TECPS'17, 13 July, 2017, Santa Barbara, USA 6

7. U-Test Uncertainty
Model
Locality
0..1
Uncertainty
Lifetime
0..1
Random
0..1
0..*
Effect
0..*
Risk
0..1
0..*
Pattern
0..1
0..*
Cause
0..*
0..*
Measurement
0
..
*
substatement
0..*
Uncertainty
Indeterminacy
Source
1..*
source
0..*
Measure
«enumeration»
IndeterminacyNature
nondeterminism
insufficientResolution
missingInfo
composite
unclassified
type1..*
0..*
Belief
Measurement 0..*
0..* beliefDegree
Belief
Statement
Characterizing Uncertainty
Belief Model
TECPS'17, 13 July, 2017, Santa Barbara, USA 7
Figure sources and relevant technical reports:
https://www.simula.no/file/d12pdf/download
Core Uncertainty
Conceptual Model
Infrastructure Level
Uncertainty Taxonomy
<<import>>
Application Level
Uncertainty Taxonomy
<<import>>

8. Data
ElasticityGovernance
Important infrastructure uncertainty
classes
TECPS'17, 13 July, 2017, Santa Barbara, USA 8
Storage
Uncertainty
Infrastructure
Uncertainty
DataDelivery
Uncertainty
Actuation
Uncertainty
DeploymentTime
Uncertainty
ExecutionEnvironment
Uncertainty
Storage
ComplianceUncertainty
StorageQuality
Uncertainty
StorageDependability
Uncertainty
DataQuality
Uncertainty
DataDelivery
ComplianceUncertainty
DataDeliveryDependability
Uncertainty
ActuationDependability
Uncertainty
EnvironmentDependability
Uncertainty
Actuation
ComplianceUncertainty
ApplicationDependability
Uncertainty
Runtime
Uncertainty

9. Modeling and Provisioning IoT Cloud
Infrastructures for Uncertainty Testing
The Topdown approach (MDE/MBT)
TECPS'17, 13 July, 2017, Santa Barbara, USA 9

10. IoT and Cloud Resource Profile
TECPS'17, 13 July, 2017, Santa Barbara, USA 10

11. Uncertainty
Profile
TECPS'17, 13 July, 2017, Santa Barbara, USA 11

12. Then Model-based Testing (MBT)
TECPS'17, 13 July, 2017, Santa Barbara, USA 12
Mostly do not
consider how to deal
with the
infrastructures of
SUT
Work with specific
static SUT
deployment
Figure source:
Mark Utting and Bruno Legeard. 2006. Practical Model-Based Testing: A Tools
Approach. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.

13. Elastic Infrastructure Testing
The bottom up approach (from IoT Cloud
systems)
Example from:
Daniel Moldovan,Hong-Linh Truong, A Platform for Run-time Health
Verification of Elastic Cyber-physical Systems, The IEEE International
Symposium on Modelling, Analysis and Simulation of Computer and
Telecommunication Systems (MASCOTS 2016), September 19-21, Imperial
College, London, UK
TECPS'17, 13 July, 2017, Santa Barbara, USA 13

14. Infrastructure-level Testing
Approach
TECPS'17, 13 July, 2017, Santa Barbara, USA 14
(From modeling/description)
*testing strategy=testing plan
Daniel Moldovan,Hong-Linh Truong, A
Platform for Run-time Health
Verification of Elastic Cyber-physical
Systems, The IEEE International
Symposium on Modelling, Analysis and
Simulation of Computer and
Telecommunication Systems
(MASCOTS 2016), September 19-21,
Imperial College, London, UK

15. Run-Time view on structure of
Elasic IoT Cloud systems/CPS
TECPS'17, 13 July, 2017, Santa Barbara, USA 15

16. Infrastructure Testing Platform
Domain-Specific Language for Test Plan
Description
name: "TestName"
description: "human readable description"
timeout: timeInSeconds
Triggers
every: number s|m
event: "Added |"Removed“ |”TestFailed” | “TestPassed”| on ComponentIdentifier
…
Execution
executor: [distinct] ComponentIdentifier for ComponentIdentifier [, ComponentIdentifier]+
where ComponentIdentifier:
• Type. [Service | Process | VirtualContainer | VirtualMachine | PhysicalMachine | PhysicalDevice]
• ID.”ComponentID”
• UUID.”ComponentUUID”
TECPS'17, 13 July, 2017, Santa Barbara, USA 16
Using this DSL: for the test plan, both deployment/configuration and
testing tasks can be described interwoven

17. Closing the gap: Combining MDE and
Elastic Execution
TECPS'17, 13 July, 2017, Santa Barbara, USA 17
Currently we are not able to leverage strength of
topdown approach and bottom-up approach for
testing CPS uncertainty

18. Public cloud infrastructures
Private cloud infrs.
Base Transceiver Station (BTS)
Case Stduy BTS
 Data uncertainty: influence both monitoring and controlling of
equipment
 Large-scale systems (1000+ BTS)
 Flexible back-end clouds
 Generic enough for other applications (e.g., in smart agriculture)
TECPS'17, 13 July, 2017, Santa Barbara, USA 18
Sensor
IoT
Gateway
MQTT
Broker
BigQuery
Cassandra
Hadoop FS
Actuator
Optimizer Analytics

19. Key issues
 Current MDE/MBT tools are not enough for
testing uncertainties
 Models, languages and transformation techniques at
the moment do not support on-demand, pay-per-use,
elasticity of IoT and cloud resources
 Elastic Execution hidden from models for
testing
 IoT and cloud infrastructures are not fixed
 SUT and their configurations are changed
TECPS'17, 13 July, 2017, Santa Barbara, USA 19

20. Key issues: Two separate worlds
TECPS'17, 13 July, 2017, Santa Barbara, USA 20
SUT Infrastructure
Development
Deployment Description
Development
IoT/Cloud Infrastructures
Infrastructure Configurations
Resource
Information
Adaptor/Tool
Deployment Scripts
Preparing CPS under Test
Requirements
Artifact
Repository
Figure source:
Mark Utting and Bruno Legeard. 2006. Practical Model-Based Testing: A Tools
Approach. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.
?

21. Combining MDE with Elastic
Execution
TECPS'17, 13 July, 2017, Santa Barbara, USA 21

22. Interwoven test exuection and
provisioning
TECPS'17, 13 July, 2017, Santa Barbara, USA 22
IoT Cloud Infrastructures
Modeling
Uncertainties
and SUT
Uncertainty
Profile
System Under
Test (SUT)
Models
Generating
Uncertainty
Test cases
Deploying/
Configuring
SUT
System Under
Test (SUT)
Test cases
Executing
Tests
IoT/Cloud
Resource
Information
Deploying/
Configuring
Testing Utilities
Test
Utilities

23. Tool pipelines: from MDE to elastic
execution for testing
TECPS'17, 13 July, 2017, Santa Barbara, USA 23
Key thoughts:
 Different algorithms to create suitable deployment configurations based
on data uncertainties, cost, and time
 Interactions between testing and elasticity control of IoT and Cloud
services

24. Our current progress
TECPS'17, 13 July, 2017, Santa Barbara, USA 24
Consider to generate provisioning configurations
from SUT models
Extracting
model
information &
Generating test
configuration
test
strategies
Infrastructural
IoT Cloud
resources
Provisioning &
Configuring
SUT
Executing
Tests
Test plans &
executors
IoT Units
Cloud
T
E
S
T
S
ProviderA
ActuatorsSensors
VMsAnalytics GWs
IoT Units
Cloud
T
E
S
T
S
ProviderB
ActuatorsSensors
VMsAnalytics GWs
Infrastructure
(Class Diagrams)
Behaviour
(State Machine
diagrams)
Uncertainty
Hong-Linh Truong, Luca Berardinelli, Ivan Pakovic, Georgiana Copil, Modeling and Provisioning IoT Cloud
Infrastructures for Testing Uncertainties, July, 2017 under submission

25. Example of BTS monitoring
TECPS'17, 13 July, 2017, Santa Barbara, USA 25

26. Elastic Test plan
Generic models for Task Executor
TECPS'17, 13 July, 2017, Santa Barbara, USA 26
Test
Configuration
Test
Executor
Provisioning
Task
Reconfiguration
Task
Test Case
Execution Task
Uncertainty
Metric

27. Concrete tool pipeline
TECPS'17, 13 July, 2017, Santa Barbara, USA 27
HINC
(runtime)Docker
Registry
Artifact
Repository
Extracted
information
(JSON)
Deployment
Description
(YAML/TOSCA)
Generate
Configurations &
Deployment
SALSA, Docker
and gcloud
utilities

28. Examples
TECPS'17, 13 July, 2017, Santa Barbara, USA 28
"MQTTConfig1": {
"name": "MQTTConfigServer",
"protocolType": "MQTT",
"qosLevel": [],
"type": "CommunicationConfiguration"
}
"MQTTConfig2": {
"name": "MQTTConfigClient",
"protocolType": "MQTT",
"clientID": "",
"serverIP": "35.189.187.208",
"portNumber": 1883,
"topics": ["/gateway/electricity"],
"qosLevel": [2],
"type": "CommunicationConfiguration"
}
services:
ingest:
build: .
volumes:
- ./:/t4u
electricitysensor:
image:
"localhost:5000/t4u/mqttsensor/realsensor:
v01"
iotgateway:
image:
"localhost:5000/t4u/cloudservice/mqttbroke
r:v01"
Enriched model information for
deployment configurations Generated deployment description

29. TECPS'17, 13 July, 2017, Santa Barbara, USA 29
Example: using SALSA for deployment and
configurations
http://tuwiendsg.github.io/SALSA/

30. Summary
 Testing uncertainties need to deal with elasticity and
virtualization
 Different tools for modeling, provisioning and testing
uncertainties based on MDE/MBT and elastic techniques
 Closing gaps in testing uncertainties by introducing
novel methods and tool pipelines
 Our future work
 Prototype our approach for uncertainty testing
 Machine learning/big data analytics for uncertainty analytics
(www.u-test.eu)
Check https://github.com/tuwiendsg/COMOT4U for new
update
TECPS'17, 13 July, 2017, Santa Barbara, USA 30

31. Thanks for your
attention!
Hong-Linh Truong
Distributed Systems Group
TU Wien
rdsea.github.io
TECPS'17, 13 July, 2017, Santa Barbara, USA 31