Internet of Things technologies and applications are evolving and continuously gaining traction in all fields and environments, including homes, cities, services, industry and commercial enterprises. However, still many problems need to be addressed. For instance, the IoT vision is mainly focused on the technological and infrastructure aspect, and on the management and analysis of the huge amount of generated data, while so far the development of front-end and user interfaces for IoT has not played a relevant role in research. On the contrary, user interfaces in the IoT ecosystem they can play a key role in the acceptance of solutions by final adopters. In this paper we present a model-driven approach to the design of IoT interfaces, by defining a specific visual design language and design patterns for IoT\ applications, and we show them at work. The language we propose is defined as an extension of the OMG standard language called IFML.

1. Firma convenzione
Politecnico di Milano e Veneranda Fabbrica del
Duomo di Milano
• Aula Magna – Rettorato
• Mercoledì 27 maggio 2015
Model-driven Development of
User Interfaces for IoT via
Domain-specific Components & Patterns
Marco Brambilla, Eric Umuhoza

2. M Brambilla, E. Umuhoza. Model-driven Development of UIs for IoT via Domain-specific Components and Patterns
Context and Motivation
 More than 30 billion IoT objects connected by 2020
 More than 2.6 billion of smart-phone users by 2020

3. M Brambilla, E. Umuhoza. Model-driven Development of UIs for IoT via Domain-specific Components and Patterns
Context and Motivation
(*) IN 2010:
Predictions said:
from 50 billions to 1 trillion (IBM) in 2020!!!
(*) NOW:
At the moment:
from 7 billions (Gartner) to 9 billions (IDC) to 18 billions (HIS)

4. M Brambilla, E. Umuhoza. Model-driven Development of UIs for IoT via Domain-specific Components and Patterns
Motivation
The role of user interfaces towards IoT

5. M Brambilla, E. Umuhoza. Model-driven Development of UIs for IoT via Domain-specific Components and Patterns
Application Modeling
Mobile Applications
Contributions
IoT-based Applications
Platform Independent
Extension
(Mobile IFML)
Platform
Specific
Extension
Design
Patterns
Modeling
Tool
Platform
Specific
Extension
IoT-
Extension
Design
Patterns
Domain
Model for IoT
Implementation
Declarative rules for
Code Generation
Monitoring
Model Interpretation
Execution
Logs
Application
Model
Data Analysis and
Visualization
Database

6. Firma convenzione
Politecnico di Milano e Veneranda Fabbrica del
Duomo di Milano
• Aula Magna – Rettorato
• Mercoledì 27 maggio 2015
UI Modeling

7. M Brambilla, E. Umuhoza. Model-driven Development of UIs for IoT via Domain-specific Components and Patterns
Interaction Flow Modeling Language (IFML)
Album
Search
«Window» AlbumSearch
Album
List
«Window» Albums
Album
Details
«Window» Album
«ParameterBindingGroup»
Title  AlbumTitle
Year  AlbumYear
«ParameterBindingGroup»
SelectedAlbum  AnAlbum
«Form» «List» «Details»
EventViewContainer
View Component
Album
Deletion
«ParameterBindingGroup»
SelectedAlbum  AnAlbum
Action
Navigation Flow
ParameterBinding

8. M Brambilla, E. Umuhoza. Model-driven Development of UIs for IoT via Domain-specific Components and Patterns
Models in software development
Models as programs
• Applications are
generated (semi)
automatically from
models

9. M Brambilla, E. Umuhoza. Model-driven Development of UIs for IoT via Domain-specific Components and Patterns
Models in software development
Models as programs
• Applications are
generated (semi)
automatically from
models

10. M Brambilla, E. Umuhoza. Model-driven Development of UIs for IoT via Domain-specific Components and Patterns
Mobile–specific extension of IFML
 Four main sets of mobile-specific
concepts
• Mobile Containers and Components
• Events
• Mobile Actions
• Mobile Context
Bottom-up approach
Investigation of
Mobile
Platforms
Extraction of
Common
Features
Abstraction
and
Conceptualization
 Meta-model excerpts:

11. M Brambilla, E. Umuhoza. Model-driven Development of UIs for IoT via Domain-specific Components and Patterns
Mobile IFML - Events
 Events generated by the
interaction of the user
(gestures, …)
 Mobile container
MobileContainer
MobileEvent
«Screen» List
«List» Lists
Options
«LongPress»
«Screen» Options
«Details» List
Edit list Delete list

12. M Brambilla, E. Umuhoza. Model-driven Development of UIs for IoT via Domain-specific Components and Patterns
Mobile IFML – Access to native features
MobileComponent
MobileAction MobileActionEvent
 Access to system features
 Native functions
 Phone sensors

13. M Brambilla, E. Umuhoza. Model-driven Development of UIs for IoT via Domain-specific Components and Patterns
Mobile IFML – the Context object
 It assumes particular relevance in mobile apps
 The context must gather all the dimensions that characterize:
• The user's intent
• The capacity of access device
• The communication network

14. Firma convenzione
Politecnico di Milano e Veneranda Fabbrica del
Duomo di Milano
• Aula Magna – Rettorato
• Mercoledì 27 maggio 2015
IoT Modeling

15. M Brambilla, E. Umuhoza. Model-driven Development of UIs for IoT via Domain-specific Components and Patterns
IoT Extensions - Components
Get Action Set Action Plan Action
Action SystemEvent
Single Information
Event
Approaching Event
 IoT Actions
 IoT Events (approaching, repeat, etc.)
 No new ViewComponent and ViewContainer
Meta-model excerpts:

16. M Brambilla, E. Umuhoza. Model-driven Development of UIs for IoT via Domain-specific Components and Patterns
IoT user interaction patterns – Set
Make
Cappuccino
Turn on
100°C
 Set patterns
• One device – One operation
• One device – More operations
• More devices – One operation
• More devices – More operations
• One device – One Program

17. M Brambilla, E. Umuhoza. Model-driven Development of UIs for IoT via Domain-specific Components and Patterns
IoT user interaction patterns – Get
• State of a device
• Details of a device
• Information from device
• Search device
• Nearby devices
• …
 Get patterns

18. M Brambilla, E. Umuhoza. Model-driven Development of UIs for IoT via Domain-specific Components and Patterns
Patterns- based UI modeling
 Get Information from the Device
 Get State of the Device
 Get Details of a Device
 Get Information from whole Category
 One Device One Operation
 Store Information
 Push Information

19. M Brambilla, E. Umuhoza. Model-driven Development of UIs for IoT via Domain-specific Components and Patterns
The full example…

20. M Brambilla, E. Umuhoza. Model-driven Development of UIs for IoT via Domain-specific Components and Patterns

21. Firma convenzione
Politecnico di Milano e Veneranda Fabbrica del
Duomo di Milano
• Aula Magna – Rettorato
• Mercoledì 27 maggio 2015
Code Generation

22. M Brambilla, E. Umuhoza. Model-driven Development of UIs for IoT via Domain-specific Components and Patterns
Code generation strategies
 CIM to several PIMs
 PIM to several PSMs
• Many other combinations
• Skipping one of the levels
Model-driven architecture (MDA)

23. M Brambilla, E. Umuhoza. Model-driven Development of UIs for IoT via Domain-specific Components and Patterns
Code generation strategies
Model-to-Model
Transformation (M2M)
Computation
Independent Model (CIM)
Platform Specific
Model (PSM)
Platform Specific
Model (PSM)
Platform Specific
Model (PSM)
M2M
Application Code
Model-to-Text
Transformation (M2T)
FSM :Cross-platform
Framework Specific Model
PIM
Native
Code
M2TM2M
PSM
PIM
Native
Code
M2T
Native
Code
M2T
PSM
PIM
Cross-platform
Code
M2T
M2T
FSM
Cross-platform
CodePIM
M2M
(1)
(2)
(3)
(4)
(5)
MDA
Platform Independent
Model (PIM)

24. M Brambilla, E. Umuhoza. Model-driven Development of UIs for IoT via Domain-specific Components and Patterns
PIM to Cross-platform Code
 From PIM it generates the code required by the cross platform framework
• PhoneGap, AppCelerator Titanium, and Xamarin
 Requires only one model and one generator
• Plus tricks and tweaks!!
 USE: Multiple platform availability (and time to market) is more important than high performance

25. M Brambilla, E. Umuhoza. Model-driven Development of UIs for IoT via Domain-specific Components and Patterns
User Behavior Analytics Approach
Example of WebRatio Runtime log line.RTXLog,
22 Jun 2016 11:10:51,761 DEBUG [http-bio-8080-exec-5] com.webratio.rtx.core.ServiceProvider:45) -
[119354A67C7C0177D4A7F411E75BCDE7][page21][pwu6Block] Creating service: WEB-INF/descr/pwu6Block.descr
Example of enriched Log line
[119354A67C7C0177D4A7F411E75BCDE7][ViewContainer[Type:page,id:page21,..]][List[id=pwu6Block][dataBinding
=Book]][[title:"...",author:"...",...]]

26. M Brambilla, E. Umuhoza. Model-driven Development of UIs for IoT via Domain-specific Components and Patterns
Approach - Analysis and Visualization

27. M Brambilla, E. Umuhoza. Model-driven Development of UIs for IoT via Domain-specific Components and Patterns
Experience: the Sem-IoT-y platform
www.semioty.com

28. M Brambilla, E. Umuhoza. Model-driven Development of UIs for IoT via Domain-specific Components and Patterns
A Predefined
Content Model

29. M Brambilla, E. Umuhoza. Model-driven Development of UIs for IoT via Domain-specific Components and Patterns
Experience: the Sem-IoT-y platform

30. M Brambilla, E. Umuhoza. Model-driven Development of UIs for IoT via Domain-specific Components and Patterns
Conclusions
 Model-driven development of mobile and IoT
applications
 Very much Domain Specific
 Content model is predefined
 Basic interaction is back to content-model driven (as
from early 2000s)

31. Thanks!
Questions?
Marco Brambilla
@marcobrambi
marco.brambilla@polimi.it
www.marco-brambilla.com
M Brambilla, E. Umuhoza. Model-driven Development of UIs for IoT via Domain-specific Components and Patterns