Presentation of the paper "End User Development in the IoT: a Semantic Approach" at the 14th International Conference on Intelligent Environments (IE '18), held in Rome, Italy.

1. End User Development in the IoT:
a Semantic Approach
Alberto Monge Roffarello
Politecnico di Torino, Italy
e-Lite research group
http://elite.polito.it/
The 14th International Conference on IntelligentEnvironments,
Rome, Italy,25th -28th June 2018

2. OUTLINE
1. PROBLEM STATEMENT AND RESEARCH GOAL
2. EUPont: End User Programming Ontology
3. EUPont IN PRACTICE
▻EUDoptimizer
▻RecRules
4. CONCLUSIONS AND FURTHER DEVELOPMENTS
2

3. Problem Statement and
Research Goal
Helping End-Users in CustomizingTheir
IoT Devices and Services
1
3

4. “The Internet of Things is a recognized
paradigm that already helps society in
many different ways, through applications
ranging in scope from the individual to the
planetary, as well as across ventures in a
variety of industries.
Vint Cerf and Max Senges, Google Research
4

5. HUMAN-COMPUTER INTERACTION IN THE IoT
However, the increasing complexity of
the IoT ecosystem raises new
challenges, especially in the interaction
with final users:
▰ TECHNOLOGY DEPENDENCY
▰ INTEROPERABILITY
▰ INFORMATION OVERLOAD
5

6. END USER DEVELOPMENT IN THE IoT
In the context of the Internet of Things,End User
Development empowers end-users with and without
programming skills to customize their own IoT devices and
service on the basis of their personal needs.
Typically, third-party EUD interfaces allow users to define
simple TRIGGER-ACTION rules
6

7. Too many rules
ISSUES
John, a manager of an important company,
is always hot, especially in summer. He
loves air conditioning, and he would like to
set a low temperature wherever it is
possible.
At home, John has an intelligent Nest
thermostat, that he controls through his
Android smartphone. John goes to work by
his BMW smart car. There, all the offices
are equipped with a Samsung air
conditioner.
Too many technologies
Too many contexts
7

8. RESEARCH GOALExplore new approaches and methodologiesable to assist
end-users in customizing their Internet of Things systems
and services, with a particular focus on End-User
Development solutions for Trigger-Action Programming.
Context-Awareness
High Level of Abstraction
User Centered Design
Semantic Web Optimization Methods
User Preferences
8
End-User Development

9. EUPont
A High-Level Approach TowardsEnd
User Development in the IoT
2
9

10. 10
IF
I enter any defined location,
THEN
set its temperature to 20 Celsius degree

11. Place your screenshot here
EUPont
End User Programming Ontology
GOALS:
▰ Higher level of abstraction
▰ Programming by functionality
▰ Context dependentrules
11

12. EUPont is available at
http://elite.polito.it/ontologies/eupont.owl
It has been integrated in a user interface for
composing trigger-action rules, and has been
evaluated in multiple user studies.
Results of an in-the-wild evaluation further
demonstrates the potentialities of the approach
12
[1] F.Corno, L. De Russis, A. Monge Roffarello, «A High-Level Approach Towards End User Development in the IoT», CHI
2017: The 35th Annual CHI Conference on Human Factors in Computing Systems
[2] F.Corno, L. De Russis, A. Monge Roffarello, «A Semantic Web Approach to Simplifying Trigger-Action Programming in
the IoT», IEEE Computer, 2017

13. EUPont in Practice
Enriching Contemporary EUD Solutions
with SemanticFeatures
3
13

14. EUDoptimizer: Assisting the Composition of IF-THEN Rules With
an Optimizer in the Loop
14
The goal is to use combinatorial
optimization methods to enhance
EUD interfaces. By using models of
human performance and layout
perception, EUDoptimizer reduces
the efforts to compose trigger-
action rules.

15. 15
min (α * SDP + β * FSM)

16. 16
min (α * SDP + β * FSM)
SEARCH DECISION POINTING
A state-of-the-art model of human performance in
linear menu search. It models:
• Search Time
• Decision Time
• Pointing Time

17. 17
min (α * SDP + β * FSM)
FUNCTIONALITY SIMILARITY MODEL
A model to measure how devices and online services
are similar in terms of EUPont functionality
SEARCH DECISION POINTING
A state-of-the-art model of human performance in
linear menu search. It models:
• Search Time
• Decision Time
• Pointing Time

18. 18
Results
Time for defining a
Trigger
Time for defining
an Action
Time for defining a
Rule

19. 19
RecRules: Recommending IF-THEN Rules to End Users
The goal is to recommend by
functionality , i.e., suggesting
trigger-action rules on the basis of
the final behaviors users would like
to define, thus abstracting any
technological details such as
brands or manufactures.

20. 20

21. 21

22. 22

23. 23
EUPont INFORMATION

24. 24
EUPont INFORMATION

25. 25
Results

26. 26
if my Nest detects a
smoke alarm, then send
me an Android
SMS
Training Set

27. 27
if my Nest detects a
smoke alarm, then send
me an Android
SMS
if my Nest detects a
carbon monoxide alarm,
then send me a
notification on my Google
Glasses
if my Nest detects a
smoke alarm, then turn
the Philips Hue on
Training Set
Recommendation Set

28. Training Set
Recommendation Set
28
if my Nest detects a
smoke alarm, then send
me an Android
SMS
if my Nest detects a
carbon monoxide alarm,
then send me a
notification on my Google
Glasses
if my Nest detects a
smoke alarm, then turn
the Philips Hue on
LET ME KNOW IF SOMETHING IS WRONG IN MY HOME...

29. Conclusions and Further
Developments
Closing the Gap Between IoT and Users
4
29

30. CONCLUSIONS
▻ I defined EUPont, an ontological model for End-User
Development in the IoT to take a step towards a high level of
abstraction.
▻ EUPont is currently used in 2 research projects, with the aim of
enriching contemporary EUD solutions for trigger-action rules
with semantic capabilities.
▻ The usage of semantic technologiesallows the modeling of
trigger-action rules according to their final functionality
30

31. FUTURE WORKS
New Tools
I will explore new tools
for helpingpeople to
customize their devices
and services, e.g., a tool
to compose and actually
execute trigger-action
rules in the EUPont
representation
New Contexts
I will explore new
contexts in which
preference-based
approaches could be
adopted in the wide
Human Computer
Interaction field
New Users
I will explore new users
that needs more
accessible and usable
tools for customizing
their IoT ecosystem,
e.g., people with
disabilities
31

32. THANK YOU!
alberto.monge@polito.it
http://elite.polito.it
32