User interaction plays a crucial role in every system. This is true for IoT too. Sensors, actuators and intelligent things connected together can cooperate and exchange information, but their ultimate goal is to provide value to people. Such value can be perceived only through appropriate user interfaces, which visualise information (through dashboard, reports, or infographics), let user navigate the information, and also interact with the devices, by setting properties or regulating their behavior. That's why it's important that in the IIoT development context we consider also user interaction. This presentation introduces IFML, the Interaction Flow Modeling Language, a new OMG standard that focuses on user interfaces and their integration with information systems, data sources, sensors and actuators. The presentation reports on some success stories from the industry, where IFML has been successfully applied. Large scale examples include consumer-oriented user interfaces, backend systems, data analysis dashboards, and interactions for command and control. Adopters include GE, Acer Computer, manufacturing, banks, utilities and military. Bio: Marco Brambilla is professor of software engineering at Polytechnic University of Milan, in Italy. He is also partner of WebRatio Inc. and founder of the startup Fluxedo. He has been visiting researcher at University of California, San Diego, CISCO, San José, and Dauphine University, Paris. He has extensive experience in model-driven methods and languages, and he published several books in the field of model-driven specifications of user interactions. He is the main promoter and designer of the language IFML (Interaction Flow Modeling Language), which has been adopted as a standard by the OMG in 2014. His research interest span model-driven development, web technologies, crowdsourcing, social network and big data analysis. WebRatio produces a model-driven tool for Web, Mobile and IoT development based on IFML, providing full code generation for enterprise-class applications.

1. Interac(on
Flow
Modeling
Language
Internet of Things and Internet of People:
The Role of User Interaction in the IIoT vision
Marco
Brambilla
marco.brambilla@polimi.it
@marcobrambi

2. Specifica(on
of
IoT
and
IIoT
so
far
focused
mainly
on
marchine-­‐
to-­‐machine
interac(ons
• Things
talking
to
things
But
ul(mately
human
contribu(on
must
be
in
the
loop!
• Repor1ng
• Data
visualiza1on
• Data
analysis
• Data
explora1on
• Decision
making
• Storytelling
Context
and
need

3. The last mile to the
user: Costly and
inefficient process
Complexity of
user interfaces (UIs)
Ineffective
design tools
Manual specification of
data and visualization
No support for human
interpretation of data
The
UI
Design
Problem
–
UIs
for
IIoT

4. " Hence
the
Interac(on
Flow
Modeling
Language
(IFML)
Standardiza(on
gap

5. No model
driven
engineering
Platform independent
description of UIs
Focused on user
interactions
No definition of
graphics and styles
Reference to
external models
The
UI
Design
solu(on:
IFML

6. IFML
Objec(ves
Binding to
Persistence Layer
(data storage)
Interaction
Options and
Navigation Paths
Binding to
Business Logic
Content
of the UI
User and
System Events

7. IFML
by
example

8. IFML
by
example
Artists
List
Artists
Artist
Event
View
Component
View Container
ParameterBinding«ParameterBindingGroup»
SelectedArtist à AnArtist

9. IFML
by
example
Album
Search
«Window» AlbumSearch
Album
List
«Window» Albums
Album
Details
«Window» Album
«ParameterBindingGroup»
Title à AlbumTitle
Year à AlbumYear
«ParameterBindingGroup»
SelectedAlbum à AnAlbum
«Form» «List» «Details»

10. IFML
by
example
Album
List
Albums
Album
Deletion
«ParameterBindingGroup»
SelectedAlbum à AnAlbum

11. Mul(ple
aspects
modeling
UML
Use
Case
BPMN
process
UML
Sequence
IFML
UML
Statechart
Handle Rental
Sales Clerk
Handle Renter
<<Include>> <<Extend>>
Handover Car
<<UML Actor>>
Sales Clerk
<<UML
Model>>
IT
system
new rental
change
days
accept
payment

12. Integra(on
with
BPMN
The
UI
of
each
ac1vity
can
be
described
by
" An
IFML
module
" Some
UML
dynamic
diagrams
(e.g.,
ac1vity,
sequence,
…)
Execute
the
payment
«ParameterBindingGroup»
Name à Name
«ParameterBindingGroup»
Name à Name
CreditCard à CC
Customer
Information
Customer Information
«Form»
Payment
Information
Payment Information
«Form»
Amount
Confirmation
Payment
Execution

13. Given
that
IFML
is
an
EXECUTABLE
model…
• Possibility
of
complete
code
genera1on
of
the
Uis
• Extremely
quick
1me-­‐to-­‐market
• Reliable
and
bug-­‐free
code
• Repeatable,
reusable,
and
coherent
resul1ng
UI
Executability

14. Successful
IFML
projects
in
industry
Energy
Finance
Research
Transportation
Government
Industry Retail and Luxury
Non
Disclosable
Logo
Non
Disclosable
Logo
Non
Disclosable
Logo
Major Worldwide
Furniture Prod.

15. Worldwide public portal
www.acer.com
Workflow system for managing
content production, approval and
visualization
60 Countries
22 Languages
Q&A and internal trouble ticket
system
System for organizing cash register
intake at the end of the day
Used in 42 Countries
Online leasing quotations
Creditworthiness evaluation
system
Building construction check-up
system
220 roles
Fleet Management System
integrated with legacy systems and
high standard security
Used in 3 countries
By 150K-200K users
Success
Stories
of
IFML
and
WebRa(o
Major Worldwide
Furniture Producer

16. A
Web
Intranet
managing
all
the
informa(on
about
the
core
objects
of
the
Fleet
Services
business:
drivers
and
vehicles.
A
set
of
dashboards,
to
allow
fleet
managers
to
monitor
the
costs
and
performance
trends
of
their
fleets.
General
Electrics
–
Fleet
Services

17. Developed
Applica(ons:
Program
and
Materials

18. Vehicle
-­‐
Billing

19. Vehicle
–
Expense
Overview

20. Developed
Applica(ons:
Performance
dashboard

21. Developed
Applica(ons:
Performance
dashboard
Developed with
85% time reduction

22. Front-­‐end
Back-­‐end
Web portal
and mobile
app for
customers
Admin
Dashboard
for the water
supplier
Cloud + Big Data
infrastructure
Smart meters and
other home sensors
SmarterWater:
online
portal

23. Dashboard for
consumption control
Online billing and
payment
Real-time data from smart
meters at home
Comparison with optimal
consumption
Teaching and
resources on water
Gamification
Leaderboard, objectives
and badges, points
SmarterWater:
online
portal

24. Performance of the
program
Alarms for spills and
exceptional consumption
Customer
segmentation
Geospatial analytics
Management of
materials and training
CRM
SmarterWater:
Admin
Dashboard

25. A
web
system
for
electrical
supply
network
suppor(ng
• Monitoring
of
the
network
• Request,
planning,
and
execu1on
of
maintenance
• Repor1ng
and
analy1cs
For
ordinary
maintenance,
malfunc(onings,
accidents
of
130,000
network
nodes
and
33,000
produc(on
units
Management
of
breakdowns
on
country-­‐wise
electrical
network

26. Management
of
breakdowns
on
country-­‐wise
electrical
network

27. UI
specifica(on
with
IFML
Integra(on
with
BPMN
business
process
specifica(on
Monitoring
of
IIoT
devices
on
the
network
nodes
Scheduling,
web
service
interac(ons
Analy(cs
and
visualiza(on:
Gan_,
Timeline,
Excel,
PDF
IFML
use
for
electric
network
case

28. Request
and
management
process

29. UML model of concepts involved

32. A
real
case:
event
analy(cs
&
engagement
MEASURE AND IMPROVE
THE INTERACTION AT YOUR EVENTS
Mobile App
Social Network Analytics
IoT Data
Attendance
profiling
sensors
People
Counting

33. Internet
of
Things
and
Sensors
Sensors to measure
(visitors flow and attendance
enviromental data as
temperature, pression and
umidity)
Sensors to interact
(proximity to points of
interest, users profiling
age, ethnicity, gender)

34. Live
event
monitoring
Enable
data
sharing
and
decision
making

35. Live
event
monitoring
Enable
data
sharing
and
decision
making

36. People
Coun(ng
Real time data from sensors
People counters + Cloud data storage
Models in IFML
Mobile and Web App
Front-end + Back-end

37. Passenger
counter
on
buses

38. Internet
of
(Green)
Things
IIoT
in
agricolture
needs
user
monitoring
and
interac(on
too
www.internetofgreens.com

39. IIo(green)T
A Web app modeled with IFML is
connected to cloud, showing real-time
parameters of sensors implanted
in the fields and pots that detect the
garden’s humidity, temperature,
conductivity and pH.
These sensors are connected to circuits
that automatically irrigate and provide
the garden with nutrients when
necessary. Special grow lamps as well as
natural lighting illuminate everything.
Users can take decisions on how to act.

40. How
it
works
Fertilizer
Irrigation
Electropump
Electropump
EC sensor
Switch
Light
Light
Switch
EC sensor
EC sensor
Plants
Ph sensor
Light sensor
Temperature sensor
Humidity sensor
Noise sensor

41. IoT
is
not
just
about
“T”s!
• People
maTer
and
need
to
be
in
the
loop
You
need
coherent
and
consistent
methods
for
covering
design
of
things
and
people
perspec(ves
IFML
can
be
the
tool
for
this
For
more
info,
look
up
www.ifml.org
Take
home
message

42. Marco
Brambilla
marcobrambi
marco.brambilla@polimi.it
Thanks!

43. Addi(onal
examples
and
resources

44. IFML
example
–
online
payment
Product
List
Shopping Cart
«List»
Customer
Information
Customer Information
«Form»
Execute
the
payment
Checkout
«ParameterBindingGroup»
Name à Name
Payment
Information
Payment Information
«Form»
«ParameterBindingGroup»
Name à Name
CreditCard à CC
Confirmation
Confirmation
Message
«Details»
«ParameterBindingGroup»
Totalà Amount

45. IFML
concrete
syntax
–
modules
IFML
Modules
-­‐
defini1on
Execute
the
payment
«ParameterBindingGroup»
Name à Name
«ParameterBindingGroup»
Name à Name
CreditCard à CC
Customer
Information
Customer Information
«Form»
Payment
Information
Payment Information
«Form»
Amount
Confirmation
Payment
Execution

46. IFML
concrete
syntax
–
modules
IFML
Modules
-­‐
usage
Shopping Cart
Checkout
«ParameterBindingGroup»
Totalà Amount
Confirmation
Confirmation
Message
«Details»
Payment
Execution
Product
List
«List»

47. Some
Ads
“Model
Driven
SoXware
Engineering
in
Prac1ce”.
Brambilla,
Cabot,
Wimmer.
Morgan&Claypool,
USA,
2012
Interac(on
Flow
Modeling
Language
Model-­‐driven
UI
Engineering
of
Web
and
Mobile
Apps
with
IFML
Morgan
Kaufmann,
The
OMG
press,
USA,
Q4
2014