Final PhD Defence presented in March 2016 at the University of Padua, Italy. 3 years PhD under the supervision of Prof. Ombretta Gaggi. Work focused on how it is possible to use smartphone to understand and analyse user behaviour, and how it is possible to use this information to further promote better lifestyle to individuals.
9. Ubiquitous Computing
4
“The most profound technologies
are those that disappear. They
veal themselves into the fabric of
everyday life until they are
indistinguishable from it”
36. • Aim at reducing the problem of market
fragmentation and different OS and APIs
• “Write once, run everywhere” approach
• Four different architectures
Cross-platform frameworks
16
45. Game with animations,
sounds, etc., developed with
all the analysed frameworks.
Considered aspects:
• Licenses
• APIs
• Community/Support
• Code Complexity
• Supported devices
• etc.
Developer point-of-view
19
46. Game with animations,
sounds, etc., developed with
all the analysed frameworks.
Considered aspects:
• Licenses
• APIs
• Community/Support
• Code Complexity
• Supported devices
• etc.
Developer point-of-view
19
jQuery PhoneGap Titanium MoSync
API 2 2 4.5 3
Complexity 4 4 5 2
IDE - - 5 4
Devices 5 4 4 2
… … … … …
47. 20
User point-of-view
Objective analysis of energy
consumption for data acquisition
from smartphone sensors.
Tests with two Android and two
Apple devices.
Results:
• Cross-platform frameworks
require higher energy from
battery
• Different frameworks
categories employ different
consumption
48. 20
User point-of-view
Objective analysis of energy
consumption for data acquisition
from smartphone sensors.
Tests with two Android and two
Apple devices.
Results:
• Cross-platform frameworks
require higher energy from
battery
• Different frameworks
categories employ different
consumption
49. 20
User point-of-view
Objective analysis of energy
consumption for data acquisition
from smartphone sensors.
Tests with two Android and two
Apple devices.
Results:
• Cross-platform frameworks
require higher energy from
battery
• Different frameworks
categories employ different
consumption
50. 20
User point-of-view
Objective analysis of energy
consumption for data acquisition
from smartphone sensors.
Tests with two Android and two
Apple devices.
Results:
• Cross-platform frameworks
require higher energy from
battery
• Different frameworks
categories employ different
consumption
74. Stress Induction Analysis
29
Initial
Relax
Relaxed
Tasks
Stressor Stressed
Tasks
ESM 1 ESM 2 ESM 3 ESM 4 ESM 5
TEST t(13) p-value
ESM 3 VS ESM 4 1.99 0,007 *
ESM 3 VS ESM 5 -2.84 0,009 *
ESM 4 VS ESM 5 -2.74 0.5
Participants were
stressed
Different stress level
at the end of tasks
Kept stressed during
stress tasks
75. Scroll and Swipe prediction model
30
F-measure for Scroll gesture models
MODEL DT KNN SVM NN BN
USER
(AVERAGE)
0.79 0.80 0.81 0.80 0.77
GLOBAL
(AVERAGE)
0.73 0.71 0.78 0.74 0.67
F-measure for Swipe gesture models
MODEL DT KNN SVM NN BN
USER
(AVERAGE)
0.86 0.86 0.79 0.87 0.85
GLOBAL
(AVERAGE)
0.92 0.75 0.81 0.82 0.77
76. User Model:
• Digits size (64% of users with strong correlation)
• Pressure/Size ratio (55% of users with strong
correlation)
Global Model:
• Wrong Words / Total words ratio (p-value =
0,028)
• Digits time distance (p-value = 0,012)
• Digit duration (p-value = 0,08)
Text writing statistical significance
31
85. • Stair steps recognition in real-time
• Single player and multi players mode
• Both competition and collaboration with other
players
ClimbTheWorld
35
98. • Data collected from 7 different users with their own
smartphone
• 8000 windows, 1500 stair steps
• Different methods from gravity removal, different
data frequencies and different learning algorithms
Stairstep recognition
37
99. • Data collected from 7 different users with their own
smartphone
• 8000 windows, 1500 stair steps
• Different methods from gravity removal, different
data frequencies and different learning algorithms
Stairstep recognition
37
0.65
0.7
0.75
0.8
0.85
0.9
Mizell Linear Our
Method
Mizell Linear Our
Method
Mizell Linear Our
Method
20Hz 30Hz 50Hz
F-score
DT KNN KOMD
101. • User test with 13 participants
• Divided into two groups, each one followed a
particular program of game usage
• 9 days length, the firsts and the last two days using
only a stair step counter without the game
• Questionnaire for user engagement, logger to
understand the number of stair steps made
depending on the game mode
User engagement and behaviour change
39
102. • User test with 13 participants
• Divided into two groups, each one followed a
particular program of game usage
• 9 days length, the firsts and the last two days using
only a stair step counter without the game
• Questionnaire for user engagement, logger to
understand the number of stair steps made
depending on the game mode
User engagement and behaviour change
39
105. • Children as Digital
Native Speakers
• Serious games and
mobile devices for
diagnosis tests instead
of normal ones
designed for adults
• More engaging games
can make tests longer
and more precise
Children and technology
41
107. • Tested with 65 children
• Evaluation both from children and diagnosis point of view
• Two children identified with eye problems
PlayWithEyes
43
108. • Tested with 65 children
• Evaluation both from children and diagnosis point of view
• Two children identified with eye problems
PlayWithEyes
43
3 years old 4 years old 5 years old
First eye (left) 4’27 5’13 3’42
Second eye (right) 3’02 3’06 3’15
Total Time 7’29 8’19 6’57
111. • Analysis of cross-platform frameworks for mobile
analysis
• Frameworks still not mature for a strong adoption
• User Interface update is the most expensive task
• Future works
• constantly monitor frameworks evolution
Cross-platform frameworks
45
112. • Stress assessment using human-smartphone
interaction, in particular using scroll, swipe and text
writing
• F-measure between 73-85% for stress recognition
• Future works
• In-the-wild approach monitoring behaviour with
smartphone
• Stress reduction strategies using mobile devices
Stress assessment
46
113. • Serious games are a good strategy for behaviour
change
• Medical diagnosis can reach same results but in a
more engaging way
• Future works
• evaluation of long term effects of serious games
for behaviour change
• Serious games usage for diagnosis or
rehabilitation for different diseases
Serious games for health
47
114. Smartphones as ubiquitous devices
for behaviour analysis
and better lifestyle promotion
PhD Student: Matteo Ciman
Supervisor: Ombretta Gaggi
Thank You!