Smartphones as ubiquitous devices for behavior analysis and better lifestyle promotion

1. Smartphones as ubiquitous devices for behaviour analysis and better lifestyle promotion PhD Student: Matteo Ciman Supervisor: Ombretta Gaggi

2. Outline

3. Outline

4. Outline

5. Outline

6. Outline

7. Ubiquitous Computing 4

9. Ubiquitous Computing 4 “The most profound technologies are those that disappear. They veal themselves into the fabric of everyday life until they are indistinguishable from it”

11. Ubiquitous computing for health 6

12. Health costs 7

13. Lifestyle problem 8

18. Wearables for health 10

19. Ubiquitous Computing for health 11

26. Smartphones for healthcare 12

27. Smartphones for healthcare 12 Gyro Sensor Proximity Sensor Accelerometer Geomagnetic Sensor Light Sensor Barometer Temperature Sensor

32. Cross-Platform frameworks

33. Users in health applications 14

34. Users in health applications 14

35. Market fragmentation 15

36. • Aim at reducing the problem of market fragmentation and different OS and APIs • “Write once, run everywhere” approach • Four different architectures Cross-platform frameworks 16

37. Cross-Platform frameworks 17

38. Cross-Platform frameworks 17 Web Approach

39. Cross-Platform frameworks 17 Web Approach Hybrid Approach

40. Cross-Platform frameworks 17 Web Approach Hybrid Approach Interpreted Approach

41. Cross-Platform frameworks 17 Web Approach Hybrid Approach Cross-Compiled ApproachInterpreted Approach

42. Frameworks analysis 18

43. Frameworks analysis 18 Developer Point-Of-View

44. Frameworks analysis 18 User Point-Of-ViewDeveloper Point-Of-View

45. Game with animations, sounds, etc., developed with all the analysed frameworks. Considered aspects: • Licenses • APIs • Community/Support • Code Complexity • Supported devices • etc. Developer point-of-view 19

46. Game with animations, sounds, etc., developed with all the analysed frameworks. Considered aspects: • Licenses • APIs • Community/Support • Code Complexity • Supported devices • etc. Developer point-of-view 19 jQuery PhoneGap Titanium MoSync API 2 2 4.5 3 Complexity 4 4 5 2 IDE - - 5 4 Devices 5 4 4 2 … … … … …

47. 20 User point-of-view Objective analysis of energy consumption for data acquisition from smartphone sensors. Tests with two Android and two Apple devices. Results: • Cross-platform frameworks require higher energy from battery • Different frameworks categories employ different consumption

52. Smartphone for stress assessment

53. Mood recognition with smartphones 22

56. The idea 23

57. The idea 23

58. Human-smartphone interaction 24

59. Human-smartphone interaction 25

60. Human-smartphone interaction 25 Tap Scroll Swipe Text Writing Double Tap Rotate Zoom Pinch Long press

61. Human-smartphone interaction 25 Tap Scroll Swipe Text Writing Double Tap Rotate Zoom Pinch Long press

62. Search & Write Task 26

67. Protocol 27 Initial Relax (5’) Relaxed Tasks (~30’) Self Assessment Negative Valence Low energy Not Stressed Positive Valence High Energy Stressed 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 ESM 1 ESM 2 ESM 3 ESM 4 ESM 5 Stressor (5’-10’) Stressed Tasks (~10’)

68. • Mathematical problems • Timing pressure • Uncontrollability • Repetition Stressor Task 28

70. • Mathematical problems • Timing pressure • Uncontrollability • Repetition Stressor Task 28 Big prime number

72. • Mathematical problems • Timing pressure • Uncontrollability • Repetition Stressor Task 28 Random decrease Vibrations and sound

73. Stress Induction Analysis 29 Initial Relax Relaxed Tasks Stressor Stressed Tasks ESM 1 ESM 2 ESM 3 ESM 4 ESM 5

74. Stress Induction Analysis 29 Initial Relax Relaxed Tasks Stressor Stressed Tasks ESM 1 ESM 2 ESM 3 ESM 4 ESM 5 TEST t(13) p-value ESM 3 VS ESM 4 1.99 0,007 * ESM 3 VS ESM 5 -2.84 0,009 * ESM 4 VS ESM 5 -2.74 0.5 Participants were stressed Different stress level at the end of tasks Kept stressed during stress tasks

75. Scroll and Swipe prediction model 30 F-measure for Scroll gesture models MODEL DT KNN SVM NN BN USER (AVERAGE) 0.79 0.80 0.81 0.80 0.77 GLOBAL (AVERAGE) 0.73 0.71 0.78 0.74 0.67 F-measure for Swipe gesture models MODEL DT KNN SVM NN BN USER (AVERAGE) 0.86 0.86 0.79 0.87 0.85 GLOBAL (AVERAGE) 0.92 0.75 0.81 0.82 0.77

76. User Model: • Digits size (64% of users with strong correlation) • Pressure/Size ratio (55% of users with strong correlation) Global Model: • Wrong Words / Total words ratio (p-value = 0,028) • Digits time distance (p-value = 0,012) • Digit duration (p-value = 0,08) Text writing statistical signiﬁcance 31

78. Activity Recognition and Serious Games for behaviour change

79. The Piano Stairs Project 33

80. The Piano Stairs Project 33

81. From stairs to smartphone 34

85. • Stair steps recognition in real-time • Single player and multi players mode • Both competition and collaboration with other players ClimbTheWorld 35

86. ClimbTheWorld 36

98. • Data collected from 7 different users with their own smartphone • 8000 windows, 1500 stair steps • Different methods from gravity removal, different data frequencies and different learning algorithms Stairstep recognition 37

99. • Data collected from 7 different users with their own smartphone • 8000 windows, 1500 stair steps • Different methods from gravity removal, different data frequencies and different learning algorithms Stairstep recognition 37 0.65 0.7 0.75 0.8 0.85 0.9 Mizell Linear Our Method Mizell Linear Our Method Mizell Linear Our Method 20Hz 30Hz 50Hz F-score DT KNN KOMD

100. Energy consumption 38 11500 12000 12500 13000 13500 14000 14500 15000 Sliding window Data segmentation Energy consumption (µAh) }1 hour of battery life

101. • User test with 13 participants • Divided into two groups, each one followed a particular program of game usage • 9 days length, the ﬁrsts and the last two days using only a stair step counter without the game • Questionnaire for user engagement, logger to understand the number of stair steps made depending on the game mode User engagement and behaviour change 39

102. • User test with 13 participants • Divided into two groups, each one followed a particular program of game usage • 9 days length, the ﬁrsts and the last two days using only a stair step counter without the game • Questionnaire for user engagement, logger to understand the number of stair steps made depending on the game mode User engagement and behaviour change 39

103. Serious games for medical diagnosis

104. Serious games for medical diagnosis

105. • Children as Digital Native Speakers • Serious games and mobile devices for diagnosis tests instead of normal ones designed for adults • More engaging games can make tests longer and more precise Children and technology 41

106. PlayWithEyes 42

107. • Tested with 65 children • Evaluation both from children and diagnosis point of view • Two children identiﬁed with eye problems PlayWithEyes 43

108. • Tested with 65 children • Evaluation both from children and diagnosis point of view • Two children identiﬁed with eye problems PlayWithEyes 43 3 years old 4 years old 5 years old First eye (left) 4’27 5’13 3’42 Second eye (right) 3’02 3’06 3’15 Total Time 7’29 8’19 6’57

110. Conclusions & Future works

111. • Analysis of cross-platform frameworks for mobile analysis • Frameworks still not mature for a strong adoption • User Interface update is the most expensive task • Future works • constantly monitor frameworks evolution Cross-platform frameworks 45

112. • Stress assessment using human-smartphone interaction, in particular using scroll, swipe and text writing • F-measure between 73-85% for stress recognition • Future works • In-the-wild approach monitoring behaviour with smartphone • Stress reduction strategies using mobile devices Stress assessment 46

113. • Serious games are a good strategy for behaviour change • Medical diagnosis can reach same results but in a more engaging way • Future works • evaluation of long term effects of serious games for behaviour change • Serious games usage for diagnosis or rehabilitation for different diseases Serious games for health 47

114. Smartphones as ubiquitous devices for behaviour analysis and better lifestyle promotion PhD Student: Matteo Ciman Supervisor: Ombretta Gaggi Thank You!

Smartphones as ubiquitous devices for behavior analysis and better lifestyle promotion

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (18)

Similar to Smartphones as ubiquitous devices for behavior analysis and better lifestyle promotion

Similar to Smartphones as ubiquitous devices for behavior analysis and better lifestyle promotion (20)

More from University of Geneva

More from University of Geneva (6)

Recently uploaded

Recently uploaded (20)

Smartphones as ubiquitous devices for behavior analysis and better lifestyle promotion