Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Holistic User Modeling for Personalized Services in Smart Cities
1. @cataldomusto
Holistic User Modeling for
Personalized Services in Smart Cities
CATALDO MUSTO, GIOVANNI SEMERARO
MARCO DE GEMMIS, PASQUALE LOPS, MARCO POLIGNANO
UNIVERSITÀ DEGLI STUDI DI BARI ‘ALDO MORO’ - ITALY
cataldo.musto@uniba.it
2. The ‘Egosystem’
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
3. Data Silos Problem
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
4. What about personalization?
X
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
5. Research Questions
‘’ Is it possible to build a
unique representation of
the user merging data
extracted from personal
devices with data
extracted from social
networks? ‘’
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
6. Holistic User Model
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
7. Holistic User Model
Affects
Demographics
Interests
Behaviors
Social Relations
Knowledge and
Skills
Physical States
Cognitive Aspects
Inspiredy by
Cena, F., Likavec, S., and Rapp, A. Real world user model: Evolution of user modeling triggered
by advances in wearable and ubiquitous computing. Information Systems Frontiers, 2018
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
8. Holistic User Model
Name Description
Demographics Models demographic data (age, weight, name, etc.)
Interests Models interests and preferences
Affects Models mood and emotions of the user
Cognitive Aspects Models cognitive traits (personality, emphaty, etc.)
Behaviors Models the activities of the user
Social Relations Models the connections and the relations
Physical States Models physical data (sleep, food, heart rate, etc.)
Knowledge and Skills Models knowledge and skills of the user
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
9. How can we build a Holistic User Model?
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
10. Holistic User Modeling Workflow
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
11. Holistic User Modeling Workflow
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
12. Data Acquisition - Sources
Twitter
Facebook LinkedIn
Android
FitBit
Instagram
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
13. Data Acquisition
Twitter
Dato Descrizione
Profile Demographic information extracted from the profile (name,
location, website, #followers, #following, etc.)
Post Textual content of each post, date, language, #likes e retweet,
latitude and longitude (if any)
Connections Username, kind of relations (following/follower)
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
14. Data Acquisition
Facebook
Dato Descrizione
Profile Demographic information extracted from the profile (name,
surname, profile pic, sex, age, location)
Post Textual content of the post, date, language, story (if any)
Friends Username
Likes Name of the page, Category of the page, Description of the page
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
15. Data Acquisition
Instagram
Dato Descrizione
Profile Demographic information extracted from the profile (name,
surname, profile pic)
Post Textual content of the post, hashtag , location (if any)
Friends Following/followers
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
16. Data Acquisition
LinkedIn
Dato Descrizione
Profile Demographic information extracted from the profile (name,
surname, profile pic, language, work category)
Position Description and category of the current work
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
17. Data Acquisition
Android Devices
Dato Descrizione
GPS Data Latitude, Longitude, Accuracy, Timestamp
Contacts Name, Phone Number, Interactions
App Name of the app, category, daily usage
Display Display mode (on/off)
Usage Network used (Wi-Fi, 4G, etc.) and traffic
Device Brand and Model of the phone
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
18. Data Acquisition
FitBit
Dato Descrizione
Profile Demographic information extracted from the profile (name,
surname, profile pic, birth date, height, weight, sex)
Activities Kind of activities (running, walking) duration, calories, distance
Heart Rate Heart rate and timestamp
Sleep Date, sleep duration, sleep Quality
Food Food, calories, date, time
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
19. Holistic User Modeling Workflow
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
20. Data Processing and Enrichment
Data are processed through a Natural Language Processing and Machine Learning pipelines
Natural Language Processing
◦ Language Detection
◦ Stop-words Removal
◦ Lemmatization
◦ Entity Linking
◦ Wikipedia Categories Identification
Machine Learning (work-in-progress)
◦ Sentiment Analysis for Italian Tweets
◦ Emphathy and Personality Detection from Text
◦ Models for Activity Detection
◦ etc.
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
21. Holistic User Modeling Workflow
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
22. Holistic User Model
(recap)
Affects
Demographics
Interests
Behaviors
Social Relations
Knowledge and
Skills
Physical States
Cognitive Aspects
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
23. Holistic Profile Builder
12 demographics attributes are modeled
◦ Name, surname, profile pic, email, gender, location, height,
weight, working position, industry, language
Many attributes are general and available in many sources
◦ Name, surname, profilePic, etc.
Other attributes are source-specific
◦ e.g., height, weight from FitBit, working position from LinkedIn
Demographics
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
24. Holistic Profile Builder
Interests can be both explicitly and implicitly modeled
◦ Explicitly: interests are obtained from the categories of the Pages
liked by the user on Facebook (e.g., sports, politics, etc.) and from the
apps she used;
◦ Implicitly: interests are inferred by mining relevant entities
mentioned in posts (with a positive or neutral sentiment) written by
the user on Twitter and Facebook, from the hashtags used on
Instagram, etc.
In both cases, the Interest facet is populated with some keywords
and entities extracted from the posts and copied from the Pages
Interests
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
25. Holistic Profile Builder
User Mood and Emotions are encoded in the ‘Affects’ facet and
are Implicitly obtained by mining textual content
Affects
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
Cognitive Aspects
Personality traits are obtained by automatically inferring Big-5 traits
(extraversion, openness to experience, Conscientiousness, neuroticism,
agreeableness) from content written by the user
26. Holistic Profile Builder
User Activities and Visited Places are encoded in the
‘Cognitive Aspects’ facet
User Activities are obtained from FitBit and Android Phones
◦ Both of them exploit device sensors to implicitly infer users’ activities
(running, walking, etc.).
Visited Places are implicitly inferred from the geotag
available in Instagram pictures
Behaviors
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
27. Holistic Profile Builder
Social Relations are currently inferred by merging all the
contacts available in the sources connected to the
platforms
Social Relations
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
Physical States of the user are explicitly extracted from
FitBit data (food, hearth rate, sleep quality, etc.)
Physical States
28. Holistic User Modeling Workflow
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
29. Data Exposure
Holistic User Profiles are made available to developers
and third-party services via a high-level REST api
◦e.g., http://90.147.102.243/api/profile/cataldo
◦Only the facets the user explicitly labeled as ‘public’ are
exposed via the endpoint
◦Developers have to request an API key to access to the
user profiles
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
30. Holistic User Modeling Workflow
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
31. Data Visualization - Login
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
32. Data Visualization – Linking Identities
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
33. Data Visualization – Privacy Settings
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
34. Data Visualization – Controlling Data Exposure
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
35. Data Visualization – User Interests
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
36. Data Visualization – Emotion Monitoring
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
37. Data Visualization – Sleep Monitoring
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
38. Conclusions
Holistic User Profiling
◦ Conceptual model based on eight different facets
◦ Built by merging the digital footprints gathered from several
heterogeneous sources
◦ Overcomes data silos problem!
Myrror
◦ Platform supporting the creation of holistic user profiles
◦ Users have full control over the data extracted and exposed
◦ Profiles are made available through a REST Api to third-party
services and developers
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
39. Future Work
Data Sources
◦Introduction of more sources for modeling user profiles
Data Processing and Enrichment
◦Room for improvement: many algorithms can be
integrated to implicitly infer users’ data from rough data
Experimental Evaluation
◦ Definition of an experimental scenario to assess about the
effectiveness of holistic user profiles.
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018
40. Thank you!
cataldo.musto@uniba.it
@cataldomusto
Want to try Myrror?
Contacts
Contact us! Working Online Demo
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018