Holistic User Modeling for Personalized Services in Smart Cities

@cataldomusto
Holistic User Modeling for
Personalized Services in Smart Cities
CATALDO MUSTO, GIOVANNI SEMERARO
MARCO DE GEMMIS, PASQUALE LOPS, MARCO POLIGNANO
UNIVERSITÀ DEGLI STUDI DI BARI ‘ALDO MORO’ - ITALY
cataldo.musto@uniba.it

The ‘Egosystem’
Cataldo Musto, Giovanni Semeraro, Marco de Gemmis, Pasquale Lops, Marco Polignano
Holistic User Modeling for Personalized Services in Smart Cities . i-CITIES 2018. L’Aquila, 19-21 September 2018

Data Silos Problem

What about personalization?
X

Research Questions
‘’ Is it possible to build a
unique representation of
the user merging data
extracted from personal
devices with data
extracted from social
networks? ‘’

Holistic User Model

Holistic User Model
Affects
Demographics
Interests
Behaviors
Social Relations
Knowledge and
Skills
Physical States
Cognitive Aspects
Inspiredy by
Cena, F., Likavec, S., and Rapp, A. Real world user model: Evolution of user modeling triggered
by advances in wearable and ubiquitous computing. Information Systems Frontiers, 2018

Holistic User Model
Name Description
Demographics Models demographic data (age, weight, name, etc.)
Interests Models interests and preferences
Affects Models mood and emotions of the user
Cognitive Aspects Models cognitive traits (personality, emphaty, etc.)
Behaviors Models the activities of the user
Social Relations Models the connections and the relations
Physical States Models physical data (sleep, food, heart rate, etc.)
Knowledge and Skills Models knowledge and skills of the user

How can we build a Holistic User Model?

Holistic User Modeling Workflow

Data Acquisition - Sources
Twitter
Facebook LinkedIn
Android
FitBit
Instagram

Data Acquisition
Twitter
Dato Descrizione
Profile Demographic information extracted from the profile (name,
location, website, #followers, #following, etc.)
Post Textual content of each post, date, language, #likes e retweet,
latitude and longitude (if any)
Connections Username, kind of relations (following/follower)

Data Acquisition
Facebook
Dato Descrizione
surname, profile pic, sex, age, location)
Post Textual content of the post, date, language, story (if any)
Friends Username
Likes Name of the page, Category of the page, Description of the page

Data Acquisition
Instagram
Dato Descrizione
surname, profile pic)
Post Textual content of the post, hashtag , location (if any)
Friends Following/followers

Data Acquisition
LinkedIn
Dato Descrizione
surname, profile pic, language, work category)
Position Description and category of the current work

Data Acquisition
Android Devices
Dato Descrizione
GPS Data Latitude, Longitude, Accuracy, Timestamp
Contacts Name, Phone Number, Interactions
App Name of the app, category, daily usage
Display Display mode (on/off)
Usage Network used (Wi-Fi, 4G, etc.) and traffic
Device Brand and Model of the phone

Data Acquisition
FitBit
Dato Descrizione
surname, profile pic, birth date, height, weight, sex)
Activities Kind of activities (running, walking) duration, calories, distance
Heart Rate Heart rate and timestamp
Sleep Date, sleep duration, sleep Quality
Food Food, calories, date, time

Data Processing and Enrichment
Data are processed through a Natural Language Processing and Machine Learning pipelines
Natural Language Processing
◦ Language Detection
◦ Stop-words Removal
◦ Lemmatization
◦ Entity Linking
◦ Wikipedia Categories Identification
Machine Learning (work-in-progress)
◦ Sentiment Analysis for Italian Tweets
◦ Emphathy and Personality Detection from Text
◦ Models for Activity Detection
◦ etc.

Holistic User Model
(recap)
Affects
Demographics
Interests
Behaviors
Social Relations
Knowledge and
Skills
Physical States
Cognitive Aspects

Holistic Profile Builder
12 demographics attributes are modeled
◦ Name, surname, profile pic, email, gender, location, height,
weight, working position, industry, language
Many attributes are general and available in many sources
◦ Name, surname, profilePic, etc.
Other attributes are source-specific
◦ e.g., height, weight from FitBit, working position from LinkedIn
Demographics

Interests can be both explicitly and implicitly modeled
◦ Explicitly: interests are obtained from the categories of the Pages
liked by the user on Facebook (e.g., sports, politics, etc.) and from the
apps she used;
◦ Implicitly: interests are inferred by mining relevant entities
mentioned in posts (with a positive or neutral sentiment) written by
the user on Twitter and Facebook, from the hashtags used on
Instagram, etc.
In both cases, the Interest facet is populated with some keywords
and entities extracted from the posts and copied from the Pages
Interests

User Mood and Emotions are encoded in the ‘Affects’ facet and
are Implicitly obtained by mining textual content
Affects
Cognitive Aspects
Personality traits are obtained by automatically inferring Big-5 traits
(extraversion, openness to experience, Conscientiousness, neuroticism,
agreeableness) from content written by the user

User Activities and Visited Places are encoded in the
‘Cognitive Aspects’ facet
User Activities are obtained from FitBit and Android Phones
◦ Both of them exploit device sensors to implicitly infer users’ activities
(running, walking, etc.).
Visited Places are implicitly inferred from the geotag
available in Instagram pictures
Behaviors

Social Relations are currently inferred by merging all the
contacts available in the sources connected to the
platforms
Social Relations
Physical States of the user are explicitly extracted from
FitBit data (food, hearth rate, sleep quality, etc.)
Physical States

Data Exposure
Holistic User Profiles are made available to developers
and third-party services via a high-level REST api
◦e.g., http://90.147.102.243/api/profile/cataldo
◦Only the facets the user explicitly labeled as ‘public’ are
exposed via the endpoint
◦Developers have to request an API key to access to the
user profiles

Data Visualization - Login

Data Visualization – Linking Identities

Data Visualization – Privacy Settings

Data Visualization – Controlling Data Exposure

Data Visualization – User Interests

Data Visualization – Emotion Monitoring

Data Visualization – Sleep Monitoring

Conclusions
Holistic User Profiling
◦ Conceptual model based on eight different facets
◦ Built by merging the digital footprints gathered from several
heterogeneous sources
◦ Overcomes data silos problem!
Myrror
◦ Platform supporting the creation of holistic user profiles
◦ Users have full control over the data extracted and exposed
◦ Profiles are made available through a REST Api to third-party
services and developers

Future Work
Data Sources
◦Introduction of more sources for modeling user profiles
Data Processing and Enrichment
◦Room for improvement: many algorithms can be
integrated to implicitly infer users’ data from rough data
Experimental Evaluation
◦ Definition of an experimental scenario to assess about the
effectiveness of holistic user profiles.

Thank you!
cataldo.musto@uniba.it
@cataldomusto
Want to try Myrror?
Contacts
Contact us! Working Online Demo

Holistic User Modeling for Personalized Services in Smart Cities

More Related Content

Similar to Holistic User Modeling for Personalized Services in Smart Cities

More from Cataldo Musto

Recently uploaded

Holistic User Modeling for Personalized Services in Smart Cities