INTELLIGENT INTERACTIVE SYSTEMS - Ambient intelligence + Mobile technology

Fly See Buy
Barry Kollee
Selvi Ratnasingam
Sylvia van Schie
Wouter Stuifmeel
Robert Jan Prick
INTELLIGENT INTERACTIVE SYSTEMS | OCTOBER 8TH, 2013

Introduction: Topics
- ambient intelligence
- mobile technology

ambient intelligence (en.wikipedia, 7/9/2013):
“ambient intelligence (AmI) refers to electronic
environments that are sensitive and responsive
to the presence of people”

Mobile technology(en.wikipedia, 7/9/2013):
“a standard mobile device has gone from being
no more than a simple two-way pager to being
a mobile phone, GPS navigation device, an
embedded web browser and instant messaging
client, and a handheld game console”.

Introduction: Vision
Weiser, M. The Computer for the 21st Century, Scientific American (1991)

Introduction: Ambient Intelligence
“Specialized elements of hardware and
software, connected by wires, radio waves and
infrared, will be
so ubiquitous that no one will notice their
presence”

Introduction: Mobile technology
“Little is more basic to human perception than
physical juxtaposition, and so ubiquitous
computers must know where they are. If a
computer knows merely what room it is in, it
can adapt its behavior in significant ways….”

Introduction: Goal
“When things disappear ….. we are freed to
use them without thinking and so to focus on
new goals”

The Big Data Challenge
● Lausanne Data Collection Campaign
● A large-scale mobile data resource
● ‘Privacy by design’
● Image logfiles
● Monitor entire smartphone (N95)
Laurila, J. K., Gatica-Perez, D., Aad, I., Blom, J., & Bornet, O.The mobile data challenge: Big data for
mobile computing research. , http://privacybydesign.ca/

Laurila, J. K., et al.The mobile data challenge: Big data for mobile computing research.

● Semantic place prediction
● Next place prediction
● Demographic attribute prediction

Applying to Fly See Buy system
“Each data type corresponds to a table in which each row represents a record
such as a phone call or an observation of a WLAN access point. User IDs and
timestamps are the basic information for each record.“

How long?
● Investigate properties of learning
● Predicting social and individual models
Altshuler, Y., (2012). Incremental learning with accuracy prediction of social and individual properties
from mobile-phone data.

‘Reality mining’
A. Pentland, in The Global Information Technology Report 2008-2009 (World Economic Forum, Geneva,
2009)

Methodology
● Classifiers
○ Personal properties (first level)
○ Social links (life-partner?)
● Correlation amount of time vs. accuracy
Altshuler, Y., Aharony, N., Fire, M., Elovici, Y., & Pentland, A. S. (2012). Incremental learning with
accuracy prediction of social and individual properties from mobile-phone data.

Methodology
● Android Application
○ GPS
○ Accelerometer
○ Third Party application
○ Cell tower ID’s
○ WIFI LAN ID’s (proximity)

Methodology
● Feature vector
○ Location
○ Sms-pattern
○ Internet usage
○ Call-pattern
○ Phone applications
○ Alarms
● Friends and family dataset (140 people)

Conclusion
● Ethnicity 60%
● Is student
● Significant other 65 %

Conclusion
Modeled using ‘gompertz function’
“At a moment I can say with an amount of
certainty who you are.”
Laurila, J. K., Gatica-Perez, D., Aad, I., Blom, J., & Bornet, O.The mobile data challenge: Big data for
mobile computing research.

AdNext: A Visit-Pattern-Aware Mobile
Advertising System | for Urban Commercial Complexes
Keywords
● Mobile advertising
● Sequential visit patterns
● Prediction models
● Wi-Fi localization
● User survey
Kim, B., Ha, J., Lee, S., Kang, S., Lee, Y., Rhee, Y., . . . Song, J. (2011). AdNext: A visit-pattern-aware
mobile advertising system for urban commercial complexes. Proceedings of the 12th Workshop on
Mobile Computing Systems and Applications, Phoenix, Arizona. 7-12.

COEX Mall
● Largest commercial complex in South Korea
● 260 stores / 100.000 visitors per day
● Customer targeting
○ Spatial relevance
○ Temporal relevance
Image: Official Site of Korea Tourism http:
//www.visitkorea.or.
kr/enu/SI/SI_EN_3_1_1_1.jsp?cid=736121
Kim, B. et al AdNext: A visit-pattern-aware mobile advertising system for urban commercial
complexes. Proceedings of the 12th Workshop on Mobile Computing Systems and Applications,
Phoenix, Arizona. 7-12.

AdNext System Architecture

Collecting Place Visit History
● store-level localization accuracy
● identify users’ current location (using accelerometer)
● detect in/out time (location change validation)

Next Visit Prediction Model 1/2
Bayesian Networks – Probabilistic graphical model
It models the joint probability P(X, Y), where X represents
features and Y represents labels.
Main features:
● visit place (P)
● visit time (T)
● visit duration (D)
● gender (G) - static
● age (A) - static

Next Visit Prediction Model 2/2

Accuracy
● Data collection
● Prediction accuracy
● Comparison
Kim, B.et al (2011). AdNext: A visit-pattern-aware mobile advertising system for urban commercial

Discussion
● Privacy concerns
● Energy consumption
Kim, B.,et al (2011). AdNext: A visit-pattern-aware mobile advertising system for urban commercial

Context-aware Media Agent for Public
Spaces | Guide visitors in Museums
Authors: Ichiro Satoh
Satoh, Ichiro. Context-aware Media Agent for Public Spaces. Lansdale PA 19446: IOS Press, doi:
10.3233/978-1-60750-606-5-407, 0. Print. Ichiro, Satoh, (2008) Context-Aware Agents to Guide
Visitors in Museums, National Institute of Informatics, Tokyo, Japan Pages 441-455,

Portable terminals are not desirable
● requires end-users to carry players along
● require explicit input
● expensive devices to lend
● require regular maintenance
● interacting difficulties
● prevent visitors from focusing on the exhibits

Agent runtime system
Mobile agent
● annotation
● navigation
● user preference

User navigation patterns

Context-aware Annotation in Museum

Conclusion
This paper only described the design and implementation
of an agent-based system for building and operating
context-aware visitor-guide services in public museums.

Need a location-based sensor to track
the user.

Most used sensors
● RFID
● NFC
● WiFi

Probability kernel regression for
WiFi localisation
How the KL-divergence kernel regression
algorithm bridges the gap with other WiFi
localisation algorithm?
Mirowski, P., Whiting, P., Steck, H., Palaniappan, R., MacDonald, M., Hartmann, D., et al. (2012). Probability kernel
regression for WiFi localisation. Journal of Location Based Services, 6(2), 81-100.

WiFi localisation based on
● Fingerprints

Fingerprint?
The received signal strength (RSS) values from multiple
access points (AP) are collected at different reference
points and these reference points with related RSS are
referred as fingerprints.

Pattern matching process
Apply the learned model to real-time RSS
sample.

Fingerprint

Apply different statistical algorithm
● K-nearest neighbour
● Artificial neural network
● KL-divergence kernel regression algorithm

Contrasting dataset
● Office building (with dense and repeated
fingerprints)
● Auditorium (a large open space)
● Public space (with mixed layouts and heavy
pedestrian traffic)

Challenges
● Device-independent measurements
● Conditional independence of the RSS from a
single AP
● Choosing parameters, like N fingerprints, N
RSS sample

Conclusion
Very flexible and generalise several existing
WiFi localisation algorithm

Ambient system
● Project goals
○ Personalised recommendations for venues
(shopping, dining, etc.) in an airport terminal
environment
○ Recommendations are represented in an ambient
system in the public space
○ More foot traffic to airport venues
○ Increased revenue of venues

How do you distinguish
an ambient display from a regular
display?

Designing interactivity awareness
for ambient displays
● Addresses particular aspects of interaction
regarding ambient displays
● How can users make out which display is
interactive, and which isn't?
● How can users tell which type of interface
the ambient display exposes?
Vatavu, R. D. (2013). On designing interactivity awareness for ambient displays. Multimedia Tools and
Applications, 1-22.

Interaction techniques
A. Using software installed
on the mobile device
B. Natural interaction via
pointing and gestures
C. Touch-based interaction
D. Combining mobile
phones and gestures
Applications, 1-22.

Example of public display
Clear Channel gaat digitaal op Amsterdam CS. (2012, December 4). Clear Channel Hillenaar -
Buitenreclame. Retrieved October 7, 2013, from http://www.clearchannel.nl/nl/Nieuws/Laatste-
nieuws/Default.aspx?PageId=535

Study
● Study to investigate perception of
interactivity of displays
● Design
○ Repeated-measures design, 3 independent
variables:
■ Display
■ Interactivity
■ Reachability
○ Dependent variable:
■ Response
Applications, 1-22.

Test data
Applications, 1-22.

Test results
Applications, 1-22.

Test results
● Accurate responses: only 54.9%
● But: participants were more accurate than
being wrong or undecided (33% wrong)
● Participants were prone to mark normal
displays as ambient, and vice versa
Applications, 1-22.

Factors for successful ambient
awareness
● Location
● Installation
● Reachability
● Content
Applications, 1-22.

Improving ambient awareness
● Clear symbols
● Instruction video
● Instruction manual, text
● Metaphorical pictures
Applications, 1-22.

Conclusions
"How we should interact with ambient content is
still to be answered"
- Vatavu R. D.
● Perceptions of ambient displays are still sometimes
wrong
● Designers need to think of clever implementation of the
system based on the four factors
● Future work is still needed for better understanding
Applications, 1-22.

How do you recommend venues
using input from a user and
contextual circumstances?

Location-based recommendation system
using bayesian user's preference model
● Goal:
○ Reflecting individual preference at a proper time
● Map-based personalised recommendation
system
○ Recommended restaurants nearby
○ User's preference modeled by Bayesian networks
● Input from user and contextual information
Park, M., Hong, J., & Cho, S. (2007). Location-based recommendation system using bayesian user's
preference model in mobile devices.4611, 1130-1139

Bayesian model
● Set of variables and their conditional
dependencies

System design

User input
● User profile
○ Age, Gender, Blood type, Has car,
Income, Food preference
● User request
○ Class, Mood, Price, Parking Area

Contextual input
● Web, system, GPS and application data will be
preprocessed into context log
● Live information (location, weather, time)
is taken into
account with user
request

Output
● Top 2 recommended
restaurants per type
of meal
○ Based on personal
preferences and
contextual
circumstances
● At least one match
on personal
preference

Conclusions
● Result is a proposition of a BN-based
recommendation system
○ It reflects user's preference using a user profile and
context information
● Usability and additional stability tests are yet
to be done

FOR OR AGAINST #1
The Fly See Buy, personal, efficient and preference
based navigation application for airports is
something I would consider using.
GREEN RED
( free beer afterwards )

FOR OR AGAINST #2
Navigation by ambient displays is a useful addition
to a mobile interface.
GREEN RED

FOR OR AGAINST #3
If one uses a portable device for navigation, can
we speak of a ambient system (definition Weiser).
GREEN RED

Final Discussion Question
Sign Post vs Mobile Device?
The mobile device is used for preference input in both cases.

INTELLIGENT INTERACTIVE SYSTEMS - Ambient intelligence + Mobile technology

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to INTELLIGENT INTERACTIVE SYSTEMS - Ambient intelligence + Mobile technology

Similar to INTELLIGENT INTERACTIVE SYSTEMS - Ambient intelligence + Mobile technology (20)

Recently uploaded

Recently uploaded (20)

INTELLIGENT INTERACTIVE SYSTEMS - Ambient intelligence + Mobile technology