This document summarizes a master's thesis on developing a personalized mobile museum tour with real-time adaptation. The thesis aimed to improve on an existing offline personalized tour system by enabling real-time user positioning and tour adaptation based on time constraints, artwork preferences, and spatial information within the museum. It investigated using WiFi radio frequency fingerprinting for real-time localization, which achieved accuracy within 1.25 meters. The mobile tour system was designed to adapt the recommended artworks and tour pathing in real-time based on the user's profile and detected location within the museum.

1. Personalized Museum Tour with Real-
Time Adaptation on a Mobile Device
with Multi-Point Touch Interface Graduation Committee:
Dr. L.M. Aroyo
Dr. A.J. Mooij
Master Thesis by Ivo Roes (0527673) Prof. Dr. P.M.E. de Bra

2. Introduction
• Current museum tours
• Information on paper
• Audio guides
• Human tour guides
• “One size fits all” mentality
• Users want the best
experience in the available
time
/ Business Information Systems July 2010 PAGE 1

3. Overview
• Context
• Related Work
• Problem Definition & Research Questions
• Requirements
• Design
• Adaptation Strategies
• Localization
• Demo
• Conclusion & Further Research
• Questions
/ Business Information Systems July 2010 PAGE 2

4. Context – Project
• CHIP project
• CATCH program
/ Business Information Systems July 2010 PAGE 3

5. Context – Existing Implementation
• Personalized Museum Tour on
a Mobile Device (PMTMD) 1
• Offline personalized PDA-tour
• User builds online profile based on
rating artworks
• Synchronization at beginning and
end of tour
• User positioning based on RFID
tags and readers
1 Master
Thesis Personalized Museum Tour on A Mobile Device (PMTMD)
Van Sambeek, R and Schuurmans, . TU/e 2007
/ Business Information Systems July 2010 PAGE 4

6. Related Work
• GUIDE
• Sotto Voce
• AgentSalon
• PEACH
• HIPPIE
• The Kubadji Project
/ Business Information Systems July 2010 PAGE 5

7. Problem Definition
• Existing Implementation (PMTMD)
• Offline Tour
• Hardware Platform
• Static user positioning
/ Business Information Systems July 2010 PAGE 6

8. Research Questions
1. What user, collection and spatial aspects play a
role in the real-time adaptation of a mobile museum
tour?
2. In what manner can the visitor be precisely
positioned in real-time and how can this
information be used in tour adaptation?
/ Business Information Systems July 2010 PAGE 7

9. Requirements
• Tour
• Adaptation
• Localization
• System
• Platform independent implementation
• Non-intrusiveness to the user
• Non-intrusiveness to the environment
/ Business Information Systems July 2010 PAGE 8

10. Design – General Scenario
• User creates online profile
• User visits physical museum
• User selects (personalized) tour
• User configures the tour to his / her preferences
• User starts tour
• During tour user can view artwork description, concepts and related
artworks.
• The user can rate artworks / concepts
• The user can add (similar) artworks
• At any moment the user can view his location on the map
/ Business Information Systems July 2010 PAGE 9

11. Design – Use Cases
• UML Case Diagrams
Rate Artwork / Concept Use Case
/ Business Information Systems July 2010 PAGE 10

12. Design – System Architecture
July 2010 PAGE 11

13. Design – Graphical User Interface
• Obstacles to tackle
• Small screen
• Rough finger touch instead of
precise stylus or mouse
• Requirements: quick
response, intuitive, easy
navigation
• Design Decisions
• Button menu bar
• Small text font
• Separation of functionalities
• AJAX communication
• Adaptive Interaction
/ Business Information Systems July 2010 PAGE 12

14. Adaptation Strategies
• CHIP mobile guide adapts
to individual user on three
levels:
• Based on time constraints
• Based on artwork preference
• Based on spatial information
/ Business Information Systems July 2010 PAGE 13

15. Adaptation Strategies – time constraints (1)
• Usually museum visitors wish to spend a certain
amount of time in the museum
• At tour configuration users can specify #artworks and
#minutes
• Each user has a different average time for viewing an
artwork tv (continously updated)
• CHIP mobile guide guards: #artworks x tv = #minutes
/ Business Information Systems July 2010 PAGE 14

16. Adaptation Strategies – time constraints (2)
• To guard the described constraint, the tour needs to
be adapted:
1. Order AWremain according to belief value in descending order
2. If (#current > #artworks)  new AWremain = first (#artworks - #Visited) of AWremain
3. Else if (#current = #artworks)  skip
4. Else  new AWremain = AWremain + (#artworks - #current) recommendations
5. Order new AWremain according to belief value in descending order
/ Business Information Systems July 2010 PAGE 15

17. Adaptation Strategies – artwork preference (1)
• Rating of artworks and concepts on a
5-star scale [-1.0, -0.5, 0.0, 0.5, 1.0]
• A belief value is calculated for all
artworks and concepts
• Belief value of concepts is built up from
child concepts
• Belief value for artworks is built up
from concepts
/ Business Information Systems July 2010 PAGE 16

18. Adaptation Strategies – artwork preference (2)
• Rating a concept or artwork triggers:
1. Calculate Beliefc for all concepts c
2. Calculate Beliefa for all artworks a
3. Order { AWall AWvisited } based on belief value in descending order
4. New AWremain = first #remain elements of sorted { AWall AWvisited }
/ Business Information Systems July 2010 PAGE 17

19. Adaptation Strategies – spatial information
• When adapting the tour, it is desirable that users are not
sent back to previous rooms
• This can be achieved by extending the algorithm:
1. Order AWremain according to belief value in descending order
2. If (#current > #artworks)  new AWremain = first (#artworks - #Visited) of AWremain
3. Else if (#current = #artworks)  skip
4. Else  new AWremain = AWremain + (#artworks - #current) recommendations
(artworks from previous rooms excluded)
5. Order new AWremain according to belief value in descending order
6. Order new AWremain according to location in museum (closest first)
/ Business Information Systems July 2010 PAGE 18

20. Adaptation Strategies – Validation (1)
• Validation of the Artwork Recommender
• Study 2 performed by CHIP team into relevant semantic
relations
• Test group rated artworks / concepts
• After presentation of recommendations, users could
rate the semantic relation used as a basis for the
recommendation
• Best performing semantic relations were based on
creator and style of an artwork
• Semantic relations based on geographic location
performed the worst
2 Semantic Relations for Content-based Recommendations, Wang, Y et al.,
In Proc. International Conference on Knowledge Capture (K-cap), 2009, Vol. September 2009
/ Business Information Systems July 2010 PAGE 19

21. Adaptation Strategies – Validation (2)
• Comparative Analysis with SpaceCHIP 3
• SpaceCHIP extends the mobile guide by implementing a
routing mechanism based on the SWI-Prolog Space package
• Main goals: minimize walking distance and take themed rooms
into account
• Introduction of connectivity graph with nodes consisting of
artworks, rooms, doors, hallways and stairs
• Set transition weights of edges to experienced distance
instead of actual distance
• Using a nearest neighbor search a suboptimal shortest tour
through the museum is calculated.
• Calculation time is exponentially related to number of artworks
3 Finding
Your Way through the Rijksmuseum with an Adaptive Mobile Museum Guide
W.R. van Hage et al., Heraklion, Greece : 7th Extended Semantic Web Conference, 2010
/ Business Information Systems July 2010 PAGE 20

22. Adaptation Strategies – Validation (3)
• Both the CHIP mobile guide and SpaceCHIP aim at
minimizing user walking distance and take themed
rooms into account
• The approach for reordering the tour differs slightly
• CHIP mobile guide orders remaining tour based on the
room where artworks are located
• SpaceCHIP uses experienced distance instead of actual
distance and therefore also takes into account
unfavorable transitioning to a different floor
• Further research is needed to determine which
method results in better remaining tours
/ Business Information Systems July 2010 PAGE 21

23. Localization
• In the exisiting implementation, localization was
achieved using RFID scanners and tags
• No real-time location information was available
• In the CHIP mobile guide we set out to see what
possibilities exist for real-time localization
• Requirements that must hold: platform-independent
implementation and non-intrusiveness.
/ Business Information Systems July 2010 PAGE 22

24. Localization – Software
• From a literature study, a Radio Frequency
Fingerprinting technique was chosen as a basis for
localization.
• Radio Frequency Fingerprinting
• Based on Received Signal Strength Indicator (RSSI)
• Calibration phase, measure RSSI values to all access
points for predefined set of points in the space. Use
average values to generate frequency radio map
• Online phase, compare live measurement of RSSI
values to all access points and compare to map
/ Business Information Systems July 2010 PAGE 23

25. Localization – Hardware technique
• RFID
• Bluetooth
• Infrared
• Ultra Wide Band (UWB)
• WiFi
/ Business Information Systems July 2010 PAGE 24

26. Localization – Hardware architecture (1)
/ Business Information Systems July 2010 PAGE 25

27. Localization – Hardware architecture (2)
July 2010 PAGE 26

28. Localization – Test setup (1)
DD-WRT firmware v2.4 4
3x Linksys WRT54GL router
Wi-viz 2.0 5
Wireless Network Visualization
4 http://www.dd-wrt.com
5 http://devices.natetrue.com/wiviz/
/ Business Information Systems July 2010 PAGE 27

29. Localization – Test setup (2)
/ Business Information Systems July 2010 PAGE 28

30. Localization – Test setup (3)
• Test results
/ Business Information Systems July 2010 PAGE 29

31. Demo
/ Business Information Systems July 2010 PAGE 30

32. Conclusion
• Focus on what aspects play a role in real-time
adaptation of a museum tour and real-time
localization of a visitor in the museum
• Adaptation in the CHIP mobile guide takes place at
three levels:
• Based on time constraints
• Based on artwork preference
• Based on spatial information
• Localization was implemented using a WiFi Radio
Frequency Fingerprinting technique, resulting in an
accuracy of up to 1,25 m
/ Business Information Systems July 2010 PAGE 31

33. Further Research
• Graphical User Interface evaluation
• Scalability of localization technique
• Social aspects
/ Business Information Systems July 2010 PAGE 32

34. Questions?
/ Business Information Systems July 2010 PAGE 33