Shoko Wakamiya, Hiroshi Kawasaki, Yukiko Kawai, Adam Jatowt, Eiji Aramaki and Toyokazu Akiyama: Lets Not Stare at Smartphones while Walking: Memorable Route Recommendation by Detecting Effective Landmarks, In Proc. of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp 2016), pp. 1136-1146, 2016. (Heidelberg, Germany, September 12-16, 2016) Navigation in unfamiliar cities often requires frequent map checking, which is troublesome for wayfinders. We propose a novel approach for improving real-world navigation by generating short, memorable and intuitive routes. To do so we detect useful landmarks for effective route navigation. This is done by exploiting not only geographic data but also crowd footprints in Social Network Services (SNS) and Location Based Social Networks (LBSN). Specifically, we detect point, area, and line landmarks by using three indicators to measure landmark's utility: visit popularity, direct visibility, and indirect visibility. We then construct an effective route graph based on the extracted landmarks, which facilitates optimal path search. In the experiments, we show that landmark-based routes out-perform the ones created by baseline from the perspectives of the lap time and the number of references necessary to check self-positions for adjusting route directions. http://dl.acm.org/citation.cfm?id=2971758&dl=ACM&coll=DL&CFID=714384679&CFTOKEN=27846987

1. Lets Not Stare at Smartphones while Walking:
Memorable Route Recommendation by
Detecting Effective Landmarks
Shoko Wakamiya1, Hiroshi Kawasaki2, Yukiko Kawai3,
Adam Jatowt4, Eiji Aramaki1, Toyokazu Akiyama3
1 Nara Institute of Science and Technology 2 Kagoshima University
3 Kyoto Sangyo University 4 Kyoto University
1
Sept. 16, 2016

2. If Eyeballing Maps on Smartphones
while Walking…
Conventional navigation systems
• may be useful
• lead to unsafe situations
• cannot be used in case when
– GPS is not available or
– users aren’t able to use any electronic devices
2
Minimum info. is preferred
A small number of navigation instructions is preferred,
as being easy to be memorized and requiring low number of map references
Pro.
Con.

3. Using Two Types of Landmarks for
Route Navigation
It is more natural and efficient to memorize
landmarks than detailed route instructions
– Local landmarks
To identify users’ locations
(e.g., gas station, post office)
– Global landmarks
To indicate approximate directions
( e.g., high-rise buildings, towers)
3
Christ the Redeemer
(global landmark)
Postbank
(local landmark)
Difficult to use these two types of landmarks together
due to their different characteristics such as visibility and position identification

4. Local, Global, and Line landmarks
e.g., “Walk toward the building to a main street. Then go straight
along the street and go toward the tower at the corner of the cafe.”
Using Three Types of Landmarks for
Route Navigation
4
Destination
building
tower
I head toward
the building freely
until a main street
Cafe

5. Goal & Approach: LandmarkNavi
To construct simple and intuitive navigation routes
– To measure landmark’s utility based on two aspects:
• Visibility using 3D geographic data
• Social recognition using SNS data
– To detect point, area, and line landmarks
5
http://www.ibe.kagoshima-u.ac.jp/~lmnavi/index_en.html
Building (visible
from far away)
Popular
Building Crowded street
River
Popular cafe
Convenience store
Point (Local) Area (Global) Line

6. Outline
• Background
• Goal and Approach
• Proposed method
– Landmark extraction
– Route graph construction
• Experiments and Evaluation
• Conclusion and Future work
6

7. Measuring Landmark’s Utility
Road network
p = <pID, name,
coordinates, catID,
checkins, users>
Places from Foursquare
t = <tweetID, userID,
content, timestamp,
coordinates>
Tweets from Twitter
Geographic data SNS data 7
Visit popularityDirect visibility Indirect visibility
Buildings
Roads & intersections
Ground

8. Visit Popularity of Places
• To extract popular places as candidates of
point and area landmarks
• Measured based on the number of users
checked in each place
8
Geographic distribution
of places in San Francisco
p = <pID, name, coordinates,
catID, checkins, users>
Schloss Heidelberg is great
I’m at Schloss Heidelberg
I’m at Schloss Heidelberg
Very nice view from
Schloss Heidelberg

9. Indirect Visibility of Places
• To find places that may not be directly visible
but are recognized from far away
• Measured based on geographic distribution of
tweets mentioning a place
9
Schloss Heidelberg
Heidelberg Hauptbahnhof
I’m gonna go to
Schloss Heidelberg
I got a ticket for
Schloss Heidelberg
signboard
Maybe we can see
Schloss Heidelberg
soon
Schloss
Heidelberg

10. Direct Visibility of Places
• To find high-rise buildings
that users can see from
far away
• Measured by analyzing
3D geographic data
Computing the number of color
pixels assigned to each building in
the rendered image
10
Rendered image
Extracting the top-N highest
buildings in each area

11. Constructing Popularity/Visibility Map
11
To check what can be recognized/seen from
intersections
Popularity map based on indirect visibility Visibility map based on direct visibility

12. To assign a landmark type based on the values
of the computed indicators
Classifying Places into
Point and Area Landmarks
12
Visit popularity Indirect visibility Direct visibility Landmark Type
1 High Low Low Point
2 High Low High Area (Geo)
3 High High Low Area (SNS+Geo)
4 High High High Area (SNS+Geo)
5 Low
/Not measured
Not measured High Area (Geo)
6 Low
/Not measured
Not measured Low -

13. Visit Popularity of Streets
To detect crowded/popular/famous streets as
line landmarks
– Using geo-tagged tweets
– Finding sequential crowded intersections
13
Mapping to
nearest
intersections
Geographic distribution of tweets Intersections weighted by tweets

14. Route Graph Construction
In order to extract switching locations of landmarks,
we generate virtual edges and nodes
Extracted landmarks in a bounding box Route graph
: Point landmark : Line landmark : Area landmark with its visible area
14
Reorientation points
Direction indicators
Virtual path

15. Route Search
• To find an optimal route
– The number of landmarks (the smaller )
– The visible ratio of area landmarks (the higher )
– The length (the shorter )
• Methods
– Genetic Algorithm (GA)
– Dijkstra Algorithm
15

16. Implementation
• Constructed route graphs
of two cities
• Datasets
16
San Francisco, US
(pop: 840K)
Kagoshima, JP
(pop: 610K)
SNS
data
Location info. 25,256 383
Geo-tagged tweets 0.6M 98K
Geo
data
Streets 1,233 61,075
Intersections 9,649 30,703
Buildings 85,116 66,189
San Francisco Kagoshima
High direct visibility
of Sakurajima made
lots of virtual edges

17. Demo: LandmarkNavi
17
http://www.ibe.kagoshima-u.ac.jp/~lmnavi/index_en.html

18. Evaluation
• To aim at evaluating landmark-based routes (LR)
• Participants walked routes that they remembered
in the virtual space (Street View) or real space
• Comparative methods
– VR (LR without indirect visibility)
– GR (Google Directions (walking mode))
18
Visit popularity
Indirect visibility
(based on SNS)
Direct visibility
(based on geo)
LR ✔ ✔ ✔
VR ✔ - ✔
GR - - -
Proposed

19. Evaluation Items
i) time (min.)
Time needed to reach the destination
ii) route ref.
Number of times a user checked the route directions
on printed materials
iii) self-position ref.
Number of times a user checked
the self-positions
19
Self-position ref. in SV

20. Results of Kagoshima Evaluation in
Virtual Space
& Real Space
20
• vs. GR: Significant differences in ii) in both spaces
• vs. VR: i) is shorter and ii) is smaller (No significant
differences)
(a) Virtual Space (b) Real Space
• #routes: 18 (6 routes×3 methods)
• Distances of routes: 1.5 ~ 2.6 km
• #navi points: LR (5.0) < VR (5.3) < GR (11.2)
• #participants: 30 students

21. Results of San Francisco Evaluation in
Virtual Space
21
• vs. GR: There are significant differences in ii) and iii)
• vs. VR: i) is shorter and ii) is smaller (No significant
differences)
• #routes: 9 (3 routes×3 methods)
• Distances of routes: 0.8 ~ 2.0 km
• #navi points: LR (3.0) < VR (4.0) < GR (6.3)
• #participants: 36 students

22. Qualitative Evaluation
Participants answered 4 questions
q1) Did you think that the route directions were easy to remember
when you check them in the printed map at first?
q2) Did you think that the directions you memorized were reliable?
q3) Did you think that it is easy to follow the memorized route
directions?
q4) Did you think that the route directions were useful as route
navigation?
22
q1
(memorability)
q2
(reliability)
q3
(easiness)
q4
(usefulness)
LR 4.5 4.1 3.9 3.9
VR 4.2 3.8 3.6 3.8
GR 3.1 3.2 2.6 3.2

23. Discussions
• Exploiting SNS data
– We could detect useful landmarks even if
wayfinders could not directly see them
• Generated routes
– Consisting of a small number of landmarks (3~5)
– Most adults’ short-term memory is 7±2
• Virtual space evaluation with SV
– Although SV has limitations, we observed similar
patterns of results in both the spaces
23

24. Conclusions
• An effective route search that recommends
easy-to-remember and short routes
– Extracted three types of landmarks
– Demonstrated effective combination of real space
data with the one harvested and computed from SNS
• Future work
– To investigate patterns of visit popularity and visibility
depending on several conditions
Time, weather, etc.
24