Proliferation of GPS-enabled mobile devices has brought a plurality of location-aware applications leveraging the location characteristics in the shared content, like photos and check-ins. While these applications provide contextual and relevant information, they also assume geo-tagged contents to be representative of the geo-bounded characteristics of location. In this paper, however, we show that the characteristics geo-tagged contents capture about a location can vary based on the familiarity of user (sharing the content) with the location. Using a large dataset of geo-tagged photos, we learn descriptive spatial photo characteristics and user temporal-location-familiarity to highlight unique characteristics photos capture of location, which vary significantly if taken by locals versus tourists. We then propose a ranking-approach to find most representative photos for a given city. A user-based evaluation shows photos are more diverse and characteristic of location compared to other popular baselines while being representative of how locals and tourists would describe the city.

1. Adaptive City Characteristics:
How Location Familiarity Changes
What is Regionally Descriptive
Vikas Kumar (University of Minnesota)
Saeideh Bakhshi (Facebook Inc.)
Lyndon Kennedy (FutureWei Tech)
David A. Shamma (CWI Amsterdam)
This research was performed when the authors worked at Yahoo Inc!

2. A city is a collection of diverse people with diverse perspectives
and, photos capture these perspectives.

3. A city is a collection of diverse people with diverse perspectives
and, photos capture these perspectives.
How does this affect location-aware models?

4. Photos
Users Location

5. Photos
Users Location
●Visual representation.
●Widely available.
●Influence travel
decisions.

6. Photos
Users Location
Familiarity varies:
●Locals are more familiar than tourists.
●Some places are only known to locals.

7. Photos
Users Location
Different user different needs:
Photos can help illustrate these needs.

8. Problem:
● Assumption of localness is vague [Johnson et al. CHI 2016]
○ Not every geo-tagged content with location is local.
● Limited representation of city (focus only on tourists):
○ Landmarks [Kennedy et al, WWW ‘08]
○ Hero-images [Chen et al., MM ‘15, Jenkins Tourism
Research, ‘99]

9. In this paper:
We analyze:
○ Geo-Tagged Photos, and
○ User’s Familiarity with location
to identify and define:
○ descriptive characteristics of photos, (content)
○ users who are local or tourist.
and, show:
○ that characteristics vary based on user’s familiarity (locals Vs. tourists),
○ a model to find representative photos for a city.

10. DATA: YFCC100M
● Publicly Available geo-tagged photo dataset from Flickr [Thomee
et al. 2016]
● photos that were taken in the US and had at least 10 favorites and
10 views [Bakhshi et al. ICWSM, 2015]
● Overall around 4.5M Geo-Tagged Photos

11. Identify locals vs. tourists
We categorize people based on their spatial-temporal patterns of activity.
We identify a person as local to a location if they have at least 30 days of activity at a given
location.
Analysis of photos posted by locals vs. tourists
We label photos taken by locals and tourists and analyze the descriptive content of the photos
(tags).
Photo tags are driven from computer vision classifications using deep learning methods.
Locals Tourists

12. Descriptive Characteristics of photos taken by locals vs. tourists
● Using photo tags we identify the
most unique description of each
city.
● We calculate conditional entropy:
● certainty of tag T given location L

13. Locals and tourists vary in what they capture
San Francisco Seattle
locals tourists locals tourists
texture urban ocean latte
graffiti architecture sunset urban
people skyline sidewalk architecture
monochrome
ferrybuilding
urban decay skyline
portrait bridge biking outdoor

14. Locals and tourists vary in what they capture
San Francisco Seattle
locals tourists locals tourists
texture urban ocean latte
graffiti architecture sunset urban
people skyline sidewalk architecture
monochrome
ferrybuilding
urban decay skyline
portrait bridge biking outdoor

15. Locals and Tourist capture distinct characteristics
In their shorter stay, tourists tend to capture more of popular locations.
Locals, instead, reflect on daily activities.

16. What does this mean for location
and its photos/representation?
Goal: Identify and rank diverse
representative photos of location

17. Location-Aware Characteristic Score
● A score for a photo is likelihood of photo being most representative of
location.
● Measures how well it captures the unique characteristics (tags) of location.

18. Location-Aware Characteristic Score
● A score for a photo is likelihood of photo being most representative of
location.
● Measures how well it captures the unique characteristics (tags) of location.
a proxy to quality of photo
average conditional
entropy for tags: the
higher uncertainty leads
to lower likelihood

19. Evaluation:
• We use mechanical turk and social media platforms to recruit survey
participants.

20. Evaluation:
• We use mechanical turk and social media platforms to recruit survey
participants.
• We created four groups of photos:
• localR: Candidate photos taken by locals and sorted by
characteristic score.
• touristR: Photos taken by tourists.
• geo-popularR (baseline): Most favorited photos of the city.
• popularR (baseline): Most favorited photos in the dataset,
irrespective of location.

21. Evaluation:
• We use mechanical turk and social media platforms to recruit survey
participants.
• We created four groups of photos:
• localR: Candidate photos taken by locals and sorted by
characteristic score.
• touristR: Photos taken by tourists.
• geo-popularR (baseline): Most favorited photos of the city.
• popularR (baseline): Most favorited photos in the dataset,
irrespective of location.
• We evaluated characteristics of the photos, beauty and aesthetic
values, diversity of the location and perceptions of both locals and
tourists.

22. Photos taken by tourists are more characteristic of the location.
Tourists capture what is the most relevant/known of the city.

23. Photos taken by tourists have higher aesthetic value than those
taken by locals.

24. Photos taken by tourists are more diverse than those taken by
locals.

25. Users who are local to the location, though with similar opinion about tourist
photos, have very different opinion for local-set. Locals find local-set (photos taken
by other locals) to be more characteristic and delightful of location than tourists.
(a) localR (b) touristR
(c) localR (d) touristR
Figure 6: Dierence in opinion of Locals (marked as Resident) Vs Tourists (marked as Non-Resident): (a) Locals nd localR

26. ● We challenged assumptions of localness!
● Localness of content is proportional to familiarity of
user sharing the content.
○ Locals and Tourist capture varying characteristics
● A location-aware characteristic-score helps us to find
more representative and diverse photos of a location.
Summary:

27. • As intuitive as it can be, with only 38% of photos taken by tourists, in
their short stay, are able to capture what is more popular, diverse and
representative of the city.
• More data is not always more efficient
• More familiarity does not always work in favor of representativeness.
Take-aways:

28. Thank You!
Questions?
vikas@cs.umn.edu; Twitter: @vikasnitr
saeideh@gatech.edu; Twitter: @sdhbkh