How Flickr Helps us Make Sense of the World

How Flickr Helps us Make Sense of the World: Context and Content in Community-Contributed Media Collections Lyndon Kennedy Mor Naaman* Share Ahern Rahul Nair Tye Rattenbury Yahoo! Research Berkeley Yahoo! Advanced Development Research *me

Community-contributed?
Media
Descriptive text (title, caption, tag)
Discussions and comments
Views and view patterns
Item use and feedback
Reuse and remix
Micro- and explicit recommendations
“ Context Metadata”
…

Research Challenges
Content is still hard…
Unstructured data (no semantics)
Noise
Scale
Computation
Long tail implies no supervised learning
Bias/feedback/Spam

Foremost Challenge:
What’s the user problem?
Navigation/exploration
Recommendation
New application
Other?
Grounded in real needs
What impact on the
community?
“ Social Media Cycle”

In Particular…
No tigers, beaches and sunsets. Please.

Flickr Tigers

Good news! Patterns That Make Sense:
Semantic space
Activity and viewing data
User/personal data
Social network
And, location/time:

Data Description

That Noise….
Noisy data
Photographer biases
Wrong data
6 kms 5 kms

Tag Patterns

Experiments byobw We can derive tag semantics using location and time metadata. Also see [Rattenbury et al, SIGIR 2007]

Can We Create Useful Applications?
Flickr “geotagged” in San Francisco

Intuition
More “activity” in a certain location indicates importance of that location
Tag that are unique to a certain location can represent the location better

Tag Maps - SF

Make a World Explorer http://tagmaps.research.yahoo.com Also see [Ahern et al., JCDL 2007]

Rolling in Content
So far, we leveraged metadata patterns to find
What are the geo-driven features
Where people take photos of these features
Can we utilized content analysis?
Hmmm….

Handling scale
Reduce computation requirements
Filter using metadata
Unsupervised methods
Effective for long tail without training

Problem: Better Visual Summaries Locations and Landmarks Raw Data Visual Summary?

The Problem, in Short and more of this… … without explicitly knowing the difference. Find less of this…

Location can help Enough visual similarity for learning?

Finding Representative Photos

Visual Features
Color : moments over a 5x5 grid
Texture : Gabor over global image
Interest points : SIFT

Ranking images: point-wise links Form links between images via matching SIFT points. Rank by degree of connectivity.

Landmark Graph Structure More connected Less connected

Results: Palace of Fine Arts Tags-only Tags+Location Tags+Location+Visual X X X X X X X

Initial Evaluation
Select 10 landmarks to evaluate
Identify landmarks region(s) of relevance
Apply visual approach to discover representative images
Evaluate using Precision @ 10

Performance Average +45% from visual +30% from location

Evaluation Issues
Degrees of “Representativeness”

Evaluation Issues
Diversity of Results

Conclusions
Noise can be handled (sometimes)
Can generate some structure from the unstructured
Content can help with the right tasks
Bias and Spam?

Thanks
With: Lyndon Kennedy, Shane Ahern, Rahul Nair, Tye Rattenbury
Jeannie Yang, Nathan Good, Simon King
In the papers: MIR06, JCDL07, SIGIR07
Have a Nokia phone? Check out ZoneTag and Zurfer
Read more, follow: http://www.whyrb.com
Slides: http://slideshare.net/mor
Mor Naaman: mor@yahoo-inc.com

APIs for all!
Everything we can do, you can do (better). APIs include:
Cell Tower ID database
Suggested Tags
TagMaps data
TagMaps Widget
ZoneTag RSS feeds, Action Tags
http://developer.yahoo.com/yrb/

Tag Maps - Paris

How Flickr Helps us Make Sense of the World

by mor on Sep 26, 2007

Accessibility

Categories

Tags

Upload Details

Usage Rights

2 Embeds 6

Statistics

How Flickr Helps us Make Sense of the World — Presentation Transcript

http://www.slideshare.net	5
http://chamberexecs.com	1