• Save
Extracting event and place semantics from Flickr tags
Upcoming SlideShare
Loading in...5

Extracting event and place semantics from Flickr tags



Presentation at SIGIR 2007.

Presentation at SIGIR 2007.



Total Views
Views on SlideShare
Embed Views



9 Embeds 184

http://yahooresearchberkeley.com 94
http://www.yahooresearchberkeley.com 79
http://wildfire.gigya.com 4
http://feeds.feedburner.com 2 1
http://blog.yahooresearchberkeley.com 1
http://www.netvibes.com 1 1
http://www.slideshare.net 1



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

CC Attribution License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
  • Dog photo by CC license from http://flickr.com/photos/pellegrini/5419830/
    Are you sure you want to
    Your message goes here
  • Seems like Slideshare still don't have some Gotham fonts. Sorry, reader!
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment
  • Mor… star interns One way to extract knowledge from Flickr tags. This is not about image search - it’s not even about photos

Extracting event and place semantics from Flickr tags Extracting event and place semantics from Flickr tags Presentation Transcript

  • Tye Rattenbury, Nathan Good, Mor Naaman* Yahoo! Research Berkeley (Yahoo! Advanced Development Division) Towards Automatic Extraction of Event and Place Semantics from Flickr Tags
  • This Talk is Not About Photos
    • Geo-referenced data
      • Photos
      • Blog posts (GeoRSS)
      • Geo-annotated Web pages
      • KML-based collections
    • Similar issues, similar potential
  • Find the Differences “ Yahoo! Headquarters” “ Dog” “ SIGIR 2007”
  • Data Description
  • Simple Model (photo_id, user_id, time, latitude, longitude) (photo_id, tag)
  • Information?
      • Flickr “geotagged” in San Francisco
  • Tag Maps - San Francisco
  • Tag-based Semantics
    • Derive meaningful data about individual tags
    • Based on the tag’s metadata patterns
    • E.g., ”Yahoo! Headquarters” , “SIGIR2007” , “Dog” .
  • Tag Semantics: Why?
    • Improved image (and web!) search through query semantics
    • Automatic place- and event-gazetteers
    • Association of missing time/place data based on tags
    • Collection visualization
  • Next Few Minutes
    • Extended model
    • Sample data
    • Problem definition
  • Extended Model (photo_id, user_id, time, latitude, longitude) (photo_id, tag) (tag, location) (tag, time)
  • Tag Patterns
  • Tag Patterns
  • Tag Patterns
  • Definitions
    • Event tag: expected to exhibit significant time-based patterns
    • Location tag: expected to exhibit significant place-based patterns
  • Problem Definition
    • Can we extract event/place semantics from unstructured tags given their usage distribution (time and location patterns)?
  • Fundamental Issues
    • “ Regions” of semantic relevance
      • “ church” in a neighborhood is a place
      • “ carnival” in Rio de Janeiro is an event
      • “ California” in San Francsico is not a place
    • Semantic ambiguity
      • “ apple”
      • “ turkey”
      • “ pride”
  • Proposed Solutions
    • Naïve Scan
      • Burst detection
    • Spatial Scan Statistics
      • Borrowed from disease outbreak detection
    • Scale-structure method
      • Examine the “structure” of the tag’s patterns in multiple scales
  • Next Few Minutes
    • Intuition behind the three methods
    • Evaluation
    • Summary
  • Naïve Scan
  • Spatial Scan
  • Issues
    • First two methods just look for a local burst in usage (in either time or space)
  • Scale-structure Identification
    • Select a scale
      • Find the structure of the data for the scale
      • Calculate the entropy of the structure
    • Increase scale, repeat
    • Is the information “well organized”?
  • Scale-structure Identification
  • Scale-structure Identification Entropy = 1.421621 Entropy = 0.766263 Entropy = 0.555915 Entropy = 0.062760 “Barry Bonds” tag
  • San Francisco Experiments
    • ~43k photos
    • ~800 tags
    San Francisco Dataset: 42,000 Photos 800+ popular tags
  • Experiments byobw Ground Truth: BYOBW?
  • Experiments
    • Given a ground truth:
      • Run the three methods, vary thresholds
      • Precision vs. recall
  • The Obligatory Slide Where I Show You that Our Curve is Above the Other Curves
  • Experiments
  • Future Work
    • Multiple regions
    • Different spatially and temporally encoded data (e.g., blogs)
    • Other metadata domains besides time and space
      • Visual features: shape primitives, color
      • Semantic features
  • Questions?
    • Tye Rattenbury, Nathan Good, Mor Naaman
    • Y!RB / Y!A.D.D.
    • Read more, follow: http://www.whyrb.com
    • Slides: http://slideshare.net/mor
    • Contact: mor@yahoo-inc.com
  • Tag Maps - Paris - Les Blogs?
  • Issues
    • Noisy data
    • Photographer biases
      • In locations
      • In Tags
    • Wrong data
    Whatever, color, city, spectrum, santa barbara, california, usa, Lookatme, Herbert Bayer Chromatic Gate