Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

What's in a place? Adventures with Location-Aware Media


Published on

Can we automatically create an “attraction map” of the world from Flickr geotagged images and their associated tags? We performed an analysis of Flickr data and developed a visualization technique called Tag Maps to do exactly that. Using the analysis and the Tag Maps visualization, we created an exploration tool called World Explorer that allows one to, well, explore the world like never before.

The idea behind the data analysis is simple: by taking a photo, photographers essentially express their interest in a particular place, and implicitly “vote” in favor of that location. This gives us a set of highly representative tags associated with each map location. The World Explorer visualization is facilitated by placing these representative tags on a map (“a Tag Map”). We augment the Tag Map with photos that represent each tag at its specific location. Together, World Explorer effectively provides a sense of the important concepts and attractions embodied in each map area and zoom level, and allows users – tourists planning a trip, virtual world-discoverers or just some bored individuals – to explore the world via photos.

I’ll also give a brief demo and overview of Zurfer, a novel mobile phone context-aware software prototype that enables access to images on the go. It utilizes the channel metaphor to give users contextual access to media of interest according to key dimensions: spatial, social, and topical. Zurfer attempts to be playful and simple to use, yet provide powerful and comprehensive media access. A temporally-driven sorting scheme for media items allows quick and easy access to items of interest in any dimension. For novice users, and more complicated tasks, we extend the application incorporating keyword search to deliver the long tail of media and images.

Published in: Technology, Education
  • Be the first to comment

What's in a place? Adventures with Location-Aware Media

  1. 1. What’s in a place? Adventures with Location-Aware Media Rahul Nair Yahoo! Research Berkeley
  2. 2. Flickr “geotagged” <ul><ul><li>20+ million images </li></ul></ul>Can we do better?
  3. 3. Talk Outline <ul><li>Extracting information from geo-tagged photos </li></ul><ul><li>World Explorer: Visualization </li></ul><ul><li>ZoneTag: Creation </li></ul><ul><li>Zurfer: Consumption </li></ul>
  4. 4. Attraction Map of Paris <ul><ul><li>Stanley Milgram, 1976. </li></ul></ul><ul><ul><li>Psychological Maps of Paris </li></ul></ul>
  5. 5. Attraction Map of Paris <ul><ul><li>Y!RB, 2007. </li></ul></ul>
  6. 6. Location-driven Modeling <ul><li>Derive meaningful data about map regions </li></ul><ul><li>E.g., representative tags, photos </li></ul>
  7. 7. Data Description
  8. 8. Issues <ul><li>Sparse data set </li></ul><ul><li>Photographer bias </li></ul><ul><ul><li>In location </li></ul></ul><ul><ul><li>In tags </li></ul></ul><ul><li>Incorrect data </li></ul>
  9. 9. Heuristics <ul><li>Number of photographs denotes the “importance” of a location </li></ul><ul><li>Users will use a common subset of tags to describe objects/locations </li></ul><ul><li>Concentrated tag usage indicates descriptiveness </li></ul>
  10. 10. Algorithm <ul><li>Clustering: k-Means, get set of k clusters </li></ul><ul><li>“ Document” C is bag of all tags in cluster </li></ul><ul><li>For each tag in C calculate: </li></ul><ul><ul><li>TF = |P(C,t)| </li></ul></ul><ul><ul><li>IDF = |P(R)| / |P(R, t)| </li></ul></ul><ul><ul><li>UF = |U(C,t)|/|U(C)| </li></ul></ul>
  11. 11. Scoring <ul><li>Score (t) = TF * IDF * UF </li></ul><ul><li>Threshold values </li></ul><ul><ul><li>30+ photographs </li></ul></ul><ul><ul><li>Minimum 3 users </li></ul></ul><ul><ul><li>Score > 1 </li></ul></ul><ul><li>Final dataset: (tag, score, latitude, longitude) </li></ul>
  12. 12. Talk Outline <ul><li>Mining information from geo-tagged photos </li></ul><ul><li>World Explorer: Visualization </li></ul><ul><li>ZoneTag: Creation </li></ul><ul><li>Zurfer: Consumption </li></ul>
  13. 13. DEMO
  14. 14. Precomputation <ul><li>Divide the world into equal sized non-overlapping tiles </li></ul><ul><li>Compute and store the tags for each tile </li></ul><ul><li>Repeat for different zoom </li></ul><ul><li>levels </li></ul>
  15. 15. Retrieval <ul><li>Find the tile level closest in size to the request area </li></ul><ul><li>Select the tiles that fully cover the request area </li></ul><ul><li>Return the tags that fall within the request area </li></ul>
  16. 16. User Study <ul><li>10 subjects </li></ul><ul><li>6 female, 4 male </li></ul><ul><li>Ages 20-60 </li></ul><ul><li>Varying technical knowledge </li></ul><ul><li>No geotagged photos of their own </li></ul>
  17. 17. Experiment tasks <ul><li>Vacation recap </li></ul><ul><li>San Francisco tour </li></ul><ul><li>Explore a new city </li></ul>
  18. 18. Recall <ul><li>Reminded the subject about locations </li></ul><ul><li>“It brings out memories” </li></ul><ul><li>“Oh my God! This place has the best restaurants” </li></ul><ul><li>“We wanted to see the Polynesian Cultural Center&quot; </li></ul>
  19. 19. Discovery <ul><li>Participants discovered previously unknown locations and events </li></ul><ul><ul><li>“I’ve never heard of this festival” </li></ul></ul><ul><ul><li>“There is car racing which I'd probably go see” </li></ul></ul>
  20. 20. Needle & Haystack <ul><li>Excellent visualization of the Haystack </li></ul><ul><li>Hard to find specific information </li></ul><ul><ul><li>“Where was Culver City again?” </li></ul></ul><ul><li>No way to search </li></ul><ul><ul><li>“I guess what I’m looking for are bull fighting pictures” </li></ul></ul>
  21. 21. Other Responses <ul><li>Gets the “vibe” of a place </li></ul><ul><li>Share with other people </li></ul><ul><li>Tags did not always match the mental model of a location </li></ul><ul><li>Wanted more tags </li></ul><ul><li>Want more info about tags </li></ul>
  22. 22. Conclusions <ul><li>Extract meaningful aggregate information from georeferenced data </li></ul><ul><li>Allows users to explore locations in a new way </li></ul><ul><li>Users like using the overview but also want the ability to search </li></ul>
  23. 23. Future work <ul><li>Adding search capability </li></ul><ul><li>Show photos in places with no tags </li></ul><ul><li>Differentiate locations and events </li></ul><ul><li>Apply to other types of georeferenced data </li></ul>
  24. 24. Talk Outline <ul><li>Mining information from geo-tagged photos </li></ul><ul><li>World Explorer: Visualization </li></ul><ul><li>ZoneTag: Creation </li></ul><ul><li>Zurfer: Consumption </li></ul>
  25. 25. Current Mobile Experience <ul><li>Difficult to share (or even save!) </li></ul><ul><li>Hard to find </li></ul><ul><ul><li>No context </li></ul></ul><ul><ul><li>No semantic information </li></ul></ul>Current mobile experience?
  26. 26. Current Mobile Experience <ul><li>Difficult to share (or even save!) </li></ul><ul><li>Hard to find </li></ul><ul><ul><li>No context </li></ul></ul><ul><ul><li>No semantic information </li></ul></ul>
  27. 27. ZoneTag Experience <ul><li>2-click upload (same key!) </li></ul><ul><li>Photo uploaded with location and time metadata </li></ul>
  28. 28. Where does location come from? <ul><li>Bluetooth GPS (when available) </li></ul><ul><li>User-contributed cell tower mapping </li></ul>
  29. 29. ZoneTag Experience <ul><li>Tagging made easy </li></ul><ul><ul><li>Tag/annotate your photos from the phone </li></ul></ul>
  30. 30. Where do tags come from? <ul><li>Tags I used in this context (`home’) </li></ul><ul><li>Tags my friends used in this context </li></ul><ul><li>Tags other people used in this context (‘Ricoh’, ‘California Research Center’) </li></ul><ul><ul><li>E.g., TagMaps data </li></ul></ul>
  31. 31. Where do tags come from? <ul><li>Stuff around you: </li></ul><ul><ul><li>Yahoo! Local </li></ul></ul><ul><ul><li> </li></ul></ul><ul><li>Stuff from you (any RSS 2.0 feed): </li></ul><ul><ul><li>Calendar </li></ul></ul><ul><ul><li>Favorite hangouts (Wayfaring, Plazes, Socialight) </li></ul></ul>
  32. 32. Example of Tagging
  33. 33. I’m Too Lazy (you’re not alone) <ul><li>Tagging is the means, not the goal </li></ul><ul><li>Benefits even if you never tagged a single image </li></ul>(Bradley Horowitz, (?) (?) (?)
  34. 34. Talk Outline <ul><li>Mining information from geo-tagged photos </li></ul><ul><li>World Explorer: Visualization </li></ul><ul><li>ZoneTag: Creation </li></ul><ul><li>Zurfer: Consumption </li></ul>
  35. 35. Flickr “geotagged” <ul><ul><li>20+ million images </li></ul></ul>Can we do mobile?
  36. 36. Strengths of Mobile <ul><li>Personal </li></ul><ul><li>Easy to access </li></ul><ul><li>Networked </li></ul><ul><li>Context aware </li></ul><ul><li>The ultimate photo wallet </li></ul>
  37. 37. Design Goals <ul><li>Engagement and Discovery </li></ul><ul><ul><li>Spatial </li></ul></ul><ul><ul><li>Social </li></ul></ul><ul><li>Allow customization </li></ul><ul><li>Complete access to your photos </li></ul><ul><li>Search and Filtering capability </li></ul>
  38. 38. Channel Metaphor <ul><li>Each row is a single channel </li></ul><ul><li>Navigate using 4 way </li></ul><ul><ul><li>Left & right to browse channel (infinite scroll) </li></ul></ul><ul><ul><li>Up & down to change channels </li></ul></ul>
  39. 39. Detail view <ul><li>Enlarge photo </li></ul><ul><ul><li>Scroll through channel </li></ul></ul><ul><li>Photo details </li></ul><ul><li>Add comments </li></ul><ul><li>Add to favourites </li></ul>
  40. 40. Nearby photos channel <ul><li>Photos from the users current location </li></ul><ul><li>Local Highlights </li></ul><ul><li>My Nearby </li></ul>
  41. 41. Social Channels <ul><li>Contacts photos </li></ul><ul><ul><li>Expanded view </li></ul></ul><ul><li>Recent Activity </li></ul>
  42. 42. My Stuff Channel <ul><li>My Photos </li></ul><ul><ul><li>By Location </li></ul></ul><ul><ul><li>By Tag </li></ul></ul><ul><li>My Photo Wallet </li></ul><ul><li>My Favourites </li></ul>
  43. 43. Custom Channels <ul><li>Users can create channels to match their interests </li></ul><ul><ul><li>Tags </li></ul></ul><ul><ul><li>Groups </li></ul></ul><ul><ul><li>Location aware (dogs near me) </li></ul></ul>
  44. 44. Search and Filter
  45. 45. Try it out <ul><li> </li></ul>
  46. 46. Conclusions <ul><li>It is possible to extract information from georeferenced media </li></ul><ul><li>Users like browsing the extracted data </li></ul><ul><li>It can help users tag new media </li></ul><ul><li>We hope it helps them browse on a mobile device </li></ul>
  47. 47. Questions? Rahul Nair [email_address]