Photos, Mobile, Location and the Social Media Cycle


Published on

A talk at NYU / ITP

Published in: Technology, Travel
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Good timing - took 50mins with questions in the middle, and a bit hurried at the end.
  • Photos, Mobile, Location and the Social Media Cycle

    1. 1. Photos, Mobile, Location and the Social Media Cycle Mor Naaman Yahoo! Research Berkeley
    2. 2. Attraction Map of Paris <ul><ul><li>Stanley Milgram, 1976. </li></ul></ul><ul><ul><li>Psychological Maps of Paris </li></ul></ul>
    3. 3. Attraction Map of Paris <ul><ul><li>Y!RB, 2007. </li></ul></ul>
    4. 4. Social Media Cycle How? What? Why?
    5. 5. Talk Outline <ul><li>Wisdom of the tags </li></ul><ul><ul><li>mining information from geo-tagged photos </li></ul></ul><ul><li>Applications </li></ul><ul><ul><li>World Explorer </li></ul></ul><ul><ul><li>ZoneTag </li></ul></ul><ul><li>Why we tag </li></ul><ul><ul><li>A user study of tagging motivations </li></ul></ul>
    6. 6. Information Overload? <ul><ul><li>Flickr “geotagged” </li></ul></ul>
    7. 7. What can we derive? <ul><li>Location-driven model </li></ul><ul><li>Tag-driven model </li></ul>Dataset: (photo_id, user_id, time, latitude, longitude) (photo_id, tag)
    8. 8. Issues to Tackle <ul><li>Noisy data </li></ul><ul><li>Photographer biases </li></ul><ul><ul><li>In locations </li></ul></ul><ul><ul><li>In Tags </li></ul></ul><ul><li>Wrong data </li></ul>Whatever, color, city, spectrum, santa barbara, california, usa, Lookatme, Herbert Bayer Chromatic Gate
    9. 9. Location-driven Modeling <ul><li>Derive meaningful data about map regions </li></ul><ul><li>E.g., representative tags, photos </li></ul>
    10. 10. Intuition <ul><ul><li>More “activity” in a certain location indicates importance of that location </li></ul></ul><ul><ul><li>Tag that are unique to a certain location can represent the location better </li></ul></ul>
    11. 11. Translation into simple algorithm <ul><li>Clustering of photos </li></ul><ul><li>Scoring of tags </li></ul><ul><ul><li>TF / IDF / UF </li></ul></ul>
    12. 12. Tag Maps - Paris
    13. 13. Tag Maps - SF
    14. 14. Tag Maps - Know this place?
    15. 15. Summary of San Francisco Golden Gate Bridge TransAmerica AT&T Baseball Park Golden Gate Twin Peaks Golden Gate Bay Bridge Ocean Beach Chinatown
    16. 16. Tag-based Modeling <ul><li>Derive meaningful data about individual tags </li></ul><ul><li>Based on the tag’s metadata patterns </li></ul><ul><li>E.g., Yahoo! Research Berkeley , CHI2007 . </li></ul>
    17. 17. Tag Maps - Paris - Les Blogs?
    18. 18. Tag Semantics <ul><li>Improved image search through query semantics </li></ul><ul><li>Automatic place- and event-gazetteers </li></ul><ul><li>Association of missing time/place data based on tags </li></ul><ul><li>… </li></ul>
    19. 19. Definitions <ul><li>Event tag: expected to exhibit significant time-based patterns </li></ul><ul><li>Location tag: expected to exhibit significant place-based patterns </li></ul>
    20. 20. Tag patterns
    21. 21. Tag patterns
    22. 22. Intuition for Solution <ul><li>Burstiness does not work </li></ul><ul><ul><li>Data too sparse </li></ul></ul><ul><ul><li>Goodbye, traditional techniques </li></ul></ul><ul><li>Scale-structure method </li></ul><ul><ul><li>Examine the “structure” of the tag’s patterns in multiple scales </li></ul></ul>
    23. 23. Scale-structure Identification Entropy = 1.421621 Entropy = 0.766263 Entropy = 0.555915 Entropy = 0.062760
    24. 24. Scale-structure Identification
    25. 25. Outline How? What? Why?
    26. 26. Outline <ul><li>Wisdom of the tags </li></ul><ul><ul><li>mining information from geo-tagged photos </li></ul></ul><ul><li>Applications </li></ul><ul><ul><li>World Explorer </li></ul></ul><ul><ul><li>ZoneTag </li></ul></ul><ul><li>Why we tag </li></ul><ul><ul><li>a user study of tagging </li></ul></ul>
    27. 27. Another Look at the Tag Map
    28. 28. Make it into World Explorer
    29. 29. ZoneTag(?) <ul><li>How would we: </li></ul><ul><li>Create/store? </li></ul><ul><li>Find? </li></ul><ul><li>Share? </li></ul><ul><li>Discover? </li></ul>“ Everything in the world exists to end up in a photograph” -- Susan Sontag
    30. 30. Why Cameraphones? <ul><li>Numbers, numbers… </li></ul><ul><li>Programmable </li></ul><ul><li>Context-aware </li></ul><ul><li>Network-connected </li></ul><ul><li>Quality… it’s getting there </li></ul><ul><ul><li>500,000,000 </li></ul></ul>(Source: Future Image Inc.)
    31. 31. Current Mobile Experience <ul><li>Difficult to share (or even save!) </li></ul><ul><li>Hard to find </li></ul><ul><ul><li>No context </li></ul></ul><ul><ul><li>No semantic information </li></ul></ul>Current mobile experience?
    32. 32. ZoneTag Experience <ul><li>2-click upload (same key!) </li></ul><ul><li>Photo uploaded with location and time metadata </li></ul>
    33. 33. ZoneTag Experience <ul><li>Tagging made easy </li></ul><ul><ul><li>Tag/annotate your photos from the phone </li></ul></ul>
    34. 34. Where do locations come from? <ul><li>Bluetooth GPS (when available) </li></ul><ul><li>User-contributed cell tower mapping </li></ul>
    35. 35. Where do tags come from? <ul><li>Tags I used in this context (e.g., `home’) </li></ul><ul><li>Tags my friends used in this context (e.g., `Yahoo! Sunnyvale’) </li></ul><ul><li>Tags other people used in this context (`Golden Gate’, `ITP’, ‘Mor Naaman’, `This talk sucks’) </li></ul>
    36. 36. Where do tags come from? <ul><li>Stuff around you: </li></ul><ul><ul><li>Yahoo! Local </li></ul></ul><ul><ul><li> (`Mor Naaman @ ITP’, `NYU’): USE IT! </li></ul></ul><ul><li>Stuff from you (any RSS 2.0 feed): </li></ul><ul><ul><li>Calendar ( G,,… ) </li></ul></ul><ul><ul><li>Favorite hangouts (Wayfaring, Plazes, Socialight) </li></ul></ul>
    37. 37. Suggested Tags “ When I went to upload, there were already all these exotic tags like &quot;Bill Graham Civic Auditorium&quot; and &quot;Bob dylan live&quot; sure was convenient to just select and go…”
    38. 38. Where do tags go? <ul><li>Back to the original RSS items </li></ul><ul><ul><li> </li></ul></ul><ul><li>Action Tags </li></ul><ul><ul><li>Trigger a call to a web service </li></ul></ul><ul><ul><ul><li>With parameters </li></ul></ul></ul><ul><ul><li>Command line - from your phone </li></ul></ul>Rotate:right Email:dad Group:zonetag Scanr:document
    39. 39. I’m Too Lazy (you’re not alone) <ul><li>Tagging is the means, not the goal </li></ul><ul><li>Benefits even if you never tagged a single image </li></ul>(Bradley Horowitz, (?) (?) (?)
    40. 40. I’m Too Lazy (you’re not alone) <ul><li>Tagging is the means, not the goal </li></ul><ul><li>What is the goal? </li></ul>
    41. 41. Outline How? What? Why?
    42. 42. Outline <ul><li>Wisdom of the tags </li></ul><ul><ul><li>mining information from geo-tagged photos </li></ul></ul><ul><li>Applications </li></ul><ul><ul><li>World Explorer </li></ul></ul><ul><ul><li>ZoneTag </li></ul></ul><ul><li>Why we tag </li></ul><ul><ul><li>a user study of tagging </li></ul></ul>
    43. 43. Inspiration <ul><li>Kinberg, Spasojevic, Fleck, Sellen </li></ul><ul><ul><li>Social uses of camera-phones </li></ul></ul><ul><li>Nancy van House </li></ul><ul><ul><li>Research methodology </li></ul></ul>
    44. 44. A few motivating facts <ul><li>ZoneTag used by 500+ users; 50,000+ photos </li></ul><ul><li>61% of users - at least one tag per photo on average </li></ul><ul><ul><li>Some more than five </li></ul></ul><ul><li>Some users only tag on phone </li></ul><ul><ul><li>some never tag on phone </li></ul></ul>
    45. 45. A few motivating facts Average Tags, Public ZoneTag Photo: 2.2 Average Tags, Public Shozu Photo: 0.97 Average Tags, Private ZoneTag Photo: 1.85
    46. 46. Tag Affordances on Flickr <ul><li>Displayed next to photo </li></ul><ul><li>Can be used to search: </li></ul><ul><ul><li>Your own photos </li></ul></ul><ul><ul><li>Others’ photos </li></ul></ul><ul><ul><li>Public photos </li></ul></ul>
    47. 47. Tags on ZoneTag <ul><li>By default, tags are kept from one photo to next </li></ul><ul><li>Tags used in a location will be suggested in that location </li></ul><ul><ul><li>To you </li></ul></ul><ul><ul><li>To others </li></ul></ul><ul><li>Immediate (i.e., during event) </li></ul><ul><li>Place name tags added automatically </li></ul>
    48. 48. Why Tag? Bay Bridge, Fog, [San Francisco, 94105]
    49. 49. Why Tag? RedSox, Fenway, GreenMonster, [Boston, 02215], …
    50. 50. Why Tag? Horsetails, handicapped, sign, [Stanford, 94305], …
    51. 51. User Study <ul><li>13 ZoneTag users (23-45, 9m, 4f) </li></ul><ul><li>All “taggers” (no use to ask non-taggers why they tag) </li></ul><ul><li>Structured interviews </li></ul>
    52. 52. Expected… <ul><li>Organization/retrieval </li></ul><ul><li>Creator/synthesizer/consumer </li></ul><ul><li>“Tagging for community” </li></ul>
    53. 53. Found… <ul><li>Motivation taxonomy </li></ul><ul><li>No “synthesizers” </li></ul><ul><li>Inspired by community </li></ul><ul><li>Organization/retrieval </li></ul><ul><li>Creator/Synthesizer </li></ul><ul><li>“ Tagging for community” </li></ul>X
    54. 54. Motivation Taxonomy “ If I tagged ahead of time I can go back and get all my pictures of [my children]…” “… I then think “well, maybe I should tag this” so I can find it again later” “ I’m obsessive-compulsive”
    55. 55. Motivation Taxonomy “ I want at least one hook of association in there that can help me reconstruct…”
    56. 56. Motivation Taxonomy “ I tag photos with what I think might be interesting to other people, stuff I think people will like” “ I know that tagging can connect my photos to activities, and get more interest” “ I want to look at all [my neighborhood’s] tags. … That’s definitely a reason I’m putting these tags in ”
    57. 57. Motivation Taxonomy “… so that my friends can see what I am up to” I can tell my mom [with the tag] “look, we went to…” “ I left reviews of places – like at the airport, when my flight was delayed, I tagged “Aloha Air sucks.”
    58. 58. Breakdown
    59. 59. Suggested Tags <ul><li>Useful for entry, when they “work” </li></ul><ul><ul><li>Critical mass needed </li></ul></ul><ul><ul><li>Better filtering </li></ul></ul><ul><li>Inspire and give example </li></ul><ul><li>Overtagging may be issue </li></ul>“ Tag suggestions were huge for me; they really cut down on typing” “ I thought: this is how I want it to work all the time!” “ This person was in my phone for a month - who is she???” “ I try to use as many suggested tags that apply. … I also use it for auto-completion – I type “s” to get San Francisco”
    60. 60. Tags/Contribution <ul><li>Derived from affordances </li></ul><ul><ul><li>Social engineering </li></ul></ul><ul><li>HT06, tagging paper, taxonomy, Flickr, academic article, to read (HyperText 2006) </li></ul>
    61. 61. Open Questions <ul><li>Those who don’t… </li></ul><ul><li>Better filtering, tag suggestions </li></ul><ul><li>Longer usage trends </li></ul><ul><li>Effect of community on usage </li></ul><ul><li>… </li></ul>
    62. 62. Enough already with this stupid figure!
    63. 63. Final Notes <ul><li>All Flickr/ZT photos, Creative Commons or with author permission </li></ul><ul><li> </li></ul><ul><li> </li></ul><ul><li> </li></ul><ul><li> </li></ul><ul><li> </li></ul>
    64. 64. Final Notes: APIs For All <ul><li>Everything we can do, you can do (better). Use our APIs: </li></ul><ul><ul><li>Cell ID translation </li></ul></ul><ul><ul><li>Suggested Tags </li></ul></ul><ul><ul><li>TagMaps data </li></ul></ul><ul><ul><li>TagMaps Widget </li></ul></ul><ul><ul><li>Extend ZoneTag with RSS feeds, ActionTags </li></ul></ul><ul><ul><li>… </li></ul></ul><ul><li>Email me </li></ul>
    65. 65. Thanks <ul><li>With: Shane Ahern, Simon King, Rahul Nair, Nathan Good, Marc Davis, Morgan Ames, Dean Eckles, Alex Jaffe, Tamir Tassa, Jeannie Yang, Tye Rattenbury </li></ul><ul><li>Read more, follow: </li></ul><ul><li>Slides: </li></ul><ul><li>What are you doing this summer? </li></ul><ul><li>Mor Naaman: </li></ul>
    66. 66. Things We Learned <ul><li>“ Best” feature: 2-click upload </li></ul><ul><ul><li>Even users that often tag </li></ul></ul><ul><li>For some, last chance to annotate </li></ul><ul><li>Mobile tagging - not just experts, not all experts… </li></ul>
    67. 67. Things We Learned <ul><li>Tagging behavior: </li></ul><ul><ul><li>All over the place </li></ul></ul><ul><ul><li>More than Shozu </li></ul></ul><ul><ul><li>Connected to privacy </li></ul></ul>
    68. 68. International Remix <ul><li>Demo? </li></ul><ul><li>Extracting patterns </li></ul>
    69. 69. International Remix
    70. 70. Key Framing
    71. 71. Reuse Patterns
    72. 72. Reuse Patterns
    73. 73. How to use? <ul><li>How can Intl Remix leverage the community to help the individual? </li></ul>
    74. 74. Flickr Example <ul><li>Distribution of “number of photos in account” </li></ul>Number of photos Number of users 6 12 18 … Huh??
    75. 75. Case Study - Flickr <ul><li>Nobody tags other people’s content </li></ul><ul><li>Why? </li></ul>Not collected Not identified Not prominent In user’s account As coming from the tagger In the interface, as “opinion” Not aggregated Can’t “vote” on tag/item pair