CBMI 2013 Presentation: User Intentions in Multimedia
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
868
On Slideshare
865
From Embeds
3
Number of Embeds
1

Actions

Shares
Downloads
3
Comments
1
Likes
2

Embeds 3

https://twitter.com 3

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. User Intentions inVisual InformationRetrieval &MultimediaInformation SystemsMathias Lux
  • 2. User Intentions inVisual InformationRetrieval &MultimediaInformation SystemsMathias Lux
  • 3. Query By Example• User has particular information need• Need reflected by example image• Query is expressed visually
  • 4. We all know that ...• Some features work better than others• Features have different characteristics• Some features work out well for somedomains, while others don’t
  • 5. PHOG & Flowers
  • 6. ColorLayout & Sunsets
  • 7. EdgeHistogram & Portraits
  • 8. JCD & Portraits
  • 9. ColorLayout & Landscapes
  • 10. PHOG & Birds on the Water
  • 11. Which one is right?• How to determine the right feature?• What are the necessary characteristics?• How do I define visual similaritywithin the domain?• What is visual similarityfor the user?
  • 12. Why is there a differentranking?
  • 13. Users in Context
  • 14. Definition: Context“Context is any information that can be usedto characterize the situation of an entity.An entity is a person, place, or object thatis considered relevant to the interactionbetween a user and an application, includingthe user and applications themselves.”Ref. G. Abowd et al., “Towards a better understandingof context and context-awareness. In Handheld andUbiquitous Computing, vol. 1707 LNCS, 1999.
  • 15. Definition: Intentionnoun(1) thing intended; an aim or plan(2) Medicine the healing process of a wound(3) (intentions) Logic conceptions formedby directing the mind towards an object
  • 16. Context vs. Intention?Context is anyinformation that canbe used tocharacterize thesituation of anentity. An entity is aperson, […]noun(1) thing intended; anaim or plan[…]
  • 17. A User’s Intention is• part of a user’s context• of manageable size (verb & frame)• related to the information need in searchExamples• I want to download a new background for my mobile.• I want to share the first laugh of my daughter.• I want to see what a Lancia Lyra looks like.
  • 18. User Intentions in the WebUnderlying goals of web searches• Informational– to learn / know something• Navigational– to go to a specific place (on the web)• Transactional– to go somewhere to ultimately buy sth.Ref. Broder: A Taxonomy of Web Search. SIGIR Forum,2002
  • 19. User Intentions in the WebRef. Rose & Levinson: Understanding user goals in websearch, WWW 2004Broder Survey Broder LogAnalysisR&L St udy 1 R&L St udy 2 R&L St udy 324,5 2014,7 11,7 13,539 48 60,961,3 61,5363024,3 27 25Navigat ionalInf ormat ionalTransact ional
  • 20. User Intentions nMultimedia• Search• Production• Sharing• Archiving• Image• Video• Audio• Multiple modalities
  • 21. Hand-picked ExamplesRight now there is no all-in-one publicationon user intentions in multimedia …
  • 22. User Intentions inImage SearchRef. Lux, Kofler & Marques: A classification scheme foruser intentions in image search, CHI 2010beautiful sunset for thebackground of my mobileold vs. newStarbucks logomy friend’s newcar on flickr.comLatest “explore”photos
  • 23. Do queries help with thesearch intention?User information need vs. query formulation invideo search.• How to support users with video indexing andsearch methods?• Search goal failure is (partially) predictable– Based on keywords and– Based on natural languageRef. Kofler, Larson & Hanjalic: To Seek, Perchance toFail: Expressions of User Needs in Internet VideoSearch, ECIR 2011
  • 24. Asking for the “Why?”behind the “What?”Ref. Hanjalic, Kofler & Larson: Intent and itsDiscontents: The User at the Wheel of the Online VideoSearch Engine, ACM MM 2012Hear others singing …Learn to sing …
  • 25. Asking for the “Why?”behind the “What?”• Information– news, commercial, advertisement, documentary, science,commentary, education, learning, …• Experience– tutorial, how-to, advise, help, training• Affect– books, podcast, music, comedy, series, art, movie, action,gaming, film, episode, entertainment, …Ref. Hanjalic, Kofler & Larson: Intent and itsDiscontents: The User at the Wheel of the Online VideoSearch Engine, ACM MM 2012
  • 26. Helping with clever Uis?Ref. Lagger, Lux & Marques: An Adaptive Video RetrievalSystem Based On Recent Studies On User IntentionsWhile Watching Videos Online. ACM CIE, online.
  • 27. Who are the Users in aVideo Search System?• Study on users of– YouTube– BBC iPlayer– Uitzending GemistRef. Kemman, Kleppe& Beunders: Who are the users ofa video search system? Classifying a heterogeneousgroup with a profile matrix, WIAMIS 2012
  • 28. Who are the Users in a VideoSearch System?Ref. Kemman, Kleppe & Beunders: Who are the users ofa video search system? Classifying a heterogeneousgroup with a profile matrix, WIAMIS 2012
  • 29. Why do People make Videos?• Study on four main goals:– Affection, Function, Sharing & Preservation.Ref. Lux & Huber: Why did you record this video?WIAMIS 2012Sharing Af f ect ion Funct ion- 0,59 - 0,7 8 - 0,36- 0,05 - 0,26 - 0,360,39 0,84 0,55- 0,50 - 0,930,25 - 0,070,46 0,21- 0,43- 0,210,47Preservat ionSharingAf f ect ion
  • 30. Finding User Intentions &Goals is a hard task ….
  • 31. Demand Media – TheAnswer Factory• Demand mined from search queries• Requests for content put on auction• Contractors create content• Crowd does quality controlsee i.e. eHow.comThe Answer Factory: Demand Media and the Fast,Disposable, and Profitable as Hell Media Model,http://www.wired.com/magazine/2009/10/ff_demandmedia/
  • 32. Human Computation
  • 33. Human Computation
  • 34. Human Computation• Is crowd-sourcing of any help here?– cp. ACM MM & work of Kofler, Larson &Hanjalic
  • 35. Crowd-Sourcing• It’s hard to judge intentions of others– That makes it error prone“a reminder of the beautiful Islandwere [sic] my father came from”o Recall situationo Preserve good feelingso Publish onlineo Show to family & friendso Support task of mineo Preserve bad feeling
  • 36. turkersRecall situation 2 2 0 1 0 2Preserve good feeling 2 -2 1 0 0 1Publish online 2 0 0 0 1 2Show to family & friends 2 1 2 1 1 0Support task of mine 0 -2 1 1 1 -2Preserve bad feeling -2 0 -2 0 -2 -2“a reminder of the beautiful Islandwere [sic] my father came from”o Recall situationo Preserve good feelingso Publish onlineo Show to family & friendso Support task of mineo Preserve bad feeling
  • 37. Crowd-Sourcing• Turkers disagreed with original publishers.• But pretests had better inter-rateragreements.Ref. Lux, Taschwer & Marques: A Closer Look atPhotographers’ Intentions: a Test Dataset, CrowdMM WSat ACM MM 2012Int ent ions Ot hert urkers 0,147 0,232pret est s 0,57 1 0,510
  • 38. Human Computation• How about motivating people, i.e. withfun & rewarding experience?
  • 39. Games with aadditional Purpose
  • 40. Games with aadditional Purpose• Tag a Tune• Popvideo• Matchin• Flip It• Verbosity
  • 41. Games with aadditional Purpose• How to go beyond annotation?– classical applications are focused on annotation• How to infer user intentions?– proves to be hard to “guess” intentions of others– even “own” intentions may not be explicit• How to leverage user intentions?– finding which intentions can be leveraged and which goalscan be supported
  • 42. Leveraging EducationalNeeds …
  • 43. Where did we go?• CBIR & QBE• User Intentions– Search– Production– Sharing• Games with additionalPurpose
  • 44. What is left?• Lots of loose ends & open grounds forresearch …… let me propose four different PhDtheses …
  • 45. Open PhD Theses I• General Model for User Intentions &Goals in Multimedia.– Is there a unified model?– What are the class cardinalities?– How to map production, archiving, search andsharing intentions?
  • 46. Open PhD Theses II• GWAP, HC & UIs for determining & inferring &utilizing User Intentions & Goals– Which UI elements, game mechanics and HC mechanicshelp in this scenario?– What are appropriate design patterns and scenarios?– What is an appropriate research methodology andhow to (easily) evaluate?
  • 47. Open PhD Theses III• Bringing Context to the Query inMultimedia Information Systems.– How to utilize Intentions & Goals withinsearch and indexing methodology?– Building MMIS around a model for userintentions.
  • 48. Open PhD Theses IV• Adaptable Applications– How to adapt an application to users’intentions?– Which elements & process to display, etc.?
  • 49. Thanks for listening …• Mathias Lux• Klagenfurt University, AT• mlux@itec.uni-klu.ac.at