0
User Intentions inVisual InformationRetrieval &MultimediaInformation SystemsMathias Lux
User Intentions inVisual InformationRetrieval &MultimediaInformation SystemsMathias Lux
Query By Example• User has particular information need• Need reflected by example image• Query is expressed visually
We all know that ...• Some features work better than others• Features have different characteristics• Some features work o...
PHOG & Flowers
ColorLayout & Sunsets
EdgeHistogram & Portraits
JCD & Portraits
ColorLayout & Landscapes
PHOG & Birds on the Water
Which one is right?• How to determine the right feature?• What are the necessary characteristics?• How do I define visual ...
Why is there a differentranking?
Users in Context
Definition: Context“Context is any information that can be usedto characterize the situation of an entity.An entity is a p...
Definition: Intentionnoun(1) thing intended; an aim or plan(2) Medicine the healing process of a wound(3) (intentions) Log...
Context vs. Intention?Context is anyinformation that canbe used tocharacterize thesituation of anentity. An entity is aper...
A User’s Intention is• part of a user’s context• of manageable size (verb & frame)• related to the information need in sea...
User Intentions in the WebUnderlying goals of web searches• Informational– to learn / know something• Navigational– to go ...
User Intentions in the WebRef. Rose & Levinson: Understanding user goals in websearch, WWW 2004Broder Survey Broder LogAna...
User Intentions nMultimedia• Search• Production• Sharing• Archiving• Image• Video• Audio• Multiple modalities
Hand-picked ExamplesRight now there is no all-in-one publicationon user intentions in multimedia …
User Intentions inImage SearchRef. Lux, Kofler & Marques: A classification scheme foruser intentions in image search, CHI ...
Do queries help with thesearch intention?User information need vs. query formulation invideo search.• How to support users...
Asking for the “Why?”behind the “What?”Ref. Hanjalic, Kofler & Larson: Intent and itsDiscontents: The User at the Wheel of...
Asking for the “Why?”behind the “What?”• Information– news, commercial, advertisement, documentary, science,commentary, ed...
Helping with clever Uis?Ref. Lagger, Lux & Marques: An Adaptive Video RetrievalSystem Based On Recent Studies On User Inte...
Who are the Users in aVideo Search System?• Study on users of– YouTube– BBC iPlayer– Uitzending GemistRef. Kemman, Kleppe&...
Who are the Users in a VideoSearch System?Ref. Kemman, Kleppe & Beunders: Who are the users ofa video search system? Class...
Why do People make Videos?• Study on four main goals:– Affection, Function, Sharing & Preservation.Ref. Lux & Huber: Why d...
Finding User Intentions &Goals is a hard task ….
Demand Media – TheAnswer Factory• Demand mined from search queries• Requests for content put on auction• Contractors creat...
Human Computation
Human Computation
Human Computation• Is crowd-sourcing of any help here?– cp. ACM MM & work of Kofler, Larson &Hanjalic
Crowd-Sourcing• It’s hard to judge intentions of others– That makes it error prone“a reminder of the beautiful Islandwere ...
turkersRecall situation 2 2 0 1 0 2Preserve good feeling 2 -2 1 0 0 1Publish online 2 0 0 0 1 2Show to family & friends 2 ...
Crowd-Sourcing• Turkers disagreed with original publishers.• But pretests had better inter-rateragreements.Ref. Lux, Tasch...
Human Computation• How about motivating people, i.e. withfun & rewarding experience?
Games with aadditional Purpose
Games with aadditional Purpose• Tag a Tune• Popvideo• Matchin• Flip It• Verbosity
Games with aadditional Purpose• How to go beyond annotation?– classical applications are focused on annotation• How to inf...
Leveraging EducationalNeeds …
Where did we go?• CBIR & QBE• User Intentions– Search– Production– Sharing• Games with additionalPurpose
What is left?• Lots of loose ends & open grounds forresearch …… let me propose four different PhDtheses …
Open PhD Theses I• General Model for User Intentions &Goals in Multimedia.– Is there a unified model?– What are the class ...
Open PhD Theses II• GWAP, HC & UIs for determining & inferring &utilizing User Intentions & Goals– Which UI elements, game...
Open PhD Theses III• Bringing Context to the Query inMultimedia Information Systems.– How to utilize Intentions & Goals wi...
Open PhD Theses IV• Adaptable Applications– How to adapt an application to users’intentions?– Which elements & process to ...
Thanks for listening …• Mathias Lux• Klagenfurt University, AT• mlux@itec.uni-klu.ac.at
CBMI 2013 Presentation: User Intentions in Multimedia
CBMI 2013 Presentation: User Intentions in Multimedia
Upcoming SlideShare
Loading in...5
×

CBMI 2013 Presentation: User Intentions in Multimedia

622

Published on

1 Comment
2 Likes
Statistics
Notes
No Downloads
Views
Total Views
622
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
4
Comments
1
Likes
2
Embeds 0
No embeds

No notes for slide

Transcript of "CBMI 2013 Presentation: User Intentions in Multimedia"

  1. 1. User Intentions inVisual InformationRetrieval &MultimediaInformation SystemsMathias Lux
  2. 2. User Intentions inVisual InformationRetrieval &MultimediaInformation SystemsMathias Lux
  3. 3. Query By Example• User has particular information need• Need reflected by example image• Query is expressed visually
  4. 4. We all know that ...• Some features work better than others• Features have different characteristics• Some features work out well for somedomains, while others don’t
  5. 5. PHOG & Flowers
  6. 6. ColorLayout & Sunsets
  7. 7. EdgeHistogram & Portraits
  8. 8. JCD & Portraits
  9. 9. ColorLayout & Landscapes
  10. 10. PHOG & Birds on the Water
  11. 11. Which one is right?• How to determine the right feature?• What are the necessary characteristics?• How do I define visual similaritywithin the domain?• What is visual similarityfor the user?
  12. 12. Why is there a differentranking?
  13. 13. Users in Context
  14. 14. Definition: Context“Context is any information that can be usedto characterize the situation of an entity.An entity is a person, place, or object thatis considered relevant to the interactionbetween a user and an application, includingthe user and applications themselves.”Ref. G. Abowd et al., “Towards a better understandingof context and context-awareness. In Handheld andUbiquitous Computing, vol. 1707 LNCS, 1999.
  15. 15. Definition: Intentionnoun(1) thing intended; an aim or plan(2) Medicine the healing process of a wound(3) (intentions) Logic conceptions formedby directing the mind towards an object
  16. 16. Context vs. Intention?Context is anyinformation that canbe used tocharacterize thesituation of anentity. An entity is aperson, […]noun(1) thing intended; anaim or plan[…]
  17. 17. A User’s Intention is• part of a user’s context• of manageable size (verb & frame)• related to the information need in searchExamples• I want to download a new background for my mobile.• I want to share the first laugh of my daughter.• I want to see what a Lancia Lyra looks like.
  18. 18. User Intentions in the WebUnderlying goals of web searches• Informational– to learn / know something• Navigational– to go to a specific place (on the web)• Transactional– to go somewhere to ultimately buy sth.Ref. Broder: A Taxonomy of Web Search. SIGIR Forum,2002
  19. 19. User Intentions in the WebRef. Rose & Levinson: Understanding user goals in websearch, WWW 2004Broder Survey Broder LogAnalysisR&L St udy 1 R&L St udy 2 R&L St udy 324,5 2014,7 11,7 13,539 48 60,961,3 61,5363024,3 27 25Navigat ionalInf ormat ionalTransact ional
  20. 20. User Intentions nMultimedia• Search• Production• Sharing• Archiving• Image• Video• Audio• Multiple modalities
  21. 21. Hand-picked ExamplesRight now there is no all-in-one publicationon user intentions in multimedia …
  22. 22. User Intentions inImage SearchRef. Lux, Kofler & Marques: A classification scheme foruser intentions in image search, CHI 2010beautiful sunset for thebackground of my mobileold vs. newStarbucks logomy friend’s newcar on flickr.comLatest “explore”photos
  23. 23. Do queries help with thesearch intention?User information need vs. query formulation invideo search.• How to support users with video indexing andsearch methods?• Search goal failure is (partially) predictable– Based on keywords and– Based on natural languageRef. Kofler, Larson & Hanjalic: To Seek, Perchance toFail: Expressions of User Needs in Internet VideoSearch, ECIR 2011
  24. 24. Asking for the “Why?”behind the “What?”Ref. Hanjalic, Kofler & Larson: Intent and itsDiscontents: The User at the Wheel of the Online VideoSearch Engine, ACM MM 2012Hear others singing …Learn to sing …
  25. 25. Asking for the “Why?”behind the “What?”• Information– news, commercial, advertisement, documentary, science,commentary, education, learning, …• Experience– tutorial, how-to, advise, help, training• Affect– books, podcast, music, comedy, series, art, movie, action,gaming, film, episode, entertainment, …Ref. Hanjalic, Kofler & Larson: Intent and itsDiscontents: The User at the Wheel of the Online VideoSearch Engine, ACM MM 2012
  26. 26. Helping with clever Uis?Ref. Lagger, Lux & Marques: An Adaptive Video RetrievalSystem Based On Recent Studies On User IntentionsWhile Watching Videos Online. ACM CIE, online.
  27. 27. Who are the Users in aVideo Search System?• Study on users of– YouTube– BBC iPlayer– Uitzending GemistRef. Kemman, Kleppe& Beunders: Who are the users ofa video search system? Classifying a heterogeneousgroup with a profile matrix, WIAMIS 2012
  28. 28. Who are the Users in a VideoSearch System?Ref. Kemman, Kleppe & Beunders: Who are the users ofa video search system? Classifying a heterogeneousgroup with a profile matrix, WIAMIS 2012
  29. 29. Why do People make Videos?• Study on four main goals:– Affection, Function, Sharing & Preservation.Ref. Lux & Huber: Why did you record this video?WIAMIS 2012Sharing Af f ect ion Funct ion- 0,59 - 0,7 8 - 0,36- 0,05 - 0,26 - 0,360,39 0,84 0,55- 0,50 - 0,930,25 - 0,070,46 0,21- 0,43- 0,210,47Preservat ionSharingAf f ect ion
  30. 30. Finding User Intentions &Goals is a hard task ….
  31. 31. Demand Media – TheAnswer Factory• Demand mined from search queries• Requests for content put on auction• Contractors create content• Crowd does quality controlsee i.e. eHow.comThe Answer Factory: Demand Media and the Fast,Disposable, and Profitable as Hell Media Model,http://www.wired.com/magazine/2009/10/ff_demandmedia/
  32. 32. Human Computation
  33. 33. Human Computation
  34. 34. Human Computation• Is crowd-sourcing of any help here?– cp. ACM MM & work of Kofler, Larson &Hanjalic
  35. 35. Crowd-Sourcing• It’s hard to judge intentions of others– That makes it error prone“a reminder of the beautiful Islandwere [sic] my father came from”o Recall situationo Preserve good feelingso Publish onlineo Show to family & friendso Support task of mineo Preserve bad feeling
  36. 36. turkersRecall situation 2 2 0 1 0 2Preserve good feeling 2 -2 1 0 0 1Publish online 2 0 0 0 1 2Show to family & friends 2 1 2 1 1 0Support task of mine 0 -2 1 1 1 -2Preserve bad feeling -2 0 -2 0 -2 -2“a reminder of the beautiful Islandwere [sic] my father came from”o Recall situationo Preserve good feelingso Publish onlineo Show to family & friendso Support task of mineo Preserve bad feeling
  37. 37. Crowd-Sourcing• Turkers disagreed with original publishers.• But pretests had better inter-rateragreements.Ref. Lux, Taschwer & Marques: A Closer Look atPhotographers’ Intentions: a Test Dataset, CrowdMM WSat ACM MM 2012Int ent ions Ot hert urkers 0,147 0,232pret est s 0,57 1 0,510
  38. 38. Human Computation• How about motivating people, i.e. withfun & rewarding experience?
  39. 39. Games with aadditional Purpose
  40. 40. Games with aadditional Purpose• Tag a Tune• Popvideo• Matchin• Flip It• Verbosity
  41. 41. Games with aadditional Purpose• How to go beyond annotation?– classical applications are focused on annotation• How to infer user intentions?– proves to be hard to “guess” intentions of others– even “own” intentions may not be explicit• How to leverage user intentions?– finding which intentions can be leveraged and which goalscan be supported
  42. 42. Leveraging EducationalNeeds …
  43. 43. Where did we go?• CBIR & QBE• User Intentions– Search– Production– Sharing• Games with additionalPurpose
  44. 44. What is left?• Lots of loose ends & open grounds forresearch …… let me propose four different PhDtheses …
  45. 45. Open PhD Theses I• General Model for User Intentions &Goals in Multimedia.– Is there a unified model?– What are the class cardinalities?– How to map production, archiving, search andsharing intentions?
  46. 46. Open PhD Theses II• GWAP, HC & UIs for determining & inferring &utilizing User Intentions & Goals– Which UI elements, game mechanics and HC mechanicshelp in this scenario?– What are appropriate design patterns and scenarios?– What is an appropriate research methodology andhow to (easily) evaluate?
  47. 47. Open PhD Theses III• Bringing Context to the Query inMultimedia Information Systems.– How to utilize Intentions & Goals withinsearch and indexing methodology?– Building MMIS around a model for userintentions.
  48. 48. Open PhD Theses IV• Adaptable Applications– How to adapt an application to users’intentions?– Which elements & process to display, etc.?
  49. 49. Thanks for listening …• Mathias Lux• Klagenfurt University, AT• mlux@itec.uni-klu.ac.at
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×