Your SlideShare is downloading. ×
  • Like
Evaluating Tag-Based Information Access in Image Collections
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Evaluating Tag-Based Information Access in Image Collections

  • 541 views
Published

The availability of social tags has greatly enhanced access to information. …

The availability of social tags has greatly enhanced access to information.
Tag clouds have emerged as a new “social” way to find
and visualize information, providing both one-click access to information
and a snapshot of the “aboutness” of a tagged collection.
A range of research projects explored and compared different tag
artifacts for information access ranging from regular tag clouds to
tag hierarchies. At the same time, there is a lack of user studies that
compare the effectiveness of different types of tag-based browsing
interfaces from the users point of view. This paper contributes to
the research on tag-based information access by presenting a controlled
user study that compared three types of tag-based interfaces
on two recognized types of search tasks – lookup and exploratory
search. Our results demonstrate that tag-based browsing interfaces
significantly outperform traditional search interfaces in both performance
and user satisfaction. At the same time, the differences
between the two types of tag-based browsing interfaces explored in
our study are not as clear.

Published in Education , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
541
On SlideShare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
10
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Graz University of Technology Evaluating Tag-Based Information Access in Image Collections Christoph Trattner*, Yiling Lin, Denis Parra, Zhen Yue, Peter Brusilovsky *Graz University of Technology, Austria University of Pittsburgh, USA Christoph Trattner Hypertext 2012 1
  • 2. Graz University of Technology Tagging Systems “Tagging gained tremendously in popularity over the past few years” Christoph Trattner Hypertext 2012 2
  • 3. Graz University of Technology Tags Christoph Trattner Hypertext 2012 3
  • 4. Graz University of Technology Tags Christoph Trattner Hypertext 2012 4
  • 5. Graz University of Technology Tags Christoph Trattner Hypertext 2012 5
  • 6. Graz University of Technology Problem Statement  Not surprisingly, there was a lot of research in the past few years that for instance investigated the value of tags for efficient search and information retrieval in online information systems  Surprisingly, most of the studies only use information retrieval or network-theoretic measures and ignore the user side  To contribute to this field of research we conducted a controlled user study 1 2 3 Christoph Trattner Hypertext 2012 6
  • 7. Graz University of Technology What will be presented? Are tags useful to be used in search interfaces? Christoph Trattner Hypertext 2012 7
  • 8. Graz University of Technology Dataset  ~ 2,000 images  ~ 4,200 tags  ~ 16,000 tag assignments Interesting Fact: Tags were generated by ~100 users from Amazon Mechanical Turk Christoph Trattner Hypertext 2012 8
  • 9. Graz University of Technology Interfaces1 Baseline 2 Tag Cloud Search Interface Faceted Tag Cloud Search Interface 3 Christoph Trattner Hypertext 2012 9
  • 10. Graz University of Technology How were the interfaces evaluated? Christoph Trattner Hypertext 2012 10
  • 11. Graz University of Technology Evaluation  Within-subject design, i.e. all of or subjects evaluated all interfaces during the study.  Interfaces were counter balancedBaseline Tag Cloud Faceted Tag Cloud1 2 3 Christoph Trattner Hypertext 2012 11
  • 12. Graz University of Technology Evaluation  2 types of tasks:  Look-up search task  9 images with different difficulty level hard 200 Rank Pos 150 medium easy  Exploratory search task 100 50  3 tasks with different difficulty level 0 1 118 235 352 469 586 703 820 937 1054 1171 1288 1405 1522 1639 1756 1873 Sample Task: “Find at least 8 different types of stores/shops in Pittsburgh! Each type of store/shop should have at least two images from different locations, i.e. in total you will have to find at least 16 images.” Christoph Trattner Hypertext 2012 12
  • 13. Graz University of Technology Evaluation: Look-up TaskLook-up task Christoph Trattner Hypertext 2012 13
  • 14. Graz University of Technology Evaluation: Exploratory Search TaskExploratroy searchtask Christoph Trattner Hypertext 2012 14
  • 15. Graz University of Technology Evaluation  All in all, 24 subjects  Median age 31  19 reported to be familiar with tagging systems  All reported to be to use computers more than 5 hours a day  All of them reported to be familiar with search engines  One session took 90 mins Christoph Trattner Hypertext 2012 15
  • 16. Graz University of Technology What are the results? Christoph Trattner Hypertext 2012 16
  • 17. Graz University of Technology Results: Performance (1/2) Variables: Question 1: What interface performs best? • Total Actions • Search Time 1 2 3  Look-up: no sign. differences between interfaces  Exploratory: Tag Cloud Interface out-performs baseline Faceted Tag Cloud Interface almost as slow as baseline Christoph Trattner Hypertext 2012 17
  • 18. Graz University of Technology Results: Performance (2/2) Variables: Question 2: What is the effect of familiarity and difficulty • Total Actions on the performance of the interfaces? • Search Time 1 2 3  On medium difficultly level… Tag Cloud Interface out-performs baseline interface in search time Christoph Trattner Hypertext 2012 18
  • 19. Graz University of Technology Results: Usage (1/2) Question 3: How are the interfaces used? => Log analysis Results:  Search action and click image action used most often  Add tag action sign. more used in facet  Show more results sign. less used in tag cloud Christoph Trattner Hypertext 2012 19
  • 20. Graz University of Technology Results: Usage Question 4: Does tag grouping by semantic category affect the usage of these categories? 50.00% Baseline 45.00% Tag Cloud 40.00% Facet 35.00% 30.00% 25.00% 20.00% Answer: Yes, 15.00% We found sign. differences 10.00% between the baseline and the 5.00% faceted tag cloud interface 0.00% who where when what other Christoph Trattner Hypertext 2012 20
  • 21. Graz University of Technology Results: Participants„ perception of the interfaces Question 5: What was the perception of the particpants regarding the interfaces?  Post-questionaires after each interface Scale: 1=very bad….5=very good Christoph Trattner Hypertext 2012 21
  • 22. Graz University of Technology Results: Preference and Rating Question 6: What was the preference of the users? • Post-questionair was handed out to the subjects with overall 7 questions. Question 7: How are the interfaces rated? Scale: 1 = very bad….5=very good Christoph Trattner Hypertext 2012 22
  • 23. Graz University of Technology Questions?  Why did people like the tag cloud interfaces more than the baseline?  Why was the tag cloud interface better rated than the faceted tag cloud interface?  Why did people recommend the faceted tag cloud interface even if they rated the tag cloud interface higher? Christoph Trattner Hypertext 2012 23
  • 24. Graz University of Technology Results: Comment Analysis (1/3)  Why did people like the tag cloud interfaces? “The tag cloud provided more information than search only interface” “I like tag cloud because it gives me new ideas and it is easier to use” Christoph Trattner Hypertext 2012 24
  • 25. Graz University of Technology Results: Comment Analysis (1/3)  Why did people prefer the faceted tag cloud interface? “It is easy to find the tags that I needed in faceted tag cloud” “I like faceted tag cloud interface, because the interface is clearer and I always know where to find the tag” Christoph Trattner Hypertext 2012 25
  • 26. Graz University of Technology Results: Comment Analysis (2/3)  Why did people prefer the tag cloud interface over the faceted tag cloud interface? “The facet did not seem to identify tags well” “I think the categorization was not good, it was not relevant to the task” ? Christoph Trattner Hypertext 2012 26
  • 27. Graz University of Technology Conclusions of this work  In general, tags are useful in search interfaces They help the user to find information faster Less clicks, less search time They give the users hints They make the user happier However, depending on the tag interface design different results Take home message can be observed… Not always the most advanced interface design is the best choice… Christoph Trattner Hypertext 2012 27
  • 28. Graz University of Technology End of Presentation Thank you! Christoph Trattner ctrattner@iicm.edu Graz University of Technology, Austria Christoph Trattner Hypertext 2012 28