Improving Personal Tagging Consistency Through Visualization Of Tag


Published on

Published in: Technology, Education
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Improving Personal Tagging Consistency Through Visualization Of Tag

  1. 1. HCI International 2009<br />19-24 July 09, San Diego, CA, USA<br />Improving Personal Tagging Consistency through Visualization of Tag Relevancy<br />Dr. Qin Gao*, Yusen Dai, and Kai Fu<br />Institute of Human Factors & Ergonomics<br />Dept. of Industrial Engineering, Tsinghua University<br />
  2. 2. Content<br />Introduction<br />Research Question<br />Tag A<br />Tag B<br />Methodology<br />Results & Discussion<br />Conclusion<br />Tagging consistency is important for users to organize things effectively and to retrieve them efficiently later on.<br />
  3. 3. Introduction<br />Tagging has emerged as a new means of information organization and retrieval<br />Tagging is easy to use, flexible, able to harvest the intelligence of the crowd<br />But there are many inconsistencies in tagging systems!<br />Tripartite model of tagging system, <br />from Halpin, Robu, & Sherpherd, 2007<br />
  4. 4. Introduction<br />Vocabulary problems<br />“Bad” tags: misspelt tags, badely encoded tags, mixed use of singulars and plurals, and etc.<br />Inevitable semantic inconsistency: polysemy, synonym, and basic level variations. (Golder & Huberman 2006)<br />Consistency between taggers<br />The extent to which different users agree on selection for certain tags for specific content.<br />Allowing a true representation of knowledge and multiple interpretations of the same content.<br />Trends towards stabilization (Golder & Huberman, 2005)<br />Consistency within individual taggers<br />The extent to which individual users agree on selection for certain tags for specific content at different point in time.<br />
  5. 5. Introduction<br />Consistency within individual taggers is important to individual users and to the system.<br />Affecting efficiency of information organization and retrieval tasks for individual users<br />Organizing information is one of the most motivation for tagging (Ames and Naaman, 2007; Marlow, et al., 2006).<br />Indexing research shows that reliance on consistently used indexing cues is desired for effective access of information<br />Impacts on users’ perceived usefulness of the system and their satisfaction.<br />How to improve individual tagging consistency?<br />Providing tag suggestions based on existing tagging pattern can shape users’ tagging behavior (Sen et al, 2006; Binkowski, 2006)<br />How to present such suggestions?<br />How to select tags for suggestion?<br />
  6. 6. Visualization of Tags<br />The first generation <br />of tag clouds<br />The second generation <br />of tag clouds<br />Tag popularity is represented by visual cues <br />Semantic relations among tags is revealed by visualization<br />Semantically clustering of tags <br />by Montero & Solana (2006)<br />Tag clouds from Amazon, from Bateman 2007<br />Nielson, 2007<br />
  7. 7. Research Question<br />Goal of the study: to examine the effect of tag frequency visualization and semantically clustering on users’ tagging consistency<br />Hypothesis 1: visualization of occurrence frequency of tags improves personal tag consistency and reduces users’ workload.<br />Hypothesis 2: visualization of inter-tag relevancy improves personal tag consistency.<br />
  8. 8. Methodology<br />2*2 experiment design<br />
  9. 9. Methodology<br />Frequency visualization by font size<br /> the font size was determined by the following logarithm function<br />Definition of font size levels<br />Currenti is the font size level of the current tag<br />Oiis the use frequency of the current tag<br />The relationship between font size level and tag frequency<br />
  10. 10. Methodology<br />ti=(d1i, d2i, d3i, …, dni)<br />Visualization of tag relevancy – Semantically clustering<br />Clusters of relevant tags were calculated based on co-occurrence similarity with K-means algorithm developed by Montero and Solana (2006).<br />The approach was proved to reduce semantically density of tag clouds significantly.<br />Definition of the vector space:<br />ti=(d1i, d2i, … dni)<br />cosine (t1, t2)=(t1·t2)/‖t1‖*‖t2‖<br />
  11. 11. Methodology<br />Let <br /> and <br /> in two sessions<br /> in two sessions<br />Dependent variables<br />Tagging consistency<br />Let Ai and Bi denote the sets of tags that assigned to the same document in two sessions, then tagging consistency with this document:<br />The overall tagging consistency:<br />Workload measured by NASA-TLX<br />Ai and Bi denote the sets of tags that assigned to the same document in two different tagging sessions<br />
  12. 12. Methodology<br />Stimuli<br />100 pictures selected from Flickr, tagged as “nature”, “city”, or “people”<br />20 were stimuli, and other 80 were filler pictures<br />Participants<br />40 participants, including 10 females and 30 males, aged from 20 to 31<br />All are experienced tagging users<br />Procedure<br />Two tagging sessions, with a disruptive interval in between.<br />
  13. 13. Results<br />aKruskal-Wallis-test.*Significant differences at p&lt;.05<br />aKruskal-Wallis-test.*Significant differences at p&lt;.05<br />aKruskal-Wallis-test.*Significant differences at p&lt;.05<br />aKruskal-Wallis-test.*Significant differences at p&lt;.05<br />Testing of hypothesis 1<br />
  14. 14. Results<br />Frequency visualization has no significant impact on tagging consistency.<br />Frequency visualization reduces perceived physical demand significantly, but also increases mental demand.<br />An interaction effect on physical demand (χ2 = 6.4, p = .01)<br />
  15. 15. Results<br />Testing of Hypothesis 2<br />aKruskal-Wallis-test.*Significant differences at p&lt;.05<br />
  16. 16. Results<br />Semantically clustering improves personal tagging significantly. H2 was supported.<br />But no significant difference in workload or the number of tags given by participants.<br />The consistency level of participants tagging with semantically clustering is 12% higher than that of participants tagging without such visualization.<br />
  17. 17. Discussion<br />Two types of tags<br />General categorical tags, influenced by the basic level<br />High recall but low accuracy<br />Users have a strong bias to use them as first tags (Golder & Huberman, 2005).<br />Relatively more consistent.<br />Descriptive/specific tags, ego-centered<br />High accuracy but low recall<br />All participants expressed their intention to tag consistently, but often failed to do so due to limited memory.<br />Major source of inconsistencies<br />
  18. 18. Discussion<br />Semantically clustering of tags helps users’ tag formulation tasks and improves their consistency in identifying and deciding on specific tags<br />It improves the performance of specific search and increase the attention towards tags in small fonts compared to other layouts (Schrammel et al., 2009).<br />Frequency visualization does not provide support for search of specific tags. <br />When used in combination with semantically clustering, it help reduce perceived physical demand.<br />
  19. 19. Conclusion<br />Visualizing the relevancy among tags has a significant positive effect on tagging consistency, whereas visualizing tagging frequency does not. <br />Empirical support for the effort of visualizing semantic relationships among tags<br />When the tag relevancy is visualized, highlight frequently used tags can reduce perceived physical demands; however, it increases perceived mental demands as well.<br />Implications for professional indexer aid design.<br />
  20. 20. Thank you for your attention.<br />Contact:<br /><br /><br />Q & A<br />