The presence of emergent semantics in social annotation systems (e.g., Flickr, Delicious or BibSonomy) has been reported in numerous studies. Two important problems in this context are (i) the induction of semantic relations among tags and (ii) the discovery of dierent senses of a given tag. While a number of approaches for discovering tag senses exist, little is known about which factors influence the tag sense discovery process. In this paper, we analyze pragmatic factors, in particular we analyze if and to what extent different kinds of users and user behavior (i.e. how people use tags) influence a tag sense discovery task. In our experiments, we divide taggers into different pragmatic distinctions, including categorizers, describers, specialists and generalists. Based on these distinctions, we identify a small subset of users whose annotations allow for a more precise and complete discovery of tag senses, evaluated against Wikipedia disambiguation pages as ground truth. Our results provide further empirical evidence for a causal link between tagging pragmatics and semantics, and make a broader argument for including pragmatic factors in future semantic extraction methods. Our work is relevant for improving search, retrieval and browsing in social annotation systems, as well as for optimizing ontology learning algorithms based on tagging data.

1. How Tagging Pragmatics Influence Tag
Sense Discovery in Social Annotation
Systems
Thomas Niebler, Philipp Singer, Dominik Benz,
Christian Körner, Andreas Hotho, Markus Strohmaier
Data Mining and Information Retrieval Group, University of Würzburg, Germany
Knowledge Management Institute and Know Center, Graz University of Technology, Austria
Knowledge and Data Engineering Group (KDE), University of Kassel, Germany

2. Social Annotation Systems
Thomas Niebler 2
Common Underlying Model:
Tags
Ressourcen
User
TAS
Folksonomy model

3. Social Annotation Systems: BibSonomy
Niebler et al.: How Tagging Pragmatics Influence Tag Sense Discovery in Social Annotation Systems 3
Resource (Publication)
Tags
User
 http://www.bibsonomy.org

4. The Problem
4Niebler et al.: How Tagging Pragmatics Influence Tag Sense Discovery in Social Annotation Systems
 Searching for „swing“ on BibSonomy:
 Tag „Swing“ occurs several times, but with different meanings!

5. Swing is not the same as Swing
Niebler et al.: How Tagging Pragmatics Influence Tag Sense Discovery in Social Annotation Systems 5
 How does user behaviour influence a tag sense discovery
process?

6. 6
Agenda
Niebler et al.: How Tagging Pragmatics Influence Tag Sense Discovery in Social Annotation Systems
1. Tag Sense Discovery
2. Tagging Pragmatics
3. Pragmatic Influences on Tag Sense Discovery
4. Results & Discussion

7. Solving „The Problem“
7Niebler et al.: How Tagging Pragmatics Influence Tag Sense Discovery in Social Annotation Systems
 http://www.bibsonomy.org/tag/swing
 How can we find out the intended meaning?

8. Semantic Relatedness
 Tag Co-Occurence
 Calculate Co-Occurrence Counts
 Put these into a matrix
 Columns are the context vectorsNiebler et al.: How Tagging Pragmatics Influence Tag Sense Discovery in Social Annotation Systems 8
Development – Java: 1
Java – Swing: 1
Development – Swing: 1
Tanz – Swing: 1
Tanz – Boogie: 1
Tanz – Dresden: 1
Swing – Boogie: 1
Swing – Dresden: 1
Boogie – Dresden: 1

9. Tag Sense Discovery: Sense Context
 Idea: Different Tag senses
reflected within co-occuring tags
 Sense Context SCt = (Vt, Et) for a
tag t
 Vt = 20 most frequently cooccuring
tags (together with t)
 Et = edges weighted by (semantic)
tag context similarity (e.g. cosine)
 Then: Identify „Sense Groups“ by
hierarchical agglomerative
clustering on the sense context
graph
Niebler et al.: How Tagging Pragmatics Influence Tag Sense Discovery in Social Annotation Systems 9
dresden
usa
Boogie-woogie
1920 dance
java
gui
development
fun
friday

10. 10
Agenda
Niebler et al.: How Tagging Pragmatics Influence Tag Sense Discovery in Social Annotation Systems
1. Tag Sense Discovery
2. Tagging Pragmatics
3. Pragmatic Influences on Tag Sense Discovery
4. Results & Discussion

11. Tagging Pragmatics: Types of Taggers
Niebler et al.: How Tagging Pragmatics Influence Tag Sense Discovery in Social Annotation Systems 11
Categorizer
Small vocabulary
Categorization of
resources by tags
Describer
Uncontrolled vocabulary
Describing content freely
swingdresden
usa
Boogie-woogie
1920
dance
fun
friday
swing
Boogie-woogie
1920
dance

12. Tagging Pragmatics: Types of Taggers
Niebler et al.: How Tagging Pragmatics Influence Tag Sense Discovery in Social Annotation Systems 12
Generalists
More general tags
Variety of topics
Not necessarily deep
knowledge of each topic
Specialists
More specific tags
Deep knowledge in a topic
Concentrates on few topics
Dancing
Sports
Programming
Music
Computer
Tennis
Normal_families
Residue
Holomorphism
Riemann_mapping_theorem
Unit_disc

13. Tagging Pragmatics: Measures Categorizers/Describers
 Example: Measure for Categorizers/Describers:
 Intuition: Describer use open set of many tags, Categorizers only
a small set of controlled tags
 Tag/Resource Ratio:
 High TRR  Describer, Low TRR  Categorizer
 Measures also exist for Generalists/Specialists:
 Mean Degree Centrality
 Tag Entropy
 Similarity Score
Niebler et al.: How Tagging Pragmatics Influence Tag Sense Discovery in Social Annotation Systems 13
How can we determine what kind of tagger a given user u is?
Number of
used tags
Number of
annotated
resources

14. 14
Agenda
Niebler et al.: How Tagging Pragmatics Influence Tag Sense Discovery in Social Annotation Systems
1. Tag Sense Discovery
2. Tagging Pragmatics
3. Pragmatic Influences on Tag Sense Discovery
4. Results & Discussion

15. Influence of Tagging Pragmatics on Tag Sense Discovery
 Idea: Is it possible to discover the same or better tag senses
from subfolksonomies induced by a subset of
describers/categorizers/specialists/generalists?
Niebler et al.: How Tagging Pragmatics Influence Tag Sense Discovery in Social Annotation Systems 15
Extreme
Categorizers/
Specialists Extreme
Describers/
Generalists
Complete folksonomy
Subset of 30% categorizers
= user

16. Experimental setup
1. Apply the aforementioned pragmatic measures to each user
2. Create folksonomy subsets with i% of Categorizers/Describers
and Specialists/Generalists (i = 10, 20, … 100)
3. Apply tag sense discovery on each subset
4. Evaluate results by comparing with Wikipedia
1. Find a disambiguation page for each tag
2. Compare found clusters with sense descriptions
3. If at least 1 matching word is found, match the tag with the
sense
Niebler et al.: How Tagging Pragmatics Influence Tag Sense Discovery in Social Annotation Systems 16

17. Datasets
 From Social Bookmarking Sites Delicious / BibSonomy
 Two filtering steps (to make measures more meaningful):
 Restrict to top 10.000 tags  FULL
 Keep only users with > 100 resources (Delicious) / > 5 resources
(BibSonomy)  MIN100RES / MIN5RES
Niebler et al.: How Tagging Pragmatics Influence Tag Sense Discovery in Social Annotation Systems 17
dataset |T| |U| |R| |Y|
DEL FULL 10,000 511,348 14,567,465 117,319,016
DEL MIN100RES 9,944 100,363 12,125,176 96,298,409
BIBS FULL 10,000 275,584 6,229,611 47,430,890
BIBS MIN5RES 9042 106,110 6,018,708 46,020,286

18. 18
Agenda
Niebler et al.: How Tagging Pragmatics Influence Tag Sense Discovery in Social Annotation Systems
1. Tag Sense Discovery
2. Tagging Pragmatics
3. Pragmatic Influences on Tag Sense Discovery
4. Results & Discussion

19. Results: Categorizers/Specialists (Delicious)
Niebler et al.: How Tagging Pragmatics Influence Tag Sense Discovery in Social Annotation Systems 19
Precision Recall

20. Results: Categorizers/Specialists (Delicious)
Niebler et al.: How Tagging Pragmatics Influence Tag Sense Discovery in Social Annotation Systems 20
Precision
 Nearly all sub-folksonomies perform worse than complete
dataset / random baseline
 Categorizers / Specialists seem not well-suited for sense
discovery

21. Results: Describers/Generalists (Delicious)
Niebler et al.: How Tagging Pragmatics Influence Tag Sense Discovery in Social Annotation Systems 21
Precision
 Describers (trr) and Generalists (mqdc / ten) provide best
results in tag sense discovery
 Especially small partitions seem well-suited

22. Discussion & Implications
 Disambiguation quality not very high (but not focus of this
work!!)
 Small sub-folksonomies based on Describers / Generalists
show globally best performance
 Categorizers / Specialists seem to be unsuited for improving
tag sense discovery
 Relevant for ontology learning, tag recommendation, query
expansion, assisted browsing, …
Niebler et al.: How Tagging Pragmatics Influence Tag Sense Discovery in Social Annotation Systems 22

23. 23
Agenda
Niebler et al.: How Tagging Pragmatics Influence Tag Sense Discovery in Social Annotation Systems
1. Tag Sense Discovery
2. Tagging Pragmatics
3. Pragmatic Influences on Tag Sense Discovery
4. Results & Discussion

24. Results: Describers/Generalists (Delicious)
Niebler et al.: How Tagging Pragmatics Influence Tag Sense Discovery in Social Annotation Systems 24
Precision Recall
 Describers (trr) and Generalists (mqdc / ten) provide best
results
 Especially small partitions seem well-suited