Organizing Resources on Tagging Systems using T-ORG Rabeeh Abbasi Steffen Staab (University of Koblenz-Landau, Germany) Philipp Cimiano (University of Karlsruhe, Germany) Bridging the Gap between Semantic Web and Web 2.0 Innsbruck, Austria June 07, 2007

Overview Social Tagging Systems Browsing a Tagging System T-ORG T-KNOW Experiments Results Conclusion and Future Work

Social Tagging Systems

Browsing a Tagging System

T-ORG

T-KNOW

Experiments

Results

Conclusion and Future Work

Social Tagging Systems / Folksonomies In a social tagging system, people add keywords (called tags ) to their resources and share these resources with others Advantages low-cost classification, improve search, reputation systems, personal organization, no fixed vocabulary, collaboration…

In a social tagging system, people add keywords (called tags ) to their resources and share these resources with others

Advantages

low-cost classification, improve search, reputation systems, personal organization, no fixed vocabulary, collaboration…

Social Tagging Systems – Browsing? I want to “ browse ” vehicle images!!! how can I do it? can I do it using a Tag Cloud? Perhaps I need to structure the tags and resources! how can I do it? Put them into categories (like Vehicles, People, etc)! Do it Manually or with Training? Might not be possible on a large scale! Automatically and without any training! Using T-ORG!

I want to “ browse ” vehicle images!!!

how can I do it?

can I do it using a Tag Cloud?

Perhaps I need to structure the tags and resources!

how can I do it?

Put them into categories (like Vehicles, People, etc)!

Do it Manually or with Training?

Might not be possible on a large scale!

Automatically and without any training!

Using T-ORG!

T-ORG – Classification Organize resources by putting their tags into categories depending upon their context Users can browse categories to retrieve required resources President Gerald Ford Nixon Pardon Group 2 Group 1 Eiffel Eiffel tower Big Eyeful Paris France Miniatures Singen Cars Motors Ford 1955 Person Location Vehicle Categories User A User B

Organize resources by putting their tags into categories depending upon their context

Users can browse categories to retrieve required resources

T-ORG Tag Organization using T-ORG Select ontologies related to the categories (e.g. Vehicle, People, etc.) Prune and refine these ontologies according to the desired categories (add missing concepts, filter existing concepts) Apply the classification algorithm T-KNOW to classify the tags and resources Browse the categories to explore the tags and resources

Tag Organization using T-ORG

Classifying the tags using T-KNOW Use well-known linguistic patterns to generate queries Search these patterns on Google and download search results Compare each Google search result with the context of the tag and extract the concept Select the concept which has the highest similarity with the context of the tag

T-KNOW – Computing Similarity Compute similarity using cosine measure between Bag of Words (BOW) representation of “Tag Context” and “Search Result” 1955 = 1 as = 0 cars = 1 ford = 1 foundation = 0 international = 0 motors = 1 organizations = 0 singen = 1 such = 0 1955 = 0 as = 1 cars = 0 ford = 1 foundation = 2 international = 1 motors = 0 organizations = 1 singen = 0 such = 1 Tag Context singen cars motors ford 1955 cos( ĉ,â ) = ĉ x â / |ĉ||â| = 0.15 ĉ â Only consider the results having similarity above a certain Threshold Result having the highest similarity is considered as final Search Result BOW

Compute similarity using cosine measure between Bag of Words (BOW) representation of “Tag Context” and “Search Result”

Only consider the results having similarity above a certain Threshold

Result having the highest similarity is considered as final

T-KNOW – Computing Similarity – Resource Context Getting the context of the tag “Ford” from middle image using Resource Context Select all tags of the current resource President, Gerald, Nixon, Pardon President Gerald Ford Nixon Pardon Eiffel Eiffel tower Big Eyeful Paris France Miniatures Singen Cars Motors Ford 1955

Getting the context of the tag “Ford” from middle image using

Resource Context

Select all tags of the current resource

President, Gerald, Nixon, Pardon

T-KNOW – Computing Similarity – Tag Context Getting the context of the tag “Ford” from middle image using Tag Context Select all tags of all the resources having this tag “Ford” President, Gerald, Nixon, Pardon, Singen, Cars, Motors, 1955 President Gerald Ford Nixon Pardon Eiffel Eiffel tower Big Eyeful Paris France Miniatures Singen Cars Motors Ford 1955

Getting the context of the tag “Ford” from middle image using

Tag Context

Select all tags of all the resources having this tag “Ford”

President, Gerald, Nixon, Pardon, Singen, Cars, Motors, 1955

T-KNOW – Computing Similarity – User Context Getting the context of the tag “Ford” from middle image using User Context Select all tags of all the resources from the user who use this resource President, Gerald, Nixon, Pardon, Eiffel, Eiffel tower, Big, Eyeful, Paris, France, Miniatures President Gerald Ford Nixon Pardon Eiffel Eiffel tower Big Eyeful Paris France Miniatures Singen Cars Motors Ford 1955 User A User B

Getting the context of the tag “Ford” from middle image using

User Context

Select all tags of all the resources from the user who use this resource

President, Gerald, Nixon, Pardon, Eiffel, Eiffel tower, Big, Eyeful, Paris, France, Miniatures

T-KNOW – Computing Similarity – Group Context Getting the context of the tag “Ford” from middle image using Group Context Select all tags of all the resources present in the group to which this resource belong President, Gerald, Nixon, Pardon, Eiffel, Eiffel tower, Big, Eyeful, Paris, France, Miniatures, Singen, Cars, Motors, Ford, 1955 President Gerald Ford Nixon Pardon Group 2 Group 1 Eiffel Eiffel tower Big Eyeful Paris France Miniatures Singen Cars Motors Ford 1955

Getting the context of the tag “Ford” from middle image using

Group Context

Select all tags of all the resources present in the group to which this resource belong

President, Gerald, Nixon, Pardon, Eiffel, Eiffel tower, Big, Eyeful, Paris, France, Miniatures, Singen, Cars, Motors, Ford, 1955

Experimental Setup Person Location Vehicle Organization Other Author, Singer, Human, … Country, District, City, Village,… Vehicle, Car, Truck, Motorbike, Train, … Company, Organization, Firm, Foundation, … 4+1 Categories 932 Concepts 189 random Images from 9 Flickr groups 1754 Tags

Experimental Setup – Classifiers Two human classifiers: K (gold standard) and S T-KNOW

Two human classifiers: K (gold standard) and S

T-KNOW

Experimental Setup – Evaluation F-Measure A = set of correct classification by test (user S or T-KNOW) B = set of all classification by Gold Standard (user K) C = set of all classifications by test Precision = A / C Recall = A / B F-Measure = 2 * Precision * Recall / (Precision + Recall) Cohen’s Kappa Considers classification done by chance Used to measure classifiers reliability P 0 = observed agreement between classifiers P c = agreement occurred due to chance

F-Measure

A = set of correct classification by test (user S or T-KNOW)

B = set of all classification by Gold Standard (user K)

C = set of all classifications by test

Precision = A / C

Recall = A / B

F-Measure = 2 * Precision * Recall / (Precision + Recall)

Cohen’s Kappa

Considers classification done by chance

Used to measure classifiers reliability

P 0 = observed agreement between classifiers

P c = agreement occurred due to chance

Results – F-Measure - Results comparable to Human Classification 51% 79%

Results – Cohen’s Kappa - Might be a good measure when there is a chance of classification by chance 0% 53%

Conclusion and Future Work -Austria -Germany -Pakistan -USA +Animals +Cameras +Colours +Events +Languages +People +Places +Programming +Resources +Cities +Countries +Lakes +Markets +Universities

Questions/Comments? Q & A