Folksonomies and Tagging: Libraries & the Hive Mind Tom Reamy Chief Knowledge Architect KAPS Group Knowledge Architecture Professional Services http://www.kapsgroup.com

Agenda Introduction - Themes Essentials of Folksonomies Advantages, Disadvantages, and Dangers of Folksonomies Improving the Quality of Folksonomies Facets and Flickr Del.icio.is – Topics, Popularity and Findability LibraryThing – facets, topics, and weirdness Folksonomies and Libraries What Won’t Work – What Might Work Semantic Infrastructure Approach Evolving Folksonomies Conclusion

Introduction - Themes

Essentials of Folksonomies

Advantages, Disadvantages, and Dangers of Folksonomies

Improving the Quality of Folksonomies

Facets and Flickr

Del.icio.is – Topics, Popularity and Findability

LibraryThing – facets, topics, and weirdness

Folksonomies and Libraries

What Won’t Work – What Might Work

Semantic Infrastructure Approach

Evolving Folksonomies

Conclusion

KAPS Group: General Knowledge Architecture Professional Services Virtual Company: Network of consultants – 12-15 Partners – Convera, Inxight, Siderean, FAST, etc. Consulting, Strategy, Knowledge architecture audit Taxonomies: Enterprise, Marketing, Insurance, etc. Services: Taxonomy development, consulting, customization Technology Consulting – Search, CMS, Portals, etc. Metadata standards and implementation Knowledge Management: Collaboration, Expertise, e-learning Applied Theory – Faceted taxonomies, complexity theory, natural categories

Knowledge Architecture Professional Services

Virtual Company: Network of consultants – 12-15

Partners – Convera, Inxight, Siderean, FAST, etc.

Consulting, Strategy, Knowledge architecture audit

Taxonomies: Enterprise, Marketing, Insurance, etc.

Services:

Taxonomy development, consulting, customization

Technology Consulting – Search, CMS, Portals, etc.

Metadata standards and implementation

Knowledge Management: Collaboration, Expertise, e-learning

Applied Theory – Faceted taxonomies, complexity theory, natural categories

2.0 Themes “ Tags are great because you throw caution to the wind, forget about whittling down everything into a distinct set of categories and instead let folks loose categorizing their own stuff on their own terms." - Matt Haughey - MetaFilter “ It’s MySpace meets YouTube meets Wikipedia meets Google – on steroids.” “ It’s ignorance meets egotism meets bad taste meets mob rule – on steroids.” – The Cult of the Amateur – Andrew Keen “ Things fall apart; the center cannot hold; Mere anarchy is loosed upon the world,… The best lack all conviction, while the worst Are full of passionate conviction.” - The Second Coming – W.B. Yeats

“ Tags are great because you throw caution to the wind, forget about whittling down everything into a distinct set of categories and instead let folks loose categorizing their own stuff on their own terms." - Matt Haughey - MetaFilter

“ It’s MySpace meets YouTube meets Wikipedia meets Google – on steroids.”

“ It’s ignorance meets egotism meets bad taste meets mob rule – on steroids.” – The Cult of the Amateur – Andrew Keen

“ Things fall apart; the center cannot hold; Mere anarchy is loosed upon the world,… The best lack all conviction, while the worst Are full of passionate conviction.” - The Second Coming – W.B. Yeats

Essentials of Folksonomies? Wikipedia: A folksonomy is an Internet-based information retrieval methodology consisting of collaboratively generated, open-ended labels that categorize content such as Web pages, online photographs, and Web links. A folksonomy is most notably contrasted from a taxonomy – done by users, not professionals, Example sites – Del.icio.us and Flickr (not really – no feedback) It is just metadata that users add Key – social mechanism for seeing other tags

Wikipedia: A folksonomy is an Internet-based information retrieval methodology consisting of collaboratively generated, open-ended labels that categorize content such as Web pages, online photographs, and Web links.

A folksonomy is most notably contrasted from a taxonomy – done by users, not professionals,

Example sites – Del.icio.us and Flickr (not really – no feedback)

It is just metadata that users add

Key – social mechanism for seeing other tags

Advantages of Folksonomies Simple (no complex structure to learn) No need to learn difficult formal classification system Lower cost of categorization Distributes cost of tagging over large population Open ended – can respond quickly to changes Relevance – User’s own terms Support serendipitous form of browsing Easy to tag any object – photo, document, bookmark Better than no tags at all Getting people excited about metadata!

Simple (no complex structure to learn)

No need to learn difficult formal classification system

Lower cost of categorization

Distributes cost of tagging over large population

Open ended – can respond quickly to changes

Relevance – User’s own terms

Support serendipitous form of browsing

Easy to tag any object – photo, document, bookmark

Better than no tags at all

Getting people excited about metadata!

Disadvantages of Folksonomies - Quality They don’t work very well for finding Re-finding is of marginal value No structure, no conceptual relationships Flats lists do not a onomy make Issues of scale – popular tags already showing a million hits Limited applicability – only useful for non-technical or non-specialist domains Either personal tags (other’s can’t find) or popularity tags – lose interesting terms (Power law distribution) Most people can’t tag very well – learned skill Errors – misspellings, single words or bad compounds, single use or idiosyncratic use

They don’t work very well for finding

Re-finding is of marginal value

No structure, no conceptual relationships

Flats lists do not a onomy make

Issues of scale – popular tags already showing a million hits

Limited applicability – only useful for non-technical or non-specialist domains

Either personal tags (other’s can’t find) or popularity tags – lose interesting terms (Power law distribution)

Most people can’t tag very well – learned skill

Errors – misspellings, single words or bad compounds, single use or idiosyncratic use

Dangers of Folksonomies Unwisdom of Crowds “ We find that whole communities suddenly fix their minds upon one object, and go mad in its pursuit; that millions of people become simultaneously impressed with one delusion, and run after it, till their attention is caught by some new folly more captivating than the first.” From witch hunts to tulipomania to stock market crash Extraordinary Popular Delusions and the Madness of Crowds Tyranny of the majority Popularity drowns quality Narrowing of choices, lost content Belief that hierarchy, taxonomy not needed

Unwisdom of Crowds

“ We find that whole communities suddenly fix their minds upon one object, and go mad in its pursuit; that millions of people become simultaneously impressed with one delusion, and run after it, till their attention is caught by some new folly more captivating than the first.”

From witch hunts to tulipomania to stock market crash

Extraordinary Popular Delusions and the Madness of Crowds

Tyranny of the majority

Popularity drowns quality

Narrowing of choices, lost content

Belief that hierarchy, taxonomy not needed

Will Social Networking make better Folksonomies? Not so far – example of Del.icio.us – same tags Quality and Popularity are very different things Most people don’t tag, don’t re-tag Study – folksonomies follow NISO guidelines – nouns, etc – but do they actually work – see analysis (hint – no) Most tags deal with computers and are created by people that love to do this stuff – not regular users and infrequent users – Beware true believers!

Not so far – example of Del.icio.us – same tags

Quality and Popularity are very different things

Most people don’t tag, don’t re-tag

Study – folksonomies follow NISO guidelines – nouns, etc – but do they actually work – see analysis (hint – no)

Most tags deal with computers and are created by people that love to do this stuff – not regular users and infrequent users – Beware true believers!

Flickr Facets

Flickr Facets Faceted navigation – extremely powerful, easy to use to find Combine strength of structure and personal perspectives Basic Facets – over 90% of content Place – Amsterdam to Beach – 40% Events, Date, People, Things / Animals, Color Subject Matter – less than 1% Works on lower level scales: Artparade, tourofbritain, stgilesfair, hideoutblockparty (last weeks)

Faceted navigation – extremely powerful, easy to use to find

Combine strength of structure and personal perspectives

Basic Facets – over 90% of content

Place – Amsterdam to Beach – 40%

Events, Date, People, Things / Animals, Color

Subject Matter – less than 1%

Works on lower level scales:

Artparade, tourofbritain, stgilesfair, hideoutblockparty (last weeks)

Del.icio.us Tags Design blog software music tools reference art video programming webdesign web2.0 mac howto linux tutorial web free news photography shopping blogs css imported education travel javascript food games Development inspiration politics flash apple tips java google osx business windows iphone science productivity books toread helath funny internet wordpress ajax ruby research humor fun technology search opensource Photoshop media recipes cool work article marketing security mobile jobs rails lifehacks tutorials resources php social download diy ubuntu freeware portfolio photo movies writing graphics youtube audio online

Design blog software music tools reference art video programming webdesign web2.0 mac howto linux tutorial web free news photography shopping blogs css imported education travel javascript food games

Development inspiration politics flash apple tips java google osx business windows iphone science productivity books toread helath funny internet wordpress ajax ruby research humor fun technology search opensource

Photoshop media recipes cool work article marketing security mobile jobs rails lifehacks tutorials resources php social download diy ubuntu freeware portfolio photo movies writing graphics youtube audio online

Del.icio.us - Topics, not Facets High level topics - photography, news, education Get related terms by popularity, not conceptual Photography Synonyms - photo, photos Related – art, design, images, camera Related Facet – howto, tutorial, photoshop Popularity is not quality Dominance of computer terms Tyranny of the majority – design (1 MIL), interior design – 3,909 Top 25 – same set, slight order shift – social inertia New terms - important – iphone, ipod, .net, ebooks,facebook Dropped terms – adult, babes, britney, naked, sex, sexy

High level topics - photography, news, education

Get related terms by popularity, not conceptual

Photography

Synonyms - photo, photos

Related – art, design, images, camera

Related Facet – howto, tutorial, photoshop

Popularity is not quality

Dominance of computer terms

Tyranny of the majority – design (1 MIL), interior design – 3,909

Top 25 – same set, slight order shift – social inertia

New terms - important – iphone, ipod, .net, ebooks,facebook

Dropped terms – adult, babes, britney, naked, sex, sexy

Del.icio.us - Folksonomy Findability Too many hits (where have we heard that before?) Design – 1 Mil, software – 931,259, sex – 129,468 No plurals, stemming (singular preferred) Folksonomy – 14,073, folksonomies – 3,843, both – 1,891 Blog-1.7M, blogs – 516,340, Weblog- 155,917, weblogs – 36,434, blogging – 157,922, bloging – 697 Taxonomy – 9.683, taxonomies – 1,574 Personal tags – cool, fun, funny, etc Good for social research, not finding documents or sites How good for personal use? Funny is time dependent

Too many hits (where have we heard that before?)

Design – 1 Mil, software – 931,259, sex – 129,468

No plurals, stemming (singular preferred)

Folksonomy – 14,073, folksonomies – 3,843, both – 1,891

Blog-1.7M, blogs – 516,340, Weblog- 155,917, weblogs – 36,434, blogging – 157,922, bloging – 697

Taxonomy – 9.683, taxonomies – 1,574

Personal tags – cool, fun, funny, etc

Good for social research, not finding documents or sites

How good for personal use? Funny is time dependent

Improving the Quality – Internet Sites Add automatic facets - Flickr Design facet system – one time cost, some monitoring Cluster tags – Taxonomy / Ontology Entity Extraction, populate facets and subjects Types of relationships – ubuntu – tutorial, howto, reference, tips, install Add broad general taxonomy of most popular tags Tags as natural categories – build up and down Start – evolve a simple 2 level taxonomy Evolve quality of tags and emerging structure of tags Preferred term = popular (Blog/blogs – Books/book) Add mechanisms – rank tags, taggers, categories

Add automatic facets - Flickr

Design facet system – one time cost, some monitoring

Cluster tags – Taxonomy / Ontology

Entity Extraction, populate facets and subjects

Types of relationships – ubuntu – tutorial, howto, reference, tips, install

Add broad general taxonomy of most popular tags

Tags as natural categories – build up and down

Start – evolve a simple 2 level taxonomy

Evolve quality of tags and emerging structure of tags

Preferred term = popular (Blog/blogs – Books/book)

Add mechanisms – rank tags, taggers, categories

Folksonomy and Libraries Three contexts: Library Catalog Internet Service Enterprise (KM) contributor Univ of Pennsylvania Penn Tags – academic tagging Stanford – educate patrons about library resources Tool for librarians to identify good Internet sites Research – ALA, Louise Spiteri, Margaret Kipp Library Thing

Three contexts:

Library Catalog

Internet Service

Enterprise (KM) contributor

Univ of Pennsylvania

Penn Tags – academic tagging

Stanford – educate patrons about library resources

Tool for librarians to identify good Internet sites

Research – ALA, Louise Spiteri, Margaret Kipp

Library Thing

Library Thing Book people aren’t much better at tagging High level concepts – psychology (55,000), religion (120,000), science (101,000) Issue – variety of terms – cognitive science – need at least 40 other tags to cover the actual field of cognitive science Strange tags – book (19,000) – it’s a book site? Combination of facets and topics Facets – Date (16 th century, 1950’s, 2007) // Function (owned, not read) // Type (graphic novel, novel) // Genre (horror, mystery) Topics – majority like Del.icio.us

Book people aren’t much better at tagging

High level concepts – psychology (55,000), religion (120,000), science (101,000)

Issue – variety of terms – cognitive science – need at least 40 other tags to cover the actual field of cognitive science

Strange tags – book (19,000) – it’s a book site?

Combination of facets and topics

Facets – Date (16 th century, 1950’s, 2007) // Function (owned, not read) // Type (graphic novel, novel) // Genre (horror, mystery)

Topics – majority like Del.icio.us

Library Thing – Book on Neuroscience 1) (Location: dining room) (1) biological (1) biology (8) box74 (1) Brain (1) brain research (1) brains (1) cognitive neuroscience (1) cognitive science (1) consciousness (1) currently reading (1) HelixHealth (1) kognitionswissenschaft (1) medical (1) medicine (1) neuroscience (19) non-fiction (5) partread (1) Psychology (4) Science (10) textbook (10) theory (1) Too General: Science, Psychology, biology, textbook Too specific: Location: dining room, box74 Facets: currently reading, partread

1) (Location: dining room) (1) biological (1) biology (8) box74 (1) Brain (1) brain research (1) brains (1) cognitive neuroscience (1) cognitive science (1) consciousness (1) currently reading (1) HelixHealth (1) kognitionswissenschaft (1) medical (1) medicine (1) neuroscience (19) non-fiction (5) partread (1) Psychology (4) Science (10) textbook (10) theory (1)

Too General: Science, Psychology, biology, textbook

Too specific: Location: dining room, box74

Facets: currently reading, partread

Library Thing – Harry Potter (51) 2003 (55) 2007 (44) adventure (118) Boarding school (86) British (193) children (292) children's (624) Children's books (39) Children's Fiction (118) children's literature (255) coming of age (41) england (142) English (48) Fantasy (3,412) favorite (54) Favorites (57) Fiction (2,277) film (47) friendship (47) hardcover (136) harry potter (1,499) Hogwarts (90) hp (80) J.K. Rowling (60) juvenile (97) juvenile fiction (40) kids (81) magic (922) Mystery (53) Novel (182) own (175) Owned (51) potter (56) Read (419) rowling (95) school (105) series (435) witchcraft (47) witches (125) Wizardry (68) wizards (409) young adult (823) youth (50)

(51) 2003 (55) 2007 (44) adventure (118) Boarding school (86) British (193) children (292) children's (624) Children's books (39) Children's Fiction (118) children's literature (255) coming of age (41) england (142) English (48) Fantasy (3,412) favorite (54) Favorites (57) Fiction (2,277) film (47) friendship (47) hardcover (136) harry potter (1,499) Hogwarts (90) hp (80) J.K. Rowling (60) juvenile (97) juvenile fiction (40) kids (81) magic (922) Mystery (53) Novel (182) own (175) Owned (51) potter (56) Read (419) rowling (95) school (105) series (435) witchcraft (47) witches (125) Wizardry (68) wizards (409) young adult (823) youth (50)

Library Thing – Harry Potter Redundant: Harry Potter, J.K. Rowling Inconsistent: 51 say 2003, 55 say 2007 Too General: British, Fantasy, Fiction Too many variations – children, children’s, children’s books, children’s fiction, children’s literature Strange: school, series, England, Mystery Personal: Favorites, owned, Read

Redundant: Harry Potter, J.K. Rowling

Inconsistent: 51 say 2003, 55 say 2007

Too General: British, Fantasy, Fiction

Too many variations – children, children’s, children’s books, children’s fiction, children’s literature

Strange: school, series, England, Mystery

Personal: Favorites, owned, Read

What Won’t Work Traditional Library Strategies – improve users Recommendations about count-non-count nouns or singular – plural Link to online dictionary or Wikipedia – extra work, whole focus is on ease of tagging – any help has to be immediate and integrated - or done by a central group Social Networking Social networking good for socializing, not tagging Social Network sites – getting worse not better Either / Or – folksonomies or LCSH

Traditional Library Strategies – improve users

Recommendations about count-non-count nouns or singular – plural

Link to online dictionary or Wikipedia – extra work, whole focus is on ease of tagging – any help has to be immediate and integrated - or done by a central group

Social Networking

Social networking good for socializing, not tagging

Social Network sites – getting worse not better

Either / Or – folksonomies or LCSH

What Might Work Semantic Infrastructure and Evolution Environment and dynamic social rules Integrated Evolving Solution: Content Structures, People, Technology, Policies and Procedures // with Feedback with consequences New Relationship of Central and Crowd Not top down or bottom up Interpenetration of opposites Reduce Folk and Increase ‘Onomy’ Wikipedia – Social plus 2,000 editors Increase Folk – add discussions and social context to tags

Semantic Infrastructure and Evolution

Environment and dynamic social rules

Integrated Evolving Solution: Content Structures, People, Technology, Policies and Procedures // with Feedback with consequences

New Relationship of Central and Crowd

Not top down or bottom up

Interpenetration of opposites

Reduce Folk and Increase ‘Onomy’

Wikipedia – Social plus 2,000 editors

Increase Folk – add discussions and social context to tags

Semantic Infrastructure: New Library Roles Library 2.0 New librarians – social and intellectual context New relationship of center and users – more sophisticated support, more freedom, more suggestions, more user input - New roles – for users (taggers, part of variety of communities – both distributed and central) New roles for central – create feedback system, tweak the evolution of the system, Develop initial candidates Communities of Practice – apply to tagging, ranking Community Maps – formal and informal Map tags to communities – more useful suggestions Use tags to uncover communities

Library 2.0

New librarians – social and intellectual context

New relationship of center and users – more sophisticated support, more freedom, more suggestions, more user input

- New roles – for users (taggers, part of variety of communities – both distributed and central)

New roles for central – create feedback system, tweak the evolution of the system, Develop initial candidates

Communities of Practice – apply to tagging, ranking

Community Maps – formal and informal

Map tags to communities – more useful suggestions

Use tags to uncover communities

Semantic Infrastructure: Technology Enterprise Content Management, KM Platforms Place to add metadata – of all kinds, not just keywords Policy support – important, part of job performance Add tag clouds to input page More sophisticated displays Tag clouds mapped to community map Tag clusters, taxonomy location Semantic Software – Inxight, SchemaLogic etc. Suggest terms based on text, on tag clouds Social Networking – add semantics SNA – apply to people and tags

Enterprise Content Management, KM Platforms

Place to add metadata – of all kinds, not just keywords

Policy support – important, part of job performance

Add tag clouds to input page

More sophisticated displays

Tag clouds mapped to community map

Tag clusters, taxonomy location

Semantic Software – Inxight, SchemaLogic etc.

Suggest terms based on text, on tag clouds

Social Networking – add semantics

SNA – apply to people and tags

Semantic Infrastructure: Putting it all together Complexity Theory and Folksonomies: Feedback Ranking Methods Explicit – people rank directly Categories, tags, taggers Good tags, best bets for terms or categories? Implicit – software evaluation, reverse relevance Ranking Roles Taggers – everyone (rewards, make it easy and fun) Meta-taggers – everyone (but levels of meta-taggers) Editors – tagging system, integration with taxonomy, resolve disputes, Wikipedia model

Ranking Methods

Explicit – people rank directly

Categories, tags, taggers

Good tags, best bets for terms or categories?

Implicit – software evaluation, reverse relevance

Ranking Roles

Taggers – everyone (rewards, make it easy and fun)

Meta-taggers – everyone (but levels of meta-taggers)

Editors – tagging system, integration with taxonomy, resolve disputes, Wikipedia model

Best of Both Worlds: Integrated Solutions Start and end with a formal taxonomy / Ontology Findability vastly superior Communication with others – share tags Take advantage of conceptual relationships Tagging experience – folksonomies plus Users can type any word – system looks it up – plurals, synonyms, preferred terms, spelling variations Software suggestions – based on content of bookmark, document and on popular user tags – natural level not top down New terms flagged and routed to central team Facets – for both things and documents (faceted taxonomy) Software suggests facet values, user override Cognitively simpler task than own value, complex hierarchy

Start and end with a formal taxonomy / Ontology

Findability vastly superior

Communication with others – share tags

Take advantage of conceptual relationships

Tagging experience – folksonomies plus

Users can type any word – system looks it up – plurals, synonyms, preferred terms, spelling variations

Software suggestions – based on content of bookmark, document and on popular user tags – natural level not top down

New terms flagged and routed to central team

Facets – for both things and documents (faceted taxonomy)

Software suggests facet values, user override

Cognitively simpler task than own value, complex hierarchy

Conclusions: Folksonomies and Libraries Library 2.0 – focus on social collaboration, not tagging More complex use of folksonomies Folksonomies can help – but they need help to evolve better quality Fundamental contradiction of ease of tagging and findability will limit usefulness of Internet folksonomies 90% of what you hear about Folksonomies (2.0) is hype – again Folksonomies are a great source for first drafts and social research Evolution, not Revolution! Evolve folksonomies, taxonomies, ontologies Semantic Infrastructure solution (people, policy, technology, semantics) and feedback is best approach

Library 2.0 – focus on social collaboration, not tagging

More complex use of folksonomies

Folksonomies can help – but they need help to evolve better quality

Fundamental contradiction of ease of tagging and findability will limit usefulness of Internet folksonomies

90% of what you hear about Folksonomies (2.0) is hype – again

Folksonomies are a great source for first drafts and social research

Evolution, not Revolution!

Evolve folksonomies, taxonomies, ontologies

Semantic Infrastructure solution (people, policy, technology, semantics) and feedback is best approach

Questions? Tom Reamy [email_address] KAPS Group Knowledge Architecture Professional Services http://www.kapsgroup.com