Content Findability in a Portable Content World

Content Findability in a Portable Content World Lise Kreps, M.S.L.S. Relevant Information Services [email_address]

Prologue: Who am I?
Master of Library Science, 1987
Academic & public librarian
Taught at University of Washington’s iSchool
20 years in technical documentation, usability and e-commerce
Software manual & online Help indexing
Cataloging books, images, and audio
Amazon, Microsoft, Corbis, National Public Radio

Content Findability in a Portable Content World
Act I: What makes information worth finding?
Act II: What’s it all about?
Act III: Too much of a good thing?
Act IV: Can’t machines do this?
Act V: What’s findability worth to you?

Act I: What makes information worth finding?
It satisfies my need well enough
Not more trouble than it’s worth to get it
Sounds simple, eh?
I know what I want, so why doesn’t it just magically appear?
Let’s look more closely…

The information satisfies my need well enough
The info I need -- not someone else needs
…for my specific purpose
… at this particular time
…in my particular context

Contains enough useful info to be worth my while
From a source I trust
In language or style appropriate for my need
In a format I can use for this need
No legal or financial barriers to my using it

I didn’t miss anything too important
In the Library & Information Science world, this is called Recall :
Number of relevant items retrieved [divided by the] Total number of relevant items available
“ Do I think I got enough of the good stuff that’s probably out there?”

Not more trouble than it’s worth to get the information
Understands my question
Search interpreter (human or computer) speaks my language at my level
Doesn’t make me guess or learn its terminology
Doesn’t give me results that seem unrelated to my question

Not more trouble than it’s worth
Helps me make good choices
I can tell what each menu item means and how it differs from its neighbours
Asks me to clarify my intention (“did you mean… or …”)
Shows other useful search terms within good items I find
Offers useful ways to change and narrow my search
Let’s shop for shoes on LandsEnd …
And narrow our choices by
Women’s
Sandals
Leather etc.

Not more trouble than it’s worth
Most of all: I didn’t have to wade through too much JUNK
In the InfoSci world, this is called Precision :
Number of relevant items retrieved [divided by the] Total number of items retrieved
“ Out of all the stuff I got, how much of it was what I really wanted ?”
“ Info-Noise:” the biggest barrier to findability

Retrieval Effectiveness
High Recall + High Precision =
High Information Retrieval Effectiveness =
I found all of and only the info worth finding
But here are the Gotchas…

Gotchas…
# 1: You can’t have it both ways.
Recall and Precision are inversely related:
Better Recall = worse Precision, and vice versa
# 2: You don’t know what you’re missing.
In the real world, Recall is hard to assess;
you may never know what relevant information you didn’t find.

Gotchas…
# 3: Size matters.
The way you seek information changes depending on how much info you think you’re dealing with
# 4: You don’t know what you want...
Until you know what your choices are.
Finding out what’s available redefines your information need.
Good searching is iterative

What’s a searcher to do?
In the InfoGlut world, we care most about Precision
Make educated guesses about
what is in the collection
whether we’re missing something important
Refine our search strategies, until we
Get “enough” good results for this information need…and then we quit

What’s a content producer to do?
Make your content very smart about how it presents itself
So it is findable in the contexts where it is most useful, and
It doesn’t become more of the InfoNoise

Act II: What’s it all about? [email_address]

Tags
Smart content uses metadata tags, or database fields
Tags contain information such as the content’s
Creator
Date of creation
Format
Location or destination
Title
Subject area
Title might not be descriptive, e.g.
Metaphoric or idiomatic: “Your new bundle of joy”
Generated from filename: 2008030601.jpg

Subject area categorization
“ Aboutness”
Sometimes called “keywords,”
Often the most useful “findability” access point
Subjective: aboutness is in the eye of the beholder
Each item is usually “about” multiple subjects
What do you think this image is about?

What do the Corbis catalogers think it’s about?
About 20 keywords, for
“ Foreground” subjects
Fishing boat
Harbor
Ocean
Implied subjects
Marine scenes
Industry
Travel
Geographic location
Goose Cove
Newfoundland
Canada
Image composition attributes
Nobody
Reflection
“ Emotional” attributes
Serenity
Simplicity

Weighting
Are all these keywords equally important?
Which are most important?
Best practice: “weight” each “aboutness” tag
Items with “high aboutness” get ranked higher in search results for that keyword

Size (still) matters
If you had just one bookcase, you could organize it by colour like this
But imagine if a public library was like this Unshelved comic
As collection size increases, you need an increasingly complex system of subject categorization

Size (still) matters
The Problem of the World’s Biggest Bookstore (Amazon.com)
Subject headings converge from multiple sources
Publishers of all sizes
Library of Congress
Users’ tags
“ Education” is okay for some small publishers, but
Not usually specific enough for the Amazon Universe

What words should I use for “aboutness” keywords?
Prominent or frequent location
Featured in title, headings, description or summary
Appears frequently in the content
“ Foreground” or main subject of image
High semantic value
Nouns (“snow”)
Gerund verbs for activities (“skiing”)
Short modifier phrases (“cross-country skis”)

What words should I use for “aboutness” keywords?
Differentiates this content from other content
Users want this keyword
Appears frequently in user search logs
Often suggested by users
Similar content uses this keyword
Competitors’ websites
Published thesauri

Which keywords have I already used?
You need to be able to
Browse all keywords as alphabetical list
Use this list when tagging new content
Edit the keywords -- both in the list and in the content
Tagging consistently increases Precision

Big 3 Problems Inherent In Language
OK now I have a list. Am I done yet?
No. Uncontrolled lists like this, and folksonomies also, do not handle the…
Big 3 Problems Inherent In Language:
Equivalent relationships
Homonyms (look the same but aren’t)
Hierarchical relationships and other related concepts

In a book index, these are “See” references
Decide on a preferred form of the keyword, and lead the other forms to that
Word variations and synonyms are the most common

Word variations
Spelling variations
color = colour
Chanuka = Hanukkah
Word ending variations (word stemming)
Canad* = Canada, Canadian
Immigra* = immigrant, immigrate, immigration
Plural and tense variations
goose = geese
run = ran, running

Synonyms
baby = infant
purchasing = buying
pupil = student – but not if it’s Pupil (Eye)
Equivalency control increases Recall

Homonyms
Words that are spelled alike but have different meanings
Disambiguate by appending clarifier terms
Turkey (Bird), Turkey (Meat), or Turkey (Country)
Play (Dramatic work), Play (Imaginative activity), or Play (Sports activity)
Can also ask searcher to choose one (“Did you mean… or …”)
Homonym control increases Precision
Now you have a “Controlled Vocabulary”
But you’re still not done yet…

Hierarchies
Grouping related categories together, e.g.
Restaurant menus
Yellow Pages
File folders in hanging files in filing cabinets
Command menus in software applications
“ Browse” trees on e-commerce websites
Especially handy for browsing if you’re not sure how to describe (or spell) what you want

Hierarchies
Broader/narrower (parent/child) concepts
Instrumental music > Piano sonatas
Canada > British Columbia > Vancouver > Burnaby > Capitol Hill
Broader term should retrieve all its child terms
Birds = Sparrows + Penguins + Ostriches etc.
Or, if too many results, narrow search by selecting one or more child terms
Child term may have multiple parents (polyhierarchy)
In a book index, these are “main entries” and “subentries”
Controlled Vocabulary + Hierarchy = Taxonomy

Hierarchies
Related concepts (“cousins”)
Not (usually) broader/narrower
In a book index, these are “See also” s
Parenting See also Child development
School supplies See also Office supplies
Searching one keyword should suggest the other keyword but not automatically retrieve its results
Taxonomy + Related concepts = Thesaurus

Hierarchies
Scope note
Tells taggers and searchers where this concept stops and other concepts begin
Often differentiates related concepts
Medieval. Use for European history of the 5 th through 15 th centuries. For earlier periods, consider Classical antiquity. For later periods, consider Renaissance.
Office supplies. Use only for business contexts. For educational or home contexts, use School supplies.
Thesaurus + Scope notes + further rules for how to apply your keywords = Ontology

Act III: Too much of a good thing? [email_address]

The Perils of Polylingual Hierarchy, Or, Will It Play in Paris?
“Grandchild” terms may not always fit the “grandparent” categories
Hierarchies are different in other languages
Two real-life examples
Part I: “Not the Comfy Chair!”
Part II: The Event of Dessert

“Not the Comfy Chair!”
Furniture
Chairs
Dining chairs
Armchairs
But what about Dentist chairs? Electric chairs? Thrones? They’re chairs but not domestic Furniture

“Not the Comfy Chair!”
In European languages, there is no general “Chair”
Only approximately “Comfy chair” and “Uncomfy chair”
“ Comfy” may have arms, upholstery, be in living room
“ Uncomfy” may be armless, unupholstered, in kitchen
But not always…
So is this chair comfy?

The Event of Dessert
Wedding cake is not really a Dessert
Brownies may be Snacks
Ice cream cones are definitely Snacks
Civilised countries that have cake at Tea time prefer cheese or fruit for dessert

Foods
Dessert
Cake
Wedding cake?
Ice cream
Ice cream cones?
Brownies?
Cheese??
Quark???

Quark is a soft cheese
In Germany they eat Quark with Fruit for Dessert…
…but they also eat Quark on Toast for Breakfast .

Dessert moved from a Food to an Event
Marked by tableware (plate, fork)
The sweet Dessert foods moved to Foods > Sweets
Here we have Brownie as Snack…
… and here we have Brownie as Dessert
Dessert foods without tableware are Snacks
Snacks and Breakfast are also Events

How deep should I get?
How far down the hierarchy should go on making narrower terms?
In the InfoSci World, this is called Specificity
Keywords should be specific enough to
Make useful distinctions between items
Accurately describe each item
Not split hairs and make useless distinctions
Specificity increases Precision

How deep should I get?
The level of specificity you need depends on
Subject domain (type of information)
More technical domains -> more specificity
Collection size (size matters again!)
As your collection grows -> more specific keywords
Otherwise, too many items share a particular keyword
Precision plummets
Often a problem with folksonomies, which tend to use a lot of broad keywords

The more the merrier?
If a few keywords are good for findability, then a lot of keywords must be better, right?
In the InfoSci world, this is called Exhaustivity:
How many different keywords each item gets
Tip: when you’re stretching for keywords that aren’t much “about” this item, it’s time to stop plastering keywords
Otherwise, your Precision tanks
Again, often a problem with folksonomies

Will my keywords play nicely with others?
Good “aboutness” keywords from a consistent taxonomy can integrate your content very well
This is the biggest reason to invest in a taxonomy
You can use your “aboutness” tags to automatically:
Retrieve relevant images to accompany text
Suppress inappropriate images
Suggest appropriate products to accompany an article
Offer highly-relevant related topics
Sensitively combine professional- with user-generated content
And that’s not all…

Will my keywords play nicely with others?
You can use your “aboutness” tags to automatically:
Generate your website’s browse menus
Categorize search results into narrower categories
Increase users’ “personalization” experience, by offering them tags to
Identify their personal attributes in their website profiles
Subscribe to feeds of new content that will interest them
Suggest topics most relevant to them
Make your “tag cloud” retrieve more relevant results
Improve findability!

What happens when worlds collide?
In the Portable Content world, collections of different content often merge
Acquiring new collections
Selling your collection to others
Continuously incorporating new content from suppliers or users

Keywords from different content collections often don’t merge well, because the collections differ in:
Uncontrolled vs. controlled vocabularies
Preferred term to use
Synonym control
Word variation control
Hierarchy construction
Decisions about what belongs with what

Keywords from different content collections often don’t merge well, because the collections differ in:
Specificity levels
Collection size
Subject area domain
Exhaustivity
Tagging quality standards
And when your collection grows, your own keywords will need to get more specific

Accept others’ keywords in special metadata tags
Review these and “map” them to your keywords
Understand that your keywords will need to change
Publish your keywords to your suppliers
Expose your keywords to your users as suggestions for their own tagging

Standardize on a publicly-available thesaurus, e.g.
Library of Congress subject headings
National Library of Medicine’s subject headings (MeSH)
Getty Art & Architecture Thesaurus
If you can find one that matches your content and meets your users’ search needs
Your competitors won’t sell you theirs

Act IV: Can’t machines do this? [email_address]

If I can search the full text, why bother with keywords?
Full-text search generally can’t cope with
Synonyms
Homonyms
Trivial occurrences (low “aboutness”)
Inferences (high “aboutness” not explicit in the text), e.g.
Intended audience
Prerequisite knowledge
African rainforest animals (Which countries? Which animals?)
Non-textual content (images, audio, video, etc.)
Online Help indexes vs. full-text search
Indexes are selective
Like travel guide rather than phone book
In my usability tests, users got to the best answers faster via the index

Can computers automatically generate keywords and tag content?
Uses linguistic analysis mathematical rules to churn through text and try to
Comprehend all the “aboutnesses” and
Categorize the content
A huge, complex and difficult field of Information Science
Analysis rules are different for each subject area domain
There are no “magic” fits-all, out-of-the-box solutions
How do they do it? Here are a couple of methods

Term Frequency-Inverse Document Frequency (TF-IDF)
Term occurs frequently within a document = more “aboutness”, right?
But if that term occurs in lots of your documents, it’s not a good discriminator for finding the most relevant documents; e.g.
“ Software” may be
rare on a travel website, but
on nearly every page of a technical publisher’s website
TF-IDF is a statistical weighting method
Determines a term’s relative importance within a document and within the collection of documents
If term is frequent in doc AND rare in collection , then
High TF-IDF (high “aboutness” and good discriminator)
For each term, calculates numerical values (vector space)
Analyses compare these values to other documents’ values to retrieve similar documents

Term Frequency-Inverse Document Frequency (TF-IDF)
Some drawbacks of TF-IDF
Needs lots of sophisticated programming to be smart about your particular content collection and subject area domain
Dictionaries of stop words, word variants, phrases, synonyms, etc.
Can miss a low-occurring term that is crucial to this document, e.g.
Article on baby-proofing your home
Mentions “danger,” “injuries,” and “stairs” only once, but they are the main point of the article
Does not mention “safety” or “falling” – important concepts to capture
Does not mention “toddlers” or “parenting” – the implied subject and audience of the article
A human indexer would immediately catch these

Inductive Learning Algorithms
Humans teach computer what “aboutness” looks like
For each term, professional experts hand-tag a set of “good example” docs
Use mathematical linguistic analysis to compare new docs to “good example” docs for similarity and categorization
More expensive than straight TF-IDF, but
Better results
Good at capturing concrete concepts (“auto repair”)
Poor at capturing implied concepts (intended audience)

Does clickstream = “aboutness”?
Are pages “about” the same thing if users:
Click through them in the same sequence?
“ Customers who viewed this also viewed…”
Buy the same items on them?
“ Customers who bought this also bought…”
Write reviews of them?
Add them to a “recommended” list?
Tag them with the same tag?
Let’s look at Amazon again…
How relevant are each of this item’s “related” items?
How relevant are the professional and user tags?

Does clickstream = “aboutness”?
Info that users consciously connect is more likely to be related than passive clickstream trails
The more effort they put into categorization, the better the categorization is likely to be
But “Related to” ≠ “about” the same subject
Connections or tags very useful to one person ≠ tags useful to everyone; e.g. users’ personal tags like
“ Me” or “Home” or “Cynthia” (recipient of this gift)

Act V: What’s findability worth to you? [email_address]

Human effort is expensive
Effort “up front”: expense to producer
Creating and maintaining thesaurus
Tagging by trained staff
And/or designing (and redesigning) smart automatic categorization and retrieval systems
Better findability = happier customers

How much money are you willing to put into your content’s findability?
Not all content is equally worth the money
Professionally- vs. user-generated content
“ Personalized” content vs. general content
New content vs. old
“ Push” content you most want to sell
Be careful of “pushing” irrelevant content and
Losing your customers’ confidence

Effort “at the end”: expense to customers
Users do the tagging: e.g. folksonomies
Users slog through junk (low Precision)
Only works in contexts where users are motivated enough to volunteer their time and effort
Find your balance

Epilogue: Q & A
Questions? Comments?
Further questions or comments? Want a copy of this presentation? Email [email_address]
Thank you!

Content Findability in a Portable Content World

by Lise Kreps on Mar 27, 2008

Accessibility

Categories

Tags

Upload Details

Usage Rights

Statistics

Content Findability in a Portable Content World — Presentation Transcript