Taxonomy Fundamentals - SLA 2014

Taxonomy Fundamentals
Why build a taxonomy?
SLA – Vancouver – June 7, 2013
www.accessinn.com
www.dataharmony.com
505-998-0800
Marjorie M.K. Hlava
President and Chief Scientist
Bob Kasenchak
Project Coordinator
Access Innovations, Inc.
Copyright © 2013 Access Innovations, Inc.

A fast moving and powerful introduction to both the theoretical
and practical aspects of building a taxonomy, thesaurus, and
ontology. A well-built taxonomy is part of the foundation of the
information architecture underlying web sites, corporate
Intranets, search/retrieval, and access to relevant content in
databases. After defining controlled vocabularies and identifying
core standards, you will explore key concepts of taxonomy,
thesaurus, indexing, classification, and filtering. Discussion will
include the basics of a taxonomy records and fundamental term
relationships. Attendees will put concepts into practice through
multiple exercises, taxonomy, indexing, and related software
tools will be demonstrated.
Introduction To
Taxonomy Concepts

About Access Innovations
Access Innovations are experts in content creation, enrichment, and
conversion services. We provide services to semantically enrich and tag raw
text into highly structured data. We deliver clean, well-formed, metadata-
enriched content so our clients can reuse, repurpose, store, and find their
knowledge assets. We go beyond the standards to build taxonomies and
other data control structures as a solid foundation for your information.
Our services and software allow organizations to use and present their
information to both internal and external constituents by leveraging search,
presentation, and e-commerce. We change search to found!
Quick Facts
• Founded in 1978
• Headquartered in Albuquerque, NM
• Privately held
• Delivered more than 2000 engagements

What we do
 Access Innovations
 Ensure clean, well formed content
 Create Knowledge Organization Systems (KOS)
 Data Harmony Tools
 To automatically index content
 To manage KOS and more
 To semantically enrich the content
 To organize the content
 Visualization tools to portray the data
4

Outline of the Day
 Why the excitement
 What is a Taxonomy
 Card Sort – Slide 39
 How to build a taxonomy
 Term relationships
 Thesaurus Examples
 Pre and Post
Coordination
 What are we controlling
 Vocabulary Options
 TaxoMatch - Slide 189
 Term Forms
 Facets / Notation / Roles /
Treatment/ Weighting
 Auto Indexing
 A Taxing Situation - Slide
315
 Search
 Where do I use it?
 Standards and references

Why The Excitement?
 Makes information findable!
 Cut search time by 50%! (The Weather Channel)
 Leverages information in new ways
 User satisfaction
 Organizes topical areas and web sites
 Provides better online help
 Customer support 30x more costly than web self-
service*
*(Forrester Research "Tier Zero Customer Support" 1999)

Taxonomies are found…
• In “indexing”, tagging, categorizing, subject metadata
• In search - precision, recall
• In content management systems, web sites
• In SharePoint to replace term tree, tag uploads
• In mashups, repackaging, repurposing data
• In social networking sites
• In author tagging - peer reviewer selection
• In filtering data – e.g., spam filters and RSS feeds
• In web crawlers
• In text analytics – trend analysis
• … and much more
Because taxonomies make them work

Where Does
Implementation Happen?
 At the backend
 When the records / articles are added to
the production system
 When the search software’s “inverted file”
is created
 When the HTML for the web page is
created

Heart Of The “Big Data”
Production Process

From the
production side
to the website
display, carry
the taxonomy
descriptors for
use in precision
search

Taxonomy

Authors at a place
MASHUP locations to a
GPS grid of an area
Two data points
GPS Coordinates
Taxonomy description of the place

Watch Crime In Action

Two data points
GPS Coordinates
Taxonomy description of the crime

17
Visualization Strategies
Matrix
Visualization
Software

All Data Up-posted
To The Top Level

Pattern Analysis
Indexing Clusters

Pattern Analysis
Domain Associations

Pattern Analysis
Domain Correlations

Pattern Analysis
Gap Analyses

Pattern Analysis
Component Gaps

More Like This - Recommender
Cancer Epidemiology Biomarkers & Prevention
Vol. 12, 161-164,
February 2003
© 2003 American Association for Cancer Research
Short Communications
Alcohol, Folate, Methionine, and Risk of Incident Breast Cancer in the
American Cancer Society Cancer Prevention Study II Nutrition Cohort
Heather Spencer Feigelson1, Carolyn R. Jonas, Andreas S. Robertson,
Marjorie L. McCullough, Michael J. Thun and Eugenia E. Calle Department
of Epidemiology and Surveillance Research, American Cancer Society,
National Home Office, Atlanta, Georgia 30329-4251
Recent studies suggest that the increased risk of breast cancer associated
with alcohol consumption may be reduced by adequate folate intake. We
examined this question among 66,561 postmenopausal women in the
American Cancer Society Cancer Prevention Study II Nutrition Cohort.
Related Press Releases
•How What and How Much We Eat (And Drink) Affects Our
Risk of Cancer
•Novel COX-2 Combination Treatment May Reduce Colon
Cancer Risk Combination Regimen of COX-2 Inhibitor and
Fish Oil Causes Cell Death
•COX-2 Levels Are Elevated in Smokers
Related AACR Workshops and Conferences
•Frontiers in Cancer Prevention Research
•Continuing Medical Education (CME)
•Molecular Targets and Cancer Therapeutics
Related Meeting Abstracts
•Association between dietary folate intake, alcohol intake, and
methylenetetrahydrofolate reductase C677T and A1298C
polymorphisms and subsequent breast
•Folate, folate cofactor, and alcohol intakes and risk for
colorectal adenoma
•Dietary folate intake and risk of prostate cancer in a large
prospective cohort study
Related Working Groups
•Finance
•Charter
•Molecular Epidemiology
Related Education Book Content
Oral Contraceptives, Postmenopausal Hormones,
and Breast Cancer
Physical Activity and Cancer
Hormonal Interventions: From Adjuvant Therapy to
Breast Cancer Prevention
Related Awards
•AACR-GlaxoSmithKline Clinical Cancer Research Scholar Awards
•ACS Award
•Weinstein Distinguished Lecture
Webcasts
Related Webcasts
Think Tank Report
Related Think Tank Report
Content

Link to Society Resources
Journal
Article on
Topic A
Other
Journal
Articles on
Topic A
Upcoming
Conference
on Topic A
Podcast Interview
with Researcher
Working on Topic A
Grant Available
for Researchers
Working on
Topic A
CME
Activity on
Topic A
Job Posting
for Expert
on Topic A

Author Connections

What is a taxonomy?
Albuquerque, NM 87110
www.accessinn.com
www.dataharmony.com
505-998-0800
Marjorie M.K. Hlava

Vocabulary Control - Options
 Classification
systems*
 Authority files
 Controlled term lists
 Uncontrolled term
lists
 Thesauri
[*We will concentrate on taxonomies and
thesauri, first, and then cover the others as
time permits.]

Taxonomy Standards
 Z39.19 (2005) Controlled Vocabularies
 BS 8723 Parts 1 – 5
 ISO25964 Parts 1 - 4
 TAG 37 and 46 standards
 SKOS - Simple Knowledge Organization
System
 OWL - Web Ontology Language
 AND more!

A Taxonomy is a
Knowledge Organization System (KOS)
 Uncontrolled list
 Name authority file
 Synonym set/ring
 Controlled vocabulary
 Taxonomy
 Thesaurus
 Ontology
 Semantic network
Not complex
Highly complex

Structure Of
Controlled Vocabularies
Lists Synonyms Taxonomy Thesaurus Ontology
Ambiguity Ambiguity Ambiguity Specifies a KOS
Synonym Synonym Additional kinds of
Hierarchy Hierarchy Relationships
Relationships
relationships
INCREASING COMPLEXITY and CONTROL

What is a Taxonomy?
ANSI/NISO Z39.19-2005
“A collection of controlled vocabulary terms
organized into a
hierarchical structure.”
controlled
Missing:
equivalence, homographic, and associative relationships
and notes
Yes!

Taxonomy? Thesaurus?
 Often used interchangeably
 Thesaurus is a taxonomy with extras
 Related Terms
 Non-preferred Terms (USE/Used for)
 Scope Notes
 More
 Taxonomies often have the actual
information object at the final node.
 CMS and SharePoint tend to the
hierarchical view only, definition, and USE

Taxonomy? Thesaurus?
 Main Term (MT)
 Top Term (TT)
 Broader Terms (BT)
 Narrower Terms (NT)
 Related Terms (RT)
 See also (SA)
 Non-Preferred Term (NP)
 Used for (UF), See (S)
 Scope Note (SN)
 History (H)
= subject term, heading, node,
category, descriptor, class
TAXONOMY
THESAURUS
OWL can specify

The Semantic Roadmap:
Knowledge Organization Systems
 Semantic network
 Ontology
 Thesaurus
 Taxonomy
 Controlled vocabulary
 Synonym set/ring
 Name authority file
 Uncontrolled list
•Unrelated Entities
•Ambiguity
•Linked Entities
•Contextual Specificity
•Simple
•Low Value
•Complex
•High value
Uncontrolled
list has the
Highest Cost
over Time!

Copyright © 2005 - Access Innovations, Inc.
Taxonomy
view
Thesaurus
Term Record
view

Taxonomy 101
How do you build a taxonomy?
Albuquerque, NM 87110
www.accessinn.com
www.dataharmony.com
505-998-0800
Marjorie M.K. Hlava

How Do You Build a
Taxonomy ?
• Define subject field
• Collect terms
• Organize terms
• Fill in gaps
• Flesh out and interrelate terms
• Apply to your data
You’re done!

Foundations
 Start with what is known
 Build from there
 Use the literature, your data
 Use the lists you already have internally
 Built-in continuous review throughout the
process, and beyond
 Who is involved?
 Taxonomists
 Subject matter experts (SME)
 Project management
 Users

Define Subject Field
 Review representative collection of content
 Determine:
 Core areas
 Peripheral topics
Psychology
Education
Sociology
Law
 Scope can be modified later

Where Do I Get the
Terms?
 Your documents and databases
 Departmental terminology
 Text books and their indexes
 Book tables of contents and indexes
 Journal quarterly indexes
 Encyclopedias
 Lexicons, glossaries on the topic
 Web resources
 Users and experts
 Search logs

How Do You Choose
Terms?
 Importance in the subject area
 Use in the literature, by the organization
or community
 Necessary degree of specificity or detail
 Relationship with other controlled
vocabularies
 Single concept = single term

Build, Buy, Augment?
 Survey existing thesaurus/taxonomy resources for your
domain
 Test for
• Scope
• Depth
• Make-or-break terms
• Cost
 Adoption of existing taxonomies
 Term registries
 Taxobank
 Taxonomy Warehouse
 Other resources
Don’t reinvent the wheel!

Gather Terms From
Search Logs
 Top ~100 search terms from search logs
 Terms used more than 50 times
 Match to web site with appropriate
answer
 Basis for favorites or best bets, presented
at the top of results list
 Behavior-based taxonomy

Vocabulary Control –
How?
 Use unambiguous terms, clear to the user
group
 Distinguish between terms that appear
similar
 Use Scope Notes when necessary
 Use terms as elements that can be
coordinated in a flexible manner
 Create compound terms, if necessary

Term Format
 KISS – Keep it short and simple
• 1-2-3 words
• Effect on search
• Pre and Post Coordination
 Establish a policy
• follow Chicago Manual of Style
 Grammatical issues
• Nouns and noun phrases
• Verbs  Gerunds
• Adjectives - no
• Adverbs - no
• Initial articles – no

Thesaurus - Format
 Main Entries
 Top Terms - TT
 Broader Terms - BT
 Narrower Terms - NT
 Related Terms - RT
 Scope Notes - SN
 History - HI
 Date term added/changed - DA

Thesaurus - Format
 Related terms - RT
 See - S
 See also - SA
 Use - U
 Preferred Term PT
 Use for - UF
 Non Preferred Term NP
 ..

Definitions
 Index term
 the representation of a concept
 Preferred term (International)
 a term used consistently to index a concept
 descriptor (USE)
 what the “USED FOR” reference points to

Definitions
 Non preferred term (International)
 synonym or quasi synonym of a preferred term
 non-descriptor (USE)
 the “USE” reference
 the “SEE” reference
 Related term
 the “SEE ALSO”

Indexing Terms
 Three main categories
 concrete entities
 abstract concepts
 proper nouns

One Term / One Concept
 Importance in the subject area
 Use in the literature, by the organization
or community
 Necessary degree of specificity or detail
 Relationship with other controlled
vocabularies

One Term / One Concept
 Terms represent simple or unitary concept
 A unit of thought
 Can be a single-word term
 Can be a multiword term, if required to
represent the concept
 Three main categories
– Concrete entities
– Abstract concepts
– Proper nouns
“A unit of thought, formed by
mentally combining some or all of
the characteristics of a concrete or
abstract, real or imaginary object.
Concepts exist in the mind as
abstract entities independent of
terms used to express them.”

Concrete Entities
 Things and their physical parts
 primates
 head
 buildings
 floors
 islands

Concrete Entities as Terms
• Things and their physical parts
– Birds
• Feathers
• Buildings
• Floors
• Materials
– Cement
– Wood
– Lead
– Cards and Chips

Concrete Entities
 Materials
 cement
 wood
 lead
 cars
 refrigerators

Abstract Concepts
 Actions and events
 evolution
 respiration
 skating
 management
 wars
 ceremonies

Abstract Concepts
 Abstract entities, properties of things,
materials and actions
 law
 theory
 strength
 efficiency
 lead (management)

Abstract Concepts
 Disciplines and sciences
 physics
 meteorology
 mathematics
 psychology

Abstract Concepts
 Units of measurement
 kilograms
 pounds
 meters
 miles

Abstract Concepts as Terms
• Actions and events
– evolution, skating, management, ceremonies
• Abstract entities
– law, theory
• Properties of things, materials, and
actions
– strength, efficiency
• Disciplines and sciences
– physics, meteorology, mathematics
• Units of measurement
– pounds, kilograms, miles, meters, nanoseconds

Proper Nouns*
 Individual entities, or “classes of one”,
expressed as proper nouns
 San Francisco
 United States of America
 Lake Michigan
* Proper names – of persons – are not
included

Proper Nouns as Terms
 Individual entities – “classes of one” –
expressed as proper nouns
 San Francisco, Lake Michigan
Thesaurus standards exclude proper names,
persons, and trade names  authority files.
Taxonomies include them as final nodes.

Most Terms Are Nouns
 Nouns or simple noun phrases
 Adj + Noun – Art history (ANSI/NISO standard)
 Noun + Prep + Noun – History of art (ISO standard)
 Exceptions – Burden of proof, Coats of arms,
Prisoners of war, Birds of prey, etc.

About “and”
 Avoid “and” in terms – not a single concept
Instead of: Children and television
Factor and postcoordinate
USE Media influence + Television + Children
“And” is not in the standard
In real life—need for granularity may dictate your choice

Compound Terms – Nope!
 “Terms in a thesaurus should represent
simple or unitary concepts…” (ISO standard)
 “Compound terms should be factored
(split) into simple elements…” (ANSI/NISO
standard)
 Term phrases are okay (bigrams)
 Adjective Noun
 American history
 Two concepts combined are not
 Aromatherapy for bloating

Organize Terms –
Roughly
 Sort terms into several major categories –
logical groups of similar concepts as Top
Terms
 Identify core areas and peripheral topics
 10 – 20 to start
 Consider moving proper names to authority files
 Result: loose collection of terms under
several main headings
 Rough and tentative – see how it fits as you go
 Initial gap analysis
 Add / modify / delete as needed

How Do Terms Relate?
 Hierarchical relationships
-- Parents and their children
 Equivalence relationships
-- Aliases
 Associative relationships
-- Cousins
TAXONOMY
THESAURUS

Hierarchical
Relationships
 Broader Term (BT) represents the class,
whole, or genus
 Narrower Term (BT) is a member, part, or
species
 Generic relationship
 Whole-part relationship
 Instance relationship
 NT inherit all the BT characteristics
 BTs/NTs have a reciprocal relationship

Hierarchical
Relationships
 Class as a whole
 superordination
 broader term (BT)
 sometimes top term (TT)
 Members or parts of the class
 subordination
 narrower term (NT)
 Reciprocal

Hierarchical
Relationships
 BT/NT based on being part of same class
 Same fundamental category
 entities
 activities
 agents
 properties

Hierarchical
Relationships
 Museums
 Archaeological museum type of entity NT
 Ethnological museum type of entity NT
 Curators agents RT
 Museum techniques action RT
 Scientific museum type of entity NT

Hierarchy –
Whole-Part Relationships
 Four general types
1. Body systems and organs
 Ear  Middle ear
2. Geographical locations
 Bernalillo County  Albuquerque
3. Fields of study
 Geology  Physical geology
4. Hierarchical social structures
 Ontario  Manitoulin District

Hierarchy –
Instance Relationships
 General category (common noun) as BT,
with individual example (proper noun) as
Narrower Term Instance (NTI)
Seas French cathedrals
Baltic Sea Chartres Cathedral
Caspian Sea Rheims Cathedral
Mediterranean Sea Rouen Cathedral
Essentially identical to “final node” in taxonomies

Hierarchical Types
of Display
 Systematic
 Alphabetic
 other, but less common views

80
DTIC
 Hierarchy

Polyhierarchical
Relationship
• Term can logically fit under more than one
Broader Term – can have Multiple Broader
Terms (MBT)
• Part of ISO standards, new to ANSI/NISO
Nurses Health administrators
Nurse administrators Nurse administrators
Finance Careers
Accounting Accounting

Polyhierarchical
Relationships
 Great for the web click environment
 Terms occur in multiple categories
 Can be generic as well as hierarchical
Engineering Physics
NT Nanotechnology NT Nanotechnology
Nanotechnology
BT Engineering
BT Physics

83
DTIC
 Alpha

Pests
Generic Relationship Tests
Squirrels
Rodents
 ALL squirrels are rodents
x NOT ALL squirrels are pests
x NOT ALL pests are rodents

Generic Relationship
Tests
• Both terms in same fundamental category
• “All-and-some” test
SOME ALL
SOME NOT ALL
Rodents
Squirrels
Pests
Squirrels
Consider concepts of marketing and advertising

Generic Relationships
 “Identifies the link between a class or
category and its members or species.”
 Easy in biology
 Rodents
 NT Squirrels
 All and some rule

All and Some Rule
 Rodents
 NT Squirrels
 RT Pests
 Q. Is this an example of polyhierarchy?
 Q. Do you need to make RT relationships
for “Pests” to all of the NTs under
“Rodents”?

 Seas ISO
 NT Baltic Sea
 NT Caspian Sea
 NT Mediterranean Sea
 French Cathedrals NISO / ANSI
 NTI Chartres Cathedral
 NTI Rheims Cathedral
 NTI Rouen Cathedral
 RT Gothic cathedrals

 French Cathedrals NISO / ANSI
 RT Gothic cathedrals
 French Gothic Cathedral
 BT Gothic cathedrals
 Q. Why/how do these differ?

90
CABI Pages

 “…a general category of things and
events expressed by a common
noun, and an individual instance of
that category, the instance then
forming a class of one which is
represented by a proper name.”
 A way of adding the proper names
and items from the Authority files to
the thesaurus

Questions before moving on to
Associative Relationships?

Associative Relationships
 Related Terms (RTs) – cousins
 “…terms related conceptually, but not
hierarchically, and are not part of an equivalence
set” (i.e. not synonyms)
 Both terms are valid thesaurus terms for indexing
and have reciprocal relationship
 Expands user’s awareness and reflects thesaurus
coverage of unanticipated areas
 Standards describe specific types

Associated Relationships
Related terms
Physicians Medicine
(“Reciprocal posting” done
automatically is highly desirable.)

 Sibling relationships
 Examples:
 Brother : Sister
 Desk : Chair
 Easier to create within well defined
facets (e.g. AAT)
 Usual step in building process
 Can be identified automatically

 RT relationships
 Braking systems
 RT Trains
 RT Bicycle
 RT Motor vehicle
 Office furniture
 RT Office buildings
 RT Ergonomics

 Field of study and objects studied
 Seismology
 RT Earthquakes
 Meteorology
 RT Weather patterns

 Operation or process and the agent or
instrument
 Hairdressing
 RT Hair dryers
 Word processing
 RT Typing skills

 Occupation and person in occupation
 Social work
 RT Social workers
 Information science
 RT Special librarians

 Action and the product of the action
 Publishing
 RT Music scores
 Landscaping
 RT Lawn mowers
 RT Irrigation systems

 Action and its patient
 Teaching
 RT Students
 Conducting
 RT Musicians

 Concepts related to their properties
 Women
 RT Femininity
 Automobiles
 RT Automotive safety

 Concepts related to their origins
 Water
 RT Water wells
 Carpet
 RT Thread

 Concepts linked by causal dependence
 Injuries
 RT Accidents
 Cultural stress
 RT Culture shock

 Action and counter action
 Pests
 RT Pesticides
 Log on
 RT Log off

 Raw material and its product
 Hides
 RT Leather
 Clothing
 RT Fabric

 Action and associated property
 Precision instrument
 RT Accuracy
 Production processes
 RT Quality control

 Concept and its opposite
 Single People
 RT Married people
 Height
 RT Depth
 RT Weight
 If not hierarchical, probably
associative

Questions before moving on to
Equivalence Relationships?

Equivalence Relationships
 Refer to the same concept
 (Use for)
 Prefix for non-preferred terms
 (Use)
 Prefix for preferred terms
 Automobiles
 used for Cars
 Cars
 use Automobiles

Use
Use for
Physicians
Doctors

 Synonyms
 popular and scientific
 spiders - arachnida
 scientific and trade names
 Motrin (TM) - ibuprofen
 standard names and slang
 hi fi - high fidelity
 different linguistic origin
 home care - domicillary care

 Synonyms cont’d
 different cultures
 aerials - antenna
 trunk - boot
 hire - rent
 emerging concepts
 telecommuting - distance working
 outdated
 refrigerators - iceboxes

A “Term” Synonym Ring
Term
Node
Subject headingCategory
Descriptor

 Lexical variants
 variant spellings
 Muslim - Moslem
 center - centre
 direct and indirect forms
 electric power plants
 power plants, electric
 abbreviations
 ECG - electrocardiograph

 Quasi synonyms
 urban areas - cities
 gifted people - geniuses
 Antonyms
 height - depth
 literacy - illiteracy

 Up posting (generic posting)
 useful for web interfaces
 NT equivalent to their BT
 not sub species of BT

PsychInfo Rotated

 Factored terms
 express terms in their combinations
 Milk hygiene
 use milk and hygiene

Equivalence Relationship
• Preferred Term
– Thesaurus term and valid for indexing
– Thesaurus notation: USE
• Non-Preferred Term
– Not valid for indexing
– An alias or imposter
– Entry point, directs user to Preferred Term
– Thesaurus notation: UF or NPT
Spiders Plant pathology
UF Arachnids USE Phytopathology

Equivalence – When to Use
 Synonyms, slang, quasi-synonyms
 Scientific and trade names
 Ibubrofen UF Motrin™
 Lexical variants
 Fiber optics UF Fibre optics
 Mouse UF Mice
 Upward posting of narrow concepts not specified
in taxonomy or thesaurus
 Social class UF Elite, Middle class, Working class
Get equivalent terms from search logs, brainstorming…

Scope Notes (SN)
 Indicate meaning of the term in the context of
this thesaurus, for this audience
 Stress – Metal, Psychological, Physiological
 Indicate any restriction in meaning
 Indicate range of topics covered
 Provide direction for indexers; for terms often
confused, may suggest an alternative term
 Use only as needed – not for every term
 Establish and stick with consistent format
 Be concise

Scope Notes (SN)
 Restrictions on meaning
 Range of topics covered
 Instructions to indexers
 Term histories
 Reciprocal scope notes

Questions before moving on
to more thesaurus examples?

Thesaurus - Examples
 Roget's 1852
 synonyms
 COSATI - 1964
 concept linking
 NASA
 AEC - ERDA - DOE - ESA
 National Library of Medicine
 outline of a field
 Medical Subject Headings - MeSH

126
NASA
 Alphabetic

127
NASA
 Hierarchical

Thesaurus - Examples
 INSPEC - multifaceted
 Thesaurus
 Classification system
 Free text terms
 Variant spellings
 NICEM
 27 Top Terms

129
INSPEC

130
INSPEC
 Hierarchy

Merged Vocabularies
 Yahoo!
 Subject headings
 Authority files
 In a single list

132

133
Yahoo!
 Hierarchy

Merged Vocabularies -
continued
 Office.com
 Multiple broader terms
 Concept mapping

Eurovoc
Thesaurus
Pages

137
Eurovoc Thesaurus Hierarchy

138
Eurovoc Terms

So far you’ve got…
 Hierarchy
– Broader and Narrower Terms
• Polyhierarchies when needed
– Preferred/Non-Preferred Terms
– Equivalence relationships
– Related Terms
– Associative relationships
– Scope Notes
– Complete term records
– Correct term format

 Hierarchical relationships
-- Parents and their children
 Equivalence relationships
-- Aliases
 Associative relationships
-- Cousins
-- See Also’s
TAXONOMY
THESAURUS

• Term format
• Grammatical issues
• Singular and plural forms
• Spelling
• Abbreviations and acronyms
• Capitalization
• Other punctuation
• Consistency

Pre and Post Coordinate
Terms
 Pre coordinates – two concepts
 Subject headings – Library of Congress
 American history – Civil War
 Back of the book
 Put together in advance by the publisher
 Post Coordinate
 Taxonomy terms
 Single concept
 Put together by the user / searcher

Pre-coordination
 Card catalogs - printed indexes
 Links and roles defined
 Controlled vocabularies
 High input costs
 Precise recall - easier searching

Post-coordination
 Starting with punch cards
 Machine readable
 Frequently natural language
 Currency and specificity
 Exhaustive coverage - loss of precision
 Low input costs
 False drops

 Work first from the literature
 Establish literary warrant for terms
 Some one else do the clerical work
 Differentiate the lexicography work
 From the Subject Matter expert work
 Let SMEs do the review and tailoring
 Expert review ensures the proper term use and
application
 Advisory Board…advisable!
Subject Matter Experts (SME)

Again, why do we index?
 Improve precision
 define scope of terms
 Improve recall
 different terms for same concept
 Guide to a field of expertise
 Learning tool
 Richer expression

Uses?
 Indexing
 …process by which subject terms or
classification symbols are assigned to concepts in
documents
 A thesaurus is also known as an indexing
language
 M.A.I.™ is an automated indexing system

What are We Controlling?
 Synonyms
 different terms same concept
 Polysemes or Homonyms
 same word different meanings
 lead or mercury

How?
 Meaning
 delineation of scope of a term
 Term equivalence
 linking of synonyms
 Disambiguation of homonyms
 lead (metal)
 lead (element)
 lead (management)

Disambiguation
Bridge Structure
Bridge Dentistry
Bridge Game
Bridge Concept

Disambiguation
 Restriction and clarification of meaning
 Cells
 biological microsystems
 electrical equipment
 prison housing
 Reading
 town in England
 communication process

Disambiguation
Bill Invoice
Bill Legislative
Bill Sport
Bill Person

Disambiguation: Pre-Coordinate vs.
Post-Coordinate Forms
 Cells (biology)
 Cells (electric)
 Cells (prison)
 Reading (place)
 Reading (process)
 Biological cells
 Electric cells

Precision Options
 Language specificity
 Coordination
 Compound terms - level of
precoordination
 Homographs and scope notes
 Word distance indication

Precision Options
 Structural relationships
 Links and roles
 Treatment and aspect codes
 Weighting

Maintenance of a
Controlled Vocabulary
 Allow for new jargon to be added
 Any living field will have new terms
 Identifier field
 Candidate terms
 Consider multiple broader terms

Review, edit, test, edit,
use, edit, and maintain, i.e., edit
 Review
 Users
 Expert reviewers
 Test
 Index 500+ documents
(more for variable writing
style; fewer for strict
style)
 Monitor search log
 Edit and maintain
 Add term
 Change existing term
 Change term status
 Delete term
 Add term relationship
 Delete term relationship
 Add/modify Scope Note
 Change overall structure
Consider automated / assisted indexing software

When Do You Add
More Terms?
 On demand
 When usage changes
 Stewardess – flight attendant
 As the field evolves
 8 changes to 64 colors
 In Use
 Don’t freeze waiting for perfection

 Classification
systems
 Authority files
 Controlled term lists
lists
 Thesauri

Classification Systems - Defined
 Are used to put an object in a specific
place. In the traditional classification
system each item has a single spot to go.
 Follow an outline of knowledge
 Used to shelve books in a library

Catalog Systems - Defined
 Used to catalog the object to identify its
contents
 Based on perception
 Multiple terms are used to identify a
single object
 Not natural language
 Pre-coordinated - subheadings

Classification Systems - Examples
 Classification of actual collections
 New York State Library - Dewey
 810.01
 Cutter - Universities 1800 - 1960’s
 Z34
 Lan
 Thomas Jefferson - Library of Congress
 z34.18
 la
 Government Documents Numbers
 based on government structure

Catalog Systems - Examples
 Library of Congress Subject Headings
 Sears Subject Headings
 (used with Dewey)

King of Catalogers
 Charles Ammi Cutter
 rules for alphabetical subject indexing
 most specific heading
 put two topics under two headings
 use English if possible
 x ref antonyms
 careful with homographs
 1895 ALA Subject Headings following
Cutter

Politics in Libraries
 In 1905 Dewey was president of ALA
(American Library Association)
 LC adopted DDC
 Threw out Cutter
 The two never spoke again.

Types of Headings
 Single word
 Botany or Ethics
 Adjective noun
 Capital punishment
 Noun - noun
 Death penalty
 American Standard
 Noun preposition noun
 Penalty of death
 International Standard
 Noun conjunction noun
 Nurses and nursing

Cutter Guidelines
 File under the phrase “as it reads”
 Use the most significant words
 Reduce adjective nouns to noun
phrases
 Use singular rather than plural
 File compound words under the first
word
 No subheadings

Cross References
 Cross reference synonyms
 main heading should be what the class uses
 use the common term
 use the unambiguous heading
 prefer the one which brings relations
 “…with a well defined network of cross
references the mob becomes an army.. “
 C.A. Cutter

Library of Congress (LC)
Subject Headings
 1911 - List of Subject Headings
 extensive use of sub-headings
 invert phrases for main subject
 file under the noun not the adjective
 see references not cross filing
 place holder terms
 homographs defined parenthetically

Classification vs.
Subject Headings
 Classification
 single spot or placement
 browse physical list
 often a numbering system
 clear hierarchy
 no or few cross references
 Like Yahoo!

Classification vs.
Subject Headings
 Subject headings
 generic search
 hidden classification system
 related terms and cross references in heavy use
 usually the inverted form
 cells, electric

Authority Systems - Defined
 Frequently have cross references
 Widely available
 Frequently coded lists
 Brand names
 .. Lists of terms in the preferred format for
use.

Authority Files - Defined
 People
 Places
 Things
 ……..NOT
 Concepts
 Methods
 Processes

Authority Files - Examples
 ISO Country Name and Code
 International Standards Organization
 ISO Language list
 NAICS (SIC)
 Standard Industrial Classification Code (SIC)
Replaced by
 North American Industrial Classification
System (NAICS)

Authority Lists - Format
 Belgian Congo
 use Congo
 Bill Gates
 use William F. Gates, III (computer scientist)
 see also
 William Gates (basketball player)

Authority Lists -
Need Style Sheets
 Names
 AACR2
 Anglo American Cataloging Rules
 AAP
 American Association of Publishers
 Chicago Manual of Style
 Dun & Bradstreet
 Style Sheet

 Classification
systems
 Authority files
 Controlled term
lists
lists
 Thesauri

Controlled Term Lists -
Defined
 State the preferred terms
 Provide allowed term entry
 Heavily cross referenced
 Not generally hierarchical
 Popular
 Easy to create

Controlled Term Lists -
Examples
 ABI/Inform
 Predicasts
 RDS - Responsive Data Services
 Back of book indexes
 Art and Architecture Thesaurus
 …....These are not FULL thesauri

Controlled Term List -
Format
 Cars
 use Automobiles
 Personal Computer
 use Microcomputer

Uncontrolled List - Define
 Add terms as they occur
 No cross reference
 Simple flat structure

Uncontrolled List - Example
 List of names
 Grocery list
 Candidate term list

Uncontrolled List - Format
 Laundry
 Trim bushes
 Cat box needs cleaning
 Tommy’s birthday (bake cake)
 Iron
 Water plants
 ….other natural language lists

Trying to Impose Control...
 Do laundry
 Trim bushes
 Clean cat box
 Bake birthday cake
 Iron shirts
 Water plants

 Designed to enhance understanding and retention of the
vocabulary concepts necessary for creating a taxonomy,
ontology, thesaurus, or controlled vocabulary.
 Game supplies:
 1 Deck of Orange Question and Challenge Cards
 1 Deck of Green Answer Cards
 Game setup:
 Shuffle the deck of Green Answer cards,
 Deal the entire deck to the players.
 Shuffle the deck of Orange Question and Challenge cards
 Place them facedown in a pile in the middle of the table so that all
players can reach the pile.
 Reinforce what you just heard!
 Have fun!

1. Play moves to the left of the dealer
2. Draw a card from the top of the Orange cards.
Read it aloud to all of the players.
3. The player who read the card says out loud
what they think the answer is.
4. Each player looks at the Green Answer cards
in their hand.
1. If they have the correct answer to the
Question or Challenge, they show their
card to everyone at the table.
2. If everyone agrees that the answer is
correct, the player holding the correct
answer card gives it to the player who
read the Question or Challenge card.
5. The player places their associated pair of
cards – one Orange Question and Challenge
card and one Green Answer card – face up on
the table in front of them.
6. Play passes to the person who held the correct
Green Answer card in their hand. Play
continues as in step 2 above.
7. Discussion among the players to arrive at the
correct answer is permissible and encouraged!
8. If players do not arrive at a consensus
regarding the correct answer, the Orange
Question and Challenge card may be returned
to the bottom of the pile, and play passes to
the person to the left of the player who drew
the previous card.
9. When all of the Orange Question and
Challenge cards have been drawn, read aloud,
and matched with their Green Answer cards,
the game ends.
10. If there are any Orange Question and
Challenge cards remaining to which players
cannot agree on an answer, players may
consult their notes or ask the session speaker.

Term Forms
 Nouns
 Prepositional forms
 Adjectives
 Adverbs
 Initial Articles
 Singular and plural

Term Forms - Noun and
Noun Phrases
 Nouns and noun phrases
 print media
 carpet

Term Forms - Prepositional
Forms
 Prepositional forms are seldom used
 okay in International Standard ISO
 Philosophy of Education
 ANSI / NISO
 Educational philosophy

Term Forms – Adjectives
 Adjectives
 not used in isolation
 may be used for coordination
 Miniature paintings
 USE PAINTINGS AND MINIATURE
 Portable typewriters
 USE TYPEWRITERS AND PORTABLE

Term Forms – Adjectives
 Adjectives
 may convert to noun forms
 MINIATURE SIZE
 PORTABLE DEVICES
 TRIANGULAR SHAPE

Term Forms - Adverbs
 Adverbs
 not used unless part of a compound term
 VERY LARGE ARRAY RADIO TELESCOPE
 Used for VLA

Term Forms - Verbs
 Verbs
 no infinitive or participle forms
 for actions that can be expressed as nouns and retain
clear meaning, use noun form or gerunds
 Examples
 Speaking (not Speech)
 Walking (not Ambulation)
 Communication (not Communicate)
 Administration (not Administer)

Term Forms - Initial Articles
 AVOID THEM
 Example
 Theater not The theater
 State (political entity) not The state
 Use if part of a proper name
 Le Mans
 El Salvador

Term Forms - Singular and
Plural
 Concrete entities
 count nouns are plurals - how many?
 planets
 children
 non count nouns - how much?
 nickel
 snow
 lace

Term Forms - Singular
and Plural
 fully formed organism
 eyes
 mouth
 objects are singular
 lamp
 classes of things
 fruits

Plural
 Abstract concepts
 Show in the singular form
 authority
 socialism
 packaging
 biochemistry

Plural
 Unique entities
 Show in the singular
 Big Ben
 Grand Canyon

Other Formatting
 Spelling
 Punctuation
 Capitalization
 Abbreviations
 ...

Spelling
 Use what the users will use and cross
post for multilingual
 fiber - fibre
 center - centre
 organization - organisation
 hemo - haemo
 Pediatrics - paediatrics

Punctuation
 Parentheses only for qualifiers
 Apostrophes are retained
 Hyphens - avoid
 avoid
 avoid
 avoid
 avoid

Capitalization
 NISO = initial only
 AACR2 format
 Practice is to follow a manual of style
 Chicago Manual of Style
 Associated Press
 American Association of Publishers

Abbreviations
 Use only when well known
 Always include the full meaning
 LASER
 Scope Note Light Amplification by Stimulated
Emission of Radiation
 WHO
 World Health Organization

Other Ways of Adding Value
 Cross references
 Facets
 Notation
 Roles
 Treatment
 Term weighting

Cross References
 See - S
 See also - SA
 Not related or associated
 Not opposite
 Just helpful guides

Synthesis in Classification
 S.R.Ranganathan 1933
 Colon Classification
 analytico-syntactic classification
 analyze subject into component parts
(facets)
 arrange facets into schedules
 combine facets to express subject
complexity

Ranganathan
 A General Properties
 Ab Configuration
 Ac Tubular
 B Materials
 Bc Metals
 Bcc ferrous
 Bcd steels
 Bcf Chromium steels
 Bcfi Chromium-nickel steels
 K Modes of failure
 Kg Creep
 Kgb Creep rupture
 L Stresses and loads
 Lb Tensile

Ranganathan
 Tubular Chromium Nickel steel creep
rupture Tensile strength
 Ac Bcfi Kgb Bb
 Chain indexing
 Tubular
 Chromium Nickel steel
 creep rupture
 Tensile strength

Facets
 Additional ways to add meaning
 Divide terms into categories using a
single characteristic
 Limited number of categories

Facets and Roles
 PRECIS - Austin 1984
 order of terms
 post-coordinate indexing system
 role of the term is important
 tomato
 living plant?
 marketable product?
 Facet role indicator
 organism
 end product

Many Faceted Vocabularies
 UMLS Semantic Network
 Unified Medical Language System - 49
 BLISS Classification Association
 British Library Information Science System
 Dewey Decimal Classification System
 Universal Decimal Classification
System
 Art and Architecture Thesaurus

MeSH and Tree Pages

219
MeSH Alpha

Order of Facets
 Post-coordinate
 Means before order
 Notation becomes important
 Breaks down for large classes
 (more than 5,000 terms)

Notation Options
 Expressive
 Ordinal
 Synthetic
 Enumeration
 Many style options

Expressive Notation
 83Hazards
 831 Fire
 831.5 Fire fighting
 831.53 Fire fighting equipment
 831.532 Fire extinguishers
 831.532.5 Carbon dioxide fire extinguishers
 832 Explosions

Ordinal and Semi-ordinal
Notation
 HK Hazards
 HL Fire
 HM Fire fighting
 HN Fire fighting equipment
 HNB Fire extinguishers
 HNE Carbon dioxide fire extinguishers
 HO Explosions
 Indention is the sole indication of hierarchy

Synthetic and Enumeration
Notation
 Need to allow the classification system to
grow
 Synthetic example
 P Architecture
 PAT Architectural information
 PAT.M Architectural information services

226
Notation Examples
- AAT Facets

Systematic Display
 Paints
 (By composition)
 Oil paints
 Water paints
 Cement paints
 (By use)
 Primers
 Undercoats
 Top coats

228
AAT Pages
 Notice faceted
 indentions

229
AAT Term

Alphabetical Display
 Paints
 NT
 Cement paints
 Oil paints
 Primers
 Top coats
 Undercoats
 Water paints

Roles
 ERIC Thesaurus - role indicators
 Adjectives - bibliographic terms
 Input or raw material
 Output or product
 Undesirables
 Indicated uses
 Materials “In which”
 Affects
 Primary topics of discussion
 Passive recipients, possessors, location
 Means used

Roles
 CAS - Super roles
 Analytical study
 Biological study
 Formation, nonpreparative
 Occurrence
 Preparation
 Process
 Uses
 CAS Specific roles
 Miscellaneous
 Properties
 Reactant

Subheadings as Roles
 MeSH
 Therapeutic use
 Drug treatment (disease)
 Adverse effect (drug treatment)
 Diagnosis

Treatment and
Aspect Codes
 Apply codes or types at article level
 Theoretical
 New development
 Experimental
 Practical

Cranfield Project -
Cleverdon 1966
 Concepts in the main theme 9/10
 Major subsidiary theme 7/8
 Minor subsidiary theme 5/6

Internet Engines
 Complex weighting of terms
 Use term frequency
 Rank output wholly automatic
 Output based on input term weights
 Can also use “well formed” data -
 like a thesaurus hierarchy
 field formatted data
 XML files

Automatic and Semi-automatic
Classification?
 Data Harmony® M.A.I.™
 Semio
 Autonomy - Muscat
 Net Owl - Names
 n-Stein
 Quiver
 Smart Logic

Machine Aided Indexing Goals
 Improve
 Indexing efficiency
 Indexing consistency
 Reduce editorial drift
 Depth of Indexing
 Reduce
 Over and under indexing
 Term over use and under use

Machine Aided Indexing Goals
 Improve productivity
 Indexer
 Information worker
 Disambiguate terms
 Increase clarity

Machine Aided Indexing -
Intellectual Components
 Word List or Thesaurus
 Knowledge base
 Rules based
 Natural Language (Semantic)
 Editorial evaluation

Example:
M.A.I.™ Software Components
 Rule Builder
 Concept Extractor
 Statistics Collector

Taxonomies in
Search

Do the Data FIRST
 What do you have?
 What does it need?
 How would you LIKE to access it?
 Look at the data BEFORE you create the
specifications
 DTD built without data is not going to work
 Then choose the system that will support
your data

My Main Frustration
1. Select hardware
2. Select software
3. Design system
4. Try to load the data
5. Add the taxonomy, if at all
 That’s BACKWARDS

Why Does Search Fail?
 Most large organizations have 5 different
search 7
 All disappointing and sitting on the shelf
 Inconsistent results
 Unclear path to results
 Lack of single unified clear consistent
vocabulary
 Not tied to data governance
 Taxonomy
 Other metadata

SEARCH
 How search works
 Measuring accuracy in search
 Precision
 Recall
 Relevance
 Search theoretical basis
 Bayes, Boole, and the rest of the guys
 The taxonomy effect

Parts of Search
 Search software
 Inverted Index
 Search algorithms
 Presentation layer
 Search box
 Autocompletion
 Related and narrower terms
 Hierarchical display

Hierarchical Display
Inverted
File
Index
Searchable Index
Taxonomy
Thesaurus
Inverted Files and Boolean
are Basic to ALL Search
Note: not available in all systems!

“Outline of Presentation”
1 Define key terminology
2 Thesaurus tools
 Features
 Functions
3 Costs
 Thesaurus construction
 Thesaurus tools
4 Why & when?
Creating
an
Inverted
File
Index
Sample DOCUMENT

Simple Inverted File Index of
the Terms from the “Outline”
&
1
2
3
4
construction
costs
define
features
functions
key
of
outline
presentation
terminology
thesaurus
tools
when
why

& - Stop
1 - Stop
2 - Stop
3 - Stop
4 - Stop
construction - L7, P2, SH
costs - L6, P1, H
define - L2, P1, H
features - L4, P1, SH
functions - L5, P1, SH
key - L2, P2, H
of - Stop
outline - L1, P1, T
presentation - L1, P3, T
terminology - L2, P3, H
thesaurus - (1) - L3, P1, H
(2) - L7, P1, SH
(3) - L8, P1, SH
tools - (1) - L3, P2, H
(2) - L8, P2, SH
when - L9, P3, H
why - L9, P1, H
Complex Inverted File Index -
Placement, Location added

Search Presentation Layer
Automatic completion
And type ahead
from Thesaurus

Related
Narrower

The Hierarchical view of the thesaurus is
also a browse able view of the content.
The numbers include the number of hits
1. For the term
2. For the branch

Many parts
 Search software – of course
 Computer network
 Parsing of text – the “inverted file”
 Well formed or structured text
 CLEAN DATA
 Computer software – network
 Computer hardware
 Telecommunications connection
 Training sets for statistical systems
How Does Search Work?

Technical Parts of Search
 Search technology
 Ranking algorithms
 Query language
 Federators
 Cache
 Inverted index – as discussed above
 Other enhancements
 Presentation Layer

Access Innovations – Complex
Farm With Perfect Search
Source
Data
Query
Search
Harmony
Presentation
Layer
Repository XIS
(cache)
Cleanup, etc.
Federators
Query Servers
Index
Builders
Deploy
Hub
Cache
Builders

QUERYAPI
CUSTOM
CONNECTOR
EMAIL
CONNECTOR
Core Architectural Components
Pipeline
SEARCH
SERVER
QUERY
PROCESSOR
Query
Results
Vertical
Applications
Portals
Custom
Front-Ends
Mobile
DevicesContent
Push
DOCUMENT
PROCESSOR
Web
Content
Files,
Documents
Databases
Custom
Applications
CONTENTAPI
MANAGEMENT API
Index DB
DATABASE
CONNECTOR
FILE
TRAVERSER
WEB
CRAWLER
Pipeline
Email,
Groupware
Administrator’s
Dashboard
FILTER
SERVER
Agent DB
Alerts
Data Harmony Governance API
MAIstro
Searchharmony
FAST Search Example

Measuring Accuracy in Search
 Relevance
 Recall
 Precision
 Accuracy – Hits, miss, noise
 Ranking
 Linguistics
 Query Processing
 Results Processing
 Display
 Search refinement
 Usability
 Business Rules
263

Relevance
 How well a set of returned documents answers
the information need
 “Accuracy”
 Related to objective of search
 Different user communities
 Information resources
 Tension of user needs and context available
 A confidence “guesstimate”

Recall = Number of relevant items retrieved
Number of relevant items in the collection
Precision = Number of relevant items retrieved
Number of items retrieved
Relevance = Germane (Precision)
Pertinent (Recall)
The Formulas

Measuring Relevance
 Concepts
 Context
 Age of documents
 Completeness (recall)
 Quality
 Statistically determined ?
 Nope, it is subjective
 Someone has to determine the rightness of the item
 A confidence factor = canard!

Kinds of Search
 Bayesian –
 FAST
 Lucene
 Autonomy / Verity
 Boolean
 Dialog
 Endeca
 Perfect Search
 Ranking algorithms
 Google
267

George Boole
and Boolean Algebra
 George Boole
 Mathematician
 1815-1864
 Boolean algebra
 An algebraic system of logic
 AND, OR, NOT, ANDNOT,
 Dialog, BRS, Stairs
268

Boolean Representation
 Venn diagram showing
the intersection of sets A
AND B (in violet),
 The union of sets A OR
B (all the colored
regions),
 And set A XOR B (all the
colored regions except
the violet).
 The "universe" is
represented by the
rectangular frame.
269

Bayes and Bayes’ Theorem
 Thomas Bayes
 Mathematician
 1702 - 1761
 Bayesian theorem
 Uses probability inductively
 Established a mathematical basis for probability inference
 WHAT?
 A means of calculating,
 from the number of times an event has not occurred,
 the probability that it will occur in future trials
270

Bayesian Methods –
Cautions
 A user might wish to change the distribution of
probabilities.
 A user will make a novel request for information
in a previously unanticipated way.
 The computational difficulty of exploring a
previously unknown network.
 The quality and extent of the prior beliefs used
in Bayesian inference processing.

Bayesian Methods -
Cautions (continued)
 A Bayesian network is only as useful as the
prior knowledge is reliable.
 An optimistic or pessimistic expectation of the
quality of these prior beliefs will distort the
entire network and invalidate the results.
 Must ensure the selection of the statistical
distribution induced in modeling the data.
 Must have the proper distribution model to
describe the data.
 That is… you have to constantly train and
retrain the data

Basic Areas of Natural
Language Processing (NLP)
 Syntactic
 Semantic
 Morphological
 Phraseological
 Lemmatization (stemming)
 Statistical
 Grammatical
 Common Sense

Basic Areas of Automatic
Language Processing (ALP)
 Auto Translation
 Auto Indexing
 Auto Abstracting
 Artificial Intelligence
 Searching
 Spell Checking
 Semantic Web
 Natural Language Processes (NLP)
 Computational Linguistics

Statistical Search
 Cluster analysis
 Neural networks
 Co-occurrence
 Bayesian inference
 Latent Semantic
 Etc.
275

Word and Term Parsing
 Stemming
 -ing, -ed, -es, -’s, -s’, etc.
 Depluralization
 Truncation
 Left and right
 Wild cards
 Organi*ation
 Variant Spellings
 Centre, Center
 Hyphens

The Taxonomy Effect
 Where do the terms go?
 How are they used in search
 What other ways can I use the taxonomy
in search?

For search all publications
Search database
for Journals and
pubs
Bookstore search
Search of 53 crawled
sites including
journals, books, web
site, conference
sites, etc.
Site search
Navigation

Taxonomy Driven
Search Presentation
Navigate
the full
taxonomy
“tree”
BROWSE
Auto-completion using the
taxonomy
Guide the user

Subject Browsing

Targeted Resources Based
on Subject or User Role
CONFIDENTIAL

Member Profile Tagging
User pastes or
uploads CV
Button to auto-
extract taxonomy
attributes

TaxoTerm Server
DataHarmony
(M.A.I.)
Returns subject
metadata
Microsoft
SharePoint
Server2010
User uploads a document
to SharePoint space
Before uploading to
SharePoint server, the
EventHandler sends the
document to Data
Harmony.
Data Harmony
automatically attaches
indexing terms before
uploading to MOSS
Adding Terms
to SharePoint

SharePoint 2010 Only Shows
10 Lines of the Taxonomy
284
This add on makes it all viewable

QUERYAPI
CUSTOM
CONNECTOR
EMAIL
CONNECTOR
Core Architectural Components
Pipeline
SEARCH
SERVER
QUERY
PROCESSOR
Query
Results
Vertical
Applications
Portals
Custom
Front-Ends
Mobile
DevicesContent
Push
DOCUMENT
PROCESSOR
Web
Content
Files,
Documents
Databases
Custom
Applications
CONTENTAPI
FAST MANAGEMENT API
Index DB
DATABASE
CONNECTOR
FILE
TRAVERSER
WEB
CRAWLER
Pipeline
Email,
Groupware
Administrator’s
Dashboard
FILTER
SERVER
Agent DB
Alerts
Use taxonomy terms here
Data Harmony Governance API
MAIstro
Searchharmony
Taxonomies Added in
Search Example

Auto suggestion of
Taxonomy Terms
Populate
Keywords,
Descriptors,
Indexing terms,
etc.
Allow for manual
review of auto-
tagging for
quality
assurance.

Where do I use a taxonomy?

Thesaurus
Master
Machine Aided
Indexer
(M.A.I.™)
Database
Repository
Search
Presentation
Layer
Increases
accuracy
Browse by
Subject
Auto-completion
Broader Terms
Narrower Terms
Related Terms
Client Taxonomy
Inline Tagging
Metadata and
Entity
Extractor
Automatic
Summarization
Search
Software
Client Data
Full Text
HTML, PDF,
Data Feeds,
etc.
Client
taxonomy
The Workflow
288
Tag and
Create
metadata
Put in
data base
with tags
Build
Search
inverted index
Create
user
interface
Gather
source
data

Thesaurus
Master
Machine Aided
Indexer
(M.A.I.™)
Repository
Search
Presentation:
90%
accuracy
Browse by
Subject
Auto-
completion
Broader Terms
Narrower Terms
Related Terms
Client Taxonomy
Inline Tagging
Metadata and
Entity
Extractor
Automatic
Summarization
Search
Software
Client Data
Full Text
HTML, PDF,
Data Feeds, etc.
Client
taxonomy
Taxonomy In Sharepoint
[Data Harmony fully integrated with MOSS.]

Adding Terms to
Information Objects
 Part of the record
 XML
 MARC
 A relational table pointing the terms to a
record ID number (Secondary key)
 Adding data to the HTML
 META NAME KEYWORD Element
 Many other options

Part of the Record - XML
 Added as an element in the XML record
 Need an element to put the data in
 <Taxonomy Term>
 Capture the terms when creating the
records

The author pastes
the data to the
document
template,
attaching images,
graphs, as
necessary:
Author
Submission
Module

Editorial Workflow Integration
Author Submission Module
The author fills in the data to the document template, attaching images
and graphs as necessary.
An API calls Data Harmony and generates a list of indexing terms
based on the content.

Authors review the
indexing and may
change it.
Content is stored
into a data
repository as
HTML, XML, etc.
Editorial Workflow Integration
Author Submission Module

In the HTML Record
 Makes it crawlable for the internet
 Used in CMS applications
 Content Management Systems
 Add to the HTML
 Manually
 In Dreamweaver
 In your CMS like Extron
 Author Submissions Example
 Do the same with SharePoint

META NAME “KEYWORDS”

In Relational Database
Table
 Primary Key – the record
 Secondary key all the metadata
 Like taxonomy terms
 Like author
 Like publication date
 Used in Oracle, SQL, etc
 Need a field to put the taxonomy data in
 Supports “Faceted Search”
 each item in a separate field or element or table

RDBMS Connection
Taxonomy
term table

Using Taxonomies
in Applications
• Improve search
• Subject browsing
• Mobile intelligence
• Targeted resources based
on subject or user role
• Link to society resources
• Author submission module
• Author authority database
• Expert reviewer identification
• Member profiles
• Data visualization
• More like this
• In “indexing” or categorizing,
as subject metadata
• In content management
systems
• In SharePoint
• In mashups
• In social networking sites
• In author tagging
• In filtering data – e.g., spam
filters and RSS feeds
• In web crawlers
• Social media - community

A Quick Look
Behind the Scenes
Database
Management
System
Thesaurus
tool
Indexing
tool•Validate terms
•Add terms and rules
•Change terms and rules
•Delete terms and rules
•Search thesaurus
•Validate term entry
•Block invalid terms
•Record candidates
•Establish rules for
term use
•Suggest indexing
terms

Taxonomy
view
Thesaurus
Term Record
view

Where Does the
Subject Metadata Go?
 Apply to content itself
 Use meta name field in HTML header
 Connect search to the keywords in the SQL or
other database tables

HTML Header

Suggested taxonomy descriptors

Integrate Taxonomy
to Enhance Find-ability
 Browsable categories of a directory
 Browsable faceted navigation
 Smart search for term equivalents
 Taxonomy terms (original or modified) as labels
 Navigation aids incorporate taxonomy terms
and relationships

More Taxonomy
Enrichment
 Spelling alternatives and correction
 Related concepts
 Statistical information about the metadata
 Navigation or drill downs
 Search refinement
 Recursive sets
 Concept linking
 Dictionary lookup (in taxonomy glossary)

Brand is repeated in several spots and
tied to search as well

Raw Full
text data
feeds
XIS™
Creation
Taxonomy
Thesaurus
Master®
Printed
source
materials
Taxonomy
terms
M.A.I.™
Concept
Extractor
M.A.I.™
Rule Base
Load to
Perfect Search
Search
Harmon™
Display
Search
Database Plus Search Workflow
Data
Crawls on
53+
sources
Add
metadata
XIS™
repository
SQL for
ecommerce
Save data to search and
repositories at the same timeCopyright © 2013 Access Innovations, Inc.

Raw Full
text data
feeds XIS
Creation
Taxonomy
Thesaurus
Master
Printed
source
materials
Taxonomy
terms
MAI Rule
Base
Load to
Search
Search
Harmony
Display
Search
Data Base Plus Search Workflow
Data
Crawls on
data
sources
Add
metadata
XIS
repository
SQL for
ecommerce
MAI Concept
Extractor
Source data
Clean and enhance data
Search data

Use Case: Inline Tagging
Show the exact point where the
concept is mentioned
Mouse-over to view the term record
Statistical summary, showing the
number of times each term is
mentioned in the article

Inline Tagging HTML View

XML View for
Inline Tagging

 The New Board Game
 Applications
 Implementation
 The taxonomy

The Changing Faces of
Web Taxonomies
 ….and how the information is delivered
 From current site
 To new version
 Depends on TAXONOMY
 Personalization
 Feeding ads
 Consistent information

HTML Headers
META NAME KEYWORD
Use the taxonomy here

More Innovations!
 Link topic to article to author to event
 Make visual links within domain
 Enable authors to submit and categorize conference
submissions
 Create author authority database linking to co-authors,
topics, locations, etc.
 Create expert reviewer database
 Create member profiles with alternate names, publications,
tagged by topic
 Visualize data and domain distribution
 Display interest connections in social network
 Deliver accurate targeted information through mobile
applications
 Etc.

Change to Ready, Aim, Fire!
 Follow the data
 Look at the data, format and content
 Design taxonomy for data
 Leverage the standards
 Use taxonomy to tag data
 Choose search and repository software for data
 Load the data into the system
 Keep your eye on the target

Standards for
Monolingual Thesauri
 TEST - Thesaurus of engineering and
scientific terms - COSATI 1967
 ARNOR NFZ 47-100 1981 French
 DIN 1463 German 1987-1993
 NISO Z39.19 - 1993 - American

Where Can I Get
Taxonomy Standards?
 www.niso.org
 Z39.19 (2010) Controlled Vocabularies
 www.ISO.ce
 ISO 25964 parts 1 and 2 (2012 and 2013)
 www.bsi.uk.co
 www.w3c.org SKOS and OWL
 www.accessinn.com/library

Suggested Reading
 F.W. Lancaster - 1986
 Vocabulary Control 1986
 Aitchison, Gilchrist and Bawden
 Thesaurus construction and use: a practical manual
4th edition
 Accidental Taxonomist
 Heather Heddon
 TaxoDiary.com Blog site

Suggested Reading
 Introduction to any thesaurus
 INSPEC
 NICEM
 Pychological Abstracts
 etc.

It Just Takes
a Little
Imagination
Thank you
Marjorie M.K. Hlava, President
Bob Kasenchak, Project Coordinator
Access Innovations
505-998-0800
mhlava@accessinn.com
Bob_kasenchak@accessinn.com

Taxonomy Fundamentals - SLA 2014

More Related Content

What's hot

Viewers also liked

Similar to Taxonomy Fundamentals - SLA 2014

More from Access Innovations, Inc.

Taxonomy Fundamentals - SLA 2014