Taxonomies are often thought of as hard to use and needing specialized applications or IT skills. Not so. Explore how taxonomies, auto-classification, and multi-term metadata generation unburden the IT team, eliminate end user tagging, and empower business users. Understand the Return on Investment from an effective infrastructure solution for search, security, compliance, eDiscovery, records management, knowledge management, collaboration, and migration activities. • Watch multi-term metadata being automatically generated. • Learn how easy it is to use taxonomy tools and interactive features, such as auto-clue suggestion, instant feedback, and assigning weights to terms. • Discover the value of dynamic screen updating to immediately see the impact of taxonomy changes. • View how document movement feedback enables you to see the cause and effect of changes without re-indexing. Understand must-have functionality, to help you evaluate classification and taxonomy software. Starting with the importance of multi-term metadata, learn about the pros and cons of differing technologies, which questions to ask vendors, and what suits your organization. Go beyond the basics, to find out what it takes to manage a taxonomy and integrate it with the SharePoint Term Store. Take away an understanding of: • Metadata generation – why it is so important. • Auto-classification – why you can’t live without it. • Taxonomy approaches that are manageable – by the staff you already have.

1. © Concept Searching 2017
The Nuts and Bolts of Metadata Tagging
and Taxonomies Made Easy
Michael Paye
Chief Technology Officer
Concept Searching
mikep@conceptsearching.com
www.conceptsearching.com
marketing@conceptsearching.com
Twitter @conceptsearch
Paul Billingham
Director of Sales, Europe
Concept Searching
paulb@conceptsearching.com

2. © Concept Searching 2017
Michael Paye – Chief Technology Officer at Concept Searching
has been the driving force behind many of the company's recent
innovations, including the SharePoint Add-in and hybrid search
products. He has a wealth of experience across the Microsoft
platform and related technologies, and oversees all product
development.
Paul Billingham – Director of Sales, Europe at Concept Searching
is one of the company founders, and has over 20 years’ sales
experience, working primarily within the document management and
workflow industry. He has a technical background, which is a major
benefit when selling complex technology through a partner channel.

3. © Concept Searching 2017
Agenda
• Who we are and what we do
• What’s the problem?
• What does it impact?
• How do you measure performance?
• Metadata generation
• Auto-classification – What does it do?
• Taxonomies – What kinds are there?
• SharePoint Term Store
• Calculating return on investment

4. © Concept Searching 2017
• Company founded in 2002
• Product launched in 2003
• Focus on management of structured and unstructured information
• Profitable, debt free
• Technology Platform
• Delivered as a web service
• Automatic concept identification, content tagging, auto-classification,
taxonomy management
• Only statistical vendor that can extract conceptual metadata
• 8 years KMWorld ‘100 Companies that Matter in Knowledge Management’
8 years KMWorld ‘Trend Setting Product’
• Authority to Operate enterprise wide US Air Force, NETCON US Army,
and Canadian SLSA
• Client base: Fortune 500/1000 organizations in Healthcare,
Financial Services, Manufacturing, Energy, Professional Services,
Pharmaceutical, Public sector and DoD
• Microsoft Gold Certification in Application Development
• Member of SharePoint PAC and TAP programs
• Suitable for all versions of SharePoint on-premises and SharePoint Online,
including the latest vNext dedicated platform and the government cloud
The Global Leader in
Managed Metadata Solutions

5. © Concept Searching 2017
Concept Searching’s technology platforms deliver
semantic metadata generation, auto-classification and
taxonomy/Term Store management, and are fully
integrated with all versions of SharePoint on-premises,
Microsoft Online/Office 365, and OneDrive for Business
What Do We Do?
These infrastructure platforms integrate not only with
SharePoint but also other content repositories, search
engines and file shares, enabling our clients to add
structure and manage their enterprise content,
regardless of environment
The resulting classification metadata is used by clients
to deliver ‘intelligent metadata solutions’ in areas such
as enhanced search, migration, data privacy, records
management, policy enforcement, compliance, text
analytics, and business and social collaboration

6. © Concept Searching 2017
Definition
• Metadata describes other data, it
provides information about a certain
item's content
• For example, an image may include
metadata that describes how large
the picture is, the color depth, the
image resolution, when the image
was created, and other data
• A text document's metadata may
contain information about how long
the document is, who the author is,
when the document was written, and
a short summary of the document
TechTerms.com
Metadata

7. © Concept Searching 2017
Types of Metadata
Intrinsic
• Information that can be extracted directly
from an object (file name, size)
Administrative/Management
• Information used to manage the document
(author, date created, date to be reviewed)
Descriptive
• Information that describes the object (title,
subject, audience)
Semantic
• Ability to extract concepts from within
content and generate the metadata
(intelligent metadata)

8. © Concept Searching 2017
“Over 80% of business decisions are made using unstructured data.”
IDC
What’s the Problem?

9. © Concept Searching 2017
• 91% use manual metadata tagging
• Free-for-all mode
• Drop down lists
• 15% maintain a homegrown manual taxonomy
• 77% have no rhyme or reason for managing content
Information Chaos
• Unstructured data is growing at the rate of 62% per year IDG
• By 2022, 93% of all data in the digital universe will be unstructured IDG
• Data volume is set to grow 800% over the next five years and 80% of it
will reside as unstructured data Gartner
What’s the Problem?

10. © Concept Searching 2017
It’s not just about search
What Does it Impact?

11. © Concept Searching 2017
How do you measure performance?

12. © Concept Searching 2017
Precision Versus Recall
• Usually used by academics
• Precision
• Positive predictive value
• Fraction of retrieved instances that are relevant
• Recall
• Sensitivity
• Correct number of documents that are relevant
• Fraction of relevant instances that are retrieved
• In a perfect world, they should be balanced
• Commercial evaluation criteria also take into account
• Order of the returned results
• Overall ability of a user to find an answer rather
than relying on a search being submitted only once

13. © Concept Searching 2017
• Automated metadata generation is
difficult to achieve consistently with
high precision and recall
• Many products on the market today
require complex rules to be generated,
often involving search syntax and
complicated Boolean expressions
• Some require a document training set
for every term to be processed
• Some of these products employ
linguistic techniques that will not
perform consistently across different
vertical markets
Result is very high initial cost in terms of
time and level of qualified staff
Precision Versus Recall

14. © Concept Searching 2017
A manual metadata approach will fail 95% of the time
Why is it So Hard to Get Metadata Right?
Issue Organizational Impact
Inconsistent Less than 50% of content is correctly indexed, meta-tagged or
efficiently searchable rendering it unusable to the organization. (IDC)
Subjective Highly trained Information Specialists will agree on meta tags between
33% - 50% of the time. (C. Cleverdon)
Cumbersome – expensive Average cost of manually tagging one item runs from $4-$7 per
document and does not factor in the accuracy of the meta tags nor the
repercussions from mis-tagged content. (Hoovers)
Malicious compliance End users select first value in list.
(Perspectives on Metadata, Sarah Courier)
No perceived value for end user What’s in it for me? End user creates document, does not see value for
organization nor risks associated with litigation and non-conformance to
policies.
What have you seen Metadata will continue to be a problem due to inconsistent human
behavior.

15. © Concept Searching 2017
• A feature found in some content management
systems or records management applications
that will scan the contents of a document and
automatically assign metadata, categories,
and keywords based on the document
contents
• Content-based assignment of one or more
pre-defined categories to documents
(records), usually machine learning, statistical
pattern recognition, or neural network
approaches that are used to construct
classifiers automatically
What is Auto-classification?

16. © Concept Searching 2017
Automatic generation of compound term metadata
Set up a taxonomy node, suggest clues for class, document feedback

17. © Concept Searching 2017
Auto-classification Systems – What Do They Do?
Document
Preparation
• Split into language
blocks (paragraphs,
headings),
formatting, layout
Parsing
• Entity extraction
• NLP: parts of speech,
phrases
• Terms, variants
Weighting
• Frequency
• Location in text,
phrase
• Proximity
• Combination
• Format of text
Classification
• If threshold reached
• Can influence search
results
This is where rules
vs statistics come
into play…
Not all classification solutions are created equal

18. © Concept Searching 2017
Auto-classification Systems
Keyword
• Boolean operators add a degree of sophistication,
but also tend to improve precision at the expense
of recall, because any document that does not
match the Boolean expression is ignored
• The majority of search users are unable to
formulate even basic Boolean expressions
Linguistic
• No commitment to a taxonomic tree
• Related to parts of speech, syntactic parses,
or semantic interpretations
• Typically not scalable
• Usually delivered as pre-configured for an
industry, hard to integrate your unique
organizational vocabulary

19. © Concept Searching 2017
Semantic Networks
• Refers to a set of relationships between
concepts and words, including parts of
speech and real-world relationships
• These can include rules of various types,
not just Boolean
Machine Learning
• Subfield of computer science (CS)
and artificial intelligence (AI) that deals with
the construction and study of systems that
can learn from data, rather than follow only
explicitly programmed instructions
Auto-classification Systems

20. © Concept Searching 2017
Auto-classification in action

21. © Concept Searching 2017
Taxonomies
Taxonomy
• A taxonomy is an organized set of
concepts or definitions, usually labeled
keywords
• For search engines, a taxonomy can
also be a set of organized searches
• Taxonomies are typically nested in a
hierarchical manner, often called a ‘tree’
• Subject-based taxonomy – created by
domain experts
• Content-based taxonomy – organizing
the data you already have
• Behavior-based taxonomy – driven by
search analytics, user tagging, or
vocabulary analysis

22. © Concept Searching 2017
Types of Taxonomies
List, Picklist, Controlled Vocabulary, Authority Files
List of lead or preferred terms, selected by the end
user, may or may not have relationships among the
terms, can include a synonym ring
Synonym Lists
The use of synonyms allows one concept to be
instantiated as the same as the other, but still
allows a term to be preferred over another
Hierarchical
Each content item resides in only one category,
referred to as a ‘tree’
• Piano
• Musical instrument

23. © Concept Searching 2017
Types of Taxonomies
Polyhierarchical, Faceted, Thesauri
Content items can exist in more than one category,
more structured controlled vocabulary, provides
information about each term and its relationship to
other terms, features of a hierarchical taxonomy
plus associative relationships
• Piano
• Musical instrument
• Stringed instrument
• Percussion instrument
Ontology
Multiple taxonomies with additional relationships
added to specify concepts within a domain
Marlene Rockmore – The Taxonomy Blog
Heather Hedden – The Accidental Taxonomist

24. © Concept Searching 2017
SharePoint Term Store
• Introduced in 2010
• Provides infrastructure for
taxonomy management
• Managed metadata properties
designed for hierarchical
metadata
• Integrated with search via the
refinement panel
• Utilizes GUIDs for term/tag
identification
SharePoint has no automatic generation of metadata
SharePoint has no auto-classification capability
SharePoint has no facility to generate concepts

25. © Concept Searching 2017
Automatic, real-time update of the SharePoint Term Store

26. © Concept Searching 2017
Return On Investment

27. © Concept Searching 2017
Return On Investment – Real World Savings
Pique Solutions
The Business Solutions
• Search
• Records Management
• Intelligent Migration
• Data Security/Confidentiality
• eDiscovery/Litigation
Support, FOIA
• Information Governance
• Text Analytics
• Business Social Networking
• Collaboration
• Content Lifecycle
Management
• Metadata Management
• Research
• Knowledge Management

28. © Concept Searching 2017
Thank You
Michael Paye
Chief Technology Officer
Concept Searching
mikep@conceptsearching.com
www.conceptsearching.com
marketing@conceptsearching.com
Twitter @conceptsearch
Paul Billingham
Director of Sales, Europe
Concept Searching
paulb@conceptsearching.com