Taxonomies and Search for Chicago SharePoint User Group

Taxonomies, Metadata and Search Seth Earley 781-4820-8080 Seth@earley.com Earley & Associates, Inc. | Classification: CONFIDENTIAL USE, NO REPRINTS Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

SharePoint Call Series Architecting for Business ValueStarting March 3rd, 2011 (Recordings will be available)Register at:www.earley.com/webinars/jumpstarts/sharepoint-2010-architecting-business-value2 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Jumpstart Series – Architecting SharePoint for Business Value• Session 1 - SharePoint 2010 – Best Practices for Creating Business Value  March 3rd, 12:00- 1:00 pm• Session 2: Methods and Tools for Better SharePoint Search  March 10th, 12:00- 1:00 pm• Session 3: Practical Approaches to Developing Rich Information Architectures  March 17th, 12:00- 1:00 pm• Session 4: The Role of Governance in Ensuring Success  March 24th, 12:00- 1:00 pm3 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Earley & Associates Highlights Founded 1994 Focus Areas Holistic approach to specific business contexts and goals for: • Retail • Manufacturing • Pharmaceuticals & Life Sciences • Public Sector • Media & Entertainment Personnel Core team of 30 consultants Locations Stow, MA headquarters, consultants in US, UK & Canada, global projects Services • Taxonomy & Information Architecture • Search Strategy for Enterprise & Web • ECM, DAM & Information Lifecycle • Program Management & Governance4 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Seth Earley, Founder & President, Earley & Associates • Co-author of Practical Knowledge Management from IBM Press • 17 years experience building content and knowledge management systems, 20+ years experience in technology • Former Co-Chair, Academy of Motion Picture Arts and Sciences, Science and Technology Council Metadata Project Committee • Founder of the Boston Knowledge Management Forum • Former adjunct professor at Northeastern University • Guest speaker for US Strategic Command briefing on knowledge networks • Currently working with enterprises to develop knowledge and digital asset management systems, taxonomy and metadata governance strategies • Founder of Taxonomy Community of Practice – host monthly conference calls of case studies on taxonomy derivation and application. http://finance.groups.yahoo.com/group/TaxoCoP 100+ calls since 2005 • Co-founder Search Community of Practice: http://tech.groups.yahoo.com/group/SearchCoP5 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Session ObjectiveFrom Session Abstract• High level review of basic concepts related to taxonomy, metadata and search• How are taxonomies integrated with metadata management and standards and• The relationship between taxonomy and information architecture• How taxonomy, metadata and IA relate to SharePoint• Options for creating good information architectures within 2010.• How to leverage taxonomy and metadata to improve navigation and search in your SharePoint portal.• Techniques for implementation using native SharePoint functionality.6 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Agenda • Change is constant • Taxonomy definition • Information and semantic architecture • The challenge of search • Five basic truths about search • The role of metadata • Taxonomy and navigation • Case Study • Conclusion7 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Change is constant • Snap shot versus movie • Business changes faster than IT can • Systems grow up to solve specific problems without a view toward integration • Integrated environments • Solution to application proliferation…?8 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Same Term, Different Expressions… Problems: Library Cardiology • Difficulty finding relevant information • Federated search configuration is cumbersome • Inability to view consolidated results Web site Cardiac Care • Limited ability to control shared vocabularies • Weak governance or demonstrated control • Costly/cumbersome administrative overhead Healthday Heart Health9 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Taxonomy is an enabler… • Every organization is struggling with findability • Content management applications, search tools, workflow applications, customer relationship management systems, etc all strive to create views of information that are in the context of work processes What is the key component to any of these initiatives? Having a common language in which to: •Describe •Communicate •Translate information between applications and between user audiences10 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Information architecture versus Semantic architecture • Information architecture describes the ways in which systems capture, manage, organize and present information  Metadata fields describe information about a document or piece of content.  Identifiers of various kinds: Name, account number, part id, price, etc  Conditions or status of the content: Workflow approval state, Date created, review date, etc • Semantic architecture is about meaning and nuance  Terms can have multiple contexts and meanings.  People use different terms to describe the same thing11 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Info Architecture Semantic ArchitectureA single concept can have A single expression can representdifferent Expressions different Concepts Person we do business with Pitch • Cust_Name • the property of sound • Cust_ID • Customer ID • the throwing of a baseball • Customer • a vendors position (especially on the • Client sidewalk) • sales talk Person who writes a document • degree of deviation from a horizontal • Contributor plane • Author • dark heavy viscid substance • Creator • a high approach shot in golf What we buy or sell a product for: • a card game • Price • abrupt up-and-down motion • Cost • the action of throwing something • …12 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved. Source: Fred Leise

Taxonomy definition• Taxonomy is a system for Products organizing concepts and categorizing content Action  Expresses hierarchical figures Games relationships (parent/child)  Arranged in a tree-like Board games Card games structure, with top level categories that branch out to reveal sub-categories and Brands terms in varying levels of depth  Dictionary of preferred Disney Milton Bradley terminology Battleship Scrabble13 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Taxonomy definition • Taxonomy: system for organizing concepts and categorizing content • Expresses hierarchical relationships (parent/child) • Expresses other relationships Car Preferred term SYN: Automobile Vehicle Synonyms fr-CA: Voiture en-UK: Auto Translations es-CO: Carro and regional variants Sample taxonomy record14 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Taxonomy is a foundation…• It is a system for classification• It allows for a means to organize documents and web content• Helps us fine tune search tools and mechanisms• Creates a common language for sharing concepts• Allows for a coherent approach to integrate information sources• It is a common language for business processes15 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Taxonomy as a common business language Browsing & filtering Related documents Financial reporting Compare product Case Example: Motorola‟s Global Business intelligence Program Management Taxonomy Framework Served Multiple Processes Product Lifecycle Management16 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Enterprise taxonomy drivers Application Primary “Clock Constituencies Technology driver speed” challenges Web Content Consistency in Medium Web developers, Exposing taxonomy to branding, to fast content managers, CMS, integration with internal content creators search efficiencies Enterprise data Cross platform Very slow Data architects, “Source of truth”, difficulty standards integration, to slow standards boards, integrating metadata business data modelers, standards intelligence, business intelligence metadata modeling, data warehousing E Commerce Web site sales. Very fast Merchandisers, e Commerce platforms do Need to support commerce not necessarily leverage customer development team, capabilities. Updates to experience marketing classification are not a priority Product Product Fast Engineering, Product Product life cycle development development development, management systems efficiencies, product marketers usually self contained speed to market Intranet Internal Slow to Intranet managers, Difficulty unifying access development efficiencies Medium functional managers to multiple repositories, sheer volume of sources17 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Search as Utility • “search as a utility has become deeply ingrained into peoples everyday lives.“ – Study by Nielsen/Net Ratings • “search software, hardware, and support bundle or search appliance has become very popular since being introduced in early 2002" – Goebel Group These are misleading concepts. Search is used as a utility, but contexts vary so widely that “plugging search in” does not always produce satisfactory results.19 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Truth #1. We have to change our definition of search. • Search is no longer just a white box. • Search is an experience. • Search is about information access & capabilities.20 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Truth #2. Search algorithms are getting better, but they cannot infer human context & intent. • A search engine doesn‟t know if I‟m an engineer, an attorney, or a high school student. • Perspective has an impact on whether a set of search results are useful & appropriate.21 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Truth #3. Taxonomy, metadata and information architecture are key aspects of search. • Search is fundamentally about metaata • Some content is structured, some isn‟t and needs help • Advanced search functionalities require taxonomy22 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Truth #4. Search is increasingly looking like navigation. • What happens when you click on a link? • Guided navigation & faceted search are really the same thing23 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

“…search terms are short, ambiguous and an approximation of the searcher’s real information need…” Source: http://research.microsoft.com/~ryenw/papers/WhiteCONTEXT2002.pdf Ryen W. White, Joemon M. Jose and Ian Ruthven25 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Rising Expectations Plus Increased Complexity • Search seems to be a „given‟ – we expect it to be there • Most enterprise search is less than optimal – too many results, irrelevant results, missing results • It was not so long ago that organizations were starved for information • A puzzling fact: as information environments have grown more complex, users expectations have grown that search should be simpler26 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Search is complex Customer Relationship Mgt Custom databases and applicationsBusiness Intelligence/Analytics Document repositories Intranets/web pages Enterprise search is diverse – need to access multiple applications and contexts – both structured and unstructured27 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Search is Heterogeneous Data Sources Business Intelligence Document repositories Intranets/web pages Customer Relationship Mgt Custom databases and applications Search/Tagging/Taxonomy Integration Framework Auto categorization/ Entity Faceted Semantic Appliances Federated Clustering Extraction Search Search Search Search Mechanisms 28 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

What is the right mechanism for accessing information?• Content can be created in structured or unstructured contexts• It‟s value can vary depending on audience, context or process• Some content is extremely nuanced and requires more precise access (according to audience or task, solution, etc…)• Search can be based on inherent structure and content of a document (implicit metadata) or on information applied to that content (explicit metadata)29 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Different tools are appropriate depending upon degree of collaboration and creation versus structured access Less More Structured Structured Knowledge Creation Knowledge Access/Reuse Chaotic Processes Emergent Value Controlled Processes Email Wikki‟s Online Workflow Records Mgt Collaborative Learning Systems Workspaces systems Instant Blogs Messages Instructor Content Mgt Doc Mgt Discussions Led Systems Courses30 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Relative value Lower Value Higher Value (More difficult to access) (Easier to access) Unfiltered Reviewed/Vetted/Approved Lower Cost Formal Tagging/Organizing Processes Higher Cost External News Interim Example Benchmarks Best deliverables deliverables Practices Message Discussion text postings Content Success Approved Repositories Stories Methods Social tagging Structured tagging (“folksonomy”) (taxonomy) 31 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

The Role of Metadata Metadata drives content processes Taxonomies provide the organizing principles behind metadata Earley & Associates, Inc. | Classification: CONFIDENTIAL USE, NO REPRINTS Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

What is metadata? • It is the “is –ness” of a piece of content • And the “about- ness” of a piece of content • This is a Product Description • It is about the Motorola Android Taxonomies are the organizing principle behind metadata and the values that populate metadata fields`33 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

What is a content model? • Content is structured with body information and a wrapper that formats and tags that information • Also called a “content object model”* Title Simple content object model Description*Content model refers to overall frameworkContent object model refers to a specific model for a setof document typesI.e., an overall “Content Model” includes multipleContent Object Models” 34 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

FAQ Product “is – ness” Press release Specification Promotion Title Doc_ID Doc_Type Author Date “about – ness” Product_Name Features Metadata for a product page in a content Category management system35 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Meta data allows for various views of content Standard Header • Web pages are made up of assembled items of content Title Doc_ID Author Date • These are comprised of metadata elements that are Product_Name Content_ID assembled together into Features Date “content types” Comp_Features Category Product content type Promotion_ID Promotion content type Promo_Type Content_ID Date Related_Products Related Products content type Content_ID Date36 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

The User Experience (UX) is at the intersection of taxonomies, metadata and content objects • Taxonomy: system for organizing and classifying content • Metadata: information about our content, housekeeping, as well as semantic and structural information • Content Objects: groups of metadata that are assembled into components that are then assembled into pages or documentsHow will taxonomysurface on the front-facing application?What do the wireframessuggest?How do people interactwith it?How does the contentarchitecture deliver thefront-end design?37 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Taxonomy and the User Experience• Define what the user interface will eventually look like• Identify how content is laid out on the page• Faceted Search: Misspellings Synonyms Best Bets Taxonomy Results Facets Document Preview38 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Taxonomy in a content management application1. Filtering products / search results 1 22. Dynamic relationships 43. Tagging & categorization of content4. Dynamic navigation5. Feature consistency / compare product3 4 5 39 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

When is it metadata and when is it taxonomy? • Taxonomy can be applied as metadata • Typically this is expressed as a drop down “controlled vocabulary” list (also called “reference data”) • Some controlled vocabularies are very simple, with a few unambiguous choices • Some are specific to a particular system or tool and will not change frequently • There is a tendency to lump all metadata into a technology bucket and assume this is owned and managed by IT • Not a good approach (since we need business ownership and participation)40 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Who owns the taxonomy? A question of governance• Metadata Management (IT or application owner)  Unambiguous  Limited number of values  Not frequently changing  Housekeeping or administration role  Specific to an application• Taxonomy Management (business or functional owner)  Ambiguous meaning  Subject to frequent changes or updates  Common across multiple applications or contexts  Requires specific knowledge of field (subject matter expertise)41 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Metadata and SearchAll search leverages metadataExplicit versus implicit metadata Earley & Associates, Inc. | Classification: CONFIDENTIAL USE, NO REPRINTS Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

All search leverages metadata… …but not all metadata is explicit • Full text search derives metadata about documents • Creates an index of terms that occur in a document collection • Associates documents with those index entries43 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Explicit metadata versus implicit metadata Content Type = License ABC Company Explicit metadata Organization = DEF Company Forward Index – Words per document Topic = Support Inverted Index – Documents per word ABC shall provide first level technical support to all Licensed Product end users and/or ABC Sublicensed Product customers/users. DEF customers release condition will provide second level support. DEF shall customer support section 7 provide to ABC a primary and a secondary customer support team secondary support support person to act as the primary interface with ABC‟s technical and customer support DEF SLA team. DEF shall provide direct technical DEF software SLA failure support to ABC for all uses of the DEF end users software Software. Support level definitions and escrow agreement. source code responsibilities are set forth in Exhibit C. An escrow agent support level “SLA Failure” as defined in Exhibit C shall exhibit c sublicensed product qualify as a Release Condition sufficient to authorize the Escrow Agent to release to licensed product technical support Source Code to ABC pursuant to Section 7 and the Escrow Agreement. Used to derive implicit metadata44 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Search index points to document In which documents do these words occur? Forward Index – Words per document Inverted Index – Documents per word Term Document Acme 1, 2, 3, 4 customers 2, 3 escrow 3, 4 exhibit c 2 1 license 1, 4 2 …etc …etc 3 4 A search index becomes derived metadata about a collection of documents45 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

All Search Leverages Metadata…but not all metadata is explicit• Full text search derives metadata about documents• Creates an index of terms that occur in a document collection• Associates documents with those index entries• Occurrence of certain words in a document and the relative value of those occurrences, including:  Weighting  Relative positioning  Semantic relationships……becomes information about the document that is cached in the index and served by the search engine• Search algorithms vary in how metadata is derived and exposed to users. Relevance ranking, for example, is additional metadata for a result that is ‘implied’ or derived based on incoming connections to a piece of content.46 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Examples of implicit metadata:• „Structure‟ and format of content – a piece of content may be „unstructured‟ and not contain metadata, but it is well organized.  Example : Newspaper story contains a headline, sub head, and first paragraph with who, what, where, when, etc.  Clear editorial standards• Context of content – Where did the content come from? If from a particular web site, file share, data source or intranet location the domain of knowledge provides context.  How can we disambiguate the term “diamond”?  Sports site – baseball diamond  Commerce site – diamond ring  Sales context for „feature‟ versus engineering context for „feature‟  “Adapter” – power cord  “Adapter” – blue tooth headset47 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Context as metadata• If we maintain context of a piece of information in our search results, this is equivalent to having additional metadata on that content Search results organized by repository This is a form of “federated” search – a single search term fed to multiple repositories Example courtesy of Morrison and Foerster48 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Why you will not “just get Google” • Google leverages linkages on the web that are not typically duplicated internally in the organization • Search engines cannot infer intent or know what is important to you in the context of your work task • Information relevance is dependant on who you are and your level of expertise as well as what you are trying to accomplish • Not all content is equal - Google is fine for broad search results or less precise information, may not work as well if large numbers of documents with finer granularity of differences50 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

“Sound bite” definitions• A Taxonomy is a list of terms that enable classification of information  Method used to organize Subject/Topic metadata  Typically expresses hierarchical relationships (parent/child)  Emphasizes context• A Thesaurus is a specialized taxonomy  Equivalence relationships (synonyms)  Associative relationships (related terms – “see also”)  Preferred terms, variant terms• An Ontology is a collection of taxonomies and thesauri  A body of knowledge is represented by multiple lists of categories  Categories of various types are conceptually related54 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Definitions • Classification Scheme - A preordained structure of words or symbols used to organize information content • Index - A list organized in a standardized sequential fashion Types of indexes may include: back-of-the-book, telephone directory, computerized look-up tables (e.g. b-tree, file system), card catalog, meeting roster of attendees, customer list, to name a few. An index is a classification scheme A taxonomy is a classification scheme But… a classification scheme is not necessarily a taxonomy…55 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Classification versus Taxonomy TAX TAX ITEMS TAX PAYERS Assets Assets Individuals Individuals Real Estate Single Corporations Vehicles Married Liabilities Liabilities Organizations Individuals Loans Corporations Corporations Debts Associations56 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Types of Term Relationships Increasing complexity Equivalence Hierarchical Associative Used in thesauri. Purist definition of Things that are related a taxonomy – conceptually. Also called terms have parent/child “entry types” of terms. Associative relation types relationship. are context and audience Synonyms. specific. This is how we might relate multiple taxonomies.57 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Relationship Examples Relationship Types E Equivalence Computer H Hierarchical Manufacturers A Associative H International Business E IBM E Big Blue Machines ? H ? Software Hardware ? A Software Group A58 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Equivalence Terms Associative Terms• Common misspellings • See also• Other terms used • Related products• Abbreviations • Language spoken• Internal names • Products for market • Available in region • Risks in region59 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Goals of a taxonomy • Allow for knowledge discovery • Improve usability of applications as well as learnability of applications • Reduce the cost of delivering services, developing products and conducting operations • Improve operational efficiencies by allowing for reuse of information rather than recreation • Improve search results and applicability (both precision and recall)61 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Taxonomy Challenges• Taxonomy means many things in SharePoint  Site organization  Content types  Controlled vocabularies for tagging documents• Challenges  Typically integration of legacy content requires significant tagging effort  Users wanted to leverage hierarchy in search in the form of faceted navigation62 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Taxonomy Solutions• Taxonomy Technology  Leveraging Hierarchy and Taxonomy in both tagging and faceted search  True taxonomy management is beyond the scope of SharePoint 2010• Taxonomy in Context  Auto-populate metadata fields with taxonomy values based on the overall architecture of the site and users roles  Reduce the burden on users allow Locations, Departments, Roles to be filled in automatically63 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Recall versus Precision• The goal of effective search is to pull back lots of relevant results• This is measured by “recall” and “precision”• Recall: I am getting the documents that contain my term• Precision: These results are relevant to me When trying to improve recall, precision can suffer and vice versa Precision can also be subjective – based on who we are and what we are doing, in other words, context and task64 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Ratio of number of relevant A Recall items retrieved to total number A+ B X 100 % of relevant items in database Irrelevant items Precision C Items Ratio of number of relevant retrievedRelevant items retrieved to total numberitems not B of irrelevant and relevant itemsretrieved retrieved Relevant items A A retrieved X 100 % Relevant items A+ C in a database Goal is to improve recall without sacrificing precision 65 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Taxonomy & search strategies Six strategies you should know about  Tuned search  Relevance ranking  Faceted search  Related terms  Clustering  Disambiguation66 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Taxonomy and Search Strategies • Pre Search processing  Search engine applies taxonomy or thesaurus to narrow or expand search before retrieving results  Tuned search “Best Bets”  Relevance ranking  Faceted search • Post Search Processing  Search results are narrowed or organized after they are retrieved  Related terms  Clustering  Disambiguation67 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Applying a taxonomy to search We need a mechanism to improve search • A Taxonomy can be used to  Define search terms and map those terms to specific locations of information (need to integrate with a search engine)  Apply terms to a document so that relevant and consistent search results are returned (need to integrate with a content management system) • A Thesaurus can be used to define term synonyms and related terms in order to improve the recall of information.  We may define “proposal” and “statement of work” and “SOW” as meaning the same thing. If I enter SOW, I can pull back documents that are labeled with (or contain) the other terms. This is referred to as “term expansion”68 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Tuned Search What is Tuned Search? • Search terms are defined in a taxonomy and mapped back to specific locations of information (ie. Specific web pages). • Eg. A user searching on a broad term like cell phones would be first pointed to a landing page (a “best bet”), or presented a box of hand-picked links above regular search results.70 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Tuned Search “Best Bets” • The same search using just keyword matching could a have retrieved a list of pages with the words “phone” or “cell” e.g.  Home phones  Cordless phones  12 cell batteries  Etc. • Reading through pages of possible matches is time consuming and frustrating72 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Tuned Search “Best Bets” How Does a Taxonomy Help? • Using the taxonomy categories as landing pages assures that users are strategically directed to the content that is most important. • Best bets are done in conjunction with a taxonomy/thesaurus, not just a list of search terms…  Eg. Circuit City73 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Relevance ranking boost• Can assign more weight to specific metadata fields in the engine‟s ranking algorithms• If search term matches metadata field, higher relative weight than full text hit and boosted rank• E.g. Best Buy boosts taxonomy category• E.g. Motorola could boost the product category content index Metadata field Relative Weighting: 4578 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Leverage the taxonomy terms as metadata - faceted search What is Faceted Search? • Attribute based search (guided navigation) approach to create precise, targeted search results. Each parameter narrows the search result to the most appropriate content.  Also commonly referred to as “advanced searching” or “parametric searching” • Users think they are browsing, but they are actually searching • Allows for multiple navigation schemes based on taxonomy80 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Navigational taxonomy Challenge is there is no “one way” to navigate that is correct. Motorola.com Modems & Mobile phones 2-way radios gateways Unlocked With service Accessories GSM Batteries Headsets Bluetooth headsetsTaxonomy can be a hierarchicalgrouping of navigational nodes Is this the “correct” way?on a web site81 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Navigational taxonomy Motorola.com Motorola.com Modems & Modems & Mobile phones 2-way radios Mobile phones 2-way radios gateways gateways Unlocked Bluetooth Camera Bluetooth With service phones phones GSM accessories Bluetooth Sunglasses Headsets accessories Sunglasses Headsets Or is this one “correct”? Or is this one?82 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Motorola.com United Canada United States Kingdom Enterprise Government Consumers Mobile Portable computers radiosMotorola.com => United States => Government => Portable Radios Motorola.com Mobile Portable Mobile radios computers radios United Canada United States Kingdom Enterprise Government ConsumerMotorola.com => Portable Radios => United States => Government Motorola.com Government Enterprise Consumers Mobile Portable computers radios United Canada United States Kingdom Motorola.com => Government => Portable Radios => United States83 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Navigating with “facets” • Two way radios  Portable “Facet” is a top level category  Fixed in the taxonomy  Mobile  Motorcycle Product type • Vertical market Target document:  Government P = Portable radio  Manufacturing G = United States V = Government  Wholesale retail • Country Vertical market  Canada  United Kingdom  United States Geographic Just three nodes with 5 region terms each could have 3 to the 5th power (243) possible combinations 84 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Taxonomy and Search • Post Search Processing- Search results are narrowed after they are retrieved  Related terms  Clustering  Disambiguation87 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Clustering• Adds context to large result sets• Clusters are similar to facets but based on derived attributes• Derived attributes based on concepts contained in result set mapped to taxonomy89 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Clustering How do I implement Clustering? • Build out your taxonomy, then extract entities from content and categorize based on derived metadata (facets) 91 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Categorizing content Rules-based These documents look similar based on some rule that have created (they Statistical/linguistic contain marketing plans and are These documents look similar about the newest widget) lets put due to an analysis of word them into the same group patterns – lets put them into the same group92 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Clustering based on Taxonomy Slice from a set of Search Results: Tagged Tagged Documents Tagged Documents TaxonomyPath Taxonomy Tagged Documents TreeBuilder “Slice” Tagged Documents Documents 93 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Disambiguation of search resultsWhat is Disambiguation?• If a user enters a broad term (like “mobile”) the taxonomy can return terms that help the user select a more precise terms• Includes multiple approaches:  Term expansion  Complex lookups95 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Disambiguation methods• Show related search terms with check boxes in the search results page.• Show additional search terms as links, perhaps with a prompt - "You might also be interested in:"• Expand the query and show the expanded words in the search box• Expand the query invisibly96 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Disambiguation of search results mobile Mobile data terminals Handheld computers Network Infrastructure Presenting term Mobile switches in multiple Phones Fixed mobile car phones contexts Mobile phones Software applications Mobile applications Two way radios Mobile radios Intelligent video solutions Mobile video enforcer Mobile video sharing MESH Solutions Multi-radio mobile broadband Mobile Computing Mobile application97 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Disambiguation of search resultsHow Do I Implement Disambiguation Methods?• Need to integrate thesaurus with search engine• Can be accomplished through custom frameworks, web services, API calls• Thesaurus values can live inside of search engine, in taxonomy management tool, in spreadsheets or databases or in public sources99 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Taxonomy and Navigation**Taxonomy is not the same as navigation Earley & Associates, Inc. | Classification: CONFIDENTIAL USE, NO REPRINTS Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Applying a taxonomy to navigation We need to improve navigation for our site • A Taxonomy can be used to  Inform navigation (though it is not the same as navigation)  Define metadata and the information architecture of the site.103 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Navigation – Sales Node Sales Tools Analyst Reports … … … Case Studies … Competition … Customer References FAQ‟s Pricing & Licensing White Papers Presentations104 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Navigation – Sales Node Doc Types • Analyst Reports • Assessment • BenchmarksSales Tools • Best Practice Analyst Reports • Brochures … • Campaign … • Case studies • Competition … • Configuration Guide Case Studies • Contracts … • Customer References Competition • Data sheet • Event … • FAQ Customer References • Guides FAQ‟s • License Agreements • Migration Pricing & Licensing • Presentations White Papers • Press Releases Presentations • Price Lists • Quick Reference Guide • White papers105 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Why you will not just “use a folksonomy” • All content is not equal • Higher value content requires more rigor • Social tagging is still immature • May be appropriate for some kinds of content • On systems open to large user groups, esoteric tags which are understood by a only minority of users tend to proliferate  burdens users  decreases system efficiency • Core to folksonomies are the flaws that formal classification systems are designed to eliminate, such as redundancy, misspelling, etc. • Taxonomists/ontologists argue that an agreed-to set of tags enables more efficient indexing and searching of content106 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

earley taxanomic classification of the taxonomy definedearley & associates freycinetia taxonomy deploymentearley & associates inc taxonimic classification of humans taxonomy development processearley & associates needham, massachusets taxonomic and dichotomus taxonomy implementationearley & associates taxonomy taxonomic classification taxonomy iqpcearley & associates, inc taxonomic classification human taxonomy job descriptionearley & associates, inc. taxonomic genus of king cobra taxonomy maintenanceearley & earley associates taxonomic implementation taxonomy managementearley and associates taxonomies of knowledge taxonomy management job titleearley and associates inc taxonomies project roadmap taxonomy management toolsearley and associates seth taxonomist job description taxonomy metadata taxonomy metadataearley and associates taxonomy taxonomy models for project taxonomy & metadata jumpstart - 2007 managementearley assoc taxonomy and false drops taxonomy of global executivesearley associates taxonomy and classifiation examples taxonomy of manearley associates address of animalsearley associates boston taxonomy search taxonomy and metadataearley associates wordmap taxonomy seth early taxonomy and metadata jumpstartearley financial taxonomy structure business taxonomy c organisationearley jumpstart taxonomy classification taxonomy trainingearley taxonomy taxonomy classification charts taxonomy validationearley taxonomy & metadata jumpstart call: taxonomy community of practice taxonomy(2007)managing structured metadata and taxonomies taxonomy consulting taxonomy, mlisearley.com taxonomy creation taxonomy/classification.onlineearly & associates taxonomy creation managementearly and associates

Conclusions • Search engines, no matter how sophisticated, do not obviate the need for taxonomies • Content value in the context of a work process will determine the level of required structure • There is no “one size fits all” • Taxonomy, content strategy and search all work together to improve the findability of content. • Google doesn‟t always get it right…108 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

Questions? Jeff Carr Seth Earley Senior Information Architect & Search CEO Consultant 781-820-8080 780-819-7275 seth@earley.com Follow me on twitter: sethearley jeff@earley.com Connect with me on LinkedIn: www.linkedin.com/in/sethearley110 Copyright © 2011 Earley & Associates, Inc. All Rights Reserved.

http://www.earley.com	740
http://earley.com	4
https://www.earley.com	4

Taxonomies and Search for Chicago SharePoint User Group

by Earley on Mar 04, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

3 Embeds 748

Statistics

Taxonomies and Search for Chicago SharePoint User Group — Presentation Transcript