Enterprise Information Architecture: Because users don't care about your org chart

1. Enterprise Information Architecture Because Users Don’t Care About Your Org Chart March 30, 2010 Merit Network Louis Rosenfeld www.louisrosenfeld.com ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 1

2. About Me Independent IA consultant and blogger (www.louisrosenfeld.com) Founder, Rosenfeld Media, UX publishing house (www.rosenfeldmedia.com) Work primarily with Fortune 500s and other large enterprises Co-author, Information Architecture for the World Wide Web (1998, 2002, 2006) Founder and past director, the Information Architecture Institute (www.iainstitute.org) and User Experience Network (www.uxnet.org) Background in librarianship/information science ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 2

3. Seminar Agenda Welcome/Introduction Topic: Top-Down Navigation Break Topic: Bottom-Up Navigation (content modeling) Exercise #1: Metadata Topic: Bottom-Up Navigation (metadata) Lunch Topic: Search Exercise #2: Search Analytics Break Topic: Research Methods Topic: Governance and Organizational Change ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 3

5. Introduction: IA in one slide Definition: the art and science of structuring, organizing and labeling information to help people find and manage information •  Balances characteristics and needs of users, content and context •  Top down (questions) & bottom up (answers) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 5

6. Introduction: Only one IA rule Pareto’s Principle (“the 80/20 rule”) •  20% of content satisfies 80% of users’ needs •  20% of possible IA options address 80% of content •  20% of IA options address 80% of users’ needs IA’s goal: figure out which 20% No other rules, just guidelines ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 6

8. What an Enterprise Is Large, distributed, decentralized organization made up of multiple business units Distributed •  Functionally in many different “businesses” (e.g., HR vs. communications, or hardware vs. software) •  Geographically Decentralized •  Large degree of authority and responsibility resides in hands of business units in practice (if not officially) •  Business units often own significant infrastructure (technical, staff, expertise) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 8

9. IA and EIA: The differences The “enterprise challenge”: providing centralized access to information in a large, decentralized, distributed environment Information often organized by business function (e.g., “org chart”), not in ways users think Not “textbook” IA; highly dependent on business context ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 9

10. The Challenge of EIA: Competing trends Trend toward autonomy •  Cheap, easy-to-use democratizing technology •  Human tendency toward autonomy Trend toward centralization •  Users’ desire for single-point of access •  Management’s desire to control costs and communications These tend to cancel each other out, getting us nowhere Result: content “silos” and user confusion ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 10

11. Indicators of Problematic EIA: Intranet glitches “How come I didn’t know your department was developing a product similar to ours?” “Why couldn’t we find any relevant case studies to show that important prospect?” “Why do our sales and support staff keep giving our customers inconsistent information?” ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 11

12. Indicators of Problematic EIA: External-facing site glitches “Our customers think we’re still in the widget business; after all these M&As, why don’t they realize that we’ve diversified?” “We have so many great products that go together; why don’t we cross-sell more?” “Customers keep asking for product support through our sales channel; why don’t they use the site’s FAQs and tech support content?” ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 12

15. So How Do We Get There? Let it go •  There is no single solution •  Redemption lies within phased, modular, evolving approaches that respect 80/20 rule Your friends •  Straw men •  Your colleagues and professional networks This seminar provides straw men for •  EIA design •  EIA methods •  EIA team design and governance ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 15

18. Top-Down Challenges Top-down IA •  Anticipates questions that users arrive with •  Provides overview of content, entry points to major navigational approaches Issues •  What do we do about main pages? •  Portals: the answer? •  Other ways to navigate from the top down •  The dangers of taxonomies ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 18

21. Portal Solutions: Why they fail 1/2 Organizational challenges •  Fixation on cosmetic, political •  Inability to enforce style guide changes, portal adoption •  Lack of ownership of centralizing initiatives, or ownership in wrong hands (usually IT) Information architecture challenges •  Taxonomy design required for successful portal tool implementation •  Always harder than people imagine •  Taxonomies break down as they get closer to local content (domains become specialized) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 21

22. Portal Solutions: Why they fail 2/2 Challenges for users •  Portals are shallow (only one or two levels deep) •  Poor interface design •  Users don’t typically personalize More in James Robertson’s “Taking a business- centric approach to portals” (http:// www.steptwo.com.au/papers/ kmc_businessportals/index.html) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 22

23. Top-Down Navigation: Design approaches Main pages Supplementary navigation •  Tables of contents •  Site indices •  Guide pages Taxonomies for browsing •  Varieties: product, business function, topical •  Topic pages ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 23

24. Top-Down Navigation: Main pages Often 80% of discussion of EIA dedicated to main page •  Important real estate •  But there are other important areas •  Navigational pages •  Search interface •  Search results •  Page design (templates, contextual navigation) Divert attention from main pages by creating alternatives, new real estate: supplementary navigation ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 24

25. Top-Down Navigation: Supplementary navigation Examples •  Site maps/TOC •  Site indices Benefits: •  Create new real estate •  Can evolve and drive evolution from org-chart centered design to user-centered design •  Relatively low cost to initially implement Drawbacks: •  Often unwieldy for largest enterprises (not at IBM, Microsoft, failure at Vanguard) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 25

26. Top-Down Navigation: Site maps Condensed versions of site hierarchy •  Hierarchical list of terms and links •  Primarily used for site orientation •  Indirectly cut across subsites by presenting multi- departmental content in one place •  But still usually reflects org chart Alternative plan •  Use site map as test bed for migration to user- centric design •  Apply card sorting exercises on second and third level nodes •  Result may cut across organizational boundaries ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 26

30. Top-Down Navigation: Site indices Flat (or nearly flat) alpha list of terms and links Benefits •  Support orientation and known-item searching •  Alternative “flattened” view of content •  Can unify content across subsites Drawbacks •  Require significant expertise, maintenance •  May not be worth the effort if table of contents and search are already available Specialized indices may be preferable (shorter, narrower domain, focused audience) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 30

32. Site Index: Am. Society of Indexers example Full site index •  @1000 entries for smallish site •  Too large to easily browse •  Replace with search? ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 32

33. Specialized Site Index: CDC example Not a full site index Focuses on health topics •  Narrow domain •  Specialized terminology •  Possibly still too large to browse ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 33

36. Top-Down Navigation: Guides Single page containing selective set of important links embedded in narrative text Address important, common user needs •  Highlight content for a specific audience •  Highlight content on a specific topic •  Explain how to complete a process •  Can work as FAQs (and FAQs can serve as interface to guides) Benefits •  Technically easy to create (single HTML page) •  Cut across departmental subsites •  Gap fillers; complement comprehensive methods of navigation and search •  Can be timely (e.g., news-oriented guides, seasonal guides) •  Minimize political headaches by creating new real estate ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 36

41. Top-Down Navigation: Topic Pages “Selective taxonomy improvement” •  Portions of a taxonomy that expand beyond navigational value •  Help knit together enterprise content deeper down in taxonomy New “real estate” can be used by •  Individual business units (to reduce pressure on main page) or… •  Cross-departmental initiatives ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 41

43. Top-Down Navigation: Taxonomies & portals Can a single taxonomy unify an enterprise site? •  First: can one be built at all? •  Software tools don’t solve problems (see metadata discussion) Approaches •  Multiple taxonomies that each cover a broad swath of enterprise content: audience, subject, task/process, etc. •  “Two-step” approach: 1.  Build shallow, broad taxonomy that will answer “where will I find the information I need?” 2.  Rely on subsite taxonomies to answer “where in this area will I find the information I need?” ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 43

44. Top-Down Navigation: Impacts on the enterprise Potential of “small steps” around which to build more centralized enterprise efforts •  Site map and site index creation and maintenance •  Guide and topic page creation and maintenance •  Large editorial role, minimal technical requirements for both May be preferable to tackle more ambitious areas much later •  Developing and maintaining top-level taxonomy •  Connecting high-level and low-level taxonomies ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 44

46. Top-Down Navigation Takeaways Main pages and portals: Bypass for now, add guides over time Site hierarchy/taxonomy: Start shallow, "simple" (e.g., products); add progressively harder taxonomies (work toward faceted approach) Site map/ToC: Use as a staging ground for a more topical approach Site index: Move from generalized to specialized around a single topic, or augment with frequent search queries/best bets work Guides: Start with a handful, then expand and rotate based on seasonality or other criteria of relevance ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 46

49. Bottom-Up Navigation: The basics Focuses on extracting answers from content •  How do I find my way through this content? •  Where can I go from here? Goals •  Answers “rise to the surface” •  Leverage CMS for reuse and syndication of content across sites and platforms •  Improve contextual navigation •  Increase the effectiveness of search ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 49

50. Content Modeling: The heart of bottom-up navigation Content models •  Used to convey meaning within select, high-value content areas •  Accommodate inter-connectedness Same as data or object modeling? Absolutely not! •  Many distinctions between data and semi- structured text •  Text makes up majority of enterprise sites ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 50

51. Content Modeling: The basics Based on patterns revealed during content inventory and analysis What makes up a content model? 1.  Content objects 2.  Metadata (attributes and values) 3.  Contextual links Applies to multiple levels of granularity •  Content objects •  Individual documents ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 51

54. Content Modeling: Answer some questions What contextual navigation should exist between album pages artist bios these content objects? (see Instone’s “Navigation Stress Test”--http://user- artist descriptions album reviews experience.org/uefiles/ navstress/ ) Are there missing content objects? Can we connect objects automatically? ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 54

55. Content Modeling: Fleshing out the model concert calendar album pages artist descriptions TV listings album reviews discography artist bios ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 55

56. Content Modeling: Connecting with metadata, rules Content …link to other Content Objects… …by leveraging common Objects… Metadata Attributes album page album review, discography, artist Album Name, Artist Name, Label, Release Date… album review album page Album Name, Artist Name, Review Author, Source, Pub Date… discography album review, artist description Artist Name, Album Name, Release Date… artist artist bio, discography, concert Artist Name, Desc Author, description calendar, TV listing Desc Date… artist bio artist description Artist Name, Individual Artist Name… concert artist description Artist Name, Tour, Venue, calendar Date, Time… TV listing artist description Artist Name, Channel, Date, Time… ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 56

57. Content Modeling: Problematic borders concert calendar album pages artist descriptions TV listings album reviews discography artist bios ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 57

58. Content Modeling: When to use Use only for high value content High value content attributes based on users, content, context, including •  High volume •  Highly dynamic •  Consistent structure •  Available metadata •  Available content management infrastructure •  Willing content owners Much content can and will remain outside formal content models ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 58

59. Content Modeling: Steps for developing a model 1.  Determine key audiences (who’s using it?) 2.  Perform content inventory and analysis (what do we have?) 3.  Determine document and object types (what are the objects?) 4.  Determine metadata classes (what are the objects about?) 5.  Determine contextual linking rules (where do the objects lead us to next?) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 59

60. Content Modeling: Content object types 1/2 List known object types For each audience: •  Are there types that don’t fit? •  Examples: company executive bios, Q&A columns •  Venue reviews may be part of a separate content model ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 60

61. Content Modeling: Content object types 2/2 For each audience (continued): •  Gap analysis: are there types missing that users might expect? •  Examples: Gig reviews, Buy the CD, Links to music in the same genre •  Which types are most important to each audience? •  Fans of the band: Interviews with the band members •  Casual listener: Samples of the CD tracks ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 61

62. Content Modeling: Metadata 1/2 Determine which objects would benefit from metadata Develop three types of metadata •  Descriptive •  Intrinsic •  Administrative ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 62

63. Content Modeling: Metadata 2/2 Aim to balance utility and cost •  Answer most important questions: who, what, where, why, when, how? •  Cost-benefit analysis •  Development and maintenance costs of controlled vocabularies/thesauri •  Ability of in-house staff to apply properly ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 63

64. Content Modeling: Contextual linking rules Are there specific objects for which these questions arise again and again? •  Where would I go from here? •  What would I want to do next? •  How would I learn more? You have a rule if •  The questions apply consistently •  The answers work consistently •  Metadata can be leveraged to connect questions and answers Unidirectional links or bidirectional? ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 64

65. Content Modeling: Impacts on the enterprise Content models are a means for tying together content across business unit boundaries Content modeling is modular; over time, content models can be connected across the enterprise Major benefits to users who get beyond main page Can help justify CMS investments Not all content areas and owners are appropriate to work with ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 65

67. CMS Selection: EIA needs Support metadata management (Interwoven) Support shared metadata workflow •  Author creation/submission/tagging (distributed) •  Editorial tagging (centralized) •  Editorial review (centralized) Ability to support contextual linking logic ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 67

68. Metadata: What is metadata? Data about data Information which describes a document, a file or a CD Common metadata •  CD information: title, composer, artist, date •  MS Word document properties: time last saved, company, author ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 68

69. Metadata: Three types 1.  Intrinsic: metadata that an object holds about itself (e.g., file name or size) 2.  Descriptive: metadata that describes the object (e.g., subject, title, or audience) 3.  Administrative: metadata used to manage the object (e.g., time last saved, review date, owner) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 69

70. Metadata: Common sources Vocabularies from other parts of your organization (e.g., research library) Competitors Commercial sources (see www.taxonomywarehouse.com) Your site’s users •  Search analytics •  Folksonomies •  User studies (e.g., free listing, card sorting) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 70

71. Metadata: Value for the Enterprise 1/2 Search: cluster or filter the search by metadata, like title or keyword Browse: create topical indexes by aggregating pages with the same metadata Personalization and customization: show content to an employee based on their role or position in the company, e.g. engineer or manager ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 71

72. Metadata: Value for the Enterprise 2/2 Contextual linking: create relationships between individual or classes of content objects (e.g., cross-marketing on llbean.com) The purpose is to connect •  Content to content •  Users to content To provide value, metadata requires consistency (structural and semantic) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 72

74. Metadata: Scaling problems Barriers to enterprise metadata development: •  Volume of metadata vocabs./silos •  Complexity of semantic relationships (beyond synonyms) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 74

75. Metadata attributes: Easy to difficult 1/2 Level of Metadata Comments Difficulty Attribute Easy Business unit These are typically already names available and standardized Easy to Chronology Variations in formats (e.g., Moderate 12/31/07 versus 31/12/07) usually can be addressed by software Moderate to Place names Although many standards exist Difficult (e.g., state abbreviations and postal codes), many enterprises (and their business units) use custom terms for regions (such as sales territories) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 75

76. Metadata attributes: Easy to difficult 2/2 Level of Metadata Comments Difficulty Attribute Moderate to Product Product granularity can vary Difficult names greatly; marketing may think in terms of product families; sales in terms of items with SKU numbers, and support in terms of product parts that can be sold individually Difficult Audiences Audiences, such as customers or types of employees, vary widely from unit to unit Difficult Topics The most ambiguous type of metadata; difficult for individuals, much less business units, to come to agreement on topical metadata ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 76

77. Metadata: Structural consistency Standard formats and approaches enable interoperability, which enables sharing of metadata. Examples •  RDF (Resource Description Format) •  Topic Maps •  Dublin Core •  OAI (Open Archives Initiative) Sources •  Academia/scholarly publishing world •  Little from data management world ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 77

78. Metadata: RDF (Resource Description Format) A syntax for expressing semantic relationships Basic components 1.  Resource 3. Value 2.  Property type 4. Property 2 1 3 4 From Andy Powell: http://www.ukoln.ac.uk/metadata/presentations/ukolug98/paper/intro.html ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 78

79. Metadata: Topic Maps Potential syntax for content modeling, semantic webs Most simply, made up of associations topics (e.g., “Lucca”, “Italy”), occurrences (e.g., “map”, “book”), and associations (e.g., topics “…is in…”, “…written by…”) Source: Tao of Topic Maps, Steve Pepper (http://www.ontopia.net/ topicmaps/materials/ tao.html) occurrences ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 79

80. Metadata: The Dublin Core A schema for expressing semantic relationships Can use HTML or RDF syntax Useful tool (or model) for creating document surrogates (e.g., Best Bet records) A standard, but not a religious one •  Selecting fewer attributes may be a necessity in enterprise environment •  Attribute review can be useful as an enterprise- wide exercise ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 80

81. Metadata: Dublin Core elements 1/2 Title: A name given to the resource Creator: An entity primarily responsible for making the content of the resource Subject: A topic of the content of the resource Description: An account of the content of the resource Publisher: An entity responsible for making the resource available Contributor: An entity responsible for making contributions to the content of the resource Date: A date of an event in the lifecycle of the resource ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 81

82. Metadata: Dublin Core elements 2/2 Type: The nature or genre of the content of the resource Format: The physical or digital manifestation of the resource Identifier: An unambiguous reference to the resource within a given context Source: A Reference to a resource from which the present resource is derived Language: A language of the intellectual content of the resource Relation: A reference to a related resource Coverage: The extent or scope of the content of the resource Rights: Information about rights held in and over the resource ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 82

83. Metadata: Dublin Core in HTML Dublin Core elements identified with “DC” prefix From Andy Powell: http://www.ukoln.ac.uk/metadata/presentations/ukolug98/paper/intro.html ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 83

84. Metadata: Dublin Core and RDF Syntax and schema combination is useful But where are the metadata values? From Andy Powell: http://www.ukoln.ac.uk/metadata/presentations/ukolug98/paper/intro.html ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 84

85. Metadata: OAI and metadata harvesting OAI: Open Archives Initiative •  Comes from academic publishing world •  Provides means for central registration of “confederate repositories” •  Repositories use Dublin Core; requests between service and data providers via HTTP; replies (results) encoded in XML Metadata harvesting •  Enables improved searching across compliant distributed repositories •  Does not address semantic merging of metadata (i.e., vocabulary control) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 85

86. Metadata: Semantic consistency 1/2 Provided through controlled vocabularies. What is a controlled vocabulary? •  A list of preferred and variant terms •  A subset of natural language Why control vocabulary? •  Language is Ambiguous •  Synonyms, homonyms, antonyms, contronyms, etc. (e.g., truck, lorry, semi, pickup, UTE) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 86

87. Metadata: Semantic consistency 2/2 Users Example Personal Digital Assistant Synonyms Handheld Computer "Alternate" Spellings Persenal Digitel Asistent Abbreviations / Acronyms PDA Communication Chasm Broader Terms Wireless, Computers Narrower Terms PalmPilot, PocketPC Related Terms WindowsCE, Cell Phones Documents and Applications Control vocabulary…so your users don’t have to! ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 87

88. Metadata: Semantic relationships Three types 1.  Equivalence: Variant terms with same meaning (e.g., abbreviations and synonyms) 2.  Hierarchical: Broader term, narrower term relationships 3.  Associative: Related terms that are related to each other ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 88

89. Metadata: Levels of control (Vocabularies) Synonym Authority Classification Thesauri Rings Files Schemes Simple Complex Equivalence Hierarchical Associative (Relationships) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 89

90. Metadata semantic relationships: Hard to hardest Level of Type of Examples Difficulty Relationship Hard Synonymous Synonym rings and authority lists Harder Hierarchical Classification schemes Hardest Associative Thesauri ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 90

91. Metadata: Synonym rings Used in many search engines to expand the number of results Words that are similar to each other are linked together Example for a multinational company •  Annual leave (Australia), the holidays (US), public holidays (Australia, US), vacation (US), bank holidays (UK), holiday (Australia and UK), personal leave (all) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 91

92. Metadata: Authority files Pick list of the authorized words to use in a field Can have some equivalence relationships Example using authors •  Poe, Edgar Allan--USE FOR Poe, E.A. •  Poe, E.A.--USE Poe, Edgar Allan ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 92

93. Metadata: Classification schemes Classification •  Systematic arrangement of knowledge, usually hierarchical •  Placement of objects into a scheme which makes sense to the user and relates them to other objects Two types of classification schemes •  Enumerative classification: hierarchical organization into which objects are placed •  Faceted classification: organization by facets or attributes that describe the object ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 93

94. Metadata: Enumerative classification Really good to classify small numbers of objects or objects that can live in only one place Provides good browsing structure Can be polyhierarchical, where objects live in many places Best known: the taxonomy of life, Dewey Decimal Classification, Library of Congress Classification Most familiar on the Web: Yahoo!, Open Directory ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 94

96. Metadata: Faceted classification 1/2 Describes the object with numerous facets or attributes Each facet could have a separate controlled vocabulary of its own Can mix and match the facets to create a browsing structure Easier to manage the controlled vocabularies ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 96

97. Metadata: Faceted classification 2/2 Facets for a roast chicken recipe •  Preparation: Roast / bake •  Main ingredient: Chicken •  Course: Main dish Drawbacks of faceted classification •  Too many facets attached to an object can make indexing hard to do •  Browsing facets may not be as clear as browsing a hierarchy; many paths to the same object ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 97

100. Metadata: What is a thesaurus? Traditional use •  Dictionary of synonyms (Roget’s) •  From one word to many words Information retrieval context •  A controlled vocabulary in which equivalence, hierarchical, and associative relationships are identified for purposes of improved retrieval •  From many words to one word ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 100

102. Enterprise Metadata: Challenges Two barriers to enterprise metadata 1.  Interoperability (structural) 2.  Merging enables controlled vocabularies to work as a whole (semantic) Interoperability must come before merging (merging requires knowledge of which vocabularies to merge) Few standards in use ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 102

103. Enterprise Metadata: Structural approaches If directly marking up documents, this approach is probably impractical in the enterprise Better uses: •  Limited high value documents (e.g., content models) •  Document surrogates (e.g., Best Bet records) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 103

104. Enterprise Metadata: Merging vocabularies Extremely difficult, and currently rare Mostly found in libraries, academia, scholarly publishing, and other resource-poor environments Examples, hard to hardest •  Cross-walking vocabularies •  Switching vocabularies •  Meta-thesaurus •  Single thesaurus ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 104

105. Merging Vocabularies: Vocabulary cross-walking Map terms peer-to-peer between individual vocabularies •  Primarily handles synonyms, not relationships •  Can be handled manually or through automated means (pattern-matching) Doesn’t scale well beyond two or three vocabularies ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 105

106. Merging Vocabularies: Switching vocabulary A single vocabulary that maps to existing vocabularies (primarily synonyms) Similar to cross-walking, but better at handling translation when there are more than two or three vocabularies to connect ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 106

107. Merging Vocabularies: Meta-thesaurus A switching vocabulary which also includes thesaural relationships (essentially a thesaurus of thesauri) Example: National Library of Medicine’s UMLS (Unified Medical Language System) •  Merges over 100 vocabularies •  Describes fairly homogeneous domain (medical literature) for fairly homogeneous audience (health science professionals) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 107

109. Metadata: What is metadata? Data about data Information which describes a document, a file or a CD Common metadata •  CD information: title, composer, artist, date •  MS Word document properties: time last saved, company, author ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 109

110. Metadata: Three types 1.  Intrinsic: metadata that an object holds about itself (e.g., file name or size) 2.  Descriptive: metadata that describes the object (e.g., subject, title, or audience) 3.  Administrative: metadata used to manage the object (e.g., time last saved, review date, owner) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 110

111. Metadata: Common sources Vocabularies from other parts of your organization (e.g., research library) Competitors Commercial sources (see www.taxonomywarehouse.com) Your site’s users •  Search analytics •  Folksonomies •  User studies (e.g., free listing, card sorting) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 111

113. Metadata: Scaling problems Barriers to enterprise metadata development: •  Volume of metadata vocabs./silos •  Complexity of semantic relationships (beyond synonyms) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 113

114. Metadata in the Large Org: Challenges Two barriers to enterprise metadata 1.  Interoperability (structural) 2.  Merging enables controlled vocabularies to work as a whole (semantic) Interoperability must come before merging (which requires knowledge of which vocabularies to merge) Few standards in use ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 114

115. Metadata attributes: Easy to difficult 1/2 Level of Metadata Comments Difficulty Attribute Easy Business unit These are typically already names available and standardized Easy to Chronology Variations in formats (e.g., Moderate 12/31/07 versus 31/12/07) usually can be addressed by software Moderate to Place names Although many standards exist Difficult (e.g., state abbreviations and postal codes), many enterprises (and their business units) use custom terms for regions (such as sales territories) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 115

116. Metadata attributes: Easy to difficult 2/2 Level of Metadata Comments Difficulty Attribute Moderate to Product Product granularity can vary Difficult names greatly; marketing may think in terms of product families; sales in terms of items with SKU numbers, and support in terms of product parts that can be sold individually Difficult Audiences Audiences, such as customers or types of employees, vary widely from unit to unit Difficult Topics The most ambiguous type of metadata; difficult for individuals, much less business units, to come to agreement on topical metadata ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 116

117. Metadata: Levels of control (Vocabularies) Synonym Authority Classification Thesauri Rings Files Schemes Simple Complex Equivalence Hierarchical Associative (Relationships) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 117

118. Metadata semantic relationships: Hard to hardest Level of Type of Examples Difficulty Relationship Hard Synonymous Synonym rings and authority lists Harder Hierarchical Classification schemes Hardest Associative Thesauri ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 118

119. Metadata: Strategy for large orgs 1/2 Coordinate to ensure: •  Structural interoperability from the start •  Semantic mergability over time •  Vocabulary control and maintenance through both manual and automated means •  A workflow model and policies to support: •  Decentralized tagging and vocabulary updating (through suggestions of new terms) •  Centralized review and maintenance ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 119

120. Enterprise Metadata: Strategy for large orgs 2/2 “Serious metadata” is beyond the means of most enterprises •  Encourage local (e.g., departmental) vocabulary development •  Provides organizational learning and local benefit •  Enterprise-wide, start with “easier” vocabularies; work your way to harder ones over time; suggested sequence: 1.  Business functions 2.  Products 3.  Topics ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 120

122. Bottom-Up Navigation Takeaways 1/3 Content models •  Use to support contextual navigation •  Apply only to homogenous, high-value content •  Won't transfer easily across silos and will require significant metadata development ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 122

123. Bottom-Up Navigation Takeaways 2/3 Metadata development •  Distinguish attributes (and structural interoperability) from values (and semantic merging) •  Costs and value both increase as these increase: •  Complexity of relationships between terms (equivalence=>hierarchical=>associative) •  Level of control (synonym rings=>authority files=>classification schemes=>thesauri) •  Think small: facets instead of a single taxonomy ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 123

124. Bottom-Up Navigation Takeaways 3/3 Metadata tagging •  Make choices based on actual needs (e.g., content models) rather than exhaustive indexing •  Consider costs of application and upkeep •  Need for professional expertise •  Metadata is a moving target that matches other moving targets (users and content) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 124

126. EIA and Search Search systems are a natural enterprise IA tool •  Automated •  Crawls what you tell it to •  Doesn’t care about politics Problems with shrink-wrapped search tools •  Default settings, IT ownership minimize customization to fit the enterprise’s needs •  Results often not relevant, poorly presented Customization is the answer •  Within the realm of your team’s abilities •  … and if IT will allow it! ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 126

128. Enterprise Search Design: Potential improvements Our focus: 1.  Clear interface 2.  Enhanced queries 3.  Improved results (relevance & presentation) Basic search system components ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 128

130. Search Interface Design: The “Box” The “Box” unifies IBM.com Consistent: •  Placement •  Design •  Labeling •  Functionality ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 130

132. Search Interface Design: The role of “advanced search” 1/2 Not a likely starting point for users who are searching Continued… ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 132

133. Search Interface Design: The role of “advanced search” 2/2 Suggestions •  Use for specialized interfaces •  Reposition as “Revise Search” •  Don’t bother ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 133

135. Search Interface and Queries: Functionality and visibility Hide functionality? Consider the “Google Effect,” human nature and the LCD Don’t hide it? •  Not if users expect it •  Legacy experience (e.g., Lexis-Nexis users) •  Specialization (e.g., patent searchers) •  Not if content allows/requires it •  Specialized content and applications (e.g., staff directory) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 135

136. The Query: Query language considerations Natural language •  Usually don’t show up in search logs •  Low priority, but nice to support Operators (Booleans, proximity, wild cards) •  Booleans: use default “AND” for multi-term queries •  Less forgiving than treating as phrase, more selective than “OR” •  Most retrieval algorithms will find results for just one term •  Rely on other approaches (e.g., filtering, clustering, Best Bets) to reduce search results overload •  Low priority: Proximity operators (e.g., “enterprise (W3) architecture”), wild cards (e.g., “wom*n”) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 136

137. The Query: Query building considerations Large potential benefits to improving “intelligence” behind search queries •  Adding semantic richness to queries allows for stronger searches without “touching” content •  Overrides “enterprise bias” embedded in content •  A centralized (enterprise-wide) process Query building approaches •  Spell checking: can be automated •  Stemming: can be automated •  Concept searching: requires manual effort •  Synonyms (via thesaurus): requires manual effort, but no need to be comprehensive ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 137

142. Enterprise Search Interface: Guidelines Hide functionality on initial enterprise-wide search Cast the net widely: rely on query builders to generate larger, higher quality result sets Use filtering/clustering to narrow Use Best Bets to ensure strong initial results ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 142

143. Individual Search Results: Goals Enable users to quickly understand something about each document represented That “something”: confirm that a known-item has been found, or distinguish from other results Align to searching behaviors (determined through user testing, persona/scenario analysis, site search analytics) •  Known-item •  Open-ended/exploratory •  Comprehensive research ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 143

144. Individual Search Results: Approaches Basic approaches •  Document titling •  Displaying appropriate elements for each result These approaches have value in any context, but especially useful in enterprise setting ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 144

145. Document Titling: DaimlerChrysler example What do these document titles tell you? And what do they tell you about DaimlerChrysler? ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 145

147. Displaying Appropriate Elements: 1) Determine common elements Develop table of available elements (including metadata) for disparate documents and records •  Comes after content inventory and analysis Develop table of common elements •  Collapse similar elements (e.g., creator derived from author, artist, source…) •  Consider Dublin Core as model •  Include bare minimum elements (e.g., title and description) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 147

148. Displaying Appropriate Elements: 2) Select appropriate elements Choose common elements which match most common searching behaviors •  Known-item •  Open-ended •  Comprehensive research •  Etc. Considerations •  Which components are decision or action based? •  Which components are of informational value only? Display these elements for each search result ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 148

149. Step #1: common content elements Step #2: select elements to display Step #1 Title Description Creator Topic Date Tech. Report Y Y Y Y Y Policy Y N Y Y Y Product Y Y N Y N Sheet FAQ Y N N Y N Step #2 Title Description Creator Topic Date Known-Item Y N Y N Y Open-Ended Y Y N Y Y ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 149

150. Individual Search Results: Columbia University example Long display for open- …shorter display for ended searchers… known-item searchers ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 150

151. Individual Search Results: What happens next? Augment with “next step” actions per result •  Open in separate window •  Get more like this •  Print •  Save •  Email Determine next steps through contextual inquiry ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 151

152. Presenting Search Result Groups: Ranked results Difficulties with relevance ranking •  Depends on consistent elements across documents •  Term frequency-dependent approaches create an “apples and oranges effect” on ranking •  Google effect: benefits of popularity make less sense in enterprise context than in open web Consider alternatives •  Clustering and filtering •  Manually-derived results (aka “Best Bets”) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 152

153. Presenting Search Result Groups: Clustering & filtering “Our user studies show that all Category interfaces were more effective than List interfaces even when lists were augmented with category names for each result” —Dumais, Cutrell & Chen list results clustered results Consider using clustered results rather than list results ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 153

154. Presenting Search Result Groups: Methods of clustering and filtering Use existing metadata and other distinctions (easier) •  Document type (via file format or CMS) •  Source (author, publisher, and business unit) •  Date (creation date? publication date? last update?) •  Security setting (via login, cookies) Use explicit metadata (harder) •  Language •  Product •  Audience •  Subject/topic ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 154

160. “Best Bets”: By popular demand Recommended links •  Ensure useful results for top X (50? 100?) most popular search queries •  Useful resources for each popular query are manually determined (guided by documented logic) •  Useful resources manually linked to popular queries; automatically displayed in result page ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 160

161. “Best Bets” Example: BBC Logic for BBC Best Bets •  Is query a country name? (yes) •  Then do we have a country profile? (yes) •  Then do we have a language service? (yes) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 161

162. “Best Bets”: In the enterprise context Who does the work? •  Difficult to “assign” queries to different business units (e.g., “computing” means different things to different business units) •  Can serve as impetus for centralized effort Operational requirements •  Logic based on users’ needs (e.g., queries) and business rules •  Policy that assigns responsibilities, negotiates conflicts (e.g., who owns “computing”) Opportunity to align Best Bets to user-centric divisions (e.g., by audience: a “computing” best bet for researchers, another for IT staff) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 162

163. Enterprise Search: Impacts on the enterprise Designs •  Simple query builders (spell checker, stemming) •  Search-enhancing thesaurus Policies •  Best Bets design and selection •  Style guide (result titling, search interface implementation) Staffing needs •  Content inventory and analysis •  Interface design •  Work with IT on spidering, configuration issues •  Ongoing site search analytics •  Editorial (e.g., Best Bets creation) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 163

164. Search Tool Selection: EIA needs 1/2 To basic evaluation criteria (from SearchTools.com)… •  Price •  Platform •  Capacity •  Ease of installation •  Maintenance ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 164

165. Search Tool Selection: EIA needs 2/2 …add: •  Ability to crawl deep/invisible web •  Ability to crawl multiple file formats •  Ability to crawl secure content •  API for customizing search results •  Work with CMS •  Duplicate result detection/removal •  Ability to tweak algorithms for results retrieval and presentation •  Federated search (merge results from multiple search engines/data sources) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 165

167. Enterprise Search Takeaways Search interface and queries •  Consistent location and behavior •  Keep as simple as possible •  Use "refine search" interface instead of "advanced search" •  Soup up users’ queries (e.g., spell checking) Search results •  Feature appropriate elements for individual results •  Consider clustered results, especially if explicit, topical metadata are available •  Best bets results for top X common queries ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 167

169. EIA Research Methods: Learn about these three areas Content, users and context drive: •  IA research •  IA design •  IA staffing •  IA education •  …and everything else ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 169

170. EIA Research Methods: Sampling challenges How do you achieve representative samples in the face of these difficulties? •  Awareness: Who and what are out there? •  Volume: How much is there? Can we cover it all? •  Costs: Can we afford to investigate at this order of magnitude? •  Politics: Who will work with us? And who will try to get in the way? ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 170

171. EIA Research Methods: Reliance on alternative techniques Standard techniques may not work in enterprise settings Alternatives often incorporate traditional methods and new technologies •  Web-based surveys (e.g., SurveyMonkey) •  Remote contextual inquiry and task analysis (via WebEx) •  Web-based “card” sorting (e.g., WebSort) •  Log analysis tools (e.g., WebTrends) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 171

172. EIA Research Methods: A closer look Content-oriented methods •  Content inventories •  Content value tiers Context-oriented methods •  Sampling stakeholders •  Departmental scorecard User-oriented methods •  2-D scorecard •  Automated metadata development •  Freelisting •  Site search analytics ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 172

173. Content Inventory: Enterprise context Issues •  Even greater sampling challenges •  Content research is even more critical: serves as a cross-departmental exercise Approaches •  Balancing breadth and depth •  Talking to the right people •  Value-driven ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 173

174. Multidimensional Inventory: Incomplete yet rich EIA requires balanced, iterative sampling (where CMS implementation may require exhaustive inventory) Balance scope (breadth) with granularity (depth) Extend inventory to all discernible areas of content, functionality: •  Portals and subsites •  Application (including search systems) •  Supplemental navigation (site maps, indices, guides) •  Major taxonomies •  Structured databases •  Existing content models •  Stakeholders ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 174

175. Content Migration Strategy: Value Tier Approach Determine value tiers of content quality that make sense given your users/content/context •  Answer “what content is important to the enterprise?” •  Help determine what to add, maintain, delete How to do it? 1. Prioritize and weight quality criteria 2. Rate content areas 3. Cluster into tiers 4. Score content areas while performing content analysis ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 175

176. Value Tier Approach: Potential quality criteria Select appropriate criteria for your business context, users, and content •  Authority •  Strategic value •  Currency •  Usability •  Popularity/usage •  Feasibility (i.e., “enlightened” content owners) •  Presence of quality existing metadata ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 176

179. Assessing Stakeholders: What to learn from them Strategic •  Understanding of business mission and goals, and fit with larger enterprise mission and goals •  Theory •  Practice •  Culture: tilt toward centralization or autonomy •  Political entanglements Practical •  Staff: IT, IA, design, authoring, editorial, usability, other UX (user experience) •  Resources: budget, content, captive audiences •  Technologies: search, portal, CMS ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 179

180. Stakeholder Interviews: Triangulate your sample Org chart: business unit representatives •  Will provide strategic overview of content and whom it serves •  May have some knowledge of content •  More importantly, they know people who do in their units •  Additionally, political value in talking with unit reps Functional/audience-centered •  Subject Matter Experts (SMEs): represent power users; valuable for pointing out content that addresses major information needs •  Audience advocates (e.g., switchboard operators): can describe content with high volume usage ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 180

181. Stakeholder Interviews: Finding the low-hanging fruit Assessment should reveal degree of “enlightenment” •  Early adopters •  Successful track records visible within the enterprise •  Understand/have experience with enterprise-wide initiatives •  Willingness to benefit the enterprise as a whole •  They just plain “get it” You’ve got to play to win: lack of interest and availability mean loss of influence ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 181

182. Stakeholder Interviews: Indicators of enlightenment Technology assessment: who has/uses the “classic 3”? •  Portal •  Search engine •  CMS Staff review: who has relevant skills/expertise on their staff? IA review: what areas of enterprise site have strong architectures? These areas may indicate redundant costs, targets for centralization ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 182

183. Involving Stakeholders: Departmental Report Card Information Architecture Dept. Dept. Dept. Heuristic 1 2 3 Supports orientation B- B B Supports known-item searching A C+ C Supports associative learning B C C Supports comprehensive research A B+ B Passes “navigation stress test” C F C+ … … … … ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 183

184. “Safe” User Sampling: The 2D Scorecard Combines alternative, apolitical methods for determining segments to sample, e.g.: •  Role-based segmentation •  Demographic segmentation Distracts stakeholders from “org chart- itis,” to purify sampling Enables evaluation methods (e.g., task analysis, card sorting) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 184

185. The 2D Scorecard: Role-based segmentation Roles cut across political boundaries •  Profile core enterprise-wide business functions •  Why does the enterprise exist? •  Examples: Sell products, B2B or B2C activities, manufacture products, inform opinion, etc. •  Determine major “actors” in each process ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 185

186. The 2D Scorecard: Demographic segmentation Standard, familiar measure; also cuts across political boundaries •  Gender •  Geography •  Age •  Income level •  Education level Your marketing department probably has this data already ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 186

187. The 2D Scorecard: Combining roles & demographics TEST Demo. Demo. Demo. Demo. SAMPLE Profile Profile Profile Profile TOTAL SIZE A B C D Role 1 1 3 3 2 9 Role 2 2 2 1 1 6 Role 3 3 4 2 1 10 Role 4 0 3 4 0 7 TOTAL 6 12 10 4 32 ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 187

188. The 2D Scorecard: Incorporating contextual bias Role/demographic “scorecard” is pure •  Serves as a structure that doesn’t have to change substantially •  But how to incorporate stakeholder bias? Stakeholder bias can be accommodated •  Poll/interview stakeholders to determine how cell values should change •  Axes and totals stay mostly the same •  Distraction is our friend ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 188

189. The 2D Scorecard: After stakeholder input TEST Demo. Demo. Demo. Demo. SAMPLE Profile Profile Profile Profile TOTAL SIZE A B C D Role 1 1 2 5 1 9 Role 2 1 1 3 1 6 Role 3 3 4 2 1 10 Role 4 0 3 3 1 7 TOTAL 5 10 13 4 32 ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 189

190. Maintaining a User Pool: Build your own for fun and power Through automated surveys, lower level information architect built an enterprise- wide pool of 1,500 users •  Prescreened by demographics and skills •  Provided him with substantial leverage with others who wanted access to users •  He just got there first and did the obvious More information: http://louisrosenfeld.com/ home/bloug_archive/000408.html ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 190

191. Metadata Development: Conventional techniques Techniques •  Open card-sorting to gather terms •  Closed card-sorting to validate terms •  Can be difficult to carry out in enterprise environment (scope of vocabulary, subject sampling) Modifications for enterprise setting •  Use remote tools (e.g. IBM’s EZsort) •  Apply in “stepped” mode: test subsections of taxonomy separately •  Drawback: lack of physical cards may diminish value of data ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 191

192. Metadata Development: Classification scheme analysis Review existing schemes, looking for: •  Duplication of domain •  Overlapping domains •  Consistency or lack thereof Can some vocabularies be reused? Improved? Eliminated? ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 192

193. Automated Metadata Development: Two classes of tools Auto-categorization tools •  Can leverage pattern-matching and cluster- analysis algorithms to automatically generate categories (e.g., Autonomy, Interwoven) •  Can also use rules (i.e., concepts) to generate categories (e.g., Inktomi, Verity, Entrieva/Semio) Auto-classification tools •  Apply indexing to existing categories •  Require controlled vocabularies (generally manually-created) to index content ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 193

194. Automated Metadata Development: Pros and cons Benefits •  Apolitical applications that disregard org chart •  May be a necessary evil in a large enterprise environment Drawbacks •  Limited value in heterogeneous, multi- domain environment •  Perform better with rich text, not so good with database records and other brief documents ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 194

195. Automated Metadata Development: Semio example “E-commerce”: A human would collapse many of these categories At best, an 80% solution; none truly “automated” •  Significant manual proofing of the 80% of content indexed •  Significant manual indexing of the 20% not indexed ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 195

196. Finding Metadata: Free listing Simple technique: •  “List all of the terms you associate with ______” •  Perform pair analysis (co-occurrence) on results Benefits •  Harvests terms associated with a concept or domain •  Can be done in survey form with many subjects, multiple audiences •  Supports card sorting •  Less useful for structuring relationships between terms •  Possible alternative to site search analytics ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 196

197. The Zipf Curve: Consistent and telling Zipf distribution from Michigan State University search logs (derived from site search analytics) From http://netfact.com/rww/write/searcher/rww-searcher-msukeywords-searchdist-apr-jul2002.gif ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 197

199. Site Search Analytics: What does this data tell us? Keywords: focis; 0; 11/26/01 12:57 PM; XXX.XXX.XXX.2 Keywords: focus; 167; 11/26/01 12:59 PM; XXX.XXX.XXX.2 Keywords: focus pricing; 12; 11/26/01 1:02 PM; XXX.XXX.XXX.2 Keywords: discounts for college students; 0; 11/26/01 3:35 PM; XXX.XXX.XXX.59 Keywords: student discounts; 3; 11/26/01 3:35 PM; XXX.XXX.XXX.59 Keywords: ford or mercury; 500; 11/26/01 3:35 PM; XXX.XXX.XXX. 126 Keywords: (ford or mercury) and dealers; 73; 11/26/01 3:36 PM; XXX.XXX.XXX.126 Keywords: lorry; 0; 11/26/01 3:36 PM; XXX.XXX.XXX.36 Keywords: “safety ratings”; 3; 11/26/01 3:36 PM; XXX.XXX.XXX.55 Keywords: safety; 389; 11/26/01 3:36 PM; XXX.XXX.XXX.55 Keywords: seatbelts; 2; 11/26/01 3:37 PM; XXX.XXX.XXX.55 Keywords: seat belts; 33; 11/26/01 3:37 PM; XXX.XXX.XXX.55 ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 199

200. Site Search Analytics: Instructions Sort and count queries Identify and group similar queries (e.g., “cell phones” and “mobile phones”) Understand users’ query syntax (e.g., use of single or multiple terms, Boolean operators) and semantics (e.g., use of lay or professional terms) Determine most common queries •  Identify content gaps through 0 result queries •  Build “Best Bets” for common queries •  Map common queries to audiences through IP or login analysis ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 200

201. Site Search Analytics: Benefits for interface development Identifies “dead end” points (e.g., 0 hits, 2000 hits) where assistance could be added (e.g., revise search, browsing alternative) Syntax of queries informs selection of search features to expose (e.g., use of Boolean operators, fielded searching) …OR… ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 201

202. Site Search Analytics: Benefits for metadata development Provides a source of terms for the creation of vocabularies Provides a sense of how needs are expressed •  Jargon (e.g., “lorry” vs. “truck”) •  Syntax (e.g., Boolean, natural language, keyword) Informs decisions on which vocabularies to develop/implement (e.g., thesaurus, spell- checker) ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 202

203. Site Search Analytics: Benefits for content analysis Identifies content that can’t be found Identifies content gaps Creation of “Best Bets” to address common queries ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 203

204. Site Search Analytics: Pros and cons Benefits •  Data is real, comprehensive, available (usually) •  High volume •  Can track sessions •  Non-intrusive Drawbacks •  Lack of good commercial analysis tools •  Lack of standards makes it difficult to merge multiple search logs (not to mention server logs) •  More difficult to merge with other logs (e.g. server) •  Doesn’t tell you why users did what they did ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 204

205. Site Search Analytics: Enterprise context Makes case for EIA; usually demonstrates that users are requesting things that aren’t tied to departmental divisions (e.g., policies, products) Informs “Best Bets” Informs synonym creation Limited value if not analyzing merged logs ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 205

206. EIA Research Methods Takeaways Challenges •  Many traditional methods can be adapted to the enterprise environment •  But sampling, geography, volume and politics force a less scientific, more pragmatic approach •  Also force greater reliance on automated tools We need new methods •  Focus on minimizing politics and geographic distribution •  Most are untested •  Information architects need to be willing to experiment, innovate, and live with mistakes ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 206

208. EIA and the Enterprise: Phased, modular model Phasing is not just about roll-out and timing Should be overarching philosophy for EIA initiatives •  We can phase in whom we work with •  We can phase in whom we hire to do EIA work •  We can modularize what types of EIA we do •  We can phase in what degree of centralization we can support ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 208

209. Why a Phased Model? Because mandates don’t work “Just do it!”… •  …all (e.g., all subsites) •  …now (e.g., in 3-6 months) •  …with few resources and people (e.g., one sad webmaster) •  …in a way that minimizes organizational learning (e.g., hire an outside consultant or agency) Results of the mandated “solution”: completely cosmetic, top-down information architecture ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 209

210. The EIA Framework Seven issues 1.  EIA governance: how the work and staff are structured 2.  EIA services: how work gets done in an enterprise environment 3.  EIA staffing: who handles strategic and tactical efforts 4.  EIA funding model: how it gets paid for 5.  EIA marketing and communications: how it gets adopted by the enterprise 6.  EIA workflow: how it gets maintained 7.  EIA design and timing: what gets created and when ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 210

211. The EIA Framework Critical goals Re-balance the enterprise’s in-house IA expertise to support an appropriate degree of centralization Enable slow, scaleable, sustainable growth of internal EIA expertise Create ownership/maintenance mechanism for enterprise-wide aspects of IA (currently orphaned) Ensure institutional knowledge is retained ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 211

212. EIA Governance: Questions What sort of individuals or group should be responsible for the EIA? Where should they be located within the organization? How should they address strategic issues? Tactical issues? Can they get their work done with carrots, sticks, or both as they try to work with somewhat autonomous business units? ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 212

213. EIA Governance: A separate business unit 1/2 Logical outgrowth of •  Web or portal team •  Design or branding group •  E-services, e-business or e-commerce unit Goals •  Ensure that IA is primary goal of the unit •  Retain organizational learning •  Avoid political baggage •  Maintain independence ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 213

214. EIA Governance: A separate business unit 2/2 Ambitious, fool-hardy, unrealistic? Necessary! •  Models of successful new organizational efforts often start as separate entities •  Alternatives (none especially attractive) •  Be a part of IT or information services •  Be a part of marketing and communications •  Be a part of each business unit ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 214

215. EIA Governance: Balancing strategic and tactical Strategic: Model on Board of Directors •  Represent key constituencies •  Track record with successes, mistakes with organization’s prior centralization efforts •  Mix of visionaries, people who understand money Tactical: Start with staff who “do stuff” •  Extend as necessary by outsourcing •  Enables logical planning of hiring and use of consultants and contractors ©2010 Louis Rosenfeld LLC (www.louisrosenfeld.com). All rights reserved. 215