Aiim conference patrick lambe slides


Published on

Beyond the Black Box - An Empirical Approach to Taxonomy Development

Published in: Business, Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Aiim conference patrick lambe slides

  1. 1. #AIIM12 Better Than the Black Box:An Empirical Approach to Taxonomy Development Patrick Lambe Straits Knowledge, Singapore#AIIM12
  2. 2. Three Traps Chasing your tail Experts know best K e e p O u t !! T a x o n o m is t a t W o r k Enclosure #AIIM12
  3. 3. Being Empirical Means…  Warrant - grounding taxonomy in  Content warrant  User warrant  De facto standards in use  Testing  Open card sorts  Closed card sorts  Scenario-based tests  Load/balance tests  Consultation  Targeted on gaps, technical accuracy, NOT general opinions  Transparent, tough issue resolution processFor enterprise taxonomies, unmanaged exposure to unfocused opinions are the Taxonomist’s greatest enemy #AIIM12
  4. 4. 1. Warrant - the role of the knowledge audit#AIIM12
  5. 5. What is a knowledge audit?A knowledge audit is a survey-cum-inventory of the different kinds of knowledgethat are: currently in use needed but not currently in use (gaps and desired improvements)…to support the work and strategicobjectives of your organisation Natural #AIIM12
  6. 6. Knowledge Audit Half day workshop 2-3 operational managers per dept QuickTimeª and a Knowledge maps, culture decompressor are needed to see this picture. analysis, pain points Site visits Gallery viewingEx a mp le o f a gal ler y v ie w ing exe rcis e #AIIM12
  7. 7. Knowledge MapsInput to activity Div & Activity Output from activity Subscribing divisions Unique code of knowledge asset Type of knowledge asset Description of knowledge asset #AIIM12
  8. 8. Expected outcomesKnowledge maps give us: Evidence for how people describe their knowledge assets (content warrant) Maps linking knowledge assets to activities (context of use - user warrant)Site visits give us: Observational evidence for organizing principles in use (user warrant) Evidence (printouts, photos) of how folders, physical and digital documents, emails, shared resources are labeled and organized (content and user warrant)Analysis of the knowledge maps gives insights into Knowledge gaps, Knowledge risks, Critical knowledge requiring protection, Sharing and collaboration opportunities, Knowledge flows and blockages - and helps identify knowledge priorities, useful in designing the taxonomy #AIIM12
  9. 9. 2. Testing - evidence that people can use the taxonomy effectively #AIIM12
  10. 10. Testing Techniques Open card sorting - to find “natural categories” among users and variance between them - at the start of a taxonomy design Closed card sorting - to test usability of top level categories - to test a draft taxonomy Scenario based testing - to uncover navigation, predictability issues - to test a completed taxonomy - scenarios are derived from user warrant The conversations are Balance/load testing - to analyze the as important as the distribution of content across a results populated taxonomy in use Donna Spencer, Card Sorting Rosenfeld Media 2009 #AIIM12
  11. 11. Case Study Review of existing taxonomy in use Public health agency#AIIM12
  12. 12. Balance 53% of taxonomy folders have no content 76% of populated folders have fewer than 5 documents 9% of populated folders have more than 15 items 15% of populated folders (7% of all taxonomy folders) have 5-15 items and could be considered navigable#AIIM12
  13. 13. Balance & Accessibility 35% of content is 5 or more levels deep 83% of content is 4 or more levels deep#AIIM12
  14. 14. What we foundFrom the usability tests:Avge time Avge Avge Task Confident Consistency / task clicks / dead ends abandoned final of decisions task / task decision 2.5 18 clicks 1.5 25% 46% 35%minutes folders opened #AIIM12
  15. 15. Taxonomy design issues As Navigation Tool:  Too narrow and deep  Unpredictable, different principles of organisation applied in parallel As Controlled Vocabulary:  Same categories repeated across the different parts of the structure - ambiguity  No overall design principle to govern additions to taxonomy As Classification Scheme:  Fewer than half the taxonomy is populated, extreme lack of balance in distribution of content  Only 35% consistency in usability testsSolution: break the hierarchy into three facets: Business Activities, DocumentTypes, Health Topics, using warrant from the existing classification plusknowledge maps from a knowledge audit #AIIM12
  16. 16. Taxonomy Redesign This combination of facets gave a very big reduction in frustration levels, increased the richness of information captured about documents, and substantially increased consistency and confidence Because in a faceted system each document needs between 2 and 3 tags to achieve the same level of precision as in a single hierarchy, we have divided the actual consistency rate of individual tags by 2.5 to make a meaningful comparison. #AIIM12
  17. 17. 3. Consultation - transparentbut rigorous issue resolution process#AIIM12
  18. 18. Consultation Means… “We’d like you to examine and Focus the feedback on the areas comment on the following sections of the taxonomy where testing of the taxonomy because of your indicates that you need advice expertise area” You’ll get these anyway - but “We do not want your opinions, we you’ve made it possible to want targeted feedback on: Say “no” to opinions  Gaps without warrant  Accuracy of language  Accuracy of structure and relationships Try to ensure that all suggested  Consolidating and simplifying changes have some warrant underpinning categories” them - labels in use, standards in use, situations requiring this language; and “Please give reasons for your that feedback represents the general suggestions, based on how people user not the specialist user will actually use these categories” #AIIM12
  19. 19. Issue Resolution#AIIM12
  20. 20. Issue Resolution#AIIM12
  21. 21. Being Empirical Also Means… Being better equipped to manage the politics of taxonomy development Being able to establish credibility for the taxonomy development process and products - including being able to justify denying requests Reducing the risk of project delays resulting from claims on the taxonomy from narrowly informed special interests #AIIM12
  22. 22. #AIIM12Any Questions? #AIIM12