Practical Approaches to Sharing Information


Published on

An old presentation based on work I did at Raytheon in 2003-4. Good lessons!

Published in: Business, Technology, Education
  • Be the first to comment

Practical Approaches to Sharing Information

  1. 1. Practical Approaches to Sharing Information at Raytheon Taxonomies, Metadata and Beyond Presented by Christine JM. Connors [email_address] KMPro / KMForum, Bentley College, Waltham, MA June 30, 2004
  2. 2. Data Discovery – What we Learned <ul><li>Level of information management varies </li></ul><ul><ul><li>85% of our information is unstructured </li></ul></ul><ul><ul><li>Over 90% of information is not tagged </li></ul></ul><ul><ul><li>High proportion of tagged documents result of templates, and therefore relay bad data </li></ul></ul><ul><ul><ul><li>Ethan Frome – over 200 documents </li></ul></ul></ul><ul><ul><li>Automatically extracted data imprecise </li></ul></ul><ul><ul><ul><li>“ The flight to Dallas is cancelled” </li></ul></ul></ul><ul><li>Greater than 13% of information is exactly duplicated </li></ul><ul><ul><li>“ Near” duplication harder to determine but potentially more costly </li></ul></ul><ul><ul><li>Worst duplication in File Servers / Shared Drives </li></ul></ul><ul><li>Difficult to determine true age of document due to web scripting, date of publication to public drive </li></ul><ul><ul><li>Over 23% of sample data not modified in previous 5 years </li></ul></ul><ul><ul><li>Over 68% of sample data not modified in previous 2 years </li></ul></ul>
  3. 3. Intranet Search and Browse Survey June 2003 <ul><li>16 multiple choice questions, 1 optional free-text comment field </li></ul><ul><li>516 surveys started (clicked into) </li></ul><ul><li>199 responses over 3 weeks </li></ul><ul><li>39% completion rate </li></ul><ul><li>101 comments on “How can we improve the intranet search and browse capabilities?” </li></ul><ul><li>51% comment rate by survey participants </li></ul>
  4. 4. How can we improve the intranet search and browse capabilities? <ul><li>101 user comments frequently included: </li></ul><ul><li>Qualify searches by function, organization, and business </li></ul><ul><li>Qualify searches by date </li></ul><ul><li>Qualify searches by document type (especially web pages) </li></ul><ul><li>Provide sorting of results by date, document type </li></ul><ul><li>Provide category search </li></ul><ul><li>Do not change URLs of pages (users bookmarked) </li></ul><ul><li>Reduce number of search results </li></ul><ul><li>Google (mentioned 32 times in comments) </li></ul>
  5. 5. Results Summary <ul><li>Search/browse </li></ul><ul><li>About ¼ (26%) of respondents find the current capabilities “Good” or “Excellent” </li></ul><ul><li>About ¼ (24%) of respondents consistently locate helpful information </li></ul><ul><li>About ¼ (22%) of respondents indicate they are consistently successful using keyword searching </li></ul><ul><li>About ¼ (26%) of respondents find it consistently easy to browse </li></ul><ul><li>The advanced search page is used sparingly </li></ul><ul><li>Categories </li></ul><ul><li>62% of respondents would find categories consistently useful </li></ul><ul><li>Bookmarks </li></ul><ul><li>36% of respondents consistently return to previously found information </li></ul><ul><li>71% of respondents use bookmarks “Most of the time” or “Always” </li></ul><ul><li>65% of respondents consistently organize their bookmarks </li></ul>
  6. 6. OUCH! <ul><li>“ The search engine is poor to inadequate. I needed to find an appropriations data sheet and was returned 366 entries, none which had anything to do with appropriations. I spend far too much time looking through the search results for this engine to be effective. If I could find this document on the INTERNET I would do so, but this is an internal Raytheon document that is successfully hidden somewhere in the archives with the Ark of the Covenant.” </li></ul><ul><ul><ul><li>Unidentified search and browse survey participant, June, 2003 </li></ul></ul></ul><ul><li>“ Who gets more hits: or ? Listen up people: Our intranet is a wasteland of information. We need to unify - we need to standardize. Information is power - but only if it is on my desktop, not hidden away in some server waiting for a lucky adventurer to uncover it like some lost continent.” </li></ul><ul><ul><ul><li>Another unidentified search and browse survey participant, June, 2003 </li></ul></ul></ul>
  7. 7. Usability Testing <ul><li>Bentley College’s Design and Usability Testing Center </li></ul><ul><li>4 Focus Groups of 8-10 people each </li></ul><ul><li>They told us: </li></ul><ul><ul><li>Want to filter searches </li></ul></ul><ul><ul><ul><li>Didn’t want long list of items to select from </li></ul></ul></ul><ul><ul><li>Liked “Suggested…” boxes </li></ul></ul><ul><ul><li>Didn’t understand the taxonomy when presented like Yahoo! </li></ul></ul><ul><ul><li>Liked taxonomy as file folder metaphor </li></ul></ul><ul><ul><li>Liked thesaurus </li></ul></ul><ul><ul><li>Were confused by relational taxonomies </li></ul></ul><ul><ul><li>Liked “Categories” as the tab label, over Topics, Taxonomy, Thesaurus, Subjects or Browse. </li></ul></ul>
  8. 8. Taxonomies – Who? <ul><li>The Dream Team </li></ul><ul><ul><li>Information Scientists </li></ul></ul><ul><ul><li>Cognitive Scientists </li></ul></ul><ul><ul><li>Linguists </li></ul></ul><ul><ul><li>Programmers </li></ul></ul><ul><ul><li>Database Experts </li></ul></ul><ul><ul><li>Network Specialists </li></ul></ul><ul><ul><li>Verity Administrators </li></ul></ul><ul><ul><li>Human Computer Interaction / Usability Experts </li></ul></ul><ul><ul><li>Subject Matter Experts </li></ul></ul><ul><ul><li>Organizational Change Management </li></ul></ul><ul><li>What we got </li></ul><ul><ul><li>Information Scientists (in-house) </li></ul></ul><ul><ul><li>Cognitive Scientist / Linguist (from Verity) </li></ul></ul><ul><ul><li>Programmers, Database Experts, Network Specialists, and Verity Administrators (both in-house and from Verity) </li></ul></ul><ul><ul><li>HCI / UI Experts (in-house, Verity and Bentley College) </li></ul></ul><ul><ul><li>Subject Matter Experts (in-house) </li></ul></ul>
  9. 9. Taxonomies – What for? <ul><li>Browse & Navigation </li></ul><ul><li>Relational Taxonomies </li></ul><ul><li>Refine Search </li></ul><ul><li>Parametric Search </li></ul><ul><li>Federated Search </li></ul><ul><li>Dynamic taxonomies </li></ul><ul><li>Profiling </li></ul><ul><li>Compliance Engine </li></ul><ul><li>Classification & Categorization </li></ul><ul><li>Provide controlled vocabularies to use with Metadata Schema(s) </li></ul><ul><li>Easy selection to minimize angst over having to fill out file properties </li></ul>
  10. 10. Taxonomies 2003 <ul><li>Deployed 5 taxonomies </li></ul><ul><ul><li>Defense Technologies (based on DTIC) </li></ul></ul><ul><ul><ul><li>Purchased DTIC taxonomy </li></ul></ul></ul><ul><ul><ul><li>Revised to fit Raytheon’s data </li></ul></ul></ul><ul><ul><ul><li>Removed several categories including agriculture that are not needed </li></ul></ul></ul><ul><ul><li>Raytheon Products </li></ul></ul><ul><ul><ul><li>Revised our products listing into a hierarchical approach </li></ul></ul></ul><ul><ul><ul><li>Enlisted Raytheon Communicators as Subject Matter Experts </li></ul></ul></ul><ul><ul><li>IPDS </li></ul></ul><ul><ul><ul><li>Built using data from the IPDS web site </li></ul></ul></ul><ul><ul><ul><li>Enlisted IPDS experts as Subject Matter Experts </li></ul></ul></ul><ul><ul><li>Engineering </li></ul></ul><ul><ul><ul><li>Implemented taxonomy built by Raytheon’s Engineering Technology Network – Needs revising and enhancing </li></ul></ul></ul><ul><ul><li>Information Technology </li></ul></ul><ul><ul><ul><li>Purchased from Verity </li></ul></ul></ul><ul><ul><ul><li>Revised to fit Raytheon’s data </li></ul></ul></ul><ul><ul><ul><li>Enlisted members of Corporate IT as Subject Matter Experts </li></ul></ul></ul>
  11. 11. Taxonomies 2004 <ul><li>Human Resources taxonomy </li></ul><ul><ul><li>Deployed March 2004 </li></ul></ul><ul><ul><li>Purchased from Verity </li></ul></ul><ul><ul><li>Cross-functional team of HR representatives reviewed as Subject Matter Experts </li></ul></ul><ul><li>Six Sigma </li></ul><ul><ul><li>Will be deployed June 2004 </li></ul></ul><ul><ul><li>Built based on Raytheon Six Sigma data </li></ul></ul><ul><li>Legal Taxonomy </li></ul><ul><ul><li>Will be deployed June 2004 </li></ul></ul><ul><ul><li>Purchased from Verity </li></ul></ul><ul><ul><li>Able to create additional taxonomies for Ethics, Environmental Health & Safety, and Export/Import Compliance from the purchase of this ONE taxonomy </li></ul></ul><ul><li>Will be restructuring our top level categories : </li></ul><ul><ul><ul><li>Business Units – domestic & international </li></ul></ul></ul><ul><ul><ul><li>Functions </li></ul></ul></ul><ul><ul><ul><li>Processes </li></ul></ul></ul><ul><ul><ul><li>Products </li></ul></ul></ul><ul><ul><ul><li>Relationships </li></ul></ul></ul>
  12. 12. Taxonomies – How? <ul><li>Card sorting </li></ul><ul><ul><li>EZ Sort </li></ul></ul><ul><ul><li>3x5 cards </li></ul></ul><ul><li>Review search engine logs </li></ul><ul><ul><li>Internal logs </li></ul></ul><ul><ul><li>Webtrends </li></ul></ul><ul><li>Review organic systems </li></ul><ul><ul><li>Web and file share navigation </li></ul></ul><ul><li>Review existing taxonomies/thesauri </li></ul><ul><li>Concept Mapping </li></ul><ul><ul><li>Linguistic algorithm </li></ul></ul><ul><li>Intelligent Classifier – lots of query building behind each node </li></ul><ul><li>MultiTes </li></ul><ul><li>Mind Manager </li></ul><ul><li>TextPad </li></ul><ul><li>BUY! </li></ul>
  13. 13. Build vs. Buy <ul><li>Build to suit users </li></ul><ul><ul><li>Reflect corporate vernacular </li></ul></ul><ul><ul><li>Internal acronyms </li></ul></ul><ul><ul><li>Corporate culture </li></ul></ul><ul><ul><ul><li>How is the business structured and portrayed? </li></ul></ul></ul><ul><li>Can take a long time </li></ul><ul><ul><li>Time estimates depend on type/use of taxonomy/tools available </li></ul></ul><ul><ul><ul><li>Simple = 5 minutes to build term, 5 minutes to build category and map the topic to the taxonomy </li></ul></ul></ul><ul><ul><ul><li>Complex = 75 minutes to build term </li></ul></ul></ul><ul><ul><ul><li>PLUS Quality Assurance testing! </li></ul></ul></ul><ul><li>Buy </li></ul><ul><ul><li>Industry standard </li></ul></ul><ul><ul><li>Rapid implementation </li></ul></ul><ul><li>Need customization </li></ul><ul><li>Both decisions require maintenance </li></ul>
  14. 14. Benefits - Increased Productivity <ul><li>Delphi Group 2003 – as reported by Gartner </li></ul><ul><li>Business professionals spend more than 2 hours per day searching for information </li></ul><ul><li>Half of that time – 1 hour per day is wasted by failure to find what they seek </li></ul><ul><li>The single factor most attributed to the large amount of time wasted was </li></ul><ul><ul><li>data changes (location 35%) and </li></ul></ul><ul><ul><li>bad tools (ineffective search and lack of labeling 28%) </li></ul></ul><ul><li>If we conservatively assume only 1 hour per year would be saved per general employee and 1 hour per month per engineer, then: </li></ul>
  15. 15. Relative of starting point, growth curves represent storage acquisition cost increases over time. Benefits – Reduce Storage Costs TB TB TB TB TB
  16. 16. Is it working? <ul><li>“ New” search launched September 29 </li></ul><ul><li>Latest survey results show improvement </li></ul><ul><ul><li>Neutral rating upgraded to Good </li></ul></ul><ul><li>Metrics show increased usage of search </li></ul><ul><ul><li>17% increase in unique users per day </li></ul></ul><ul><ul><li>25% increase in searches per day </li></ul></ul><ul><li>Metrics show increasing use of categories in search </li></ul><ul><ul><li>Since launch, the categories have been used 50,000 times </li></ul></ul><ul><li>ITLT approved project funding </li></ul><ul><li>Knowledge Representation team recipients of 2003 IT Excellence in Collaboration and Knowledge Management Award </li></ul>