's Rollout of Topic Taxonomy


Published on

Emily discusses the Robert Wood Johnson Foundation's various activities in taxonomy creation and maintenance.

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  •'s Rollout of Topic Taxonomy

    1. 1.’s Rollout of Topic Taxonomy The Project from a Client View
    2. 2. In This Presentation <ul><li>About RWJF </li></ul><ul><li>Why do we need a taxonomy? </li></ul><ul><li>The project: development and validation </li></ul><ul><li>The result </li></ul><ul><li>Lessons and next steps </li></ul>
    3. 3. About RWJF <ul><li>To improve the health and health care of all Americans </li></ul><ul><li>Focus on problems that demand attention </li></ul><ul><li>Passion, perseverance and rigor </li></ul><ul><li>Support programs with measurable impact </li></ul><ul><li>Seek to create meaningful and lasting social change </li></ul>
    4. 4. RWJF By the Numbers <ul><li>Total Assets: $10 billion </li></ul><ul><li>Grants and Contracts Awarded in 2006 —928 </li></ul><ul><li>Total Active Grants: 2402 </li></ul><ul><li>7 Priorities </li></ul>
    5. 5. Why Taxonomy? A Lot of Content <ul><li>4,296 publications, articles, journal articles, webcasts, videos on today </li></ul><ul><li>400 peer-reviewed journal articles and reports added each year </li></ul><ul><li>100 end-of-grant reports added each year </li></ul><ul><li>Does not include (but will soon): news releases, feature stories, speeches, more! </li></ul>
    6. 6. Why Taxonomy? Clarity <ul><li>Guiding Principles: Be objective, rigorous and transparent and communicate clearly and openly … </li></ul><ul><li>2007: 7 Program Areas </li></ul><ul><li>2005: 11 Interest Areas </li></ul><ul><li>2003: 4 Goals </li></ul><ul><li>2001: 8 Priorities </li></ul><ul><li>Goals change! </li></ul>
    7. 7. Before The Project: Internal Taxonomy Creation <ul><li>Workgroup of grants managers, Web staff, information architect, research team, librarian </li></ul><ul><li>Back-indexing completed in early 2005 and takes place weekly on an ongoing basis </li></ul><ul><li>Approximately 600 terms used to index publications as well as active, pending and turned-down grants </li></ul><ul><li>Categories cover: </li></ul><ul><ul><li>RWJF goals and program areas </li></ul></ul><ul><ul><li>What type of work it was (demonstration, research, communications) </li></ul></ul><ul><ul><li>What population benefits from the grant and where, if any </li></ul></ul><ul><ul><li>The subject matter of the project </li></ul></ul><ul><ul><li>Any medical problem or health condition underlying the program </li></ul></ul>
    8. 8. The Project <ul><li>Part of a merger of multiple sections into a single section begun in early 2006 </li></ul><ul><li>User-focused vocabulary covering conditions, treatment, people, places and particular government and social services </li></ul><ul><li>Permanent home for content as programs change </li></ul><ul><li>Prioritizes frequently searched content </li></ul><ul><li>Must be clear that not all topics are funded </li></ul>
    9. 9. Discovery <ul><li>Reviewed consumer health taxonomies, relevant sections of NLM taxonomies </li></ul><ul><li>Reviewed our “competitors” – not much structure to parallel </li></ul><ul><li>Rapid prototype showing a year’s worth of information from the internal taxonomy </li></ul><ul><ul><li>“Wow, that’s a lot of terms” </li></ul></ul><ul><ul><li>“Let’s find out how users want to see this” </li></ul></ul>
    10. 10. Open Card Sort <ul><li>11 RWJF grantees who also were members of highest priority target audience </li></ul><ul><li>3x5 index cards of 100 terms covering the spectrum of content </li></ul><ul><li>1 hour to work as individuals to create as many/as few groups as they wanted </li></ul>
    11. 11. Open Card Sort: Results <ul><li>11 different hierarchies: some flat, some deep (4 levels) </li></ul><ul><li>Not a high degree of overlap especially on particular policy issues </li></ul><ul><li>Certain areas of agreement (conditions and diseases, health care quality, people and places) </li></ul>
    12. 12. Taxonomy Iteration <ul><li>10+ iterations between consultant and RWJF </li></ul><ul><li>First iteration had about 500 topics </li></ul><ul><li>Last iteration had 370 topics </li></ul><ul><li>Top level hovered between 6 and 11 choices </li></ul><ul><li>Hierarchy was between 3 and 5 levels </li></ul>
    13. 13. Iteration Factors <ul><li>Remove jargon: “pharmaceutical services” v. “prescription drugs” </li></ul><ul><li>The same word has different meanings in different contexts: “health care system” </li></ul><ul><li>How do hierarchy and visual presentation match up? Launching a browse without browse functionality and as a result went deeper </li></ul>
    14. 14. Closed card sort: “Taxi” (Catalyst Group Design in NY) <ul><li>2 days: 7 hourlong sessions per day </li></ul><ul><li>User, interviewer, computer </li></ul><ul><li>Question: Where would you look for an item (for example, asthma in children) in this hierarchy? </li></ul><ul><li>Users saw terms, but not content below terms, and could not go backwards </li></ul><ul><li>Users rated their confidence on a scale of 1-5 </li></ul><ul><li>Items developed from search logs and presented in varying order </li></ul><ul><li>Big changes from Day 1 to Day 2 </li></ul>
    15. 15. Post-Study Observations <ul><li>Top level still does not satisfy a number of users </li></ul><ul><li>Not showing content or allowing back-and-forth really stresses the taxonomy </li></ul><ul><li>Current iteration misses a sizeable number of topics </li></ul><ul><li>Easier to use when looking for health care items and less easy to use for public health items </li></ul><ul><li>Several topics had multiple successful paths </li></ul>
    16. 16. Post-Report Activities <ul><li>Relate topics (relate a chronic condition, such as asthma, to chronic care improvement) </li></ul><ul><li>Eliminate redundancy </li></ul><ul><li>Focus on other elements of a relaunch with a hard deadline </li></ul>
    17. 17. The result: taxonomy at launch <ul><li>6 top level categories (plus a category of Popular Topics) </li></ul><ul><li>366 terms </li></ul><ul><li>Used in our Publications & Research section and in limited amounts elsewhere </li></ul>
    18. 18. Publications & Research
    19. 19. Long-Term Care
    20. 20. Individual Articles
    21. 21. How’d We Do? <ul><li>Policy-makers seem to like the usage in popular topics but want keywords used more throughout the site </li></ul><ul><li>Twice the proportion of usage as the previous standalone sections (7.5 v. 13.5) (attributable to multiple factors) </li></ul><ul><li>Single-page visits down a few points </li></ul><ul><li>Still difficult to find things </li></ul><ul><li>Need flexible browse interface to account for multiple starting thoughts (a care setting v. a population v. a disease) </li></ul>
    22. 22. Our Lessons <ul><li>Design hierarchy and interface together </li></ul><ul><li>Incorporated more informal feedback with rapid prototyping </li></ul><ul><li>Still uncertain as to how to engage internal stakeholders </li></ul><ul><li>Assessing usability and user feedback is not enough; need additional content expertise </li></ul><ul><li>Look for alternative forms of presentation to suit multiple audience </li></ul>