Lyns knowledge journeyv3


Published on

Published in: Education, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Intranet redesign @ Medibank The old view Analysing the info needed Card sorting Site architecture New site Results 2010 plans
  • : needs analysis, developing outcomes, writing guides, producing & delivering training needs analysis, developing outcomes, writing guides, producing & delivering training Supporting clients incidents and providing solutions MPL – met through RMIT – networking Stakeholder engagement, collaboration with SMEs, building solutions FastTrack – headhunted back - Stakeholder engagement, collaboration with SMEs, building solutions
  • Communications: Ability to get consensus and collaboration across many business units; ability to explain complex concepts in layman's language; ability to generate enthusiasm; ability to communicate with all levels of management and staff. establishing straightforward, productive relationships; treating all individuals with fairness and respect, demonstrating sensitivity for cultural and gender differences; showing great drive and commitment to the organization s mission; inspires others: Maintaining high standards of personal integrity; Client Orientation: Understands clients' needs and concerns; responds promptly and effectively to client needs; Customizes services and products as appropriate Drive for Results: Makes things happen; Is proactive; balances "analysis" with "doing"; sets high standards for self; Commits to organizational goals Teamwork: Collaborates with others in own unit and across boundaries; acknowledges others' contributions; works effectively with individuals of different culture and gender; willing to seek help as needed. Influencing and resolving differences across organizational boundaries: Gaining support and commitment from others even without formal authority; resolving differences by determining needs and forging solutions that benefit all parties; promoting collaboration and facilitating teamwork across organizational boundaries. Learning and knowledge sharing: open to new ideas; shares own knowledge; applies knowledge in daily work; builds partnerships for learning and knowledge sharing Analytical Thinking and Decisive Judgment : Analyzing issues and problems systematically, gathering broad and balanced input, drawing sound conclusions and translating conclusions into timely decisions and actions.
  • Despite the pressing need for reliable and timely knowledge sharing, most professional services organizations do not have a high quality mechanism to do so. In fact, 56% of practice professionals' time is spent either searching for information or gathering information. Only 25% is spent on the actual analysis. These statistics point to the need for much improved knowledge sharing. Much of the difficulty to date has been because professional services firms have focused on 'knowledge management' (KM) systems as the answer. Knowledge management systems are meant to organize and share knowledge within a firm, but more often than not, their scope has become too large to manage, and keeping the KM systems up-to-date and useful has been an enormous challenge. it is insightful to break down the elements of a knowledge sharing culture. These can typically be broken into people, process & technology; each of these explain the root causes of why KM systems are not working:
  • P is People. Don't start a social strategy until you know the capabilities of your audience. If you're targeting college students, use social networks. If you're reaching out business travelers, consider ratings and reviews. Forrester has great  data to help with this, but you can make some estimates on your own. Just don't start without thinking about it. O is objectives. Pick one. Are you starting an application to listen to your customers, or to talk with them? To support them, or to energize your best customers to evangelize others? Or are you trying to collaborate with them? Decide on your objective before you decide on a technology. Then figure out how you will measure it. S is Strategy. Strategy here means figuring out what will be different after you're done. Do you want a closer, two-way relationship with your best customers? Do you want to get people talking about your products? Do you want a permanent focus group for testing product ideas and generating new ones? Imagine you succeed. How will things be different afterwards? Imagine the endpoint and you'll know where to begin. T is Technology. A community. A wiki. A blog or a hundred blogs. Once you know your people, objectives, and strategy, then you can decide with confidence.
  • 1a Understanding the various components that constitute the KM strategy and technology framework. Analysing and accounting for what is already in place in Telstra, then we can identify critical gaps in the existing infrastructure. Leveraging existing infrastructure is the logically, scientifically. rationally, theoretically and financially right approach. 1b KM architecture design and component selection Knowledge audit and analysis KM team design Creation of a KM blueprint tailored for your organization The actual systems development process THEY COPIED ALL THAT THEY COULD FOLLOW BUT THEY COULD NOT COPY MY MIND, AND I LEFT 'EM SWEATING AND STEALING AND A YEAR AND HALF BEHIND. -RUDYARD KIPLING
  • Each QAT consists of a group of volunteer staff from across all business divisions each with different experiences and interests.
  • New look and feel since Sept 2008
  • [George - note slide slightly changed from Final Report Exhibit 30 Page 57] 1. Grow The first opportunity to be mapped out for implementation was the potential for “bulking up in PHI” through possible acquisition, although it is accepted that there are difficult commercial and legal hurdles to be overcome because of the mutual structure of potential targets and the lack of incentive for them to merge with MPL. Further analysis to commence now. Alternative arrangements, such as the provision of back office services for other funds, will also be considered. (previously outlined) Anticipated resourcing requirements were also factored into implementation planning, taking into account both potential acquisition activity and also the establishment of a distribution alliance for general and “lite” life insurance. 2. Diversify It is expected that the design of the preferred distribution arrangement and the selection of a preferred partner will be undertaken in the later part of FY06 and early in FY07. Further analysis and detailed planning underway. The partner selection process will take into account the future potential for a manufacturing JV, once the distribution model has been established and fully tested.
  • Based on collaborative process How Staff will be surveyed to identify and prioritise the Top 10 issues that will provide the greatest gains. This will guarantee there are ‘quick wins’ in the system which will instil confidence in the system and will encourage greater participation. It is essential that the correct taxonomy is applied so users may effectively navigate the system to find information quickly. It is vital that time is spent on the design and testing of the system to ensure ease of use. ss now used by Medibank
  • Like mind mapping – ideas / areas of interest / pages on notes and then group together in like areas
  • Open Card Sorting: Participants are given cards showing site content with no pre-established groupings. They are asked to sort cards into groups that they feel are appropriate and then describe each group. Open card sorting is useful as input to information structures in new or existing sites and products. Closed Card Sorting: Participants are given cards showing site content with an established initial set of primary groups. Participants are asked to place cards into these pre-established primary groups. Closed card sorting is useful when adding new content to an existing structure, or for gaining additional feedback after an open card sort.
  • 15 June 2009
  • Lyns knowledge journeyv3

    1. 1. Knowledge Management My journey Lyn Murnane Knowledge Manager
    2. 2. Topics <ul><li>1. The journey so far </li></ul><ul><ul><li>What is a Knowledge Manager? </li></ul></ul><ul><ul><li>What is Knowledge </li></ul></ul><ul><ul><li>Some data </li></ul></ul><ul><ul><li>Where to start with a KM strategy </li></ul></ul><ul><li>2. An overview of Medibank’s change </li></ul><ul><li>3. Intranet Infrastructure – a quick how to </li></ul>
    3. 3. My journey so far
    4. 4. Quick CV <ul><li>IT Trainer </li></ul><ul><li>FastTrack Software: Product Consultant, Support Desk Team Leader </li></ul><ul><li>Medibank Private: Knowledge Management Business Consultant </li></ul><ul><li>FastTrack – Knowledge Manager </li></ul><ul><li>Telstra – Manager Knowledge Management </li></ul><ul><ul><li>Manager of KnowHow – website supporting 14,000 customer service staff </li></ul></ul>
    5. 5. What is a Knowledge Manager? <ul><li>From </li></ul><ul><ul><li>The main function of the knowledge sharing position is to help champion organization-wide knowledge sharing, so that the organization's know-how, information and experience is shared inside and (as appropriate) outside the organization with clients, partners, and stakeholders. </li></ul></ul>
    6. 6. <ul><li>Skills Required </li></ul><ul><li>Leadership </li></ul><ul><li>Communications </li></ul><ul><li>Customer / User Orientation </li></ul><ul><li>Facilitate sharing & collaboration </li></ul><ul><li>Teamwork </li></ul><ul><li>Learning and knowledge sharing </li></ul><ul><li>Analytical Thinking and Decisive Judgment </li></ul>
    7. 7. Knowledge Transfer “ I hear and I forget. I see and I remember. I do and I understand.” Confuscius
    8. 8. Knowledge is... <ul><li>Thomas Davenport defines knowledge as what happens at the moment in time when information becomes valuable to the individual seeking it. </li></ul><ul><li>In call centres, help desks, and other support environments, that individual is either the support agent seeking information to help a customer, or a customer (product user, employee, partner, or vendor) seeking answers in a web-based self-help environment. </li></ul><ul><li>Thomas Davenport, the author of several works on the subject including, Information Ecology: Mastering the Information and Knowledge Environment and Working Knowledge: How Organizations Manage What They Know. </li></ul>
    9. 9. Some distractions <ul><li>Social Media 2011 - </li></ul><ul><li>Using KM - </li></ul><ul><li>Discover what you know – 2006 </li></ul>
    10. 10. Some data <ul><li>56% of knowledge workers' time is spent either searching for information or gathering information. Only 25% is spent on the actual analysis. </li></ul><ul><li>Organisations have focused on 'knowledge management' (KM) systems as the answer. </li></ul>
    11. 11. Where to start? Let’s think about the types of people who participate in knowledge sharing
    12. 12. Participation - Australia
    13. 13. A Knowledge Strategy <ul><li>What do we need to consider when trying to define a Knowledge Strategy? </li></ul> How about we start by looking at a social media strategy?
    14. 14. POST <ul><li>People . Start by understanding what employees actually use and need today. Don’t guess and don’t rely on anecdotal interviews. Instead, start with a quantitative assessment. </li></ul><ul><li>Objectives . With that baseline of understanding in place, next decide what your business goals are. You will need to build a decision council that includes IT and business to help you do this. </li></ul><ul><li>Strategy . The strategy part of this planning process means mapping the business goals to specific collaboration scenarios that you can actually improve — no tools yet. </li></ul><ul><li>Technology . The last step is to figure out which technologies improve your most important collaboration scenarios. Choose cloud services if they make sense; on-premises if not. </li></ul>
    15. 15. Sites & Tools that might help <ul><li> </li></ul><ul><li> (understand other company’s stories) </li></ul><ul><li> </li></ul><ul><li>Working Knowledge – Thomas Davenport </li></ul>
    16. 16. Telstra <ul><li>KnowHow – an intranet based process and sales information that supports 14,000 users – onshore , offshore and industry partners. </li></ul><ul><li>KnowHow’s supports consumer customers </li></ul><ul><li>Including support for Telstra Business (Small Business) </li></ul><ul><li>Telstra has at least 10 ‘official’ KM systems </li></ul><ul><li>100’s of unofficial tools including spreadsheets, personalised web pages, databases etc </li></ul><ul><li>My focus is on KnowHow </li></ul>
    17. 17. KnowHow <ul><li>Observations – content / information is verbose and not user friendly </li></ul><ul><li>NO collaboration </li></ul><ul><li>Feedback loop is sporadic and not transparent </li></ul><ul><li>NO Governance, archiving or expiry of content unless requested </li></ul>
    18. 18. Changes <ul><li>User Feedback forums </li></ul><ul><ul><li>What does KnowHow sound like / its character </li></ul></ul><ul><ul><li>Understanding what works and what doesn’t </li></ul></ul><ul><ul><li>What’s missing? </li></ul></ul><ul><ul><li>Suggestions for inclusions </li></ul></ul><ul><ul><li>Getting engagement / buy-in </li></ul></ul>
    19. 19. Processes <ul><li>Governance model </li></ul><ul><li>Audit process </li></ul><ul><li>Expiry process </li></ul><ul><li>Writing style guide </li></ul><ul><li>Publishing style </li></ul><ul><li>New content management system should automate some of these processes </li></ul>
    20. 20. Telstra Bigger picture <ul><li>Project to create a company wide KM strategy </li></ul><ul><li>Aims to create a single source of truth </li></ul><ul><li>High level governance model </li></ul><ul><li>Has leadership support and cross business unit endorsement </li></ul><ul><li>Project currently being scoped and mapped </li></ul><ul><li>Identifying measures of success </li></ul>
    21. 21. Suggested KM Strategy <TELSTRA DOCUMENT ID> `
    22. 22. Pause for breath <ul><li>Any questions? </li></ul>
    23. 23. Medibank Private <ul><li>About private health insurance: </li></ul><ul><li>Highly government regulated – and the regulations change </li></ul><ul><li>Extremely complicated – for staff as well as customers </li></ul><ul><li>Customers often don’t really understand their cover until they claim </li></ul><ul><li>PHI is a high use compared to other insurances </li></ul>2009 Market share in private health insurance in Australia 29% Number of people covered 3.5 million Number of memberships 1.8 million Total contribution income $3.4 billion Total benefits paid $2.9 billion (84.8% of contributions) Number of customer transactions in Call Centre and Retail 6 million Number of staff 3000
    24. 24. Medibank’s culture - approach to change <ul><li>“ Empowerment for Ground crew” </li></ul><ul><li>“ We don’t need a McKinsey or a Boston Consulting to tell us how to improve the business – we’ve got over 1200 ‘ground crew’ staff who know exactly where the real gaps are to be addressed in the business,” George Savvides – MD. </li></ul>We embrace change better when we do it ourselves
    25. 25. In 2004 - The problems frontline faced <ul><li>Intranet – 1400 files, out of date, inconsistent, poor search, slow </li></ul><ul><li>Many sources of information: Lotus Notes, shared drive (40,000 files), local info, Circulars </li></ul><ul><li>20,000 internal staff helpdesk calls per month </li></ul><ul><li>Communication to frontline staff ineffective – Circulars, Manuals, Guides, many emails </li></ul><ul><li>Inconsistent information given to customers </li></ul><ul><li>One size fits all communication – 400 page fund policy document! </li></ul><ul><li>Feedback from exit interviews - staff leaving because not sufficiently supported to do their jobs effectively </li></ul>Access to knowledge is confusing, inaccurate and inconsistent .
    26. 26. Opportunity costs and benefit realisation <ul><li>Training – new starters </li></ul><ul><ul><li>$12.5Keach /30% turnover </li></ul></ul><ul><li>Staff Help Desks </li></ul><ul><ul><li>20,000 calls to 2 helpdesks. </li></ul></ul><ul><li>Call Handling Time </li></ul><ul><ul><li>The Pilot Program statistics demonstrated a reduction of 6.3% in Call Handling Time. </li></ul></ul><ul><li>Ex Gratia Payments </li></ul><ul><ul><li>Cost MPL $500,000 in FY03. Consistent, complete and accurate information in a central repository has the ability to reduce this cost. </li></ul></ul>Opportunity costs > Millions <ul><li>On-going costs 6 staff and support. </li></ul><ul><li>Benefit realisation within three months. </li></ul>Ongoing savings ~ Millions
    27. 27. Medibank’s elements of KM Knowledge Management Process Technology People Content <ul><li>databases </li></ul><ul><li>search engines </li></ul><ul><li>intranets </li></ul><ul><li>portals </li></ul><ul><li>building a learning environment </li></ul><ul><li>knowledge sharing </li></ul><ul><li>trust </li></ul><ul><li>support the capture of explicit knowledge </li></ul><ul><li>continual learning and improving </li></ul><ul><li>organisation & meaning </li></ul><ul><li>accuracy </li></ul><ul><li>consistency </li></ul><ul><li>currency </li></ul>
    28. 28. Desired state – Communication to frontline staff Knowledge Enablers
    29. 29. What worked well…initial project <ul><li>Team </li></ul><ul><ul><li>Built by staff for staff </li></ul></ul><ul><li>Frontline engagement Get the end users involved…make it a knowledge system </li></ul><ul><ul><li>focus groups (New Starters, Experts, 20+ years service) </li></ul></ul><ul><ul><li>super user group </li></ul></ul><ul><ul><li>competitions </li></ul></ul><ul><ul><li>pilot </li></ul></ul><ul><ul><li>surveys </li></ul></ul><ul><ul><li>road shows </li></ul></ul><ul><ul><li>video – of staff response to project </li></ul></ul><ul><li>Brand – identity </li></ul><ul><ul><li>stickers, soft balls, umbrellas </li></ul></ul><ul><ul><li>quick reference </li></ul></ul><ul><li>Tool </li></ul><ul><ul><li>good search </li></ul></ul><ul><ul><li>no bells and whistles </li></ul></ul><ul><ul><li>met requirements </li></ul></ul><ul><ul><li>easy to use </li></ul></ul><ul><li>Ongoing support </li></ul><ul><ul><li>Feedback mechanism was and still is the most popular feature </li></ul></ul><ul><li>Content </li></ul><ul><ul><li>Write it for the audience </li></ul></ul><ul><ul><li>Write if for how they think about it </li></ul></ul><ul><ul><li>Avoid jargon </li></ul></ul>
    30. 30. What didn’t work well….initial project <ul><li>Business experts & Management engagement - resistance </li></ul><ul><li>Approval process – subject matter experts took three times longer than expected </li></ul><ul><li>Training – self-led through a workbook doesn’t work for call centre / retail environment </li></ul>
    31. 31. A quick peek at Max
    32. 32. Growth Max / Molly / Intranet
    33. 33. ahm KM implementation <ul><li>Create a Knowledge Management System (KMS) that is the single point of reference for all learning and knowledge materials, updates and alerts so staff are not trawling multiple mediums for information. </li></ul><ul><li>Identify what knowledge is critical to the effectiveness of the Contact Centre and where gaps exist. Work closely with Operations Managers, front line staff and other stakeholders to identify the priorities for inclusion in the KMS. </li></ul>Where are we headed? “ We share knowledge with our colleagues to deliver professional excellence.”
    34. 34. “ Anyone in the organization who is not directly accountable for making a profit should be involved in creating and distributing knowledge that the company can use to make a profit”   Sir John Browne – CEO of BP Interesting article on BP’s knowledge management struggle
    35. 35. Questions? <ul><li>Are you still with me? </li></ul>
    36. 36. Time for a break?
    37. 37. Developing the right information architecture for Medibank’s Intranet
    38. 38. Topics for discussion – Medibank’s Intranet <ul><li>About Medibank, KM and the old Intranet </li></ul><ul><li>About the process we have used to design our new Intranet </li></ul><ul><ul><li>Streamlining the information flow to meet diverse user needs </li></ul></ul><ul><ul><li>Catering for intuitive user search and navigation </li></ul></ul><ul><ul><li>Collaborating with customers for user satisfaction and efficiency </li></ul></ul><ul><ul><li>Techniques and online tools for information architecture </li></ul></ul>
    39. 39. About Medibank <ul><li>About Knowledge Management: </li></ul><ul><li>Established in 2004 </li></ul><ul><li>Team of 6 including: </li></ul><ul><ul><li>Manager </li></ul></ul><ul><ul><li>KM Consultant </li></ul></ul><ul><ul><li>Senior Technical Writer </li></ul></ul><ul><ul><li>3 KM analysts </li></ul></ul><ul><li>Manage the Intranet and 2 knowledge bases </li></ul><ul><ul><li>Intranet – all staff </li></ul></ul><ul><ul><li>Max - 1200 member-serving staff (Retail, Call centre, processing) </li></ul></ul><ul><ul><li>Molly – all staff policies, processes, forms etc. </li></ul></ul>
    40. 40. Overview – new Intranet project <ul><li>The Old Intranet: </li></ul><ul><li>Intranet seen as static and not valued </li></ul><ul><li>Technology last upgraded in 2000 </li></ul><ul><li>Unsupported by vendor </li></ul><ul><li>No development environment </li></ul><ul><li>Missing standard features (eg functional search, forums, surveys, staff polls) </li></ul><ul><li>No ability to segment content for different users </li></ul><ul><li>Authoring is limited to those trained in HTML coding </li></ul><ul><li>Most of the valued information lies in a separate knowledge base (called Molly) that is not seamlessly integrated </li></ul><ul><li>Feedback from staff – “the tools are hard to use and confusing” </li></ul>
    41. 41. Aims <ul><li>Aims for the new Intranet: </li></ul><ul><li>People will be able to find people (drill down by division, location, or search) </li></ul><ul><li>Will showcase company events, jobs, and encourage employee collaboration and networking </li></ul><ul><li>Content – the top 20% of information 80% of employees need to know </li></ul><ul><li>Brand new design and architecture - user-centric </li></ul>
    42. 42. A look at the existing high level taxonomy – Intranet
    43. 43. Classifying and presenting content in a logical way <ul><li>Start by mapping what we know: </li></ul><ul><li>Map what is in scope (existing content) for Molly and Intranet </li></ul>
    44. 44. Streamlining the information flow to meet diverse user needs <ul><li>Techniques </li></ul><ul><li>Determine current use (heat map) </li></ul><ul><li>Determine what people are looking for (search log analysis) </li></ul><ul><li>Listen to and capture your users opinions (Affinity diagram) </li></ul><ul><li>Create persona’s </li></ul><ul><li>Determine what can go (ROT analysis – Redundant, out of date, trivial) </li></ul><ul><li>Determine what is missing (Gap analysis) </li></ul><ul><li>Work with the content owners to convince them of a user centric design </li></ul><ul><li>KM Employee satisfaction surveys </li></ul><ul><li>KM Strategy staff interviews </li></ul>
    45. 45. Streamlining the information flow to meet diverse user needs <ul><li>Defining your users and their needs </li></ul><ul><li>Define the different user segments </li></ul><ul><ul><li>Board </li></ul></ul><ul><ul><li>GE </li></ul></ul><ul><ul><li>SET </li></ul></ul><ul><ul><li>Managers </li></ul></ul><ul><ul><li>Corporate staff </li></ul></ul><ul><ul><li>Retail </li></ul></ul><ul><ul><li>Call Centre </li></ul></ul><ul><ul><li>Claims </li></ul></ul><ul><ul><li>Member Liaison </li></ul></ul><ul><li>Techniques </li></ul><ul><li>Determine what they need – interviews </li></ul><ul><ul><li>What they need </li></ul></ul><ul><ul><li>What they use </li></ul></ul><ul><li>Set tasks and observe use </li></ul><ul><ul><li>How they use it </li></ul></ul><ul><ul><li>How they search and find </li></ul></ul><ul><li>Get them to draw their ideal Intranet </li></ul><ul><ul><li>Crayons </li></ul></ul><ul><ul><li>Butchers Paper </li></ul></ul><ul><ul><li>Coloured Pens </li></ul></ul>
    46. 46. Segment, profile and interview our users
    47. 47. Affinity diagrams
    48. 48. Catering for intuitive user search and navigation <ul><li>Observations </li></ul><ul><li>Card sorting </li></ul><ul><li>Search log analysis </li></ul><ul><li>Search database mapping and rationalisation </li></ul><ul><li>Look at Best practice navigation designs </li></ul><ul><li>Provide multiple ways to search/navigate </li></ul>
    49. 49. Created cards from the top used content <ul><li>Create cards from the top content used (heat map) </li></ul><ul><li>Performed card sorting exercises on different segments </li></ul><ul><ul><li>Open and closed </li></ul></ul>
    50. 50. Analyse how they think – Open card sorting <ul><li>Take the top content and get staff to openly verbalise what they would name it, how they would categorise it </li></ul><ul><li>Do this for multiple staff segments and notice the language used, and common trends </li></ul>
    51. 51. Search log analysis - Molly Molly Intranet
    52. 52. Determine the new site structure (high level)
    53. 53. Start to build your new structure
    54. 54. Design multiple ways to search/navigate Wireframes
    55. 55. Collaborating with customers for user satisfaction and efficiency <ul><li>Created Wireframes – conceptualise the outcomes we wish to achieve in next three years </li></ul><ul><li>Engaged all major stakeholders – 1:1 interviews from Frontline staff to MD </li></ul><ul><li>Published samples on the Intranet </li></ul><ul><li>Road showed wireframes at all corporate inductions </li></ul><ul><li>Road showed at senior executive offsites </li></ul><ul><li>Recorded feedback – modified designs </li></ul>
    56. 56. Refine your designs <ul><li>Narrow down the designs – 1st phase (of 3) </li></ul><ul><ul><li>Tested these designs with stakeholders </li></ul></ul><ul><li>Road show the new designs with staff </li></ul><ul><ul><li>senior managers to frontline, new inductions </li></ul></ul><ul><li>Take in feedback and modify the designs </li></ul><ul><li>Create functional specification </li></ul><ul><li>Receive more feedback from the project team </li></ul><ul><li>What happened (Aug 2009) </li></ul><ul><ul><li>Project was delayed due to financial concerns </li></ul></ul><ul><ul><li>Decided to upgrade the intranet based on all this work with current systems </li></ul></ul><ul><ul><li>Built using html and then copying that into the CMS </li></ul></ul>
    57. 59. Change Management <ul><li>The writing / design / card sort process commenced early 2008 We notified of impending change in early March via Intranet bulletin board. Removed all content apart from home page and intranet bulletin board at 31 March Completed all content pages on 8 June 2009 Notified of change for 3 weeks prior to launch including an email with instructions on how to navigate Launched 15 June 2009 </li></ul>
    58. 60. Summary - Techniques used to create new design New design & architecture Affinity diagram ROT analysis Heat map use Open Card sorting Closed card sorting Map in scope content Observations Search log analysis
    59. 61. Questions?