Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Search Quality Management

580 views

Published on

Enterprise Search is still considered as a one-time IT Project in most cases, although it should rather be a Business Process, with well-defined lifecycle, metrics and analytics. Measuring and controlling success is very challenging – in most cases, it has to be a string collaboration between internal and external experts. In this session, I’m introducing several roles of this process as well as useful metrics and best practices. Attendees will get a practical plan for quality management of their Enterprise Search solution as the key takeaway.

Published in: Software
  • Be the first to comment

Search Quality Management

  1. 1. Search Quality Management Agnes Molnar Enterprise Search Consultant, Founder Search Explained
  2. 2. Introduction – Agnes Molnar  8 years MVP (SharePoint Server & Office 365)  Enterprise Search, Information Architecture  Community Leader  Enterprise Search Consultant, Founder  Worldwide Consultancy, Guidance, Trainings, Workshops  Web: http://SearchExplained.com  Yammer: http://yammer.com/SearchExplained  Twitter: @molnaragnes / @SearchExplained  E-mail: agnes.molnar@SearchExplained.com
  3. 3. Agenda  Enterprise Search – Real World Challenges  What is Enterprise Search?  Quality of Enterprise Search  Action Plan
  4. 4. Enterprise Search – Real World Challenges
  5. 5. Search is Easy!?
  6. 6. Find is the REAL Challenge!
  7. 7. Search = Users + Content + Context Content UsersContext
  8. 8. Common Requirements  ‘I want Google-like Search in my Enterprise’  ‘I want it just work.’  ‘We have 666 million documents. I want to be able to find everything.’
  9. 9. Formal Requirements Business goals Scenarios & use cases Application Lifecycle Systems to integrate User Experience Metrics Milestones Resources Experts
  10. 10. Enterprise Search – IT Project?  Content evolves  Users evolve  Context evolves  Search has to evolve, too!  Add/Modify content source  Add/Modify/Remove New metadata  Modify/Improve UI  New Search-Driven Applications  Obsolete content  … Content UsersContext
  11. 11. Enterprise Search – Business Process! Planning ImplementationAnalysis
  12. 12. What Makes a Good Enterprise Search?
  13. 13. Search Quality  Quality of Results  Relevance of Results  Quality of User Experience  Supportability and Maintainability Quality of Information Architecture!!!
  14. 14. Quality of Results  Various Content Sources  Various Types of Content  Structured and Unstructured Data  Security is critical  Metadata is essential  User tagging  Auto-tagging  Advanced features vary system by system
  15. 15. Relevance of Results  The order of results  Enterprise SEO = Information Architecture!  Ranking – METADATA based! Custom Ranking Model On-Prem only Query Rules and XRANK On-Prem + Cloud
  16. 16. Supportability  Well-defined processes  Well-defined roles  Experts involved  Consistent Management of  Content Sources  Search Metadata  Search Configuration (Result Sources, Query Rules, …)  User Interface (Results, Refiners, Hover Panel, custom components, …)
  17. 17. Enterprise Search – Roles Team Members Internal External Liaison Business Sponsors Stakeholders Project Management IT Search Admins Developers … Consultants Service Providers Product Vendors Developers Act as Internal Have Wide Search Experience and Knowledge Understands the Business Well Organized Liaise between Internal + External Quality Management “Active Insurance”
  18. 18. How we Know Search is Good (Enough)? Users don’t complain Users keep using it
  19. 19. Search Metrics Number of Searches Number of successful Searches Number of Searches with Zero results Results with no click Most popular Queries Results clicked the most Who’s performing the most searches Correlated searches Speed / latency Total distinct search terms Total distinct words Trends and historical data/charts
  20. 20. Search Analytics What Metrics? Who? How often? What to do with the results?
  21. 21. Action Plan
  22. 22. Action Plan  Start a Discussion!!  Evaluate your current Search solution  Evaluate your Content  Collect Use Cases / Stories  Identify Users / Audiences Is Your Search Good Enough???
  23. 23. Action Plan  Collect Feedbacks  Analyze  Take Actions  Search Analytics Reports  Analyze  Take Actions  Errors/warnings in the Crawl Log  Analyze  Fix  Evaluate if any new Content Source is needed  Evaluate if any new Type of Content is needed  Improve Metadata  Improve the UI Measure and Analyze! Planning Implemen tation Analysis
  24. 24. Keep in Mind: FINDABILITY is more than SEARCH
  25. 25. Thank You!  Web: http://SearchExplained.com  Yammer: http://yammer.com/SearchExplained  Twitter: @molnaragnes  E-mail: agnes.molnar@SearchExplained.com

×