Successfully reported this slideshow.

Search Quality Management

1

Share

Loading in …3
×
1 of 26
1 of 26

Search Quality Management

1

Share

Enterprise Search is still considered as a one-time IT Project in most cases, although it should rather be a Business Process, with well-defined lifecycle, metrics and analytics. Measuring and controlling success is very challenging – in most cases, it has to be a string collaboration between internal and external experts. In this session, I’m introducing several roles of this process as well as useful metrics and best practices. Attendees will get a practical plan for quality management of their Enterprise Search solution as the key takeaway.

Enterprise Search is still considered as a one-time IT Project in most cases, although it should rather be a Business Process, with well-defined lifecycle, metrics and analytics. Measuring and controlling success is very challenging – in most cases, it has to be a string collaboration between internal and external experts. In this session, I’m introducing several roles of this process as well as useful metrics and best practices. Attendees will get a practical plan for quality management of their Enterprise Search solution as the key takeaway.

More Related Content

Related Books

Free with a 14 day trial from Scribd

See all

Related Audiobooks

Free with a 14 day trial from Scribd

See all

Search Quality Management

  1. 1. Search Quality Management Agnes Molnar Enterprise Search Consultant, Founder Search Explained
  2. 2. Introduction – Agnes Molnar  8 years MVP (SharePoint Server & Office 365)  Enterprise Search, Information Architecture  Community Leader  Enterprise Search Consultant, Founder  Worldwide Consultancy, Guidance, Trainings, Workshops  Web: http://SearchExplained.com  Yammer: http://yammer.com/SearchExplained  Twitter: @molnaragnes / @SearchExplained  E-mail: agnes.molnar@SearchExplained.com
  3. 3. Agenda  Enterprise Search – Real World Challenges  What is Enterprise Search?  Quality of Enterprise Search  Action Plan
  4. 4. Enterprise Search – Real World Challenges
  5. 5. Search is Easy!?
  6. 6. Find is the REAL Challenge!
  7. 7. Search = Users + Content + Context Content UsersContext
  8. 8. Common Requirements  ‘I want Google-like Search in my Enterprise’  ‘I want it just work.’  ‘We have 666 million documents. I want to be able to find everything.’
  9. 9. Formal Requirements Business goals Scenarios & use cases Application Lifecycle Systems to integrate User Experience Metrics Milestones Resources Experts
  10. 10. Enterprise Search – IT Project?  Content evolves  Users evolve  Context evolves  Search has to evolve, too!  Add/Modify content source  Add/Modify/Remove New metadata  Modify/Improve UI  New Search-Driven Applications  Obsolete content  … Content UsersContext
  11. 11. Enterprise Search – Business Process! Planning ImplementationAnalysis
  12. 12. What Makes a Good Enterprise Search?
  13. 13. Search Quality  Quality of Results  Relevance of Results  Quality of User Experience  Supportability and Maintainability Quality of Information Architecture!!!
  14. 14. Quality of Results  Various Content Sources  Various Types of Content  Structured and Unstructured Data  Security is critical  Metadata is essential  User tagging  Auto-tagging  Advanced features vary system by system
  15. 15. Relevance of Results  The order of results  Enterprise SEO = Information Architecture!  Ranking – METADATA based! Custom Ranking Model On-Prem only Query Rules and XRANK On-Prem + Cloud
  16. 16. Supportability  Well-defined processes  Well-defined roles  Experts involved  Consistent Management of  Content Sources  Search Metadata  Search Configuration (Result Sources, Query Rules, …)  User Interface (Results, Refiners, Hover Panel, custom components, …)
  17. 17. Enterprise Search – Roles Team Members Internal External Liaison Business Sponsors Stakeholders Project Management IT Search Admins Developers … Consultants Service Providers Product Vendors Developers Act as Internal Have Wide Search Experience and Knowledge Understands the Business Well Organized Liaise between Internal + External Quality Management “Active Insurance”
  18. 18. How we Know Search is Good (Enough)? Users don’t complain Users keep using it
  19. 19. Search Metrics Number of Searches Number of successful Searches Number of Searches with Zero results Results with no click Most popular Queries Results clicked the most Who’s performing the most searches Correlated searches Speed / latency Total distinct search terms Total distinct words Trends and historical data/charts
  20. 20. Search Analytics What Metrics? Who? How often? What to do with the results?
  21. 21. Action Plan
  22. 22. Action Plan  Start a Discussion!!  Evaluate your current Search solution  Evaluate your Content  Collect Use Cases / Stories  Identify Users / Audiences Is Your Search Good Enough???
  23. 23. Action Plan  Collect Feedbacks  Analyze  Take Actions  Search Analytics Reports  Analyze  Take Actions  Errors/warnings in the Crawl Log  Analyze  Fix  Evaluate if any new Content Source is needed  Evaluate if any new Type of Content is needed  Improve Metadata  Improve the UI Measure and Analyze! Planning Implemen tation Analysis
  24. 24. Keep in Mind: FINDABILITY is more than SEARCH
  25. 25. Thank You!  Web: http://SearchExplained.com  Yammer: http://yammer.com/SearchExplained  Twitter: @molnaragnes  E-mail: agnes.molnar@SearchExplained.com

Editor's Notes

  • Display Templates
    Query Builder
    Result Sources
    Query Rules
  • “An average person today processed more information in a single day
    than a person in the 1500’s did in an entire lifetime.”
  • Identifying them is critical – but not easy at all!!!
  • Resources
    Organization and resources

  • Search has a Complex Lifecycle
  • + Maintenance
    + Administration

    Iterations
    Agile
    Start in Small, roll out to big
  • Content Freshness
  • http://blog.leeromero.org/2009/01/20/search-analytics-basic-metrics/

    Let’s make it to be more formal!
  • Starting discussion might be painful
  •  our session tomorrow

    Making Sense of the Information Jungle and the Path That Lies Ahead
  • ×