Enterprise Search. Approaches to enable effective
                                        Scalability in a secure collaborative environment.

                                        KM World 2011


                                        Concept Searching, a Microsoft Managed ISV
Martin Garland, President and Founder
Concept Searching Inc.
marting@conceptsearching.com
+1 703 531 8567
Twitter @conceptsearch



  www.conceptsearching.com
Agenda

      Is Enterprise Search doomed to failure?
      What is scalability?
      What is the ‘real’ question?
      What is the ‘real’ problem?
      What is Enterprise Content?
      How much unstructured data to you have?
      Building Block #1 – Manage your content
      Building Block #2 – Eliminate the end user
      Building Block #3 – Protect data at risk
      Building Block #4 - Identify and tag your assets for storage and preservation
      How others are doing it
      Technology
      Recommendations
      Case Studies
      Who we are




www.conceptsearching.com
Is Enterprise Search
                                                     doomed to failure?


 In spite of 10 years of advances in Enterprise Search products, less than
  22% of organizations have purchased the technology (Down from 24% in
  2008)
 Less than 10% use it searching more than four data sources
 56% rank search at the bottom third of their project lists
                                              (Go Rogue With Enterprise Search, Information Week)
 Many factors contribute to the inability of workers to find unstructured
  information including: redundant and out-of-date content, incomplete
  search scope, lack of information retrieval expertise and lack of
  information governance (Forrester)
 Solutions still fall short of end user expectations (Enterprise Search is not
  Google)
 Show me the ROI
                                Why?
www.conceptsearching.com
What is scalability?


 Scalability?
    Performance
    Number of documents
    Types of documents
    Number of users
    Number of documents, web pages, records
    Geographic footprint
    File types, number of applications
    Functionality (bells, whistles, must haves,
     nice to haves)
           Steve Weissman, Principal Analyst at consulting firm Holly Group
           and President of AIIM’s New England Chapter




www.conceptsearching.com
What is the ‘real’ question?

            What do I need for scalable Enterprise Search?
                                 OR…
 What do I need for scalable Enterprise Search with meaningful results?



       “By itself the search function has limited value. The real value of
     search and information access technologies is in the ongoing efforts
        needed to establish effective taxonomies, to index and classify
         content of all kinds, in order to provide meaningful results.”
                                         Tom Eid, Technology and Research VP at Gartner




www.conceptsearching.com
What is Enterprise Content?




www.conceptsearching.com
How much unstructured data do
                                                   you have?
80% of Enterprise Data is Unstructured (IBM)
60% of Documents are Obsolete (e.Law)                      Building Block #1
50% of Documents are Duplicates (equivio)
40%+ Annual Growth (Ventana Research)                    Manage Your Content
In 2009 there were 100,000,000 SharePoint users      Consistent Classification to the
(Microsoft)
Every day for the past 5 years 20,000 new SP users        Corporate Structure
(Microsoft)
One in five users has access to SharePoint
(Microsoft)




 www.conceptsearching.com
What happens today




             Access
             Rights


 Records
 Retention
   Code                      Server Content with
               Metadata
                           Appropriate Metadata,      Document Library 1   Document Library 2
               Tagging      Retention Codes, and
                            Rights Management
                                 Templates




                                                      Document Library 3   Document Library 4

                           www.conceptsearching.com
www.conceptsearching.com
• Limiting Factor = Human Behavior

• Incorrect Metadata  Incorrect Content Type  Incorrect Policy Application


             Access
             Rights


 Records
 Retention
   Code                        Server Content with
               Metadata
                             Appropriate Metadata,      Document Library 1   Document Library 2
               Tagging        Retention Codes, and
                              Rights Management
                                   Templates




                                                        Document Library 3   Document Library 4

                             www.conceptsearching.com
www.conceptsearching.com
You say potato I say potahto


Less than 50% of content is correctly indexed, meta
tagged or efficiently searchable (IDC)

85% of relevant documents are never
retrieved in search (IDC)

End users - subjective, in a hurry, disinterested, etc.

Align Content with Corporate Goals or Mission


    Building Block #2 – Eliminate the End User
      Address the Process not the Behavior




www.conceptsearching.com
Metadata and Transparency

     Natural Language Query on Search Solution with semantic metadata applied to all content -
                              Do caskets need to be pressurized?




www.conceptsearching.com
Data Transparency




www.conceptsearching.com
Final Result




www.conceptsearching.com
Same Query on Platform with
                           poorly applied Metadata




www.conceptsearching.com
Same Query on Platform w/no
                                     Metadata Tagging




www.conceptsearching.com
Must use keywords to find document
                                  when no metadata is applied




www.conceptsearching.com
What is more stressful than getting
                                     a divorce or losing your job?
                       72% of IT Managers felt protecting company data is more
                       stressful than getting a divorce, losing your job, managing
                       personal debt, or being in a minor car accident (Websense
                       Survey)
                       Typically IT has not been involved in the security process
                       details (Websense Survey)
                       70% of breaches are due to a mistake or malicious intent by
                       end users, 88% are attributed to negligence (Wharton Information
                       Security Best Practices Conference)
                       Average cost per exposed record is $197 and ranges from
                       $90 to $305 (Ponemon Institute)
                       Average loss in value of brand ranges from $184 million to
                       $330 million+ (17% - 31% decline) (Ponemon Institute)

                       Leverage Content Types to drive Information Rights
                       Management


         Building Block #3 – Apply Metadata Driven Policies
                      To Protect Data at Risk

www.conceptsearching.com
What happens when appropriate
                                                       policies are not applied to captured
                                                       content?

 Protected Health
                       Travel Vouchers       Alpha Rosters
   Information


Operational Security                         Documents of
                        Duty Rosters
   Information                                 Record




                       Server Content with
                          No Semantic,
                       Retention Code, and
                        Security Metadata             Web Servers/Collaboration Portals



www.conceptsearching.com
Those darn end users, they
                                                    just don’t get it!


 67% of data loss in records management is
  due to end user error (Prism Intl)
 It costs an organization $180 per document
  to recreate it when it is not tagged correctly
  and cannot be found in search (IDC)
 Large organizations lose a document every
  12 seconds (Prism Intl)
 Align corporate goals with records policies
  and file plans with content types.
 Drive Content Types with metadata



     Building Block #4 – Apply Metadata Driven Policies to Identify
             & Tag Your Assets for Storage & Preservation


                                 www.conceptsearching.com
www.conceptsearching.com
Solution: Address the
                                   Technology/Process Not the Behavior


                                                        Semantic
                                                        Metadata
                                                        Tagging            Increase
                                                                         Information
                                                                           Retrieval
                                                                         Precision for
                                                                         e-Discovery



                                                Concept
                                              Classifier for       Automatic
                                               SharePoint           Content
                                                                     Type
                                                                   Application
                                                                                  Windows
                                                                                   Rights
  Document     Document                                   Records
                                                                                 Management
   Library 1    Library 2                                 Retention                  &
                                                            Code                  Workflow
                                                           Tagging


                                                                      Appropriate
                            Backup &                                   Storage &
 Document      Document     Archived Data                             Preservation
  Library 3     Library 4


www.conceptsearching.com
Semantic
                                                      Metadata
                                                      Tagging            Increase
                                                                       Information
                                                                         Retrieval
                                                                       Precision for
                                                                       e-Discovery



                                              Concept
                                            Classifier for       Automatic
                                             SharePoint           Content
                                                                   Type
                                                                 Application
                                                                                Windows
                                                                                 Rights
  Document     Document                                 Records
                                                                               Management
   Library 1    Library 2                               Retention                  &
                                                          Code                  Workflow
                                                         Tagging


                                                                    Appropriate
                            Backup &                                 Storage &
 Document      Document     Archived Data                           Preservation
  Library 3     Library 4


www.conceptsearching.com
Summary
                                                                              Recommendations
 Its not about better search, but the Proactive
  Management of the Life cycle of content

 Find tools that run natively in SharePoint
     Reduce costs, time, & risk
     Leverages your investment
     Leverage Content Types

 Align your taxonomy(s) with your organization,                                    SharePoint
  one size does not fit all                                        Metadata Driven Automatic Application of Policies
                                                                                  & Content Types

 Identify tools that are highly interactive and do
  not require Information Scientists on staff                                     Enterprise Search


 Integration with your search solution
     Navigation and improved findability
                                                      IBM File Net P8       Opentext               SAP                 File Shares
 Look for rapid deployment and ease to
  manage and maintain
                                                      ROI – 38% to 600%
                                                                                                       (IDC)
 Vendor experience



 www.conceptsearching.com
Scalability, regardless of how you define it, is ultimately the
       intersection of technology and business processes to achieve
         quantifiable organizational improvements impacting search,
      records management, compliance, data privacy, and governance.

          Overcoming traditional challenges, relevant information is
       delivered timely, to the right stakeholder, in a secure, compliant,
                        and collaborative environment.

                                            Martin Garland KM World 2011
Martin Garland, President and Founder
Concept Searching Inc.
marting@conceptsearching.com
+1 703 531 8567
Twitter @conceptsearch



  www.conceptsearching.com

KMWorld Martin Briefing

  • 1.
    Enterprise Search. Approachesto enable effective Scalability in a secure collaborative environment. KM World 2011 Concept Searching, a Microsoft Managed ISV Martin Garland, President and Founder Concept Searching Inc. marting@conceptsearching.com +1 703 531 8567 Twitter @conceptsearch www.conceptsearching.com
  • 2.
    Agenda  Is Enterprise Search doomed to failure?  What is scalability?  What is the ‘real’ question?  What is the ‘real’ problem?  What is Enterprise Content?  How much unstructured data to you have?  Building Block #1 – Manage your content  Building Block #2 – Eliminate the end user  Building Block #3 – Protect data at risk  Building Block #4 - Identify and tag your assets for storage and preservation  How others are doing it  Technology  Recommendations  Case Studies  Who we are www.conceptsearching.com
  • 3.
    Is Enterprise Search doomed to failure?  In spite of 10 years of advances in Enterprise Search products, less than 22% of organizations have purchased the technology (Down from 24% in 2008)  Less than 10% use it searching more than four data sources  56% rank search at the bottom third of their project lists (Go Rogue With Enterprise Search, Information Week)  Many factors contribute to the inability of workers to find unstructured information including: redundant and out-of-date content, incomplete search scope, lack of information retrieval expertise and lack of information governance (Forrester)  Solutions still fall short of end user expectations (Enterprise Search is not Google)  Show me the ROI Why? www.conceptsearching.com
  • 4.
    What is scalability? Scalability?  Performance  Number of documents  Types of documents  Number of users  Number of documents, web pages, records  Geographic footprint  File types, number of applications  Functionality (bells, whistles, must haves, nice to haves) Steve Weissman, Principal Analyst at consulting firm Holly Group and President of AIIM’s New England Chapter www.conceptsearching.com
  • 5.
    What is the‘real’ question? What do I need for scalable Enterprise Search? OR… What do I need for scalable Enterprise Search with meaningful results? “By itself the search function has limited value. The real value of search and information access technologies is in the ongoing efforts needed to establish effective taxonomies, to index and classify content of all kinds, in order to provide meaningful results.” Tom Eid, Technology and Research VP at Gartner www.conceptsearching.com
  • 6.
    What is EnterpriseContent? www.conceptsearching.com
  • 7.
    How much unstructureddata do you have? 80% of Enterprise Data is Unstructured (IBM) 60% of Documents are Obsolete (e.Law) Building Block #1 50% of Documents are Duplicates (equivio) 40%+ Annual Growth (Ventana Research) Manage Your Content In 2009 there were 100,000,000 SharePoint users Consistent Classification to the (Microsoft) Every day for the past 5 years 20,000 new SP users Corporate Structure (Microsoft) One in five users has access to SharePoint (Microsoft) www.conceptsearching.com
  • 8.
    What happens today Access Rights Records Retention Code Server Content with Metadata Appropriate Metadata, Document Library 1 Document Library 2 Tagging Retention Codes, and Rights Management Templates Document Library 3 Document Library 4 www.conceptsearching.com www.conceptsearching.com
  • 9.
    • Limiting Factor= Human Behavior • Incorrect Metadata  Incorrect Content Type  Incorrect Policy Application Access Rights Records Retention Code Server Content with Metadata Appropriate Metadata, Document Library 1 Document Library 2 Tagging Retention Codes, and Rights Management Templates Document Library 3 Document Library 4 www.conceptsearching.com www.conceptsearching.com
  • 10.
    You say potatoI say potahto Less than 50% of content is correctly indexed, meta tagged or efficiently searchable (IDC) 85% of relevant documents are never retrieved in search (IDC) End users - subjective, in a hurry, disinterested, etc. Align Content with Corporate Goals or Mission Building Block #2 – Eliminate the End User Address the Process not the Behavior www.conceptsearching.com
  • 11.
    Metadata and Transparency Natural Language Query on Search Solution with semantic metadata applied to all content - Do caskets need to be pressurized? www.conceptsearching.com
  • 12.
  • 13.
  • 14.
    Same Query onPlatform with poorly applied Metadata www.conceptsearching.com
  • 15.
    Same Query onPlatform w/no Metadata Tagging www.conceptsearching.com
  • 16.
    Must use keywordsto find document when no metadata is applied www.conceptsearching.com
  • 17.
    What is morestressful than getting a divorce or losing your job? 72% of IT Managers felt protecting company data is more stressful than getting a divorce, losing your job, managing personal debt, or being in a minor car accident (Websense Survey) Typically IT has not been involved in the security process details (Websense Survey) 70% of breaches are due to a mistake or malicious intent by end users, 88% are attributed to negligence (Wharton Information Security Best Practices Conference) Average cost per exposed record is $197 and ranges from $90 to $305 (Ponemon Institute) Average loss in value of brand ranges from $184 million to $330 million+ (17% - 31% decline) (Ponemon Institute) Leverage Content Types to drive Information Rights Management Building Block #3 – Apply Metadata Driven Policies To Protect Data at Risk www.conceptsearching.com
  • 18.
    What happens whenappropriate policies are not applied to captured content? Protected Health Travel Vouchers Alpha Rosters Information Operational Security Documents of Duty Rosters Information Record Server Content with No Semantic, Retention Code, and Security Metadata Web Servers/Collaboration Portals www.conceptsearching.com
  • 19.
    Those darn endusers, they just don’t get it!  67% of data loss in records management is due to end user error (Prism Intl)  It costs an organization $180 per document to recreate it when it is not tagged correctly and cannot be found in search (IDC)  Large organizations lose a document every 12 seconds (Prism Intl)  Align corporate goals with records policies and file plans with content types.  Drive Content Types with metadata Building Block #4 – Apply Metadata Driven Policies to Identify & Tag Your Assets for Storage & Preservation www.conceptsearching.com www.conceptsearching.com
  • 20.
    Solution: Address the Technology/Process Not the Behavior Semantic Metadata Tagging Increase Information Retrieval Precision for e-Discovery Concept Classifier for Automatic SharePoint Content Type Application Windows Rights Document Document Records Management Library 1 Library 2 Retention & Code Workflow Tagging Appropriate Backup & Storage & Document Document Archived Data Preservation Library 3 Library 4 www.conceptsearching.com
  • 21.
    Semantic Metadata Tagging Increase Information Retrieval Precision for e-Discovery Concept Classifier for Automatic SharePoint Content Type Application Windows Rights Document Document Records Management Library 1 Library 2 Retention & Code Workflow Tagging Appropriate Backup & Storage & Document Document Archived Data Preservation Library 3 Library 4 www.conceptsearching.com
  • 22.
    Summary Recommendations  Its not about better search, but the Proactive Management of the Life cycle of content  Find tools that run natively in SharePoint  Reduce costs, time, & risk  Leverages your investment  Leverage Content Types  Align your taxonomy(s) with your organization, SharePoint one size does not fit all Metadata Driven Automatic Application of Policies & Content Types  Identify tools that are highly interactive and do not require Information Scientists on staff Enterprise Search  Integration with your search solution  Navigation and improved findability IBM File Net P8 Opentext SAP File Shares  Look for rapid deployment and ease to manage and maintain ROI – 38% to 600% (IDC)  Vendor experience www.conceptsearching.com
  • 23.
    Scalability, regardless ofhow you define it, is ultimately the intersection of technology and business processes to achieve quantifiable organizational improvements impacting search, records management, compliance, data privacy, and governance. Overcoming traditional challenges, relevant information is delivered timely, to the right stakeholder, in a secure, compliant, and collaborative environment. Martin Garland KM World 2011 Martin Garland, President and Founder Concept Searching Inc. marting@conceptsearching.com +1 703 531 8567 Twitter @conceptsearch www.conceptsearching.com