Importance of Enterprise Search

Enterprise Search in a User-Generated World
Strategies for Implementing & Improving Enterprise Search
| www.nonlinear.ca ottawa 613.241.2067 toronto 416.203.2997
1
Shannon Ryan – President and CEO, non~linear creations
Paul Loefstedt – Technical Specialist, Microsoft Enterprise Search Group

A Tale of Two Moore/Mooers
We all most-likely know the first Gordon Moore (1965) from his seminal paper for the journal of Electronics called “Cramming more Components onto Integrated Circuits”
Transistor density has doubled every 2 years growing from 50 in 1965 to 410 million in 2003 and close to a billion today.
Since 1965, Moore helped found Intel, personal computers, the Internet, mobile computing, etc. have a become a reality
VISIT US AT: WWW.NONLINEAR.CA

This brings us to the other Mooers, Calvin (1959)
“An information retrieval system will tend NOT to be used whenever it is more painful and troublesome for a customer to have information than for him not to have it”.
This is not a comment necessarily on usability, but rather “the impact of finding the information” and challenges it presents
Mooers law tells us that information will be used directly in proportion to how easy it is to find

What do the two Moore/Mooers tell us?
Information explosion is a by-product of Moore’s Law.
12 exabytesof information out there right now, and around 4-5 exabytes of info is now be created EVERY YEAR. (since 2005
(that sound you hear in your ears is the rising amount of information that is being created at your companies right this second.
Moore’s Law doesn’t apply to the wetware in our brains. We are still mostly running with the base models. Mooers reminds us that in designing “information retrieval systems” we need to have a deep understanding of our users and the social context they operate in. Users are complex, Users are social, so is information.

About NLC
5

Our executive summary
Founded in 1995
55+ full-time specialists
Offices Toronto, Ottawa, Calgary with global client base
Privately held, profitable & stable
Proven expertise in implementation of leading software packages, systems integration, and custom development
Our Goal: client success, full stop.
Our Approach: leverage the potential of internet technologies to deliver business value
Our Result: extensive list of reference-able clients in private and public sector
6
Technology Partnerships

NLC’s areas of focus
7
DIGITAL STRATEGY
BUSINESS PERFORMANCE
BRAND MANAGEMENT & EXPERIENCE DESIGN
CUSTOMER & EMPLOYEE SOLTIONS
DIGITAL MARKETING & ANALYTICS
ENTERPRISE CUSTOMER & EMPLOYEE SOLTIONS
Solutions Groups
EMAIL MARKETING
CONTENT MANAGEMENT
SEARCH
SEARCH ENGINE MARKETING
COLLABORATION
WEB ANALYTICS & INTEGRATION
C USTOM APPLICATIONS & BUILDS
APPLICATION DEVELOPMENT & INTEGRATION

A few of our clients
8

Our thought leadership
NLC thoughts…
Sitecore Best Practices: A Strategy Guide
Proven Approaches for Success with the Google Search Appliance
SEO and CMS: Implementing Best Practices
Planning for Success: Best Practices in CMS Governance
The NLC Performance Framework: A pragmatic guide to planning online business initiatives
Speaking engagements …
What’s New in Social Networking? at the SMX conference in New York, 2008, 2009
Enterprise 2.0: Behind the Firewall for the Toronto Board of Trade, 2008
Best Practices in Enterprise Search at Gilbane CMS San Francisco , 2007, 2008, 2009
Leveraging Social Networks at the SMX Social Marketing Conference LA, 2008
Rich Application Interfaces for Sharepoint at the Sharepoint Summit Montreal 2008
Best Practices in Microsoft Search at ECM Days in Ottawa 2008
Best Practices in Social Computing at Gilbane CMS San Francisco 2008, 2009
9

Enterprise Search Solution Group
Established in 2002
Today the majority of NLC work can be categorized as:
Fixing broken Intranet search
Implementing large-scale Enterprise & Internet search applications
Tuning existing search engines using 80:20 and Best Bets
Deploying small-scale crawlers
Currently engaged in FAST pilots from coast to coast

The Importance of Search
11

What’s the Right Question?
Should we be thing about Enterprise Search?
OR
“Does ES search actively contribute to building a better, smarter, organization that will make more qualified decisions and therefore needs to be added to the organization right now” (whew!)

Keep it simple
The average information worker spends 8.8 hours per week searching for information
As compared to only 6 hours creating content
At $75,000 per year, that’s over $14,000 per employee
* Source: IDC – Hidden Cost of Information Worker Report 2009

14
* Source: IDC – Hidden Cost of Information Worker Report 2009

Are you wasting time?
15

And the problem is only going to get worse
16

The Trouble with Search
17

Is your search broken?
Users avoid it in usability tests, and when search is specifically tested, users voice disappointment with results
Search’s business owner does not know the top ten search queries by heart
There is no ongoing maintenance process for analyzing and improving the search results
Search cannot reach all of the sections on the enterprise (and there are no plans in place for getting there)
Important queries like product names do not return all relevant information types (i.e., support as well as sales)

Why is that….
We can find out how much the Stanley Cup weighs in one query…

But…
It is almost impossible to find the last quarters sales presentation
You sure can’t find the company org. chart in a usable format
And worst of all
I found the 5th version of the .ppt template that contained out-dated information AND I just presented THAT to the board!

Answer
Enterprise search is hard
And doesn’t really exist
And it shouldn’t be your focus in the first place
Oh ya, and it isn’t about the technology – but that’s a big part!

Enterprise Search is Hard
Search engines say they use complex algorithms to help users find exactly what they want.
The legions of top Ph.Ds working for the search engines publish oodles of scientific papers on complex mathematical concepts related to search.
Recent Papers Written by some SearchEngine Guys
A complete, co-inductive syntactic theory of sequential control and state
Achieving Anonymity via Clustering in a Metric Space
An Assertional Correctness Proof of a Self-Stabilizing l-Exclusion Algorithm
On-the-fly sharing for streamed aggregation
Programmable clustering
Using Many Machines to Handle an Enormous Error-Correcting Code

Enterprise Search is Hard
Search engines can’t find what they don’t know exists
Data is often secure or proprietary in nature
High stakes
Wrong or difficult to access format
Dropping an “appliance” into an organization without knowing what you are searching for is not an answer.

Enterprise Search is Hard: Content
Clutter
Many of the same problems as the Internet – where the solution is to improve ranking
Lack of in-link analysis really hurts this for an Intranet
We have to come up with different strategies
Disparate repositories
Different technologies (databases, LOB Apps)
Managed by different groups
Different navigation and metadata schemes
“Over the wall” attitude
Content is our job, search is your job.
Not always a lot of interaction between search team and content owners
“Wild west” – no centralized strategy

Enterprise Search is Hard: Security
Five cases – for EVERY piece of content
The content shouldn’t be in the search indexes at all
The content should be in the index, but only be returned for certain individuals
The content should be returned in results, but different information should be shown in the results for different individuals, and access to the full document should be restricted
The content should be returned in results for everyone, but gaining access to the full document should be restricted
The content should be returned and accessible to everyone

Enterprise Search Doesn’t Exist Anyway
I don’t know of a single organization that has truly implemented Enterprise Search
Why?

Enterprise Search Doesn’t Exist Anyway – But it should
Improved customer service by presenting information more quickly
Can contribute to higher sales by making crucial information more available to the sales force
They can leverage intellectual capital by finding, rather than recreating, existing information
They can foster innovation through improved discovery and can contribute to shortening response and development cycle times.
BUT ULIMATELY
A well executed enterprise search solution will (hopefully) help your organization make better decisions

Enterprise Search Doesn’t Exist Anyway – But it should
Some numbers:
Average Knowledge worker spends about 9 hours a week looking for information
1000 Knowledge workers
20% increase in their ability to find the information they are looking for
1800 hour a week in savings
$75K a year worker (2080 hours - about 36 bucks an hour)
$36 X 1800 X 50 weeks
$3.24 MILLION DOLLARS
Okay, I am off by 70% but that number is still close to a million dollars!

Don’t Aim for Search – Aim for Knowledge!
Search is, somewhat, a counter-productive exercise, since search is truly a subset of information seeking, and information seeking is a subset of knowledge. (I know which one I want my company focusing on).
Lets think about the following:
Data – a string of identified, but unevaluated symbols
Information - evaluated, validated or useful data
Knowledge – information in the context of understanding
Many people prefer browsing over searching
Most truly advance search techniques are above your users.
Search is about recovery, discovery – Knowledge is about exploration

AND DON’T FORGET: it’s about the people
31

What’s the best way to search?
What’s the best UI?
What’s the best relevance engine?
32

Answer:
It depends on who’s looking, and what they’re looking for.
33

34

35

36

37

38

39

40

41

Long Live Role-Based Search
Tina has several roles within a company
As an employee, Tina wants the same results as everyone else when searching for
Holidays
Company information
Cafeteria menus
As an engineer, however, Tina assumes a different meaning for ‘Application’ than Frank in HR
42

43

A far as technology goes…
Dropping in a “search appliance” doesn’t work – trust me
Don’t default the ownership of search to IT – just because it runs on a computer doesn’t make it their problem
Good knowledge is predicated on good information and better data. Microsoft owns this part of the food chain and therefore should own knowledge, and that’s what we want to buy.

Know your Users, Context
IT’S ABOUT YOUR USERS ACCESSING RECOVERY, DISCOVERY AND EXPLORATION:
Can users find the search box?
Are we indexing the right information?
Are we protecting information that should not be searchable?
Most searchers enter only one or two-word queries, what can we do on the backend to improve that query before sending it to the search engine?
What search terms are people using that are not returning results?
What search terms are people using that are returning too many or inappropriate results?
Once the results are returned, what can be done to categorize them, describe them, rank them.
What next steps can we offer the user if their initial results do not answer their question?

Top 10 Tips for Better Search Results
1. Resist Clutter
2. Use Icons and Graphics When Appropriate
3. Offer Both Native and HTML Views of Binary Documents
4. Make it Easy to Refine Searches
5. Searching for Nothing is Common: add "Null Search" Page(s)
6. Finding Nothing is also Common: add a "Zero Results" Page
7. Explain Your Search Syntax, It's Not Obvious
8. If Your Document Dates are Always Recent, Check Them Again
9. Omit or Truncate the Display of Long URLs in the Search Results
10. Let Your Users Decide What's Relevant
46

The Search Landscape
47

‘Enterprise’ Search Engines
AskMeNow: S3 - Semantic Search Solution
Concept Searching Limited: concept search products
Dieselpoint: Search & Navigation
dtSearch: dtSearch Engine(SDK), dtSearch Web
Endeca: Information Access Platform
Exalead: exaleadone:enterprise
Expert System S.p.A.: Cogito
Fast Search & Transfer: Enterprise Search Platform (ESP),
Funnelback: Funnelback Search
ISYS Search Software: ISYS:web, ISYS:sdk
Microsoft: Search Server
Northern Light
Open Text: Hummingbird Search Server, Livelink Search
Oracle Corporation: Secure Enterprise Search 10g
SAP: TREX
TeraText: TeraText Suite
Vivisimo: Vivisimo Clustering Engine
X1 Technologies : X1 Enterprise Search
ZyLAB Technologies: ZyIMAGE Information Access Platform
* Source: wikipedia.org
48

Gartner Magic Quadrant ‘02-’06
49
Execution
Vision

Content Focus
50
Google
Open Text
Relevance Algorithm is Key
Search and synthesis
Web sites and document libraries
Un-
structured
Structured
Endeca
Browsing/discovery
Product catalogue
FAST ESP
Combined
Databases & transactions
Autonomy/Verity
Dashboard Views
Task- and role- based search
Knowledge management & disovery

Real World Results
51

Accenture - Knowledge Exchange
52

Optimizing simple processes-BluePages
Estimated to save each employee ~72 minutes per month
Employees ~400,000
At $75,000 average salary
That’s $244, 000, 000 per year!
(or .25% of revenue)
* Source: Jacob Nielsen’s 10 Best Intranets 2006
53

Search For Knowledge Management
Norton Rose: 2000 employees (legal services)
Consolidated search across internal systems and relevant external sources
Estimated return: GBP 500,000 / year
Using FAST ESP
Integrates internal and external sources
Search in a range of languages
54