Enterprise Search in SharePoint 2010

Enterprise Search in SharePoint 2010
Bob GermanTechnology ArchitectMicrosoft Technology Center | Boston

Agenda
SharePoint Search Options
SharePoint Search Engine
FAST for SharePoint
Social Search
Content Acquisition
Federation

You Asked, We Listened!
Scale – content and query scale becomes a non-issue
User experience –guide users with rich, visual navigation
Social and people search – search unlocks the value of the people and expertise
Relevance –improving our matching and ranking capabilities and language support
Platform –tools to customize, extend, or build new applications.
Management & reliability –reliable crawling, scriptable admin, improved monitoring.
Rich Content Processing–extract metadata from my enterprise content

A Family of Search Solutions
Quick, easy, powerful search (for free!)
Complete intranet search
High-end search delivered through SharePoint
Family Values
Common UI Framework
Social search features and integration
SharePoint platform integration
End user and site administrator enablement
Common Connector Framework (BDC)
APIs and developer experience
Admin & deployment capabilities
Operations advantages (SCOM, scripting)

FAST Search Extends SharePoint Server
Superset of capabilities; common base functionality and platform
Best High-End Content Search based on FAST ESP
FAST Search
for SharePoint
Common connectors and content APIs
Common query and federation APIs
End
Users
Content+
Profiles
Unique Social Search from SharePoint Server
SharePoint Search
Queries
Common developer experience
and IT Pro Foundation
People
Content

SharePoint Search Engine
What’s new in 2010?

SharePoint 2010 Shared Service Applications
SSP
Search
UserProfiles
Excel Calc
UserProfiles
Search
HRBusinessData
Visio
Excel Calc
BusinessData
OfficeWeb Apps
3rd party Service
IT Search
http://itweb/
http://hrweb/
http://hrweb/
http://itweb/
Corp Farm
Corp Farm
SharePoint 2007 Model
SharePoint 2010 Model

SharePoint Search SA
Each Search Service Application (SSA):
Administration component , database
Query Topology
Query Components (1+)
Property Databases (1+)
Crawl Topology
Crawl Components (1+)
Crawl Databases (1+)
Services (Query, Admin)

Default Search Configuration
Out of the Box:
1 Admin component
1 Crawl component
1 Query component
Databases on drive specified in post-setup configuration
Fully functional:
Crawl content
Develop customizations
Query
Crawl
Admin
Property DB
Crawl DB
Admin DB
(other SharePoint DBs)

Component Scale-out
Query
Q2
Q1
Q2
Q1
Q6
Q5
Q4
Q3
Query:
Segregate
Partition for scale
Passive mirrors provide failover
Active mirrors increase throughput
Crawl
Additional crawl components
Crawl Database
Q2
Q1
Q1
Q2
Q5
Q6
Q3
Q4
Query
Crawl
Crawl
Admin
Crawl
Crawl
100 million items!
Property DB
Crawl DB
Crawl DB
Admin DB
Crawl DB
Admin DB

Manageability – Monitor Crawl
“Levels” of reports
Current rate
Crawl rate per type
Crawl processing
Filter by SSA, time
Optimize crawl resources to achieve freshness goals.

Manageability – Monitor Query
“Levels” of reports
Current rate
Overall Query Latency
Detailed Backend Query Latency
Optimize query latency
Enable verbosequery latency, trend

FAST Search Engine
High-end Search for SharePoint

Go Beyond the Search Box
Visual, Conversational Search
Sorting on any property
Scrolling PowerPoint
Previews
Visual Best Bets
Thumbnails
Refinement with counts
on any property
Similarity Search

FAST for SharePointArchitecture
Users
SearchCenter
ContentConnectors
SharePoint
FAST
CCF
SharePointAPI’s
Document
Processing
Indexing
Search
Query &Result
Administration
FAST Search Engine Shared with SharePoint Search Engine

Content Pipeline
Advanced Content Processing Out-of-the-Box
Processing & refinement
Configurable content processing pipeline
Transform and process content automatically
Strong linguistic coverage
80+ languages detected
45 languages with advanced linguistics features
Property Extraction to create new metadata
Prebuilt extractors: Person, Location, Company
Generic extractors to extend
Extensibility
Location
Company
Company
Company
REDMOND, Wash., andOSLO, Norway — Jan. 8, 2008
Microsoft Corp. (Nasdaq “MSFT”) today announced that it will make an offer to acquire Fast Search & Transfer ASA (OSE: “FAST”), a leading provider of enterprise search solutions, through a cash tender offer for 19.00 Norwegian kroner (NOK) per share. This offer represents a 42 percent premium to the closing share price on Jan. 4, 2008 (the last trading day prior to this announcement), and values the fully diluted equity of FAST at 6.6 billion NOK (or approximately $1.2 billion U.S. ). FAST’s board of directors has unanimously recommended that its shareholders accept the offer.
Mapper
…
Configurable
Stages
EntityExtraction
Language
Detection
Format
Conversion

Enable Conversational Search
Deep Refiners with Exact Counts
Search results can be refined based on:
Metadata from SharePoint (such as Author)
Enriched meta data based on entity extraction (such as Company)
Custom refiners can be made by:
Enabling new Managed Properties to be used as refiners
Configuring the Refinement Web Part to display the new refiner

User Context Matters
Renee Lo, Engineer
What should I know about implementing ERP?
Alan Brewer, Sales
What should I know about selling ERP consulting?

Large Scale Deployment
Add columns to extend document capacity
Add rows to extend query capacity or fault tolerance
Column 0
1
2
3
Indexer/Search
Indexer/Search
Indexer/Search
Indexer/Search
Row 0
Row 1
Indexer/Search
Indexer/Search
Indexer/Search
Indexer/Search

Social Search

Go Behind the Search Box
Connect with People and Expertise
Vanity search
Recently authored content
Phonetic and nickname matching
Improved result layout and hit-highlighting
Sort by relevance, name or social distance
Refine by query type, and many other pivots

Leveraging the Managed Metadata Service for better People Search
Out of box:
Responsibilities, Interest, Skills, Schools, are all taxonomy properties
Automatic refinement on relevant pivots
Extensibility
Adding a new taxonomy property to the profile store

Click through improves relevance
The wisdom of the masses helps the masses
Documents are boosted by ‘popularity’: if many users find a document useful, it must be useful.

Search suggestions
Helping users form better queries
Pre-query suggestions
Post-query suggestions

Social tags
Recall
Results with social tags are returned.
Ranking
Results with social tags are boosted.
Refinement
Users can filter results on social tags.
Results
Top social tags are displayed as metadata on each result.

Content Acquisition
Feeding the Search Engines

Making Content Searchable
Use Query Federation
Query across external indexes
Based on OpenSearch standard(http://www.opensearch.org/)
Use Indexing Connectors
Crawl, enumerate and create local index of content
Use out-of-box connectors or write custom connector

Shared Out-of-box Indexing Connectors in SharePoint 2010
SharePoint content (Protocol Handler - PH)
File shares (PH)
Web sites (PH)
People profiles (PH)
Lotus Notes™ (Connector Framework - CF)
Exchange public folders (CF)
External systems(CF)
Documentum™ content (CF)

Fast Search out-of-box Connectors
Enterprise crawler
Good for large scale crawling across multiple nodes
Supports dynamic data including JavaScript
JDBC™ connector
Simple configuration using SQL commands
Supports push based crawling
Supports change detection through checksums

SharePoint Connector
Accesses data through SharePoint web service (using Windows credentials)
Supports full crawl through enumeration of content and incremental crawl through change log
Built-in support for security trimming (Windows and pluggable)

Connector Framework
Easy to build
Write managed code instead of PH interface
Built connectors shared across SharePoint Search and FAST search
‘No-code’ solution for Databases and Web Services
Easy to manage
Use existing user interfaces for crawl configuration
Faster iteration on connectivity using SharePoint Designer (SPD)
Shared advantages across SharePoint
Can use other BCS features such as external lists

Search page points to Profile Page or custom page to show search result
Search results page – results based on metadata
Connector Framework Architecture
Search user queries
Search Engine/Index
BCSProfile Pages or External Lists
Search BCS Runtime
Execution call routing through BCS
Connectors using BCS model file
Fetch data from repository
WCF
Databases
.NET BCSConnector
Custom BCSConnector
Protocol Handlers

Enabling Connectivity
Every indexing connector needs
A BDC model file to express connection information and the structure of the backend
A BDC Connector for code to execute when accessing the backend
Out-of-box BDC Connectors
Database
WCF (Web Services)
.NET (custom code)

Federation
Combining Search Results

Eliminate Compromise
Easily Connect to Corporate Assets that live Outside of SharePoint
Files
OpenSearch Federation
Web
Databases
SharePoint
Easy to Connect
Easy to Build
Exchange Server
Enterprise Content Mgmt Systems
Line of Business Apps
Custom Apps
Asset Mgmt systems
Documentum
Indexing Connectors
Lotus Notes
…

“Out of the Box” Federation
People Search and Related Searches
Bing and other OpenSearch providers
Windows 7 and IE8 searching SharePoint 2010
Rich RSS feeds and federation generator

How OpenSearch federation works
HTTP request
Search Client
Search Provider
RSS/Atom Results
HTTP request with query in the URL:
http://www.site.com/srchrss.aspx?q={searchTerms}
RSS / Atom results:
RSS results with <title>, <link>, <description>
Best sources also include: <pubdate>, <author>, <category>, <media:thumbnail>
Optionally include custom metadata: <recordid>, <projectname>, <contactnumber>

Deploying a federation connectorUse or create a searchable feed
Search Client
OpenSearchPage
Search Provider
Use an existing searchable feed or create your own.
Anything searchable– takes a query as a URL parameter.
Must return results in RSS or Atom formatted XML.
Creating a searchable feed is simple. Just code a basic ASPX page.

Make Search Pervasive!Deploy OpenSearch (OSDX files) Everywhere
SharePoint 2010:
Import the file.
Add it to a Search Center.
Windows 7:
Pull: Add a link to the OSDX file on the search center or team site.
Push: Deploy the OSDX file via Group Policy.
IE8
Link to Search Provider
Discoverable Search Provider

Extend and Create Search Web Parts
The OOB Web Parts are no longer sealed!

Microsoft Enterprise Search - Productivity
Products for Every Search Need

FAST Search for SharePoint and ESP
A new choice; a different world
Bringing high-end search to the mainstream market

© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

End User perspective
Go Beyond the Search Box

IT Pro perspective
Eliminate Compromise

Do More with Search
Developer Perspective

Search Technology Concepts
Concepts
Search Center - UI for users to issue queries and interact with results
Federated Source
Query Object Model
Query Servers- Accept query requests from users and return results
Index Files
Query Servers
Query Federation - Return results from non-SharePoint Indexes
Indexing - Extract information from items to enable efficient matching
Indexer
Index Partition - Subset of the overall index
Crawling - Traverse URL space to record items in search catalog
Crawler
Indexing Connectors - Know how to process different content sources
Content Sources - Host the content we want to return in main results
Content
Content
Content

http://www.slideshare.net	296
http://www.techgig.com	4
http://webcache.googleusercontent.com	2

Enterprise Search in SharePoint 2010

by bgerman on Mar 02, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

3 Embeds 302

Statistics

Enterprise Search in SharePoint 2010 — Presentation Transcript