1) The document discusses cloud hybrid search, which allows searching across on-premises and online content in Office 365 with a single search experience. 2) It highlights benefits like simplified administration and lower costs compared to traditional federated search. However, there are also limitations with the default configuration regarding features like security trimming. 3) The document provides guidance on implementing cloud hybrid search and considerations for different environments, including performance, customizations, and regulatory requirements.

1. Understanding and Applying
Cloud Hybrid Search
@jefffried
Jeff Fried
CTO, BA Insight

2. we love hybrid search - it's amazing how fast usage is growing
Jeff Teper @jeffteper

4. Focused on Search and
SharePoint since 2004
Longtime
Search Nerd
• CTO, BA Insight
• Senior PM, Microsoft
• VP, FAST
• SVP, LingoMotors
About Jeff Fried
Passionate About
• Search
• SharePoint
• Search-driven
applications
• Information Strategy
Blog:
BAinsight.com/blog
Technet Column
“A View from the
Crawlspace”
jeff.fried@bainsight.com

5. About BA Insight


– Connectivity
– Applications -
– Classification -
– Analytics


6. KCTCS (background)

8. Search is not stationary

9. Demo
9

10. 
–
–

–

–

–
Why Hybrid SharePoint?

11. The
Evolution
of
SharePoint:
HYBRID ManagementExtensibilityExperiences
| Server
Experiences ManagementExtensibility
| Server | Server
HYBRID
Team
Sites
Portals
Enterprise
Content Mngt
BI

12. Search Provides a Unified View

13. SharePoint 2013/2016 Search Architecture
Web Service (CEWS)

14. “Classic” Hybrid Search is Federated
not a single result set OOB

15. Cloud Hybrid Search

17. Benefits of Cloud Hybrid Search
2) Makes finding content easy, wherever the content lives
1) Simpler, easier, and less costly to run search

18. SharePoint Server
(On-premises or Hosted)
Office 365
SharePoint Online Content
Onedrive for Business Content
SharePoint Content
Cloud Hybrid Search

19. Case Study: Split Users with SharePoint






20. SupportSales & Marketing
Knowledge Articles
Fileshares
OneDrive
Support forum
SPO
Search Farm
SP 2013 content SP 2010 content
On-
premises
Office 365
SPO content
SP 2013/2016
Cloud SSA

21. Setting up Cloud Hybrid Search
•
•
1.
2.
3.
4.

23. Use search verticals with Cloud Hybrid Search
SharePoint Online
Custom result source using Local SharePoint results plus
a filter which excludes results from on-premises
TIP: Can be used during validation of hybrid search in
the production tenant.
Result source query:
{searchTerms}
NOT(IsExternalContent:1)

24. Result Sources are your friend
The Support Search vertical only searches sites that
are relevant to the Support team.
It uses Local SharePoint results plus a filter on
which sites to include in the search results
Result source query:
{searchTerms} (
Path:»http://sp2010» OR
Path:»file://fileshare» OR
Path:»http://demohybrid...
/../supportforum»)
SharePoint Online Support Search

25. Demo
25

26. Single node topology






VM
Crawler
CPC
(unused)
APC
(unused)
Indexer
(unused)
QPC

27. Multi-node topology





1.
2.
3.
VM
Crawler
QPC
VM
Crawler
CPC
(unused)
APC
(unused)
Indexer
(unused)
QPC

28. Reduce your footprint
Servers
Volume of Content
(indexable items) Pattern
On-prem Search
Farm
Cloud Hybrid
Search
0-10 million items small 4 App + 2 DB 1 or 2
10-40 million items medium 12 App + 2 DB 2
40-100 million items large 28 App + 4 DB 2
400 million items XL example (SP2016) 86 App + 4DB 2 or 3

29. Item Limits and Pricing
Licensing: 1M items of external content in index for every 1TB storage in O365
1TB included by default
+ 0.5 GB per licensed O365 user
No limit on number of items from O365 in the index
Default throttling at 20M external items; current threshold at 25M
2000 users x 0.5 GB = 1TB
+ 1TB default = 2 TB total
-> 2M external items indexed
+ Can also buy the “Office 365 Extra File Storage” Add-on
$0.20/GB/Month = $200/TB/Month = $200/M items/Month
50,000 users x 0.5 GB = 25TB
+ 1TB default = 26 TB total
-> 26M external items indexed

30. SharePoint 2016 Hybrid
Cloud Hybrid
Search
User Profiles Following
Extranet
Compliance
(DLP/e-
Discovery)
Config
Experience
Built on Search

31. Advantages
•



•



Disadvantages
Cloud SSA Pro/Con versus on-prem

33. External Content
(on-premises and/or
in the cloud)
SharePoint Server
(On-premises or Hosted)
Office 365
SharePoint Online Content
Onedrive for Business Content
Connectors
SharePoint Content
Adding External Content
Cloud Hybrid Search
Also drives:
• Office Graph (delve,..)
• Compliance (DLP, …)

34. Connectors to MANY Enterprise Systems
•
•
•
•
ERP and Portal Systems
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•

35. 

External Content in O365 UX
Unified view across all content
- on-premises and on-line
- inside and outside SharePoint

36. Current Caveats:
1) don’t see thumbnails, just file icons
2) Have to query for it to show up

37. 
–
–

–

–
Case Study:
Cloud SSA, external content
Large global company
in materials science

38. External Content
(on-premises and/or
in the cloud)
Custom
Processing
CEWS
Bottlenecks:
1) Source systems
2) Content Processing
3) Indexer
….

39. External Content
(on-premises and/or
in the cloud)
Bottlenecks:
1) Uplink
2) Source systems
….

40. 42
Performance

41. SCS under the hood
Crawler
Content
Indexing
API
Blob store
Document
state table
Work
queues
Backend
API
Index/Graph
On-Premises content source
Search farm
Azure
Broker
Crawler
Content
SPO content source

42. Her user token gets rehydrated with
her online claims as she is
authenticated against Office 365.
Cloud SSA
SPO
Search Index
Logical architecture: query
Corporate network
SP 2013
1
2a
Jaden issues a query from Office
365.
Her user token contains her online
identity and group memberships.
1
Jaden issues a query from a site on-
premises. This sends over her on-
premises claims to SPO
2a
2b
2b
Office 365

43. SUPPORTED
– Custom IFilter
– BCS connectors
– Partner connectors
Customizations with Cloud Hybrid Search
SUPPORTED
– Tenant level schema mapping
– Query rules
– Result sources
Cloud SSA SCS/O365
NOT SUPPORTED
• Content that requires custom
security trimming
NOT SUPPORTED
• Site collection level schema mapping
• Custom security trimming
• Custom entity extraction
• Content enrichment web service

44. Issues with Cloud Hybrid Search (1)
Cloud Hybrid Search "annoyances"
Performance Characteristics
slower query latency for on-prem queries against Cloud SSA
SharePoint Online Limitations
no synonyms
no site-level schema
no full trust code access
Hybrid Administration Weaknesses
clunky metadata mapping
can't remove on-premises search results from Cloud SSA
trickier to test & debug crawls
can't reset index from Cloud SSA
Be aware of these
& compensate for them
(Fixed in August PU)
(Semi-addressed in June PU)
And it’s getting better:

45. Should I run index reset?
NO!
DeleteAllCloudHybridSearchContent()
https://blogs.technet.microsoft.com/beyondsharepoint/2016/07/07/cloud-hybrid-search-service-application-removing-items-from-the-office-365-search-index/

46. Issues with Cloud Hybrid Search (2)
50
Content Enrichment
no CEWS
no Entity Extraction
Security
no Custom Security Trimming
Can't crawl across Multiple Domains
Can't Crawl SP in Classic Auth Mode
Data Sovereignty
export-restricted content
can't be put in O365 index
Limitations of Cloud SSA

47. External Content
(on-premises and/or
in the cloud)
SharePoint Server
(On-premises or Hosted)
SPO Content
OneDrive Content
Connectors
SharePoint Content
Connector
Framework
Office 365
AutoClassifier
(app version)
CEWS
Custom
Processing

48. Case study:
Content Enrichment











Content
Cloud
SSA
Connector
Framework
Indexing
Connectors
Smart
Pipeline
AutoClassifier
Custom
Stage A
Custom
Stage C
Custom
Stage B

49. Online
On-Prem
Cloud Hybrid Search under the covers
Security = identity sync + ACL mapping
Cloud SSA
Cloud SSA
ParseCrawl
SCS
ACL Map Process
Blob
store
queue

50. •
•
Directory Synchronization
SID S-1-5-21-1212121212-1212121212-1212
jaden@corp.hybridsearch.com
msOnline-
OnPremiseSecurity
Identifier
S-1-5-21-1212121212-1212121212-1212
PUID PUID-XXXX-XXXXXXXXXX

51. Mapping of Access Control Lists
Allow: S-1-5-21-1212121212-1212121212-1212 Allow: PUID-XXXX-XXXXXXXXXX
• User SIDs are mapped to PUIDs
• Group SIDs are mapped to Object IDs
• «Everyone» and «Authenticated users» are mapped to
«Everyone except external users»
Only AD Users and Groups,
Only from one domain

52. 






Case Study:
Crawling Cross-Domain

53. A global single index solution
Cloud SSA
Cloud SSA
Cloud SSA
Cloud SSA
Cloud SSA
BUT export-restricted content
can’t be in the global index

54. Issues with Cloud Hybrid Search OOB
Content Enrichment
no CEWS
no Entity Extraction
Security
no Custom Security Trimming
Can't crawl across Multiple Domains
Can't Crawl SP in Classic Auth Mode
Data Sovereignty
export-restricted content
can't be put in O365 index
Limitations of Cloud SSA BA Insight Solution
Connector Framework
AutoClassifier
Connector Framework
can 'map down' to AD groups
can 'map across' cross-domain
can crawl and map security
Federator

56. Key Considerations for Hybrid:
Workloads, Environment, Data, Customizations
Availability of features Online versus
On-Premises on particular workloads
Significant investments in
customization of On-Premises
workloads
Concerns over global network
performance with remote sites
Regulatory
considerations
Manageability concerns

61. Contact:
Jeff.Fried@BAinsight.com
www.BAinsight.com
Questions