Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Fried sp techcon hybrid search deeper dive


Published on

scenarios, gaps, and workarounds with the new cloud Hybrid Search

Published in: Technology
  • Be the first to comment

Fried sp techcon hybrid search deeper dive

  1. 1. Hybrid SharePoint with the new Cloud Hybrid Search Jeff Fried CTO, BA Insight
  2. 2. Cloud Search Service Application • Unified index with on-premises and cloud content • Feeds Office Graph/Delve experiences • Supports Search as a Service • Reduces search server footprint Audio text And search indexpropertiessignals Metadata extraction and processing
  3. 3. 2 Hybrid SharePoint sessions on Monday 4-5:15 – pick one, review the other later SharePoint Hybrid: The Sure Path Forward - Ben Curry Debunking the Hybrid SharePoint Infrastructure Dilemma - Jill Hannemann & Adam Levithan 2 Hybrid search sessions on Tuesday – go to one or both 11:45am The Future of Microsoft Search is Here! Cloud SSA - Jeff Fried & Ben Curry 3:45pm Hybrid SharePoint with the new Cloud Hybrid Search - Jeff Fried Hybrid Sessions at
  4. 4. Focused on Search and SharePoint since 2004 Longtime Search Nerd • CTO, BA Insight • Senior PM, Microsoft • VP, FAST • SVP, LingoMotors About Jeff Fried Passionate About • Search • SharePoint • Search-driven applications • Information Strategy Blog: Technet Column “A View from the Crawlspace”
  5. 5. About BA Insight   – Connectivity – Applications - Im – Classification - – Analytics 
  6. 6.  – –  –  –  – Why Hybrid SharePoint? 7
  7. 7. Approaches to Hybrid – by Workload Split Workload different tools in different places Split User task uses content or sites across ‘the divide’ Exchange, SharePoint, Lync OneDrive, Yammer, PowerBI, Delve Extranet, Mysites, Team Sites, Project Sites Portals, Intranet, Services/Applications Links Search
  8. 8. Online On-Prem Cloud Hybrid Search Cloud SSA Text & Metadata
  9. 9. Online On-Prem Logical Architecture: Crawling Cloud SSA Cloud SSA ParseCrawl SCS ACL Map Process Blob store queue
  10. 10. Online On-Prem Query processing Logical Architecture: Query Cloud SSA
  11. 11. Online On-Prem Logical Architecture: Query Cloud SSA Query processing
  12. 12. Online On-Prem Failure mode: what if you can’t reach the cloud? Cloud SSA
  13. 13. Online On-Prem Combination: double crawling Cloud SSA Text & Metadata
  14. 14. Mechanisms Cloud SSA Remote Result Source Cloud App model Add-ins External Content Federation Identity and Directory Sync
  15. 15. Benefits of Cloud Hybrid Search
  16. 16. External Content (on-premises and/or in the cloud) SharePoint Server (On-premises or Hosted) Office 365 SharePoint Online Content Onedrive for Business Content Connectors SharePoint Content Adding External Content
  17. 17. • • • • • • • • • • • • • • • • BA Insight Connectors Mailbox and Archiving Systems • Microsoft Exchange • Microsoft Exchange Online • IBM Lotus Notes • Symantex Evault • Autonomy EAS / (Zantaz) • • • • • • • • • • • • • • • • • ERP and Portal Systems • • • • • • • • • • • • Plus a proven architecture and process for creating new connectors to complex systems
  18. 18.   External Content in O365 UX Unified view across all content - on-premises and on-line - inside and outside SharePoint
  19. 19. Scaling
  20. 20. External Content (on-premises and/or in the cloud) Custom Processing CEWS Bottlenecks: 1) Source systems 2) Content Processing 3) Indexer ….
  21. 21. External Content (on-premises and/or in the cloud) Bottlenecks: 1) Uplink 2) Source systems ….
  22. 22. 24 Performance
  23. 23. External Content (on-premises and/or in the cloud) CEWS Custom Processing Bottlenecks: 1) Uplink 2) Source systems 3) Content Processing ….
  24. 24. Performance Monitoring and Bandwidth  (Get-Counter -ListSet "Search Gatherer Azure Plugin - SharePointServerSearch").counter   
  25. 25. 500K items crawled on an Azure D3 50 DPS   100 DPS  1 hour 
  26. 26. Less servers is OK • • •  
  27. 27. • • Directory Synchronization SID S-1-5-21-1212121212-1212121212-1212 msOnline- OnPremiseSecurity Identifier S-1-5-21-1212121212-1212121212-1212 PUID PUID-XXXX-XXXXXXXXXX
  28. 28. Mapping of Access Control Lists Allow: S-1-5-21-1212121212-1212121212-1212 PUID-XXXX-XXXXXXXXXX • User SIDs are mapped to PUIDs • Group SIDs are mapped to Object IDs • «Everyone» and «Authenticated users» are mapped to «Everyone except external users»
  29. 29. SUPPORTED – Custom IFilter – BCS connectors – Partner connectors Customizations: Supported & Unsupported SUPPORTED – Tenant level schema mapping – Query rules – Result sources On-premises In the cloud NOT SUPPORTED • Content that requires custom security trimming NOT SUPPORTED • Site collection level schema mapping • Custom security trimming • Custom entity extraction • Content enrichment web service
  30. 30. 1) 2) 3) Cloud Hybrid Search Limitations + Workarounds 33 Feature OOB Limitation BA Insight CEWS not available with Cloud SSA available via connector framework Entity Extraction not available with Cloud SSA available via autoclassifier Custom Security Trimming not available with O365 index can 'map down' to AD groups Thesaurus SharePoint Online doesn't support a thesaurus can use Federator - with SP server- based search center Removal of on-premises search results not available with Cloud SSA (could provide a custom solution)
  31. 31. External Content (on-premises and/or in the cloud) SharePoint Server (On-premises or Hosted) SPO Content OneDrive Content Connectors SharePoint Content Connector Framework Office 365 AutoClassifier (app version) CEWS Custom Processing
  32. 32. External Content (on-premises and/or in the cloud) SharePoint Server (On-premises or Hosted) SPO Content OneDrive Content Connectors SharePoint Content Connector Framework AutoClassifier Office 365 AutoClassifier (app version) CEWS
  33. 33. DLP Sensitive Data Search works with hybrid Search for sensitive data across on-premises and SharePoint Online All Built-in sensitive types Identification and export Extends to data in OneDrive Sensitive Information type detection through KQL searches Get instant statistics Preview & export results
  34. 34. Right now: only when you query for it
  35. 35. A global single index solution Cloud SSA Cloud SSA Cloud SSA Cloud SSA Cloud SSA
  36. 36.  – – –   NOT OOB …. but there’s a way to handle them all Scenarios 40
  37. 37. 41 Connectors Federator
  38. 38. OOB Federated Search User Experience Results from Cloud Results from SharePoint On-Premise Refiners from Cloud only No termset synchronization Result Blocks (not interleaved)
  39. 39. BA Insight Federator
  40. 40. 45 Full Range of Hybrid Search Configurations Scenario Most systems and portals hosted on-premises Most systems and portals hosted in the cloud Must work across borders but maintain data residency Single Single MultiSearch Search Index Search Index Across Multiple in SP Server in SP Online * Search Indices How it works Crawls SP Online and other sources from SP Server Crawls SP Server and other sources from Cloud SSA, pushes text & metadata to SP Online Searches SP Online and SP Server simultaneously; combines the results Advantages Simplest approach; best search experience Low footprint on-premises; can use online features (Delve, DLP) Only solution for some scenarios BA Insight Improvement over OOB no OOB solution Adds content outside SP Preview content outside SP2013 Supports content enrichment Provides single interleaved result set and refiners * requires Microsoft Cloud SSA Approaches for Hybrid SharePoint Configuration
  41. 41. Should I run index reset? NO!
  42. 42. Best Practice: Content Source Naming & Deletion{ { { {
  43. 43. Action CrawlDB state Office 365 index state User’s view Create contentsource1_v1 Crawl doc1 doc1 in crawldb Doc1 indexed Doc1 is searchable “index reset”  <empty> Doc1 indexed Doc1 is searchable Create a result source to exclude contentsource1_v1 from the tenant & search center site collections <empty> Doc1 indexed Doc1 is no longer searchable Tenant Admin opens SR to delete ALL cloud SSA content. <empty> <empty> All external content has been removed Create contentsource1_v2 Crawl doc1 doc1 in crawldb Doc1 indexed Doc1 searchable again Orphaned Content
  44. 44. 50 Customer Example: ACE Built on SharePoint 2013 – but couldn’t run as-is in O365
  45. 45. SharePoint Server in Azure in hybrid configuration with O365 Tenant Virtual Network Cloud Service Availability Set Active Directory & DNS Cloud ServiceCloud Service Availability Set Front End Availability Set App server Availability Set Database Microsoft Azure Gateway subnet Active VPN On-premises environment Optional!
  46. 46. Example: Using Search-First Migration with Hybrid Cloud Service Availability Sets SharePoint Services Farm Microsoft Azure SharePoint Online Site collections Office 365 Tenant SharePoint 2013 Content Farms SharePoint 2010 Farm(s) 2) Migrate / Upgrade Content Farms Each site collection can be moved independently Can be on-premises, in O365, or hosted in Azure 3) Decommission old farm(s) 1) Establish Search Service (using Azure IaaS)
  47. 47. Key Considerations for Hybrid: Workloads, Environment, Data, Customizations Availability of features Online versus On-Premises on particular workloads Significant investments in customization of On-Premises workloads Concerns over global network performance with remote sites Regulatory considerations Manageability concerns
  48. 48. References Tools/tree/master/Scripts/SharePoint.Hybrid.Search.Configuration
  49. 49. References - Blogs synchronization.aspx synchronization-password-sync-part2.aspx hybrid-search-in-office-365-sharepoint-online-part3.aspx inbound-hybrid-topology-with-office-365-and-microsoft-sharepoint-server-2013-part7.aspx
  50. 50. References – Installing with SP2016
  51. 51. Tools     62
  52. 52. New Sites to bookmark 63
  53. 53. Contact: Questions / Discussion