Enterprise search in SharePoint 2013 - Sydney 15th of January 2013
•Download as PPTX, PDF•
1 like•2,140 views
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Enterprise search in SharePoint 2013 - Sydney 15th of January 2013
Similar to Enterprise search in SharePoint 2013 - Sydney 15th of January 2013 (20)
More from Findwise
More from Findwise (20)
Enterprise search in SharePoint 2013 - Sydney 15th of January 2013
- 1. Enterprise Search in SharePoint 2013 Mattias Brunnert, Findability Expert at Findwise Sydney, 28 januari 2013
- 2. • We only do Findability! • Founded in 2005 • Offices in Europe and Sydney. Presence in the US • 150 years of search experience 80 employees (April 2012) • Hundreds of projects • Experts in technology, information management, usability and more!
- 3. Agenda 1 2 3 SP2013 Search Overview Query Management Search Analytics Search User Experience Content Processing and Search Architecture Enrichment
- 4. SharePoint 2013 Search FAST FAST Technology Search For SharePoint SharePoint 2010 Search SharePoint 2013 Search
- 9. Search Architecture 2013 Search Architecture SharePoint
- 10. Extend
- 12. Search RESTful API Keywords http://server/site/_api/search/query?querytext=`test` Selecting Properties http://server/site/_api/search/query?querytext=`test` &selectproperties=`Title,Rank` Sorting http://server/site/_api/search/query?querytext=`test` &sortlist=`LastModifiedTime:descending` http://server/site/_api/search/query?querytext=`test` &sortlist=`LastModifiedTime:descending,Rank:ascending`
- 14. Parsers
- 16. Content Enrichment Web Service Call-out
- 17. Agenda 1 2 3 Search Overview Query Management Search Analytics Search User Experience Content Processing and Search Architecture Enrichment
- 18. Governance: Search Analytics Analyze content and Understand and improve Guide users to content usage patterns search performance relevant to them
- 21. Demos
- 22. Demo 1: Managing query suggestions
- 23. Demo 2: Adding a content search web part 1 23
- 24. Demo 2: Adding a content search web part
- 25. Demo 2: Adding a content search web part
- 26. Demo 2: Adding a content search web part
- 27. Demo 2: Adding a content search web part
- 28. Summary 1 2 3
- 29. My Top 5 Picks in 2013 1 2 3 4 5
- 30. SharePoint – Intranet - Feature Tiering On-premises Online Foundation Standard CAL Enterprise P Kiosk Plan 1 Plan 2 CAL Basic Search (Site collection search) Standard Search (People & Expertise Search, Visual - People/ Expertise Previews, Visual Best Bets, OOTB Recommendations) Search Search ( No Custom Enterprise Search (Entity Extraction, Video Search, Custom Entity Query Rules, Extensible Content Flow, Item Extraction, E xtensible Recommendations, CBS) Content Flow, or CBS) www.office365.com
- 31. General Best Practice still applies • You have to work with it • Focus on user and business needs • Balance information quality and automization • Measure – use search analytics • Get help • Don’t do a project
- 32. So… what’s next? TechNet, MSDN, SharePoint Server 2013 findwise.com SharePoint Team Blog, findabilityblog.com Findwise Findability Blog slideshare.net/findwise @findwise
- 33. Offerings Overview New into search: • Implementing a search solution based on the SharePoint or FAST platforms Moving to SharePoint/FAST platform: • Migration from existing search platform to the SharePoint platform Existing SharePoint/FAST platform: • Healthcheck • Upgrading and Updating • Support & Maintenance • Improved Search User Experience • Search Editorial Support • Training Taking search to the next level – Findability by Findwise: • SharePoint by Findwise
- 35. Demo 3: Managing query rules 1 23
- 36. Demo 3: Managing query rules
- 37. Demo 3: Managing query rules
- 38. Demo 3: Managing query rules
- 39. Demo 4: Add a custom entity extractor
- 40. Demo 4: Add a custom entity extractor
- 41. Demo 4: Add a custom entity extractor
- 42. Demo 4: Add a custom entity extractor
- 43. Demo 4: Add a custom entity extractor
- 44. Demo 4: Add a custom entity extractor
- 45. Content Processing Flow (conceptual) Delete Links Security Descriptors Parse Map to documents Security Detect managed and extract Descriptors language metadata properties Ifilter sandbox Phonetic Custom Web Service Word Document name Entity Callout breaking summary variations Extraction
- 46. On Premise Online Note: SharePoint Online Plan 1 and Plan 2 also include Yammer
- 47. SharePoint – Usage Scenarios
- 48. SharePoint – Deployment Scenarios
- 49. SharePoint 2013 SKUs SharePoint SharePoint Enterprise CAL Online Plan 2 2013 SharePoint SharePoint SharePoint Standard CAL 1GB Storage Server 2013 Online Plan 1 2013 E-discovery, ACM, BI (PowerView App Catalog and Marketplace, Work Management, Social (Community Site), External Sharing, SharePoint 2013 Workflow
- 50. SharePoint 2013 Features Apps App Catalog & Marketplace ● ● ● ● ● ● ● ● ● ● ● ● ● Work Management ● ● ● ● ● Collaboration ● ● ● ● ● External sharing ● ● ● ● ● ● ● ● ● ● Search ● ● ● ● ● ● ● ● ● ● ● ● Content Management ● ● ● ● E-discovery, ACM, Compliance ● ● PowerView ●* ● Business Intelligence ● ● ● ● ● ● ● Business Solutions ● ● ● SharePoint 2013 Workflow ● ● ● ● ● ● ● ● ● ● ● ●
- 51. SharePoint – Intranet - Feature Tiering On-premises Online Foundation Standard CAL Enterprise P Kiosk Plan 1 Plan 2 CAL Apps App Catalog & Marketplace (Marketplace, Corporate Catalog) Team Sites (Drag & Drop, Notebook, Simplified Access) External sharing (share-by email) Collaboration Work Management (Project Site, My Tasks, Site Mailbox) Project Site Social (Personal Site - Newsfeed, Follow Content & People., SkyDrive Pro. Community Site– Discussions, Moderation, Reputation) SkyDrive Pro (-Personal Site) Basic Search (Site collection search) - People/Expertise Standard Search (People & Expertise Search, Visual Previews, Visual Best Bets, OOTB Recommendations) Search Search Enterprise Search (Entity Extraction, Video Search, Custom Query Rules, Extensible Content Flow, Item ( No Custom Entity Recommendations, CBS) Extraction, Extensible Content Flow, or CBS) Content Management (ECM, Document Sets, Word/PPT Automation, Translation Services) Content Records Management (in-place holds) Management E-discovery, Compliance across Exchange & SharePoint (Case manager) -ACM Excel Services, PowerPivot, PowerView BI Scorecards & Dashboards (PPS- Decomp tree, Balanced Scorecards) Access Services Visio Services Business Form Based Applications (InfoPath Forms) Solutions SharePoint “2013” Workflow Business Connectivity Services +Secure Store, External Data Search, OData Connector +Business Data Web Parts (External List, Basic Connector Framework) External Users 500 free Up to 10,000 free, Azure Provisioned Apps (Access Services, Custom code in Azure LWR) Per App, Per user per month* Add-On 10GB + additional 500MB per user, Kiosk and External users do not contribute Additional Storage storage to the overall tenant quota User Limits 50 For organizations of all sizes, 1 to hundreds of thousands www.office365.com
- 52. SharePoint – Internet Sites - Feature Tiering Online On-Premise P/E SKU SharePoint Server 2013 (Enterprise License Key) Lightweight Public-Facing Site Topic Pages Content Management Web Publishing & ECM Publishing Product Catalog Cross-site publishing Authenticated Social Social (Personal Site - Newsfeed, Follow Content & People, SkyDrive Pro. Community Site– Discussions, Moderation, Reputation) Basic Search (Site collection search) Search Standard Search (People & Expertise Search, Visual Previews, Visual Best Bets, OOTB Recommendations) Enterprise Search (Entity Extraction, Video Search, Custom Query Rules, Extensible Content Flow, Item Recommendations, CBS) Mobile and Alternate Device Rendering Limited (2 channels) Multiple Language Translation Audiences Variation Custom Design Business Connectivity Services +Secure Store, External Data Search Extensibility (External List, Basic Connector Framework) Business Data Web Parts Custom Development Domains Multiple Domains
- 53. Search Architecture 2013 Search Architecture SharePoint Hydra Content Refinement Framework
Editor's Notes
- Welcome!
- Our agenda for this presentation includes three parts. The first part out-of-the-box search experience,. This part is all about finding what the users are looking for and getting answers to their questions. The new product revolves around the user more than ever, and you will beSecond part of the presentation, which is all about extending. We will talk a bit about executing queries under the new architecture and more specifically on how we can extend the way they are executed.But what happens next to the search engine? The third part of the presentation will talk about the governance of your search solution. More specifically, we will focus on search analytics.
- Search in SharePoint 2013 focuses not just on technology, but the belief that search should help users find what they are looking for and get answers to the questions they ask. This means going beyond searching a index, which required the development of a entire new experience dedicated to the users intent, that can analyze user interactions.In the next few slides, we will show how SharePoint 2013 search has made it easier to find information and get answers to questions.
- This is how the new search interface looks!Let’s start from the search box. Query suggestion (or autocomplete) guide the user search keywords as the user types. It provides suggestions based on previously searched queries.Now, the autocomplete experience not only shows you common queries, but actually allows you to navigate back to past results you’ve interacted with.Result blocks contain a small subset of results that are related in a particular way (for example, PowerPoint documents appear in a result block when the word ‘presentation’ is one of the query terms)The refinement panel summarizes and narrows the results. For example, it allows the user to type in a broad query and then drill down in the result set.In SharePoint 2013, the refinement panel reflects more accurately the entities in your content. Until now, deep refiners were only available for FAST Search for SharePoint customers. Deep refinement allows users to see a more exact number of documents that match each refinement option in a result set. However, you will notice that the counts are not enabled for the refiners, and we will discuss how to enable them in one of our demos.Visual refiners are available out-of-the-box.Video and audio have been introduced as new content types.Document deep links provides the ability to look at the structure of the document and it works for PowerPoint and Word documents. Text in bigger fonts or underlined text gets extracted and stored, and in the search result you can quickly jump straight to it.
- The hover panel allows you for example to get a live preview of a document, dive into the document structure, and take actions on a document. A similar functionality in 2010, that allowed users to quickly review documents in the results page, are the visual previews, however these were only available to FAST Search for SharePoint customers.What is more interesting is that the hover card experience changes based on content type. Result types rules determine what type of result you’re looking at and the correct hover panel will be shown based on those rules.It is extensible and customizable to allow you to make search actionable for any content and create a richer experience.The previews are powered by Office Web Apps, and you need to have a separate server in your farm for hosting these web apps. If you don’t, you will not be able to see the document previews in the hover panel.
- GLOBAL COMPANIESPeople searchFor people/expertise search, in the hover panel we see past projects, interests, or documents that a persons has worked on (documents relevant to the query). The actions you can take are also contextual. For a person result, you can follow, send email, view profile, while for a video it can play and share.Social recommendations (introduced in this version) are powered by search, and are based on analysis of user’s behavior patterns.Also, people search is now integrated with the core results.You now have more than phonetic search for people names. You have name suggestions as you type which support exact name matching and fuzzy matches (where the spelling is similar but not exact because of phonetic misspellings or typing errors).
- The first step is finding the right information. The next step is answering to actual questions.Answering questions is about understanding the user intentIf you search for “x x video”, the search surfaces videos on top. If you search for “x x expert”, the search surfaces people. This is done through the out-of-the-box query rules, and businesses can create their own query rules that can trigger new experiences.Search can go beyond searching for documents. Here is an example of utilizing information in conversations in SharePoint for answering a question.
- So, what is powering this great search experience? Let’s take a look at the search architecture.Let’s discuss about each component in the architecture. If we start from the left, we talk about content sources, and these can be http, file shares, SharePoint itself, profiles, exchange, and so on.Connecting to these sources in this version of SharePoint is done through .NET assembly connectors (previously used for business connectivity services solutions). Previously, iFilters were used to crawl a piece of content we find in a repository. Now, this is replaced by parsers, which is a new piece introduced in this architecture.Moving to the right, we find the content processing component. There is one entry point in this content processing pipeline, and that is called the web service call-out. Here is where you can receive information about managed properties, you can modify information about the properties, and send it back into the content pipeline before they actually get in the index. The indexing component is just the box that represents the actual indexing of content, building the index that we search.A new analysis engine was integrated. It serves for a variety of jobs including processing user behavior, performing click analysis, recommendations. The search system determines the relevance of search results in part by how content is connected, how often an item appears in search results, and which search results people click. The new analytics component in SharePoint Server 2013 tracks and analyzes this information and uses it to continuously improve relevance. (http://technet.microsoft.com/en-us/library/ee667266(v=office.15).aspx)At this step, we have built an index on which we can query against. This querying happens on the right side. Query pipeline – contains the functionality necessary to execute the query and move it from user interface in the query engine where it actually gets executed against the index, and then getting the results and moving them back to the user interface. At the query pipeline level, search provides a RESTful interface that can be used to run queries.Note: Description of components in the architecture are based on the developer training (http://msdn.microsoft.com/en-US/sharepoint/fp123633)
- While the out-of-the-box SharePoint 2013 search experience sounds exciting, you may also be wondering how much customization and extensibility opportunities you have.With SharePoint 2013 developers can extend content processing and enrichment. Query rules can help combining business rules with user intent without any custom code. We will talk more about this in the next slides.The search experience can be customized and allows organizations to extend and build their own experiences through the RESTful API’s. We will talk further about the query management, the content processing, and the RESTful API.
- New to SharePoint 2013, Query Rules can be used to customize search results in a flexible manner (adapt the experience based on the user’s true intent). A Query Rule consists of three top level elements:Query Conditions: Which queries will activate the rule? A condition might be that a word in a user's query matches a term in a SharePoint term set, or that a word in a query commonly appears in queries typed on a particular site on your intranetQuery Action: What will happen when the rule fires?Publishing Options: When will the rule be used?A query rule can specify the following types of actions:Add Promoted Results (formerly called Best Bets) that appear above ranked results. For example, for the query "sick leave," a query rule could specify a particular Promoted Result, such as a link to a site that has a statement of company policy regarding time off work.Add one or more result blocks. And we have previously seen an example of how to use these.Change ranked results, such as by modifying their relevance. For example, for a query that contains "download toolbox," a query rule could recognize the word "download" as an action term and boost results from a particular download site on your intranet.
- This slide shows the REST-based API into search. While you can go into the search center and just type in a query in KQL, as developers we can use the same keyword query syntax in the REST-based API to get results. In the examples, you see that there is a URL that we can call, and _api is pointing to the restful API endpoint. And then we provide query string arguments that refer to the keyword query syntax.The RESTful API is exposing all the properties that are part of the keyword query object inside of SharePoint as query strings. So, for example, you can see in the second example we have a query text (in our case we are just looking for test) and then we are including the selectproperties parameter to allow us to return just the title and the rank from search.We also have some examples of sorting, and the point is that all the properties available in KQL are available through the REST API.And this is how the results would look like. You will probably use some code to pick up the interesting elements from the result, but the point is that you get back a document in a predictable format that has a row for each of the results, and you decide how to display this.You can use this RESTful API to develop apps for SharePoint, apps on tables or mobile devices, and more.
- We have seen what extensibility options we have on the query management side. Let’s now look at the extensibility options of the content we are working with in the first place. The content processing pipeline is the one place where you have access to all of the crawled items before the index gets created.One of the strengths of FAST was the advanced content processing capabilities and the idea of extracting and creating metadata from documents to improve search results, sorting capabilities, and refinement.There are three new items in SharePoint 2013 related to content processing and enrichment: parsers, custom entity extraction, and the web service call-out. We will now talk about each of them.
- In previous versions of SharePoint we have used something called an IFilter to allow us to crawl the individual items in a content source. Parsers are more sophisticated than iFilters.The parser detects the document format, File format detection is now done regardless the document extension. Besides automatic file format detection regardless the document extension, new parsing features include:Deep link extraction.The visual metadata extraction extracts the title, author and date based on the document and not the metadata because the metadata can sometimes be wrong. Good search results depend on users entering good metadata for documents and not all users do that.
- The custom entity extraction is an extensibility point that allows you to plug your own dictionary into the search system. This capability is useful when you want to search and refine by something that doesn’t have a managed property defined. Those of you that use FS4SP know that the some predefined extractors were available for extracting person names and locations. In SharePoint 2013, you can create custom extractors to replace those. The difference is that, in SharePoint 2013, the custom extractors are only based on dictionary entries, whereas the predefined extractors used extraction rule. The number of custom extractors that you can create is now limited to 12.The dictionary is a simple text file that you import using PowerShell. There are 12 custom extractor slots that you can use, and each slot is defined using some predefined name. There are word matching and substring matching dictionaries and then there are case insensitive and case sensitive slots that you can put them in. So 12 custom dictionaries in total.We will see in a few minutes a demo on how to add a custom entity extractor.Based on MSDN developer training
- The web service callout allows you to transform managed properties using a custom web service. It allows you to modify managed properties and/or add new properties. Can be used for data cleansing. For example, if your documents include MSFT, Microsoft, Microsoft Corporation, with the web service callout you can take a look at the managed property for company and you could normalize all these values into Microsoft Corporation. It can also be used for entity extraction, which means adding new managed properties that did not exist before. Or you can use this service for classification and tagging, by adding new managed properties based on some kind of classification.The web service client is configured with a SOAP endpoint and implements a well defined interface. This will allow for the web service to be called during the crawling process. A key thing is that there is a trigger condition that determines when to do the synchronous callout. If the trigger condition evaluates to true, the web service is called, if it evaluates to false, the web service does not perform a callout, and if no trigger condition is set, all items are sent to this web service. What this also means is that you can plug in your own custom processing web service in this step.Based on MSDN developer training
- Now, let’s talk a bit about search analytics.We have previously mentioned that SharePoint 2013 includes a new component that runs various analytics jobs to analyze not only the content in the search index but also user actions that are performed on a site – why? for example to identify items that users perceive as more relevant that others.The new functionality for displaying content recommendations based on usage patterns actually uses the information from the analyses. By including recommendations on a page, you can guide users to other content that may be relevant for them. For example, you can guide users to popular items in a category or let them know that users who viewed this item also viewed another item.
- Get insights about search through Excelreports providing usage events, user statistics, and traffic patterns on a site.Based on the information in the reports, you can make decisions about how to fine-tune the website.Then, monitor the changes you make. The reports are updated based on the changes that are made.
- Next, how does the search recommendations framework work?User actions produce usage events. When users interact with a SharePoint Server 2013 website — for example, when users click a link, press a button, or view a document — their actions are stored as usage events.The recommendations algorithm in the Analytics Processing Component counts and analyzes the usage events.After processing in the Analytics Processing Component, the information is added to the search index and the reporting database.You can thenadd recommended items and popular items web parts to a site. In SharePoint Server 2013, you can display recommendations on a site by adding one or more Recommended Items Web Parts. You can configure the Web Part to display recommendations for the document or item that a user is viewing. For example, these recommendations can be displayed under the heading Users who viewed this document also viewed.You can use the Popular Items Web Part to display the most popular items that satisfy a set of criteria. For example, these recommendations can be displayed under the heading Most popular items in this category.
- We’ve now reached the end of the presentation. Let’s summarize what we have learned during the past hour. We’ve showcased the OOTB search experience and we’ve seen that the new features do not focus only on finding information, but also on answering users questions. We’ve seen some of the possibilities of how to extend the search to meet your business needs and customize the search experience for your users. Moreover, we’ve briefly presented the search analytics features that can help you understand what your users are looking for and analyze where more powerful features are needed
- I would like to thank the SharePoint 2013 team at Findwise. They did a great job in checking out all the search functionalities that have been added or improved in SharePoint 2013. We are excited about the new SharePoint and we are still playing with the preview version. Moreover, based on what the team has seen so far, we have created a top 5 of what we like best in SharePoint 2013.- With the new people in videos column, you are now able to search for people in videos.- Via the Print to PDF option in the hover panel, Word documents are converted automatically to PDF files.- Name suggestions introduces a simple and intuitive way to find people by their names. Start typing a name and you get name suggestions. This feature support both exact matching and phonetic (fuzzy) matches, which is useful if the user misspells a person’s name or does not remember the exact spelling of a name.- The new Analytics Processing Component feeds content recommendations- Deep refiners are now out-of-the-box and visual refiners are available and can be used for example for date ranges, or continents.
- Availablesearch features is determined by yourlicensing.
- Please look at otherFindwiseresources or contactus for more information on howtosucceedwithsearch.
- Of course, we can’tcover all the search functionalities in a one hour presentation and if you would like to learn more about what changes the new SharePoint release brings, you can start by checking the TechNet material, the developer trainings on MSDN, follow the SharePoint Team Blog and the Findwise Findability Blog.Then, download the SharePoint 2013 Preview. The SharePoint Server 2013 Preview has been available for download since July this year, and in mid-October the new SharePoint has reached Release to Manufacturing (RTM).Also, attend the SharePoint Conference in Las Vegasin November. Findwise will be there; this will be a great opportunity to learn more about the upcoming SharePoint. We will also report from the conference from a findability and enterprise search perspective using social media.Then, get in contact. Either with us or Microsoft. There are certain features in the previous versions of SharePoint or in FAST that are not enabled or configured out-of-the-box. We can help you with that and more, based on your specific needs.
- ThesearesomeofFindwiseofferingsaroundMicroSoftsearch.
- Överflödig?
- Content by search
- So, what is powering this great search experience? Let’s take a look at the search architecture.Let’s discuss about each component in the architecture. If we start from the left, we talk about content sources, and these can be http, file shares, SharePoint itself, profiles, exchange, and so on.Connecting to these sources in this version of SharePoint is done through .NET assembly connectors (previously used for business connectivity services solutions). Previously, iFilters were used to crawl a piece of content we find in a repository. Now, this is replaced by parsers, which is a new piece introduced in this architecture.Moving to the right, we find the content processing component. There is one entry point in this content processing pipeline, and that is called the web service call-out. Here is where you can receive information about managed properties, you can modify information about the properties, and send it back into the content pipeline before they actually get in the index. The indexing component is just the box that represents the actual indexing of content, building the index that we search.A new analysis engine was integrated. It serves for a variety of jobs including processing user behavior, performing click analysis, recommendations. The search system determines the relevance of search results in part by how content is connected, how often an item appears in search results, and which search results people click. The new analytics component in SharePoint Server 2013 tracks and analyzes this information and uses it to continuously improve relevance. (http://technet.microsoft.com/en-us/library/ee667266(v=office.15).aspx)At this step, we have built an index on which we can query against. This querying happens on the right side. Query pipeline – contains the functionality necessary to execute the query and move it from user interface in the query engine where it actually gets executed against the index, and then getting the results and moving them back to the user interface. At the query pipeline level, search provides a RESTful interface that can be used to run queries.Note: Description of components in the architecture are based on the developer training (http://msdn.microsoft.com/en-US/sharepoint/fp123633)