MSDN Live Meeting - Introduction to FAST Search Server for SharePoint Server 2010

  • 2,703 views
Uploaded on

 

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
2,703
On Slideshare
0
From Embeds
0
Number of Embeds
10

Actions

Shares
Downloads
60
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • Elements in green are part of the Query SSA – the ones in purple are part of the Content SSAElements in blue are part of the SharePoint general infrastructureElements in purple are part of the FAST Content SSA

Transcript

  • 1. FAST Search Server 2010 for SharePoint: a developers walkthrough
    MSDN Live Meeting
    24/05/2010
    Joris Poelmans, ECM Unit manager
  • 2. About Me
    ECM Unit Manager @ RealDolmen
    SharePoint Server MVP
    Blog – http://jopx.blogspot.com
    Twitter – jopxtwits
    Co-founder BIWUG – www.biwug.be
    Co-author
    Real World SharePoint 2010 (November 2010, Wrox)
    Real World SharePoint 2007 (August 2007, Wrox)
    SharePoint 2007, The definitive guide (September 2007, O’Reilly)
  • 3. Agenda
    Introduction to FAST Search Server 2010 for developers
    FAST Search Server 2010 – common developer tasks
  • 4. About Fast Search for SharePoint
    April 2008
    Microsoft acquires Norwegian Enterprise Search Solution Fast Search & Transfer
    Overall Vision
    To provide a market leading enterprise search platform that integrates with your existing infrastructure
    Leader in the Gartner Magic Quadrant around Information Access
    Gartner MQ
    4
  • 5. Microsoft Enterprise Search - Productivity
    • Common UI Framework
    • 6. Social search features and integration
    • 7. SharePoint platform integration
    • 8. End user and site administrator enablement
    • 9. Common Connector Framework (BDC)
    • 10. Object Model, APIs and developer XP
    • 11. Admin & deployment capabilities
    • 12. Operations advantages (SCOM, scripting)
    Common across the product line
  • 13. DEMO OOTB FAST Search
  • 14. Visual, conversational search
    Promote / Demote Results
    Sorting on any property
    7
    Scrolling PowerPoint
    Previews
    Visual Best Bets
    Thumbnails
    Similarity Search
    Deep refinement with counts
    on any property
    Extractors to generate metadata
  • 15. Didyouknow?
    Thumbnail and preview feature possibletoimplementfor SharePoint Server as well – check out http://techmikael.blogspot.com/2011/05/document-thumbnails-and-powerpoint.html#more
  • 16. Contextual Relevancy Tuning
    Kevin Kennedy
    Healthcare Sales
    ”What’s the latest product news for productivity”
    Alan Brewer
    Financial Services Sales
    ”What’s the latest product news for productivity”
  • 17. FAST Search Differentiators
    The features above and beyond SharePoint Search
    Conversational
    User Experience
    Precise Control
    Content Processing
    Platform
    Visual Cues
    • Thumbnails & Previews
    • 18. Visual Best Bets
    Exploration
    • Deep refiners
    • 19. Similarity Search
    • 20. Multilevel Sorting
    • 21. …on any property
    Context-Based Results
    • Promote/Demote
    • 22. User Context
    Relevance Control
    • Multiple profiles
    • 23. Word/Term control
    • 24. Custom/Extensibility
    Property extraction
    • Prebuilt
    • 25. Custom/Extensibility
    Content coverage
    • 400+ document types
    • 26. 82 languages
    • 27. Advanced Linguistics
    • 28. Offensive content filter
    Extreme scale
    • in 3 dimensions
    • 29. at best footprint
    Query Expressiveness (FQL)
    • Relevance and sorting
    • 30. Symbol & proximity search
    Features
    Provide the best, most productive, user experience
    Maximize productivity through control
    Make the most of all your content
    Tackle the most demanding search challenges
    Benefit
  • 31. FAST Extends Capabilities of SharePoint Server
    This is the out of the box SharePoint implementation
    FAST builds on top of this
    Content search
    People search
    SharePoint Server
    End
    Users
    Content +
    Profiles
  • 32. FAST Extends Capabilities of SharePoint Server
    Content search
    FAST Search
    for SharePoint
    People search
    Unique Social Search from SharePoint Server
    SharePoint Server
    Common developer experience and IT Pro Foundation
  • 33. FAST Search for SharePoint
    Summary of architectural elements
    Administration and Schema Object Model
    Content
    Processing
    And
    Linguistics
    Connectors:
    - SharePoint
    - BDC
    - Exchange
    Indexing
    FAST Search
    Authorization
    SharePoint
    Front-end
    Content
    Content
    Query Object Model
    Custom
    front-end
    Connectors:
    - Web Crawler
    Search
    Query and
    Result
    Processing
    Query Web Service
    Content
    Federation
    Object Model
    Monitoring Services
    OpenSearch or other Sources
    People Search
  • 35. Content Processing
    A new level of search quality
    Content Processing Pipeline


    End Users
    Format
    Conversion
    Lemmatization
    Language
    Detection
    EntityExtraction
    Mapper
    OpenSearch Source
    Content
    Content Processor
    Crawler
    Indexer
    Query Processor
    Federation
    Index Partition
    Search Center
    Profiles
    Metadata
    Relevance
    Control
    User
    Context
    Indexing
    Connectivity
  • 36. Content Processing CapabilitiesLeverage existing metadata, use property extraction to add structure
    The Content Pipeline
    Processing & refinement
    • Strong linguistic coverage
    • 37. 82 languages detected
    • 38. 45 languages with advanced linguistics features
    • 39. Spell checking & synonyms improve search experience
    • 40. Unique offensive content filtering
    • 41. Configurable content pipeline
    • 42. Transform and process content automatically
    • 43. Index over 400 document types
    • 44. Property Extraction creates metadata
    • 45. Prebuilt extractors: Person, Location, Company, E-mail, Date, Time
    • 46. Generic extractors for extension
    Location
    Amount
    Date
    Company
    REDMOND, Wash., andOSLO, Norway — Jan. 8, 2008
    Microsoft Corp. (Nasdaq “MSFT”) today announced that it will make an offer to acquire Fast Search & Transfer ASA (OSE: “FAST”), a leading provider of enterprise search solutions, through a cash tender offer for 19.00 Norwegian kroner (NOK) per share. This offer represents a 42 percent premium to the closing share priceon
    Jan. 4, 2008 (the last trading day prior to this announcement), and values the fully diluted equity of FAST at 6.6 billion NOK (or approximately $1.2 billion U.S. ). FAST’s board of directors has unanimously recommended that its shareholders accept the offer.
    Mapper

    Configurable
    Stages
    EntityExtraction
    Language
    Detection
    Format
    Conversion
  • 47. Content Processing pipeline configuration
  • 48. The Power of FQL – FAST Query Language
    Used to perform exact search and narrow the scope of search to a field
    Check out – FQL Syntax reference on MSDN –
    http://msdn.microsoft.com/en-us/library/ff394606.aspx
    Two modes:
    Fast Query Language (FQL)
    FQL Simple Query Language
    Standard web parts use the FQL simple query language
    You cannot use FQL directly from OOB search box in SharePoint
  • 49. The Power of FQL -Full FQL Operators
  • 50. Using XRANK and FQL
    Boost all documents that contain the name
    Dianne
    Tibbott
    Boost all documents that contain the name
    Mary
    Baker
    Or, in the FAST Query Language:
    Single (documents containing either Mary, Diane or both gets a boost of 5000): Xrank(string(”performance”), or(person:string(”dianetibbott”), person:string(”mary baker”)), boost=5000))
    Accumulative (documents with both Mary and Diane gets a boost of 10000): Xrank(Xrank(string(”performance”), person:string(”dianetibbott”), boost=5000), person:string(”mary baker”), boost=5000)
  • 51. Result
    Content
    Processing
    Search/Index
    Query
    External data
    Sort results by distance from end-user location
    (50,100)
    Index
    Geo specific processing
    ABC
    XYZ
    Geo
    123
    Pos_1
    Pos_2
    Geo data
    Lat
    Lon
    Lon
    Lat
    Using Location Awareness and FQLGeo-search using sortformula
    Formula sorting allows complete control of distance calculations (standard or ‘warped’ geometries)
    Distance(from 50,100) =(50−latitude)2+(100−longitude)2 
     
    query=hotel&sortby=+[formula:sqrt(pow(50-latitude,2)+pow(100-longitude,2))]
  • 52. The Power of FQL - Toolset
    FAST Search for SharePoint MOSS 2010 Query Tool - http://fastforsharepoint.codeplex.com/
    FS4SP2010 Query logger - http://fs4splogger.codeplex.com/
  • 53. Demo FAST Helper Tools from Codeplex
  • 54. Agenda
    Introduction to FAST Search Server 2010 for developers
    FAST Search Server 2010 – common developer tasks
  • 55. Top Customization Scenarios
    • Add new Refinement category
    • 56. Show results from federated location
    • 57. Modify the look and feel of OOB end user experience
    • 58. Enable sorting by custom metadata
    • 59. Add visual Best Bet for upcoming sales event
    • 60. Configure different ranking for HR vs. Engineering department
    Modify the OOB End User Experience
    • Create new visual elements
    • 61. Show Location refinement on Chart/Maps
    • 62. Show tags in tag cloud
    • 63. Enable export results to Spread Sheet
    • 64. Summarize Financial Information from customers in Graphs
  • Top Customization Scenarios (Ctd.)
    • Query & Indexing Shims
    • 65. Show email results from personal mailbox on Exchange Server through the EWS
    • 66. Index content from custom repositories like Documentum
    • 67. Create content processing plug-ins to create new metadata
    • 68. Create new Search Driven Applications
    • 69. Create new customer search experience
    • 70. Indexing content
    • 71. Define content for search
    • 72. Design search experience
    • 73. Create new Audio/Video/Image search experience
  • Top Customization Scenarios (Ctd.)
    • Expand query terms based on synonyms defined in Term Store
    • 74. Augment customer results with project information
    • 75. Show popular customers/people inline with search results
    • 76. Show people results from other sources
    • 77. Query & Result Pipeline Plug-ins
    • 78. create a new customer page that shows:
    • 79. Customer Contact Details
    • 80. Customer Project Details
    • 81. Customer Contacts
    • 82. Internal Experts
    • 83. Customer related documents
    • 84. Create a new Search Verticals
  • Common development scenarios – 3 scenarios explained
    Modify the OOB End User experience
    Configure FAST Search webparts using additional search refiners
    Create new visual elements
    Create new Extend Search UI with extra webparts leveraging the SharePoint object model
    Query & Indexing Shims
    Create content processing plugins to create new metadata
  • 85. Common development scenarios – 3 scenarios explained
    Modify the OOB End User experience
    Configure FAST Search webparts using additional search refiners
    Create new visual elements
    Create new Extend Search UI with extra webparts leveraging the SharePoint object model
    Query & Indexing Shims
    Create content processing plugins to create new metadata
  • 86. Modify the OOTB end user experience
    Adding refiners
    Refiners will use Managed Properties
    Refinement webpart uses specific XML syntax see http://msdn.microsoft.com/en-us/library/ff625183.aspx
    Important fields
    Type: TaxonomyFilterGenerator, ManagedPropertyFilterGenerator,RankingModelFilterGenerator,CustomFilterGenerator
    MappedProperty: defines the name of the managed property in the service application which will be used as search refiner.
  • 87. Using Location and Person search refiners
  • 88. Common development scenarios – 3 scenarios explained
    Modify the OOB End User experience
    Configure FAST Search webparts using additional search refiners
    Create new visual elements
    Create new Extend Search UI with extra webparts leveraging the SharePoint object model
    Query & Indexing Shims
    Create content processing plugins to create new metadata
  • 89. Extend Search UI using SharedQueryManager class
    All search web parts use SharedQueryManager class
    One shared instance of the SharedQueryManager per search page
    At least one search results web part needs to be on the page
    Use SharedQueryManager to access other classes of the Federation Object Model
    SharedQueryManager.GetInstance(this.Page).QueryManager
  • 90. Build a Search result Driven Tag Cloud webpart using fast document vectors
  • 91. Common development scenarios – 3 scenarios explained
    Modify the OOB End User experience
    Configure FAST Search webparts using additional search refiners
    Create new visual elements
    Create new Extend Search UI with extra webparts leveraging the SharePoint object model
    Query & Indexing Shims
    Create content processing plugins to create new metadata
  • 92. Content Pipeline Stages
    Default
    Optional (sleeper stages)
    Format Conversion
    iFilters, OutSideIn
    Language and encoding detection
    Lemmatizer
    Linguistics normalization
    Tokenizer
    Word breaking
    Entity Extraction
    Companies, locations
    DateTimeNormalizer
    Date normalization
    Vectorizer
    Create document vector for similarity searching
    WebAnalyzer
    Anchor text and link cardinality analysis
    PropertiesMapper
    Map to crawled properties
    PropertiesReporter
    Report detected properties
    XML Properties mapper
    Offensive Content Filter
    Verbatim (wholeword) extractor
    Loads dictionary for custom extraction, e.g product names
    Field Collapsing
    Entity Extraction
    Persons


    Format
    Conversion
    Lemmatization
    Language
    Detection
    EntityExtraction
    Mapper
    The different plug-ins can either be configured from UI or from config files
    Extensibility options to add custom processing modules / 3rd party SW
  • 93. Creating content processing plugins to create new metadata
    Extending the content processing pipeline
    Configure entity extraction (companies, people, location)
    Standard property extractors available in FAST for Location, Company and Person
    Person property extractor – default not enabled – modify optionalprocessing.xml
    Whole word extraction
    Add custom property extractors using code
  • 94. Pipeline Extensibility API
    Motivation
    Straightforward way to add text analysis functionality
    Flexibility and supportability
    Example uses
    Sentiment analysis
    Translation
    Auto-Classification
    Mechanism
    Just before Mapper
    “any” binary
    Runs in sandbox with timeout
    Mapper
    Extensibility

    Standard processing
  • 95. Whole word extractionSentiment analysis using Custom property Extractor
  • 96. Summary
    FAST provides building blocks for search applications such as:
    Research portals for R&D teams
    Call center portals with 360° view on customer
    Employee self service using social search and conversational search
    ...
  • 97. Resources
  • 98. Recommended Resources
    FAST Search Top Blogs:
    On the Search - http://blogs.msdn.com/b/arnts/
    Microsoft Enterprise Search Blog - http://blogs.msdn.com/b/enterprisesearch/
    Search Unleashed - http://searchunleashed.wordpress.com
    Tech and Me (Mikael Svenson) - http://techmikael.blogspot.com/
    Knut Brandrud - http://blogs.msdn.com/b/knutbran/
  • 99. Recommended Resources (Ctd)
    SharePoint Server 2010 Virtual Labs (incl. FAST) - http://msdn.microsoft.com/en-us/cc707678.aspx
    SPS 2010 IW Trial Virtual Image (incl. FAST) - http://www.microsoft.com/downloads/en/details.aspx?FamilyID=751fa0d1-356c-4002-9c60-d539896c66ce&displaylang=en
    TechNet Survival Guide – FAST Search Server 2010 for SharePoint http://social.technet.microsoft.com/wiki/contents/articles/2149.aspx
  • 100. Thank you
  • 101. Stay up to date with MSDN Belux
    Register for our newsletters and stay up to date:http://www.msdn-newsletters.be
    Technical updates
    Event announcements and registration
    Top downloads
    Follow our bloghttp://blogs.msdn.com/belux
    Join us on Facebookhttp://www.facebook.com/msdnbehttp://www.facebook.com/msdnbelux
    LinkedIn: http://linkd.in/msdnbelux/
    Twitter: @msdnbelux
    DownloadMSDN/TechNet Desktop Gadgethttp://bit.ly/msdntngadget