Semantic search in the cloud
Upcoming SlideShare
Loading in...5
×
 

Semantic search in the cloud

on

  • 1,287 views

Presented by Roberto Masiero, Vice President ADP Innovation Lab, ADP ...

Presented by Roberto Masiero, Vice President ADP Innovation Lab, ADP

In this presentation we will cover ADP's Semantic Search strategy and implementation. From the use cases to the design to support semantic searches on a vast set of data, to crawling data from hundreds of data sources. We will also cover our architecture to scale the search service on a multi-tenant SaaS environment.

Statistics

Views

Total Views
1,287
Views on SlideShare
861
Embed Views
426

Actions

Likes
0
Downloads
20
Comments
0

5 Embeds 426

http://www.lucenerevolution.org 329
http://lucenerevolution.org 93
http://lucenerevolution.stephenz.com 2
http://lucenerevolution.com 1
http://www.lucenerevolution.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Semantic search in the cloud Semantic search in the cloud Presentation Transcript

    • Roberto MasieroADP Innovation Labs
    • 2011 –ADP #872011 - Rank #720102010 & 2011PINNACLE AWARDS 2011Global Outsourcing ServicesProvider of the YearADP #6FinancialDataServicesRank #2  >300,000 SaaS clients  >18 Million users  81 countries & expanding  Widest breadth of HCM solutions  BPO/Service offerings extend reachbeyond traditional SaaSADP is the Largest Global SaaS Provider ofCloud-based Enterprise Software2
    • ADP Innovation Labs
    • ADP Innovation Labs
    • Research and Incubation…9Current Major Research AreasPersonalCloudBig Data/ Analytics& SearchGlobalSORMobilitySocialPlatform© Copyright 2012 ADP, Inc. Proprietary Information.
    • 1010© Copyright 2013 ADP, Inc. Proprietary and Confidential Information.10
    • National EmploymentAnalytics11  Insights to US economicactivity  500,000+ companies  Several dimensions  ‘Large’ Data + R© Copyright 2013 ADP, Inc. Proprietary Information.
    • Innovation: ADP Intelligence & Analytics© Copyright 2013 ADP, Inc. Proprietary and Confidential Information.12  User Friendly Analytics  Embedded andContextualized Content  Benchmarks byIndustry/Location
    • © Copyright 2013 ADP, Inc. Proprietary and Confidential Information.13Behavioral Intelligence•  Using BigData to create an unparalleleduser experience•  Driving performance using in-memory data•  3Vs (Volume, Velocity and Variety)U i Bi D t t tPayroll – Heat map time sliceBigData - allow us to learn from dataimpressions(map)al Int(reduce)ggMary will probably check her Payroll and 401ktoday at 9:10AMPre-Cache (in-memory) payStatements andretirementPlan objects
    • 141414© Copyright 2013 ADP, Inc. Proprietary and Confidential Information.14
    • © Copyright 2013 ADP, Inc. Proprietary and Confidential Information.ADP Semantic Search: The RequirementsEnterprise Search GoalsSearch as a service across multiple ADP products and clientsSearch as the primary navigation toolSemantic Search: verbs and nounsKey Search Architecture ElementsMulti-TenancyMulti-DatacenterLarge # of sourcesSearch High-AvailabilityCrawling and Indexing HADisaster Recovery (Active-Active)Key Functional Design ElementsInstant Search across all object typesPartial text and phrase matchHighlightingFaceting by document typesQuery suggestionsUser’s click historyAccess control
    • © Copyright 2013 ADP, Inc. Proprietary and Confidential Information.ADP Semantic SearchHIRE = task/verbJOHN Doe= object/nounapplies toCANDIDATE objectis there a candidate named John Doe?invoke action ONBOARDJOHN DOEJust ask the appA new navigation paradigmFoundation for voice commandSimpler/Faster algorithm than NLPSearch objects (nouns) and tasks (verbs) to create (actions)User behavior feeds taxonomy/relevance on both positive and negative searches
    • © Copyright 2013 ADP, Inc. Proprietary and Confidential Information.ADP Semantic Search: The StackDataIndexQueryUserMetaDataUsing Lucene/Solr as search indexfor both data and meta data
    • © Copyright 2013 ADP, Inc. Proprietary and Confidential Information.ADP Semantic Search: The Flow
    • © Copyright 2013 ADP, Inc. Proprietary and Confidential Information.ADP Semantic Search: Simple APIsGET /v1_0/search/warmupHeader: Realm, Org OID, Associate OID, LocaleParams: NoneResult: Language-specific labels JSON GET /v1_0/searchHeader: Realm, Org OID, Associate OID, LocaleParams: q=<query text>, groupValue=<Document type to filter on>Result: Search Results if q is non-empty. Click history if q is empty. POST /v1_0/search/clicks/{canonicalURI}Header: Realm, Org OID, Associate OID, LocaleResult: Status OK (200)
    • © Copyright 2013 ADP, Inc. Proprietary and Confidential Information.ADP Semantic Search: SOLR Out-of-the-box vs Custom FeaturesOut-of-the-Box •  Text search using invertedindex•  REST APIs w/ JSONOutput•  Track query execution time•  Fuzzy search using Ngramsand Phonetics•  Result Grouping andFaceting •  Result highlighting•  Indexed vs. Stored fields•  Multi-lingual Search (UTF-8) Custom Features•  Multi-tenancy (enforced via predicates)•  Access Control (enforced viapredicates)•  Instant Search (regular search from Solrviewpoint)•  Dynamic relevance (via custompredicates built in Proxy)•  Synonym enrichment (via meta index)•  Smart actions via Task-Object mapping(via meta •  Index)•  Semantic Search (via custompredicates built in Proxy)•  Click-history based personalized resultboosting (via custom predicates basedon captured clicks)
    • © Copyright 2013 ADP, Inc. Proprietary and Confidential Information.ADP Semantic Search
    • © Copyright 2013 ADP, Inc. Proprietary and Confidential Information.User Experience: Changing our Mental ModelStart Here
    • © Copyright 2013 ADP, Inc. Proprietary and Confidential Information.User Experience: The Design Conversation has two aspectsIntuitiveComplexInsecurityConfidenceOverlySimplisticSpecialistsRequiredNotUser-friendlyEngaging
    • “The most profound technologies arethose that disappear. They weavethemselves into the fabric of everydaylife until they are indistinguishable.”- Mark Weiser@masieror