SlideShare a Scribd company logo
Project
PageZero
Smart Search
Antonio Gulli
Search transformed
the way we look at
the world
Search box:
Looking at the
world through a
window
Satori: Entering the
World of Entities
Words are
ambiguous
Sites
Bing uses the world of entities
as soon as you type. Not only
for refining search results
Smart Search
Windows 8.1
World Wide
SkyDrive

Xbox

Web
Local
Files

Personalized Results
Email Skype

Web
Learning to Personalize Query AutoCompletion

Milad Shokouhi

Microsoft
Relevance Labelling for Contextual Search

• For learning we need labels.
• Relevance labelling for contextual (personalized) search
(auto-completion) is not trivial.
• Previous work on personalized search [Fox et al., 2005]

• Samples search impressions from the logs
• Documents with SAT clicks are annotated with relevant labels
• The goal is to learn a re-ranking model that improves the
ranking of those relevant documents given the context.
Analogy: Auto-Completion Labels

34
Experimental Settings
• Ranker: Lambda-Mart [Burges et al., 2011]
• AOL testbed
• 657K users (Mar-May 2006)
• 128,620 queries in the prefix-tree
• Userid, query, timestamp
• Bing testbed
• 196K logged in users with Microsoft LiveID (Jan-2013)
• 699,862 queries in the prefix-tree
• Userid, query, timestamp, age, gender, zip code
• Training & testing on different sets of users
Personalized Ranking Features
• Demographics
• Age (5 groups)
• Gender (2 groups)
• Zip-code (10 groups)

• Search history
• Short (session)
• Long (all past queries)
Personalization by Age
The effectiveness of auto-completion personalization according to
the user’s age in terms of MRR. All differences are statistically
significant (P < 0.01)
Testbed

Baseline

Personalized

MRR (Gain/Loss)

Bing (age)

-

-

+3.80%

Frequently promoted suggestions for different age groups
Below 20

21-30

31-40

41-50

Above 50
Results Summary

Features
Short history
Long history
Age
Gender
Location
All

AOL
+1.95%
+4.45%
+6.45%

Bing
0.91%
5.57%
3.80%
+3.59%
+4.58%
+9.42%
London
Twitter: @gulliantonio
Project page zero, Smart Search, Learning to Personalize suggestions
Project page zero, Smart Search, Learning to Personalize suggestions

More Related Content

Similar to Project page zero, Smart Search, Learning to Personalize suggestions

Tagging search solution design
Tagging search solution designTagging search solution design
Tagging search solution design
Alexander Tokarev
 
Everything You Wish You Knew About Search
Everything You Wish You Knew About SearchEverything You Wish You Knew About Search
Everything You Wish You Knew About Search
IDEAS - Int'l Data Engineering and Science Association
 
Tagging search solution design Advanced edition
Tagging search solution design Advanced editionTagging search solution design Advanced edition
Tagging search solution design Advanced edition
Alexander Tokarev
 
Advanced google searching (1)
Advanced google searching (1)Advanced google searching (1)
Advanced google searching (1)Brenda Crawford
 
Search Strategy for Enterprise SharePoint 2013 - Vancouver SharePoint Summit
Search Strategy for Enterprise SharePoint 2013 - Vancouver SharePoint SummitSearch Strategy for Enterprise SharePoint 2013 - Vancouver SharePoint Summit
Search Strategy for Enterprise SharePoint 2013 - Vancouver SharePoint Summit
Joel Oleson
 
Deck_Rob Flaherty
Deck_Rob FlahertyDeck_Rob Flaherty
Deck_Rob Flaherty
Warren Hersch
 
Enterprise Search Strategy 101 at SEF2014 in Stockholm
Enterprise Search Strategy 101 at SEF2014 in StockholmEnterprise Search Strategy 101 at SEF2014 in Stockholm
Enterprise Search Strategy 101 at SEF2014 in Stockholm
Joel Oleson
 
IS Audit Course- Final Project ppt
IS Audit Course- Final Project pptIS Audit Course- Final Project ppt
IS Audit Course- Final Project ppt
Shahzeb Pirzada
 
SPConnections Amsterdam: Beyond the Search Center - Application or Solution? ...
SPConnections Amsterdam: Beyond the Search Center - Application or Solution? ...SPConnections Amsterdam: Beyond the Search Center - Application or Solution? ...
SPConnections Amsterdam: Beyond the Search Center - Application or Solution? ...Agnes Molnar
 
SPC.Org - SharePoint 2013 Search
SPC.Org - SharePoint 2013 SearchSPC.Org - SharePoint 2013 Search
SPC.Org - SharePoint 2013 Search
Bert Johnson
 
Microsoft Search Strategy Today - Exploring Office 365 Search in Real Life
Microsoft Search Strategy Today - Exploring Office 365 Search in Real LifeMicrosoft Search Strategy Today - Exploring Office 365 Search in Real Life
Microsoft Search Strategy Today - Exploring Office 365 Search in Real Life
Joel Oleson
 
SharePoint Jumpstart #3: Navigation, Metadata, & Faceted Search: Approaches &...
SharePoint Jumpstart #3: Navigation, Metadata, & Faceted Search: Approaches &...SharePoint Jumpstart #3: Navigation, Metadata, & Faceted Search: Approaches &...
SharePoint Jumpstart #3: Navigation, Metadata, & Faceted Search: Approaches &...
Earley Information Science
 
Fried sp techcon hybrid search deeper dive
Fried sp techcon hybrid search deeper diveFried sp techcon hybrid search deeper dive
Fried sp techcon hybrid search deeper dive
Jeff Fried
 
Agnes Molnar: Personalized Search and Collaboration in Office 365
Agnes Molnar: Personalized Search and Collaboration in Office 365Agnes Molnar: Personalized Search and Collaboration in Office 365
Agnes Molnar: Personalized Search and Collaboration in Office 365
Agnes Molnar
 
Important news from SharePoint Conference North America 2019
Important news from SharePoint Conference North America 2019Important news from SharePoint Conference North America 2019
Important news from SharePoint Conference North America 2019
Martin Albrektson
 
Cloud Hybrid Search with SharePoint
Cloud Hybrid Search with SharePointCloud Hybrid Search with SharePoint
Cloud Hybrid Search with SharePoint
Jeff Fried
 
Spcua 2013 Alexey Kozhemiakin Enterprise Search
Spcua 2013 Alexey Kozhemiakin Enterprise SearchSpcua 2013 Alexey Kozhemiakin Enterprise Search
Spcua 2013 Alexey Kozhemiakin Enterprise Search
Alex Kozhemiakin
 
SharePoint site admins leverage search
SharePoint site admins leverage searchSharePoint site admins leverage search
SharePoint site admins leverage search
C/D/H Technology Consultants
 
Effective Searching by Dominik Kornas
Effective Searching by Dominik KornasEffective Searching by Dominik Kornas
Effective Searching by Dominik KornasAEM HUB
 

Similar to Project page zero, Smart Search, Learning to Personalize suggestions (20)

Tagging search solution design
Tagging search solution designTagging search solution design
Tagging search solution design
 
Everything You Wish You Knew About Search
Everything You Wish You Knew About SearchEverything You Wish You Knew About Search
Everything You Wish You Knew About Search
 
Tagging search solution design Advanced edition
Tagging search solution design Advanced editionTagging search solution design Advanced edition
Tagging search solution design Advanced edition
 
Advanced google searching (1)
Advanced google searching (1)Advanced google searching (1)
Advanced google searching (1)
 
Search Strategy for Enterprise SharePoint 2013 - Vancouver SharePoint Summit
Search Strategy for Enterprise SharePoint 2013 - Vancouver SharePoint SummitSearch Strategy for Enterprise SharePoint 2013 - Vancouver SharePoint Summit
Search Strategy for Enterprise SharePoint 2013 - Vancouver SharePoint Summit
 
Deck_Rob Flaherty
Deck_Rob FlahertyDeck_Rob Flaherty
Deck_Rob Flaherty
 
Enterprise Search Strategy 101 at SEF2014 in Stockholm
Enterprise Search Strategy 101 at SEF2014 in StockholmEnterprise Search Strategy 101 at SEF2014 in Stockholm
Enterprise Search Strategy 101 at SEF2014 in Stockholm
 
IS Audit Course- Final Project ppt
IS Audit Course- Final Project pptIS Audit Course- Final Project ppt
IS Audit Course- Final Project ppt
 
SPConnections Amsterdam: Beyond the Search Center - Application or Solution? ...
SPConnections Amsterdam: Beyond the Search Center - Application or Solution? ...SPConnections Amsterdam: Beyond the Search Center - Application or Solution? ...
SPConnections Amsterdam: Beyond the Search Center - Application or Solution? ...
 
SPC.Org - SharePoint 2013 Search
SPC.Org - SharePoint 2013 SearchSPC.Org - SharePoint 2013 Search
SPC.Org - SharePoint 2013 Search
 
Microsoft Search Strategy Today - Exploring Office 365 Search in Real Life
Microsoft Search Strategy Today - Exploring Office 365 Search in Real LifeMicrosoft Search Strategy Today - Exploring Office 365 Search in Real Life
Microsoft Search Strategy Today - Exploring Office 365 Search in Real Life
 
SharePoint Jumpstart #3: Navigation, Metadata, & Faceted Search: Approaches &...
SharePoint Jumpstart #3: Navigation, Metadata, & Faceted Search: Approaches &...SharePoint Jumpstart #3: Navigation, Metadata, & Faceted Search: Approaches &...
SharePoint Jumpstart #3: Navigation, Metadata, & Faceted Search: Approaches &...
 
Fried sp techcon hybrid search deeper dive
Fried sp techcon hybrid search deeper diveFried sp techcon hybrid search deeper dive
Fried sp techcon hybrid search deeper dive
 
Search engines
Search enginesSearch engines
Search engines
 
Agnes Molnar: Personalized Search and Collaboration in Office 365
Agnes Molnar: Personalized Search and Collaboration in Office 365Agnes Molnar: Personalized Search and Collaboration in Office 365
Agnes Molnar: Personalized Search and Collaboration in Office 365
 
Important news from SharePoint Conference North America 2019
Important news from SharePoint Conference North America 2019Important news from SharePoint Conference North America 2019
Important news from SharePoint Conference North America 2019
 
Cloud Hybrid Search with SharePoint
Cloud Hybrid Search with SharePointCloud Hybrid Search with SharePoint
Cloud Hybrid Search with SharePoint
 
Spcua 2013 Alexey Kozhemiakin Enterprise Search
Spcua 2013 Alexey Kozhemiakin Enterprise SearchSpcua 2013 Alexey Kozhemiakin Enterprise Search
Spcua 2013 Alexey Kozhemiakin Enterprise Search
 
SharePoint site admins leverage search
SharePoint site admins leverage searchSharePoint site admins leverage search
SharePoint site admins leverage search
 
Effective Searching by Dominik Kornas
Effective Searching by Dominik KornasEffective Searching by Dominik Kornas
Effective Searching by Dominik Kornas
 

Recently uploaded

FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance
 
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Jeffrey Haguewood
 
FIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdfFIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance
 
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdfFIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance
 
Elevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object CalisthenicsElevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object Calisthenics
Dorra BARTAGUIZ
 
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*
Frank van Harmelen
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
Product School
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Albert Hoitingh
 
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
BookNet Canada
 
How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...
Product School
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Thierry Lestable
 
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdfSmart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
91mobiles
 
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 previewState of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
Prayukth K V
 
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMsTo Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
Paul Groth
 
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
Product School
 
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
UiPathCommunity
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
James Anderson
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
RTTS
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
DianaGray10
 
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
DanBrown980551
 

Recently uploaded (20)

FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
 
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
 
FIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdfFIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdf
 
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdfFIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
 
Elevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object CalisthenicsElevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object Calisthenics
 
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
 
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
 
How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
 
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdfSmart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
 
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 previewState of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
 
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMsTo Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
 
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
 
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
 
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
 

Project page zero, Smart Search, Learning to Personalize suggestions