SlideShare a Scribd company logo

Keynote: Bias in Search and Recommender Systems

•
•
Catalyst
Catalyst

Ricardo Baeza-Yates's slides from TechSEO Boost 2019

Marketing
1 of 55
Download to read offline
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost #TechSEOBoost | @CatalystSEM THANK YOU TO THIS YEAR’S SPONSORS Keynote: Bias in Search and Recommender Systems Ricardo Baeza-Yates, NTENT
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Ricardo Baeza-Yates CTO, NTENT Biases in Search & Recommender Systems
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost NTENT ntent.com Marketing Engineering Operations International Applied Research ntent.com
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Prologue
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost A Bit of History Data Volume Complexity IR DB Two different points of view for data
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Data Understanding Data Query Unstructured Structured Explicit Information Retrieval (Relational) Databases Implicit Recommender Systems Unknown Data Mining
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost What is Bias? • Statistical: significant systematic deviation from a prior (unknown) distribution; • Cultural: interpretations and judgments phenomena acquired through our life; • Cognitive: systematic pattern of deviation from norm or rationality in judgment;
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost So (Observational) Human Data has Bias • Gender • Racial • Sexual • Age • Religious • Social • Linguistic • Geographic • Political • Educational • Economic • Technological ▪ Gathering process ▪ Sampling process ▪ Validity (e.g. temporal) ▪ Completeness ▪ Noise, spam Many people extrapolate results of a sample to the whole population (e.g., social media analysis) In addition there is bias when measuring bias as well as bias towards measuring it! Attempt of an unbiased (personal) view on bias in Search & RS Cultural Biases Statistical Biases Cognitive Biases Self-selection
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Impact in Search and Recommender Systems • Most web systems are optimized by using implicit user feedback • However, user data is partly biased to the choices that these systems make • Clicks can only be done on things that are shown to us • As those systems are usually based in ML, they learn to reinforce their own biases, yielding self-fulfilled prophecies and/or sub-optimal solutions • For example, personalization and filter bubbles for users • but also echo chambers for (recommender) systems • Moreover, sometimes these systems compete among themselves, learning also biases of other systems rather than real user behavior • Even more, an improvement in one system might be just a degradation in another system that uses a different (even inversely correlated) optimization function • For example, user experience vs. monetization
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost A Non-Technical Question Algorithm Biased Data Neutral? Fair? Same Bias Garbage In Garbage Out
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost What is being fair?
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost A Non-Technical Question Algorithm Biased Data Neutral? Fair? Same Bias Not Always! Debias the input Tune the algorithm Debias the output Bias awareness!
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost January 2017 ACM US Statement on Algorithm Transparency and Accountability 1. Awareness 2. Access and redress 3. Accountability 4. Explanation 5. Data Provenance 6. Auditability 7. Validation and Testing Systems do not need to be perfect, they just need to be better than us
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Biases Everywhere!
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Data bias Biases on Search & RS: Web Case Study Web Spam
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost [Baeza-Yates, Castillo & López, Cybermetrics, 2005] Number of linked domains Exports(thousandsofUS$) Economic Bias in Links
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost 17 [Baeza-Yates & Castillo, WWW2006] Economic Bias in Links
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost 18 Minimal effortShameCultural Bias in Websites [Baeza-Yates, Castillo, Efthimiadis, TOIT 2007]
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Language Bias
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost [Bolukbasi at al, NIPS 2016] • Word embedding’s in w2vNEWS Yes, about 60 to 70% at work although at college is the inverse Gender Bias in Content Most journalists are men?
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost [E. Graells-Garrido et al,. ACM Hypertext’15] Systemic bias? Equal opportunity? Gender Bias in Content Wikipedia Partial information
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Data bias Activity bias Bias on Usage Actions People We are all in the long tail! [Goel et al., WSDM 2010]
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Popularity Bias in Recommender Systems Items Users Popular items Rest of items (long tail) • Take care to recommended items that are not too popular • Metrics • Novelty enhancement • Problem solved! …really? 𝑛𝑜𝑣 𝑖 = 1 − # ratings of 𝑖 # users Items #interactions More novel Less novel 𝑎 𝑏 [Vargas & Castells, RecSys 2011]
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost A Self-fulfilling Prophecy? Popular items (short head) Rest of items (long tail) Observed user-item interaction Unobserved preference Items Users Ratings are missing not at random (MNAR)
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Test data (relevant items) Training data Unobserved preference Items Users Popular items (short head) Rest of items (long tail) avg P@𝑘 ∼ + 𝑘 Judgments are missing not at random (MNAR) Worse yet: user-system reinforcement loop (more later) A Self-fulfilling Prophecy? [Marlin et al., RecSys 2010] [Steck, RecSys 2010, 2011] [Fleder & Hossanagar, Management Sciences 2009]
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost 1.E-06 1.E-05 1.E-04 1.E-03 0 0.2 0.4 0.6 0.8 1 A Problem for IR Evaluation Methodology! 30 TREC collections Items / documents #ratings/judgments (infraction) [Bellogín, Castells & Cantador, IRJ 2017] To how many queries is a document relevant? 25% queries can be answered with less than 1% of the URLs! [Baeza-Yates, Boldi, Chierichetti, WWW 2015]
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Get Rid of the Popularity Bias! • In the rating split [Bellogín, Castells & Cantador, IRJ 2017] • In the metrics • Stratified recall [Steck, RecSys 2011] • Importance propensity scoring [Yang et al., RecSys 2018] • In the algorithms [Steck, RecSys 2011] [Lobato et al., ICML 2014] [Jannach et al., UMUAI 2015] [Cañamares & Castells, SIGIR 2018, best paper award] Test data (relevant items) Training data Unobserved preference Items Items #ratings Flat test Popularity strata #ratings Time Temporal split
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Activity Bias also Affects Content [Baeza-Yates & Saez-Trumper, ACM Hypertext 2015] Most users are passive (i.e., more than 90% are lurkers) Then, which percentage of active users produce 50% of the content?
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost October 2015
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost [Baeza-Yates & Saez-Trumper, ACM Hypertext 2015] Which percentage of active users produce 50% of the content? Wisdom of crowds is a partial illusion Activity Bias: The Wisdom of a Few
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Content Never Seen [Baeza-Yates & Saez-Trumper, ACM Hypertext 2015] The Digital Desert
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Data bias Activity bias Sampling (size) bias Algorithmic bias Search or Recommender System Bias on the Web
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost [E. Graells-Garrido & M. Lalmas, ACM Hypertext’14] Geographical Bias in Recommender Systems
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost • If we want to estimate the frequency of queries that appear with probability at least p with a certain relative error, ∊, we can use the standard binomial error formula which works well for p near ½ but not for p near 0 • Better is the Agresti-Coull technique (also called Take 2) which gives: where Z is the inverse of the standard normal distribution, 1 − 𝛼 is the confidence interval and • If p = 0.1, 1 − 𝛼 is 80% and ∊ is 10%, the standard formula gives n = 900, while with A-C we get n = 2342. [Brown, Cai & DasGupta, Statistical Science, 2001] [Baeza-Yates, SIGIR 2015, Industry track] Sample Size?
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost • Standard technique: • A good sample should cover well all the items distribution, but this does not work with very skewed distributions. 10 0 10 1 10 2 10 3 Rank 10 0 10 1 10 2 10 3 10 4 10 5 Frequency 50M 10M 100K 1K 10 0 10 1 10 2 10 3 10 4 10 5 10 6 10 7 Rank 10 0 10 1 10 2 10 3 10 4 10 5 Frequency 50M 10M 100K 1K [Zaragoza et al, CIKM 2010] Sampling Queries
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost 36 Stratified Sampling Example [Baeza-Yates, SIGIR 2015, Industry track]
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Data bias Activity bias Sampling bias Interaction bias (Self) selection bias Bias in the User Interaction Search or Recommender System
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Position bias Ranking bias Presentation or exposure bias Social bias Interaction bias Bias in the Interaction Amazon.com
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Position bias Presentation bias Social bias Interaction bias Ranking bias Click bias Scrolling bias Mouse movement bias Data and algorithmic bias Self-selection bias Dependencies: A Cascade of Biases!
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Ranking Bias in Web Search [Mediative Study, 2014]
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Ranking Bias: Click Bias in Web Search • Ranking & next page bias Navigational queries
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost CTR (log) 1 11 21 Rank Learning to Rank with bias [Joachims et al., WSDM 2017, best paper] + many other papers Fair rankings [Zehlike et al., CIKM 2017] Clicks as implicit positive user feedback Debiasing Search Clicks and Other Biases [Dupret & Piwowarski, SIGIR 2008] [Chapelle & Zhang, WWW 2009] [Dupret & Liao, WSDM 2010, best paper] Debias the input Tune the algorithm Debias the output
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Data bias Activity bias Sampling bias Interaction bias (Self) selection bias Second-order bias Vicious Cycle of Bias Search or Recommender System
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost [Baeza-Yates, Pereira & Ziviani, WWW 2008] Person Web content is redundant (> 20%) Query Ranking bias in new content Redundancy grows (35%) Search results New page Second Order Bias in Web Content [Fortunato, Flammini, Menczer & Vespignani. PNAS 2006]
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Data bias Activity bias Sampling bias Interaction bias (Self) selection bias Vicious Cycle of Bias Search or Recommender System Feedback loop bias
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Bias due to Personalization • The effect of self-selection bias • Avoid the rich get richer and poor get poorer syndrome • Avoid the echo chamber by empowering the tail Cold start problem solution: Explore & Exploit Partial solutions: • Diversity • Novelty • Serendipity • My dark side Wikipedia [Eli Pariser, Penguin 2011]
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Users’ Eco Chambers in Recommender Systems ▪ Filter bubbles ▪ Degenerate feedback loops (e.g., YouTube autoplay) [Jiang et al., AAAI 2019]
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Eco Chamber of the Recommender System • Short-term greedy optimization, partial knowledge of the world • Long-term revenue optimization is not achieved • Views from new users should balance the exploration for new items • Disparate impact: unfair ecommerce/information markets [Baeza-Yates & Ribaudo, to appear]
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Epilogue
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Recap Bias Type Statistical Cultural Cognitive Algorithmic  ? ? Presentation  Position    Data   Sampling    Activity  Self-selection   Interaction   Social   Second order   
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost [Silberzahn et al., COS, Univ. of Virginia, 2015] Professional Bias? ➔ 61 analysts, 29 teams: 20 yes and 9 no
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost If Systems Reflects Its Designers: What we can/should do? ▪ Data ▪ Analyze for known and unknown biases, debias when possible/needed ▪ Recollect more data for difficult/sparse regions of the problem ▪ Delete attributes associated directly/indirectly with harmful bias ▪ Interaction ▪ Make sure that the user is aware of the biases all the time ▪ Give more control to the user ▪ Design and Implementation • Let experts/colleagues/users contest every step of the process ▪ Evaluation • Do not fool yourself!
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Final Take-Home Message ▪ Systems are a mirror of us, the good, the bad and the ugly ▪ The Web amplifies everything, but always leaves traces ▪ We need to be aware of our own biases! ▪ We must be aware of the biases and contrarrest them to stop the vicious bias cycle ▪ Plenty of open (research) problems! Big Data of People is huge….. ….. but it is tiny compared to the future Big Data of the Internet of Things (IoT) No activity bias!
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Thank You! Any Questions? – rbaeza@acm.org | http://www.baeza.cl/ | http://fairness-measures.org Biased Questions? ASIST 2012 Book of the Year Award (Biased Ad)
Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Thanks for Viewing the Slideshare! – Watch the Recording: https://youtube.com/session-example Or Contact us today to discover how Catalyst can deliver unparalleled SEO results for your business. https://www.catalystdigital.com/

Recommended

Fair Recommender Systems
Fair Recommender Systems Fair Recommender Systems
Fair Recommender Systems
Sharmistha Chatterjee
 
Tutorial on Advances in Bias-aware Recommendation on the Web @ WSDM 2021
Tutorial on Advances in Bias-aware Recommendation on the Web @ WSDM 2021Tutorial on Advances in Bias-aware Recommendation on the Web @ WSDM 2021
Tutorial on Advances in Bias-aware Recommendation on the Web @ WSDM 2021
Mirko Marras
 
Tutorial on Bias in Rec Sys @ UMAP2020
Tutorial on Bias in Rec Sys @ UMAP2020Tutorial on Bias in Rec Sys @ UMAP2020
Tutorial on Bias in Rec Sys @ UMAP2020
Mirko Marras
 
Fairness and Popularity Bias in Recommender Systems: an Empirical Evaluation
Fairness and Popularity Bias in Recommender Systems: an Empirical EvaluationFairness and Popularity Bias in Recommender Systems: an Empirical Evaluation
Fairness and Popularity Bias in Recommender Systems: an Empirical Evaluation
Cataldo Musto
 
Recent advances in deep recommender systems
Recent advances in deep recommender systemsRecent advances in deep recommender systems
Recent advances in deep recommender systems
NAVER Engineering
 
Tag based recommender system
Tag based recommender systemTag based recommender system
Tag based recommender system
Karen Li
 
Recommender system introduction
Recommender system introductionRecommender system introduction
Recommender system introductionLiang Xiang
 
RecSysTEL lecture at advanced SIKS course, NL
RecSysTEL lecture at advanced SIKS course, NLRecSysTEL lecture at advanced SIKS course, NL
RecSysTEL lecture at advanced SIKS course, NL
Hendrik Drachsler
 

Recommended

Fair Recommender Systems
Fair Recommender Systems Fair Recommender Systems
Fair Recommender Systems
Sharmistha Chatterjee
 
Tutorial on Advances in Bias-aware Recommendation on the Web @ WSDM 2021
Tutorial on Advances in Bias-aware Recommendation on the Web @ WSDM 2021Tutorial on Advances in Bias-aware Recommendation on the Web @ WSDM 2021
Tutorial on Advances in Bias-aware Recommendation on the Web @ WSDM 2021
Mirko Marras
 
Tutorial on Bias in Rec Sys @ UMAP2020
Tutorial on Bias in Rec Sys @ UMAP2020Tutorial on Bias in Rec Sys @ UMAP2020
Tutorial on Bias in Rec Sys @ UMAP2020
Mirko Marras
 
Fairness and Popularity Bias in Recommender Systems: an Empirical Evaluation
Fairness and Popularity Bias in Recommender Systems: an Empirical EvaluationFairness and Popularity Bias in Recommender Systems: an Empirical Evaluation
Fairness and Popularity Bias in Recommender Systems: an Empirical Evaluation
Cataldo Musto
 
Recent advances in deep recommender systems
Recent advances in deep recommender systemsRecent advances in deep recommender systems
Recent advances in deep recommender systems
NAVER Engineering
 
Tag based recommender system
Tag based recommender systemTag based recommender system
Tag based recommender system
Karen Li
 
Recommender system introduction
Recommender system introductionRecommender system introduction
Recommender system introductionLiang Xiang
 
RecSysTEL lecture at advanced SIKS course, NL
RecSysTEL lecture at advanced SIKS course, NLRecSysTEL lecture at advanced SIKS course, NL
RecSysTEL lecture at advanced SIKS course, NL
Hendrik Drachsler
 
Introduction to Recommendation Systems
Introduction to Recommendation SystemsIntroduction to Recommendation Systems
Introduction to Recommendation Systems
Trieu Nguyen
 
An introduction to Recommender Systems
An introduction to Recommender SystemsAn introduction to Recommender Systems
An introduction to Recommender Systems
David Zibriczky
 
Tutorial on sequence aware recommender systems - UMAP 2018
Tutorial on sequence aware recommender systems - UMAP 2018Tutorial on sequence aware recommender systems - UMAP 2018
Tutorial on sequence aware recommender systems - UMAP 2018
Paolo Cremonesi
 
Recommender system algorithm and architecture
Recommender system algorithm and architectureRecommender system algorithm and architecture
Recommender system algorithm and architectureLiang Xiang
 
Collaborative Recommender System for Music using PyTorch
Collaborative Recommender System for Music using PyTorchCollaborative Recommender System for Music using PyTorch
Collaborative Recommender System for Music using PyTorch
Valentin Nagacevschi
 
Recommendation Systems Basics
Recommendation Systems BasicsRecommendation Systems Basics
Recommendation Systems Basics
Jarin Tasnim Khan
 
Recommendation engines
Recommendation enginesRecommendation engines
Recommendation enginesGeorgian Micsa
 
Recommendation System Explained
Recommendation System ExplainedRecommendation System Explained
Recommendation System Explained
Crossing Minds
 
Anatomy of an eCommerce Search Engine by Mayur Datar
Anatomy of an eCommerce Search Engine by Mayur DatarAnatomy of an eCommerce Search Engine by Mayur Datar
Anatomy of an eCommerce Search Engine by Mayur Datar
Naresh Jain
 
Overview of recommender system
Overview of recommender systemOverview of recommender system
Overview of recommender system
Stanley Wang
 
Recommender system
Recommender systemRecommender system
Recommender system
Nilotpal Pramanik
 
Deep Learning for Recommender Systems
Deep Learning for Recommender SystemsDeep Learning for Recommender Systems
Deep Learning for Recommender Systems
Justin Basilico
 
Missing values in recommender models
Missing values in recommender modelsMissing values in recommender models
Missing values in recommender models
Parmeshwar Khurd
 
Recommender Systems
Recommender SystemsRecommender Systems
Recommender Systems
Girish Khanzode
 
Session-Based Recommender Systems
Session-Based Recommender SystemsSession-Based Recommender Systems
Session-Based Recommender Systems
Eötvös Loránd University
 
Knowledge Graphs and AI to Hyper-Personalise the Fashion Retail Experience at...
Knowledge Graphs and AI to Hyper-Personalise the Fashion Retail Experience at...Knowledge Graphs and AI to Hyper-Personalise the Fashion Retail Experience at...
Knowledge Graphs and AI to Hyper-Personalise the Fashion Retail Experience at...
Connected Data World
 
How to build a recommender system?
How to build a recommender system?How to build a recommender system?
How to build a recommender system?
blueace
 
Boston ML - Architecting Recommender Systems
Boston ML - Architecting Recommender SystemsBoston ML - Architecting Recommender Systems
Boston ML - Architecting Recommender Systems
James Kirk
 
Recommendation system by_arpit_sharma
Recommendation system by_arpit_sharmaRecommendation system by_arpit_sharma
Recommendation system by_arpit_sharma
Er. Arpit Sharma
 
Rishabh Mehrotra - Recommendations in a Marketplace: Personalizing Explainabl...
Rishabh Mehrotra - Recommendations in a Marketplace: Personalizing Explainabl...Rishabh Mehrotra - Recommendations in a Marketplace: Personalizing Explainabl...
Rishabh Mehrotra - Recommendations in a Marketplace: Personalizing Explainabl...
MLconf
 
Webinar - Harness the Power of Data with Tableau - 2016-02-18
Webinar - Harness the Power of Data with Tableau - 2016-02-18Webinar - Harness the Power of Data with Tableau - 2016-02-18
Webinar - Harness the Power of Data with Tableau - 2016-02-18
TechSoup
 
What is the Value of SAS Analytics?
What is the Value of SAS Analytics?What is the Value of SAS Analytics?
What is the Value of SAS Analytics?
SAS Canada
 

More Related Content

What's hot

Introduction to Recommendation Systems
Introduction to Recommendation SystemsIntroduction to Recommendation Systems
Introduction to Recommendation Systems
Trieu Nguyen
 
An introduction to Recommender Systems
An introduction to Recommender SystemsAn introduction to Recommender Systems
An introduction to Recommender Systems
David Zibriczky
 
Tutorial on sequence aware recommender systems - UMAP 2018
Tutorial on sequence aware recommender systems - UMAP 2018Tutorial on sequence aware recommender systems - UMAP 2018
Tutorial on sequence aware recommender systems - UMAP 2018
Paolo Cremonesi
 
Recommender system algorithm and architecture
Recommender system algorithm and architectureRecommender system algorithm and architecture
Recommender system algorithm and architectureLiang Xiang
 
Collaborative Recommender System for Music using PyTorch
Collaborative Recommender System for Music using PyTorchCollaborative Recommender System for Music using PyTorch
Collaborative Recommender System for Music using PyTorch
Valentin Nagacevschi
 
Recommendation Systems Basics
Recommendation Systems BasicsRecommendation Systems Basics
Recommendation Systems Basics
Jarin Tasnim Khan
 
Recommendation engines
Recommendation enginesRecommendation engines
Recommendation enginesGeorgian Micsa
 
Recommendation System Explained
Recommendation System ExplainedRecommendation System Explained
Recommendation System Explained
Crossing Minds
 
Anatomy of an eCommerce Search Engine by Mayur Datar
Anatomy of an eCommerce Search Engine by Mayur DatarAnatomy of an eCommerce Search Engine by Mayur Datar
Anatomy of an eCommerce Search Engine by Mayur Datar
Naresh Jain
 
Overview of recommender system
Overview of recommender systemOverview of recommender system
Overview of recommender system
Stanley Wang
 
Recommender system
Recommender systemRecommender system
Recommender system
Nilotpal Pramanik
 
Deep Learning for Recommender Systems
Deep Learning for Recommender SystemsDeep Learning for Recommender Systems
Deep Learning for Recommender Systems
Justin Basilico
 
Missing values in recommender models
Missing values in recommender modelsMissing values in recommender models
Missing values in recommender models
Parmeshwar Khurd
 
Recommender Systems
Recommender SystemsRecommender Systems
Recommender Systems
Girish Khanzode
 
Session-Based Recommender Systems
Session-Based Recommender SystemsSession-Based Recommender Systems
Session-Based Recommender Systems
Eötvös Loránd University
 
Knowledge Graphs and AI to Hyper-Personalise the Fashion Retail Experience at...
Knowledge Graphs and AI to Hyper-Personalise the Fashion Retail Experience at...Knowledge Graphs and AI to Hyper-Personalise the Fashion Retail Experience at...
Knowledge Graphs and AI to Hyper-Personalise the Fashion Retail Experience at...
Connected Data World
 
How to build a recommender system?
How to build a recommender system?How to build a recommender system?
How to build a recommender system?
blueace
 
Boston ML - Architecting Recommender Systems
Boston ML - Architecting Recommender SystemsBoston ML - Architecting Recommender Systems
Boston ML - Architecting Recommender Systems
James Kirk
 
Recommendation system by_arpit_sharma
Recommendation system by_arpit_sharmaRecommendation system by_arpit_sharma
Recommendation system by_arpit_sharma
Er. Arpit Sharma
 
Rishabh Mehrotra - Recommendations in a Marketplace: Personalizing Explainabl...
Rishabh Mehrotra - Recommendations in a Marketplace: Personalizing Explainabl...Rishabh Mehrotra - Recommendations in a Marketplace: Personalizing Explainabl...
Rishabh Mehrotra - Recommendations in a Marketplace: Personalizing Explainabl...
MLconf
 

What's hot (20)

Introduction to Recommendation Systems
Introduction to Recommendation SystemsIntroduction to Recommendation Systems
Introduction to Recommendation Systems
 
An introduction to Recommender Systems
An introduction to Recommender SystemsAn introduction to Recommender Systems
An introduction to Recommender Systems
 
Tutorial on sequence aware recommender systems - UMAP 2018
Tutorial on sequence aware recommender systems - UMAP 2018Tutorial on sequence aware recommender systems - UMAP 2018
Tutorial on sequence aware recommender systems - UMAP 2018
 
Recommender system algorithm and architecture
Recommender system algorithm and architectureRecommender system algorithm and architecture
Recommender system algorithm and architecture
 
Collaborative Recommender System for Music using PyTorch
Collaborative Recommender System for Music using PyTorchCollaborative Recommender System for Music using PyTorch
Collaborative Recommender System for Music using PyTorch
 
Recommendation Systems Basics
Recommendation Systems BasicsRecommendation Systems Basics
Recommendation Systems Basics
 
Recommendation engines
Recommendation enginesRecommendation engines
Recommendation engines
 
Recommendation System Explained
Recommendation System ExplainedRecommendation System Explained
Recommendation System Explained
 
Anatomy of an eCommerce Search Engine by Mayur Datar
Anatomy of an eCommerce Search Engine by Mayur DatarAnatomy of an eCommerce Search Engine by Mayur Datar
Anatomy of an eCommerce Search Engine by Mayur Datar
 
Overview of recommender system
Overview of recommender systemOverview of recommender system
Overview of recommender system
 
Recommender system
Recommender systemRecommender system
Recommender system
 
Deep Learning for Recommender Systems
Deep Learning for Recommender SystemsDeep Learning for Recommender Systems
Deep Learning for Recommender Systems
 
Missing values in recommender models
Missing values in recommender modelsMissing values in recommender models
Missing values in recommender models
 
Recommender Systems
Recommender SystemsRecommender Systems
Recommender Systems
 
Session-Based Recommender Systems
Session-Based Recommender SystemsSession-Based Recommender Systems
Session-Based Recommender Systems
 
Knowledge Graphs and AI to Hyper-Personalise the Fashion Retail Experience at...
Knowledge Graphs and AI to Hyper-Personalise the Fashion Retail Experience at...Knowledge Graphs and AI to Hyper-Personalise the Fashion Retail Experience at...
Knowledge Graphs and AI to Hyper-Personalise the Fashion Retail Experience at...
 
How to build a recommender system?
How to build a recommender system?How to build a recommender system?
How to build a recommender system?
 
Boston ML - Architecting Recommender Systems
Boston ML - Architecting Recommender SystemsBoston ML - Architecting Recommender Systems
Boston ML - Architecting Recommender Systems
 
Recommendation system by_arpit_sharma
Recommendation system by_arpit_sharmaRecommendation system by_arpit_sharma
Recommendation system by_arpit_sharma
 
Rishabh Mehrotra - Recommendations in a Marketplace: Personalizing Explainabl...
Rishabh Mehrotra - Recommendations in a Marketplace: Personalizing Explainabl...Rishabh Mehrotra - Recommendations in a Marketplace: Personalizing Explainabl...
Rishabh Mehrotra - Recommendations in a Marketplace: Personalizing Explainabl...
 

Similar to Keynote: Bias in Search and Recommender Systems

Webinar - Harness the Power of Data with Tableau - 2016-02-18
Webinar - Harness the Power of Data with Tableau - 2016-02-18Webinar - Harness the Power of Data with Tableau - 2016-02-18
Webinar - Harness the Power of Data with Tableau - 2016-02-18
TechSoup
 
What is the Value of SAS Analytics?
What is the Value of SAS Analytics?What is the Value of SAS Analytics?
What is the Value of SAS Analytics?
SAS Canada
 
Audit Webinar: Surefire ways to succeed with Data Analytics
Audit Webinar: Surefire ways to succeed with Data AnalyticsAudit Webinar: Surefire ways to succeed with Data Analytics
Audit Webinar: Surefire ways to succeed with Data Analytics
CaseWare IDEA
 
Managing Data Science | Lessons from the Field
Managing Data Science | Lessons from the Field Managing Data Science | Lessons from the Field
Managing Data Science | Lessons from the Field
Domino Data Lab
 
Big Data: Setting Up the Big Data Lake
Big Data: Setting Up the Big Data LakeBig Data: Setting Up the Big Data Lake
Big Data: Setting Up the Big Data Lake
Caserta
 
A Brief Introduction to Enterprise Architecture
A Brief Introduction to Enterprise Architecture A Brief Introduction to Enterprise Architecture
A Brief Introduction to Enterprise Architecture
Daljit Banger
 
Predictive Analytics - Big Data Warehousing Meetup
Predictive Analytics - Big Data Warehousing MeetupPredictive Analytics - Big Data Warehousing Meetup
Predictive Analytics - Big Data Warehousing Meetup
Caserta
 
Balancing Data Governance and Innovation
Balancing Data Governance and InnovationBalancing Data Governance and Innovation
Balancing Data Governance and Innovation
Caserta
 
Introduction to Data Science (Data Summit, 2017)
Introduction to Data Science (Data Summit, 2017)Introduction to Data Science (Data Summit, 2017)
Introduction to Data Science (Data Summit, 2017)
Caserta
 
Leadership beyond Agile XP2015
Leadership beyond Agile XP2015Leadership beyond Agile XP2015
Leadership beyond Agile XP2015
samililja
 
Building Information Governance Policies and Workflows
Building Information Governance Policies and WorkflowsBuilding Information Governance Policies and Workflows
Building Information Governance Policies and Workflows
kCura_Relativity
 
Simplifying Analytics - by Novoniel Deb
Simplifying Analytics - by Novoniel DebSimplifying Analytics - by Novoniel Deb
Simplifying Analytics - by Novoniel Deb
Novoniel Deb
 
Part 2 - 20 Years in Healthcare Analytics & Data Warehousing: What did we lea...
Part 2 - 20 Years in Healthcare Analytics & Data Warehousing: What did we lea...Part 2 - 20 Years in Healthcare Analytics & Data Warehousing: What did we lea...
Part 2 - 20 Years in Healthcare Analytics & Data Warehousing: What did we lea...
Health Catalyst
 
Measuring Impact: Towards a data citation metric
Measuring Impact: Towards a data citation metricMeasuring Impact: Towards a data citation metric
Measuring Impact: Towards a data citation metric
Edward Baker
 
The Data Lake - Balancing Data Governance and Innovation
The Data Lake - Balancing Data Governance and Innovation The Data Lake - Balancing Data Governance and Innovation
The Data Lake - Balancing Data Governance and Innovation
Caserta
 
Cerias talk on testing and evaluation
Cerias talk on testing and evaluationCerias talk on testing and evaluation
Cerias talk on testing and evaluation
International Center for Biometric Research
 
Audit: Breaking Down Barriers to Increase the Use of Data Analytics
Audit: Breaking Down Barriers to Increase the Use of Data AnalyticsAudit: Breaking Down Barriers to Increase the Use of Data Analytics
Audit: Breaking Down Barriers to Increase the Use of Data Analytics
CaseWare IDEA
 
Earley Executive Roundtable on Data Analytics - Session 2 - Mining Business I...
Earley Executive Roundtable on Data Analytics - Session 2 - Mining Business I...Earley Executive Roundtable on Data Analytics - Session 2 - Mining Business I...
Earley Executive Roundtable on Data Analytics - Session 2 - Mining Business I...
Earley Information Science
 
The State of Streaming Analytics: The Need for Speed and Scale
The State of Streaming Analytics: The Need for Speed and ScaleThe State of Streaming Analytics: The Need for Speed and Scale
The State of Streaming Analytics: The Need for Speed and Scale
VoltDB
 
Introduction to Data Science
Introduction to Data ScienceIntroduction to Data Science
Introduction to Data Science
Caserta
 

Similar to Keynote: Bias in Search and Recommender Systems (20)

Webinar - Harness the Power of Data with Tableau - 2016-02-18
Webinar - Harness the Power of Data with Tableau - 2016-02-18Webinar - Harness the Power of Data with Tableau - 2016-02-18
Webinar - Harness the Power of Data with Tableau - 2016-02-18
 
What is the Value of SAS Analytics?
What is the Value of SAS Analytics?What is the Value of SAS Analytics?
What is the Value of SAS Analytics?
 
Audit Webinar: Surefire ways to succeed with Data Analytics
Audit Webinar: Surefire ways to succeed with Data AnalyticsAudit Webinar: Surefire ways to succeed with Data Analytics
Audit Webinar: Surefire ways to succeed with Data Analytics
 
Managing Data Science | Lessons from the Field
Managing Data Science | Lessons from the Field Managing Data Science | Lessons from the Field
Managing Data Science | Lessons from the Field
 
Big Data: Setting Up the Big Data Lake
Big Data: Setting Up the Big Data LakeBig Data: Setting Up the Big Data Lake
Big Data: Setting Up the Big Data Lake
 
A Brief Introduction to Enterprise Architecture
A Brief Introduction to Enterprise Architecture A Brief Introduction to Enterprise Architecture
A Brief Introduction to Enterprise Architecture
 
Predictive Analytics - Big Data Warehousing Meetup
Predictive Analytics - Big Data Warehousing MeetupPredictive Analytics - Big Data Warehousing Meetup
Predictive Analytics - Big Data Warehousing Meetup
 
Balancing Data Governance and Innovation
Balancing Data Governance and InnovationBalancing Data Governance and Innovation
Balancing Data Governance and Innovation
 
Introduction to Data Science (Data Summit, 2017)
Introduction to Data Science (Data Summit, 2017)Introduction to Data Science (Data Summit, 2017)
Introduction to Data Science (Data Summit, 2017)
 
Leadership beyond Agile XP2015
Leadership beyond Agile XP2015Leadership beyond Agile XP2015
Leadership beyond Agile XP2015
 
Building Information Governance Policies and Workflows
Building Information Governance Policies and WorkflowsBuilding Information Governance Policies and Workflows
Building Information Governance Policies and Workflows
 
Simplifying Analytics - by Novoniel Deb
Simplifying Analytics - by Novoniel DebSimplifying Analytics - by Novoniel Deb
Simplifying Analytics - by Novoniel Deb
 
Part 2 - 20 Years in Healthcare Analytics & Data Warehousing: What did we lea...
Part 2 - 20 Years in Healthcare Analytics & Data Warehousing: What did we lea...Part 2 - 20 Years in Healthcare Analytics & Data Warehousing: What did we lea...
Part 2 - 20 Years in Healthcare Analytics & Data Warehousing: What did we lea...
 
Measuring Impact: Towards a data citation metric
Measuring Impact: Towards a data citation metricMeasuring Impact: Towards a data citation metric
Measuring Impact: Towards a data citation metric
 
The Data Lake - Balancing Data Governance and Innovation
The Data Lake - Balancing Data Governance and Innovation The Data Lake - Balancing Data Governance and Innovation
The Data Lake - Balancing Data Governance and Innovation
 
Cerias talk on testing and evaluation
Cerias talk on testing and evaluationCerias talk on testing and evaluation
Cerias talk on testing and evaluation
 
Audit: Breaking Down Barriers to Increase the Use of Data Analytics
Audit: Breaking Down Barriers to Increase the Use of Data AnalyticsAudit: Breaking Down Barriers to Increase the Use of Data Analytics
Audit: Breaking Down Barriers to Increase the Use of Data Analytics
 
Earley Executive Roundtable on Data Analytics - Session 2 - Mining Business I...
Earley Executive Roundtable on Data Analytics - Session 2 - Mining Business I...Earley Executive Roundtable on Data Analytics - Session 2 - Mining Business I...
Earley Executive Roundtable on Data Analytics - Session 2 - Mining Business I...
 
The State of Streaming Analytics: The Need for Speed and Scale
The State of Streaming Analytics: The Need for Speed and ScaleThe State of Streaming Analytics: The Need for Speed and Scale
The State of Streaming Analytics: The Need for Speed and Scale
 
Introduction to Data Science
Introduction to Data ScienceIntroduction to Data Science
Introduction to Data Science
 

More from Catalyst

Closing the Gap: Adopting Omnichannel Strategies for Stronger Brand-Consumer ...
Closing the Gap: Adopting Omnichannel Strategies for Stronger Brand-Consumer ...Closing the Gap: Adopting Omnichannel Strategies for Stronger Brand-Consumer ...
Closing the Gap: Adopting Omnichannel Strategies for Stronger Brand-Consumer ...
Catalyst
 
TechSEO Boost 2021 - Cultivating a Product Mindset for Success
TechSEO Boost 2021 - Cultivating a Product Mindset for SuccessTechSEO Boost 2021 - Cultivating a Product Mindset for Success
TechSEO Boost 2021 - Cultivating a Product Mindset for Success
Catalyst
 
TechSEO Boost 2021 - SEO Experimentation
TechSEO Boost 2021 - SEO ExperimentationTechSEO Boost 2021 - SEO Experimentation
TechSEO Boost 2021 - SEO Experimentation
Catalyst
 
TechSEO Boost 2021 - Rendering Strategies: Measuring the Devil’s Details in C...
TechSEO Boost 2021 - Rendering Strategies: Measuring the Devil’s Details in C...TechSEO Boost 2021 - Rendering Strategies: Measuring the Devil’s Details in C...
TechSEO Boost 2021 - Rendering Strategies: Measuring the Devil’s Details in C...
Catalyst
 
TechSEO Boost 2021 - The Future Is The Past: Tagging And Tracking Through The...
TechSEO Boost 2021 - The Future Is The Past: Tagging And Tracking Through The...TechSEO Boost 2021 - The Future Is The Past: Tagging And Tracking Through The...
TechSEO Boost 2021 - The Future Is The Past: Tagging And Tracking Through The...
Catalyst
 
10 Trends Changing Programmatic
10 Trends Changing Programmatic10 Trends Changing Programmatic
10 Trends Changing Programmatic
Catalyst
 
New Commerce Conference: Charting a Course to Success with Your Retail Media ...
New Commerce Conference: Charting a Course to Success with Your Retail Media ...New Commerce Conference: Charting a Course to Success with Your Retail Media ...
New Commerce Conference: Charting a Course to Success with Your Retail Media ...
Catalyst
 
The New Commerce Conference: The Omni-channel Imperative
The New Commerce Conference: The Omni-channel ImperativeThe New Commerce Conference: The Omni-channel Imperative
The New Commerce Conference: The Omni-channel Imperative
Catalyst
 
New Commerce Commerce: All Things Instacart
New Commerce Commerce: All Things InstacartNew Commerce Commerce: All Things Instacart
New Commerce Commerce: All Things Instacart
Catalyst
 
The Power of SEO: Protect Your Bottom Line & Future Proof Your Brand
The Power of SEO: Protect Your Bottom Line & Future Proof Your BrandThe Power of SEO: Protect Your Bottom Line & Future Proof Your Brand
The Power of SEO: Protect Your Bottom Line & Future Proof Your Brand
Catalyst
 
The Era of Omni-Commerce: New Insights for Dominating the Digital Shelf and B...
The Era of Omni-Commerce: New Insights for Dominating the Digital Shelf and B...The Era of Omni-Commerce: New Insights for Dominating the Digital Shelf and B...
The Era of Omni-Commerce: New Insights for Dominating the Digital Shelf and B...
Catalyst
 
Reignite Your Business with Performance Marketing: 4 Ways to Fuel Your Reopening
Reignite Your Business with Performance Marketing: 4 Ways to Fuel Your ReopeningReignite Your Business with Performance Marketing: 4 Ways to Fuel Your Reopening
Reignite Your Business with Performance Marketing: 4 Ways to Fuel Your Reopening
Catalyst
 
Reignite Your Business with Performance Marketing: 4 Ways to Dial-Up Brand In...
Reignite Your Business with Performance Marketing: 4 Ways to Dial-Up Brand In...Reignite Your Business with Performance Marketing: 4 Ways to Dial-Up Brand In...
Reignite Your Business with Performance Marketing: 4 Ways to Dial-Up Brand In...
Catalyst
 
Evolve Your Social Commerce Strategy: Thinking Beyond Facebook
Evolve Your Social Commerce Strategy: Thinking Beyond FacebookEvolve Your Social Commerce Strategy: Thinking Beyond Facebook
Evolve Your Social Commerce Strategy: Thinking Beyond Facebook
Catalyst
 
B2B SEO: Increase Traffic & Leads in 2020
B2B SEO: Increase Traffic & Leads in 2020B2B SEO: Increase Traffic & Leads in 2020
B2B SEO: Increase Traffic & Leads in 2020
Catalyst
 
TechSEO Boost 2019: Research Competition
TechSEO Boost 2019: Research CompetitionTechSEO Boost 2019: Research Competition
TechSEO Boost 2019: Research Competition
Catalyst
 
NLP Powered Outreach Link Building
NLP Powered Outreach Link BuildingNLP Powered Outreach Link Building
NLP Powered Outreach Link Building
Catalyst
 
Generating Qualitative Content with GPT-2 in All Languages
Generating Qualitative Content with GPT-2 in All LanguagesGenerating Qualitative Content with GPT-2 in All Languages
Generating Qualitative Content with GPT-2 in All Languages
Catalyst
 
Automate, Create Tools, & Test Ideas Quickly with Google Apps Script
Automate, Create Tools, & Test Ideas Quickly with Google Apps ScriptAutomate, Create Tools, & Test Ideas Quickly with Google Apps Script
Automate, Create Tools, & Test Ideas Quickly with Google Apps Script
Catalyst
 
NLP for SEO
NLP for SEONLP for SEO
NLP for SEO
Catalyst
 

More from Catalyst (20)

Closing the Gap: Adopting Omnichannel Strategies for Stronger Brand-Consumer ...
Closing the Gap: Adopting Omnichannel Strategies for Stronger Brand-Consumer ...Closing the Gap: Adopting Omnichannel Strategies for Stronger Brand-Consumer ...
Closing the Gap: Adopting Omnichannel Strategies for Stronger Brand-Consumer ...
 
TechSEO Boost 2021 - Cultivating a Product Mindset for Success
TechSEO Boost 2021 - Cultivating a Product Mindset for SuccessTechSEO Boost 2021 - Cultivating a Product Mindset for Success
TechSEO Boost 2021 - Cultivating a Product Mindset for Success
 
TechSEO Boost 2021 - SEO Experimentation
TechSEO Boost 2021 - SEO ExperimentationTechSEO Boost 2021 - SEO Experimentation
TechSEO Boost 2021 - SEO Experimentation
 
TechSEO Boost 2021 - Rendering Strategies: Measuring the Devil’s Details in C...
TechSEO Boost 2021 - Rendering Strategies: Measuring the Devil’s Details in C...TechSEO Boost 2021 - Rendering Strategies: Measuring the Devil’s Details in C...
TechSEO Boost 2021 - Rendering Strategies: Measuring the Devil’s Details in C...
 
TechSEO Boost 2021 - The Future Is The Past: Tagging And Tracking Through The...
TechSEO Boost 2021 - The Future Is The Past: Tagging And Tracking Through The...TechSEO Boost 2021 - The Future Is The Past: Tagging And Tracking Through The...
TechSEO Boost 2021 - The Future Is The Past: Tagging And Tracking Through The...
 
10 Trends Changing Programmatic
10 Trends Changing Programmatic10 Trends Changing Programmatic
10 Trends Changing Programmatic
 
New Commerce Conference: Charting a Course to Success with Your Retail Media ...
New Commerce Conference: Charting a Course to Success with Your Retail Media ...New Commerce Conference: Charting a Course to Success with Your Retail Media ...
New Commerce Conference: Charting a Course to Success with Your Retail Media ...
 
The New Commerce Conference: The Omni-channel Imperative
The New Commerce Conference: The Omni-channel ImperativeThe New Commerce Conference: The Omni-channel Imperative
The New Commerce Conference: The Omni-channel Imperative
 
New Commerce Commerce: All Things Instacart
New Commerce Commerce: All Things InstacartNew Commerce Commerce: All Things Instacart
New Commerce Commerce: All Things Instacart
 
The Power of SEO: Protect Your Bottom Line & Future Proof Your Brand
The Power of SEO: Protect Your Bottom Line & Future Proof Your BrandThe Power of SEO: Protect Your Bottom Line & Future Proof Your Brand
The Power of SEO: Protect Your Bottom Line & Future Proof Your Brand
 
The Era of Omni-Commerce: New Insights for Dominating the Digital Shelf and B...
The Era of Omni-Commerce: New Insights for Dominating the Digital Shelf and B...The Era of Omni-Commerce: New Insights for Dominating the Digital Shelf and B...
The Era of Omni-Commerce: New Insights for Dominating the Digital Shelf and B...
 
Reignite Your Business with Performance Marketing: 4 Ways to Fuel Your Reopening
Reignite Your Business with Performance Marketing: 4 Ways to Fuel Your ReopeningReignite Your Business with Performance Marketing: 4 Ways to Fuel Your Reopening
Reignite Your Business with Performance Marketing: 4 Ways to Fuel Your Reopening
 
Reignite Your Business with Performance Marketing: 4 Ways to Dial-Up Brand In...
Reignite Your Business with Performance Marketing: 4 Ways to Dial-Up Brand In...Reignite Your Business with Performance Marketing: 4 Ways to Dial-Up Brand In...
Reignite Your Business with Performance Marketing: 4 Ways to Dial-Up Brand In...
 
Evolve Your Social Commerce Strategy: Thinking Beyond Facebook
Evolve Your Social Commerce Strategy: Thinking Beyond FacebookEvolve Your Social Commerce Strategy: Thinking Beyond Facebook
Evolve Your Social Commerce Strategy: Thinking Beyond Facebook
 
B2B SEO: Increase Traffic & Leads in 2020
B2B SEO: Increase Traffic & Leads in 2020B2B SEO: Increase Traffic & Leads in 2020
B2B SEO: Increase Traffic & Leads in 2020
 
TechSEO Boost 2019: Research Competition
TechSEO Boost 2019: Research CompetitionTechSEO Boost 2019: Research Competition
TechSEO Boost 2019: Research Competition
 
NLP Powered Outreach Link Building
NLP Powered Outreach Link BuildingNLP Powered Outreach Link Building
NLP Powered Outreach Link Building
 
Generating Qualitative Content with GPT-2 in All Languages
Generating Qualitative Content with GPT-2 in All LanguagesGenerating Qualitative Content with GPT-2 in All Languages
Generating Qualitative Content with GPT-2 in All Languages
 
Automate, Create Tools, & Test Ideas Quickly with Google Apps Script
Automate, Create Tools, & Test Ideas Quickly with Google Apps ScriptAutomate, Create Tools, & Test Ideas Quickly with Google Apps Script
Automate, Create Tools, & Test Ideas Quickly with Google Apps Script
 
NLP for SEO
NLP for SEONLP for SEO
NLP for SEO
 

Recently uploaded

Turn Digital Reputation Threats into Offense Tactics - Daniel Lemin
Turn Digital Reputation Threats into Offense Tactics - Daniel LeminTurn Digital Reputation Threats into Offense Tactics - Daniel Lemin
Turn Digital Reputation Threats into Offense Tactics - Daniel Lemin
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
 
Digital Marketing Trends - Experts Insights on How to Gain a Competitive Edge
Digital Marketing Trends - Experts Insights on How to Gain a Competitive EdgeDigital Marketing Trends - Experts Insights on How to Gain a Competitive Edge
Digital Marketing Trends - Experts Insights on How to Gain a Competitive Edge
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
 
Traditional Store Audits are Outdated: A New Approach to Protecting Your Bran...
Traditional Store Audits are Outdated: A New Approach to Protecting Your Bran...Traditional Store Audits are Outdated: A New Approach to Protecting Your Bran...
Traditional Store Audits are Outdated: A New Approach to Protecting Your Bran...
Auxis Consulting & Outsourcing
 
SMM Cheap - No. 1 SMM panel in the world
SMM Cheap - No. 1 SMM panel in the worldSMM Cheap - No. 1 SMM panel in the world
SMM Cheap - No. 1 SMM panel in the world
smmpanel567
 
Marketing as a Primary Revenue Driver - Lee Levitt
Marketing as a Primary Revenue Driver - Lee LevittMarketing as a Primary Revenue Driver - Lee Levitt
Marketing as a Primary Revenue Driver - Lee Levitt
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
 
How to Run Landing Page Tests On and Off Paid Social Platforms
How to Run Landing Page Tests On and Off Paid Social PlatformsHow to Run Landing Page Tests On and Off Paid Social Platforms
How to Run Landing Page Tests On and Off Paid Social Platforms
VWO
 
How to Use AI to Write a High-Quality Article that Ranks
How to Use AI to Write a High-Quality Article that RanksHow to Use AI to Write a High-Quality Article that Ranks
How to Use AI to Write a High-Quality Article that Ranks
minatamang0021
 
May 2024 - VBOUT Partners Meeting Group Session
May 2024 - VBOUT Partners Meeting Group SessionMay 2024 - VBOUT Partners Meeting Group Session
May 2024 - VBOUT Partners Meeting Group Session
Vbout.com
 
Coca Cola Branding Strategy and strategic marketing plan
Coca Cola Branding Strategy and strategic marketing planCoca Cola Branding Strategy and strategic marketing plan
Coca Cola Branding Strategy and strategic marketing plan
Maswer Ali
 
Email Marketing Master Class - Chris Ferris
Email Marketing Master Class - Chris FerrisEmail Marketing Master Class - Chris Ferris
Email Marketing Master Class - Chris Ferris
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
 
The Old Oak - Press Kit - Cannes Film Festival 2023
The Old Oak - Press Kit - Cannes Film Festival 2023The Old Oak - Press Kit - Cannes Film Festival 2023
The Old Oak - Press Kit - Cannes Film Festival 2023
Pascal Fintoni
 
BLOOM_May2024 (r). Balmer Lawrie Online Monthly Bulletin
BLOOM_May2024 (r). Balmer Lawrie Online Monthly BulletinBLOOM_May2024 (r). Balmer Lawrie Online Monthly Bulletin
BLOOM_May2024 (r). Balmer Lawrie Online Monthly Bulletin
BalmerLawrie
 
Digital Marketing Trends - Experts Insights on How
Digital Marketing Trends - Experts Insights on HowDigital Marketing Trends - Experts Insights on How
Digital Marketing Trends - Experts Insights on How
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
 
Your Path to Profits - The Game-Changing Power of a Marketing OS for Your Bus...
Your Path to Profits - The Game-Changing Power of a Marketing OS for Your Bus...Your Path to Profits - The Game-Changing Power of a Marketing OS for Your Bus...
Your Path to Profits - The Game-Changing Power of a Marketing OS for Your Bus...
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
 
Monthly Social Media News Update May 2024
Monthly Social Media News Update May 2024Monthly Social Media News Update May 2024
Monthly Social Media News Update May 2024
Andy Lambert
 
Unknown to Unforgettable - The Art and Science to Being Irresistible on Camer...
Unknown to Unforgettable - The Art and Science to Being Irresistible on Camer...Unknown to Unforgettable - The Art and Science to Being Irresistible on Camer...
Unknown to Unforgettable - The Art and Science to Being Irresistible on Camer...
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
 
The New Era Of SEO - How AI Has Changed SEO Forever - Danny Leibrandt
The New Era Of SEO - How AI Has Changed SEO Forever - Danny LeibrandtThe New Era Of SEO - How AI Has Changed SEO Forever - Danny Leibrandt
The New Era Of SEO - How AI Has Changed SEO Forever - Danny Leibrandt
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
 
Unlocking Everyday Narratives: The Power of Storytelling in Marketing - Chad...
Unlocking Everyday Narratives: The Power of Storytelling in Marketing - Chad...Unlocking Everyday Narratives: The Power of Storytelling in Marketing - Chad...
Unlocking Everyday Narratives: The Power of Storytelling in Marketing - Chad...
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
 
ThinkNow 2024 Consumer Financial Wellness Report
ThinkNow 2024 Consumer Financial Wellness ReportThinkNow 2024 Consumer Financial Wellness Report
ThinkNow 2024 Consumer Financial Wellness Report
ThinkNow
 
Turn Digital Reputation Threats into Offense Tactics - Daniel Lemin
Turn Digital Reputation Threats into Offense Tactics - Daniel LeminTurn Digital Reputation Threats into Offense Tactics - Daniel Lemin
Turn Digital Reputation Threats into Offense Tactics - Daniel Lemin
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
 

Recently uploaded (20)

Turn Digital Reputation Threats into Offense Tactics - Daniel Lemin
Turn Digital Reputation Threats into Offense Tactics - Daniel LeminTurn Digital Reputation Threats into Offense Tactics - Daniel Lemin
Turn Digital Reputation Threats into Offense Tactics - Daniel Lemin
 
Digital Marketing Trends - Experts Insights on How to Gain a Competitive Edge
Digital Marketing Trends - Experts Insights on How to Gain a Competitive EdgeDigital Marketing Trends - Experts Insights on How to Gain a Competitive Edge
Digital Marketing Trends - Experts Insights on How to Gain a Competitive Edge
 
Traditional Store Audits are Outdated: A New Approach to Protecting Your Bran...
Traditional Store Audits are Outdated: A New Approach to Protecting Your Bran...Traditional Store Audits are Outdated: A New Approach to Protecting Your Bran...
Traditional Store Audits are Outdated: A New Approach to Protecting Your Bran...
 
SMM Cheap - No. 1 SMM panel in the world
SMM Cheap - No. 1 SMM panel in the worldSMM Cheap - No. 1 SMM panel in the world
SMM Cheap - No. 1 SMM panel in the world
 
Marketing as a Primary Revenue Driver - Lee Levitt
Marketing as a Primary Revenue Driver - Lee LevittMarketing as a Primary Revenue Driver - Lee Levitt
Marketing as a Primary Revenue Driver - Lee Levitt
 
How to Run Landing Page Tests On and Off Paid Social Platforms
How to Run Landing Page Tests On and Off Paid Social PlatformsHow to Run Landing Page Tests On and Off Paid Social Platforms
How to Run Landing Page Tests On and Off Paid Social Platforms
 
How to Use AI to Write a High-Quality Article that Ranks
How to Use AI to Write a High-Quality Article that RanksHow to Use AI to Write a High-Quality Article that Ranks
How to Use AI to Write a High-Quality Article that Ranks
 
May 2024 - VBOUT Partners Meeting Group Session
May 2024 - VBOUT Partners Meeting Group SessionMay 2024 - VBOUT Partners Meeting Group Session
May 2024 - VBOUT Partners Meeting Group Session
 
Coca Cola Branding Strategy and strategic marketing plan
Coca Cola Branding Strategy and strategic marketing planCoca Cola Branding Strategy and strategic marketing plan
Coca Cola Branding Strategy and strategic marketing plan
 
Email Marketing Master Class - Chris Ferris
Email Marketing Master Class - Chris FerrisEmail Marketing Master Class - Chris Ferris
Email Marketing Master Class - Chris Ferris
 
The Old Oak - Press Kit - Cannes Film Festival 2023
The Old Oak - Press Kit - Cannes Film Festival 2023The Old Oak - Press Kit - Cannes Film Festival 2023
The Old Oak - Press Kit - Cannes Film Festival 2023
 
BLOOM_May2024 (r). Balmer Lawrie Online Monthly Bulletin
BLOOM_May2024 (r). Balmer Lawrie Online Monthly BulletinBLOOM_May2024 (r). Balmer Lawrie Online Monthly Bulletin
BLOOM_May2024 (r). Balmer Lawrie Online Monthly Bulletin
 
Digital Marketing Trends - Experts Insights on How
Digital Marketing Trends - Experts Insights on HowDigital Marketing Trends - Experts Insights on How
Digital Marketing Trends - Experts Insights on How
 
Your Path to Profits - The Game-Changing Power of a Marketing OS for Your Bus...
Your Path to Profits - The Game-Changing Power of a Marketing OS for Your Bus...Your Path to Profits - The Game-Changing Power of a Marketing OS for Your Bus...
Your Path to Profits - The Game-Changing Power of a Marketing OS for Your Bus...
 
Monthly Social Media News Update May 2024
Monthly Social Media News Update May 2024Monthly Social Media News Update May 2024
Monthly Social Media News Update May 2024
 
Unknown to Unforgettable - The Art and Science to Being Irresistible on Camer...
Unknown to Unforgettable - The Art and Science to Being Irresistible on Camer...Unknown to Unforgettable - The Art and Science to Being Irresistible on Camer...
Unknown to Unforgettable - The Art and Science to Being Irresistible on Camer...
 
The New Era Of SEO - How AI Has Changed SEO Forever - Danny Leibrandt
The New Era Of SEO - How AI Has Changed SEO Forever - Danny LeibrandtThe New Era Of SEO - How AI Has Changed SEO Forever - Danny Leibrandt
The New Era Of SEO - How AI Has Changed SEO Forever - Danny Leibrandt
 
Unlocking Everyday Narratives: The Power of Storytelling in Marketing - Chad...
Unlocking Everyday Narratives: The Power of Storytelling in Marketing - Chad...Unlocking Everyday Narratives: The Power of Storytelling in Marketing - Chad...
Unlocking Everyday Narratives: The Power of Storytelling in Marketing - Chad...
 
ThinkNow 2024 Consumer Financial Wellness Report
ThinkNow 2024 Consumer Financial Wellness ReportThinkNow 2024 Consumer Financial Wellness Report
ThinkNow 2024 Consumer Financial Wellness Report
 
Turn Digital Reputation Threats into Offense Tactics - Daniel Lemin
Turn Digital Reputation Threats into Offense Tactics - Daniel LeminTurn Digital Reputation Threats into Offense Tactics - Daniel Lemin
Turn Digital Reputation Threats into Offense Tactics - Daniel Lemin
 

Keynote: Bias in Search and Recommender Systems

  • 1. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost #TechSEOBoost | @CatalystSEM THANK YOU TO THIS YEAR’S SPONSORS Keynote: Bias in Search and Recommender Systems Ricardo Baeza-Yates, NTENT
  • 2. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Ricardo Baeza-Yates CTO, NTENT Biases in Search & Recommender Systems
  • 3. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost NTENT ntent.com Marketing Engineering Operations International Applied Research ntent.com
  • 4. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Prologue
  • 5. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost A Bit of History Data Volume Complexity IR DB Two different points of view for data
  • 6. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Data Understanding Data Query Unstructured Structured Explicit Information Retrieval (Relational) Databases Implicit Recommender Systems Unknown Data Mining
  • 7. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost What is Bias? • Statistical: significant systematic deviation from a prior (unknown) distribution; • Cultural: interpretations and judgments phenomena acquired through our life; • Cognitive: systematic pattern of deviation from norm or rationality in judgment;
  • 8. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost So (Observational) Human Data has Bias • Gender • Racial • Sexual • Age • Religious • Social • Linguistic • Geographic • Political • Educational • Economic • Technological ▪ Gathering process ▪ Sampling process ▪ Validity (e.g. temporal) ▪ Completeness ▪ Noise, spam Many people extrapolate results of a sample to the whole population (e.g., social media analysis) In addition there is bias when measuring bias as well as bias towards measuring it! Attempt of an unbiased (personal) view on bias in Search & RS Cultural Biases Statistical Biases Cognitive Biases Self-selection
  • 9. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Impact in Search and Recommender Systems • Most web systems are optimized by using implicit user feedback • However, user data is partly biased to the choices that these systems make • Clicks can only be done on things that are shown to us • As those systems are usually based in ML, they learn to reinforce their own biases, yielding self-fulfilled prophecies and/or sub-optimal solutions • For example, personalization and filter bubbles for users • but also echo chambers for (recommender) systems • Moreover, sometimes these systems compete among themselves, learning also biases of other systems rather than real user behavior • Even more, an improvement in one system might be just a degradation in another system that uses a different (even inversely correlated) optimization function • For example, user experience vs. monetization
  • 10. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost A Non-Technical Question Algorithm Biased Data Neutral? Fair? Same Bias Garbage In Garbage Out
  • 11. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost What is being fair?
  • 12. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost A Non-Technical Question Algorithm Biased Data Neutral? Fair? Same Bias Not Always! Debias the input Tune the algorithm Debias the output Bias awareness!
  • 13. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost January 2017 ACM US Statement on Algorithm Transparency and Accountability 1. Awareness 2. Access and redress 3. Accountability 4. Explanation 5. Data Provenance 6. Auditability 7. Validation and Testing Systems do not need to be perfect, they just need to be better than us
  • 14. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Biases Everywhere!
  • 15. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Data bias Biases on Search & RS: Web Case Study Web Spam
  • 16. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost [Baeza-Yates, Castillo & López, Cybermetrics, 2005] Number of linked domains Exports(thousandsofUS$) Economic Bias in Links
  • 17. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost 17 [Baeza-Yates & Castillo, WWW2006] Economic Bias in Links
  • 18. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost 18 Minimal effortShameCultural Bias in Websites [Baeza-Yates, Castillo, Efthimiadis, TOIT 2007]
  • 19. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Language Bias
  • 20. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost [Bolukbasi at al, NIPS 2016] • Word embedding’s in w2vNEWS Yes, about 60 to 70% at work although at college is the inverse Gender Bias in Content Most journalists are men?
  • 21. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost [E. Graells-Garrido et al,. ACM Hypertext’15] Systemic bias? Equal opportunity? Gender Bias in Content Wikipedia Partial information
  • 22. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Data bias Activity bias Bias on Usage Actions People We are all in the long tail! [Goel et al., WSDM 2010]
  • 23. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Popularity Bias in Recommender Systems Items Users Popular items Rest of items (long tail) • Take care to recommended items that are not too popular • Metrics • Novelty enhancement • Problem solved! …really? 𝑛𝑜𝑣 𝑖 = 1 − # ratings of 𝑖 # users Items #interactions More novel Less novel 𝑎 𝑏 [Vargas & Castells, RecSys 2011]
  • 24. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost A Self-fulfilling Prophecy? Popular items (short head) Rest of items (long tail) Observed user-item interaction Unobserved preference Items Users Ratings are missing not at random (MNAR)
  • 25. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Test data (relevant items) Training data Unobserved preference Items Users Popular items (short head) Rest of items (long tail) avg P@𝑘 ∼ + 𝑘 Judgments are missing not at random (MNAR) Worse yet: user-system reinforcement loop (more later) A Self-fulfilling Prophecy? [Marlin et al., RecSys 2010] [Steck, RecSys 2010, 2011] [Fleder & Hossanagar, Management Sciences 2009]
  • 26. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost 1.E-06 1.E-05 1.E-04 1.E-03 0 0.2 0.4 0.6 0.8 1 A Problem for IR Evaluation Methodology! 30 TREC collections Items / documents #ratings/judgments (infraction) [Bellogín, Castells & Cantador, IRJ 2017] To how many queries is a document relevant? 25% queries can be answered with less than 1% of the URLs! [Baeza-Yates, Boldi, Chierichetti, WWW 2015]
  • 27. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Get Rid of the Popularity Bias! • In the rating split [Bellogín, Castells & Cantador, IRJ 2017] • In the metrics • Stratified recall [Steck, RecSys 2011] • Importance propensity scoring [Yang et al., RecSys 2018] • In the algorithms [Steck, RecSys 2011] [Lobato et al., ICML 2014] [Jannach et al., UMUAI 2015] [Cañamares & Castells, SIGIR 2018, best paper award] Test data (relevant items) Training data Unobserved preference Items Items #ratings Flat test Popularity strata #ratings Time Temporal split
  • 28. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Activity Bias also Affects Content [Baeza-Yates & Saez-Trumper, ACM Hypertext 2015] Most users are passive (i.e., more than 90% are lurkers) Then, which percentage of active users produce 50% of the content?
  • 29. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost October 2015
  • 30. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost [Baeza-Yates & Saez-Trumper, ACM Hypertext 2015] Which percentage of active users produce 50% of the content? Wisdom of crowds is a partial illusion Activity Bias: The Wisdom of a Few
  • 31. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Content Never Seen [Baeza-Yates & Saez-Trumper, ACM Hypertext 2015] The Digital Desert
  • 32. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Data bias Activity bias Sampling (size) bias Algorithmic bias Search or Recommender System Bias on the Web
  • 33. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost [E. Graells-Garrido & M. Lalmas, ACM Hypertext’14] Geographical Bias in Recommender Systems
  • 34. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost • If we want to estimate the frequency of queries that appear with probability at least p with a certain relative error, ∊, we can use the standard binomial error formula which works well for p near ½ but not for p near 0 • Better is the Agresti-Coull technique (also called Take 2) which gives: where Z is the inverse of the standard normal distribution, 1 − 𝛼 is the confidence interval and • If p = 0.1, 1 − 𝛼 is 80% and ∊ is 10%, the standard formula gives n = 900, while with A-C we get n = 2342. [Brown, Cai & DasGupta, Statistical Science, 2001] [Baeza-Yates, SIGIR 2015, Industry track] Sample Size?
  • 35. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost • Standard technique: • A good sample should cover well all the items distribution, but this does not work with very skewed distributions. 10 0 10 1 10 2 10 3 Rank 10 0 10 1 10 2 10 3 10 4 10 5 Frequency 50M 10M 100K 1K 10 0 10 1 10 2 10 3 10 4 10 5 10 6 10 7 Rank 10 0 10 1 10 2 10 3 10 4 10 5 Frequency 50M 10M 100K 1K [Zaragoza et al, CIKM 2010] Sampling Queries
  • 36. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost 36 Stratified Sampling Example [Baeza-Yates, SIGIR 2015, Industry track]
  • 37. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Data bias Activity bias Sampling bias Interaction bias (Self) selection bias Bias in the User Interaction Search or Recommender System
  • 38. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Position bias Ranking bias Presentation or exposure bias Social bias Interaction bias Bias in the Interaction Amazon.com
  • 39. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Position bias Presentation bias Social bias Interaction bias Ranking bias Click bias Scrolling bias Mouse movement bias Data and algorithmic bias Self-selection bias Dependencies: A Cascade of Biases!
  • 40. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Ranking Bias in Web Search [Mediative Study, 2014]
  • 41. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Ranking Bias: Click Bias in Web Search • Ranking & next page bias Navigational queries
  • 42. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost CTR (log) 1 11 21 Rank Learning to Rank with bias [Joachims et al., WSDM 2017, best paper] + many other papers Fair rankings [Zehlike et al., CIKM 2017] Clicks as implicit positive user feedback Debiasing Search Clicks and Other Biases [Dupret & Piwowarski, SIGIR 2008] [Chapelle & Zhang, WWW 2009] [Dupret & Liao, WSDM 2010, best paper] Debias the input Tune the algorithm Debias the output
  • 43. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Data bias Activity bias Sampling bias Interaction bias (Self) selection bias Second-order bias Vicious Cycle of Bias Search or Recommender System
  • 44. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost [Baeza-Yates, Pereira & Ziviani, WWW 2008] Person Web content is redundant (> 20%) Query Ranking bias in new content Redundancy grows (35%) Search results New page Second Order Bias in Web Content [Fortunato, Flammini, Menczer & Vespignani. PNAS 2006]
  • 45. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Data bias Activity bias Sampling bias Interaction bias (Self) selection bias Vicious Cycle of Bias Search or Recommender System Feedback loop bias
  • 46. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Bias due to Personalization • The effect of self-selection bias • Avoid the rich get richer and poor get poorer syndrome • Avoid the echo chamber by empowering the tail Cold start problem solution: Explore & Exploit Partial solutions: • Diversity • Novelty • Serendipity • My dark side Wikipedia [Eli Pariser, Penguin 2011]
  • 47. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Users’ Eco Chambers in Recommender Systems ▪ Filter bubbles ▪ Degenerate feedback loops (e.g., YouTube autoplay) [Jiang et al., AAAI 2019]
  • 48. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Eco Chamber of the Recommender System • Short-term greedy optimization, partial knowledge of the world • Long-term revenue optimization is not achieved • Views from new users should balance the exploration for new items • Disparate impact: unfair ecommerce/information markets [Baeza-Yates & Ribaudo, to appear]
  • 49. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Epilogue
  • 50. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Recap Bias Type Statistical Cultural Cognitive Algorithmic  ? ? Presentation  Position    Data   Sampling    Activity  Self-selection   Interaction   Social   Second order   
  • 51. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost [Silberzahn et al., COS, Univ. of Virginia, 2015] Professional Bias? ➔ 61 analysts, 29 teams: 20 yes and 9 no
  • 52. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost If Systems Reflects Its Designers: What we can/should do? ▪ Data ▪ Analyze for known and unknown biases, debias when possible/needed ▪ Recollect more data for difficult/sparse regions of the problem ▪ Delete attributes associated directly/indirectly with harmful bias ▪ Interaction ▪ Make sure that the user is aware of the biases all the time ▪ Give more control to the user ▪ Design and Implementation • Let experts/colleagues/users contest every step of the process ▪ Evaluation • Do not fool yourself!
  • 53. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Final Take-Home Message ▪ Systems are a mirror of us, the good, the bad and the ugly ▪ The Web amplifies everything, but always leaves traces ▪ We need to be aware of our own biases! ▪ We must be aware of the biases and contrarrest them to stop the vicious bias cycle ▪ Plenty of open (research) problems! Big Data of People is huge….. ….. but it is tiny compared to the future Big Data of the Internet of Things (IoT) No activity bias!
  • 54. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Thank You! Any Questions? – rbaeza@acm.org | http://www.baeza.cl/ | http://fairness-measures.org Biased Questions? ASIST 2012 Book of the Year Award (Biased Ad)
  • 55. Ricardo Baeza-Yates © | @polarbearby | #TechSEOBoost Thanks for Viewing the Slideshare! – Watch the Recording: https://youtube.com/session-example Or Contact us today to discover how Catalyst can deliver unparalleled SEO results for your business. https://www.catalystdigital.com/