SlideShare a Scribd company logo
1 of 12
The Ethics Of A/B Testing
What is A/B Testing
 Also named randomized controlled experiments, online controlled
experiments, and split testing.
 In marketing, A/B testing is a simple randomized experiment with two
variants, A and B.
 It sees if different versions of website with little changes make us use
more, visit more, click more or buy more.
1
Example
 This example for a website that make two version with little changes
2
Facebook feeling A/B test
 PNAS magazine published results of an experiment on 689,003 users to
see if showing them more positive or negative sentiment posts in the
News Feed would affect their happiness levels as deduced by what they
posted.
3
The Experiment Had Almost No Effect
 Lets take a look on this experiment.
 Experiment duration : 1 Week
From January 11–18, 2012
 People shown fewer positive posts (a more
depressing feed) posted 0.1% fewer positive
words in their posts.
 People shown fewer negative posts (a happier
feed) post 0.07% fewer negative words.
4
Experiment risk and fails
 the National Institute Of Mental Health says 9.5% of Americans have mood
disorders, which can often lead to depression.
 Some people who are at risk of depression were almost surely part of
Facebook’s study group.
 Status update isn’t the real emotion, may this people just following the
trends they see on Facebook.
 Another research says that some people feel sad when they see lots of
happy status, because they feels that them life isn’t happy as they see on
Facebook.
6
Facebook Didn’t Get Ethics Board
Approval
 Facebook only did an internal review to decide if the study was ethical.
 the study would likely fail to meet many of the pre-requisites.
 But actually Facebook manipulates the News Feed all the time to test what
types of stories and designs makes you spend time, money like ,comment
or generally more active.
 Facebook technically has consent from all users, as its Data Use Policy
people automatically agree to when they sign up says
“we may use the information we receive about you…for…data
analysis, testing, research and service improvement.”
5
Everyone Is A/B Testing
 there are lots of A/B Tests but most are pushing for more business-oriented
results like increasing usage or clicks or purchases.
 A lot of times we don’t use the “real” Facebook. Or Twitter. Or Google, Yahoo,
or LinkedIn. We are almost all part of experiments they quietly.
7
Regulation, Or At Least Safeguards
 Offering an opt out Not for every test, but ones with some real risk like
showing users a more depressing feed.
8
Regulation, Or At Least Safeguards
(cont.)
 Independent protection against harmful emotional manipulation on a
grand scale.
 At the very least, the tech companies should educate their data scientists
and others designing A/B tests about the ethical research methods
associated with having experiments approved by the IRB
9
Till this happen
 Till this regulations happen, all of us are Laboratory rats
10
Thank you :D
11

More Related Content

Similar to The ethics of A/B test

11 Ways To Leverage Crowdvibe
11 Ways To Leverage Crowdvibe11 Ways To Leverage Crowdvibe
11 Ways To Leverage Crowdvibe
Peanut Labs
 
Assignment 1 ExamplesHere are a few examples of submissions for .docx
Assignment 1 ExamplesHere are a few examples of submissions for .docxAssignment 1 ExamplesHere are a few examples of submissions for .docx
Assignment 1 ExamplesHere are a few examples of submissions for .docx
cargillfilberto
 
Pp chapter 02 reading the news revised
Pp chapter 02 reading the news revisedPp chapter 02 reading the news revised
Pp chapter 02 reading the news revised
Michael Hill
 
Basics of the Proposal EssayOverview In the proposa.docx
Basics of the Proposal EssayOverview In the proposa.docxBasics of the Proposal EssayOverview In the proposa.docx
Basics of the Proposal EssayOverview In the proposa.docx
JASS44
 
Running Head LASA 1 Final Project Early Methods Section .docx
Running Head LASA 1 Final Project Early Methods Section        .docxRunning Head LASA 1 Final Project Early Methods Section        .docx
Running Head LASA 1 Final Project Early Methods Section .docx
charisellington63520
 
WRITERSocial networking dilemma arises when the society is una.docx
WRITERSocial networking dilemma arises when the society is una.docxWRITERSocial networking dilemma arises when the society is una.docx
WRITERSocial networking dilemma arises when the society is una.docx
ambersalomon88660
 

Similar to The ethics of A/B test (20)

11 Ways To Leverage Crowdvibe
11 Ways To Leverage Crowdvibe11 Ways To Leverage Crowdvibe
11 Ways To Leverage Crowdvibe
 
Creative Media BTEC- "Unit 3- Research Techniques"
Creative Media BTEC- "Unit 3- Research Techniques"Creative Media BTEC- "Unit 3- Research Techniques"
Creative Media BTEC- "Unit 3- Research Techniques"
 
Building Wellness Interventions Into Facebook
Building Wellness Interventions Into Facebook Building Wellness Interventions Into Facebook
Building Wellness Interventions Into Facebook
 
Analytical Essay - 6+ Examples, For. Online assignment writing service.
Analytical Essay - 6+ Examples, For. Online assignment writing service.Analytical Essay - 6+ Examples, For. Online assignment writing service.
Analytical Essay - 6+ Examples, For. Online assignment writing service.
 
Critical Thinking 2
Critical Thinking 2Critical Thinking 2
Critical Thinking 2
 
Why does Amazon knows me better than my doctor?
Why does Amazon knows me better than my doctor? Why does Amazon knows me better than my doctor?
Why does Amazon knows me better than my doctor?
 
Upworthy & Bill and Melinda Gates Foundation - All 7 Billion - Research Summary
Upworthy & Bill and Melinda Gates Foundation - All 7 Billion - Research SummaryUpworthy & Bill and Melinda Gates Foundation - All 7 Billion - Research Summary
Upworthy & Bill and Melinda Gates Foundation - All 7 Billion - Research Summary
 
Assignment 1 ExamplesHere are a few examples of submissions for .docx
Assignment 1 ExamplesHere are a few examples of submissions for .docxAssignment 1 ExamplesHere are a few examples of submissions for .docx
Assignment 1 ExamplesHere are a few examples of submissions for .docx
 
Pp chapter 02 reading the news revised
Pp chapter 02 reading the news revisedPp chapter 02 reading the news revised
Pp chapter 02 reading the news revised
 
Mmc5427 elsie puig message testing presentation
Mmc5427 elsie puig message testing presentationMmc5427 elsie puig message testing presentation
Mmc5427 elsie puig message testing presentation
 
Marketing to Baby Boomers
Marketing to Baby BoomersMarketing to Baby Boomers
Marketing to Baby Boomers
 
Social Media for Veterinarians - EBMS International
Social Media for Veterinarians - EBMS InternationalSocial Media for Veterinarians - EBMS International
Social Media for Veterinarians - EBMS International
 
COMPUTER APPLICATIONS IN PHARMACY LAB MANUAL.pdf
COMPUTER APPLICATIONS IN PHARMACY LAB MANUAL.pdfCOMPUTER APPLICATIONS IN PHARMACY LAB MANUAL.pdf
COMPUTER APPLICATIONS IN PHARMACY LAB MANUAL.pdf
 
Basics of the Proposal EssayOverview In the proposa.docx
Basics of the Proposal EssayOverview In the proposa.docxBasics of the Proposal EssayOverview In the proposa.docx
Basics of the Proposal EssayOverview In the proposa.docx
 
TalkonHowToDesignQuestionnaireDesign.pptx
TalkonHowToDesignQuestionnaireDesign.pptxTalkonHowToDesignQuestionnaireDesign.pptx
TalkonHowToDesignQuestionnaireDesign.pptx
 
Questionnaire analysis
Questionnaire analysisQuestionnaire analysis
Questionnaire analysis
 
Running Head LASA 1 Final Project Early Methods Section .docx
Running Head LASA 1 Final Project Early Methods Section        .docxRunning Head LASA 1 Final Project Early Methods Section        .docx
Running Head LASA 1 Final Project Early Methods Section .docx
 
WRITERSocial networking dilemma arises when the society is una.docx
WRITERSocial networking dilemma arises when the society is una.docxWRITERSocial networking dilemma arises when the society is una.docx
WRITERSocial networking dilemma arises when the society is una.docx
 
Social Media Analysis for Government Communicators
Social Media Analysis for Government CommunicatorsSocial Media Analysis for Government Communicators
Social Media Analysis for Government Communicators
 
Tips Pre Test Post Test
Tips   Pre Test Post TestTips   Pre Test Post Test
Tips Pre Test Post Test
 

Recently uploaded

Call Girls In Model Towh Delhi 💯Call Us 🔝8264348440🔝
Call Girls In Model Towh Delhi 💯Call Us 🔝8264348440🔝Call Girls In Model Towh Delhi 💯Call Us 🔝8264348440🔝
Call Girls In Model Towh Delhi 💯Call Us 🔝8264348440🔝
soniya singh
 
Low Rate Young Call Girls in Sector 63 Mamura Noida ✔️☆9289244007✔️☆ Female E...
Low Rate Young Call Girls in Sector 63 Mamura Noida ✔️☆9289244007✔️☆ Female E...Low Rate Young Call Girls in Sector 63 Mamura Noida ✔️☆9289244007✔️☆ Female E...
Low Rate Young Call Girls in Sector 63 Mamura Noida ✔️☆9289244007✔️☆ Female E...
SofiyaSharma5
 
Call Girls Service Chandigarh Lucky ❤️ 7710465962 Independent Call Girls In C...
Call Girls Service Chandigarh Lucky ❤️ 7710465962 Independent Call Girls In C...Call Girls Service Chandigarh Lucky ❤️ 7710465962 Independent Call Girls In C...
Call Girls Service Chandigarh Lucky ❤️ 7710465962 Independent Call Girls In C...
Sheetaleventcompany
 
CALL ON ➥8923113531 🔝Call Girls Lucknow Lucknow best sexual service Online
CALL ON ➥8923113531 🔝Call Girls Lucknow Lucknow best sexual service OnlineCALL ON ➥8923113531 🔝Call Girls Lucknow Lucknow best sexual service Online
CALL ON ➥8923113531 🔝Call Girls Lucknow Lucknow best sexual service Online
anilsa9823
 
Lucknow ❤CALL GIRL 88759*99948 ❤CALL GIRLS IN Lucknow ESCORT SERVICE❤CALL GIRL
Lucknow ❤CALL GIRL 88759*99948 ❤CALL GIRLS IN Lucknow ESCORT SERVICE❤CALL GIRLLucknow ❤CALL GIRL 88759*99948 ❤CALL GIRLS IN Lucknow ESCORT SERVICE❤CALL GIRL
Lucknow ❤CALL GIRL 88759*99948 ❤CALL GIRLS IN Lucknow ESCORT SERVICE❤CALL GIRL
imonikaupta
 
Delhi Call Girls Rohini 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip Call
Delhi Call Girls Rohini 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip CallDelhi Call Girls Rohini 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip Call
Delhi Call Girls Rohini 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip Call
shivangimorya083
 
₹5.5k {Cash Payment}New Friends Colony Call Girls In [Delhi NIHARIKA] 🔝|97111...
₹5.5k {Cash Payment}New Friends Colony Call Girls In [Delhi NIHARIKA] 🔝|97111...₹5.5k {Cash Payment}New Friends Colony Call Girls In [Delhi NIHARIKA] 🔝|97111...
₹5.5k {Cash Payment}New Friends Colony Call Girls In [Delhi NIHARIKA] 🔝|97111...
Diya Sharma
 

Recently uploaded (20)

INDIVIDUAL ASSIGNMENT #3 CBG, PRESENTATION.
INDIVIDUAL ASSIGNMENT #3 CBG, PRESENTATION.INDIVIDUAL ASSIGNMENT #3 CBG, PRESENTATION.
INDIVIDUAL ASSIGNMENT #3 CBG, PRESENTATION.
 
VIP 7001035870 Find & Meet Hyderabad Call Girls LB Nagar high-profile Call Girl
VIP 7001035870 Find & Meet Hyderabad Call Girls LB Nagar high-profile Call GirlVIP 7001035870 Find & Meet Hyderabad Call Girls LB Nagar high-profile Call Girl
VIP 7001035870 Find & Meet Hyderabad Call Girls LB Nagar high-profile Call Girl
 
Enjoy Night⚡Call Girls Dlf City Phase 3 Gurgaon >༒8448380779 Escort Service
Enjoy Night⚡Call Girls Dlf City Phase 3 Gurgaon >༒8448380779 Escort ServiceEnjoy Night⚡Call Girls Dlf City Phase 3 Gurgaon >༒8448380779 Escort Service
Enjoy Night⚡Call Girls Dlf City Phase 3 Gurgaon >༒8448380779 Escort Service
 
Call Girls In Model Towh Delhi 💯Call Us 🔝8264348440🔝
Call Girls In Model Towh Delhi 💯Call Us 🔝8264348440🔝Call Girls In Model Towh Delhi 💯Call Us 🔝8264348440🔝
Call Girls In Model Towh Delhi 💯Call Us 🔝8264348440🔝
 
@9999965857 🫦 Sexy Desi Call Girls Laxmi Nagar 💓 High Profile Escorts Delhi 🫶
@9999965857 🫦 Sexy Desi Call Girls Laxmi Nagar 💓 High Profile Escorts Delhi 🫶@9999965857 🫦 Sexy Desi Call Girls Laxmi Nagar 💓 High Profile Escorts Delhi 🫶
@9999965857 🫦 Sexy Desi Call Girls Laxmi Nagar 💓 High Profile Escorts Delhi 🫶
 
Low Rate Young Call Girls in Sector 63 Mamura Noida ✔️☆9289244007✔️☆ Female E...
Low Rate Young Call Girls in Sector 63 Mamura Noida ✔️☆9289244007✔️☆ Female E...Low Rate Young Call Girls in Sector 63 Mamura Noida ✔️☆9289244007✔️☆ Female E...
Low Rate Young Call Girls in Sector 63 Mamura Noida ✔️☆9289244007✔️☆ Female E...
 
Call Girls Service Chandigarh Lucky ❤️ 7710465962 Independent Call Girls In C...
Call Girls Service Chandigarh Lucky ❤️ 7710465962 Independent Call Girls In C...Call Girls Service Chandigarh Lucky ❤️ 7710465962 Independent Call Girls In C...
Call Girls Service Chandigarh Lucky ❤️ 7710465962 Independent Call Girls In C...
 
CALL ON ➥8923113531 🔝Call Girls Lucknow Lucknow best sexual service Online
CALL ON ➥8923113531 🔝Call Girls Lucknow Lucknow best sexual service OnlineCALL ON ➥8923113531 🔝Call Girls Lucknow Lucknow best sexual service Online
CALL ON ➥8923113531 🔝Call Girls Lucknow Lucknow best sexual service Online
 
Call Girls Ludhiana Just Call 98765-12871 Top Class Call Girl Service Available
Call Girls Ludhiana Just Call 98765-12871 Top Class Call Girl Service AvailableCall Girls Ludhiana Just Call 98765-12871 Top Class Call Girl Service Available
Call Girls Ludhiana Just Call 98765-12871 Top Class Call Girl Service Available
 
Nanded City ( Call Girls ) Pune 6297143586 Hot Model With Sexy Bhabi Ready ...
Nanded City ( Call Girls ) Pune  6297143586  Hot Model With Sexy Bhabi Ready ...Nanded City ( Call Girls ) Pune  6297143586  Hot Model With Sexy Bhabi Ready ...
Nanded City ( Call Girls ) Pune 6297143586 Hot Model With Sexy Bhabi Ready ...
 
Pune Airport ( Call Girls ) Pune 6297143586 Hot Model With Sexy Bhabi Ready...
Pune Airport ( Call Girls ) Pune  6297143586  Hot Model With Sexy Bhabi Ready...Pune Airport ( Call Girls ) Pune  6297143586  Hot Model With Sexy Bhabi Ready...
Pune Airport ( Call Girls ) Pune 6297143586 Hot Model With Sexy Bhabi Ready...
 
VIP Model Call Girls Hadapsar ( Pune ) Call ON 9905417584 Starting High Prof...
VIP Model Call Girls Hadapsar ( Pune ) Call ON 9905417584 Starting  High Prof...VIP Model Call Girls Hadapsar ( Pune ) Call ON 9905417584 Starting  High Prof...
VIP Model Call Girls Hadapsar ( Pune ) Call ON 9905417584 Starting High Prof...
 
Lucknow ❤CALL GIRL 88759*99948 ❤CALL GIRLS IN Lucknow ESCORT SERVICE❤CALL GIRL
Lucknow ❤CALL GIRL 88759*99948 ❤CALL GIRLS IN Lucknow ESCORT SERVICE❤CALL GIRLLucknow ❤CALL GIRL 88759*99948 ❤CALL GIRLS IN Lucknow ESCORT SERVICE❤CALL GIRL
Lucknow ❤CALL GIRL 88759*99948 ❤CALL GIRLS IN Lucknow ESCORT SERVICE❤CALL GIRL
 
Russian Call Girls in %(+971524965298 )# Call Girls in Dubai
Russian Call Girls in %(+971524965298  )#  Call Girls in DubaiRussian Call Girls in %(+971524965298  )#  Call Girls in Dubai
Russian Call Girls in %(+971524965298 )# Call Girls in Dubai
 
Delhi Call Girls Rohini 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip Call
Delhi Call Girls Rohini 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip CallDelhi Call Girls Rohini 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip Call
Delhi Call Girls Rohini 9711199171 ☎✔👌✔ Whatsapp Hard And Sexy Vip Call
 
₹5.5k {Cash Payment}New Friends Colony Call Girls In [Delhi NIHARIKA] 🔝|97111...
₹5.5k {Cash Payment}New Friends Colony Call Girls In [Delhi NIHARIKA] 🔝|97111...₹5.5k {Cash Payment}New Friends Colony Call Girls In [Delhi NIHARIKA] 🔝|97111...
₹5.5k {Cash Payment}New Friends Colony Call Girls In [Delhi NIHARIKA] 🔝|97111...
 
Russian Call girl in Ajman +971563133746 Ajman Call girl Service
Russian Call girl in Ajman +971563133746 Ajman Call girl ServiceRussian Call girl in Ajman +971563133746 Ajman Call girl Service
Russian Call girl in Ajman +971563133746 Ajman Call girl Service
 
Call Now ☎ 8264348440 !! Call Girls in Sarai Rohilla Escort Service Delhi N.C.R.
Call Now ☎ 8264348440 !! Call Girls in Sarai Rohilla Escort Service Delhi N.C.R.Call Now ☎ 8264348440 !! Call Girls in Sarai Rohilla Escort Service Delhi N.C.R.
Call Now ☎ 8264348440 !! Call Girls in Sarai Rohilla Escort Service Delhi N.C.R.
 
Call Now ☎ 8264348440 !! Call Girls in Green Park Escort Service Delhi N.C.R.
Call Now ☎ 8264348440 !! Call Girls in Green Park Escort Service Delhi N.C.R.Call Now ☎ 8264348440 !! Call Girls in Green Park Escort Service Delhi N.C.R.
Call Now ☎ 8264348440 !! Call Girls in Green Park Escort Service Delhi N.C.R.
 
𓀤Call On 7877925207 𓀤 Ahmedguda Call Girls Hot Model With Sexy Bhabi Ready Fo...
𓀤Call On 7877925207 𓀤 Ahmedguda Call Girls Hot Model With Sexy Bhabi Ready Fo...𓀤Call On 7877925207 𓀤 Ahmedguda Call Girls Hot Model With Sexy Bhabi Ready Fo...
𓀤Call On 7877925207 𓀤 Ahmedguda Call Girls Hot Model With Sexy Bhabi Ready Fo...
 

The ethics of A/B test

  • 1. The Ethics Of A/B Testing
  • 2. What is A/B Testing  Also named randomized controlled experiments, online controlled experiments, and split testing.  In marketing, A/B testing is a simple randomized experiment with two variants, A and B.  It sees if different versions of website with little changes make us use more, visit more, click more or buy more. 1
  • 3. Example  This example for a website that make two version with little changes 2
  • 4. Facebook feeling A/B test  PNAS magazine published results of an experiment on 689,003 users to see if showing them more positive or negative sentiment posts in the News Feed would affect their happiness levels as deduced by what they posted. 3
  • 5. The Experiment Had Almost No Effect  Lets take a look on this experiment.  Experiment duration : 1 Week From January 11–18, 2012  People shown fewer positive posts (a more depressing feed) posted 0.1% fewer positive words in their posts.  People shown fewer negative posts (a happier feed) post 0.07% fewer negative words. 4
  • 6. Experiment risk and fails  the National Institute Of Mental Health says 9.5% of Americans have mood disorders, which can often lead to depression.  Some people who are at risk of depression were almost surely part of Facebook’s study group.  Status update isn’t the real emotion, may this people just following the trends they see on Facebook.  Another research says that some people feel sad when they see lots of happy status, because they feels that them life isn’t happy as they see on Facebook. 6
  • 7. Facebook Didn’t Get Ethics Board Approval  Facebook only did an internal review to decide if the study was ethical.  the study would likely fail to meet many of the pre-requisites.  But actually Facebook manipulates the News Feed all the time to test what types of stories and designs makes you spend time, money like ,comment or generally more active.  Facebook technically has consent from all users, as its Data Use Policy people automatically agree to when they sign up says “we may use the information we receive about you…for…data analysis, testing, research and service improvement.” 5
  • 8. Everyone Is A/B Testing  there are lots of A/B Tests but most are pushing for more business-oriented results like increasing usage or clicks or purchases.  A lot of times we don’t use the “real” Facebook. Or Twitter. Or Google, Yahoo, or LinkedIn. We are almost all part of experiments they quietly. 7
  • 9. Regulation, Or At Least Safeguards  Offering an opt out Not for every test, but ones with some real risk like showing users a more depressing feed. 8
  • 10. Regulation, Or At Least Safeguards (cont.)  Independent protection against harmful emotional manipulation on a grand scale.  At the very least, the tech companies should educate their data scientists and others designing A/B tests about the ethical research methods associated with having experiments approved by the IRB 9
  • 11. Till this happen  Till this regulations happen, all of us are Laboratory rats 10