SlideShare a Scribd company logo
1 of 33
Download to read offline
Doing Data the NFLX way
Satya Kunapuli
July 2017
“Half the money I spend on advertising is
wasted; trouble is I don’t know which
half…”
John Wanamaker
Father of modern advertising and pioneer in modern marketing
https://www.linkedin.com/in/satya-kunapuli-159b161/
@satya_kunapuli
satyakunapuli
countries
190
100
members worldwide
45
MILLION
view hours per year
BILLION
PROGRAMMATIC
ADVERTISING
?
y = f(x)
Co-relation is NOT causation!
CPA
Click-through rates
Conversion rates
Media mix modeling
etc.
Traditional Marketing is co-relational
*Figure from Google Analytics
Method Definition
Last Touch
Conversion credit is given to the last
impression/click
Equal Attribution
Same conversion credit is given to all
the impressions the users were
exposed to
Attributions do not capture causality
• Burst & Delay tests
• Spend mix tests
The Final Frontier!
Titles?
Markets?
Channels?
Audience &
Creative?
H
U
M
A
N
S
M
A
C
H
I
N
E
S
Auction
Creative CCreative BCreative A
AdIndividual
Bid
Netflix
Advertiser
A
Advertiser
B
Advertiser
C
Bid
Bid
Bid
The future will be a
partnership between
humans and machines…
Recap
Thank you.

More Related Content

Similar to Doing Data the NFLX Way: How Netflix Uses Testing and Machine Learning for Programmatic Advertising

Programmatic Advertising for Beginners
Programmatic Advertising for BeginnersProgrammatic Advertising for Beginners
Programmatic Advertising for BeginnersSyed M Salman
 
Ask an Expert: How to Measure the Effectiveness of Your branding Campaigns
Ask an Expert: How to Measure the Effectiveness of Your branding CampaignsAsk an Expert: How to Measure the Effectiveness of Your branding Campaigns
Ask an Expert: How to Measure the Effectiveness of Your branding CampaignsTinuiti
 
Evaluating True Value Vs Hype In Bto B Media
Evaluating True Value Vs Hype In Bto B MediaEvaluating True Value Vs Hype In Bto B Media
Evaluating True Value Vs Hype In Bto B Mediaelanorafagan
 
Analyzing and Enhancing Your Social Engagement
Analyzing and Enhancing Your Social EngagementAnalyzing and Enhancing Your Social Engagement
Analyzing and Enhancing Your Social Engagementahorsepubs
 
5 pillars of the Infinite Marketer
5 pillars of the Infinite Marketer 5 pillars of the Infinite Marketer
5 pillars of the Infinite Marketer Mathew Sweezey
 
A look at client-agency relationships.
A look at client-agency relationships. A look at client-agency relationships.
A look at client-agency relationships. Mark Linder
 
The evolution of client-agency relationships
The evolution of client-agency relationshipsThe evolution of client-agency relationships
The evolution of client-agency relationshipsMark Linder
 
Digital Dealer 23 - Settling the Traditional v Digital Debate
Digital Dealer 23 - Settling the Traditional v Digital DebateDigital Dealer 23 - Settling the Traditional v Digital Debate
Digital Dealer 23 - Settling the Traditional v Digital DebateDave Spannhake
 
Basics Of Online Advertising
Basics Of Online AdvertisingBasics Of Online Advertising
Basics Of Online AdvertisingArpit Dhariwal
 
Digital Dealer 22 - Building a Budget Around Today's Now Buyer - 2017-04
Digital Dealer 22 - Building a Budget Around Today's Now Buyer - 2017-04Digital Dealer 22 - Building a Budget Around Today's Now Buyer - 2017-04
Digital Dealer 22 - Building a Budget Around Today's Now Buyer - 2017-04Dave Spannhake
 
Invite media playbook report
Invite media playbook reportInvite media playbook report
Invite media playbook reportAdCMO
 
Invite media playbook
Invite media playbookInvite media playbook
Invite media playbookAdCMO
 
Digital marketing strategy playbook
Digital marketing strategy playbookDigital marketing strategy playbook
Digital marketing strategy playbookAdCMO
 
Ms inspire odr02_digital-marketing-growth
Ms inspire odr02_digital-marketing-growthMs inspire odr02_digital-marketing-growth
Ms inspire odr02_digital-marketing-growthChristi Olson
 
HatlingFlint KNOW Series
HatlingFlint KNOW SeriesHatlingFlint KNOW Series
HatlingFlint KNOW SeriesFlint Group
 
Real estate lead generation how to build a pipeline using social media
Real estate lead generation how to build a pipeline using social mediaReal estate lead generation how to build a pipeline using social media
Real estate lead generation how to build a pipeline using social mediaThe Big Unit
 
MetricsBeyondTheClick_BestPractices
MetricsBeyondTheClick_BestPracticesMetricsBeyondTheClick_BestPractices
MetricsBeyondTheClick_BestPracticesChris Rash
 

Similar to Doing Data the NFLX Way: How Netflix Uses Testing and Machine Learning for Programmatic Advertising (20)

Programmatic Advertising for Beginners
Programmatic Advertising for BeginnersProgrammatic Advertising for Beginners
Programmatic Advertising for Beginners
 
Ask an Expert: How to Measure the Effectiveness of Your branding Campaigns
Ask an Expert: How to Measure the Effectiveness of Your branding CampaignsAsk an Expert: How to Measure the Effectiveness of Your branding Campaigns
Ask an Expert: How to Measure the Effectiveness of Your branding Campaigns
 
Evaluating True Value Vs Hype In Bto B Media
Evaluating True Value Vs Hype In Bto B MediaEvaluating True Value Vs Hype In Bto B Media
Evaluating True Value Vs Hype In Bto B Media
 
Analyzing and Enhancing Your Social Engagement
Analyzing and Enhancing Your Social EngagementAnalyzing and Enhancing Your Social Engagement
Analyzing and Enhancing Your Social Engagement
 
5 pillars of the Infinite Marketer
5 pillars of the Infinite Marketer 5 pillars of the Infinite Marketer
5 pillars of the Infinite Marketer
 
Maloney-3P Track
Maloney-3P TrackMaloney-3P Track
Maloney-3P Track
 
A look at client-agency relationships.
A look at client-agency relationships. A look at client-agency relationships.
A look at client-agency relationships.
 
The evolution of client-agency relationships
The evolution of client-agency relationshipsThe evolution of client-agency relationships
The evolution of client-agency relationships
 
Social Banking
Social BankingSocial Banking
Social Banking
 
Digital Dealer 23 - Settling the Traditional v Digital Debate
Digital Dealer 23 - Settling the Traditional v Digital DebateDigital Dealer 23 - Settling the Traditional v Digital Debate
Digital Dealer 23 - Settling the Traditional v Digital Debate
 
Basics Of Online Advertising
Basics Of Online AdvertisingBasics Of Online Advertising
Basics Of Online Advertising
 
Digital Dealer 22 - Building a Budget Around Today's Now Buyer - 2017-04
Digital Dealer 22 - Building a Budget Around Today's Now Buyer - 2017-04Digital Dealer 22 - Building a Budget Around Today's Now Buyer - 2017-04
Digital Dealer 22 - Building a Budget Around Today's Now Buyer - 2017-04
 
Invite media playbook report
Invite media playbook reportInvite media playbook report
Invite media playbook report
 
Invite media playbook
Invite media playbookInvite media playbook
Invite media playbook
 
Digital marketing strategy playbook
Digital marketing strategy playbookDigital marketing strategy playbook
Digital marketing strategy playbook
 
Ms inspire odr02_digital-marketing-growth
Ms inspire odr02_digital-marketing-growthMs inspire odr02_digital-marketing-growth
Ms inspire odr02_digital-marketing-growth
 
HatlingFlint KNOW Series
HatlingFlint KNOW SeriesHatlingFlint KNOW Series
HatlingFlint KNOW Series
 
Real estate lead generation how to build a pipeline using social media
Real estate lead generation how to build a pipeline using social mediaReal estate lead generation how to build a pipeline using social media
Real estate lead generation how to build a pipeline using social media
 
MetricsBeyondTheClick_BestPractices
MetricsBeyondTheClick_BestPracticesMetricsBeyondTheClick_BestPractices
MetricsBeyondTheClick_BestPractices
 
MnSearch Summit 2018 - Purna Virji – From Search to Audiences: 3 Powerful St...
MnSearch Summit 2018 - Purna Virji –  From Search to Audiences: 3 Powerful St...MnSearch Summit 2018 - Purna Virji –  From Search to Audiences: 3 Powerful St...
MnSearch Summit 2018 - Purna Virji – From Search to Audiences: 3 Powerful St...
 

Recently uploaded

代办国外大学文凭《原版美国UCLA文凭证书》加州大学洛杉矶分校毕业证制作成绩单修改
代办国外大学文凭《原版美国UCLA文凭证书》加州大学洛杉矶分校毕业证制作成绩单修改代办国外大学文凭《原版美国UCLA文凭证书》加州大学洛杉矶分校毕业证制作成绩单修改
代办国外大学文凭《原版美国UCLA文凭证书》加州大学洛杉矶分校毕业证制作成绩单修改atducpo
 
How we prevented account sharing with MFA
How we prevented account sharing with MFAHow we prevented account sharing with MFA
How we prevented account sharing with MFAAndrei Kaleshka
 
Industrialised data - the key to AI success.pdf
Industrialised data - the key to AI success.pdfIndustrialised data - the key to AI success.pdf
Industrialised data - the key to AI success.pdfLars Albertsson
 
Schema on read is obsolete. Welcome metaprogramming..pdf
Schema on read is obsolete. Welcome metaprogramming..pdfSchema on read is obsolete. Welcome metaprogramming..pdf
Schema on read is obsolete. Welcome metaprogramming..pdfLars Albertsson
 
VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...
VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...
VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...Suhani Kapoor
 
Call Girls In Dwarka 9654467111 Escorts Service
Call Girls In Dwarka 9654467111 Escorts ServiceCall Girls In Dwarka 9654467111 Escorts Service
Call Girls In Dwarka 9654467111 Escorts ServiceSapana Sha
 
9654467111 Call Girls In Munirka Hotel And Home Service
9654467111 Call Girls In Munirka Hotel And Home Service9654467111 Call Girls In Munirka Hotel And Home Service
9654467111 Call Girls In Munirka Hotel And Home ServiceSapana Sha
 
Dubai Call Girls Wifey O52&786472 Call Girls Dubai
Dubai Call Girls Wifey O52&786472 Call Girls DubaiDubai Call Girls Wifey O52&786472 Call Girls Dubai
Dubai Call Girls Wifey O52&786472 Call Girls Dubaihf8803863
 
꧁❤ Greater Noida Call Girls Delhi ❤꧂ 9711199171 ☎️ Hard And Sexy Vip Call
꧁❤ Greater Noida Call Girls Delhi ❤꧂ 9711199171 ☎️ Hard And Sexy Vip Call꧁❤ Greater Noida Call Girls Delhi ❤꧂ 9711199171 ☎️ Hard And Sexy Vip Call
꧁❤ Greater Noida Call Girls Delhi ❤꧂ 9711199171 ☎️ Hard And Sexy Vip Callshivangimorya083
 
Kantar AI Summit- Under Embargo till Wednesday, 24th April 2024, 4 PM, IST.pdf
Kantar AI Summit- Under Embargo till Wednesday, 24th April 2024, 4 PM, IST.pdfKantar AI Summit- Under Embargo till Wednesday, 24th April 2024, 4 PM, IST.pdf
Kantar AI Summit- Under Embargo till Wednesday, 24th April 2024, 4 PM, IST.pdfSocial Samosa
 
04242024_CCC TUG_Joins and Relationships
04242024_CCC TUG_Joins and Relationships04242024_CCC TUG_Joins and Relationships
04242024_CCC TUG_Joins and Relationshipsccctableauusergroup
 
VIP Call Girls Service Miyapur Hyderabad Call +91-8250192130
VIP Call Girls Service Miyapur Hyderabad Call +91-8250192130VIP Call Girls Service Miyapur Hyderabad Call +91-8250192130
VIP Call Girls Service Miyapur Hyderabad Call +91-8250192130Suhani Kapoor
 
办理(Vancouver毕业证书)加拿大温哥华岛大学毕业证成绩单原版一比一
办理(Vancouver毕业证书)加拿大温哥华岛大学毕业证成绩单原版一比一办理(Vancouver毕业证书)加拿大温哥华岛大学毕业证成绩单原版一比一
办理(Vancouver毕业证书)加拿大温哥华岛大学毕业证成绩单原版一比一F La
 
办理学位证纽约大学毕业证(NYU毕业证书)原版一比一
办理学位证纽约大学毕业证(NYU毕业证书)原版一比一办理学位证纽约大学毕业证(NYU毕业证书)原版一比一
办理学位证纽约大学毕业证(NYU毕业证书)原版一比一fhwihughh
 
dokumen.tips_chapter-4-transient-heat-conduction-mehmet-kanoglu.ppt
dokumen.tips_chapter-4-transient-heat-conduction-mehmet-kanoglu.pptdokumen.tips_chapter-4-transient-heat-conduction-mehmet-kanoglu.ppt
dokumen.tips_chapter-4-transient-heat-conduction-mehmet-kanoglu.pptSonatrach
 
Saket, (-DELHI )+91-9654467111-(=)CHEAP Call Girls in Escorts Service Saket C...
Saket, (-DELHI )+91-9654467111-(=)CHEAP Call Girls in Escorts Service Saket C...Saket, (-DELHI )+91-9654467111-(=)CHEAP Call Girls in Escorts Service Saket C...
Saket, (-DELHI )+91-9654467111-(=)CHEAP Call Girls in Escorts Service Saket C...Sapana Sha
 
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptx
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM  TRACKING WITH GOOGLE ANALYTICS.pptxEMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM  TRACKING WITH GOOGLE ANALYTICS.pptx
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptxthyngster
 

Recently uploaded (20)

代办国外大学文凭《原版美国UCLA文凭证书》加州大学洛杉矶分校毕业证制作成绩单修改
代办国外大学文凭《原版美国UCLA文凭证书》加州大学洛杉矶分校毕业证制作成绩单修改代办国外大学文凭《原版美国UCLA文凭证书》加州大学洛杉矶分校毕业证制作成绩单修改
代办国外大学文凭《原版美国UCLA文凭证书》加州大学洛杉矶分校毕业证制作成绩单修改
 
How we prevented account sharing with MFA
How we prevented account sharing with MFAHow we prevented account sharing with MFA
How we prevented account sharing with MFA
 
Industrialised data - the key to AI success.pdf
Industrialised data - the key to AI success.pdfIndustrialised data - the key to AI success.pdf
Industrialised data - the key to AI success.pdf
 
VIP Call Girls Service Charbagh { Lucknow Call Girls Service 9548273370 } Boo...
VIP Call Girls Service Charbagh { Lucknow Call Girls Service 9548273370 } Boo...VIP Call Girls Service Charbagh { Lucknow Call Girls Service 9548273370 } Boo...
VIP Call Girls Service Charbagh { Lucknow Call Girls Service 9548273370 } Boo...
 
Schema on read is obsolete. Welcome metaprogramming..pdf
Schema on read is obsolete. Welcome metaprogramming..pdfSchema on read is obsolete. Welcome metaprogramming..pdf
Schema on read is obsolete. Welcome metaprogramming..pdf
 
VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...
VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...
VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...
 
Call Girls In Dwarka 9654467111 Escorts Service
Call Girls In Dwarka 9654467111 Escorts ServiceCall Girls In Dwarka 9654467111 Escorts Service
Call Girls In Dwarka 9654467111 Escorts Service
 
9654467111 Call Girls In Munirka Hotel And Home Service
9654467111 Call Girls In Munirka Hotel And Home Service9654467111 Call Girls In Munirka Hotel And Home Service
9654467111 Call Girls In Munirka Hotel And Home Service
 
Call Girls in Saket 99530🔝 56974 Escort Service
Call Girls in Saket 99530🔝 56974 Escort ServiceCall Girls in Saket 99530🔝 56974 Escort Service
Call Girls in Saket 99530🔝 56974 Escort Service
 
E-Commerce Order PredictionShraddha Kamble.pptx
E-Commerce Order PredictionShraddha Kamble.pptxE-Commerce Order PredictionShraddha Kamble.pptx
E-Commerce Order PredictionShraddha Kamble.pptx
 
Dubai Call Girls Wifey O52&786472 Call Girls Dubai
Dubai Call Girls Wifey O52&786472 Call Girls DubaiDubai Call Girls Wifey O52&786472 Call Girls Dubai
Dubai Call Girls Wifey O52&786472 Call Girls Dubai
 
꧁❤ Greater Noida Call Girls Delhi ❤꧂ 9711199171 ☎️ Hard And Sexy Vip Call
꧁❤ Greater Noida Call Girls Delhi ❤꧂ 9711199171 ☎️ Hard And Sexy Vip Call꧁❤ Greater Noida Call Girls Delhi ❤꧂ 9711199171 ☎️ Hard And Sexy Vip Call
꧁❤ Greater Noida Call Girls Delhi ❤꧂ 9711199171 ☎️ Hard And Sexy Vip Call
 
Kantar AI Summit- Under Embargo till Wednesday, 24th April 2024, 4 PM, IST.pdf
Kantar AI Summit- Under Embargo till Wednesday, 24th April 2024, 4 PM, IST.pdfKantar AI Summit- Under Embargo till Wednesday, 24th April 2024, 4 PM, IST.pdf
Kantar AI Summit- Under Embargo till Wednesday, 24th April 2024, 4 PM, IST.pdf
 
04242024_CCC TUG_Joins and Relationships
04242024_CCC TUG_Joins and Relationships04242024_CCC TUG_Joins and Relationships
04242024_CCC TUG_Joins and Relationships
 
VIP Call Girls Service Miyapur Hyderabad Call +91-8250192130
VIP Call Girls Service Miyapur Hyderabad Call +91-8250192130VIP Call Girls Service Miyapur Hyderabad Call +91-8250192130
VIP Call Girls Service Miyapur Hyderabad Call +91-8250192130
 
办理(Vancouver毕业证书)加拿大温哥华岛大学毕业证成绩单原版一比一
办理(Vancouver毕业证书)加拿大温哥华岛大学毕业证成绩单原版一比一办理(Vancouver毕业证书)加拿大温哥华岛大学毕业证成绩单原版一比一
办理(Vancouver毕业证书)加拿大温哥华岛大学毕业证成绩单原版一比一
 
办理学位证纽约大学毕业证(NYU毕业证书)原版一比一
办理学位证纽约大学毕业证(NYU毕业证书)原版一比一办理学位证纽约大学毕业证(NYU毕业证书)原版一比一
办理学位证纽约大学毕业证(NYU毕业证书)原版一比一
 
dokumen.tips_chapter-4-transient-heat-conduction-mehmet-kanoglu.ppt
dokumen.tips_chapter-4-transient-heat-conduction-mehmet-kanoglu.pptdokumen.tips_chapter-4-transient-heat-conduction-mehmet-kanoglu.ppt
dokumen.tips_chapter-4-transient-heat-conduction-mehmet-kanoglu.ppt
 
Saket, (-DELHI )+91-9654467111-(=)CHEAP Call Girls in Escorts Service Saket C...
Saket, (-DELHI )+91-9654467111-(=)CHEAP Call Girls in Escorts Service Saket C...Saket, (-DELHI )+91-9654467111-(=)CHEAP Call Girls in Escorts Service Saket C...
Saket, (-DELHI )+91-9654467111-(=)CHEAP Call Girls in Escorts Service Saket C...
 
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptx
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM  TRACKING WITH GOOGLE ANALYTICS.pptxEMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM  TRACKING WITH GOOGLE ANALYTICS.pptx
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptx
 

Doing Data the NFLX Way: How Netflix Uses Testing and Machine Learning for Programmatic Advertising