SlideShare a Scribd company logo
KNOW WHAT YOU MEASURE.
AND WHY.
KIRIL TSEMEKHMAN
CHIEF DATA SCIENTIST,
INTEGRAL AD SCIENCE
1
WHAT MATTERS TO YOU?
• Measuring performance:
incrementality
• Attribution: rewarding rightly
• Driving performance: the importance
of environment
• Taking it in: influencing the user
• Revealing what matters
2
ATTRIBUTION: REWARDING RIGHTLY
• Establishing the baseline
• Measuring and attributing only incremental actions
• Giving credit where credit is really due
3
4
5
6
LESSONS LEARNED
• The treasure chest had “baseline coins”
• Pirates had to find a way to separate baseline coins from their
“incremental loot”
• Knowing each pirate’s effort leads to rewarding rightly
7
DRIVING PERFORMANCE:
THE IMPORTANCE OF ENVIRONMENT
• Is viewability all that matters?
• Other dimensions that make a difference
– How many times driver sees the billboard?
– How often?
– Is there enough time to read it?
8
9
10
LESSONS LEARNED
• Picking the right place is not that simple
• Consider what’s happening at any given moment
• How big is the opportunity to be noticed and remembered?
11
TAKING IT IN: INFLUENCING THE USER
• Opportunity does not equal viewability
• Viewability does not equal engagement
• Engagement does not equal action…yet
12
13
14
15
LESSONS LEARNED
• You are not cured by getting a prescription
• You are also not cured by filling a prescription
• You have a better chance to be cured by taking the
medication as directed
– but then again, different dosages may yield better results
16
CAUSAL IMPACT OF USER ENGAGEMENT
WITH DIGITAL ADS
• “Natural” A/B test
• Unviewable ads instead of PSAs
• Causal impact of different treatment levels
– Number of ads shown to user
– Total exposure time
17
AND NOW…REVEALING WHAT MATTERS
Measuring incremental (causal) actions:
• Does the number of exposures matter?
• Does the frequency of exposures matter?
• Does the total exposure duration matter?
• Are two long ads better than ten short ads?
18
600K
200K
1 2 3 4 5 6 7 8 9 10 11
Impressions per User
NumberofUsersOverOneWeek
7% of users were served over 5 impressions
2.5% of users were served over 10 impressions
AD FREQUENCY: MOST USERS SAW 1-2 ADS
19
700K
120K
1-5 5-15 15-30 30-45 45-60 60-75 75-90 90-120 120-150 >150
Total User Exposure Time (seconds)
NumberofUsersOverOneWeek 57% of users have no viewable impressions
USER EXPOSURE TIME: MOST EXPOSED <5 SEC
20
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0 50 100 150
Gaming Telcom
Exposure to Advertising (seconds)
AdditiveLiftinConversions
EFFECT OF TOTAL EXPOSURE TIME ON
INCREMENTAL CONVERSIONS
21
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0 2 4 6 8 10
Gaming Telecom
Number of Impressions
AdditiveLiftinConversions
EFFECT OF REPEATED EXPOSURES ON
INCREMENTAL CONVERSIONS
22
0
0.05
0.1
0.15
1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10
Number of Impressions
AdditiveLiftinConversionsCOMBINED EFFECT OF EXPOSURE
FREQUENCY & DURATION
23
LESSONS LEARNED
• Number and frequency of exposures matter
• Total exposure duration matters
• One long exposure does not quite reach the goal
24
ULTIMATE LESSONS LEARNED
• Remember what matters and ask the
right questions. Then…
• Measure the right things and measure
them right.
25
THANK YOU
KIRIL TSEMEKHMAN
kiril@integralads.com

More Related Content

Similar to Omni.Digital 2015 - "Know What You Measure. And Why."

Growth Hacking Workshop: AAU
Growth Hacking Workshop: AAUGrowth Hacking Workshop: AAU
Growth Hacking Workshop: AAU
Mark Andersen
 
0 RIVA Lean Enterprise Overview 12.11.15
0 RIVA Lean Enterprise Overview 12.11.150 RIVA Lean Enterprise Overview 12.11.15
0 RIVA Lean Enterprise Overview 12.11.15
•Joe Perillo
 
Presentations - Zarget CRO meetup 2017
Presentations - Zarget CRO meetup 2017Presentations - Zarget CRO meetup 2017
Presentations - Zarget CRO meetup 2017
ZargetHQ
 
E-learning and data: two peas in a pod
E-learning and data: two peas in a podE-learning and data: two peas in a pod
E-learning and data: two peas in a pod
Xylos
 
Ver 1.10 the venture capital ecosystem feb 2015
Ver 1.10   the venture capital ecosystem feb 2015Ver 1.10   the venture capital ecosystem feb 2015
Ver 1.10 the venture capital ecosystem feb 2015
Andrew Waitman
 
Innovation accounting and key metrics for startups
Innovation accounting and key metrics for startupsInnovation accounting and key metrics for startups
Innovation accounting and key metrics for startups
Blaz Kos
 
Design of Work Experience
Design of Work ExperienceDesign of Work Experience
Design of Work Experience
Chet Marchwinski
 
Assessing the quality of usability
Assessing the quality of usabilityAssessing the quality of usability
Assessing the quality of usability
Aditya Karle
 
A bigger view of UX
A bigger view of UXA bigger view of UX
A bigger view of UX
Eric Reiss
 
Voice of Customer Program Design
Voice of Customer Program Design Voice of Customer Program Design
Voice of Customer Program Design
Mohamad El-Hinnawi, CCXP
 
ICEM Presentation-Istanbul-September 2014_LinkedIn
ICEM Presentation-Istanbul-September 2014_LinkedInICEM Presentation-Istanbul-September 2014_LinkedIn
ICEM Presentation-Istanbul-September 2014_LinkedIn
Mohamad El-Hinnawi, CCXP
 
Safeabilty: Analyzing the Relationship between Safety and Reliability
Safeabilty: Analyzing the Relationship between Safety and Reliability Safeabilty: Analyzing the Relationship between Safety and Reliability
Safeabilty: Analyzing the Relationship between Safety and Reliability
PlantEngineering
 
Learning to Be Lean (LDS.org Homepage Redesign Case Study)
Learning to Be Lean (LDS.org Homepage Redesign Case Study)Learning to Be Lean (LDS.org Homepage Redesign Case Study)
Learning to Be Lean (LDS.org Homepage Redesign Case Study)
James Valentine, MSC
 
2012 777 The Seven Blind Spots in Business and How to Prevent Them
2012 777   The Seven Blind Spots in Business and How to Prevent Them2012 777   The Seven Blind Spots in Business and How to Prevent Them
2012 777 The Seven Blind Spots in Business and How to Prevent Them
Smith Family Business Initiative at Cornell
 
ASSESSING THE QUALITY OF USABILTY
ASSESSING THE QUALITY OF USABILTYASSESSING THE QUALITY OF USABILTY
ASSESSING THE QUALITY OF USABILTY
Radhika Dilip Kale
 
The Perfect Storm
The Perfect StormThe Perfect Storm
The Perfect Storm
xiaoqunclever
 
Analytics in business
Analytics in businessAnalytics in business
Analytics in business
Niko Vuokko
 
Understanding our experiences
Understanding our experiencesUnderstanding our experiences
Understanding our experiences
Nikita Singh
 
Measuring Digital Signage Networks - Quividi
Measuring Digital Signage Networks - QuividiMeasuring Digital Signage Networks - Quividi
Measuring Digital Signage Networks - Quividi
BroadSign
 
Eventex move roi victor neyndorff
Eventex move roi victor neyndorffEventex move roi victor neyndorff
Eventex move roi victor neyndorff
MOVE
 

Similar to Omni.Digital 2015 - "Know What You Measure. And Why." (20)

Growth Hacking Workshop: AAU
Growth Hacking Workshop: AAUGrowth Hacking Workshop: AAU
Growth Hacking Workshop: AAU
 
0 RIVA Lean Enterprise Overview 12.11.15
0 RIVA Lean Enterprise Overview 12.11.150 RIVA Lean Enterprise Overview 12.11.15
0 RIVA Lean Enterprise Overview 12.11.15
 
Presentations - Zarget CRO meetup 2017
Presentations - Zarget CRO meetup 2017Presentations - Zarget CRO meetup 2017
Presentations - Zarget CRO meetup 2017
 
E-learning and data: two peas in a pod
E-learning and data: two peas in a podE-learning and data: two peas in a pod
E-learning and data: two peas in a pod
 
Ver 1.10 the venture capital ecosystem feb 2015
Ver 1.10   the venture capital ecosystem feb 2015Ver 1.10   the venture capital ecosystem feb 2015
Ver 1.10 the venture capital ecosystem feb 2015
 
Innovation accounting and key metrics for startups
Innovation accounting and key metrics for startupsInnovation accounting and key metrics for startups
Innovation accounting and key metrics for startups
 
Design of Work Experience
Design of Work ExperienceDesign of Work Experience
Design of Work Experience
 
Assessing the quality of usability
Assessing the quality of usabilityAssessing the quality of usability
Assessing the quality of usability
 
A bigger view of UX
A bigger view of UXA bigger view of UX
A bigger view of UX
 
Voice of Customer Program Design
Voice of Customer Program Design Voice of Customer Program Design
Voice of Customer Program Design
 
ICEM Presentation-Istanbul-September 2014_LinkedIn
ICEM Presentation-Istanbul-September 2014_LinkedInICEM Presentation-Istanbul-September 2014_LinkedIn
ICEM Presentation-Istanbul-September 2014_LinkedIn
 
Safeabilty: Analyzing the Relationship between Safety and Reliability
Safeabilty: Analyzing the Relationship between Safety and Reliability Safeabilty: Analyzing the Relationship between Safety and Reliability
Safeabilty: Analyzing the Relationship between Safety and Reliability
 
Learning to Be Lean (LDS.org Homepage Redesign Case Study)
Learning to Be Lean (LDS.org Homepage Redesign Case Study)Learning to Be Lean (LDS.org Homepage Redesign Case Study)
Learning to Be Lean (LDS.org Homepage Redesign Case Study)
 
2012 777 The Seven Blind Spots in Business and How to Prevent Them
2012 777   The Seven Blind Spots in Business and How to Prevent Them2012 777   The Seven Blind Spots in Business and How to Prevent Them
2012 777 The Seven Blind Spots in Business and How to Prevent Them
 
ASSESSING THE QUALITY OF USABILTY
ASSESSING THE QUALITY OF USABILTYASSESSING THE QUALITY OF USABILTY
ASSESSING THE QUALITY OF USABILTY
 
The Perfect Storm
The Perfect StormThe Perfect Storm
The Perfect Storm
 
Analytics in business
Analytics in businessAnalytics in business
Analytics in business
 
Understanding our experiences
Understanding our experiencesUnderstanding our experiences
Understanding our experiences
 
Measuring Digital Signage Networks - Quividi
Measuring Digital Signage Networks - QuividiMeasuring Digital Signage Networks - Quividi
Measuring Digital Signage Networks - Quividi
 
Eventex move roi victor neyndorff
Eventex move roi victor neyndorffEventex move roi victor neyndorff
Eventex move roi victor neyndorff
 

More from Ad Exchanger

PROGRAMMATIC I/O 2015 - "Putting Facebook’s People-Based Marketing to the Test"
PROGRAMMATIC I/O 2015 - "Putting Facebook’s People-Based Marketing to the Test"PROGRAMMATIC I/O 2015 - "Putting Facebook’s People-Based Marketing to the Test"
PROGRAMMATIC I/O 2015 - "Putting Facebook’s People-Based Marketing to the Test"
Ad Exchanger
 
PROGRAMMATIC I/O 2015 - "Fulfilling Programmatic’s Promise" - Panel
PROGRAMMATIC I/O 2015 - "Fulfilling Programmatic’s Promise" - PanelPROGRAMMATIC I/O 2015 - "Fulfilling Programmatic’s Promise" - Panel
PROGRAMMATIC I/O 2015 - "Fulfilling Programmatic’s Promise" - Panel
Ad Exchanger
 
PROGRAMMATIC I/O 2015 - "And You Thought 2015 Was Just About Viewability!" - ...
PROGRAMMATIC I/O 2015 - "And You Thought 2015 Was Just About Viewability!" - ...PROGRAMMATIC I/O 2015 - "And You Thought 2015 Was Just About Viewability!" - ...
PROGRAMMATIC I/O 2015 - "And You Thought 2015 Was Just About Viewability!" - ...
Ad Exchanger
 
Omni.Digital 2015 - "Attribution Fraud: Know If You are Getting Gamed and How...
Omni.Digital 2015 - "Attribution Fraud: Know If You are Getting Gamed and How...Omni.Digital 2015 - "Attribution Fraud: Know If You are Getting Gamed and How...
Omni.Digital 2015 - "Attribution Fraud: Know If You are Getting Gamed and How...
Ad Exchanger
 
Omni.Digital 2015 - "On the Path to Accurate Cross-Channel Attribution"
Omni.Digital 2015 - "On the Path to Accurate Cross-Channel Attribution"Omni.Digital 2015 - "On the Path to Accurate Cross-Channel Attribution"
Omni.Digital 2015 - "On the Path to Accurate Cross-Channel Attribution"
Ad Exchanger
 
Industry Preview 2015 - "Mobile's Unmet Potential" from Melissa Parrish, AdEx...
Industry Preview 2015 - "Mobile's Unmet Potential" from Melissa Parrish, AdEx...Industry Preview 2015 - "Mobile's Unmet Potential" from Melissa Parrish, AdEx...
Industry Preview 2015 - "Mobile's Unmet Potential" from Melissa Parrish, AdEx...
Ad Exchanger
 
Industry Preview 2015 - "The Future of Buying TV Everywhere" from AdExchanger...
Industry Preview 2015 - "The Future of Buying TV Everywhere" from AdExchanger...Industry Preview 2015 - "The Future of Buying TV Everywhere" from AdExchanger...
Industry Preview 2015 - "The Future of Buying TV Everywhere" from AdExchanger...
Ad Exchanger
 

More from Ad Exchanger (7)

PROGRAMMATIC I/O 2015 - "Putting Facebook’s People-Based Marketing to the Test"
PROGRAMMATIC I/O 2015 - "Putting Facebook’s People-Based Marketing to the Test"PROGRAMMATIC I/O 2015 - "Putting Facebook’s People-Based Marketing to the Test"
PROGRAMMATIC I/O 2015 - "Putting Facebook’s People-Based Marketing to the Test"
 
PROGRAMMATIC I/O 2015 - "Fulfilling Programmatic’s Promise" - Panel
PROGRAMMATIC I/O 2015 - "Fulfilling Programmatic’s Promise" - PanelPROGRAMMATIC I/O 2015 - "Fulfilling Programmatic’s Promise" - Panel
PROGRAMMATIC I/O 2015 - "Fulfilling Programmatic’s Promise" - Panel
 
PROGRAMMATIC I/O 2015 - "And You Thought 2015 Was Just About Viewability!" - ...
PROGRAMMATIC I/O 2015 - "And You Thought 2015 Was Just About Viewability!" - ...PROGRAMMATIC I/O 2015 - "And You Thought 2015 Was Just About Viewability!" - ...
PROGRAMMATIC I/O 2015 - "And You Thought 2015 Was Just About Viewability!" - ...
 
Omni.Digital 2015 - "Attribution Fraud: Know If You are Getting Gamed and How...
Omni.Digital 2015 - "Attribution Fraud: Know If You are Getting Gamed and How...Omni.Digital 2015 - "Attribution Fraud: Know If You are Getting Gamed and How...
Omni.Digital 2015 - "Attribution Fraud: Know If You are Getting Gamed and How...
 
Omni.Digital 2015 - "On the Path to Accurate Cross-Channel Attribution"
Omni.Digital 2015 - "On the Path to Accurate Cross-Channel Attribution"Omni.Digital 2015 - "On the Path to Accurate Cross-Channel Attribution"
Omni.Digital 2015 - "On the Path to Accurate Cross-Channel Attribution"
 
Industry Preview 2015 - "Mobile's Unmet Potential" from Melissa Parrish, AdEx...
Industry Preview 2015 - "Mobile's Unmet Potential" from Melissa Parrish, AdEx...Industry Preview 2015 - "Mobile's Unmet Potential" from Melissa Parrish, AdEx...
Industry Preview 2015 - "Mobile's Unmet Potential" from Melissa Parrish, AdEx...
 
Industry Preview 2015 - "The Future of Buying TV Everywhere" from AdExchanger...
Industry Preview 2015 - "The Future of Buying TV Everywhere" from AdExchanger...Industry Preview 2015 - "The Future of Buying TV Everywhere" from AdExchanger...
Industry Preview 2015 - "The Future of Buying TV Everywhere" from AdExchanger...
 

Recently uploaded

Learn more about affiliate marketing as a beginner
Learn more about affiliate marketing as a beginnerLearn more about affiliate marketing as a beginner
Learn more about affiliate marketing as a beginner
MichaelGiles34
 
Boost Your Instagram Views Instantly Proven Free Strategies.pptx
Boost Your Instagram Views Instantly Proven Free Strategies.pptxBoost Your Instagram Views Instantly Proven Free Strategies.pptx
Boost Your Instagram Views Instantly Proven Free Strategies.pptx
InstBlast Marketing
 
Top Strategies for Building High-Quality Backlinks in 2024 PPT.pdf
Top Strategies for Building High-Quality Backlinks in 2024 PPT.pdfTop Strategies for Building High-Quality Backlinks in 2024 PPT.pdf
Top Strategies for Building High-Quality Backlinks in 2024 PPT.pdf
1Solutions Pvt. Ltd.
 
Viapulsa - Content Strategist Assignment
Viapulsa - Content Strategist AssignmentViapulsa - Content Strategist Assignment
Viapulsa - Content Strategist Assignment
Amal Agung Cahyadi
 
Compitive analysis on Noise pvt Ltd.pptx
Compitive analysis on Noise pvt Ltd.pptxCompitive analysis on Noise pvt Ltd.pptx
Compitive analysis on Noise pvt Ltd.pptx
SauravDey45
 
3 Best “Add to Calendar” Link Generator Tools (2024)
3 Best  “Add to Calendar”  Link Generator Tools (2024)3 Best  “Add to Calendar”  Link Generator Tools (2024)
3 Best “Add to Calendar” Link Generator Tools (2024)
Y
 
What is Digital Marketing: A Comprehensive Guide
What is Digital Marketing: A Comprehensive GuideWhat is Digital Marketing: A Comprehensive Guide
What is Digital Marketing: A Comprehensive Guide
V-tech Marketing
 
Title: Making Money the Easy Way: A Quick Guide to Generating Income
Title: Making Money the Easy Way: A Quick Guide to Generating IncomeTitle: Making Money the Easy Way: A Quick Guide to Generating Income
Title: Making Money the Easy Way: A Quick Guide to Generating Income
WilliamZinsmeister
 
Boost Your Instagram Views Instantly Proven Free Strategies.
Boost Your Instagram Views Instantly Proven Free Strategies.Boost Your Instagram Views Instantly Proven Free Strategies.
Boost Your Instagram Views Instantly Proven Free Strategies.
InstBlast Marketing
 
THE STORY COMMUNICATION Credential 2024.pptx
THE STORY COMMUNICATION Credential 2024.pptxTHE STORY COMMUNICATION Credential 2024.pptx
THE STORY COMMUNICATION Credential 2024.pptx
huyenngo62
 
Lyrics Video Document for artist Kevin Aguirre
Lyrics Video Document for artist Kevin AguirreLyrics Video Document for artist Kevin Aguirre
Lyrics Video Document for artist Kevin Aguirre
rawlensproductionsco
 
SEO and Google Ads Fundamentals Gokce Yesilbas
SEO and Google Ads Fundamentals Gokce YesilbasSEO and Google Ads Fundamentals Gokce Yesilbas
SEO and Google Ads Fundamentals Gokce Yesilbas
gokceyesilbas
 
在线办理(worc毕业证书)伍斯特大学毕业证成绩单一模一样
在线办理(worc毕业证书)伍斯特大学毕业证成绩单一模一样在线办理(worc毕业证书)伍斯特大学毕业证成绩单一模一样
在线办理(worc毕业证书)伍斯特大学毕业证成绩单一模一样
5ys5mvlp
 
What Software is Used in Marketing in 2024.
What Software is Used in Marketing in 2024.What Software is Used in Marketing in 2024.
What Software is Used in Marketing in 2024.
Ishaaq6
 
How to Maximize Sales Using Social Commerce
How to Maximize Sales Using Social CommerceHow to Maximize Sales Using Social Commerce
How to Maximize Sales Using Social Commerce
Vbout.com
 
2024 case study-SHEIN Analysis-Final.pdf
2024 case study-SHEIN Analysis-Final.pdf2024 case study-SHEIN Analysis-Final.pdf
2024 case study-SHEIN Analysis-Final.pdf
Sun.Lee
 
2025 Adventur ehomes Product Guide released Jun 1 2024
2025 Adventur ehomes Product Guide released Jun 1 20242025 Adventur ehomes Product Guide released Jun 1 2024
2025 Adventur ehomes Product Guide released Jun 1 2024
JimWilliams206540
 
01 Field+Guide+to+Human-Centered+Design_IDEOorg_English GUIA COMPLETA DETALLA...
01 Field+Guide+to+Human-Centered+Design_IDEOorg_English GUIA COMPLETA DETALLA...01 Field+Guide+to+Human-Centered+Design_IDEOorg_English GUIA COMPLETA DETALLA...
01 Field+Guide+to+Human-Centered+Design_IDEOorg_English GUIA COMPLETA DETALLA...
Jorge Calmett
 
How to Start Affiliate Marketing with ChatGPT- A Step-by-Step Guide (1).pdf
How to Start Affiliate Marketing with ChatGPT- A Step-by-Step Guide (1).pdfHow to Start Affiliate Marketing with ChatGPT- A Step-by-Step Guide (1).pdf
How to Start Affiliate Marketing with ChatGPT- A Step-by-Step Guide (1).pdf
SimpleMoneyMaker
 
一比一原版哥伦比亚大学毕业证(Columbia毕业证书)学历如何办理
一比一原版哥伦比亚大学毕业证(Columbia毕业证书)学历如何办理一比一原版哥伦比亚大学毕业证(Columbia毕业证书)学历如何办理
一比一原版哥伦比亚大学毕业证(Columbia毕业证书)学历如何办理
omywaf
 

Recently uploaded (20)

Learn more about affiliate marketing as a beginner
Learn more about affiliate marketing as a beginnerLearn more about affiliate marketing as a beginner
Learn more about affiliate marketing as a beginner
 
Boost Your Instagram Views Instantly Proven Free Strategies.pptx
Boost Your Instagram Views Instantly Proven Free Strategies.pptxBoost Your Instagram Views Instantly Proven Free Strategies.pptx
Boost Your Instagram Views Instantly Proven Free Strategies.pptx
 
Top Strategies for Building High-Quality Backlinks in 2024 PPT.pdf
Top Strategies for Building High-Quality Backlinks in 2024 PPT.pdfTop Strategies for Building High-Quality Backlinks in 2024 PPT.pdf
Top Strategies for Building High-Quality Backlinks in 2024 PPT.pdf
 
Viapulsa - Content Strategist Assignment
Viapulsa - Content Strategist AssignmentViapulsa - Content Strategist Assignment
Viapulsa - Content Strategist Assignment
 
Compitive analysis on Noise pvt Ltd.pptx
Compitive analysis on Noise pvt Ltd.pptxCompitive analysis on Noise pvt Ltd.pptx
Compitive analysis on Noise pvt Ltd.pptx
 
3 Best “Add to Calendar” Link Generator Tools (2024)
3 Best  “Add to Calendar”  Link Generator Tools (2024)3 Best  “Add to Calendar”  Link Generator Tools (2024)
3 Best “Add to Calendar” Link Generator Tools (2024)
 
What is Digital Marketing: A Comprehensive Guide
What is Digital Marketing: A Comprehensive GuideWhat is Digital Marketing: A Comprehensive Guide
What is Digital Marketing: A Comprehensive Guide
 
Title: Making Money the Easy Way: A Quick Guide to Generating Income
Title: Making Money the Easy Way: A Quick Guide to Generating IncomeTitle: Making Money the Easy Way: A Quick Guide to Generating Income
Title: Making Money the Easy Way: A Quick Guide to Generating Income
 
Boost Your Instagram Views Instantly Proven Free Strategies.
Boost Your Instagram Views Instantly Proven Free Strategies.Boost Your Instagram Views Instantly Proven Free Strategies.
Boost Your Instagram Views Instantly Proven Free Strategies.
 
THE STORY COMMUNICATION Credential 2024.pptx
THE STORY COMMUNICATION Credential 2024.pptxTHE STORY COMMUNICATION Credential 2024.pptx
THE STORY COMMUNICATION Credential 2024.pptx
 
Lyrics Video Document for artist Kevin Aguirre
Lyrics Video Document for artist Kevin AguirreLyrics Video Document for artist Kevin Aguirre
Lyrics Video Document for artist Kevin Aguirre
 
SEO and Google Ads Fundamentals Gokce Yesilbas
SEO and Google Ads Fundamentals Gokce YesilbasSEO and Google Ads Fundamentals Gokce Yesilbas
SEO and Google Ads Fundamentals Gokce Yesilbas
 
在线办理(worc毕业证书)伍斯特大学毕业证成绩单一模一样
在线办理(worc毕业证书)伍斯特大学毕业证成绩单一模一样在线办理(worc毕业证书)伍斯特大学毕业证成绩单一模一样
在线办理(worc毕业证书)伍斯特大学毕业证成绩单一模一样
 
What Software is Used in Marketing in 2024.
What Software is Used in Marketing in 2024.What Software is Used in Marketing in 2024.
What Software is Used in Marketing in 2024.
 
How to Maximize Sales Using Social Commerce
How to Maximize Sales Using Social CommerceHow to Maximize Sales Using Social Commerce
How to Maximize Sales Using Social Commerce
 
2024 case study-SHEIN Analysis-Final.pdf
2024 case study-SHEIN Analysis-Final.pdf2024 case study-SHEIN Analysis-Final.pdf
2024 case study-SHEIN Analysis-Final.pdf
 
2025 Adventur ehomes Product Guide released Jun 1 2024
2025 Adventur ehomes Product Guide released Jun 1 20242025 Adventur ehomes Product Guide released Jun 1 2024
2025 Adventur ehomes Product Guide released Jun 1 2024
 
01 Field+Guide+to+Human-Centered+Design_IDEOorg_English GUIA COMPLETA DETALLA...
01 Field+Guide+to+Human-Centered+Design_IDEOorg_English GUIA COMPLETA DETALLA...01 Field+Guide+to+Human-Centered+Design_IDEOorg_English GUIA COMPLETA DETALLA...
01 Field+Guide+to+Human-Centered+Design_IDEOorg_English GUIA COMPLETA DETALLA...
 
How to Start Affiliate Marketing with ChatGPT- A Step-by-Step Guide (1).pdf
How to Start Affiliate Marketing with ChatGPT- A Step-by-Step Guide (1).pdfHow to Start Affiliate Marketing with ChatGPT- A Step-by-Step Guide (1).pdf
How to Start Affiliate Marketing with ChatGPT- A Step-by-Step Guide (1).pdf
 
一比一原版哥伦比亚大学毕业证(Columbia毕业证书)学历如何办理
一比一原版哥伦比亚大学毕业证(Columbia毕业证书)学历如何办理一比一原版哥伦比亚大学毕业证(Columbia毕业证书)学历如何办理
一比一原版哥伦比亚大学毕业证(Columbia毕业证书)学历如何办理
 

Omni.Digital 2015 - "Know What You Measure. And Why."

  • 1. KNOW WHAT YOU MEASURE. AND WHY. KIRIL TSEMEKHMAN CHIEF DATA SCIENTIST, INTEGRAL AD SCIENCE
  • 2. 1
  • 3. WHAT MATTERS TO YOU? • Measuring performance: incrementality • Attribution: rewarding rightly • Driving performance: the importance of environment • Taking it in: influencing the user • Revealing what matters 2
  • 4. ATTRIBUTION: REWARDING RIGHTLY • Establishing the baseline • Measuring and attributing only incremental actions • Giving credit where credit is really due 3
  • 5. 4
  • 6. 5
  • 7. 6
  • 8. LESSONS LEARNED • The treasure chest had “baseline coins” • Pirates had to find a way to separate baseline coins from their “incremental loot” • Knowing each pirate’s effort leads to rewarding rightly 7
  • 9. DRIVING PERFORMANCE: THE IMPORTANCE OF ENVIRONMENT • Is viewability all that matters? • Other dimensions that make a difference – How many times driver sees the billboard? – How often? – Is there enough time to read it? 8
  • 10. 9
  • 11. 10
  • 12. LESSONS LEARNED • Picking the right place is not that simple • Consider what’s happening at any given moment • How big is the opportunity to be noticed and remembered? 11
  • 13. TAKING IT IN: INFLUENCING THE USER • Opportunity does not equal viewability • Viewability does not equal engagement • Engagement does not equal action…yet 12
  • 14. 13
  • 15. 14
  • 16. 15
  • 17. LESSONS LEARNED • You are not cured by getting a prescription • You are also not cured by filling a prescription • You have a better chance to be cured by taking the medication as directed – but then again, different dosages may yield better results 16
  • 18. CAUSAL IMPACT OF USER ENGAGEMENT WITH DIGITAL ADS • “Natural” A/B test • Unviewable ads instead of PSAs • Causal impact of different treatment levels – Number of ads shown to user – Total exposure time 17
  • 19. AND NOW…REVEALING WHAT MATTERS Measuring incremental (causal) actions: • Does the number of exposures matter? • Does the frequency of exposures matter? • Does the total exposure duration matter? • Are two long ads better than ten short ads? 18
  • 20. 600K 200K 1 2 3 4 5 6 7 8 9 10 11 Impressions per User NumberofUsersOverOneWeek 7% of users were served over 5 impressions 2.5% of users were served over 10 impressions AD FREQUENCY: MOST USERS SAW 1-2 ADS 19
  • 21. 700K 120K 1-5 5-15 15-30 30-45 45-60 60-75 75-90 90-120 120-150 >150 Total User Exposure Time (seconds) NumberofUsersOverOneWeek 57% of users have no viewable impressions USER EXPOSURE TIME: MOST EXPOSED <5 SEC 20
  • 22. 0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0 50 100 150 Gaming Telcom Exposure to Advertising (seconds) AdditiveLiftinConversions EFFECT OF TOTAL EXPOSURE TIME ON INCREMENTAL CONVERSIONS 21
  • 23. 0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0 2 4 6 8 10 Gaming Telecom Number of Impressions AdditiveLiftinConversions EFFECT OF REPEATED EXPOSURES ON INCREMENTAL CONVERSIONS 22
  • 24. 0 0.05 0.1 0.15 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 Number of Impressions AdditiveLiftinConversionsCOMBINED EFFECT OF EXPOSURE FREQUENCY & DURATION 23
  • 25. LESSONS LEARNED • Number and frequency of exposures matter • Total exposure duration matters • One long exposure does not quite reach the goal 24
  • 26. ULTIMATE LESSONS LEARNED • Remember what matters and ask the right questions. Then… • Measure the right things and measure them right. 25