SlideShare a Scribd company logo
Online Marketing Summit Philadelphia | July 8, 2010 Changing the way we measure success Tami DalleyDirector, User Experience OptimizationROILabs 1
7/9/2010 3 Cultural Cheat Sheet ,[object Object]
 Behaviour = Behavior
 Overuse of new lingo: “awesome,” “totally,” “LIKE!” “SNAP”,[object Object]
7/9/2010 Quick Testing Basics 5
7/9/2010 6 Types of Tests   A/B/N Testing
7/9/2010 7 Vs. Vs.
7/9/2010 8 Types of Tests   Multivariate  Testing (MVT)
Headlines/Images Calls To Action Copy Display *
7/9/2010 Why Test? 10
7/9/2010 11
7/9/2010 12 *
7/9/2010 How to Test? 13
[object Object]
Wide and Shallow, then Narrow and Deep
Rinse, repeat
Have a testing strategy
1 primary success metric7/9/2010 14 Testing Strategies
7/9/2010 15 Only 1 Success Metric
7/9/2010 16 Which one to Pick? *
7/9/2010 17
7/9/2010 18
“Success means different things to different people” 7/9/2010 19 *
7/9/2010 So what to do? Use a weighted metric 20
7/9/2010 21 What Is It? Assigning value to behaviors for “assists.”  Home Runs: 18 Home Runs: 15 Runs Batted In: 66 Runs Batted In: 103 Which player is more valuable to you? *
7/9/2010 Why Use A Weighted Metric? 22
7/9/2010 23 Balanced view of success
7/9/2010 24 Multiple desired outcomes
7/9/2010 25 Looking for extra optimization *
7/9/2010 When to use it 26
7/9/2010 27
7/9/2010 28
7/9/2010 29
Can be used to optimize: ,[object Object]
PPC
SEO
Email
Social
Etc,etc7/9/2010 30 *1/2
7/9/2010 How do you do it? 31
7/9/2010 32 The Process Step 1:Identify online behaviors that reflect engagement (and correlate with leads or sales) Step 2:Confirm correlation, assign  value and calculate the weighted metric Step 3:Optimizeusing this weighted metric “point” system
7/9/2010 33 Step 1.1: ID Behavioral Indicators Identify online behaviors that correlate to leads: ,[object Object]
 Whitepaper Download
 Free Trials
 Online Chat
 Product Page View,[object Object]
7/9/2010 35 Ideal = Integration Step 1.2: Visibility into “Success”
7/9/2010 36 Reality = ?
7/9/2010 37
7/9/2010 38
7/9/2010 39 Pull a Goal Report Goals Week
7/9/2010 40 Analyze Trends ,[object Object],[object Object]
 Assign a valueto those behaviorsOption 1 – Gut instinct Option 2 – Statistical Analysis “ When this behavior happens, leads increase by x.”
7/9/2010 42 Step 3: Analyze Output ,[object Object]
So, 10 video views = 2.6 Leads
Translate into points (x100)

More Related Content

Similar to How to Use an Index to Measure Page Testing Success - Tami Dalley, ROI Labs

The 7 step problem solving methodology
The 7 step problem solving methodologyThe 7 step problem solving methodology
The 7 step problem solving methodology
quest_pune
 
The Testing Planet Issue 7
The Testing Planet Issue 7The Testing Planet Issue 7
The Testing Planet Issue 7
Rosie Sherry
 
Final Presentation JAM
Final Presentation JAMFinal Presentation JAM
Final Presentation JAM
Jason Vineis
 
Design Process YouTube
Design Process YouTubeDesign Process YouTube
Design Process YouTube
Hong Qu
 

Similar to How to Use an Index to Measure Page Testing Success - Tami Dalley, ROI Labs (20)

UX Process
UX ProcessUX Process
UX Process
 
Always Be Testing - Learn from Every A/B Test (Hiten Shah)
Always Be Testing - Learn from Every A/B Test (Hiten Shah)Always Be Testing - Learn from Every A/B Test (Hiten Shah)
Always Be Testing - Learn from Every A/B Test (Hiten Shah)
 
The 7 step problem solving methodology
The 7 step problem solving methodologyThe 7 step problem solving methodology
The 7 step problem solving methodology
 
Why You Need to Do a Pilot - Mitch Weisburgh, Founder, Games4Ed & Scott Brews...
Why You Need to Do a Pilot - Mitch Weisburgh, Founder, Games4Ed & Scott Brews...Why You Need to Do a Pilot - Mitch Weisburgh, Founder, Games4Ed & Scott Brews...
Why You Need to Do a Pilot - Mitch Weisburgh, Founder, Games4Ed & Scott Brews...
 
Make Your Games Play, Teach, and Socialize Better: Usability & Playability Te...
Make Your Games Play, Teach, and Socialize Better: Usability & Playability Te...Make Your Games Play, Teach, and Socialize Better: Usability & Playability Te...
Make Your Games Play, Teach, and Socialize Better: Usability & Playability Te...
 
ux flow and process
ux flow and processux flow and process
ux flow and process
 
Benchmarking new
Benchmarking newBenchmarking new
Benchmarking new
 
MEGA 2011 NSW Info night
MEGA 2011 NSW Info nightMEGA 2011 NSW Info night
MEGA 2011 NSW Info night
 
You Can Do Program Evaluation
You Can Do Program EvaluationYou Can Do Program Evaluation
You Can Do Program Evaluation
 
The ways to fuck up ab testing (from data products meetup)
The ways to fuck up ab testing (from data products meetup)The ways to fuck up ab testing (from data products meetup)
The ways to fuck up ab testing (from data products meetup)
 
Growing Your Business Through Experimentation
Growing Your Business Through ExperimentationGrowing Your Business Through Experimentation
Growing Your Business Through Experimentation
 
Retooling Bootcamp 2018: How Reviews are Disrupting Homebuilding
Retooling Bootcamp 2018: How Reviews are Disrupting HomebuildingRetooling Bootcamp 2018: How Reviews are Disrupting Homebuilding
Retooling Bootcamp 2018: How Reviews are Disrupting Homebuilding
 
A/B Testing and Experimentation in Data Science
A/B Testing and Experimentation in Data ScienceA/B Testing and Experimentation in Data Science
A/B Testing and Experimentation in Data Science
 
Social marketing monitoring & evaluation October 2010
Social marketing monitoring & evaluation October 2010 Social marketing monitoring & evaluation October 2010
Social marketing monitoring & evaluation October 2010
 
Optimizely Workshop: Mobile Walkthrough
Optimizely Workshop: Mobile Walkthrough Optimizely Workshop: Mobile Walkthrough
Optimizely Workshop: Mobile Walkthrough
 
The Testing Planet Issue 7
The Testing Planet Issue 7The Testing Planet Issue 7
The Testing Planet Issue 7
 
Products and People Over Process and Dogma
  Products and People Over Process and Dogma  Products and People Over Process and Dogma
Products and People Over Process and Dogma
 
Final Presentation JAM
Final Presentation JAMFinal Presentation JAM
Final Presentation JAM
 
Design Process YouTube
Design Process YouTubeDesign Process YouTube
Design Process YouTube
 
Putting Customers First: How To Build Data-Driven Strategies To Ensure Custom...
Putting Customers First: How To Build Data-Driven Strategies To Ensure Custom...Putting Customers First: How To Build Data-Driven Strategies To Ensure Custom...
Putting Customers First: How To Build Data-Driven Strategies To Ensure Custom...
 

More from Online Marketing Summit

Social Media Empowering People, Empowering Business - Emma Roffey
Social Media Empowering People, Empowering Business -  Emma RoffeySocial Media Empowering People, Empowering Business -  Emma Roffey
Social Media Empowering People, Empowering Business - Emma Roffey
Online Marketing Summit
 
The current state of email, social, and mobile integration oms at dma - al...
The current state of email, social, and mobile integration   oms at dma  - al...The current state of email, social, and mobile integration   oms at dma  - al...
The current state of email, social, and mobile integration oms at dma - al...
Online Marketing Summit
 

More from Online Marketing Summit (20)

Social Integration - Sundeep Kapur
Social Integration - Sundeep KapurSocial Integration - Sundeep Kapur
Social Integration - Sundeep Kapur
 
Reputation Management - Giliian Muessig
Reputation Management  - Giliian MuessigReputation Management  - Giliian Muessig
Reputation Management - Giliian Muessig
 
Opening for TFMA - Aaron Kahlow
Opening for TFMA - Aaron KahlowOpening for TFMA - Aaron Kahlow
Opening for TFMA - Aaron Kahlow
 
Social Media Empowering People, Empowering Business - Emma Roffey
Social Media Empowering People, Empowering Business -  Emma RoffeySocial Media Empowering People, Empowering Business -  Emma Roffey
Social Media Empowering People, Empowering Business - Emma Roffey
 
OMS London PPT Accountability - Jen Brady
OMS London PPT Accountability - Jen BradyOMS London PPT Accountability - Jen Brady
OMS London PPT Accountability - Jen Brady
 
Conversion Content - Scott Brinker
Conversion Content - Scott BrinkerConversion Content - Scott Brinker
Conversion Content - Scott Brinker
 
OMI Workshop @ SES London 2011 - Search & Social, a Global Perspective
OMI Workshop @ SES London 2011 - Search & Social, a Global PerspectiveOMI Workshop @ SES London 2011 - Search & Social, a Global Perspective
OMI Workshop @ SES London 2011 - Search & Social, a Global Perspective
 
The current state of email, social, and mobile integration oms at dma - al...
The current state of email, social, and mobile integration   oms at dma  - al...The current state of email, social, and mobile integration   oms at dma  - al...
The current state of email, social, and mobile integration oms at dma - al...
 
Improving SEO Conversions Through Social Content - Gillian Muessig
Improving SEO Conversions Through Social Content - Gillian MuessigImproving SEO Conversions Through Social Content - Gillian Muessig
Improving SEO Conversions Through Social Content - Gillian Muessig
 
Generating New Business Leads with Corporate Blogging - Compendium
Generating New Business Leads with Corporate Blogging - CompendiumGenerating New Business Leads with Corporate Blogging - Compendium
Generating New Business Leads with Corporate Blogging - Compendium
 
The SEO Value of Social Media - Catfish Comstock
The SEO Value of Social Media - Catfish ComstockThe SEO Value of Social Media - Catfish Comstock
The SEO Value of Social Media - Catfish Comstock
 
From "Sorta" to Portable: Five Actionable Steps to Embrace On-The-Go Custome...
From "Sorta" to Portable:  Five Actionable Steps to Embrace On-The-Go Custome...From "Sorta" to Portable:  Five Actionable Steps to Embrace On-The-Go Custome...
From "Sorta" to Portable: Five Actionable Steps to Embrace On-The-Go Custome...
 
Using Data-Driven Decisions to Enhance Cross-Channel Marketing - Experian Ma...
Using Data-Driven Decisions to Enhance Cross-Channel Marketing  - Experian Ma...Using Data-Driven Decisions to Enhance Cross-Channel Marketing  - Experian Ma...
Using Data-Driven Decisions to Enhance Cross-Channel Marketing - Experian Ma...
 
Empathy in Ecommerce: Achieving Breakthrough Conversion Rates (and Profit!) b...
Empathy in Ecommerce: Achieving Breakthrough Conversion Rates (and Profit!) b...Empathy in Ecommerce: Achieving Breakthrough Conversion Rates (and Profit!) b...
Empathy in Ecommerce: Achieving Breakthrough Conversion Rates (and Profit!) b...
 
Integrated Marketing - Susan Prater, Masco Cabinetry
Integrated Marketing - Susan Prater, Masco CabinetryIntegrated Marketing - Susan Prater, Masco Cabinetry
Integrated Marketing - Susan Prater, Masco Cabinetry
 
Inbound Marketing: Lead Generation at 60% Lower Cost - HubSpot, Ellie Mirman
Inbound Marketing: Lead Generation at 60% Lower Cost - HubSpot, Ellie MirmanInbound Marketing: Lead Generation at 60% Lower Cost - HubSpot, Ellie Mirman
Inbound Marketing: Lead Generation at 60% Lower Cost - HubSpot, Ellie Mirman
 
Building a Content Marketing Engine to Accelerate Revenue Growth - BreakingPoint
Building a Content Marketing Engine to Accelerate Revenue Growth - BreakingPointBuilding a Content Marketing Engine to Accelerate Revenue Growth - BreakingPoint
Building a Content Marketing Engine to Accelerate Revenue Growth - BreakingPoint
 
Inbound Marketing Works: Spend 60% Less on Lead Gen - HubSpot, Kirsten Knipp
Inbound Marketing Works: Spend 60% Less on Lead Gen - HubSpot, Kirsten KnippInbound Marketing Works: Spend 60% Less on Lead Gen - HubSpot, Kirsten Knipp
Inbound Marketing Works: Spend 60% Less on Lead Gen - HubSpot, Kirsten Knipp
 
Closing the Gap on ROI Measurement - Spur Interactive, Steve Interactive
Closing the Gap on ROI Measurement - Spur Interactive, Steve InteractiveClosing the Gap on ROI Measurement - Spur Interactive, Steve Interactive
Closing the Gap on ROI Measurement - Spur Interactive, Steve Interactive
 
Design For Your Subscribers: Tips and Tricks to Increase Email Marketing ROI ...
Design For Your Subscribers: Tips and Tricks to Increase Email Marketing ROI ...Design For Your Subscribers: Tips and Tricks to Increase Email Marketing ROI ...
Design For Your Subscribers: Tips and Tricks to Increase Email Marketing ROI ...
 

Recently uploaded

Adversarial Attention Modeling for Multi-dimensional Emotion Regression.pdf
Adversarial Attention Modeling for Multi-dimensional Emotion Regression.pdfAdversarial Attention Modeling for Multi-dimensional Emotion Regression.pdf
Adversarial Attention Modeling for Multi-dimensional Emotion Regression.pdf
Po-Chuan Chen
 

Recently uploaded (20)

Gyanartha SciBizTech Quiz slideshare.pptx
Gyanartha SciBizTech Quiz slideshare.pptxGyanartha SciBizTech Quiz slideshare.pptx
Gyanartha SciBizTech Quiz slideshare.pptx
 
Basic Civil Engineering Notes of Chapter-6, Topic- Ecosystem, Biodiversity G...
Basic Civil Engineering Notes of Chapter-6,  Topic- Ecosystem, Biodiversity G...Basic Civil Engineering Notes of Chapter-6,  Topic- Ecosystem, Biodiversity G...
Basic Civil Engineering Notes of Chapter-6, Topic- Ecosystem, Biodiversity G...
 
Basic Civil Engg Notes_Chapter-6_Environment Pollution & Engineering
Basic Civil Engg Notes_Chapter-6_Environment Pollution & EngineeringBasic Civil Engg Notes_Chapter-6_Environment Pollution & Engineering
Basic Civil Engg Notes_Chapter-6_Environment Pollution & Engineering
 
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
 
slides CapTechTalks Webinar May 2024 Alexander Perry.pptx
slides CapTechTalks Webinar May 2024 Alexander Perry.pptxslides CapTechTalks Webinar May 2024 Alexander Perry.pptx
slides CapTechTalks Webinar May 2024 Alexander Perry.pptx
 
How to Create Map Views in the Odoo 17 ERP
How to Create Map Views in the Odoo 17 ERPHow to Create Map Views in the Odoo 17 ERP
How to Create Map Views in the Odoo 17 ERP
 
Phrasal Verbs.XXXXXXXXXXXXXXXXXXXXXXXXXX
Phrasal Verbs.XXXXXXXXXXXXXXXXXXXXXXXXXXPhrasal Verbs.XXXXXXXXXXXXXXXXXXXXXXXXXX
Phrasal Verbs.XXXXXXXXXXXXXXXXXXXXXXXXXX
 
Adversarial Attention Modeling for Multi-dimensional Emotion Regression.pdf
Adversarial Attention Modeling for Multi-dimensional Emotion Regression.pdfAdversarial Attention Modeling for Multi-dimensional Emotion Regression.pdf
Adversarial Attention Modeling for Multi-dimensional Emotion Regression.pdf
 
UNIT – IV_PCI Complaints: Complaints and evaluation of complaints, Handling o...
UNIT – IV_PCI Complaints: Complaints and evaluation of complaints, Handling o...UNIT – IV_PCI Complaints: Complaints and evaluation of complaints, Handling o...
UNIT – IV_PCI Complaints: Complaints and evaluation of complaints, Handling o...
 
How to Break the cycle of negative Thoughts
How to Break the cycle of negative ThoughtsHow to Break the cycle of negative Thoughts
How to Break the cycle of negative Thoughts
 
Application of Matrices in real life. Presentation on application of matrices
Application of Matrices in real life. Presentation on application of matricesApplication of Matrices in real life. Presentation on application of matrices
Application of Matrices in real life. Presentation on application of matrices
 
Palestine last event orientationfvgnh .pptx
Palestine last event orientationfvgnh .pptxPalestine last event orientationfvgnh .pptx
Palestine last event orientationfvgnh .pptx
 
Sectors of the Indian Economy - Class 10 Study Notes pdf
Sectors of the Indian Economy - Class 10 Study Notes pdfSectors of the Indian Economy - Class 10 Study Notes pdf
Sectors of the Indian Economy - Class 10 Study Notes pdf
 
MARUTI SUZUKI- A Successful Joint Venture in India.pptx
MARUTI SUZUKI- A Successful Joint Venture in India.pptxMARUTI SUZUKI- A Successful Joint Venture in India.pptx
MARUTI SUZUKI- A Successful Joint Venture in India.pptx
 
Introduction to Quality Improvement Essentials
Introduction to Quality Improvement EssentialsIntroduction to Quality Improvement Essentials
Introduction to Quality Improvement Essentials
 
2024_Student Session 2_ Set Plan Preparation.pptx
2024_Student Session 2_ Set Plan Preparation.pptx2024_Student Session 2_ Set Plan Preparation.pptx
2024_Student Session 2_ Set Plan Preparation.pptx
 
Forest and Wildlife Resources Class 10 Free Study Material PDF
Forest and Wildlife Resources Class 10 Free Study Material PDFForest and Wildlife Resources Class 10 Free Study Material PDF
Forest and Wildlife Resources Class 10 Free Study Material PDF
 
Solid waste management & Types of Basic civil Engineering notes by DJ Sir.pptx
Solid waste management & Types of Basic civil Engineering notes by DJ Sir.pptxSolid waste management & Types of Basic civil Engineering notes by DJ Sir.pptx
Solid waste management & Types of Basic civil Engineering notes by DJ Sir.pptx
 
PART A. Introduction to Costumer Service
PART A. Introduction to Costumer ServicePART A. Introduction to Costumer Service
PART A. Introduction to Costumer Service
 
Home assignment II on Spectroscopy 2024 Answers.pdf
Home assignment II on Spectroscopy 2024 Answers.pdfHome assignment II on Spectroscopy 2024 Answers.pdf
Home assignment II on Spectroscopy 2024 Answers.pdf
 

How to Use an Index to Measure Page Testing Success - Tami Dalley, ROI Labs

Editor's Notes

  1. MethodologyMore effective optimization Greater Marketing Success
  2. http://www.istockphoto.com/stock-photo-10999426-strategy-implementation.php
  3. Incrementally improve campaigns that have already been optimized using a single measure
  4. Incrementally improve campaigns that have already been optimized using a single measure
  5. Vacation exampleTrue for landing pages - a LP needs to do so many things. Not just for internal stakeholders but for consumers at totally different point in the shopping cycle. So who’s right? Well everyone is… but how do you address the problem?
  6. Balanced View of Success – not one dimensionalhttp://www.istockphoto.com/stock-photo-11846527-balancing-stones.php
  7. Better optimize for multiple desired outcomeshttp://www.istockphoto.com/stock-photo-3470760-blank-signpost.php
  8. Incrementally improve campaigns that have already been optimized using a single measureAston Martin DB9 - $289K. Pretty good street car – “how can we make it better?” There is always more you can do…
  9. Long Sales CyclesConsidered purchasesEducation goals?
  10. Multiple possible outcomes from a single path
  11. Integrate offline data or data from other sourceshttp://www.istockphoto.com/stock-photo-713960-bridge.php
  12. These are obvious online examples
  13. Ideal - (Omniture GENESIS integration w/sales force. Call tracking integration - Need a unique identifierOffline SalesCRM Call centreValidation:Contact correctInterestDecision 1:Option 1) Tie secondary conversion to offline salesOption 2) Tie to online leads
  14. Ideal - (Omniture GENESIS integration w/sales force. Call tracking integration - Need a unique identifierOffline SalesCRM Call centreValidation:Contact correctInterestDecision 1:Option 1) Tie secondary conversion to offline salesOption 2) Tie to online leads
  15. Ideal - (Omniture GENESIS integration w/sales force. Call tracking integration - Need a unique identifierOffline SalesCRM Call centreValidation:Contact correctInterestDecision 1:Option 1) Tie secondary conversion to offline salesOption 2) Tie to online leads
  16. Look at least at 2 rounds of screeningShit lead or good leadCalled/contacted and lead was interested
  17. Following example is assuming you have no access to offline data
  18. Now, if you HAVE integration of offline data – this step is not necessary. Because you clearly know the relationship between the online engagement behaviours and ultimate conversion
  19. Use same arithmetic on each of the behaviors
  20. Limitation of GWO (can use some GA to get at the details) plus stat sign calculatorUse revenue filed in T&T
  21. Use revenue filed in T&T