This slide presentation is from a webcast sponsored by ClearSaleing and Search Marketing Now. The webcast focuses on using advanced statistical models to build attribution models that can then be tested. The webinar is co-hosted by Adam Goldberg, ClearSaleing co-founder and Chief Innovation Officer, and Dr. Purush Papatla, President of Vetra Analytics, which is a high-end statistical consultancy group that is partnered with ClearSaleing. Dr. Papatla shows how using high-end statistical models can allow one to build attribution models that specifically address how to account for: * Social media * Word of Mouth * The differences between short Purchase Paths and long Purchase Paths As in previous webinars, there are some interactive elements. To view the webinar in its entirety, please visit www.ClearSaleing.com or: http://searchmarketingnow.com/on-demand

1. Attribution Management Forum 3.0:
How To Build Accurate Models
To Solve Attribution
Tuesday May 5, 2009
1 PM EDT
Speakers: Adam Goldberg,
Dr. PurushPapatla
©2009 Third Door Media, Inc.

2. Viewing Tips
 Turn Off Pop-Up Blockers
 Technical difficulties?
Click on “Help?” link

Use Q+A box

 Submitting questions to speaker
Q+A session at end of webcast

Use “Ask a Question” box to submit

questions
Send questions at any time

©2009 Third Door Media, Inc.

3. Adam Goldberg,
Chief Innovation Officer
 Co-Founded ClearSaleing Inc. in 2006
Columbus, OH
 Started Google’s inside sales organization in
2003-2006
New York City, NY
 Started Actuate’s inside sales organization in
2000-2003
San Francisco, CA
 Worked for Oracle Corp. in 1998-2000 in
Major Account Sales
Redwood Shores, CA
 Speaker and Trainer at events such as:
(SMX), (SES), (DMA)
©2009 Third Door Media, Inc.

4. Dr. PurushPapatla
 Ph.D. from Kellogg School of Management at
Northwestern University
 Associate Professor, Marketing
Sheldon B. Lubar School of Business
 President and Founder; Vetra Analytics
 Published in top-tier marketing journals
» Marketing Science
» Journal of Marketing Research
» Journal of Business Research
» Journal of Retailing
» Journal of Interactive Marketing
©2009 Third Door Media, Inc.

5. EVOLUTION OF ONLINE ADVERTISING
Number of Online Offline Attribution Portfolio
Clicks Conversions Conversions Management Management
©2009 Third Door Media, Inc.

6. ATTRIBUTION MANAGEMENT HIERARCHY
©2009 Third Door Media, Inc.

7. THE CONSUMER BUYING CYCLE
©2009 Third Door Media, Inc.

8. THE CONSUMER BUYING CYCLE
©2009 Third Door Media, Inc.

9. THE CONSUMER BUYING CYCLE
©2009 Third Door Media, Inc.

10. Poll Question 1
• Currently, how are you
attributing conversion credit to
your various ad sources?
1. Last click
2. Other attribution method
©2009 Third Door Media, Inc.

11. Recap of Modeling Framework From
“Measuring the Immeasurable”
www.AttributionManagement.com
©2009 Third Door Media, Inc.

12. CONSUMER DECISIONS
©2009 Third Door Media, Inc.

13. DECISION INFLUENCER
What we know What we don’t know yet
Our Communications Communications
┼Competitor
Search search
┼Paid ┼Consumer
U
┼Banner Ads Site visits to competitors
┼
n
Product trials
┼e-mail ┼
c
Promotions …….
┼Onsite ┼
┼Comparison Shopping e
┼Affiliate ad
r
sources
┼Other
Social Media t
┼
Consumer Search
Word of mouth
┼
a
┼Organic search
Opinion sites
┼
i
┼Site visits to us
Expert opinions
┼
n
Traditional Mass Media
┼
t
y
©2009 Third Door Media, Inc.

14. MODELING CONSUMER DECISIONS
Build a mathematical model to predict consumer decisions
┼
Using data on influencers that we are able to track and measure
┼
Representing data on influencers that we can’t yet track and measure - our
┼
uncertainty - through a statistical distribution
Calibrate the model on observed consumer decisions
┼
Purchase - yes/no
┼
Purchase size - dollar volume, # of units
┼
Repeat purchases
┼
Word of mouth
┼
Etc.
┼
Test the model’s quality by comparing predicted and actual behavior
┼
©2009 Third Door Media, Inc.

15. CONSUMER DECISION MODEL
Consumer’s Decision = f (Our Communications, Consumer Search,
Competitor Communications, Other Sources)
= f ([Paid Search, Banner Ads, e-mail,
Onsite Promotions, Comparison
Shopping, Affiliate ads],
[Organic search, Site visits to us],
[uncertainty])
©2009 Third Door Media, Inc.

16. MEASURING THE EFFECTS OF KNOWN FACTORS?
We assume that each of the known
influencers
has an influence potential
©2009 Third Door Media, Inc.

17. MATHEMATICAL MODEL FOR CONSUMER’S
DECISION
* The β’s are the attribution weights
©2009 Third Door Media, Inc.

18. GETTING THE ATTRIBUTIONS
We calibrate the model on data from the
ClearSaleingplatform
The data includes but is not limited to:
Purchase Path™ data
Record of consumer’s decisions
• Purchase/non-purchase
•Product(s) purchased
• Amount spent
• Repeat visits and purchases
©2009 Third Door Media, Inc.

19. GETTING THE ATTRIBUTIONS
Calibrate the model on the ClearSaleing data
• Find the values of β’s which will help us predict
consumer decisions as accurately as possible
Model is calibrated using:
•Maximum Likelihood
•Bayesian methods
Theβ’s are the attribution weights!
©2009 Third Door Media, Inc.

20. MODELING THE INFLUENCE POTENTIAL
Influence potentialof an influencer = f (# of exposures,
when each of the exposures occurred,
decay rate of the effect of exposures)
©2009 Third Door Media, Inc.

21. Poll Question 2
• What challenges have you run into when trying to build an
attribution model?
1. Our technology cannot track beyond the last ad clicked
2. We cannot build a sound mathematical model
3. We cannot incorporate offline, social media, and word of
mouth advertising
4. All of the above
5. We haven’t tried to build an attribution model
©2009 Third Door Media, Inc.

22. PROGRESS SINCE LAST WEBINAR
1. Selection of businesses for the first round
of model testing
2. Identification of unique influencers
3. Set up the data for calibrating and testing
the model
4. Calibrate and test multiple versions of
the model
©2009 Third Door Media, Inc.

23. SELECTING BUSINESSES
 Selecting businesses that:
•Have a high level of ad spend
•Wide array of advertising sources (paid search,
email, banner, etc)
•We have a least 6 month of data
We have 2+ years of data in some cases
•Seasonal variations
©2009 Third Door Media, Inc.

24. SELECTED BUSINESS VERTICALS
We will be developing and testing the model on nine
businesses in the following verticals
• Retail – web only
• Retail – multi-channel
• Insurance
• Financial Services
©2009 Third Door Media, Inc.

25. PROGRESS SINCE LAST WEBINAR
Identification of unique influencers
©2009 Third Door Media, Inc.

26. INFLUENCER CATEGORIES
 We organized the influencers into the following categories:
•Direct
• Organic Referrers (e.g., Google)
• Paid Search
• Comparison Shopping
•e-mail
• Display advertising
• Affiliate
• Social Media
• Video
©2009 Third Door Media, Inc.

27. UNIQUE INFLUENCERS
Each category was further sub-categorized into a number of
unique influencers:
•Direct - 1
• Organic Referrers – 11 (Google, Yahoo, MSN, etc)
• Paid Search Engine – 11 (Ex: Brand vs. Non-Brand)
• Comparison Shopping – 3 (Ex: Model Number vs. Product Name)
• e-mail - 3 (Ex: Direct Response vs. Brand)
• Display advertising - 4
• Affiliate - 2
• Social Media - 1
• Video - 1
©2009 Third Door Media, Inc.

28. OVERALL
We have:
• 9 categories of influencers
•37 types of unique influencers across the nine
categories
 Our model develops attributions for these 37 unique
influencers across the nine businesses.
©2009 Third Door Media, Inc.

29. PROGRESS SINCE LAST WEBINAR
Set up the data for calibrating
and testing the model
©2009 Third Door Media, Inc.

30. PROGRESS SINCE LAST WEBINAR
135 predictors
64,653 Purchase Paths™
•11,353 paths resulting in a purchase
•53,300 abandoned paths that did not end in a purchase
oA path was defined as abandoned based on some
proprietary criteria
 Model can explain abandonment too
 Another frontier: Attributions for abandonment
©2009 Third Door Media, Inc.

31. RESULTS TO DATE
 To date, we have calibrated over 70 versions of the model
 We plan to calibrate and test the model at least 500 more
times in various forms before firming up our
conclusions
•45,000 models run across the nine data sets
 Testing
• Do the estimated attributions make intuitive sense?
• Is the model able to predict consumer behavior?
o Can it predict purchases?
o Can it predict non-purchases?
©2009 Third Door Media, Inc.

32. RESULTS TO DATE
 Intuitive assessment of attributions
• Findings: not yet firmed up since we have 37 unique
influencers to assess across hundreds of model runs
 Predictive Testing
• 85% or more of the purchases being predicted correctly
• 95% or more of the non-purchases predicted correctly
• Lift charts for calibration and prediction samples
o Top decile indices average between 450 and 500
©2009 Third Door Media, Inc.

33. TYPES OF NON-CLICK/
PASSIVE INFLUENCERS
 Consumer ratings and reviews
 Social networks
 Blogging
 Social commerce
 Instant messaging
 Twitter
 You Tube
 RSS and multiple feeds
©2009 Third Door Media, Inc.

34. ATTRIBUTIONS FOR NON-CLICK INFLUENCERS
 Why do we need to?
• Benefits of including non-click influencers in
attribution models
• Risks of not-including non-click influencers in
attribution models
 Challenges
• How do we include them, if they don’t click and we
don’t track??
©2009 Third Door Media, Inc.

35. VETRA PASSIVE SURVEY™
 Use survey data on the use of non-click influencers
• Statistically infer the likelihood of the use of each type
of passive influencers by different demographic,
lifestyle and psychographic segments
•Vetra is currently working on this approach
oVetra Passive Survey™
•Vetra Passive Survey™ can be used to include passive
influencers in attribution models
©2009 Third Door Media, Inc.

36. VETRA PASSIVE PROPORTIONS™
Monitor sources of passive influence
• YouTube
•Facebook
•Myspace
• epinions.com
Statistically infer the proportions of buyers who engage in
discussions and exchanges regarding products
•Vetra has completed preliminary work on a model for this
inference
Vetra Passive Proportions™ can also be used to include
passive influencers in attribution models
©2009 Third Door Media, Inc.

37. ONGOING RESEARCH ON ATTRIBUTION
AND NEXT STEPS
Vetra and ClearSaleingwill:
•Continue to test a number of attribution models and
influencers
• Analyze the performance of models across different
verticals
• Identify the best attribution models for different verticals
• Expand attribution models to include passive influencers using
oVetra Passive Survey™
oVetra Passive Proportions™
©2009 Third Door Media, Inc.

38. Poll Question 3
• What is your timeframe in switching from a
last click model to an advanced attribution
model?
1. Less than 6 months
2. Within a year
3. Within 2 years
4. More than 2 years
5. No timeframe
©2009 Third Door Media, Inc.

39. ©2009 Third Door Media, Inc.

40. QUESTIONS?
Adam Goldberg- www.attributionmanagement.com
info@attributionmanagement.com
www.ClearSaleing.com
www.searchmarketingnow.com
webcasts@searchmarketingnow.com
Upcoming SMN webcast:May12: Ask iProspect: Strategies &Tactics
for World-Class SEO
May 14: Search Marketing for Small Business:
The Basics for Online Success
©2009 Third Door Media, Inc.