Big Data @ Hippo - GetTogether 2014

follow the Hippo trail
Hippo GetTogether 2014
Big Data @ Hippo
Hippo GetTogether 2014 - Trouw
Frank van Lankvelt

Co-occurrence
Relating Attributes

Scary Math

Contingency Table
A not A
B x 20 - x 20
not B 40 - x 140 + x 180
40 160 200
Documents A, B
total # visitors
visitors of B
visitors of A
x
P(x >= 8) ≈ 3%
visitors of A & B

Co-occurrence Insights
Insight: a high cohesion of page visits in the partner section
standing out from the regular ‘.com’ visitor cluster suggests that
visitors looking for a partner go through every single page and
probably can’t find what they’re looking for.
Action: Hippo suggests to improve navigation, search or filtering.
● attribute / url
relatedness
find partner
/fr
.com.org
genericrelease
notes

Recommendations
Alice Bob Charlie
Star Wars 3 4
Finding
Nemo
3 4
Sound of
Music
5 1 2
genre stars
Star Wars sci-fi Portman
Finding
Nemo
animation DeGeneres
Sound of
Music
musical Andrews
user - item (rating)
collaborative filtering
content
(meta) data
which documents are interesting for ME?
find docs similar to visited documents find docs co-occurring with visited documents

Implementation
combine in search index:
Recommendation Query
Content-based:
(meta) data
Collaborative Filtering:
co-occurrence

Recommended For You
1.Collect ID of viewed content
2.Calculate co-occurrences
3.Index, along with content
IDs of co-viewed documents
4.Search with recent IDs, similarity

Patterns
Beyond Co-occurrence

Patterns in the Data
customers that buy diapers often buy beer as well
(young dads rewarding themselves?)

Itemsets Rules
Find the patterns (association rule mining):
1.sets of items that are bought together
P(beer,diapers) > 1%
(support)
1.subsets that are good predictors
> 4 (lift)P(beer,diapers)
P(beer) P(diapers)

http://www.onehippo.com/en/thankyou - Thank You
Beer? Diapers? Conversions!!!

http://www.onehippo.com/en/thankyou
will a visitor go there?
P(conversion|request log)
what are the relevant “signals”?
which configuration performs best?

Patterns For Conversion
single item:
referrer www.google.com
pattern/itemset:
visited demo
2014 week 4
correlations

Scary Data
Structure

1.Build Frequent Prefix Tree
(FPGrowth)
2.Extract patterns relevant for conversion
(using contingencies)
Finding Frequent Itemsets

Pattern Contingency Table
converted not converted
pattern
matches
pattern
does not
match
converted
● visited /thankyou
sample pattern
● visited demo
● in 2014 week 4

Sub-Pattern Filtering
Problem:
when pattern (A, B, C) is relevant, patterns
(A), (B), (C), (A, B), (A, C), (B, C)
(likely) also match. E.g. with C meta-data on page B.
Solution:
test for independence using contingency!

Actionable Insights?
The found itemsets are quite numerous and
seem to contain a lot of redundancy.
But they are certainly interesting, e.g. for a
periodic evaluation.

Personalization
Putting Patterns to Use

Naive A/B Testing
The naive solution:
route some traffic to alternative configuration
A (old config): 80%
B (new config): 20%
run for some time
see if B has relatively more conversions

Problems With Naive Solution
if B is drastically worse,
20% of traffic is LOST
marketer must regularly check and decide
when has a new config PROVEN itself?
number of concurrent experiments is LOW
no user context

Predict Conversion
Conversion rate depends on context:
x the patterns
w the “weights”
ϕ cdf of normal dist.

Experimental Setup
Split data set (.org + .com)
1.training set
189660 visitors, 435 conversions
2.test set
27013 visitors, 40 conversions

Can We Predict Conversion?
1260 itemsets
ROC curve
TPR versus FPR
@ false positive rate 10%
: 96% true positive rate

Towards Actionable Insights
Use
A utomatic
R elevance
D etermination
to prune the patterns
(optimize the prior)
σ
μ
relevant
irrelevant
weights (w)

Top 20 Patterns For Conversion
referer.go.onehippo.com
.pathInfo./resources/whitepapers/forrester-market-
overview-web-content-management-systems.html
.pathInfo./resources/whitepapers/cms---a-critical-
solution-for-todays-ecommerce.html
.pathInfo./resources/whitepapers/hippo-cms-for-the-
enterprise.html
.pathInfo./resources/whitepapers/web-content-
management-in-the-cloud.html
.collectorData.channel.One Hippo English Site
.collectorData.audience.terms.
referer.www.onehippo.com
.collectorData.categories.terms.cms
.pathInfo./mobile-cms
.pathInfo./ressourcen/demo
.pathInfo./resources/videos/hippo-cms-grand-
tour.html
.collectorData.audience.terms.
.collectorData.categories.terms.cms
.pathInfo./ressources/demo
.pathInfo./what_to_buy/compare.html
referer.www.cmswire.com
.pathInfo./resources/demo
.collectorData.categories.terms.mobile
.pathInfo./resources/whitepapers/understanding-hippo-cms-7-
software-architecture.html
.pathInfo./resources/whitepapers/selecting-today’s-
enterprise-web-content-management-system.html
referer.www.google.nl
referer.www.onehippo.com .pathInfo./resources/videos/a-
quick-overview-of-hippo-cms-in-just-under-3-minutes.html
.collectorData.categories.terms.repository
.pathInfo./resources/whitepapers/selecting-today’s-
enterprise-web-content-management-system.html
.collectorData.categories.terms.
.collectorData.categories.terms.relevance

Actionable Insights!
we can find a
small model
that can be used for
human interpretation
and
automated personalization

Product Challenge
KISS
# parameters should be minimal

Parameters
Recommendations
1 hyper-param
Personalization
idem
NICE!

Questions?

Big Data @ Hippo - GetTogether 2014

Recommended

Recommended

More Related Content

Similar to Big Data @ Hippo - GetTogether 2014

Similar to Big Data @ Hippo - GetTogether 2014 (12)

Recently uploaded

Recently uploaded (20)

Big Data @ Hippo - GetTogether 2014