Mahout and Recommendations

1©MapR Technologies 2013- Confidential
Introduction to Mahout
And How To Build a Recommender

Me, Us
 Ted Dunning, Chief Application Architect, MapR
Committer PMC member, Mahout, Zookeeper, Drill
Bought the beer at the first HUG
 MapR
Distributes more open source components for Hadoop
Adds major technology for performance, HA, industry standard API’s
 Tonight
Hash tag - #dfwbd #mapr
See also - @ApacheMahout @ApacheDrill
@ted_dunning and @mapR

Requested Topic For Tonight
 What is Mahout?
 What makes it different?
 How can big data technology solve impossible problems?
 How is big data affecting the world?

Also
 What is MapR?
 What is MapR doing?
 How does MapR’s technology work?
 How are customers making use of MapR?
 How can anyone make use of MapR to solve problems?

Oh … Also This
 Detailed break-down of a live machine learning system running
with Mahout on MapR
 With code examples

I may have to
summarize

I may have to
summarize
just a bit

Part 1:
5 minutes of math

Part 2:
12 minutes: I want a pony

Part 3:
A working example

What Does Machine Learning Look Like?

What Does Machine Learning Look Like?
A1 A2
é
ë
ù
û
T
A1 A2
é
ë
ù
û=
A1
T
A2
T
é
ë
ê
ê
ù
û
ú
ú
A1 A2
é
ë
ù
û
=
A1
T
A1 A1
T
A2
AT
2A1 AT
2A2
é
ë
ê
ê
ù
û
ú
ú
r1
r2
é
ë
ê
ê
ù
û
ú
ú
=
A1
T
A1 A1
T
A2
AT
2A1 AT
2A2
é
ë
ê
ê
ù
û
ú
ú
h1
h2
é
ë
ê
ê
ù
û
ú
ú
r1 = A1
T
A1 A1
T
A2
é
ëê
ù
ûú
h1
h2
é
ë
ê
ê
ù
û
ú
ú
O(κ k d + k3 d) = O(k2 d log n + k3 d) for small k,
high quality
O(κ d log k) or O(d log κ log k) for larger k,
looser quality
But tonight we’re going to show you how to keep it simple yet powerful…

Comparison of Three Main ML Topics
 Recommendation:
– Involves observation of interactions between people taking action (users)
and items for input data to the recommender model
– Goal is to suggest additional appropriate or desirable interactions
– Applications include: movie, music or map-based restaurant choices;
suggesting sale items for e-stores or via cash-register receipts

Part 1:
A bit of math
(the math of bits)

Mahout Math
 Goals are
– basic linear algebra,
– and statistical sampling,
– and good clustering,
– decent speed,
– extensibility,
– especially for sparse data
 But not
– totally badass speed
– comprehensive set of algorithms
– optimization, root finders, quadrature

Matrices and Vectors
 At the core:
– DenseVector, RandomAccessSparseVector
– DenseMatrix, SparseRowMatrix
 Highly composable API
 Important ideas:
– view*, assign and aggregate
– iteration
m.viewDiagonal().assign(v)

Assign? View?
 Why assign?
– Copying is the major cost for naïve matrix packages
– In-place operations critical to reasonable performance
– Many kinds of updates required, so functional style very helpful
 Why view?
– In-place operations often required for blocks, rows, columns or diagonals
– With views, we need #assign + #views methods
– Without views, we need #assign x #views methods
 Synergies
– With both views and assign, many loops become single line

Examples
A =a
A =aB+ b
double alpha; a.assign(alpha);
a.assign(b, Functions.chain(
Functions.plus(beta),
Functions.times(alpha));

More Examples
 The trace of a matrix
 Set diagonal to zero
 Set diagonal to negative of row sums

Examples
m.viewDiagonal().zSum()

Examples
m.viewDiagonal().assign(0)

Examples
 Set diagonal to negative of row sums excluding the diagonal
m.viewDiagonal().assign(0)
Vector diag = m.viewDiagonal().assign(0);
diag.assign(m.rowSums().assign(Functions.MINUS));

Clustering and Such
 Streaming k-means and ball k-means
– streaming reduces very large data to a cluster sketch
– ball k-means is a high quality k-means implementation
– the cluster sketch is also usable for other applications
– single machine threaded and map-reduce versions available
 SVD and friends
– stochastic SVD has in-memory, single machine out-of-core and map-reduce
versions
– good for reducing very large sparse matrices to tall skinny dense ones
 Spectral clustering
– based on SVD, allows massive dimensional clustering

Mahout Math Summary
 Matrices, Vectors
– views
– in-place assignment
– aggregations
– iterations
 Functions
– lots built-in
– cooperate with sparse vector optimizations
 Sampling
– abstract samplers
– samplers as functions
 Other stuff … clustering, SVD

Part 2:
How recommenders work
(I still want a pony)

Recommendations
Behavior of a
crowd helps us
understand what
individuals will do

Recommendations
Alice got an apple and a
puppy
Charles got a bicycle
Alice
Charles

Recommendations
Alice got an apple and a
puppy
Charles got a bicycle
Bob got an apple
Alice
Bob
Charles

Recommendations
What else would Bob like??
Alice
Bob
Charles

Recommendations
What if everybody gets a
pony?
Now what does Bob want?
?
Alice
Bob
Charles

Log Files
Alice
Bob
Charles
Alice
Bob
Charles
Alice

Log Files
u1
u3
u2
u1
u3
u2
u1
t1
t2
t3
t4
t3
t3
t1

Log Files and Dimensions
u1
u3
u2
u1
u3
u2
u1
t1
t2
t3
t4
t3
t3
t1
t1
t2
t3
t4
Things
u1 Alice
Bob
Charles
u3
u2
Users

History Matrix
Alice
Bob
Charles
✔ ✔ ✔
✔ ✔
✔ ✔

Cooccurrence Matrix
1 2
1 1
1
1
2 1

Indicator Matrix
✔

Indicator Matrix
✔
id: t4
title: puppy
desc: The sweetest little puppy ever.
keywords: puppy, dog, pet
indicators: (t1)

Problems with Raw Cooccurrence
 Very popular items co-occur with everything
– Welcome document
– Elevator music
 That isn’t interesting
– We want anomalous cooccurrence

Recommendation Basics
 Coocurrence
t3 not t3
t1 2 1
not t1 1 1

Spot the Anomaly
 Root LLR is roughly like standard deviations
A not A
B 13 1000
not B 1000 100,000
A not A
B 1 0
not B 0 2
A not A
B 1 0
not B 0 10,000
A not A
B 10 0
not B 0 100,000
0.44 0.98
2.26 7.15

A Quick Simplification
 Users who do h (a vector of things a user has done)
 Also do r
Ah
AT
Ah( )
AT
A( )h
User-centric recommendations
(transpose translates back to things)
Item-centric recommendations
(change the order of operations)
A translates things into users

Symmetry Gives Cross Recommentations
AT
A( )h
BT
A( )h
Conventional recommendations
with off-line learning
Cross recommendations

For example
 Users enter queries (A)
– (actor = user, item=query)
 Users view videos (B)
– (actor = user, item=video)
 ATA gives query recommendation
– “did you mean to ask for”
 BTB gives video recommendation
– “you might like these videos”

The punch-line
 BTA recommends videos in response to a query
– (isn’t that a search engine?)
– (not quite, it doesn’t look at content or meta-data)

Real-life example
 Query: “Paco de Lucia”
 Conventional meta-data search results:
– “hombres del paco” times 400
– not much else
 Recommendation based search:
– Flamenco guitar and dancers
– Spanish and classical guitar
– Van Halen doing a classical/flamenco riff

Real-life example

Hypothetical Example
 Want a navigational ontology?
 Just put labels on a web page with traffic
– This gives A = users x label clicks
 Remember viewing history
– This gives B = users x items
 Cross recommend
– B’A = label to item mapping
 After several users click, results are whatever users think they
should be

Nice. But we
can do better?

Ausers
things

A1 A2
é
ë
ù
û
users
thing
type 1
thing
type 2

A1 A2
é
ë
ù
û
T
A1 A2
é
ë
ù
û=
A1
T
A2
T
é
ë
ê
ê
ù
û
ú
ú
A1 A2
é
ë
ù
û
=
A1
T
A1 A1
T
A2
AT
2A1 AT
2A2
é
ë
ê
ê
ù
û
ú
ú
r1
r2
é
ë
ê
ê
ù
û
ú
ú
=
A1
T
A1 A1
T
A2
AT
2A1 AT
2A2
é
ë
ê
ê
ù
û
ú
ú
h1
h2
é
ë
ê
ê
ù
û
ú
ú
r1 = A1
T
A1 A1
T
A2
é
ëê
ù
ûú
h1
h2
é
ë
ê
ê
ù
û
ú
ú

Part 3:
What about that worked
example?

Metrics and
logs (5)
Cooccurrence
analysis (7)
Post to
search
engine (8)
Search
engine (4)
Presentation
tier (2)
User behavior
generator (1)
Session
collector
(3)
History collector
(6)
Diagnostic
browsing (9)
http://bit.ly/18vbbaT

SolR
Indexer
SolR
Indexer
Solr
indexing
Cooccurrence
(Mahout)
Item meta-
data
Index
shards
Complete
history
Analyze with Map-Reduce

SolR
Indexer
SolR
Indexer
Solr
search
Web tier
Item meta-
data
Index
shards
User
history
Deploy with Conventional Search System

Objective Results
 At a very large credit card company
 History is all transactions
 Development time to minimal viable product about 4 months
 General release 2-3 months later
 Search-based recs at or equal in quality to other techniques

Summary
 Input: Multiple kinds of behavior on one set of things
 Output: Recommendations for one kind of behavior with a
different set of things
 Cross recommendation is a special case

Objective Results
 At a very large credit card company
 History is all transactions
 Development time to minimal viable product about 4 months
 General release 2-3 months later
 Search-based recs at or equal in quality to other techniques

Me, Us
 Ted Dunning, Chief Application Architect, MapR
Committer PMC member, Mahout, Zookeeper, Drill
Bought the beer at the first HUG
tdunning@{apache.org,maprtech.com} ted.dunning@gmail.com
 MapR
Distributes more open source components for Hadoop
Adds major technology for performance, HA, industry standard API’s
 Tonight
Hash tag - #dfwbd #mapr
See also - @ApacheMahout @ApacheDrill
@ted_dunning and @mapR

Mahout and Recommendations

More Related Content

What's hot

Viewers also liked

Similar to Mahout and Recommendations

More from Ted Dunning

Recently uploaded

Mahout and Recommendations

Editor's Notes