Presentation by Ted Dunning core Mahout commiter, architect at MapR, and author of Mahout in Action at Data Science London 23/05/12

1. Super-Fast Clustering
Report from MapR workshop
©MapR Technologies - Confidential 1

2.  Contact:
– tdunning@maprtech.com
– @ted_dunning
 Twitter for this talk
– #mapr_uk
 Slides and such:
– http://info.mapr.com/ted-uk-05-2012
©MapR Technologies - Confidential 2

3. Company Background
 MapR provides the industry’s best Hadoop Distribution
– Combines the best of the Hadoop community
contributions with significant internally
financed infrastructure development
 Background of Team
– Deep management bench with extensive analytic,
storage, virtualization, and open source experience
– Google, EMC, Cisco, VMWare, Network Appliance, IBM,
Microsoft, Apache Foundation, Aster Data, Brio, ParAccel
 Proven
– MapR used across industries (Financial Services, Media,
Telcom, Health Care, Internet Services, Government)
– Strategic OEM relationship with EMC and Cisco
– Over 1,000 installs
©MapR Technologies - Confidential 3

4. We Also Do …
 Open source development
– Zookeeper
– Hadoop
– Mahout
– Stuff
 Partner workshops
– Machine learning
– Information architecture
– Cluster design
©MapR Technologies - Confidential 4

5. We Also Do …
 Open source development
– Zookeeper
– Hadoop
– Mahout
– Stuff
 Partner workshops
– Machine learning
– Information architecture
– Cluster design
©MapR Technologies - Confidential 5

6. The Problem
 A certain bank
– had lots of customers
– had lots of prospective customers
– had a non-trivial number of fraudulent customers
– had a non-trivial number of fraudulent merchants
 They also
– collected data
– built models
– collected more data
– built more models
©MapR Technologies - Confidential 6

7. But …
 These models were arduous to build
 And hard to test
 So people suggested something simpler
 Like k-nearest neighbor
©MapR Technologies - Confidential 7

8. What’s that?
 Find the k nearest training examples
 Use the average value of the target variable from them
 This is easy … but hard
– easy because it is so conceptually simple and you don’t have knobs to turn
or models to build
– hard because of the stunning amount of math
– also hard because we need top 50,000 results
 Initial prototype was massively too slow
– 3K queries x 200K examples takes hours
– needed 20M x 25M in the same time
©MapR Technologies - Confidential 8

9. What We Did
 Mechanism for extending Mahout Vectors
– DelegatingVector, WeightedVector, Centroid
 Searcher interface
– ProjectionSearch, KmeansSearch, LshSearch, Brute
 Super-fast clustering
– Kmeans, StreamingKmeans
©MapR Technologies - Confidential 9

10. Projection Search
©MapR Technologies - Confidential 10

11. K-means Search
©MapR Technologies - Confidential 11

12. But These Require k-means!
 Need a new k-means algorithm to get speed
 Streaming k-means is
– One pass (through the original data)
– Very fast (20 us per data point with threads)
– Very parallelizable
©MapR Technologies - Confidential 12

13. How It Works
 For each point
– Find approximately nearest centroid (distance = d)
– If d > threshold, new centroid
– Else possibly new cluster
– Else add to nearest centroid
 If centroids > K ~ C log N
– Recursively cluster centroids with higher threshold
 Result is large set of centroids
– these provide approximation of original distribution
– we can cluster centroids to get a close approximation of clustering original
– or we can just use the result directly
©MapR Technologies - Confidential 13

14. Parallel Speedup?
200
Non- threaded
✓
100
2
Tim e per point (μs)
Threaded version
3
50
4
40 6
5
8
30
10 14
12
20 Perfect Scaling 16
10
1 2 3 4 5 20
Threads
©MapR Technologies - Confidential 14

15. Warning, Recursive Descent
 Inner loop requires finding nearest centroid
 With lots of centroids, this is slow
 But wait, we have classes to accelerate that!
©MapR Technologies - Confidential 15

16. Warning, Recursive Descent
 Inner loop requires finding nearest centroid
 With lots of centroids, this is slow
 But wait, we have classes to accelerate that!
(Let’s not use k-means searcher, though)
©MapR Technologies - Confidential 16

17.  Contact:
– tdunning@maprtech.com
– @ted_dunning
 Slides and such:
– http://info.mapr.com/ted-uk-05-2012
©MapR Technologies - Confidential 17

18. Thank You
©MapR Technologies - Confidential 18