Machine Learning with Apache Mahout

http://twitter.com/danielglauser http://www.linkedin.com/in/danglauser danglauser@gmail.com

What is Machine Learning?

What is Machine Learning?A branch of Artiﬁcial Intelligence

What is Machine Learning?A branch of Artiﬁcial IntelligenceCreative use of statistics

What is Machine Learning?A branch of Artiﬁcial IntelligenceCreative use of statisticsSmart decisions from large data sets

What is Machine Learning?A branch of Artiﬁcial IntelligenceCreative use of statisticsSmart decisions from large data setsAll of the above

Common Applications

Common Applications ?

SpamFiltering

CreditCardFraud

Medical Diagnostics

Search Engines

Sentiment Analysis

Math AlertIf you want to go big with Machine Learningmath is necessaryWhat math?

Statistics Discrete Math Linear Algebra Probability

Apache MahoutA platform for Machine LearningRoll your own algorithm, use the platformEasy integration with Hadoop

History• 2005 The Taste framework• 2008 Services built on Lucene

Mahout is composed of...Recommender EnginesClassiﬁcationClusteringFrequent itemsets

A brief intro to:Recommender EnginesClassiﬁcationClustering

RecommendationsFor a given set of input, make arecommendation

RecommendationsRank the best out of many possibilities

Recommenders are typicallyUser basedorItem based

NeighborhoodNearest N Users Threshold

Similarity

PearsonCorrelationSimilarity Produces a value between 1 and -1 Tendency of two series to move together

PearsonCorrelationSimilarity 1 - the two series are similar 0 - no similarity -1 - opposite similarity

PearsonCorrelationSimilarity Problems Doesn’t take into account how many items overlap between users Cannot ﬁnd similarity between two users if they only have one item in common Undeﬁned if two users have identical preferences

Similarity AlgorithmsPersonCorrelationSimilarityEuclidianDistanceSimilarityTanimotoCoefﬁcientSimilarityLogLikelyhoodSimilarity

To the code!

How big is a Java Object?

GenericPreferenceuser id - long - 8 bytesitem id - long - 8 bytespreference value - ﬂoat - 4 bytes

PreferenceArrayWhy not just use an array or an ArrayList?A little overhead x millions of items = a *lot* of overhead

GenericUserPreferenceArray item id - long - 8 bytes preference value - ﬂoat - 4 bytes ] x millions - one user id - long - 8 bytes

Clustering

ClusteringSurface naturally occurring groups of dataA notion of similarity (and dissimilarity)

ClusteringAlgorithms do not require trainingStopping condition - iterate until closeenough

Common Clustering AlgorithmsK-MeansFuzzy K-MeansMeanshiftCentroid generationDirechlet clustering

Representing DataFeature SelectionVectorization

Feature SelectionFigure out what features of your data areinteresting

VectorizationRepresent the interesting features in an n-dimensional space

N-Dimensional SpaceEvery word in a group of documentsSize, shape, color of an object

Representing VectorsDenseVectorRandomAccessSparseVectorSequentialAccessSparseVector

Representing VectorsDenseVector Random SeekRandomAccessSparseVectorSequentialAccessSparseVector

Hadoop SequenceFilesInput vectors SequenceFile(s) Initial SequenceFile(s) Centoids

K-Means50+ years old, in commonly used for 25 yearsSet the number of clusters - kWorks well even if you don’t pick a good - k

K-MeansGuess at initial placement of the centers (centroids) ]-Expectation - assign the nearest Wash,points to each centroid rinse,Maximization - reposition the centroid repeat

C1C2 C3

C1 C1C2 C3 C3 C2

C1 C3C2

C1 C1 C3 C3C2 C2

C1 C3C2

C1 C1 C3 C3 C2C2

C1 C3C2

C1 C1 C3 C3C2 C2

Stop! C1 C3C2

Clustering

Classiﬁcation

ClassiﬁcationBFF39D 577335 B3E631 D0F5B0 90B073 AFCF3C

ClassiﬁcationBFF39D 577335 B3E631 D0F5B0 90B073 AFCF3C Green

Attributes of Classiﬁcation Algorithms Require training (supervised) Make a single decision with a very limited set of outcomes

ClassiﬁcationTypical answers naturally ﬁt into categories

Examples of Classiﬁcation Credit card fraud prediction Customer attrition Diabetes detector Search Engine

Training - learned process that produces a modelModel - output of the training algorithm

Predictor variable - input for classiﬁcation modelTarget variable - what we are trying to predict

Classiﬁcation

Common AlgorithmsStochastic Gradient Decent (SGD)Support Vector Machine (SVM)Naive BayesComplementary Naive BayesRandom Forrest

Going Distributed

OverheadParallel processing requires managementoverheadEspecially when spread over multiple machines

Vector SequenceFile Keys Values Implements Implements WritableComparable Writeable

JavaWritableComparable Comparable Writeable Serializable

RecommenderRank large datasets

ClusteringGroup your data

ClassiﬁcationTrain me to think like you

Integration with HadoopThrough SequenceFiles and Map/Reduce jobs

Resources

n-dimensional space http://en.wikipedia.org/wiki/File:Coord_system_CA_0.svgBatman http://www.flickr.com/photos/farukahmet/3005752670/sizes/l/in/photostream/duke http://kenai.com/projects/duke/pages/Homemahout logo http://mahout.apache.org/scalability diagram http://manning.com/owen/ Thanks!classification diagram http://manning.com/owen/phew http://www.flickr.com/photos/iain/1022210850/clouds http://www.flickr.com/photos/spazzo_1493/3682989696/spam http://www.flickr.com/photos/johotravels/4334224546/credit card http://www.flickr.com/photos/thetruthabout/4542026865/medical diagnostics http://www.flickr.com/photos/adrianclarkmbbs/3063516728/search engines http://www.flickr.com/photos/enda/144377951/angry http://www.flickr.com/photos/jmgasalla/3467458535/crystal ball http://www.flickr.com/photos/mache/142561526/glasses http://www.flickr.com/photos/nickwheeleroz/2220008689/coffee http://www.flickr.com/photos/mr_t_in_dc/2818254382/

http://twitter.com/danielglauser http://www.linkedin.com/in/danglauser danglauser@gmail.com

http://www.scoop.it	826
http://www.sozialpapier.com	19
http://localhost	19
http://sozialpapier.com	5
http://us-w1.rockmelt.com	3
http://tweetedtimes.com	2
http://www.linkedin.com	2
http://pinterest.com	1

Machine Learning with Apache Mahout

by Daniel Glauser on Feb 16, 2012

Accessibility

Categories

Upload Details

Usage Rights

8 Embeds 877

Statistics

Machine Learning with Apache Mahout Presentation Transcript