Intro to Apache Mahout

Machine Learning with Apache Mahout ,[object Object],Grant Ingersoll,[object Object],July 8, 2010,[object Object]

Introduction,[object Object],You,[object Object],Machine learning experience?,[object Object],Business Intelligence?,[object Object],Natural Lang. Processing?,[object Object],ApacheHadoop?,[object Object],Me,[object Object],Co-founder Apache Mahout,[object Object],Apache Lucene/Solr committer,[object Object],Co-founder Lucid Imagination,[object Object]

Topics,[object Object],What is Machine Learning?,[object Object],ML Use Cases,[object Object],What is Mahout?,[object Object],What can I do with it right now?,[object Object],Where’s Mahout headed?,[object Object]

Amazon.com,[object Object],What is Machine Learning?,[object Object],Google News,[object Object]

Really it’s…,[object Object],“Machine Learning is programming computers to optimize a performance criterion using example data or past experience”,[object Object],Intro. To Machine Learning by E. Alpaydin,[object Object],Subset of Artificial Intelligence,[object Object],Lots of related fields:,[object Object],Information Retrieval,[object Object],Stats,[object Object],Biology,[object Object],Linear algebra,[object Object],Many more,[object Object]

Common Use Cases,[object Object],Recommend friends/dates/products,[object Object],Classify content into predefined groups,[object Object],Find similar content based on object properties,[object Object],Find associations/patterns in actions/behaviors,[object Object],Identify key topics in large collections of text,[object Object],Detect anomalies in machine output,[object Object],Ranking search results,[object Object],Others?,[object Object]

Useful Terminology,[object Object],Vectors/Matrices,[object Object],Weights,[object Object],Sparse,[object Object],Dense,[object Object],Norms,[object Object],Features,[object Object],Feature reduction,[object Object],Occurrences and Co-occurrences,[object Object]

Getting Started with ML,[object Object],Get your data,[object Object],Decide on your features per your algorithm,[object Object],Prep the data,[object Object],Different approaches for different algorithms,[object Object],Run your algorithm(s),[object Object],Lather, rinse, repeat,[object Object],Validate your results,[object Object],Smell test, A/B testing, more formal methods,[object Object]

Apache Mahout,[object Object],http://dictionary.reference.com/browse/mahout,[object Object],An Apache Software Foundation project to create scalable machine learning libraries under the Apache Software License,[object Object],http://mahout.apache.org,[object Object],Why Mahout?,[object Object],Many Open Source ML libraries either:,[object Object],Lack Community,[object Object],Lack Documentation and Examples,[object Object],Lack Scalability,[object Object],Lack the Apache License,[object Object],Or are research-oriented,[object Object]

Focus: Machine Learning,[object Object],Applications,[object Object],Examples,[object Object],Recommenders,[object Object],Clustering,[object Object],Classification,[object Object],Freq. Pattern,[object Object],Mining,[object Object],Genetic,[object Object],Math,[object Object],Vectors/Matrices/SVD,[object Object],Utilities,[object Object],Lucene/Vectorizer,[object Object],Collections (primitives),[object Object],Apache Hadoop,[object Object],See http://cwiki.apache.org/confluence/display/MAHOUT/Algorithms,[object Object]

Focus: Scalable,[object Object],Goal: Be as fast and efficient as possible given the intrinsic design of the algorithm,[object Object],Some algorithms won’t scale to massive machine clusters,[object Object],Others fit logically on a Map Reduce framework like Apache Hadoop,[object Object],Still others will needalternative distributed programming models,[object Object],Be pragmatic,[object Object],Most Mahout implementations are Map Reduce enabled,[object Object],(Always a) Work in Progress,[object Object]

Prepare Data from Raw content,[object Object],Data Sources:,[object Object],Lucene integration,[object Object],bin/mahout lucenevector …,[object Object],Document Vectorizer,[object Object],bin/mahout seqdirectory …,[object Object],bin/mahout seq2sparse …,[object Object],Programmatically,[object Object],See the Utils module in Mahout,[object Object],Database,[object Object],File system,[object Object]

Recommendations,[object Object],Extensive framework for collaborative filtering,[object Object],Recommenders,[object Object],User based,[object Object],Item based,[object Object],Online and Offline support,[object Object],Offline can utilize Hadoop,[object Object],Many different Similarity measures,[object Object],Cosine, LLR, Tanimoto, Pearson, others,[object Object]

Clustering,[object Object],Document level,[object Object],Group documents based on a notion of similarity,[object Object],K-Means, Fuzzy K-Means, Dirichlet, Canopy, Mean-Shift,[object Object],Distance Measures,[object Object],Manhattan, Euclidean, other,[object Object],Topic Modeling ,[object Object],Cluster words across documents to identify topics,[object Object],Latent Dirichlet Allocation,[object Object]

Categorization,[object Object],Place new items into predefined categories:,[object Object],Sports, politics, entertainment,[object Object],Mahout has several implementations,[object Object],Naïve Bayes,[object Object],Complementary Naïve Bayes,[object Object],Decision Forests,[object Object],Logistic Regression (Almost done),[object Object]

Freq. Pattern Mining,[object Object],Identify frequently co-occurrent items,[object Object],Useful for:,[object Object],Query Recommendations,[object Object],Apple -> iPhone, orange, OS X,[object Object],Related product placement,[object Object],“Beer and Diapers”,[object Object],Spam Detection,[object Object],Yahoo: http://www.slideshare.net/hadoopusergroup/mail-antispam,[object Object],http://www.amazon.com,[object Object]

Evolutionary,[object Object],Map-Reduce ready fitness functions for genetic programming,[object Object],Integration with Watchmaker,[object Object],http://watchmaker.uncommons.org/index.php,[object Object],Problems solved:,[object Object],Traveling salesman,[object Object],Class discovery,[object Object],Many others,[object Object]

Singular Value Decomposition,[object Object],Reduces a big matrix into a much smaller matrix by amplifying the important parts while removing/reducing the less important parts,[object Object],Mahout has fully distributed Lanczos implementation,[object Object],<MAHOUT_HOME>/bin/mahout svd -Dmapred.input.dir=path/to/corpus --tempDir path/for/svd-output --rank 300 --numColumns <numcols> --numRows <num rows in the input>,[object Object],<MAHOUT_HOME>/bin/mahout cleansvd --eigenInput path/for/svd-output --corpusInput path/to/corpus --output path/for/cleanOutput --maxError 0.1 --minEigenvalue 10.0 ,[object Object],https://cwiki.apache.org/confluence/display/MAHOUT/Dimensional+Reduction,[object Object]

How To: Recommenders,[object Object],Data: ,[object Object],Users (abstract),[object Object],Items (abstract),[object Object],Ratings (optional),[object Object],Load the data model,[object Object],Ask for Recommendations:,[object Object],User-User,[object Object],Item-Item,[object Object]

Ugly Demo I,[object Object],Group Lens Data: http://www.grouplens.org,[object Object],http://lucene.apache.org/mahout/taste.html#demo,[object Object],http://localhost:8080/RecommenderServlet?userID=1&debug=true,[object Object],In other words: the reason why I work on servers, not UIs!,[object Object]

How to: Command Line,[object Object],Most algorithms have a Driver program,[object Object],Shell script in $MAHOUT_HOME/bin helps with most tasks,[object Object],Prepare the Data,[object Object],Different algorithms require different setup,[object Object],Run the algorithm,[object Object],Single Node,[object Object],Hadoop,[object Object],Print out the results,[object Object],Several helper classes: ,[object Object],LDAPrintTopics, ClusterDumper, etc.,[object Object]

Ugly Demo II - Prep,[object Object],Data Set: Reuters,[object Object],http://www.daviddlewis.com/resources/testcollections/reuters21578/,[object Object],Convert to Text via http://www.lucenebootcamp.com/lucene-boot-camp-preclass-training/,[object Object],Convert to Sequence File:,[object Object],bin/mahout seqdirectory –input <PATH> --output <PATH> --charset UTF-8,[object Object],Convert to Sparse Vector:,[object Object],bin/mahout seq2sparse --input <PATH>/content/reuters/seqfiles/ --norm 2 --weight TF --output <PATH>/content/reuters/seqfiles-TF/ --minDF 5 --maxDFPercent 90,[object Object]

Ugly Demo II: Topic Modeling,[object Object],Latent Dirichlet Allocation,[object Object],./mahout lda --input <PATH>/content/reuters/seqfiles-TF/vectors/ --output <PATH>/content/reuters/seqfiles-TF/lda-output --numWords 34000 –numTopics 10,[object Object],./mahout org.apache.mahout.clustering.lda.LDAPrintTopics --input <PATH>/content/reuters/seqfiles-TF/lda-output/state-19 --dict <PATH>/content/reuters/seqfiles-TF/dictionary.file-0 --words 10 --output <PATH>/content/reuters/seqfiles-TF/lda-output/topics --dictionaryTypesequencefile,[object Object],Good feature reduction (stopword removal) required,[object Object]

Ugly Demo III: Clustering,[object Object],K-Means,[object Object],Same Prep as UD II, except use TFIDF weight,[object Object],./mahout kmeans --input <PATH>/content/reuters/seqfiles-TFIDF/vectors/part-00000 --k 15 --output <PATH>/content/reuters/seqfiles-TFIDF/output-kmeans --clusters <PATH>/content/reuters/seqfiles-TFIDF/output-kmeans/clusters,[object Object],Print out the clusters: ./mahout clusterdump --seqFileDir <PATH>/content/reuters/seqfiles-TFIDF/output-kmeans/clusters-15/ --pointsDir <PATH>/content/reuters/seqfiles-TFIDF/output-kmeans/points/ --dictionary <PATH>/content/reuters/seqfiles-TFIDF/dictionary.file-0 --dictionaryTypesequencefile --substring 20,[object Object]

Ugly Demo IV: Frequent Pattern Mining,[object Object],Data: http://fimi.cs.helsinki.fi/data/,[object Object],./mahout fpg -i <PATH>/content/freqitemset/accidents.dat -o patterns -k 50 -method mapreduce -g 10 -regex [],[object Object], ./mahout seqdump --seqFile patterns/fpgrowth/part-r-00000 ,[object Object]

What’s Next?,[object Object],Google Summer of Code (GSOC),[object Object],Restricted Boltzmann Machines,[object Object],SVM,[object Object],EigenCuts (spectral clustering),[object Object],Neural Networks,[object Object],SVD-based Recommender,[object Object],0.4 release in Fall (after GSOC),[object Object],Stabilize API’s for 1.0 release,[object Object],Benchmarking,[object Object],http://cwiki.apache.org/MAHOUT/howtocontribute.html,[object Object]

Resources,[object Object],Slides will be posted at http://lucene.grantingersoll.com,[object Object],Examples at:,[object Object],http://lucene.grantingersoll.com/2010/02/13/intro-to-mahout-slides-and-demo-examples/ ,[object Object],More Examples in Mahout SVN in the examples directory,[object Object]

Resources,[object Object],http://mahout.apache.org,[object Object],http://cwiki.apache.org/MAHOUT,[object Object],{user|dev}@mahout.apache.org,[object Object],http://svn.apache.org/repos/asf/mahout/trunk,[object Object],http://hadoop.apache.org,[object Object]

Resources,[object Object],“Mahout in Action” by Owen and Anil,[object Object],http://lucene.li/E,[object Object],“Taming Text” by Ingersoll, Morton and Farris,[object Object],http://lucene.li/D,[object Object],“Introducing Apache Mahout” ,[object Object],http://lucene.li/F,[object Object],“Programming Collective Intelligence” by Toby Segaran,[object Object],http://lucene.li/G,[object Object],“Data Mining - Practical Machine Learning Tools and Techniques” by Ian H. Witten and Eibe Frank,[object Object],http://lucene.li/H,[object Object]

References,[object Object],HAL: http://en.wikipedia.org/wiki/File:Hal-9000.jpg,[object Object],Terminator: http://en.wikipedia.org/wiki/File:Terminator1984movieposter.jpg,[object Object],Matrix: http://en.wikipedia.org/wiki/File:The_Matrix_Poster.jpg,[object Object],Google News: http://news.google.com,[object Object],Amazon.com: http://www.amazon.com,[object Object],Facebook: http://www.facebook.com,[object Object],Mahout: http://lucene.apache.org/mahout,[object Object],Beer and Diapers: http://www.flickr.com/photos/baubcat/2484459070/,[object Object],http://www.theregister.co.uk/2006/08/15/beer_diapers/,[object Object],DMOZ: http://www.dmoz.org,[object Object]

Intro to Apache Mahout

Intro to Apache Mahout

Recommended

More Related Content

What's hot

What's hot(20)

Similar to Intro to Apache Mahout

Similar to Intro to Apache Mahout(20)

More from Grant Ingersoll

More from Grant Ingersoll(20)

Recently uploaded

Recently uploaded(20)

Intro to Apache Mahout

Editor's Notes