This document discusses machine learning and the use of MapReduce and Apache Mahout for scalable machine learning. It provides an introduction to machine learning concepts like supervised and unsupervised learning. It then describes how MapReduce can be used to parallelize machine learning algorithms across large datasets. Finally, it discusses Apache Mahout, an open source machine learning library that provides implementations of popular algorithms optimized for MapReduce.

1. MapReduce for Machine
Learning
by
pranya prabhakar
S4 MCA
05

2. CONTENTS
 Introduction
 Machine Learning
 MapReduce
 ML on MapReduce
 Apache mahout and its installation
steps
 Conclusion

3. Introduction
• Data increasing rapidly
• It is necessary to process and to analyze the
data
• Analyzing the data by machine as a human
being. …Different

4. Machine Learning
 Supervised Learning:
Generate a function based upon assigned
labels that maps inputs to desired outputs.
 Unsupervised Learning:
Looks for patterns native to a dataset, and
models it like clustering (e.g. Data mining
&knowledge discovery).
 Reinforcement Learning:
Learns how to act given reward(or
punishment) from the world.

7. Types of problems
 Classification:
data is labeled means it assigned a class
- Learn a model from a manually classified data
- Predict the class of a new object based on its
features and the learned model
e.g.: spam/non-spam, fraud/non-fraud
 Clustering
data is not labelled,but can be divided into groups based on
similarity
- Group similar looking objects
- Notion of similarity: Distance measure:
eg:organizing pictures by faces without names.
 Regression
 Data is labeled with real value rather than a label
eg:time series data like the price of a stock over time.

8. Supervised Learning
Algorithms
 Decision Trees
 k-Nearest Neighbours
 Naive Bayes
 Logistic Regression
 Perceptron and Multi-level
Perceptions
 Neural Networks
 SVM and Kernel estimation

9. Unsupervised Learning
Algorithms
 Clustering
◦ k-Means, MinHash, Hierarchical
Clustering
 Hidden Markov Models
 Feature Extraction methods
 Self-organizing Maps (Neural Nets)

10. uses
 Spam filtering
 Credit card Fraud detection
 Face recognition(computer vision)
 Speech understanding
 Medical diagnosis
and so on…

11. Current state of ML libraries
 Lack scalability
 Lack documentations and examples
 Lack Apache licensing
 Are not well tested
 Are Research oriented
 Not built over existing production
quality libraries
 Lack “Deployability”

12. MapReduce
 It’s a programming framework
 Used for parallel processing over large
data sets
 Application divided into small
fragments of works and distributed
across the cluster
 Computation unit of Hadoop
 Two functions: Map() and Reduce()

13. Apache mahout
 The starting place for MapReduce-
based machine learning
 A disparate collection of algorithms for
 Recommendation
 Clustering
 Classification
 Frequency item Mining

14. Mahout installation
 Prerequisites
java
Hadoop
maven
 Java installation
1. sudo apt-get install sun java jdk
2. sudo gedit .bashrc
set JAVA_HOME in .bashrc file
 Installation of maven
1. sudo apt-get install maven2
2. open .bashrc and add the lines
############## Apache-Maven #########
export M2_HOME=/usr/local/apache-maven-3.0.4
export M2=$M2_HOME/bin
export PATH=$M2:$PATH
export JAVA_HOME=$HOME/programs/jdk

15. Contd..
 Run mvn --version to verify that it is
correctly installed.

16.  Hadoop installation
single node hadoop cluster has been set up as how java
installed
 Installation of Mahout
1. http://www.apache.org/dyn/closer.cgi/lucene/mahout/
2. Create a folder and move the download file to the created directory
say, mkdir usr/local/mahout
3.Mvn install..it shows as

19. Example showing 20news
group’s database

20. Application of Mahout
 Collaborative Filtering
Matrix factorization based recommenders
A user based Recommender
 Clustering
Canopy Clustering
K-Means Clustering
Fuzzy K-Means
Affinity Propagation Clustering
 Classification
Naive Bayes

21. Conclusion
 By using the mapReduce framework,
we could parallelize a wide range of
machine learning algorithms and
apache mahout provide s a platform
for machine learning in mapReduce
paradigm.