Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING
www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING
Agenda for today’s Session
 MapReduce Way
 Class...
www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING
MapReduce Example on Word Count Process
www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING
MapReduce Way
www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING
MapReduce Way – Word Count Process
www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING
Input/Output Classes in MapReduce
www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING
Input Format – Class Hierarchy
www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING
Output Format – Class Hierarchy
www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING
Packages and Classes in Word Count
MapReduce Examp...
www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING
Packages to Import
import java.io.IOException;
imp...
www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING
Mapper Class
public static class Map extends
Mappe...
www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING
Reducer Class
public static class Reduce extends
R...
www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING
Its Time to see some MapReduce Examples
www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING
MapReduce is useful in a wide range of application...
www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING
Let us see few MapReduce Examples
on Analytics
www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING
MapReduce Temperature Example
www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING
Weather Forecasting
 Problem Statement:
» Analysi...
www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING
Temperature Example - Weather Dataset
6th Column
M...
www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING
MapReduce Example
www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING
Last.fm Example
is an online music website where u...
www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING
Let us see a MapReduce Example
on Testing
www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING
MRUnit Testing Framework
 Provides 4 drivers for ...
www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING
MapReduce MRUnit Example
www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING
Learning Resources
 Hadoop Tutorial: www.edureka....
www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING
Thank You …
Questions/Queries/Feedback
Upcoming SlideShare
Loading in …5
×

MapReduce Example | MapReduce Programming | Hadoop MapReduce Tutorial | Edureka

1,355 views

Published on

This Hadoop tutorial on MapReduce Example ( Mapreduce Tutorial Blog Series: https://goo.gl/w0on2G ) will help you understand how to write a MapReduce program in Java. You will also get to see multiple mapreduce examples on Analytics and Testing.

Check our complete Hadoop playlist here: https://goo.gl/ExJdZs

Below are the topics covered in this tutorial:

1) MapReduce Way
2) Classes and Packages in MapReduce
3) Explanation of a Complete MapReduce Program
4) MapReduce Examples on Analytics
5) MapReduce Example on Testing - MRUnit

Published in: Technology
  • Be the first to comment

MapReduce Example | MapReduce Programming | Hadoop MapReduce Tutorial | Edureka

  1. 1. www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING
  2. 2. www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING Agenda for today’s Session  MapReduce Way  Classes and Packages in MapReduce  Explanation of a Complete MapReduce Program  MapReduce Examples on Analytics  MapReduce Example on Testing
  3. 3. www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING MapReduce Example on Word Count Process
  4. 4. www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING MapReduce Way
  5. 5. www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING MapReduce Way – Word Count Process
  6. 6. www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING Input/Output Classes in MapReduce
  7. 7. www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING Input Format – Class Hierarchy
  8. 8. www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING Output Format – Class Hierarchy
  9. 9. www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING Packages and Classes in Word Count MapReduce Example
  10. 10. www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING Packages to Import import java.io.IOException; import java.util.*; import org.apache.hadoop.fs.Path; import org.apache.hadoop.conf.*; import org.apache.hadoop.io.*; import org.apache.hadoop.mapreduce.*; import org.apache.hadoop.mapreduce.lib.input.FileInputFormat; import org.apache.hadoop.mapreduce.lib.input.TextInputFormat; import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat; import org.apache.hadoop.mapreduce.lib.output.TextOutputFormat; All these packages are present in hadoop-common.jar All these packages are present in hadoop-mapreduce- client-core.jar
  11. 11. www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING Mapper Class public static class Map extends Mapper<LongWritable, Text, Text, IntWritable> { Name of the Mapper Class which inherits Super Class Mapper Mapper Class takes 4 Arguments i.e. Mapper<KEYIN, VALUEIN, KEYOUT, VALUEOUT>
  12. 12. www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING Reducer Class public static class Reduce extends Reducer<Text, IntWritable, Text, IntWritable> { Name of the Reducer Class which inherits Super Class Reducer Reducer Class takes 4 Arguments i.e. Reducer <KEYIN, VALUEIN, KEYOUT, VALUEOUT>
  13. 13. www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING Its Time to see some MapReduce Examples
  14. 14. www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING MapReduce is useful in a wide range of applications in multiple domains. It is majorly used for 2 things:  Analytics: Process the data and give the desired results  Testing: Perform few test cases using MRUnit
  15. 15. www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING Let us see few MapReduce Examples on Analytics
  16. 16. www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING MapReduce Temperature Example
  17. 17. www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING Weather Forecasting  Problem Statement: » Analysing weather data of Austin to determine Hot and Cold Days. We have weather data set of Austin by NCIE. NOAA's National Centres for Environmental Information (NCEI) (previously NCDC) is responsible for preserving, monitoring, assessing, and providing public access to the Nation's treasure of climate and historical weather data and information. Refer -> ftp://ftp.ncdc.noaa.gov/pub/data/uscrn/products/daily01 Temperature Example
  18. 18. www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING Temperature Example - Weather Dataset 6th Column Max Temp 6th Column Min Temp
  19. 19. www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING MapReduce Example
  20. 20. www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING Last.fm Example is an online music website where users listen to various tracks, the data gets collected like shown below. Write a map reduce program to get the Number of unique listeners. The data is coming in log files and looks like as shown below: UserId TrackId Shared Radio Skip 100001 150 1 1 0 100005 103 0 0 1 100142 78 1 0 0 110005 289 1 0 1
  21. 21. www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING Let us see a MapReduce Example on Testing
  22. 22. www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING MRUnit Testing Framework  Provides 4 drivers for separately testing MapReduce code » MapDriver » ReduceDriver » MapReduceDriver » PipelineMapReduceDriver  Helps in filling the gap between MapReduce programs and JUnit*  Better control on log messages with JUnit Integration *JUnit is a simple framework to write repeatable tests.
  23. 23. www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING MapReduce MRUnit Example
  24. 24. www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING Learning Resources  Hadoop Tutorial: www.edureka.co/blog/hadoop-tutorial  MapReduce Tutorial: www.edureka.co/blog/mapreduce-tutorial  MapReduce Interview Questions: www.edureka.co/blog/interview-questions/hadoop-interview-questions-mapreduce
  25. 25. www.edureka.co/big-data-and-hadoopEDUREKA HADOOP CERTIFICATION TRAINING Thank You … Questions/Queries/Feedback

×