Hhheeeeeee

1. Finding maximum temperature of each
city in a state using MapReduce

2. If you are familiar with creating a new project and adding external jar
files, then do it and start from slide number

3. Click on File ->New -> Java Project.

4. Enter project name and click Next>

5. Click on libraries.

6. Click on Add External JARs.

7. Click on File System in the Places pane-> open usr folder.

8. Open lib folder.

9. Open Hadoop folder.

10. Select all the jar files and click ok.

11. Click on Add External JARs again.

12. Open client folder.

13. Select all JAR files and click ok.

14. Under your project folder, right click on src->select new->select class.

15. Enter the name of mapper class and click finish.

16. Similarly create reducer class. ( right click on src->new->class, then enter reducer class name)

17. Similarly create driver class.

19. import java.io.IOException;
import java.util.StringTokenizer;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Mapper;
public class Map
extends Mapper<LongWritable, Text, Text, IntWritable>{
private IntWritable max = new IntWritable();
private Text word = new Text();
@Override
protected void map(LongWritable key, Text value, Context context)
throws IOException, InterruptedException {
StringTokenizer line = new StringTokenizer(value.toString(),",t");
word.set(line.nextToken());
max.set(Integer.parseInt(line.nextToken()));
context.write(word,max);
}
}
Mapper Program

21. import java.io.IOException;
import java.util.Iterator;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Reducer;
public class Reduce
extends Reducer<Text, IntWritable, Text, IntWritable>{
private int max_temp = Integer.MIN_VALUE;
private int temp = 0;
@Override
protected void reduce(Text key, Iterable<IntWritable> values,
Context context)
throws IOException, InterruptedException {
Iterator<IntWritable> itr = values.iterator();
temp = 0;
max_temp = Integer.MIN_VALUE;
while (itr.hasNext()) {
temp = itr.next().get();
if( temp > max_temp)
{
max_temp = temp;
}
}
context.write(key, new IntWritable(max_temp));
}
}
Reducer Program

23. import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
public class MaxTempDriver {
public static void main(String[] args) throws Exception {
// Create a new job
Job job = new Job();
// Set job name to locate it in the distributed environment
job.setJarByClass(MaxTempDriver.class);
job.setJobName("Max Temperature");
// Set input and output Path, note that we use the default input format
// which is TextInputFormat (each record is a line of input)
FileInputFormat.addInputPath(job, new Path(args[0]));
FileOutputFormat.setOutputPath(job, new Path(args[1]));
// Set Mapper and Reducer class
job.setMapperClass(Map.class);
job.setCombinerClass(Reduce.class);
job.setReducerClass(Reduce.class);
// Set Output key and value
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(IntWritable.class);
System.exit(job.waitForCompletion(true) ? 0 : 1);
}
}
Driver Program

24. After saving all 3 programs -> right click on your project in package explorer pane and click on export.

25. Under Java -> select JAR file –> click Next>

26. Click on Browse

27. Enter Name and select the location where you wish to store JAR file.

28. Click on finish.

29. Open terminal and Use ls command to check if the jar file is present.

30. Create a text file and enter sample data into the text file. Press ctrl +Z when u have finished entering the
data.

31. Create a directory in hdfs.

32. Move the txt file into hdfs .

33. Check if the file is moved properly by displaying the file contents.

34. Now give the txt file store in HDFS as input to the MapReduce program.
( /output is the directory in which the output will be stored.)
hadoop jar /home/cloudera/MaxTemp.jar MaxTemp /maxtemp_ip/MaxTemp_Data.txt /maxtemp_op

36. Check the contents of the output directory. It will contain a file named “part-r-00000” which contains the
output of the program.
Display the contents of that file to check the output obtained.

37. Thank You