The document outlines how to run an Elastic MapReduce job on AWS. It discusses uploading input data and a MapReduce jar file to S3 storage. It then shows how to create an EMR Hadoop cluster with a specified number of workers and hardware configuration. Finally, it demonstrates submitting a MapReduce job to the cluster, monitoring its status, and terminating the cluster once complete.

1. ELASTIC MAP REDUCE OF AWS
Rim Moussa
ENI-Carthage
University of Carthage
2017/2018
1

2. DEMO OUTLINE
 Simple Storage Service: S3
 Job
 Jar: Map Reduce code
 Input: input data files
 Output: output data files
 All data must be on S3 including jar and input data
 Create Hadoop Cluster
 Size: number of workers
 Hardware configuration
 Stat the job
2

3. S3
3

4. CREATE A BUCKET FOR INPUT DATA ON S3
4

5. CREATE A NEW S3 BUCKET
5

6. UPLOAD DATA INTO THE BUCKET
6

7. UPLOAD IN PROGRESS
7

8. UPLOAD JAR INTO S3
8

9. CREATE EMR HADOOP CLUSTER
9

10. 10

11. 11

12. 12

13. 13

14. CREATE A NEW KEY PAIR
14
Click here

15. 15

16. DOWNLOAD NEW KEY AND CHANGE ACCESS RIGHTS
16

17. CLUSTER PROVISIONING
17

18. HARDWARE TAB
18

19. BOOTSTRAPPING
19

20. CONNECT TO MASTER USING SSH
20

21. RESIZE A CLUSTER
21
Add or drop instances

22. INSTANCES ARE RUNNING
22

23. SUBMIT A MAPREDUCE JOB
23

24. JOB DETAILS
24

25. JOB IS RUNNING
25

26. 26

27. 27

28. /! TERMINATE CLUSTER
28

29. FIND OUT
29
 S3
 How to upload data from a terminal to S3
 Scenario where data is some where in the net
 Hadoop Master
 Compile the job on the master
 Submit the job from a terminal on the master
 Performance Tuning
 Hadoop cluster configuration
 RAM allocated to each Mapper, Reducer
 Data Compression
 Code
 Input Split Size
 Adjust the number of reducers