• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
An Introduction to MapReduce 2 and YARN
 

An Introduction to MapReduce 2 and YARN

on

  • 1,707 views

 

Statistics

Views

Total Views
1,707
Views on SlideShare
1,706
Embed Views
1

Actions

Likes
4
Downloads
45
Comments
0

1 Embed 1

http://lanyrd.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    An Introduction to MapReduce 2 and YARN An Introduction to MapReduce 2 and YARN Presentation Transcript

    • An Introduction toMapReduce 2 andYARNTom WhiteApril 25, 2012Seattle Hadoop / Scalability / NoSQL MeetupWednesday, April 25, 2012
    • First, whatʼsMapReduce 1?Wednesday, April 25, 2012
    • Wednesday, April 25, 2012
    • Whatʼs wrong withMR1?Wednesday, April 25, 2012
    • Motivation•Scaling >4000 nodes•HA of Job Tracker•Poor resource utilizationWednesday, April 25, 2012
    • Yet Another Resource NegotiatorWednesday, April 25, 2012
    • Wednesday, April 25, 2012
    • Wednesday, April 25, 2012
    • Node Manageris a generalized Task Tracker• Task Tracker• fixed number of map or reduceslots• Node Manager• containers with variable resourcelimitsWednesday, April 25, 2012
    • Wednesday, April 25, 2012
    • Wednesday, April 25, 2012
    • MR is user spaceYARN is kernelWednesday, April 25, 2012
    • Bonus Apps•Distributed shell•MPI (MAPREDUCE-2911)•Master-worker(MAPREDUCE-3315)•Apache Giraph, HamaWednesday, April 25, 2012
    • Wednesday, April 25, 2012
    • Wednesday, April 25, 2012
    • Old API ≠ MR1New API ≠ MR2Wednesday, April 25, 2012
    • Old APIo.a.h.mapredNew APIo.a.h.mapreduceMR1 ✓ ✓MR2 ✓ ✓Wednesday, April 25, 2012
    • Wednesday, April 25, 2012
    • Try out MR2•Apache Hadoop 0.23.1•CDH4 Beta 2Wednesday, April 25, 2012
    • <dependency><groupId>org.apache.hadoop</groupId><artifactId>hadoop-client</artifactId><version>1.0.2</version></dependency><dependency><groupId>org.apache.hadoop</groupId><artifactId>hadoop-client</artifactId><version>0.23.1</version></dependency>MR1MR2Wednesday, April 25, 2012
    • TODO• Still alpha status• Performance tuning• Usability bug fixes• RM recovery• Security in MR2 not completeWednesday, April 25, 2012
    • Further ReadingWednesday, April 25, 2012
    • Thank You!Wednesday, April 25, 2012