• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Hadoop Release Plan Feb17
 

Hadoop Release Plan Feb17

on

  • 1,797 views

 

Statistics

Views

Total Views
1,797
Views on SlideShare
1,295
Embed Views
502

Actions

Likes
1
Downloads
0
Comments
0

5 Embeds 502

http://developer.yahoo.net 430
http://developer.yahoo.com 63
https://developer.yahoo.com 6
http://feeds.developer.yahoo.net 2
http://static.slidesharecdn.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Fleshed out Context object API include JobClient

Hadoop Release Plan Feb17 Hadoop Release Plan Feb17 Presentation Transcript

  • Hadoop Release Plans February 17th, 2010 Owen O'Malley, Yahoo! Hadoop Team
  • Yahoo and 0.20
    • Yahoo back ported a lot of features from trunk into Yahoo’s 0.20
        • Back port features to support Yahoo needs
        • Keep our branch in Git and publish to GitHub
        • Has been deployed on all 25,000 nodes
        • Improved Capacity scheduler
        • Run tasks as users
        • Currently have Yahoo 0.20.8 deployed and Yahoo 0.20.9 being tested
    • I submitted Apache 0.20.2 rc2 today.
  • The current state of Hadoop 0.21
    • Branched on 19 Sep 2009
    • Has a couple of big ticket items
        • HDFS sync/flush/append
        • Improved MapReduce schedulers
        • Run tasks as user
    • Lots of blockers still open 3 + 5 + 20
    • Some missing back ports
        • libhdfs is still in MapReduce!
    • Lots of grunt work left to do
  • Yahoo and Security
    • Yahoo needs Hadoop Security
        • Finish development this month
        • Deploy first integration cluster in April
        • Deploy to production in August
        • Can’t afford to slip dates
        • Building on Yahoo 0.20 was lower risk
    • Created a new branch Yahoo 0.20.100
    • We will publish to GitHub.
  • Going Forward
    • For Yahoo, Hadoop 0.21 will be very stale by the time it could be deployed.
    • For our customers, moving to 0.21 isn’t worth the cost in incompatibility.
    • Yahoo is planning on jumping straight from 0.20 with security to 0.22.
    • Is anyone outside of Yahoo interested enough in 0.21 to get it released?