Tom White discusses release plans for Apache Hadoop 0.21, which includes fixing blockers, creating build artifacts, testing, voting, and later bug fix releases. The 0.21.0 release is aimed at improving quality and should not be used for production. It features a new MapReduce API, file context symlinks, and hundreds of bug fixes and other improvements.

2. Apache Hadoop 0.21
Release Plans
Tom White
tomwhite@apache.org
19 May 2010

3. Avoiding the quality death spiral
Solaris went through a very interesting transition. Prior to Solaris 2.5, there was much
more of a, for lack of a better word, waterfall model in terms of the way new releases
were dispersed to people. As a result, people would not run the latest bits on their
desktop or on the server; they would develop their own little bits and integrate them
into a whole that they never saw. Solaris was in the quality death spiral because once
people refused to use the latest stuf because it was known to be broken, then people
used the latest stuf less and less and it got to be more and more broken.
To break the quality death spiral, you’ve got to force people to use the latest stuf. I
think it’s much more important when you’re in a distributed environment where you
don’t necessarily have the kind of immediate peer pressure to do that.
--Bryan Cantrill, http://queue.acm.org/detail.cfm?id=1413258

4. Release History

5. Source Compatibility

7. Process
▪ Fix blockers
– http://bit.ly/common21blockers, hdfs21blockers, mapreduce21blockers
▪ Create build artifacts
– First post split release
▪ Test
– http://wiki.apache.org/hadoop/ReleaseTesting
▪ Vote and release
▪ Later 0.21.x bug fx releases for critical bugs or regressions

8. How to help out
▪ Fix blockers
▪ Try compiling your code against 0.21
▪ Try out release candidates on your (test) cluster
▪ Try out releases on your (test) cluster
▪ Caveat emptor: 0.21.0 should not be used for production

9. What's in it?
▪ New, comprehensive MapReduce API
▪ Symlinks FileContext API (although not integrated with
MapReduce)
▪ New shufe
▪ Hundreds of bug fxes and other improvements
– https://svn.apache.org/repos/asf/hadoop/{common,hdfs,mapreduce}/branches/branch-
0.21/CHANGES.txt
▪ Not security (0.22 at end of year)

10. (c) 2008 Cloudera, Inc. or its licensors. "Cloudera" is a registered trademark of Cloudera, Inc.. All rights reserved. 1.0