Help! My Hadoop doesn't work!


Published on

How to report a bug in apache hadoop -and when not to

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Help! My Hadoop doesn't work!

  1. 1. Help! My Hadoop doesn't work! Steve Loughran– Hortonworks stevel at @steveloughran January 2014 © Hortonworks Inc. 2014
  2. 2. Reporting bugs in Apache Hadoop TM © Hortonworks Inc. 2014 TM Page 2
  3. 3. Terminology JIRA An issue (bug or feature request) in the JIRA database, referenced by its ID, i.e. HADOOP-8545 Feature Request A proposal for adding something new to the codebase. Implicit promise to implement the feature. Bug A flaw in the Hadoop codebase which causes it to not behave as planned –in a well configured cluster. © Hortonworks Inc. Page 3
  4. 4. Is it a Bug? Probably Not • Look at your own code • Look at the logs and any stack traces • Follow any URLs in the exception messages • Search for the exception messages • Commercial Big Data Stacks: use their tools, forums, support services • Before filing: ask on Invalid issues will be closed: © Hortonworks Inc. Page 4
  5. 5. Tools • Hadoop Documentation, Javadocs, Wiki • Books • Network tools: ping, telnet, nslookup, ifconfig • Browsers & search engines • Stack traces • kill -QUIT • Hadoop source code • Dev tool chain: svn, mvn, git, patch, JDK © Hortonworks Inc. Page 5
  6. 6. Be able to start a Minecraft server before trying to bring up a Hadoop cluster © Hortonworks Inc. 2014 Page 6
  7. 7. Apache JIRA © Hortonworks Inc. Page 7
  8. 8. DEMO 1: Bad JIRA HADOOP-10226 © Hortonworks Inc. 2014 Page 8
  9. 9. Don't • File “critical” issue when it is just your server not working. • File the same issue repeatedly. • File issues against non ASF-releases • File bug reports against old releases • Include confidential information in emails or bug reports (including stack traces). • Send “help!” emails to development lists or developers • Send those “help!” emails every 15 minutes © Hortonworks Inc. Page 9
  10. 10. DEMO 2: Good JIRA HADOOP-10227 © Hortonworks Inc. 2014 Page 10
  11. 11. Special Case: Security Bugs Report security bugs to These get fixed before being announced Kerberos not liking you is not a bug © Hortonworks Inc. Page 11
  12. 12. What happens to an issue? • People who care about it will fix it when they have time. • You are probably the person who cares the most. • You may be the only person who can replicate it • You are the best placed person to fix it. • No fix will ship until a new major/minor Hadoop release. • So come up with a workaround! © Hortonworks Inc. Page 12
  13. 13. Summary • Apache JIRA is for reporting bugs in ASF code • Search for the problem on the web & in JIRA first. • Ask for help via user@hadoop or support channels • Replicate in most recent release • Create an informative JIRA Then one of 1. Wait 2. Provide a fix with tests Either way: a workaround is the only timely solution © Hortonworks Inc. Page 14
  14. 14. How do I fix a bug? To follow… © Hortonworks Inc. 2014 Page 15