Hadoop World 2011: Preview of the New Cloudera Management Suite - Phil Zeyliger & Henry Robinson, Cloudera

1,370 views

Published on

This session will preview what is new in the latest release of the Cloudera Management Suite. We will cover the common problems we've seen in Hadoop management and will do a demonstration of several new features designed to address these problems.

Published in: Technology, Design
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,370
On SlideShare
0
From Embeds
0
Number of Embeds
195
Actions
Shares
0
Downloads
0
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Hadoop World 2011: Preview of the New Cloudera Management Suite - Phil Zeyliger & Henry Robinson, Cloudera

  1. 1. November 8, 2011Preview of Cloudera ManagerHenry Robinson, Vinithra Varadharajan, Philip ZeyligerSoftware Engineers
  2. 2. Live from NY, it’s Hadoop Debugging Live!
  3. 3. (intentionally left blank)
  4. 4. Managing Services and Configuration Manage lifecycle of your Hadoop cluster just like any other system in your data center
  5. 5. What Cloudera Manager Does...• Assists installation• Manages configuration• Supervises processes• Executes workflows• Searches logs• Escalates events• Monitors CDH
  6. 6. SCM Express• Easiest, fastest way to install CDH• Available for freehttp://www.cloudera.com/products-services/scm-express
  7. 7. Adding hosts to your cluster...
  8. 8. Installing packages...
  9. 9. Inspecting host details...
  10. 10. Inspecting host details... Processes
  11. 11. Inspecting host details... Roles Processes
  12. 12. Inspecting host details... Host Info Roles Processes
  13. 13. Services
  14. 14. Service Selection
  15. 15. “Casting”
  16. 16. Service Config Recommendations
  17. 17. Service Actions
  18. 18. Service Actions Generic
  19. 19. Service Actions Generic Specific
  20. 20. Roles
  21. 21. Role Actions
  22. 22. Configuration Management 35% of Cloudera support tickets are due to misconfiguration of Hadoop clusters
  23. 23. Configuration Management
  24. 24. Config Search
  25. 25. Annotating Config Changes
  26. 26. Configuration History
  27. 27. Configuration Diff/Revert
  28. 28. Multiple Users
  29. 29. Audit Trail
  30. 30. Monitoring • Ganglia, Nagios, Hyperic, etc.?25 ©2011 Cloudera, Inc. All Rights Reserved.
  31. 31. It’s nice to know your config...26 ©2011 Cloudera, Inc. All Rights Reserved.
  32. 32. What else is there?27 ©2011 Cloudera, Inc. All Rights Reserved.
  33. 33. What else is there? Current Status27 ©2011 Cloudera, Inc. All Rights Reserved.
  34. 34. What else is there? Current Status Evaluated Health Tests27 ©2011 Cloudera, Inc. All Rights Reserved.
  35. 35. What else is there? Current Status Summaries of Daemons Evaluated Health Tests27 ©2011 Cloudera, Inc. All Rights Reserved.
  36. 36. What else is there? Current Status Summaries of Daemons Performance Graphs Evaluated Health Tests27 ©2011 Cloudera, Inc. All Rights Reserved.
  37. 37. Canaries28 ©2011 Cloudera, Inc. All Rights Reserved.
  38. 38. Time Travel!29 ©2011 Cloudera, Inc. All Rights Reserved.
  39. 39. Log Search
  40. 40. Alerts! • Emails for important events. SE H BA WN DO31 ©2011 Cloudera, Inc. All Rights Reserved.
  41. 41. Monitoring Activities32 ©2011 Cloudera, Inc. All Rights Reserved.
  42. 42. Monitoring Activities Jobs run in this period32 ©2011 Cloudera, Inc. All Rights Reserved.
  43. 43. Monitoring Activities Cluster- Jobs wide and per-job graphs. run in this period32 ©2011 Cloudera, Inc. All Rights Reserved.
  44. 44. Monitoring Activities Search, Filters Cluster- Jobs wide and per-job graphs. run in this period32 ©2011 Cloudera, Inc. All Rights Reserved.
  45. 45. Task Distribution33 ©2011 Cloudera, Inc. All Rights Reserved.
  46. 46. Task Distribution Look for skew and underperforming nodes33 ©2011 Cloudera, Inc. All Rights Reserved.
  47. 47. Reports34 ©2011 Cloudera, Inc. All Rights Reserved.
  48. 48. Reports HDFS & MR Usage34 ©2011 Cloudera, Inc. All Rights Reserved.
  49. 49. Closing thoughts... • Managing a distributed system has unique challenges; tool up! • Cloudera Manager eases learning curve; cuts down on preventable errors. • Configuration, Process Supervision, and Monitoring for Hadoop, all in one piece of software. • Thanks! & Questions?35 ©2011 Cloudera, Inc. All Rights Reserved.
  50. 50. How’s it work? • Central server to hold configuration data. • Agents heartbeat to server to find out what to do. • Agents send monitoring data to monitoring sinks.36 ©2011 Cloudera, Inc. All Rights Reserved.
  51. 51. Services

×