The document summarizes the key features and capabilities of Cloudera Manager, a tool for managing Hadoop clusters. Cloudera Manager assists with installation, manages configuration, supervises processes, executes workflows, searches logs, escalates events, and monitors the Cloudera Distribution including Hadoop. It helps install and manage the lifecycle of Hadoop clusters like any other system. It provides configuration management, monitoring, alerting and log search capabilities to ease management of distributed Hadoop clusters.

1. November 8, 2011
Preview of Cloudera Manager
Henry Robinson, Vinithra Varadharajan, Philip Zeyliger
Software Engineers

2. Live from NY,
it’s Hadoop Debugging Live!

3. (intentionally left blank)

4. Managing Services and Configuration
Manage lifecycle of your Hadoop
cluster just like any other system in
your data center

5. What Cloudera Manager Does...
• Assists installation
• Manages configuration
• Supervises processes
• Executes workflows
• Searches logs
• Escalates events
• Monitors CDH

6. SCM Express
• Easiest, fastest way to install CDH
• Available for free
http://www.cloudera.com/products-services/scm-express

7. Adding hosts to your cluster...

8. Installing packages...

9. Inspecting host details...

10. Inspecting host details...
Processes

11. Inspecting host details...
Roles
Processes

12. Inspecting host details...
Host
Info
Roles
Processes

13. Services

14. Service Selection

15. “Casting”

16. Service Config Recommendations

17. Service Actions

18. Service Actions
Generic

19. Service Actions
Generic
Specific

20. Roles

21. Role Actions

22. Configuration Management
35% of Cloudera
support tickets are due
to misconfiguration of
Hadoop clusters

23. Configuration Management

24. Config Search

25. Annotating Config Changes

26. Configuration History

27. Configuration Diff/Revert

28. Multiple Users

29. Audit Trail

30. Monitoring
• Ganglia, Nagios, Hyperic, etc.?
25
©2011 Cloudera, Inc. All Rights Reserved.

31. It’s nice to know your config...
26
©2011 Cloudera, Inc. All Rights Reserved.

32. What else is there?
27
©2011 Cloudera, Inc. All Rights Reserved.

33. What else is there?
Current
Status
27
©2011 Cloudera, Inc. All Rights Reserved.

34. What else is there?
Current
Status
Evaluated
Health Tests
27
©2011 Cloudera, Inc. All Rights Reserved.

35. What else is there?
Current
Status
Summaries
of Daemons
Evaluated
Health Tests
27
©2011 Cloudera, Inc. All Rights Reserved.

36. What else is there?
Current
Status
Summaries
of Daemons
Performance
Graphs
Evaluated
Health Tests
27
©2011 Cloudera, Inc. All Rights Reserved.

37. Canaries
28
©2011 Cloudera, Inc. All Rights Reserved.

38. Time Travel!
29
©2011 Cloudera, Inc. All Rights Reserved.

39. Log Search

40. Alerts!
• Emails for
important
events.
SE
H BA
WN
DO
31
©2011 Cloudera, Inc. All Rights Reserved.

41. Monitoring Activities
32
©2011 Cloudera, Inc. All Rights Reserved.

42. Monitoring Activities
Jobs
run in this period
32
©2011 Cloudera, Inc. All Rights Reserved.

43. Monitoring Activities
Cluster-
Jobs
wide and per-job graphs.
run in this period
32
©2011 Cloudera, Inc. All Rights Reserved.

44. Monitoring Activities
Search,
Filters
Cluster-
Jobs
wide and per-job graphs.
run in this period
32
©2011 Cloudera, Inc. All Rights Reserved.

45. Task Distribution
33
©2011 Cloudera, Inc. All Rights Reserved.

46. Task Distribution
Look
for skew and
underperforming
nodes
33
©2011 Cloudera, Inc. All Rights Reserved.

47. Reports
34
©2011 Cloudera, Inc. All Rights Reserved.

48. Reports
HDFS
& MR Usage
34
©2011 Cloudera, Inc. All Rights Reserved.

49. Closing thoughts...
• Managing a distributed system has unique
challenges; tool up!
• Cloudera Manager eases learning curve;
cuts down on preventable errors.
• Configuration, Process Supervision, and
Monitoring for Hadoop, all in one piece of
software.
• Thanks! & Questions?
35
©2011 Cloudera, Inc. All Rights Reserved.

50. How’s it work?
• Central server to hold configuration data.
• Agents heartbeat to server to find out what
to do.
• Agents send monitoring data to monitoring
sinks.
36
©2011 Cloudera, Inc. All Rights Reserved.

51. Services