Ambari Meetup: Ambari Futures


Published on

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Ambari Meetup: Ambari Futures

  1. 1. AmbariFuturesApril 2, 2013Jeff Sposetti @ Hortonworks
  2. 2. Ambari Futures Make Hadoop easier to manage Cluster Operations Job Diagnostics Extensible Platform Extend core capabilities to Enable insight into job Expose integration and include the critical tasks performance and reduce the customization points so Hadoop associated with provisioning burden on specialized Hadoop can interoperate with existing and operating Hadoop clusters. skills and knowledge. operational tooling. 2
  3. 3. What’s Planned for 1.3.0• Improved Configuration Mgmt with host-level overrides• HBase Multi-Master + Heatmaps• Multi-tenancy support via Capacity Scheduler• More Databases (MySQL and Oracle)• Centralized Stack Upgrade• Manage Kerberos Secure Cluster• External group mappings (LDAP/AD)• Job Diagnostics Improvements – New Visualizations (swimlane, scatter plots)
  4. 4. Configuration Mgmt + Host Exceptions• Allows you to handle “heterogeneous” clusters – For example: “Not all my DataNodes are the same”• Set configuration property exceptions for one or more hosts P ag e
  5. 5. Capacity Scheduler UI• Basic UI for adding and configuring queues• Set scheduler properties and see capacity allocations P ag e
  6. 6. More Databases• Ambari to support Postgres, MySQL or Oracle• Configure Hive and Oozie to use MySQL or Oracle P ag e
  7. 7. HBase• New Heatmaps specific to HBase Region Servers• Ability to designate Multiple Hbase Masters P ag e
  8. 8. Other Goodies• Add slaves components to hosts• Stop/Start All Services• Re-assign Master Components• Host status filtering P ag e
  9. 9. Job Diagnostics• Enhanced swimlane visualizations• See job DAG with task overlay• See task scatter plot across jobs P ag e
  10. 10. Beyond 1.3.0• Rack awareness• Log Aggregation• HDFS Rebalance• HBase Compaction• Manage Full Stack HA• Finer-grain Ambari user roles• Customizable Metric Graphs and Heatmaps• Capacity Scheduler Usage Reporting• HDFS Mirroring• Ambari Server HA 10
  11. 11. Stack Definitions• Design Goals – Ambari should be able to support choice of Hadoop stacks – Ambari should enable adding new components to a stack• Define which Services are available (services)• Define where to get the packages (repos) repos Stack A services S S S S S repos Stack B services S S S S 11
  12. 12. Stack Inheritance• Provide ability to “extend a stack”• Define clear wrapper around services to have consistent mgmt interface and configuration controls repos Stack C extends Stack A S S S S S S services + 12
  13. 13. APIs APIs and More APIs• Consistent front-end REST API• Service Provider plugin architecture “ZERO TOUCH” INSTALLS Blueprint Ambari Cluster “LIGHTS OUT” WORKFLOWS Alarm Decom Inform “BRING MY OWN” SCENARIOS Custom Provider 13
  14. 14. Cluster Blueprints• Perform “Headless Install”• Export blueprint from cluster Ambari Blueprint• Boot wizard with blueprint Server• Save wizard as blueprint MANIFEST <host> BLUEPRINT <meta> <stack> <host> <service> <component> <config> <configs> <configs> <config> 14
  15. 15. Thank You