Apache Ambari - What's New in 1.6.1

2,213 views

Published on

Learn about the new features of Apache Ambari 1.6.1.

http://ambari.apache.org/

Published in: Technology
0 Comments
6 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,213
On SlideShare
0
From Embeds
0
Number of Embeds
16
Actions
Shares
0
Downloads
62
Comments
0
Likes
6
Embeds 0
No embeds

No notes for slide

Apache Ambari - What's New in 1.6.1

  1. 1. © Hortonworks Inc. 2014 Apache Ambari 1.6.1 July 2014
  2. 2. © Hortonworks Inc. 2014 What’s New in 1.6.1 • New Host Checks • External Database Connection Testing • Stack Repository Mgmt Enhancements • Performance Improvements • For a complete list of changes https://issues.apache.org/jira/browse/AMBARI/fixforversion/12326545 Page 2
  3. 3. © Hortonworks Inc. 2014 New Host Checks Custom JDK Hostname resolution Reverse hostname Page 3
  4. 4. © Hortonworks Inc. 2014 Background: Current JDK Options • Ambari Server setup provides three JDK options • Use options [1] or [2] for Ambari to automatically download + install the JDK on all hosts –Requires Internet access • Use option [3] to specify an already installed JDK –This is a must when performing a no-Internet / local install –Ambari expects this JDK to already be installed on ALL hosts Page 4
  5. 5. © Hortonworks Inc. 2014 Custom JDK Host Check • Problem: when selecting option [3], if a host did not already have the JDK installed, service and component starts would fail late in the Cluster Install Wizard process –Difficult to troubleshoot • Solution: New “Custom JDK” Host Check performed during Host Registration –Confirms the Custom JDK is installed on each host Page 5
  6. 6. © Hortonworks Inc. 2014 Hostname Resolution Check • Checks that each host can resolve hostname for other hosts in the cluster Page 6
  7. 7. © Hortonworks Inc. 2014 Reverse Hostname Check • On each host, check hostname matches reverse lookup Page 7
  8. 8. © Hortonworks Inc. 2014 External Database Connections • Simplified JDBC Driver Setup • Ability to “test” Database Connections Page 8
  9. 9. © Hortonworks Inc. 2014 Background: External Database Drivers Page 9 • Ambari supports using an external database for Hive Metastore and Oozie • Ambari 1.6.0 (or earlier) experience required user to put the JDBC Driver ON the PLANNED host for Hive Metastore or Oozie Server PRIOR to starting cluster install • Potential chances for error – Wrong host – JDBC Driver in wrong location – Miss this step completely
  10. 10. © Hortonworks Inc. 2014 Simplified JDBC Driver Setup • User provides driver during ambari-server setup… ambari-server setup --jdbc-db=oracle --jdbc-driver=/path/to/odjcb6.jar • During Cluster Install Wizard… –Ambari checks that user provided the driver during setup –Ambari copies the driver to the correct destination host Page 10 AMBARI SERVER Hive Metastore Oozie JDBC Driver JAR
  11. 11. © Hortonworks Inc. 2014 JDBC Driver Setup Reminder Page 11
  12. 12. © Hortonworks Inc. 2014 Test Database Connection • Get JDBC driver from Ambari Server • Distribute to target machine (Oozie or Hive Metastore) • Test the database connection (show success/fail) • Popup warning if user proceeds without testing Page 12 1 2 3
  13. 13. © Hortonworks Inc. 2014 Stack Repository Mgmt HDP and HDP-UTILS managed per Stack Template-driven .repo files Manage Base URLs from Ambari Web Page 13
  14. 14. © Hortonworks Inc. 2014 Background: Repositories Page 14 AMBARI Repo HDP-UTILS Repo HDP Repo AMBARI HDP STACK Ambari Server and Agent Packages Hadoop Stack Packages Stack Utils LZO, ExtJS, Snappy, Nagios, Ganglia
  15. 15. © Hortonworks Inc. 2014 Background: .repo Files Page 15 AMBARI Repo HDP-UTILS Repo HDP Repo AMBARI HDP STACK ambari.repo HDP.repo BUT HDP-UTILS is Stack specific
  16. 16. © Hortonworks Inc. 2014 ambari.repo Changes Page 16 The ambari.repo file no longer includes HDP-UTILS, only AMBARI [root@c6402 ~]# more /etc/yum.repos.d/ambari.repo [AMBARI.1.6.1-1.x] name=Ambari 1.x baseurl=http://public-repo-1.hortonworks.com/ambari/centos6/1.x/updates/1.6.1/ gpgcheck=1 gpgkey=http://public-repo-1.hortonworks.com/ambari/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins enabled=1 priority=1 1
  17. 17. © Hortonworks Inc. 2014 HDP + HDP-UTILS Stack Repositories Page 17 Prompted to provide HDP and HDP-UTILS Stack repository Base URLs2
  18. 18. © Hortonworks Inc. 2014 HDP.repo and HDP-UTILS.repo Page 18 Ambari Server Ambari Agent 4) Server sends HDP.repo & HDP-UTILS.repo 2) Reads Stack Base URL values 6) Agent uses HDP.repo HDP-UTILS.repo when installing packages 1) Agent reports 3) Creates HDP.repo & HDP-UTILS.repo ** 5) Agent puts HDP.repo & HDP-UTILS.repo in place ** Generated from Stack template file
  19. 19. © Hortonworks Inc. 2014 Stack .repo Template File • HDP.repo and HDP-UTILS.repo template file /var/lib/ambari- server/resources/stacks/HDP/2.0.6/hooks/before- INSTALL/templates/repo_suse_rhel.j2 https://github.com/apache/ambari/blob/trunk/ambari- server/src/main/resources/stacks/HDP/2.0.6/hooks/before- INSTALL/templates/repo_suse_rhel.j2 Page 19 Ambari generates the Stack .repo files from a template3
  20. 20. © Hortonworks Inc. 2014 Managing Repositories Page 20 Manage repository Base URLs from Ambari Web4
  21. 21. © Hortonworks Inc. 2014 Performance + Scale Page 21
  22. 22. © Hortonworks Inc. 2014 2,000 Node Clusters • Testing Ambari with 2,000 node clusters • Continued to move Web UI logic to Server • Big focus on Hosts page (filtering, sorting, paging) • Ambari Session at Hadoop Summit 2014 http://www.slideshare.net/Hadoop_Summit/managing-2000-node-cluster-with-ambari http://www.youtube.com/watch?v=sVsHfmZ4s2M Page 22
  23. 23. © Hortonworks Inc. 2014 Learn More Page 23 Resource Location Apache Ambari Project Page http://ambari.apache.org Ambari Project Wiki https://cwiki.apache.org/confluence/display/AMBARI Ambari Project JIRA https://issues.apache.org/jira/browse/AMBARI

×