Learn about the new features of Apache Ambari 1.6.1. http://ambari.apache.org/

1. © Hortonworks Inc. 2014
Apache Ambari
1.6.1
July 2014

2. © Hortonworks Inc. 2014
What’s New in 1.6.1
• New Host Checks
• External Database Connection Testing
• Stack Repository Mgmt Enhancements
• Performance Improvements
• For a complete list of changes
https://issues.apache.org/jira/browse/AMBARI/fixforversion/12326545
Page 2

3. © Hortonworks Inc. 2014
New Host Checks
Custom JDK
Hostname resolution
Reverse hostname
Page 3

4. © Hortonworks Inc. 2014
Background: Current JDK Options
• Ambari Server setup provides three JDK options
• Use options [1] or [2] for Ambari to automatically
download + install the JDK on all hosts
–Requires Internet access
• Use option [3] to specify an already installed JDK
–This is a must when performing a no-Internet / local install
–Ambari expects this JDK to already be installed on ALL hosts
Page 4

5. © Hortonworks Inc. 2014
Custom JDK Host Check
• Problem: when selecting option [3], if a host did not
already have the JDK installed, service and component
starts would fail late in the Cluster Install Wizard process
–Difficult to troubleshoot
• Solution: New “Custom JDK” Host Check performed
during Host Registration
–Confirms the Custom JDK is installed on each host
Page 5

6. © Hortonworks Inc. 2014
Hostname Resolution Check
• Checks that each host can resolve hostname for other
hosts in the cluster
Page 6

7. © Hortonworks Inc. 2014
Reverse Hostname Check
• On each host, check hostname matches reverse
lookup
Page 7

8. © Hortonworks Inc. 2014
External Database Connections
• Simplified JDBC Driver Setup
• Ability to “test” Database Connections
Page 8

9. © Hortonworks Inc. 2014
Background: External Database Drivers
Page 9
• Ambari supports using an
external database for Hive
Metastore and Oozie
• Ambari 1.6.0 (or earlier)
experience required user to put
the JDBC Driver ON the
PLANNED host for Hive
Metastore or Oozie Server
PRIOR to starting cluster install
• Potential chances for error
– Wrong host
– JDBC Driver in wrong location
– Miss this step completely

10. © Hortonworks Inc. 2014
Simplified JDBC Driver Setup
• User provides driver during ambari-server setup…
ambari-server setup --jdbc-db=oracle --jdbc-driver=/path/to/odjcb6.jar
• During Cluster Install Wizard…
–Ambari checks that user provided the driver during setup
–Ambari copies the driver to the correct destination host
Page 10
AMBARI
SERVER
Hive
Metastore
Oozie
JDBC Driver JAR

11. © Hortonworks Inc. 2014
JDBC Driver Setup Reminder
Page 11

12. © Hortonworks Inc. 2014
Test Database Connection
• Get JDBC driver from Ambari Server
• Distribute to target machine (Oozie or Hive Metastore)
• Test the database connection (show success/fail)
• Popup warning if user proceeds without testing
Page 12
1
2
3

13. © Hortonworks Inc. 2014
Stack Repository Mgmt
HDP and HDP-UTILS managed per Stack
Template-driven .repo files
Manage Base URLs from Ambari Web
Page 13

14. © Hortonworks Inc. 2014
Background: Repositories
Page 14
AMBARI Repo HDP-UTILS Repo
HDP Repo
AMBARI HDP STACK
Ambari Server
and Agent
Packages
Hadoop Stack
Packages
Stack Utils
LZO, ExtJS,
Snappy,
Nagios, Ganglia

15. © Hortonworks Inc. 2014
Background: .repo Files
Page 15
AMBARI Repo HDP-UTILS Repo
HDP Repo
AMBARI HDP STACK
ambari.repo
HDP.repo
BUT
HDP-UTILS is
Stack specific

16. © Hortonworks Inc. 2014
ambari.repo Changes
Page 16
The ambari.repo file no longer includes HDP-UTILS, only AMBARI
[root@c6402 ~]# more /etc/yum.repos.d/ambari.repo
[AMBARI.1.6.1-1.x]
name=Ambari 1.x
baseurl=http://public-repo-1.hortonworks.com/ambari/centos6/1.x/updates/1.6.1/
gpgcheck=1
gpgkey=http://public-repo-1.hortonworks.com/ambari/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
1

17. © Hortonworks Inc. 2014
HDP + HDP-UTILS Stack Repositories
Page 17
Prompted to provide HDP and HDP-UTILS Stack repository Base URLs2

18. © Hortonworks Inc. 2014
HDP.repo and HDP-UTILS.repo
Page 18
Ambari Server Ambari Agent
4) Server sends HDP.repo &
HDP-UTILS.repo
2) Reads Stack Base URL
values
6) Agent uses HDP.repo
HDP-UTILS.repo when
installing packages
1) Agent reports
3) Creates HDP.repo &
HDP-UTILS.repo **
5) Agent puts HDP.repo &
HDP-UTILS.repo
in place
** Generated from Stack template file

19. © Hortonworks Inc. 2014
Stack .repo Template File
• HDP.repo and HDP-UTILS.repo template file
/var/lib/ambari-
server/resources/stacks/HDP/2.0.6/hooks/before-
INSTALL/templates/repo_suse_rhel.j2
https://github.com/apache/ambari/blob/trunk/ambari-
server/src/main/resources/stacks/HDP/2.0.6/hooks/before-
INSTALL/templates/repo_suse_rhel.j2
Page 19
Ambari generates the Stack .repo files from a template3

20. © Hortonworks Inc. 2014
Managing Repositories
Page 20
Manage repository Base URLs from Ambari Web4

21. © Hortonworks Inc. 2014
Performance + Scale
Page 21

22. © Hortonworks Inc. 2014
2,000 Node Clusters
• Testing Ambari with 2,000 node clusters
• Continued to move Web UI logic to Server
• Big focus on Hosts page (filtering, sorting, paging)
• Ambari Session at Hadoop Summit 2014
http://www.slideshare.net/Hadoop_Summit/managing-2000-node-cluster-with-ambari
http://www.youtube.com/watch?v=sVsHfmZ4s2M
Page 22

23. © Hortonworks Inc. 2014
Learn More
Page 23
Resource Location
Apache Ambari Project Page http://ambari.apache.org
Ambari Project Wiki https://cwiki.apache.org/confluence/display/AMBARI
Ambari Project JIRA https://issues.apache.org/jira/browse/AMBARI