Presentation by StackIQ's Director of Open Source Engineering, Joe Kaiser, at StackiFest 2017. Step 1 of every Hadoop vendor’s documentation reads something like this: “First install a cluster.” Without a consistent group of installed machines, a Hadoop installation is prone to failure. Open source Stacki installs machines to a ping and a prompt enabling the consistency and configuration required for a functioning Hadoop installation. StackIQ released a new open source Hortonworks bridge pallet to enable the installation of Hortonworks through the Ambari appliance at the beginning of 2017. In this presentation, Joe will show you how to set-up Stacki, the HDP Bridge pallet, Ambari, and then install Hadoop on a running cluster.

1. Stacki Lab
Joe Kaiser
Director of Open Source Engineering

2. Open Source Stack Installer
Stacki is a very fast and ultra reliable Linux server provisioning tool … at scale.
With zero prerequisites for taking systems from bare metal to a ping and prompt.

3. Stuff it does
 Installs to bare metal or VMs that look like bare metal
 Kickstart based
 Parallel formatting of disk
 Parallel Sharing of RPMs
 CentOS/RHEL
 Networking
 Multiple subnets, vlaning, bonding.
 Storage
 Controller config
 Partitioning
Whatever you can do in Linux, you can do with Stacki only clustered

4. Stacki and Hortonworks Data Platform

5. Hortonworks – stacki-hdp-bridge pallet
 Add/enable/run
 Stacki creates ambari appliance
 Download software as pallets (isos)
 Add/enable HDP and Ambari
 Run gethdp script
 Or download (links in documentation)
 Assign a node to be Ambari deployment.
 Do partitions and preps backend nodes for Hadoop.
 Install all.
 Go to Ambari interface to deploy Hadoop.
 Current versions: (updated /export/HDP/hdp.cfg for new versions)
 distribution = 2.x
 os = centos7
 ambari = 2.4.2.0
 hdp = 2.5.3.0

6. Augment the “default” Box
1) Replace the “os” pallet with the “CentOS” and “CentOS-
Updates” pallets
2) Add the Hortonworks pallets: “HDP”, “HDP-UTILS”,
“Updates-ambari” or run /export/HDP/gethdp.py
3) Add a pallet to glue the two layers together: “stacki-
hdp-bridge”

7. Specify One Host as the “Ambari” appliance
“backend-0-0” will be automatically configured with Ambari
Install the node and then point your web browser at it

8. Reinstall All Backend Nodes
Wipe all hardware disk array configuration and rebuild all the LUNs:
◦ stack set host attr ambari backend attr=nukecontroller value=true
Remove all partitions then repartition and reformat the disks:
◦ stack set host attr ambari backend attr=nukedisks value=true
Instruct nodes to install on next PXE boot:
◦ stack set host boot ambari backend action=install

9. Try It
 Website
www.stacki.com
 Vagrant tire kick
https://github.com/rfkrocktk/vagrant-stacki
Source Code and docs
github.com/stackiq/stacki
github.com/StackIQ/stacki-hdp-bridge
Slack Channel (because everyone)
Google Groups
groups.google.com/forum/#!forum/stacki

10. Finis
Thanks