Workshop: Building a Cluster with Stacki - Greg Bruno VP Engineering/ Co-Founder @ StackIQ StackiFest - March 24, 2016

1. Greg Bruno
VP of Engineering

2. Workshop: Building a Cluster with Stacki

3. Datacenter Architecture

4. Frontend Services
Services to build backend nodes
◦ DHCP
◦ TFTP
◦ Named (optional)
Services to access backend nodes
◦ SSH key management
◦ Parallel execution shell

5. Host Configuration Spreadsheet

6. Backend Installation
Save your Host Configuration spreadsheet as a CSV
Import CSV on frontend
◦ “stack load hostfile file=hosts.csv”
Tell backend nodes to install on their next PXE boot
◦ “stack set host boot backend action=install”
PXE boot all backend nodes
Done!

7. BitTorrent-Inspired Package Installation
Stacki

8. Custom Partitioning

9. Storage Partition Spreadsheet

10. Custom Partitioning
We will substitute the “os” pallet with the full CentOS 6.7 distribution
◦ “stack list storage partition”
◦ “stack load storage partition file=partition.csv”
Prep the host for reinstall
◦ “stack list host partition”
◦ “stack remove host partition backend-0-0”
◦ “stack set host attr backend-0-0 attr=nukedisks value=true”
◦ “stack set host boot backend-0-0 action=install”

11. Adding a Pallet

12. Adding CentOS to Default Box
We will substitute the “os” pallet with the full CentOS 6.7 distribution
◦ “stack list pallet”
◦ “stack add pallet CentOS*iso”
◦ “stack enable pallet CentOS”
◦ “stack disable pallet os”

13. Create a New Box with CentOS
Making a new box
◦ “stack list box”
◦ “stack add box centos”
◦ “stack enable pallet CentOS box=centos”
◦ “stack enable pallet stacki box=centos”
Assign a host to a new box
◦ “stack list host”
◦ “stack set host box backend-0-0 box=centos”

14. Boxes
Stacki Pallet
CentOS
CentOS PalletOS Pallet
Stacki Pallet
Default

15. Boxes
OS Pallet
Stacki Pallet
Default
Stacki Pallet
CentOS
CentOS Pallet

16. Why is this hard and important?

17. The “Step 0” Problem
Check namenodes are
empty
Format/start HDFS
Create all directories
Create all metastores
Start services (Hbase, Hive,
Oozie, Sqoop, Impala, etc)
Deploy client configuration Configure database
Setup/assign monitors
(activity, services, and host)
Test database connections
Validate/resolve hostnamesConsistent host timezones
No bad kernel versions
running
(CDH) version consistency
Java version consistency
Daemons versions
consistency
Mgmt Agents versions
consistency
Host specification/SSH
ports
MUCH MORE …
DHCP Server/Client setup TFTP/PXE configuration
Server OS installation
Node OS Install
RAID configuration
Boot configuration
System/data disk
partitioning
Monitoring system setup
and config
Lights Out/IPMI setup
User accounts added and
synced
SSH keys on all hosts
Network node configuration
Config Mgmt install and
configuration
Route configurationOS upgrades/updates
Site specific software and
configuration
Host specification/SSH
ports
Security
Firewall setupCluster Mgmt utility Database install and config
Multiple network configPackage installation MUCH MORE …

18. Clusters are Different
Adding new servers does require coordination
Newly added servers must:
• Have same software stack as original
servers
• Have same configuration as original
servers
• Know about original servers
And, original servers must:
• Know about new servers
Result: The management complexity added to the
Operations staff is “exponential”

19. Exponential Complexity
Number of Servers
ManagementComplexity
General Data Center
Clusters

20. The Pain Curve
Number of Servers
ManagementComplexity
General Data Center
Clusters
PAIN

21. The Pain Threshold
The pain threshold differs for every
organization
Function of:
• cluster(s) size
• number of people in Operations
• Operations staff cluster expertise

22. Moore’s Law

23. Moore’s Law and Infrastructure Value

24. What it Means for You

25. Time is Money
The clock starts ticking when hosts land on your
loading dock
Without your applications online, you have an
paper weight that consumes power, cooling, and
management’s attention