0-DAY DEPLOYMENT
UK VMUG
A little about myself
• Maish Saidel-Keesing - @maishsk
• Blogger – Technodrone
(http://technodrone.blogspot.com)
• Platform Architect – Cisco, Israel
• Co-Author of VMware vSphere
Design
•

vExpert, VCAP5-DCD/DCA blah blah blah …..
We will cover…
• The problem …

• The Journey …
• Why was it so difficult?

• Tools of the trade …
Things you won’t get ( Today)
• An End-to-End Solution

• Code Samples (unless they are
already public).
• The pot at the end of the rainbow.
The Typical Datacenter #1
• Mine is better
The Typical Datacenter #2
• What do they all have in common?

• Existing Infrastructure
• Storage

• Network
• Virtual Infrastructure
What if that wasn’t
the case?
VMware vCenter Server

Manage
Manage

And all you had
wasVMware vSphere
this ??

VMware vSphere

VMware vSphere

Docklandsboy (flickr)
First THE PROBLEM…..
• Install Hardware

• Servers
• Network
• Storage
• Virtualization Stack
• Operating Systems
• Applications
……
• ~0 margin for error

• Easy to implement in the field
• Rinse and repeat (100’s of times)
This is what we got.
What is a must for Automation?
What comes first?

Jasonunbound (flickr)
Uplink Switches and FI’s
UCS configuration
• Houston we have an API!

• Cisco UCS and VMware Integration
(http://bit.ly/1bFIDdT)
Hypervisor

Power On Server

• Kickstart

Boot from CDROM/PXE

• USB Key

Provide Kickstart
File

Engineer
Engineer

ESXi Kickstart installation

• Deployment appliance (Razor)
Static IP’s + Hostnames
• External CSV file

• Plant the hostname
after kickstart in the host
• Set static IP
Deploy vCenter
• 2 predefined OVF’s
• vCenter (http://bit.ly/18Osy1l)
• Oracle Database
• No VCSA
• External Oracle DB
• Was not production ready until
vSphere 5.5

vSphere Client

Deploy vCenter and vCenterDB
Images

vSphere Client

Configure IP
vCenter VM

Engineer
Engineer

Engineer
Engineer

vSphere Client

Powershell

Engineer
Engineer

Configure IP
vCenterDB VM

Setup VMware environment
How to connect the ESXi hosts
to vCenter?
• vCenter deployment script

• PS:> PowerCLI
• Kickstart

• Puppet vCenter Module
Dude where’s my Storage?
• Standard Arrays (NetApp/EMC)

• Predefined scripts for configuration
• IP’s

• LUNs / Volumes / NFS
The VM’s ….
• Clone / Deploy ?

• Razor – handoff to Puppet Broker

aarongilson (flickr)
(Nickapedia)
Deploy Applications
• Tags were already there (Razor)

• Active / Passive Nodes
• Puppet Modules
Tools
• How To Deal With a Complex
Project – (http://bit.ly/1hWoy8t)
• PowerShell/bash/Python/…

• Razor
• Puppet
Looking forward

• Handling the Workflow
• Wrapping this in a
normal interface
• Questions???

0-day deployment (UKVMUG)

  • 1.
  • 2.
    A little aboutmyself • Maish Saidel-Keesing - @maishsk • Blogger – Technodrone (http://technodrone.blogspot.com) • Platform Architect – Cisco, Israel • Co-Author of VMware vSphere Design • vExpert, VCAP5-DCD/DCA blah blah blah …..
  • 3.
    We will cover… •The problem … • The Journey … • Why was it so difficult? • Tools of the trade …
  • 4.
    Things you won’tget ( Today) • An End-to-End Solution • Code Samples (unless they are already public). • The pot at the end of the rainbow.
  • 5.
    The Typical Datacenter#1 • Mine is better
  • 6.
    The Typical Datacenter#2 • What do they all have in common? • Existing Infrastructure • Storage • Network • Virtual Infrastructure
  • 7.
    What if thatwasn’t the case? VMware vCenter Server Manage Manage And all you had wasVMware vSphere this ?? VMware vSphere VMware vSphere Docklandsboy (flickr)
  • 8.
    First THE PROBLEM….. •Install Hardware • Servers • Network • Storage • Virtualization Stack • Operating Systems • Applications
  • 9.
    …… • ~0 marginfor error • Easy to implement in the field • Rinse and repeat (100’s of times)
  • 10.
    This is whatwe got.
  • 11.
    What is amust for Automation?
  • 12.
  • 13.
  • 14.
    UCS configuration • Houstonwe have an API! • Cisco UCS and VMware Integration (http://bit.ly/1bFIDdT)
  • 15.
    Hypervisor Power On Server •Kickstart Boot from CDROM/PXE • USB Key Provide Kickstart File Engineer Engineer ESXi Kickstart installation • Deployment appliance (Razor)
  • 16.
    Static IP’s +Hostnames • External CSV file • Plant the hostname after kickstart in the host • Set static IP
  • 17.
    Deploy vCenter • 2predefined OVF’s • vCenter (http://bit.ly/18Osy1l) • Oracle Database • No VCSA • External Oracle DB • Was not production ready until vSphere 5.5 vSphere Client Deploy vCenter and vCenterDB Images vSphere Client Configure IP vCenter VM Engineer Engineer Engineer Engineer vSphere Client Powershell Engineer Engineer Configure IP vCenterDB VM Setup VMware environment
  • 18.
    How to connectthe ESXi hosts to vCenter? • vCenter deployment script • PS:> PowerCLI • Kickstart • Puppet vCenter Module
  • 19.
    Dude where’s myStorage? • Standard Arrays (NetApp/EMC) • Predefined scripts for configuration • IP’s • LUNs / Volumes / NFS
  • 20.
    The VM’s …. •Clone / Deploy ? • Razor – handoff to Puppet Broker aarongilson (flickr) (Nickapedia)
  • 21.
    Deploy Applications • Tagswere already there (Razor) • Active / Passive Nodes • Puppet Modules
  • 22.
    Tools • How ToDeal With a Complex Project – (http://bit.ly/1hWoy8t) • PowerShell/bash/Python/… • Razor • Puppet
  • 23.
    Looking forward • Handlingthe Workflow • Wrapping this in a normal interface
  • 24.

Editor's Notes

  • #5 As close as I may be to god – the last one I cannot provide…
  • #6 Convergence10Gb
  • #8 If you only have an empty rack – with no supporting infrastructure.
  • #10 Yes there are companies that do this – such as VCENot everyone can afford a vBlock
  • #11 We broke it down into steps, and this is what we gotOf course not everything can be automated – but as much as possible was desired
  • #12 Proper automation without an API is really difficult.So we have one for UCS (except the initial setup)Nexus – NopeStorage – Partially – because additional Storage Vendor software is needed
  • #13 Have to lay out all the stepsNetwork connectivityFabric InterconnectsUplink Switches
  • #14 A set of instructions for the engineer, for basic cabling – with a standard of ports – both on the Uplink switches – and the FI’sBasic VLAN definitionsFC uplinks
  • #15 Cisco have a very rich API which allows for the configuration of a full UCS serverAlan Renouf and Eric Williams
  • #16 USB Key was the less of two evils – less dependency on external components.Powershell script to create the kickstart files – which accepts input from CSV file.Razor is better for bigger environments
  • #17 Static IP’s + Hostnames are not are not possible in razor – so we needed to go around it.
  • #18 VM’s were deployed and configured with IP addresses – then vCenter was installed.Time to install vCenter – was less than 10 minutes – automated scripts that put IP’s
  • #19 Python scripts – William Lam has a great post – but not suitable – no vCenter at the time of kickstartThe Puppet environment was the preferred choice – because it allows for the definition of a full environment – also the guests as wellvCenter installation could also build the environment
  • #20 How could I actually start without attached storage – with local disks – and svMotion after the fact.EMCStorage IntegratorData ONTAP PowerShell Toolkit
  • #21 Clone SizeHave to keep it updated.But what if I want the build to be a physical node – or on another cloud / datacenter ?Deploy - always consistentQuickerVery portable – the RHEL ISO is always available…The empty shell VM’s could be create either with PowerCLI or through puppet (cloning only)Razor and Puppet go hand in hand
  • #22 Razor already has the tags in place – that is what it does extremely well. Plant these tags as a custom fact and have Puppet apply modules according to the fact discovery.
  • #23 The whole process had a large number of steps – and many dependencies. Break it down piece by piece and work you way through the problem.Oracle RAC installation script for example (~1000 lines) and growing.Razor re-write has caused a lot of pain. But a great learning process.Puppet – great tool – very powerful – but not all encompassing.