Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

OSDC 2018 | Spicing up VMWare with Ansible and InSpec by Martin Schurz and Sebastian Gumprich

165 views

Published on

VMWare is a common hypervisor choice in large organizations, and it comes with a zoo of additional tools, options and licenses. But once you add a little bit of OpenSource to the mix, things start to get interesting. There are open APIs which integrate well with Tools like Ansible and InSpec. So it is easy to write your own scripts to verify a configuration or harden a ESX host. We will show you, how we automated our deployment, what problems we encountered and how we added some nice features.

Published in: Software
  • Be the first to comment

OSDC 2018 | Spicing up VMWare with Ansible and InSpec by Martin Schurz and Sebastian Gumprich

  1. 1. Spicing up VMware with Ansible and InSpec T-Systems Multimedia Solutions GmbH
  2. 2. Martin Schurz Sebastian Gumprich T-Systems MMS T-Systems Multimedia Solutions GmbH
  3. 3. Ops: the old days (tm) T-Systems Multimedia Solutions GmbH
  4. 4. Ops: the old days (tm) T-Systems Multimedia Solutions GmbH
  5. 5. Ops: the old days (tm) handcrafted and sometimes arcane con guration clusters parameters for Oracle the "one" server someone installed virtualization is just lift and shift T-Systems Multimedia Solutions GmbH
  6. 6. Ops: slowly improving reliance on enterprise tools vSphere / vRealize / vCloud T-Systems Multimedia Solutions GmbH
  7. 7. Ops: slowly improving reliance on enterprise tools vSphere / vRealize / vCloud T-Systems Multimedia Solutions GmbH
  8. 8. VMware T-Systems Multimedia Solutions GmbH
  9. 9. We have a lot of pets, but we need more cattle T-Systems Multimedia Solutions GmbH
  10. 10. Mantra: manual work is a bug! T-Systems Multimedia Solutions GmbH
  11. 11. T-Systems Multimedia Solutions GmbH
  12. 12. Why Ansible? because we don't like Puppet Ansible is simple, agent-less easy to learn straight-forward in task execution Not written in Ruby (looking @ you, Puppet) T-Systems Multimedia Solutions GmbH
  13. 13. Ansible - quick rundown T-Systems Multimedia Solutions GmbH
  14. 14. Ansible modules - many of them T-Systems Multimedia Solutions GmbH
  15. 15. ... except Oracle pet, not cattle. T-Systems Multimedia Solutions GmbH
  16. 16. Jenkins incoming already reliable application deployments now reliable con guration of servers, too T-Systems Multimedia Solutions GmbH
  17. 17. Automation is fun! ... or so they say ... T-Systems Multimedia Solutions GmbH
  18. 18. Automation is fun! ... or so they say ... due to "unfortunate circumstances" we lost half our servers “ “ T-Systems Multimedia Solutions GmbH
  19. 19. Automation is fun! ... or so they say ... ... and we did recover easily due to "unfortunate circumstances" we lost half our servers “ “ T-Systems Multimedia Solutions GmbH
  20. 20. Automation is fun! ... or so they say ... ... and we did recover easily due to "unfortunate circumstances" we lost half our servers “ “ T-Systems Multimedia Solutions GmbH
  21. 21. Managing ESX Hosts Prebuild modules for all basic tasks: Network Storage Cluster vCenter VM tasks T-Systems Multimedia Solutions GmbH
  22. 22. Managing ESX Hosts (example) I want to con gure all VLANs for my ESX Cluster All Hosts should have correct VLAN con guration All Hosts should be con gured from one source Adding Hosts and VLANs should be easy (like a distributed vSwitch) T-Systems Multimedia Solutions GmbH
  23. 23. Managing ESX Hosts (example) create a host group (e.g. esx-servers ) Add group_vars: vlans: customer1-vlan: tag: 4006 vswitch: vSwitch0 customer2-vlan: tag: 4007 vswitch: vSwitch0 ... T-Systems Multimedia Solutions GmbH
  24. 24. Managing ESX Hosts (example) Add a playbook task: - hosts: esx-servers tasks: - name: "Add VLANs" local_action: module: vmware_portgroup hostname: '{{ ansible_hostname }}' username: root password: '{{ esxi_pass }}' switch_name: "{{ item.value.vswitch }}" portgroup_name: "{{ item.key }}" vlan_id: "{{ item.value.tag }}" validate_certs: false with_dict: "{{ vlans }}" T-Systems Multimedia Solutions GmbH
  25. 25. Managing ESX Hosts (example) rst Ansible run TASK [Add VLANs] ***************************************** ok: [esx_server] => (item={'value': {u'vswitch': u'vSwitch0', u'tag': 4006}, 'key': u'customer1-vlan'}) changed: [esx_server] => (item={'value': {u'vswitch': u'vSwitch0', u'tag': 4007}, 'key': u'customer2-vlan'}) PLAY RECAP *********************************************** esx_server : ok=1 changed=1 unreachable=0 failed=0 T-Systems Multimedia Solutions GmbH
  26. 26. Managing ESX Hosts (example) second Ansible run TASK [Add VLANs] ***************************************** ok: [esx_server] => (item={'value': {u'vswitch': u'vSwitch0', u'tag': 4006}, 'key': u'customer1-vlan'}) ok: [esx_server] => (item={'value': {u'vswitch': u'vSwitch0', u'tag': 4007}, 'key': u'customer2-vlan'}) PLAY RECAP *********************************************** esx_server : ok=1 changed=0 unreachable=0 failed=0 T-Systems Multimedia Solutions GmbH
  27. 27. Managing ESX Hosts (example) I want to con gure all VLANs for my ESX Cluster All Hosts should have correct VLAN con g All Hosts should be con gured from one source Adding Hosts and VLANs should be easy (like a distributed vSwitch) T-Systems Multimedia Solutions GmbH
  28. 28. Creating VMs - Host variables vm_cpu: 8 vm_ram: 8 vm_storage: srv_live_vmdata1 vm_host: srv-live-vh07 vm_disksize: 80 default_gateway: 172.31.225.1 network_ether_interfaces: - vm_net: srv-lgen-app device: eth0 bootproto: static address: 172.31.225.36 netmask: 255.255.255.128 onboot: "yes" dns1: "{{ srv_dns1 }}" dns2: "{{ srv_dns2 }}" domain: "{{ srv_domain }}" T-Systems Multimedia Solutions GmbH
  29. 29. Creating VMs - the Ansible task - name: Create new VM vmware_guest: hostname: "{{ vcenter_host }}" username: "{{ vcenter_user }}" password: "{{ vcenter_pass }}" datacenter: "{{ vcenter_dc }}" name: "{{ item }}" template: "{{ vm_template }}" state: poweredon wait_for_ip_address: yes hardware: memory_mb: "{{hostvars[item]['vm_ram']}}" num_cpus: "{{hostvars[item]['vm_cpu']}}" disk: - size_gb: "{{hostvars[item]['vm_disksize']}}" datastore: "{{hostvars[item]['vm_storage']}}" T-Systems Multimedia Solutions GmbH
  30. 30. Adding Security to the mix Telekom security guideline requires all servers to be hardened also VMware security guideline: https://www.vmware.com/security/hardening- guides.html (beware Excel!) T-Systems Multimedia Solutions GmbH
  31. 31. Hardening an ESX host (example) VMware Requirement: Guideline ID: ESXi.disable-mob: The managed object browser (MOB) provides a way to explore the object model used by the VMkernel to manage the host; it enables con gurations to be changed as well. This interface is meant to be used primarily for debugging the vSphere SDK. In Sphere 6.0 this is disabled by default T-Systems Multimedia Solutions GmbH
  32. 32. Hardening an ESX host (example) Ansible implementation: # Guideline ID: ESXi.disable-mob - name: get | disable MOB shell: "vim-cmd hostsvc/advopt/view Config.HostAgent.plugins.solo.enableMob | grep value | cut -d ' ' -f 9" register: mob_status changed_when: mob_status.rc > 0 - name: set | disable MOB shell: "vim-cmd hostsvc/advopt/update Config.HostAgent.plugins.solo.enableMob bool {{ mob }}" when: mob not in mob_status.stdout T-Systems Multimedia Solutions GmbH
  33. 33. Hardening VMs - nding them all! - name: Find all .vmx files on local store shell: | find /vmfs/volumes/datastore/ -name *.vmx register: found_vms changed_when: False T-Systems Multimedia Solutions GmbH
  34. 34. Hardening VMs - changing them - name: Set VM parameters lineinfile: path: "{{ item[1] }}" regexp: "{{ item[0].key }}" backrefs: yes line: "{{ item[0].key }} = "{{ item[0].value }}"" with_nested: - "{{ parameters_add }}" - "{{ found_vms }}" parameters_add: - { key: isolation.tools.copy.disable, value: TRUE } - { key: isolation.tools.paste.disable, value: TRUE } T-Systems Multimedia Solutions GmbH
  35. 35. Managing VMs - deleting them - name: delete VM vmware_guest: vcenter_hostname: "{{ vcenter_host }}" username: "{{ vcenter_user }}" password: "{{ vcenter_pass }}" validate_certs: false guest: "{{ item }}" force: true state: absent # deletion! with_items: "{{ vm_name }}" T-Systems Multimedia Solutions GmbH
  36. 36. Managing VMs - making snapshots - name: Create snapshot of {{vm_name}} vmware_guest_snapshot: folder: "/vm/" name: "{{ vm_name }}" state: present snapshot_name: "snap_{{ '%Y-%m-%d-%M' | strftime }}" T-Systems Multimedia Solutions GmbH
  37. 37. Not everything out of the box moving VMs not implemented in Ansible :( but Ansible is extensible with Python code so just write your own module VMware vSphere API Bindings for Python (https://github.com/vmware/pyvmomi) VMware API Docs Python + API = T-Systems Multimedia Solutions GmbH
  38. 38. Not everything out of the box we started with Ansible code: - name: Move VM to target host and DS delegate_to: localhost vm_move: vc_host: "{{ vcenter_host }}" vc_pass: "{{ vcenter_pass }}" vc_user: "{{ vcenter_user }}" vm_name: "{{ inventory_hostname }}" ds_name: "{{ vm_storage }}" esx_host: "{{ vm_host }}" T-Systems Multimedia Solutions GmbH
  39. 39. Not everything out of the box then think about implementation what needs to be done: Locate the VM we want to move T-Systems Multimedia Solutions GmbH
  40. 40. Not everything out of the box then think about implementation what needs to be done: Locate the VM we want to move Locate the target ESX host / storage T-Systems Multimedia Solutions GmbH
  41. 41. Not everything out of the box then think about implementation what needs to be done: Locate the VM we want to move Locate the target ESX host / storage check what needs to be changed T-Systems Multimedia Solutions GmbH
  42. 42. Not everything out of the box then think about implementation what needs to be done: Locate the VM we want to move Locate the target ESX host / storage check what needs to be changed move the VM T-Systems Multimedia Solutions GmbH
  43. 43. Not everything out of the box some boilerplate is needed: def main(): module = AnsibleModule( argument_spec=dict( vc_host = dict(required=True, type='str'), ... esx_host = dict(required=False, type='str'), ), ) result = dict( changed=False, original_message='', message='' ) # do something module.exit_json(**result) T-Systems Multimedia Solutions GmbH
  44. 44. Not everything out of the box vm = get_obj(content, [vim.VirtualMachine], vm_name) vm_datastore = get_obj(content, [vim.Datastore], ds_name) dest_host = get_obj(content, [vim.HostSystem], esx_host) vm_relocate_spec = vim.vm.RelocateSpec() T-Systems Multimedia Solutions GmbH
  45. 45. Not everything out of the box vm = get_obj(content, [vim.VirtualMachine], vm_name) vm_datastore = get_obj(content, [vim.Datastore], ds_name) dest_host = get_obj(content, [vim.HostSystem], esx_host) vm_relocate_spec = vim.vm.RelocateSpec() if vm.datastore[0] != vm_datastore: result['changed'] = True vm_relocate_spec.datastore = vm_datastore T-Systems Multimedia Solutions GmbH
  46. 46. Not everything out of the box vm = get_obj(content, [vim.VirtualMachine], vm_name) vm_datastore = get_obj(content, [vim.Datastore], ds_name) dest_host = get_obj(content, [vim.HostSystem], esx_host) vm_relocate_spec = vim.vm.RelocateSpec() if vm.datastore[0] != vm_datastore: result['changed'] = True vm_relocate_spec.datastore = vm_datastore if vm.runtime.host != dest_host: result['changed'] = True vm_relocate_spec.host = dest_host T-Systems Multimedia Solutions GmbH
  47. 47. Not everything out of the box vm = get_obj(content, [vim.VirtualMachine], vm_name) vm_datastore = get_obj(content, [vim.Datastore], ds_name) dest_host = get_obj(content, [vim.HostSystem], esx_host) vm_relocate_spec = vim.vm.RelocateSpec() if vm.datastore[0] != vm_datastore: result['changed'] = True vm_relocate_spec.datastore = vm_datastore if vm.runtime.host != dest_host: result['changed'] = True vm_relocate_spec.host = dest_host if result['changed']: task = vm.Relocate(spec=vm_relocate_spec) wait_for_task(module, task, si) T-Systems Multimedia Solutions GmbH
  48. 48. Not everything out of the box VMWare has a tool called govc https://github.com/vmware/govmomi/tree/mast er/govc pretty easy to use from the command line this can also be included in Ansible scripts but do I really need to write all this python code? I'm not a programmer! “ “ T-Systems Multimedia Solutions GmbH
  49. 49. Testing T-Systems Multimedia Solutions GmbH
  50. 50. Testing with inSpec written by Chef guys originally a fork of serverspec diverged since then and has gotten many new features T-Systems Multimedia Solutions GmbH
  51. 51. Testing with inSpec - the test control 'VM.disable-console-drag-n-drop' do title 'Explicitly disable copy/paste operations' vsphere.datacenters.each { |dc| dc.vms.each { |vm| describe vm_advancedsetting) do its(['isolation.tools.dnd.disable']) { should eq true } end } } end T-Systems Multimedia Solutions GmbH
  52. 52. Testing with inSpec - results VM.disable-console-drag-n-drop isolation.tools.dnd.disable should eq true Profile Summary: 136 successful controls, 0 failures Test Summary: 136 successful, 0 failures, 0 skipped T-Systems Multimedia Solutions GmbH
  53. 53. Bonus - ansible-cmdb T-Systems Multimedia Solutions GmbH
  54. 54. The End Now grab some food! T-Systems Multimedia Solutions GmbH
  55. 55. Ansible logo from redbubble.com VMWare logo from fujitsu InSpec logo from sdtimes Fry from ickr user liliana_von_k success kid from instagram user laneymg automate from ickr user Amber Case Ansible works image from tutorialspoint.com T-Systems Multimedia Solutions GmbH

×