Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Deploying Applications in Today’s Network Infrastructure

2,721 views

Published on

This presentation prepares networking engineers for the fundamentals of deploying application in today’s server virtualization infrastructure. The objectives for this presentation is to share best practices, tips and tricks on how best to implement Cisco technology such as Cisco UCS and Cisco Nexus 1000v with any virtualization stack. During this presentation we will analyze and dissect two server virtualization use cases recently architected. These use cases consist of a multi -tenant private cloud and virtual desktop infrastructure for thousands of users.

Published in: Technology
  • Be the first to comment

Deploying Applications in Today’s Network Infrastructure

  1. 1. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Connect 11© 2012 Cisco and/or its affiliates. All rights reserved.Deploying Applicationsin Todays NetworkInfrastructure
  2. 2. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 2Why did I create this Presentation?Prepares networking professionals for the fundamentals of deploying applicationin today’s server virtualization infrastructure2Gartner sees virtualization workloads become software defined Infrastructure integration is leading to traditionalsilos merging High percentage of virtualization abstractsworkloads & increases portability Drive toward x86 servers standardization Workloads consume infrastructure and have apersonality defined by:– Function, (e.g., Web App, Database, VDI)– State (e.g., transaction, publish, share)– Size (e.g., small, medium, large)– Availability (e.g., portability & clustering)– Complexity, security ....Source: Philip Dawson Gartner
  3. 3. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 33A Fabric Resource Pool Manager (FRPM) FRPM is typically hosted by top ofthe rack switch or dedicatedmanagement server See FRPM as a "uber-managementsuite,“ enabling easier componentaggregation/disaggregation FRPM may be implemented singly orin conjunction with each otherCisco UCS Manager and UCS Central are examples of aFabric Resource Pool Manager (FRPM)Virtualization Drives Hardware Abstraction
  4. 4. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 4Workloads are the use cases of infrastructureServer virtualization: Are we there yet?4Penetration Has Reached Critical Mass: 2012 58% of all installed x86 serverworkloads are running in a VMWhich Workloads DO You or Do not Virtualize?1. Large OLTP DBMS2. Large application servers3. Large ERP projects4. Complex BI/DW workloads5. Large email instances6. Commercial issues (support and licensing)7. Clustered environments8. I/O-intensive applications9. Workloads that scale above a single socketFrom "Virtual Machines Will Slow in the Enterprise, Grow in the Cloud," 4 March 2011, Gartner
  5. 5. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 5Example Workloads is Hosted Virtual DesktopsUnderstanding the architectures as workloads increase5Have You Thought About … Typically 4-5 users per core (pre-Nehalem), 7- 9 users per core (Nehalem) I/O - sufficient bandwidth and throughput? Memory configurations (2GB to 4GB per VMrunning Windows) Server type; rack, blade, stand-alone, etc Server density may cause data centerpower/cooling issues Windows 7 images from 15GB to 45GB. Withdeduplication technologies from 2GB to 15GB Expandability Often a step function in net-new server andstorage infrastructures Highly dense zones Space Power Cooling Latency (<150 ms) Bandwidth - from100kbps up to 5mbpsRecommend Reading: Workload Considerations for Virtual Desktop Reference Architectures byVMware - http://www.vmware.com/files/pdf/VMware-WP-WorkloadConsiderations-WP-EN.pdf
  6. 6. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 6Application Deployment Case StudyInitial POD VDI deployment. Scale to additional PODs6 VMware View 5.1 Deployment “ Design should be scalable with no significant change in performance or stability,compared to current physical workloads per pod – “Deploy and user will comemodel” focusing on 6K POD deployment Knowledge Worker Profile– This is a middle of the range performance profile tier– Applicable to many generic types of users– Suitable to run basic corporate application suites– Linked cloned desktop– Non-persistent type of desktop
  7. 7. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 7VMware View POD Logical DesignStorage is broken down into two (2) NetApp array’s which will each service up to3,000 users using NFS7VDI Cluster Management VDI Cluster VDI ClusterBlock 1 & 2 VDI Storage Management Storage Block 3 & 4 VDI StorageUCS B230 UCS B230UCS B200NetApp 3270Storage ArrayNetApp 3270Storage Array2 x 318 GBServer Data storeNexus 5K Nexus 5K Nexus 5K500GBDesktopTemplateData store500GBDesktopTemplateData store2500GBUser DataStore Per250 User2500GBUser DataStore Per250 User530GBDesktopData storesper 250VMs530GBDesktopData storesper 250VMs
  8. 8. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 8VMware View POD - Logical Architecture DesignExample of a VMware View Pod8 A VMware View pod integrates multiple1,500-user building blocks into a ViewManager installation that you can manageas one entity A pod is a unit of organization determinedby VMware View scalability limits. Tablelists the components of a View POD– View building blocks 4– Each block consists of 2 ESX Hosts– View Connection Servers 4 (3 active and 1failover)– 10Gb Fabric and Cisco Nexus 1kv DSPod Architecture for 6000 View DesktopsPODs change based on requirements - Consult the VMware View Architecture Planning
  9. 9. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 9VMware View POD - Logical Architecture DesignVMware view POD broken down into management and compute blocks9Management B200 M3 Small Blade ConfigSupports small to medium size VMs / PhysicalCompute B230 M2 Small Blade ConfigSupports small to medium size VMs / Physical
  10. 10. Application Deployment Tips and Tricks10
  11. 11. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 11Application Deployment Tips and TricksA collection of hints and tips gathered from over three years of UCS/Nexus 1Kvdeployments11What do we see on site? Majority of deployments (80%+) run a mix of Hypervisors and bare-metal– Around 20% run with no bare-metal at all Hypervisor is for the large part (80%) VMware’s vSphere ESXi– ESXi 4.1 Update 2 primarily, customer moving to ESXi 5.1– Microsoft’s Hyper-V comes second– Xenserver comes third– Open Stack including XEN and KVM increasing in popularity Bare-metal deployments consist mostly of Windows 2008 R2 server and RHEL– Either bare-metal deployment imposed by application vendor– Or virtualization isn’t fully trusted yet (or misunderstood?)
  12. 12. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 12Application Deployment Tips and TricksA collection of hints and tips gathered from over three years of UCS/Nexus 1Kvdeployments12 Boot from SAN has literally exploded … but can be tricky to implement– From virtually non-existent in mid-2008 to 90% today– Valid for all OS: Windows, ESXi and Linux Fairly limited expertise in “advanced” OS deployments– ESXi HA cluster design options, Nexus 1000V, how many vNICs per blade, etc. Misunderstanding of certain networking options in UCS– The infamous Native VLAN checkbox anyone? Customers love automation of repetitive tasks … but often don’t know how– OS deployments, configuration of networking in ESXi Which sensors and objects should I be monitoring?Recurring patterns, questions and concerns
  13. 13. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 13How do Servers Communicate?Converged Network Adapters converge the functionality of network and storageadapters13 Servers have at least two adapters – FC HBA (Fiber Channel Host Bus Adapter) &Ethernet NIC (Network Interface Card) to connect to the storage network (FiberChannel) and computer network (Ethernet) Servers have at least two adapters – FC HBA (Fiber Channel Host Bus Adapter) &Ethernet NIC (Network Interface Card) to connect to the storage network (FiberChannel) and computer network (Ethernet)
  14. 14. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 14Future Proofing for VirtualizationBuilding an environment which can scale14Cisco UCS adds support for flexible VLAN configurations on Fabric Interconnect(FI’s) uplink ports while using End Host Mode. This feature provides support to allcombinations of upstream network configurations: End Host Mode and Switch Mode End Host Mode is similar to the hardwareimplementations of VMware vSwitches – nospanning tree, no loops, and does not look likeit is switched to the external network Switch Mode, means the FIs can act like anormal switch (use spanning tree, etc.)I most always recommend usingEnd Host Mode (default mode)
  15. 15. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 15Inter-Fabric Traffic using Cisco UCSUCS Release 2.0(2m) adds support for Nexus 2232 fabric extender15In order for Cisco UCS to provide the benefits, interoperability and managementsimplicity it does, the networking infrastructure is handled in a unique fashion: UCS rack-mount and blade servers areconnected to a pair of FI’s which handle theswitching and management The rack-mount servers connect to Nexus2232s providing local connectivity point 10GEFCoE without expanding management Not shown in this diagram are the I/O Modules(IOM) in the back of the UCS chassis. Theseextend to the Fabric Interconnects providingmanagement
  16. 16. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 16Cisco UCS Logical ConnectivityUCS is a Layer 2 system so any routing (L3 decisions) must occur upstream16UCS hardware is designed for low latency environments, such as high performancecomputing, and perfect for today’s applications: All switching occurs at the Fabric Interconnectand no intra-chassis switching occurs The only connectivity between FI’s is thecluster links. Both FI’s are active from aswitching perspective but management UCSManager (UCSM) is an Active/Standbyclustered application. This clustering occursacross L1 and L2 links. These links do notcarry data traffic
  17. 17. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 17Cisco UCS Fabric FailoverWhen deploying applications in the network, multipath and pinning configurationis critical17Fabric Failover is a capability found in Cisco UCS that allows a server to have ahighly available paths without using NIC teaming drivers or any NIC failoverconfiguration required in the OS, hypervisor, or virtual machine Fabric Failover provides the servers with avirtual cable that can be quickly andautomatically be moved from one upstreamswitch to another interface identifier and MAC address remainthe sameFabric failover is simple!! Perfect for PXE, Linuxand Windows installs. Just check the box!
  18. 18. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 18VMware ESX vNIC’s for UCSTo minimize error and ensuring uniform service profiles leverage vNIC templates.vNIC templates provide a mechanism to define interfaces and their policies. Aninterface contains a list of VLANs (or a single VLAN if that’s required), whetherCDP is enabled, a QoS policy, which pool the MAC address comes from, thelogical FI routing and the MTU18 Eight vNIC templates for ESX host;– ESX-Mgmt-A vmnic0 management for the host and Nexus 1Kv– ESX-Mgmt-B vmnic1 fabric B– ESX-NFS-A vmnic2 NFS mounts for fabric A - 9000 MTU– ESX-NFS-B vmnic3 fabric B– ESX-PROD-A vmnic4 data traffic for fabric A– ESX-PROD-B vmnic5 fabric B– ESX-Vmotion-A vmnic6 Vmotion for fabric A - 9000 MTU– ESX-Vmotion-B vmnic7 Fabric B Additional options explored at end of session!!!12
  19. 19. 19UCS QoS Made EasyCisco UCS, Palo, ESX, and Nexus 1000v using QoS
  20. 20. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 20Windows 2K8 vNIC’s for UCSTo minimize error and ensuring uniform service profiles leverage vNIC templates.vNIC templates provide a mechanism to define interfaces and their policies. Aninterface contains a list of VLANs (or a single VLAN if that’s required), whetherCDP is enabled, a QoS policy, which pool the MAC address comes from, thelogical FI routing and the MTU20 Two vNIC templates for Windows 2008 host– WIN2K8-PROD-AB windows management interface. Enable Fabric Failover. Pin to fabric A formanagement– WIN2K8-NFS-AB Windows NFS interface. MTU set to 9000. Enable Fabric Failover. Pin tofabric B for NFS Use a unique MAC resource pool1 2
  21. 21. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 21World Wide Names (WWPN/WWNN)Cisco UCS allows the users to create custom values for World Wide Names21 Used to logically identify resources for storage fabric zoning,array LUN masking Similar to MAC addresses for Ethernet 2 types:– World Wide Node Name (WWNN) – Identifies node– World Wide Port Name (WWPN) – Identifies a port on a node Visible in name server and FLOGI tables 8 bytes, representing:– Format 1,2, or 5 with the first 2 bytes (ex. 20:00)– Vendor unique OUI with bytes 3 through 5 (ex. 00:25:B5)– Assigned serial number with bytes 6 through 8 (ex. 00:01:02)
  22. 22. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 22Suggested WWNN/WWPN Octet Values22
  23. 23. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 23Suggested WWNN/WWPN Best Practice23 Always create pools that are multiple of 16 and contain less than 128 entries– This ensures vHBA-A (SAN A) and vHBA-B (SAN B) have the same low-order byte Counter-example using 233-entries pools Much better for both vHBAs to have the same low-order byte and a unique SANFabric identifier– Presence of “0A” or “0B” in the port WWN indicates SAN Fabric
  24. 24. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 24Port WWN poolsUse Expert setting when creating vHBAs24
  25. 25. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 25Suggested MAC Format for UCSMCisco UCS allows the users to create custom values for MAC address25
  26. 26. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 26Before we move on …The Native VLAN checkbox26 When defining VLANs on a given vNIC inside a SP, there’s a Native VLANcolumn When are you supposed to check that box?
  27. 27. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 27Native VLAN on a vNICWhen to check it27 The Native VLAN checkbox here is link-local only– It has zero impact on network uplinks or other SPs Behind the scenes vNICs are trunk (802.1Q) ports– FCoE VLAN + classical Ethernet VLAN(s) A vNIC can have one to N VLANs defined on it but only one can be Native Native VLAN checked means traffic is sent to the OS with no tag on that VLAN– Typical with single VLAN vNICs– The OS just receives traffic on the corresponding interface, no need to define VLAN-basedsub-interfaces Native VLAN unchecked means the OS must be able to handle 802.1Q tags
  28. 28. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 28Native VLAN examplesWill this work?28 This Service Profile is associated to a bladerunning ESXiThis won’t work! All traffic issent tagged to ESXi. A VLANmust be defined to handlemanagement traffic!
  29. 29. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 29Native VLAN examplesHow about this one?29 This Service Profile is associated to a blade running Windows 2008 R2 (not aVM!)This will work. Traffic on the“backbone” VLAN arrivesuntagged and is handled by“Cisco VIC EthernetInterface”
  30. 30. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 30Boot ProcessBooting is an involved, hacky, multi-stage affair – fun stuff30 Outline of the typical boot process:  Outline of the typical boot process: Once the motherboard is powered upit initializes its own firmware – chipset CPU will begin the bootstrapprocessor (BSP) that runs all of theBIOS and kernel initialization code Pre-Execution "pixie" is anenvironment to boot computers usinga network interface independently ofdata storage devices (like hard disks)or installed operating systems
  31. 31. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 31Unattended OS Installation and Boot ProcessBooting is simplified using UCS over the network or SAN31 UCS solves the booting complexity Create Boot Policy Complete control of system boot policyseparate from the BIOS settings– PXE, FC SAN– Virtual media (CDROM, ISO, USB, floppy) Control of how to un-provision serversto factory default when no longerrequired– Called “Scrub Policy”– Optionally clear BIOS settings– Optionally wipe local disks Allows for removing the low-levelconfiguration state on server– Easier automation possible
  32. 32. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 32 Cisco Server Provisioner automatically installs operating environments forphysical virtual servers and blades, a process known as bare metal provisioning Simple product to installation Easy to use & well-documented (w/ graphical tutorials) 3-step process to provision1. Prepare the ISO (Windows, Linux, ESX)2. Use Web UI to create:Provisioning Role Templates (MAC-Spec Provisioning)MAC-Independent Provisioning menus3. Assign templates and values to systems based on requirementsCisco Server ProvisionerAutomated System Provisioning, Recovery, and Cloning32
  33. 33. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 33Cisco Server ProvisionerMAC-Independent ("Pull“) Provisioning MAC-Specific (“Push”) Provisioning:33 Outline of the typical boot process:  Outline of the typical boot process: MAC address-specific push provisioning can be used in situations whereusers rarely touch the computer systems and rely on a provisioningdashboard to remotely provision servers and blades
  34. 34. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 34My Cloupia Solution “Demo”34 Key Components Of Cloupia Solution– Cloupia Unified Infrastructure Controller– Cloupia Network Services Appliance The Cloupia Unified Infrastructure Controller (CUIC) is a multi-tenant,multi-hypervisor provisioning and management solution that providecomprehensive virtual infrastructure control, management and monitoringvia single pane of glass The Cloupia Network Services Appliance provides PXE boot capabilitiesfor bare metal provisioning and acts as a PXE repository
  35. 35. 35What Can I Do in Cloupia Unified Infrastructure Controller Adding Physical Accounts Adding Virtual Accounts Discovery Policies/Policy Creation Virtual Data Center (vDC) Catalog (self-service catalog)Adding a Cisco UCSM Account 2You can also add other Compute/Network/Storage platformsAdding a Cisco UCSM AccountAdding a NetApp OnTapDiscovery Virtual Discovery Physical1344 5Adding a NetApp OnTap
  36. 36. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 363667 8CUIC Policies - A policy is a group of rules which determineswhere and how a new VM will be provisioned within theinfrastructure based on the availability of system resourcesCUIC needs four policies to be setup by sysadmin in order toprovision VM(s) Adding Physical Accounts Adding Virtual Accounts Discovery Policies/Policy Creation Virtual Data Center (vDC) Catalog (self-service catalog)CUIC VDC - An environment that combines– Infrastructure and virtual resources– Rules and Policies– Business Operational Processes– Cost Model– Enable/Disable Storage Efficiency– End User Self Service Option– Network,– Storage,– Computing,– Service Delivery/System PolicyCUIC Catalogs is an catalog combines:– Group and images– Application category, application type– Additional options such as Credentials,Guest customization, Remote access etc.– And overall presents as a single ‘Menu Item’to ‘Self Service’ user to a group(s)What Can I Do in Cloupia Unified Infrastructure Controller
  37. 37. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 37Preparing Server for ApplicationsBoot from San Tasks – You don’t need to be a storage guru!37UCS Manager Tasks– Create a Service Profile Template with x number of vHBAs– Create a Boot Policy that includes SAN Boot as the first device and link it to the Template– Create x number of Service Profiles from the Template– Use Server Pools, or associate servers to the profiles– Let all servers attempt to boot and sit at the “Non-System Disk” style message that UCS servers returnSwitch Tasks– Zone the server WWPN to a zone that includes the storage array controller’s WWPN– Zone the second fabric switch as well. Note: For some operating systems (Windows for sure), you need tozone just a single path during OS installation so consider this step optionalArray Tasks– On the array, create a LUN and allow the server WWPNs to have access to the LUN– Present the LUN to the host using a desired LUN number (typically zero, but this step is optional and notavailable on all array models)
  38. 38. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 38Boot from SANSteps required to configure boot from SAN381 25 4 36 7 8Note – If you are installing a new OS on the boot LUN youmight need to add a CDROM drive to the Boot Policy
  39. 39. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 39Tune your BIOS policyLet the server speak up39 Boot from SAN involves several key components working hand in hand– Correct UCSM boot-from-SAN policy with the right target port WWNs– Correct SAN zoning and LUN masking are imperative– SAN array must present a LUN (storage groups, initiator groups, etc.) During your first trial a component won’t work the way it’s supposed to UCSM lets you create BIOS policies that you can attach to the Service Profile Best Practice: for Boot-from-SAN you always want Quiet Boot disabled
  40. 40. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 40Build your boot policyOne path works, but if resiliency matters40 UCS can boot from 4 different paths– You can boot with just a single target boot policy, but not ideal for resiliency Typically, you’ll want a boot policy that goes like this: That policy says:– First try vHBA fc0 pWWN “63” via fc0  Storage Processor A, port A3– Then try vHBA fc0 pWWN “6B” via fc0  Storage Processor B, port B3– If those fail, then try fc1 (first pWWN “64” on SP A; then pWWN “6C” on SP B) Don’t forget to append CD-ROM or PXE after the SAN boot targets
  41. 41. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 41Let’s boot the serverKeep an eye out41 Associate the boot policy you just defined then boot the server With a M81KR adapter, this is what you’ll see for each vHBAIf you do not see the array show uphere, there’s probably a zoning ormasking error
  42. 42. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 42Booting from SAN TroubleshootingBooting from SAN is not necessarily the easiest configuration42 UCS removes the complexity of booting from SAN by using service profiles,templates and associated boot policieslogging into an array which has aWWPN of 20:00:00:1F:93:00:12:9E andit’s Service Profile is associated to blade1/1 in chassis 1 slot 1123First connect to the VICfirmware:Now list the vNIC ID’sand force the VIC to loginto the SAN fabric:Successfully logged into the fabric aswe’ve got a successful PLOGI and reportlUNs available
  43. 43. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 43Preparing Server for ApplicationsVMware vSphere 5 Auto Deploy and Cisco UCS43 A new stateless functionality that ships with vSphere 5– Stateless PXE boot of bare-metal hosts– Assign a specific configuration to PXE-booted hosts PXE-booted hosts receive a configuration at boot time– OS is not installed on disk, it runs from RAM Configuration applied through Host Profiles “Connect to vCenter” Auto Deploy works in tandem with a vCenter Server, a DHCP and a TFTP server– DHCP and TFTP server not part of Auto Deploy. They have to be configured to point to AutoDeploy (explained in this slide deck) Auto Deploy can be installed on a Windows VM, on vCenter Server directly. It alsoships with the vCenter Server appliance Auto Deploy is registered during installation with a vCenter Server instance
  44. 44. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 44Auto DeployTechnical Overview: 6 step process44 Host PXE boots and gets an IP address from DHCP DHCP points the host to the TFTP server via option #66 TFTP server downloads a gPXE configuration file as specified in option #67 gPXE config file instructs host to make HTTP boot request to Auto Deploy Server Auto Deploy queries the rules engine for information about host An Image Profile and Host Profile is attached to the host based on a rule set ESXi is installed into host RAM, is added to vCenter and is configured in the cluster vCenter maintains Image Profile and Host Profile for each host in its database2 3451DHCP Option 66DHCP Option 676
  45. 45. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 45UCS Manager Configuration Recommendations451. If you will be installing a lot of systems, know and understand goUCS, CLIscripting - It dramatically simplifies the setup of several complex objects2. Always use Policies, Pools, and Templates - Ive seen a lot of cases weremanually configured settings for specific service profiles are used.Always recommend using updating templates.If you use an actual policy you can quickly see whichelements are using the policy through the "showpolicy usage“ action under each policy3. Fixing service profiles where Policies, Pools, and Templates were not used - IfService profiles were created without policies, pools, and templates you canadd them later. Since often the systems being "fixed" are probably in productionyou have to be very careful with the process
  46. 46. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 46Use Updating Templates for vNICs or vHBAs46 To update or not to update? With vNIC/vHBA templates is always a question When creating a vNIC or vHBA templatealways use "updating template" optionunless you want to lock down changes With updating templates all virtualinterfaces bound to the template will beupdated immediately with any change This can be very powerful with it comesto adding new VLANs Take for instance you need to add a new VLAN to your UCS environment. If thetemplate is updating, all you do is add the new VLAN to the global VLAN listwithin UCS and then update your interface template VLAN list
  47. 47. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 47UCS Manager Configuration474. When cloning a the service profile the clone WILL NOT have the same MACaddress, UUID, WWNN, and WWPNs as the original one5. Be careful about switching or modifying vNICs and vHBAs since the MACaddress and WWPNs could change if you dont follow the right process. Do notDelete the current ones then re-add the templates. This is likely to change theaddresses. Could possibly break Storage zoning, boot from San and PXE setup6. Always and ONLY use vNIC and vHBA templates - You loose a lot of controland dramatically increase the complexity of troubleshooting and monitoring yourenvironment7. Always use a maintenance policy - I suggest using a user-ack policy againstEVERY service profile and EVERY service profile template. Personally I onlyuse user-ack policy
  48. 48. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 48VMware BIOS Settings48 Here are the best practices for VMware ESXi on Cisco UCS for deployingapplications in the network
  49. 49. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 49VMware BIOS Settings49Enhanced Intelspeed step cannotbe disabled on theB230 and B440Manage BIOSfirmware versionsand settings on aper-service basis
  50. 50. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 50UCS Manager configuration Recommendations508. Always set all blade firmware versions in the policy - I always selecting allfirmware options and version even for hardware that you may not have in theenvironment. Sometimes it turns out you might have a component the firmwareapplies to and you dont want to leave it at some random firmware versionSecondly you are likely toacquire new hardware and youwill have to remember to modifyyour firmware policy to includethis new hardware. It takes aseconds to select everythingNote - Newest firmware version is not necessarily at the top or the bottom of thelist. You will need to pay attention to the firmware version values to find the bestchoice
  51. 51. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 51Upgrade UCS with Production Applications51Firmware Upgrade Process- All production applicationmust remain unaffectedPreparation should prior the tothe upgrade - Collect theoperating equivalent of an IOSshow technical support;system logs, from UCSMThe firmware upgrade process isbroken down into two phases -upgrading the chassis versusupgrading the blade firmwareService profile associated with aparticular blade must be rebooted inorder to affect a firmware upgrade onthat bladeTwo firmware binaries which can belongto either domain and so land in a no-man’s land of sorts: the Adaptorsoftware on the Converged networkadaptor, and the CIMC firmware
  52. 52. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 52Upgrade UCS with Production Applications52Activating the FabricInterconnects - This is theonly step in the processwhere all data connectionsin the UCSM domain on aparticular path: A or B will beaffectedPreparation should prior the tothe upgrade - firmwareupgrade where service profilesare managed by serviceprofile templates - No UpdateTemplateDirect Updates - following thesteps below to upgrade the UCSinfrastructure without takingaffecting the application runningon the bladesMake sure multipathing issetup correctly prior toupgrade
  53. 53. Nexus 1000v and Nexus 1100
  54. 54. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 54Why Nexus 1000v ArchitectureComparison of a standard physical switch, wherenetwork administrators manage the physical switchand the server administrators manage the serversconnected to that switchMoving towards a virtualized environment, theserver administrators still manage the physicalESXi servers and network administratorsmanage the switchComparison to a Physical Switch Moving to a Virtual Environment
  55. 55. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 55Why Nexus 1000v Architecture55With the Nexus 1000V, the network administratorwill still manage the VSMs of the Nexus 1000V,along with the physical switchVEMs are managed by the networkadministrators since the port-profilesconfigurations are configured on the VSM. Thisallows the server administrators to manage theESXi hosts without worrying about the“networking” portion within the ESXi serverVSM: Virtual Supervisor ModuleVEM: Virtual Ethernet ModuleVSM: Virtual Supervisor Module
  56. 56. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 56Host Connectivity Requirements for Nexus 1000vEach Physical Host Is Typically on Several Networks56 Management to talk to vCenter Storage iSCSI and NFS vMOTION for moving VMs VSM to VEM communication the “backplane” Virtual machine networks—(why we are all here) Port channels for physical NICs– Many configurations possible– From dual 10G to many 1GVirtual SidePort GroupPhysical Side
  57. 57. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 57Uplink Port Profiles from VMware ESX vNIC’s57 The port-profiles of type “Ethernet” are utilized for the physical NIC interfaces onthe host. There are two things to note for the uplink port-profile– N1K Control and Packet VLAN is used for communication between the VSM to the VEM andMGMT VLAN is used for the service console of the ESXi servers. Those 2 VLANs need to beconfigured as “system vlans”. System VLANs are brought up on their ports before talking withthe VSM– The “channel-group” configuration needs to be configured for “macpinning” since the UCSblade servers is not able to be configured utilizing a LACP port-channel. Recommendedconfiguration is to use mac pinningTypical Nexus 1000v deployment with UCS Recommended Nexus 1000v deployment with high traffic
  58. 58. 58Uplink Port Profiles from VMware ESX vNIC’s exampleData Uplinks port-profileVmotion Uplinks port-profileManagement/Control/Packet port-profileNFS Uplinks port-profilevmnic 0 and 1 used for mgmt and N1K control and packettraffic only, and will use the following Port-ProfilePort-profile type ethernet system-uplinkvmware port-groupswitchport mode trunkswitchport trunk allowed vlan 300,406channel-group auto mode on mac-pinningno shutdownsystem vlan 300,406state enabledvmnic 2 and 3 used for NFS traffic only, and will usethe following Port-Profileport-profile type ethernet NFS-uplinkvmware port-groupswitchport mode trunkswitchport trunk allowed vlan 402,403channel-group auto mode on mac-pinningmtu 9000no shutdownsystem vlan 402state enabledvmnic 4 and 5 will be used to carry data productiontraffic, and will use the following Port-Profilevmnic 6 and 7 will be used to vMOTION traffic, andwill use the following Port-ProfilePort-profile type ethernet data-uplinkvmware port-groupswitchport mode trunkswitchport trunk allowed vlan 410-460channel-group auto mode on mac-pinningno shutdownstate enabledport-profile type ethernet Vmotion-uplinkvmware port-groupswitchport mode accessswitchport access vlan 400channel-group auto mode on mac-pinningmtu 9000no shutdownsystem vlan 400state enabled
  59. 59. 59Uplink Port Profiles from VMware ESX vNIC’s exampleData Uplinks port-profilemgmt vethernet port-profileThe service console or mgmt port-profile will be createdfor service console (vmkernel) interface. It is critical thatthis port-profile is also configured as a “system vlan”port-profile type vethernet mgmtvmware port-groupswitchport mode accessswitchport access vlan 300pinning id 1no shutdownsystem vlan 300state enabledport-profile type vethernet NFS-1vmware port-groupswitchport mode accessswitchport access vlan 402pinning id 0no shutdownsystem vlan 402state enabledport-profile type vethernet NFS-2vmware port-groupswitchport mode accessswitchport access vlan 403pinning id 1no shutdownsystem vlan 403state enabledSystem VLANs may be used for StorageVLANs (NFS/iSCSI) and vMotionAssigns (or pins) a vethernetinterface to a specific portControl and Packet vethernet port-profileport-profile type vethernet control-packetvmware port-groupswitchport mode accessswitchport access vlan 406pinning id 0no shutdownsystem vlan 406state enabled
  60. 60. 60Uplink Port Profiles from VMware ESX vNIC’s exampleSystem VLANs must also be used in vethernet port profiles for VSMManagement (console) VLAN and Nexus 1000V Control VLANVmotion vethernet port-profileData vethernet port-profileThe Vmotion port-profile will be created for the Vmotion (vmkernel) interfaces for each of the ESXi serversport-profile type vethernet vmotionvmware port-groupswitchport mode accessswitchport access vlan 400pinning id 0no shutdownsystem vlan 400state enabledport-profile type vethernet Client-Onevmware port-groupswitchport access vlan 410switchport mode accessno shutdownstate enabledport-profile type vethernet Client-Twovmware port-groupswitchport access vlan 411switchport mode accessno shutdownstate enabledport-profile type vethernet Client-Threevmware port-groupswitchport access vlan 412switchport mode accessno shutdownstate enabledport-profile type vethernet Client-Fourvmware port-groupswitchport access vlan 413switchport mode accessno shutdownstate enabled
  61. 61. 61Uplink Port Profiles from VMware ESX vNIC’s exampleWithout a VMkernel port none of these services can be used on the ESX server A VMkernel port is required on each ESX host where the following services will beused:– vMotion– iSCSI– NFS– Fault ToleranceChoose "VMkernel" for the connection type,Click Next.231Give the VMKernel port a label (e.g. iSCSI - if it willpurely be used for iSCSI)Enter an IP address to assign to the VMkernel port.No routing!!
  62. 62. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 62Introducing Cisco N1Kv Into VMware EnvironmentOnly migrate data port-profiles on the Cisco N1Kv62 We keep the management vmknic ina regular vSwitch and place N1KVcontrol and packet Create a vSwitch (or DVS, or N1KV)for vMotion and make one vNICstandby so local switching takesplace The 3rd pair of vNICs are for N1KV Provisions a 4th pair of interfaces forfuture use such as NFS. Set correctMTU sizeSome networking teams struggle to get Cisco N1Kv into a VMware environment
  63. 63. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 63Why I love Nexus 1000vDeploying Virtual Machines with Nexus 1000V63 Network admin sets up port profiles in advance based on requirements– All features are specified that will be needed– Goes to get coffee or on vacation Server admin creates VM templates– Template virtual NICs use port profiles Server admin clones templates– Clones bring port profiles along for the ride Server admin starts up VMs Nexus 1000V sets up ports from port profiles– Communicated by VMware on VM startupPossibly Thousands of VMs!
  64. 64. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 64Nexus 1000v Gotchas641. You cannot change system VLAN command if the port-profile is in used. Useshow port-profile name <name> usage command to check if port-profile is in useTried to remove system vlan from port-profile no system vlan 351 This will remove allsystem vlans from this port profile. Do you really want to proceed(yes/no)? [yes] ERROR:Cannot remove system vlans, port-profile currently in use by interface eth72. Never use the same Nexus 1000v domain id when installing a new Nexus 1000venvironment. Double check to make sure domain id is unique!!Workaround - Create another port-profile with the same settings, then change the vmnicport-profile to the new port-profile3. VSM gets migrated on same host/storage - preventive/failureThis is driven by vCenter anti-affinity rules. In order for this to occur, an ITIL change would have tobe made to disable this policy. There are no alarms on N1kv to detect this, it would have to besomething within the virtualization tool set to ensure the rules that have been defined are beingfollowedrevent this!
  65. 65. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 65Cisco Nexus 1100 SeriesCisco Nexus 1100 Series with Four VSBs: Cisco VSMs, VSGs, NAM, and DCNM65 Nexus 1100 Manager: Cisco managementexperience Manages a total of 5 virtual service blades(ie. 4 VSMs and 1 NAM) Each VSM can manage up to 64 VEMs(256 total VEMs) A dedicated NX-OS appliance for deployingmultiple Virtual Appliances / Virtual Services It is NOT a general purpose server to deployany VMCisco Nexus 1100 Series High-Availability Pair
  66. 66. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 66Network Connectivity OptionsCisco Nexus 1100 Series can be connected to the network in five ways66 Network Connection Option 4– Option 4 uses the two LOM interfaces for management traffic, two of the four PCI interfaces forcontrol and packet traffic, and the other two PCI interfaces for data traffic. Each of these pairs ofinterfaces should be split between two upstream switches for redundancy Option 4 is well suited for customers who want to use the Cisco NAM but requireseparate data and control networks. Separating the control from the data networkhelps ensure that Cisco NAM traffic does not divert cycles from control traffic andtherefore affect connectivity
  67. 67. Accelerate Workloads with POD Deployment
  68. 68. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 68Reference Architecture Can Accelerate Any WorkloadsStandard Operation Procedures (SOPs) for operational excellence
  69. 69. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 69The Perfect POD for any workload69Reference Architecture Can Accelerate Any WorkloadsEasy Jet, like Ryan air, borrows its business modelfrom United States carrier Southwest AirlinesUCS 5108 ChassisWire-once to UCS fabric – fastscale up / scale downB440 M2 Large Blade ConfigSupports small to XL size VMs /Physical (req’d for large missioncritical apps and DB hosting)B230 M2 Small Blade ConfigSupports small to medium sizeVMs / PhysicalCisco Nexus 7k/5k CoreCore + aggregation back toenterprise networkUCS 6248XP FabricInterconnectShared connectivity / uplink tonetwork, storage, backupUnified management of UCSfabric
  70. 70. Complete Your Paper“Session Evaluation”Give us your feedback and you could win1 of 2 fabulous prizes in a random draw.Complete and return your paperevaluation form to the room attendantas you leave this session.Winners will be announced today.You must be present to win!..visit them at BOOTH# 100
  71. 71. © 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 71Thank you.

×