More Related Content
Similar to Data Center Network Trends - Lin Nease
Data Center Network Trends - Lin Nease
- 1. © 2004 Hewlett-PackardDevelopment Company, L.P. The information contained herein is subject to change without notice. HP DUTCHWORLD 2008 OUTSMART THE FUTURE! Data Center Network Trends Data Center Network Trends Lin NeaseLin Nease
- 2. HP ProCurve DataCenter Solutions 2008 2HP ProCurve Confidential What’s driving data center transformation? Customer priorities and expected outcomes Business pressures • New Apps & Services – growth • Integrate acquisitions • Support business innovation • Compliance • Reduce costs and risks • Mission critical Data center constraints • Communications cost structure • Power and cooling • Facilities • Personnel • Relocations • Legacy infrastructure • Workflow
- 3. 3 Tools to addressdata center challenges Legacy Data Center Next Generation Data Center Standardization Virtualization Automation Reduced maintenance costs, better asset utilization, lower power consumption, and higher system uptime – resulting in better business outcomes Heterogeneous environment Application-specific clusters Manual element configuration Fast, reliable deployments Shared, flexible resources Pods – Ruthless Standards
- 4. Types of DCNetworks “Back Office”: • typical 3-tier business applications • diverse application requirements • these customers buy tools from vendors High-performance clustering • east-west performance with line rate • in some cases, reliable multicast is very important • Infiniband is prevalent today “Front Office”: • big on-line web presence • lots of server load balancing & application acceleration “Cloud”: • huge web-based service infrastructure • think Google, MSN • totally self-sufficient, cost-driven customers Each Imposes Different Priorities And Demands
- 5. 10Gb growth atthe edge brings new topology approaches • 10G low-cost connectivity will evolve repeatedly for the next 3 years • 10GbaseT not practical yet – power & latency excessive, non-standard cables • Home-runs directly to core switches carry huge optics cost with 10G • Fiber & transceiver costs • core switches – higher $ per port – many more ports • choice between lack of flexibility or low capacity utilization • Using TOR/Pod-based switches greatly reduces 10G costs • CX-4 today, SFP+ direct connect next year – 7M practical, 10M theoretical passive limit • Oversubscribe in the pod – reduce costs of more complex infrastructure 10G Home run approach (up to $4k more per server!) 10G Pod-based approach = cheap copper cabling = expensive fiber cabling Expensive Ports, fixed capacity Lower cost Ports, Flexible growth Expensive fiber Cheap Copper in pods
- 6. Pod-based design –split the network into separate lifecycles • Network access & aggregation components on same lifecycle as servers • De-couple IT processes • More network bandwidth and energy contained lower in the network • “Crop rotation” used to replace old with new technologies over time • power/cooling contained in pods, allowing adoption of future efficiencies “Pod 7” “Pod 8”“Pod 6” “Pod 9”
- 7. Coupled with FacilitiesDesign: Configure Once, Allocate on Demand – Division of Labor - De-couple IT processes (PS – we still need silos ;-) - Minimize change - Standardize & automate to the maximum possible extent The “DNA” approach: -Standard VLANs, Addressing, ACLs at every port -Server admins assign and manage server context -Assigned context “activates” network personality POD Types: 1. Server 1. Normal Power 2. High Power 2. Storage (3B) 1. SAN 2. Tape/Backup/Archive 3. Network Services 4. Utility (Floor Mount Computing) 5. Legacy 6. Vacant 7. Development/Test Network substrate configured the same regardless of future pod usage.
- 8. Very-Large Layer 2domains Value propositions: • Allow very flexible mapping of virtual LANs • Allow dynamic plumbing of applications • Multipathing at L2 • Scalability versus spanning tree • Simpler configuration and troubleshooting than spanning tree Potential futures: • Trill • Shortest-Path Bridging • Extended Meshing approaches VLL2 network
- 9. Fibre Channel /Ethernet Convergence? FCoE Value Propositions: -Replace current NICs and HBAs in servers with fewer consolidated, specialized NICs (logic goes that 10G has plenty of headroom for consolidation – at least for now) -Homogenize server I/O to enable more standard server configurations -Allow for greater flexibility to “match make” any LUN to any server -Solve interoperability problems with Fibre Channel via the Ethernet ecosystem Problems: 1) T11 standards are not finalized, issues remain 2) A “new Ethernet” will be required for storage; convergence will not be as easy as VOIP 3) Network complexity will increase significantly 4) No new entrants are likely to emerge for FC director products 5) Interoperability problems will probably get worse, not better FC / Ethernet converged network
- 10. Possible Convergence Stages– Today = tcp/ip links = FC native links Data Center Distribution SLB Firewall LUN farm FC SAN IP Data Network FC switches E-net switches server farm
- 11. Step 1? –Standardize NIC types (still dedicated SANs, but using Ethernet) = tcp/ip links = FC native links = FCoE SAN links Data Center Distribution SLB Firewall LUN farm FC SAN IP Data Network FCoE switches E-net switches server farm Value Proposition 1) Homogenize Server I/O 2) Homogenize Switch Types (This is likely how FCoE will end up, as this is exactly how iSCSI is deployed today)
- 12. Step 2? –homogenize server I/O – Universal NICs with TOR splitting = tcp/ip links = FCoE SAN links = FC native links = Converged Server NICs SLB Firewall LUN farm FC SAN IP Data Network converged Ethernet switches server farm F_ports Value Prop (beyond step 1) 1) Consolidate Server I/O 2) Reduce number of NICs/HBAs 3) Aggregate server cabling 4) Homogenize Server Types N_ports Issues: • complex switch management • standards not complete • all NICs must be 10Gb for benefit • it’s still Fibre Channel! • ‘010 or later for volume • 10Gb won’t be overkill by then (once you need 4 NICs per server, why do it this way?)
- 13. Step 3? –Ethernet enhances SAN flexibility – Managed Virtual HBAs “on-a-rope” = tcp/ip links = FCoE SAN links = FC native links = Converged Server NICs SLB Firewall LUN farm FC SAN IP Data Network FC SAN N_ports N_ports N_ports N_ports Ethernet/FC switches converged Ethernet switches Value Prop (beyond step 2) • Virtualize any server to any LUN • …while preserving FC SANs
- 14. Step 4? –Fully Converged Network? 2015? FC Ethernet, or Ethernet FC? = tcp/ip links = FCoE SAN links = FC native links = Converged Server NICs SLB Firewall LUN farm Legacy FC SAN Converged Network VE_ports VF_ports E_ports E_ports VN_ports Issues: • It’s still Fibre Channel • Fibre Channel is not growing • Different security model • Different flow model • Different mgmt model • Higher cost structure • Same 2 director vendors • More interoperability problems • Complexity of merged networks
- 15. Meanwhile – iSCSIis real and growing fast today • iSCSI growing 30-40% YOY, FC is flat • HP just acquired Left-Hand Networks • turns a general-purpose server or virtual machine into an iSCSI storage system • offers compelling feature set in very low-cost solution • Very recent growth is driven by VMware • Use case is “vanilla storage for vmotion-able servers” • Ethernet switch requirements • good buffering • flow control capability • jumbo frames • (emerging) 10G density • Typically customers separate iSCSI switches from data TCP/IP
- 16. Virtualization – whereis the network going? • Both ends of the network are starting to look alike! • virtual switches are much like access points • Port-centric edge management model is challenged • constraining • labor-intensive • static • costly infrastructure Virtual Servers Virtual Servers Data Center Campus VM soft switches: “Access Points?”
- 17. Virtualization & Agility– HP Strategy -Make forwarding decisions in the network -Use simple mechanisms to make sure all traffic can be classified -Leverage the “Lite AP” architectural framework -Allow real switches to do ACLs, lookups, QOS, classification -Don’t force NICs to be bridges -Allows the NICs to focus on the host/IO membrane -Use standards-based mechanisms to track VM’s connections -Use MAC addresses and 802.1X tokens -Out-of-band service, authentication, tracking, and automation approaches -Actively drive new standards that help solve the problem -ProCurve vice chairs, participates in multiple standards bodies Virtual Machines Server farms
- 18. DC Network SwitchThermal Design: Side-to-side cooling Front-to-back cooling Cool Aisle Hot Aisle Side-cooled switch draws rising warm air from inside rack Side-cooled switch draws re-circulated hot air from inside rack Extra heat is leakage that must be moved Hot air exhausted inside rack Cool Aisle Hot Aisle F2B cooled switch draws air directly from the cool aisle F2B cooled switch exhausts air directly Into hot aisle No extra heat is built up inside the rack Side-to-side Cooled Switches Front-to-Back Cooled Switches
- 19. Common Network Automation usecases today Snapshot and store device configuration information Real-time change detection for all activities Keystroke level audit trail Auto-discover the network and capture detailed audit trail of all device changes Catalog & Diagramming Mass configuration changes Software updates Bare-metal provisioning ACL deployments Automate changes across thousands of devices Deploying Large Scale Changes Out of the box reports on ITIL, PCI, HIPAA & more Enforce best practices and security standards Easily remediate violations Enforce, audit and report on compliance Maintaining Compliance Automate multi-step processes across disparate systems Automate validation of complex network changes Ensure best practices are followed Integrated process automation to automate network operations Process-powered Network Automation
- 20. Data Center Network Network complexityin data centers is exploding… • many functional types of networks • virtual servers • application acceleration • multiple firewall/NAT layers • HA / load balancing • multi-tier applications • explosion of VLANs • workload mobility • compliance • change management • high availability …which intertwines too deeply with server provisioning… VMs Where does the “server” end and the “network” begin?
- 21. Data Center Network Intelligent Switches Intelligent Switches Intelligent Switches Also, PhysicalIssues Exist with Virtual Networking: 1) Unlike discrete items like CPU cycles or LUNs, the network is a single shared resource. 2) The network cannot be fully virtualized, due to laws of physics and the fact that it’s shared. (i.e. bandwidth & connectivity cannot be created out of thin air, and a network that offers infinite configurable bandwidth between all endpoints is not cost-effective)
- 22. Data Center Network Intelligent Switches Intelligent Switches Intelligent Switches However… - wecan “carve up” the network into pre-allocated virtual connections, and allocate them as if they were discrete items like servers and LUNs. - by choosing how many virtual connections to offer where, we can virtualize the network while still obeying laws of physics; Pre-allocated virtual chunks
- 23. Data Center Network Intelligent Switches Intelligent Switches Intelligent Switches What ifthe network was presented as an inventory of virtual connections – that abstract the network complexities? Inventories of Virtual connections Demarcation between “Network” and “Server”
- 24. Data Center Network Blade Servers Intelligent Switches Intelligent Switches Server Pod VMs Blade Servers Intelligent Switches …andthe network access was securely automated & configured one server at a time, as a service?
- 25. Policy-Based Provisioning, Automation& Accounting for Virtual Data Centers Server Configuration -IP Address / Subnet -Gateway -DNS Servers Network Edge Configuration -VLAN -ACL -QoS -Rate-limit Virtual Connection Network Core Configuration -Load-balancer -Firewall -Router
- 26. federate 26 12/11/2008 Datacenter ConnectionManagement 2) Select an available connection 4) Configure Server according to subscription policies 1) Set up policies Place new connections in the inventory Network Admin Server Admin 3) Subscribe to the connection Network Infrastructure (dynamically deployed for each connection) 7) Automate other Network Policies via events Subscription, Registration or IP allocation event UCMDB Compliance Checking Fault Management Capacity Management info info info new server or VM 5) Register Server @ L2 6) Enforce L3 registration Policy enforcement responses Policies for Router, Firewall, Load Balancer, DLP, IDS, etc Connection Inventory
- 27. Example: 3-tier datacenter network is represented as… Serial SCSI VLAN (iSCSI or FCoE) Internet VLANs Web/App VLANs App/DB VLANs Organization “a” Organization “b”
- 28. Serial SCSI VLAN(iSCSI or FCoE) Internet VLAN 1 Web/App VLAN 1 App/DB VLANs Organization “a” Organization “b” Zone 1 Internet VLAN 2 Internet VLAN 3 Internet VLAN 4 Web/App VLAN 2 Web/App VLAN 3 Web/App VLAN 4 Deployed Servers -org_a.webserver1 -org_a.webserver2 -org_a.webserver3 -org_a.webserver4 -org_a.webserver5 Future Servers -org_a.webserver6 -org_a.webserver7 -org_a.webserver8 Deployed Servers -org_b.webserver1 -org_b.webserver2 -org_b.webserver3 -org_b.webserver4 -org_b.webserver5 -org_b.webserver6 Future Servers -org_b.webserver7 -org_b.webserver8 Example: 3-tier data center network is represented as pools of connections
- 29. Pools of connections -example inventory status Internet VLAN 1 Web/App VLAN 1 (app) Organization “a” Organization “b” Zone 1 Internet VLAN 2 Internet VLAN 3 (dmz) Internet VLAN 4 Web/App VLAN 2 Pod 1 Connection Status Server ID VLAN IP MAC Policy Forms org_a.webserver.dmz.1 In Use org_a.webserver1 Internet VLAN 3 org_a.webserver.app.1 In Use org_a.webserver1 Web/App VLAN 1 org_a.webserver.dmz.2 In Use org_a.webserver2 Internet VLAN 3 org_a.webserver.app.2 In Use org_a.webserver2 Web/App VLAN 1 org_a.webserver.dmz.3 In Use org_a.webserver3 Internet VLAN 3 org_a.webserver.app.3 In Use org_a.webserver3 Web/App VLAN 1 . org_a.webserver.dmz.4 In Use org_a.webserver4 Internet VLAN 3 . org_a.webserver.app.4 In Use org_a.webserver4 Web/App VLAN 1 org_a.webserver.dmz.5 In Use org_a.webserver5 Internet VLAN 3 org_a.webserver.app.5 In Use org_a.webserver5 Web/App VLAN 1 org_a.webserver.dmz.6 AVAILABLE org_a.webserver.app.6 AVAILABLE org_a.webserver.dmz.7 AVAILABLE org_a.webserver.app.7 AVAILABLE org_a.webserver.dmz.8 AVAILABLE org_a.webserver.app.8 AVAILABLE org_b.webserver.dmz.1 In Use org_b.webserver1 Internet VLAN 4 org_b.webserver.app.1 In Use org_b.webserver1 Web/App VLAN 2 . . . . . .
- 30. Long-term Vision –Virtualize SANs also iSCSI LUN farm FC LUN farm Production “C: drives” VMware VMFS LUNs Virtual Tape devices Oracle DB LUNs Test/dev LUNs iSCSI SAN Exchange LUNs LUNs for NAS heads J2EE/SAP LUNs FC SAN Ethernet • Physical servers are exposed to resources only upon subscription • Inventories are adjusted per capacity management • Physical servers are not confined to any particular SANs • SANs can be smaller, more specialized, more manageable, with faster & simpler recovery Server Farm LU N /server binding
- 31. Evolution to Virtualization ApplicationSilos Dedicated Infrastructure – Designed, Procured, & Built Separately for each Application Virtual Resource Virtual Resource Virtual Resource Virtual Resource Application Stacks Configured one App at a Time Shared Infrastructure Design-to-Order Infrastructure Configure-to-Order Infrastructure
- 32. Evolution to InfrastructureClouds Virtual Resource Virtual Resource Virtual Resource Virtual Resource Application Stacks Configured one App at a Time Shared Infrastructure Allocate-to-Order Infrastructure Configure-to-Order Infrastructure Application Stacks Pools of Pre-Configured Standard Resources
- 33. Infrastructure Standards Menu Each Silo providesa portfolio of well-known “products” that are standard in that IT shop: Server Menu: - “small web server” - “medium web server” - “standard exchange server” - “small J2EE server” - “large ESX server” - “NFS cluster node” - “standard .Net app server” - “SAP financials container” - “large database node” Network Menu: - “DMZ connection” - “App-to-DB” connection” - “Backup network connection” - “Mgmt network connection” - “Vmotion network connection” Storage Menu: - “small VMFS LUN” - “High-perf DB LUN” - “J2EE LUN” - “small Windows C: clone” - “small NAS LUN” - “test/dev LUN” - “Exchange LUN”
- 34. Infrastructure Standards areLayered Small .web Server iSCSI Network Connection VM container Type 2 Large ESX Server BL585 Server Blade w/Flex10 NIC Vmotion Network connection Pods of blade chassis, servers, storage, and networking VMFS LUN Archive Network Connection DMZ Network Connection AppServer Network Connection
- 35. LUN Inventory Network Connection Inventory VirtualConnect Domain Small Web Server 1 J2EE Server 2 J2EE Server 1 Inventory Managed by Network Team Server Inventory Inventory Managed by Server Team Inventory Managed by Storage Team, Applications HP Vision – Adaptive Infrastructure Clouds Virtual Connect provides a Firewall between change domains Demand Supply
- 36. LUN Inventory Network Connection Inventory VirtualConnect Domain Small Web Server 1 J2EE Server 2 J2EE Server 1 Server Inventory Applications Thin-Provision Subscriptions – Pre-approve “Standard” Changes Forecast Demand Provision or Change Servers – “Subscribe” Allocate & Activate Subscriptions On-Demand 1. 2. 3. 4. Infrastructure Service Lifecycle
- 37. LUN Inventory Network Connection Inventory VirtualConnect Domain Small Web Server 1 J2EE Server 2 J2EE Server 1 Server Inventory Applications Domains are Separated & Simplified Server Change & Configuration Domain Network Change & Configuration Domain Storage Change & Configuration Domain Firewall between Domains
- 38. Lifecycle of anApplication Requirements Design Build Deploy Operate Optimize
- 39. Lifecycle of ApplicationsUsing Infrastructure Clouds (Standard Menus) Requirements Design Build Deploy Operate Optimize Connection Inventory Infrastructure Standards Menu Browse Menu vs. Requirements Select best Fit standard elements Subscribe to infrastructure from inventoryConfigure specifics on top of standard infrastructure Monitor & Maintain Remediate As Necessary