Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Designing Scalable SAN using MDS 9396S

2,593 views

Published on

Find out how the unique architecture of MDS 9396S innovative, next-generation switch enables you to design a high-performing, scalable Fibre Channel SAN. Learn best practices for supporting flash storage applications as well as a wide variety of deployment scenarios. See how the new Cisco MDS 9396S can meet your SAN challenges today and tomorrow as we reveal:

• Architectural innovations inside the Cisco MDS 9396S

• Enterprise-class features and scale options

• Design and deployment scenarios

• Customer-tested best practices for implementation

Published in: Technology
  • Be the first to comment

Designing Scalable SAN using MDS 9396S

  1. 1. using Cisco MDS 9396S Designing Scalable SAN Paresh Gupta, paregupt@cisco.com Technical Marketing Engineer, Cisco August 2015
  2. 2. Cisco Multi-Protocol Product Portfolio: SAN, LAN, and Compute 10+ Years of Proven NX-OS Operating System Cisco Prime Data Center Network Manager (DCNM) CONSISTENT AND SIMPLIFIED Features, Management, and Programmability Cisco UCS C-Series Rack Servers Cisco UCS B-Series Blade Servers Cisco UCS Fabric Interconnects Cisco UCS 6248UP Cisco UCS 6296UP LAN/SAN Cisco Nexus 9000 Cisco Nexus 7000 Cisco Nexus 5600 Cisco Nexus 5500 Cisco Nexus 3000 Cisco Nexus 2000 Cisco MDS 9700 48x16G Line-Rate FC Cisco MDS 9250i Cisco MDS 9148S 48x10G Line-Rate FCoE SAN 16G COMPUTE Cisco MDS 9396S July 2015
  3. 3. Methodology for a continuous life-cycle of a SAN Prepare Plan Design Implement Operate Optimize Storage Area Network
  4. 4. Propose an architecture to meet business requirements and goals Preparation phase Prepare Top quality user experience and application performance Maximize availability (Reduced downtime) Lower TCO (Reduced CapEx & OpEx) Expandable architecture to support future business growth Define Business Goals Create business plan with cost-benefit analysis
  5. 5. Gather network requirements, existing gap analysis and project plan Planning phase Human resource planning Environmental planning (Power, Real State, etc) Gap analysis, Risks and Constraints Project management, timeline and mile stones Create Project Plan Sourcing options (Vendor selection) Plan
  6. 6. MDS 9396S achieves your business goals Prepare Plan
  7. 7. • Best of both worlds: Performance of a Director in a fabric switch! • 96 ports at line rate 16 Gbps Fibre Channel in compact 2 RU • Non-blocking, non-oversubscribed architecture • Consistent and Predictable performance across all ports • No local switching dependency and restrictions Top Quality User Experience & Application Performance MDS 9700 48 port, 16Gbps FC line card MDS 9396S delivered by High Performance of MDS 9396S
  8. 8. • Two hot swappable fan trays • Choice of reversible airflow • Port side exhaust at FCS • 2 hot swappable power supplies • 1+1 grid redundancy • In Service Software Upgrade • 16 member ports in single port-channel (Link Aggregation) • High Availability is achieved by spreading members across different ASIC, port-groups, etc • Add or remove links as needed without traffic loss • Dual Redundant Power Supply (for Grid Redundancy) • Dual Redundant Fan Trays Maximum availability delivered by Robust Architecture of MDS 9396S
  9. 9. • Base configuration: 48 ports or 96 ports • 48 port base can be upgraded with 12 port on demand license • Optional Licenses : • Enterprise License for Extended B2B credits, QOS, IVR, VSAN based access control • DCNM License for Fabric Management • DCNM + Enterprise license bundle Expandable architecture: Support future business Growth delivered by Flexible Licensing 48 Ports 12 Ports Increment 12 Ports Increment 12 Ports Increment 12 Ports Increment
  10. 10. • Leadership across Collaboration, Security, High-End Routing, Switching, Wireless, Cloud and Data Center Verticals, not just point products • Leadership across LAN, SAN & Compute in Data Center • SAN: Industry partnership and Ecosystem • SAN: Strong Investment Protection, backward compatibility Solution provider, not just a point product vendor delivered by Cisco’s ability to develop and support complete enterprise IT infrastructure Programmable: Integration with Industry-Leading Cloud Platforms EMC ViPR Software Defined Storage OpenStack Microsoft System Center VMM IBM PowerVC Automate Zoning for faster application deployments User-Centric Models for Cloud- Scale Workloads Cisco MDS 9000 Family: Extensive Industry Partnerships Cisco Channel Partners MDS 9300/9200/9100 Only Other Flash Vendors Storage Array interop only The Power of Ecosystem Vblock FlexPod HDS UCP Select VSPEX Versa Stack UCS for Red Hat OpenStack Smart Stack Pre-Packaged Converged Infrastructures
  11. 11. Tight integration between Cisco UCS and MDS 9396S F-Port Channel VSAN Trunking F-Port Trunking for Multi-Tenancy • Servers log in & participate in separate VSANs • Any UCS blade can connect over any N-Port uplink across which the VSAN is trunked F-Port Channel for Resiliency • Port Channel between UCS & MDS: No Port-Pinning • No host relogin on individual link failure in Port Channel • No Manual re-balancing when new link added or removed Operational Simplicity: NX-OS, DCNM • Common operating system for MDS & UCS Fabric Interconnect (FI) • Visibility of UCS FI and MDS in DCNM • Cisco TAC as common support system MDS 9396S UCS FI Cisco UCS and Cisco MDS 9000 Family: Better Together White Paper http://www.cisco.com/c/en/us/products/collateral/storage-networking/mds-9700-series-multilayer-directors/white- paper-c11-734381.html
  12. 12. Designing a network to meet Business and technical Requirements Design phase Design High Availability Scalability Security Manageability Design the network Flexibility
  13. 13. Designing small single-switch SAN using MDS 9396S Consolidation of SAN Islands E-mail Database BackupE-mail E-mail Database Start with 48 port base license with single VSAN for E-Mail Servers Add 12 port license to accommodate Database servers Add another 12 port license to add Backup servers MDS 9396S Minimizes no. of SAN switches Centralized management Blue VSAN Red VSAN Yellow VSAN Reduced TCO with best user experience & application performance Secure & Flexible Design 48 – 96 Ports
  14. 14. Designing small multi-switch SAN using MDS 9396S Reduced TCO with support for business expansion Secure & Flexible Design MDS 9396S MDS 9396S • Deploy in N-Port Virtualization (NPV) or N-Port ID Virtualization (NPIV) mode to reduce number of managed switches • Maximize localization of servers and Storage on single switch: Reduced ISL traffic BackupE-mail MDS 9396S MDS 9148S E-mail Database 12 – 192 Ports
  15. 15. Designing enterprise class SAN using MDS 9396S Reduced OpEx & CapEx by reduction in no. of edge switches, power, cooling &cabling requirements with readiness for future growth Massive Scalable Design • MDS 9396S as Middle of Row (MoR) switch to consolidate server connections • Even reduced number of managed devices with MDS 9396S as NPV & MDS 9700 as NPIV • Quality of Service (QoS) for traffic prioritization MDS 9700 MDS 9396S MDS 9396S MDS 9700 Edge-Core Architecture – Consolidation at Access Layer 1000s of Ports
  16. 16. SAN Extension using MDS 9396S Business Continuity and Disaster Recovery Design to connect distant SAN islands • Up to 25 KM: Use of CWDM optics at 8G or ELW optics at 16G • 512 KM @ 16G FC with DWDM infrastructure (Up to 4095 B2B credits per port with enterprise license) Primary Site MDS 9396S MDS 9396S MDS 9250i MDS 9250i Native Fibre Channel based MAN / WAN IP MAN / WAN for FCIP Backup Site Disk replication, Tape backups FC TrustSec Encryption IPSec Encryption
  17. 17. MDS 9396S achieves your business goals Prepare Plan Design
  18. 18. Quick provisioning of network as per design Implementation phase Implement Connect the devices as per design Configure the devices Validate the configuration and connectivity Training of support personals Implement the Design Quick & automated implementation to minimize time to deploy
  19. 19. • Intuitive approach to configure and Manage a Single Switch • Point browser to the management IP address of the switch to access Device Manager Cisco Device Manager Watch Full demo (and many other videos) at youtube.com/ciscomds9000
  20. 20. • Unlike LAN switches, SAN switches require zoning for Servers and Storage Array to communicate. • Quick Config Wizard configures zoning within minutes Quick Configuration Wizard Expedite Return on Investment
  21. 21. Quick Configuration Wizard
  22. 22. As number of switches increase, Configuring Single switch at a time is not a viable option
  23. 23. Power on Auto Provisioning (POAP) on MDS 9396S Rapid Error Free Deployment • Automation • Consistency • ‘Go-Live’ time in minutes • No Human Errors • Accuracy • Reliability MDS 9396S ………………………………………………………………….. Provides: •IP Address, Gateway •Software Image •Configuration File
  24. 24. Maintain Network Health to support business operations Operating a SAN Operate Provisioning of new Servers & Storage Performance Monitoring Health Monitoring Incident Management Day-to-Day Operations Fault Detection
  25. 25. MDS 9396S achieves your business goals Prepare Plan Design Operate
  26. 26. • Options of zoning via Intuitive NX-OS CLI • Zoning can be done easily via DCNM • Zoning Automation available via Industry leading Platforms: Cisco UCS Director, EMC ViPR, Microsoft System Center VMM, IBM Power PC, OpenStack. Simplified & Efficient Zoning operations using MDS 9396S
  27. 27. Cisco Enhanced zoning on MDS 9396S Secure and Resilient OperationsEnhanced Zoning Serializes Configuration to Streamline Parallel Config Attempts from Multiple Users HOST1 Users Acquire Fabric Lock One at a Time to Make Zone Config Changes TARGET1 TARGET2 TARGET3 SWITCH1 SWITCH3 SWITCH2 SWITCH4 HOST2 HOST3 HOST4 TARGET4
  28. 28. • 8 Initiators & 4 targets in single zone create 132 entries (12 x (12-1)) within a SAN switch • Ideal number of entries = 32 • SAN admins pay the price on internal inefficiency by creating 2-member zones. Too much of overhead for day-to-day operation! • Solution: Cisco Smart Zoning • Create single zone with all initiators and targets. MDS 9396S is smart to create just ideal number of entries (32) Cisco Smart Zoning on MDS 9396S Simpler, Efficient, IntuitiveSmart Zoning I T I I I I I I I T T T
  29. 29. Visualizing the benefit of Smart Zoning on MDS 9396S Smart Zoning Requires only 1 zone with 12 hosts and 2 X-Brick Zoneset name Smart-VDI vsan 200 zone name x-brick1 vsan 200 pwwn 15:10:10:10:10:10:10:01 initiator pwwn 15:10:10:10:10:10:10:02 initiator pwwn 15:10:10:10:10:10:10:03 initiator pwwn 15:10:10:10:10:10:10:04 initiator …….. pwwn 15:10:10:10:10:10:10:12 initiator pwwn 25:20:20:20:20:20:20:01 target pwwn 25:20:20:20:20:20:20:02 target Traditional Zoning (Requires 24 two-member zones) Zoneset name VDI vsan 100 zone name host1-x-brick1a vsan 100 pwwn 10:10:10:10:10:10:10:01 pwwn 20:20:20:20:20:20:20:01 zone name host1-x-brick1b vsan 100 pwwn 10:10:10:10:10:10:10:01 pwwn 20:20:20:20:20:20:20:02 zone name host2-x-brick1a vsan 100 pwwn 10:10:10:10:10:10:10:02 pwwn 20:20:20:20:20:20:20:01 zone name host2-x-brick1b vsan 100 pwwn 10:10:10:10:10:10:10:02 pwwn 20:20:20:20:20:20:20:02 . . . . . . . zone name host12-x-brick1a vsan 100 pwwn 10:10:10:10:10:10:10:12 pwwn 20:20:20:20:20:20:20:01 zone name host12-x-brick1b vsan 100 pwwn 10:10:10:10:10:10:10:12 pwwn 20:20:20:20:20:20:20:02 Adding another VDI cluster would require an additional 72 zones to be configured per fabric. Adding another VDI cluster would require adding the new storage ports and new hosts to the existing zone or adding the 2 new storage pwwns to the first zone and creating one additional zone with all 4 storage ports together with the new hosts.
  30. 30. Resilient Operations on MDS 9396S Operate • Faulty equipment, loose SFPs, dirty or damaged cables, can result in packets getting corrupted • CRC checking drops corrupted frames from end devices or internally corrupted frames • FEC corrects frames corrupted in-flight to preserve frames Ingress CRC checking Internal CRC checking Forward Error Correction Drop frame Drop frame CRC Detection and 16 Gbps FC Forward Error Correction (FEC)
  31. 31. MDS 9396S for Self Healing SAN Misbehaving device “slowly drains” frames destined to it. Unrelated flows and devices get affected MDS auto-detects slow drain devices and takes recovery actions Potential Reasons for Slow-Drain Devices • Speed Mismatch • HBA issues • Server Performance issues – application/OS • Non graceful Virtual Machine exit Affected Host Misbehaving HBA Detection 2.5 µs Automatic Recovery Immediate Troubleshooting DCNM Slow Drain Diagnostics
  32. 32. Detecting and Troubleshooting Slow Drain using DCNM Automates Troubleshooting Reduces False Positives Shows Fluctuations in Counters Watch Full demo (and many other videos) at youtube.com/ciscomds9000
  33. 33. Programmability using MDS 9396S OpenStack Integration Full benefits of OpenStack along with seamless zoning Embedded Event Manager (EEM) NX-OS infrastructure to take user defined events on actions Native on-switch TCL Interpreter Near-Infinite possibilities with your own custom script Scheduler Capability to schedule any task on the switch (like Linux cron)
  34. 34. Programmability Use Case: Config Checkpoint Config backup at remote server after each change Switchname_Timestamp.cfg Functionality Behind the Scene EEM catches config change event Triggers TCL script on switch bootflash Script copies running_config to remote server
  35. 35. Maximize the performance with proactive management Optimizing a SAN Network Capacity Trending Network Health Trending Aim to improve uptime Aim to reduce number of incidents Day-to-Day Operations Take all the learnings to Prepare Phase Optimize
  36. 36. MDS 9396S Health Score on DCNM HEALTH SCORE TRENDING Identify and Remediate HEALTH SCORE TRENDING IF YOU CAN MEASURE IT, YOU CAN FIX IT 80% Health 60 % Health Policy-Based Health Index Quickly Determine Level of Risk View Alerts Contributing to Low Score Measure Success Overtime Restore Health back Normal MDS 9396S MDS 9396S
  37. 37. MDS 9396S achieves your business goals Prepare Plan Design OperateImplement Optimize
  38. 38. • Current Infrastructure unable to support user growth and data explosion • Planning for Access Network Consolidation to reduce CapEx • Already have large UCS install base and growing • Due to pressure on OpEx: Need for Programmability, Automation and Manageability • Unhappy with long history of Fork Lift upgrades with every speed change. Business Goals • Best in class user experience: Support high demanding application (Today & Future) • Investment Protection for 10+ years • Minimize CapEx and OpEx • Consolidated global IT strategy and architecture Design under exploration: Large 16 Gbps FC customer Prepare Plan
  39. 39. Design under exploration MDS 9396S MDS 9706 MDS 9710 . . . . . 80 • MDS 9706 as Storage Edge • MDS 9710 as Core (Logically Server Edge) • MDS 9396S as Top of Row (TOR) switch shared among servers at multiple racks • Single rack has 20 – 30 servers • F-Port Channel between NPV MDS 9396S & MDS 9710 NPIV for high availability • Single NPIV MDS9710 will be attached to 80 NPV MDS 9396S in a single fabric • POAP to provision hundreds of MDS 9396S Design
  40. 40. Customer meets Business Goals using Cisco MDS MDS 9396S MDS 9706 MDS 9710 . . . . . 80 Best in Class User Experience High performance, non-oversubscribed, no-local switching Cisco MDS as choice for multiple data centers across globe along with Cisco Nexus and UCS Consolidated global IT strategy Large CapEx saving by using MDS 9396S as Server Edge. OpEx saving by POAP, DCNM Minimize CapEx & OpEx MDS 9700 ready for 32 G FC & 40 G FCoE today. ISLs can easily be upgraded without forklift Investment Protection By high performing, non- oversubscribed, predictable, consistent, no-local switching architecture Best in Class User Experience
  41. 41. • Existing Infrastructure unable to support user growth and data explosion • Existing infrastructure has shown inconsistent and unpredictable behavior under load • Existing infrastructure known to forward corrupt frames causing fabric wide disruption Business Goals • Increase efficiency of VDI application running at Point of Sale (POS) to reduce time for each customer • Stores located every few KMs. Maintain backup at central Data Center • Reduced CapEx and OpEx • Capacity for future expansion Design under exploration: Flash Storage for VDI Prepare Plan
  42. 42. Flash Storage for VDI using MDS 9396S • MDS 9396S as SAN switch between VDI application servers and Flash Storage • Predictable & Consistent Performance provides freedom to connect Storage & Servers to any port on the switch without worrying local-switching complexity. • Ingress CRC checking & dropping of corrupt frames will limit failure domains just to the affected device • Operational simplicity by Smart zoning • End-to-End visibility (VM & LUN awareness) by DCNM. • MDS 9396S for long distance FC native synchronous replication over dark fiber to central data center Design Access VSAN Replication VSAN Dark Fibre to central Data Center – Continuous backup of store operations FC TrustSec Encryption
  43. 43. MDS 9396S achieves Business Goals for Flash Storage Best in Class User Experience High performance, non-oversubscribed, no-local switching By unmatched licensing flexibility of MDS 9396S. Customer starts with 48-port base & increments with 12-ports on-demand license Capacity for Future Expansion By consolidation of Access & Replication SAN on same switch using Cisco VSANs. OpEx saving by DCNM Minimize CapEx & OpEx By synchronous replication between store & Central Data Center. Long distance FC link possible due to large number of B2B credits on MDS 9396S Backup at Central Data Center By high performing, non-oversubscribed, predictable, consistent, no-local switching architecture of MDS 9396S. Cisco MDS + UCS + EMC XtremeIO has been validated for > 1M IOPS Increased VDI Application Performance Access VSAN Replication VSAN Dark Fibre to central Data Center – Continuous backup of store operations FC TrustSec Encryption
  44. 44. Summary
  45. 45. Prepare Plan Design Implement Operate Optimize MDS 9396S Coordinated Planning and Strategy Assess Readiness Design the Solution Implement the Solution Maintain Network Health Operational Excellence
  46. 46. 2RU form factor NPIV No Oversubscription POAP Slow Drain Automatic Recovery Robust Architecture FEC Industry wide EcoSystems 16 Gbps FC Line Rate Affordable Non-Disruptive Software Upgrade Flexible Licensing NPV 16 member PortChannel Link Encryption Predictable Performance Redundant Fan Trays Quick ConfigurationFEC Consistent Latency Cisco MDS + UCS Synergies Slow Drain detection at 2.5µs MDS 9396SSmart Zoning Deploy today! 96 ports
  47. 47. Reference
  48. 48. B2B credit requirement increases with Extending optical FC SAN : B2B credit requirement Frame Size 1 Gbps 2 Gbps 4 Gbps 8 Gbps 10 Gbps 16 Gbps 512 Bytes 2 BB/km 4 BB/km 8 BB/km 16 BB/km 24 BB/km 32 BB/km 1024 Bytes 1 BB/km 2 BB/km 4 BB/km 8 BB/km 12 BB/km 16 BB/km 2112 Bytes 0.5 BB/km 1 BB/km 2 BB/km 4 BB/km 6 BB/km 8 BB/km Distance Speed Frame size B2Bcredit requirement
  49. 49. Understanding B2B credits on MDS 9396S • All ports in a port group can have maximum of 500 B2B credits • Enterprise license enables extended credits that means up to 4095 B2B per port in a port group • Best Practice: Avoid grouping all E ports in same port group/IOSlice • Generic Formula: 0.5 B2B credits are required for every 1 KM distance at 1Gbps for standard FC frame (2112 bytes). Refer to previous slide • CLI command: show port-resources module 1 Port Group 1 Port Group 2 Port Group 3 Port Group 4 Port Group 5 Port Group 6

×