VMworld 2013: How SRP Delivers More Than Power to Their Customers

376 views

Published on

VMworld 2013

Sheldon Brown, SRP
Girish Manmadkar, VMware

Learn more about VMworld and register at http://www.vmworld.com/index.jspa?src=socmed-vmworld-slideshare

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
376
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

VMworld 2013: How SRP Delivers More Than Power to Their Customers

  1. 1. How SRP Delivers More Than Power to Their Customers Sheldon Brown, SRP Girish Manmadkar, VMware VAPP5105 #VAPP5105
  2. 2. 2 Agenda  Introduction  Before We Start - Homework  Figuring It All Out  What It All Looks Like Now  After the Dust Settled  Summary  Questions
  3. 3. 3 Introduction
  4. 4. 4 Girish Manmadkar – Consulting Architect With VMware since 2007 in various roles, focusing on providing VMware solutions around Business Critical Application with major focus around SAP More than 22+ years experience starting with SAP R4.6 Regular speaker at VMworld since 2007 Recent VMware Partner Exchange, EMCWorld events
  5. 5. 5 Sheldon Brown – Manager SAP Technology  Responsible for SAP Technology (BASIS,Development/Integration)  21 years with SRP  10 years with Server and Storage Team  2 years with SAP implementation team  9 of 10 years at VMWorld  sheldon.brown@srpnet.com
  6. 6. 6 Who We Are – Salt River Project Agricultural and Power District (SRP)  Power and Water supplier to the 1,000,000+ customers in the Phoenix metro area  Third largest public power company in the country  100+ years old  5000+ employees  Slow moving, risk adverse  70% virtualized  http://www.srpnet.com
  7. 7. 7 Home Work VMware Best Practices
  8. 8. 8 Best Practices  Use the latest processor generations, especially with EPT (Intel XEON 55xx, 65xx, 75xx and newer) or RVI (AMD Opteron 83xx, 84xx, 61xx and newer) which bring performance benefits  Assigning vCPUs to virtual machines • ESXi 5.1: shows scaling for virtual machines > 8 vCPUs • Understand the v/NUMA architecture for larger VMs  You can overcommit CPUs, but reasonably • Start with no CPU overcommitment. When the average utilization of SAP instances and the average idle time on the host are known, accordingly raise the number of vCPUs on the host (for example, adding virtual machines). Use VMware vSphere Distributed Resource Scheduler (DRS) capabilities to control and balance overall workload. • If performance problems are seen, first reserve CPU to see if the performance problems are a result of resource contention
  9. 9. 9 Best Practices (cont.) • Do not overcommit memory. This is not recommended at all with SAP. Reserve 100% of the assigned memory. • Installation of VMware Tools is mandatory to avoid time conflicts and have all necessary drivers (for example, new network cards) • Timekeeping: Use NTP (Network Time Protocol) on the ESX host and guest as per Timekeeping best practices for Windows, including NTP and SAP Note 989963 http://kb.vmware.com/kb/1318 • Use multiple virtual SCSI controllers for the database virtual machine and virtual machines with high I/O load. The use of multiple virtual SCSI controllers allows the execution of several parallel I/O operations inside the guest OS. • SAP requires the implementation of the new virtualization aware monitoring. See SAP Note 1409604. • When using VMware snapshots, follow Best practices for virtual machine snapshots in the VMware environment. Make sure that the virtual machine has no active snapshot before reporting performance problems. http://kb.vmware.com/kb/1025279
  10. 10. 10 Best Practices for Linux  Disabling the Linux I/O scheduler • Set kernel parameter "elevator=noop" • Red Hat KB 5428 (applies to RHEL and SLES)  Choose the optimal SAP memory model • MAP memory model (SAP Note 386605) for CPUs without EPT/RVI • STD memory model (SAP Note 941735) for CPUs with EPT/RVI
  11. 11. 11 vSphere – ESX CPU Scheduling  Determine size of hardware NUMA nodes on the server  Size the virtual machine as small as possible • Too many vCPUs creates scheduling overhead • Too much RAM results in accessing RAM from a far NUMA node  Enable simultaneous multithreading • More hardware execution contexts are available  Generally CPU affinity not required
  12. 12. 12 Best Practices for Databases  Oracle • SAP Note 1173954 states support with minor restrictions • Oracle on VMware vSphere Essential Database Deployment Tips http://www.vmware.com/resources/techresources/10101 • VMware Alliances statement http://www.vmware.com/solutions/partners/alliances/oracle-vmware- support.html
  13. 13. 13 Best Practices for Databases – I/O Subsystem  Store data files and log files on separate physical devices and distribute data files as per SAN Volume Controller as defined by the storage vendor’s best practices  Quote from SAP on SQL Server best practices guide • “The number of disks is determined by the throughput in I/O operations per seconds (IOPS). Therefore discussions around storage investments or configurations should be driven by IOPS” • http://www.sdn.sap.com/irj/scn/index?rid=/library/uuid/4ab89e84-0d01-0010- cda2-82ddc3548c65  IOPS used by storage and OEM vendors to size SAP storage on VMware – Similar to physical  The standard SAP rules for databases apply! • SAP Note 806554 for general I/O intensive DB tasks • SAP Note 592393 and 793113 for Oracle
  14. 14. 15 Best Practices for vStorage APIs (cont.)  VMware vSphere® Storage APIs – Array Integration (VAAI) • Has nothing to do with backup – it is a better way of integrating storage capabilities into vSphere • vStorage APIs for Array Integration FAQ http://kb.vmware.com/kb/1021976 • Follows the vSphere lifecycle • Offloads specific storage operations to compliant storage hardware • Write Same/Zero • Eliminating redundant and repetitive write commands • Fast/Full Copy • Leveraging array ability to mass copy, snapshot, and move blocks • Hardware Offloaded Locking (ATS) • Stop locking LUNs and start locking only blocks
  15. 15. 16 Best Practices for Network  Standard vSphere networking guidelines apply • Separate infrastructure traffic from virtual machine traffic: use physical NICs or VLANs • Use NIC teaming for availability and load balancing (group physical NICs connected to the same physical network) • Take advantage of network I/O controls to converge network and storage I/O onto 10GbE • Take advantage of the dvSwitch and/or Cisco Nexus 1000v to simplfy the architecture and support  Use VMXNET3 adapter in guest OS • Improved performance • Helps with network traffic in three-tier setup between application and DB VM • SAP Batch Job Performance on vSphere http://blogs.vmware.com/performance/2010/02/sap-batch-job-performance-on- vsphere.html
  16. 16. 17 Best Practices – Further Information  SDN Forum SAP on VMware • http://forums.sdn.sap.com/forum.jspa?forumID=471 • Most complete link collection around SAP on VMware  SAP Notes • Following the SAP Notes is mandatory!  VMworld Sessions • Various documents – from high level overview to technical deep dive  Performance Tests • Continuous performance measurement for databases and SAP software
  17. 17. 18 Figuring It All Out
  18. 18. 19 Figuring It All Out – SAP Architecture and Components
  19. 19. 20 Figuring It All Out – the Team  Team Effort  Consultants • Accenture • SAP • HP • VMware • NetApp • Cisco  SRP Employees • Server • Storage • Network • BASIS • Strategy and Planning • Executive Management
  20. 20. 21 Figuring It All Out – What we needed to decide  Datacenters  Host Servers  Virtual Servers  Storage  Network  Database  Operating System  Infrastructure Security
  21. 21. 22 Figuring It All Out – SAP Environment Description ECC, SRM, PI, BW, SolMan, BOBJ, GRC, Portal, Data Services Bolt Ons • Vertex • OpenText • BPC Landscapes (Production, Quality, Test, Development, Sandbox, Training) • Production • Quality • Test • Development • Sandbox (Technical and Functional) • Training 5000+ users 850+ unique per day
  22. 22. 23 Figuring It All Out – Quick Sizer Results (nothing “quick about it”) User Based SAPs Memory and Disk Sizing
  23. 23. 24 What It Looks Like Now
  24. 24. 25 What It All Looks Like Now – Datacenters and Network  Two datacenters • 30 miles by the wire apart • Connected by 20Gbps (Layer 2) 16 Gbps (Layer 3)  Network • Cisco 7k series core • Cisco 5k series edge • 10Gbps network
  25. 25. 26 What It All Looks Like Now – Physical Servers  24 ESX Hosts (2 clusters of 12 at each data center) • HP DL580 G7 • vSphere 5.1 U1 • 4 proc 10 core • 512 GB RAM • 2 10GBps ports  6 ESX Host Stretch Cluster (3 at each data center) • HP DL580 G7 • vSphere 5.1 U1 • 4 proc 10 core • 256GB RAM • 2 10GBps ports
  26. 26. 27 What It All Looks Like Now – Virtual Servers  Production – 76 servers  Quality – 59 servers  Test – 27 servers  Development – 32 servers  Sandbox (Technical and Functional) – 40 servers  Training – 3 servers  Administration – 3 servers  Total – 240  Operating Systems – Red Hat Enterprise Linux (RHEL) 5.x to 6.2 and Windows 2008 R2
  27. 27. 28 What It All Looks Like Now – Storage  4 NetApp 6280’s (2 at each data center) • NetApp Data ONTAP Release 8.1.2P3 7-Mode • Theoretical capacity 77,000 IOPS (440 disk * 175 IOPS) • Highest recorded capacity 8212 IOPS • Raw Capacity:154,493GB or 150TB • Consumed Capacity: 61519GB or 60TB • Extensive use of Snap Shot Manager for SAP
  28. 28. 29 What It All Looks Like Now – Network Architecture Datacenter 2Datacenter 1
  29. 29. 30 What It All Looks Like Now – SAP vSphere Host Configuration Datacenter 1 Datacenter 2
  30. 30. 31 What It All Looks Like Now – Storage Architecture
  31. 31. 32 What It All Looks Like Now – Monitoring  SAP Solution Manager – Application  Quest vFoglight – Virtual Server Performance  Top, Satellite – Operating System  CA Wily Interscope – Java  NetApp ONCommand Unified Manager – Storage  HP SIM – Server Hardware  Q1, Cascade Shark, eHealth – Network  CA Spectrum – Operations
  32. 32. 33 What It All Looks Like Now – Database  Oracle 11g (Corporate Standard)  DataGuard for replication  One virtual server per database  Unlimited Oracle for SAP
  33. 33. 34 What It All Looks Like Now – Disaster Recovery/High Availability  Completed a Business Impact Analysis  12 hour RTO  Close to zero RPO  Load-balanced dialog instances  Central Messaging instance at primary datacenter  Database instance at primary datacenter  Database replication happening real time
  34. 34. 35 Datacenter 1 What It All Looks Like Now – Disaster Recovery/High Availability Replication Log Shipping Datacenter 1 Datacenter 2
  35. 35. 36 What It All Looks Like Now – Why We Virtualized SAP on VMware  Cost Effective  Flexibility  Higher Availability  Faster Deployment  Easier to recover in disaster  Higher Server Density = Lower Data Center Costs  Virtualization is the SRP corporate standard  VMware is the leader  Favorable license agreement with SAP
  36. 36. 37 What It All Looks Like Now – Refresh Methodology  Prepare the source and target systems  Snap Clone production using Snapshot Manager for SAP • Mount cloned copies to targeted servers • Changes the SID • Minutes instead of hours  Post clone cleanup  Validate target systems  Scrub production data  Enjoy
  37. 37. 38 After the Dust Settled
  38. 38. 39 After the Dust Settled - Lessons Learned  Overbuild  Plan for disaster now  Plan out storage configuration early  Pay attention to security early • Infrastructure • OS and Application hardening
  39. 39. 40 After the Dust Settled - Successes  Flexibility  Speed in reaction  Environment Refresh rate  Spin up environments quickly  Saved $1,000,000 in hardware by virtualizing Oracle
  40. 40. 41 After the Dust Settled - Things to Come  Release 2 • Work Management? • HR • More Supply Chain • More Finance • Customer Information?  Mobility?  HANA?
  41. 41. 42 Summary  Now that we are done – don’t forget your home work  Figure it all out  What It All Looks Like Now  After the Dust Settled
  42. 42. 43 Questions
  43. 43. 44 Other VMware Activities Related to This Session  HOL: HOL-SDC-1301 Applied Cloud Operations
  44. 44. THANK YOU
  45. 45. How SRP Delivers More Than Power to Their Customers Sheldon Brown, SRP Girish Manmadkar, VMware VAPP5105 #VAPP5105

×