Successfully reported this slideshow.
Performance Monitoring and Testingin the Salesforce CloudYour Success. Our CloudRaj Advani, salesforce.com, Sr. Technical ...
Raj AdvaniSr. Technical Solution Architectradvani@salesforce.com
Randy CaseDirector, Customer Centric Engineeringrcase@salesforce.com
Safe Harbor Safe harbor statement under the Private Securities Litigation Reform Act of 1995: This presentation may contai...
Agenda What is performance testing The scalable Salesforce architecture Internal Salesforce performance testing Why do you...
What is Performance Testing?    ‘Performance testing is typically done to help identify bottlenecks in asystem, establish ...
What is Performance Testing?        ‘Performance testing is typically done to help identify bottlenecks in a    system, es...
Agenda What is performance Testing The Scalable Salesforce Architecture Internal Salesforce Performance Testing Why do you...
Transactional Volumes
It’s in the numbers!                                     104,000+ businesses   >60   billion              requests/quarter...
Multi-tenant, Cloud Computing Platform
Pod Architecture Enhances Availability, Scale, and Platformfor Future Growth     NA0                                APAC  ...
A Scalable, High Performing Architecture Database cluster   8 Node RAC cluster   Scalability and fault tolerance Applica...
Scalable, High Performing Architecture Network   10 carriers for geographic    performance and redundancy.   Data center...
Agenda What is performance Testing The Scalable Salesforce Architecture Internal Salesforce Performance Testing Why do you...
Performance Engineering      Primarily focuses on benchmarking,      simulation, tuning, analysis, and      monitoring/pro...
Performance Engineering Performance testing typically falls into two major categories.   Playback-based:      • Plays bac...
What’s Tested?    Full       Database                               Programmatic                            User Interface...
Capacity Planning                                              Multi year growthProactive Approach    Capacity alerts     ...
Agenda What is performance Testing The Scalable Salesforce Architecture Internal Salesforce Performance Testing Why do you...
Candidates for Performance TestingHeavy Custom Development • Testing should be geared towards exercising the custom code o...
What Is Performance Testing             Response Time Based Testing      Confirm that a set of VF Pages and Apex      cont...
What is Performance Testing? Performance Testing categories   Load, Endurance, Spike, Configuration, Isolation. Performan...
Metrics Collected Page Response      Server Response     Transaction     Times               Times        Throughput Rate ...
Agenda What is performance Testing The Scalable Salesforce Architecture Internal Salesforce Performance Testing Why do you...
Recommended Performance Testing Approach      Build Test Plan  •Identify Key Business     Run Baseline Test  Transactions ...
Performance Testing Tools Salesforce supports most all major performance testing tools Tools capture all data and metrics ...
Engaging Salesforce for Performance Testing Create tests that measure transaction and business process time. Create a test...
Wrap Up • Most clients will not need to conduct their own testing  • Average of 60 billion transactions a day with an aver...
Raj Advani                  Randy Case Sr. Technical Architect,     Director, Softwareradvani@salesforce.com           Eng...
Performance Monitoring and Testing in the Salesforce Cloud
Upcoming SlideShare
Loading in …5
×

Performance Monitoring and Testing in the Salesforce Cloud

18,465 views

Published on

  • Hi,

    How does one generate with real world type data and populate Salesforce test instances with this data that may be needed for performance testing? Does Salesforce provide any services to generate this type of data if one's own production org does not have it currently?
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Performance Monitoring and Testing in the Salesforce Cloud

  1. 1. Performance Monitoring and Testingin the Salesforce CloudYour Success. Our CloudRaj Advani, salesforce.com, Sr. Technical Solution ArchitectRandy Case, salesforce.com, Director, Customer Centric Engineering
  2. 2. Raj AdvaniSr. Technical Solution Architectradvani@salesforce.com
  3. 3. Randy CaseDirector, Customer Centric Engineeringrcase@salesforce.com
  4. 4. Safe Harbor Safe harbor statement under the Private Securities Litigation Reform Act of 1995: This presentation may contain forward-looking statements that involve risks, uncertainties, and assumptions. If any such uncertainties materialize or if any of the assumptions proves incorrect, the results of salesforce.com, inc. could differ materially from the results expressed or implied by the forward-looking statements we make. All statements other than statements of historical fact could be deemed forward-looking, including any projections of product or service availability, subscriber growth, earnings, revenues, or other financial items and any statements regarding strategies or plans of management for future operations, statements of belief, any statements concerning new, planned, or upgraded services or technology developments and customer contracts or use of our services. The risks and uncertainties referred to above include – but are not limited to – risks associated with developing and delivering new functionality for our service, new products and services, our new business model, our past operating losses, possible fluctuations in our operating results and rate of growth, interruptions or delays in our Web hosting, breach of our security measures, the outcome of intellectual property and other litigation, risks associated with possible mergers and acquisitions, the immature market in which we operate, our relatively limited operating history, our ability to expand, retain, and motivate our employees and manage our growth, new releases of our service and successful customer deployment, our limited history reselling non-salesforce.com products, and utilization and selling to larger enterprise customers. Further information on potential factors that could affect the financial results of salesforce.com, inc. is included in our annual report on Form 10-Q for the most recent fiscal quarter ended July 31, 2012. This documents and others containing important disclosures are available on the SEC Filings section of the Investor Information section of our Web site. Any unreleased services or features referenced in this or other presentations, press releases or public statements are not currently available and may not be delivered on time or at all. Customers who purchase our services should make the purchase decisions based upon features that are currently available. Salesforce.com, inc. assumes no obligation and does not intend to update these forward- looking statements.
  5. 5. Agenda What is performance testing The scalable Salesforce architecture Internal Salesforce performance testing Why do you need to test? Performance testing approach and testing tools Questions and Answers
  6. 6. What is Performance Testing? ‘Performance testing is typically done to help identify bottlenecks in asystem, establish a baseline for future testing, support a performancetuning effort, determine compliance with performance goals andrequirements, and/or collect other performance-related data to helpstakeholders make informed decisions related to the overall quality of theapplication being tested. In addition, the results from performance testingand analysis can help you to estimate the hardware configurationrequired to support the application(s) when you “go live” to productionoperation. ‘ WOW!6
  7. 7. What is Performance Testing? ‘Performance testing is typically done to help identify bottlenecks in a system, establish a baseline for future testing, support a performance tuning effort, determine compliance with performance goals and requirements, and/or collect other performance-related data to help stakeholders make informed decisions related to the overall quality of the application being tested. WOW!7
  8. 8. Agenda What is performance Testing The Scalable Salesforce Architecture Internal Salesforce Performance Testing Why do you need to test? Performance Testing Approach and Testing Tools
  9. 9. Transactional Volumes
  10. 10. It’s in the numbers! 104,000+ businesses >60 billion requests/quarter < 275 ms avg. response time 100s thousands users 100s gigabytes of data >800 million >40 million account records operations/day million contact records >20 >30 million opportunity records 100s millions custom object records
  11. 11. Multi-tenant, Cloud Computing Platform
  12. 12. Pod Architecture Enhances Availability, Scale, and Platformfor Future Growth NA0 APAC Pod Pod Network NA1 Sandbox Pod Services Pod NA2 Storage EMEA2 Pod Services Pod NA3 Backup NA6 Pod Pod Services NA4 NA7 Pod Monitoring Pod Services NA5 Pod Security Services EMEA “N” Pod Pod
  13. 13. A Scalable, High Performing Architecture Database cluster  8 Node RAC cluster  Scalability and fault tolerance Application Servers  Web requests, API calls, Batch processes, Search requests  30 plus servers, load balanced for performance  Scale vertically and horizontally
  14. 14. Scalable, High Performing Architecture Network  10 carriers for geographic performance and redundancy.  Data centers located at core Internet hubs.  Access to thousands of global Internet peering points.  Redundant routers, switches, firewalls and load balancers
  15. 15. Agenda What is performance Testing The Scalable Salesforce Architecture Internal Salesforce Performance Testing Why do you need to test? Performance Testing Approach and Testing Tools
  16. 16. Performance Engineering Primarily focuses on benchmarking, simulation, tuning, analysis, and monitoring/profiling of the applications. Uses sophisticated simulation methodologies, automation, tools, and environments to discover the problems.
  17. 17. Performance Engineering Performance testing typically falls into two major categories.  Playback-based: • Plays back the Production traffic logs within a secure environment. • Private information appropriately obfuscated. • Capture real-world data skews, volumes, and transactions.  Synthetic: • Custom tools to generate and profile workloads and data shapes • Simulate current loads and beyond that are currently in use in the production systems.
  18. 18. What’s Tested? Full Database Programmatic User Interface SearchEnvironment Performance Features
  19. 19. Capacity Planning Multi year growthProactive Approach Capacity alerts plan Customer Scale Reactive transaction vertically or monitoring monitoring horizontally Excess capacity always exists!
  20. 20. Agenda What is performance Testing The Scalable Salesforce Architecture Internal Salesforce Performance Testing Why do you need to test? Performance Testing Approach & Testing Tools
  21. 21. Candidates for Performance TestingHeavy Custom Development • Testing should be geared towards exercising the custom code of an applicationHigh Activity • Large Transaction based testing • Large User based testing • Browser Rendering • Real-time monitoring
  22. 22. What Is Performance Testing Response Time Based Testing Confirm that a set of VF Pages and Apex controllers perform in such a manner as to give the end user a positive experience Load Based Testing Confirm that a set of Apex code, VF Pages will be able to support a particular load.22
  23. 23. What is Performance Testing? Performance Testing categories  Load, Endurance, Spike, Configuration, Isolation. Performance Testing Architecture
  24. 24. Metrics Collected Page Response Server Response Transaction Times Times Throughput Rate Measurement Measurement Measurement of number of of End User of SFDC POD completed perspective of Response transactions in tested pages Times a time period
  25. 25. Agenda What is performance Testing The Scalable Salesforce Architecture Internal Salesforce Performance Testing Why do you need to test? Performance Testing Approach & Tools
  26. 26. Recommended Performance Testing Approach Build Test Plan •Identify Key Business Run Baseline Test Transactions •Test Tool •Environment for •Low User Test Identify X testing •Used to confirm script •Projected X Load for •Get baseline numbers Run .5X, .75X, and X each script •Use baseline numbers Tests •Data analysis from existing sources •Make sure number is •Run your full scale realistic tests •Gradually increasing loadX = Projected realistic goal based on script andwait time
  27. 27. Performance Testing Tools Salesforce supports most all major performance testing tools Tools capture all data and metrics for reporting More popular tools: Commercial Open Source •LoadRunner •Jmeter •Silk Performer •Fiddler •RadView •OpenSTA
  28. 28. Engaging Salesforce for Performance Testing Create tests that measure transaction and business process time. Create a test plan Open a Salesforce support case with the following:  Brief explanation of the test  When test will be conducted  Sandbox instance and organization ID where the test will be conducted.  Contact information during the test  Attach your test plan If plan or timeframe changes, let us know.
  29. 29. Wrap Up • Most clients will not need to conduct their own testing • Average of 60 billion transactions a day with an average response time of 250ms • Multi-Tenet architecture supports scalability and high performance • Capacity planning and Performance engineer teams conduct performance test to ensure application and platform performance • Heavy use of custom code or high user load clients may conduct performance tests • Main collected metrics include, response times, server times, and transaction throughput rate • Identify realistic load, create test plan and run tests • Engage Salesforce by creating a support case
  30. 30. Raj Advani Randy Case Sr. Technical Architect, Director, Softwareradvani@salesforce.com Engineering rcase@salesforce.com

×