This document discusses performance monitoring and testing in the Salesforce cloud. It describes how Salesforce uses a scalable, multi-tenant architecture to support over 60 billion transactions per quarter with an average response time of under 275ms. It also outlines how Salesforce's performance engineering team conducts internal testing to benchmark and monitor the platform, using both playback of production logs and synthetic workloads. The document provides guidance for customers on when and how to conduct their own performance tests, including identifying key transactions to test, using tools to capture metrics, and engaging Salesforce support for assistance.

1. Performance Monitoring and Testing
in the Salesforce Cloud
Your Success. Our Cloud
Raj Advani, salesforce.com, Sr. Technical Solution Architect
Randy Case, salesforce.com, Director, Customer Centric Engineering

2. Raj Advani
Sr. Technical Solution Architect
radvani@salesforce.com

3. Randy Case
Director, Customer Centric Engineering
rcase@salesforce.com

4. Safe Harbor
Safe harbor statement under the Private Securities Litigation Reform Act of 1995:
This presentation may contain forward-looking statements that involve risks, uncertainties, and assumptions. If any such uncertainties
materialize or if any of the assumptions proves incorrect, the results of salesforce.com, inc. could differ materially from the results
expressed or implied by the forward-looking statements we make. All statements other than statements of historical fact could be
deemed forward-looking, including any projections of product or service availability, subscriber growth, earnings, revenues, or other
financial items and any statements regarding strategies or plans of management for future operations, statements of belief, any
statements concerning new, planned, or upgraded services or technology developments and customer contracts or use of our services.
The risks and uncertainties referred to above include – but are not limited to – risks associated with developing and delivering new
functionality for our service, new products and services, our new business model, our past operating losses, possible fluctuations in our
operating results and rate of growth, interruptions or delays in our Web hosting, breach of our security measures, the outcome of
intellectual property and other litigation, risks associated with possible mergers and acquisitions, the immature market in which we
operate, our relatively limited operating history, our ability to expand, retain, and motivate our employees and manage our growth, new
releases of our service and successful customer deployment, our limited history reselling non-salesforce.com products, and utilization
and selling to larger enterprise customers. Further information on potential factors that could affect the financial results of
salesforce.com, inc. is included in our annual report on Form 10-Q for the most recent fiscal quarter ended July 31, 2012. This
documents and others containing important disclosures are available on the SEC Filings section of the Investor Information section of
our Web site.
Any unreleased services or features referenced in this or other presentations, press releases or public statements are not currently
available and may not be delivered on time or at all. Customers who purchase our services should make the purchase decisions based
upon features that are currently available. Salesforce.com, inc. assumes no obligation and does not intend to update these forward-
looking statements.

5. Agenda
What is performance testing
The scalable Salesforce architecture
Internal Salesforce performance testing
Why do you need to test?
Performance testing approach and testing tools
Questions and Answers

6. What is Performance Testing?
‘Performance testing is typically done to help identify bottlenecks in a
system, establish a baseline for future testing, support a performance
tuning effort, determine compliance with performance goals and
requirements, and/or collect other performance-related data to help
stakeholders make informed decisions related to the overall quality of the
application being tested. In addition, the results from performance testing
and analysis can help you to estimate the hardware configuration
required to support the application(s) when you “go live” to production
operation. ‘
WOW!
6

7. What is Performance Testing?
‘Performance testing is typically done to help identify bottlenecks in a
system, establish a baseline for future testing, support a performance
tuning effort, determine compliance with performance goals and
requirements, and/or collect other performance-related data to help
stakeholders make informed decisions related to the overall quality of the
application being tested.
WOW!
7

8. Agenda
What is performance Testing
The Scalable Salesforce Architecture
Internal Salesforce Performance Testing
Why do you need to test?
Performance Testing Approach and Testing Tools

9. Transactional Volumes

10. It’s in the numbers!
104,000+ businesses
>60 billion requests/quarter
< 275 ms avg. response time 100s thousands users
100s gigabytes of data
>800 million >40 million account records
operations/day million contact records
>20
>30 million opportunity records
100s millions custom object records

11. Multi-tenant, Cloud Computing Platform

12. Pod Architecture Enhances Availability, Scale, and Platform
for Future Growth
NA0 APAC
Pod Pod
Network
NA1 Sandbox
Pod Services Pod
NA2 Storage EMEA2
Pod Services Pod
NA3 Backup NA6
Pod Pod
Services
NA4 NA7
Pod Monitoring Pod
Services
NA5
Pod
Security
Services
EMEA “N” Pod
Pod

13. A Scalable, High Performing Architecture
Database cluster
 8 Node RAC cluster
 Scalability and fault tolerance
Application Servers
 Web requests, API calls, Batch processes, Search requests
 30 plus servers, load balanced for performance
 Scale vertically and horizontally

14. Scalable, High Performing Architecture
Network
 10 carriers for geographic
performance and redundancy.
 Data centers located at core
Internet hubs.
 Access to thousands of global
Internet peering points.
 Redundant routers, switches,
firewalls and load balancers

15. Agenda
What is performance Testing
The Scalable Salesforce Architecture
Internal Salesforce Performance Testing
Why do you need to test?
Performance Testing Approach and Testing Tools

16. Performance Engineering
Primarily focuses on benchmarking,
simulation, tuning, analysis, and
monitoring/profiling of the applications.
Uses sophisticated simulation
methodologies, automation, tools, and
environments to discover the problems.

17. Performance Engineering
Performance testing typically falls into two major categories.
 Playback-based:
• Plays back the Production traffic logs within a secure environment.
• Private information appropriately obfuscated.
• Capture real-world data skews, volumes, and transactions.
 Synthetic:
• Custom tools to generate and profile workloads and data shapes
• Simulate current loads and beyond that are currently in use in the production
systems.

18. What’s Tested?
Full Database Programmatic
User Interface Search
Environment Performance Features

19. Capacity Planning
Multi year growth
Proactive Approach Capacity alerts
plan
Customer Scale
Reactive
transaction vertically or
monitoring
monitoring horizontally
Excess capacity always exists!

20. Agenda
What is performance Testing
The Scalable Salesforce Architecture
Internal Salesforce Performance Testing
Why do you need to test?
Performance Testing Approach & Testing Tools

21. Candidates for Performance Testing
Heavy Custom Development
• Testing should be geared towards exercising the custom code of an
application
High Activity
• Large Transaction based testing
• Large User based testing
• Browser Rendering
• Real-time monitoring

22. What Is Performance Testing
Response Time Based Testing
Confirm that a set of VF Pages and Apex
controllers perform in such a manner as to
give the end user a positive experience
Load Based Testing
Confirm that a set of Apex code, VF Pages
will be able to support a particular load.
22

23. What is Performance Testing?
Performance Testing categories
 Load, Endurance, Spike, Configuration, Isolation.
Performance Testing Architecture

24. Metrics Collected
Page Response Server Response Transaction
Times Times Throughput Rate
Measurement
Measurement Measurement
of number of
of End User of SFDC POD
completed
perspective of Response
transactions in
tested pages Times
a time period

25. Agenda
What is performance Testing
The Scalable Salesforce Architecture
Internal Salesforce Performance Testing
Why do you need to test?
Performance Testing Approach & Tools

26. Recommended Performance Testing Approach
Build Test Plan
•Identify Key Business Run Baseline Test
Transactions
•Test Tool
•Environment for •Low User Test Identify X
testing •Used to confirm script
•Projected X Load for •Get baseline numbers Run .5X, .75X, and X
each script •Use baseline numbers Tests
•Data analysis from
existing sources
•Make sure number is •Run your full scale
realistic tests
•Gradually increasing
load
X = Projected realistic goal based on script and
wait time

27. Performance Testing Tools
Salesforce supports most all major performance testing tools
Tools capture all data and metrics for reporting
More popular tools:
Commercial Open Source
•LoadRunner •Jmeter
•Silk Performer •Fiddler
•RadView •OpenSTA

28. Engaging Salesforce for Performance Testing
Create tests that measure transaction and business process time.
Create a test plan
Open a Salesforce support case with the following:
 Brief explanation of the test
 When test will be conducted
 Sandbox instance and organization ID where the test will be conducted.
 Contact information during the test
 Attach your test plan
If plan or timeframe changes, let us know.

29. Wrap Up
• Most clients will not need to conduct their own testing
• Average of 60 billion transactions a day with an average response time of 250ms
• Multi-Tenet architecture supports scalability and high performance
• Capacity planning and Performance engineer teams conduct performance test to
ensure application and platform performance
• Heavy use of custom code or high user load clients may conduct
performance tests
• Main collected metrics include, response times, server times, and transaction
throughput rate
• Identify realistic load, create test plan and run tests
• Engage Salesforce by creating a support case

30. Raj Advani Randy Case
Sr. Technical Architect, Director, Software
radvani@salesforce.com Engineering
rcase@salesforce.com