Performance problems are one of the most cited concerns about to the cloud. But is it really the cloud or the application? What does performance mean anyway when you can scale to thousands of servers? This session will discuss why traditional means of performance management and troubleshooting no longer work and how this affects everything. Most importantly we will look at how to identify the root cause of performance problems in such dynamic environments. Finally we will explain how to assess and manage performance when capacity is no longer the issue.

1. What does performance mean in the
cloud?

2. What are the risks of moving to the cloud?
IDC(Survey Q4 „09) Results from actual pilots (March 2010)
Perception Primary Benefits Biggest Issues
Before Reduced IT costs Security
“The Maturing Cloud: What It
Will Take to Win” (Published Mar After Scalability Performance
2010)
What are the major risks in the Agility SLA Management
Cloud?
• Security – 87.5%
• Availability – 83.3%
• Performance – 82.9%
(88.6% stated that cloud
“All About The Cloud” Conference (May 2010)
service providers need to
“Security in the Cloud isn‟t any harder than it is in the
provide SLAs)
Enterprise – it‟s just different” (Unisys)
“[Application] Performance Management in the Cloud is
becoming the hot topic” (THINKstrategies)
 Projects fail to deliver acceptable performance
 Moving Legacy Applications is harder than thought

3. What is Performance?

4. Performance ≠ Scalability
The Cloud scales, but does it perform?

5. How do we measure Performance
 Response Time
 Transaction Level Metric
 Don’t use averages  High Volatility
 Be specific  Which type of transaction
 Throughput
 Volume of Transactions per Timeframe
 Average Speed of Transaction
 Be specific  Which type of transactions

6. What does Scalability mean
 More concurrent Transactions with same response time
 Linear growing Throughput with linear more hardware
Scalability depends on Performance

7. Performance in the cloud
 “Pure Performance” is never better in a Cloud!
 Co Tenancy
 Resource sharing
 Commodity and generally smaller hardware
 Scalability can be better in the Cloud
 Rapid elasticity
 Depends on Application Design and Performance
 Legacy Applications have limitations
 End User Performance depends on both and more
 Web Delivery Chain
 Network!
 Can be better than on premise!

8. Performance Management in the
Cloud

9. Traditional Performance Management - Fails
 Sniffing and other appliances do not work
 Are based on System metrics which are
 Corrupted
 Do not answer application performance questions
 Are not manageable
 To many unrelated metrics
 Does not deal well Exponential Complexity Increase

10. Why is Cloud Monitoring not enough?
 Only System and High Level Response Metrics
 No Visibility into Application
(Regressions, MTTR, Application Dependencies)
 No Visibility into End User Impact  Business Impact
We need Application Focus

11. What we really care about
Availability and Baseline
Performance
Web 2.0
Load Balancer WebServer Frontend(s) Backend(s) Private Datacenter
Detailed Contribution
End User Performance
Times

12. Key Challenge - Volatility
Real vs. Measured
Performance ^= F(Capacity)
60
Utilization
40
20
0

13. Measure Performance where it matters
But
slow is
bad
Faster
is not
better

14. Understand your Transactions

15. End To End: Don„t forget the Chain
User
Click
On the Web
Server
In the
In the
Application
Cloud

16. Details, Details, Details, but be aware…
High
Volatility
 Steal Time
 Shared I/O Virtualization Aware Timers
 Shared network

17. Monitoring the Complex

18. Cloud Designs are simple, yet…
 Everything Fails!
 Tight Couple End User Delivery Components
 Few Tiers
 Response Time
 Scale Upfront
100.000s
users

19. Cloud Designs are simple, yet…
 Everything Fails!
 Tight Couple End User Delivery Components
 Few Tiers
 Response Time
 Scale Upfront
 Loosely Couple everything else
 Throughput
 Scale everything independent
Simple Designs still lead to Complex Systems
Complex Systems are hard to manage

20. Monitoring Complex Systems – Look at what matters

21. Context matters
 Too much Aggregation will blur the picture
Buying
Books
Buying Context
DVDs matters!
Buying
Cloth

22. Measure what Matters
 The Application and its Business Transactions
 Measure End User Performance
 Measure Throughput on Transaction Type Level
 How Performance effects your business
 e.g. Conversion Rate
 SLA Window
 Cost vs. Gain
 Prioritize based on what matters most

23. Identify cause of End User Impact
Flow of single
Transaction
Response Time
Hotspots

24. Cloud vs. Application
Application shows
otherwise
Cloud Monitoring
would show CPU

25. Application or Cloud Instance?
Application Hotspots:
CPU, Wait, I/O, Sync, Susp
ension?
Cause for Volatility?

26. putting Cloud Monitoring in Context
Steal Time or
out of CPU?
Cause for
Latency

27. We want to scale the Application and not the Cloud
 Auto Scaling on System metrics
 Is indirect and not goal oriented
 Fails when application changes
 Scale on Application Metrics and Application Components
 Transaction Load
 Response Time Contribution and Trend
 Throughput Goals

28. Rapid Deployment and Availability

29. Understand your Flow
 Understand the Application Flow
 Always Capture Performance Data
 Everything is transitory
 Reproducing problems is hard
 Analyze offline
 Identify Contributors

30. Automatically detect Regressions
 Deploy
 Compare
 Fix small
 Start again

31. Reacting Automatically to Issues
 Disk Latency Degradation
 Too much steal time
 Hardware Issues
 Detect “Application” Degradation
Terminate!
And start new

32. Make sure you are not blind
 Application Monitoring must be high available
 Outside and Inside
 Failover
 not in the same zone.
 Automated Deployments
 Zero Configuration Monitoring

33. Assessing Performance/Value

34. What is the goal?
 Performance and Scalability are not self serving
 “Desired” End User Experience
 Faster than that is not better
 Using less resources is cheaper!

35. A Price Performance Index
 Dollar Value for acceptable Performance:
90th response time/(Total Cost/Number of Transactions)
Desired Throughput/Total Cost
 Mind Volatility
 Price Performance Index is comparable
 Cost Scalability
 Cost per Transaction must remain stable
Performance is not based on Capacity
It is a function of desired User
Experience and associated Cost

36. Questions
Michael Kopp
Michael.kopp@dynaTrace.com
http://blog.dynatrace.com
@mikopp