A presentation from the London Openstack Meetup on February 2nd by Darryl Weaver. Public Cloud Data was provided by Striking Cloud and donated by public and private cloud deployments and then anonymised. The demo was available online here: https://clouddatademo.stackevolution.com/ (password required).

1. Measuring Performance:
the basics
by Stack Evolution

2. 2 © Stack Evolution Ltd
Why?
I just finished building my IaaS cloud.
But…
Is it any good?

3. 3 © Stack Evolution Ltd
Lots of metrics and measures
System monitoring
System metric graphing
Log file analysis
These are focused on the infrastructure, not the
tenant workload

4. 4 © Stack Evolution Ltd
IaaS is a Shared Platform
Tunable memory/processor/disk overcommit
Dynamic workloads
Batch jobs
Time of day/month/year
Many variables

5. 5 © Stack Evolution Ltd
Measuring the tenant workloads
Tenant only cares about workload
performance
How does the disk perform?
Is the processor faster or slower?
What is performance of a specific Database?
How do HPC jobs perform?

6. 6 © Stack Evolution Ltd
Benchmarks
Many standard benchmark tools available:
Iperf
Unixbench
Fio
Sysbench_OLTP
Aerospike
Redis
Scimark2
HPCC

7. 7 © Stack Evolution Ltd
Standardise Suite of Benchmarks
Select flavours to directly compare memory
Volumes of standard size (20GB)
Network: Internal and External
Specific applications
Database
Logging
Streaming
HPC

8. 8 © Stack Evolution Ltd
Run on multiple platforms
Collect data from
Public Clouds
Private cloud deployments
Staging/Test Clouds

9. 9 © Stack Evolution Ltd
Compare
Now compare the data to evaluate how good
your cloud really is.

10. Demo

11. 11 © Stack Evolution Ltd
Variability of performance for similar flavours across cloud providers, notice the
latency and boot time graphs higher is worse, others higher is better

12. 12 © Stack Evolution Ltd
Overall Index score for combined benchmarks, notice the variability in providers
is as much as 7 times as much performance for the same flavour size

13. 13 © Stack Evolution Ltd
Mimicking real world workloads: Database and Logging and measuring disk
performance using a 20GB Volume of a standard type

14. 14 © Stack Evolution Ltd
A High Performance Computing Benchmark, then weighted by flavour hourly
cost and finally weighted by Volume monthly cost (for 20GB Volume).
Notice how much the graphs change once you add in pricing information to the
performance data.

15. 15 © Stack Evolution Ltd
Data provided by:
Thanks
Contact darryl@stackevolution.com if you want to find out more.