1. COSBench: A Benchmark Tool for
Cloud Object Storage Services
Jiangang Duan (段建钢)
2012/8
1
2. Agenda
• Self introduction
• Cloud Storage in tomorrow’s Data Center
• COSBench Introduction
• Case Study to evaluate OpenStack* swift
performance with COSBench
• Summary
2
3. Self introduction
• Jiangang Duan
• Working in Cloud Infrastructure Technology
Team (CITT) of Intel APAC R&D Ltd.
• We are software team, good at performance
• Try to understand how to build an efficient/scale
Cloud Solution with Open Source software
(OpenStack*, Xen*, KVM*)
• All of work will be contributed to Community
• Today we will talk about some efforts we try to
measure OpenStack* performance and know people
who want to contribute to OpenStack and work
together
3
4. Data Centers are Evolving
Compute • Data centers are built
Flexible
upon three fundamental
Workloads pillars:
– Compute
– Storage
Virtualized – Networking
Storage Infrastructure • All three are critical for
Open efficient data center
Platforms
Network operations
Common – Balanced in
Fabrics
performance and
utilization
A Balanced Data Center is Essential for Efficiency
4
5. IDC Storage Capacity Growth†
EB
90
Structured data (23.6% CAGR)
Traditional enterprise database 80
Replicated data (24.2% CAGR) 70
Backups
60
Data warehouses
50
Unstructured data (54.8% CAGR)
Archives 40
Content Depots (75.6% CAGR) 30
Web
20
Email
Document sharing 10
Social network content (pictures/videos)
2009 2010 2011 2012 2013 2014
2012 Deployment ~7.6 million drives
Estimate: ~500,000 storage systems‡
†Source: IDC, Worldwide Enterprise Storage Systems 2010–2014 Forecast: Recovery, Efficiency, and Digitization Shaping Customer
Requirements for Storage Systems, Doc
‡Source: Internal estimates based on the IDC Worldwide Enterprise Storage Systems Forecast # 223234., May 2011
5
6. Usage Models Dictate the Solutions
Business DB Performance
Storage Performance
Random small
(OLTP, OLAP) Storage
Content distribution
network (CDN)
Requirement
(Objects per second)
Application data store
(e.g. e-mail, VM/Boot, Sharepoint*)
Large Relational DB
(e.g. NoSQL, non ACID)
Large analytics
(e.g Hadoop*/HDFS)
Sequential Large
High performance compute
(e.g. pNFS, Luster*) Capacity
Storage
Cloud Object storage
COSBench (e.g. photos/videos)
Backup and archive
(server and client)
Gigabytes Terabytes Petabytes Exabytes
Storage Capacity Requirement
Key Storage Usage Models Have Differing Requirements
Thus Need New Benchmarks
6
7. COSBench Introduction
• COSBench is an Intel developed benchmark
to measure Cloud Object Storage Service
performance
• Cloud end user can use COSBench to
compare different public Cloud Object
Storage service performance
• Cloud provider can use it to
– Compare different Hardware/Software Stacks
– Identify bottleneck and make optimization
COSBench is the IOMeter for Cloud Object Storage
service
7
8. COSBench Key Component
Config.xml:
– define workload with flexibility. Web
Console
Controller
Controller:
– Control all drivers Config.xml
– Collect and aggregate stats.
COSBench
Driver:
– generate load w/ config.xml parameters. Driver
Driver
– can run tests w/o controller.
Web Console: Controller
Node
– Manage controller
– Browse real-time stats Storage
– Communication is based on HTTP (RESTful
style)
Cloud
Storage
Node
8
9. Web Console
Driver list
Workload List
History list
Intuitive UI to get Overview.
9
10. Workload Configuration
Flexible load control
object size distribution
Read/Write Operations
Workflow for complex stages
Flexible configuration parameters is capable of complex
Cases
10
11. Performance Metrics
Throughput (Operations/s): the operations completed in one
second
Response Time (in ms): the duration between operation
initiation and completion.
Bandwidth (KB/s): the total data in KiB transferred in one
second
Success Ratio (%): the ratio of successful operations
11
12. OpenStack* Swift overview
OpenStack* is open source software to build private and public
clouds.
OpenStack Object Store (Swift): Create petabytes of reliable
storage using standard servers
†Source: docs.openstack.org
12
13. OpenStack* Swift Overview
Entities RING physical location (zone/device/partition/…)
Proxy Node
Proxy Server
Account Container Object
Ring Ring Ring
Account Server Container Server Object Server
Storage Node metadata
Object
Account Container
file
DB DB
13
15. Test OpenStack* Swift performance
Software and workloads used in performance tests may have been optimized for performance only on Intel® microprocessors. Performance tests, such as
SYSmark* and MobileMark*, are measured using specific computer systems, components, software, operations and functions. Any change to any of those
15 factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated
purchases, including the performance of that product when combined with other products. For more information go to http://www.intel.com/performance.
16. Swift characterization
- Insufficient processing power throttles overall performance.
Baseline (ObjectSize=64KB, Concurrency=512, 12 Disks per node)
Op/s RT (ms) BW (KB/s)
Performance 5,644 91 361,220
Proxy Node 6 Node 7 Node 8 Node 9 Node 10Client
r/s 90.31 91.16 90.64 91.01 91.17
w/s 0.01 0.01 0.01 0.00 0.01
rKB/s 5,633 5,378 5,384 5,379 5,381
wKB/s 0.09 0.09 0.09 0.08 0.08
data disk await 5.16 5.05 5.13 5.16 5.14
rxkB/s 356,225 1,813 1,826 1,844 1,757 1,710
Internal txkB/s 8,356 71,910 73,393 74,265 74,559 72,641
rxkB/s 3,524
External txkB/s 357,506
user% 79.70 14.35 1.42
cpu% system% 19.40 4.26 1.74
iowait% 0.00 21.85 0.13
Software and workloads used in performance tests may have been optimized for performance only on Intel® microprocessors. Performance tests, such as
SYSmark* and MobileMark*, are measured using specific computer systems, components, software, operations and functions. Any change to any of those
16 factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated
purchases, including the performance of that product when combined with other products. For more information go to http://www.intel.com/performance.
17. Summary
• New storage Usage model rises for Cloud
Computing age, which need new benchmark
• COSBench is a new benchmark developed by Intel
to measure Cloud Object Storage service
performance
• COSBench is useful to analyze Cloud Object
Service system performance, identify bottleneck
and conduct optimization
17
18. Next Step and call for action
• We are WIP developing COSBench to support more Cloud
Object Storage service
• Our final goal is to open source COSBench to make it
available for industry and community use to make better
Object Storage service design
• We will continue to use COSBench to analyze the optimize
OpenStack* Swift performance and share back our finding to
community
• Any question, feedback, please contact me at:
• Jiangang.duan@intel.com
18
21. GET/HEAD@Proxy Node
How proxy node cooperate with storage nodes to obtain
object data ? Consult the
Retrieve Ring for
container candidate Return the
information nodes result
Perform GET/HEAD
A&A using candidate
pre-hooked 200 server(s)
facilities 0 (R&H)
no response
5xx 314
server error ? 507 404 unmodified
disk error 412
file not found
precondition failed or not synchronized
21
22. PUT@Proxy Node – Part I
How proxy cooperate with storage nodes to create an
object ? Consult the
Retrieve Ring for Check
container candidate various
information nodes constraints
Perform Create the
A&A using timestamp
pre-hooked header
facilities
413 411 404 400
Invalid
Path
Object Length Container
Too Large Required not Found Invalid
Object
Name
22
23. PUT@Proxy Node – Part II
How proxy cooperate with storage nodes to create an
object ?
Try making Forward 3 Phased Workflow
R conns to data to
storage storage Return the
servers servers result
Assign Collect
each conn resps from
a container storage
server
0 201 servers Time out:
86400 secs
no response
5xx 408
time out
server error
507 422
disk error data corrupted
23