1. COSBench: A Benchmark Tool for
Cloud Object Storage Services
Jiangang Duan (段建钢)
2012/8
1

2. Agenda
• Self introduction
• Cloud Storage in tomorrow’s Data Center
• COSBench Introduction
• Case Study to evaluate OpenStack* swift
performance with COSBench
• Summary
2

3. Self introduction
• Jiangang Duan
• Working in Cloud Infrastructure Technology
Team (CITT) of Intel APAC R&D Ltd.
• We are software team, good at performance
• Try to understand how to build an efficient/scale
Cloud Solution with Open Source software
(OpenStack*, Xen*, KVM*)
• All of work will be contributed to Community
• Today we will talk about some efforts we try to
measure OpenStack* performance and know people
who want to contribute to OpenStack and work
together
3

4. Data Centers are Evolving
Compute • Data centers are built
Flexible
upon three fundamental
Workloads pillars:
– Compute
– Storage
Virtualized – Networking
Storage Infrastructure • All three are critical for
Open efficient data center
Platforms
Network operations
Common – Balanced in
Fabrics
performance and
utilization
A Balanced Data Center is Essential for Efficiency
4

5. IDC Storage Capacity Growth†
EB
90
Structured data (23.6% CAGR)
Traditional enterprise database 80
Replicated data (24.2% CAGR) 70
Backups
60
Data warehouses
50
Unstructured data (54.8% CAGR)
Archives 40
Content Depots (75.6% CAGR) 30
Web
20
Email
Document sharing 10
Social network content (pictures/videos)
2009 2010 2011 2012 2013 2014
2012 Deployment ~7.6 million drives
Estimate: ~500,000 storage systems‡
†Source: IDC, Worldwide Enterprise Storage Systems 2010–2014 Forecast: Recovery, Efficiency, and Digitization Shaping Customer
Requirements for Storage Systems, Doc
‡Source: Internal estimates based on the IDC Worldwide Enterprise Storage Systems Forecast # 223234., May 2011
5

6. Usage Models Dictate the Solutions
Business DB Performance
Storage Performance
Random small
(OLTP, OLAP) Storage
Content distribution
network (CDN)
Requirement
(Objects per second)
Application data store
(e.g. e-mail, VM/Boot, Sharepoint*)
Large Relational DB
(e.g. NoSQL, non ACID)
Large analytics
(e.g Hadoop*/HDFS)
Sequential Large
High performance compute
(e.g. pNFS, Luster*) Capacity
Storage
Cloud Object storage
COSBench (e.g. photos/videos)
Backup and archive
(server and client)
Gigabytes Terabytes Petabytes Exabytes
Storage Capacity Requirement
Key Storage Usage Models Have Differing Requirements
Thus Need New Benchmarks
6

7. COSBench Introduction
• COSBench is an Intel developed benchmark
to measure Cloud Object Storage Service
performance
• Cloud end user can use COSBench to
compare different public Cloud Object
Storage service performance
• Cloud provider can use it to
– Compare different Hardware/Software Stacks
– Identify bottleneck and make optimization
COSBench is the IOMeter for Cloud Object Storage
service
7

8. COSBench Key Component
Config.xml:
– define workload with flexibility. Web
Console
Controller
Controller:
– Control all drivers Config.xml
– Collect and aggregate stats.
COSBench
Driver:
– generate load w/ config.xml parameters. Driver
Driver
– can run tests w/o controller.
Web Console: Controller
Node
– Manage controller
– Browse real-time stats Storage
– Communication is based on HTTP (RESTful
style)
Cloud
Storage
Node
8

9. Web Console
Driver list
Workload List
History list
Intuitive UI to get Overview.
9

10. Workload Configuration
Flexible load control
object size distribution
Read/Write Operations
Workflow for complex stages
Flexible configuration parameters is capable of complex
Cases
10

11. Performance Metrics
Throughput (Operations/s): the operations completed in one
second
Response Time (in ms): the duration between operation
initiation and completion.
Bandwidth (KB/s): the total data in KiB transferred in one
second
Success Ratio (%): the ratio of successful operations
11

12. OpenStack* Swift overview
OpenStack* is open source software to build private and public
clouds.
OpenStack Object Store (Swift): Create petabytes of reliable
storage using standard servers
†Source: docs.openstack.org
12

13. OpenStack* Swift Overview
Entities  RING  physical location (zone/device/partition/…)
Proxy Node
Proxy Server
Account Container Object
Ring Ring Ring
Account Server Container Server Object Server
Storage Node metadata
Object
Account Container
file
DB DB
13

14. Test Configuration
14

15. Test OpenStack* Swift performance
Software and workloads used in performance tests may have been optimized for performance only on Intel® microprocessors. Performance tests, such as
SYSmark* and MobileMark*, are measured using specific computer systems, components, software, operations and functions. Any change to any of those
15 factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated
purchases, including the performance of that product when combined with other products. For more information go to http://www.intel.com/performance.

16. Swift characterization
- Insufficient processing power throttles overall performance.
Baseline (ObjectSize=64KB, Concurrency=512, 12 Disks per node)
Op/s RT (ms) BW (KB/s)
Performance 5,644 91 361,220
Proxy Node 6 Node 7 Node 8 Node 9 Node 10Client
r/s 90.31 91.16 90.64 91.01 91.17
w/s 0.01 0.01 0.01 0.00 0.01
rKB/s 5,633 5,378 5,384 5,379 5,381
wKB/s 0.09 0.09 0.09 0.08 0.08
data disk await 5.16 5.05 5.13 5.16 5.14
rxkB/s 356,225 1,813 1,826 1,844 1,757 1,710
Internal txkB/s 8,356 71,910 73,393 74,265 74,559 72,641
rxkB/s 3,524
External txkB/s 357,506
user% 79.70 14.35 1.42
cpu% system% 19.40 4.26 1.74
iowait% 0.00 21.85 0.13
Software and workloads used in performance tests may have been optimized for performance only on Intel® microprocessors. Performance tests, such as
SYSmark* and MobileMark*, are measured using specific computer systems, components, software, operations and functions. Any change to any of those
16 factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated
purchases, including the performance of that product when combined with other products. For more information go to http://www.intel.com/performance.

17. Summary
• New storage Usage model rises for Cloud
Computing age, which need new benchmark
• COSBench is a new benchmark developed by Intel
to measure Cloud Object Storage service
performance
• COSBench is useful to analyze Cloud Object
Service system performance, identify bottleneck
and conduct optimization
17

18. Next Step and call for action
• We are WIP developing COSBench to support more Cloud
Object Storage service
• Our final goal is to open source COSBench to make it
available for industry and community use to make better
Object Storage service design
• We will continue to use COSBench to analyze the optimize
OpenStack* Swift performance and share back our finding to
community
• Any question, feedback, please contact me at:
• Jiangang.duan@intel.com
18

19. backup
19

20. Storage Layout
How data is stored in each node ?
/dev/swift/a /dev/swift/b /dev/swift/c
• accounts • accounts • accounts
• containers • containers • containers
• objects • objects • objects
• async_pending • async_pending • async_pending
object
1025
1027
partition
DG3 12C
1026 G1J 45A1…12C SFT3…12C
hash suffix hash
20

21. GET/HEAD@Proxy Node
How proxy node cooperate with storage nodes to obtain
object data ? Consult the
Retrieve Ring for
container candidate Return the
information nodes result
Perform GET/HEAD
A&A using candidate
pre-hooked 200 server(s)
facilities 0 (R&H)
no response
5xx 314
server error ? 507 404 unmodified
disk error 412
file not found
precondition failed or not synchronized
21

22. PUT@Proxy Node – Part I
How proxy cooperate with storage nodes to create an
object ? Consult the
Retrieve Ring for Check
container candidate various
information nodes constraints
Perform Create the
A&A using timestamp
pre-hooked header
facilities
413 411 404 400
Invalid
Path
Object Length Container
Too Large Required not Found Invalid
Object
Name
22

23. PUT@Proxy Node – Part II
How proxy cooperate with storage nodes to create an
object ?
Try making Forward 3 Phased Workflow
R conns to data to
storage storage Return the
servers servers result
Assign Collect
each conn resps from
a container storage
server
0 201 servers Time out:
86400 secs
no response
5xx 408
time out
server error
507 422
disk error data corrupted
23

24. Disclaimers
INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. NO LICENSE, EXPRESS OR
IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT
AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY
WHATSOEVER AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL
PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY,
OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT.
A "Mission Critical Application" is any application in which failure of the Intel Product could result, directly or indirectly, in
personal injury or death. SHOULD YOU PURCHASE OR USE INTEL'S PRODUCTS FOR ANY SUCH MISSION CRITICAL
APPLICATION, YOU SHALL INDEMNIFY AND HOLD INTEL AND ITS SUBSIDIARIES, SUBCONTRACTORS AND AFFILIATES, AND
THE DIRECTORS, OFFICERS, AND EMPLOYEES OF EACH, HARMLESS AGAINST ALL CLAIMS COSTS, DAMAGES, AND EXPENSES
AND REASONABLE ATTORNEYS' FEES ARISING OUT OF, DIRECTLY OR INDIRECTLY, ANY CLAIM OF PRODUCT LIABILITY,
PERSONAL INJURY, OR DEATH ARISING IN ANY WAY OUT OF SUCH MISSION CRITICAL APPLICATION, WHETHER OR NOT
INTEL OR ITS SUBCONTRACTOR WAS NEGLIGENT IN THE DESIGN, MANUFACTURE, OR WARNING OF THE INTEL PRODUCT OR
ANY OF ITS PARTS.
Intel may make changes to specifications and product descriptions at any time, without notice. Designers must not rely on the
absence or characteristics of any features or instructions marked "reserved" or "undefined". Intel reserves these for future
definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them. The
information here is subject to change without notice. Do not finalize a design with this information.
The products described in this document may contain design defects or errors known as errata which may cause the product to
deviate from published specifications. Current characterized errata are available on request.
Contact your local Intel sales office or your distributor to obtain the latest specifications and before placing your product order.
Copies of documents which have an order number and are referenced in this document, or other Intel literature, may be
obtained by calling 1-800-548-4725, or go to: http://www.intel.com/design/literature.htm%20
This document contains information on products in the design phase of development.
Intel and the Intel logo are trademarks of Intel Corporation in the U.S. and/or other countries.
*Other names and brands may be claimed as the property of others.
Copyright © 2012 Intel Corporation. All rights reserved.
24