S104876 ibm-cos-jburg-v1809b

IBM Systems Technical University © 2018 IBM Corporation
2018 IBM Systems
Technical University
Save the World! Save your IT
budget with IBM Cloud Object
Storage
—
Tony Pearson
Master Inventor and
Senior IT Architect,
IBM Corporation
Sep 11-13, 2018
Johannesburg, South Africa

Abstract
2
Can't afford to save your world's
collection of data on your IT budget?
This session explains why object storage
is becoming so popular and how it differs
from block or file-level storage.
The speaker will provide an overview of
IBM Cloud Object Storage System and its
clever, highly scalable, highly reliable use
of Erasure Coding to reduce storage costs
by up to 70 percent from traditional disk
arrays

This week with Tony Pearson
3
Day Time Topic
Tuesday 11:30 Improve your NAS environment in One Day!
15:00
Dip your TOE in our Pool!
iSER and Data Reduction with Storage Virtualization
Wednesday 11:30
Save the World! Save your IT Budget with
IBM Cloud Object Storage System
15:00 How big is your NAS? Sizing, Management, and Deployment
16:15
Nightmares and Dreams:
Manage your entire Storage Infrastructure
Thursday 9:00
What else can you use that data for?
Adventures in Data Reuse
10:15 Be Persistent in your Journey to Private Cloud
13:45 Get ready for the NVMe Revolution

Agenda
4
Why Object Store?
What is IBM Cloud Object
Storage System?
How to use it: Applications
and Use Cases

How is Object Storage Different?
5
Block and File Storage
– Decide where to put it
• For block, which array/volume/LUN
• For file, which filer/subdirectory
– Remember where it is to get it back
– Don’t let anyone else move it
– Read/Write records, append data
– Limits on LUN size, number of files
Object Storage
– Provide data over to the Object storage
• Get “claim stub” reference locator
– Use or share “claim stub” to access
data HTTP, Openstack Swift, S3
– Get/Put/Delete object in its entirety
– Effectively “unlimited” scalability

Object Store for Unstructured data
6
Hot Data
High-IOPS and Low-Latency
All-Flash and Hybrid Flash/Disk
Information Lifecycle Management
(ILM)
Structured data / Random-Access
Active logs and traces
Virtual Machines and VDI
Single-Tenancy
Static and Stable data
Backups, Files, Archives
Seismic, Research, Telemetry, HPC
Video, Animation, Body Cams
Photos, Images, CAD/CAM, GIS
Music, Audio
Genomic, Medical Images
Multi-tenancy
Object Store provides a
• Secure
• Reliable
• Scalable
• Cost Effective
Platform For Unstructured data
Object Store
is not designed for
• High IOPS workflows
• Transaction
Processing
• Inherent ILM

Object Storage is Simpler for Application Development
7
POSIX – over 60 commands NFS – over 30 commands
HEAD
• Read metadata
GET
• Read content and
metadata
PUT
• Write content and
metadata
DELETE
• Remove object or
empty container
POST
• Update metadata
Object – 5 commands
SMB – over 80 commands

Object storage is dominant in cloud
8
0
50
100
150
200
250
2012 2013 2014 2015 2016 2017 2018
File-based software, appliances, and gateways
Object-based software and appliances (includes self-built/SaaS/cloud NAS)
IDC: Worldwide file- and object-based storage
2014–2018 Forecast (EB)

Why Object Storage?
9
It enables my
business and reduces
costs!
• Handles 80% of my
data
• Scales easily, growth
does not impact my
business plan
• Less expensive than
traditional flash and
disk arrays
It is easy to deploy and
manage!
• Easily provision to users
and teams
• Location-independent,
Flat namespace
• Software-Defined or
Pre-built systems
• Ethernet-based and
Cloud friendly
• Online backup/archive
alternative to tape and
virtual tape libraries
It is great for
developing software!
• GET/PUT interface
reduces coding effort
• Robust metadata for
search of
documents, images,
and videos
• Can be used for
Analytics, IoT,
Mobile, Web and
Cloud Applications
IT
Director
Software
Developer
Storage
Administrator

Agenda
10
Why Object Store?
Storage System?
and Use Cases

Storage Positioning – Filling a Gap
11
Unified file and
object storage.
Optimized for high
performance, across
flash and disk
Flash 15K
Unified file and object
storage on tape
Information Lifecycle
Management (ILM) across tiers
HighestPerformance
Lowest cost
Tape
IBM was looking to offer easy to
manage, scalable disk-based
object storage for unstructured
data
• Moderate performance
• Moderate cost
10K 7200 rpm

Data Protection Schemes
12
Tolerate 1 drive failure Tolerate 2 drive failures Tolerate “M” failures
RAID-1 / RAID-10
K pieces 2 x K slices
RAID-5
K pieces K + 1 slices
2.0X
1.2X
3.0X
1.5X
1.7XTriplication
K pieces 3 x K slices
RAID-6
K pieces K + 2 slices
Erasure Coding
K pieces K+M = N
slices

Data Growth at Petabyte (PB) Scale
13
PB of data
3 to 5x
Data Protection
RAID, Mirrors,
Replication, Tape
Data Protection
High Availability & Disaster Recovery
Geo-Distribution & Erasure Coding
Infrastructure
Proprietary, specialized
hardware, multiple systems
Operations
More than 1 FTE per PB
Maintenance outages
Infrastructure
Software Defined,
Commodity Hardware,
Single System
Operations
Less than 1 FTE per 6 PB
Single system, Secure
Self-healing
1.7 x 60% Less
Hardware &
Rack space
Traditional
Approach
IBM Cloud
Object Storage
Approach
70%
Lower
TCO

IBM Cloud Object Storage System economics beat
legacy NAS storage and Amazon Web Services (AWS)
14
$8,400
$4,210
$1,613
$1,053
Legacy NAS DR protected
Legacy NAS single copy
IBM Cloud Object + NAS
gateway
dsNet object protectedIBM Cloud Object
IBM Cloud Object vs NAS
Cost: 80% lower
$0
$2,000,000
$4,000,000
$6,000,000
$8,000,000
$10,000,000
480 TB 960 TB 1920 TB 3840 TB
S3
Cost: 10 to 60% lower
IBM Cloud Object vs AWS S3
IBM Cloud Object
Amazon S3

IBM Cloud Object Storage System – Terminology
15
HTTP
Application
Server
IBM COS
Manager
Accesser
Slicestor
Device
Set
Accesser PoolVault
Load
Balancer
Site A Site B Site C
End user
Global Namespace
IBM Cloud Object Storage
System organizes objects into
“Vaults” and “Containers”
Storage Pools can hold multiple
vaults
Vaults can have millions of
containers
OpenStack Swift
S3 Protocol

The Math Behind Reed-Solomon Erasure Coding
16
“K” variables of input data: a, b, c, d, e
Example: K=5 M=3 K+M= 8
Here we create “K+M”
equations, adding and
subtracting by different
co-factors
Results in “K+M” slices that
can be geographically
dispersed
We can tolerate losing up to
“M” slices of data, and still
solve for the original
“K” pieces of data.

Information Dispersal Algorithm (IDA)
17
01 02 03 04 05 06 07 08 09 10 11 12
A B C D
E F G
IDA is represented as Width/Read/Write
This example is 12/7/9
Original data is received by
IBM COS Accesser and
chopped up into 7 pieces
(Read threshold)
Erasure Coding:
7 variables in,
12 equation results out
(Width)
12 slices written in parallel
to IBM COS Slicestor devices
A B C D
E F G
IBM SmartWrite™
At least 9 slices (Write Threshold) must be
written for the I/O to be considered complete,
rest will be written later. IBM COS periodically
scans and rebuilds slices as needed
IBM SmartRead™
To read the data back,
only 7 slices are used
to solve 7 equations
(Read threshold)
Original data is
re-assembled

Dedicated
(Private)
IBM Cloud Object Storage System –
Single-site, Two-site and Multi-Site
18
Single Site
Better performance,
when site-tolerance
not a factor, better
than traditional
RAID-5 / RAID-6
Two Site Vault Mirroring
Allows customers to leverage existing two-site
infrastructure. Provides concurrent reads and
writes despite communication disruption
between locations. Local data better than
traditional RAID-5 / RAID-6
Geographically Dispersed
Three to Nine Sites
Consider adding IBM Cloud or leverage existing
datacenter locations to provide a broader
distribution of data for higher availability, site-
tolerance and scalability

Two Dispersal Modes
19
Standard Dispersal
One or more Accessers
Each slice on its own
Slicestor
Concentrated Dispersal (CD Mode)
Entry Level System (1-3 sites)
Accessers can be embedded into
Slicestor (Optional)
Each slice stored on separate HDD
within Slicestor for availability
01 02 03
04 05 06
07 08 09
10 11 12
01 02
03 04
05 06
07 08
09 10
11 12

Scalability
20
Site 1 Site 2 Site 3
S3 Compatible API Openstack Swift
Compatible API
Simple Object API
Slicestor
Software
Accesser
Software
Scalability – Scale performance and/or
capacity at any time with no downtime to
operations
Need more Performance?
Add more Accesser nodes
Need more Capacity?
Add more disks to existing Slicestor nodes,
add more device sets to existing storage
pools, or add more storage pools
– All nodes in single device set must have
same amount of storage
• New device sets can have different size
drives than existing drives sets
– All device sets in a storage pool must
have the same IDA
• Standard and Concentrated IDA can be
intermixed in same storage pool
– Different storage pools can have
different IDA settings
Device
Set 1
Device
Set 3
Device
Set 2

Encryption Options for IBM Cloud Object Storage
21
A
A B C D
E F G
Data
A
A#
AONT
Data
SecureSlice™
System-generated key
Used to encrypt data
Data
A
#
Hash taken of
encrypted data
Key wrapped with hash makes
All-or-Nothing Transform (AONT)
Information
Dispersal
Algorithm (IDA)
User sends data
Data
B
Data
B
Server-Side Encryption with Customer key
System-generated key used to encrypt data
Key wrapped with customer-supplied key
User sends data and key
SSE-C
Data
CData
C AMK
Application-managed key
User sends encrypted data
Encryption disabled in IBM COS

Compliance-Enabled Vault
22
Standard
Vault
Compliance
Enabled
Vault
Government Regulations specify that
“electronic storage must preserve the
records exclusively in a non-erasable,
non-rewriteable, format” (SEC 17a-4)
Various Terms have been used:
• Write Once Read Many (WORM)
for tape and optical media
• Fixed-Content or Content-
Addressable for disk-based storage
• Immutable for file systems
• Non-erasable, Non-rewriteable
(NENR) to cover all of the above
Compliance-Enabled Vaults prevent
objects from being deleted or modified
for a specified retention period
Vaults can co-exist
in same
storage pool

IBM Cloud Object Storage System –
Software, Pre-Built Systems or Cloud Services
23
Software
• ClevOS – IBM Cloud
Object Storage System
software packaged with
Debian Linux OS
• Software-defined,
hardware-aware model
for flexibility of x86
platform choice
• Can be deployed on
qualified vendor
hardware
Pre-built Systems
• Fully integrated appliance
models for easy
deployment and support
• On-premises object storage
solution
Cloud Services
• IBM Cloud Object
Storage System as a
service for customers
that want security and
controlled performance

IBM Spectrum Storage Suite
24
Bundle license for
flash, disk, tape and
storage-rich servers
Per TiB
IBM Spectrum
Storage Suite
Control Protect
Archive
Virtualize Accelerate
Scale
I’ve deployed Spectrum Storage Suite. Now
what?
1. Well, since you already own entitlement
to all capabilities in the Spectrum Storage
suite, you are free to download any of the
software you like, including IBM Cloud
Object Storage!
2. To help facilitate quick adoption of
additional use cases, Spectrum Storage
Suite licensing offers the ability to
perform extended tests in an evaluation
sandbox proving ground without
additional charge. Tebibytes (TiB) in this
sandbox are not counted.
So go ahead, experiment with your next use
case. Prove it, become familiar with it, pay for
it only when it’s deployed for productive use.

IBM Cloud Object Storage System – Deployment Options
25
Infrastructure
Dedicated
(Private)
Public
Local
(Private)
On-premises
Off-premises
Cloud
Infrastructure
Off-premises
(IBM Cloud)
Public
• Standard regional
• Nearline regional
• Nearline geo-
dispersed
Consumed
pricing
Dedicated
• IBM Managed
• Client managed
• Hybrid / Mixed
Allocated
pricing
Client
infrastructure
On-premises
Locally managed
• Software
• Pre-built systems
Entry Level
System as
small as 72 TB

Pre-Built Systems for IBM Cloud Object Storage System
26
IBM COS Manager 2105 / 3105
Accesser 2100 / 3105 / 4105
Slicestor 2212
Slicestor 2448
Health and performance monitoring
GUI and API access
Configuration and security
Zero downtime upgrades
12 drives @ 4, 6, 8 or 10 TB
48 to 120 TB Nearline HDD
in 2U rack space
16, 32 or 48 drives @ 4, 6, 8 or 10 TB
64 to 480 TB Nearline HDD
in 4U rack space
Slices, Disperses and Retrieves data

VersaStack Solution for Object Storage
27
Cisco UCS S3260 Storage Server
Dual nodes in 4U space
56 hot-swappable 3.5” LFF HDD
4, 6, 8, or 10 TB 7200-rpm NL-SAS
(28 drives per COS Slicestor)
Cisco UCS 6300 Fabric Interconnect
Low-latency, lossless 10 and 40 GbE
Cisco UCS 220 M4 servers
1U with 36 cores, 24 DDR4 memory DIMMs
For COS Manager and Accessers
Cisco Validated Design (CVD)
784 to 1960 TB
Usable Capacity

IBM Cloud Object Storage public cloud services
28
IBM Cloud Object Storage Flex
DataAccessPattern
Standard
IBM Cloud Object
Storage
Vault
Variable data
access pattern /
Mixed workloads
Predictable
data access
pattern /
workload
Frequent
data access
Less frequent
data access
Minimal
data access
IBM Cloud Object
Storage
Cold Vault
• High Availability – Cross Regional USA
• Low Latency – Regional (Dallas)

Agenda
29
Why Object Store?
Storage System?
and Use Cases

A broad spectrum of strategies to build, buy, or assemble an end to end solution
Integrations, Connectors, Gateways Ready to RunCustom Applications
Off-PremisesHybridOn-Premises
and/or and/or
How to Consume IBM Cloud Object Storage
30

Large
multimedia
files
Video, Audio,
Images
Store
telemetric,
instrumentatio
n and sensor
data
Web generated
and social
media data
Backup, Active
archive, DR,
inactive and
compliance
data
Security
audits, Log
files and
Analytics
Collaboration
and content
distribution
Global file
services,
mobile access,
and StaaS
Genomics,
HPC, Geo-
spatiial
8
7
6
5
4
1
2
3
Object Storage Typical Use Cases Storage Services
31

Market Industry Verticals
32
Secure
customer
trust and
business
compliance
.
Financial
Services &
Insurance
Incorporate
into IoT
processes,
analytics
Industrial,
Distribution
Manage the
data
essential to
serving the
good of the
public.
Public Sector,
Government
Scale your
market
offering
without
worry.
Telco &
Service
Providers
Create and
Deliver
Content on
demand.
Media &
Entertainmen
t
Heath Care &
Life Sciences
Put
medical
progress
before
everything
else.
Every industry does backup, all have some
amount of static-and-stable unstructured data,
most need file or content archive…

Client Experience – IBM Cloud Object Storage
33
Users upload photo
and video content via
web based application
Photo and video objects are sent to
IBM COS via REST based protocols
Metadata is
captured and
stored
• Scale – 150 petabytes and growing: more than 50 Billion images stored
• Manageability – 3 Administrators manage entire environment
• Security – 50,000+ uploads per minute with zero touch security
• Always-on availability – SLA of 100% download on demand – even during
California to Nevada datacenter move
• Economics – Operating costs reduced by more than 70%
• Key decision makers – Technical team backed by financial cost cutting
mandates

IBM Spectrum Protect –
Cloud and Object Store Capabilities
Client nodes
Off-premises:
• IBM Cloud
• Amazon Web Services
• Microsoft Azure
On-premises
IBM Spectrum
Protect Server
• IBM Cloud Object Storage System
(using S3-compatible API)
34

Backup Performance Comparison
35
0
5
10
15
20
25
IBM Tape IBM COS IBM ESS
Throughput Rates
Backup TB/hr Restore TB/hr
8 drives
1 drive 30 sessions
Backup and Restore rates vary depending on
network connection, workload, file size and other
environmental factors
Costs may vary, based on discounted street prices of LTO-8, IBM COS, and
ESS GLxS models as configured for “Large” IBM Spectrum Protect blueprint on AIX
8 file systems

IBM Spectrum Storage and IBM Cloud Object
36
storage. Optimized for
high performance, across
flash, disk and object
store
Flash
Object
Store
15K
Object storage on disk
( File, backup and archive interfaces
available through variety of options )
IBM Cloud
Amazon Web Services
Microsoft Azure
Swift S3 emulation
OpenStack Swift
storage on tape
Transparent Cloud Tiering
Information Lifecycle
Management (ILM) across tiers
HighestPerformance
Lowest cost
Tape10K 7200

Summary
37
Object Storage scales easily to
handle your unstructured data, easy
to manage, ideal for software
development
IBM is the #1 leading vendor of
Object Storage, with over 400
patents, and several clients >100PB
IBM Cloud Object Storage can be
used on-premises, hybrid, or in the
Cloud
Integrates with the rest of the IBM
Spectrum Storage suite

Please
complete
the session
survey!
38

Session reference links
40
http://www.ibm.com/training/events
https://www-03.ibm.com/systems/storage/
http://www.redbooks.ibm.com/
https://www-03.ibm.com/systems/storage/spectrum/suite/
http://www-03.ibm.com/software/products/en/spectrum-
protect
https://www-935.ibm.com/services/us/en/it-
services/business-continuity/draas/
https://www-
03.ibm.com/systems/storage/solutions/backup-and-
disaster-recovery/
http://www.ibm.com/systems/clientcenterdemonstrations
https://ibm.biz/client-experience-portal

About the Speaker
Tony Pearson is a Master Inventor and Senior IT Architect for the IBM Storage product line. Tony joined
IBM Corporation in 1986 in Tucson, Arizona, USA, and has lived there ever since. In his current role,
Tony presents briefings on storage topics covering the entire IBM Storage product line, IBM Spectrum
Storage software products, and topics related to Cloud Computing, Analytics and Cognitive Solutions.
He interacts with clients, speaks at conferences and events, and leads client workshops to help clients
with strategic planning for IBM’s integrated set of storage management software, hardware, and
virtualization solutions.
Tony writes the “Inside System Storage” blog, which is read by thousands of clients, IBM sales reps and
IBM Business Partners every week. This blog was rated one of the top 10 blogs for the IT storage
industry by “Networking World” magazine, and #1 most read IBM blog on IBM’s developerWorks. The
blog has been published in series of books, Inside System Storage: Volume I through V.
Over the past years, Tony has worked in development, marketing and consulting for various storage
hardware and software products. Tony has a Bachelor of Science degree in Software Engineering, and a
Master of Science degree in Electrical Engineering, both from the University of Arizona. Tony is an
inventor or co-inventor of 19 patents in the field of electronic data storage.
9000 S. Rita Road
Bldg 9032 Floor 1
Tucson, AZ 85744
+1 520-799-4309
(Office)
tpearson@us.ibm.com
Tony Pearson
Master Inventor
Senior IT Architect
IBM Storage
41

IBM Tucson Client Experience Center
Tucson, Arizona is headquarters
for IBM storage hardware and
software design and development
IBM Tucson Client Experience
Center offers:
–Technology briefings
–Product demonstrations
–Solution workshops
–Lab tours
Take a video tour!
–http://youtu.be/CXrpoCZAazg
https://www.ibm.com/it-infrastructure/services/client-centers

Email:
tpearson@us.ibm.com
Twitter:
twitter.com/az990tony
Blog:
ibm.co/Pearson
Books:
www.lulu.com/spotlight/990_tony
IBM Expert Network on Slideshare:
www.slideshare.net/az990tony
Facebook:
www.facebook.com/tony.pearson.16121
LinkedIn:
https://www.linkedin.com/in/az990tony
Additional Resources from Tony Pearson

This presentation uses the IBM Plex™ font
44
IBM Plex™ is our new typeface. It’s global, it’s versatile and it’s
distinctly IBM.
IBM Plex
Sans
The IBM company is freeing itself from the cold, modernist cliché
and replacing Helvetica with a new corporate typeface. Also
replaces Arial, Calibri, Lucida Grande, Trebuchet, etc.
IBM Plex
Mono
A little something for developers. Replaces
Courier New, Letter Gothic, Lucida Console, etc.
IBM Plex
Serif
A hybrid of the third kind (combining the best of Plex, Bodoni,
and Janson into a contemporary serif). Replaces Cambria,
Garamond, Lucida Bright, Times New Roman, etc.
IBM Plex is freely available as TrueType and OpenType at: https://github.com/IBM/plex/releases

Notices and disclaimers
45
© 2018 International Business Machines Corporation. No part of this
document may be reproduced or transmitted in any form without
written permission from IBM.
U.S. Government Users Restricted Rights — use, duplication or
disclosure restricted by GSA ADP Schedule Contract with IBM.
Information in these presentations (including information relating to
products that have not yet been announced by IBM) has been reviewed
for accuracy as of the date of initial publication and could include
unintentional technical or typographical errors. IBM shall have no
responsibility to update this information. This document is distributed
“as is” without any warranty, either express or implied. In no event,
shall IBM be liable for any damage arising from the use of this
information, including but not limited to, loss of data, business
interruption, loss of profit or loss of opportunity. IBM products and
services are warranted per the terms and conditions of the agreements
under which they are provided.
IBM products are manufactured from new parts or new and used parts.
In some cases, a product may not be new and may have been previously
installed. Regardless, our warranty terms apply.”
Any statements regarding IBM's future direction, intent or product
plans are subject to change or withdrawal without notice.
Performance data contained herein was generally obtained in a
controlled, isolated environments. Customer examples are presented as
illustrations of how those
customers have used IBM products and the results they may have
achieved. Actual performance, cost, savings or other results in other
operating environments may vary.
References in this document to IBM products, programs, or services
does not imply that IBM intends to make such products, programs or
services available in all countries in which IBM operates or does
business.
Workshops, sessions and associated materials may have been
prepared by independent session speakers, and do not necessarily
reflect the views of IBM. All materials and discussions are provided
for informational purposes only, and are neither intended to, nor shall
constitute legal or other guidance or advice to any individual
participant or their specific situation.
It is the customer’s responsibility to insure its own compliance
with legal requirements and to obtain advice of competent legal
counsel as to the identification and interpretation of any
relevant laws and regulatory requirements that may affect the
customer’s business and any actions the customer may need to take
to comply with such laws. IBM does not provide legal advice
or represent or warrant that its services or products will ensure that
the customer follows any law.
Information concerning non-IBM products was obtained from the
suppliers of those products, their published announcements or other
publicly available sources. IBM has not tested those products about
this publication and cannot confirm the accuracy of performance,
compatibility or any other claims related to non-IBM
products. Questions on the capabilities of non-IBM products should
be addressed to the suppliers of those products. IBM does not
warrant the quality of any third-party products, or the ability of
any such third-party products to interoperate with IBM’s products.
IBM expressly disclaims all warranties, expressed or implied,
including but not limited to, the implied warranties of
merchantability and fitness for a purpose.
The provision of the information contained herein is not intended to,
and does not, grant any right or license under any IBM patents,
copyrights, trademarks or other intellectual property right.
IBM, the IBM logo, ibm.com and [names of other referenced IBM
products and services used in the presentation] are trademarks of
International Business Machines Corporation, registered in many
jurisdictions worldwide. Other product and service names might
be trademarks of IBM or other companies. A current list of IBM
trademarks is available on the Web at "Copyright and trademark
information" at: www.ibm.com/legal/copytrade.shtml.

S104876 ibm-cos-jburg-v1809b

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to S104876 ibm-cos-jburg-v1809b

Similar to S104876 ibm-cos-jburg-v1809b (20)

More from Tony Pearson

More from Tony Pearson (20)

Recently uploaded

Recently uploaded (20)

S104876 ibm-cos-jburg-v1809b