IBM Cloud Object Storage: How it works and typical use cases

1. IBM Cloud Object Storage: How it works and typical use cases Tony Pearson IBM Master Inventor, Senior IT Management Consultant, TechU Content Manager 2019 IBM Systems Technical University 10-12 Sep 2019 | Johannesburg, SA

2. Abstract Can't afford to save your world's collection of data on your IT budget? This session explains why object storage is becoming so popular and how it differs from block or file-level storage. The speaker will provide an overview of IBM Cloud Object Storage System and its clever, highly scalable, highly reliable use of Erasure Coding to reduce storage costs by up to 70 percent from traditional disk arrays IBM Systems Technical University © Copyright IBM Corporation 2019 2

3. Agenda Why Object Store? What is IBM Cloud Object Storage System? How to use it: Applications and Use Cases IBM Systems Technical University © Copyright IBM Corporation 2019 3

4. How is Object Storage Different? IBM Systems Technical University © Copyright IBM Corporation 2019 4 Block and File Storage – Decide where to put it • For block, which array/volume/LUN • For file, which filer/subdirectory – Remember where it is to get it back – Don’t let anyone else move it – Read/Write records, append data – Limits on LUN size, number of files Object Storage – Provide data over to the Object storage • Get “claim stub” reference locator – Use or share “claim stub” to access data HTTP, Openstack Swift, S3 – Get/Put/Delete object in its entirety – Effectively “unlimited” scalability

5. Object Storage for Static and Stable data IBM Systems Technical University © Copyright IBM Corporation 2019 5 Hot Data High-IOPS and Low-Latency All-Flash and Hybrid Flash/Disk Information Lifecycle Management (ILM) Structured data / Random-Access Active logs and traces Virtual Machines and VDI Single-Tenancy Static and Stable data Backups, Files, Archives Seismic, Research, Telemetry, HPC Video, Animation, Body Cams Photos, Images, CAD/CAM, GIS Music, Audio Genomic, Medical Images Multi-tenancy Object Store provides a • Secure • Reliable • Scalable • Cost Effective Platform For Unstructured data Object Store is not designed for • High IOPS workflows • Transaction Processing • Inherent ILM

6. Object Storage is Simpler for Application Development IBM Systems Technical University © Copyright IBM Corporation 2019 6 POSIX – over 60 commands NFS – over 30 commands HEAD • Read metadata GET • Read content and metadata PUT • Write content and metadata DELETE • Remove object or empty container POST • Update metadata Object – 5 commands SMB – over 80 commands

7. Why Object Storage? IBM Systems Technical University © Copyright IBM Corporation 2019 7 It enables my business and reduces costs! • Handles 80% of my data • Scales easily, growth does not impact my business plan • Less expensive than traditional flash and disk arrays It is easy to deploy and manage! • Easily provision to users and teams • Location-independent, Flat namespace • Software-Defined or Pre-built systems • Ethernet-based and Cloud friendly • Online backup/archive alternative to tape and virtual tape libraries It is great for developing software! • GET/PUT interface reduces coding effort • Robust metadata for search of documents, images, and videos • Can be used for Analytics, IoT, Mobile, Web and Cloud Applications IT Director Software Developer Storage Administrator

8. Agenda IBM Systems Technical University © Copyright IBM Corporation 2019 8 Why Object Store? What is IBM Cloud Object Storage System? How to use it: Applications and Use Cases

9. Moderate Performance, Moderate Cost IBM Systems Technical University © Copyright IBM Corporation 2019 9 Unified file and object storage. Optimized for high performance, across flash and disk Flash 15K Unified file and object storage on tape Information Lifecycle Management (ILM) across tiers HighestPerformance Lowest cost Tape IBM was looking to offer easy to manage, scalable disk-based object storage for unstructured data • Moderate performance • Moderate cost 10K 7200 rpm

10. RAID versus Erasure Coding IBM Systems Technical University © Copyright IBM Corporation 2019 10 Tolerate 1 drive failure Tolerate 2 drive failures Tolerate “M” failures RAID-1 / RAID-10 K pieces : 2 x K slices RAID-5 K pieces : K + 1 slices 2.0X 1.2X 3.0X 1.5X 1.7X Triplication K pieces : 3 x K slices RAID-6 K pieces : K + 2 slices Erasure Coding K pieces : K+M = N slices

11. Data Growth at Petabyte (PB) Scale IBM Systems Technical University © Copyright IBM Corporation 2019 11 PB of data 3 to 5x Data Protection RAID, Mirrors, Replication, Tape Data Protection High Availability & Disaster Recovery Geo-Distribution & Erasure Coding Infrastructure Proprietary, specialized hardware, multiple systems Operations More than 1 FTE per PB Maintenance outages Infrastructure Software Defined, Commodity Hardware, Single System Operations Less than 1 FTE per 6 PB Single system, Secure Self-healing 1.7 x 60% Less Hardware & Rack space Traditional Approach IBM Cloud Object Storage Approach 70% Lower TCO

12. IBM Cloud Object Storage System economics beat legacy NAS storage and Amazon Web Services (AWS) IBM Systems Technical University © Copyright IBM Corporation 2019 12 $8,400 $4,210 $1,613 $1,053 Legacy NAS DR protected Legacy NAS single copy IBM Cloud Object + NAS gateway dsNet object protected IBM Cloud Object IBM Cloud Object vs NAS Cost: 80% lower $0 $2,000,000 $4,000,000 $6,000,000 $8,000,000 $10,000,000 480 TB 960 TB 1920 TB 3840 TB Cost: 10 to 60% lower IBM Cloud Object vs AWS S3 IBM Cloud Object Amazon S3

13. IBM Cloud Object Storage System – Terminology IBM Systems Technical University © Copyright IBM Corporation 2019 13 HTTP Application Server IBM COS Manager Accesser Slicestor Device Set Accesser Pool Vault Load Balancer Site A Site B Site C End user Global Namespace IBM Cloud Object Storage System organizes objects into “Vaults” and “Containers” Storage Pools can hold multiple vaults Vaults can have millions of containers OpenStack Swift S3 Protocol

14. Global Manager UI Changes – Top Navigation New changes in the IBM Cloud Object Storage control portal (Manager) make the visual experience consistent with other IBM Storage products Top navigation bar is now textual based main navigation with icons for secondary information IBM Systems Technical University © Copyright IBM Corporation 2019 14

15. The Math Behind Reed-Solomon Erasure Coding IBM Systems Technical University © Copyright IBM Corporation 2019 15 “K” variables of input data: a, b, c, d, e Example: K=5 M=3 K+M= 8 Here we create “K+M” equations, adding and subtracting by different co-factors Results in “K+M” slices that can be geographically dispersed We can tolerate losing up to “M” slices of data, and still solve for the original “K” pieces of data.

16. Information Dispersal Algorithm (IDA) IBM Systems Technical University © Copyright IBM Corporation 2019 16 01 02 03 04 05 06 07 08 09 10 11 12 A B C D E F G IDA is represented as Width/Read/Write This example is 12/7/9 Original data is received by IBM COS Accesser and chopped up into 7 pieces (Read threshold) Erasure Coding: 7 variables in, 12 equation results out (Width) 12 slices written in parallel to IBM COS Slicestor devices A B C D E F G IBM SmartWrite™ At least 9 slices (Write Threshold) must be written for the I/O to be considered complete, rest will be written later. IBM COS periodically scans and rebuilds slices as needed IBM SmartRead™ To read the data back, only 7 slices are used to solve 7 equations (Read threshold) Original data is re-assembled

17. Dedicated (Private) IBM Cloud Object Storage System – Single-site, Two-site and Multi-Site IBM Systems Technical University © Copyright IBM Corporation 2019 17 Single Site Better performance, when site-tolerance not a factor, better than traditional RAID-5 / RAID-6 Two Site Vault Mirroring Allows customers to leverage existing two-site infrastructure. Provides concurrent reads and writes despite communication disruption between locations. Local data better than traditional RAID-5 / RAID-6 Geographically Dispersed Three to Nine Sites Consider adding IBM Cloud or leverage existing datacenter locations to provide a broader distribution of data for higher availability, site- tolerance and scalability

18. Two Dispersal Modes IBM Systems Technical University © Copyright IBM Corporation 2019 18 — Standard Dispersal — One or more Accessers — Each slice on its own Slicestor — Concentrated Dispersal (CD Mode) — Entry Level System (1-3 sites) — Accessers can be embedded into Slicestor (Optional) — Each slice stored on separate HDD within Slicestor for availability 01 02 03 04 05 06 07 08 09 10 11 12 01 02 03 04 05 06 07 08 09 10 11 12

19. Scalability IBM Systems Technical University © Copyright IBM Corporation 2019 19 Site 1 Site 2 Site 3 S3 Compatible API Openstack Swift Compatible API Simple Object API Slicestor Software Accesser Software Scalability – Scale performance and/or capacity at any time with no downtime to operations Need more Performance? Add more Accesser nodes Need more Capacity? Add more disks to existing Slicestor nodes, add more device sets to existing storage pools, or add more storage pools –All nodes in single device set must have same amount of storage • New device sets can have different size drives than existing drives sets –All device sets in a storage pool must have the same width • Standard and Concentrated IDA can be intermixed in same storage pool –Different storage pools can have different widths Device Set 1 Device Set 2 Device Set 3

20. Encryption Options for IBM Cloud Object Storage IBM Systems Technical University © Copyright IBM Corporation 2019 20 A A B C D E F G Data A A# AONT Data SecureSlice™ System-generated key Used to encrypt data Data A # Hash taken of encrypted data Key wrapped with hash makes All-or-Nothing Transform (AONT) Information Dispersal Algorithm (IDA) User sends data Data B Data B Server-Side Encryption with Customer key System-generated key used to encrypt data Key wrapped with customer-supplied key User sends data and key SSE-C Data CData C AMK Application-managed key User sends encrypted data Encryption disabled in IBM COS

21. Compliance-Enabled Vault IBM Systems Technical University © Copyright IBM Corporation 2019 21 Standard Vault Compliance Enabled Vault Government Regulations specify that “electronic storage must preserve the records exclusively in a non-erasable, non-rewriteable, format” (SEC 17a-4) Various Terms have been used: • Write Once Read Many (WORM) for tape and optical media • Fixed-Content or Content-Addressable for disk- based storage • Immutable for file systems Non-erasable, Non-rewriteable (NENR) adopted to cover all of the above Compliance-Enabled Vaults prevent objects from being deleted or modified for a specified retention period Vaults can co-exist in same storage pool

22. Compliance-Enabled Objects in Container Mode - Features IBM Cloud Object Storage now supports Compliance-Enabled Objects (NENR/WORM) in the Container Mode configuration  Retention Policy (min, max, default) - per bucket  Retention Period - per object level  Extending Retention - per object  Temporary Legal Hold - per object  Event based Legal Hold (indefinite) – per object  Application owned & initiated Event trigger -per object  Permanent Retention - per bucket/object  UI support  API support  SDK support IBM Systems Technical University © Copyright IBM Corporation 2019 22

23. The Global Data Privacy Regulation (GDPR) Readiness A comprehensive readiness program took place to ensure the enablement of IBM Cloud Object Storage to be used by customers processing European Union citizen personal data — New features added to handle retention and redaction of log data and enhance security access controls — Risk assessments of data processing activities completed — Potential IBM hardware parts with EU customer data identified for additional destruction and secure handling IBM Systems Technical University © Copyright IBM Corporation 2019 23

24. IBM Cloud Object Storage System – Software, Pre-Built Systems or Cloud Services IBM Systems Technical University © Copyright IBM Corporation 2019 24 Software • ClevOS – IBM Cloud Object Storage System software packaged with Debian Linux OS • Software-defined, hardware- aware model for flexibility of x86 platform choice • Can be deployed on qualified vendor hardware Pre-built Systems • Fully integrated appliance models for easy deployment and support • On-premises object storage solution Cloud Services • IBM Cloud Object Storage System as a service for customers that want security and controlled performance

25. IBM Spectrum Storage Suite IBM Systems Technical University © Copyright IBM Corporation 2019 25 Bundle license for flash, disk, tape and storage- rich servers Per TiB IBM Spectrum Storage Suite Control Protect Archive Virtualize Accelerate Scale IBM Cloud Object Storage I’ve deployed Spectrum Storage Suite. Now what? 1. Well, since you already own entitlement to all capabilities in the Spectrum Storage suite, you are free to download any of the software you like, including IBM Cloud Object Storage! 2. To help facilitate quick adoption of additional use cases, Spectrum Storage Suite licensing offers the ability to perform extended tests in an evaluation sandbox proving ground without additional charge. Tebibytes (TiB) in this sandbox are not counted. So go ahead, experiment with your next use case. Prove it, become familiar with it, pay for it only when it’s deployed for productive use.

26. Spectrum Discover Integration - Architecture IBM Cloud Object Storage now supports integration with IBM’s Spectrum Discover through metadata scanning and event notifications October 11, 2018 IBM & IBM Business Partner Internal Use Only Scanning and Event Notifications I BM Spectrum Discover File and Object Storage Data Activation/OptimizationData Insight Analyze Governance Optimize •Data discovery •Dataset identification •Data pipeline progression •Data inspection •Data classification •Data clean-up •Archive / tiering •Duplicate data removal •Trivial data removal Use Cases •Simple to deploy (VMware virtual appliance) •Metadata curation •Custom metadata tagging •Automatic indexing •Policy-Engine •Action Agent API Reporting DashboardSearch Planned for 2019 and other 3rd parties IBM Systems Technical University © Copyright IBM Corporation 2019 26

27. IBM Cloud Object Storage public cloud services IBM Cloud Object Storage Flex DataAccessPattern IBM Cloud Object Storage Standard IBM Cloud Object Storage Vault Variable data access pattern / Mixed workloads Predictable data access pattern / workload Frequent data access Less frequent data access Minimal data access IBM Cloud Object Storage Cold Vault • High Availability – Cross Regional USA • Low Latency – Regional (Dallas) IBM Systems Technical University © Copyright IBM Corporation 2019 27

28. Agenda IBM Systems Technical University © Copyright IBM Corporation 2019 28 Why Object Store? What is IBM Cloud Object Storage System? How to use it: Applications and Use Cases

29. A broad spectrum of strategies to build, buy, or assemble an end to end solution Integrations, Connectors, Gateways Ready to RunCustom Applications Off-PremisesHybridOn-Premises and/or and/or How to Consume IBM Cloud Object Storage IBM Systems Technical University © Copyright IBM Corporation 2019 29

30. Large multimedia files Video, Audio, Images Store telemetric, instrumentatio n and sensor data Web generated and social media data Backup, Active archive, DR, inactive and compliance data Security audits, Log files and Analytics Collaboration and content distribution Global file services, mobile access, and StaaS Genomics, HPC, Geo- spatiial 8 7 6 5 4 1 2 3 Object Storage Typical Use Cases Storage Services IBM Systems Technical University © Copyright IBM Corporation 2019 30

31. Market Industry Verticals IBM Systems Technical University © Copyright IBM Corporation 2019 31 Secure customer trust and business compliance . Financial Services & Insurance Incorporate into IoT processes, analytics Industrial, Distribution Manage the data essential to serving the good of the public. Public Sector, Government Scale your market offering without worry. Telco & Service Providers Create and Deliver Content on demand. Media & Entertainment Heath Care & Life Sciences Put medical progress before everything else. Every industry does backup, all have some amount of static-and-stable unstructured data, most need file or content archive…

32. Client Experience – IBM Cloud Object Storage IBM Systems Technical University © Copyright IBM Corporation 2019 32 Users upload photo and video content via web based application Photo and video objects are sent to IBM COS via REST based protocols Metadata is captured and stored • Scale – 150 petabytes and growing: more than 50 Billion images stored • Manageability – 3 Administrators manage entire environment • Security – 50,000+ uploads per minute with zero touch security • Always-on availability – SLA of 100% download on demand – even during California to Nevada datacenter move • Economics – Operating costs reduced by more than 70% • Key decision makers – Technical team backed by financial cost cutting mandates

33. IBM Spectrum Protect – Cloud and Object Store Capabilities IBM Systems Technical University © Copyright IBM Corporation 2019 33 Client nodes Off-premises: • IBM Cloud • Amazon Web Services • Microsoft Azure On-premises IBM Spectrum Protect Server • IBM Cloud Object Storage System (using S3-compatible API)

34. IBM Spectrum Storage and IBM Cloud Object IBM Systems Technical University © Copyright IBM Corporation 2019 34 Unified file and object storage. Optimized for high performance, across flash, disk and object store Flash Object Store 15K Object storage on disk ( File, backup and archive interfaces available through variety of options ) IBM Cloud Amazon Web Services Microsoft Azure Swift S3 emulation OpenStack Swift Unified file and object storage on tape Transparent Cloud Tiering Information Lifecycle Management (ILM) across tiers HighestPerformance Lowest cost  Tape10K 7200

35. Summary IBM Systems Technical University © Copyright IBM Corporation 2019 35 Object Storage scales easily to handle your unstructured data, easy to manage, ideal for software development IBM is the #1 leading vendor of Object Storage, with over 400 patents, and several clients >100PB IBM Cloud Object Storage can be used on-premises, hybrid, or in the Cloud Integrates with the rest of the IBM Spectrum Storage suite

37. Special Thanks I would like to thank the following colleagues who contributed charts, insights, and review comments for these presentation materials — John Shubeck — Chris Maestas IBM Systems Technical University © Copyright IBM Corporation 2019 37

38. COS Knowledge Center – Overview — This feature creates a public COS Knowledge Center (KC) at https://www.ibm.com/support/knowledgecenter/STXNRM for on-premises documentation. — This feature embeds a standalone KC application in the Manager UI for use as a help system. The embedded KC replaces the current contextual help and Help Index. P R E - K N O W L E D G E C E N T E R P O S T - K N O W L E D G E C E N T E R — Access issues to documentation — No search capability — Help content and product documentation can be out of sync — Non-Compliance with IBM Documentation Standards — Slow documentation update time — Publicly accessible — Enterprise search capability — All document versions housed in one place — Consistent with other IBM product documentation — Shorter times for documentation updates — Mobile support IBM Systems Technical University © Copyright IBM Corporation 2019 38

39. About the Speaker 39 Tony Pearson is a Master Inventor, Senior IT Management Consultant, and Content Manager for the IBM Systems Technical University events. Tony joined IBM Corporation in 1986 in Tucson, Arizona, USA, and has lived there ever since. Tony presents briefings on storage topics covering the entire IBM Storage product line, IBM Spectrum Storage software products, and topics related to Cloud Computing, Analytics and Cognitive Solutions. He interacts with clients, speaks at conferences and events, and leads client workshops to help clients with strategic planning for IBM’s integrated set of storage management software, hardware, and virtualization solutions. Tony writes the “Inside System Storage” blog, which is read by thousands of clients, IBM sales reps and IBM Business Partners every week. This blog was rated one of the top 10 blogs for the IT storage industry by “Networking World” magazine, and #1 most read IBM blog on IBM’s developerWorks. The blog has been published in series of books, Inside System Storage: Volume I through V. Over the past years, Tony has worked in development, marketing and consulting for various IBM Systems hardware and software products. Tony has a Bachelor of Science degree in Software Engineering, and a Master of Science degree in Electrical Engineering, both from the University of Arizona. Tony is an inventor or co-inventor of 19 patents in the field of IBM Systems and electronic data storage. 9000 S. Rita Road Bldg 9032 Floor 1 Tucson, AZ 85744 +1 520-799-4309 (Office) tpearson@us.ibm.com Tony Pearson Master Inventor Senior Management Consultant, IBM Systems La Services IBM Storage IBM Systems Technical University © Copyright IBM Corporation 2019

40. My Social Media Presence Blog*: ibm.co/Pearson LinkedIn: https://www.linkedin.com/in/az990tony Books: www.lulu.com/spotlight/990_tony IBM Expert Network on Slideshare: www.slideshare.net/az990tony Twitter: twitter.com/az990tony Facebook: www.facebook.com/tony.pearson.16121 Instagram: www.instagram.com/az990tony/ Email: tpearson@us.ibm.com * Not a typo. This is short URL for https://www.ibm.com/developerworks/mydeveloperworks/blogs/InsideSystemStorage/ IBM Systems Technical University © Copyright IBM Corporation 2019 40

41. Notices and disclaimers — © 2019 International Business Machines Corporation. No part of this document may be reproduced or transmitted in any form without written permission from IBM. — U.S. Government Users Restricted Rights — use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM. — Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. This document is distributed “as is” without any warranty, either express or implied. In no event, shall IBM be liable for any damage arising from the use of this information, including but not limited to, loss of data, business interruption, loss of profit or loss of opportunity. IBM products and services are warranted per the terms and conditions of the agreements under which they are provided. — IBM products are manufactured from new parts or new and used parts. In some cases, a product may not be new and may have been previously installed. Regardless, our warranty terms apply.” — Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice. — Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented as illustrations of how those — customers have used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary. — References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business. — Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation. — It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer’s business and any actions the customer may need to take to comply with such laws. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer follows any law. IBM Systems Technical University © Copyright IBM Corporation 2019 41

42. Notices and disclaimers continued — Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products about this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non- IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the ability of any such third-party products to interoperate with IBM’s products. IBM expressly disclaims all warranties, expressed or implied, including but not limited to, the implied warranties of merchantability and fitness for a purpose. — The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual property right. — IBM, the IBM logo, ibm.com and [names of other referenced IBM products and services used in the presentation] are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml IBM Systems Technical University © Copyright IBM Corporation 2019 42

43. This presentation uses the IBM Plex™ font IBM Plex™ is our new typeface. It’s global, it’s versatile and it’s distinctly IBM. IBM Plex Sans The IBM company is freeing itself from the cold, modernist cliché and replacing Helvetica with a new corporate typeface. Also replaces Arial, Calibri, Lucida Grande, Trebuchet, etc. IBM Plex Mono A little something for developers. Replaces Courier New, Letter Gothic, Lucida Console, etc. IBM Plex Serif A hybrid of the third kind (combining the best of Plex, Bodoni, and Janson into a contemporary serif). Replaces Cambria, Garamond, Lucida Bright, Times New Roman, etc. IBM Plex is freely available as TrueType and OpenType at: https://github.com/IBM/plex/releases and looks consistently good across Windows, Linux and Mac IBM Systems Technical University © Copyright IBM Corporation 2019 43