Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Storage 200

284 views

Published on

In this course, you will learn advanced storage features like NVMe storage options and performance. You will understand how to use volume groups and File Storage Service to improve performance of your workload.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Storage 200

  1. 1. 1Copyright © 2018, Oracle and/or its affiliates. All rights reserved.Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Storage Level 200 Flavio Pereira November 2018
  2. 2. 2Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
  3. 3. 3Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Objectives • Explain local NVMe SSD devices • Block Storage Volume Groups and Performance • File Storage Service Performance • Object Storage and Amazon S3 combability API • Data Backup and Migration Options
  4. 4. 4Copyright © 2018, Oracle and/or its affiliates. All rights reserved. OCI Storage Services Local NVMe Block Volume File Storage Object Storage Archive Storage Type NVMe SSD based temporary storage NVMe SSD based block storage NFSv3 compatible file system Highly durable Object storage Long-term archival and backup Durability Non-persistent; survives reboots Durable (multiple copies in an AD) Durable (multiple copies in an AD) Highly durable (multiple copies across ADs) Highly durable (multiple copies across ADs) Capacity Terabytes+ Petabytes+ Exabytes+ Petabytes+ Petabytes+ Unit Size 51.2 TB for BM, 6.4-25.6 TB for VM 50 GB to 32 TB/vol 32 vols/instance Up to 8 Exabyte 10 TB/object 10 TB/object Use cases Big Data, OLTP, high performance workloads Apps that require SAN like features (Oracle DB, VMW, Exchange) Apps that require shared file system (EBS, HPC) Unstructured data incl. logs, images, videos Long term archival and backups (Oracle DB backups)
  5. 5. 5Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Local NVMe SSD Devices Instance type NVMe SSD Devices BM.DenseIO2.52 8 drives = 51.2 TB raw VM.DenseIO2.8 2 drive = 6.4 TB raw VM.DenseIO2.16 4 drives = 12.8 TB raw VM.DenseIO2.24 8 drives = 25.6 TB raw • DenseIO instance shapes include locally attached NVMe devices • The acronym NVMe stands for Non-Volatile Memory Express • Designed for high performance and non-volatile storage media, NVMe is the only protocol that stands out in highly demanding and compute intensive enterprise, cloud and edge data ecosystems.
  6. 6. 6Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Data deleted on instance reboot or pause, not usable for primary data Data saved on instance reboot or pauseLocal NVMe SSD 101011010101010 100101010101010 010101010010101 00010011110101 Instance (VM/BM) Local NVMe SSD 101011010101010 100101010101010 010101010010101 00010011110101 Instance (VM/BM) “With Oracle Cloud Infrastructure, companies can leverage NVMe for persistent storage to host databases and applications. However, other cloud providers typically do not offer such a capability. In cases where NVMe storage was an option with other vendors, it was not persistent. This meant that the multi-terabyte database that researchers loaded to this storage was lost when the server stopped. Accenture NVMe SSD Persisted - Reboot/Pause
  7. 7. 7Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Protecting NVMe SSD Devices RAID 1: An exact copy (or mirror) of a set of data on two or more disks RAID 10: Stripes data across multiple mirrored pairs. As long as one disk in each mirrored pair is functional, data can be retrieved RAID 6: Block-level striping with two parity blocks distributed across all member disks
  8. 8. 8Copyright © 2018, Oracle and/or its affiliates. All rights reserved. What to do when an NVMe Device Fails • If an NVMe device fails while the instance is in service, you should start another instance with the same amount of storage or more and then copy the data onto the new instance, replacing the old instance. • There are multiple toolsets for copying large amounts of data, with rsync being the most popular. Since the connectivity between instances is a full 10Gb/sec, copying data should be quick. • Remember that with a failed device, your array may no longer be protected, so you should copy the data off of the impacted instance as quickly as possible.
  9. 9. 9Copyright © 2018, Oracle and/or its affiliates. All rights reserved. NVMe Performance Using a Bare Metal shape with 52 CPUs- BM.DenseIO2.52 the following command was executed following the cloud harmony test suite and we are getting ~500K IOPS (50/50) on a Read and Write Mix test for a single NVMe device. # run.sh --target=/dev/nvme1n1 --test=iops --nopurge --noprecondition --fio_direct=1 --fio_size=10g -- skip_blocksize 512b --skip_blocksize 1m --skip_blocksize 8k --skip_blocksize 16k --skip_blocksize 32k -- skip_blocksize 64k --skip_blocksize 128k
  10. 10. 10Copyright © 2018, Oracle and/or its affiliates. All rights reserved. NVMe Performance Using a Bare Metal shape with 52 CPUs- BM.DenseIO2.52 the following command was executed following the cloud harmony test suite and we are getting ~ 3.0MM IOPS (50/50) on a Read and Write Mix test for all NVMe devices combined # run.sh `ls /dev/nvme[0-9]n1 | sed -e 's///--target=//'` --test=iops --nopurge --noprecondition -- fio_direct=1 --fio_size=10g --skip_blocksize 512b --skip_blocksize 1m --skip_blocksize 8k --skip_blocksize 16k --skip_blocksize 32k --skip_blocksize 64k --skip_blocksize 128k
  11. 11. 11Copyright © 2018, Oracle and/or its affiliates. All rights reserved. SLA for NVMe Performance Shape Minimum Supported IOPS VM.DenseIO1.4 200k VM.DenseIO1.8 250k VM.DenseIO1.16 400k BM.DenseIO1.36 2.5MM VM.DenseIO2.8 250k VM.DenseIO2.16 400k VM.DenseIO2.24 800k BM.DenseIO2.52 3.0MM • OCI provides a service-level agreement (SLA) for NVMe performance • Measured against 4k block sizes with 100% random write workload on Dense IO shapes where the drive is in a steady-state of operation • Run test on Oracle Linux shapes with 3rd party Benchmark Suites, https://github.com/cloudharmony/block- storage SLA: https://cloud.oracle.com/en_US/iaas/sla
  12. 12. 12Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Block Volume
  13. 13. 13Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Block Volume Service • Block Volume Service let you store data on block volumes independently and beyond the lifespan of compute instances • Block volumes operates at the raw storage device level and manages data as a set of numbered, fixed-size blocks using a protocol such as iSCSI • You can create, attach, connect, and move volumes, as needed, to meet your storage and application requirements • Typical Scenarios – Persistent and Durable Storage – Expand an Instance's Storage – Instance Scaling
  14. 14. 14Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Volume Groups • Group together block and boot volumes from multiple compartments across multiple compute instances in a volume group. • You can use volume groups to create volume group backups and clones that are point-in-time and crash-consistent. • Manually trigger a full or incremental backup of all the volumes in a volume group leveraging a coordinated snapshot across all the volumes.
  15. 15. 15Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Volume Groups for Coordinate Backups • A coordinated point-in-time crash consistent backup of the entire set of volumes that are in a volume group. This operation creates a volume group backup. • From a volume group a manual backup can be created. This can be either a full or incremental backup (a full backup is required to exist before an incremental backup can be triggered). • When backing up a volume group an aggregated state for the backup is available to enable applications to query the progress of the backup in particular to figure out when the coordinated backups have been completed.
  16. 16. 16Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Volume Groups for Coordinate Clones • Deep disk-to-disk copy of an entire volume group and all the volumes in the volume group. • This creates a new completely isolated volume group and new set of volumes that are also completely isolated from their corresponding source volumes. • A clone can be attached and used as regular volume when its lifecycle state becomes "available”, usually within seconds. • Hydration will continue happening in the background. There may be latency spikes for blocks of data which are not yet copied over.
  17. 17. 17Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Volume Groups Console
  18. 18. 18Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Block Volume Cross Region Backup You can copy backups to a different region and It will be possible apply cross region copy to manual or scheduled backups so that backups created in one region are seamlessly copied to another region. Cross region backups are just like regular backups encrypted during transit and at rest. What are the main use cases? • Disaster recovery – by copying backups to another region at a regular interval customers can, in the case of disaster, restore their applications in a second region using the cross region backups. • Migration – using cross region backups customers can migrate and expand their applications into another region.
  19. 19. 19Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Block Volume Performance • There are 3 aspects to the block volume performance: • IOPS • Throughput • Latency • The Block Volume service uses NVMe-based storage infrastructure, and is designed for consistency. You just need to provision the capacity needed and performance scales linearly per GB volume size up to the service maximums. • Throughput performance on VM instances is dependent on the network bandwidth that is available to the instance, and further limited by that bandwidth for the volume. • IOPS performance is independent of the instance type or shape, so is applicable to all bare metal and VM shapes, for iSCSI attached volumes.
  20. 20. 20Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Block Volume Performance Metric Characteristic Volume Size 50 GB to 32 TB, in 1 GB increments IOPS 60 IOPS/GB , up to 25,000 IOPS Throughput 480 KBPS/GB, up to 320 MBPS Latency Sub-millisecond latencies. Per-instance Limits 32 attachments per instance, up to 1 PB. Up to 620K or more IOPS, near line rate throughout. • The following table describes the performance characteristics of the service: • IOPS performance is independent of the instance type or shape, so is applicable to all bare metal and VM shapes, for iSCSI attached volumes. • Due to the overhead of virtualization, paravirtulized attachments reduces the maximum IOPS performance for larger block volumes
  21. 21. 21Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Block Volume Performance Here is a sample FIO commands we use for throughput measurement on a single volume: Read-only: # sudo fio --direct=1 --ioengine=libaio --size=10g --bs=4k --runtime=60 --numjobs=8 --iodepth=64 --time_based -- rw=randread --group_reporting --filename=/dev/sdb --name=iops-test Write-only: # sudo fio --direct=1 --ioengine=libaio --size=10g --bs=4k --runtime=60 --numjobs=8 --iodepth=64 --time_based -- rw=randwrite --group_reporting --filename=/dev/sdb --name=iops-test Read/write Mix: # sudo fio --direct=1 --ioengine=libaio --size=10g --bs=4k --runtime=60 --numjobs=8 --iodepth=64 --time_based -- rw=randrw --group_reporting --filename=/dev/sdb --name=iops-test Note: In read/write case, you need to add the read result and write result for duplex traffic. Also, please note that all volumes attached to an instance share the same network bandwidth with the instance. If there is heavy network traffic or other volumes are under I/O pressure, the apparent performance of single volume may look degraded
  22. 22. 22Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Block Volume Performance Our block volume service performance measurement methodology and characteristics are covered in detail here: https://blogs.oracle.com/cloud-infrastructure/block-volume-performance-analysis https://docs.us-phoenix-1.oraclecloud.com/Content/Block/Concepts/blockvolumeperformance.htm http://www.storagereview.com/oracle_cloud_infrastructure_compute_bare_metal_instances_review WARNING: Before running any tests, protect your data by making a backup of your data and operating system environment to prevent any data loss. Do not run FIO tests directly against a device that is already in use, such as /dev/sdX. If it is in use as a formatted disk and there is data on it, running FIO with a write workload (readwrite, randrw, write, trimwrite) will overwrite the data on the disk, and cause data corruption. Run FIO only on unformatted raw devices that are not in use.
  23. 23. 23Copyright © 2018, Oracle and/or its affiliates. All rights reserved. File Storage Service
  24. 24. 24Copyright © 2018, Oracle and/or its affiliates. All rights reserved. File Storage Performance • When performing reads and writes of large blocks (~1MB), for each terabyte of data stored you can expect: • Overall read performance of at least 100 MB/sec • Overall write performance of at least 50 MB/sec • At least 2,500 read IOPs • The highest levels of performance assume concurrent access that can be achieved by using multiple clients, multiple threads, and multiple mount targets.
  25. 25. 25Copyright © 2018, Oracle and/or its affiliates. All rights reserved. File Storage Performance This table describes the level of performance expected for different size file systems. Although we do not guarantee these numbers, we expect that customers should be able to achieve at least this level of performance in the current File Storage service. File System Read Bandwidth (1MB Blocks) Write Bandwidth (1MB Blocks) Read IOPS (8K Blocks) 1 TB 100 MB/s 50 MB/s 2,500 10 TB 1 GB/s 500 MB/s 25,000 100 TB 10 GB/s 5GB/s 250,000 More details you can find on File Storage Performance Guide: https://cloud.oracle.com/iaas/whitepapers/file_storage_performance_guide.pdf
  26. 26. 26Copyright © 2018, Oracle and/or its affiliates. All rights reserved. File Storage Performance To optimize the performance of File Storage, consider the following guidelines: • File Storage scales based on consumed capacity, with larger file systems receiving more available bandwidth. • File Storage performance increases with parallelism. Increase concurrency by using multiple threads, multiple clients, and multiple mount targets. • To minimize latency, clients and file systems should be in the same availability domain • For best performance, don’t set any mount options such as rsize or wsize when mounting the file system. The system automatically negotiates optimal window sizes
  27. 27. 27Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Object Storage Service
  28. 28. 28Copyright © 2018, Oracle and/or its affiliates. All rights reserved. High Availability for Object Storage • Oracle Object Storage was designed to be highly durable. Data is stored redundantly across multiple storage servers across multiple Availability Domains. • Data integrity is actively monitored using checksums and corrupt data is detected and automatically repaired. • Any loss in data redundancy is detected and remedied, without customer intervention or impact. ORACLE CLOUD INFRASTRUCTURE (REGION) Availability Domain 1 Availability Domain 2 Availability Domain 3 Storage Server Storage Server Storage Server The service is designed for 99.95% availability. Multiple safeguards have been built into the platform to monitor the health of the service to guard against unplanned downtime.
  29. 29. 29Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Amazon S3 Compatibility API Keys • S3 API compatibility implies that customers and partners can continue to use existing S3 tools (SDK clients, third party tools like Cloudberry) with OCI Object Storage services. • Partners will be able to qualify their application to work the OCI Object Storage services with minimal changes. • Not all of the available Amazon S3 APIs are supported. See the Amazon S3 Compatibility API documentation for a complete list of currently supported Amazon APIs.
  30. 30. 30Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Partner Backup Options
  31. 31. 31Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Data Backup and Migration Options
  32. 32. 32Copyright © 2018, Oracle and/or its affiliates. All rights reserved. OCI Storage Gateway • A frictionless on-ramp to OCI Object Storage. A virtual storage platform than looks and behaves like traditional NAS • Exposes object storage buckets as filesystem (NFS endpoint). Traditional applications can start using cloud storage right away • Supports file to object transparency - Breaks the metadata dependency. The format of the original source data in the cloud is maintained as-is. Once data has been uploaded to the cloud. You can use any supported Object Storage tooling to manage data • Near local NAS storage performance. A configurable read/write cache supports fast local shared storage like performance
  33. 33. 33Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Storage Gateway Service – How it works OCI Object or Archive Storage • Object-aware applications store and retrieve objects from Oracle Cloud Infrastructure Object Storage through filesystems that you create in Storage Gateway. Application ServersEnd Users Storage Gateway HTTPS NFSv4 • Storage Gateway is installed in an Oracle Cloud Infrastructure compute instance or as a Linux docker instance on one or more hosts in your on-premise data center.
  34. 34. 34Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Storage Gateway Service – Use cases Cloud Tiering • Use Storage Gateway to expand the capacity of on-premises storage solutions without capital expenditures. Backups • Use Storage Gateway to move files to Oracle Cloud Infrastructure Archive Storage as a cost-effective backup solution. You can move individual files and compressed or uncompressed ZIP or TAR archives. Storing secondary copies of data is an ideal use case for Storage Gateway. Archival • Storage Gateway is ideal for archive use cases. Disaster Recovery • Storage Gateway lets traditional applications move data to a highly durable object storage. When there is a need to recover data, a new instance of Storage Gateway is created and data can be easily recovered.
  35. 35. 35Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Data Transfer Service Data Transfer Disk • You send your data as files on encrypted commodity hard disk drives to an Oracle transfer site. • Operators at the Oracle transfer site upload the files into your designated Object Storage bucket in your tenancy. • This transfer solution requires you to source and purchase the disks used to transfer data to Oracle Cloud Infrastructure. The disks are shipped back to you after the data is successfully uploaded. Data Transfer Appliance • Oracle-supplied storage appliances to an Oracle transfer site. • Operators at the Oracle transfer site upload the data into your designated Object Storagebucket in your tenancy. • This solution supports data transfer when you are migrating a large volume of data and when using disks is not a practical alternative.
  36. 36. 36Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Interstate Highway Bandwidth >> Internet Bandwidth Approximate Data Upload Time Dataset Size 10 Mbps 100 Mbps 1 Gbps 10 Gbps Using DTS 10 TB 92 Days 9 Days 22 Hrs 2 Hrs ~1 week 100 TB 1,018 Days 101 Days 10 Days 24 Hrs ~1 week 500 TB 5,092 Days 509 Days 50 Days 5 Days ~ 1 week 1 PB 10,185 Days 1,018 Days 101 Days 10 Days ~ 2 weeks Sweet Spot for DTS
  37. 37. 37Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Summary • Describe and validate storage performance; both NVMe and Block Volumes • Use volume groups to manage snapshot and cloning activities for logical volumes spanning multiple block volumes • Describe File Storage Service Performance options • Utilize the S3 Compatibility API to enable interoperability with Amazon S3 • Understand Data Backup and Migration Options
  38. 38. 38Copyright © 2018, Oracle and/or its affiliates. All rights reserved. cloud.oracle.com/iaas cloud.oracle.com/tryit

×