Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Oracle Cloud Infrastructure – Storage

280 views

Published on

Prezentace z webináře, 3.9.2020
Prezentoval: Rostislav Gemrot, Arrow

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Oracle Cloud Infrastructure – Storage

  1. 1. OCI Storage Services L100 Rohit Rahi Oracle Cloud Infrastructure Oct 2019 prezentuje Rostislav Gemrot Arrow ECS, a.s.
  2. 2. OCI Storage Services Local NVMe Block Volume File Storage Object Storage Archive Storage Type NVMe SSDbased temporary storage NVMe SSDbased block storage NFSv3 compatible file system Highly durable Object storage Long-term archival and backup Durability Non-persistent; survives reboots Durable (multiple copies in anAD) Durable (multiple copies in anAD) Highly durable (multiple copies across ADs) Highly durable (multiple copies across ADs) Capacity Terabytes+ Petabytes+ Exabytes+ Petabytes+ Petabytes+ Unit Size 51.2TB for BM, 6.4-25.6 TB for VM 50 GB - 32 TB/vol. 32 vols/instance Up to 8 Exabyte 10 TB/object 10 TB/object Use cases Big Data, OLTP, high performance workloads Apps that require SAN like features (Oracle DB, VMW, Exchange) Apps that require shared file system (EBS, HPC) Unstructured data incl. logs, images, videos Long term archival and backups (Oracle DB backups)
  3. 3. Local NVMe Storage
  4. 4. Local NVMe SSD Devices Instance type NVMe SSDDevices BM.DenseIO2.52 8 drives = 51.2TBraw VM.DenseIO2.8 2 drive = 6.4 TBraw VM.DenseIO2.16 4 drives = 12.8 TBraw VM.DenseIO2.24 8 drives =25.6 TB raw • Some instance shapes in OCI include locally attached NVMe devices • Local NVMe SSD can be used for workloads that have high storage performance requirements • Locally attached SSDs are not protected and OCI provides no RAID, snapshots, backups capabilities for these devices • Customers are responsible for the durability of data on the local SSDs [opc@nvme ~]$ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT nvme0n1 259:0 0 5.8T 0 disk nvme1n1 259:3 0 5.8T 0 disk nvme2n1 259:1 0 5.8T 0 disk nvme3n1 259:2 0 5.8T 0 disk nvme4n1 259:5 0 5.8T 0 disk nvme5n1 259:6 0 5.8T 0 disk nvme6n1 259:4 0 5.8T 0 disk nvme7n1 259:7 0 5.8T 0 disk sda 8:0 0 46.6G 0 disk ├─sda2 8:2 0 8G 0 part [SWAP] ├─sda3 8:3 0 38.4G 0 part / └─sda1 8:1 0 200M 0 part /boot/efi
  5. 5. SLA for NVMe Performance Shape Minimum Supported IOPS VM.DenseIO1.4 200k VM.DenseIO1.8 250k VM.DenseIO1.16 400k BM.DenseIO1.36 2.5MM VM.DenseIO2.8 250k VM.DenseIO2.16 400k VM.DenseIO2.24 800k BM.DenseIO2.52 3.0MM • OCI provides a service-level agreement (SLA) for NVMe performance • Measured against 4k block sizes with 100% random write workload on Dense IO shapes where the drive is in a steady-state of operation • Run test on Oracle Linux shapes with 3rd party Benchmark Suites, https://github.com/cloudharmony/block- storage
  6. 6. Block Volume Intro
  7. 7. Block Volume Service • Block Volume Service let you store data on block volumes independently and beyond the lifespan of compute instances • Block volumes operates at the raw storage device level and manages data as a set of numbered, fixed- size blocks using a protocol suchas iSCSI • You can create, attach, connect, and move volumes, as needed, to meet your storage and application requirements • Typical Scenarios • Persistent and Durable Storage • Expand an Instance's Storage • Instance Scaling
  8. 8. Block Volume Service (contd.) Capacity Configurable: 50 GB to 32 TB(1GBincrements) Perf: disk type NVMe SSDbased Perf: IOPS 60 IOPS/GB - up to 25K IOPS* Perf: Throughput/Vol 480 KBPS/GB - up to 320MBPS** Perf: Latency (P95) Sub-millisecond latencies Perf: Per-instance Limits • 32 attachments/instance, up to 1 PB (32 TB/volume x 32 volumes/instance) • Up to 620K or more IOPS, near line rate throughout. Durability Multiple replicas across multiple storage servers within the AD Security Encrypted at rest andtransit * For Bare Metal or 8-core+ VM compute instance, using 4KB blocks. VM perf is limited by VM network bandwidth. ** 256 KB block size
  9. 9. Creating and Attaching a Block Volume Paravirtualization is a light virtualization technique where a VM utilizes hypervisor APIs to access remote storage directly as if it were a local device iSCSI block storage attachment utilizes the internal storage stack in the guest OS and network hardware virtualization to access block volumes. Hypervisor is not involved in the iSCSI attachment process By default, all Block Volumesare Read/Write Block Volume can also be read-only to prevent against accidental modification
  10. 10. Detaching and Deleting Block Volumes • When an instance no longer requires a block volume, you can disconnect and then detach it from the instance without any loss of data • When you attach the same volume to another instance or to the same instance, DO NOT FORMAT the disk volume. Otherwise, you will lose all the data on the volume • When the volume itself is no longer needed, you can delete the block volume • You cannot undo a delete operation. Any data on a volume will be permanently deleted once the volume is deleted
  11. 11. Block Volume Offline Resize The Oracle Cloud Infrastructure Block Volume service lets you expand the size of block volumes and boot volumes. You have three options to increase the size of your volumes: • Expand an existing volume in place with offline resizing (cannot resize an attachedvolume) • Restore from a volume backup to a larger volume. • Clone an existing volume to a new, larger volume. You can only increase the size of the volume, you cannot decrease the size
  12. 12. Backup and Restoration
  13. 13. Backup and Restoration • Complete point-in-time snapshot copy of your block volumes • Encrypted and stored in the Object Storage Service, and can be restored as new volumes to any Availability Domain within the same region (for multi-AD regions) • Can copy block volume backups from one-region to another ORACLE CLOUDINFRASTRUCTURE (REGION) Availability Domain 1 Availability Domain 2 Subnet A Server SubnetB Server Block Storage (Backup) Block Storage (Restore) Object Storage
  14. 14. Backup and Restoration • Backups are done using point-in-time snapshot; therefore, while the backup is being performed in the background asynchronously, your applications can continue to access your data without any interruption or performanceimpact • For a 2 TB volume being backed up for the first time, ~30 mins • For a 50 GB boot volume being backed up for the first time, ~ few mins On-demand, one-off block volume backups provide a choice of incremental versus full backup options Backup options: • On-demand, one-off: point-in-time snapshot • Automated policy-based: backups automatically on a schedule and retain them based on the selected backup policy. Three backuppolicies: • Bronze: monthly incremental backups, retained for twelve months (+full yearly backup, retained for 5years) • Silver: weekly incremental backups, retained for four weeks (+ Bronze) • Gold: daily incremental backups, retained for seven days (+Silver, + Bronze) • Customized
  15. 15. Clone and Volume Groups
  16. 16. Clone • Cloning allows copying an entire existing block volume to a new volume without needing to go through a backup andrestore process • Clone is a point-in-time direct disk-to-disk deep copy an of entire volume • The clone operation is immediate, but actual copying of data happens in the background and can take up to 15minutes for 1TBvolume • A clone can only be created in the same AD with no need of detaching the source volume before cloning it • Clones cannot be copied to another region • A clone can be attached and used as regular volume when its lifecycle state changes from ”PROVISIONING” to "AVAILABLE", usually withinseconds • Clone and backup operations are mutually exclusive • Number ofclones created simultaneously • If the source volume is attached: You can create one clone at a time • If the source volume is detached: You can create up to 10 clones from the same source volume simultaneously
  17. 17. Volume Groups • Group together block and boot volumes from multiple compartments across multiple compute instances in a volumegroup. • You can use volume groups to create volume group backups and clones that are point-in-time and crash-consistent. • Manually trigger a full or incremental backup of all the volumes in a volume group leveraging a coordinated snapshot across all thevolumes. • This is ideal for the protection and lifecycle management of enterprise applications, which typically require multiple volumesacross multiple compute instances to functioneffectively • Volume Group feature is available with no additional charge
  18. 18. Boot Volumes • A compute instance is launched using OS image stored on a remote boot volume • Boot volume is created automated and associated with an instance until you terminate the instance • Boot volumes are encrypted, have faster performance, lower launch times, and higher durability for BM and VM instances • Launch another instance with a boot volume • First create a custom image of your boot volume and then using the custom image launch the instance • Alternately, you can launch a new instance directly from an unattached boot volume if you don't wish to create a custom image • Delete boot volume • You can delete an unattached boot volume. • You can optionally chose to automatically delete the boot volume when terminating an instance by selecting the checkbox in the delete confirmation dialog. • OCI does not allow you to delete the boot volume currently attached to an instance. • Possible to take a manual backup, assign backup policy or create clone of boot volumes • You have the option of specifying a custom boot volume size
  19. 19. File Storage Service Intro
  20. 20. OCI Storage Services Local NVMe Block Volume File Storage Object Storage Archive Storage Type NVMe SSDbased temporary storage NVMe SSDbased block storage NFSv3 compatible file system Highly durable Object storage Long-term archival and backup Durability Non-persistent; survives reboots Durable (multiple copies in anAD) Durable (multiple copies in anAD) Highly durable (multiple copies across ADs) Highly durable (multiple copies across ADs) Capacity Terabytes+ Petabytes+ Exabytes+ Petabytes+ Petabytes+ Unit Size 51.2TB for BM, 6.4-25.6 TB for VM 50 GB to 32TB/vol 32 vols/instance Up to 8 Exabyte 10 TB/object 10 TB/object Use cases Big Data, OLTP, high performance workloads Apps that require SAN like features (Oracle DB, VMW, Exchange) Apps that require shared file system (EBS, HPC) Unstructured data incl. logs, images, videos Long term archival and backups (Oracle DB backups)
  21. 21. File Storage service Features • AD-local service, available in all OCI regions and Availability Domains • Supports NFS v.3 • Network Lock Management (NLM) for file locking • Full POSIXsemantics • Data Protection: Snapshots capabilities; 10,000 snapshots per file system • Security: 128-bit, data-at-rest encryption for all file systems & metadata • Console management, APIs, CLI, data-path commands, andTerraform • Create 100 file systems and 2 mount targets per AD per account
  22. 22. Mount Target Mount Target • NFS endpoint that lives in your subnet of choice;AD-specific • Mount target has an IP address and DNS name that you can use in your mount command. E.g.10.0.0.6 • Requires three private IP addressesin the subnet (don’t use /30 or smaller subnets for theFSS) • Two of the IP addresses are used during mount target creation;3rd IP used for HA OCIREGION AVAILABILITY DOMAIN-2AVAILABILITY DOMAIN-1 NFSclient 10.0.1.0/24 NFSclient 10.0.0.0/24 VCN, 10.0.0.0/16
  23. 23. Mount Target OCIREGION AVAILABILITY DOMAIN-2AVAILABILITY DOMAIN-1 10.0.0.0/24 10.0.1.0/24 NFSclient NFSclient VCN, 10.0.0.0/16 Mount Target • Placing NFS clients and mount target in the same subnet can result in IP conflicts, as users are not shown which private IPs are used for mount target • Place FSSmount target in its own subnet, where it can consume IPs as it needs 10.0.2.0/24
  24. 24. File System OCIREGION AVAILABILITY DOMAIN-2AVAILABILITY DOMAIN-1 NFSclient 10.0.1.0/24 NFSclient 10.0.0.0/24 VCN, 10.0.0.0/16 10.0.2.0/ 24 File System • Primary resources for storing files in FSS • To access your file systems, you create a new (or use an existing) mount target • 100 File Systems per Mount Target • AD-specific • Accessible from OCI VM/BM instances • Accessible from on-premises through FastConnect/VPN
  25. 25. FSS Paths • Export Path: unique path specified when the file system is associated with a mount target during creation • No two File systems associated with the same mount target can have overlapping export paths (e.g. FS paths like /example and /example/path are not allowed) Mount target (NFS endpoint): 10.0.0.6 Export Path1: /example1/path Export Path1 2: /example2/path • Export path, along with the mount target IP address, is used to mount the file system to an instance • sudo mount 10.0.0.6:/example1/path /mnt/mountpointA • sudo mount 10.0.0.6:/example2/path /mnt/mountpointB • /mnt/mountpointA and /mnt/mountpointB are path to the directory on the NFS client instance on which the external file systems are mounted
  26. 26. Mounting an OCI File System • Launch OCI instance from console • Use NFSv3 protocol to mountthe FSSvolume • Install nfs-utils (Oracle Linux and CentOS) or nfs-common (Ubuntu) in your Linux system • Create a directory • On the FSS console, click on Mount Targets • Use the Private IP address information to mount the volume using nfscommand: opc@node01:~$ sudo mkdir -p /<user’s target directory> opc@node01:~$ sudo mount <IPaddress>:<path-name> /<user’s target directory> opc@node01:~$ sudo yum install nfs-utils opc@node01:~$ sudo mkdir -p /mnt/nfs opc@node01:~$ sudo mount 10.0.0.3:/fss-shared /mnt/nfs NOTE: We recommend not to pass mount options to achieve best performance with File Storage Service. This approach leaves it to the client and server to negotiate the window size for Read & Write operations.
  27. 27. Security Security layer Uses these.. To control actions like these.. IAM Service OCI users, policies Creating instances (NFS clients) and FSSVCNs.Creating, listing, and associating file systems and mount targets Security Lists CIDR blocks Connecting the NFS client instance to the mount target Export Options Export options, CIDR blocks Applying access control per-file system based on source IP CIDR blocks that bridges the Security Lists layer and the NFS v.3 Unix Security layer NFS v3. Unix Security Unix users Mounting file systems1, reading the writing files, file access security 1When mounting file systems, don't use mount options such as nolock, rsize, or wsize. These options cause issues with performance and file locking Four distinct and separate layers of security with its own authorization entities and methods to consider when using FSS
  28. 28. File Storage Service Snapshot • Snapshots provide a read-only, space efficient, point-in-time backup of a file system • Snapshots are created under the root folder of file system, in a hidden directory named .snapshot • You can take up to 10,000 snapshots per file system • You can restore a file within the snapshot, or an entire snapshot using the cp or rsync command – cp -r .snapshot/snapshot_name/* destination_directory_name • If nothing has changed within the target file system and you take a snapshot, it does not consume any additional storage
  29. 29. Object Storage Intro
  30. 30. Local NVMe Block Volume File Storage Object Storage Archive Storage Type NVMe SSDbased temporary storage NVMe SSDbased block storage NFSv3 compatible file system Highly durable Object storage Long-term archival and backup Durability Non-persistent; survives reboots Durable (multiple copies in anAD) Durable (multiple copies in anAD) Highly durable (multiple copies across ADs)* Highly durable (multiple copies across ADs)* Capacity Terabytes+ Petabytes+ Exabytes+ Petabytes+ Petabytes+ Unit Size 51.2TB for BM, 6.4-25.6 TB for VM 50 GB to 32TB/vol 32 vols/instance Up to 8 Exabyte 10 TB/object 10 TB/object Use cases Big Data, OLTP, high performance workloads Apps that require SAN like features (Oracle DB, VMW, Exchange) Apps that require shared file system (EBS, HPC) Unstructured data incl. logs, images, videos Long term archival and backups (Oracle DB backups) OCI Storage Services
  31. 31. Object Storage service • An internet-scale, high-performance storageplatform • Ideal for storing unlimited amount of unstructured data (images, media files, logs, backups) • Data is managed as objects using an API built on standard HTTP verbs • Regional service, not tied to any specific compute instance • Offers two distinct storage classes to address the need for performant, frequently accessed "hot" storage, and less frequently accessed "cold" storage • Supports private access from Oracle Cloud Infrastructure resources in a VCN through a Service Gateway • Supports advanced features such as cross-region copy, pre-authenticated requests, lifecycle rules and multipart upload
  32. 32. Object Storage Resources Object • All data, regardless of content type, is managed as objects (e.g. logs, videos) • Each Object is composed of object itself and metadata of the object Bucket • A logical container for storing objects; Each object is stored in a bucket Namespace • A logical entity that serves as a top-level container for all buckets and objects • Each tenancy is provided one unique namespace that is global, spanning all compartments and regions • Bucket names must be unique within your tenancy, but can be repeated across tenancies • Within a namespace, buckets and objects exist in flat hierarchy, but you can simulate a directory structure using prefixes and hierarchies
  33. 33. Object Naming • Service prepends the Object Storage namespace string and bucket name to object name, /n/<object_storage_namespace>/b/<bucket>/o/<object_name> • https://objectstorage.us-phoenix- 1.oraclecloud.com/n/gse00014346/b/DatabaseBackup/o/database1.dbf • Flat hierarchy • For large number of objects, use prefixes and hierarchies, • /n/ansh8tvru7zp/b/event_photos/o/marathon/finish_line.jpg • /n/ansh8tvru7zp/b/event_photos/o/marathon/participants/p_21.jpg • You can use the CLI to perform bulk downloads and bulk deletes of all objects at a specified level of the hierarchy, without affecting objects in levels above or below • E.g. above, you can use CLI to download or delete all objects at the marathon/ level without downloading or deleting objects at the marathon/participants sublevel
  34. 34. Object Storage Tiers Standard StorageTier(Hot) • Fast, immediate, and frequent access • Object Storage Service always serves the most recent copy of the data when retrieved • Data retrieval is instantaneous • Standard buckets can’t be downgraded to archive storage Archive StorageTier (Cold) • Seldom or rarely accessed data but must be retained and preserved for long periods of time • Minimum retention requirement for Archive Storage is 90 days • Objects need to be restored before download • Archive Bucket can’t be upgraded to Standard storage tier • Time To First Byte (TTFB) after Archive Storage restore request is made: 4 Hours
  35. 35. 17 © 2019 Oracle Oracle Cloud always freetier: oracle.com/cloud/free/ OCItraining and certification: https://www.oracle.com/cloud/iaas/training/ https://www.oracle.com/cloud/iaas/training/certification.html education.oracle.com/oracle-certification-path/pFamily_647 OCIhands-on labs: ocitraining.qloudable.com/provider/oracle Oracle learning library videos onYouTube: youtube.com/user/OracleLearning
  36. 36. Thank you

×