This document discusses flash implications for enterprise storage array designs. It begins with an overview of common storage array functions and challenges posed by increasing workload randomization from virtualized servers (the "I/O blender" effect). It then examines how features like deduplication, snapshots, thin provisioning and data protection are limited by designs optimized for sequential hard drive I/O. Simply substituting flash does not unlock full potential due to unnecessary writes harming endurance and inability to fully optimize for random access. A clean-sheet flash-optimized design is needed to improve performance and fully leverage advanced features in all-flash arrays.
In a mixed-application workload running in a cost-conscious environment, such as our chargeback scenario, IT organizations must be able to meet performance SLAs and consolidate applications. Database VMs need plenty of storage capacity and performance to handle the increased workload demands users place on them. Measures such as purchasing new arrays to meet these demands can be costly.
Thanks to its data reduction technologies, the EMC XtremIO storage array 4.0 saved storage space while supporting additional development-level database VMs in a VMware vSphere 6.0 environment. XtremIO substantially saved capacity by leveraging inline compression, inline deduplication, and virtual copies. The addressable capacity in our largest test run was 7,213 GB but took only 1,565 GB of physical space on the all-flash array. At our most I/O-intensive and largest-scale performance level, our 13 workloads generated 207,927 IOPS at an average latency of .9 milliseconds. Although we focused on increasing IOPS, latency remained under one millisecond in all of our mixed-application workload tests.
Based on our findings, scaling workloads, saving storage capacity, and delivering speedy all-flash performance can improve the value of the array. In the small capacity footprint at the 13-database level, the cost per addressable GB shrunk by 73 percent. Had our tests been larger and used more XtremIO capacity, we could have potentially found greater reduction in terms of price per GB. We also calculated the cost per IOPS and saw a 43 percent reduction at the 13-database level from baseline.
Introduction to the EMC XtremIO All-Flash ArrayEMC
This white paper introduces the EMC XtremIO's storage array and provides detailed descriptions of the system architecture, theory of operation, and its features.
This technical paper provides the essential technical information about the advanced storage management solution for VMware virtual infrastructure using the VMware vSphere 5.0 Storage DRS feature with the IBM SONAS storage system. To know more about the VMware vSphere, visit http://ibm.co/Lx6hfc.
This White Paper provides an overview of EMC VFCache. It describes the implementation details of the product and provides performance, usage considerations, and major customer benefits when using VFCache.
Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage EMC
This white paper describes how a multi-tiered, multi-site backup solution can be deployed to protect data in a VMware vSphere environment using vSphere data protection APIs along with Symantec NetBackup 7.5 and EMC Isilon SyncIQ. This paper also outlines the setup, configuration, and functional testing of a disk-to-disk backup environment.
White Paper: EMC FAST Cache — A Detailed Review EMC
This white paper describes EMC's FAST Cache technology in EMC CLARiiON, Celerra unified, and VNX storage systems. It describes the implementation of the FAST Cache feature and how to use it with Unisphere Manager and CLI.
In a mixed-application workload running in a cost-conscious environment, such as our chargeback scenario, IT organizations must be able to meet performance SLAs and consolidate applications. Database VMs need plenty of storage capacity and performance to handle the increased workload demands users place on them. Measures such as purchasing new arrays to meet these demands can be costly.
Thanks to its data reduction technologies, the EMC XtremIO storage array 4.0 saved storage space while supporting additional development-level database VMs in a VMware vSphere 6.0 environment. XtremIO substantially saved capacity by leveraging inline compression, inline deduplication, and virtual copies. The addressable capacity in our largest test run was 7,213 GB but took only 1,565 GB of physical space on the all-flash array. At our most I/O-intensive and largest-scale performance level, our 13 workloads generated 207,927 IOPS at an average latency of .9 milliseconds. Although we focused on increasing IOPS, latency remained under one millisecond in all of our mixed-application workload tests.
Based on our findings, scaling workloads, saving storage capacity, and delivering speedy all-flash performance can improve the value of the array. In the small capacity footprint at the 13-database level, the cost per addressable GB shrunk by 73 percent. Had our tests been larger and used more XtremIO capacity, we could have potentially found greater reduction in terms of price per GB. We also calculated the cost per IOPS and saw a 43 percent reduction at the 13-database level from baseline.
Introduction to the EMC XtremIO All-Flash ArrayEMC
This white paper introduces the EMC XtremIO's storage array and provides detailed descriptions of the system architecture, theory of operation, and its features.
This technical paper provides the essential technical information about the advanced storage management solution for VMware virtual infrastructure using the VMware vSphere 5.0 Storage DRS feature with the IBM SONAS storage system. To know more about the VMware vSphere, visit http://ibm.co/Lx6hfc.
This White Paper provides an overview of EMC VFCache. It describes the implementation details of the product and provides performance, usage considerations, and major customer benefits when using VFCache.
Backup and Recovery Solution for VMware vSphere on EMC Isilon Storage EMC
This white paper describes how a multi-tiered, multi-site backup solution can be deployed to protect data in a VMware vSphere environment using vSphere data protection APIs along with Symantec NetBackup 7.5 and EMC Isilon SyncIQ. This paper also outlines the setup, configuration, and functional testing of a disk-to-disk backup environment.
White Paper: EMC FAST Cache — A Detailed Review EMC
This white paper describes EMC's FAST Cache technology in EMC CLARiiON, Celerra unified, and VNX storage systems. It describes the implementation of the FAST Cache feature and how to use it with Unisphere Manager and CLI.
This white paper describes EMC® FAST Cache technology in CLARiiON®, Celerra® unified, and VNXTM storage systems. It describes the implementation of the FAST Cache feature and how to use it with UnisphereTM Manager and CLI.
Boosting virtualization performance with Intel SSD DC Series P3600 NVMe SSDs ...Principled Technologies
When it comes time to make your server purchase or if you’re looking for an easy way to boost performance of existing infrastructure, consider upgrading your server’s internal storage. As our hands-on tests with a Dell EMC PowerEdge R630 environment running VMware Virtual SAN proved, Intel SSD DC P3600 Series NVMe SSDs could increase virtualized mixed-workload performance by as much as 59.9 percent compared to SATA SSDs while allowing you to run a large additional number of VMs. When you improve performance for your virtualized workloads, your employees and customers will benefit. By increasing performance with Intel NVMe SSDs on your Dell EMC PowerEdge R630 servers, you can potentially slash wait times and do more work on your servers without having to expand your infrastructure with additional storage arrays, which can translate to happier users and a more efficient infrastructure.
White paper: IBM FlashSystems in VMware EnvironmentsthinkASG
Drive performance in VMware environments with IBM FlashSystem. IBM flash storage delivers extreme, scalable performance for virtualized infrastructure.
White Paper - EMC IT's Oracle Backup and Recovery-4X Cheaper, 8X Faster, and ...EMC
Migrating from a legacy availability infrastructure for Backup and Recovery
creates challenges in terms of what are the best practices for a new Backup and
Recovery deployment with EMC‘s Oracle databases for Global Data Warehouse
and mission-critical Oracle applications. This white paper will illustrate the
transformation of EMC IT Oracle Backup and Recovery Infrastructure and
highlight how the Data Domain appliance transforms EMC IT Oracle Backup
infrastructure
Early adopters of Microsoft Windows Server 2012 will find this guide to be a useful resource. It provides an explanation of new features in Windows Server 2012 and details upgrade paths, installations options and directions, hardware requirements, supported drivers and more.
Backup and Recovery. The Capstone engineered solution is built on Unitrends, Inc. exceptional data protection appliance technology.
Unitrends offers data protection appliances that provide the lowest total cost of ownership (TCO) in the industry in terms of protecting and restoring critical data and systems. The Unitrends family of disk-to-disk (D2D) data protection appliances provides unmatched backup and rapid recovery of a server’s OS, applications, structured and unstructured data, as well as provide for disaster recovery protection using electronic vaulting.
This document is an introduction to Disk Storage technologies and its terminology. Within this
document basic disk and storage architectures as well as storage protocols and common fault
tolerance technologies will be discussed. It is not intended as a comprehensive guide for planning
and configuring storage infrastructures, nor as a storage training handbook.
Due to scope, this guide provides some device-specific information. For additional device- specific
configuration, Citrix suggests reviewing the storage vendor‘s documentation, the storage vendor‘s
hardware compatibility list, and contacting the vendor‘s technical support if necessary.
For design best practices and planning guidance, Citrix recommends reviewing the Storage Best
Practices and Planning Guide (http://support.citrix.com/article/CTX130632)
Accelerating Oracle on Red Hat Enterprise Linux with ioMemorySumeet Bansal
Oracle on RHEL is a great combination. The pot gets even sweeter with Fusion-io's iomemory gets added to the mix. Team Red Hat has done some excellent benchmarking to show that a single commodity server with RHEL and ioDrives can deliver mind-blowing throughput and IOPS. If you have a read-heavy Oracle workload on RHEL and can't use Oracle Smart Flash Cache, just use directCache from Fusion-io and get it done.
I am presenting this at the Red Hat mini-theatre at the Oracle Open World 2012.
EMC Isilon Best Practices for Hadoop Data StorageEMC
This white paper describes the best practices for setting up and managing the HDFS service on an Isilon cluster to optimize data storage for Hadoop analytics.
This white paper describes EMC® FAST Cache technology in CLARiiON®, Celerra® unified, and VNXTM storage systems. It describes the implementation of the FAST Cache feature and how to use it with UnisphereTM Manager and CLI.
Boosting virtualization performance with Intel SSD DC Series P3600 NVMe SSDs ...Principled Technologies
When it comes time to make your server purchase or if you’re looking for an easy way to boost performance of existing infrastructure, consider upgrading your server’s internal storage. As our hands-on tests with a Dell EMC PowerEdge R630 environment running VMware Virtual SAN proved, Intel SSD DC P3600 Series NVMe SSDs could increase virtualized mixed-workload performance by as much as 59.9 percent compared to SATA SSDs while allowing you to run a large additional number of VMs. When you improve performance for your virtualized workloads, your employees and customers will benefit. By increasing performance with Intel NVMe SSDs on your Dell EMC PowerEdge R630 servers, you can potentially slash wait times and do more work on your servers without having to expand your infrastructure with additional storage arrays, which can translate to happier users and a more efficient infrastructure.
White paper: IBM FlashSystems in VMware EnvironmentsthinkASG
Drive performance in VMware environments with IBM FlashSystem. IBM flash storage delivers extreme, scalable performance for virtualized infrastructure.
White Paper - EMC IT's Oracle Backup and Recovery-4X Cheaper, 8X Faster, and ...EMC
Migrating from a legacy availability infrastructure for Backup and Recovery
creates challenges in terms of what are the best practices for a new Backup and
Recovery deployment with EMC‘s Oracle databases for Global Data Warehouse
and mission-critical Oracle applications. This white paper will illustrate the
transformation of EMC IT Oracle Backup and Recovery Infrastructure and
highlight how the Data Domain appliance transforms EMC IT Oracle Backup
infrastructure
Early adopters of Microsoft Windows Server 2012 will find this guide to be a useful resource. It provides an explanation of new features in Windows Server 2012 and details upgrade paths, installations options and directions, hardware requirements, supported drivers and more.
Backup and Recovery. The Capstone engineered solution is built on Unitrends, Inc. exceptional data protection appliance technology.
Unitrends offers data protection appliances that provide the lowest total cost of ownership (TCO) in the industry in terms of protecting and restoring critical data and systems. The Unitrends family of disk-to-disk (D2D) data protection appliances provides unmatched backup and rapid recovery of a server’s OS, applications, structured and unstructured data, as well as provide for disaster recovery protection using electronic vaulting.
This document is an introduction to Disk Storage technologies and its terminology. Within this
document basic disk and storage architectures as well as storage protocols and common fault
tolerance technologies will be discussed. It is not intended as a comprehensive guide for planning
and configuring storage infrastructures, nor as a storage training handbook.
Due to scope, this guide provides some device-specific information. For additional device- specific
configuration, Citrix suggests reviewing the storage vendor‘s documentation, the storage vendor‘s
hardware compatibility list, and contacting the vendor‘s technical support if necessary.
For design best practices and planning guidance, Citrix recommends reviewing the Storage Best
Practices and Planning Guide (http://support.citrix.com/article/CTX130632)
Accelerating Oracle on Red Hat Enterprise Linux with ioMemorySumeet Bansal
Oracle on RHEL is a great combination. The pot gets even sweeter with Fusion-io's iomemory gets added to the mix. Team Red Hat has done some excellent benchmarking to show that a single commodity server with RHEL and ioDrives can deliver mind-blowing throughput and IOPS. If you have a read-heavy Oracle workload on RHEL and can't use Oracle Smart Flash Cache, just use directCache from Fusion-io and get it done.
I am presenting this at the Red Hat mini-theatre at the Oracle Open World 2012.
EMC Isilon Best Practices for Hadoop Data StorageEMC
This white paper describes the best practices for setting up and managing the HDFS service on an Isilon cluster to optimize data storage for Hadoop analytics.
This white paper examines the XDP implementation and discusses its benefits and advantages over RAID, with special consideration given to the unique requirements of enterprise flash storage arrays.
VMware vSphereTM 5.0 (“vSphere”) brings many new capabilities to extend the benefits of vSphere 4.1. These new features and enhancements to core capabilities in vSphere provide more performance optimization and easier provisioning, monitoring and troubleshooting. This paper focuses on the storage-specific features and enhancements that are available in vSphere 5.0 and provides an overview of how they optimize storage utilization, ease monitoring, and increase operational efficiency. Wherever possible, we will also provide use cases and requirements that might apply to these new functions.
In a globally dispersed enterprise with private cloud environment, where unstructured data is exponentially growing, there is a need to provide 24x7 accesses to business-critical data and be able to restore in case of loss of data. IBM Scale Out Network Attached Storage (IBM SONAS) with its integrated IBM Tivoli Storage Manager client enables enterprises to back up and restore data seamlessly and the IBM Active Cloud Engine offers the capability to replicate data to remote sites.
This paper provides an end-to-end solution for users using an application, such as Apple Final Cut Pro (a video editing s/w) and one of the currently supported protocols from SONAS, for example Network File System (NFS). To know more about the Network Attached Storage, visit http://ibm.co/QblYtC.
This white paper introduces EMC Fully Automated Storage Tiering for Virtual Pools (FAST VP) technology and describes its features and implementation. Details on how to use the product in Unisphere are discussed, and usage guidance and major customer benefits are also included.
WHITE PAPER▶ Software Defined Storage at the Speed of FlashSymantec
Solid State drives are changing the shape of the data center. Myths around lifespan and reliability are a thing of the past and as $/GB continues to decline, the value of SSDs is now unquestionable. While a few years ago a few GB was the norm, today’s Intel® Solid-State Drive Data Center P3700 Series drives possess capacities up to 2TB with incredible performance and resiliency. At the same time, servers are maximizing the number of drive bays. Combining denser, more powerful CPUs, and several terabytes of high performance storage is driving a shift away from SAN to software-defined infrastructures.
This white paper describes how to use Symantec software to consume and protect Intel storage, resulting in a small, fast, low-cost architecture ideal for running multiple high-performance Oracle databases.
The result is a pair of fast, flexible application 1U platforms that achieve better performance (up to 184%) than alternative architectures using All Flash Arrays at a fraction of the cost (5-10%)
With the football season in full swing, the baseball season heading into the playoffs, and the hockey season just starting, it is time to raid the refrigerator for snacks, head for the most comfortable chair in the family room, and settle in for a full day of viewing sports. Unfortunately, it is not always easy to turn on the myriad number of devices required to watch a game broadcast over cable, on that wide-screen hi-def TV, with the wrap-around sound from the latest audio system available. There is the re-mote for the cable system; there is a remote for the TV; there is one for the satellite dish; there is anoth-er for the sound system. There are so many remote controls on the coffee table that there is hardly room for the snacks! What you need is a universal remote; a single, simplified command center that can control all of the hi-tech equipment in the family room. Unfortunately, even that universal remote will not do the job for any device released after the remote was manufactured. What is required is a universal remote with a learning capability to take the complexity out of turning on the TV, one than can reprogram itself from the remote that comes with every new device.
This is a paper was written by David Reine, an IT analyst for The Clipper Group, and highlights IBM’s SAN Volume Controller new features, capabilities and benefits. These new capabilities were announced on October 20, 2009 If you have a heterogeneous storage architecture in your data center that is under-utilized and costing the enterprise on the bottom line, IBM SVC 5 may be the solution that you have
While apps may display several symptoms indicative of slow or erratic response after being virtualized, the problem boils down to contention for shared storage resources; contention
that did not occur when the apps had the storage all to themselves.
These so called “bottlenecks” occur in spurts as application requests collide randomly, resulting in spikes of sluggish, unpredictable latency. The more frequent, the greater the
users’ dissatisfaction. You may recall that one of the primary reasons these business critical apps were originally
sequestered on separate physical machines was to avoid such collisions
VMworld 2013
John Dodge, VMware
Andre Leibovici, VMware
Learn more about VMworld and register at http://www.vmworld.com/index.jspa?src=socmed-vmworld-slideshare
Why is Virtualization Creating Storage Sprawl? By Storage SwitzerlandINFINIDAT
Desktop and server virtualization have brought many benefits to the data center. These two initiatives have allowed IT to respond quickly to the needs of the organization while driving down IT costs, physical footprint requirements and energy demands. But there is one area of the data center that has actually increased in cost since virtualization started to make its way into production… storage. Because of virtualization, more data centers need "ash to meet the random I/O nature of the virtualized environment, which of course is more expensive, on a dollar per GB basis, than hard disk drives. The single biggest problem however is the signi!cant increase in the number of discrete storage systems that service the environment. This “storage sprawl” threatens the return on investment (ROI) of virtualization projects and makes storage more complex to manage.
Learn more at www.infinidat.com.
Similar to Flash Implications in Enterprise Storage Array Designs (20)
INDUSTRY-LEADING TECHNOLOGY FOR LONG TERM RETENTION OF BACKUPS IN THE CLOUDEMC
CloudBoost is a cloud-enabling solution from EMC
Facilitates secure, automatic, efficient data transfer to private and public clouds for Long-Term Retention (LTR) of backups. Seamlessly extends existing data protection solutions to elastic, resilient, scale-out cloud storage
Transforming Desktop Virtualization with Citrix XenDesktop and EMC XtremIOEMC
With EMC XtremIO all-flash array, improve
1) your competitive agility with real-time analytics & development
2) your infrastructure agility with elastic provisioning for performance & capacity
3) your TCO with 50% lower capex and opex and double the storage lifecycle.
• Citrix & EMC XtremIO: Better Together
• XtremIO Design Fundamentals for VDI
• Citrix XenDesktop & XtremIO
-- Image Management & Storage
-- Demonstrations
-- XtremIO XenDesktop Integration
EMC FORUM RESEARCH GLOBAL RESULTS - 10,451 RESPONSES ACROSS 33 COUNTRIES EMC
Explore findings from the EMC Forum IT Study and learn how cloud computing, social, mobile, and big data megatrends are shaping IT as a business driver globally.
Reference architecture with MIRANTIS OPENSTACK PLATFORM.The changes that are going on in IT with disruptions from technology, business and culture and so IT to solve the issues has to change from moving from traditional models to broker provider model.
Force Cyber Criminals to Shop Elsewhere
Learn the value of having an Identity Management and Governance solution and how retailers today are benefiting by strengthening their defenses and bolstering their Identity Management capabilities.
Container-based technology has experienced a recent revival and is becoming adopted at an explosive rate. For those that are new to the conversation, containers offer a way to virtualize an operating system. This virtualization isolates processes, providing limited visibility and resource utilization to each, such that the processes appear to be running on separate machines. In short, allowing more applications to run on a single machine. Here is a brief timeline of key moments in container history.
This white paper provides an overview of EMC's data protection solutions for the data lake - an active repository to manage varied and complex Big Data workloads
This infographic highlights key stats and messages from the analyst report from J.Gold Associates that addresses the growing economic impact of mobile cybercrime and fraud.
This white paper describes how an intelligence-driven governance, risk management, and compliance (GRC) model can create an efficient, collaborative enterprise GRC strategy across IT, Finance, Operations, and Legal areas.
The Trust Paradox: Access Management and Trust in an Insecure AgeEMC
This white paper discusses the results of a CIO UK survey on a“Trust Paradox,” defined as employees and business partners being both the weakest link in an organization’s security as well as trusted agents in achieving the company’s goals.
Let's dive deeper into the world of ODC! Ricardo Alves (OutSystems) will join us to tell all about the new Data Fabric. After that, Sezen de Bruijn (OutSystems) will get into the details on how to best design a sturdy architecture within ODC.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Flash Implications in Enterprise Storage Array Designs
1. FLASH IMPLICATIONS IN
ENTERPRISE STORAGE
ARRAY DESIGNS
ABSTRACT
This white paper examines some common practices in enterprise storage array
design and their resulting trade-offs and limitations. The goal of the paper is to
help the reader understand the opportunities XtremIO found to improve
in mind, and to illustrate why similar advancements simply cannot be achieved
enterprise storage systems by thinking with random, rather than sequential I/O
using a hard drive or hybrid SSD/hard drive array design.
3. TABLE OF CONTENTS
TABLE OF CONTENTS ............................................................................................................................. 3
EXECUTIVE SUMMARY ........................................................................................................................... 4
STORAGE ARRAY DESIGN PRIMER ......................................................................................................... 5
THE I/O BLENDER ................................................................................................................................. 6
STORAGE ENLIGHTENMENT ................................................................................................................... 7
DEDUPLICATION ................................................................................................................................... 7
SNAPSHOTS ........................................................................................................................................... 9
THIN PROVISIONING .......................................................................................................................... 10
DATA PROTECTION .............................................................................................................................. 11
THE INEVITABLE CONCLUSION............................................................................................................ 12
HOW TO LEARN MORE ......................................................................................................................... 12
3
FLASH IMPLICATIONS IN ENTERPRISE STORAGE ARRAY DESIGNS
4. EXECUTIVE SUMMARY
Enterprise storage arrays are highly sophisticated systems that have evolved over decades to provide levels of performance,
reliability, and functionality unavailable in the underlying hard disk drives upon which they’re based. A significant aspect of the
engineering work to make this possible is designing the array hardware and software stack to extract the most capability out of
hard drives, which are physical machines with mechanically moving parts and vastly different performance profiles for
sequentially (good performance) versus randomly (poor performance) accessed data. A good enterprise storage array attempts
to sequentialize the workload seen by the hard drives, regardless of the native data pattern at the host level.
In recent years, Solid State Drives (SSDs) based on flash media and not bound by mechanical limitations have come to market.
A single SSD is capable of random I/O that would require hundreds of hard drives to match. However, obtaining this
performance from an SSD, much less in an enterprise storage array filled with them, is challenging. Existing storage arrays
designed to sequentialize workloads are simply optimized along a dimension that is no longer relevant. Furthermore, since an
array based on flash media is capable of approximately two orders of magnitude greater performance than an array with a
comparable number of hard drives, array controller designs must be entirely rethought, or they simply become the bottleneck.
The challenge isn’t just limited to performance. Modern storage arrays offer a wide variety of features such as deduplication,
snapshots, clones, thin provisioning, and replication. These features are built on top of the underlying disk management engine,
and are based on the same rules and limitations favoring sequential I/O. Simply substituting flash for hard drives won’t break
these features, but neither does it enhance them.
XtremIO has developed a new class of enterprise data storage system based entirely on flash media. XtremIO’s approach was
not simply to substitute flash in an existing storage controller design or software stack, but rather to engineer an entirely new
array from the ground-up to unlock flash’s full performance potential and deliver array-based capabilities that are
counterintuitive when thought about in the context of current storage system features and limitations. This approach requires a
100% commitment to flash (or more accurately a storage media that can be accessed randomly without performance penalty)
and XtremIO’s products use flash SSDs exclusively.
This white paper examines some common practices in enterprise storage array design and their resulting trade-offs and
limitations. The goal of the paper is to help the reader understand the opportunities XtremIO found to improve enterprise
storage systems by thinking with random, rather than sequential I/O in mind, and to illustrate why similar advancements simply
cannot be achieved using a hard drive or hybrid SSD/hard drive array design.
4
FLASH IMPLICATIONS IN ENTERPRISE STORAGE ARRAY DESIGNS
5. STORAGE ARRAY DESIGN PRIMER
At their core, storage arrays perform a few basic functions:
•
Aggregate several physical disk drives for combined performance
•
Provide data protection in the event of a disk drive failure (RAID)
•
Present several physical disk drives as a single logical volume to host computers
•
Enable multiple hosts to share the array
Since physical disk drives perform substantially better when accessed sequentially, array designs hold incoming data in cache
until such time as enough data accumulates to enable a sequential write to disk.
When multiple hosts share the array the workload is inherently randomized since each host is acting independently of the others
and the array sees a mix of traffic from all hosts.
I/O RANDOMIZATION
Multiple hosts sharing storage
causes I/O randomization. Here,
three hosts, A, B, and C each have
logical volumes on the same
physical disk drives. As the array
fulfills I/O requests to each host,
the drive heads must seek to the
portion of the drive allocated to
each host, lowering performance.
Maintaining sequential I/O to the
drives is all but impossible in this
common scenario.
Storage arrays worked well enough for many years but have become increasingly challenged due to advancements in server
design and virtualization technology that create what has come to be known in the industry as the “I/O Blender”. In short, the
I/O Blender is an effect that causes high levels of workload randomization as seen by a storage array.
5
FLASH IMPLICATIONS IN ENTERPRISE STORAGE ARRAY DESIGNS
6. THE I/O BLENDER
CPUs have historically gained power through increases in transistor count and clock speed. More recently, a shift has been made
to multi-core CPUs and multi-threading. This, combined with server virtualization technology, allows massive consolidation of
applications onto a single physical server. The result is intensive randomization of the workload as seen by the storage array.
Imagine a dual socket server with six cores per socket and two threads per core. With virtualization technology this server can
easily present shared storage with a workload that intermixes twenty-four unique data streams (2 sockets x 6 cores per socket
x 2 threads per core). Now imagine numerous servers on a SAN sharing that same storage array. The array’s workload very
quickly becomes completely random I/O coming from hundreds or thousands of intermixed sources. This is the I/O Blender.
THE IO BLENDER
Multi-core CPUs, multithreading, and virtualization
all randomize the workload
seen by the storage array.
Three physical hosts no longer
present three unique
workloads, instead presenting
dozens.
Storage array designs, which depend on the ability to sequentialize I/O to disk, very quickly face performance challenges in
these increasingly common server environments. Common techniques to address this challenge are:
•
Increase array cache sizes: It is not uncommon for enterprise arrays to support 1TB or more of cache. With more cache
capacity there is a better chance to aggregate disparate I/O since more data can be held in cache, opportunistically waiting
for enough random data to accumulate to allow sequential writes. The downside to this approach is that cache memory is
very expensive and read performance is not addressed (random reads result in cache misses and random disk I/O).
•
Spread the workload across more spindles: If the workload becomes more randomized, then more disk IOPS are needed
to service it. Adding more spindles to the array yields more potential IOPS. The downsides to this approach are numerous –
cost, space, power consumption, inefficient capacity utilization (having much more capacity than the applications require,
simply because so many spindles are needed for performance), and the need to plan application deployments based on how
their I/O requirements will map to the array.
•
Add a flash cache or flash tier: This approach adds a higher performing media as a tier inside the array. There are
numerous benefits to this approach, however array controllers designed to work with hard drives are internally optimized,
both in hardware and software, to work with hard drives. Adding a flash cache or tier simply allows the performance
potential of the array controller to be reached with fewer drives, but it doesn’t improve the overall capabilites of the array.
Of note, when flash is added as a cache, it will experience write cycles more frequently (data is moved in and out of the cache as
it becomes “hot” or “cold”), necessitating expensive SLC flash that provides longer endurance.
The efficacy of caching and tiering can vary based on application requirements. Even if only 10% of I/Os go to spinning disk, the
application will often become limited by the response time for those transactions due to serialization. Maintaining consistently
high and predicatable performance is not always possible in caching/tiering designs.
6
FLASH IMPLICATIONS IN ENTERPRISE STORAGE ARRAY DESIGNS
7. Attempting to resolve the I/O Blender problem with these methods helps, but what happens next year when CPU core counts
double again? And again the year after that? Eventually the size of the array RAM cache and flash cache tiers need to become so
large that effectively all data (excepting purely archival information) will need to reside on them. With array controller designs
sized and optimized for the performance of hard drives, the full potential of flash will not be realized.
This fact alone is enough to warrant a ground-up new storage system design. But so far we have only discussed the drivers
around performance. Enterprise storage systems also have many advanced features for data management. Yet just as
performance can be enhanced by a ground-up flash approach, so may advanced array features.
STORAGE ENLIGHTENMENT
In this section we’ll examine four common array-based data management
features: deduplication, snapshots, thin provisioning and data protection. For
each feature we’ll examine how it works, how hard disk drives impact the
performance of the feature, and why a simple substitution of flash media for
hard drives doesn’t unlock the full potential of the feature. While we’ve chosen
four examples here for illustrative purposes, the implications are common
across nearly all array-based features including replication, cloning, and more.
DEDUPLICATION
Deduplication has predominately been deployed as a backup technology for two
reasons. First, it is clear that multiple backup jobs run over a period of days,
weeks, and months will deduplicate very well. But more importantly for this
discussion, deduplication has never been suitable in primary storage
applications because it has a negative impact on performance, always for
reading data, and in most implementations, on writing it.
Deduplication affects access performance on primary data because it leads to
logical fragmentation of the data volume. Whenever duplicate data is eliminated
and replaced with a pointer to a unique data block, the data stream for the
duplicate blocks is no longer sequentially laid out on disk.
7
FLASH IMPLICATIONS IN ENTERPRISE STORAGE ARRAY DESIGNS
FLASH MANAGEMENT
All types of flash media support a
limited number of write cycles.
And while SSD vendors are adept
at managing endurance within the
drive itself, much more can be
done at the array level to extend
the useful life of the drives. A
flash-based array must be
designed to:
• Minimize the number of writes to
flash by performing as much realtime data processing as possible
and avoiding post-processing
operations
• Spread writes across all SSDs in
the array to ensure even wear
• Avoid movement of data once it is
written
These guiding principles are
another reason why a clean-sheet
design is warranted with flash.
The processing operations and
data placement algorithms of
disk-based storage systems did
not need to consider flash
endurance.
8. DEDUPLICATION PERFORMANCE IMPACT
Once data is deduplicated, it is no longer possible to perform sequential reads as any deduplicated block requires a disk
seek to the location of the stored block. In this example, what would have required a single disk seek followed by a
sequential data read now requires 21 disk seeks to accomplish. With a disk seek taking roughly 15ms, a single read
has just increased from 15ms to 315ms, far outside acceptable performance limits.
But how does deduplication affect write performance? Most deduplication implementations are post-process operations.
Incoming data is first written to disk without being deduplicated. Then, the array (either on a schedule or by administrator
command) processes the data to deduplicate it. This processing necessitates reading the data and writing it back to disk in
deduplicated form. Every I/O operation consumed during the deduplication processing is unavailable for servicing host I/O.
Furthermore, the complex processing involved in deduplication is computationally intensive and slows the performance of the
array controller. This has the additional drawback of requiring a large amount of capacity to “land” the data while waiting for the
deduplication process to reduce it.
When flash is substituted for hard drives, performance improves as the random I/O deduplication creates is more favorably
handled by SSDs. And the extra operations of the deduplication post-processing could be completed more quickly. But the postprocessing operation itself requires multiple writes to disk – the initial landing of data on disk, and then the subsequent write for
the deduplicated data. This consumes IOPS that would otherwise be available for host I/O, but more importantly the extra writes
negatively impact flash endurance (see sidebar on previous page), which is an undesirable outcome. Other problems relating to
the computational burden deduplication places on the controller, the inability of many deduplication implementations to operate
globally across all array volumes or to scale across any array capacity are also not addressed by merely swapping from disk
drives to flash.
8
FLASH IMPLICATIONS IN ENTERPRISE STORAGE ARRAY DESIGNS
9. SNAPSHOTS
Snapshots are point-in-time images of a storage volume allowing the volume to be rolled back or replicated within the array.
While some snapshot implementations literally make full copies of the source volume (split mirror snapshots), we will focus our
discussion on space-efficient snapshots, which only write changes to the source volume. There are two types of space-efficient
snapshots; copy-on-write and redirect-on-write.
In a copy-on-write snapshot, changes to the source volume invoke a copy operation in the array. The original block is first
copied to a designated snapshot location, and then the new block is written in its place. Copy-on-write snapshots keep the
source volume layout intact, but “old” blocks in the snapshot volume are heavily fragmented.
COPY-ON-WRITE
SNAPSHOT
A change to an existing data
block (1) requires first that
the existing data block be
copied to the snapshot
volume (2). The new data
block then takes its place (3).
This results in two penalties. First, there are two array writes for every host write – the copy operation and the new write. This
severely impacts write performance. Secondly, reads from the snapshots become highly fragmented, causing disk I/O both from
the source volume and random locations in the snapshot reserve volume. This makes the snapshots unsuitable for any
performance oriented application (for example, as a test/development copy of a database volume).
The other type of space-efficient snapshot is redirect-on-write. In a redirect-on-write snapshot, once the snapshot is taken, new
writes are redirected to a snapshot reserve volume rather than copying the existing volume’s data. This avoids the extra write
penalty of copy-on-write designs.
REDIRECT-ON-WRITE
SNAPSHOT
In a redirect-on-write snapshot, new
writes (1) are redirected to the
snapshot reserve volume. This avoids
the extra write penalty of copy-onwrite snapshots, but results in
fragmentation of the active volume and
poor read performance.
However, now reads from the source volume must occur from two locations; the original source volume and the snapshot
reserve volume. This results in poor read performance as changes in the source volume accumulate and as more snapshots are
taken since the data is no longer stored contiguously and numerous disk seeks ensue.
As with deduplication, a simple substitution of flash in place of hard drives will yield some benefits, but the full potential
of flash cannot be achieved. In a copy-on-write design, there are still two writes for every new write (bad for flash endurance).
A redirect-on-write design can take better advantage of flash, but challenges come in other areas. For example, mixing
deduplication with snapshots is extremely complex from a metadata management perspective. Blocks of data may exist on disk,
or only as pointers. They may be in a source volume or in a snapshot. Figuring out which blocks need to be kept in the face of
changing data, deleting data, and creating, deleting, and cloning snapshots is non-trivial. In deduplicating storage arrays, the
performance impact when snapshots are utilized can be acute.
9
FLASH IMPLICATIONS IN ENTERPRISE STORAGE ARRAY DESIGNS
10. A final area of concern in snapshot technology is cloning. A clone is a copy of a volume creating using a snapshot, but that may
be subsequently altered and can diverge from the source volume. Clones are particularly challenging to implement with harddrive based storage because changes to the source volume and the clone must both be tracked and lead to fragmentation of
both volumes.
CLONES
When clones (writeable snapshots) are
created, the clone may diverge from the
source volume. This causes inherent
fragmentation as both the source volume and
clone now contain their own unique change
sets, neither of which is complete without the
source volume’s snapshot. Reading from the
source volume or the clone now requires
seeking to various locations on disk.
Cloning is an especially complex feature that often does not live up to performance expectations (and thus intended use cases)
in disk-based arrays. As previously noted, merely substituting flash media doesn’t solve the inherent way in which snapshots or
clones are implemented (e.g. copy-on-write or redirect-on-write) or the underlying metadata structures, and thus does not solve
the underlying problems.
THIN PROVISIONING
Thin provisioning allows volumes to be created of any arbitrary size without having to pre-allocate storage space to the volume.
The array dynamically allocates free space to the volume as data is written, improving storage utilization rates. Storage arrays
allocate free space in fairly large chunks, typically 1MB or more, in order to preserve the ability to write sequentially into the
chunks. However, most operating system and application I/O does not align perfectly to the storage system’s allocation
boundaries. The result is thin provisioning “creep”, where the storage system has allocated more space to the volume than the
OS or application thinks it has used. Some more advanced thin provisioning arrays have post-processing operations that
periodically re-pack data to reclaim the dead space, but this results in performance issues, both from the repacking operation
itself, and the resultant volume fragmentation and random I/O.
Substituting flash for hard drives provides no benefit because the underlying allocation size in the storage array hasn’t changed.
Allocation sizes are not simple parameters that can be tuned. They are deeply woven into the storage system design, data
structures, and layouts on disk.
Thin
Provisioning
Substituting flash for hard drives provides no benefit because the underlying allocation size in the
storage array hasn’t changed. Allocation sizes are not simple parameters that can be tuned. They
are deeply woven into the storage system design, data structures, and layouts on disk.
10 FLASH IMPLICATIONS IN ENTERPRISE STORAGE ARRAY DESIGNS
Problems
resulting from
large allocation
sizes – wasted
space (creep)
and volume
fragmentation
leading to
additional disk
seeks during
read operations.
11. DATA PROTECTION
The RAID algorithms developed for disk-based storage arrays protect data in the event of one or more disk drive failures. Each
algorithm has a trade-off between its protection level, its performance and its capacity overhead. Consider the following
comparison of typical RAID levels used in current storage systems:
RAID Algorithm
Relative Read
Relative Write
Relative Capacity
Performance
Performance
Overhead
Data Protection
RAID 1 – Mirroring
Excellent
Good
Poor
Single disk failure
RAID 5 – single parity
Good
OK
Good
Single disk failure
RAID 6 – double parity
Good
Poor
OK
Single disk failure
Regardless of which RAID level is chosen, there is no way to achieve the best performance, the lowest capacity overhead, and
the best data protection simultaneously. Much of the reason for this is rooted in the way these RAID algorithms place data on
disk and the need for data to be laid out sequentially.
Using standard RAID algorithms with flash media is possible, but it doesn’t change these trade-offs. There are clear gains to be
realized by rethinking data protection algorithms to leverage random-access media that would be all but impossible to replicate
using hard drives. In fact, a storage system designed for flash can provide nearly optimal read performance, write performance,
capacity overhead, and data protection simultaneously.
Another important consideration in the data protection design is how to handle flash “hiccups”, or periods where the flash drive
does not respond as expected due to internal operations. RAID schemes created for hard drives sometimes have “on-the-fly”
drive rebuilding and/or journaling to handle hard drive “hiccups”. However, these hiccup management techniques are
implemented within the constraints of the core RAID algorithm and would not function properly with flash media.
11 FLASH IMPLICATIONS IN ENTERPRISE STORAGE ARRAY DESIGNS