Challenges and Opportunities for the Storage Industry

230

Published on

In politics, one often hears: “Something must be done.” Invariably, the “something” refers to a significant, known set of problems and a paucity of answers, and it is simply apparent that things cannot go on as they are. In the world of SMBs and enterprise storage, the first part of the statement holds true. There are indeed abundant known challenges—data growth, consumerization, clouds, big data, and so on. And often, they come with even bigger expectations. The silver lining to the dark clouds threatening the storage ecosystem is that plenty of tools and methods either exist or are well understood and can control or remove much of the problem … if both vendors and users are willing to admit the issues and embrace change before it is too late.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
230
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Challenges and Opportunities for the Storage Industry"

  1. 1. Storage Systems Brief Challenges and Opportunities for the Storage Industry Date: May 2012 Author: Mark Peters, Senior Analyst Abstract: In politics, one often hears: “Something must be done.” Invariably, the “something” refers to a significant, known set of problems and a paucity of answers, and it is simply apparent that things cannot go on as they are. In the world of SMBs and enterprise storage, the first part of the statement holds true. There are indeed abundant known challenges—data growth, consumerization, clouds, big data, and so on. And often, they come with even bigger expectations. The silver lining to the dark clouds threatening the storage ecosystem is that plenty of tools and methods either exist or are well understood and can control or remove much of the problem … if both vendors and users are willing to admit the issues and embrace change before it is too late.Storage Challenges Are Mounting “Faced with the choice of changing one’s mind and proving that there is no need to do so, almost everybody gets busy on the proof.” —J.K. GalbraithThe challenges that managers of organizational data and storage face today are daunting. First, there is the constantlyincreasing need for additional resources to support growing data volumes stemming from natural application growthand new workloads. (Think of the impact of the web, social media, mobility, and big data.) In many cases, organizationstry to handle it by simply throwing more hardware at the problem, often ending up with massively underutilized assets.Next, operational processes often have not caught up with technology innovations, so as IT service delivery becomesmore agile (and cloud enters the fray, for instance), administrators struggle to manage with the same old, inflexibleprocesses.Budget constraints are almost always an issue, typically even more so in recent years. In addition, as virtual servers andcloud computing are improving provisioning and providing a higher level of service, users are beginning to expect“instant IT.” In the old days (five years ago!), if a user wanted to launch a new application to support a business process,he or she had an expectation that it would take a while—weeks or months—to get through the normal channels. Butwith fast, easy provisioning made possible by virtualization, IT can spin up a new virtual machine in minutes.However, having the correct, appropriate, available, and affordable storage infrastructure behind that VM andapplication can be a whole different story. As with any problem in life, the first two stages typically requireacknowledging the problem and establishing the desire to address it. Storage needs “something to be done” before anunsustainable model—the one largely deployed today—begins to cause real damage to IT and the business.Key Challenges • Data Growth—Data growth is probably the only absolute constant in IT, and arguably, it is the inevitable cause of most operating problems. Data growth never abates; it occurs whether the economy is up or down. One can have a delightful academic debate about whether the demand for capacity will always exceed and drive the technologies (in terms of both capacity and performance), or whether a declining raw price drives latent demands that become economically viable at the new cost level. Frankly, it does not matter. When you’re at the top of a roller coaster, regardless of whether you were pulled or pushed up the incline, massive acceleration and g-forces are coming. The whole ecosystem is on the ride. We now live in a zettabyte world, and it strains systems, people, opex, and capex. • New IT Architectures—Virtualization drives and stresses storage capacity and performance needs equally— and a poor storage infrastructure can be an anchor on the success of server virtualization and VDI projects. At the same time, the use of cloud models is increasing. (ESG research pins cloud expenditures in 2012 at 7% © 2012, The Enterprise Strategy Group, Inc. All Rights Reserved.
  2. 2. Storage Systems Brief: Challenges and Opportunities for the Storage Industry 2 of all IT spending.) While the attraction of both virtualization and cloud is beguiling, users all too often find that storage and its overall management can cause a type of “Ponzi” complexity behind the scenes. Very often, specific storage exists for a specific application, and huge operational issues arise in making data available to others within or outside an organization. (Imagine if you had to change laptops to go from Word to PowerPoint!) And yet, that is how storage is often deployed. • Uncertainty—With much of the world’s new data produced outside of data centers—unstructured and rich media being the new norm—the ability to predict what storage is needed within those data centers is crucial but increasingly difficult. On the other hand, it is certain that users have high expectations in terms of speed, availability, and immediacy … irrespective of geography and time zones. • Operational Struggles—The lack of flexibility in, and between, many storage systems simply makes completing the jigsaw puzzle harder. Everything must still be protected and backed up … whether it’s a scale-up or scale-out environment, distributed or converged. Many users have resorted (perhaps a better choice of words would be “are resigned”) to overprovisioning and underutilization just to get the job done, even knowing that such an approach equates to throwing money down the drain.The Crux of the ProblemDisk technology is 56 years old this year, and tape is 60. The underlying essence of what we’re using for storage hasn’tchanged. There’s an old joke about giving someone directions that ends with, “But if I were you, I wouldn’t start fromhere!” But is that so with storage? Are we really just rearranging the deckchairs on the Titanic, or is there room foroptimism? The truth is that storage is complex and tough. And getting the job done effectively has for years oftentrumped getting it done efficiently.But two things have changed lately: The economic downturn has put a laser focus on IT efficiency in general and storagein particular, and the lockstep between new storage technologies and new demands for capacity and performance hasbeen well and truly broken. Without change and innovation, storage costs will escalate to an unacceptable percentageof IT budgets, probably without providing acceptable service levels either. As the saying goes, “Something has tochange.”The good news is that there are plenty of reasons to be optimistic. Before examining them, here’s a reminder of why weare where we are.How We Got Here, and Where We Will Go NextHow We Got HereCommercial computing took hold when one infrastructure stack executed one specific application for one specificpurpose. The original mainframe was a glorified calculator. Centralized computing was predictable and controllable,albeit expensive. It was manageable: one processor system and one I/O subsystem.Decentralized (or distributed) computing was developed largely to try to solve the economic (essentially capex)challenges of centralized computing. It yielded low-cost, commodity servers—which we promptly plugged intoproprietary, large, expensive, monolithic storage boxes. Servers became cheaper and more interoperable, while storageremained proprietary and expensive. In the old days, the server was the thing that cost all the money. You picked yourserver by your OS. You picked your OS by your application. Storage was a “peripheral.”Today, servers are cheap and interoperable, while storage is still expensive, complex, incompatible, and difficult. Inmany respects, it is the last bastion of IT awkwardness: the peripheral tail wagging the purposeful dog!Where to Next?Let’s take for granted that we want to virtualize IT in general because we can gain efficiencies in asset utilization, takeadvantage of the commoditization of hardware, leverage common infrastructures, provide seamless mobility options,etc. © 2012, The Enterprise Strategy Group, Inc. All Rights Reserved.
  3. 3. Storage Systems Brief: Challenges and Opportunities for the Storage Industry 3If we do this, then we need to provide an infrastructure that: • Self-tunes: Storage that manages/optimizes/reconfigures itself for the workloads that are presented • Self-heals: Infrastructure that handles faults autonomously, so that applications are not affected • Scales dynamically: Devices that can extend—virtually—to whatever requirements the workload(s) presents • Self-manages: Adapts to changing scenarios based on policy and enforces those policies via automationThis set of values is nirvana when it comes to IT and storage infrastructures today, but it is exactly what we are movingtoward. In so doing, we will address the levels of manual intervention, inflexibility, and wasted resources that bedeviltoday’s storage world. If we do not adopt these changes, then the storage scenario is simply unsustainable.In layman’s terms, what is needed is storage that has more automation, flexibility, application and business linkages,resource utilization, and management /tuning ease.There is, however, one thing we consistently need storage to have less of. Cost!The Importance of EconomicsNothing in storage makes sense if it doesn’t make economic sense. We have storage tiers because storage isn’t free. Andusing them efficiently has been an aim (that we’re finally beginning to deliver against) for decades. Different types ofstorage exist only because of cost differentials; after all, if storage were free (or all the same price, at least), we wouldnaturally put everything on the fastest devices possible. Only because storage has a price—indeed, a range of prices—dowe think about where to put different data. We often disguise that as a conversation about performance, but that’s asymptom rather than a cause. The cause is money. Figure 1 shows very simply, yet starkly, how we arrived where weare. It also shows why a need exists for more complete tiering at all levels of the data hierarchy. Figure 1. The Evolution of the Storage Hierarchy Source: Enterprise Strategy Group, 2012. © 2012, The Enterprise Strategy Group, Inc. All Rights Reserved.
  4. 4. Storage Systems Brief: Challenges and Opportunities for the Storage Industry 4The first picture displayed in Figure 1 portrays the “perfect story” spun for decades; however, it has never properlyexisted and would actually be better represented by the wedding-cake diagram shown in the second picture, in whichthe “steps” from one storage tier to another are large. In the real world, such moves (such as that from disk to tape)were for decades also awkward, largely manual, and hence, to be avoided as much as possible. As shown in Picture 3,the moves were pretty much happening in one direction, creating what could almost be called a storage “lower-archy!”Over the years, we have added more gradations for active data (Picture 4) in terms of differing types of disk drive(speed, capacity, caching, etc.), but the top and bottom layers of the data “cake”—solid-state disk and tape—have beenessentially left alone. What the industry needs is to extend additional layers (see Picture 5) in these places, too.What you end up with is a gradual smoothing of the layer cake from top to bottom so that it begins to look (as Picture 6shows) rather like the original vision we had decades ago. In some instances, it will be abundantly clear (for applicationsknown to have extreme I/O performance or latency-sensitive needs) that certain data should reside permanently on anultra-performance tier. Another option is to employ some form of tiering software that moves the most deserving ordemanding data at any given time to the highest available storage tier. Both options have their place. The former is veryspecific and absolutely guarantees the best service for certain data. The latter is a more generic approach that generatesless performance improvement across a wider amount of data. The final element is shown in Picture 7, in whichtechnologies such as deduplication, thin provisioning, and solid-state/caching shrink the pyramid for a given workload.Now that we’ve seen where we have come from in storage, and what’s needed to avoid going over the precipice ofinefficiency and financial disaster, should we throw our hands up in defeat? No. And users don’t have to wait forartificial-DNA-based storage or the next storage-class-memory technology to appear. Future technologies hold muchpromise, but the storage challenge is real today, and a number of technologies are available or emerging that hold thepromise to alleviate the issues.Areas of Immediate OpportunityWhile nothing alone is a panacea, there is reason for optimism among users who will embrace some change … andamong vendors that have the wherewithal to develop and improve the technologies that can make storage a useful—hopefully almost an invisible—tool, rather than a pain-in-the-IT-neck. Some of the main opportunities are: 1. Embracing an automated storage hierarchy: While any mention of terms such as “information lifecycle management” must be made in hushed tones (because its marketing preceded a real delivery capability years ago), the value of using tiers and caching for appropriate data placement is irrefutable—both in terms of logic and finances. 2. Storing less data: This means not only a backend reduction (although this is still far from endemic), but also using newer technology on the front end of storage (in-line compression and deduplication, for instance) to prevent unnecessary blocks of data ever being written anywhere in a storage system. 3. Store the data you have more efficiently: This step includes using straightforward tools such as thin provisioning (which can apply to prime and replicated data), that are still deployed by a surprisingly small group of users (around 50%). Unified storage platforms can also reduce the overall capacities needed by establishing pools of storage for better utilization and more flexibility. Additionally, the use of sophisticated management tools allows for efficient deletion of unnecessary data. 4. Storage virtualization: This step can occur within devices (enabling thin provisioning, snapshots, etc.) and across (heterogeneous) platforms. The latter approach can be termed a “storage hypervisor,” and it is another method enabling users to deploy less actual storage to store the same amount of data—mainly via better utilization. 5. Automation and analytics: Having systems that are “application aware” and can implement policies is another emerging route to better storage efficiency. It is entirely possible that the “storage administrator” function as we know it will not exist in five to ten years. Storage will be given specific requirements by the overall system administrator, and it will then—automatically, autonomically—set performance, protection, tiering, and recovery policies. Even “set and forget” will be surpassed by “get and forget.” © 2012, The Enterprise Strategy Group, Inc. All Rights Reserved.
  5. 5. Storage Systems Brief: Challenges and Opportunities for the Storage Industry 5The Bigger TruthThe storage world is dealing with a “burning platform.” The fuel for the fire is the unrelenting growth in demand fordata, and the flames are fanned by the complexity and inefficiency that is endemic in many storage infrastructuresdeployed today.There has been massive consolidation on the vendor side (with well over $10 billion spent on acquisitions in the last twoto three years) as big vendors seek to have the right tools in their bag … which only goes to show that many of thenecessary tools are available already to at least ameliorate—and possibly extinguish—the flames. The situation justneeds users to be more aggressive in using what’s available, and vendors to invest ever-more in the orchestration andautomated management of advanced storage functionality … so that storage can become a valuable, unseen foundationrather than an expensive frustration.If neither of those changes happen, then we are headed for a catastrophe. If only one or the other happens, then we willhave at least bought our industry some time. If both happen, then we should be able to return to a balanced storagesystem.While the list of requirements for the “good outcome” to be achieved seems onerous (make storage less expensive;easier; more self-managing; more scalable; and more sharable within and across applications, people, andorganizations), it is all eminently achievable. The value for users, and therefore the opportunity for vendors, is so greatthat it will—hopefully—ensure these advances occur. But the time for both users and vendors to act is now. All trademark names are property of their respective companies. Information contained in this publication has been obtained by sources The Enterprise Strategy Group (ESG) considers to be reliable but is not warranted by ESG. This publication may contain opinions of ESG, which are subject to change from time to time. This publication is copyrighted by The Enterprise Strategy Group, Inc. Any reproduction or redistribution of this publication, in whole or in part, whether in hard-copy format, electronically, or otherwise to persons not authorized to receive it, without the express consent of The Enterprise Strategy Group, Inc., is in violation of U.S. copyright law and will be subject to an action for civil damages and, if applicable, criminal prosecution. Should you have any questions, please contact ESG Client Relations at 508.482.0188. © 2012, The Enterprise Strategy Group, Inc. All Rights Reserved.

×