Enabling Storage Automation for Cloud Computing


Published on

This paper looks at the requirements of both sets of customers and the challenges that each faces. It then overlays the NetApp strategy as a storage supplier in serving both sets of customers by providing policy-based storage automation and thus enabling IT service automation.

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Enabling Storage Automation for Cloud Computing

  1. 1. WHITE P APER Enabling Storage Automation for Cloud Computing Sponsored by: NetApp Laura DuBois January 2012 EXECUTIVE SUMMARYwww.idc.com Today, most datacenters are trying to achieve operational excellence by providing the right service at the right time to the right application. However, the growth in storage capacity, datacenter infrastructure, and virtual machine sprawl, in concert with constrained IT budgets, is driving firms to consider policy-based automation as a means to gain operational excellence and improve service delivery.F.508.935.4015 Large enterprises and service providers seeking to build out either private or public cloud offerings view automation as a central tenet in executing on the vision for IT service automation. For this audience, service-level integration between management and orchestration frameworks and technology components (storage, server, network,P.508.872.8200 applications) is required. For the midsize enterprise, policy-based automation is a way to gain operational efficiencies by automating and simplifying complex, manual, and time-consuming processes for provisioning, backup, restoration, recovery, and cloning of physical or virtual application-specific assets.Global Headquarters: 5 Speen Street Framingham, MA 01701 USA This paper looks at the requirements of both sets of customers and the challenges that each faces. It then overlays the NetApp strategy as a storage supplier in serving both sets of customers by providing policy-based storage automation and thus enabling IT service automation. SITUATION OVERVIEW Todays Datacenter Trends Economic Hurdles with Data Growth and Storage Costs Firms are seeing, on average, a doubling of data storage requirements every year. To keep up with this reality, firms are making the storage budget an increasingly larger component of the overall IT budget. This challenge is leading to increased investment in storage optimization and efficiency technologies. Another core challenge is the management of growing storage infrastructure. On average, every incremental 500–700TB of storage installed requires an additional storage administrator to manage this capacity, according to IDC. This equates to spending an incremental $110,000 in operating costs (fully burdened storage administrator) to manage every incremental 600TB or $654,000 in storage cost. Storage capacity is increasing an average of 52% annually, but storage management is not scaling to match this growth. The last overarching economic storage challenge is low utilization in concert with infrastructure costs, specifically floor tile, power, and cooling requirements.
  2. 2. The average storage utilization rate remains low, varying between 30% and 50%depending on the application and representing a multibillion-dollar challenge inexcess storage capacity.Virtualization Brings Efficiency But Introduces Need for AutomationThe ongoing investment in server virtualization has curbed power, cooling, andphysical server costs but offered no material gains in reducing operating costs. Infact, virtualization has created a management challenge. As the ratio of virtual serversto physical servers scales and the growth rate for shipments of virtual serversdrastically exceeds that of physical servers, the management of virtual infrastructure(VI) has become the next big hurdle for datacenter managers. The managementprocesses, best practices, and integration of existing management tools are notmature enough to satisfy the long-term demands of a highly dynamic virtualinfrastructure. Existing frameworks, built upon the need to manage physicalinfrastructure, need to be maintained. Virtualization-specific management tools arebeing deployed but require integration with higher-level management or orchestrationframeworks. Lastly, the management across virtual infrastructure, servers, storage,and networks is still not integrated or automated. This integration is essential toachieving the dynamic, pooled resource environment that is the underpinning of ITservice management.Delivering the Right Service at the Right Time to the Right ApplicationToday, most datacenters managers are seeking operational excellence by providingthe right IT service at the right time to the right application. Midsize enterprises seekto automate manual IT processes and empower application or VI administrators toperform various tasks such as provisioning, snapshot creation, and restorationaccording to consistent, well-defined policies. These firms need to ensure thatservice-level objectives for availability, performance, and recovery are provided to theapplication, application owner, and business unit. Often, with an investment invirtualization, these same firms need to integrate these tools with the underlyingstorage infrastructure. This enables use of familiar virtualized management consolesto perform application- and/or virtual machine–specific storage tasks. However, thelargest enterprises and service providers seek to go several steps further, not only inautomating storage processes and integrating physical and virtual infrastructure butalso in coordinating the management of IT from higher-level orchestration frameworksto ultimately enable IT service automation. IT service automation forms the underlyingfoundation to effectively providing scale-out, dynamic cloud services.The Evolution to Private Clouds for Large Enterprises and CloudService ProvidersThe largest enterprises may consider public cloud services for specific applications,business units, and use cases but are less likely to move wholesale to public clouds.The largest firms, however, will evolve their datacenter infrastructure to deliver privatecloud services. A central tenet to the effective delivery of either public or private cloudservices is enabling IT service automation. To achieve this goal of IT serviceautomation, these customers must enable service-level integration betweenmanagement and orchestration frameworks and the underlying physical technologycomponents (storage, server, network, applications). This is necessary to provide2 #231836 ©2012 IDC
  3. 3. end-to-end provisioning of workload, server, hypervisor, network, and storage againsta service delivery objective. Once the service is provisioned, ongoing monitoring andmanagement of the infrastructure as well as service quality and levels must beconducted in an automated fashion. When out-of-policy conditions are detected, thereshould be automated workload and resource balancing or tuning to return theworkload or service to expected service levels. The goal of IT service automation is todynamically allocate workloads across public and private cloud virtual and physicalresources based on service-level objectives.THE ROLE OF IT SERVI CE AUTOMATION:WHAT CAN IT OFFER AN D TO WHOM?So, what is IT service automation? For the largest enterprises and service providers,it is part of a journey or foundation to enabling cloud services. For smaller firms,perhaps its a concept so broad and sweeping that they perceive it does not apply tothem. Service providers and large entities interested in offering public or private cloudservices recognize that automation — and, more specifically, IT service automation— is a foundational requirement to truly enabling cloud-based services.IT service automation is the integration of technology, process, and policy to allow abusiness or service provider to rapidly create and deploy on-demand IT services,according to established central IT policy, on dynamic, shared infrastructureresources, in either a public cloud or a private cloud. IT service management seeks toprovide end-to-end provisioning of resources (workload, server compute, hypervisor,network, and storage) against a set service delivery objective. The practical reality isthat myriad management tools reside in most datacenters to manage a given asset orgroup of similar assets. The challenge that both users and suppliers face is theintegration of these different management stacks across the technology domains.Policy-Based AutomationInherent within IT service automation and IT service management is the concept ofpolicy-based automation. Automation is the critical enabler to large-scale virtualizeddatacenter operations and cloud computing environments. Policy-based automationleverages an extensible policy definition to allow a firm to control exactly how ITworkloads and resources are matched. Policy-based automation includes constraint-,event-, and rule-based execution policies. Automation can be applied to productionenvironments for physical and virtual machine deployments, workload migrations,patching and distribution, change and configuration management, storageconfiguration, data protection, and security and compliance enforcement.The benefits of policy-based automation include: Reduce operational costs (do more with current staff) Reallocate IT resources Create more time for strategic planning Mitigate human error©2012 IDC #231836 3
  4. 4.  Reduce infrastructure costs Improve speed of application/system deploymentOrganizations that seek to move to a less manual process for management of their ITenvironment are increasingly considering policy-based automation as virtualizedinfrastructure becomes mainstream. Automation is a critical element in an overallstrategy to improve workload automation and make more efficient use of staff andcapital resources. But gaining buy-in for automation is not without challenges. In orderfor automation of any IT task to be accepted and proliferate, trust in the underlyingtechnology needs to be gained. Administrators typically gain this trust in thetechnology by monitoring the tasks that are automated and observing the results overtime. Another challenge organizations face in automating routine tasks is gaining thebenefits of offloading humans from mundane processes without losing accuracy in theexecution of the task. Again, this assurance that the automation provides adequatelevels of accuracy is something that a human will monitor and gain comfort with overtime. Lastly, automation of tasks requires organizational change and acceptance,probably the largest hurdle for a firm to overcome.The largest enterprise IT organizations and major service providers are alreadyleveraging policy-based automation as a means to execute on a broader vision for ITservice automation.ATTRIBUTES OF POLICY -BASEDAUTOMATIONPolicy-based automation can occur only when configurations, policies, workflows, anddeliverables are highly standardized for ongoing repetition. Policies must be welldefined and consistently applied. Key attributes of effective policy-based automationinclude: Policy granularity Breadth and depth of integration Ease of use Automation templates to enable consistency Application integration Service-level aware Virtualization integrated Secure4 #231836 ©2012 IDC
  5. 5. THE NETAPP AP PRO ACH TO POLICY-BASEDSTORAGE AUTO MATION AS AN ENABLER TOIT SERVICE AUTOMATIO NNetApps strategy for IT service automation is built upon the objective of helpingcustomers raise the efficiency of managing NetApp storage. NetApp is differentiatedas a pure-play storage provider not only in serving customer need by providing policy-based storage automation but also in taking on an open approach to enabling ITservice automation. NetApps policy-based storage automation in turn enables abroader IT service automation. In taking this approach, NetApp is able to serveseveral distinct customers sets and their requirements.Policy-Based Storage Automation for MidsizeEnterprise DatacentersServing NetApp customers without a previous investment in management ororchestration frameworks, NetApps policy-driven storage automation strategyincludes the automated management of NetApp storage to drive cost containment,storage efficiency and operational excellence. NetApps offerings include theSnapManager family of application-aware plug-ins and OnCommand UnifiedManager, which includes the capabilities of the former Provisioning Manager,Protection Manager, and Operations Manager products as well as OnCommandSystem Manager. A central component of NetApps policy-based storage automationapproach is exposing a storage service catalog where administrators can defineservice levels (Gold, Silver, Bronze storage service levels). These service levels canthen be applied to a new data set to automate storage provisioning while alsoapplying consistent policies for attributes such as thin provisioning, snapshots,deduplication, RAID, and storage type. Additionally, for firms making broad use ofvirtualization management consoles, NetApp integrates with virtualizationmanagement frameworks such as VMwares vCenter or Microsoft Virtual MachineManager (VMM) or vSphere with its Virtual Storage Console (VSC) and OnCommandPlug-in for Microsoft.NetApps approach to management and automation of storage is taken in the contextof specific IT roles. For example, NetApp storage administrators can automatemanual storage tasks by leveraging the protection and provisioning capabilities ofUnified Manager. Application administrators can automate storage tasks associatedwith their application assets by leveraging the application integration provided bySnapManager products. Virtual machine administrators can automate storage tasksassociated with VMDK or Virtual Disk assets by leveraging the VI integration providedby VSC and the OnCommand Plug-in for Microsoft. NetApp management softwareprovides integration between the application or virtual server and the storageinfrastructure to provide an administration experience in the context of the informationasset (e.g., a database, a VMDK, a SharePoint object, or an Exchange message).©2012 IDC #231836 5
  6. 6. Enabling IT Service Automation for Privateand Public CloudsFor the largest corporations and service providers en route to IT service automation,NetApps strategy is to provide integration between the management controls forNetApp storage with higher-level management and orchestration frameworks. NetAppmanagement functions are integrated with orchestration frameworks through theNetApp Manageability SDK, OnCommand Unified Manager, and orchestration-specific adaptors. This allows administrators using management and orchestrationframeworks to control, execute, and automate NetApp storage routines based onservices exposed through the NetApp storage service catalog. Thus, the IT servicemanagement administrator who manages the datacenter infrastructure in the contextof service levels can automate management of NetApp storage in that frameworkmodel, leveraging the NetApp Manageability SDK.The NetApp Manageability SDK provides all the functionality necessary to monitorand manage NetApp hardware, plus full integration with the OnCommand UnifiedManager. Using the NetApp Manageability SDK allows customers and serviceproviders to offer a technical service catalog for storage. Through the XML-RPC andSOAP APIs provided with the SDK, those seeking to offer IT as a service canleverage storage services, instrumented by the NetApp service catalog, for use withhigher-level orchestration frameworks. Service providers or storage subscribers canimplement a set of services that abstract lower-level tasks for data protection,provisioning, and multitenant configuration.As an example, defined storage services and associated configurations can bepushed to a management framework or custom application for self-service selectionof storage without involving a storage administrator. On the back end, the NetAppManageability SDK, in concert with the OnCommand Unified Manager, coordinatesthe creation of the data sets, volumes, and Qtrees. The resulting effect is that the roleof the storage administrator has changed from performing these tasks manually todefining and standardizing storage services that, when activated, call the appropriateNetApp technology to automate the tasks.CUSTOMER AND SUPPLIE R CHALLENGESPolicy-based storage automation is a reality today, providing real operational benefits,while true IT service automation remains a vision and longer-term objective for themost mature organizations. As an industry, we will continue to move closer and closerto automating tasks that have linkages between technology domains. However, thisroute presents challenges for end users and suppliers alike. These challengesinclude: Need for technology integration and linkages between policy logic and execution at the infrastructure level and across infrastructure domains. Multiple management stacks remain, including legacy frameworks and virtualization management tools. These disparate tools require onsite integration that customers need to plan and budget for.6 #231836 ©2012 IDC
  7. 7.  Moving firms through organizational transformation as IT roles change and/or converge because of increased reliance on technology to automate IT tasks. Policy setting and standardization, specifically catalog definitions to describe workloads, service objectives, and ongoing monitoring and measurement. This requires extensive planning, policy setting, and detailed ongoing communications between business units and various IT domains. High levels of customization and architectural planning required to achieve IT service automation.SUMMARYStorage is playing an increasingly important role in datacenter infrastructure. As largefirms seek to deliver IT as a service and smaller and midsize firms begin to considerpublic and private cloud services, the role of IT service automation and thereforestorage service automation is paramount. NetApps multipronged strategy to providepolicy-based automation storage offerings as well as integrate with higher-levelvirtualization, IT management, and orchestration frameworks will position it well asthe market continues to evolve. In the short run, management and automation ofstorage still need to occur and increasingly need to be done in the context of specificIT roles. This is where NetApp excels and is doing more to integrate the managementof storage with higher-level frameworks than other suppliers.Copyright NoticeExternal Publication of IDC Information and Data — Any IDC information that is to beused in advertising, press releases, or promotional materials requires prior writtenapproval from the appropriate IDC Vice President or Country Manager. A draft of theproposed document should accompany any such request. IDC reserves the right todeny approval of external usage for any reason.Copyright 2012 IDC. Reproduction without written permission is completely forbidden.©2012 IDC #231836 7