Gartner evaluation criteria_for_clou_security_networking

G00259331
Evaluation Criteria for Cloud Infrastructure as a
Service
Published: 7 May 2014
Analyst(s): Kyle Hilgendorf
This document provides a comprehensive set of criteria for assessing
infrastructure as a service cloud providers such as Amazon Web Services,
Microsoft Azure Infrastructure Services, Rackspace Public Cloud, VMware
vCloud Hybrid Service, Google Compute Engine or IBM SoftLayer Cloud.
Table of Contents
Evaluation Criteria...................................................................................................................................2
Baseline Criteria................................................................................................................................3
Compute.......................................................................................................................................... 3
Required.....................................................................................................................................3
Preferred.................................................................................................................................... 6
Optional......................................................................................................................................7
Storage............................................................................................................................................ 7
Required.....................................................................................................................................8
Preferred.................................................................................................................................... 9
Optional....................................................................................................................................10
Network......................................................................................................................................... 11
Required...................................................................................................................................11
Preferred.................................................................................................................................. 13
Optional....................................................................................................................................14
Security and Access.......................................................................................................................15
Required...................................................................................................................................15
Preferred.................................................................................................................................. 18
Optional....................................................................................................................................20
Service Offerings............................................................................................................................ 21
Required...................................................................................................................................22
Preferred.................................................................................................................................. 23

Optional....................................................................................................................................24
Support and Service Levels............................................................................................................ 24
Required...................................................................................................................................24
Preferred.................................................................................................................................. 26
Optional....................................................................................................................................27
Management and DevOps..............................................................................................................28
Required...................................................................................................................................28
Preferred.................................................................................................................................. 30
Optional....................................................................................................................................31
Price and Billing..............................................................................................................................32
Required...................................................................................................................................32
Preferred.................................................................................................................................. 33
Optional....................................................................................................................................33
Using the Criteria Toolkit.......................................................................................................................34
Gartner Recommended Reading..........................................................................................................34
Notes................................................................................................................................................... 34
Evaluation Criteria
There are numerous choices for cloud infrastructure as a service (IaaS) providers today, and
choosing the right service for an organization's technical and business needs is critical. However,
some industry experts argue that cloud IaaS is a commodity and that price is the only significant
consideration. Gartner strongly opposes this belief and finds that service features and
configurations differ greatly, even among the leading providers in the market. For example, different
providers offer many kinds of security and management configurations and gravitate toward
compatibility with certain on-premises software platforms for hybrid cloud enablement — which
may drive customers into the arms of different providers. Service parity should never be assumed,
and it will therefore remain crucial for end-user organizations to identify their critical requirements
and map those to the capabilities of prospective IaaS providers.
This document presents comprehensive evaluation criteria to use when making cloud IaaS provider
selections and architecture decisions. Gartner developed this evaluation framework to address
current and future needs of its customers, categorizing cloud IaaS service features as:
■ Required: Essential/must-have features needed to develop, deploy and manage a broad range
of use cases, including production applications at cloud IaaS providers
■ Preferred: Supplementary features not necessary to satisfy the minimum requirements of the
typical large enterprise, but frequently desired to address specific needs such as larger scales,
better management and improved availability
Page 2 of 35 Gartner, Inc. | G00259331

■ Optional: Requirements-driven features necessary for specific deployment scenarios, but not
needed in all deployments
Gartner considers satisfaction of 100% of the required features as indicative of mature cloud IaaS
services that are ready for deployment among the largest and most critical of workload portfolios,
including both existing applications with legacy architectures and new cloud-native applications.
Although enterprise organizations can select and use providers that do not meet 100% of the
required criteria, the organization must be fully aware of the trade-offs associated with deploying a
solution that is missing features or that has service gaps. Gartner recommends evaluating each
potential workload for fit at the provider to determine which scenarios will not be suitable based on
the missing provider features.
Cloud IaaS providers can be evaluated by using the accompanying Excel spreadsheet
"Cloud_IaaS_Criteria." The Using the Criteria Toolkit section of this document provides instructions
on working with and modifying the spreadsheet.
Baseline Criteria
The following criteria are baseline capabilities, assumed to be part of every cloud IaaS provider in
the market. This list can be considered "table stakes" or minimum requirements that a provider
must meet to even be considered a cloud IaaS provider. This list may be helpful in delineating
whether a provider is a managed hosting provider or a cloud IaaS provider:
■ On-demand, self-service provisioning of compute instances and storage volumes
■ Remote access connectivity into compute instances (e.g., Secure Shell [SSH], Remote Desktop
Protocol [RDP])
■ Internet accessibility for all services
■ Publicly accessible and downloadable terms of service
■ Pay-per-use models for services (e.g., per hour, per GB/month)
Compute
The compute portion of an IaaS offering is often the most requested and deployed service.
Providers most commonly use server virtualization to provide virtual machines (VMs) as a service.
However, because some providers use bare-metal provisioning or containerization software, the
compute node will be referred to as "instance" in this document. Because most IaaS offerings are
based on x86 server virtualization and hypervisors, note that this evaluation criteria will deliberately
not overlap with "Evaluation Criteria for Server Virtualization Infrastructure Platforms."
Required
■ Rapid, self-service provisioning: The cloud service must offer self-service provisioning of
instances either through a programmatic interface (for example, an API and command line
interface [CLI]) or through a management console. Provisioning must be simultaneous, not
Gartner, Inc. | G00259331 Page 3 of 35

sequential — the provider's self-service interfaces and control plane must be able to provision a
large number of instances for multiple customers at the same time, without dependencies
between those provisioning jobs. Finally, the provisioning capabilities must be rapid, meaning
the time between the provisioning action and the point where the customer can log in to the
server must match all of the following:
■ A single Linux instance (1 virtual CPU [vCPU], ~4GB RAM) in less than 5 minutes
■ A single Windows server instance (1vCPU, ~4GB RAM) in less than 10 minutes
■ 20 Linux instances (each with 1vCPU, ~4GB RAM) in less than 15 minutes (in total)
■ Image customization: As a self-service capability, a customer can provision an image onto a
compute instance, customize that instance (altering OS files, installing additional software, etc.),
and save it as a new, privately available image that can then be used to provision other
instances in the future. This new image must either be saved into the image catalog or
otherwise be persisted beyond the lifetime of the instance itself.
■ Bring your own image/instance import: As a selfservice capability (requiring no intervention
from the service provider to customimport an image), a customer with an existing image in a
supported format, can import that image and save it as a new, privately available image that
can then be used to provision instances in the future. The provider must be able to support
Open Virtualization Format (OVF)-, Virtual Machine Disk Format (VMDK)- or virtual hard disk
(VHD)-based images.
■ Two-generation OS support: The provider must offer, in its catalog of OS images, the current
and second-most-recent long-term supported versions for at least one enterprise Linux
distribution (such as Red Hat or SUSE), one commonly used free Linux distribution (such as
Ubuntu, Debian, or CentOS) and the two latest major Windows Server releases (e.g., Windows
Server 2008 and Windows Server 2012).
■ Large instance support: Providers must offer customers instances with a large number of
processor cores and memory for processor- or memory-intensive use cases. The provider must
be able to provide instances that support at least eight vCPUs or Cores and 64GB of RAM.
■ No compute starvation or resource prioritization across tenants: Providers must not
employ any amount of resource starvation or prioritization across tenants unless the customer
clearly subscribes to such a variable-performing tier of service. In the standard service, it is not
acceptable to impose resource starvation of customer instances in order to rebalance or
improve the performance of another tenant.
■ Hot-swappable virtual hardware: Providers must support the customer ability to "hot swap"
(that is, resize, add or remove) core virtual resources — processors, memory, disk and network
configurations — on the virtual instance. Providers must not force customers to reprovision a
new instance with new characteristics. Providers can offer this functionality in a granular
manner, with controls for each core resource, or through prebundled "sizes" of instances in the
service catalog. Customers should note that some guest OS families will require a guest reboot.
■ Instance maintenance mitigation: The cloud service must be architected in such a way to
avoid instance outages or downtime when the provider is performing any kind of hardware or

service maintenance. The provider must support this through one of the two following instance
maintenance mitigations or another technique that clearly avoids instance outages:
■ Live migration: A running instance may be moved from one physical host to another
physical host without downtime or reboot.
■ Memory preserving maintenance: During planned maintenance, the provider may
suspend actively running instances for no more than 60 seconds in order to accomplish
hardware or host activities. The instance must not lose state after recovering from
suspension.
■ Instance failure recovery: The cloud service must be architected in such a way to
automatically restart instances on a healthy host if the original physical host fails. This is often
referred to as "instance restart" or "VM restart." It is acceptable for an automatic attempt to be
made to reboot the original physical host and restart instances on it. If that fails, the service
must support restarting the instances on a healthy host. The failure detection must occur within
one minute, and if the host cannot be immediately recovered, the automatic instance restart
process on another host must begin within 5 minutes.
■ Instance maintenance/failure notifications: If a compute resilience event occurs (such as live
migration, instance restart or memory preserving maintenance), the cloud service must support
the ability for customers to be notified that such an event occurred. At a minimum, an option for
email notification must exist. The customer must be able to opt in or out of this communication
via self-service means.
■ Instance restart flexibility: In all non-host-specific service maintenance situations, customer
instances must never incur a service downtime or outage. If that occurs, this criterion is
violated. However, in the event where host maintenance must occur and an instance restart is
necessary due to a guest OS requirement (e.g., patch or driver update), providers must offer
customers flexibility in selecting restart windows per instance. For example, if a security
vulnerability patch is required at the infrastructure level and a customer instance reboot is
required to apply the patch, providers must offer customers at least three different choices of
downtime windows or communicate a procedure by which they have flexibility to take their own
action. The flexibility window must be at least 30 days long with the exception of critical
vulnerabilities.
■ Instance anti-affinity: Some enterprise use cases require that two or more instances be split
and hosted across different physical hosts/clusters/locales. This assurance allows customers to
adhere to higher degrees of application availability by ensuring that a single physical failure
event does not take all instances offline. These scenarios become increasingly important when
load balancing multiple instances. Customers must have the ability to configure individual
instances to have anti-affinity rules — through self-service interfaces — for other instances on
an instance-setting basis.
■ Basic autoscaling support: The cloud service must provide functionality to automatically scale
instance pools horizontally (i.e., adding or subtracting compute instances and reconfiguring
load balancing accordingly) based on schedules and triggers. This must be a service and

cannot require direct application instrumentation. For more details, please refer to "Technology
Overview for Autoscaling."
Preferred
■ Instance affinity: Opposite the "instance anti-affinity" requirement, some enterprise
requirements mandate that two or more instances must share a single physical server to
improve performance, minimize latency or maximize security. Therefore, the cloud service must
support enforcing specific instances sharing a physical host through a self-service interface.
■ Extra-large instance support: Providers must offer customers instances with a large number
of processor cores and memory for processor- or memory-intensive use cases. The provider
must be able to provide instances that support at least 32 vCPUs or Cores and 128GB of RAM.
■ Restart priority: For providers that support instance restart, customers can choose what order
they would like instances restarted in or otherwise can indicate a priority level for restart that
determines the order in which the platform restarts instances.
■ Console-level access to instances: The customer can, via the portal, access the local virtual
console of the instance. Although the specific implementation and protocol used for this may
vary, it must emulate the local video and accept mouse and keyboard inputs. It must include
access to basic input/output system (BIOS) and OS load/boot procedures. Finally, the console
must work from Google Chrome in addition to Internet Explorer, Safari and/or Firefox. Many
customers prefer this feature, but many development organizations are not mandating it as
required.
■ Single-tenant compute instances: Single-tenant compute instances ensure that instances
reside on a physical host that is not shared with any other customer. For this specific offering,
storage and network may be shared or isolated. If this capability is not directly integrated into
the fully multitenant service, then customers must have the option of stretching a virtual LAN
(VLAN) or subnet across both environments and sharing Internet Protocol (IP) address space
range with the multitenant service. Providers can provide and price the dedicated or isolated
offering separately from the multitenant offering.
■ Compute performance baseline: Performance benchmarks across the IaaS industry are not
well-defined or standardized. However, enterprise customers need to know what is "expected"
or "normal" in terms of a compute performance baseline at their provider. Providers must
annually publish a documented performance test scenario/script against three or more of the
most popular instance sizes and across both a standard Linux configuration and a standard
Windows configuration. The results of the baseline provider test must also be published as well
as a statement indicating the standard deviation that is to be considered normal from the
baseline. Customers must have access to the test script and suite, thereby making it easy to
repeat or run the test scenario on their own.
■ Advanced autoscaling support: To achieve advanced autoscaling status, the cloud service
must meet the basic autoscaling support mentioned earlier as well as add vertical instance
autoscaling (i.e., resizing instances), health maintenance events and monitoring metric trigger
support.

Optional
■ HPC offering: Many scientific and financial services organizations are increasingly looking to
take advantage of IaaS offerings for high-performance computing (HPC) projects. Therefore,
leading providers must include a unique HPC offering or service tier within their services. HPC
offerings must include the following features:
■ Low-latency, high-bandwidth network connections among the cluster peers of an
architecture of at least 10 Gigabit Ethernet or 10 Gbps throughput
■ Identical and published processor architectures among the cluster peers
■ Support for 250 or more nodes
■ Graphics processing unit (GPU) support: The provider must document which GPU
technology is used so that customers can program specifically against the GPU model.
■ Export instance image: The cloud service must support the ability to take an existing running
instance or a copy of an instance and export the instance into a VMDK, OVF or VHD image
format.
■ Bare-metal provisioning: The cloud service must offer bare-metal compute servers — as a
service — that do not run on top of server virtualization platforms. The bare-metal offering must
be fully automated. Occasionally, enterprise needs require bare-metal provisioning support.
■ Customer-controlled overprovisioning: The provider must offer at least one tier of service by
which a customer can decide to overprovision compute resources. This must be an opt-in
service or a separate service and must be priced at a lower rate than the standard service.
■ Subminute provisioning times: At least one version of a Linux image (with minimum specs of 1
vCPU, 2GB RAM, 40GB persistent storage) must be able to provision, boot and be accessible
in less than one minute.
■ Provider-offered Linux distribution: The provider must offer and maintain a specific Linux
distribution/image that is optimized, tuned and preintegrated for use within the cloud platform.
The provider must regularly update the Linux image with security updates and configurations to
stay current with other cloud platform features. Furthermore, the provider must offer Linux OS
support for customers that opt into support offerings of the cloud service.
Storage
Providers offer IaaS storage in several different flavors, and they are often a complementary service
to a compute offering. Block and object storage services are the most common offerings found in
the industry. Block storage services typically are not exposed via the Internet, whereas object
services typically are. For the purposes of this document, the following definitions apply:
■ Block storage service: Instance-independent block storage whereby the customer can obtain
block storage volumes that are networkattached and independent of any specific compute
instance. The customer can then mount this storage volume on a compute instance.

■ Object storage service: Internet-accessible storage offering whereby the customer can
upload, download and interact with files and objects across.
Required
■ Bulk data import/export: Migrating large amounts of data to any external hosting location is
challenging when dealing with networks. The challenge is due in large part to bandwidth
restrictions, network latency, overall reliability and Internet backbone charges. Therefore,
providers must offer a bulk data import and export service for moving large amounts of data
both into and out of the cloud service. Bulk data movement includes the ability to physically
ship datasets via traditional express-mailing techniques (for example, FedEx, United Parcel
Service [UPS] and United States Postal Service [USPS]). The provider must support any of the
following device interfaces: External SATA (eSATA), SATA I, SATA II or USB 2.0/3.0.
■ Block storage service criteria:
■ Scalable instance-independent block storage service: Customers require the ability to
connect compute instances to network block storage volumes. The main purpose for this
storage service is to provide customers with storage that can persist beyond the life of an
individual instance and to enable advanced functionality such as snapshots and data
replication. To meet large-scale requirements, the block storage service offering must
support at least 10 volumes of at least 1TB in size per customer as well as allow the overall
capacity of the storage service to be unlimited per customer.
■ Block storage snapshots: The cloud service must support a pointintime copy of a storage
volume known as a snapshot. The customer must be able to create a snapshot of any
storage volume through self-service means. Furthermore, snapshots must be able to be
used as an image for the purpose of self-service provisioning new compute instances.
■ Expandable block storage volumes: The customer must be able to increase the size of an
existing block storage volume without having to provision a new volume and copy the data.
■ Object storage service criteria:
■ Scalable object storage service: The provider must provide a distributed, multi-data-
center object storage service where objects (individual files) can be stored and retrieved via
a Web services API. Scalable includes both individual object size scalability and overall
storage scalability. For individual object sizes, the storage service must be able to support
at least 1TB file sizes, and the overall amount of cloud storage capacity per customer must
be unlimited.
■ Object storage replication: The provider must automatically replicate objects across
multiple data centers. This replication must either not cross country boundaries, or the
customer must have the ability to prevent it from doing so or to control the country
placement.
■ Logging of administrative object service requests: The provider must offer the capability
for customers to choose to log, and be able to audit, every administrative object storage
service request. This must include logging of create, read, write, copy and delete events.

■ Provider-enabled encryption services: The object storage services must offer customers
the self-service ability to opt into provider-enabled server side encryption (SSE) for objects
or object hierarchies within the storage service. Customers should note that they can
always manage their own encryption keys and encrypt their own data prior to uploading
and storing within a public cloud storage service.
■ File/object-versioning support: The object storage service must provide a feature to
protect against data overwrite or data deletion using prior file/object versions. Versioning
ultimately allows customers to protect themselves from accidental data overwrite or data
deletion, but too much versioning leads to storage sprawl. Customers must therefore have
self-service configuration control to turn versioning on/off per object or container.
Preferred
■ Cross-geography copy/replication: The provider must offer a replication service offering for
both object and block storage services that traverse regional boundaries. This offering must be
a service customers can opt into for geographic/global data protection, and the customer must
have control over country placement. Due to the distance between regional boundaries, this
replication is assumed to be asynchronous.
■ Block storage criteria:
■ Block storage interconnect transparency: The interconnect between storage services
and the rest of the cloud service can be implemented in a variety of different ways and with
different characteristics. This architecture can have a dramatic effect on the success or
failure of performance requirements. Therefore, the provider must document the storage
interconnect architecture for customers to understand, including the following key points:
■ The explicit, average latency between compute instances and block storage
■ The use of software such as rate limiting or traffic shaping to avoid storage contention
and starvation and the relevant effect or improvement such software has
■ Multiple instance mount: Customers must be able to simultaneously mount a block
storage volume on multiple compute instances. It is acceptable for these volumes to be in
readonly mode.
■ Performance target/tier block storage: Customers must be able to choose to purchase
an explicit performance target or performance tier on block storage volumes, such as a
certain number of I/O operations per second (IOPS), or Mbps of throughput. Because of
limitations in present-day technology, the target need not be an absolute guarantee, but the
provider should document how much variance is typical in the target.
■ Instance-encrypted volumes: A customer must be able to elect to have data on the
instance block storage volume encrypted at rest. This must be a simple, selfservice option
when the instance is provisioned.
■ SSD-based block storage: The provider must offer block storage that is solid-state drive
(SSD)-based. It must provide higher I/O than the non-SSD service offering if one exists, and

the performance target or percentage improvement over the standard offering must be
documented.
■ Snapshot copy/replication: The customer must be able to replicate snapshots on demand
to a different data center, including data centers in a different geographic region. This
feature must be directly supported by the portal and/or API, not by the customer manually
copying files around.
■ Object storage criteria:
■ Automatic object durability: To provide ample data protection, all customer-written data
objects must be automatically replicated to three or more locations, and the provider must
implement erasure coding that tolerates multiple concurrent failures. The service must not
inform the customer that a data-put operation has been successful until at least two copies
have been successfully written. Erasure coding is a method of data protection by which
data is fragmented, expanded and encoded with redundant data pieces and stored across
a set of different locations.
■ Bulk object delete and retirement policies: The object storage service must support the
ability for customers to bulk delete all objects in a container or based on metadata assigned
to objects. For example, if objects are tagged with a metadata tag of "Project X," customers
must have the self-service ability to bulk delete all objects with that tag. Furthermore,
providers must offer customers the ability to define object retirement policies to assist with
data life cycle management. At a minimum, these policies must support object deletion
based on date/age policies.
■ Bulk data import/export with encryption: The provider must meet the required criterion
for bulk data import/export, but must additionally offer a service by which the customer can
encrypt the data prior to transport and then decrypt it upon arrival. This encryption service
must be a provider service offering and not left to the customer.
■ Tiered storage service(s): The provider must offer a lower tier (e.g., less redundant) of
storage service for use cases like long-term archival and facilitate the movement of data
into the lower tier from normal tiers or directly into the lower tier from the outside. Gartner
expects that SLAs, price and performance, specifically access times, may differ greatly
from the traditional tiers of storage services.
Optional
■ Single-tenant storage service: Providers must offer a tier of service that physically isolates
storage for an individual customer while still providing the service interfaces and management
console to enable customer control. Furthermore, the provider must offer connectivity between
the isolated service and the multitenant service should a customer desire to leverage both
service offerings.
■ Instance-specific storage: This storage is directly associated with a single compute instance,
and it persists only for the lifetime of the instance (but does persist across reboots). If the
instance is terminated, either by the customer or by the provider (for instance, because the

underlying hardware fails), the data is lost. Usually, this storage is local to the compute
instance.
■ Array and tape bulk data import and export: Migrating large amounts of data to any external
hosting location is challenging when dealing with wide-area networks (WANs). For very large
datasets within an IT organization, providers must support the ability to receive and ship Fibre
Channel (FC)-based and/or Internet Small Computer System Interface (iSCSI)-based disk arrays
as well as Linear Tape-Open (LTO) tapes. For LTO tapes, the provider must be able to isolate
the physical LTO tape and drive to a specific customer instance so that the customer can install
the corresponding backup software on a cloud instance and initiate the data restore from tape
into the cloud storage service.
■ Static Web hosting support: Providers must offer customers the ability to self-service enable
the object storage service and its containers to act like a fully functioning website. With this
feature, customers can easily upload root and static Web content and serve out websites
without configuring Web servers on top of compute instances. Providers may front end this
capability with a website service outside of the object storage API or interface.
■ Cloud storage gateway (CSG) support: The provider must support one or more on-premises
CSGs at customer's sites in terms of a physical or virtual appliance. The provider may offer its
own CSG or support a leading CSG such as Panzura or TwinStrata. The CSG must offer API
compatibility and provide some level of compression, acceleration or encryption services on top
of simple file movement.
Network
IaaS cloud offerings must operate at extreme scales to satisfy the complex requirements of multiple
customers simultaneously. Therefore, the networking of all IaaS components is crucial to the overall
viability of each service. Too little throughput, not enough isolation between tenants or too much
latency among tiers of service, and customers will likely take their business elsewhere. Therefore,
evaluating the network capabilities of a cloud IaaS cloud provider is important in an overall cloud
service assessment.
Required
■ Customer-defined hierarchical LAN topology: Customers require the ability to design
hierarchical network infrastructure at the provider and choose their Request for Comments
(RFC) 1918
1
IP addressing scheme without dependency on having instances in place at the
provider. Prior to deploying any compute instances, cloud customers must have the ability to
design the following network components and layouts:
■ Firewalls
■ Subnet or VLANs
■ VPN
■ Network address translation (NAT)

■ Load balancing
■ Multiple virtual NICs and network segments: Providers must offer the ability to connect
instances, as allowed by the guest OS, to at least two different virtual network interface cards
(NICs) and routing networks simultaneously. Therefore, the provider must also support at least
two network segments.
■ Isolated network segments and private-network interface instances: Providers must offer
network segments that are fully isolated and not routable externally. Furthermore, instance
configurations must exist that can reside only on these isolated network segments and without
having any public IP address or Internet routing. To qualify for this criterion, an instance must
be able to be deployed with only a private-facing network address, or the provider must allow
for customers to remove a public-facing network address. Firewall support is not sufficient for
qualifying for this criterion.
■ Static IPs: To qualify as having static IPs, the provider must support several capabilities. First, if
a compute instance is dynamically assigned an IP address, that address must remain the same
across the lifetime of the instance (unless the customer wants to change it). Second, the
customer must be able to obtain an IP address from the provider (including an Internet facing
public IP) that can be assigned to a compute instance or load-balancing pool, that can be
moved between compute instances or load-balancing pools and that persists as long as the
customer wants.
■ Choose your own RFC 1918 address space: Customers must have the ability to choose or
define a customized RFC 1918 IP address space within the cloud service environment. This is
critical for many hybrid cloud networking designs.
■ Customer VPN connectivity: Providers must allow customers to access the cloud service via
an IPsec VPN tunnel or Secure Sockets Layer (SSL) VPN tunnel over the public Internet. This
must be a self-service capability from the provider side, although customers will have to make
configurations on their end.
■ Private customer connectivity: The provider must support the ability for customers to make
private WAN connections into the cloud service from the customer data center. In this scenario,
customers must be able to obtain private WAN connectivity (such as Ethernet, Multiprotocol
Label Switching [MPLS], virtual private LAN service [VPLS] or Cross Connect) from carrier or
colocation facilities of their choice into the cloud service. Providers may limit this service to
specific carriers or colocation facilities.
■ Customer network segments across data centers: Providers must enable customers to build
multiple, defined internal networks in the form of network segments (for example, IP subnets) at
the provider that can span two or more physical data centers.
■ Multiple private-network connections per customer: The cloud provider must support two or
more private network connections (such as, VPN, MPLS, direct connect) per customer and per
geographic region within the cloud service. It is not acceptable to require a customer to
maintain multiple cloud accounts to satisfy this criterion. This is important for customers with
multiple offices that do not want to backhaul all traffic into a single private network connection.

■ Front-end load balancing: Providers must support the ability to load balance front-end (that is,
external-facing) instances. The load-balancing service must be self-service and support up to
25 nodes (instances) per load-balancing group. The service must support health checks (that is,
avoid sending requests to nonresponsive compute instances) and round robin, weighted or
metrics-driven algorithms. Some providers may also support virtual appliance-based load
balancers, but this requirement is for self-service load balancing as a service.
■ Back-end load balancing: Providers must support the ability to load balance back-end
instances (that is, instances with private IP addresses, or where the traffic is passed from
compute instance to instance). The load-balancing service must be self-service and support up
to 25 nodes (instances) per load-balancing group. The service must support health checks
(that is, avoid sending requests to nonresponsive compute instances) and round robin,
weighted or metrics-driven algorithms. Some providers may also support virtual appliance-
based load balancers, but this requirement is for a self-service load balancing as a service.
■ DNS-based global load balancing: The customer can, as a self-service capability, configure
global load balancing, where requests are directed to endpoints that can be located in different
data centers (and those endpoints can be different front-end load-balancer pools). This is
usually a DNS-based service. Furthermore, the service must support endpoint health check
routing (that is, the service will automatically avoid sending requests to endpoints that are
unresponsive) and at least one of the two following algorithms:
■ Latency-based request routing: Where the request is directed to the endpoint with the
lowest latency between the requestor and the location.
■ Geographic request routing: Where the request is directed to the endpoint based on the
location of the request origination. Typically, this will be used to route a request to
endpoints based on the country of origin.
Preferred
■ Five network segments: Providers must support five or more virtual networks or routing
segments simultaneously per customer and per geographic region. Simple application
architectures require only one or two network segments, but more complex architectures with
several applications often require five or more virtual networks. These networks must be a
standard offering and accessible via self-service interfaces to apply.
■ Interregion private network: The provider must offer customers the ability to communicate
between all cloud data centers by private network. This service offering must route data-center-
to-data-center traffic, including across geographic regions, over a private network. This private
network can be logically private and does not have to include private fiber.
■ Session affinity load balancing: In addition to meeting the load-balancing requirement
previously detailed, providers must support session affinity features within the load-balancing
service. This is typically done by setting a session cookie and continuing to route requests
associated with that cookie to the same compute instance.

■ Metrics-driven load balancing: In addition to meeting the load-balancing requirement
previously detailed, providers must support metrics-driven load-balancing features. Metrics-
driven load balancing is when the load-balancing service has insight into various metrics at the
compute instances (such as utilization, latency and responsiveness) and routes requests based
on those metrics. Furthermore, customers must have the self-service ability to choose which
load-balancing algorithm they want to use for a given configuration (for example, round robin,
weighted and metrics-driven).
■ Network performance tiers of service: Customers must be able to choose to purchase an
explicit network performance target or performance tier within the cloud service, such as a
certain guarantee of latency or bandwidth throughput. Because of limitations in present-day
technology, the target need not be an absolute guarantee, but the provider should document
how much variance is typical in the target.
■ Multiple private-network connections per network segment: The cloud provider must
support two or more private network connections (such as VPN, MPLS, direct connect) per
network segment. For example, a cloud customer must be able to connect two VPN
connections into a single virtual network simultaneously. If the provider supports only one
private network connection per network segment, the provider must support internal network
security access control lists (ACLs)/firewalls between network segments and the ability to route
across the network segments.
Optional
■ WAN traffic encryption: The provider must encrypt all WAN traffic between cloud data centers,
regardless of what is used for interdata center connectivity.
■ LAN traffic encryption: Recent revelations of government surveillance have increased
customer concerns about data privacy in the cloud, raising the featureroad-map priority of end-
toend encryption for many service providers. To meet this criterion, the provider must encrypt
all LAN traffic between compute instances within the data center.
■ Instance support for five or more network interfaces and IPs: To satisfy some customer
application use cases, providers must offer the ability to connect instances, as allowed by the
guest OS, to at least five different virtual NICs and routing networks simultaneously and support
up to five different IP addresses.
■ IPv6 support: Providers must support Internet Protocol version 6 (IPv6) at either the gateway
(for example, load balancer) or instance level and expose this functionality to customers.
■ Real-time network performance visibility: Customers want the ability to see real-time
performance monitoring of network throughput and latency between services (for example,
compute-compute or compute-storage) at the provider. Having insight into the current levels of
service performance among tiers enables cloud consumers to understand whether the
performance levels they are seeing within their own applications match the overall cloud service
health. Customers understand that variance of performance among tiers does occur, so real-
time insight is important for some customers.

■ High-speed cross-geography networking: Providers must optimize the network traffic
performance across geographies/regions to achieve the following sustained metrics between
two geographies/regions. Tests will be run from one United States geography to one European
geography and from one United States geography to one Asia/Pacific geography:
■ Ping round-trip time (RTT) less than or equal to 75 milliseconds (ms)
■ Transfer of a 1GB file over Secure Copy (SCP) at speeds greater than or equal to 2MB/sec
Security and Access
Security and access control often top the list of biggest customer concerns about using cloud IaaS
solutions. Cloud IaaS providers are quickly improving security and access configurations as well as
sharing more information about the ways in which their environments are secured.
Required
■ Document user control considerations: There is a division of responsibility between the
provider and the customer, which the provider must make explicit by documenting a set of user
control expectations. These are the assumptions that providers make about the operational and
security risk controls that the customer must employ. These must be documented and are often
provided in the form of a checklist or documented in a shared responsibility model.
■ Block storage data eradication: The cloud service must support one of the two following
forms of data eradication:
■ Immediate eradication: The cloud service offers the capability to immediately overwrite the
data associated with a storage volume when the storage volume is deprovisioned or
otherwise released by the customer. This allows the customer to be assured that the data
has been eradicated from the physical disks. Your platform may do this automatically for all
storage volumes, or customers may be offered the choice to force an overwrite (which
might incur an extra fee). This must be a platform feature and not left to manual customer
responsibility.
■ Eventual overwrite: This guarantees that data deleted by a customer cannot be accessed,
and that deleted data is eventually overwritten. For block storage, this means that nobody
has access to read that block again until the block is overwritten. Furthermore, if a
customer deprovisions or otherwise releases a storage volume, those blocks must be
overwritten before that storage is made available again (whether to the same customer or to
another customer).
■ Data sanitization: The provider must have documented evidence that it adheres to the
Department of Defense (DoD) 522022M or National Institute of Standards and Technology
Special Publication (NIST SP) 80088 processes for data sanitization and disposal when the
provider retires or otherwise disposes of physical storage devices (such as disks and arrays).
This policy must apply to all storage services.

■ External network security ACLs/firewall: The provider must offer customers the ability to
allow and deny external network traffic based on source and destination address and port.
Customers must also be able to group infrastructure elements and assign a set of ACLs to the
group rather than to individual instances one at a time. The service must be stateful.
■ Internal network security ACLs/firewall: The provider must offer customers the ability to allow
and deny internal network traffic (within the customer's instance deployments) based on source
and destination address and port. Customers must also be able to group infrastructure
elements and assign a set of ACLs to the group rather than to individual instances one at a
time. The service must be stateful.
■ Allow firewall policy to be changed per instance or group: Providers must not force
customers to shut down, reboot or redeploy any customer infrastructure involved in the change
of a network security ACL or group.
■ Annual SOC 1 and SOC 2 Reports: Providers must have an annual, completed Service
Organization Control 1 (SOC 1) and SOC 2 audit report that customers and prospects can see
on demand. Gartner allows providers to require a signed confidentiality agreement to meet this
requirement. Customers should realize that SOC 1 focuses more on financial reporting controls,
whereas SOC 2 focuses on a "business's non-financial reporting controls as they relate to
security, availability, processing integrity, confidentiality and privacy of a system."
2
■ Published compliance assistance: Organizations have a wide variety of regulatory, privacy
and compliance requirements that cross-section industry verticals and individual countries. The
onus of adhering to individual compliance requirements always falls upon the shoulders of the
application owner (that is, the organization). Therefore, for any regulatory, privacy or compliance
certification that the provider claims to support, the provider must have published guidance that
lays out for the customer how individual applications or deployments can adhere to the named
certification. The provider must have a consolidated listing of all such certifications for
customers to see along with the supporting documentation.
■ Customer control over data locale residency: Providers must assure customers that the
provider will not arbitrarily move data across international boundaries. This means that the
cloud service is designed in such a way that customers are clearly aware where each service is
physically hosted (that is, the country), and the service allows customers to physically move
data between locales, either through service interfaces or through management consoles. If a
provider offers a redundancy service that distributes data across international boundaries, it
must be opt-in rather than opt-out.
■ ISO/IEC 27001 certification: ISO and IEC published an information management standard in
2005 referred to as International Organization for Standardization/International Electrotechnical
Commission (ISO/IEC) 27001:2005 or ISO/IEC 27001:2013, often shortened to ISO 27001. ISO
27001 mandates specific requirements to organizations about the management of information
security. Providers must have achieved ISO 27001 certification to assure customers that the
management of information security is handled according to the high process standards set in
ISO 27001. The certification must cover all worldwide data centers. For more information, refer
to "ISO/IEC 27001:2013 Shifts Focus From Effectiveness of Controls to Risk Treatment Plans."

■ Customer data ownership: Although the provider owns the rights to the overall service and
physical infrastructure, the general terms of service and enterprise agreements must specify
that ownership rights to all data, inputs and outputs of consuming the service are retained by
the cloud customer. The ownership must be retained by the customer even through a provider
acquisition or bankruptcy event. In the event of bankruptcy, closing of business, or retirement of
service, the provider must provide 90 days for customers to get their data out of the system and
migrate applications.
■ Provider personnel protections: The provider must document the measures taken to protect
customer data from employees and personnel. This must include background checks, logging
of administrative access to the cloud service and network separation between the cloud service
and the general-purpose LAN of the provider company. It is acceptable for this documentation
to be included in an audit report such as Statement on Standards for Attestation Engagements
(SSAE) 16, ISO 27001 or to be published in a stand-alone security paper.
■ Secure instance access credentials: The default administrative access for every deployed
instance in the cloud service must be automatically generated or chosen at the point of
provisioning by the customer. If automatically generated, the access credentials must be
communicated to the cloud administrator either through an API service call or visually in the
management console. Passwords must never be emailed.
■ SSL-secured API endpoints: Providers must offer API endpoints (that is, customer access
points) that are secured with SSL.
■ Multiple API keys per customer: Providers must allow customers to generate and own
multiple API keys per customer.
■ Local identity management and granular role-based authorization — compute: Providers
must include, at minimum, a local identity management system (that is, local accounts) with
granular role-based authorization for compute services in both the service interfaces and
management console. At a minimum, the role-based authorization must support assigning
authorization based on individual users and groups of users and must support, as applicable,
delineation of the following granular actions: create, delete, restart/reboot, shutdown, startup
and back up/snapshot.
■ Local identity management and granular role-based authorization — storage: Providers
granular role-based authorization for storage services in both the service interfaces and
authorization based on individual users and groups of users, and delineation must be assigned
per volume or container and support, as applicable, the following granular actions: create,
delete, attach, detach and back up/snapshot/copy.
■ Local identity management and granular role-based authorization — network: Providers
granular role-based authorization for network services in both the service interfaces and
authorization based on individual users and groups of users and delineation must be assignable

per firewall, load balancer, IP address and network segment and support, as applicable, the
following granular actions: create, delete and configure.
■ Private image catalog: The provider must allow customers to maintain a private image catalog
of instances that is not accessible to other cloud customers. Furthermore, the provider must
offer basic role-based authorization into the catalog so that granular rights (i.e., create, deploy,
delete) can be handled on an image-by-image basis and on a user-by-user basis.
■ Federated SSO to management console: Providers must offer single-sign on (SSO) login
access to the graphical user interface (GUI) management console through Security Assertion
Markup Language (SAML) assertions.
■ MFA administrative access control (root and admin): Providers must offer customers the
ability to control root (that is, parent- or master-level) administrative access to the cloud service
through multifactor authentication (MFA). Multifactor authentication does not need to be the
default configuration, but providers must present an offering for customers to secure the root or
master account. Examples of satisfactory multifactor authentication include hardware- or
software-based secure tokens based on the HMAC-based one-time password (HOTP; hash
message authentication code [HMAC]based, RFC 4226) or time-based one-time password
(TOTP; timebased, RFC 6238) algorithm.
Preferred
■ Advanced firewall functionality: Providers must support the following advanced firewall
features:
■ Firewall policy hierarchy: Customers must have the ability to implement levels of firewall
policies. This is helpful for implementing overarching firewall policy restrictions that other
administrators cannot override.
■ Multiple firewall policy assignment: Customers must have the ability to assign three or
more firewall policies simultaneously to an instance or group of instances.
■ Cloud security guideline matrix: Many different security standards and guidelines exist in the
industry. IT organizations deem it important for public IaaS providers to respond to two or more
of the leading standards as a compromise to responding to every cloud security document in
existence. Providers must create a document detailing how their cloud service meets or does
not meet every detail from one of the following cloud computing security guidelines:
■ Cloud Security Alliance (CSA) — "Security Guidance"
■ Federal Risk and Authorization Management Program (FedRAMP)
■ NIST SP 800-53 — Security and Privacy Controls for Federal Information Systems and
Organizations
■ European Network and Information Security Agency (ENISA) — "Cloud Computing Risk
Assessment"

■ Penetration testing request process: IT organizations that implement critical applications
often employ a security organization or firm to conduct a penetration test against an
application. Penetration tests seek to expose security vulnerabilities, scalability limitations or
general configuration errors. A penetration test may be intrusive and impactful to infrastructure
at a cloud provider, and some provider intrusion detection systems (IDSs) may even signal
alarms indicating that the penetration test is an actual attack. With that in mind, providers must
be able to support customer requests to conduct a penetration test. Providers can qualify for
this criterion, even if they request that customers follow a request process for initiating a
penetration test.
■ SIEM integration or service: Security information and event management (SIEM) provides real-
time analysis of security alerts generated by applications or network equipment. Providers must
offer out-of-the-box integration with leading SIEM products or provide a self-service, turnkey
offering by which customers can configure real-time analysis and alerting of security events. At
a minimum, the integration or service must support alerting, log retention and some form of
forensic analysis that is able to search across logs and periods of time for patterns.
■ Enterprise directory integration: Providers must offer integration with on-premises Active
Directory (AD) or Lightweight Directory Access Protocol (LDAP) directories for cloud service
account management. Providers must support importing and synchronizing users with the
customer's AD or LDAP directory and/or allowing users to authenticate with the customer's
directory. Such functionality must be a service offering and not a customized customer solution.
■ API support for federated authentication: Providers must enable identity federation for
service interfaces (that is, APIs) and provide documentation for customers to implement the
federation. Organizations that build new applications within an IaaS environment prefer to be
able to leverage identity federation through service interfaces and into the application. In order
to build this functionality, customers need more than federated SSO to the management
console; they need federated authentication into the service interfaces underlying the cloud
infrastructure. Due to the complexity of integrating cloud service interfaces with federation,
Gartner does not require that this federation be implemented with leading open standards.
■ Role-based authorization based on dynamic group or tag: Providers must offer the ability for
role-based authorization permissions to be specific to elements of the service rather than
applying across the entire account and its associated resources. Permissions may apply per-
group, per-tag or per-element. Role-based authorization in such fashion is described as:
■ Group: This type of permission applies to the elements under a group. For instance, a group
permission could cover the ability to restart compute instances within the "prod" group.
■ Tag: This type of permission applies to the elements that have a particular metadata tag.
For instance, a tag permission could cover the ability to restart compute instances that have
the "dev" tag.
■ Element: This type of permission applies to an individual item like a single instance, single
load balancer or single firewall.

Optional
■ Approval workflow: Providers must offer customers a self-service approval workflow service.
Customers sometimes want the ability to add an approval step to cloud service functions,
especially those that accrue cost. Approval workflow means that when a self-service user
makes a provisioning request, the account owner (or some other delegated manager) must
approve or deny the request, and the request is automatically executed if approved. Approval
workflows must be able to be designed with a self-service interface and must allow for flexibility
of approval assignment based upon the master account holder or delegate.
■ Patch management service: The cloud service must offer an automated patch management
service for Windows instances and enterprise Linux instances running on the cloud platform.
This should be an optional service that customers can opt into as they see fit. The service must
include approval processes for patches and target or deadline dates for installation
enforcement.
■ Network forensic tools as a service: Visibility into networking at a cloud provider is often very
limited. Some customers are asking cloud providers to provide network forensic tools as a
service. To support this criterion, the provider must offer a self-service capability to opt into
network forensics and alerting services such as intrusion detection and intrusion prevention
services as well as analytics and historical trending. This must be above and beyond firewall
services.
■ HSM support: Providers must offer support for hardware security module (HSM) that assists
with the most stringent regulatory requirements by offering cryptographic key management. To
meet the regulatory requirements, the HSM service or support should include at least one
offering or tier that has dedicated access. The HSM should comply with NIST Federal
Information Processing Standard (FIPS) 140-2.
3
■ Published CSA STAR documentation: "The objective and mission of CSA Security, Trust &
Assurance Registry (STAR) are improving trust in the cloud and ICT [information and
communication technology] market by offering transparency and assurance."
4
CSA STAR is a
generally available and free repository of cloud provider controls aimed at assisting end-user
organizations in assessing cloud providers. CSA has garnered attention and legitimacy among
IT organizations. Providers must complete the necessary steps to register at STAR in direct
relationship to the cloud offering at the provider and not colocation or managed hosting
services that the provider might also offer.
■ Instance vulnerability scanning: The provider must offer a service whereby customer
instances are scanned for guest vulnerabilities. This service must provide a report/dashboard
for customers to see any vulnerabilities that exist. Providers may charge extra for these
services, but the service must be accessible via self-service interfaces.
■ Adaptive authorization based on time and location: Providers must offer customers adaptive
authorization to cloud services — based on time and location — that is configurable through a
self-service interface. Adaptive authorization in such fashion is described as:

■ Time: This type of permission restricts access to the functionality to a certain time. For
example, a time permission could allow the restarting of compute instances only between 3
a.m. and 5 a.m.
■ Location: This type of permission restricts access to functionality based on the IP address
of the source of the request. For instance, a location permission could restrict the restarting
of compute instances to just requests coming from a particular corporate network IP
address block.
■ Annual SOC 3 published publicly: SOC 3 is a higher-level (that is, more general-purpose)
report than a SOC 2. Due to the nature of this type of report, auditing agencies allow providers
to make the report freely available on a public website. Therefore, to meet this requirement, the
provider must have an annual SOC 3 report and publicly publish on the provider's website.
■ PCI DSS Level 1 compliance: The provider must be Payment Card Industry Data Security
Standard (PCI DSS) Level 1 compliant for the cloud service and have supporting documentation
for customer assistance.
■ HIPAA with BAA: The provider must support Health Insurance Portability and Accountability
Act (HIPAA) compliance and sign a business associate agreement (BAA) with customers. The
provider must have supporting documentation for customer assistance.
■ U.S. government certifications: U.S. government agencies have a variety of certification
needs. Each is listed as an individual optional criterion:
■ FedRAMP, including JAB P-ATO
■ PPT-Moderate Personnel
■ ITAR Compliant
■ FISMA Low
■ FISMA Moderate
■ FISMA High
■ FIPS 140-2
Service Offerings
Compute, storage and network are the three main pillars in IaaS offerings. However, many criteria
exist that cannot be isolated into any of the three specific areas. Therefore, Gartner has highlighted
criteria not specific to any of the three pillars into this section. This category generally focuses on
service implementations of the bundled IaaS stack, including data center architecture, availability,
scalability and value-added services.

Required
■ Data center dispersion: Providers must have at least three data centers a minimum of 200
kilometers (125 miles) apart from one another. These data centers must be on different power
grids. Offering multiple data centers with this level of dispersion allows enterprise customers to
develop availability options that can sustain local outage-causing issues like storms.
■ Data center proximity: Providers must also have at least two data centers (per geography) that
are within 100 kilometers (60 miles) of each other to support synchronous replication
requirements. This is the maximum distance for which the latencies are low enough to
accomplish synchronous replication. Furthermore, customer networking must be able to span
across these local data centers so that customers may build highly available architectures.
■ Multiple continents and data centers: Providers must include service offerings to satisfy high
availability (HA) and disaster recovery (DR) on multiple continents and for multinational
customers. At a bare minimum, the provider must have at least two distinct geographic
offerings in the United States (e.g., east and west coast), one in Europe and one in Asia/Pacific.
Furthermore, for each geographic service offering, the implementation must include at least two
separate, distinct hosting locations (for example, data centers or zones) to qualify. Within a
geographic locale, IT organizations must have the ability to distribute applications across two or
more physical data centers.
■ Offer negotiation for custom terms of service/cloud-hosting agreement: IT organizations
often begin using an IaaS cloud solution based on a publicly accessible terms-of-service
agreement; however, most of the same organizations eventually desire signing a more robust
enterprise agreement for cloud hosting. Providers must offer such customers the ability to
negotiate a custom agreement. To qualify for this criterion, a provider must offer a request
process by which a customer can initiate a custom-negotiated agreement, and the provider
must be willing to materially alter the terms of the standard agreement.
■ Published architecture transparency: In order for customers to assess the risk of using a
cloud IaaS environment — in addition to impacts of migration, compliance, licensing,
configuration and performance — providers must publish certain infrastructure details to
customers. Providers must offer published documentation with infrastructure configuration
details that include all of the following:
■ The region and country of each data center location (e.g., northern California or western
Germany)
■ Data center availability configurations for power, cooling and network
■ The underlying hypervisor product/technology
■ The current relative physical CPU processor architecture series and life cycle strategy, or
the ability to query the actual processor architecture on an instance-by-instance basis
■ A description of the configuration details of what constitutes a vCPU at the provider
■ Any variability, oversubscription or throttling of vCPUs

■ Data resiliency, protection, replication or durability strategies — essentially, how often data
is replicated and protected by the provider automatically on behalf of the customer
■ Enterprise customer case studies: Case studies and customer references are an important
aspect of evaluating providers. The provider must have a repository of customer case studies
published on its website. The list must include at least three enterprises with annual revenue
over $1 billion. The repository must also include at least one case study from each region of the
world in which the provider has a cloud presence.
■ Published reference architecture: The provider must maintain a published list of reference
architecture diagrams and descriptions on its website. This material should be a set of
recommended patterns that customers can emulate when deploying common scenarios. The
provider must have at least five reference architecture examples, and the examples must be
inclusive and illustrative of combining multiple services (e.g., compute, storage and networking)
to solve a common customer problem with the cloud service.
■ Relational DBaaS: The provider must offer a relational database as a service (DBaaS), provided
as a fully automated, self-service turnkey offering. In this service, the customer should not have
access to the underlying instance, and the database maintenance must be done entirely by the
provider. At a minimum, the service must support one open-source database (either MySQL or
PostgreSQL) and one enterprise database (either Microsoft SQL Server or Oracle).
■ In-memory caching: In memory caching offers customers the ability to optimize performance
for read operations as well as maintain important availability information such as session affinity
and state. The provider must offer an in-memory caching service that is delivered via a self-
service, turnkey offering. The customer should not have access to any underlying infrastructure
running the caching service, and maintenance must be done entirely by the provider. The
service should support standard specifications (such as Memcached and Redis).
Preferred
■ Content delivery network: Many enterprise customers desire that a provider also offer a
service for global content delivery networking. Content delivery networks (CDNs) are global-
caching or acceleration networks that attempt to improve end-user access performance around
the world. Popular existing CDN vendors include Akamai, EdgeCast, Level 3 Communications
and Limelight Networks. Providers may look to implement their own CDN or collaborate with a
CDN vendor and resell such functionality as a service to their customers. The CDN service must
be offered in self-service fashion with all maintenance offered by the provider. To qualify for this
criterion, the CDN offering must provide 15 or more edge locations including presences in North
America, South America, Europe, Asia/Pacific and Australia.
■ NoSQL DBaaS: NoSQL databases are increasingly popular for massive scale-out application
architectures. The provider must offer a fully automated, self-service NoSQL DBaaS offering
that is accessible from the rest of the cloud IaaS offering. The provider may support a common
NoSQL platform like MongoDB, Cassandra or CouchDB or build its own.
■ Relational DBaaS with redundancy: Providers must offer additional services to the relational
DBaaS offering previously listed by optionally offering the DBaaS in a locally redundant fashion,

meaning that the customer database is automatically replicated across multiple data centers
within a single geography. Opting into this service must be simple and through self-service
means, not requiring any additional replication configurations on the customer's part.
Optional
■ Hadoop as a service: Providers must offer a Hadoop environment that is provided for the
customer as a fully automated self-service turnkey offering. This must be a full service, not
simply a "one click install" of Hadoop or the like.
Support and Service Levels
Self-service and on-demand features helped define IaaS offerings. However, IT organizations need
support and service levels from providers that will accommodate enterprise-grade hosting
requirements. Enterprise customers care about support and service levels because they
demonstrate provider commitment and integrity and help build trust in the relationship between
providers and customers. SLAs do not prevent cloud outages. Rather, they define the terms of an
outage, incident or degradation of service; they explain the response expectation to issues; and
they set customer expectations for service performance.
Required
■ 24/7 support, 15-minute response: Providers must offer at least one support service tier that
includes 15 minutes or less of incident response time while also being offered 24/7. This tier of
support is required by many IT organizations in order to align with internal SLAs and
operational-level agreements (OLAs) for production and critical workloads.
■ Member of TSANet: Providing IaaS is a complex initiative that includes integrating many
different pieces of IT hardware and software. Furthermore, IT organizations marry internal
hardware and software components with various cloud services. When problems arise, it is
challenging for providers or IT organizations to gather the various companies together to solve
complex issues. Technical Support Alliance Network (TSANet) offers organized collaboration
across vendors in order to solve complex problems. The provider must be a member of
TSANet.
■ Live support offering (English): Providers must offer live support via phone and instant
messaging. Furthermore, the live support must be offered in English. Native languages for each
hosting location are covered in the Preferred category.
■ Free online self-service support: Providers must offer free, online self-service support that
includes FAQs, a knowledge base and discussion forums. The discussion forums must have
evidence of regular participation and moderation from cloud service support staff.
■ Online error/bug reporting: Providers must offer all customers an online-based mechanism for
reporting errors or bugs with the service. Providers must offer the mechanism through an online
process posted on their public cloud service websites.

■ One year parallel support for API changes: In the event of an API retirement, upgrade or
substantial change, providers must guarantee in writing at least one year of parallel support for
both the old and new APIs.
■ Cloud service partner registry: Providers must have an online listing of established and official
partners that offer value-added functionality such as increased security, management,
integration, audit, control or configurations. The partner list must be organized and broken
down by the service that a partner provides.
■ Published DR plan and test results: Providers operate very large data centers and large
amounts of infrastructure. Many customers want to deploy critical applications in these
environments and need surety that a provider has a well-planned and well-tested DR plan in
place. Therefore, providers must conduct at least one DR test annually and publish the results
to customers on demand. Providers may require that customers sign a confidentiality
agreement in order to see the assessment.
■ Dedicated account manager offering: Providers must offer at least one support service tier
that includes a dedicated account manager. Large and complex IT organizations prefer a
concierge-level support experience for production and critical workloads and a direct
relationship with an account manager who understands the unique requirements and business
challenges of the customer.
■ Cloud offboarding support: Providers must offer customers the ability to completely sever all
existing assets, deployments and spend with the provider. This must be a process that
terminates all cost-accruing assets rather than making the customer individually terminate each
asset. Due to the high-impact nature, it is acceptable for providers to inject a workflow to
confirm that this action is desirable through a phone call or email confirmation.
■ 90-day SLA change notice: In the event of an SLA change in language, exclusions, terms or
measurements, providers must provide at least a 90-day advanced notice before implementing
an SLA change. If the SLA measurement change is advantageous to the customer (for example,
availability increases from 99.9% to 99.99%), an advanced notice is not required. To qualify, the
provider must define the SLA change notice in the terms of service.
■ SLA versioning and revision history: Providers must make the SLAs for standard service
accessible for review at any time and include versioning control as well as revision history.
Enterprise customers need the ability to see the revision history of all SLA changes for proper
auditing and continuous assessment.
■ 60-day service health and SLA history: Providers must offer a dashboard or snapshot of
service health and standard SLA status for customers to view at any time. The dashboard must
contain at least 60 days of trailing health history so that customers have more than one billing
period to review health and SLA status against the prior month's billing reports.
■ Downtime calculation starts immediately: Customers require that downtime calculations
begin immediately when the downtime starts. However, it is acceptable for up to five minutes to
pass before reporting engines take notice of the downtime. Delays longer than five minutes are
not acceptable for this criterion.

■ Compute service availability SLA — 30 minutes: The cloud compute service must offer at
least one tier of service that has a maximum allowed outage time of less than or equal to 30
minutes per month. Measured monthly, an uptime availability SLA of 99.95% or higher achieves
the threshold of 30 minutes per month.
■ Single-instance/single-data center availability SLA: Providers must offer and document a
cloud compute single instance or single data center SLA. This SLA must be applicable for a
single instance or a single data center and cannot contain any requirements mandating the use
of multiple instances or multiple data centers. Customers should note that the availability
percentage for a single instance could be considerably less than the availability SLA for
multiple/pooled instance sets.
■ Storage service availability SLA — 30 minutes: The cloud storage service must offer at least
one tier of service that has a maximum allowed outage time of less than or equal to 30 minutes
per month. Measured monthly, an uptime availability SLA of 99.95% or higher achieves the
threshold of 30 minutes per month.
■ Service credits/refunds have no limits: During the time that a service is unavailable,
customers must not be charged for the use of that service. SLA penalties must be in the form of
service credits or refunds, and must not cap at less than 100% of the previous month's bill.
■ Notification window for customer to submit SLA miss >= two billing cycles: Providers must
grant customers two billing cycles worth of time to submit a claim after the occurrence of an
SLA miss. Enterprise IT customers have complicated organizational structures, including
accounts receivable/payable, procurement, operations and architecture. Such customers need
at least two billing cycles in order to submit SLA miss claims to a cloud service provider.
■ No maintenance downtime exceptions in the SLA: Providers must count all noncustomer-
initiated downtime events as outages, no matter how the downtime occurs. This means that any
scheduled, announced, planned, unplanned or malicious events all count against documented
SLAs.
Preferred
■ Live support offering (native language): Providers must offer live support via phone and
instant messaging. Furthermore, the live support must be offered in the native languages for
each hosting location. When public IaaS providers expand globally, IT organizations in the
different countries require language support native to them.
■ Tiers of support assignable per consumption and by granularity: Providers must allow
customers to self-assign different tiers of support to assets based on granular classification and
not by maintaining separate master cloud accounts. Examples of classification might include
the user, a component (for example, an instance) or a metadata tag that can be assigned to any
asset. Customers must furthermore be able to change service support tiers on demand through
a service interface or management console.
■ Technical certification program: Providers design environments differently and utilize different
underlying architectures and configurations. Therefore, providers must offer a technical
certification for consumers or administrators of the service. The technical certification must

involve an exam or test to obtain certification and may or may not involve instructor-led or
individual-led study.
■ MSP and consulting ecosystem: Providers must have an online listing of established and
official managed service provider (MSPs) and consultants that offer customized services such
as architecture, design, implementation, integration or support. The partner list must be
organized and broken down by the services that an ecosystem partner provides.
■ 365-day service health and SLA history: Providers must offer a dashboard or snapshot of
service health and standard SLA status for customers to view at any time. The dashboard must
contain at least 365 days of trailing health history so that customers have ample time to review
health and SLA status against the prior year's billing reports.
■ Compute service availability SLA — five minutes: The cloud compute service must offer at
least one tier of service that has a maximum allowed outage time of less than or equal to five
minutes per month. Measured monthly, an uptime availability SLA of 99.99% or higher achieves
the threshold of five minutes per month.
■ Storage service availability SLA — five minutes: The cloud storage service must offer at least
one tier of service that has a maximum allowed outage time of less than or equal to five minutes
per month. Measured monthly, an uptime availability SLA of 99.99% or higher achieves the
threshold of five minutes per month.
■ Data reliability SLA >= 99.99%: The cloud object storage service shall have a minimum
documented data reliability SLA of 99.99%. Gartner differentiates data reliability from overall
uptime availability because a storage service can be online while still failing in terms of the
reliability of specific customer data. For example, an SLA of 99.99% data reliability equates to
tolerance of one out of 10,000 files/objects being unavailable/corrupt at any point during the
billing period. This must be an actual SLA and not simply a definition of how often providers
replicate objects.
■ Customer view of SLA dashboard: Providers must offer a dashboard or snapshot of SLA
status on a per-customer basis. A customer-specific dashboard must take into account all of
the cloud assets a customer has deployed as well as the various locations where those assets
reside. The dashboard should therefore be able to pinpoint whether or not a specific customer
asset was affected by a specific service issue based on location or region. The dashboard must
contain at least 365 days of trailing health history so that customers have ample time to review
health and SLA status against the prior year's billing reports. This is in addition to the service-
wide health dashboard.
Optional
■ Compute service availability SLA — three minutes: The cloud compute service must offer at
least one tier of service that has a maximum allowed outage time of less than or equal to three
minutes per month. Measured monthly, an uptime availability SLA of 99.995% or higher
achieves the threshold of three minutes per month.

■ Storage service availability SLA — three minutes: The cloud storage service must offer at
least one tier of service that has a maximum allowed outage time of less than or equal to three
minutes per month. Measured monthly, an uptime availability SLA of 99.995% or higher
achieves the threshold of three minutes per month.
■ SLAs are offered in a programmatically readable format: Emerging requirements from
organizations will increasingly compel providers to offer their SLAs to customers in
programmatically readable formats such as XML. IT organizations leverage many different cloud
providers and brokers, and manually keeping track of each individual SLA is challenging. IT
organizations will be looking to automate SLA tracking and so will require that providers offer
SLAs in a programmatically readable format or through a service interface that integrates into
an SLA tracking system or dashboard.
Management and DevOps
The strength or weakness of an IT organization often has nothing to do with technology
implementation or selection, but instead with the organization's processes and people. When IT
organizations desire to move critical workloads to cloud IaaS environments, it is paramount that
providers offer them mature enterprise management capabilities and DevOps solutions to further
automate and control the various customer assets.
Required
■ Self-service CLI and API for all cloud functions: Providers must provide all cloud customers
with command-line tools for both Linux and Windows and a REST-based API for all cloud
service functions. A cloud service that is not built on service calls is not enterprise-ready and is
difficult to automate. Providers must offer a development center that includes documentation
for all APIs.
■ GUI management console support: Providers must offer a rich, configurable management
Web-based GUI console for interacting with the cloud service. Providers must demonstrate that
most critical cloud service functions are represented in the GUI management console and that
— within 180 days of release — all new services are represented in the GUI management
console. Finally, the Web-based GUI console must support current versions of Internet
Explorer, Firefox, Chrome and Safari as well as mobile browsers in leading iOS and Android
tablets (for example, Mobile Safari and WebKit).
■ Self-service incident logging system: Providers must offer an incident management system
for identifying, submitting and tracking cloud service incidents. The system must be available
online and accessible via API to paying customers. It must also include the capability to submit
incidents and track incident status.
■ Custom tagging and grouping of resources: Providers must offer customers the self-service
ability to group or tag all of their assets. The provider must support the ability to attach at least
three custom metadata tags or group memberships per asset with no limit on total number of
metadata tags or groups per customer. All custom metadata tags must be searchable and
filterable within both the service interfaces and the management console.

■ Self-service templating: Providers must offer customers the self-service ability to create
infrastructure templates. A template in this scenario is a blueprint that allows a collection of
different resources to be provisioned together, such as compute instances, storage volumes,
network elements, monitoring configurations and security configurations. The customer must be
able to create, store and provision from templates, and the templates must be able to be built in
the cloud service without having to learn a configuration language. A template is not just the
combination of compute instance size with a particular image. It is a build manifest that
provisions multiple elements, and it must be able to include more than the specification of how
to provision one or more compute instances.
■ Real-time performance-monitoring service: Providers must offer a real-time performance-
monitoring service that must be accessible from a service interface or the management console
and not require that customers build their own performance-monitoring hooks or integration
points. The services must support monitoring of compute metrics: (such as CPU utilization,
memory utilization, disk I/O and network I/O), storage metrics (such as utilization, IOPS and
queue length), network metrics (such as bandwidth and packet loss) and loadbalancer metrics
(such as request latency and connection count).
■ Real-time performance health checks, thresholds and alerts: Providers must offer
customers the ability to monitor the general health of their infrastructure (that is, compute,
storage and network) for failures and send an alert if a failure occurs. Providers must offer
customers the ability to receive alerts when a performance-monitoring threshold is reached or
exceeded within one minute of the threshold occurring. Furthermore, customers must be able
to define at least three different performance thresholds for which to be alerted, and customers
must be able to assign these thresholds on a component-by-component basis (for example,
instances, storage or networks). At a minimum, the provider must support email and SMS alerts
and is encouraged to support URL triggers.
■ API access to monitoring data: Providers must expose customer-accessible monitoring data
from the provider's monitoring service via the provider's API. This functionality allows
customers to programmatically integrate cloud monitoring data with in-house monitoring
systems or to export monitoring data for analysis. The provider must support current health
status, current metrics and historical data via the API.
■ Account management logging: Providers must offer logging of account management
activities, including user create/delete events, user grouping/tagging events and miscellaneous
account events (such as assigning users to roles, assigning quotas to users and changing user
passwords). Customers must have access to these logs through self-service interfaces. Logs
should be provided by default for at least three months, and customers should have the ability
to export these logs for longer retention.
■ Provisioning and catalog action logging: Providers must offer logging of provisioning and
catalog actions. In this case, "provisioning" refers to both the provisioning (creation) and de-
provisioning (termination) of infrastructure elements. The log service must include logging
provisioning events for compute instances, logging provisioning events for storage volumes,
logging when changes are made to a customer's image catalog (such as adding, deleting or
updating an image) and logging when changes are made to a customer's template catalog

(such as adding, deleting or updating a template). Customers must have access to these logs
through self-service interfaces. Logs should be provided by default for at least three months,
and customers should have the ability to export these logs for longer retention.
■ Security configuration logging: Providers must offer logging of security configurations,
including changes in network ACLs and changes in firewall configurations. Customers must
have access to these logs through self-service interfaces. Logs should be provided by default
for at least three months, and customers should have the ability to export these logs for longer
retention.
Preferred
■ SDK library: Providers must offer a rich set of software development kit (SDKs) for three or
more of the leading programming languages: Java, .NET, PHP, Ruby, Node.js, Perl, PowerShell
and Python. These SDKs should at a minimum provide support for the core services: compute,
storage and networking. The SDK must also include documentation and code samples. It is
acceptable for the provider to support an open-source library instead of creating one
independently.
■ Historical performance monitoring: Providers must offer historical performance data of
services to customers. At a minimum, the provider must be able to support up to three months
of historical performance data with at most five-minute collection intervals. The provider can
offer this as an add-on service or require that the customer use cloud-based storage to store
the historical data. The service should also support standard charts to show historical
performance graphically and should support all standard metrics in the real-time performance
monitoring service. Finally, providers must offer customers the ability to export the performance
data for longer-term storage.
■ Customer-defined monitoring metrics: Providers must allow customers to extend or custom-
define performance-monitoring metrics that are not standard in the provider's performance-
monitoring service. In order to qualify for this criterion, a provider must allow customer-defined
metrics in at least the compute, storage and network services. Providers may require that a
monitoring agent be installed into the assets being monitored.
■ Predefined action events: Providers must offer a service whereby customers have the ability
to execute a specific cloud action (that is, call a service interface action) when a standard or
customer-defined performance threshold is passed or an event occurs. These scenarios tend to
manifest themselves in terms of elasticity when components of the infrastructure need to grow
or shrink dynamically based on load or performance.
■ Complex, multi-data-center templating: Providers must expand their basic templating service
to support multi-data-center deployments within or across geographic regions. As previously
mentioned, a template is a blueprint that allows a collection of different resources to be
provisioned together, such as compute instances, storage volumes, network elements,
monitoring configurations and security configurations. The customer must be able to create,
store and provision from templates, and the templates must be able to be built in the cloud
service without having to learn a configuration language. A template is not just the combination
of compute instance size with a particular image. It is a build manifest that provisions multiple

elements, and it must be able to include more than the specification of how to provision one or
more compute instances.
■ Community image catalog: Providers must maintain an image catalog of community-supplied
images that have been created by customers and that those customers have made publicly
available. Community images must be directly self-service accessible, without use of a third-
party service or tool.
■ Self-service, postprovisioning events: Providers must offer customers the self-service ability
to perform a user-specified action for both Windows and Linux when a certain provisioning or
configuration event takes place. One particularly useful event is a postprovisioning hook, where
the platform can automatically perform a user-specified action when the boot process is
finished on a newly provisioned compute instance. To qualify, providers must allow customers
to define an action to be taken upon the completion of provisioning of a compute instance.
■ Professional developers program: Providers must offer a professional developers program
that provides certification for both developers and applications. A professional developers
program validates a comprehensive set of skills that are necessary to develop applications
successfully by using the programmatic interfaces of the provider. Supporting and bolstering
such a development program will create an ecosystem of trusted third parties and individuals
that will increase the adoption of the cloud service.
Optional
■ Mobile SDK support: Providers must offer development SDKs for two of the following mobile
platforms: Android, iOS, Windows and JavaScript/HTML5. These SDKs should at a minimum
provide support for the core services: compute, storage and networking. The SDKs must also
include documentation and code samples. It is acceptable for the provider to support an open-
source library instead of creating one independently.
■ GUI-based network design/inventory mapping: Providers must provide customers the ability
to create infrastructure architecture with a GUI-based interface. In this scenario, customers can
drag and drop servers, storage and networks to build a logical design that automatically
deploys from the layout of the design. Providers can accomplish this through a native graphical
interface or through the import of a Microsoft Visio diagram.
■ GUI-based network architecture export: Providers must offer customers the ability to export
a graphical representation of their infrastructure architecture, which includes all servers, storage
and networks. Providers can export this into an image file (such as JPG or BMP), PDF,
Microsoft Visio or Microsoft PowerPoint format.
■ Configuration management capabilities as a service: Providers must offer configuration
management tools as a fully standardized service. Configuration management tools, such as
Puppet, Chef and SaltStack, have become increasingly popular. Although customers can build
their own configuration management at any provider, some customers prefer a service, and
providers should not require customers to operate or maintain configuration management
servers on their own. Nor should this offering be a managed service. Providers may wrap or
extend the configuration management tool with additional capabilities.

Gartner evaluation criteria_for_clou_security_networking

Gartner evaluation criteria_for_clou_security_networking

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Gartner evaluation criteria_for_clou_security_networking

Similar to Gartner evaluation criteria_for_clou_security_networking (20)

Recently uploaded

Recently uploaded (20)

Gartner evaluation criteria_for_clou_security_networking