Security Within a Virtualized Environment:Document Transcript
Security Within a
A New Layer in Layered Security
730 Glenridge Dr. • Suite 104
Atlanta, Georgia 30328 USA
The use of Information Technology can help organizations improve employee productivity, business pro-
cess automation and other functions. However, it can also create management, operational and security
challenges. These problems and the associated costs can be substantial in large enterprise or service
provider data centers that employ large numbers of computer servers.
A potential solution to these problems is “server virtualization”, which can allow the workloads of 20 or
more servers to be consolidated onto a single physical machine.
However, while virtualized server configurations provide many potential benefits, they create unique
network security issues that are not addressed by conventional security products. This can result in the
spread of computer viruses, theft of data, denial of service, regulatory compliance conflicts or other con-
sequences within the virtualized environment.
This paper will outline the special issues and solution criteria required to deploy effective security in a
virtualized server environment. It will also present a discussion of new patent-pending technology from
Reflex Security that provides a solution to these challenges.
The Business Case for Server Virtualization
Traditionally, as computing needs increase within an organization, additional physical computers are
installed to handle incremental applications and processing workloads. However, dedicating machines to
specific computing applications often causes a proliferation of hundreds or thousands of physical com-
puters. The resulting “server farm” sprawl leaves many enterprises and service providers saddled with a
large operational, logistical and total cost of ownership (TCO) burden. For example, each physical server
incurs incremental demands for data center floor space, electricity, cooling, networking, administration
and other resources.
This computing model also tends to waste capital resources, because application requirements and
server resources are almost never a perfect match. In practice, servers typically have substantial excess
capacity. According to Gartner, most x86-based servers in a 1 : 1 application / server configuration only
use 10% of their CPU capacity . Similarly, fixed-capacity server resources such as RAM, disk, network-
ing and power supplies are typically configured to handle peak loads that may occur infrequently. All this
idle capacity means many organizations purchase and maintain computing resources that are redundant
These challenges have created a need for technologies that help organizations consolidate and optimize
their server resources. A potential solution to these computing problems is “server virtualization”. Server
virtualization uses specially-designed software to create multiple “virtual machines” and other virtualized
resources that run simultaneously on, and share the resources of, a single physical machine (a host). By
allowing virtual machines to share host computer resources, virtualized configurations can make more
efficient use of existing computing capacity and consolidate the number of physical computers that must
be purchased, installed and maintained. This can help organizations reduce management, logistical and
operational costs by as much as 64% .
In a virtualized server environment, a single physical host machine can simultaneously provide computing
resources for twenty or more resident virtual machines. Since virtual machines are logically isolated from
each other, virtualization makes it possible for them to have heterogeneous operating systems, applica-
tions, access policies and other attributes. As an example, these capabilities could allow an organization
to replace a server farm of 100 physical Windows and Linux servers with perhaps five host machines,
each running a mixed set of Windows and Linux virtual machines.
For more information, visit www.reflexsecurity.com or call 888-872-7555
The benefits of virtualization are striking and well-documented. A recent Microsoft study found the 22-25 days re-
quired to provision a physical server could be reduced to one day with a virtualized approach. Microsoft found these
and other virtualization benefits would yield a 30% cost savings over three years. A VMware case study found similar
results; virtualization cut server provisioning time from 14 days to 30 minutes, reduced the number of physical servers
by 90% and increased average server CPU utilization by more than 500%.
These advantages are fueling rapid advances in virtualized deployments. IDC estimates 45% of new servers pur-
chased in 2006 will be virtualized and most will run business-critical workloads . Further, IDC estimates the virtualiza-
tion market will reach nearly $15 billion worldwide by 2009 and virtual servers will be deployed by approximately 75%
of companies with 500 or more employees.
Virtualization can provide a variety of solutions and benefits. For example:
• Server Consolidation: Virtual machines can consolidate the computing tasks of several under-utilized servers
onto a smaller number of physical machines. Benefits of this may include hardware, administrative and opera-
tional cost savings.
• Multiple OS Support: Virtual machines can allow multiple or incompatible operating systems to be run on a
single platform simultaneously.
• Legacy Application Support: Virtual machines can provide an effective platform to run legacy operating sys-
tems and/or applications that may be incompatible with modern hardware and software. This can help organiza-
tions extend the service life of valued applications and avoid costly upgrade and migration costs.
• “Sandboxing” Untrusted Applications: Virtual machines can create a secure, isolated environment (a “sand-
box”) that confines the impact of untrusted, malfunctioning or potentially compromised applications. For example,
a “sandboxed” web browser could allow access to the Internet while quarantining potentially malicious web con-
tent from other applications running on the same physical machine. A sandboxed machine could also be used to
evaluate the behavior of a computer virus or software malfunction.
• Resource / QoS Provisioning: Virtual machines can be used to create operating systems or environments with
a pre-defined set of resources and/or resource limits. This can be useful in a variety of shared or Service Level
Agreement (SLA) environments.
• Resource Simulation: Virtual machines can be used to simulate computer equipment that is not physically avail-
able, such as a network switch, a SCSI drive, a network of computers, etc. This can be useful for testing purposes
or to provide cost-efficient functionality.
• Software “Appliance” Support: Virtual machines can be used to deploy consistent, packaged “software appli-
ances” that can be easily installed on diverse hardware platforms. This avoids the complexities and compromises
typically required to install and operate an application on distinct platforms.
Virtualization: A New Layer in a Layered Security Model
Mounting pressure from computer security threats, regulatory compliance requirements and other issues have made
network and data asset protection a paramount concern for IT departments. Traditionally, organizations have created
a gauntlet of defensive layers to protect their networks.
For example, a firewall might be deployed at a perimeter gateway to limit access to via certain ports, hosts, etc. Com-
munications that made it through the firewall might then be inspected by an intrusion prevention system, an anti-virus
engine and other security technologies. As standalone measures, these layers would create formidable obstacles for
external threats, though any intruder that made it past these screens would find the internal network relatively soft and
Since the concept of the network perimeter has been eroded by the advent of wireless networks, mobile devices,
VPNs and other technologies, organizations often find it necessary to deploy additional layers of security on internal
network segments. In these configurations, firewalls, IPS, access control and other security tools to provide en-
hanced protection to protect critical equipment and data.
Just as the concept of the network perimeter
was altered by new technologies, the con-
cept of a network endpoint and endpoint se-
curity is being altered by server virtualization.
Where at one time a machine might have
been viewed as an indivisible “atom” on the
corporate network, virtualization has opened
up an entire universe of invisible “subpar-
ticle” endpoints that can live inside the host
machine. While the new realities brought
on by virtualization have created remarkable
opportunities for performance and efficiency
improvements, they have also changed the
rules for network security professionals.
While traditional perimeter and internal
security devices continue to be effective
measures for securing conventional network
architectures, they are not capable of fully
protecting virtualized resources which are
deployed within such a network. A host ma-
chine with “N” associated virtual resources
creates new layers of network infrastructure
which, in turn, require new layers of security controls.
Protecting the Virtualized Environment
At a broad level, hosts and virtualized components require the same physical and network security precautions
as any critical, non-virtualized IT resource. However, virtual environments have unique attributes and incremental
security challenges that are not addressed by traditional security solutions.
In a virtualized environment, security threats may originate externally or from within the host machine. Of these,
intra-host threats present the toughest challenges for legacy security solutions.
When a host or virtual machine communicates with resources outside the host (such as an Internet site or a ma-
chine on the local network) it may be exposed to hostile content or users. As with any such connection, this can
result in an intrusion, malware infection or other unwanted result. Externally-based threats can be controlled with
devices installed outside or within the host machine.
Since externally-based threats transit network segments outside the host machine, they can typically be routed
through layered firewall, anti-virus and/or intrusion detection system defenses. This provides an opportunity for
these or related devices to secure the traffic and protect the host/virtual machine, as they would any physical ma-
chine on the local area network. Depending on the proximity of anticipated threats to the host machine, external
protection systems can be comprised of a single network-wide control point, separate control points in front of
each critical host, or variations in between.
Externally-based threats can also be controlled with a virtualized security system deployed inside a host machine.
Because this solution is typically executed in software, it eliminates the need for additional hardware or network
reconfiguration. It also facilitates granular control that can be specific to an individual host machine or virtual
server. As a result, the virtualized approach may have significant security, cost, configuration and deployment
For more information, visit www.reflexsecurity.com or call 888-872-7555
In contrast to external threats, intra-host threat vectors are difficult or impossible to secure with traditional security
tools, because they typically use virtual LAN infrastructure and/or other channels unseen outside the host. As a
result, conventional firewalls and other security tools outside the host can’t inspect or control the traffic. This cre-
ates an unmonitored, unprotected security hole that may expose virtual machines to unauthorized or undesirable
communication originating from other virtual machines.
To illustrate, a physical host machine may have multiple virtual servers, applications and owner/administrators.
Because the owner and/or applications on one virtual machine may pose a threat to owners and/or applications
on adjacent virtual machines, there is a need to protect virtual machines from their most immediate neighbors.
Failure to effectively protect virtual machines from each other can result in the spread of computer viruses, theft
of data, denial of service, regulatory compliance conflicts or other consequences.
Intra-host threats may come from various vectors, including:
• Legitimate Intra-Host Communications: Virtual machines may have a legitimate need to communi-
cate or share data with each other. If these communications are not monitored or controlled, they may
enable the spread of viruses, theft of data or other issues. For example, a virtual machine infected with a
computer worm may spread the worm to other virtual machines within the host when it communicates via
an unprotected intra-host virtual LAN.
• Unauthorized Intra-Host Communications: Although virtualization technology can give virtual ma-
chines a logical partition level comparable to the “air gap” separation between physical machines, this
software-defined barrier can potentially be breached by a penetration of the host platform or other un-
expected mechanisms. This may create a potential “back door” entry point for intruders or other hostile
• Intra-Host Denial of Service: A malicious or infected virtual machine could potentially inflict a Denial
of Service attack on other local virtual machines by consuming shared host and/or virtual LAN resources.
For example, a virtual machine might flood the virtual LAN with malformed or high volume traffic that pre-
cludes legitimate access by other virtual machines.
• Intra-Host Spyware Applications: If the virtual LAN or host environment is compromised with spy-
ware technology, data sent from virtual machines could potentially be intercepted and made available to
an unauthorized third party. Examples include technology that intercepts keyboard inputs, video output,
unencrypted memory images, unencrypted IP communications, file transfers, etc.
Special Requirements for Virtualized Security Challenges
The emerging prevalence of virtualized computing environments creates a need for network security enhance-
ments that address the special needs of these platforms.
Within the context of a host machine supporting multiple virtualized resources, an effective security solution
should meet several criteria, including:
• Awareness of the Virtual Environment: Effective security requires administrator awareness of the
active components, services and communication flows. This information can be difficult to discover in the
physical network world; it is even harder to discern in a virtualized environment. In an intangible, invisible
virtual network, administrators can’t rely on a “walk-by” inspection to identify legitimate devices, rogues,
flows or other installed elements. Therefore, a security solution should have the capability to accurately
detect and profile the virtual environment and provide administrators with timely and actionable configura-
• Complete, Multi-Function Security: The security solution should protect virtual machines from a full
spectrum of threats. In most cases, a solution that integrates many security functions will be more ef-
ficient and practical to deploy and manage in a virtualized environment.
• Wire-Speed Performance: The virtualized environment typically facilitates high-speed, efficient commu-
nications between virtual machines. A security solution should be able to protect virtual resources without
introducing unreasonable latencies.
• Minimize Application Performance Degradation: Because virtual servers share host resources, a security
solution needs to make efficient use of CPU cycles, memory and other finite host capabilities. A security solution
should provide protection without unreasonably impacting or degrading the performance of the applications it is
intended to protect.
• Ease of Deployment and Administration: A security solution should be easy to deploy and facilitate efficient
• Compatibility and Interoperability: A security solution should be able to interoperate with other networking and
security technologies in the surrounding virtual and physical environments.
The Reflex Security VSATM Solution
Reflex Security’s patent-pending Reflex VSATM creates a virtualized security appliance and virtual security infra-
structure inside a host machine. This allows organizations to provide appropriate security to virtual machines that
would otherwise be exposed to risk.
The Reflex VSA resides within a host machine and applies multiple network security and policy enforcement con-
trols to protect virtual machines, virtualized networks and the underlying host and virtualization platform. If need-
ed, it can also safeguard communications between virtual components and resources outside the host machine.
The resulting functionality provides more a complete security perimeter around and between virtual machines
and reduces the risk of virtual machine intrusion, infection or other consequences. Depending on requirements,
security services provided by Reflex VSA may include firewall, intrusion detection, intrusion prevention, anti-virus,
anti-spyware, denial of service mitigation, network quarantine and network discovery.
Reflex VSA operates from within a virtual machine that replicates the operational attributes and interfaces of a
physical network security appliance and supports the hardened Linux OS and proprietary Reflex Security soft-
Reflex VSA can be configured to operate as an in-band virtualized Layer 2 network bridge or it can be deployed
as a transparent, out-of-band monitoring and control device. In the latter configuration, Reflex VSA would typi-
cally collect data via a mirrored port on a virtualized switch and relay traffic control instructions to the switch or
other devices via 802.1x or comparable protocols. Various Reflex VSA configurations are illustrated in Section 5
of this document.
One or more reflex VSA virtual security appliances can be deployed within a virtualized environment. Depending
on requirements, a Reflex VSA virtual appliance could be placed in front of key virtual servers, between virtual-
ized LAN segments and/or between virtual servers and the physical world outside the host. Multiple Reflex VSA
appliances can also be configured behind a virtualized load balancer to address performance or high-availability
As Reflex VSA encounters network traffic within the host, it performs deep packet inspection and content analysis
to identify threats or other unwanted elements (such as intrusion attempts, viruses, spyware and related items)
within the data stream. When Reflex VSA encounters threats or other unwanted content, it can block the content,
issue alerts and/or initiate other defined actions. As noted above, Reflex VSA has the capability to block most
threats and content directly; it can also interoperate with other elements of the network environment to block traf-
fic, quarantine unauthorized or infected virtual machines, or execute other actions.
Key Reflex VSA Advantages
Complete Awareness of the Virtual Environment
Reflex VSA security begins with a complete awareness of the surrounding virtual environment. Reflex VSA de-
ploys the Reflex Network DiscoveryTM application, which dynamically profiles network state, assets, services and
communication flows and provides essential context for security threats and attacks. It has the capability to report
on these attributes, create visual representations and/or initiate appropriate alerts.
This provides administrators with a complete, accurate understanding of their virtualized network environment
and facilities more effective resource protection.
Complete Protection for the Virtual Environment
The Reflex VSA virtual security appliance delivers the Tolly-Certified Reflex ThreatIQTM (Threat Inspection and
Quarantine) system, which provides superior network protection via an integrated suite of intrusion prevention,
anti-virus, anti-spyware, network discovery and network policy enforcement components.
Within Reflex ThreatIQ, the flagship Reflex IPSTM solution delivers patent-pending intrusion prevention (IPS) and
unified threat management (UTM) technologies for enterprise and Managed Security Services Provider (MSSP)
applications. It provides a flexible, easy-to-use approach to maintain network security and regulatory compliance.
Reflex IPS employs deep packet inspection and multiple threat interdiction modules. Reflex IPS analyzes net-
work traffic with a combination of signature and anomaly-based algorithms and quickly adapts and responds to
new security threats. In addition, the Reflex ThreatIQ system employs a unique access permission engine that
filters unauthorized user/host traffic before it reaches a targeted system. The combined capabilities allow Reflex
IPS to provide comprehensive protection.
In addition, the integrated Reflex Network DefenderTM policy enforcement module works in conjunction with Reflex
Network Discovery to quarantine infected, disruptive or unauthorized virtual devices that could jeopardize security
or compliance standards.
Granular Security Control
Reflex VSA’s unique deployment points in front of and between virtual servers allow it to provide superior security
and more granular control as compared to systems that reside outside the host or solely within a virtual machine.
For example, Reflex VSA has the capability to mitigate intra-host DoS attacks or quarantine virtual devices, nei-
ther of which could be readily accomplished from conventional vantage points.
Wire-Speed Performance and High-Availability
Reflex VSA’s highly-efficient processing capabilities provide wire-speed performance in demanding virtual envi-
ronments. Reflex VSA also supports multiple fail-open and fail-closed options. Depending on requirements, it
can be configured in a variety of in-band, out-of-band and load-balanced configurations.
The Reflex VSA is deployed as a pre-configured, software-based security appliance. This allows it to be quickly
deployed on any platform that supports a virtual Linux machine and eliminates most installation and configuration
tasks. This simplifies provisioning and deployment issues in large data center environments.
High-performance Reflex VSA applications may also be configured with a hardware accelerator card that is com-
patible with standards-based x86 hardware architectures.
Advanced Reporting and Administrative Capabilities
Reflex VSA is administered through the Reflex Command Center (RCC) management console, which can be
deployed on a virtual server or external physical appliance. Reflex RCC provides an extensive, out-of-the box
reporting capability that supplies intuitive and actionable security information.
The RCC correlates and aggregates event data and presents a flexible array of real-time, interactive, 3-D graphs
and integrated tabular reports. On the fly filtering, threat classification and alert triggering capabilities make it
easy to spot intrusion attempts and distinguish high risk events from simple background traffic. The RCC also
supplies abundant historical reporting to support forensic analysis and compliance documentation.
More than a passive observer, the Reflex Command Center reporting system also allows immediate “right click”
access to essential attack information and control resources. Suspect activity warnings are accompanied with
CERT, BugTraq and Reflex interpretations that help administrators make informed security decisions. Further, the
RCC right-click capabilities also allow users to define blocking policies and special event triggers without leaving
the reporting interface.
Reflex’s highly integrated information and control tools facilitate effective, easy-to-use security in the virtualized
environment. The RCC console also provides integrated control for various network security applications de-
ployed on one or more Reflex VSA virtual appliances, as well as integrated controls for select third-party devices.
Economical; Low Total Cost of Ownership (TCO)
Reflex VSA offers an economical and efficient solution that avoids or reduces the incremental hardware, adminis-
trative and operational costs associated with physical data center security products.
The patent-pending Reflex VSA delivers superior security for virtualized server environments. A distinctive com-
bination of performance, security, manageability and TCO advantages make it a credible candidate for protecting
virtualized computing resources, maintaining regulatory compliance and reducing operational expenses.
Sample Reflex VSA Deployment Configurations
Sample Configuration 1: Virtual network with Reflex VSA deployed in-band.
Sample Configuration 1 demonstrates a virtual network of eight servers that exists within a host machine. The
Reflex VSA virtualized security appliance is deployed in-band and provides multi-functional security (firewall,
intrusion prevention, etc.) to protect the virtualized network from threats originating outside the host.
Sample Configuration 2: Virtual network with Reflex VSA deployed out-of-band
Sample Configuration 2 demonstrates a virtual network of five servers that exists within a host machine. The virtual
servers are connected to one virtual switch. Reflex VSA is deployed out-of-band and listens to a mirrored port on the
virtual switch. In this configuration, Reflex VSA analyzes traffic on the virtual network traffic and provides complete
intrusion detection (IDS) functionality. It could also be configured to provide additional functions such as network
discovery or quarantine.
For more information, visit www.reflexsecurity.com or call 888-872-7555
Sample Configuration 3: Multi-subnet virtual network with Reflex VSA deployed out-of-band
Sample Configuration 3 demonstrates one virtual network with three virtual subnets. Each virtual subnet is con-
nected to a virtual switch. In this example, Reflex VSA is configured to run out-of-band and monitors traffic on a
mirrored port on the virtual switch. The Reflex VSA analyzes all of the subnets on the virtual switch and provides
complete intrusion detection (IDS) functionality. It could also be configured to provide additional functions such as
network discovery or quarantine.
For more information, visit www.reflexsecurity.com or call 888-872-7555
Sample Configuration 4: Virtual network with hybrid, multiple Reflex VSA configuration
Sample Configuration 4 demonstrates a virtual network of seven servers that exists within a host machine. The virtual-
ized network is segmented into two zones. Zone 1 (Figure 4, bottom left) consists of four virtual servers which are the
critical servers in the network. Zone 2 (Figure 1, bottom right) consists of three virtual servers that are less critical
than Zone 1.
Two Reflex VSA appliances provide intrusion prevention (IPS) and firewall protection. One Reflex VSA protects the
virtual network from external threats. The second Reflex VSA protects the critical Zone 1 segment from threats origi-
nating from Zone 2. The third Reflex VSA is configured to run out-of-band and it is connected to a mirrored port on the
virtual switch in Zone 2. The Reflex VSA analyzes the entire virtual network traffic on the switch that is running in the
less critical Zone 2 and provides complete intrusion detection (IDS) functionality. The hybrid configuration provides
maximum information and protection.
This configuration could reflect an application where virtualized resources are shared by a diverse mix of users, virtual
servers, and policy requirements.
A physical computer. As used in this document, a Host Machine (“Host”) is defined as the equipment that provides the physi-
cal environment and computing resources used to support one or more Virtual Machine or network environments.
Virtual Load Balancer:
A virtualized device that assigns workloads to a set of virtual devices operating within a Virtual Network environment so that
computing resource usage is optimized.
A virtualized computing environment running on a Host Machine platform, on which a guest operating system and associated
application software can run. Multiple Virtual Machines can operate on a Host Machine concurrently. A Virtual Machine is
typically defined and implemented in software rather than hardware, though it may also be possible to define a Virtual environ-
ment via hardware.
Virtual Network (also Virtual LAN):
A virtualized local area network infrastructure running on a Host Machine platform, on which a virtualized networked commu-
nication environment that includes virtual switches, segments, network interface cards or other elements can run. A Virtual
Network is typically defined and implemented in software rather than hardware, though it may also be possible to define a
Virtual Network via hardware.
Virtual Network Device:
A virtualized representation of the functionality and interface provided by a physical network component such as a switch,
router, network interface card or other element.
As used in this document, a computer server deployed within a Virtual Machine.
Network security technology designed to identify, thwart and/or eliminate “spyware” software programs. Spyware is a broad
category of malicious software intended to intercept or take partial control of a computer’s operation without the user’s in-
formed consent, typically for the benefit of a third party.
Network security technology designed to identify, thwart and/or eliminate computer viruses and other malicious software (mal-
Denial of Service Mitigation: (also, DoS or DDoS Mitigation)
Network security technology designed to identify, thwart and/or eliminate attacks on a computer system or network that denies
user or application access to services, such as network connectivity or computational capacity, by consuming the bandwidth of
the victim network or overloading the computational resources of a victim system.
Network security technology designed to limit access between two or more networks. Normally, a Firewall is deployed between
a trusted, protected private network and an untrusted public network.
Intrusion detection (Also “IDS”) :
Network security technology designed to gather and analyze information from various areas within a computer or a network
to identify possible security breaches emanating from external or internal sources. When a breach attempt is discovered, the
intrusion detection system can log the activity and/or issue an alert. Typically a subset of Intrusion Prevention.
Intrusion Prevention (Also “NIPS”):
Network security technology designed to gather and analyze information from various areas within a computer or a network to
identify possible security breaches emanating from external or internal sources. When a breach attempt is discovered, the in-
trusion prevention system can block the attack or initiate other appropriate actions. Typically a superset of Intrusion Detection.
Network Access Control:
Network security technology designed to ensure appropriate compliance with defined network, security and access policies.
Network infrastructure technology designed to identify and profile the presence, configuration and activity of network assets.
For more information, visit www.reflexsecurity.com or call 888-872-7555