IBM zEnterprise System - Network Security

IBM zEnterprise System - Network Security
July 2010

IBM zEnterprise System - Network
Security

July 2010

Table of Contents
Abstract............................................................................................................................. 3
Network Firewall Introduction............................................................................................ 3
IBM zEnterprise System Network Security Overview........................................................ 6
zEnterprise Security Framework .................................................................................... 8
Physical Infrastructure ................................................................................................... 9
Logical Security ............................................................................................................. 9
IEDN Workloads and Security Zones .......................................................................... 12
External Network Access............................................................................................. 13
Exploiting External Firewalls ........................................................................................... 14
Summary ......................................................................................................................... 17
Acknowledgments and Contributions............................................................................. 18
About the Authors: .......................................................................................................... 19

2

July 2010

Abstract
This paper is provided for both IBM and IBM customers who have an interest in the
IBM zEnterprise™ System “Network Security” topic. It is assumed that readers
already have a basic background in both the zEnterprise System and fundamental
Network Security concepts. The primary purpose of this paper is to describe why,
for many customers, traditional network firewalls will not be required for their
network traffic associated with multi-tier application workloads within a zEnterprise
Ensemble. This subject is organized into the following three topics:

1. Network Firewall Introduction
General introduction and overview of the concepts of network security zones and
how firewalls can be used to control security zone crossings for network traffic
related to multi-tier workloads
2. IBM zEnterprise System - Network Security Overview
Overview of the zEnterprise System internal (intraensemble) data network and how
the IBM zEnterprise Unified Resource Manager (zManager) provides innovative
network management features such as network virtualization, access control and
network isolation to protect the heterogeneous platforms within the Ensemble
3. Exploiting External Firewalls
How (when required for compliance, mandated standards, and / or other business
imperatives) customers can continue to exploit their existing external firewalls to
provide the same level of protection for resources within the Ensemble (i.e. when
network traffic crosses intraensemble security zones).

Network Firewall Introduction
One of the core security technologies common in most, if not all, network-attached
computing environments, large or small, is the firewall. Firewalls take many shapes
and forms, from host-based solutions targeting the personal computer as an
integrated security suite to large dedicated purpose built appliance hardware
protecting high volume traffic at the network’s edge. There are hundreds of
variations on firewall solutions and their uses, each with their own value add or
benefit in a particular situation, but there is one clear requirement that firewalls
bring to the table no matter what size or how many bells and whistles are present.
Firewalls must have the ability to block access or connectivity that is deemed as
unauthorized, while still letting authorized traffic reach the intended target system or
application.

In its simplest form the firewall acts as a basic packet filter, looking at each packet
and checking a set of rules or policy to determine which packets are granted access,
passing through the firewall, and which packets are denied. This basic packet
filtering capability can be found in both network firewalls (either hardware or
software based) and host firewalls. Host solutions, like that found in IBM’s
Proventia® Server for Linux® on IBM System z® or z/OS® Communications IP Filters,

3

July 2010

run within the server image and are used to protect network traffic flowing into and
out of the server. These types of host solutions are targeted at self-protection.
Another firewall solution that might be found on the host is an application firewall,
designed to protect a particular application or server, such as a Web server, FTP,
database, Telnet, etc. from unauthorized or malicious attack.

Security Zone 1 Security Zone 2 Security Zone 3
Publicly Available
Application
Private
External
Network
Network Perimeter
Network
F F
i i
r r
e e
w w
a a
l DMZ l
l l

Figure 1: Basic DMZ

It is the job of a network firewall to ensure that only the intended traffic passes
though the firewall from one security zone to another security zone. One of the
classic deployments of a network firewall is the Demilitarized Zone, or DMZ, which
can be defined as a perimeter network, located between an external network and a
private or protected network, that provides isolation for a publicly available service
in the perimeter network, with the ultimate goal of protecting the private network,
data and services in the enterprise. An example of a DMZ can be seen in figure 1.
The yellow perimeter network, identified in security zone 2 might include a web
server, which is available to the Internet or external network shown in the red
security zone 1. There is no path directly from the untrusted red external network to
the trusted green private network found in security zone 3. In this case the DMZ,
with its two firewalls that bound the perimeter network, isolates the enterprise
resources from the threats of the Internet.

As virtualization continues to drive consolidation on all platforms, the network in
these environments will need to be evaluated for compliance with the corporate
security policy as well as with possible regulatory requirements. Different network
architectures might need to be explored including the use of proxy servers, virtual
LANs, Network Address Translation (NAT), and other technologies in conjunction
with a basic packet filter or stateful firewalls. In a distributed environment it might
be simpler to segregate security zones, deploying each zone on isolated, inexpensive

4

July 2010

hardware. As you explore the benefits of deploying diverse workloads on System z
and the advantages the platform has to offer, it becomes necessary to reexamine the
goals, intentions and requirements of network security in the light of this
environment. It is always important to question and revise security decisions as
environments and threats evolve. Reevaluating the placement and need for firewalls
is no different. They need to be placed where they make sense and were they
provide value.

System z is now an integration of new multiple architectural technologies with the
introduction of the IBM zEnterprise System. It is comprised of the IBM zEnterprise
196 (z196), the IBM zEnterprise BladeCenter® Extension (zBX) Model 002, the
Unified Resource Manager, and optimizers and IBM blades. The heterogeneous
resources of the zEnterprise System are managed and virtualized through the Unified
Resource Manager as a single pool of resources, providing integrated system and
workload management across this multisystem, multitier, multiarchitecture
environment.

One function of the Unified Resource Manager is network virtualization
management, including the provisioning of a secure private data network called the
Intraensemble Data Network (IEDN). The IEDN is designed to ensure the safety,
security and isolation of network traffic into and out of applications running within
this environment. It is important to ensure the right technologies are used in
network flows to minimize latency while providing the required level of security and
isolation of intellectual property and mission critical applications and data.
Understanding the level of security required and the isolation provided by the
network virtualization management function of the Unified Resource Manager in
collaboration with other firmware elements of the IBM zEnterprise System will help
clients determine what, if any, additional security devices, like firewalls, are required
in their enterprise solutions.

As the end-to-end solutions that clients build or consolidate on a System z
Enterprise System are explored, the security requirements of that solution must be
understood. Careful consideration should be given to the security requirements.
These requirements might be born out of a security policy that identifies various
security zones and transitions, or it might be based on regulatory requirements such
as PCI DSS (Payment Card Industry Data Security Standard). Either way, the
requirements are real and must be addressed in a way that satisfies the client and in
many cases the auditor. The determination of the requirement for, and placement
of, network firewalls needs to be reevaluated with a new understanding of the
security and isolation inherently provided by the IBM zEnterprise System as an
integrated and optimized multiplatform environment.

5

July 2010

IBM zEnterprise System Network Security Overview
This section provides an overview of the zEnterprise physical infrastructure
associated with network communications. Key concepts such as the node, how a
cluster of nodes can be formed into an “Ensemble”, and finally how network
communication is provided for within the Ensemble are also introduced in this
section. The resources within the ensemble are managed across heterogeneous
platforms by an innovative zEnterprise function called “Unified Resource Manager”.
Unified Resource Manager will orchestrate various forms of platform management
and virtualization by interacting with various elements of platform firmware and
hardware.

Figure 2: System zEnterprise

Figure 2 illustrates a z196 with an attached zBX. A z196 can support up to four
locally attached racks in a zBX. Together the CPC and the optional zBX are
considered a single logical node. Individual nodes (up to eight) can be grouped into
an Ensemble which is defined at the HMC. The Unified Resource Manager (HMC)
can then manage the resources of the entire Ensemble by communicating with the
Support Element (SE) of each node.

The zEnterprise provides a dedicated system data network. This data network spans
all nodes within the Ensemble reaching all servers within each node across the
entire ensemble. The security attributes and considerations associated with
zEnterprise network communications is the primary focus of this document.

6

July 2010

Figure 3: System zEnterprise Node

Figure 3 illustrates a zEnterprise node and the various networks associated with the
node.

Within the zEnterprise node there are two new “internal networks” which are both
private and dedicated to specific communication purposes. The customer managed
“external” networks (data and management) are not changed. zCPC access to the
customer’s external data network is through an OSA configured as OSD. The access
to the new internal networks is controlled by the Unified Resource Manager
(zManager).

The zEnterprise provides the following two new internal networks:
1. Intranode management network (INMN) – a 1GbE network used by zEnterprise
Unified Resource Management firmware to communicate with the various virtual
servers and hypervisors within the node. From a zCPC z this network is accessed
by an OSA configured as an OSM CHPID. It is restricted to zManager related
functions and therefore can not be used or accessed by customer management
applications.

2. Intraensemble data network (IEDN) – a 10 GbE flat layer 2 network that spans
across all physical and virtual resources of all of the nodes within the ensemble
and is used by customer application workloads fully contained within the ensemble
to provide normal network communications for these workloads. From a zCPC
this network is accessed by an OSA configured as an OSX CHPID. Layer 3 IP
routing is not required to communicate with resources within the ensemble.

7

July 2010

This security and access control attributes related to the intra-ensemble data network
is the underlying topic of this paper.

zEnterprise Security Framework
The industry leading system security related features of the zEnterprise System are
achieved by providing a security framework that spans multiple tiers of the
zEnterprise platform. This multiple layer security model is very similar to previous
System z platforms, but it is enhanced with Unified Resource Manager network
related functions associated with the IEDN. The following figure provides an
overview of the multi-tier security model.

The IBM Security Framework
Security Governance, Risk Management
and Compliance
Security Governance, Risk Management
and Compliance

People and Identity

Data and Information

Application and Process

Network, Server, and End-point

Physical Infrastructure

Common Policy, Event Handling and Reporting

Professional Managed Hardware
Services Services & Software

Figure 4: IBM Security Framework

Figure 4 illustrates the existing IBM security framework and how this framework is
preserved and enhanced for zEnterprise. All security functions provided at the
“Application and Process” layer and above are not affected by zEnterprise and will
still be leveraged and used by customers without change. However, the lower two
layers “Physical Infrastructure” and “Network, Server and End-point” are affected
and enhanced by the zEnterprise environment. Both layers require closer
examination.

8

July 2010

Physical Infrastructure
The physical security provided by the customer’s secure system environment will
continue to apply to zEnterprise. This typically consists of aspects such as:

Lock and key areas (i.e. badge access to the physical systems lab and restricted
areas)
Locked systems and physical infrastructure (locked covers and access panels)
Employee access control (IDs and passwords) to system administrative functions
at the system consoles, HMC and SE
This existing physical security is enhanced by the following new networking
hardware features offered in the zEnterprise System:

Dedicated and private networking hardware equipment which reside within the
frames of the zEnterprise System and zBX under locked covers that reduces the
number of typical physical network hops reducing the scope of security
vulnerability
All administration and management interfaces for the new networking hardware
equipment are provided exclusively via the Unified Resource Manager (HMC)
New Unified Resource Manager (HMC) administrative roles and passwords for
secure access to the HMC for network virtualization configuration settings
New OSA-Express3 OSM and OSX CHPID types for the Intranode Management
Network and Intraensemble Data Network which contain new system Unified
Resource Manager firmware that provide secure access control to the internal
networks (which cannot be defined on the same physical CHPIDs/ports used as
OSD connections to access the customer managed external data networks)
An OSX CHPIDs identifies and verifies the physical switch to which it is
connected. If the switch is not the expected/supported zBX TOR switch, then an
alert is raised, and a Call Home event is generated.

Logical Security
The next layer in the framework describes security features related to the network,
server and End-point. System z supports many security features related to this layer
including:

Network security – Identification, authentication, and encryption using TLS/SSL
SSH and IPSec, network isolation and access control using IP Filters, Firewalls,
VLANs, and other technologies.
Sserver and end-point security – Operating System and middleware/application
identification, authentication and access controls, security managers such as
RACF, administrative roles and passwords, Operating System specific security
features, zVM security features, I/O configuration (e.g. using IOCDS NOTPART
in device candidate list or HCD) security features, logging, and other capabilities.

9

July 2010

The existing features (listed above) continue to be available and can be deployed
within a zEnterprise system. However, zEnterprise also introduces several key
advanced network security features for the IEDN in support of the new
heterogeneous multi-tier, multiplatform, virtualized workloads that can be deployed
in a zEnterprise ensemble.

These new capabilities are provided by the Network Virtualization Management
(NVM) component of the Unified Resource Manager and include the following
network virtualization and isolation features:

The ability to define multiple distinct virtual networks in the IEDN with platform
enforced access controls for all network access points in the IEDN. These virtual
networks can be viewed as distinct security zones and exploit VLAN technology
for strict isolation of these networks across a common shared physical network
fabric (IEDN). The OSA-Express3 OSX CHPID also includes the OSA ISOLATE
function (assures virtual server isolation for shared OSA).
The ability to associate virtual servers and optimizers within an ensemble with
one or more virtual networks. All virtual servers and optimizers must be explicitly
associated with a virtual network in order to access the IEDN.
When combined, these two features provide administrators the ability to deploy
workloads with distinct security and isolation requirements into zEnterprise nodes
within the ensemble while retaining complete network separation of the servers
comprising each workload in the IEDN.

The following figure provides a summary of the previous topics illustrating the
network access control system architecture provided by the zEnterprise Unified
Resource Manager.

10

July 2010

Figure 5: zEnterprise Network Access Control

As virtual networks are defined and provisioned, the Unified Resource Manager will
push all relevant network configuration information to the virtual switch (hypervisor)
and physical switches within each node of the ensemble. These virtual and physical
switches within the Ensemble will then serve as the access control points for the
IEDN. All ensemble network traffic must pass through one or more of these
applicable network access control points.

Operating Systems that are to be loaded into virtual servers must coordinate their
network VLAN configurations with the Unified Resource Manager (VLAN
configuration). If the Operating System attempts to use to a virtual network to which
it does not have access to (authorization) it will fail to connect.

The NVM component of the Unified Resource Manager also supports the following
additional network security features for the IEDN:

Access controls are provided for the enablement of each external port on the zBX
TOR switch. Using the Unified Resource Manager an administrator must enable
physical TOR switch ports by defining the specific VLANs (and MAC addresses
for access to zBX system resources by servers that are not part of the ensemble)
that are to be granted access to the TOR switch (ports).

11

July 2010

The Unified Resource Manager controls all dynamic MAC address generation by
assigning a MAC address prefix to all hypervisors and virtual switches (Note:
OSX is also considered a virtual switch of the PR/SM™ hypervisor). This central
configuration approach eliminates MAC address conflicts and unauthorized
virtual MAC generation.

IEDN Workloads and Security Zones
Groups of related virtual servers can be isolated into smaller networks by defining
multiple virtual networks and then restricting virtual servers to specific virtual
networks. This isolation can be based on workload, line of business, or other related
security criteria that customers define. Virtual servers can be System z Logical
Partitions (LPARs), guests virtualized by the various hypervisors within the
Ensemble or optimizers. Virtual networks have no physical boundaries within the
IEDN; any virtual server within the IEDN can be connected to an IEDN virtual
network.

The following figure illustrates how virtual servers can be isolated by deploying
multiple virtual networks.

Figure 6: Multiple Virtual Networks

12

July 2010

Within the IEDN security zones can be viewed as virtual networks. When a new
zone is required, the user can define a new virtual network and then grant access to
the new network to the appropriate virtual servers. zManager will then orchestrate
the virtualization among the server, network, hypervisor and the underlying VLAN
technology (within the physical switches) in a manner to provide a complete and
secure network solution.

This approach provides significant additional flexibility and isolation controls when
compared with traditional deployments of multi-tier multiplatform workloads across
physical servers connected via one or more external networks. In this latter scenario,
unrelated workloads may be hosted on the same external physical networks and
firewalls may need to be deployed to ensure isolation across the various workloads
and network boundaries. With the zEnterprise virtual network isolation features,
multi-tier, multiplatform workloads hosted within an ensemble can now be
associated with a single virtual network to maintain strict isolation from other
workloads hosted within the ensemble. When this type of deployment is possible the
requirement for traversing a firewall between all server tiers may be reduced or
eliminated. Each OS can continue to implement their existing IP filters as necessary.
Coupling the IP filters with the virtual network isolation provides a very strong form
of access control for secure communications within servers that are part of the same
virtual network in the IEDN. Traditional firewall requirements will however continue
to exist for certain scenarios, such as traffic entering/leaving the zEnterprise System
or scenarios where workloads belonging to different virtual networks need to be able
to communicate with one another.

The focus of this paper has been on the network isolation and firewall considerations
for the zEnterprise IEDN and a discussion of the new physical and logical network
security features being introduced. In addition to these topics, the use of network
security protocols like SSL/TLS, IPsec and SSH should also be carefully considered
within the intraensemble data network. Some workloads may only require the
endpoint or packet-based authentication offered by these protocols while others
might still require full encryption of network traffic for privacy or regulatory
requirements. Careful analysis of the specific security requirements within the
confines of the intraensemble data network could show that more selective use of
network encryption is warranted in some cases.

External Network Access
The following figure illustrates the two options customers can use to connect their
external network traffic to the zEnterprise (i.e. connecting blades to the outside

13

July 2010

world). Both options involve leveraging the customer’s traditional external Firewall
and access controls used to protect their enterprise systems.

Figure 7: External Network Access

In most cases customers will use both forms of external access based on traffic
destination, load balancing, or other QoS criteria. The access control from the
customer’s external network remains within the customer’s scope of control. Once
network traffic is within the Ensemble, then the IEDN virtual networking access
control provisions take control.

Exploiting External Firewalls
It is recognized that in some environments customers will still be required to force
traffic back out of the IEDN to route some network connections through a specific
firewall. This can be achieved by using standard network routing within the OS of
the virtual servers.

If you want to ensure that all packets that cross VLAN boundaries in the IEDN go
through a firewall router, create a static default route whose next hop is the address
of the firewall router. If you require all traffic to go through this firewall, this is

14

July 2010

sufficient. If traffic that is not crossing VLAN boundaries does not have to go
through the firewall, you can create a static subnet route to direct that traffic to go
directly where it needs to go on the VLAN.

For example if you are attached to a VLAN with subnet, and traffic to destinations
within that subnet do not require firewall use but all packets leaving that VLAN do,
you create the following static routes:

Route destination Outgoing interface Next Hop comment
0.0.0.0 Your interface attached The firewall router's IP Default route. Causes all
to the VLAN address on the VLAN you packets not routed by any
are attached to other routes to be routed
to the firewall
The VLAN’s subnet Your interface attached none Direct route to
address to the VLAN destinations in the VLAN
subnet, will ARP for the
destination address and
go directly there,
bypassing the firewall.

Table 1: Routing Table Overview – Accessing External Firewall

Referencing the Figure 7 (external network access), a server within the zBX could
use the option 2 external network path to access the customer’s external firewall and
then re-enter the intraensemble data network for connecting to the next tier server.
The following figure illustrates this approach.

15

July 2010

Figure 8: Exploiting an External Firewall

Figure 8 provides an example illustrating the network path of a blade server
accessing a System z server via an external firewall. IP addresses are now shown in
Table 2 (below) that correspond to the example in figure 8. Some key points to note
are:

1. Server 72B (IBM Blade virtual server) has two virtual network interfaces defined
as follows:
a. Eth1 – IP Subnet 192.12.144. 0/24 VLAN B - used to access the external
network and Firewall via zBX TOR switch – note the red network path
b. Eth2 – IP Subnet 10.24.104.0/24 VLAN C - used to connect directly to
servers via the IEDN within the Ensemble (e.g. Linux 55 is also defined to
access VLAN C and same IP subnet) – note the direct blue network path
2. Server 22A (z/OS LP) has a virtual network interface subnet 10.67.124.0/24
VLAN A (used to access the external network and Firewall via zBX TOR switch)
that is defined to support connections from the blades which must use the
external Firewall. This z/OS server would most likely have other virtual NICs
(VLANs) defined for direct IEDN access to other virtual servers (both system z
and IBM blades) within the Ensemble.

16

July 2010

Route destination Outgoing interface Next Hop comment
0.0.0.0 Your interface attached The firewall router's IP Default route. Causes all
to the VLAN B address on the VLAN you packets not routed by any
are attached to other routes to be routed
192.12.144.100 (Eth1) to the firewall
192.12.144.1
10.24.104.0/24 Your interface attached none Direct route to
to the VLAN C destinations in the VLAN
subnet, will ARP for the
destination address and
go directly there,
10.24.104.108 (Eth2) bypassing the firewall.
Table 2: Sample Routing Table in Virtual Server 72B (Figure 8)

Summary
This paper has provided a basic review of the notion of a network firewall and
considerations regarding the requirements for deploying one in a zEnterprise
environment. It has also described the internal networking support introduced with
the IBM zEnterprise and how, due to its enhanced physical and logical security, in
many cases it may eliminate the need for a network firewall to protect network traffic
within a zEnterprise environment. Finally, it has described how you can use an
external firewall if it is deemed necessary, e.g. for regulatory reasons or due to
general mandated corporate policy, to utilize a specific network firewall solution to
protect traffic between virtual servers in a zEnterprise environment.

17

July 2010

Acknowledgments and Contributions
This paper was a collaborative effort. Thanks to the following individuals for their
contributions to this paper.

Kim Bailey
Bill Carey
Anna Coffey
John Dayka
Gwen Dente - IBM S&D Advanced Technical Skills (ATS)
Patty Driever
Mike Fox
Gus Kassimis
Gary McAfee
Christopher Meyer
Linwood Overby

18

July 2010

About the Authors:

Jerry Stevens is a Senior Technical Staff Member with IBM
SWG and works in AIM Enterprise Networking Solutions
Architecture Strategy and Design with a focus on
communications hardware architecture. He has 25+ years
experience with z/OS network communications. Jerry can be
reached at sjerry@us.ibm.com.

Peter Spera is a Senior Software Engineer with IBM Corp.
He is focused on security for Linux on the System z platform,
but he is also involved with other areas, such as system
integrity and vulnerability reporting for System z. Peter can
be reached at spera@us.ibm.com.

19


Copyright IBM Corporation 2010
IBM Systems and Technology Group
Route 100
Somers, New York 10589
U.S.A.
Produced in the United States of America,
05/2010
All Rights Reserved

IBM, IBM logo, BladeCenter, Proventia, PR/SM, System z, zEnterprise, z/OS and z/VM are trademarks or
registered trademarks of the International Business Machines Corporation.

Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks
of Adobe Systems Incorporated in the United States, and/or other countries.

Cell Broadband Engine is a trademark of Sony Computer Entertainment, Inc. in the United States, other
countries, or both and is used under license therefrom.

InfiniBand and InfiniBand Trade Association are registered trademarks of the InfiniBand Trade Association.
Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other
countries, or both.

Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the
United States, other countries, or both.

Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel
SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its
subsidiaries in the United States and other countries.

UNIX is a registered trademark of The Open Group in the United States and other countries.

Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.

ITIL is a registered trademark, and a registered community trademark of the Office of Government
Commerce, and is registered in the U.S. Patent and Trademark Office.

IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency,
which is now part of the Office of Government Commerce.

All statements regarding IBM’s future direction and intent are subject to change or withdrawal without notice,
and represent goals and objectives only.

Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using
standard IBM benchmarks in a controlled environment. The actual throughput that any user will experience
will vary depending upon considerations such as the amount of multiprogramming in the user’s job stream,
the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can
be given that an individual user will achieve throughput improvements equivalent to the performance ratios
stated here.

ZSW03167-USEN-00

1

IBM zEnterprise System - Network Security

More Related Content

What's hot

Similar to IBM zEnterprise System - Network Security

More from IBM India Smarter Computing

Recently uploaded

IBM zEnterprise System - Network Security