DELL EMC XTREMIO X2 CSI PLUGIN INTEGRATION WITH PIVOTAL PKS A Detailed Review

Dell EMC XtremIO X2 CSI Plugin Integration with Pivotal PKS
© 2019 Dell Inc. or its subsidiaries.
Dell Customer Communication - ConfidentialDell Customer Communication - Confidential
DELL EMC XTREMIO X2 CSI PLUGIN INTEGRATION
WITH PIVOTAL PKS
A Detailed Review
Abstract
This white paper evaluates the best-in-class performance and scalability
delivered by Dell EMC XtremIO X2, using the Container Storage Interface
(CSI) plugin to deliver persistent storage for Cloud-Native Applications and
DB workloads managed by Pivotal PKS platform.
July 2019
WHITE PAPER

2 | Dell EMC XtremIO X2 CSI Plugin Integration with Pivotal PKS
Contents
Abstract............................................................................................................................................................. 1
Intended Audience............................................................................................................................................. 3
Introduction........................................................................................................................................................ 3
Business Challenge........................................................................................................................................... 4
Key Benefits ...................................................................................................................................................... 5
Pivotal Container Service (PKS) Overview ........................................................................................................ 7
VMware NSX-T.................................................................................................................................................. 7
Dell EMC XtremIO X2: The Next Generation All-Flash Array............................................................................. 7
What is Kubernetes? ......................................................................................................................................... 8
What is Container Storage Interface?................................................................................................................ 8
Dell EMC XtremIO and Pivotal PKS – Better Together ...................................................................................... 9
Integrating Dell EMC XtremIO CSI Plugin with Pivotal PKS............................................................................. 11
Platform and Software Dependencies..........................................................................................................................11
Supported XtremIO Versions: ..................................................................................................................................11
Installing PKS..............................................................................................................................................................11
Downloading and Extracting the Installer .................................................................................................................12
Configuring the Installer...........................................................................................................................................12
Generating Dell EMC XtremIO CSI Plugin YAML.....................................................................................................14
Deploying Mandatory BOSH Add-Ons............................................................................................................. 16
Managing PKS with the PKS CLI..................................................................................................................... 16
Connecting to the Kubernetes Dashboard ....................................................................................................... 20
Health Monitoring and Troubleshooting ........................................................................................................... 21
Deploying Persistent Volumes and Applications .............................................................................................. 23
Example Use Case: PostgreSQL.................................................................................................................................24
Conclusion....................................................................................................................................................... 28
References...................................................................................................................................................... 29
How to Learn More .......................................................................................................................................... 30

Intended Audience
This white paper is designed for developers, system architects and storage administrators tasked with evaluating or
deploying Pivotal PKS, and interested in persistent storage within their deployment. While this document focuses on Dell
EMC XtremIO X2 products, the concepts discussed and implemented can be applied to other storage vendor products.
Understanding the material in this document requires prior knowledge of containers, Kubernetes, PKS and of the Dell
EMC XtremIO X2 cluster type. These concepts are not covered in detail in this document. However, links to helpful
resources on these topics are provided throughout the document and in the References section. In-depth knowledge of
Kubernetes is not required.
Introduction
To improve competitiveness in a fast-paced world, organizations are embracing software as a prime differentiator. A
software-driven business is more agile and efficient, innovates faster and responds dynamically to changing market and
customer demands. Increasingly, gaining these compelling advantages requires adopting cloud-native applications.
Kubernetes is the de facto standard for container orchestration for microservices and applications. However, enterprise
adoption of big data and databases using containers and Kubernetes is hindered by multiple challenges, such as
complexity of persistent storage, availability, and application lifecycle management. Kubernetes provides the agility and
scale that modern enterprises require. However, Kubernetes provides the building blocks for infrastructure, but not a
turnkey solution.
Originally containers and Kubernetes were optimized for stateless applications, restricting enterprises to use container
technology for workloads, such as websites and user interfaces. More recently, stateful workloads have been increasingly
run on Kubernetes. Whereas it is relatively easy to run stateless microservices using container technology, stateful
applications require slightly different treatment.
There are multiple factors which need to be taken into account when considering handling persistent data using
containers, including:
• Containers are ephemeral by nature, so the data that needs to be persistent must survive through the restart/re-
scheduling of a container.
• When containers are rescheduled, they can die on one host, and might get scheduled on a different host. In such
cases, the storage should also be shifted and made available on a new host for the container to start gracefully.
• The application should not have to deal with the volume/data and underlying infrastructure, and should handle the
complexity of unmounting and mounting, etc.
• Certain applications have a strong sense of identity (such as Kafka, Elastic, etc.), and the disk used by a container
with a specific identity is tied to it. If, for any reason, a container with a specific identity becomes rescheduled, it is
important that the disk specifically associated with that identity, gets reattached on a new host.
In this white paper, we present a solution that provides the robust capabilities and built-in high availability and scalability of
XtremIO and PKS, bringing simplicity to Kubernetes, and creating a turnkey solution for data-heavy workloads. XtremIO
CSI Plugin provides Pivotal PKS technology with built-in enterprise-grade container storage and uncompromising
performance that extends Kubernetes’ multi-cloud portability to the private cloud.
With this proven integration, organizations can accelerate their digital business objectives via cloud-native applications,
from development through testing and into production, on a single platform with ease and agility. Most importantly,
eliminating the risk of application downtime preserves critical revenue streams, helps ensure customer satisfaction and
strengthens your competitive advantage.

Business Challenge
Planning, designing and building a private or hybrid cloud to support cloud-native applications can be a complex and
lengthy project that does not address immediate business needs. IT also must maintain and manage the infrastructure
that supports these new applications, to ensure that the environment is reliable, secure and upgradeable.
Kubernetes offers great capabilities. However operationalization of these capabilities is not a trivial task. There are many
layers involved, such as hardware, operating system, container runtime, software-defined networking, and Kubernetes
itself. And all the layers require Day 0, Day 1 and Day 2 activities. Most of these layers also require constant patching (for
example, Kubernetes ships a new version every three months), so the effort to maintain this infrastructure grows
exponentially.
Kubernetes is the de facto standard for container orchestration platforms. However, Kubernetes can be difficult to
operationalize and maintain for the following reasons:
• High availability — There is no out-of-the-box fault-tolerance for the cluster components themselves (masters and
etcd nodes).
• Scaling — Kubernetes clusters handle scaling the pod/service within the nodes, but do not provide a mechanism to
scale masters and etcd Virtual Machines (VMs).
• Health checks and healing — The Kubernetes cluster performs routine health checks for node health only.
• Upgrades — Rolling upgrades on a large fleet of clusters is difficult. Who manages the system it runs on?
• Operating systems — A Kubernetes installation requires multiple machines where an OS is installed. Installing and
patching these systems is an overhead.
Pivotal Container Service (PKS) is a purpose-built product that enables enterprises to deploy and consume container
services with production-grade Kubernetes, built with high availability, security, multi-tenancy and operational efficiency
across private and public clouds.
It effectively solves the complexities around Kubernetes infrastructure and enables you to focus on your business
applications. Furthermore, it benefits your developers, providing them with the latest version of Kubernetes and simple
integrations with monitoring and logging tools.

Key Benefits
With Pivotal Container Service (PKS), not only do you receive a vanilla Kubernetes distribution supported by Pivotal and
VMware, but you also receive the automation to maintain the Kubernetes stack with reduced effort, including the operating
system, container runtime, software-defined networking and Kubernetes itself. The automation provided by BOSH
enables you to install (Day 0), configure (Day 1), patch and maintain your Kubernetes infrastructure in a healthy state over
time (Day 2).
This automation also extends your capabilities to have an effective Kubernetes as a service offering, and lets you create
Kubernetes clusters with a single command, to either expand or shrink the clusters, or to repair them as needed. This is a
unique capability that enables you to provide your Kubernetes users with as many clusters as they need, and to resize
them, leveraging automation as your workload requirements change.
PKS also simplifies the networking configuration, by providing you with VMware NSX-T, which is a software-defined
network infrastructure to build cloud-native application environments.
PKS also provides you with a container registry to host your containers, namely, Harbor, a graduated CNCF project.
Pivotal PKS Components

For the above reasons, Pivotal and VMware, in partnership with Google, launched PKS with the vision of offering a
turnkey solution to provision, operate and manage enterprise-grade Kubernetes clusters:
• Built with open-source Kubernetes — Constant compatibility with the current stable release of Kubernetes, operated
by BOSH. No proprietary extensions.
• Production-ready — Highly available from apps to infrastructure with no single points of failure. Built-in health checks,
scaling, auto-healing and rolling upgrades.
• Multicloud — BOSH provides a reliable and consistent operational experience for any cloud.
• Network management and security — Out-of-the-box with VMware NSX-T. Multi-cloud, multi-hypervisor.
• Fully automated Ops — Fully automated deploy, scale, patch and upgrade. No downtime. Can also use CD pipelines
to deploy your platform.
• Harbor — Enterprise-class container registry server. Harbor provides vulnerability scanning and identity management
for Docker images, and supports multiple registries.
• Storage persistence for stateful applications — PKS leverages open source projects Hatchway to implement
persistent storage for K8s. No additional manual installation is required. The plugin supports static and dynamic
provisioning of volumes for Kubernetes.
• Monitoring integration — Monitoring integrations with vRLI and vROPs, and eventually Wavefront. Also, as PKS
delivers OSS Kubernetes, any monitoring tool that works with Kubernetes can be integrated into clusters provisioned
by PKS.
• Load Balancing — Load balancing functionalities for Kubernetes are provided via NSX-T, and HA for the load
balancing solution is achieved thanks to the vSphere infrastructure powering PKS and NSX-T.
By running Kubernetes on top of BOSH, the following benefits are achieved:
DAY 1 – BUILD
• Deploy Kubernetes cluster via BOSH.
• Deploy Kubernetes clusters on demand.
DAY 2 – OPERATE
• Self-healing VMs and monitoring.
• Elastic scaling for clusters.
• Rolling upgrades to latest Kubernetes release.
• High-availability and multi-availability Zones support.

Pivotal Container Service (PKS) Overview
PKS runs enterprise-grade Kubernetes containers and uses BOSH to build and maintain highly-available Kubernetes
clusters. PKS uses software-defined networking that is incorporated with VMware NSX and uses a Harbor-based
container registry. Developers receive on-demand managed vanilla Kubernetes clusters, and operators can easily run
secure Kubernetes clusters.
PKS is production-ready and highly available, and includes dynamic load balancing, zero downtime updates, automated
health checks and more. PKS enables IT teams to flexibly deploy and consume Kubernetes on-premises with vSphere, or
in a public cloud. Due to PKS being powered by BOSH, you get a reliable and consistent operating experience on any
cloud.
PKS uses NSX-T to programmatically manage software-defined virtual networks, and to keep your Kubernetes network
secure. IT operators can use BOSH to deploy on-demand Kubernetes clusters through the command line and the API.
BOSH constantly monitors the health of Kubernetes clusters deployed by PKS.
PKS integrates easily with VMware software like vRealize Operations Manager, VMware Wavefront, vRealize Automation
and vRealize Log Insight. PKS comes with Harbor; an enterprise-class container registry server. Harbor provides
vulnerability scanning and identity management for Docker images and supports multiple registries.
VMware NSX-T
VMware NSX-T provides an agile, software-defined network infrastructure to build cloud-native application environments.
NSX-T network virtualization programmatically creates, snapshots, deletes, and restores software-based virtual networks.
With network virtualization, the functional equivalent of a network hypervisor reproduces the complete set of Layer 2
through Layer 7 networking services, such as switching, routing, access control, firewalling, and QoS in software. These
services can be programmatically assembled in any combination to produce unique, isolated virtual networks in a matter
of seconds.
NSX-T works by implementing three separate but integrated planes: management, control, and data. The three planes are
implemented as a set of processes, modules, and agents residing on three types of nodes: manager, controller, and
transport nodes.
NSX-T is designed for management, operation and consumption by development organizations. NSX-T enables IT and
development teams to choose the technologies best-suited for their applications. IaaS providers can use NSX-T to
provide self-service functionality to their users, by deploying it on-premises. NSX-T is built for developer-ready
infrastructure and it can automatically integrate with Kubernetes.
Dell EMC XtremIO X2: The Next Generation All-Flash Array
Dell EMC XtremIO is a purpose-built all-flash array that provides consistent high performance with low latency;
unmatched storage efficiency with inline, all-the-time data services; rich application integrated copy services; and
unprecedented management simplicity.
The next-generation platform, XtremIO X2, builds upon unique features of XtremIO to provide even more agility and
simplicity for your data center and business. Content-aware in-memory metadata, and inline all-the time data services
have made XtremIO the ultimate platform for virtual server and desktop environments and workloads that benefit from
efficient copy data management. The architecture offers VMware XCOPY operations that are twice as fast, which can
reach as high as 40 GB/s bandwidth, due to them being in-memory operations.
X2 includes inline, all-the-time data services, thin provisioning, deduplication, compression, replication, D@RE, XtremIO
Virtual Copies (XVC), and double SSD failure protection with zero performance impact. Only XtremIO’s unique in-memory
metadata architecture can accomplish this.

What is Kubernetes?
Kubernetes is a container orchestration system that automates lifecycle operations such as deployment, scaling and
management for containerized applications. Initially developed by Google and later open-sourced, Kubernetes is now
managed by Cloud Native Computing Foundation (CNCF).
Kubernetes is built using master-slave architecture common to most distributed computing systems. A Kubernetes cluster
consists of at least one master node and multiple worker nodes. Master nodes are responsible for running the master
components that provide the cluster’s control plane. Master components can run on multiple master nodes, but, for
simplicity, they are started on the same node using set up scripts. The designated master node does not run any user
containers. On the slave side, node components run on each worker node, and are responsible for maintaining running
pods and providing the Kubernetes runtime environment.
What is Container Storage Interface?
The CSI is a community-led effort to develop a standardized API for
container orchestration platforms (COs) to talk to storage plugins.
Theoretically, with a standardized communication protocol, storage vendors
only need to write a plugin to a single specification. CSI sets out the
definition of how to talk to the plugin. Exactly how the plugin is managed or
operates is up to the storage provider.
CSI provides the following capabilities:
• Dynamic provisioning and decommissioning of volumes
• Attachment and detachment of volumes from a host node
• Mounting and unmounting of a volume from a host node
CSI makes installing new volume plugins as easy as deploying a pod and enables third-party storage providers to develop
their plugins without needing to add code to the core Kubernetes code base.
Various community members from Kubernetes, Mesosphere and Docker worked on this specification together. This new
interface is a major benefit to the container ecosystem, because it standardizes the model for integrating storage systems
across container orchestrators. Specifically, it frees the storage system driver from the Kubernetes release schedule, as
the driver is no longer incorporated in the same code base (in-tree). With CSI, you can install storage system drivers
independently of the container orchestration releases, providing faster bug fixes and features, while reducing code sprawl.
With all major orchestration engines working toward supporting CSI, DevOps teams have only to worry about one
configuration for their environments, which helps in keeping everything portable regardless of where you are running your
computer. This new plugin model further eases development and provides peace of mind for users running persistent
workloads on Dell EMC XtremIO, both in terms of performance and availability.
Before beginning with provisioning storage, it is important to understand the Kubernetes concepts of a persistent volume
and a persistent volume claim, and how they work together in a cluster. You can find detailed information in the Dell EMC
XtremIO CSI White Paper.

Dell EMC XtremIO and Pivotal PKS – Better Together
Over the last few years, software containers have become a big deal. According to Gartner, by the end of 2016, 25% of
workloads were deployed using containers. The wide adoption of containers has been primarily driven by developers and
yet challenges remain, such as how to operate and manage an environment in which these containers run securely, at
scale.
PKS is exactly that environment; a platform that allows application teams to self-serve provision Kubernetes clusters that
are fully managed on their behalf, enabling them to focus on the workloads that serve their consumers rather than the
substrate needed to run those workloads.
The platform is enterprise grade, providing the following capabilities:
• Production-ready — Highly available from apps to infrastructure, with no single points of failure. Built-in health checks,
scaling, auto-healing and rolling upgrades.
• Fully automated Ops — Fully automated deploy, scale, patch, upgrade. No downtime. Can also use CD pipelines to
deploy your platform.
• Multicloud infrastructure automation — BOSH for automated infrastructure provisioning, configuration, operation, and
remediation on any cloud or IaaS.
• Commercial support — Enterprise grade assistance from Pivotal’s support network.
There are several factors to consider when running stateful apps in Kubernetes:
• Pods are ephemeral by nature. Data that needs to be persisted must be accessible upon restart and rescheduling of
a pod.
• When a pod is rescheduled, it may be on a different host. Storage must be available on the new host for the pod
to start gracefully.
• The app should not manage the volume and data. The underlying infrastructure should handle the complexity of
unmounting and mounting.
• Certain apps have a strong sense of identity. When a container with a certain ID uses a disk, the disk becomes
tied to that container. If a pod with a certain ID gets rescheduled, the disk associated with that ID must be reattached
to the new pod instance.
The integration between Pivotal PKS and Dell EMC XtremIO merges the benefits of the PKS and the unmatched
performance and data services of XtremIO to the world of containers. PKS is highly available up and down the stack, from
apps to infrastructure. The service also includes dynamic load balancing, zero downtime updates, automated health
checks and much more. With PKS, you can deploy, scale, patch and upgrade all the Kubernetes clusters in your system
without downtime, and rapidly apply security fixes when new vulnerabilities are detected. Combining Dell EMC XtremIO
CSI Plugin with PKS enables customers to deliver container-based applications on-premises for both development and
production scale. XtremIO CSI Plugin support for Kubernetes enables container-based applications to be deployed,
managed and scaled. With Dell EMC XtremIO X2’s unmatched levels of performance, availability and TCO, organizations
can now achieve the same levels of business agility offered by the public cloud, with the benefits of running them on
premises.

The solution presented in this white paper is fully engineered and pre-tested to ensure consistency across multiple
platform deployments and reliable operation from day one, through its entire life cycle of upgrades and enhancements.
Container’s high-availability feature keeps your most critical cloud-native applications running twenty-four hours a day,
seven days a week. Using built-in health monitoring and orchestrated self-healing, this solution eliminates the risk of
application downtime and the costly damage it could inflict on revenue streams and customer confidence.
Pivotal PKS and Dell EMC XtremIO X2 Architecture

Integrating Dell EMC XtremIO CSI Plugin with Pivotal PKS
Platform and Software Dependencies
• Pivotal PKS 1.4 and above
• iSCSI Portal configured at the XtremIO level
• iSCSI vSphere network for PKS virtual machines
Note: Pivotal PKS with NSX-T has not tested.
Supported XtremIO Versions:
• XtremIO X2 XIOS 6.2.0
• XtremIO X2 XIOS 6.2.1
• XtremIO X1 XIOS 6.2.0 (limited functionally, please contact Dell EMC support for specific details)
• XtremIO X1 XIOS 6.2.1 (limited functionally, please contact Dell EMC support for specific details)
Note: Multi-cluster functionally is supported. Note that all Dell EMC XtremIO X-Bricks must be of the same generation.
Installing PKS
To install PKS:
1. Download the product file from Pivotal Network.
2. Open a browser and navigate to https://YOUR-OPS-MANAGER-FQDN/ to log in to the Ops Manager Installation
Dashboard.
3. Click Import a Product to upload the product file.
4. In the left column, under Pivotal Container Service, click the plus sign to add this product to your staging area.
5. Click the orange Pivotal Container Service icon, as shown in Figure 3, to start the configuration process.
Pivotal Container Service Icon
The complete installation instructions are in the VMware Enterprise PKS Documentation.

Downloading and Extracting the Installer
Clone the latest version of the XtremIO CSI installer bundle from the CSI XtremIO Deploy GitHub Repository to the Linux
machine with network connectivity to the XMS, and extract it by running the following commands:
Configuring the Installer
Edit the csi.ini configuration file.
Note: The installation script is configured to download the required Docker images from the Internet by default. The
installation script gets the current XtremIO CSI Docker image from the Dell EMC Storage Docker Hub Repository.
Alternatively, if your environment has no Internet access directly or via proxy, you can use 33 (or any other local registry)
as an internal Docker repository, and then upload the images to the registry and set plugin_path and csi_.._path
variables, accordingly. See Figure 4 on page 13.
Mandatory Fields:
• management_ip — The management IP address or FQDN of the XtremIO Management Server.
• csi_user and csi_password — The credentials used by the plugin to connect to the XtremIO Management
Server. csi_user and csi_password can be created in advance (as suggested above) or they can be created by
an installation script. If user creation is left to the script, the user is required to provide initial_user and
initial_password in the csi.ini. For example; the credentials of the Storage Administrator.
• kubelet_root=/var/vcap/data/kubelet — The Kubelet path for PKS deployments.
• ephemeral_node=True — This parameter is set to True when the Kubernetes nodes are ephemeral.
$ git clone https://github.com/dell/csi-xtremio-deploy.git
$ cd csi-xtremio-deploy && git checkout 1.2.0-1

Figure 4 presents the case when the Storage Administrator has the credentials admin/password123 and the plugin uses it
to create the CSI user during the installation - csiuser/password456.
The example in Figure 4 also shows that the PKS nodes do not have direct access to the Internet. The required Docker
images have been downloaded, and uploaded to VMware Harbor, so the plugin is set to get the necessary Docker images
from the local Harbor registry.
csi.ini Example File points to VMware Harbor Registry
Note: The initial user is no longer in use by the plugin following the installation.
management_ip=vxms-xbrick1100.pks.xtremio
initial_username=admin
initial_password=password123
csi_username=csiuser
csi_password=password456
force=No
verify=Yes
kubelet_root=/var/vcap/data/kubelet
plugin_path=harbor.pks.xtremio/csi/csi-xtremio:pks
csi_attacher_path=harbor.pks.xtremio/csi/csi-attacher:v1.0.1
csi_cluster_driver_registrar_path=harbor.pks.xtremio/csi/csi-cluster-driver-registrar:v1.0.1
csi_node_driver_registrar_path=harbor.pks.xtremio/csi/csi-node-driver-registrar:v1.0
csi_provisioner_path=harbor.pks.xtremio/csi/csi-provisioner:v1.0.1
csi_snapshotter_path=harbor.pks.xtremio/csi/csi-snapshotter:V1
storage_class_name=csi-xtremio-sc
list_of_clusters=xbrick1100
list_of_initiators=
csi_qos_policy_id=
csi_high_qos_policy=15m
csi_medium_qos_policy=5m
csi_low_qos_policy=1m
csi_xtremio_debug=0
ephemeral_node=True

Generating Dell EMC XtremIO CSI Plugin YAML
Unlike native Kubernetes deployment, in PKS the CSI plugin is provided as Custom Workloads. Custom Workloads define
what is included in a cluster out-of-the-box, and can be attached to a PKS plan that defines a set of resource types that
are used for deploying clusters.
PKS uses BOSH to install the XtremIO CSI plugin for each new Kubernetes environment deployed by PKS from that
specific plan, which includes the CSI Plugin YAML file.
To generate Dell EMC XtremIO CSI Plugin YAML:
1. Navigate to the template directory, and update the secret-template.yaml with the XMS CSI user’s password.
CSI Secret YAML File
2. After specifying the environment parameters, run the install-csi-plugin.sh script with -g parameter to
generate the plugin YAML file. This script also configures the CSI user and the CSI QoS policies on the XMS.
Upon completion, the YAML file is in the current location, under the csi-tmp directory.
3. Using a text editor, open the plugin.yaml file and, at the beginning of the file, add the secret content that you
edited in the previous step.
Make sure to add the ‘---’ to separate the sections.
4. Navigate to the Ops manager website and click Enterprise PKS, as shown in Figure 6.
Ops Manager Web Interface
apiVersion: v1
kind: Secret
metadata:
name: csixtremio
namespace: kube-system
stringData:
password: "#XMS_PASSWORD#"
$ ./install-csi-plugin.sh -g
plugin.sh -g

5. Select Active to activate the plan and to make it available to developers deploying clusters.
6. Enter a unique name and description in the respective fields, as shown in Figure 7.
PKS Configuration Plan
7. At the bottom of the page, under (Optional) Add-ons - Use with caution, enter the additional YAML
configuration file that you generated earlier, including the Kubernetes secret, as shown in Figure 8.
This enables PKS to add custom workloads to each cluster in this plan.
8. Select Allow Privileged to enable the CSI plugin containers to access physical devices, as shown in Figure 8.
9. Click Save.
PKS Configuration Plan – Add-ons Sections

Deploying Mandatory BOSH Add-Ons
This section explains how to deploy BOSH Add-ons as a part of installing the Pivotal Cloud Foundry (PCF) platform.
BOSH Add-ons are optional platform extensions that you deploy with BOSH, rather than installing them as Ops Manager
tiles. Pivotal recommends installing BOSH Add-ons immediately after deploying the BOSH and Ops Manager, prior to
using Ops Manager to add tiles to the platform.
To integrate PKS with the XtremIO CSI Plugin, it is necessary to install the iSCSI add-on, located at:
https://github.com/svrc-pivotal/iscsi-release
Clone the latest version of the iSCSI-release bundle from the svrc-pivotal GitHub Repository to BOSH.
Edit the applying.sh script according to the instructions in the README file, and then run it on your BOSH machine.
This script installs the necessary packages to support the CSI plugin on every Kubernetes cluster deployed by PKS.
Managing PKS with the PKS CLI
To manage PKS-deployed clusters, use the PKS Command Line Interface (CLI). When you log into PKS successfully for
the first time, the PKS CLI generates a local creds.yml file containing the API endpoint, refresh token, access token and
CA certificate (if applicable). By default, creds.yml is saved to the ~/.pks directory on your local system. You can use
the PKS_HOME environment variable to override this location, and store creds.yml in any directory on your system.
You will use PKS to get credential information from the deployed Kubernetes cluster, and set the context for kubectl.
The context associates cluster and authentication information to a context name and makes that name the current
context.
To Manage PKS with PKS CLI:
1. Install PKS CLI on your local system. See Installing the PKS CLI.
2. See Configuring PKS API Access for a username and password with access to the PKS API.
3. Run the following command in the CLI to authenticate to the PKS API:
4. Run the following command to create a new Kubernetes cluster. Make sure you specify the correct plan name
(‘XtremIO’, as shown in the example):
pks login -a #PKS UAA ip/fqdn# -u #username# -p #password#
pks create-cluster #cluster name# --external-hostname #cluster ip/fqdn# --plan XtremIO

5. Run the following command to monitor the cluster status:
‘pks cluster’ Command Output
Once completed, the Last Action Description changes to Instance provisioning completed.
6. Create a DNS record with the Kubernetes Master Host and its IP address and make sure you can ping it.
‘pks cluster’ Command Output
7. Run the pks get-credentials command to perform the following actions:
o Fetch the cluster’s kubeconfig.
o Add the cluster’s kubeconfig to the existing kubeconfig.
o Create a new kubeconfig (if non-existent).
o Switch the context to the CLUSTER-NAME provided.
‘pks get-credentials’ Command Output
pks cluster #cluster name#
pks get-credentials CLUSTER-NAME

8. Run the following command and verify that all nodes are up and running:
'kubectl get nodes’ Showing All Kubernetes Nodes
During the PKS initial configuration, PKS uses BOSH to deploy the Kubernetes master and node virtual machines
on the resources specified in advance for each new Kubernetes cluster.
PKS Virtual Machines
The XtremIO CSI plugin is installed on each Kubernetes virtual machine, as well as the necessary iSCSI and
multipath drivers to support this configuration during the deployment.
An iSCSI initiator group is created by the XMS for each PKS Kubernetes node. This initiator group is used by the
CSI plugin for mapping volumes to the PKS Kubernetes nodes during static and dynamic provisioning operations,
on Kubernetes volumes.
The XMS Showing Automatically Created PKS Kubernetes Node Initiator Groups
kubectl get nodes

9. Run the following command and verify that the CSI plugin deployment is completed successfully:
‘kubectl get pods –all-namespaces’ Showing All Kubernetes Pods

Connecting to the Kubernetes Dashboard
The easiest way to access the Kubernetes dashboard is via kube proxy. We will run the proxy command on RDSH in
PowerShell. This redirects all traffic destined for 127.0.0.1:{port_number} on this server, to the Kubernetes Master Node.
To connect to the Kubernetes dashboard:
1. Run the following command in order to run kube proxy:
Running the above command binds the Kubernetes dashboard to an available port on localhost. In our case, this
turned out to be 8001.
‘kubectl proxy’ Command Output
2. In your web browser, navigate to: 127.0.0.1:{port_number}/ui
The screen shown in Figure 17 opens.
Kubernetes enables authentication to the UI by either using the Kubeconfig file or Token.
For authentication in this lab, we select Kubeconfig, as shown in Figure 17.
The Kubernetes Dashboard Login Page
3. Navigate to C:Users<username>.kube and select the config file.
Note: You must use your username to locate your Kubeconfig file.
kubectl proxy

Health Monitoring and Troubleshooting
PKS leverages BOSH to provision infrastructure. BOSH receives periodic heartbeats from the nodes or VMs it has
provisioned and ensures that all the required processes on these nodes are running. If BOSH detects a failure, it repairs
the nodes in the background, using various methods. For example, if a node or VM is not responding, BOSH detaches its
persistent disk, spins up a new VM, attaches the persistent disk to this new VM, and then starts the required processes on
this node. All this is performed in the background and the issue is resolved without user interference.
Figure 18 shows all current BOSH tasks.
‘bosh tasks’ Command Output
Figure 19 shows all BOSH managed virtual machines.
‘bosh vms’ Command Output

Figure 20 shows all BOSH deployments.
‘bosh deployments’ Command Output
PKS Administrators can collect logs for each Kubernetes cluster or for the individual nodes, for advanced debugging.
The command shown in Figure 21 collects logs from a Kubernetes cluster into a single tarball file.
Generating BOSH Log Bundle

Deploying Persistent Volumes and Applications
Although it is relatively easy to run stateless microservices using container technology, stateful applications require
slightly different treatment. There are multiple factors which need to be considered when handling persistent data using
containers. Such considerations include:
• Kubernetes pods are ephemeral by nature, so the data that needs to remain persistent must survive through the
restart and/or rescheduling of a pod.
• When pods are rescheduled, they can die on one host, and might get scheduled on a different host. In such cases,
the storage should also be shifted and made available on the new host, for the pod to start gracef`ully.
• The application should not have to be concerned about the volume and data. The underlying infrastructure should
handle the complexity of unmounting and mounting.
• Certain applications have a strong sense of identity (such as Kafka and Elastic), and the disk used by a container with
a certain identity is tied to it. It is important that if for any reason a pod with a certain ID gets rescheduled, the disk
associated with that ID is reattached to the new pod instance.
The following section demonstrates the successful deployment of PostgreSQL, using the XtremIO CSI driver with PKS.
In the example we will look at, PKS leverages Dell EMC XtremIO Storage for Kubernetes to enable pods to use
enterprise-grade persistent storage. Before beginning with provisioning storage, it is important to understand the
Kubernetes concept of a persistent volume and a persistent volume claim, and how they work together in a cluster. You
can find detailed information about Kubernetes storage building blocks in the Dell EMC XtremIO CSI White Paper.
To demonstrate the capabilities of managing Kubernetes persistent storage applications using the XtremIO CSI plugin
with Pivotal PKS, we deployed a Postgres multi-node cluster on top of a Pivotal PKS Kubernetes cluster, that is
connected to a single XtremIO X2 cluster type X-Brick.
In the XtremIO plan we created in PKS, we chose to deploy a three-node cluster with the ‘2xlarge.cpu’ type in order to
eliminate any bottleneck at the virtual machine level, and to highlight the great performance of the XtremIO X2 cluster type
storage array.
Worker Node Configuration in Pivotal PKS XtremIO Plan

Example Use Case: PostgreSQL
PostgreSQL is a general purpose and object-relational database management system. It is
one of the widely-used open-source database systems.
PostgreSQL was designed to run on UNIX-like platforms. However, PostgreSQL was also
designed to be portable, so that it could run on various platforms such as Mac OS X, Solaris
and Windows. PostgreSQL is designed to be extensible. In PostgreSQL, a user can define
their own data types, index types, functional languages, etc.
Before creating the application, we create the Kubernetes ConfigMap and Service, by running
the following kubectl commands.
Postgres Config Map YAML File
kubectl create -f config.yaml
Postgres Config Map
apiVersion: v1
kind: ConfigMap
metadata:
name: postgres-config-demo
labels:
app: postgres
data:
POSTGRES_DB:demodb
POSTGRES_USER: admin
POSTGRES_PASSWORD: Password123!

Postgres Service YAML File
The following example shows that we set the Storage Class Name to ‘csi-xtremio-high’, which is generated
automatically during the CSI Plugin installation. This feature enables the user to automatically assign QoS policies at the
XMS level to Kubernetes volumes, to guarantee service for critical applications and prevent a noisy neighbor. You can
find detailed information about Kubernetes QoS in the Dell EMC XtremIO CSI White Paper.
For each new pod, we request a 30 GB volume with ReadWriteOnce accessModes.
When creating our first volume, the XtremIO CSI plugin creates the iSCSI Initiator Groups at the XMS level, and runs the
iSCSI login, discovery, and multipath configurations for each node.
kubectl create -f service.yaml
Postgres Service
apiVersion: v1
kind: Service
metadata:
name: postgres
labels:
app: postgres
spec:
ports:
- port: 5432
name: postgres
clusterIP: None
selector:
app: postgres

To create the Postgres DB, we run the following command, which deploys the Postgres stateful set below:
Postgres Stateful Set YAML File
kubectl create -f postgres.yaml
Postgres Stateful Set
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: postgres-demo
spec:
serviceName: "postgres"
replicas: 3
selector:
matchLabels:
app: postgres
template:
metadata:
labels:
app: postgres
spec:
containers:
- name: postgres
image: postgres:latest
envFrom:
- configMapRef:
name: postgres-config-demo
ports:
- containerPort: 5432
name: postgredb
volumeMounts:
- name: postgredb
mountPath: /var/lib/postgresql/data
subPath: postgres
volumeClaimTemplates:
- metadata:
name: postgredb
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: csi-xtremio-high
resources:
requests:
storage: 30Gi

The following steps verify that all Postgres pods are up, by running the following kubectl command:
‘kubectl get pods’ Command Showing the Postgres Pods
Verify that the PersistentVolumeClaims were created for all Postgres pods, by running the following kubectl command:
‘kubectl get pods’ Command Showing the Postgres Persistent Volume Claims
Verify that the PersistentVolumes were created, formatted and mounted on all Postgres pods, by running the following
kubectl command:
‘kubectl get pods’ Command Showing the Postgres Persistent Volumes
At this point, all volumes were automatically created at the XtremIO X2 cluster level, mapped to the relevant Kubernetes
worker nodes, formatted with a filesystem, and mounted onto the relevant Postgres application pod.
By navigating to the XMS, we can see the dynamically created volumes by the CSI plugin and their characteristics.
The XtremIO CSI plugin automatically assigned the desired QoS Policy (QoS_High) to the Postgres pod’s Persistent
Volumes:
The XMS Showing the Dynamically Created Volumes via a Kubernetes PersistentVolume Claim
In order to demonstrate XtremIO’s unique performance capabilities, we run a simple benchmark tool called pgbench.
Running pgbench runs the same sequence of SQL commands over and over, in multiple concurrent database sessions
against our running PostgreSQL cluster, which resides on multiple XtremIO X2 LUNs. Each command represents a
Persistent Volume, which is bound to a single container.
More information about pgbench can be found here.

Figure 30 shows the XMS Dashboard view of a Postgres DB workload. The Postgres DB workload drives about 770 MB/s
throughput in sub-millisecond latency, while the workload is distributed across the Postgres pods. Each Kubernetes pod
has its dedicated persistent volume.
The XMS – Postgres Workload Performance
Conclusion
Containers present significant benefits for supporting cloud-native apps and the transformation to micro-services, as they
continue to grow in popularity and adoption. Several important challenges associated with managing containers have
been addressed by popular orchestration solutions, such as Kubernetes. However, deploying production-ready
Kubernetes clusters faces additional issues. The most noteworthy challenge is determining the right persistent storage
solution for containers.
The integration between Pivotal PKS and Dell EMC XtremIO brings together the benefits of PKS and the unmatched
performance and data services of XtremIO to the containers world. PKS is highly available up and down the stack, from
apps to infrastructure. The service also includes dynamic load balancing, zero downtime updates, automated health
checks, and much more. With PKS, you can deploy, scale, patch and upgrade all the Kubernetes clusters in your
system—without downtime— and rapidly apply security fixes whenever new vulnerabilities are detected. Combining Dell
EMC XtremIO CSI Plugin with PKS enables customers to deliver container-based applications on-premises, for both
development and production scale. XtremIO CSI Plugin support for Kubernetes enables container-based applications to
be deployed, managed and scaled. With Dell EMC XtremIO X2’s unmatched levels of performance, availability and TCO,
organizations can now achieve the same levels of business agility offered by the public cloud, with the benefits of running
them on-premises. The solution presented in this white paper is fully engineered and pre-tested to ensure consistency
across multiple platform deployments, and reliable operations from day one, through its entire life cycle of upgrades and
enhancements. Container’s high-availability feature keeps your most critical cloud-native applications running twenty-four
hours a day, seven days a week. Through built-in health monitoring and orchestrated self-healing, this solution eliminates
the risk of application downtime and the costly damage it could inflict on revenue streams and customer confidence.

References
• Dell EMC XtremIO CSI White Paper:
– https://www.dellemc.com/resources/en-us/asset/white-papers/solutions/white-paper_xtremio-
x2_interface_plugin_for_kubernetes_h17583.pdf
• VMware Enterprise PKS Documentation:
– https://docs.vmware.com/en/VMware-Enterprise-PKS/index.html
• Dell EMC XtremIO CSI Docker Repository:
– https://hub.docker.com/r/dellemcstorage/csi-xtremio
• Dell EMC XtremIO CSI Deployment Script:
– https://github.com/dell/csi-xtremio-deploy
• Harbor Cloud Native Registry:
– https://goharbor.io/
• Cloud Foundry BOSH:
– https://bosh.io/docs/
• ISCSI BOSH Add-On:
– https://github.com/svrc-pivotal/iscsi-release
• Dell EMC XtremIO Main Page:
– https://github.com/svrc-pivotal/iscsi-release
• Dell EMC XtremIO X2 Datasheet:
– http://www.emc.com/collateral/data-sheet/h16095-xtremio-x2-next-generation-all-flash-array-ds.pdf
• Introduction to Dell EMC XtremIO X2 Storage Array:
– https://www.emc.com/collateral/white-papers/h16444-introduction-xtremio-x2-storage-array-wp.pdf
• CSI Spec:
– https://github.com/container-storage-interface/spec/blob/master/spec.md
• CSI GitHub Repository:
– https://kubernetes-csi.github.io/docs/Home.html
• Kubernetes Documentation site:
– https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/
• CSI Explanation:
– https://blog.architecting.it/container-storage-interface/

How to Learn More
For a detailed presentation explaining XtremIO X2 Storage Array's capabilities and how XtremIO substantially improves
performance, operational efficiency, ease-of-use and total cost of ownership, please contact XtremIO, at
XtremIO@emc.com.
We will schedule a private briefing, in person or via a web meeting. XtremIO X2 provides benefits in many environments
and mixed workload consolidations, including virtual server, cloud, virtual desktop, database, analytics and business
applications.
Learn more about Dell
EMC XtremIO
Contact a Dell EMC Expert View more resources Join the conversation
@DellEMCStorage and
#XtremIO
© 2019 Dell Inc. or its subsidiaries. All Rights Reserved. Dell, EMC and other trademarks are trademarks of Dell Inc. or
its subsidiaries. Other trademarks may be trademarks of their respective owners. Reference Number H17839

DELL EMC XTREMIO X2 CSI PLUGIN INTEGRATION WITH PIVOTAL PKS A Detailed Review

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to DELL EMC XTREMIO X2 CSI PLUGIN INTEGRATION WITH PIVOTAL PKS A Detailed Review

Similar to DELL EMC XTREMIO X2 CSI PLUGIN INTEGRATION WITH PIVOTAL PKS A Detailed Review (20)

More from Itzik Reich

More from Itzik Reich (9)

Recently uploaded

Recently uploaded (20)

DELL EMC XTREMIO X2 CSI PLUGIN INTEGRATION WITH PIVOTAL PKS A Detailed Review