1. Kubernetes and Windows: At
Scale with Enterprise PKS
Kartik Lunkad, Product Lead, .NET+Windows, Pivotal
Michael Michael, Director of Product, VMware
October 7–10, 2019
Austin Convention Center

2. Unless otherwise indicated, these slides are © 2013-2019 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Safe Harbor Statement
The following is intended to outline the general direction of Pivotal's offerings. It is intended for information
purposes only and may not be incorporated into any contract. Any information regarding pre-release of
Pivotal offerings, future updates or other planned modifications is subject to ongoing evaluation by Pivotal
and is subject to change. This information is provided without warranty or any kind, express or implied, and
is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making
purchasing decisions regarding Pivotal's offerings. These purchasing decisions should only be based on
features currently available. The development, release, and timing of any features or functionality described
for Pivotal's offerings in this presentation remain at the sole discretion of Pivotal. Pivotal has no obligation to
update forward looking information in this presentation.

3. Unless otherwise indicated, these slides are © 2013-2019 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Three
takeaways
- It’s a good time to evaluate your .NET/Windows
workloads and make decisions about
invest/migrate/retire
- Managing Windows worker nodes in PKS is
mostly the same as managing Linux workers.
- Several/Active investments across Pivotal &
VMware for Windows support in Kubernetes w/
Pivotal Platform

4. Unless otherwise indicated, these slides are © 2013-2019 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Agenda
- Windows Containers: What and Why?
- Windows + Kubernetes Ecosystem
- Windows in Pivotal Container Service (PKS)
- Decision Making: PAS or PKS? Invest or Migrate?

5. Unless otherwise indicated, these slides are © 2013-2019 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Windows Containers: What and Why

6. Unless otherwise indicated, these slides are © 2013-2019 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Evolution of Windows Containers Ecosystem
2016
Sept
2017
Sept
2018
March
2018
Oct
2019
March
PASW 2.1 supports
“real” Windows
containers with
Windows Server 1709
2019
August
PKS 1.5 supports
Windows containers
with Windows Server
2019
2020
March
PASW 2.5
supports
Windows
Server 2019
Windows GA in
Kubernetes 1.14
Microsoft introduces
Windows containers in
Windows Server 2016
Windows Server
2019 launch

7. Unless otherwise indicated, these slides are © 2013-2019 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
What are Windows Containers?
Types of Windows Container Isolation
Process Isolation
• Similar to containers on Linux
• Containers share the same kernel with the host, as well as
each other
• Used in PASW & PKS
Hyper-V Isolation
• Hyper-V role required (supports nested virtualization, but
can’t mix hypervisors)
• Containers run in a special Virtual Machine, stripped down
to bare minimum
• Minimal driver/kernel support
• Provides kernel level isolation between each container as
well as the container host
Windows Container Images Published (by Microsoft)
Nano Server
• Leanest Windows operating system, purpose built for
cloud native apps
• Supports .NET core
• PKS-compatible
Windows Server Core
• For applications that require the full .NET stack
• Used in PASW; PKS-compatible
Windows Server
• Full dependency on Windows, including all libraries
and dependencies
• Least likely to be used
• PKS-compatible

8. Unless otherwise indicated, these slides are © 2013-2019 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Why Windows Containers? In the Kubernetes
ecosystem?
- End-of-support for Windows Server 2008 coming up.
- PASW has proven .NET workloads run well in Windows Containers
- Leverage your investment in using Kubernetes for all types of workloads
- Opportunity to streamline your cloud migration strategy on a single platform
(lift-or-shift and/or migrate)
- Operational efficiencies with using a single platform to manage legacy
workloads

9. Unless otherwise indicated, these slides are © 2013-2019 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Windows + Kubernetes

10. Unless otherwise indicated, these slides are © 2013-2019 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Windows + Kubernetes OSS Interestings
- Flannel overlay support
- Beta support for gMSA in Kubernetes 1.16
- Stable release of Windows Server 2019 support in Kubernetes 1.14
- Alpha: Improvements to setup & node join experience with kubeadm
- Alpha: Introducing support for Container Storage Interface (CSI)

11. Unless otherwise indicated, these slides are © 2013-2019 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Future Kubernetes + Windows OSS Investments
1. CRI-ContainerD (sig-node collaboration)
2. Kubeadm support to enable Cluster API support (sig-cluster-lifecycle
collaboration)
3. Continuing advancements in gMSA and Windows workload identity
(sig-node/sig-api/sig-auth collaboration)
4. More CNIs and Storage plugins
5. Envoy support

12. Unless otherwise indicated, these slides are © 2013-2019 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Windows in PKS

13. Unless otherwise indicated, these slides are © 2013-2019 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
PKS: A Runtime for Windows Containers
BOSH
Harbor
NSX-T
Kubernetes
K8s Cluster
K8s ClusterLinux &
Windows
K8s Clusters
Built with open-source Kubernetes — Constant compatibility
with the current stable release of Kubernetes, operated by
BOSH. No proprietary extensions.
Production-ready — Highly available from apps to infrastructure,
no single points of failure with master, etcd and worker nodes
spanning multiple availability zones. Built-in health checks,
scaling, auto-healing and rolling upgrades.
Multicloud — BOSH provides a reliable and consistent
operational experience. For any cloud.
Network management and security out-of-the-box with VMware
NSX-T. Automated microsegmentation and multi-tenant isolation.
Fully automated Ops — Fully automated deploy, scale, patch,
upgrade. No downtime. Use CD pipelines to deploy your
platform, too.
Zero downtime upgrades — Upgrade to the latest version of
Kubernetes, apply maintenance without impacting availability
using BOSH’s Day 2 capabilities.
VMware GCP Azure AWS
PKSController

14. Windows K8s Cluster (dedicated Windows Workers)
Platform User
PKSControlPlane
CLI
API
PKS CREATE CLUSTER
BOSH
deploy
Kubernetes cluster
Create
Harbor
Master
Worker
WorkerWorker
etcd
Windows
Worker
Master
etcd

15. Unless otherwise indicated, these slides are © 2013-2019 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Windows (beta) in PKS 1.5 Release Update
Features
- vSphere support yet
- Flannel networking mode support
- Support for Windows Server 2019
- Kubernetes v1.14
- Single common stemcell for both PASW & Windows in PKS
- Windows support in PKS… no separate PKSW tile!

16. Unless otherwise indicated, these slides are © 2013-2019 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Windows in PKS Outcomes in upcoming releases
- Enable latest security patching for Windows Worker Nodes
- Enable Windows worker-based cluster creation in internet-less environment
- Windows and Linux Worker support in a single Kubernetes Cluster
- Azure and AWS Support for Windows in PKS

17. Unless otherwise indicated, these slides are © 2013-2019 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Kubernetes + Windows Investments @ Pivotal & VMware
Windows w/ PKS GA
VMware and Pivotal have partnered to
drive forward Windows support in
Kubernetes 1.14 and PKS 1.5.
Integrated Windows Authentication
support for .NET framework apps
Windows authentication is a pervasive
technology for legacy .NET apps. We are
exploring how legacy apps can continue
to leverage it.
Windows w/ Pivotal Build Service
Pivotal Build Service provides a
declarative configuration model,
consistent and up to date container
images and operators the ability to
restrict usage of buildpacks in the apps
they supervise
Dev Productivity: Log & Metrics Sinks
for .NET/Windows Apps
PAS provides aggregated metrics &
logging across your application instances
out of the box. K8s doesn’t. We are
investing in making this a reality for
.NET/Windows Apps.
NSX-T w/ Windows in Kubernetes
Pivotal & VMWare NSX-T team are
collaborating on introducing compatibility
of NSX-T with Windows in PKS.
Envoy support for Windows
This work aims to provide
container-to-container networking plus
the Istio weighted routing and other
features of the service mesh.
K8s + Windows OSS Contributions
We are making contributions to the
open-source community in the k8s +
Windows space
Windows in PAS-on-k8s
Pivotal have committed to exploring the
Eirini Kubernetes-based scheduler for
Pivotal Application Service. We are
working to introduce Windows support as
part of this story.
Persistent disk support for .NET
framework apps
Pivotal along with the OSS Kubernetes
community, is exploring persistent volume
support for Windows apps.

18. Unless otherwise indicated, these slides are © 2013-2019 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Decision making: PAS or PKS? Invest or
Migrate?

19. Hardware
IaaS
Container Orchestrator
Application Platform
Serverless
Functions
Strategic goal: Push as many workloads as technically
feasible to the top of the platform hierarchy
Higher flexibility and
less enforcement of
standards
Lower development
complexity and higher
operational efficiency

20. .NET Portfolio Landscape
Invest : Greenfield Invest/Maintain: Modernize Maintain: Legacy, Lift-and-shift Divest
Depends heavily on non-cloud-native patterns
(local state, file system dependencies, etc.).
Use a variety of Windows / IIS technologies
(COM+ assemblies, GAC, registry).
Leverages Integrated Windows Authentication
in a way that’s difficult to refactor.
Older workloads for which there is no
dedicated team.
The .NET Core future is here.
Opportunity for new patterns like
microservices, functions,
event-driven architectures, etc.
More easily transformable to a
cloud-friendly (and eventually, cloud-native)
state.
The platform provides support for common
IIS, .NET, Windows technologies.
Flexibility to decide the modernization
strategy.

21. Unless otherwise indicated, these slides are © 2013-2019 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
When to choose PKS for .NET/Windows workloads?
If one or more apply from the below!
- Legacy .NET Framework dependencies (3.5 or maybe 2.0?)
- Application has specific environment needs (IIS, Registry etc)
- Application needs administrative access inside the Windows container
- Application development team would like to manage the entire container image
creation & management lifecycle
- No source code for the application

22. Unless otherwise indicated, these slides are © 2013-2019 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Want to learn more about
migrating patterns for your
.NET framework portfolio?
Learn from the experts.

23. Unless otherwise indicated, these slides are © 2013-2019 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Three
takeaways
- It’s a good time to evaluate your .NET/Windows
workloads and make decisions about
invest/migrate/retire
- Managing Windows VMs in PKS is mostly the
same as managing Linux VMs.
- Several active investments across Pivotal &
VMWare for Windows support in Kubernetes &
PKS

24. Stay Connected
@kartikkl, klunkad@pivotal.io
@michmike77, michaelmi@vmware.com
#springone@s1p

25. Unless otherwise indicated, these slides are © 2013-2019 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
What do you want to “own”
DIY k8s or container stack
Embedded OS
OS Image
Runtime Layer
Service Brokerage
Application Layer
Platform
Provided
App
Team
provided
Embedded OS
OS Image
Runtime Layer
Service Brokerage
Application Layer
Platform
Provided
App
Team
Provided
Embedded OS
OS Image
Runtime Layer
Service Brokerage
Application Layer
App
Team
Provided

26. Unless otherwise indicated, these slides are © 2013-2019 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
2
6


27. > kubectl
StorageCompute
Pivotal Container Service (PKS) Provides:
Kubernetes Dashboard
Dev / Apps
PKS Control Plane
App User
vRealize Ops*
*integration
IT / Platform Ops
> pks
Operations
Manager
vRealize Operations*

28. > kubectl
Multi-Cloud PKS
Kubernetes Dashboard
vRealize Ops
PKS Control Plane
> pks
Operations
Manager
vRealize Operations
Dev / Apps
App User
IT / Platform Ops

29. Windows K8s Cluster (dedicated Windows Workers)
Platform User
PKSControlPlane
CLI
API
PKS CREATE CLUSTER
BOSH
deploy
Kubernetes cluster
Create
Harbor
Master
Worker
WorkerWorker
etcd
Windows
Worker
Master
etcd