Virtualization technolegys for amdocs

Copyright © 2018 LOGTEL
Technology
overview
Samuel Dratwa
Samuel.dratwa@gmail.com

 Introduction to virtualization
 Cloud architecture
 Openstack
 Docker Containers
 Spring Cloud
 K8S/Kubernetes
 Red Hat OpenShift
 Docker containers
Agenda
2

Copyright © 2011 LOGTEL 3
What are we selling ?
It’s all about
customer satisfaction
customer satisfaction !

Why do we need virtualization ?
 Reduce OPEX & CAPEX
 Isolation
 Abstraction
 Reuse

A virtual machine is implemented by adding software to an
execution platform to give it the appearance of a different
platform, or for that matter, to give the appearance of multiple
platforms.
What does a VM look like
Without VMs: Single OS owns all hardware resources With VMs: Multi-OSes share hardware resources

 Virtualization is a term that refers to the abstraction of
computer resources, or the act of creating a virtual (rather
than actual) version of something, including
virtual computer hardware platforms, operating
systems, storage devices, and computer network resources.
What is virtualization
https://en.wikipedia.org/wiki/Virtualization
Physical computer resources
Virtualized computer resources
abstraction virtualization

 In the 1960s
 Firstly appeared in IBM mainframe computer OS.
 Used as a method of dividing system resources between
different applications.
 Since then, before 1999
 the development of virtualization was almost at a standstill
about 30 years.
 1999-, develop to climax
 Vmware, the first virtual platform for Intel x86-32 architecture
computers.
 2003, UCCL, Xen, the first open-source, drive more academia
research on virtualization.
 2007, KVM, a linux OS kernel-based VM
 2007, hardware support: Intel VT, AMD-V
 Now ~50 virtualization tools or types
History

 Hardware virtualization
 Software virtualization
 Other virtualization
 Desktop virtualization
 Nested virtualization
Virtualization technology types

 Hardware virtualization (most fundamental)
 or platform virtualization refers to the creation of a virtual
machine (VM) that acts like a real computer with an
operating system (OS).
 Software executed on these VMs is separated from the
underlying hardware resources.
 Host with Microsoft Windows  VM  Ubuntu linux OS;
Hardware virtualization (HV)
Host machine
Guest machine
Hypervisor
 Concepts
 Host/guest
 Hypervisor or
Virtual Machine
Manager (VMM)

Virtual machine monitor (VMM)
VMM transforms the single machine interface into the illusion of many
interfaces. Each of these interfaces (virtual machines) is an efficient replica
of the original computer system, complete with all of the processor
instructions.
Robert P. Goldberg.

Two Hypervisor Types
Vmware Esx server
Xen ctrix server
VMware Esx workstation
Xenserver

 Nested virtualization
 refers to the ability of running a virtual machine within
another, having this general concept extendable to an
arbitrary depth.
 Can be used to deploy a cloud platform in VMs.
Nested virtualization
VM
VM
VM
VM

 Equivalence
 Essentially identical virtual platform, except
 Differences caused by the availability of system resources
 E.g. amount of memory available to VM
 Differences caused by timing dependencies.
 Isolation, or resource control
 VMM is in complete control of system resources
 VM can’t access any resources not explicitly allocated to it
 VMM may regain control of resources already allocated
 Efficiency
 At worst only minor decreases in speed
 Rules out traditional emulators and complete software interpreters
(simulators)
Essential characteristics of VMM
Source from Gerald J. Popek & Robert P. Goldberg, “Formal
Requirements for Virtualizable Third Generation Architectures”

 How to virtualize CPU resource?
 VM is set one or more vCPUs and each vCPU is allocated to run
applications of Guest OS.
 Each application program is denoted by an ordered set of
instructions and finally executed by physical CPU.
 During the running of application, app instructions will firstly
submitted to vCPU, then the vCPUs are mapped to or allocated with
physical CPU to run instructions through VMM.
CPU virtualization
vCPU vCPU …... vCPU
CPU CPU …… CPU
VMs
PM
VMnVM0
MAP / allocate
APP (Instructions ) APP (Instructions )
VMM

 Binary Translation (BT)
 Translate guest OS binary on the fly to solve virtualization issue
 Paravirtualization (PV)
 Hardware-assisted virtualization
CPU virtualization approaches

 Translate guest OS binary
 OS binary is still visible to guest, but the
executed guest code is actually in translation
cache.
 Need to steal guest address space to hold
the translated cache.
 be able to run unmodified guest OS
 All the instructions are virtualized for
translation.
 Frequent translations impact on
performance
 Benefit: not modify guest OS
Binary Translation (BT)
Guest Operating
System
binary
Physical
instructions
Translated by VMM

 Modify guest OS source to cooperatively work with
hypervisor for performance, simplicity etc.
 Hypercall to request for hypervisor service
 Share the global resource
 Event channel to receive asynchronous notification from hypervisor
 Share memory for massive information communication
 Virtio for directly access IO device
 Widely used in device driver by commercial VMMs
 Vmware
 Xen/KVM
 Hyper-V
Paravirtualization (PV)

 Hardware extension
 Intel® Virtualization Technology (Intel® VT)
 AMD-V
 Hardware support to enable run
 Guest OS runs in de-privileged mode to execute instructions
 Guest access to privileged resources triggers exit from VM to VMM
 Be able to run unmodified guest OS
Hardware-assisted virtualization

 In OS, memory management has the rules
 OS expect to see physical memory starting from 0
 BIOS/Legacy OS are designed to boot from address low 1M
 OS expect to see contiguous memory in address space
 OS kernel requires minimal contiguous low memory
 OS, such as Linux, requires minimal contiguous low memory
 OS components may require certain level of contiguous memory
 For example, DMA pages, identical mapped kernel data structure…
 Efficient page frame management
 Less management overhead
 Efficient TLB (Translation lookaside buffer)
 Super page TLB
Memory virtualization

Physical page frame redirection

 Partitioning
 Simple and high performance
 $ to buy more memory
 Contents based sharing
 Pages with same contents can share the physical frame
 Such as zero page, guest code pages etc.
 Can improve memory usage
 Ballooning
 A VMM-aware balloon driver running in guest OS to dynamically allocate
memory from OS and release them to VMM, and vice versa
 Host swapping
 The physical frames for guest pages may be swapped out
 Shadow page table
 Guest memory <-- physical machine memory
Page frame allocation

Ballooning

 Xen
 KVM/Qemu
 Vmware
 virtualbox
 LXC & Docker
Commercial virtualization tools

 Produced in 2003.
 Open-source virtualization tool
 The University of Cambridge Computer Laboratory
developed the first versions of Xen
 Xen is currently available for the IA-32, x86-64 and ARM
instruction sets.
 Xen supports five different approaches to running the guest
operating system:
 HVM (hardware virtual machine),
 HVM with PV drivers,
 PVHVM (HVM with PVHVM drivers),
 PVH (PV in an HVM container) and PV (paravirtualization)
Xen

 Produced in 2007.
 KVM (Kernel-based Virtual Machine)
 is a virtualization infrastructure for the Linux kernel that turns it into
a hypervisor.
 supports x86 processors
 Paravirtualization support for certain devices
 is available for Linux, OpenBSD, FreeBSD, NetBSD, Plan 9 and
Windows guests using the Virt-IO API.
 This supports a para virtual Ethernet card, a para virtual disk I/O
controller, a balloon device for adjusting guest memory usage, and
a VGA graphics interface using SPICE or VMware drivers.
KVM

 QEMU (short for Quick Emulator)
 is a free and open-source hosted hypervisor that performs
hardware virtualization (not to be confused with hardware-assisted
virtualization).
 QEMU is a hosted virtual machine monitor
 It emulates CPUs through dynamic binary translation and provides a
set of device models, enabling it to run a variety of unmodified
guest operating systems.
 It also can be used together with KVM in order to run virtual
machines at near-native speed (requiring hardware virtualization
extensions (VT-x) on x86 machines).
 QEMU can also be used purely for CPU emulation for user-level
processes, allowing applications compiled for one architecture to be
run on another.
Qemu

 Produced in 1999. VMware Inc. product. It is non-open
source.
 VMware's desktop software runs on Microsoft Windows, Linux, and
Mac OS X.
 Its enterprise software hypervisors for servers, VMware ESX and
VMware ESXi, are bare-metal hypervisors that run directly on server
hardware without requiring an additional underlying operating
system.
 Desktop software
 VMware workstation, VMware Fusion
 VMware player: is freeware for non-commercial use
 Server software
 VMware vSphere
 VMware server
VMware

 is a hypervisor for x86 computers from Oracle Corporation.
 VirtualBox may be installed on a number of host operating systems,
including: Linux, OS X, Windows Vista, Solaris, and OpenSolaris.
 Support Software-based virtualization and Hardware-assisted
virtualization
virtualbox

 LXC (Linux Containers)
 is an operating-system-level virtualization environment for running
multiple isolated Linux systems (containers) on a single Linux
control host.
 The Linux kernel provides the cgroups functionality that allows
limitation and prioritization of resources (CPU, memory, block I/O,
network, etc.)
 Docker
 a project automating deployment of applications inside software
containers
 can also use LXC as one of its execution drivers, enabling image
management and providing deployment services.
LXC (linux container) and Docker

 VM Image file stores all contents of a VM.
 Two modes: mirror and sparse mode
 The mirror mode image stores all byte data, includes invalid data for
users. Raw format is a mirror mode image.
 The sparse mode image stores the user or system valid data, only
occupy a special necessary size of storage space. The qcow2 and
vmdk are sparse mode file formats.
 File format
 General image: qcow, qcow2 & raw
 Commercial image: vmdk, vdi, …
 Each commercial virtualization has its own image format.
VM image file

 raw format (KVM/Xen)
 (linux default) the raw format is a plain binary image of the disc image,
and is very portable.
 On file systems that support sparse files, images in this format only use the
space actually used by the data recorded in them.
 qcow2 (QEMU copy-on-write)
 QEMU copy-on-write format with a range of special features, including the
ability to take multiple snapshots, smaller images on file-systems that don’t
support sparse files, optional AES encryption, and optional zlib compression.
 Other
 vmdk: VMWare file format
 vdi: virtualbox file format
 Image File format convert
 In general a commercial VMM can convert its format file to the raw formant
Choosing Image format

Network Communication
PM2
HW
PM1
HW
VM1 VM2
VM1 VM2
Application
Disperse placement
(DP)
Network
VMM VMM
Through NIC
Communication
PM
Hardware(HW)
VM1 VM2
VM1 VM2
Application
Colocation-placement
(CP)
VMM Virtual
Network
Pass through vNIC
but not NIC

 Physical network
 NIC, switch
 Virtual Network
 vNIC, vSwitch
 Virtual network devices
 TAP/TUN
 Linux Bridge
Virtual network concepts

Physical network
 Many computers with distinct
geolocations are connected together,
creating a physical network.
 Network connection needs network
devices.
 Network interface controller(NIC)
 A computer or server must be network-capable with a
working NIC or a network card installed.
 The NIC enables the computer to interact with a network.
 Switch and local area network (LAN)
 Computers are usually connected to a device called a switch,
which creates a local area network (LAN).
 Switches are responsible for intelligently routing this
network traffic to the appropriate destination.
 Network protocol: TCP/IP

Virtual network
 In virtualization network devices are virtualized, such as
vNIC and vSwitch.
 Many VMs connected together with virtual network
devices creates a virtual network.
 Virtual network interface card (vNIC)
 Hypervisor can create one or more vNICs for each VM.
 The vNIC provides the networking capabilities of the VM.
 Each vNIC is identical to a physical NIC.
 Virtual Switch(vSwitch)
 Switch also can be virtualized as a virtual switch.
 Each vNIC is connected to the vSwitch port, and these vSwitch access
external physical network through the physical NIC of Physical Server.
 e.g., TAP/TUN

TAP/TUN
 TAP / TUN
 is a pair of virtual network devices based on Linux kernel
implementation.
 is widely used to realize the connection between the VM and PM.
 TAP
 is equivalent to an Ethernet device, which operates the second-
layer packets such as Ethernet data frames. (can see in linux by
ifconfig)
 TUN
 simulates the network layer devices, and operate the third
layer packet such as IP data packets. router or switch (vSwitch)

What is
Cloud Computing?

 Wikipedia - a type of Internet-based computing that
provides shared computer processing resources and
data to computers and other devices on demand. It is a
model for enabling ubiquitous, on-demand access to a
shared pool of configurable computing resources
(computer networks, servers, storage, applications and
services),which can be rapidly provisioned and released
with minimal management effort.
 Academic (simple) - provide on-demand resources or
services over a network, often the Internet, with the
scale and reliability of a data center.
 Mine – distributed mainframe, reachable from any
device connected to the internet
Definition
38

The “old way”
39
Server(s)
PC(s) / Laptop(s)Software
Buy, Install,
Maintain, Upgrade

The “new way”
40
Server(s)
PC(s) / Laptop(s)Software
Buy, Install,
Maintain, Upgrade

It’s not just above the line
41

Also (almost) all the rest
42
X
X
X
X

Cloud Ecosystem
Public
Cloud
Enterprise
Private Cloud Private Cloud
External
Internal
Virtual Private Cloud
43

Three layers of services
IaaS
Infrastructure as a Service
PaaS
Platform as a Service
SaaS
Software as a Service

SaaS
PaaS
IaaS
IaaS, PaaS and SaaS Point of View
Platform as a Service
PRODUCT: storage, compute and
other services to simplify application
development, especially of web
applications.
USERS: Application Developers
Software as a Service
PRODUCT: Finished
application available on
demand to end user
USERS: Software consumer
Infrastructure as a Service
PRODUCT: Compute power, storage
and networking infrastructure over the
internet, provided as a virtual machine
image
USERS: Developers

SaaS examples
46

 Pay per use
 Instant Scalability
 Security
 Reliability
 Disaster recovery
 APIs
SaaS - Advantages
SaaS

PaaS examples

Amazon EC2
49

Amazon pricing
50

 Pay per use
 Security
 Reliability
 APIs
PaaS - Advantages
PaaS

IaaS - infrastructure delivery model
IaaS
 Access to infrastructure stack:
 Full OS access
 Firewalls
 Routers
 Load balancing
 Commercial Services:
 Flexiscale
 AWS: EC2
 RackSpace

IaaS - Advantages
IaaS
 Pay per use
 Security
 Reliability
 APIs

 Pay per use
 Security
 Reliability
 APIs
Common Factors
IaaS
PaaS
SaaS

 Lower cost of ownership
 Reduce infrastructure management
responsibility
 Allow for unexpected resource loads
 Faster application rollout
Advantages
IaaS
PaaS
SaaS

Aren’t we missing something ?
 NaaS – Network (connectivity) as a Service
 Manage the network itself
 Having on demand bandwidth
 Network abstraction
56

 Security
 Downtime
 Access
 Dependency
 Interoperability
 Regulation
Risks
IaaS
PaaS
SaaS

Cloud and the “security problem”
 The good news
 The cloud providers are better at security than we are
(hey, it’s their specialty…)
 The bad news
 It’s a different ball game
 With bigger threats
58

Moving from “not so smart” to
“the smarter”
59

Standards are on the way here
60

Cloud Security Alliance
61

The players
63

Three Cloud Models – You Choose.
Über-cloud
Applications
Database
OS
Server
Storage
Middleware
Verticalization
Applications
Database
OS
Middleware
Virtual Infrastructure
VirtualizationAggregation

The big players
65

Cloud Architecture
66

Cloud computing is kind of hosting
67

What is there in a Cloud?
Individuals Corporations Non-Commercial
Cloud provisioning
Storage
Provisioning
OS
Provisioning
Network
Provisioning
Service(apps)
Provisioning
SLA(monitor),
Security, Billing,
Payment
Services Storage Network OS
Resources

What is Cloud OS ?
69
Physical
Node
Physical
Node
Storag
e
Server
Storag
e
Server
Physical
Node
Physical
Node
Storage
Server
Storage
Server
Mail
Virtual
Cluster
Compute
Nodes
Backup
Virtual
Cluster
HC
Virtual
Cluster
AppX
Virtual
Cluster
Data NodesService Nodes
System
Service
daemons
Cloud
OS
agents
• System Management Software layer
– Physical Resource Provisioning
– Virtual Resource Management
• Improve manageability of massive
Cloud Data Center
• Enhance self-provisioning
• Optimize physical resource utilization
• High Availability for any single point
of failure
• Energy management
– Highly Available Distributed
Storage Management
– Service Load Balancing
– Security
– High Speed Networking
• What is it not?
– It’s not Operating System
– It’s not Virtualization Hypervisor

CloudOS Virtualization Level
70
… PM
OSAPs
vm
OSAPs
vm
OSAPs
vm
OSAPs
vm
…
PM
OSAPs
vm
OSAPs
vm
OSAPs
vm
OSAPs
vm
…
PM
OSAPs
vm
OSAPs
vm
OSAPs
vm
OSAPs
vm
…
PM
OSAPs
vm
OSAPs
vm
OSAPs
vm
OSAPs
vm
…
VCluster VCluster VCluster
VDC VDC

Load balancing
71

Consolidation plan
72

Google’s Chrome OS
74

Cloud Data Center

Google data center
76

IBM
77

Amazon
78

Infrastructure
AsaService
Compute Storage Network
Physical
Infrastructure
IaaS
Compute
(Nova*)
Block Storage
(Cinder*)
Object Storage
(Swift*)
Network
(Neutron*)
Dashboard (Horizon*)
OS Images
(Glance*)
Open-Source (OpenStack*)
Manageability
Monitoring
AsaService
Watcher
(Nagios*, Shinken*,
Heat*)
Decider
(Heat)
Collector
(Hadoop*)
Actor
(Puppet*, Cfengine*)
Open-Source Foundation
Interfaces
GUI
(Graphical User Interface)
API
(Application Programming Interface)
AppPlatform
Services
PaaS
Analytics Messaging Data Web
Open Cloud Components
82

OpenStack components
83

Conceptual architecture
84

Open Stack – A Cloud Operating System
Images
CentOS
XMS VNFc
CentOS
MRB VNFc
CentOS
Load Balancer
VNFc
Flavors
4 CPU
8GB RAM
50GB Disk
8 CPU
16GB RAM
50GB Disk
12 CPU
16GB RAM
50GB Disk
 Stack: Collection of
resources
 Virtual machines,
networks, auto scaling
rules
 Template: Definition of
resources that make up
the Stack
 Four sections
 Resources – Objects
that will be created -
like a server
 Properties – image,
flavor
 Parameters –
Property values
 Output – Information
passed back to user or
dashboard
Stacks
Auto scaling
policy
Parameters
Stack
Heat Templates contain info to create stacks

Resources

Parameters

The future
88

April 10th 2015 – Amazon start to push Docker (!)

Multiplicityof
Goods
Multipilicityof
methodsfor
transporting/storing
DoIworryabout
howgoodsinteract
(e.g.coffeebeans
nexttospices)
CanItransport
quicklyandsmoothly
(e.g.fromboatto
traintotruck)
Cargo Transport Pre-1960

MultiplicityofGoods
Multiplicityof
methodsfor
transporting/storing
DoIworryabout
howgoodsinteract
(e.g.coffeebeans
nexttospices)
CanItransport
quicklyand
smoothly
(e.g.fromboatto
traintotruck)
Solution: Intermodal Shipping Container
…in between, can be loaded and
unloaded, stacked, transported
efficiently over long distances,
and transferred from one mode
of transport to another
A standard container that is
loaded with virtually any
goods, and stays sealed
until it reaches final delivery.

Static website
Web frontend
User DB
Queue Analytics DB
Background workers
API endpoint
nginx 1.5 + modsecurity + openssl + bootstrap 2
postgresql + pgv8 + v8
hadoop + hive + thrift + OpenJDK
Ruby + Rails + sass + Unicorn
Redis + redis-sentinel
Python 3.0 + celery + pyredis + libcurl + ffmpeg + libopencv + nodejs +
phantomjs
Python 2.7 + Flask + pyredis + celery + psycopg + postgresql-client
Development VM
QA server
Public Cloud
Disaster recovery
Contributor’s laptop
Production Servers
The Challenge
Multiplicityof
Stacks
Multiplicityof
hardware
environments
Production Cluster
Customer Data Center
Doservicesand
appsinteract
appropriately?
CanImigrate
smoothlyand
quickly?

Static website Web frontendUser DB Queue Analytics DB
Development
VM
QA server Public Cloud Contributor’s
laptop
Docker is a shipping container system for
code
MultiplicityofStacks
Multiplicityof
hardware
environments
Production
Cluster
Customer Data
Center
Doservicesand
appsinteract
appropriately?
CanImigrate
smoothlyand
quickly
…that can be manipulated using
standard operations and run
consistently on virtually any
hardware platform
An engine that enables any
payload to be encapsulated
as a lightweight, portable,
self-sufficient container…

Docker’s mission is to
build tools of mass innovation

Cloud Market
PublicHybridPrivate
IT Pros Devops DevelopersArchitects

Low MTBIAMSH
MTBIAMSH (Mean Time Between Idea And Making Stuff Happen)

Why Developers Care
• Build once…(finally) run anywhere*
• A clean, safe, hygienic and portable runtime environment for your app.
• No worries about missing dependencies, packages and other pain points during
subsequent deployments.
• Run each app in its own isolated container, so you can run various versions of
libraries and other dependencies for each app without worrying
• Automate testing, integration, packaging…anything you can script
• Reduce/eliminate concerns about compatibility on different platforms, either
your own or your customers.
• Cheap, zero-penalty containers to deploy services? A VM without the overhead
of a VM? Instant replay and reset of image snapshots? That’s the power of
Docker
* Almost ;-)

Why Devops Cares?
• Configure once…run anything
• Make the entire lifecycle more efficient, consistent, and repeatable
• Increase the quality of code produced by developers.
• Eliminate inconsistencies between development, test, production, and
customer environments
• Support segregation of duties
• Significantly improves the speed and reliability of continuous
deployment and continuous integration systems
• Because the containers are so lightweight, address significant
performance, costs, deployment, and portability issues normally
associated with VMs

Why it works—separation of concerns
• Dan the Developer
• Worries about what’s “inside”
the container
• His code
• His Libraries
• His Package Manager
• His Apps
• His Data
• All Linux servers look the same
• Oren the Ops Guy
• Worries about what’s “outside”
the container
• Logging
• Remote access
• Monitoring
• Network configuration
• All containers start, stop, copy,
attach, migrate, etc. the same
way

Containers before Docker……

Containers after Docker ……

Docker Containers as a Service Platform
Docker
Toolbox
Docker
Trusted Registry
Docker Universal
Control Plane
Build Ship Run
• “docker push”
with image signing
• Search/browse repos
• Teams-based RBAC
• View signed images
• Deleting tags
• Authentication
• Deploy and scale-out app
• Monitor stats
• Secrets management

App
A
Containers vs. VMs
Hypervisor (Type 2)
Host OS
Server
Gues
t
OS
Bins/
Libs
App
A’
Gues
t
OS
Bins/
Libs
App
B
Gues
t
OS
Bins/
Libs
AppA’
Docker
Host OS
Server
Bins/Libs
AppA
Bins/Libs
AppB
AppB’
AppB’
AppB’
VM
Container
Containers are isolated,
but share OS and, where
appropriate, bins/libraries
Gues
t
OS
Gues
t
OS
…result is significantly faster deployment,
much less overhead, easier migration,
faster restart

Why are Docker containers lightweight?
Bins/
Libs
App
A
Original App
(No OS to take
up space, resources,
or require restart)
AppΔ
Bins
/
App
A
Bins/
Libs
App
A’
Gues
t
OS
Bins/
Libs
Modified App
Copy on write
capabilities allow
us to only save the diffs
Between container A
and container
A’
VMs
Every app, every copy of an
app, and every slight modification
of the app requires a new virtual server
App
A
Gues
t
OS
Bins/
Libs
Copy of
App
No OS. Can
Share bins/libs
App
A
Gues
t
OS
Gues
t
OS
VMs Containers

Docker engine Architecture……
• Docker Engine
– CLI
– Docker Daemon
– Docker Registry
• Docker Hub
– Cloud service
• Share Applications
• Automate workflows
• Assemble apps from components
• Docker images
• Docker containers

Isolation using Linux kernel features
namespaces
 pid
 mnt
 net
 uts
 ipc
 user
cgroups
 memory
 cpu
 blkio
 devices

Linux Cgroups ……
• Kernel Feature
• Groups of processes
• Control resource allocations
– CPU
– Memory
– Disk
– I/O
• May be nested

• Kernel Feature
• Restrict your view of the system
– Mounts (CLONE_NEWNS)
– UTS (CLONE_NEWUTS)
• uname() output
– IPC (CLONE_NEWIPC)
– PID (CLONE_NEWPID)
– Networks (CLONE_NEWNET)
– User (CLONE_NEWUSER)
• Not supported in Docker yet
• Has privileged/unprivileged modes today
• May be nested
Linux Kernel Namespaces ……

What are the basics of the Docker
system?
Source
Code
Repositor
y
Dockerfil
e
For
A
Docker Engine
Docker
Container
Image
Registry
Build
Docker
Host 2 OS (Linux)
ContainerA
Container
B
Container
C
ContainerA
Push
Search
Pull
Run
Host 1 OS (Linux)

Changes and Updates
Docker Engine
Docker
Container
Image
Registry
Docker Engine
Push
Update
Bins/
Libs
App
A
AppΔ
Bins
/
Base
Container
Image
Host is now running A’’
Container
Mod A’’
AppΔ
Bins
/
Bins/
Libs
App
A
Bins
/
Bins/
Libs
App
A’’
Host running A wants to upgrade to A’’.
Requests update. Gets only diffs
Container
Mod A’

Docker for developers / Dockerfile
https://registry.hub.docker.com/_/java/

• Like a Makefile (shell script with keywords)
• Extends from a Base Image
• Results in a new Docker Image
• Imperative, not Declarative
• A Docker file lists the steps needed to build an images
• docker build is used to run a Docker file
• Can define default command for docker run, ports to expose, etc
Dockerfile ……

docker-compose: running multiple
containers
 Run your stack with one command: docker-compose
up
 Describe your stack with one file: docker-
compose.ymlweb:
build: .
command: python app.py
ports:
- "5000:5000"
volumes:
- .:/code
links:
- redis:redis
redis:
image: redis

docker-machine
docker-machine create -d azure
—azure-subscription-id="c4f51be3-784c-xxx-
7c50ad9e1b7c"
--azure-subscription-cert="/Users/pat/.ssh/docker-
azure-cert.pem"
--azure-location="East US"
--azure-size=Small
--azure-username="pat"
pat-docker-machine-n

Docker Hub
Dev & QA
ColleaguesDevelopers
QA
Build & Ship

Docker Trusted Registry … now with Docker Content
Trust
Docker
Toolbox
Build Ship
Docker
Trusted Registry

Swarm
Scheduler plugins
Engine
Volumes plugins
Network plugins
Service discovery
plugins
Engine
Volumes plugins
Network plugins
Service discovery
plugins
mesos
flockerglusterfs
weavecalico
consuletcdzookeeper
midokuraciscoazurenuagenetworks
Docker Plugins
Batteries included but removable

Docker
CLI
Docker
CLI
Docker
CLI
Docker Engine

Engine 1.9 Release
• Network management
•Top-level docker network command
•Out of the box support for multi host overlay
networking
•Extensibility through plugins
• Volume management
•Top-level docker volume command
•Extensibility through plugins
• Built-in nodes discovery
•Daemon --cluster-store and --cluster-
advertise

Engine 1.9 Release
• Builder
•Refactoring toward client-side build support
•ARG: build-time arguments (provides support for
HTTP_PROXY)
•STOPSIGNAL: choose the stop signal for the
contained process
• Miscellaneous
•Quality, stability, performance improvements
•awslog logging driver
• Experimental
•User namespaces phase 1 (daemon-wide root
remap)

• The Life of a Container
– Conception
• BUILD an Image from a Dockerfile
– Birth
• RUN (create+start) a container
– Reproduction
• COMMIT (persist) a container to a new image
• RUN a new container from an image
– Sleep
• KILL/stop a running container
– Wake
• START a stopped container
– Death
• RM (delete) a stopped container
• Extinction
– RMI a container image (delete image)
Docker Container Lifecycle

Docker is a “Shipping Container”
132

Monolithic App
(Various Components linked together)

MicroServices
separate single purpose services

Monolithic Architecture
Load Balancer
Monolithic App
Account
Component
Catalog
Component
Recommendation
Component
Customer Service
Component
Database

Microservices Architecture
Load Balancer
Account Service Catalog
Service
Recommendation
Service
Customer Service
Service
Catalog
DB
API Gateway
Customer
DB

Concept -> Service Dependency Graph
Your App/Service
Service X
Service Y
Service Z
Service L
Service M

Why ?
• Faster and simpler deployments and rollbacks
• Independent Speed of Delivery (by different teams)
• Right framework/tool/language for each domain
• Recommendation component using Python?, Catalog
Service in Java ..
• Greater Resiliency
• Fault Isolation
• Better Availability
• If architected right 

Spring: the source for modern java
141

Spring Boot
142

Spring Cloud
144

us-west us-east
Docker
CLI
Docker
CLI
Docker Swarm
Swarm

Ecosystem Support
• Operating systems
• Virtually any distribution with a 2.6.32+ kernel
• Red Hat/Docker collaboration to make work across RHEL 6.4+, Fedora, and other members of
the family (2.6.32 +)
• CoreOS—Small core OS purpose built with Docker
• OpenStack
• Docker integration into NOVA (& compatibility with Glance, Horizon, etc.) accepted for Havana
release
• Private PaaS
• OpenShift
• Solum (Rackspace, OpenStack)
• Other TBA
• Public PaaS
• Deis, Voxoz, Cocaine (Yandex), Baidu PaaS
• Public IaaS
• Native support in Rackspace, Digital Ocean,+++
• AMI (or equivalent) available for AWS & other
• DevOps Tools
• Integrations with Chef, Puppet, Jenkins, Travis, Salt, Ansible +++
• Orchestration tools
• Mesos, Heat, ++
• Shipyard & others purpose built for Docker
• Applications
• 1000’s of Dockerized applications available at index.docker.io

Advanced topics
• Data
• Today: Externally mounted volumes
• Share volumes between containers
• Share volume between a containers and underlying hosts
• high-performance storage backend for your production database
• making live development changes available to a container, etc.
• Optional: specify memory limit for containers, CPU priority
• Device mapper/ LVM snapshots in 0.7
• Futures:
• I/O limits
• Container resource monitoring (CPU & memory usage)
• Orchestration (linking & synchronization between containers)
• Cluster orchestration (multi-host environment)
• Networking
• Supported today:
• UDP/TCP port allocation to containers
• specify which public port to redirect. If you don’t specify a public port, Docker will revert to allocating a random
public port.
• Docker uses IPtables/netfilter
• IP allocation to containers
• Docker uses virtual interfaces, network bridge,
• Futures:
• See Pipework (Upstream) : Software-Defined Networking for Linux Containers
(https://github.com/jpetazzo/pipework)
• Certain pipework concepts will move from upstream to part of core Docker
• Additional capabilities come with libvirt support in 0.8-0.9 timeframe

Docker is a “Shipping Container”
154

What’s missing
155

Options for Container Cloud Orchestration on Power
156
Docker Swarm/Datacenter KubernetesMesos
Docker Inc GoogleMesosphere
• Strengths
• Built-in to Docker 1.12 Engine
• Easy to use for Small Clouds
• Weaknesses
• Full Docker DC not on Power Yet
• Strengths
• Good for Batch and Analytics
• Lots of Apps in Catalog
• Weaknesses
• Less usage in Web Applications
• Requires Marathon Framework for
Web Apps
• Strengths
• Lots of Industry usage and
experience for Web Apps
• Synergy with Other parts of Client
Business for X86 Container Mgmt
• Weaknesses
• Significant Integration of many
components for Production Cloud

Kubernetes Layout
157

Swarm
158

Kubernetes
 Pronounced /koo-ber-nay'-tace/
 A Greek term for “ship master”.
 Developed at/by Google.
 The third iteration of container management.
 Daddy was Omega.
 Grandaddy was Borg.
 Kubernetes is not a PaaS, just orchestration
(Docker is the PaaS)
 Kubernetes built to “planet scale”.
 Google wants us to stop writing Kubernetes and use k8s
instead
 100% Open source, written in Go
159

 Kubernetes is an open-source system for automating
deployment, scaling, and management of containerized
applications.
 Planet Scale
 Designed on the same principles that allows Google to run billions
of containers a week, Kubernetes can scale without increasing your
ops team.
 Never Outgrow
 Whether testing locally or running a global enterprise, Kubernetes
flexibility grows with you to deliver your applications consistently
and easily no matter how complex your need is
 Run Anywhere
 on-premise, hybrid, or public cloud infrastructure, letting you
effortlessly move workloads to where it matters to you. 160

Kubernetes Features
161

Kubernetes Layout
162

Master
 API Server—nearly all the components on the master and
nodes accomplish their respective tasks by making API
calls. These are handled by the API Server running on the
master.
 Etcd—Etcd is a service whose job is to keep and replicate
the current configuration and run state of the cluster. It is
implemented as a lightweight distributed key-value store
and was developed inside the CoreOS project.
 Scheduler and Controller Manager—These processes
schedule containers (actually pods) onto target nodes. They
also make sure that the correct numbers of these pods are
running at all times.
163

Node
 Kubelet - A special background process (daemon) that
runs on each node whose job is to respond to commands
from the master to create, destroy, and monitor the
containers on that host.
 Proxy - This is a simple network proxy that’s used to
separate the IP address of a target container from the
name of the service it provides.
 cAdvisor (optional) - Container Advisor is a special
daemon that collects, aggregates, processes, and exports
information about running containers. This information
includes information about resource isolation, historical
usage, and key network statistics.
164

Pods
 A pod is a collection of containers and volumes that are
bundled and scheduled together because they share a
common resource, usually a filesystem or IP address.
165

Why not just run multiple programs
in a single (Docker) container?
 Transparency. Making the containers within the pod
visible to the infrastructure enables the infrastructure to
provide services to those containers, such as process
management and resource monitoring. This facilitates a
number of conveniences for users.
 Decoupling software dependencies. The individual
containers may be versioned, rebuilt and redeployed
independently. Kubernetes may even support live updates
of individual containers someday.
 Ease of use. Users don’t need to run their own process
managers, worry about signal and exit-code propagation,
etc.
 Efficiency. Because the infrastructure takes on more
responsibility, containers can be lighter weight.
166

Kubernetes Architecture
168

Descriptors
apiVersion: v1
kind: Pod
metadata:
name: ""
labels:
name: ""
namespace: ""
annotations: []
generateName: ""
spec:
? "// See 'The spec schema' for
details."
: ~
{
"kind": "Pod",
"apiVersion": "v1",
"metadata": {
"name": "",
"labels": {
"name": ""
},
"generateName": "",
"namespace": "",
"annotations": []
},
"spec": {
// See 'The spec schema' for details.
}
}
169
YAML is the clear winner. Especially in the context of Shannon’s Information Theory.
The same density of information can be transmitted in less lines with YAML.

Pod Spec
spec:
containers:
-
args:
- ""
command:
- ""
env:
-
name: ""
value: ""
image: ""
imagePullPolicy: ""
name: ""
ports:
-
containerPort: 0
name: ""
protocol: ""
resources:
cpu: ""
memory: ""
restartPolicy: ""
volumes:
-
emptyDir:
medium: ""
name: ""
secret:
secretName: ""
170

Pod spec example
apiVersion: v1
kind: Pod
metadata:
name: redis-django
labels:
app: web
spec:
containers:
- name: key-value-store
image: redis
ports:
- containerPort: 6379
- name: frontend
image: django
ports:
- containerPort: 8000
171

The Pod Lifecycle in a Cluster
Let’s say you want to fire up a pod. With kubectl you would:
1. Make a Pod request to the API server using a local pod
definition file.
2. The API server saves the info for the pod in ETCD.
3. The scheduler finds the unscheduled pod and schedules it
to a node.
4. Kubelet sees the pod scheduled and fires up Docker.
5. Docker runs the container.
 The entire lifecycle state of the pod is stored in ETCD.
Most of the things in Kubernetes are built on top of
Pods
172

kubectl create - Create a resource from a file
// Create a service using the definition in example-service.yaml.
$ kubectl create -f example-service.yaml
// Create a replication controller using the definition in example-
controller.yaml.
$ kubectl create -f example-controller.yaml
// Create the objects that are defined in any .yaml, .yml, or .json file
within the <directory> directory.
$ kubectl create -f <directory>
173

Labels
 A label is a key-value pair that is assigned to objects in k8s.
 Pods, services, etc
 Labels can be used to organize and to select subsets of objects
 Labels can be attached to objects at creation time and subsequently
added and modified at any time
174

Label Selectors
 Two kinds of label selectors
 equality-based (IS/IS NOT)
 tier = frontend
 tier != frontend
 tier != frontend, game = super-shooter-2
 set-based (IN/NOT IN)
 environment in (production, qa)
 tier notin (frontend, backend)
175

Services
 A Kubernetes Service is an abstraction which defines a
logical set of Pods and a policy by which to access them
 The set of pods gathered by a Service is determined by a
label selector
 A service is a long-lived, well-known endpoint that points to
a set of pods in a cluster. It consists of three things:
 external IP address (known as a portal, or sometimes a portal IP)
 port
 label selector.
176

Service Example
177

Basic Cluster
178
 When you launch a cluster, you get some built in services.
 Each one of these has their own endpoints and / or UIs.
 They run on the master directly though you could schedule
them across the cluster or other masters.
 To find the endpoints type: kubectl cluster-info

Heapster
 Heapster is a cluster-wide aggregator of monitoring and event data. It
supports Kubernetes natively and works on all Kubernetes setups,
including our Deis Workflow setup. Heapster runs as a pod in the
cluster, similar to how any other Kubernetes application would run.
 The Heapster pod discovers all nodes in the cluster and queries usage
information from each node's Kubelet—the on-machine Kubernetes
agent. The Kubelet itself fetches the data from cAdvisor.
 Heapster groups the information by pod along with the relevant labels.
This data is then pushed to a configurable backend for storage and
visualization.
179

InfluxDB & Grafana
 InfluxDB is an open source database written in Go specifically to handle
time series data with high availability and high performance
requirements. It exposes an easy to use API to write and fetch time
series data. Heapster is set up to use InfluxDB as the storage backend
by default on most Kubernetes clusters.
 Grafana is the data visualization component in our monitoring setup. It
is available out of the box in a Kubernetes cluster. The default
dashboard displays resource usage of the cluster and the pods inside of
it. This dashboard can easily be customized and expanded.
 InfluxDB and Grafana already run in pods exposing themselves as
Kubernetes services, making it easy for Heapster to discover them.
With all of this already in place, there is minimal setup required to get
things up and running.
180

The base – container & spec
 The only required field in spec is containers.
it requires (mandatory only) two entries
 name
 image
 Part of the spec is for all containers in a pod:
 restartPolicy
 Volumes
 The spec is very extensible by design
 Specs don’t do anything by themselves;
for that you need a pod .
183

Pods
 Pods are just collections of containers that share a few
things:
 Access to volumes
 Networking
 Are co-located
 Restart together
 Usualy scale togther
 Etc.
 Pods can be run by themselves but have no guarantee to
restart or stay running or scale or do anything useful
184

Ingress Service
 Services point to a Pod or to an external source.
 With Pods a virtual endpoint is created then routed to using the
kube-proxy
 For non-pod services a virtual IP in the cluster is used to route
externally .
 Ingress Service = AWS API Gateway
 An Ingress Controller sits at the boundary of the cluster and routes
requests to Services
 One Ingress Controller can handle multiple domains
 Each route can point to a different Service
 Relies on the creation of an Ingress Controller in the cluster
(another service that is not enabled by default).
185

Daemon sets
 Daemons is an object that ensures that a copy of each Pod
runs on each node. This is commonly used to make sure
side-car containers are running across the cluster.
 If new nodes come up they’ll get a copy of the daemon set
and will come up.
 Daemon sets don’t have scaling rules.
186

Pet Sets – Statefull pods
 Introduced 1.3, Pet Sets allow you to create complex
microservices across the cluster.
 They have the ability to set dependency on other
containers.
 They require:
 A stable hostname, available in DNS
 An ordinal index
 Stable storage: linked to the ordinal & hostname
 It is used for statefull pods
187

Replcia Set
 A Replication Controller (deprecated) was the best way
to run Pods.
 You set a number of pods to run and the Replication Controller
made sure that the number was running across the cluster.
 Rolling updates could be performed by starting a new Replication
Controller and scaling up.
 A Replica Set differs from the Replication Controller
because it can be updated.
 If you update the Replica Set template you can fire and update and
automatically roll changes
 Rollbacks are also built in.
 Replica Set are not designed to be used directly.
For that you need Deployments.
188

replication controller ReplicaSet
apiVersion: v1
kind: ReplicationController
metadata:
name: rcweb
labels:
name: rcweb
spec:
replicas: 4
selector:
app: rcweb
phase: production
template:
metadata:
labels:
app: rcweb
role: frontend
phase: production
name: rcwebpod
spec:
containers:
- name: staticweb 189

Deployments, The king of the hill
 A Deployment controls the running state of Pods and
Replica Sets.
 In k8s it is the primary object you should be manipulating.
 Deployments have:
 History
 Rolling updates
 Pausing updates
 Roll-backs
190

A Virtual Cluster in Your Cluster
 A namespace as an isolated section of a cluster.
 It’s a virtual cluster in your cluster.
 Each cluster can have multiple namespaces.
 The root services have their own.
 Namespaces are in network isolation from each other and
can (are normally) used to house different environments on
the same cluster.
 resources (pods, services, replication controllers, etc.)
 policies (who can or cannot perform actions in their community)
 constraints (quota, etc.)
 Kubernetes starts with two initial namespaces:
 default - The default namespace for objects with no other
namespace.
 kube-system - The namespace for objects created by the
Kubernetes system
191

ResourceQuota (for namespace)
apiVersion: v1
kind: ResourceQuota
metadata:
name: quota
spec:
hard:
cpu: "20"
memory: 1Gi
pods: "10"
replicationcontrollers: "20"
resourcequotas: "1"
services: "5"
192

ConfigMap
 ConfigMap is a resource available in kubernetes for managing
application configuration.
 The goal is to decouple the app configuration from the image content
in order to keep the container portable and k8s agnostic.
 ConfigMap are key value pairs of configuration data.
193

Kubernetes on public clouds
 Availability on the major cloud providers:
 There are a variety of CLI tools available
 automate deployment of Kubernetes
 enable production-quality Kubernetes clusters on AWS & other
clouds to be deployed quickly & easily
 some include node auto-scaling
 Also third-party companies providing Kubernetes-as-a-
Service on multiple public clouds
Native support Node auto-scaling
Google ✔ ✔
Azure ✔ ✗
AWS ✗ ✗

Windows Server Container Architecture
Source: Microsoft

Windows Server Node in Kubernetes
kubelet kube-proxy
Kubernetes
Master
Components
(unchanged)
Kubectl
(unchanged)
Windows Server 2016 Node
docker
Infra
container POD
container
Infra
container POD
container

Standalone
 Minikube
 https://github.com/kubernetes/minikube
 Runs a Kubernetes node on top of your favorite (probably
Virtualbox) VM.
 Lots of involvement from the K8s community.
 Kube-solo
 https://github.com/TheNewNormal/kube-solo-osx
 Uses the Corectl app to run a Kube VM
 Also has a multi-node version.
197

Routes/Routers
 Route exposes a service at a host name, like
www.example.com, so that external clients can reach it by
name.”
 Administrator can deploy routers to nodes in an OpenShift
Enterprise cluster, which enable routes created by
developers to be used by external clients.”
 The default Router in Openshift is an actual HAProxy
container providing reverse proxy capabilities.
 The route is an Openshift construct that defines the rules
you want to apply to incoming connections.
199

Project
200

FLEXIBILITY ACROSS DEPLOYMENTTARGETS
Building A Trusted, Consistent Environment Everywhere
RED HAT ENTERPRISE LINUX IS THE FOUNDATION

Virtualization technolegys for amdocs

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Virtualization technolegys for amdocs

Similar to Virtualization technolegys for amdocs (20)

More from Samuel Dratwa

More from Samuel Dratwa (16)

Recently uploaded

Recently uploaded (20)

Virtualization technolegys for amdocs

Editor's Notes