SlideShare a Scribd company logo
INTRODUCTION TO KUBERNETES
OpenNMS DevJam 2019
by Alejandro Galue
WHY CONTAINERS
• Build, Ship, and Run any App, Anywhere.
• No more dependency problems; e.x. no RPM/DEB or
dynamic libraries incompatibility issues.
• No more: "but it works on my machine", as the same
image used in dev is deployed to prod.
• Serverless implementations are mainly backed by
containers.
WHY KUBERNETES
• Be able to manage hundreds or thousands of containers,
on a fleet of hundred or thousands of nodes.
• Be able to deploy an application without worrying about
the hardware infrastructure.
• Be able to guarantee that applications will stay running
according with the specifications.
• Be able to facilitate scaling and upgrades not only for the
workloads but for the k8s cluster itself.
WHAT IS KUBERNETES
• A distributed cluster technology that manages container-based systems
in a declarative manner using an API (a container orchestrator).
• Designed from the ground-up as a loosely coupled collection of
components centered around deploying, maintaining and scaling
workloads.
• Abstracts away the underlying hardware of the nodes and provides a
uniform interface for workloads to be both deployed and consume the
shared pool of resources.
• Works as an engine for resolving state by converging actual and the
desired state of the system (self-healing).
A COUPLE OF KEY CONCEPTS…
PODS
• Atomic unit or smallest “unit of
work”of Kubernetes.
• Pods are one or MORE
containers that share volumes, a
network namespace, and are a
part of a single context.
Host A
Pod
ExternalVolume
container container
Network Namespace 10.255.16.3
Pod Network
Ephemeral
SERVICES
• Unified method of
accessing the exposed
workloads of Pods.
• Durable resource
- static cluster IP
- static namespaced DNS
name
Host A
Labels:
app=nginx
env=prod
Labels:
app=nginx
env=prod
Labels:
app=mysql
env=dev
Host B
Labels:
app=nginx
env=prod
Labels:
app=mysql
env=prod
Labels:
app=nginx
env=dev
Service
app=nginx
env=prod
NOT
Ephemeral
ARCHITECTURE OVERVIEW
Control Plane - 1,2,…n
kube
controller manager
kube
scheduler
kube
apiserver
kubectl
Node 1
Containers
System Services
kubelet
kube-proxy
Container Runtime
Pod Pod
Node 2
Containers
System Services
kubelet
kube-proxy
Container Runtime
Pod Pod
cloud
controller manager
Cloud
Provider
API
Load
Balancer
End Users
Cloud Provider
Network Edge
Control Plane - 1,2,…n
Containers
kube-control-manager
kube-scheduler
kube
apiserver
kubectl
Node 1
Containers
System Services
kubelet
kube-proxy
Container Runtime
Pod Pod
Node 2
Containers
System Services
kubelet
kube-proxy
Container Runtime
Pod Pod
cloud-controller-manager
Cloud
Provider
API
Load
Balancer
End Users
Cloud Provider
Network Edge
System Services
kubelet
Container Runtime
additional services
kube-proxy
KUBE-APISERVER
• Provides a forward facing REST interface into the
kubernetes control plane and datastore.
• All clients and other applications interact with
kubernetes strictly through the API Server.
• Acts as the gatekeeper to the cluster by handling
authentication and authorization, request validation,
mutation, and admission control in addition to being
the front-end to the backing datastore.
master
• etcd acts as the cluster datastore.
• Purpose in relation to Kubernetes is to provide a
strong, consistent and highly available key-value store
for persisting cluster state.
• Stores objects and config information.
• Uses “Raft Consensus” among a quorum of systems to
create a fault-tolerant consistent “view” of the cluster.
master
KUBE-CONTROLLER-MANAGER
• Serves as the primary daemon that manages
all core component control loops.
• Monitors the cluster state via the apiserver
and steers the cluster towards the desired
state.
master
KUBE-SCHEDULER
• Component on the master that watches newly
created pods that have no node assigned, and
selects a node for them to run on.
• Factors taken into account for scheduling decisions
include individual and collective resource
requirements, hardware/software/policy
constraints, affinity and anti-affinity specifications,
data locality, inter-workload interference and
deadlines.
master
CLOUD-CONTROLLER-MANAGER
• Daemon that provides cloud-provider specific
knowledge and integration capability into the
core control loop of Kubernetes.
• The controllers include Node, Route, Service,
and add an additional controller to handle
things such as PersistentVolume Labels.
master
Optional
KUBE-PROXY
• Manages the network rules on each node.
• Performs connection forwarding or load
balancing for Kubernetes cluster services.
worker
KUBELET
• An agent that runs on each node in the cluster.
It makes sure that containers are running in a
pod.
• The kubelet takes a set of PodSpecs that are
provided through various mechanisms and
ensures that the containers described in those
PodSpecs are running and healthy.
worker
CONTAINER RUNTIME ENGINE
A container runtime is a CRI (Container Runtime
Interface) compatible application that executes and
manages containers.
• containerd (Docker)
• cri-o
• rkt
• kata (formerly clear and hyper)
• virtlet (VM CRI compatible runtime)
worker
KUBERNETES NETWORKING
• Pod Network
- Cluster-wide network used for pod-to-pod
communication managed by a CNI
(Container Network Interface) plugin.
• Service Network
- Cluster-wide range of Virtual IPs managed by
kube-proxy for service discovery.
FUNDAMENTAL NETWORKING RULES
• All containers within a pod can communicate
with each other unimpeded.
• All Pods can communicate with all other Pods
without NAT.
• All nodes can communicate with all Pods (and
vice-versa) without NAT.
• The IP that a Pod sees itself as is the same IP
that others see it as.
What if I want to limit
communication within
Pods ?
Learn about Network
Policies (and make sure
the chosen CNI supports
it)
API
API OVERVIEW
• The REST API is the true
keystone of Kubernetes.
• Everything within Kubernetes
is as an API Object.
• Referenced within an object as
the apiVersion and kind.
Format:
/apis/<group>/<version>/<resource>
Examples:
/apis/apps/v1/deployments
/apis/batch/v1beta1/cronjobs
OBJECT MODEL
• Objects are a “record of intent” or a
persistent entity that represent the desired
state of the object within the cluster.
• All objects MUST have  apiVersion, kind, and
poses the nested fields metadata.name,
metadata.namespace, and metadata.uid.
OBJECT EXPRESSION -YAML
• Files or other representations
of Kubernetes Objects are
generally represented in YAML.
• Three basic data types:
- mappings  - hash or
dictionary,
- sequences - array or list
- scalars - string, number,
boolean etc
apiVersion: v1
kind: Pod
metadata:
name: sample
namespace: test
spec:
containers:
- name: container1

image: nginx
- name: container2

image: alpine
YAMLVS JSON
{
  "apiVersion": "v1",
  "kind": "Pod",
  "metadata": {
    "name": "pod-example"
  },
  "spec": {
    "containers": [
      {
        "name": "nginx",
        "image": "nginx:stable-alpine",
        "ports": [
{ "containerPort": 80 }
]
      }
    ]
  }
}
apiVersion: v1
kind: Pod
metadata:
  name: pod-example
spec:
  containers:
  - name: nginx
    image: nginx:stable-alpine
    ports:
    - containerPort: 80
Are you wondering about theYAML schema ?
kubectl explain is your friend ;)
What about kinds or versions ?
kubectl api-versions is your friend ;)
CORE CONCEPTS
NAMESPACES
Namespaces are a logical cluster or
environment, and are the primary method of
partitioning a cluster or scoping access.
apiVersion: v1
kind: Namespace
metadata:
  name: prod
  labels:
    app: MyBigWebApp
$ kubectl get ns --show-labels
NAME          STATUS    AGE       LABELS
default       Active    11h       <none>
kube-public   Active    11h       <none>
kube-system   Active    11h       <none>
prod          Active    6s        app=MyBigWebApp
Known as
Projects in
OpenShift
PODS
• Atomic unit or smallest
“unit of work”of
Kubernetes.
• Foundational building
block of Kubernetes
Workloads.
• Pods are one or MORE
containers that share
volumes, a network
namespace, and are a
part of a single context.
Host A
Pod
ExternalVolume
container container
Network Namespace 10.255.16.3
Pod Network
apiVersion: v1
kind: Pod
metadata:
name: sample
spec:
containers:
- name: nginx
image: nginx:stable-alpine
ports:
  - containerPort: 80
    name: http
    protocol: TCP
env:
- name: MYVAR
value: isAwesome
command: [“/bin/sh”, “-c”]
args: [“echo ${MYVAR}”]
readinessProbe:
tcpSocket:
port: http
initialDelaySeconds: 10
periodSeconds: 10
livenessProbe:
httpGet:
path: /
port: http
initialDelaySeconds: 30
periodSeconds: 60
Array of
ports to expose
Name of the
container
Container
ImageArray of
environment
variables
Entrypoint
Array
~ Docker
ENTRYPOINT
Arguments
to pass to the
command
~ Docker CMD
Tells
when the
application is ready
to receive
requests
Checks if the
container is still
alive (running)
LABELS
• key-value pairs that
are used to identify,
describe and group
together related sets of
objects or resources.
• NOT characteristic of
uniqueness.
• Have a strict syntax
with a slightly limited
character set*.
https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set
Host A
Pod
container
Network Namespace 10.255.16.3
Pod Network
Labels:
gpu=nvidia
Labels:
app=nginx
env=prod
apiVersion: v1
kind: Pod
metadata:
  name: pod-label-example
  labels:

    app: nginx

    env: prod
spec:
  containers:
  - name: nginx
    image: nginx:stable-alpine
    ports:
    - containerPort: 80
SELECTORS
Selectors use
labels to filter or
select objects, and
are used
throughout
Kubernetes.
Host A
Pod
mlapp
Network Namespace 10.255.16.3
Pod Network
Labels:
gpu=nvidia
Labels:
app=mlapp
env=prodapiVersion: v1
kind: Pod
metadata:
  name: mlapp
  labels:

    app: mlapp

    env: prod
spec:
  nodeSelector:
  - gpu: nvidia
  containers:
  - name: nginx
    image: tensorflow/tensorflow
Host B
Labels:
gpu=amd
SERVICES
• Unified method of accessing the
exposed workloads of Pods.
• Durable resource (unlike Pods)
- static cluster-unique IP
- static namespaced DNS name
<service name>.<namespace>.svc.cluster.local
There are 4 major service types:
• ClusterIP (default)
• NodePort
• LoadBalancer
• ExternalName
CLUSTER IP SERVICE
Labels:
app=nginx
env=prod
kube-proxy
Host A
Labels:
app=nginx
env=prod
Host B
Pod
Host C
Pod
Network
kube-proxy
kube-proxy
Name:         example-prod
Selector:     app=nginx,env=prod
Type:         ClusterIP
IP:           10.96.28.176
Port:         <unset>  80/TCP
TargetPort:   80/TCP
Endpoints:    10.255.16.3:80,

              10.255.16.4:80
• The Pod on host C requests the service.
• Hits host iptables and it load-balances
the connection between the endpoints
residing on Hosts A, B
/ # nslookup example-prod.default.svc.cluster.local
Name:      example-prod.default.svc.cluster.local
Address 1: 10.96.28.176 example-prod.default.svc.cluster.local
NODE PORT SERVICE
Labels:
app=nginx
env=prod
kube-proxy
Host A
Labels:
app=nginx
env=prod
Host B
Labels:
app=nginx
env=dev
Host C
Name:         example-prod
Selector:     app=nginx,env=prod
Type:         NodePort
IP:           10.96.28.176
Port:         <unset>  80/TCP
TargetPort:   80/TCP
NodePort:     <unset>  32410/TCP
Endpoints:    10.255.16.3:80,

              10.255.16.4:80
• User can hit any host in cluster on
NodePort IP and get to service.
• Does introduce extra hop if hitting a
host without instance of the pod.
32410
32410kube-proxy
32410kube-proxy
LOAD BALANCER SERVICE
• LoadBalancer services extend NodePort.
• Works in conjunction with an external
system to map a cluster external IP to the
exposed service.
Labels:
app=nginx
env=prod
kube-proxy
Host A
Labels:
app=nginx
env=prod
Host B
Labels:
app=nginx
env=dev
Host C
32410
32410kube-proxy
32410kube-proxy
Load
Balancer
Cloud Provider
Edge Network
Name:         example-prod
Selector:     app=nginx,env=prod
Type:         LoadBalancer
IP:           10.96.28.176
LoadBalancer

Ingress:      172.17.18.43
Port:         <unset>  80/TCP
TargetPort:   80/TCP
NodePort:     <unset>  32410/TCP
Endpoints:    10.255.16.3:80,

              10.255.16.4:80
INGRESSKnown as
Routes in
OpenShift
• An API object that manages external access
to the services in a cluster, through a single
LoadBalancer.
• Provides load balancing, SSL termination and
name/path-based virtual hosting
• Gives services externally-reachable URLs.
• Multiple implementations to choose from.
service:
grafana
port=3000
Host A
service:
kibana
port=5601
Host B
service:
opennms
port=8980
Host C
grafana.example.com
kibana.example.com
Cloud Provider
Edge Network
Ingress
Controller
Host D
Load
Balancer
onms.example.com
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-rules
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: onms.example.com
http:
paths:
- path: /
backend:
serviceName: opennms
servicePort: 8980
…
EXTERNAL NAME SERVICE
• ExternalName is used to
reference endpoints
OUTSIDE the cluster.
• Creates an internal CNAME
DNS entry that aliases
another.
apiVersion: v1
kind: Service
metadata:
  name: example-prod
spec:
  type: ExternalName
spec:
  externalName: example.com
WORKLOADS
WORKLOADS
• Workloads within Kubernetes
are higher level objects that
manage Pods or other higher
level objects.
• In ALL CASES a Pod Template
is included, and acts the base
tier of management.
•ReplicaSet
•Deployment
•DaemonSet
•StatefulSet
•Job
•CronJob
PODTEMPLATE
• Workload Controllers
manage instances of Pods
based off a provided
template.
• Pod Templates are Pod
specs with limited metadata.
• Controllers use Pod
Templates to make actual
pods
apiVersion: v1
kind: Pod
metadata:
  name: pod-example
  labels:

    app: nginx
spec:
  containers:
  - name: nginx
    image: nginx
template:
  metadata:
    labels:

      app: nginx
  spec:
    containers:
    - name: nginx
      image: nginx
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-example
  labels:

    app: nginx
spec:
replicas: 3
  selector:
    matchLabels:

      app: nginx
template:
  metadata:
    labels:

      app: nginx
  spec:
    containers:
    - name: nginx
      image: nginx
RESOURCE MODEL
• Request: amount of a resource
allowed to be used, with a strong
guarantee of availability.
- CPU (seconds/second), RAM
(bytes)
- Scheduler will not over-commit
requests
• Limit: max amount of a resource that
can be used, regardless of guarantees
- Scheduler ignores limits
apiVersion: v1
kind: Pod
metadata:
  name: pod-example
  labels:

    app: nginx
spec:
  containers:
  - name: nginx
    image: nginx
  resources:
  requests:
  memory: "64Mi"
    cpu: "250m"
  limits:
  memory: "128Mi"
    cpu: "500m"
INIT CONTAINERS
If we compare a container template with a Class
definition in Java, an initContainer would be the
constructor of a class; while a running Pod, would be
an instance of that class.
Can be used for everything that should happen
before the containers within a Pod start running. For
example: wait for dependencies, initialize volumes or
databases, verify requirements, etc.
REPLICA SET
• Primary method of managing
pod replicas and their lifecycle.
• Includes their scheduling,
scaling, and deletion.
• Their job is simple: Always
ensure the desired number
of pods are running.
ReplicaSet Pod
• replicas: The desired number of instances of
the Pod.
• selector: The label selector for the
ReplicaSet will manage ALL Pod instances
that it targets; whether it’s desired or not.
apiVersion: apps/v1
kind: ReplicaSet
metadata:
  name: rs-example
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx

      env: prod
  template:
    <pod template>
DEPLOYMENT
• Declarative method of managing Pods via
ReplicaSets.
• Provide rollback functionality and update
control.
• Updates are managed through the pod-
template-hash label.
• Each iteration creates a unique label that is
assigned to both the ReplicaSet and
subsequent Pods.
PodReplicaSetDeployment
• revisionHistoryLimit: The number of previous iterations of the
Deployment to retain.
• strategy: Describes the method of  updating the Pods based on
the type.Valid options are:
- Recreate: All existing Pods are killed before the new ones
are created.
- RollingUpdate: Cycles through updating the Pods according
to the parameters: maxSurge and maxUnavailable.
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-example
spec:
  replicas: 3

  revisionHistoryLimit: 3
  selector:
    matchLabels:
      app: nginx

      env: prod
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  template:
    <pod template>
DAEMON SET
• Ensure that all nodes matching certain
criteria will run an instance of the
supplied Pod.
• They bypass default scheduling
mechanisms.
• Are ideal for cluster wide services such
as log forwarding, or health monitoring.
• Revisions are managed via a
controller-revision-hash label.
DaemonSet Pod
• spec.template.spec.nodeSelector: The
primary selector used to target nodes.
• Default Host Labels:
- kubernetes.io/hostname
- beta.kubernetes.io/os
- beta.kubernetes.io/arch
• Cloud Host Labels:
- failure-domain.beta.kubernetes.io/zone
- failure-domain.beta.kubernetes.io/region
- beta.kubernetes.io/instance-type
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: ds-example
spec:
  revisionHistoryLimit: 3
  selector:
    matchLabels:
      app: nginx
  updateStrategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1
  template:
    spec:
      nodeSelector:
        nodeType: edge
     <pod template>
STATEFUL SET
• Tailored to managing Pods that must persist or maintain state.
• Pod identity including hostname, network, and storage
WILL be persisted.
• Assigned a unique ordinal name following the convention of
‘<statefulset name>-<ordinal index>’.
• Naming convention is also used in Pod’s network Identity and
Volumes.
• Pod lifecycle will be ordered and follow consistent patterns.
• Revisions are managed via a controller-revision-hash label.
Pod
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: sts-cassandra
spec:
  replicas: 3
  selector:
    matchLabels:
      app: cassandra
  serviceName: cassandra
  updateStrategy:
   type: RollingUpdate
   rollingUpdate:
     partition: 0
  template:
    metadata:
      labels:
        app: cassandra
    spec:
      containers:
      - name: cassandra-node
        image: cassandra:3.11.4
        env:
        - name: CASSANDRA_SEEDS
        value: sts-cassandra-0.cassandra
        - name: CASSANDRA_CLUSTER_NAME
        value: my-cluster
        ports:
        - containerPort: 7000
        - containerPort: 7199
        - containerPort: 9042
        volumeMounts:
        - name: data
          mountPath: /cassandra_data
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: standard
      resources:
        requests:
          storage: 100Gi
The name of the
associated headless service;
or a service without a
ClusterIP
Pods with an
ordinal greater than the
partition value will be
updated one-by-one in
reverse order
StatefulSet
Template of the
persistent volume(s)
request to use for each
instance of the
StatefulSet.
HEADLESS SERVICE
<StatefulSet Name>-<ordinal>.<service name>.<namespace>.svc.cluster.local
apiVersion: v1
kind: Service
metadata:
  name: cassandra
spec:
  clusterIP: None
  selector:
    app: cassandra
  ports:
  - name: intra-node
    port: 80
  - name: jmx
    port: 7199
  - name: cql
    port: 9042
/ # dig cassandra.default.svc.cluster.local +noall +answer
; <<>> DiG 9.9.4-RedHat-9.9.4-74.el7_6.1 <<>> cassandra.default.svc.cluster.local +noall +answer
;; global options: +cmd
cassandra.default.svc.cluster.local. 5 IN A 172.17.0.5
cassandra.default.svc.cluster.local. 5 IN A 172.17.0.4
cassandra.default.svc.cluster.local. 5 IN A 172.17.0.6
/ # dig sts-cassandra-0.cassandra.default.svc.cluster.local +noall +answer
; <<>> DiG 9.9.4-RedHat-9.9.4-74.el7_6.1 <<>> sts-cassandra-0.cassandra.default.svc.cluster.local +noall +answer
/ # dig sts-cassandra-1.cassandra.default.svc.cluster.local +noall +answer
; <<>> DiG 9.9.4-RedHat-9.9.4-74.el7_6.1 <<>> sts-cassandra-1.cassandra.default.svc.cluster.local +noall +answer
/ # dig sts-cassandra-2.cassandra.default.svc.cluster.local +noall +answer
; <<>> DiG 9.9.4-RedHat-9.9.4-74.el7_6.1 <<>> sts-cassandra-2.cassandra.default.svc.cluster.local +noall +answer
$ kubectl get pods
NAME            READY     STATUS    RESTARTS   AGE
sts-cassandra-0   1/1       Running   0          11m
sts-cassandra-1   1/1       Running   0          11m
sts-cassandra-2   1/1       Running   0          11m
VOLUME CLAIMTEMPLATE
<Volume Name>.<StatefulSet Name>-<ordinal>
 volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: standard
      resources:
        requests:
          storage: 1Gi
Persistent Volumes associated with a
StatefulSet will NOT be automatically
garbage collected when its associated
StatefulSet is deleted. They must
manually be removed.
$ kubectl get pvc
NAME                STATUS    VOLUME                                    CAPACITY   ACCESS MODES   STORAGECLASS   AGE
data-cassandra-sts-0 Bound pvc-8baec2a5-8c77-11e9-a6be-0800275ddaf0 100Gi RWO standard 17m
data-cassandra-sts-1 Bound pvc-95463714-8c77-11e9-a6be-0800275ddaf0 100Gi RWO standard 17m
data-cassandra-sts-2 Bound pvc-9807c42a-8c77-11e9-a6be-0800275ddaf0 100Gi RWO standard 17m
JOB
• Job controller ensures one or more
pods are executed and successfully
terminate.
• Will continue to try and execute the
job until it satisfies the completion
and/or parallelism condition.
• Pods are NOT cleaned up until the
job itself is deleted.
PodJob
apiVersion: batch/v1
kind: Job
metadata:
  name: job-example
spec:
  backoffLimit: 4
  completions: 4
  parallelism: 2
  template:
    spec:
      restartPolicy: Never
    template:
    <pod-template>
• backoffLimit: The number of failures before
the job itself is considered failed.
• completions: The total number of successful
completions desired.
• parallelism: How many instances of the pod
can be run concurrently.
• spec.template.spec.restartPolicy:Jobs only
support a restartPolicy of type Never or
OnFailure.
CRONJOB
• An extension of the Job Controller,
it provides a method of executing
jobs on a cron-like schedule.
• CronJobs within Kubernetes

 use UTC ONLY.
PodJobCronJob
apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: cronjob-example
spec:
  schedule: "*/1 * * * *"
  successfulJobsHistoryLimit: 3
  failedJobsHistoryLimit: 1
  jobTemplate:
    spec:
      completions: 4
      parallelism: 2
      template:
        <pod template>
The cron
schedule for
the job
The number of
successful jobs to
retain
The number of
failed jobs to
retain
OPERATORS
• Might be the most powerful feature of
K8s.
• When none of the workloads fits the
needs of your app, you can create your
own controllers with your own
specifications (CRD).
• Should be implemented in Go, but there
are alternatives.
• A given controller usually requires a
ServiceAccount with proper RBAC Role
to be able to manage workloads (e.x.
create and maintain Pods, Services, etc.)
apiVersion: acid.zalan.do/v1
kind: postgresql
metadata:
name: opennms-database
spec:
teamId: OpenNMS
volume:
size: 100Gi
numberOfInstances: 3
enableMasterLoadBalancer: false
enableReplicaLoadBalancer: false
users:
opennms:
- superuser
- createdb
databases:
opennms: opennms
postgresql:
version: "10"
STORAGE
VOLUMES
• Storage that is tied to the Pod’s Lifecycle.
• A pod can have one or more types of volumes
attached to it.
• Can be consumed by any of the containers
within the pod.
• Survive Pod restarts; however their durability
beyond that is dependent on the Volume Type.
PERSISTENTVOLUMES
• A PersistentVolume (PV) represents a storage resource. 
• PVs are a cluster wide resource linked to a backing
storage provider: NFS, EBS, GCEPersistentDisk, etc.
• Generally provisioned by an administrator.
• Their lifecycle is handled independently from a pod
• CANNOT be attached to a Pod directly. Relies on a 
PersistentVolumeClaim (PVC).
• A PVC is a namespaced request for storage.
STORAGE CLASS
• Satisfies a set of requirements instead of mapping
to a storage resource directly.
• Ensures that an application’s claim for storage is
portable across numerous backends or
providers.
CONFIGURATION
CONFIG MAPS
• Externalized data stored within
kubernetes.
• Can be referenced through several
different means:
- environment variable
- a command line argument (via
environment variable)
- injected as a file into a volume mount
• Can be created from a manifest, literals,
directories, or files directly.
apiVersion: v1
kind: ConfigMap
metadata:
  name: manifest-example
data:
  state: Minnesota
  city: Minneapolis
  content: |
    Look at this,
    its multiline!
SECRETS
• Functionally identical to a ConfigMap.
• Stored as base64 encoded content.
• Encrypted at rest within etcd (if
configured!).
• Ideal for username/passwords,
certificates or other sensitive information
that should not be stored in a container.
• Can be created from a manifest, literals,
directories, or from files directly.
apiVersion: v1
kind: Secret
metadata:
  name: manifest-secret
type: Opaque
data:
  username: ZXhhbXBsZQ==
  password: bXlwYXNzd29yZA==
• type: There are three different types of
secrets within Kubernetes:
- docker-registry - credentials used to
authenticate to a container registry
- generic/Opaque - literal values from
different sources
- tls - a certificate based secret
• data: Contains key-value pairs of base64
encoded content.
INSTALLATION
BARE-METAL, CLOUD, OR

LOCAL INSTALLATION
Install
OpenShift
is hard ;)
Local PC Installation

For development and testing
• minikube
• minishift (for OKD)
• oc cluster up (for OKD)
• Docker for Mac or Windows
• microk8s (for Linux via Snap)
Cloud Installation

Single Command Experience
• gcloud container clusters create
• kops create cluster
• eksctl create cluster
• az aks create
• az openshift create
OpenShift 4 promises this for the first
time since its conception.
Bare-Metal / On-Premise
• kubeadm (single command
to spin up a master or a
worker)
• OpenShift/OKD Ansible
Tools
On-Premise vs Cloud Managed
• Masters are managed for you, and the only additional resources are the instances for worker nodes
• Complete integration with cloud services (volumes, network, load balancers, etc.)
SECURITY
Here is where
OpenShift is
Excellent!!!
https://kubernetes.io/blog/2018/07/18/11-ways-not-to-get-hacked/
THINGS TO KEEP IN MIND
• Users (to access cluster through the API). Usually implemented through certificates.
• Service Accounts (for controllers and operators, to make changes from within K8s).
• RBAC: Roles and RoleBinding, ClusterRole and ClusterRoleBinding (for users, groups
and service accounts).
• Never run containers as root, and block privilege escalations (OpenShift enforce this by
default using SecurityContextConstraint; known I k8s as PodSecurityPolicy).
• Limit resources per namespace (total CPUs and memory, number of Pods, …).
• Create your own images and keep a private registry (OpenShift offers this by default).
• Research about NetworkPolicy (which requires a CNI that supports this feature).
• Research about applications like vault (to enhance protection for your secrets).
• Research about clair (a tool for vulnerability scans on containers).
• Research about kube-bench (a tool that analyzes your cluster for security best practices).
• Research about gvisor.dev (a container sandbox focused on security and efficiency).
Some inspired
by OpenShift
ACCESS
On user the certificate (signed by the CA
used on the API server), the Common
Name (CN) will be interpreted as the
username, and the Organization (O) as
the group where this user belongs to.
Use the kubectl command to
create the kubeconfig based
on the user’s certificate, to
access the cluster resources.
Define an RBAC Role to
connect resources
(deployments, pods,
ingress, …) with operations
(create, delete, list, …) on a
given namespace.
Define an RBAC
RoleBinding to connect a
role with subjects (i.e.
group, user, service-
account)
Use a
ClusterRole for
cluster wide access
For a
ClusterRole use
ClusterRoleBinding
Now you
have
access!
cluster-admin
O=system:masters
MONITORING
HOWTO MONITOR K8S?
• Based on Google’s Four Golden Signals: latency, errors, traffic, saturation, use
Brendan Gregg’s USE method (Utilization, Saturation, Errors) for Resources,
and Tom Wilkie’s RED method (Rate, Errors, Duration) for Services.
• Metric Server collects metrics such as CPU and Memory by each pod and
node from the Summary API, exposed by Kubelet (via cAdvisor) on each
node.This allows the usage of K8s features like the HorizontalPodAutoscaler.
• The usage of Prometheus is widely adopted.This solution is based on their
own node_exporter (deployed as DaemonSet) which exposes lots of Linux
metrics in Prometheus format.
MONITOR K8S WITH OPENNMS
• OpenNMS has been used to monitor network infrastructure (or
network devices).
• OpenNMS can monitor servers at some degree, and can do some
level of application monitoring; but traditionally other applications
are used in this context.
• For Kubernetes, Prometheus is wildly adopted, on a level that it is
considered the standard.
• OpenNMS can be used to expand the functionality of the already
existing observability/monitoring tools in Kubernetes world.
MONITOR K8S WITH OPENNMS
• Have an operator forwarding important K8s events to OpenNMS, so you
can use all the event processing infrastructure to react upon changes on
your K8s cluster.
• There is no such thing as fixed IP addresses in Kubernetes, so forget
about Pods. But, if providing support FQDN support in OpenNMS is
possible, that can make monitoring K8s services a reality in the future.
• Build a solution based on the Metrics API / cAdvisor, or Prometheus
(directly or indirectly through the node_exporter), to get data from the
running Nodes, Pods and Services.
• SNMP could run on Kubernetes nodes; but that’s not an option for Pods.
HOWTO DEPLOY OPENNMS IN K8S?
• Use a StatefulSet for OpenNMS (1 replica), to take advantage of auto-deployed PVCs, as
the dynamic nature of the configuration directory makes OpenNMS a stateful application.
Keep in mind "kubectl cp" for transferring files.
• Use a StatefulSet for Minions, to take advantage of the persistent network identity, and use
Pod name as the minion-ID.This is recommended, even if Minions are stateless by nature.
• Use a Deployment for Sentinels, as they are stateless in nature and having a unique
identity is not relevant (for now). Although a StatefulSet won’t hurt.
• A combination of initContainers, config-maps, and secrets can be used to pre-configure
the mandatory settings on all the applications.The PersistentVolume will keep the custom
settings added by the user during the lifetime of the product.
OPENNMS DEPENDENCIES
It depends… a given team should decide if it
it worth running PostgreSQL, Kafka, Cassandra
and Elasticsearch within K8s.
Keep in mind that all databases are stateful in
nature, and managing the life-cycle is
different on each case. Solutions based on
operators when available are preferred.
INTEGRATE OPENNMS WITH K8S
• OpenNMS can forward everything to Kafka, and
even consume events from Kafka. From there,
Serverless technologies or Kafka Streams apps
deployed into K8s, can be used to create your own
horizontally scalable integrations triggered by
events, alarms or collected metrics.
• What about using OpenNMS as the source of metrics
for Horizontal Pod Autoscaling or Cluster Workers
auto-scaling ?
DEMO
OVERWHELMED?
Do not forget we haven’t talked about …
• Several k8s features not discussed here ;)
• Helm (the k8s package manager; the yum or apt-get for k8s)
• Service Meshes (Envoy, Istio, Linkerd, …)
• Serverless (Kubeless, Fission, Knative, …)
• Machine Learning (kubeflow, …)
• VMs and Kubernetes (KubeVirt)
• Operators In Depth:

https://operatorhub.io

https://github.com/operator-framework
And more…

Not a big enough lis ? Check https://landscape.cncf.io
QUESTIONS?
THANKYOU!
This work has been inspired by a few presentations from:
https://github.com/cncf/presentations/tree/master/kubernetes
Demo:
https://github.com/OpenNMS/opennms-drift-kubernetes

More Related Content

What's hot

Kubernetes Architecture | Understanding Kubernetes Components | Kubernetes Tu...
Kubernetes Architecture | Understanding Kubernetes Components | Kubernetes Tu...Kubernetes Architecture | Understanding Kubernetes Components | Kubernetes Tu...
Kubernetes Architecture | Understanding Kubernetes Components | Kubernetes Tu...
Edureka!
 
Kubernetes Introduction
Kubernetes IntroductionKubernetes Introduction
Kubernetes Introduction
Martin Danielsson
 
Kubernetes 101 for Beginners
Kubernetes 101 for BeginnersKubernetes 101 for Beginners
Kubernetes 101 for Beginners
Oktay Esgul
 
Kubernetes - A Comprehensive Overview
Kubernetes - A Comprehensive OverviewKubernetes - A Comprehensive Overview
Kubernetes - A Comprehensive Overview
Bob Killen
 
Getting Started with Kubernetes
Getting Started with Kubernetes Getting Started with Kubernetes
Getting Started with Kubernetes
VMware Tanzu
 
Introduction to Kubernetes Workshop
Introduction to Kubernetes WorkshopIntroduction to Kubernetes Workshop
Introduction to Kubernetes Workshop
Bob Killen
 
Kubernetes Architecture
 Kubernetes Architecture Kubernetes Architecture
Kubernetes Architecture
Knoldus Inc.
 
Introduction to kubernetes
Introduction to kubernetesIntroduction to kubernetes
Introduction to kubernetes
Michal Cwienczek
 
Introduction to Kubernetes and Google Container Engine (GKE)
Introduction to Kubernetes and Google Container Engine (GKE)Introduction to Kubernetes and Google Container Engine (GKE)
Introduction to Kubernetes and Google Container Engine (GKE)
Opsta
 
An overview of the Kubernetes architecture
An overview of the Kubernetes architectureAn overview of the Kubernetes architecture
An overview of the Kubernetes architecture
Igor Sfiligoi
 
Kubernetes Concepts And Architecture Powerpoint Presentation Slides
Kubernetes Concepts And Architecture Powerpoint Presentation SlidesKubernetes Concepts And Architecture Powerpoint Presentation Slides
Kubernetes Concepts And Architecture Powerpoint Presentation Slides
SlideTeam
 
Evolution of containers to kubernetes
Evolution of containers to kubernetesEvolution of containers to kubernetes
Evolution of containers to kubernetes
Krishna-Kumar
 
Kubernetes for Beginners: An Introductory Guide
Kubernetes for Beginners: An Introductory GuideKubernetes for Beginners: An Introductory Guide
Kubernetes for Beginners: An Introductory Guide
Bytemark
 
Kubernetes: A Short Introduction (2019)
Kubernetes: A Short Introduction (2019)Kubernetes: A Short Introduction (2019)
Kubernetes: A Short Introduction (2019)
Megan O'Keefe
 
Introduction to kubernetes
Introduction to kubernetesIntroduction to kubernetes
Introduction to kubernetes
Gabriel Carro
 
Introduction to kubernetes
Introduction to kubernetesIntroduction to kubernetes
Introduction to kubernetes
Rishabh Indoria
 
Kubernetes security
Kubernetes securityKubernetes security
Kubernetes security
Thomas Fricke
 
Kubernetes - Security Journey
Kubernetes - Security JourneyKubernetes - Security Journey
Kubernetes - Security Journey
Jerry Jalava
 
Kubernetes Introduction
Kubernetes IntroductionKubernetes Introduction
Kubernetes Introduction
Peng Xiao
 
Introduction to Kubernetes
Introduction to KubernetesIntroduction to Kubernetes
Introduction to Kubernetes
Paris Apostolopoulos
 

What's hot (20)

Kubernetes Architecture | Understanding Kubernetes Components | Kubernetes Tu...
Kubernetes Architecture | Understanding Kubernetes Components | Kubernetes Tu...Kubernetes Architecture | Understanding Kubernetes Components | Kubernetes Tu...
Kubernetes Architecture | Understanding Kubernetes Components | Kubernetes Tu...
 
Kubernetes Introduction
Kubernetes IntroductionKubernetes Introduction
Kubernetes Introduction
 
Kubernetes 101 for Beginners
Kubernetes 101 for BeginnersKubernetes 101 for Beginners
Kubernetes 101 for Beginners
 
Kubernetes - A Comprehensive Overview
Kubernetes - A Comprehensive OverviewKubernetes - A Comprehensive Overview
Kubernetes - A Comprehensive Overview
 
Getting Started with Kubernetes
Getting Started with Kubernetes Getting Started with Kubernetes
Getting Started with Kubernetes
 
Introduction to Kubernetes Workshop
Introduction to Kubernetes WorkshopIntroduction to Kubernetes Workshop
Introduction to Kubernetes Workshop
 
Kubernetes Architecture
 Kubernetes Architecture Kubernetes Architecture
Kubernetes Architecture
 
Introduction to kubernetes
Introduction to kubernetesIntroduction to kubernetes
Introduction to kubernetes
 
Introduction to Kubernetes and Google Container Engine (GKE)
Introduction to Kubernetes and Google Container Engine (GKE)Introduction to Kubernetes and Google Container Engine (GKE)
Introduction to Kubernetes and Google Container Engine (GKE)
 
An overview of the Kubernetes architecture
An overview of the Kubernetes architectureAn overview of the Kubernetes architecture
An overview of the Kubernetes architecture
 
Kubernetes Concepts And Architecture Powerpoint Presentation Slides
Kubernetes Concepts And Architecture Powerpoint Presentation SlidesKubernetes Concepts And Architecture Powerpoint Presentation Slides
Kubernetes Concepts And Architecture Powerpoint Presentation Slides
 
Evolution of containers to kubernetes
Evolution of containers to kubernetesEvolution of containers to kubernetes
Evolution of containers to kubernetes
 
Kubernetes for Beginners: An Introductory Guide
Kubernetes for Beginners: An Introductory GuideKubernetes for Beginners: An Introductory Guide
Kubernetes for Beginners: An Introductory Guide
 
Kubernetes: A Short Introduction (2019)
Kubernetes: A Short Introduction (2019)Kubernetes: A Short Introduction (2019)
Kubernetes: A Short Introduction (2019)
 
Introduction to kubernetes
Introduction to kubernetesIntroduction to kubernetes
Introduction to kubernetes
 
Introduction to kubernetes
Introduction to kubernetesIntroduction to kubernetes
Introduction to kubernetes
 
Kubernetes security
Kubernetes securityKubernetes security
Kubernetes security
 
Kubernetes - Security Journey
Kubernetes - Security JourneyKubernetes - Security Journey
Kubernetes - Security Journey
 
Kubernetes Introduction
Kubernetes IntroductionKubernetes Introduction
Kubernetes Introduction
 
Introduction to Kubernetes
Introduction to KubernetesIntroduction to Kubernetes
Introduction to Kubernetes
 

Similar to DevJam 2019 - Introduction to Kubernetes

Kubernetes presentation
Kubernetes presentationKubernetes presentation
Kubernetes presentation
GauranG Bajpai
 
Container Orchestration using kubernetes
Container Orchestration using kubernetesContainer Orchestration using kubernetes
Container Orchestration using kubernetes
Puneet Kumar Bhatia (MBA, ITIL V3 Certified)
 
01 - VMUGIT - Lecce 2018 - Fabio Rapposelli, VMware
01 - VMUGIT - Lecce 2018 - Fabio Rapposelli, VMware01 - VMUGIT - Lecce 2018 - Fabio Rapposelli, VMware
01 - VMUGIT - Lecce 2018 - Fabio Rapposelli, VMware
VMUG IT
 
DevNetCreate - ACI and Kubernetes Integration
DevNetCreate - ACI and Kubernetes IntegrationDevNetCreate - ACI and Kubernetes Integration
DevNetCreate - ACI and Kubernetes Integration
Hank Preston
 
Building Cloud-Native Applications with Kubernetes, Helm and Kubeless
Building Cloud-Native Applications with Kubernetes, Helm and KubelessBuilding Cloud-Native Applications with Kubernetes, Helm and Kubeless
Building Cloud-Native Applications with Kubernetes, Helm and Kubeless
Bitnami
 
01. Kubernetes-PPT.pptx
01. Kubernetes-PPT.pptx01. Kubernetes-PPT.pptx
01. Kubernetes-PPT.pptx
TamalBanerjee16
 
Kubernetes: від знайомства до використання у CI/CD
Kubernetes: від знайомства до використання у CI/CDKubernetes: від знайомства до використання у CI/CD
Kubernetes: від знайомства до використання у CI/CD
Stfalcon Meetups
 
Docker kubernetes fundamental(pod_service)_190307
Docker kubernetes fundamental(pod_service)_190307Docker kubernetes fundamental(pod_service)_190307
Docker kubernetes fundamental(pod_service)_190307
Inhye Park
 
Kubernetes-Presentation-Syed-Murtaza-Hassan
Kubernetes-Presentation-Syed-Murtaza-HassanKubernetes-Presentation-Syed-Murtaza-Hassan
Kubernetes-Presentation-Syed-Murtaza-Hassan
Syed Murtaza Hassan
 
Kubernetes Internals
Kubernetes InternalsKubernetes Internals
Kubernetes Internals
Shimi Bandiel
 
Containers and Kubernetes -Notes Leo
Containers and Kubernetes -Notes LeoContainers and Kubernetes -Notes Leo
Containers and Kubernetes -Notes Leo
Léopold Gault
 
(Draft) Kubernetes - A Comprehensive Overview
(Draft) Kubernetes - A Comprehensive Overview(Draft) Kubernetes - A Comprehensive Overview
(Draft) Kubernetes - A Comprehensive Overview
Bob Killen
 
Kubernetes fundamentals
Kubernetes fundamentalsKubernetes fundamentals
Kubernetes fundamentals
Victor Morales
 
An Introduction to Kubernetes and Continuous Delivery Fundamentals
An Introduction to Kubernetes and Continuous Delivery FundamentalsAn Introduction to Kubernetes and Continuous Delivery Fundamentals
An Introduction to Kubernetes and Continuous Delivery Fundamentals
All Things Open
 
Kubernetes for Java developers
Kubernetes for Java developersKubernetes for Java developers
Kubernetes for Java developers
Robert Barr
 
Docker and kubernetes
Docker and kubernetesDocker and kubernetes
Docker and kubernetes
Meiyappan Kannappa
 
Kubernetes 101
Kubernetes 101Kubernetes 101
Kubernetes 101
Huy Vo
 
Kubernetes a comprehensive overview
Kubernetes   a comprehensive overviewKubernetes   a comprehensive overview
Kubernetes a comprehensive overview
Gabriel Carro
 
Kubernetes acomprehensiveoverview
Kubernetes acomprehensiveoverviewKubernetes acomprehensiveoverview
Kubernetes acomprehensiveoverview
Ankit Shukla
 
Getting started with google kubernetes engine
Getting started with google kubernetes engineGetting started with google kubernetes engine
Getting started with google kubernetes engine
Shreya Pohekar
 

Similar to DevJam 2019 - Introduction to Kubernetes (20)

Kubernetes presentation
Kubernetes presentationKubernetes presentation
Kubernetes presentation
 
Container Orchestration using kubernetes
Container Orchestration using kubernetesContainer Orchestration using kubernetes
Container Orchestration using kubernetes
 
01 - VMUGIT - Lecce 2018 - Fabio Rapposelli, VMware
01 - VMUGIT - Lecce 2018 - Fabio Rapposelli, VMware01 - VMUGIT - Lecce 2018 - Fabio Rapposelli, VMware
01 - VMUGIT - Lecce 2018 - Fabio Rapposelli, VMware
 
DevNetCreate - ACI and Kubernetes Integration
DevNetCreate - ACI and Kubernetes IntegrationDevNetCreate - ACI and Kubernetes Integration
DevNetCreate - ACI and Kubernetes Integration
 
Building Cloud-Native Applications with Kubernetes, Helm and Kubeless
Building Cloud-Native Applications with Kubernetes, Helm and KubelessBuilding Cloud-Native Applications with Kubernetes, Helm and Kubeless
Building Cloud-Native Applications with Kubernetes, Helm and Kubeless
 
01. Kubernetes-PPT.pptx
01. Kubernetes-PPT.pptx01. Kubernetes-PPT.pptx
01. Kubernetes-PPT.pptx
 
Kubernetes: від знайомства до використання у CI/CD
Kubernetes: від знайомства до використання у CI/CDKubernetes: від знайомства до використання у CI/CD
Kubernetes: від знайомства до використання у CI/CD
 
Docker kubernetes fundamental(pod_service)_190307
Docker kubernetes fundamental(pod_service)_190307Docker kubernetes fundamental(pod_service)_190307
Docker kubernetes fundamental(pod_service)_190307
 
Kubernetes-Presentation-Syed-Murtaza-Hassan
Kubernetes-Presentation-Syed-Murtaza-HassanKubernetes-Presentation-Syed-Murtaza-Hassan
Kubernetes-Presentation-Syed-Murtaza-Hassan
 
Kubernetes Internals
Kubernetes InternalsKubernetes Internals
Kubernetes Internals
 
Containers and Kubernetes -Notes Leo
Containers and Kubernetes -Notes LeoContainers and Kubernetes -Notes Leo
Containers and Kubernetes -Notes Leo
 
(Draft) Kubernetes - A Comprehensive Overview
(Draft) Kubernetes - A Comprehensive Overview(Draft) Kubernetes - A Comprehensive Overview
(Draft) Kubernetes - A Comprehensive Overview
 
Kubernetes fundamentals
Kubernetes fundamentalsKubernetes fundamentals
Kubernetes fundamentals
 
An Introduction to Kubernetes and Continuous Delivery Fundamentals
An Introduction to Kubernetes and Continuous Delivery FundamentalsAn Introduction to Kubernetes and Continuous Delivery Fundamentals
An Introduction to Kubernetes and Continuous Delivery Fundamentals
 
Kubernetes for Java developers
Kubernetes for Java developersKubernetes for Java developers
Kubernetes for Java developers
 
Docker and kubernetes
Docker and kubernetesDocker and kubernetes
Docker and kubernetes
 
Kubernetes 101
Kubernetes 101Kubernetes 101
Kubernetes 101
 
Kubernetes a comprehensive overview
Kubernetes   a comprehensive overviewKubernetes   a comprehensive overview
Kubernetes a comprehensive overview
 
Kubernetes acomprehensiveoverview
Kubernetes acomprehensiveoverviewKubernetes acomprehensiveoverview
Kubernetes acomprehensiveoverview
 
Getting started with google kubernetes engine
Getting started with google kubernetes engineGetting started with google kubernetes engine
Getting started with google kubernetes engine
 

More from Ronny Trommer

DevJam 2019 - OpenNMS Integration API
DevJam 2019 - OpenNMS Integration APIDevJam 2019 - OpenNMS Integration API
DevJam 2019 - OpenNMS Integration API
Ronny Trommer
 
Dev-Jam 2019 - New since Dev-Jam 2018
Dev-Jam 2019 - New since Dev-Jam 2018Dev-Jam 2019 - New since Dev-Jam 2018
Dev-Jam 2019 - New since Dev-Jam 2018
Ronny Trommer
 
Dev-Jam 2019 - Container & OpenNMS
Dev-Jam 2019 - Container & OpenNMSDev-Jam 2019 - Container & OpenNMS
Dev-Jam 2019 - Container & OpenNMS
Ronny Trommer
 
Dev-Jam 2019 - Developing OpenNMS on Apache Karaf
Dev-Jam 2019 - Developing OpenNMS on Apache KarafDev-Jam 2019 - Developing OpenNMS on Apache Karaf
Dev-Jam 2019 - Developing OpenNMS on Apache Karaf
Ronny Trommer
 
Dev-Jam 2019 - Developing & Contributing to OpenNMS
Dev-Jam 2019 - Developing & Contributing to OpenNMSDev-Jam 2019 - Developing & Contributing to OpenNMS
Dev-Jam 2019 - Developing & Contributing to OpenNMS
Ronny Trommer
 
DevJam 2019 - Building an ALEC Time Engine
DevJam 2019 - Building an ALEC Time EngineDevJam 2019 - Building an ALEC Time Engine
DevJam 2019 - Building an ALEC Time Engine
Ronny Trommer
 
OUCE Reporting Enhancements in OpenNMS
OUCE Reporting Enhancements in OpenNMSOUCE Reporting Enhancements in OpenNMS
OUCE Reporting Enhancements in OpenNMS
Ronny Trommer
 
Who Pulls the Strings?
Who Pulls the Strings?Who Pulls the Strings?
Who Pulls the Strings?
Ronny Trommer
 
OSMC 2011 - Introduction to OpenNMS
OSMC 2011 - Introduction to OpenNMSOSMC 2011 - Introduction to OpenNMS
OSMC 2011 - Introduction to OpenNMS
Ronny Trommer
 

More from Ronny Trommer (9)

DevJam 2019 - OpenNMS Integration API
DevJam 2019 - OpenNMS Integration APIDevJam 2019 - OpenNMS Integration API
DevJam 2019 - OpenNMS Integration API
 
Dev-Jam 2019 - New since Dev-Jam 2018
Dev-Jam 2019 - New since Dev-Jam 2018Dev-Jam 2019 - New since Dev-Jam 2018
Dev-Jam 2019 - New since Dev-Jam 2018
 
Dev-Jam 2019 - Container & OpenNMS
Dev-Jam 2019 - Container & OpenNMSDev-Jam 2019 - Container & OpenNMS
Dev-Jam 2019 - Container & OpenNMS
 
Dev-Jam 2019 - Developing OpenNMS on Apache Karaf
Dev-Jam 2019 - Developing OpenNMS on Apache KarafDev-Jam 2019 - Developing OpenNMS on Apache Karaf
Dev-Jam 2019 - Developing OpenNMS on Apache Karaf
 
Dev-Jam 2019 - Developing & Contributing to OpenNMS
Dev-Jam 2019 - Developing & Contributing to OpenNMSDev-Jam 2019 - Developing & Contributing to OpenNMS
Dev-Jam 2019 - Developing & Contributing to OpenNMS
 
DevJam 2019 - Building an ALEC Time Engine
DevJam 2019 - Building an ALEC Time EngineDevJam 2019 - Building an ALEC Time Engine
DevJam 2019 - Building an ALEC Time Engine
 
OUCE Reporting Enhancements in OpenNMS
OUCE Reporting Enhancements in OpenNMSOUCE Reporting Enhancements in OpenNMS
OUCE Reporting Enhancements in OpenNMS
 
Who Pulls the Strings?
Who Pulls the Strings?Who Pulls the Strings?
Who Pulls the Strings?
 
OSMC 2011 - Introduction to OpenNMS
OSMC 2011 - Introduction to OpenNMSOSMC 2011 - Introduction to OpenNMS
OSMC 2011 - Introduction to OpenNMS
 

Recently uploaded

WeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation TechniquesWeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation Techniques
Postman
 
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
Edge AI and Vision Alliance
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
Alex Pruden
 
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their MainframeDigital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Precisely
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Tosin Akinosho
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
Ivanti
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
Tatiana Kojar
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
Jason Yip
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
Antonios Katsarakis
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
saastr
 
The Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptxThe Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptx
operationspcvita
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
DanBrown980551
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
Miro Wengner
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
Edge AI and Vision Alliance
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
panagenda
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Brandon Minnick, MBA
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying AheadDigital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Wask
 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
Ivo Velitchkov
 

Recently uploaded (20)

WeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation TechniquesWeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation Techniques
 
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
 
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their MainframeDigital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
 
The Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptxThe Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptx
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying AheadDigital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying Ahead
 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
 

DevJam 2019 - Introduction to Kubernetes

  • 1. INTRODUCTION TO KUBERNETES OpenNMS DevJam 2019 by Alejandro Galue
  • 2. WHY CONTAINERS • Build, Ship, and Run any App, Anywhere. • No more dependency problems; e.x. no RPM/DEB or dynamic libraries incompatibility issues. • No more: "but it works on my machine", as the same image used in dev is deployed to prod. • Serverless implementations are mainly backed by containers.
  • 3. WHY KUBERNETES • Be able to manage hundreds or thousands of containers, on a fleet of hundred or thousands of nodes. • Be able to deploy an application without worrying about the hardware infrastructure. • Be able to guarantee that applications will stay running according with the specifications. • Be able to facilitate scaling and upgrades not only for the workloads but for the k8s cluster itself.
  • 4. WHAT IS KUBERNETES • A distributed cluster technology that manages container-based systems in a declarative manner using an API (a container orchestrator). • Designed from the ground-up as a loosely coupled collection of components centered around deploying, maintaining and scaling workloads. • Abstracts away the underlying hardware of the nodes and provides a uniform interface for workloads to be both deployed and consume the shared pool of resources. • Works as an engine for resolving state by converging actual and the desired state of the system (self-healing).
  • 5. A COUPLE OF KEY CONCEPTS…
  • 6. PODS • Atomic unit or smallest “unit of work”of Kubernetes. • Pods are one or MORE containers that share volumes, a network namespace, and are a part of a single context. Host A Pod ExternalVolume container container Network Namespace 10.255.16.3 Pod Network Ephemeral
  • 7. SERVICES • Unified method of accessing the exposed workloads of Pods. • Durable resource - static cluster IP - static namespaced DNS name Host A Labels: app=nginx env=prod Labels: app=nginx env=prod Labels: app=mysql env=dev Host B Labels: app=nginx env=prod Labels: app=mysql env=prod Labels: app=nginx env=dev Service app=nginx env=prod NOT Ephemeral
  • 9. Control Plane - 1,2,…n kube controller manager kube scheduler kube apiserver kubectl Node 1 Containers System Services kubelet kube-proxy Container Runtime Pod Pod Node 2 Containers System Services kubelet kube-proxy Container Runtime Pod Pod cloud controller manager Cloud Provider API Load Balancer End Users Cloud Provider Network Edge
  • 10. Control Plane - 1,2,…n Containers kube-control-manager kube-scheduler kube apiserver kubectl Node 1 Containers System Services kubelet kube-proxy Container Runtime Pod Pod Node 2 Containers System Services kubelet kube-proxy Container Runtime Pod Pod cloud-controller-manager Cloud Provider API Load Balancer End Users Cloud Provider Network Edge System Services kubelet Container Runtime additional services kube-proxy
  • 11. KUBE-APISERVER • Provides a forward facing REST interface into the kubernetes control plane and datastore. • All clients and other applications interact with kubernetes strictly through the API Server. • Acts as the gatekeeper to the cluster by handling authentication and authorization, request validation, mutation, and admission control in addition to being the front-end to the backing datastore. master
  • 12. • etcd acts as the cluster datastore. • Purpose in relation to Kubernetes is to provide a strong, consistent and highly available key-value store for persisting cluster state. • Stores objects and config information. • Uses “Raft Consensus” among a quorum of systems to create a fault-tolerant consistent “view” of the cluster. master
  • 13. KUBE-CONTROLLER-MANAGER • Serves as the primary daemon that manages all core component control loops. • Monitors the cluster state via the apiserver and steers the cluster towards the desired state. master
  • 14. KUBE-SCHEDULER • Component on the master that watches newly created pods that have no node assigned, and selects a node for them to run on. • Factors taken into account for scheduling decisions include individual and collective resource requirements, hardware/software/policy constraints, affinity and anti-affinity specifications, data locality, inter-workload interference and deadlines. master
  • 15. CLOUD-CONTROLLER-MANAGER • Daemon that provides cloud-provider specific knowledge and integration capability into the core control loop of Kubernetes. • The controllers include Node, Route, Service, and add an additional controller to handle things such as PersistentVolume Labels. master Optional
  • 16. KUBE-PROXY • Manages the network rules on each node. • Performs connection forwarding or load balancing for Kubernetes cluster services. worker
  • 17. KUBELET • An agent that runs on each node in the cluster. It makes sure that containers are running in a pod. • The kubelet takes a set of PodSpecs that are provided through various mechanisms and ensures that the containers described in those PodSpecs are running and healthy. worker
  • 18. CONTAINER RUNTIME ENGINE A container runtime is a CRI (Container Runtime Interface) compatible application that executes and manages containers. • containerd (Docker) • cri-o • rkt • kata (formerly clear and hyper) • virtlet (VM CRI compatible runtime) worker
  • 19. KUBERNETES NETWORKING • Pod Network - Cluster-wide network used for pod-to-pod communication managed by a CNI (Container Network Interface) plugin. • Service Network - Cluster-wide range of Virtual IPs managed by kube-proxy for service discovery.
  • 20. FUNDAMENTAL NETWORKING RULES • All containers within a pod can communicate with each other unimpeded. • All Pods can communicate with all other Pods without NAT. • All nodes can communicate with all Pods (and vice-versa) without NAT. • The IP that a Pod sees itself as is the same IP that others see it as. What if I want to limit communication within Pods ? Learn about Network Policies (and make sure the chosen CNI supports it)
  • 21. API
  • 22. API OVERVIEW • The REST API is the true keystone of Kubernetes. • Everything within Kubernetes is as an API Object. • Referenced within an object as the apiVersion and kind. Format: /apis/<group>/<version>/<resource> Examples: /apis/apps/v1/deployments /apis/batch/v1beta1/cronjobs
  • 23. OBJECT MODEL • Objects are a “record of intent” or a persistent entity that represent the desired state of the object within the cluster. • All objects MUST have  apiVersion, kind, and poses the nested fields metadata.name, metadata.namespace, and metadata.uid.
  • 24. OBJECT EXPRESSION -YAML • Files or other representations of Kubernetes Objects are generally represented in YAML. • Three basic data types: - mappings  - hash or dictionary, - sequences - array or list - scalars - string, number, boolean etc apiVersion: v1 kind: Pod metadata: name: sample namespace: test spec: containers: - name: container1
 image: nginx - name: container2
 image: alpine
  • 25. YAMLVS JSON {   "apiVersion": "v1",   "kind": "Pod",   "metadata": {     "name": "pod-example"   },   "spec": {     "containers": [       {         "name": "nginx",         "image": "nginx:stable-alpine",         "ports": [ { "containerPort": 80 } ]       }     ]   } } apiVersion: v1 kind: Pod metadata:   name: pod-example spec:   containers:   - name: nginx     image: nginx:stable-alpine     ports:     - containerPort: 80 Are you wondering about theYAML schema ? kubectl explain is your friend ;) What about kinds or versions ? kubectl api-versions is your friend ;)
  • 27. NAMESPACES Namespaces are a logical cluster or environment, and are the primary method of partitioning a cluster or scoping access. apiVersion: v1 kind: Namespace metadata:   name: prod   labels:     app: MyBigWebApp $ kubectl get ns --show-labels NAME          STATUS    AGE       LABELS default       Active    11h       <none> kube-public   Active    11h       <none> kube-system   Active    11h       <none> prod          Active    6s        app=MyBigWebApp Known as Projects in OpenShift
  • 28. PODS • Atomic unit or smallest “unit of work”of Kubernetes. • Foundational building block of Kubernetes Workloads. • Pods are one or MORE containers that share volumes, a network namespace, and are a part of a single context. Host A Pod ExternalVolume container container Network Namespace 10.255.16.3 Pod Network apiVersion: v1 kind: Pod metadata: name: sample spec: containers: - name: nginx image: nginx:stable-alpine ports:   - containerPort: 80     name: http     protocol: TCP env: - name: MYVAR value: isAwesome command: [“/bin/sh”, “-c”] args: [“echo ${MYVAR}”] readinessProbe: tcpSocket: port: http initialDelaySeconds: 10 periodSeconds: 10 livenessProbe: httpGet: path: / port: http initialDelaySeconds: 30 periodSeconds: 60 Array of ports to expose Name of the container Container ImageArray of environment variables Entrypoint Array ~ Docker ENTRYPOINT Arguments to pass to the command ~ Docker CMD Tells when the application is ready to receive requests Checks if the container is still alive (running)
  • 29. LABELS • key-value pairs that are used to identify, describe and group together related sets of objects or resources. • NOT characteristic of uniqueness. • Have a strict syntax with a slightly limited character set*. https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set Host A Pod container Network Namespace 10.255.16.3 Pod Network Labels: gpu=nvidia Labels: app=nginx env=prod apiVersion: v1 kind: Pod metadata:   name: pod-label-example   labels:
     app: nginx
     env: prod spec:   containers:   - name: nginx     image: nginx:stable-alpine     ports:     - containerPort: 80
  • 30. SELECTORS Selectors use labels to filter or select objects, and are used throughout Kubernetes. Host A Pod mlapp Network Namespace 10.255.16.3 Pod Network Labels: gpu=nvidia Labels: app=mlapp env=prodapiVersion: v1 kind: Pod metadata:   name: mlapp   labels:
     app: mlapp
     env: prod spec:   nodeSelector:   - gpu: nvidia   containers:   - name: nginx     image: tensorflow/tensorflow Host B Labels: gpu=amd
  • 31. SERVICES • Unified method of accessing the exposed workloads of Pods. • Durable resource (unlike Pods) - static cluster-unique IP - static namespaced DNS name <service name>.<namespace>.svc.cluster.local There are 4 major service types: • ClusterIP (default) • NodePort • LoadBalancer • ExternalName
  • 32. CLUSTER IP SERVICE Labels: app=nginx env=prod kube-proxy Host A Labels: app=nginx env=prod Host B Pod Host C Pod Network kube-proxy kube-proxy Name:         example-prod Selector:     app=nginx,env=prod Type:         ClusterIP IP:           10.96.28.176 Port:         <unset>  80/TCP TargetPort:   80/TCP Endpoints:    10.255.16.3:80,
               10.255.16.4:80 • The Pod on host C requests the service. • Hits host iptables and it load-balances the connection between the endpoints residing on Hosts A, B / # nslookup example-prod.default.svc.cluster.local Name:      example-prod.default.svc.cluster.local Address 1: 10.96.28.176 example-prod.default.svc.cluster.local
  • 33. NODE PORT SERVICE Labels: app=nginx env=prod kube-proxy Host A Labels: app=nginx env=prod Host B Labels: app=nginx env=dev Host C Name:         example-prod Selector:     app=nginx,env=prod Type:         NodePort IP:           10.96.28.176 Port:         <unset>  80/TCP TargetPort:   80/TCP NodePort:     <unset>  32410/TCP Endpoints:    10.255.16.3:80,
               10.255.16.4:80 • User can hit any host in cluster on NodePort IP and get to service. • Does introduce extra hop if hitting a host without instance of the pod. 32410 32410kube-proxy 32410kube-proxy
  • 34. LOAD BALANCER SERVICE • LoadBalancer services extend NodePort. • Works in conjunction with an external system to map a cluster external IP to the exposed service. Labels: app=nginx env=prod kube-proxy Host A Labels: app=nginx env=prod Host B Labels: app=nginx env=dev Host C 32410 32410kube-proxy 32410kube-proxy Load Balancer Cloud Provider Edge Network Name:         example-prod Selector:     app=nginx,env=prod Type:         LoadBalancer IP:           10.96.28.176 LoadBalancer
 Ingress:      172.17.18.43 Port:         <unset>  80/TCP TargetPort:   80/TCP NodePort:     <unset>  32410/TCP Endpoints:    10.255.16.3:80,
               10.255.16.4:80
  • 35. INGRESSKnown as Routes in OpenShift • An API object that manages external access to the services in a cluster, through a single LoadBalancer. • Provides load balancing, SSL termination and name/path-based virtual hosting • Gives services externally-reachable URLs. • Multiple implementations to choose from. service: grafana port=3000 Host A service: kibana port=5601 Host B service: opennms port=8980 Host C grafana.example.com kibana.example.com Cloud Provider Edge Network Ingress Controller Host D Load Balancer onms.example.com apiVersion: extensions/v1beta1 kind: Ingress metadata: name: ingress-rules annotations: kubernetes.io/ingress.class: nginx spec: rules: - host: onms.example.com http: paths: - path: / backend: serviceName: opennms servicePort: 8980 …
  • 36. EXTERNAL NAME SERVICE • ExternalName is used to reference endpoints OUTSIDE the cluster. • Creates an internal CNAME DNS entry that aliases another. apiVersion: v1 kind: Service metadata:   name: example-prod spec:   type: ExternalName spec:   externalName: example.com
  • 38. WORKLOADS • Workloads within Kubernetes are higher level objects that manage Pods or other higher level objects. • In ALL CASES a Pod Template is included, and acts the base tier of management. •ReplicaSet •Deployment •DaemonSet •StatefulSet •Job •CronJob
  • 39. PODTEMPLATE • Workload Controllers manage instances of Pods based off a provided template. • Pod Templates are Pod specs with limited metadata. • Controllers use Pod Templates to make actual pods apiVersion: v1 kind: Pod metadata:   name: pod-example   labels:
     app: nginx spec:   containers:   - name: nginx     image: nginx template:   metadata:     labels:
       app: nginx   spec:     containers:     - name: nginx       image: nginx apiVersion: apps/v1 kind: Deployment metadata:   name: deployment-example   labels:
     app: nginx spec: replicas: 3   selector:     matchLabels:
       app: nginx template:   metadata:     labels:
       app: nginx   spec:     containers:     - name: nginx       image: nginx
  • 40. RESOURCE MODEL • Request: amount of a resource allowed to be used, with a strong guarantee of availability. - CPU (seconds/second), RAM (bytes) - Scheduler will not over-commit requests • Limit: max amount of a resource that can be used, regardless of guarantees - Scheduler ignores limits apiVersion: v1 kind: Pod metadata:   name: pod-example   labels:
     app: nginx spec:   containers:   - name: nginx     image: nginx   resources:   requests:   memory: "64Mi"     cpu: "250m"   limits:   memory: "128Mi"     cpu: "500m"
  • 41. INIT CONTAINERS If we compare a container template with a Class definition in Java, an initContainer would be the constructor of a class; while a running Pod, would be an instance of that class. Can be used for everything that should happen before the containers within a Pod start running. For example: wait for dependencies, initialize volumes or databases, verify requirements, etc.
  • 42. REPLICA SET • Primary method of managing pod replicas and their lifecycle. • Includes their scheduling, scaling, and deletion. • Their job is simple: Always ensure the desired number of pods are running. ReplicaSet Pod • replicas: The desired number of instances of the Pod. • selector: The label selector for the ReplicaSet will manage ALL Pod instances that it targets; whether it’s desired or not. apiVersion: apps/v1 kind: ReplicaSet metadata:   name: rs-example spec:   replicas: 3   selector:     matchLabels:       app: nginx
       env: prod   template:     <pod template>
  • 43. DEPLOYMENT • Declarative method of managing Pods via ReplicaSets. • Provide rollback functionality and update control. • Updates are managed through the pod- template-hash label. • Each iteration creates a unique label that is assigned to both the ReplicaSet and subsequent Pods. PodReplicaSetDeployment • revisionHistoryLimit: The number of previous iterations of the Deployment to retain. • strategy: Describes the method of  updating the Pods based on the type.Valid options are: - Recreate: All existing Pods are killed before the new ones are created. - RollingUpdate: Cycles through updating the Pods according to the parameters: maxSurge and maxUnavailable. apiVersion: apps/v1 kind: Deployment metadata:   name: deployment-example spec:   replicas: 3
   revisionHistoryLimit: 3   selector:     matchLabels:       app: nginx
       env: prod   strategy:     type: RollingUpdate     rollingUpdate:       maxSurge: 1       maxUnavailable: 0   template:     <pod template>
  • 44. DAEMON SET • Ensure that all nodes matching certain criteria will run an instance of the supplied Pod. • They bypass default scheduling mechanisms. • Are ideal for cluster wide services such as log forwarding, or health monitoring. • Revisions are managed via a controller-revision-hash label. DaemonSet Pod • spec.template.spec.nodeSelector: The primary selector used to target nodes. • Default Host Labels: - kubernetes.io/hostname - beta.kubernetes.io/os - beta.kubernetes.io/arch • Cloud Host Labels: - failure-domain.beta.kubernetes.io/zone - failure-domain.beta.kubernetes.io/region - beta.kubernetes.io/instance-type apiVersion: apps/v1 kind: DaemonSet metadata:   name: ds-example spec:   revisionHistoryLimit: 3   selector:     matchLabels:       app: nginx   updateStrategy:     type: RollingUpdate     rollingUpdate:       maxUnavailable: 1   template:     spec:       nodeSelector:         nodeType: edge      <pod template>
  • 45. STATEFUL SET • Tailored to managing Pods that must persist or maintain state. • Pod identity including hostname, network, and storage WILL be persisted. • Assigned a unique ordinal name following the convention of ‘<statefulset name>-<ordinal index>’. • Naming convention is also used in Pod’s network Identity and Volumes. • Pod lifecycle will be ordered and follow consistent patterns. • Revisions are managed via a controller-revision-hash label. Pod apiVersion: apps/v1 kind: StatefulSet metadata:   name: sts-cassandra spec:   replicas: 3   selector:     matchLabels:       app: cassandra   serviceName: cassandra   updateStrategy:    type: RollingUpdate    rollingUpdate:      partition: 0   template:     metadata:       labels:         app: cassandra     spec:       containers:       - name: cassandra-node         image: cassandra:3.11.4         env:         - name: CASSANDRA_SEEDS         value: sts-cassandra-0.cassandra         - name: CASSANDRA_CLUSTER_NAME         value: my-cluster         ports:         - containerPort: 7000         - containerPort: 7199         - containerPort: 9042         volumeMounts:         - name: data           mountPath: /cassandra_data   volumeClaimTemplates:   - metadata:       name: data     spec:       accessModes: [ "ReadWriteOnce" ]       storageClassName: standard       resources:         requests:           storage: 100Gi The name of the associated headless service; or a service without a ClusterIP Pods with an ordinal greater than the partition value will be updated one-by-one in reverse order StatefulSet Template of the persistent volume(s) request to use for each instance of the StatefulSet.
  • 46. HEADLESS SERVICE <StatefulSet Name>-<ordinal>.<service name>.<namespace>.svc.cluster.local apiVersion: v1 kind: Service metadata:   name: cassandra spec:   clusterIP: None   selector:     app: cassandra   ports:   - name: intra-node     port: 80   - name: jmx     port: 7199   - name: cql     port: 9042 / # dig cassandra.default.svc.cluster.local +noall +answer ; <<>> DiG 9.9.4-RedHat-9.9.4-74.el7_6.1 <<>> cassandra.default.svc.cluster.local +noall +answer ;; global options: +cmd cassandra.default.svc.cluster.local. 5 IN A 172.17.0.5 cassandra.default.svc.cluster.local. 5 IN A 172.17.0.4 cassandra.default.svc.cluster.local. 5 IN A 172.17.0.6 / # dig sts-cassandra-0.cassandra.default.svc.cluster.local +noall +answer ; <<>> DiG 9.9.4-RedHat-9.9.4-74.el7_6.1 <<>> sts-cassandra-0.cassandra.default.svc.cluster.local +noall +answer / # dig sts-cassandra-1.cassandra.default.svc.cluster.local +noall +answer ; <<>> DiG 9.9.4-RedHat-9.9.4-74.el7_6.1 <<>> sts-cassandra-1.cassandra.default.svc.cluster.local +noall +answer / # dig sts-cassandra-2.cassandra.default.svc.cluster.local +noall +answer ; <<>> DiG 9.9.4-RedHat-9.9.4-74.el7_6.1 <<>> sts-cassandra-2.cassandra.default.svc.cluster.local +noall +answer $ kubectl get pods NAME            READY     STATUS    RESTARTS   AGE sts-cassandra-0   1/1       Running   0          11m sts-cassandra-1   1/1       Running   0          11m sts-cassandra-2   1/1       Running   0          11m
  • 47. VOLUME CLAIMTEMPLATE <Volume Name>.<StatefulSet Name>-<ordinal>  volumeClaimTemplates:   - metadata:       name: data     spec:       accessModes: [ "ReadWriteOnce" ]       storageClassName: standard       resources:         requests:           storage: 1Gi Persistent Volumes associated with a StatefulSet will NOT be automatically garbage collected when its associated StatefulSet is deleted. They must manually be removed. $ kubectl get pvc NAME                STATUS    VOLUME                                    CAPACITY   ACCESS MODES   STORAGECLASS   AGE data-cassandra-sts-0 Bound pvc-8baec2a5-8c77-11e9-a6be-0800275ddaf0 100Gi RWO standard 17m data-cassandra-sts-1 Bound pvc-95463714-8c77-11e9-a6be-0800275ddaf0 100Gi RWO standard 17m data-cassandra-sts-2 Bound pvc-9807c42a-8c77-11e9-a6be-0800275ddaf0 100Gi RWO standard 17m
  • 48. JOB • Job controller ensures one or more pods are executed and successfully terminate. • Will continue to try and execute the job until it satisfies the completion and/or parallelism condition. • Pods are NOT cleaned up until the job itself is deleted. PodJob apiVersion: batch/v1 kind: Job metadata:   name: job-example spec:   backoffLimit: 4   completions: 4   parallelism: 2   template:     spec:       restartPolicy: Never     template:     <pod-template> • backoffLimit: The number of failures before the job itself is considered failed. • completions: The total number of successful completions desired. • parallelism: How many instances of the pod can be run concurrently. • spec.template.spec.restartPolicy:Jobs only support a restartPolicy of type Never or OnFailure.
  • 49. CRONJOB • An extension of the Job Controller, it provides a method of executing jobs on a cron-like schedule. • CronJobs within Kubernetes
  use UTC ONLY. PodJobCronJob apiVersion: batch/v1beta1 kind: CronJob metadata:   name: cronjob-example spec:   schedule: "*/1 * * * *"   successfulJobsHistoryLimit: 3   failedJobsHistoryLimit: 1   jobTemplate:     spec:       completions: 4       parallelism: 2       template:         <pod template> The cron schedule for the job The number of successful jobs to retain The number of failed jobs to retain
  • 50. OPERATORS • Might be the most powerful feature of K8s. • When none of the workloads fits the needs of your app, you can create your own controllers with your own specifications (CRD). • Should be implemented in Go, but there are alternatives. • A given controller usually requires a ServiceAccount with proper RBAC Role to be able to manage workloads (e.x. create and maintain Pods, Services, etc.) apiVersion: acid.zalan.do/v1 kind: postgresql metadata: name: opennms-database spec: teamId: OpenNMS volume: size: 100Gi numberOfInstances: 3 enableMasterLoadBalancer: false enableReplicaLoadBalancer: false users: opennms: - superuser - createdb databases: opennms: opennms postgresql: version: "10"
  • 52. VOLUMES • Storage that is tied to the Pod’s Lifecycle. • A pod can have one or more types of volumes attached to it. • Can be consumed by any of the containers within the pod. • Survive Pod restarts; however their durability beyond that is dependent on the Volume Type.
  • 53. PERSISTENTVOLUMES • A PersistentVolume (PV) represents a storage resource.  • PVs are a cluster wide resource linked to a backing storage provider: NFS, EBS, GCEPersistentDisk, etc. • Generally provisioned by an administrator. • Their lifecycle is handled independently from a pod • CANNOT be attached to a Pod directly. Relies on a  PersistentVolumeClaim (PVC). • A PVC is a namespaced request for storage.
  • 54. STORAGE CLASS • Satisfies a set of requirements instead of mapping to a storage resource directly. • Ensures that an application’s claim for storage is portable across numerous backends or providers.
  • 56. CONFIG MAPS • Externalized data stored within kubernetes. • Can be referenced through several different means: - environment variable - a command line argument (via environment variable) - injected as a file into a volume mount • Can be created from a manifest, literals, directories, or files directly. apiVersion: v1 kind: ConfigMap metadata:   name: manifest-example data:   state: Minnesota   city: Minneapolis   content: |     Look at this,     its multiline!
  • 57. SECRETS • Functionally identical to a ConfigMap. • Stored as base64 encoded content. • Encrypted at rest within etcd (if configured!). • Ideal for username/passwords, certificates or other sensitive information that should not be stored in a container. • Can be created from a manifest, literals, directories, or from files directly. apiVersion: v1 kind: Secret metadata:   name: manifest-secret type: Opaque data:   username: ZXhhbXBsZQ==   password: bXlwYXNzd29yZA== • type: There are three different types of secrets within Kubernetes: - docker-registry - credentials used to authenticate to a container registry - generic/Opaque - literal values from different sources - tls - a certificate based secret • data: Contains key-value pairs of base64 encoded content.
  • 59. BARE-METAL, CLOUD, OR
 LOCAL INSTALLATION Install OpenShift is hard ;) Local PC Installation
 For development and testing • minikube • minishift (for OKD) • oc cluster up (for OKD) • Docker for Mac or Windows • microk8s (for Linux via Snap) Cloud Installation
 Single Command Experience • gcloud container clusters create • kops create cluster • eksctl create cluster • az aks create • az openshift create OpenShift 4 promises this for the first time since its conception. Bare-Metal / On-Premise • kubeadm (single command to spin up a master or a worker) • OpenShift/OKD Ansible Tools On-Premise vs Cloud Managed • Masters are managed for you, and the only additional resources are the instances for worker nodes • Complete integration with cloud services (volumes, network, load balancers, etc.)
  • 60. SECURITY Here is where OpenShift is Excellent!!! https://kubernetes.io/blog/2018/07/18/11-ways-not-to-get-hacked/
  • 61. THINGS TO KEEP IN MIND • Users (to access cluster through the API). Usually implemented through certificates. • Service Accounts (for controllers and operators, to make changes from within K8s). • RBAC: Roles and RoleBinding, ClusterRole and ClusterRoleBinding (for users, groups and service accounts). • Never run containers as root, and block privilege escalations (OpenShift enforce this by default using SecurityContextConstraint; known I k8s as PodSecurityPolicy). • Limit resources per namespace (total CPUs and memory, number of Pods, …). • Create your own images and keep a private registry (OpenShift offers this by default). • Research about NetworkPolicy (which requires a CNI that supports this feature). • Research about applications like vault (to enhance protection for your secrets). • Research about clair (a tool for vulnerability scans on containers). • Research about kube-bench (a tool that analyzes your cluster for security best practices). • Research about gvisor.dev (a container sandbox focused on security and efficiency). Some inspired by OpenShift
  • 62. ACCESS On user the certificate (signed by the CA used on the API server), the Common Name (CN) will be interpreted as the username, and the Organization (O) as the group where this user belongs to. Use the kubectl command to create the kubeconfig based on the user’s certificate, to access the cluster resources. Define an RBAC Role to connect resources (deployments, pods, ingress, …) with operations (create, delete, list, …) on a given namespace. Define an RBAC RoleBinding to connect a role with subjects (i.e. group, user, service- account) Use a ClusterRole for cluster wide access For a ClusterRole use ClusterRoleBinding Now you have access! cluster-admin O=system:masters
  • 64. HOWTO MONITOR K8S? • Based on Google’s Four Golden Signals: latency, errors, traffic, saturation, use Brendan Gregg’s USE method (Utilization, Saturation, Errors) for Resources, and Tom Wilkie’s RED method (Rate, Errors, Duration) for Services. • Metric Server collects metrics such as CPU and Memory by each pod and node from the Summary API, exposed by Kubelet (via cAdvisor) on each node.This allows the usage of K8s features like the HorizontalPodAutoscaler. • The usage of Prometheus is widely adopted.This solution is based on their own node_exporter (deployed as DaemonSet) which exposes lots of Linux metrics in Prometheus format.
  • 65. MONITOR K8S WITH OPENNMS • OpenNMS has been used to monitor network infrastructure (or network devices). • OpenNMS can monitor servers at some degree, and can do some level of application monitoring; but traditionally other applications are used in this context. • For Kubernetes, Prometheus is wildly adopted, on a level that it is considered the standard. • OpenNMS can be used to expand the functionality of the already existing observability/monitoring tools in Kubernetes world.
  • 66. MONITOR K8S WITH OPENNMS • Have an operator forwarding important K8s events to OpenNMS, so you can use all the event processing infrastructure to react upon changes on your K8s cluster. • There is no such thing as fixed IP addresses in Kubernetes, so forget about Pods. But, if providing support FQDN support in OpenNMS is possible, that can make monitoring K8s services a reality in the future. • Build a solution based on the Metrics API / cAdvisor, or Prometheus (directly or indirectly through the node_exporter), to get data from the running Nodes, Pods and Services. • SNMP could run on Kubernetes nodes; but that’s not an option for Pods.
  • 67. HOWTO DEPLOY OPENNMS IN K8S? • Use a StatefulSet for OpenNMS (1 replica), to take advantage of auto-deployed PVCs, as the dynamic nature of the configuration directory makes OpenNMS a stateful application. Keep in mind "kubectl cp" for transferring files. • Use a StatefulSet for Minions, to take advantage of the persistent network identity, and use Pod name as the minion-ID.This is recommended, even if Minions are stateless by nature. • Use a Deployment for Sentinels, as they are stateless in nature and having a unique identity is not relevant (for now). Although a StatefulSet won’t hurt. • A combination of initContainers, config-maps, and secrets can be used to pre-configure the mandatory settings on all the applications.The PersistentVolume will keep the custom settings added by the user during the lifetime of the product.
  • 68. OPENNMS DEPENDENCIES It depends… a given team should decide if it it worth running PostgreSQL, Kafka, Cassandra and Elasticsearch within K8s. Keep in mind that all databases are stateful in nature, and managing the life-cycle is different on each case. Solutions based on operators when available are preferred.
  • 69. INTEGRATE OPENNMS WITH K8S • OpenNMS can forward everything to Kafka, and even consume events from Kafka. From there, Serverless technologies or Kafka Streams apps deployed into K8s, can be used to create your own horizontally scalable integrations triggered by events, alarms or collected metrics. • What about using OpenNMS as the source of metrics for Horizontal Pod Autoscaling or Cluster Workers auto-scaling ?
  • 70. DEMO
  • 71. OVERWHELMED? Do not forget we haven’t talked about … • Several k8s features not discussed here ;) • Helm (the k8s package manager; the yum or apt-get for k8s) • Service Meshes (Envoy, Istio, Linkerd, …) • Serverless (Kubeless, Fission, Knative, …) • Machine Learning (kubeflow, …) • VMs and Kubernetes (KubeVirt) • Operators In Depth:
 https://operatorhub.io
 https://github.com/operator-framework And more…
 Not a big enough lis ? Check https://landscape.cncf.io
  • 73. THANKYOU! This work has been inspired by a few presentations from: https://github.com/cncf/presentations/tree/master/kubernetes Demo: https://github.com/OpenNMS/opennms-drift-kubernetes