While developers see and realize the benefits of Kubernetes, how it improves efficiencies, saves time, and enables focus on the unique business requirements of each project; InfoSec, infrastructure, and software operations teams still face challenges when managing a new set of tools and technologies, and integrating them into existing enterprise infrastructure. This is especially true for environments where security and governance requirements are so strict as to come into conflict with the cloud-native reference architectures. This deck will outline a plan that leverages Kubernetes as an infrastructure abstraction (hint: there is a lot more to it than just container orchestration!). Such an approach allows enterprises to untie themselves from infrastructure provider-specific technology stack and free development to use whichever tool fits their use case best. But how do you implement open source cloud-native technologies while meeting enterprise security and governance requirements? We’ll summarize common prerequisites for running Kubernetes in production, and how to leverage fine-grained controls and separation of responsibilities to meet enterprise governance and security needs; what’s needed for a general architecture of a centralized Kubernetes operations layer based on open source components such as Prometheus, Grafana, ELK Stack, Keycloak, etc.

1. Kubernetes as Infrastructure Abstraction
Cloud-Native Implementation Strategy in an Enterprise Environment

2. Introductions
Oleg Chunikhin
CTO, Kublr
 20+ years in software architecture & development
 Working w/ Kubernetes since its release in 2015
 CTO at Kublr—an enterprise ready container
management platform
 Twitter @olgch; @kublr
Like what you hear? Tweet at us!

3. Kubernetes & Cloud Native
in Restrictive/Enterprise Environments
@olgch; @kublr

4. Cloud Native Attributes
Lightweight containers
Language agnostic
Microservices
API
Stateless/stateful separation
Self-service infrastructure
Isolated from OS/server deps
Agile DevOps processes
Highly automated
Declarative resource management
@olgch; @kublr

5. Applications & Architecture
Digital –
Web/Mobile
Data Science &
Machine Learning
Digital
Transformation
and App
Modernization
Hybrid Cloud
Multi-Cloud
Edge
Computing
Private
Cloud
Internet of
Things
Microservices
CI / CD@olgch; @kublr

6. Cloud Native
Cloud native precursors
SRE, DevOps, 12factor app
SOA / Microservices, API
(management)
Containers, Cloud, Virtualization
Benefits
Empower IT teams to respond to
business requirements quickly,
reliably, and predictably
Larger enterprises can benefit
most, but adoption is lagging
behind
@olgch; @kublr

7. “Restrictive” Environment
Due to scale and/or nature of business:
requires governance
limits access and options
@olgch; @kublr

8. Enterprise Requirements
Multiple/complex environments
on-prem, multi-cloud, hybrid
Centralized/unified management and governance
Provisioning, monitoring, log collection, IdM/AAA, Cost
Integration with existing, often legacy, components
Security infrastructure
OS, IdM/AAA
Software management
patches, packages, images
@olgch; @kublr

9. Enterprise Challenges & Constraints
Separation of responsibilities
Infrastructure, Operations, Security, Legal
Network access
white/black-listing, air gap
Security tools and processes
infra, OS, platform, apps
OS, platform, and software practices and standards
Vendor and version certification; configuration practices; custom package
repositories; etc
Regulations
Complexity

11. What is Kubernetes?
• Container orchestrator?
• A step in the evolution from mainframes to serverless?
• Microservices platform?
Infrastructure/cloud abstraction and platform
@olgch; @kublr

12. Automation
Ingress
Custom Cfg
Self-service
Infrastructure
Logging Monitoring
Observability
API
Usage
Reporting
RBAC IAM
Air Gap TLS
Secret
Management
Audit
Storage Networking
Repos &
Registries
CI / CD App Mgmt
Infrastructure
Container Runtime Kubernetes
OPERATIONS SECURITY &
GOVERNANCE
Cloud Native in an Enterprise
@olgch; @kublr
MANAGED
SERVICES
APPLICATION
LIFECYCLE
Scanning
Service MeshDataAPI Mgmt
Services and Applications
Backup & DR
Network
Policies

13. Automation
Ingress
Custom Cfg
Self-service
Infrastructure
Logging Monitoring
Observability
API
Usage
Reporting
RBAC IAM
Air Gap TLS
Secret
Management
Audit
Storage Networking
Repos &
Registries
CI / CD App Mgmt
Infrastructure
Container Runtime Kubernetes
OPERATIONS SECURITY &
GOVERNANCE
K8S as Infrastructure Abstraction
@olgch; @kublr
MANAGED
SERVICES
APPLICATION
LIFECYCLE
Scanning
Service MeshDataAPI Mgmt
Services and Applications
Backup & DR
Network
Policies

14. Kubernetes Management
Platform
@olgch; @kublr

15. Kubernetes Management
K8S Clusters
Cloud(s)
Data
center
API UI
Log collection
Operations
Monitoring
IAM, RBAC, SSO, Federation
DR Binary Repos
Infrastructure management
Image Mgmt
Dev
K8S API
Cloud/Infra API
Prod
PoC
Dev
@olgch; @kublr

16. Identity Broker: AAA, SSO, Federation
K8S Clusters
Cloud(s)
Data
center
API UI
Log collection
Operations
Monitoring
IAM, RBAC, SSO, Federation
DR Binary Repos
Infrastructure management
Image Mgmt
Dev
K8S API
Cloud/Infra API
Prod
PoC
Dev
Enterprise IdM
@olgch; @kublr

17. Kubernetes Services
K8S Clusters
Cloud(s)
Data
center
API UI
Log collection
Operations
Monitoring
IAM, RBAC, SSO, Federation
DR Binary Repos
Infrastructure management
Image Mgmt
Dev
K8S API
Cloud/Infra API
Prod
PoC
Dev
@olgch; @kublr

18. Infrastructure
Automation
Cluster Architecture
Control
Center
MASTER
OPS AGENT
overlay network, discovery,
connectivity
K8s Master Components:
etcd, scheduler, API, controller
Docker
KUBELET OPS AGENTKUBELET
NODE
Docker
overlay network, discovery,
connectivity
Infrastructure and
Application containers
Orchestration
Store
Discovery &
orchestration
Orchestration and
configuration agent
@olgch; @kublr

19. K8S Monitoring with Prometheus
• Discover nodes, services, pods
via K8S API
• Query metrics from discovered
endpoints
• Endpoint are accessed directly
via internal cluster addresses
Kubernetes Cluster
Prometheus
Nodes
K8S API
Grafana
Pods
Discovery
Srv
Metrics
@olgch; @kublr

20. Centralized Monitoring
Cluster registry
PROMETHEUSGrafana
K8S Proxy API
nodes, pods,
service endpoints
Ship externally
Ship externally
Prometheus
config
Prometheus
data
Configurator
Control plane
KUBERNETES CLUSTER
Prometheus
(collector)
Prometheus
(collector)
@olgch; @kublr

21. K8S Logging with Elasticsearch
• Fluentd runs on nodes
• OS, K8S, and container logs
collected and shipped to
Elasticsearch
• Kibana for visualization
Kubernetes Cluster
Elasticsearch
Kibana
Pods
Logs
@olgch; @kublr

22. Prometheus
(collector)
RabbitMQ
Centralized Log Collection
Cluster registry
K8S Proxy API
Port
forwarding
MQTT
Ship externally
Messaging
config
Configurator
Control
Plane
RabbitMQ
Shovel
ElasticsearchLogstash
Fluentd
KUBERNETES CLUSTER
filter
filter analyze
Ship externally
MQTT
Forwarder
filter
@olgch; @kublr

23. Automation
Ingress
Custom Cfg
Self-service
Infrastructure
Logging Monitoring
Observability
API
Usage
Reporting
RBAC IAM
Air Gap TLS
Secret
Management
Audit
Storage Networking
Repos &
Registries
CI / CD App Mgmt
Infrastructure
Container Runtime Kubernetes
OPERATIONS SECURITY &
GOVERNANCE
Enterprise Kubernetes
@olgch; @kublr
MANAGED
SERVICES
APPLICATION
LIFECYCLE
Scanning
Service MeshDataAPI Mgmt
Services
Backup & DR
Network
Policies

24. Q&A
Take Kublr for a test drive!
kublr.com/deploy
@olgch; @kublr

25. Stay in touch! Signup for our
newsletter at kublr.com
Oleg Chunikhin
CTO, Kublr
oleg@kublr.com
@olgch
Kublr | kublr.com
@kublr