SlideShare a Scribd company logo
ING Container Hosting Platform, 3 years onward
ING Tech Infra&Engineering
Growing and sustaining a Private Cloud Container Hosting Platform
October 2022
2
Introductions
ING is a global bank with a strong European base. Our more than
57,000 employees serve around 38 million customers, corporate
clients and financial institutions in over 40 countries. Our
purpose is to empower people to stay a step ahead in life and in
business.
* ING is undergoing a transition to close our Wholesale Banking offices in Argentina, Brazil and Kazakhstan as
announced on 5 November 2020. ING has exited the Austrian and Czech retail banking markets as of the end of
2021. We announced in December 2021 that we will leave the retail banking market in France after a strategic
review of our retail banking operations there. We announced June 24th 2022 we will leave the Philippines retail
banking market as of then end of 2022.
For ING’s position regarding Russia and the Ukraine see our Feb 28 2022 statement
Thijs Ebbers
Architecting Cloud Native @ING
since 2016 (employee since 2001)
Architecture Lead for the
Runtime Domain (“VM & Container
Hosting”), for ING Private & Public
Clouds
Sandeep Kaul
IT Area Lead (“Head of IT”) for
Cloud/PaaS, a.o. responsible for
the delivery of ICHP towards
internal customers.
3
Short Recap
At 2019’s San Diego OpenShift Commons ING presented “Running Containers in Production, the ING Story”
where we introduced ING’s Container Hosting Platform (“ICHP”) and explained:
• the drivers behind our transformation
• the “Cloud Native ecosystem Kube” we use(d) to communicate with our stakeholders
• ICHP’s deployment model & its delivery patterns
• which topics took the most time during the first 2 years of engineering and operation.
So for the remainder of this presentation we assume this content is known ;-)
and if not catch up afterwards :
4
So what happened since ? (aka agenda)
• Covid…
• Expansions
• Divisions
• Security
• LCM
• Exponential growth
• Bugs
• Resilience
• Partnerships
• (Refined) Strategy
5
ICHP –> ICH* (aka abstracting for multi cloud)
ING Container Hosting Platform (“ICHP”) was ING’s first standardized Container Hosting environment, but
it has evolved into a family we now call ICH which consists of:
• The private cloud ICHP platform based on OCP
• The ICHA platform on Azure AKS (still being engineered)
• The ICHG platform on GCP GKE (still being engineered)
All these platforms abide to the same defined interfaces to enable application portability between the
ICH family members, for both re-useability as well as regulatory reasons. And making deployments to
multiple target clouds with defined interfaces mandatory will empower our engineers with the skills they
need.
Why not use (managed) OpenShift on Azure & GCP ?
• ING believes the value of public clouds is in the integrated ecosystem and the cloud vendors native
services will be the 1st to benefit from innovations in those ecosystems. This means we should
consume SaaS rather then PaaS rather then IaaS to make use of those innovations as soon as possible.
• Hence we consume Azure and GCP native services wherever possible and we try to avoid IaaS services
For the remainder of this presentation we’ll focus on ICHP
Note : 2 of our ING colleagues are presenting this KubeCon on “HPC Batch computing for Analytics Workloads “
(Wednesday 2:30 pm), attend and learn but be aware this environment is not (yet) hosted on ICH*
K8s technology = Amazing
• Almost infinite # use cases…
But
• Mixing multiple distinct use cases
together in the same clusters =
a recipe for (regulatory) disaster…
This table shows the attempt of ING to
categorize in order to give each service
its appropriate (risk&security) treatment.
white = (partially) available as a service
grey = currently not available as a service
This table is not intended to be static…
Meaning ING is very opinionated on
what workloads should be hosted on our
Immutable container hosting service…
Don’t make any wrong assumptions,
we do get a lot of requests for hosting
software which in our opinion belongs
on services currently unavailable (grey)
“No” is a valid answer…, as ING Tech chooses to focus on a limited set (since we have scarce engineering capacity). Spinning up those other
services would require additional DevOps squads to engineer and run those services. Requesters can still run that software on VM’s...
Service Typical use case(s)
Immutable
Containers
Capability to host immutable container workloads like 12 Factor Apps (commonly: “API’s”)
Functions as a
Service
Capability to execute backend code directly without having to manage any servers
Data Services
Capability to provide data persistence as a service e.g. Object Storage, Elastic, Kafka, MS-SQL,
Oracle, Cassandra, Redis, etc.
HPC
Capability to provide High Performance Compute (e.g. GPU, fast local cache, etc.) as a service
for machine learning and analytics.
Centralized End-
User
Capability to provide a managed, dedicated set-up for specific user(s) e.g. for data science,
educational and training set-ups etc.
Infrastructure
Management
Capability to manage IT assets as Desired State Custom Resources (e.g. Crossplane)
Non-12 factor
apps (“VM”s) – K8S
Hosting of non-12 factor apps (“VM’s in a container package”), typically large images
containing monolithic applications, not ready for hands off operations, could be requiring
managed local persistency. Hosted on a K8s or equivalent platform.
Standalone - Non-
12 factor apps
(“VM”s)
Hosting of non-12 factor apps (“VM’s in a container package”), typically large images
containing monolithic applications, not ready for hands off operations, could be requiring
managed local persistency. Hosted on Standalone “Docker” instances.
6
Divide & Conquer (aka Distinct Services deserve Distinct Treatment)
7
Security
• ING Container Security Standard in place as of Q2
2019
(“Zero Privilege”, do not trust “Zero Trust”…)
• Comprehensive Vulnerability Scanning in
OnePipeline enforced as of Q4 2021
(thank you Log4J for the priority…)
• Immutable Workloads, enforced as of Q1 2022
(privileged NPA’s removed from production namespaces,
Break Glass available, mandatory redeploy afterwards)
• Immutable Platform (Clusters), as of Q3 2022
(OCP 4.10.30 IPI for Bare Metal dependency)
• Anomaly Detection & Policy-as-Code capability as
of Q3 2022 (OCP 4.10/ K8s kernel version dependency)
• As of Q3 2022 ICHP has implemented all controls
as specified in the Container Security Standard
8
OpenShift LCM
Generic learnings:
• Not all consuming DevOps teams understand the full scope of going Cloud Native and might be unaware of their
shortcomings
• In particular we noticed the value of automated testing wasn't always recognized, which did hamper our Platform
LCM
• Application redeployment should be “the new normal”, not “something to be scared of”…
Migrating OpenShift 3.11 OKD to OpenShift 3.11 Enterprise (Q1 2020):
• Clusters were redeployed (1 DC at a time), forcing downtime (in practice a DR-excersize) upon our consumers. This
wasn’t well received and led us to rethink our migration strategy for the next LCM moment…
Migrating OpenShift 3.11 Enterprise to OpenShift 4.10 Enterprise (“ICHP v2”) (Q3 2022):
• Running on Bare Metal nodes doesn’t help if new OpenShift
versions are first released for VM based clusters… It took Red Hat
until version 4.10.30 to deliver the quality ING needed to do a
fully automated IPI based install in our DC’s (which was kind of
challenging looking at the EOS of OCP 3.11…)
• This time we opted to spin up new clusters in parallel to the old
environment, and offer our customers a migration window to
redeploy their workloads. We opened the migration Sep 15th 2022
for our Non-Prod clusters and our Prod clusters are planned to go live this week
More details on how we build our NaaS on top of OCP 4.10 IPI in the other ING presentation today
In Q3 2019 ICHP had 18 FTE and hosted :
• 250 Prod namespaces whose API’s only
processed a marginal number of transactions
(most still preparing for actual workloads)
• about 1000 Non-Prod Namespaces
In Q3 2022 ICHP hosted:
• 600 namespaces in Production whose
API’s processed 50% of ING retail customer
transactions
• 1690 namespaces in Non-Production
And did this with the same 18 FTE, we only
added an additional Ops squad to manage
the environment (+6 FTE) from a partner
(hired as of Q1 2022, for new total of 24 FTE)
9
Growing pains : exponential growth for Immutable App hosting
So looking back ICHP coped with a sustained 100% growth each year (measured in installed CPU/memory
capacity) (for the last 3 years) & this growth is expected to continue into 2023. As a result the price/unit
has now become very competitive and ICHP’s Immutable Container hosting in this way supports the
broader ING Tech Strategy to enable to grow our business at marginal cost
Green line is actual ICHP Prod DC1
usage (Prod DC2 usage is similar)
Red line is available ICHP Prod DC1
capacity (Prod DC2 capacity is identical)
Drop in capacity end of August ’21
due to bug in OpenShift 3.11 icw
Atomic requiring ICHP to scale
back from 250 pods to 160 pods
per node to become stable again
(with OpenShift 4/CoreOS its fixed)
Customers were impacted (mainly
in the non-Prod environment) in
the Q3 2021 due to this shortage
As a provider we had to scavenge
additional nodes everywhere to
remain stable…
The drop in Feb 2022 was due to an
unintended reboot of multiple nodes by
one of our suppliers
Growing pains : Bugs (pending pods / pods per node)
10
Pending pods ->
Pods/node back to 160
Running out of
capacity…
Blue Line
measuring
“pending pods”
introduced
Q1 2022
Multiple outages impacting ING customers on workloads hosted on ICHP (For the record : ICHP has had zero
unplanned platform downtime…). Analysis concluded most were avoidable… by properly defining K8s
application deployment configs. Lesson learned : Developer Autonomy is good thing, but never forget to
validate the results…
Q2 2022
• ING CTO&CIO’s decided on mandatory (not yet enforced) use of :
canary releasing
resilient pod placement
liveness & readiness probes
• ICHP published ING specific workload resilience documentation
Q3 2022
• ICHP provides Dashboards giving ING CIO’s (& developers) insight in
the quality of the deployments on ICHP
• Additional internal templates & resilience training materials being developed (by partner enabling teams)
Future Step : Policy enforcement via Policy-as-Code (K8s deployment config insufficient -> deploy job fails)
11
Growing pains : workload resilience
12
ICHP Partnerships
Ingested products
(products or data provided by other platforms
that are exposed through this platform)
supplied-services
(provided by other platforms
required for platform functions)
engineered product builds
(developed and maintained by one or multiple
squads)
CONSUME ENGINEER
INGEST
Exposed platform
services
EXPOSE
Exposed product builds
(semi finished product used as supplied
product build by the consuming
platform)
Complement
(add value)
ING “BTP” (Topics & Service Mesh & Workload Deployment Templates)
ING “IPC” (DBaaS, Keyspace-aaS, Object Storage, Cloud Automation)
ING “Security Tribe” (IAM & Workload Security Monitoring & Workload Certs)
ING “MDPL” (Workload Observability)
ING “One Pipeline” (Workload CI/CD)
ING Banking Technology Platform
ING Group Services
ING Retail Banks
ING Wholesale
ING Infra&Engineering
ING “DCN” (Networking)
ING “IPC” (Object Storage)
ING “MDPL” (Platform Observability)
ING “One Pipeline” (Platform CI/CD)
ING “Security Tribe” (IAM & Platform Security Monitoring & Platform Certs)
ING “IPC” (Nodes BM & VM)
Red Hat (OpenShift)
Portworx (local persistency)
13
Strategy (part 1 : why do we do what we do ?)
ING’s Tech strategy : “To build Scaleable Tech which enables Growing our business at marginal cost”
• ICHP is a core example of technologies engineered by ING to support this goal, as explained earlier
Additional Strategy goals are “Self-Service”, “Re-useability” & Automation
• The immutable container hosting is delivered as a Namespace-aaS from our private cloud portal
• The ICHP platform hosts applications from almost every ING segment and for most of the countries
• The ICHP platform is fully automated, both for Namespace & Cluster provisioning (“Immutable”)
ICHP’s Unique Selling Point : Compliance-out-of-the-Box
• Immutability is the key tool in achieving this goal. Hence ICHP focused on Immutable container hosting
over the past period and this will stay it’s core delivery
ING Tech is looking into “Team Topologies” as a way to optimize the internal
organization and as it turns out ICHP ticks all the boxes for a
“Thinnest Viable Platform” (since 2018, the book was published in 2019…)
Our platform is compelling for consumers, with our NaaS approach which reduces
cognitive load, it simplifies the risk evidencing burden & we are strongly partnered
with key “stream aligned”, “enabling” & “platform” teams in different ING segments
to help guide our backlog/roadmap
14
Strategy (part 2 : what’s next?)
For the Immutable container hosting service there’s still work to be done:
• Pets -> Cattle for the clusters, which is aided by
• Hypervisor based Nodes (currently all production is on Bare Metal)
• Multi-Cluster Management (“MCM”) : Pattern(s)/Solution(s) to be investigated
• Anomaly Detection and Policy-as-Code capabilities have been released with ICHPv2 based on
OpenShift 4.10, but scope wise we have lots of opportunities to expand rulesets… Workload resilience
will be high on that backlog…
But ING does recognize other K8s use cases as explained before:
• ICHP is hosting Elastic & Prometheus environments on dedicated
OpenShift clusters (with local persistency)
• ICHP is working on a Cluster-aaS delivery for Platform Teams
• And ICHP is tracking the maturity of the market:
• To determine when best to start investing our scarce engineering resources (Scarcity does create focus...).
• The Multi-Cluster Services (“MCS”) developments have our special attention (perhaps Skupper + KCP…?)
In the mean time non-Immutable workloads can still be hosted on our Private Cloud VM offering…
(no exceptions/no specials in our multi-tenant Immutable Container hosting, or else we risk losing our USP…)
Last but not least we strive to be a more engaged participant in the community, more on that topic in the
other ING presentation today…
Of course we’re hiring ;-)
https://www.ing.jobs/tech
15
Thank you / Questions
Slides will be made available at
https://www.slideshare.net/ThijsEbbers/
Appendices
Additional materials for your understanding
Whilst we treaded the eye of the needle
in our Production clusters when we hit the
“Pending Pods bug”, we weren’t so “lucky”
with our Non-Production clusters.
On several occasions consumer (DTA)
workloads did encounter negative impact,
and since hardware replenishment was
difficult we moved D/T workloads to
newly added VM based nodes (which
explains the decline in the green line) in
order to free up Bare Metal nodes to be
used to stabilize the Production clusters
as well as to keep the Acceptance
workloads as stable as possible on the
remaining Bare Metal nodes
Meanwhile, in Non-Prod…
17
Pods/node
back to 160
Pending
pods
Introduction
of additional
(VM based)
nodes
Non-Prod
Environment
stabilized
• From ING’s Daily Banking Tribe talk at GOTO 2022 conference (“Spotify Model : On Radical
Improvements From The Trenches Of Change”):
• Deployment time : 62% build & deploy speed improvement
• Development capacity : 27% more Backlog items done (measured per year in 2020/21)
• More Coding : 16% more coding time per team
• No more VM’s : Less infra costs, no patching, more secure, less issues with auditing the stack
(More cool ING Tech talks on our YouTube Channel)
• Building a Global Investments Platform in 12 Months (2020-21)
• Building a Digital Platform for Insurance Products (“ING-AXA partnership”) in 9 Months (2019)
• Building a (Mobile Only Digital) Bank in 9 Months (“The Manila Miracle”) (2018)
• And more to come…
ICHP(‘s-ecosystem) enabled our Internal ING Customers to achieve :
18
Platform (“ICHP”) Adds e.g. :
• ING Compliance
• ING Integrations (SEM,
Monitoring, AZDO, …)
• ING Hardening
Distribution (e.g. “OpenShift”, “RKE”, “GKE”, “AKS”)
Adds e.g. : RBAC, SDN
Container Management Framework
(“Kubernetes”)
Adds e.g. :
Scaling, resilience
Application Containers to Container Hosting Platforms
19
Containers
• Important aspect to be taken into consideration: The ING Container Hosting Platform Immutable
Container hosting is NOT aiming to rehost VM’s to containers… Purpose is to have the best possible
hosting environment for immutable workloads!
• If a DevOps teams manages to properly refactor their VM into a set of container images they are
welcome
• Hands-off approach in production (which implies a mature team in all aspects, e.g. CI/CD !)
If teams feel not comfortable with this they should stay on VM’s!
ICHP Immutable App hosting Intended Use
20
ING aims to make our DevOps teams autonomous (As explained in ING’s Way-of-Working in 2019) .
We also want to enable those DevOps teams to deliver maximum value to the business, by not bothering them
with IT-Infrastructure concerns, nor bothering them with having to deliver compliancy evidence for the hosting
platform.
Hence offering a Namespace-as-a-Service is for us the sweet spot:
• Clear demarcation between (Infra)Provider and Consumer
• Enabling hand-over of compliance evidence
• Enabling multi-tenancy (hence fast time-to-market/self-service consumption, and the potential for efficient
utilization of resources)
• The DevOps team can assume responsibility for (almost) the full stack (only the kernel stays shared). They have
liberty/responsibility to choose/maintain their (versions of) base image, runtime engine, libraries, etc. (within
the boundaries set by the ING corporate risk&compliancy rules !)
ING will offer a K8S Cluster-as-a-Service in the future, however this will be a limited offering only available to
selected platform teams.
Why ICHP only offers a “Namespace-as-a-Service”
21
Key Objective for ING is to remain capable to exit a Public Cloud provider.
Determining the right abstractions is key to meeting this objective for a container hosting platform,
hence:
• We choose an open/industry standard (in this case: the Kubernetes ecosystem)
• We choose an intended use (in this case: Immutable Workloads)
• We clearly define the allowed interfaces (in the ICH – Interfaces standard)
• We automate our deployments by the mandatory use of One Pipeline
Note that these abstractions are design time and not runtime (as we still use the AKS service native to
Azure or the GKE service native to GCP)
However there will always be debate on the details as we try to balance innovation, time-to-market,
risk/security and/or cost-income ambitions. Therefore actually running production workloads on at
least one Kubernetes stack outside of the primary (Public) Cloud provider is the best recipe to “keep
everyone honest”, as this will prevent Cloud Provider specific “optimizations” from slipping into ING
code. Meaning there should always be at least 2 ICH standard solutions in use.
Cloud Abstractions
22
We do know how to build those services in grey… Our engineers are more then capable to build and
run those. But not all of them together at the same time…
Hence for each year ING determines where to allocate that engineering capacity:
• Improving existing services
• e.g. Pets -> Cattle
• Supporting additional use cases on existing services
• e.g. Data Services we today only support ELK & Prometheus, if we start to support Data Persistence Services
more critical to the bank (e.g. message bus, (non-)relational databases) we’d actually like the availability of
those Services to be independent from K8s cluster uptime (hence our interest in CNCF MCS developments)
• Introducing New Services
• We do have active instantiations of Centralized End-User & HPC use cases on K8s in ING (one is even presenting
@KubeCon Detroit). But those are currently not hosted as a centralized/standardized Service. So we could also
choose to include those in the ICH catalogue
• The one defined service we’re not likely to invest time in any time soon is FaaS, that has everything to do
with the controls currently enforced from a regulatory perspective and little with the actual technology…
Dive and Conquer additional information
23
IPC/ICHP hosting services (1/2) (Grey is currently not available)
24
Service Typical use case(s)
Immutable Containers Capability to host immutable container workloads like 12 Factor Apps (commonly: “API’s”)
Functions as a Service Capability to execute backend code directly without having to manage any servers
Data Services
Capability to provide data persistence as a service e.g. Object Storage, Elastic, Kafka, MS-SQL, Oracle,
Cassandra, Redis, etc.
HPC
Capability to provide High Performance Compute (e.g. GPU, fast local cache, etc.) as a service for machine
learning and analytics.
Centralized End-User
Capability to provide a managed, dedicated set-up for specific user(s) e.g. for data science, educational
and training set-ups etc.
Infrastructure
Management
Capability to manage IT assets as Desired State Custom Resources (e.g. Crossplane)
Non-12 factor apps
(“VM”s) – K8S
Hosting of non-12 factor apps (“VM’s in a container package”), typically large images containing
monolithic applications, not ready for hands off operations, could be requiring managed local persistency.
Hosted on a K8s or equivalent platform.
Standalone - Non-12
factor apps (“VM”s)
Hosting of non-12 factor apps (“VM’s in a container package”), typically large images containing
monolithic applications, not ready for hands off operations, could be requiring managed local persistency.
Hosted on Standalone “Docker” instances.
IPC/ICHP hosting services (2/2) (Grey is currently not available)
25
Service Unit of Delivery
(/Security)
Local
Persistency
GPU Ingress Exposed
interface(s)
End-user access to
SSH/Terminal Interfaces
Immutable
Containers
Namespace Not Available Not Available
Shared per
Traffic type
(Internal/DMZ/…
)
API (HTTPS ingress),
Eventing (“Kafka”),
File (/Object) (“S3”)
Not Allowed (in Production)
(currently still tolerated in Non-
Production clusters)
Functions as a Service Function
Not Available
TBD Shared
See “Immutable
Containers”
Not Possible
Data Services
(set of)
Namespace(s)
Available Not Available
Dedicated per
workload
TBD Not Allowed
HPC Namespace Optional Optional TBD
See “Immutable
Containers”
Not Allowed
Centralized End-User Namespace TBD TBD TBD SSH/https/… Allowed
Infrastructure
Management
Control plane Not Available Not Available Not Applicable K8s API Not Allowed
Non-12 factor apps
(“VM”s) – K8S
Namespace, VM ? TBD Not Available TBD TBD TBD
Standalone - Non-12
factor apps (“VM”s)
VM Available Not Available Dedicated No restrictions Available
“Immutable” =
• Any change can only be executed via an ING pipeline which has all required checks &
balances in place (and which facilitates automated evidencing for regulatory purposes)
• In short : no direct access for natural persons, no change privileges for other systems
And yes, ING’s Container Security Standard was in place before “Solarwinds” reached the
headlines in 2020…
And although most people in the IT(-Security) industry are now aware of the software
lineage issue exposed by it (that doesn’t mean its anywhere near solved…), far too few
understand it’s 2nd lesson about the dangers of overprivileged software… (what enabled
those attackers to move laterally so easily…?)
For these 2 reason ING does not trust “Zero Trust” labelled implementations as very few of
those have actually covered these concerns properly...
Security additional explanation
26
Range Of Container Technology Usage Models
27
28
Container Technology ING Security Reference Architecture
Image Storage and Retrieval
Container Deployment,
Management, and Operations.
Image Creation, Testing, and
Accreditation
DevOps
DevOps
DevOps
Host with
Containers
Host with
Containers
Host with
Containers
Host with
Containers
Build, Deploy,
Test, Release
Development
Registry
Any Other
Registry
Authorized
Registry
Deployment
Registry
Orchestrator
Container
Management
Framework
No Connection
No Connection
W.01 W.02 W.03
W.04 W.05
I.06 I.07 I.08 R.09 R.10 O.11
P.14 P.17 P.20 H.21
P.15 C.12
P.18 H.22
P.16 C.13
P.19 H.23
29
Container Technology Security Measures Overview
Design & Code
(digitize)
Build & Test
(create)
Operate & Maintain
(detect & correct)
Deploy & Test
(assembly)
Release
(orchestrate)
Onboard and Plan
(define)
Engineering Journey
R.09
R.10
Registry
Measures
O.11
Orchestrator
Measures
I.06
I.07
I.08
Image
Measures
Measures to be implemented by
providers of CI/CD Pipeline for container
technology e.g. GEP
P.14
P.15
P.16
P.17
P.18
P.19
P.20
Platform
Measures
H.21
H.22
H.23
Host
Measures
C.12
C.13
Container
Measures
Measures to be implemented by
providers of container hosting platform
(except H.23) e.g. ICHP
W.01
W.02
W.03
W.04
W.05
Workload
Measures
Measures to be implemented by
providers of container workloads e.g.
DevOps
• Want to understand how to :
• Use Canary (Blue/Green) releasing using K8s cluster external loadbalancing services
• Use Canary (Blue/Green) releasing using K8s cluster internal capabilities
• Revert my traffic 100% to my stable version in case of unexpected issues
• Rollback my release to the N-1 version in case of unexpected issues with version N
• Perform Resilient Placement so a single Red or Blue outage in the underlying IaaS
layers will not affect my application availability
• Implement Liveness Probes so my pods will restart automatically in case of applicative
issues
• Implement Readiness Probes so I have the input to automatically direct my traffic in
case of applicative issues
• And would like to see examples & templates how to use above features in ING’s One
Pipeline
Workload Resilience : As an ICHP Consuming DevOps Team, I
30
The following slides come from ING Investor Update June 13th 2022 Tech& Operations
33
Strategy additional explanation
34
35
Construct your own Kube
36
Follow us
SlideShare.net/ING
@ING_News LinkedIn.com/company/ING
YouTube.com/ING Flickr.com/INGGroup
Facebook.com/ING
ing.com
ingwb.com
Medium.com/ing-blog
ING Container Hosting Platform - 3 years onward_with Kube_for distribution.pdf
ING Container Hosting Platform - 3 years onward_with Kube_for distribution.pdf

More Related Content

What's hot

Streaming all over the world Real life use cases with Kafka Streams
Streaming all over the world  Real life use cases with Kafka StreamsStreaming all over the world  Real life use cases with Kafka Streams
Streaming all over the world Real life use cases with Kafka Streams
confluent
 
Netflix Data Pipeline With Kafka
Netflix Data Pipeline With KafkaNetflix Data Pipeline With Kafka
Netflix Data Pipeline With Kafka
Allen (Xiaozhong) Wang
 
Kubernetes Deployment Strategies
Kubernetes Deployment StrategiesKubernetes Deployment Strategies
Kubernetes Deployment Strategies
Abdennour TM
 
Getting Started with Kubernetes
Getting Started with Kubernetes Getting Started with Kubernetes
Getting Started with Kubernetes
VMware Tanzu
 
Containers Anywhere with OpenShift by Red Hat
Containers Anywhere with OpenShift by Red HatContainers Anywhere with OpenShift by Red Hat
Containers Anywhere with OpenShift by Red Hat
Amazon Web Services
 
MicroK8s
MicroK8sMicroK8s
Apache Iceberg - A Table Format for Hige Analytic Datasets
Apache Iceberg - A Table Format for Hige Analytic DatasetsApache Iceberg - A Table Format for Hige Analytic Datasets
Apache Iceberg - A Table Format for Hige Analytic Datasets
Alluxio, Inc.
 
Confluent REST Proxy and Schema Registry (Concepts, Architecture, Features)
Confluent REST Proxy and Schema Registry (Concepts, Architecture, Features)Confluent REST Proxy and Schema Registry (Concepts, Architecture, Features)
Confluent REST Proxy and Schema Registry (Concepts, Architecture, Features)
Kai Wähner
 
Cloud Native Bern 05.2023 — Zero Trust Visibility
Cloud Native Bern 05.2023 — Zero Trust VisibilityCloud Native Bern 05.2023 — Zero Trust Visibility
Cloud Native Bern 05.2023 — Zero Trust Visibility
Raphaël PINSON
 
Gitops: the kubernetes way
Gitops: the kubernetes wayGitops: the kubernetes way
Gitops: the kubernetes way
sparkfabrik
 
Migrating Hundreds of Legacy Applications to Kubernetes - The Good, the Bad, ...
Migrating Hundreds of Legacy Applications to Kubernetes - The Good, the Bad, ...Migrating Hundreds of Legacy Applications to Kubernetes - The Good, the Bad, ...
Migrating Hundreds of Legacy Applications to Kubernetes - The Good, the Bad, ...
QAware GmbH
 
Apache Kafka 0.8 basic training - Verisign
Apache Kafka 0.8 basic training - VerisignApache Kafka 0.8 basic training - Verisign
Apache Kafka 0.8 basic training - Verisign
Michael Noll
 
CI-CD Jenkins, GitHub Actions, Tekton
CI-CD Jenkins, GitHub Actions, Tekton CI-CD Jenkins, GitHub Actions, Tekton
CI-CD Jenkins, GitHub Actions, Tekton
Araf Karsh Hamid
 
Azure kubernetes service
Azure kubernetes serviceAzure kubernetes service
Azure kubernetes service
Vishwas N
 
Spark on Kubernetes
Spark on KubernetesSpark on Kubernetes
Spark on Kubernetes
datamantra
 
Why to Cloud Native
Why to Cloud NativeWhy to Cloud Native
Why to Cloud Native
Karthik Gaekwad
 
Microservices Docker Kubernetes Istio Kanban DevOps SRE
Microservices Docker Kubernetes Istio Kanban DevOps SREMicroservices Docker Kubernetes Istio Kanban DevOps SRE
Microservices Docker Kubernetes Istio Kanban DevOps SRE
Araf Karsh Hamid
 
Azure Synapse Analytics
Azure Synapse AnalyticsAzure Synapse Analytics
Azure Synapse Analytics
WinWire Technologies Inc
 
AKS
AKSAKS
Cloud Native Landscape (CNCF and OCI)
Cloud Native Landscape (CNCF and OCI)Cloud Native Landscape (CNCF and OCI)
Cloud Native Landscape (CNCF and OCI)
Chris Aniszczyk
 

What's hot (20)

Streaming all over the world Real life use cases with Kafka Streams
Streaming all over the world  Real life use cases with Kafka StreamsStreaming all over the world  Real life use cases with Kafka Streams
Streaming all over the world Real life use cases with Kafka Streams
 
Netflix Data Pipeline With Kafka
Netflix Data Pipeline With KafkaNetflix Data Pipeline With Kafka
Netflix Data Pipeline With Kafka
 
Kubernetes Deployment Strategies
Kubernetes Deployment StrategiesKubernetes Deployment Strategies
Kubernetes Deployment Strategies
 
Getting Started with Kubernetes
Getting Started with Kubernetes Getting Started with Kubernetes
Getting Started with Kubernetes
 
Containers Anywhere with OpenShift by Red Hat
Containers Anywhere with OpenShift by Red HatContainers Anywhere with OpenShift by Red Hat
Containers Anywhere with OpenShift by Red Hat
 
MicroK8s
MicroK8sMicroK8s
MicroK8s
 
Apache Iceberg - A Table Format for Hige Analytic Datasets
Apache Iceberg - A Table Format for Hige Analytic DatasetsApache Iceberg - A Table Format for Hige Analytic Datasets
Apache Iceberg - A Table Format for Hige Analytic Datasets
 
Confluent REST Proxy and Schema Registry (Concepts, Architecture, Features)
Confluent REST Proxy and Schema Registry (Concepts, Architecture, Features)Confluent REST Proxy and Schema Registry (Concepts, Architecture, Features)
Confluent REST Proxy and Schema Registry (Concepts, Architecture, Features)
 
Cloud Native Bern 05.2023 — Zero Trust Visibility
Cloud Native Bern 05.2023 — Zero Trust VisibilityCloud Native Bern 05.2023 — Zero Trust Visibility
Cloud Native Bern 05.2023 — Zero Trust Visibility
 
Gitops: the kubernetes way
Gitops: the kubernetes wayGitops: the kubernetes way
Gitops: the kubernetes way
 
Migrating Hundreds of Legacy Applications to Kubernetes - The Good, the Bad, ...
Migrating Hundreds of Legacy Applications to Kubernetes - The Good, the Bad, ...Migrating Hundreds of Legacy Applications to Kubernetes - The Good, the Bad, ...
Migrating Hundreds of Legacy Applications to Kubernetes - The Good, the Bad, ...
 
Apache Kafka 0.8 basic training - Verisign
Apache Kafka 0.8 basic training - VerisignApache Kafka 0.8 basic training - Verisign
Apache Kafka 0.8 basic training - Verisign
 
CI-CD Jenkins, GitHub Actions, Tekton
CI-CD Jenkins, GitHub Actions, Tekton CI-CD Jenkins, GitHub Actions, Tekton
CI-CD Jenkins, GitHub Actions, Tekton
 
Azure kubernetes service
Azure kubernetes serviceAzure kubernetes service
Azure kubernetes service
 
Spark on Kubernetes
Spark on KubernetesSpark on Kubernetes
Spark on Kubernetes
 
Why to Cloud Native
Why to Cloud NativeWhy to Cloud Native
Why to Cloud Native
 
Microservices Docker Kubernetes Istio Kanban DevOps SRE
Microservices Docker Kubernetes Istio Kanban DevOps SREMicroservices Docker Kubernetes Istio Kanban DevOps SRE
Microservices Docker Kubernetes Istio Kanban DevOps SRE
 
Azure Synapse Analytics
Azure Synapse AnalyticsAzure Synapse Analytics
Azure Synapse Analytics
 
AKS
AKSAKS
AKS
 
Cloud Native Landscape (CNCF and OCI)
Cloud Native Landscape (CNCF and OCI)Cloud Native Landscape (CNCF and OCI)
Cloud Native Landscape (CNCF and OCI)
 

Similar to ING Container Hosting Platform - 3 years onward_with Kube_for distribution.pdf

Pivotal Container Service Overview
Pivotal Container Service Overview Pivotal Container Service Overview
Pivotal Container Service Overview
VMware Tanzu
 
Making Cloud Native CI_CD Services.pdf
Making Cloud Native CI_CD Services.pdfMaking Cloud Native CI_CD Services.pdf
Making Cloud Native CI_CD Services.pdf
Rakuten Group, Inc.
 
[API World 2021 ] - Understanding Cloud Native Deployment
[API World 2021 ] - Understanding Cloud Native Deployment[API World 2021 ] - Understanding Cloud Native Deployment
[API World 2021 ] - Understanding Cloud Native Deployment
WSO2
 
ANIRBAN_GHOSH_RESUME
ANIRBAN_GHOSH_RESUMEANIRBAN_GHOSH_RESUME
ANIRBAN_GHOSH_RESUME
Anirban Ghosh
 
Ultimate Guide to Microservice Architecture on Kubernetes
Ultimate Guide to Microservice Architecture on KubernetesUltimate Guide to Microservice Architecture on Kubernetes
Ultimate Guide to Microservice Architecture on Kubernetes
kloia
 
Building Cloud Native Applications with Oracle Autonomous Database.
Building Cloud Native Applications with Oracle Autonomous Database.Building Cloud Native Applications with Oracle Autonomous Database.
Building Cloud Native Applications with Oracle Autonomous Database.
Oracle Developers
 
Infrastructure design for Kubernetes
Infrastructure design for KubernetesInfrastructure design for Kubernetes
Infrastructure design for Kubernetes
Guillaume Morini
 
Continuous Everything in a Multi-cloud and Multi-platform Environment
Continuous Everything in a Multi-cloud and Multi-platform EnvironmentContinuous Everything in a Multi-cloud and Multi-platform Environment
Continuous Everything in a Multi-cloud and Multi-platform Environment
VMware Tanzu
 
Tim Hall [InfluxData] | InfluxDB Roadmap | InfluxDays Virtual Experience NA 2020
Tim Hall [InfluxData] | InfluxDB Roadmap | InfluxDays Virtual Experience NA 2020Tim Hall [InfluxData] | InfluxDB Roadmap | InfluxDays Virtual Experience NA 2020
Tim Hall [InfluxData] | InfluxDB Roadmap | InfluxDays Virtual Experience NA 2020
InfluxData
 
Migrating a Large Fortune 100 Healthcare Company to Kubernetes in 7 months
Migrating a Large Fortune 100 Healthcare Company to Kubernetes in 7 monthsMigrating a Large Fortune 100 Healthcare Company to Kubernetes in 7 months
Migrating a Large Fortune 100 Healthcare Company to Kubernetes in 7 months
Konveyor Community
 
Transforming mission-critical applications on mainframes for innovation
Transforming mission-critical applications on mainframes for innovationTransforming mission-critical applications on mainframes for innovation
Transforming mission-critical applications on mainframes for innovation
Eranea
 
Connectivity is here (5 g, swarm,...). now, let's build interplanetary apps! (1)
Connectivity is here (5 g, swarm,...). now, let's build interplanetary apps! (1)Connectivity is here (5 g, swarm,...). now, let's build interplanetary apps! (1)
Connectivity is here (5 g, swarm,...). now, let's build interplanetary apps! (1)
Samy Fodil
 
GCP Meetup #3 - Approaches to Cloud Native Architectures
GCP Meetup #3 - Approaches to Cloud Native ArchitecturesGCP Meetup #3 - Approaches to Cloud Native Architectures
GCP Meetup #3 - Approaches to Cloud Native Architectures
nine
 
Bandwidth: Use Cases for Elastic Cloud on Kubernetes
Bandwidth: Use Cases for Elastic Cloud on Kubernetes Bandwidth: Use Cases for Elastic Cloud on Kubernetes
Bandwidth: Use Cases for Elastic Cloud on Kubernetes
Elasticsearch
 
Agile integration: Decomposing the monolith
Agile integration: Decomposing the monolithAgile integration: Decomposing the monolith
Agile integration: Decomposing the monolith
Judy Breedlove
 
Javantura v4 - Self-service app deployment with Kubernetes and OpenShift - Ma...
Javantura v4 - Self-service app deployment with Kubernetes and OpenShift - Ma...Javantura v4 - Self-service app deployment with Kubernetes and OpenShift - Ma...
Javantura v4 - Self-service app deployment with Kubernetes and OpenShift - Ma...
HUJAK - Hrvatska udruga Java korisnika / Croatian Java User Association
 
Technology insights: Decision Science Platform
Technology insights: Decision Science PlatformTechnology insights: Decision Science Platform
Technology insights: Decision Science Platform
Decision Science Community
 
OpenStack and Kubernetes - A match made for Telco Heaven
OpenStack and Kubernetes - A match made for Telco HeavenOpenStack and Kubernetes - A match made for Telco Heaven
OpenStack and Kubernetes - A match made for Telco Heaven
Trinath Somanchi
 
Preparing for Neo - Singapore OutSystems User Group October 2022 Meetup
Preparing for Neo - Singapore OutSystems User Group October 2022 MeetupPreparing for Neo - Singapore OutSystems User Group October 2022 Meetup
Preparing for Neo - Singapore OutSystems User Group October 2022 Meetup
YashrajNayak4
 
Pivotal Container Service (PKS) at SF Cloud Foundry Meetup
Pivotal Container Service (PKS) at SF Cloud Foundry MeetupPivotal Container Service (PKS) at SF Cloud Foundry Meetup
Pivotal Container Service (PKS) at SF Cloud Foundry Meetup
cornelia davis
 

Similar to ING Container Hosting Platform - 3 years onward_with Kube_for distribution.pdf (20)

Pivotal Container Service Overview
Pivotal Container Service Overview Pivotal Container Service Overview
Pivotal Container Service Overview
 
Making Cloud Native CI_CD Services.pdf
Making Cloud Native CI_CD Services.pdfMaking Cloud Native CI_CD Services.pdf
Making Cloud Native CI_CD Services.pdf
 
[API World 2021 ] - Understanding Cloud Native Deployment
[API World 2021 ] - Understanding Cloud Native Deployment[API World 2021 ] - Understanding Cloud Native Deployment
[API World 2021 ] - Understanding Cloud Native Deployment
 
ANIRBAN_GHOSH_RESUME
ANIRBAN_GHOSH_RESUMEANIRBAN_GHOSH_RESUME
ANIRBAN_GHOSH_RESUME
 
Ultimate Guide to Microservice Architecture on Kubernetes
Ultimate Guide to Microservice Architecture on KubernetesUltimate Guide to Microservice Architecture on Kubernetes
Ultimate Guide to Microservice Architecture on Kubernetes
 
Building Cloud Native Applications with Oracle Autonomous Database.
Building Cloud Native Applications with Oracle Autonomous Database.Building Cloud Native Applications with Oracle Autonomous Database.
Building Cloud Native Applications with Oracle Autonomous Database.
 
Infrastructure design for Kubernetes
Infrastructure design for KubernetesInfrastructure design for Kubernetes
Infrastructure design for Kubernetes
 
Continuous Everything in a Multi-cloud and Multi-platform Environment
Continuous Everything in a Multi-cloud and Multi-platform EnvironmentContinuous Everything in a Multi-cloud and Multi-platform Environment
Continuous Everything in a Multi-cloud and Multi-platform Environment
 
Tim Hall [InfluxData] | InfluxDB Roadmap | InfluxDays Virtual Experience NA 2020
Tim Hall [InfluxData] | InfluxDB Roadmap | InfluxDays Virtual Experience NA 2020Tim Hall [InfluxData] | InfluxDB Roadmap | InfluxDays Virtual Experience NA 2020
Tim Hall [InfluxData] | InfluxDB Roadmap | InfluxDays Virtual Experience NA 2020
 
Migrating a Large Fortune 100 Healthcare Company to Kubernetes in 7 months
Migrating a Large Fortune 100 Healthcare Company to Kubernetes in 7 monthsMigrating a Large Fortune 100 Healthcare Company to Kubernetes in 7 months
Migrating a Large Fortune 100 Healthcare Company to Kubernetes in 7 months
 
Transforming mission-critical applications on mainframes for innovation
Transforming mission-critical applications on mainframes for innovationTransforming mission-critical applications on mainframes for innovation
Transforming mission-critical applications on mainframes for innovation
 
Connectivity is here (5 g, swarm,...). now, let's build interplanetary apps! (1)
Connectivity is here (5 g, swarm,...). now, let's build interplanetary apps! (1)Connectivity is here (5 g, swarm,...). now, let's build interplanetary apps! (1)
Connectivity is here (5 g, swarm,...). now, let's build interplanetary apps! (1)
 
GCP Meetup #3 - Approaches to Cloud Native Architectures
GCP Meetup #3 - Approaches to Cloud Native ArchitecturesGCP Meetup #3 - Approaches to Cloud Native Architectures
GCP Meetup #3 - Approaches to Cloud Native Architectures
 
Bandwidth: Use Cases for Elastic Cloud on Kubernetes
Bandwidth: Use Cases for Elastic Cloud on Kubernetes Bandwidth: Use Cases for Elastic Cloud on Kubernetes
Bandwidth: Use Cases for Elastic Cloud on Kubernetes
 
Agile integration: Decomposing the monolith
Agile integration: Decomposing the monolithAgile integration: Decomposing the monolith
Agile integration: Decomposing the monolith
 
Javantura v4 - Self-service app deployment with Kubernetes and OpenShift - Ma...
Javantura v4 - Self-service app deployment with Kubernetes and OpenShift - Ma...Javantura v4 - Self-service app deployment with Kubernetes and OpenShift - Ma...
Javantura v4 - Self-service app deployment with Kubernetes and OpenShift - Ma...
 
Technology insights: Decision Science Platform
Technology insights: Decision Science PlatformTechnology insights: Decision Science Platform
Technology insights: Decision Science Platform
 
OpenStack and Kubernetes - A match made for Telco Heaven
OpenStack and Kubernetes - A match made for Telco HeavenOpenStack and Kubernetes - A match made for Telco Heaven
OpenStack and Kubernetes - A match made for Telco Heaven
 
Preparing for Neo - Singapore OutSystems User Group October 2022 Meetup
Preparing for Neo - Singapore OutSystems User Group October 2022 MeetupPreparing for Neo - Singapore OutSystems User Group October 2022 Meetup
Preparing for Neo - Singapore OutSystems User Group October 2022 Meetup
 
Pivotal Container Service (PKS) at SF Cloud Foundry Meetup
Pivotal Container Service (PKS) at SF Cloud Foundry MeetupPivotal Container Service (PKS) at SF Cloud Foundry Meetup
Pivotal Container Service (PKS) at SF Cloud Foundry Meetup
 

Recently uploaded

Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Brandon Minnick, MBA
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
Edge AI and Vision Alliance
 
dbms calicut university B. sc Cs 4th sem.pdf
dbms  calicut university B. sc Cs 4th sem.pdfdbms  calicut university B. sc Cs 4th sem.pdf
dbms calicut university B. sc Cs 4th sem.pdf
Shinana2
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Zilliz
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
Chart Kalyan
 
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
alexjohnson7307
 
SAP S/4 HANA sourcing and procurement to Public cloud
SAP S/4 HANA sourcing and procurement to Public cloudSAP S/4 HANA sourcing and procurement to Public cloud
SAP S/4 HANA sourcing and procurement to Public cloud
maazsz111
 
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - HiikeSystem Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
Hiike
 
AWS Cloud Cost Optimization Presentation.pptx
AWS Cloud Cost Optimization Presentation.pptxAWS Cloud Cost Optimization Presentation.pptx
AWS Cloud Cost Optimization Presentation.pptx
HarisZaheer8
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
akankshawande
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
Miro Wengner
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
saastr
 
Public CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptxPublic CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptx
marufrahmanstratejm
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
Zilliz
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
Ivanti
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 

Recently uploaded (20)

Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
 
dbms calicut university B. sc Cs 4th sem.pdf
dbms  calicut university B. sc Cs 4th sem.pdfdbms  calicut university B. sc Cs 4th sem.pdf
dbms calicut university B. sc Cs 4th sem.pdf
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
 
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
 
SAP S/4 HANA sourcing and procurement to Public cloud
SAP S/4 HANA sourcing and procurement to Public cloudSAP S/4 HANA sourcing and procurement to Public cloud
SAP S/4 HANA sourcing and procurement to Public cloud
 
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - HiikeSystem Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
 
AWS Cloud Cost Optimization Presentation.pptx
AWS Cloud Cost Optimization Presentation.pptxAWS Cloud Cost Optimization Presentation.pptx
AWS Cloud Cost Optimization Presentation.pptx
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
 
Public CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptxPublic CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptx
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 

ING Container Hosting Platform - 3 years onward_with Kube_for distribution.pdf

  • 1. ING Container Hosting Platform, 3 years onward ING Tech Infra&Engineering Growing and sustaining a Private Cloud Container Hosting Platform October 2022
  • 2. 2 Introductions ING is a global bank with a strong European base. Our more than 57,000 employees serve around 38 million customers, corporate clients and financial institutions in over 40 countries. Our purpose is to empower people to stay a step ahead in life and in business. * ING is undergoing a transition to close our Wholesale Banking offices in Argentina, Brazil and Kazakhstan as announced on 5 November 2020. ING has exited the Austrian and Czech retail banking markets as of the end of 2021. We announced in December 2021 that we will leave the retail banking market in France after a strategic review of our retail banking operations there. We announced June 24th 2022 we will leave the Philippines retail banking market as of then end of 2022. For ING’s position regarding Russia and the Ukraine see our Feb 28 2022 statement Thijs Ebbers Architecting Cloud Native @ING since 2016 (employee since 2001) Architecture Lead for the Runtime Domain (“VM & Container Hosting”), for ING Private & Public Clouds Sandeep Kaul IT Area Lead (“Head of IT”) for Cloud/PaaS, a.o. responsible for the delivery of ICHP towards internal customers.
  • 3. 3 Short Recap At 2019’s San Diego OpenShift Commons ING presented “Running Containers in Production, the ING Story” where we introduced ING’s Container Hosting Platform (“ICHP”) and explained: • the drivers behind our transformation • the “Cloud Native ecosystem Kube” we use(d) to communicate with our stakeholders • ICHP’s deployment model & its delivery patterns • which topics took the most time during the first 2 years of engineering and operation. So for the remainder of this presentation we assume this content is known ;-) and if not catch up afterwards :
  • 4. 4 So what happened since ? (aka agenda) • Covid… • Expansions • Divisions • Security • LCM • Exponential growth • Bugs • Resilience • Partnerships • (Refined) Strategy
  • 5. 5 ICHP –> ICH* (aka abstracting for multi cloud) ING Container Hosting Platform (“ICHP”) was ING’s first standardized Container Hosting environment, but it has evolved into a family we now call ICH which consists of: • The private cloud ICHP platform based on OCP • The ICHA platform on Azure AKS (still being engineered) • The ICHG platform on GCP GKE (still being engineered) All these platforms abide to the same defined interfaces to enable application portability between the ICH family members, for both re-useability as well as regulatory reasons. And making deployments to multiple target clouds with defined interfaces mandatory will empower our engineers with the skills they need. Why not use (managed) OpenShift on Azure & GCP ? • ING believes the value of public clouds is in the integrated ecosystem and the cloud vendors native services will be the 1st to benefit from innovations in those ecosystems. This means we should consume SaaS rather then PaaS rather then IaaS to make use of those innovations as soon as possible. • Hence we consume Azure and GCP native services wherever possible and we try to avoid IaaS services For the remainder of this presentation we’ll focus on ICHP Note : 2 of our ING colleagues are presenting this KubeCon on “HPC Batch computing for Analytics Workloads “ (Wednesday 2:30 pm), attend and learn but be aware this environment is not (yet) hosted on ICH*
  • 6. K8s technology = Amazing • Almost infinite # use cases… But • Mixing multiple distinct use cases together in the same clusters = a recipe for (regulatory) disaster… This table shows the attempt of ING to categorize in order to give each service its appropriate (risk&security) treatment. white = (partially) available as a service grey = currently not available as a service This table is not intended to be static… Meaning ING is very opinionated on what workloads should be hosted on our Immutable container hosting service… Don’t make any wrong assumptions, we do get a lot of requests for hosting software which in our opinion belongs on services currently unavailable (grey) “No” is a valid answer…, as ING Tech chooses to focus on a limited set (since we have scarce engineering capacity). Spinning up those other services would require additional DevOps squads to engineer and run those services. Requesters can still run that software on VM’s... Service Typical use case(s) Immutable Containers Capability to host immutable container workloads like 12 Factor Apps (commonly: “API’s”) Functions as a Service Capability to execute backend code directly without having to manage any servers Data Services Capability to provide data persistence as a service e.g. Object Storage, Elastic, Kafka, MS-SQL, Oracle, Cassandra, Redis, etc. HPC Capability to provide High Performance Compute (e.g. GPU, fast local cache, etc.) as a service for machine learning and analytics. Centralized End- User Capability to provide a managed, dedicated set-up for specific user(s) e.g. for data science, educational and training set-ups etc. Infrastructure Management Capability to manage IT assets as Desired State Custom Resources (e.g. Crossplane) Non-12 factor apps (“VM”s) – K8S Hosting of non-12 factor apps (“VM’s in a container package”), typically large images containing monolithic applications, not ready for hands off operations, could be requiring managed local persistency. Hosted on a K8s or equivalent platform. Standalone - Non- 12 factor apps (“VM”s) Hosting of non-12 factor apps (“VM’s in a container package”), typically large images containing monolithic applications, not ready for hands off operations, could be requiring managed local persistency. Hosted on Standalone “Docker” instances. 6 Divide & Conquer (aka Distinct Services deserve Distinct Treatment)
  • 7. 7 Security • ING Container Security Standard in place as of Q2 2019 (“Zero Privilege”, do not trust “Zero Trust”…) • Comprehensive Vulnerability Scanning in OnePipeline enforced as of Q4 2021 (thank you Log4J for the priority…) • Immutable Workloads, enforced as of Q1 2022 (privileged NPA’s removed from production namespaces, Break Glass available, mandatory redeploy afterwards) • Immutable Platform (Clusters), as of Q3 2022 (OCP 4.10.30 IPI for Bare Metal dependency) • Anomaly Detection & Policy-as-Code capability as of Q3 2022 (OCP 4.10/ K8s kernel version dependency) • As of Q3 2022 ICHP has implemented all controls as specified in the Container Security Standard
  • 8. 8 OpenShift LCM Generic learnings: • Not all consuming DevOps teams understand the full scope of going Cloud Native and might be unaware of their shortcomings • In particular we noticed the value of automated testing wasn't always recognized, which did hamper our Platform LCM • Application redeployment should be “the new normal”, not “something to be scared of”… Migrating OpenShift 3.11 OKD to OpenShift 3.11 Enterprise (Q1 2020): • Clusters were redeployed (1 DC at a time), forcing downtime (in practice a DR-excersize) upon our consumers. This wasn’t well received and led us to rethink our migration strategy for the next LCM moment… Migrating OpenShift 3.11 Enterprise to OpenShift 4.10 Enterprise (“ICHP v2”) (Q3 2022): • Running on Bare Metal nodes doesn’t help if new OpenShift versions are first released for VM based clusters… It took Red Hat until version 4.10.30 to deliver the quality ING needed to do a fully automated IPI based install in our DC’s (which was kind of challenging looking at the EOS of OCP 3.11…) • This time we opted to spin up new clusters in parallel to the old environment, and offer our customers a migration window to redeploy their workloads. We opened the migration Sep 15th 2022 for our Non-Prod clusters and our Prod clusters are planned to go live this week More details on how we build our NaaS on top of OCP 4.10 IPI in the other ING presentation today
  • 9. In Q3 2019 ICHP had 18 FTE and hosted : • 250 Prod namespaces whose API’s only processed a marginal number of transactions (most still preparing for actual workloads) • about 1000 Non-Prod Namespaces In Q3 2022 ICHP hosted: • 600 namespaces in Production whose API’s processed 50% of ING retail customer transactions • 1690 namespaces in Non-Production And did this with the same 18 FTE, we only added an additional Ops squad to manage the environment (+6 FTE) from a partner (hired as of Q1 2022, for new total of 24 FTE) 9 Growing pains : exponential growth for Immutable App hosting So looking back ICHP coped with a sustained 100% growth each year (measured in installed CPU/memory capacity) (for the last 3 years) & this growth is expected to continue into 2023. As a result the price/unit has now become very competitive and ICHP’s Immutable Container hosting in this way supports the broader ING Tech Strategy to enable to grow our business at marginal cost
  • 10. Green line is actual ICHP Prod DC1 usage (Prod DC2 usage is similar) Red line is available ICHP Prod DC1 capacity (Prod DC2 capacity is identical) Drop in capacity end of August ’21 due to bug in OpenShift 3.11 icw Atomic requiring ICHP to scale back from 250 pods to 160 pods per node to become stable again (with OpenShift 4/CoreOS its fixed) Customers were impacted (mainly in the non-Prod environment) in the Q3 2021 due to this shortage As a provider we had to scavenge additional nodes everywhere to remain stable… The drop in Feb 2022 was due to an unintended reboot of multiple nodes by one of our suppliers Growing pains : Bugs (pending pods / pods per node) 10 Pending pods -> Pods/node back to 160 Running out of capacity… Blue Line measuring “pending pods” introduced
  • 11. Q1 2022 Multiple outages impacting ING customers on workloads hosted on ICHP (For the record : ICHP has had zero unplanned platform downtime…). Analysis concluded most were avoidable… by properly defining K8s application deployment configs. Lesson learned : Developer Autonomy is good thing, but never forget to validate the results… Q2 2022 • ING CTO&CIO’s decided on mandatory (not yet enforced) use of : canary releasing resilient pod placement liveness & readiness probes • ICHP published ING specific workload resilience documentation Q3 2022 • ICHP provides Dashboards giving ING CIO’s (& developers) insight in the quality of the deployments on ICHP • Additional internal templates & resilience training materials being developed (by partner enabling teams) Future Step : Policy enforcement via Policy-as-Code (K8s deployment config insufficient -> deploy job fails) 11 Growing pains : workload resilience
  • 12. 12 ICHP Partnerships Ingested products (products or data provided by other platforms that are exposed through this platform) supplied-services (provided by other platforms required for platform functions) engineered product builds (developed and maintained by one or multiple squads) CONSUME ENGINEER INGEST Exposed platform services EXPOSE Exposed product builds (semi finished product used as supplied product build by the consuming platform) Complement (add value) ING “BTP” (Topics & Service Mesh & Workload Deployment Templates) ING “IPC” (DBaaS, Keyspace-aaS, Object Storage, Cloud Automation) ING “Security Tribe” (IAM & Workload Security Monitoring & Workload Certs) ING “MDPL” (Workload Observability) ING “One Pipeline” (Workload CI/CD) ING Banking Technology Platform ING Group Services ING Retail Banks ING Wholesale ING Infra&Engineering ING “DCN” (Networking) ING “IPC” (Object Storage) ING “MDPL” (Platform Observability) ING “One Pipeline” (Platform CI/CD) ING “Security Tribe” (IAM & Platform Security Monitoring & Platform Certs) ING “IPC” (Nodes BM & VM) Red Hat (OpenShift) Portworx (local persistency)
  • 13. 13 Strategy (part 1 : why do we do what we do ?) ING’s Tech strategy : “To build Scaleable Tech which enables Growing our business at marginal cost” • ICHP is a core example of technologies engineered by ING to support this goal, as explained earlier Additional Strategy goals are “Self-Service”, “Re-useability” & Automation • The immutable container hosting is delivered as a Namespace-aaS from our private cloud portal • The ICHP platform hosts applications from almost every ING segment and for most of the countries • The ICHP platform is fully automated, both for Namespace & Cluster provisioning (“Immutable”) ICHP’s Unique Selling Point : Compliance-out-of-the-Box • Immutability is the key tool in achieving this goal. Hence ICHP focused on Immutable container hosting over the past period and this will stay it’s core delivery ING Tech is looking into “Team Topologies” as a way to optimize the internal organization and as it turns out ICHP ticks all the boxes for a “Thinnest Viable Platform” (since 2018, the book was published in 2019…) Our platform is compelling for consumers, with our NaaS approach which reduces cognitive load, it simplifies the risk evidencing burden & we are strongly partnered with key “stream aligned”, “enabling” & “platform” teams in different ING segments to help guide our backlog/roadmap
  • 14. 14 Strategy (part 2 : what’s next?) For the Immutable container hosting service there’s still work to be done: • Pets -> Cattle for the clusters, which is aided by • Hypervisor based Nodes (currently all production is on Bare Metal) • Multi-Cluster Management (“MCM”) : Pattern(s)/Solution(s) to be investigated • Anomaly Detection and Policy-as-Code capabilities have been released with ICHPv2 based on OpenShift 4.10, but scope wise we have lots of opportunities to expand rulesets… Workload resilience will be high on that backlog… But ING does recognize other K8s use cases as explained before: • ICHP is hosting Elastic & Prometheus environments on dedicated OpenShift clusters (with local persistency) • ICHP is working on a Cluster-aaS delivery for Platform Teams • And ICHP is tracking the maturity of the market: • To determine when best to start investing our scarce engineering resources (Scarcity does create focus...). • The Multi-Cluster Services (“MCS”) developments have our special attention (perhaps Skupper + KCP…?) In the mean time non-Immutable workloads can still be hosted on our Private Cloud VM offering… (no exceptions/no specials in our multi-tenant Immutable Container hosting, or else we risk losing our USP…) Last but not least we strive to be a more engaged participant in the community, more on that topic in the other ING presentation today…
  • 15. Of course we’re hiring ;-) https://www.ing.jobs/tech 15 Thank you / Questions Slides will be made available at https://www.slideshare.net/ThijsEbbers/
  • 17. Whilst we treaded the eye of the needle in our Production clusters when we hit the “Pending Pods bug”, we weren’t so “lucky” with our Non-Production clusters. On several occasions consumer (DTA) workloads did encounter negative impact, and since hardware replenishment was difficult we moved D/T workloads to newly added VM based nodes (which explains the decline in the green line) in order to free up Bare Metal nodes to be used to stabilize the Production clusters as well as to keep the Acceptance workloads as stable as possible on the remaining Bare Metal nodes Meanwhile, in Non-Prod… 17 Pods/node back to 160 Pending pods Introduction of additional (VM based) nodes Non-Prod Environment stabilized
  • 18. • From ING’s Daily Banking Tribe talk at GOTO 2022 conference (“Spotify Model : On Radical Improvements From The Trenches Of Change”): • Deployment time : 62% build & deploy speed improvement • Development capacity : 27% more Backlog items done (measured per year in 2020/21) • More Coding : 16% more coding time per team • No more VM’s : Less infra costs, no patching, more secure, less issues with auditing the stack (More cool ING Tech talks on our YouTube Channel) • Building a Global Investments Platform in 12 Months (2020-21) • Building a Digital Platform for Insurance Products (“ING-AXA partnership”) in 9 Months (2019) • Building a (Mobile Only Digital) Bank in 9 Months (“The Manila Miracle”) (2018) • And more to come… ICHP(‘s-ecosystem) enabled our Internal ING Customers to achieve : 18
  • 19. Platform (“ICHP”) Adds e.g. : • ING Compliance • ING Integrations (SEM, Monitoring, AZDO, …) • ING Hardening Distribution (e.g. “OpenShift”, “RKE”, “GKE”, “AKS”) Adds e.g. : RBAC, SDN Container Management Framework (“Kubernetes”) Adds e.g. : Scaling, resilience Application Containers to Container Hosting Platforms 19 Containers
  • 20. • Important aspect to be taken into consideration: The ING Container Hosting Platform Immutable Container hosting is NOT aiming to rehost VM’s to containers… Purpose is to have the best possible hosting environment for immutable workloads! • If a DevOps teams manages to properly refactor their VM into a set of container images they are welcome • Hands-off approach in production (which implies a mature team in all aspects, e.g. CI/CD !) If teams feel not comfortable with this they should stay on VM’s! ICHP Immutable App hosting Intended Use 20
  • 21. ING aims to make our DevOps teams autonomous (As explained in ING’s Way-of-Working in 2019) . We also want to enable those DevOps teams to deliver maximum value to the business, by not bothering them with IT-Infrastructure concerns, nor bothering them with having to deliver compliancy evidence for the hosting platform. Hence offering a Namespace-as-a-Service is for us the sweet spot: • Clear demarcation between (Infra)Provider and Consumer • Enabling hand-over of compliance evidence • Enabling multi-tenancy (hence fast time-to-market/self-service consumption, and the potential for efficient utilization of resources) • The DevOps team can assume responsibility for (almost) the full stack (only the kernel stays shared). They have liberty/responsibility to choose/maintain their (versions of) base image, runtime engine, libraries, etc. (within the boundaries set by the ING corporate risk&compliancy rules !) ING will offer a K8S Cluster-as-a-Service in the future, however this will be a limited offering only available to selected platform teams. Why ICHP only offers a “Namespace-as-a-Service” 21
  • 22. Key Objective for ING is to remain capable to exit a Public Cloud provider. Determining the right abstractions is key to meeting this objective for a container hosting platform, hence: • We choose an open/industry standard (in this case: the Kubernetes ecosystem) • We choose an intended use (in this case: Immutable Workloads) • We clearly define the allowed interfaces (in the ICH – Interfaces standard) • We automate our deployments by the mandatory use of One Pipeline Note that these abstractions are design time and not runtime (as we still use the AKS service native to Azure or the GKE service native to GCP) However there will always be debate on the details as we try to balance innovation, time-to-market, risk/security and/or cost-income ambitions. Therefore actually running production workloads on at least one Kubernetes stack outside of the primary (Public) Cloud provider is the best recipe to “keep everyone honest”, as this will prevent Cloud Provider specific “optimizations” from slipping into ING code. Meaning there should always be at least 2 ICH standard solutions in use. Cloud Abstractions 22
  • 23. We do know how to build those services in grey… Our engineers are more then capable to build and run those. But not all of them together at the same time… Hence for each year ING determines where to allocate that engineering capacity: • Improving existing services • e.g. Pets -> Cattle • Supporting additional use cases on existing services • e.g. Data Services we today only support ELK & Prometheus, if we start to support Data Persistence Services more critical to the bank (e.g. message bus, (non-)relational databases) we’d actually like the availability of those Services to be independent from K8s cluster uptime (hence our interest in CNCF MCS developments) • Introducing New Services • We do have active instantiations of Centralized End-User & HPC use cases on K8s in ING (one is even presenting @KubeCon Detroit). But those are currently not hosted as a centralized/standardized Service. So we could also choose to include those in the ICH catalogue • The one defined service we’re not likely to invest time in any time soon is FaaS, that has everything to do with the controls currently enforced from a regulatory perspective and little with the actual technology… Dive and Conquer additional information 23
  • 24. IPC/ICHP hosting services (1/2) (Grey is currently not available) 24 Service Typical use case(s) Immutable Containers Capability to host immutable container workloads like 12 Factor Apps (commonly: “API’s”) Functions as a Service Capability to execute backend code directly without having to manage any servers Data Services Capability to provide data persistence as a service e.g. Object Storage, Elastic, Kafka, MS-SQL, Oracle, Cassandra, Redis, etc. HPC Capability to provide High Performance Compute (e.g. GPU, fast local cache, etc.) as a service for machine learning and analytics. Centralized End-User Capability to provide a managed, dedicated set-up for specific user(s) e.g. for data science, educational and training set-ups etc. Infrastructure Management Capability to manage IT assets as Desired State Custom Resources (e.g. Crossplane) Non-12 factor apps (“VM”s) – K8S Hosting of non-12 factor apps (“VM’s in a container package”), typically large images containing monolithic applications, not ready for hands off operations, could be requiring managed local persistency. Hosted on a K8s or equivalent platform. Standalone - Non-12 factor apps (“VM”s) Hosting of non-12 factor apps (“VM’s in a container package”), typically large images containing monolithic applications, not ready for hands off operations, could be requiring managed local persistency. Hosted on Standalone “Docker” instances.
  • 25. IPC/ICHP hosting services (2/2) (Grey is currently not available) 25 Service Unit of Delivery (/Security) Local Persistency GPU Ingress Exposed interface(s) End-user access to SSH/Terminal Interfaces Immutable Containers Namespace Not Available Not Available Shared per Traffic type (Internal/DMZ/… ) API (HTTPS ingress), Eventing (“Kafka”), File (/Object) (“S3”) Not Allowed (in Production) (currently still tolerated in Non- Production clusters) Functions as a Service Function Not Available TBD Shared See “Immutable Containers” Not Possible Data Services (set of) Namespace(s) Available Not Available Dedicated per workload TBD Not Allowed HPC Namespace Optional Optional TBD See “Immutable Containers” Not Allowed Centralized End-User Namespace TBD TBD TBD SSH/https/… Allowed Infrastructure Management Control plane Not Available Not Available Not Applicable K8s API Not Allowed Non-12 factor apps (“VM”s) – K8S Namespace, VM ? TBD Not Available TBD TBD TBD Standalone - Non-12 factor apps (“VM”s) VM Available Not Available Dedicated No restrictions Available
  • 26. “Immutable” = • Any change can only be executed via an ING pipeline which has all required checks & balances in place (and which facilitates automated evidencing for regulatory purposes) • In short : no direct access for natural persons, no change privileges for other systems And yes, ING’s Container Security Standard was in place before “Solarwinds” reached the headlines in 2020… And although most people in the IT(-Security) industry are now aware of the software lineage issue exposed by it (that doesn’t mean its anywhere near solved…), far too few understand it’s 2nd lesson about the dangers of overprivileged software… (what enabled those attackers to move laterally so easily…?) For these 2 reason ING does not trust “Zero Trust” labelled implementations as very few of those have actually covered these concerns properly... Security additional explanation 26
  • 27. Range Of Container Technology Usage Models 27
  • 28. 28 Container Technology ING Security Reference Architecture Image Storage and Retrieval Container Deployment, Management, and Operations. Image Creation, Testing, and Accreditation DevOps DevOps DevOps Host with Containers Host with Containers Host with Containers Host with Containers Build, Deploy, Test, Release Development Registry Any Other Registry Authorized Registry Deployment Registry Orchestrator Container Management Framework No Connection No Connection W.01 W.02 W.03 W.04 W.05 I.06 I.07 I.08 R.09 R.10 O.11 P.14 P.17 P.20 H.21 P.15 C.12 P.18 H.22 P.16 C.13 P.19 H.23
  • 29. 29 Container Technology Security Measures Overview Design & Code (digitize) Build & Test (create) Operate & Maintain (detect & correct) Deploy & Test (assembly) Release (orchestrate) Onboard and Plan (define) Engineering Journey R.09 R.10 Registry Measures O.11 Orchestrator Measures I.06 I.07 I.08 Image Measures Measures to be implemented by providers of CI/CD Pipeline for container technology e.g. GEP P.14 P.15 P.16 P.17 P.18 P.19 P.20 Platform Measures H.21 H.22 H.23 Host Measures C.12 C.13 Container Measures Measures to be implemented by providers of container hosting platform (except H.23) e.g. ICHP W.01 W.02 W.03 W.04 W.05 Workload Measures Measures to be implemented by providers of container workloads e.g. DevOps
  • 30. • Want to understand how to : • Use Canary (Blue/Green) releasing using K8s cluster external loadbalancing services • Use Canary (Blue/Green) releasing using K8s cluster internal capabilities • Revert my traffic 100% to my stable version in case of unexpected issues • Rollback my release to the N-1 version in case of unexpected issues with version N • Perform Resilient Placement so a single Red or Blue outage in the underlying IaaS layers will not affect my application availability • Implement Liveness Probes so my pods will restart automatically in case of applicative issues • Implement Readiness Probes so I have the input to automatically direct my traffic in case of applicative issues • And would like to see examples & templates how to use above features in ING’s One Pipeline Workload Resilience : As an ICHP Consuming DevOps Team, I 30
  • 31.
  • 32.
  • 33. The following slides come from ING Investor Update June 13th 2022 Tech& Operations 33 Strategy additional explanation
  • 34. 34
  • 35. 35
  • 37. Follow us SlideShare.net/ING @ING_News LinkedIn.com/company/ING YouTube.com/ING Flickr.com/INGGroup Facebook.com/ING ing.com ingwb.com Medium.com/ing-blog