SlideShare a Scribd company logo
Kubernetes & AI
👸Beauty and the 👾Beast!?!
Tobias Schneck
@ tobi@kubermatic.com
@toschneck
Principal Architect
@toschneck
As Kubernetes folks
why should we care about AI?
🤨
… will it be the next big thing?!
😨
By 2028, the adoption of AI will culminate in
over 50% of cloud compute
resources devoted to AI workload, up from
less than 10% in 2023.
Gartner® states, 2023
OK … so what’s about this AI thingy?
🤔󰞵
Drawings created with Excalidraw, thanks Koray Oksay (@korayoksay) for the hint 🙏
😳
… a lot of Data and Math for an
Infrastructure guy 🧐
… how does such data get compute?
🖥 💽
Credits to Andrej Karpathy 👉 Awesome Intro to LLMs
[1hr Talk] Intro to Large Language Models
Credits to Andrej Karpathy 👏
Credits to Andrej Karpathy 👏
Credits to Andrej Karpathy 👏
Credits to Andrej Karpathy 👏
Credits to Andrej Karpathy 👏
How does our normal Job look like?
󰻶
Based on Adel Zaalouk (@ZaNetworker) drawings from the CNCF Cloud Native AI white paper 🙏
What will change in our Infra?
🏗
Based on Adel Zaalouk (@ZaNetworker) drawings from the CNCF Cloud Native AI white paper 🙏
Based on Adel Zaalouk (@ZaNetworker) drawings from the CNCF Cloud Native AI white paper 🙏
So, Why Kube ?
Flexibility & Standardization
Standard
Container
High-Cube
Container
Hardtop
Container
Open Top
Container
Flat Platform (Plat) Ventilated
Container
Cooling Container Bulk Container
Tank
Container
Container Types
Data Center I
Infrastructure Layer
Standardization with Kubernetes
App Services
IT Space I IT Space II IT Space III
Backend Services
DB Services Analytics Observability
Data Center II
Data Center III
Edge
Cloud Providers
Caches AI / ML DDoS
Protect
Managed
Services
Real Time
Analysis
Intelligent
Edge Devices
Smart
Automation
Data
Processing
Kube for AI ⇔ Kube for Applications
👉 Kube is a de facto standard as “cloud operating systems”
✅ API abstraction layer for multiple types of network, storage, and compute resources
📦 Standard interfaces for support of DevOps best practices like GitOps
🌈 Variation of Cloud Providers and Services are consumable via standard APIs
… and OpenAI uses Kube already 2017 🤯
So …. 😅
How to build an AI Platform?
🏗 🤔
How does the ecosystem look
like!?
🛒 🛍
AI Frameworks
Options to Adapt the Frameworks 󰞵
Feature
“Kube Native”
via Kubeflow
Managed Platforms
(e.g., SageMaker)
Focused Tools
(e.g., MLflow)
Scope End-to-end MLOps platform Managed MLOps service
Specific functionalities within
ML lifecycle
Open Source Yes No Yes
Scalability & Portability High Depends on cloud provider Moderate
Setup & Management Complex Simpler Simpler
Portability Everywhere Mostly Cloud Mostly Machine based
Vendor Lock-in No Yes (to specific cloud provider) No
AI Frameworks
Could use
KubeFlow ⁉
👉 Currently the most
feature complete
choice for Kube
🥴 But Setup is complex!
KubeFlow ⁉
The Beauty 👸:
● Incubating CNCF Project
● Serving AI Platform in Multi-Tenancy
● Popularity 13.7k ⭐ ~ long-term Maintenance Chance
● Alternatives like MLflow / KServe are integrated
The Beast 👾
● Mostly vendor specific installer instructions 🥴
○ No maintained automated installer for generic Kubernetes
○ Helm chart issue #3173
● Dependency “hell”
○ A lot of different 3rd party dependencies constraints
○ Hard to adapt again to existing company defaults
● Only support EOL Kubernetes <= 1.26❗
○ Usability is then questionable in production
Sounds good, but what about
on-prem / offline cases?
🤔
[Cloud] Data Center I
GPU / TPU Powered Services
based on Argo CD
AI Model Serving
[AI] Application Service
Application
⚙ Separate Model Training / Model Usage Example
Infrastructure Layer
Data Center II
Data Center III
Edge
Cloud Providers
Real Time
Analysis
Intelligent
Edge Devices
Smart
Automation
Data
Processing
Data Delivery
Model
Export
Local AI
Consume
Scale for
Training
Vanilla Setup
Starting a POC
󰳘
github.com/toschneck/kubernetes-and-ai
Kubeflow | Katib Architecture for Hyperparameter
Tuning (aka optimization run)
Server & Local Model with LocalAI
Any Questions?
THANKS FOR JOINING!
kubermatic
@toschneck
tobi@kubermatic.com

More Related Content

Similar to Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024

AWS vs Azure vs Google (GCP) - Slides
AWS vs Azure vs Google (GCP) - SlidesAWS vs Azure vs Google (GCP) - Slides
AWS vs Azure vs Google (GCP) - Slides
TobyWilman
 
Info Session GDSC USICT
Info Session GDSC USICTInfo Session GDSC USICT
Info Session GDSC USICT
DSCUSICT
 
Serverless Computing with Python
Serverless Computing with PythonServerless Computing with Python
Serverless Computing with Python
wesley chun
 
Citizen Developer Tools are not just for Citizen Developers (session at Share...
Citizen Developer Tools are not just for Citizen Developers (session at Share...Citizen Developer Tools are not just for Citizen Developers (session at Share...
Citizen Developer Tools are not just for Citizen Developers (session at Share...
Antti Koskela
 
DEVNET-1125 Partner Case Study - “Project Hybrid Engineer”
DEVNET-1125	Partner Case Study - “Project Hybrid Engineer”DEVNET-1125	Partner Case Study - “Project Hybrid Engineer”
DEVNET-1125 Partner Case Study - “Project Hybrid Engineer”
Cisco DevNet
 
NZCS Cloud Computing Presentation
NZCS Cloud Computing PresentationNZCS Cloud Computing Presentation
NZCS Cloud Computing Presentation
Abhinav Keswani
 
Citizen Developer Tools - session at SPS New England 10/20/2018
Citizen Developer Tools - session at SPS New England 10/20/2018Citizen Developer Tools - session at SPS New England 10/20/2018
Citizen Developer Tools - session at SPS New England 10/20/2018
Antti Koskela
 
Citizen Developer Tools (session at SharePoint Saturday Twin Cities 4/14/2018...
Citizen Developer Tools (session at SharePoint Saturday Twin Cities 4/14/2018...Citizen Developer Tools (session at SharePoint Saturday Twin Cities 4/14/2018...
Citizen Developer Tools (session at SharePoint Saturday Twin Cities 4/14/2018...
Antti Koskela
 
Agile Architecture in a Modern Cloud-Native Ecosystem
Agile Architecture in a Modern Cloud-Native EcosystemAgile Architecture in a Modern Cloud-Native Ecosystem
Agile Architecture in a Modern Cloud-Native Ecosystem
Cloud Study Network
 
Agile architectures in a modern cloud-native ecosystem
Agile architectures in a modern cloud-native ecosystemAgile architectures in a modern cloud-native ecosystem
Agile architectures in a modern cloud-native ecosystem
Turja Narayan Chaudhuri
 
Cloud Application Development Lifecycle
Cloud Application Development LifecycleCloud Application Development Lifecycle
Cloud Application Development Lifecycle
Suhas Kelkar
 
Forget becoming a Data Scientist, become a Machine Learning Engineer instead
Forget becoming a Data Scientist, become a Machine Learning Engineer insteadForget becoming a Data Scientist, become a Machine Learning Engineer instead
Forget becoming a Data Scientist, become a Machine Learning Engineer instead
Data Con LA
 
London Oracle Developer Meetup April 18
London Oracle Developer Meetup April 18London Oracle Developer Meetup April 18
London Oracle Developer Meetup April 18
Phil Wilkins
 
Infrastructure Agnostic Machine Learning Workload Deployment
Infrastructure Agnostic Machine Learning Workload DeploymentInfrastructure Agnostic Machine Learning Workload Deployment
Infrastructure Agnostic Machine Learning Workload Deployment
Databricks
 
Demystifying IoT skills : What does it take to become a FullStack IoT engineer?
Demystifying IoT skills : What does it take to become a FullStack IoT engineer?Demystifying IoT skills : What does it take to become a FullStack IoT engineer?
Demystifying IoT skills : What does it take to become a FullStack IoT engineer?
Emertxe Information Technologies Pvt Ltd
 
Azure Meetup: Novità CosmosDB modalità Serverless e Cognitive Services
Azure Meetup: Novità CosmosDB modalità Serverless e Cognitive ServicesAzure Meetup: Novità CosmosDB modalità Serverless e Cognitive Services
Azure Meetup: Novità CosmosDB modalità Serverless e Cognitive Services
dotnetcode
 
ODSC East 2020 Accelerate ML Lifecycle with Kubernetes and Containerized Da...
ODSC East 2020   Accelerate ML Lifecycle with Kubernetes and Containerized Da...ODSC East 2020   Accelerate ML Lifecycle with Kubernetes and Containerized Da...
ODSC East 2020 Accelerate ML Lifecycle with Kubernetes and Containerized Da...
Abhinav Joshi
 
2019-10-15 - the future of cloud-native Java - Bert Ertman
2019-10-15 - the future of cloud-native Java - Bert Ertman2019-10-15 - the future of cloud-native Java - Bert Ertman
2019-10-15 - the future of cloud-native Java - Bert Ertman
Apeldoorn JUG
 
Cloud Native (Bert Ertman)
Cloud Native (Bert Ertman)Cloud Native (Bert Ertman)
Cloud Native (Bert Ertman)
Anton de Ruiter
 
Docker - Scripting the PayPal Cloud
Docker - Scripting the PayPal CloudDocker - Scripting the PayPal Cloud
Docker - Scripting the PayPal Cloud
Abraham Hoffman
 

Similar to Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024 (20)

AWS vs Azure vs Google (GCP) - Slides
AWS vs Azure vs Google (GCP) - SlidesAWS vs Azure vs Google (GCP) - Slides
AWS vs Azure vs Google (GCP) - Slides
 
Info Session GDSC USICT
Info Session GDSC USICTInfo Session GDSC USICT
Info Session GDSC USICT
 
Serverless Computing with Python
Serverless Computing with PythonServerless Computing with Python
Serverless Computing with Python
 
Citizen Developer Tools are not just for Citizen Developers (session at Share...
Citizen Developer Tools are not just for Citizen Developers (session at Share...Citizen Developer Tools are not just for Citizen Developers (session at Share...
Citizen Developer Tools are not just for Citizen Developers (session at Share...
 
DEVNET-1125 Partner Case Study - “Project Hybrid Engineer”
DEVNET-1125	Partner Case Study - “Project Hybrid Engineer”DEVNET-1125	Partner Case Study - “Project Hybrid Engineer”
DEVNET-1125 Partner Case Study - “Project Hybrid Engineer”
 
NZCS Cloud Computing Presentation
NZCS Cloud Computing PresentationNZCS Cloud Computing Presentation
NZCS Cloud Computing Presentation
 
Citizen Developer Tools - session at SPS New England 10/20/2018
Citizen Developer Tools - session at SPS New England 10/20/2018Citizen Developer Tools - session at SPS New England 10/20/2018
Citizen Developer Tools - session at SPS New England 10/20/2018
 
Citizen Developer Tools (session at SharePoint Saturday Twin Cities 4/14/2018...
Citizen Developer Tools (session at SharePoint Saturday Twin Cities 4/14/2018...Citizen Developer Tools (session at SharePoint Saturday Twin Cities 4/14/2018...
Citizen Developer Tools (session at SharePoint Saturday Twin Cities 4/14/2018...
 
Agile Architecture in a Modern Cloud-Native Ecosystem
Agile Architecture in a Modern Cloud-Native EcosystemAgile Architecture in a Modern Cloud-Native Ecosystem
Agile Architecture in a Modern Cloud-Native Ecosystem
 
Agile architectures in a modern cloud-native ecosystem
Agile architectures in a modern cloud-native ecosystemAgile architectures in a modern cloud-native ecosystem
Agile architectures in a modern cloud-native ecosystem
 
Cloud Application Development Lifecycle
Cloud Application Development LifecycleCloud Application Development Lifecycle
Cloud Application Development Lifecycle
 
Forget becoming a Data Scientist, become a Machine Learning Engineer instead
Forget becoming a Data Scientist, become a Machine Learning Engineer insteadForget becoming a Data Scientist, become a Machine Learning Engineer instead
Forget becoming a Data Scientist, become a Machine Learning Engineer instead
 
London Oracle Developer Meetup April 18
London Oracle Developer Meetup April 18London Oracle Developer Meetup April 18
London Oracle Developer Meetup April 18
 
Infrastructure Agnostic Machine Learning Workload Deployment
Infrastructure Agnostic Machine Learning Workload DeploymentInfrastructure Agnostic Machine Learning Workload Deployment
Infrastructure Agnostic Machine Learning Workload Deployment
 
Demystifying IoT skills : What does it take to become a FullStack IoT engineer?
Demystifying IoT skills : What does it take to become a FullStack IoT engineer?Demystifying IoT skills : What does it take to become a FullStack IoT engineer?
Demystifying IoT skills : What does it take to become a FullStack IoT engineer?
 
Azure Meetup: Novità CosmosDB modalità Serverless e Cognitive Services
Azure Meetup: Novità CosmosDB modalità Serverless e Cognitive ServicesAzure Meetup: Novità CosmosDB modalità Serverless e Cognitive Services
Azure Meetup: Novità CosmosDB modalità Serverless e Cognitive Services
 
ODSC East 2020 Accelerate ML Lifecycle with Kubernetes and Containerized Da...
ODSC East 2020   Accelerate ML Lifecycle with Kubernetes and Containerized Da...ODSC East 2020   Accelerate ML Lifecycle with Kubernetes and Containerized Da...
ODSC East 2020 Accelerate ML Lifecycle with Kubernetes and Containerized Da...
 
2019-10-15 - the future of cloud-native Java - Bert Ertman
2019-10-15 - the future of cloud-native Java - Bert Ertman2019-10-15 - the future of cloud-native Java - Bert Ertman
2019-10-15 - the future of cloud-native Java - Bert Ertman
 
Cloud Native (Bert Ertman)
Cloud Native (Bert Ertman)Cloud Native (Bert Ertman)
Cloud Native (Bert Ertman)
 
Docker - Scripting the PayPal Cloud
Docker - Scripting the PayPal CloudDocker - Scripting the PayPal Cloud
Docker - Scripting the PayPal Cloud
 

More from Tobias Schneck

Kubernetes in the Manufacturing Line @KubeCon EU Valencia 2022
Kubernetes in the Manufacturing Line  @KubeCon EU Valencia 2022 Kubernetes in the Manufacturing Line  @KubeCon EU Valencia 2022
Kubernetes in the Manufacturing Line @KubeCon EU Valencia 2022
Tobias Schneck
 
$ kubectl hacking @DevOpsCon Berlin 2019
$ kubectl hacking @DevOpsCon Berlin 2019$ kubectl hacking @DevOpsCon Berlin 2019
$ kubectl hacking @DevOpsCon Berlin 2019
Tobias Schneck
 
Will ARM be the new Mainstream in our Data Centers? @Rejekts Paris 2024
 Will ARM be the new Mainstream in our Data Centers? @Rejekts Paris 2024 Will ARM be the new Mainstream in our Data Centers? @Rejekts Paris 2024
Will ARM be the new Mainstream in our Data Centers? @Rejekts Paris 2024
Tobias Schneck
 
Kubermatic How to Migrate 100 Clusters from On-Prem to Google Cloud Without D...
Kubermatic How to Migrate 100 Clusters from On-Prem to Google Cloud Without D...Kubermatic How to Migrate 100 Clusters from On-Prem to Google Cloud Without D...
Kubermatic How to Migrate 100 Clusters from On-Prem to Google Cloud Without D...
Tobias Schneck
 
ClusterAPI Overview - Managing multi-cloud Kubernetes Clusters - k8s Meetup@v...
ClusterAPI Overview - Managing multi-cloud Kubernetes Clusters - k8s Meetup@v...ClusterAPI Overview - Managing multi-cloud Kubernetes Clusters - k8s Meetup@v...
ClusterAPI Overview - Managing multi-cloud Kubernetes Clusters - k8s Meetup@v...
Tobias Schneck
 
Creating Kubernetes multi clusters with ClusterAPI @ Stuttgart Kubernetes Meetup
Creating Kubernetes multi clusters with ClusterAPI @ Stuttgart Kubernetes MeetupCreating Kubernetes multi clusters with ClusterAPI @ Stuttgart Kubernetes Meetup
Creating Kubernetes multi clusters with ClusterAPI @ Stuttgart Kubernetes Meetup
Tobias Schneck
 
KubeCI - Cloud Native Continuous Delivery for Kubernetes
KubeCI - Cloud Native Continuous Delivery for KubernetesKubeCI - Cloud Native Continuous Delivery for Kubernetes
KubeCI - Cloud Native Continuous Delivery for Kubernetes
Tobias Schneck
 
Kubernetes Cluster API - managing the infrastructure of multi clusters (k8s ...
Kubernetes Cluster API - managing the infrastructure of  multi clusters (k8s ...Kubernetes Cluster API - managing the infrastructure of  multi clusters (k8s ...
Kubernetes Cluster API - managing the infrastructure of multi clusters (k8s ...
Tobias Schneck
 
UI Testing - Selenium? Rich-Clients? Containers? (SwanseaCon 2018)
UI Testing - Selenium? Rich-Clients? Containers? (SwanseaCon 2018)UI Testing - Selenium? Rich-Clients? Containers? (SwanseaCon 2018)
UI Testing - Selenium? Rich-Clients? Containers? (SwanseaCon 2018)
Tobias Schneck
 
Creating Kubernetes multi clusters with ClusterAPI in the Hetzner Cloud
Creating Kubernetes multi clusters with ClusterAPI in the Hetzner CloudCreating Kubernetes multi clusters with ClusterAPI in the Hetzner Cloud
Creating Kubernetes multi clusters with ClusterAPI in the Hetzner Cloud
Tobias Schneck
 
OpenShift Build Pipelines @ Lightweight Java User Group Meetup
OpenShift Build Pipelines @ Lightweight Java User Group MeetupOpenShift Build Pipelines @ Lightweight Java User Group Meetup
OpenShift Build Pipelines @ Lightweight Java User Group Meetup
Tobias Schneck
 
OpenShift-Build-Pipelines: Build -> Test -> Run! @JavaForumStuttgart
OpenShift-Build-Pipelines: Build -> Test -> Run! @JavaForumStuttgartOpenShift-Build-Pipelines: Build -> Test -> Run! @JavaForumStuttgart
OpenShift-Build-Pipelines: Build -> Test -> Run! @JavaForumStuttgart
Tobias Schneck
 
OpenShift-Build-Pipelines: Build ► Test ► Run!
OpenShift-Build-Pipelines: Build ► Test ► Run!OpenShift-Build-Pipelines: Build ► Test ► Run!
OpenShift-Build-Pipelines: Build ► Test ► Run!
Tobias Schneck
 
Kotlin for backend development (Hackaburg 2018 Regensburg)
Kotlin for backend development (Hackaburg 2018 Regensburg)Kotlin for backend development (Hackaburg 2018 Regensburg)
Kotlin for backend development (Hackaburg 2018 Regensburg)
Tobias Schneck
 
UI-Testing - Selenium? Rich-Clients? Containers? @APEX connect 2018
UI-Testing - Selenium? Rich-Clients? Containers? @APEX connect 2018UI-Testing - Selenium? Rich-Clients? Containers? @APEX connect 2018
UI-Testing - Selenium? Rich-Clients? Containers? @APEX connect 2018
Tobias Schneck
 
Continuous Testing: Integration- und UI-Testing mit OpenShift-Build-Pipelines
Continuous Testing: Integration- und UI-Testing mit OpenShift-Build-PipelinesContinuous Testing: Integration- und UI-Testing mit OpenShift-Build-Pipelines
Continuous Testing: Integration- und UI-Testing mit OpenShift-Build-Pipelines
Tobias Schneck
 
Testing - Selenium? Rich-Clients? Containers?
Testing - Selenium? Rich-Clients? Containers?Testing - Selenium? Rich-Clients? Containers?
Testing - Selenium? Rich-Clients? Containers?
Tobias Schneck
 
OOP2017: Containerized End-2-End Testing – automate it!
OOP2017: Containerized End-2-End Testing – automate it!OOP2017: Containerized End-2-End Testing – automate it!
OOP2017: Containerized End-2-End Testing – automate it!
Tobias Schneck
 
Containerized End-2-End Testing - Agile Testing Meetup at Süddeutsche Zeitung...
Containerized End-2-End Testing - Agile Testing Meetup at Süddeutsche Zeitung...Containerized End-2-End Testing - Agile Testing Meetup at Süddeutsche Zeitung...
Containerized End-2-End Testing - Agile Testing Meetup at Süddeutsche Zeitung...
Tobias Schneck
 
Containerized End-2-End-Testing - ContainerConf Mannheim
Containerized End-2-End-Testing - ContainerConf MannheimContainerized End-2-End-Testing - ContainerConf Mannheim
Containerized End-2-End-Testing - ContainerConf Mannheim
Tobias Schneck
 

More from Tobias Schneck (20)

Kubernetes in the Manufacturing Line @KubeCon EU Valencia 2022
Kubernetes in the Manufacturing Line  @KubeCon EU Valencia 2022 Kubernetes in the Manufacturing Line  @KubeCon EU Valencia 2022
Kubernetes in the Manufacturing Line @KubeCon EU Valencia 2022
 
$ kubectl hacking @DevOpsCon Berlin 2019
$ kubectl hacking @DevOpsCon Berlin 2019$ kubectl hacking @DevOpsCon Berlin 2019
$ kubectl hacking @DevOpsCon Berlin 2019
 
Will ARM be the new Mainstream in our Data Centers? @Rejekts Paris 2024
 Will ARM be the new Mainstream in our Data Centers? @Rejekts Paris 2024 Will ARM be the new Mainstream in our Data Centers? @Rejekts Paris 2024
Will ARM be the new Mainstream in our Data Centers? @Rejekts Paris 2024
 
Kubermatic How to Migrate 100 Clusters from On-Prem to Google Cloud Without D...
Kubermatic How to Migrate 100 Clusters from On-Prem to Google Cloud Without D...Kubermatic How to Migrate 100 Clusters from On-Prem to Google Cloud Without D...
Kubermatic How to Migrate 100 Clusters from On-Prem to Google Cloud Without D...
 
ClusterAPI Overview - Managing multi-cloud Kubernetes Clusters - k8s Meetup@v...
ClusterAPI Overview - Managing multi-cloud Kubernetes Clusters - k8s Meetup@v...ClusterAPI Overview - Managing multi-cloud Kubernetes Clusters - k8s Meetup@v...
ClusterAPI Overview - Managing multi-cloud Kubernetes Clusters - k8s Meetup@v...
 
Creating Kubernetes multi clusters with ClusterAPI @ Stuttgart Kubernetes Meetup
Creating Kubernetes multi clusters with ClusterAPI @ Stuttgart Kubernetes MeetupCreating Kubernetes multi clusters with ClusterAPI @ Stuttgart Kubernetes Meetup
Creating Kubernetes multi clusters with ClusterAPI @ Stuttgart Kubernetes Meetup
 
KubeCI - Cloud Native Continuous Delivery for Kubernetes
KubeCI - Cloud Native Continuous Delivery for KubernetesKubeCI - Cloud Native Continuous Delivery for Kubernetes
KubeCI - Cloud Native Continuous Delivery for Kubernetes
 
Kubernetes Cluster API - managing the infrastructure of multi clusters (k8s ...
Kubernetes Cluster API - managing the infrastructure of  multi clusters (k8s ...Kubernetes Cluster API - managing the infrastructure of  multi clusters (k8s ...
Kubernetes Cluster API - managing the infrastructure of multi clusters (k8s ...
 
UI Testing - Selenium? Rich-Clients? Containers? (SwanseaCon 2018)
UI Testing - Selenium? Rich-Clients? Containers? (SwanseaCon 2018)UI Testing - Selenium? Rich-Clients? Containers? (SwanseaCon 2018)
UI Testing - Selenium? Rich-Clients? Containers? (SwanseaCon 2018)
 
Creating Kubernetes multi clusters with ClusterAPI in the Hetzner Cloud
Creating Kubernetes multi clusters with ClusterAPI in the Hetzner CloudCreating Kubernetes multi clusters with ClusterAPI in the Hetzner Cloud
Creating Kubernetes multi clusters with ClusterAPI in the Hetzner Cloud
 
OpenShift Build Pipelines @ Lightweight Java User Group Meetup
OpenShift Build Pipelines @ Lightweight Java User Group MeetupOpenShift Build Pipelines @ Lightweight Java User Group Meetup
OpenShift Build Pipelines @ Lightweight Java User Group Meetup
 
OpenShift-Build-Pipelines: Build -> Test -> Run! @JavaForumStuttgart
OpenShift-Build-Pipelines: Build -> Test -> Run! @JavaForumStuttgartOpenShift-Build-Pipelines: Build -> Test -> Run! @JavaForumStuttgart
OpenShift-Build-Pipelines: Build -> Test -> Run! @JavaForumStuttgart
 
OpenShift-Build-Pipelines: Build ► Test ► Run!
OpenShift-Build-Pipelines: Build ► Test ► Run!OpenShift-Build-Pipelines: Build ► Test ► Run!
OpenShift-Build-Pipelines: Build ► Test ► Run!
 
Kotlin for backend development (Hackaburg 2018 Regensburg)
Kotlin for backend development (Hackaburg 2018 Regensburg)Kotlin for backend development (Hackaburg 2018 Regensburg)
Kotlin for backend development (Hackaburg 2018 Regensburg)
 
UI-Testing - Selenium? Rich-Clients? Containers? @APEX connect 2018
UI-Testing - Selenium? Rich-Clients? Containers? @APEX connect 2018UI-Testing - Selenium? Rich-Clients? Containers? @APEX connect 2018
UI-Testing - Selenium? Rich-Clients? Containers? @APEX connect 2018
 
Continuous Testing: Integration- und UI-Testing mit OpenShift-Build-Pipelines
Continuous Testing: Integration- und UI-Testing mit OpenShift-Build-PipelinesContinuous Testing: Integration- und UI-Testing mit OpenShift-Build-Pipelines
Continuous Testing: Integration- und UI-Testing mit OpenShift-Build-Pipelines
 
Testing - Selenium? Rich-Clients? Containers?
Testing - Selenium? Rich-Clients? Containers?Testing - Selenium? Rich-Clients? Containers?
Testing - Selenium? Rich-Clients? Containers?
 
OOP2017: Containerized End-2-End Testing – automate it!
OOP2017: Containerized End-2-End Testing – automate it!OOP2017: Containerized End-2-End Testing – automate it!
OOP2017: Containerized End-2-End Testing – automate it!
 
Containerized End-2-End Testing - Agile Testing Meetup at Süddeutsche Zeitung...
Containerized End-2-End Testing - Agile Testing Meetup at Süddeutsche Zeitung...Containerized End-2-End Testing - Agile Testing Meetup at Süddeutsche Zeitung...
Containerized End-2-End Testing - Agile Testing Meetup at Süddeutsche Zeitung...
 
Containerized End-2-End-Testing - ContainerConf Mannheim
Containerized End-2-End-Testing - ContainerConf MannheimContainerized End-2-End-Testing - ContainerConf Mannheim
Containerized End-2-End-Testing - ContainerConf Mannheim
 

Recently uploaded

Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
Daiki Mogmet Ito
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
Recommendation System using RAG Architecture
Recommendation System using RAG ArchitectureRecommendation System using RAG Architecture
Recommendation System using RAG Architecture
fredae14
 
Project Management Semester Long Project - Acuity
Project Management Semester Long Project - AcuityProject Management Semester Long Project - Acuity
Project Management Semester Long Project - Acuity
jpupo2018
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
Octavian Nadolu
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
Matthew Sinclair
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
DianaGray10
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
Matthew Sinclair
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
innovationoecd
 
Mariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceXMariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceX
Mariano Tinti
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
名前 です男
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
Hiroshi SHIBATA
 
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing InstancesEnergy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Alpen-Adria-Universität
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
Quotidiano Piemontese
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
IndexBug
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Brandon Minnick, MBA
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
Edge AI and Vision Alliance
 

Recently uploaded (20)

Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
Recommendation System using RAG Architecture
Recommendation System using RAG ArchitectureRecommendation System using RAG Architecture
Recommendation System using RAG Architecture
 
Project Management Semester Long Project - Acuity
Project Management Semester Long Project - AcuityProject Management Semester Long Project - Acuity
Project Management Semester Long Project - Acuity
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
 
Mariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceXMariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceX
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
 
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing InstancesEnergy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
 

Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024

  • 1. Kubernetes & AI 👸Beauty and the 👾Beast!?!
  • 3. As Kubernetes folks why should we care about AI? 🤨
  • 4. … will it be the next big thing?! 😨
  • 5. By 2028, the adoption of AI will culminate in over 50% of cloud compute resources devoted to AI workload, up from less than 10% in 2023. Gartner® states, 2023
  • 6. OK … so what’s about this AI thingy? 🤔󰞵
  • 7. Drawings created with Excalidraw, thanks Koray Oksay (@korayoksay) for the hint 🙏
  • 8.
  • 9.
  • 10.
  • 11.
  • 12. 😳
  • 13. … a lot of Data and Math for an Infrastructure guy 🧐 … how does such data get compute? 🖥 💽
  • 14. Credits to Andrej Karpathy 👉 Awesome Intro to LLMs [1hr Talk] Intro to Large Language Models
  • 15. Credits to Andrej Karpathy 👏
  • 16. Credits to Andrej Karpathy 👏
  • 17. Credits to Andrej Karpathy 👏
  • 18. Credits to Andrej Karpathy 👏
  • 19. Credits to Andrej Karpathy 👏
  • 20. How does our normal Job look like? 󰻶
  • 21. Based on Adel Zaalouk (@ZaNetworker) drawings from the CNCF Cloud Native AI white paper 🙏
  • 22. What will change in our Infra? 🏗
  • 23. Based on Adel Zaalouk (@ZaNetworker) drawings from the CNCF Cloud Native AI white paper 🙏
  • 24. Based on Adel Zaalouk (@ZaNetworker) drawings from the CNCF Cloud Native AI white paper 🙏
  • 25.
  • 27. Flexibility & Standardization Standard Container High-Cube Container Hardtop Container Open Top Container Flat Platform (Plat) Ventilated Container Cooling Container Bulk Container Tank Container Container Types
  • 28. Data Center I Infrastructure Layer Standardization with Kubernetes App Services IT Space I IT Space II IT Space III Backend Services DB Services Analytics Observability Data Center II Data Center III Edge Cloud Providers Caches AI / ML DDoS Protect Managed Services Real Time Analysis Intelligent Edge Devices Smart Automation Data Processing
  • 29. Kube for AI ⇔ Kube for Applications 👉 Kube is a de facto standard as “cloud operating systems” ✅ API abstraction layer for multiple types of network, storage, and compute resources 📦 Standard interfaces for support of DevOps best practices like GitOps 🌈 Variation of Cloud Providers and Services are consumable via standard APIs
  • 30. … and OpenAI uses Kube already 2017 🤯
  • 31. So …. 😅 How to build an AI Platform? 🏗 🤔
  • 32.
  • 33.
  • 34.
  • 35.
  • 36.
  • 37.
  • 38. How does the ecosystem look like!? 🛒 🛍
  • 40. Options to Adapt the Frameworks 󰞵 Feature “Kube Native” via Kubeflow Managed Platforms (e.g., SageMaker) Focused Tools (e.g., MLflow) Scope End-to-end MLOps platform Managed MLOps service Specific functionalities within ML lifecycle Open Source Yes No Yes Scalability & Portability High Depends on cloud provider Moderate Setup & Management Complex Simpler Simpler Portability Everywhere Mostly Cloud Mostly Machine based Vendor Lock-in No Yes (to specific cloud provider) No
  • 42. KubeFlow ⁉ 👉 Currently the most feature complete choice for Kube 🥴 But Setup is complex!
  • 43. KubeFlow ⁉ The Beauty 👸: ● Incubating CNCF Project ● Serving AI Platform in Multi-Tenancy ● Popularity 13.7k ⭐ ~ long-term Maintenance Chance ● Alternatives like MLflow / KServe are integrated The Beast 👾 ● Mostly vendor specific installer instructions 🥴 ○ No maintained automated installer for generic Kubernetes ○ Helm chart issue #3173 ● Dependency “hell” ○ A lot of different 3rd party dependencies constraints ○ Hard to adapt again to existing company defaults ● Only support EOL Kubernetes <= 1.26❗ ○ Usability is then questionable in production
  • 44. Sounds good, but what about on-prem / offline cases? 🤔
  • 45. [Cloud] Data Center I GPU / TPU Powered Services based on Argo CD AI Model Serving [AI] Application Service Application ⚙ Separate Model Training / Model Usage Example Infrastructure Layer Data Center II Data Center III Edge Cloud Providers Real Time Analysis Intelligent Edge Devices Smart Automation Data Processing Data Delivery Model Export Local AI Consume Scale for Training Vanilla Setup
  • 47. Kubeflow | Katib Architecture for Hyperparameter Tuning (aka optimization run)
  • 48. Server & Local Model with LocalAI
  • 49.