SlideShare a Scribd company logo
About Me
Kişisel Bilgilerim
<<12 Yıllık Tecrübeye Sahip Yazılım Mühendisiyim. Siemens AŞ de Cloud Mimarı >>
Eğitim Bilgilerim
<<Kocaeli Üniversitesi Bilgisayar Mühendisliği – 2011 >>
İletişim Bilgilerim
<<emre.gundogdu@siemens.com - gundogdu.emre@yahoo.com>>
Hobilerim
<<Basketbol Oynamak , Takip Etmek>>
Skills Language
<<Software - English>>
Kubecon &
CloudNative
Conf 2024 -
Paris
Linux Foundation
900
Opensource
Project
3M+
Developers
Trained
777K
Developeres
Contributing
Code
51M
Lines of Code
added weekly
17K
Contributing
Organization
The Cloud Native
Computing Foundation
(CNCF)
185 Projects
239K
Contributors
16.2M
Contributions
190
Countries
Projects
SANDBOX
The CNCF Sandbox is the entry
point for early stage projects
INCUBATING
Graduated and incubating
projects are considered stable
and are used successfully in
production environments
GRADUATED
Graduated and incubating
projects are considered stable
and are used successfully in
production environments
Projects
GRADUATED
INCUBATING
Recap of KubeCon 2024 - Paris
ATTENDEE OVER
12,000 PEOPLE
ARTIFICIAL
INTELLIGENT
PLATFORM
ENGINEERING
GREENER
COMPUTING
Key Components of an AI Model
Data
The foundation of
AI models. Models
are trained on large
datasets to learn
patterns and
relationships
Algorithms
The set of rules or
procedures the
model uses to
process data and
make decisions.
Training
The process of
feeding data into
the model and
adjusting its
parameters to
minimize errors and
improve accuracy
Inference
The phase where
the trained model
makes predictions
or decisions based
on new, unseen
data.
What We
Talk on
Today ?
Training an
AI Model
•Gather and preprocess data relevant to the task
Data Collection
•Choose the appropriate algorithm or architecture for the problem
Model Selection
•Feed the data into the model, adjust parameters using optimization
techniques (e.g., gradient descent)
Training
•Assess the model’s performance using metrics such as accuracy, precision,
recall, F1 score
Evaluation
•Implement the trained model in a real-world application for inference
Deployment
CloudNative Artificial
Intelligence (CNAI)
• Cloud Native
• Cloud Native technologies empower organizations to build
and run scalable applications in modern, dynamic
environments such as public, private, and hybrid clouds.
Containers, service meshes, microservices, immutable
infrastructure, and declarative APIs exemplify this
approach. These techniques enable loosely coupled
systems that are resilient, manageable, and observable.
Combined with robust automation, they allow engineers
to make high-impact changes frequently and predictably
with minimal toil.
• Cloud Native Artificial Intelligence (CNAI)
Refers to approaches and patterns for building and deploying AI
applications and workloads using the principles of Cloud Native.
Enabling repeatable and scalable AI-focused workflows allows AI
practitioners to focus on their domain
Evolution of
Artificial
Intelligence
Discriminative AI
Generative AI
Convolutional Neural Networks
Transformers
Large Language Model
arge anguage odel – LLM is
just a larger version of a
language model
• Why LLM ?
: Number of
Parameters, billion
parameters
• : Self-supervised
learning
Language Model
Large
Language
Model
Level of LLMs
Prompt Engineering
Model Fine Tuning
Build Your Own LLM
Step for Building LLM Application
• Identifiy Problem to
Solve
PLAN
• Choose the LLM
• Customize the LLM
• Setup the
Application
Architecture
BUILD • Implement evalution
and feedback
RUN
One Way to Deploy your LLM in CloudNative
Model Definition
Model Consumption (Local or API)
Package LLM
Containerize
Serve Multiple Model
Model Definition
Define Your Problem
Conversational Chatbot
• Text Summarization
• Classification
• Question Answering
Pick Your Model
Strategy
• Foundational Model
(General Knowledge)-
70B/7B
• Fine tune a model
(Context knowledge)
Own Data
• Retrival Augumented
Generation - RAG
Find Tools
• Hugging Face
• LangChain
Model Consumption
Local External
Package LLMs
Business
needs a
unified way to
interact with
models
Business needs
different types of
LLMs
Each model has
different
compute/storage
requirements
Each model has a
different way to
interact
Exposing LLMs: LangChain
LangChain: A framework for building apps
powered by LLMs
• Python and JS/Typescript library
• Native support for 80+ LLMs, open source
models supported by templates
• Supports RAG pipelines, 75+ vector stores
• LangServe: Deploy LangChain chains as REST
API
• LangSmith: Developer platform
LLMLogic: Local
Tell me about K8S
Pompt Template
CHAIN
Model
(Llama2
~26GB)
Pipeline
LLMLogic: Local Optimized
Tell me about K8S
Pompt Template
CHAIN
Pipeline
Llama Optimized
Model (~7GB)
chain = prompt|pipeline
question = "Tell me about K8s"
result = chain.invoke({"query": question})
LLMLogic: External
Tell me about K8S
Pompt Template
CHAIN
External
LLM
API
Model Client
chain = prompt|pipeline
question = "Tell me about K8s"
result = chain.invoke({"query": question})
Integrating Multiple Models Users
UI
LLM PROXY
LLAMA2
Finance Fine
Tunned Model
Other Local
LLAMA2
Legal Fine
Tuned
LLAMA2_Optimi
zed
Private
General
Knowledge
Model
(Small/Big)
External LLM API
General
Knowledge
Model
Integrated Multiple Model:
Multipod
Recap of Demo:
Pre - Local Model Downloaded
at each container launch
UI (Frontend) send messages to
the LLM Proxy
LLM_proxy send messages to
the selected LLM (INVOKE)
After processed the answer
send back to the UI
DEMO
Merging of
Cloud Native
and Artificial
Intelligence
Predictive and generative AI needs
across computing, networking, and
storage
Challenges/Need Generative AI Predictive AI
Computational Power Extremely high. Requires specialized hardware. Moderate to high. General-purpose hardware can suffice.
Data Volume and Diversity Massive, diverse datasets for training. Specific historical data for prediction
Model Training and Fine-
tuning
Complex, iterative training with specialized compute. Moderate training.
Scalability and Elasticity Highly scalable and elastic infrastructure (variable and
intensive computational demands)
Scalability is necessary but lower elasticity demands. Batch
processing or event-driven tasks
Storage and Throughput High-performance storage with excellent throughput.
Diverse data types. Requires high throughput and
lowlatency access to data.
Efficient storage with moderate throughput. It focuses
more on data analysis and less on data generation; data is
mostly structured.
Networking High bandwidth and low latency for data transfer and
model synchronization (e.g., during distributed training).
Consistent and reliable connectivity for data access.
Enabling Tools and Techniques
LinuxFoundation AI Landscape
Distributed
Training
Kubeflow
Training
Operator
Pytorch DDP
Torchx
Tensorflow
Distributed
So on …
General
Orchestratio
n
Kubernetes
Volcana
Armada
Kuberay
Nvidia Nemo
Yunikorn
Kueue
So on..
ML Serving
Kserve
Seldon
VLLM
So on..
CI/CD
KubeFlow
Pipeline
ML FLow
TFX
BentoML
MLRun
so on ...
Data Science
Juypter
Kubeflow
Notebook
Pytorch
TensorFlow
Apache
Zeeplin
so on ..
Workload
Observabilit
y
Prometheus
Graphana
InfluxDB
Open
Telemetery
so on ...
Auto ML
Hyperopt
Optuna
Kubeflow
Katib
NNI
so on ...
Governance
& Policy
Kyverno
Optuna
OPA/
Gateway
so on ...
Data
Architecture
Clickhouse
Apache Pinot
Apache Druid
Cassandra
Hadoop HDFS
Apache
HBASE
Apache Spark
Apache Flink
Apache
Pulsar
Vektor
Database
Milvus
Chroma
Quadrant
Pinecone
LLM
Observabilit
y
Trulens
Langfuse
OpenLLMetry
Distributed
Training
Kubeflow
Training
Operator
Pytorch DDP
Torchx
Tensorflow
Distributed
OpenMPI
Deepspeed
Megatron
CHALLENGES FOR
CLOUD NATIVE
ARTIFICIAL
INTELLIGENCE
The typical ML pipeline is comprised of:
• Data Preparation (collection, cleaning/pre-processing, feature engineering)
• Model Training (model selection, architecture, hyperparameter tuning)
• CI/CD, Model Registry (storage)
• Model Serving
• Observability (usage load, model drift, security)
Benefits of Kubernetes for ML
REPEATABILITY PIPELINE PORTABILITY SCALING
Right Tools for ML/AI Jobs
GPU
Huge amount of core
Good for Lightweight task
Design for graphics computation task
CPU
Small amount of Core
Good for Heavy Task
Design for Common Computation
How GPUs
work with
K8s
K8S GPU Worker Node
How GPUs are Actually Used
Training environment
Notebooks
MLOps pipelines
Data processing
Tests
Inference environment
ML model serving
Online operations
Data pre-processing
DPUs
AI Landscape & Ecosystem
Modeling
Deployment
Versioning
Orchestration
Compute
Data
How
Infrastructure
is
Needed
How
Much
Data
Scientist
Care
AI Landscape & Ecosystem
Cloud Native Production-Ready AI Platform
Summary
What is LF , CNCF and Kubecon2024 Paris
Cloud Native Artifical Intelligence
What is LLM ?
Multi Model LLM Demo on Kubernetes
Cloud Native Production Ready AI Platform Components
References
https://www.cncf.io/reports/cloud-native-artificial-intelligence-whitepaper/
https://www.youtube.com/watch?v=1u5LtsJqyrA&list=PLj6h78yzYM2N8nw1YcqqKveySH6_0VnI0
https://www.youtube.com/watch?v=Ek0eU_H9AoQ&list=PLj6h78yzYM2PWGv34W6w5ssq1b1meRmY7
https://huggingface.co/
https://www.langchain.com/langchain
https://ollama.com/
Q & A
Thank you

More Related Content

Similar to KubeCon & CloudNative Con 2024 Artificial Intelligent

Best Practices for Building Hybrid-Cloud Architectures | Hans Jespersen
Best Practices for Building Hybrid-Cloud Architectures | Hans JespersenBest Practices for Building Hybrid-Cloud Architectures | Hans Jespersen
Best Practices for Building Hybrid-Cloud Architectures | Hans Jespersen
confluent
 
End to end Machine Learning using Kubeflow - Build, Train, Deploy and Manage
End to end Machine Learning using Kubeflow - Build, Train, Deploy and ManageEnd to end Machine Learning using Kubeflow - Build, Train, Deploy and Manage
End to end Machine Learning using Kubeflow - Build, Train, Deploy and Manage
Animesh Singh
 
Scaling AI/ML with Containers and Kubernetes
Scaling AI/ML with Containers and Kubernetes Scaling AI/ML with Containers and Kubernetes
Scaling AI/ML with Containers and Kubernetes
Tushar Katarki
 
The Future of Cloud Innovation, featuring Adrian Cockcroft
The Future of Cloud Innovation, featuring Adrian CockcroftThe Future of Cloud Innovation, featuring Adrian Cockcroft
The Future of Cloud Innovation, featuring Adrian Cockcroft
Dun & Bradstreet Cloud Innovation Center
 
Red hat's updates on the cloud & infrastructure strategy
Red hat's updates on the cloud & infrastructure strategyRed hat's updates on the cloud & infrastructure strategy
Red hat's updates on the cloud & infrastructure strategy
Orgad Kimchi
 
Ohio LinuxFest: Crash Course in Open Source Cloud Computing
Ohio LinuxFest:  Crash Course in Open Source Cloud ComputingOhio LinuxFest:  Crash Course in Open Source Cloud Computing
Ohio LinuxFest: Crash Course in Open Source Cloud Computing
Mark Hinkle
 
The REMICS model-driven process for migrating legacy applications to the cloud
The REMICS model-driven process for migrating legacy applications to the cloudThe REMICS model-driven process for migrating legacy applications to the cloud
The REMICS model-driven process for migrating legacy applications to the cloud
Marcos Almeida
 
Serverless machine learning architectures at Helixa
Serverless machine learning architectures at HelixaServerless machine learning architectures at Helixa
Serverless machine learning architectures at Helixa
Data Science Milan
 
Session
SessionSession
Session
Amr Salah
 
Cloud Aware Large Scale Distributed SOA
Cloud Aware Large Scale Distributed SOACloud Aware Large Scale Distributed SOA
Cloud Aware Large Scale Distributed SOA
Christophe Hamerling
 
Cloudify: Open vCPE Design Concepts and Multi-Cloud Orchestration
Cloudify: Open vCPE Design Concepts and Multi-Cloud OrchestrationCloudify: Open vCPE Design Concepts and Multi-Cloud Orchestration
Cloudify: Open vCPE Design Concepts and Multi-Cloud Orchestration
Cloudify Community
 
OS for AI: Elastic Microservices & the Next Gen of ML
OS for AI: Elastic Microservices & the Next Gen of MLOS for AI: Elastic Microservices & the Next Gen of ML
OS for AI: Elastic Microservices & the Next Gen of ML
Nordic APIs
 
From OpenStack.... towards an Open cloud architecture
From OpenStack.... towards an Open cloud architecture From OpenStack.... towards an Open cloud architecture
From OpenStack.... towards an Open cloud architecture
Claude Riousset
 
Modern Cloud-Native Streaming Platforms: Event Streaming Microservices with A...
Modern Cloud-Native Streaming Platforms: Event Streaming Microservices with A...Modern Cloud-Native Streaming Platforms: Event Streaming Microservices with A...
Modern Cloud-Native Streaming Platforms: Event Streaming Microservices with A...
confluent
 
Optimizing the Ops in DevOps
Optimizing the Ops in DevOpsOptimizing the Ops in DevOps
Optimizing the Ops in DevOps
Gordon Haff
 
Confluent Partner Tech Talk with Synthesis
Confluent Partner Tech Talk with SynthesisConfluent Partner Tech Talk with Synthesis
Confluent Partner Tech Talk with Synthesis
confluent
 
Hybrid Cloud, Kubeflow and Tensorflow Extended [TFX]
Hybrid Cloud, Kubeflow and Tensorflow Extended [TFX]Hybrid Cloud, Kubeflow and Tensorflow Extended [TFX]
Hybrid Cloud, Kubeflow and Tensorflow Extended [TFX]
Animesh Singh
 
Feec telecom-nw-softwarization-aug-2015
Feec telecom-nw-softwarization-aug-2015Feec telecom-nw-softwarization-aug-2015
Feec telecom-nw-softwarization-aug-2015
Christian Esteve Rothenberg
 
Data Streaming with Apache Kafka & MongoDB
Data Streaming with Apache Kafka & MongoDBData Streaming with Apache Kafka & MongoDB
Data Streaming with Apache Kafka & MongoDB
confluent
 
Oracle Keynote Cloud Expo 11-04-09
Oracle Keynote Cloud Expo 11-04-09Oracle Keynote Cloud Expo 11-04-09
Oracle Keynote Cloud Expo 11-04-09
Rex Wang
 

Similar to KubeCon & CloudNative Con 2024 Artificial Intelligent (20)

Best Practices for Building Hybrid-Cloud Architectures | Hans Jespersen
Best Practices for Building Hybrid-Cloud Architectures | Hans JespersenBest Practices for Building Hybrid-Cloud Architectures | Hans Jespersen
Best Practices for Building Hybrid-Cloud Architectures | Hans Jespersen
 
End to end Machine Learning using Kubeflow - Build, Train, Deploy and Manage
End to end Machine Learning using Kubeflow - Build, Train, Deploy and ManageEnd to end Machine Learning using Kubeflow - Build, Train, Deploy and Manage
End to end Machine Learning using Kubeflow - Build, Train, Deploy and Manage
 
Scaling AI/ML with Containers and Kubernetes
Scaling AI/ML with Containers and Kubernetes Scaling AI/ML with Containers and Kubernetes
Scaling AI/ML with Containers and Kubernetes
 
The Future of Cloud Innovation, featuring Adrian Cockcroft
The Future of Cloud Innovation, featuring Adrian CockcroftThe Future of Cloud Innovation, featuring Adrian Cockcroft
The Future of Cloud Innovation, featuring Adrian Cockcroft
 
Red hat's updates on the cloud & infrastructure strategy
Red hat's updates on the cloud & infrastructure strategyRed hat's updates on the cloud & infrastructure strategy
Red hat's updates on the cloud & infrastructure strategy
 
Ohio LinuxFest: Crash Course in Open Source Cloud Computing
Ohio LinuxFest:  Crash Course in Open Source Cloud ComputingOhio LinuxFest:  Crash Course in Open Source Cloud Computing
Ohio LinuxFest: Crash Course in Open Source Cloud Computing
 
The REMICS model-driven process for migrating legacy applications to the cloud
The REMICS model-driven process for migrating legacy applications to the cloudThe REMICS model-driven process for migrating legacy applications to the cloud
The REMICS model-driven process for migrating legacy applications to the cloud
 
Serverless machine learning architectures at Helixa
Serverless machine learning architectures at HelixaServerless machine learning architectures at Helixa
Serverless machine learning architectures at Helixa
 
Session
SessionSession
Session
 
Cloud Aware Large Scale Distributed SOA
Cloud Aware Large Scale Distributed SOACloud Aware Large Scale Distributed SOA
Cloud Aware Large Scale Distributed SOA
 
Cloudify: Open vCPE Design Concepts and Multi-Cloud Orchestration
Cloudify: Open vCPE Design Concepts and Multi-Cloud OrchestrationCloudify: Open vCPE Design Concepts and Multi-Cloud Orchestration
Cloudify: Open vCPE Design Concepts and Multi-Cloud Orchestration
 
OS for AI: Elastic Microservices & the Next Gen of ML
OS for AI: Elastic Microservices & the Next Gen of MLOS for AI: Elastic Microservices & the Next Gen of ML
OS for AI: Elastic Microservices & the Next Gen of ML
 
From OpenStack.... towards an Open cloud architecture
From OpenStack.... towards an Open cloud architecture From OpenStack.... towards an Open cloud architecture
From OpenStack.... towards an Open cloud architecture
 
Modern Cloud-Native Streaming Platforms: Event Streaming Microservices with A...
Modern Cloud-Native Streaming Platforms: Event Streaming Microservices with A...Modern Cloud-Native Streaming Platforms: Event Streaming Microservices with A...
Modern Cloud-Native Streaming Platforms: Event Streaming Microservices with A...
 
Optimizing the Ops in DevOps
Optimizing the Ops in DevOpsOptimizing the Ops in DevOps
Optimizing the Ops in DevOps
 
Confluent Partner Tech Talk with Synthesis
Confluent Partner Tech Talk with SynthesisConfluent Partner Tech Talk with Synthesis
Confluent Partner Tech Talk with Synthesis
 
Hybrid Cloud, Kubeflow and Tensorflow Extended [TFX]
Hybrid Cloud, Kubeflow and Tensorflow Extended [TFX]Hybrid Cloud, Kubeflow and Tensorflow Extended [TFX]
Hybrid Cloud, Kubeflow and Tensorflow Extended [TFX]
 
Feec telecom-nw-softwarization-aug-2015
Feec telecom-nw-softwarization-aug-2015Feec telecom-nw-softwarization-aug-2015
Feec telecom-nw-softwarization-aug-2015
 
Data Streaming with Apache Kafka & MongoDB
Data Streaming with Apache Kafka & MongoDBData Streaming with Apache Kafka & MongoDB
Data Streaming with Apache Kafka & MongoDB
 
Oracle Keynote Cloud Expo 11-04-09
Oracle Keynote Cloud Expo 11-04-09Oracle Keynote Cloud Expo 11-04-09
Oracle Keynote Cloud Expo 11-04-09
 

Recently uploaded

PSD to Wordpress Service Providers in 2024
PSD to Wordpress Service Providers in 2024PSD to Wordpress Service Providers in 2024
PSD to Wordpress Service Providers in 2024
Bestdesign2hub
 
How-to-Diagnose-Hard-Drives-by-DFL-DDP-2024.pdf
How-to-Diagnose-Hard-Drives-by-DFL-DDP-2024.pdfHow-to-Diagnose-Hard-Drives-by-DFL-DDP-2024.pdf
How-to-Diagnose-Hard-Drives-by-DFL-DDP-2024.pdf
Dolphin Data Lab
 
Cyber Security Course & Guide. X.GI. pdf
Cyber Security Course & Guide. X.GI. pdfCyber Security Course & Guide. X.GI. pdf
Cyber Security Course & Guide. X.GI. pdf
RohitRoshanBengROHIT
 
Corporate Minimal Newspaper Headline Style Newsletter.pptx
Corporate Minimal Newspaper Headline Style Newsletter.pptxCorporate Minimal Newspaper Headline Style Newsletter.pptx
Corporate Minimal Newspaper Headline Style Newsletter.pptx
byubyu7
 
Dewanstudio Project Portfolio 2023 show case
Dewanstudio Project Portfolio 2023 show caseDewanstudio Project Portfolio 2023 show case
Dewanstudio Project Portfolio 2023 show case
DEWANSTUDIO.COM
 
Quiz Quiz Hota Hai (School Quiz 2018-19)
Quiz Quiz Hota Hai (School Quiz 2018-19)Quiz Quiz Hota Hai (School Quiz 2018-19)
Quiz Quiz Hota Hai (School Quiz 2018-19)
Kashyap J
 
Trading Strategy for London silver bullet
Trading Strategy for London silver bulletTrading Strategy for London silver bullet
Trading Strategy for London silver bullet
OkgatoSemadi1
 
Information Systems Auditing, Controls and Assurance , tanapat limsaiprom
Information Systems Auditing, Controls and Assurance , tanapat limsaipromInformation Systems Auditing, Controls and Assurance , tanapat limsaiprom
Information Systems Auditing, Controls and Assurance , tanapat limsaiprom
TanapatLimsaiprom1
 
Tarun Gaur On Data Breaches and Privacy Fears
Tarun Gaur On Data Breaches and Privacy FearsTarun Gaur On Data Breaches and Privacy Fears
Tarun Gaur On Data Breaches and Privacy Fears
Tarun Gaur
 
Build a Professional Resume using Canva , Tanapat Limsaiprom
Build a Professional Resume using Canva , Tanapat LimsaipromBuild a Professional Resume using Canva , Tanapat Limsaiprom
Build a Professional Resume using Canva , Tanapat Limsaiprom
TanapatLimsaiprom1
 
Chennai Girls Call ServiCe X00XXX00XX Tanisha Best High Class Chennai Available
Chennai Girls Call ServiCe X00XXX00XX Tanisha Best High Class Chennai AvailableChennai Girls Call ServiCe X00XXX00XX Tanisha Best High Class Chennai Available
Chennai Girls Call ServiCe X00XXX00XX Tanisha Best High Class Chennai Available
shamrisumri
 
Jarren Duran Fuck EM T shirts Jarren Duran Fuck EM T shirts
Jarren Duran Fuck EM T shirts Jarren Duran Fuck EM T shirtsJarren Duran Fuck EM T shirts Jarren Duran Fuck EM T shirts
Jarren Duran Fuck EM T shirts Jarren Duran Fuck EM T shirts
exgf28
 
202254.com全网最高清影视香蕉影视,热门电影推荐,热门电视剧在线观看,免费电影,电影在线,在线观看。球华人在线電視劇,免费点播,免费提供最新高清的...
202254.com全网最高清影视香蕉影视,热门电影推荐,热门电视剧在线观看,免费电影,电影在线,在线观看。球华人在线電視劇,免费点播,免费提供最新高清的...202254.com全网最高清影视香蕉影视,热门电影推荐,热门电视剧在线观看,免费电影,电影在线,在线观看。球华人在线電視劇,免费点播,免费提供最新高清的...
202254.com全网最高清影视香蕉影视,热门电影推荐,热门电视剧在线观看,免费电影,电影在线,在线观看。球华人在线電視劇,免费点播,免费提供最新高清的...
ffg01100
 
202254.com免费观看《长相思第二季》免费观看高清,长相思第二季线上看,《长相思第二季》最新电视剧在线观看,杨紫最新电视剧
202254.com免费观看《长相思第二季》免费观看高清,长相思第二季线上看,《长相思第二季》最新电视剧在线观看,杨紫最新电视剧202254.com免费观看《长相思第二季》免费观看高清,长相思第二季线上看,《长相思第二季》最新电视剧在线观看,杨紫最新电视剧
202254.com免费观看《长相思第二季》免费观看高清,长相思第二季线上看,《长相思第二季》最新电视剧在线观看,杨紫最新电视剧
ffg01100
 
SlideEgg_200767-ICC Mens T20 World Cup 2024.pptx
SlideEgg_200767-ICC Mens T20 World Cup 2024.pptxSlideEgg_200767-ICC Mens T20 World Cup 2024.pptx
SlideEgg_200767-ICC Mens T20 World Cup 2024.pptx
NandakumarP24
 
Bai-Tập-Tiếng-Anh-On-Tập-He lớp 1- lớp 5 hot nhất
Bai-Tập-Tiếng-Anh-On-Tập-He lớp 1- lớp 5  hot nhấtBai-Tập-Tiếng-Anh-On-Tập-He lớp 1- lớp 5  hot nhất
Bai-Tập-Tiếng-Anh-On-Tập-He lớp 1- lớp 5 hot nhất
Thiên Đường Tình Yêu
 
AWS Networking Basic , tanapat limsaiprom
AWS Networking Basic , tanapat limsaipromAWS Networking Basic , tanapat limsaiprom
AWS Networking Basic , tanapat limsaiprom
ธนาพัฒน์ ลิ้มสายพรหม
 
Web development Platform Constraints.pptx
Web development Platform Constraints.pptxWeb development Platform Constraints.pptx
Web development Platform Constraints.pptx
ssuser2f6682
 
@Girls @Call Chennai 🛬 XXXXXXXXXX 🛬 available 24*7 cash payment book now pay ...
@Girls @Call Chennai 🛬 XXXXXXXXXX 🛬 available 24*7 cash payment book now pay ...@Girls @Call Chennai 🛬 XXXXXXXXXX 🛬 available 24*7 cash payment book now pay ...
@Girls @Call Chennai 🛬 XXXXXXXXXX 🛬 available 24*7 cash payment book now pay ...
shamrisumri
 
Female Service Girls Call Delhi 9873940964 Provide Best And Top Girl Service ...
Female Service Girls Call Delhi 9873940964 Provide Best And Top Girl Service ...Female Service Girls Call Delhi 9873940964 Provide Best And Top Girl Service ...
Female Service Girls Call Delhi 9873940964 Provide Best And Top Girl Service ...
elbertablack
 

Recently uploaded (20)

PSD to Wordpress Service Providers in 2024
PSD to Wordpress Service Providers in 2024PSD to Wordpress Service Providers in 2024
PSD to Wordpress Service Providers in 2024
 
How-to-Diagnose-Hard-Drives-by-DFL-DDP-2024.pdf
How-to-Diagnose-Hard-Drives-by-DFL-DDP-2024.pdfHow-to-Diagnose-Hard-Drives-by-DFL-DDP-2024.pdf
How-to-Diagnose-Hard-Drives-by-DFL-DDP-2024.pdf
 
Cyber Security Course & Guide. X.GI. pdf
Cyber Security Course & Guide. X.GI. pdfCyber Security Course & Guide. X.GI. pdf
Cyber Security Course & Guide. X.GI. pdf
 
Corporate Minimal Newspaper Headline Style Newsletter.pptx
Corporate Minimal Newspaper Headline Style Newsletter.pptxCorporate Minimal Newspaper Headline Style Newsletter.pptx
Corporate Minimal Newspaper Headline Style Newsletter.pptx
 
Dewanstudio Project Portfolio 2023 show case
Dewanstudio Project Portfolio 2023 show caseDewanstudio Project Portfolio 2023 show case
Dewanstudio Project Portfolio 2023 show case
 
Quiz Quiz Hota Hai (School Quiz 2018-19)
Quiz Quiz Hota Hai (School Quiz 2018-19)Quiz Quiz Hota Hai (School Quiz 2018-19)
Quiz Quiz Hota Hai (School Quiz 2018-19)
 
Trading Strategy for London silver bullet
Trading Strategy for London silver bulletTrading Strategy for London silver bullet
Trading Strategy for London silver bullet
 
Information Systems Auditing, Controls and Assurance , tanapat limsaiprom
Information Systems Auditing, Controls and Assurance , tanapat limsaipromInformation Systems Auditing, Controls and Assurance , tanapat limsaiprom
Information Systems Auditing, Controls and Assurance , tanapat limsaiprom
 
Tarun Gaur On Data Breaches and Privacy Fears
Tarun Gaur On Data Breaches and Privacy FearsTarun Gaur On Data Breaches and Privacy Fears
Tarun Gaur On Data Breaches and Privacy Fears
 
Build a Professional Resume using Canva , Tanapat Limsaiprom
Build a Professional Resume using Canva , Tanapat LimsaipromBuild a Professional Resume using Canva , Tanapat Limsaiprom
Build a Professional Resume using Canva , Tanapat Limsaiprom
 
Chennai Girls Call ServiCe X00XXX00XX Tanisha Best High Class Chennai Available
Chennai Girls Call ServiCe X00XXX00XX Tanisha Best High Class Chennai AvailableChennai Girls Call ServiCe X00XXX00XX Tanisha Best High Class Chennai Available
Chennai Girls Call ServiCe X00XXX00XX Tanisha Best High Class Chennai Available
 
Jarren Duran Fuck EM T shirts Jarren Duran Fuck EM T shirts
Jarren Duran Fuck EM T shirts Jarren Duran Fuck EM T shirtsJarren Duran Fuck EM T shirts Jarren Duran Fuck EM T shirts
Jarren Duran Fuck EM T shirts Jarren Duran Fuck EM T shirts
 
202254.com全网最高清影视香蕉影视,热门电影推荐,热门电视剧在线观看,免费电影,电影在线,在线观看。球华人在线電視劇,免费点播,免费提供最新高清的...
202254.com全网最高清影视香蕉影视,热门电影推荐,热门电视剧在线观看,免费电影,电影在线,在线观看。球华人在线電視劇,免费点播,免费提供最新高清的...202254.com全网最高清影视香蕉影视,热门电影推荐,热门电视剧在线观看,免费电影,电影在线,在线观看。球华人在线電視劇,免费点播,免费提供最新高清的...
202254.com全网最高清影视香蕉影视,热门电影推荐,热门电视剧在线观看,免费电影,电影在线,在线观看。球华人在线電視劇,免费点播,免费提供最新高清的...
 
202254.com免费观看《长相思第二季》免费观看高清,长相思第二季线上看,《长相思第二季》最新电视剧在线观看,杨紫最新电视剧
202254.com免费观看《长相思第二季》免费观看高清,长相思第二季线上看,《长相思第二季》最新电视剧在线观看,杨紫最新电视剧202254.com免费观看《长相思第二季》免费观看高清,长相思第二季线上看,《长相思第二季》最新电视剧在线观看,杨紫最新电视剧
202254.com免费观看《长相思第二季》免费观看高清,长相思第二季线上看,《长相思第二季》最新电视剧在线观看,杨紫最新电视剧
 
SlideEgg_200767-ICC Mens T20 World Cup 2024.pptx
SlideEgg_200767-ICC Mens T20 World Cup 2024.pptxSlideEgg_200767-ICC Mens T20 World Cup 2024.pptx
SlideEgg_200767-ICC Mens T20 World Cup 2024.pptx
 
Bai-Tập-Tiếng-Anh-On-Tập-He lớp 1- lớp 5 hot nhất
Bai-Tập-Tiếng-Anh-On-Tập-He lớp 1- lớp 5  hot nhấtBai-Tập-Tiếng-Anh-On-Tập-He lớp 1- lớp 5  hot nhất
Bai-Tập-Tiếng-Anh-On-Tập-He lớp 1- lớp 5 hot nhất
 
AWS Networking Basic , tanapat limsaiprom
AWS Networking Basic , tanapat limsaipromAWS Networking Basic , tanapat limsaiprom
AWS Networking Basic , tanapat limsaiprom
 
Web development Platform Constraints.pptx
Web development Platform Constraints.pptxWeb development Platform Constraints.pptx
Web development Platform Constraints.pptx
 
@Girls @Call Chennai 🛬 XXXXXXXXXX 🛬 available 24*7 cash payment book now pay ...
@Girls @Call Chennai 🛬 XXXXXXXXXX 🛬 available 24*7 cash payment book now pay ...@Girls @Call Chennai 🛬 XXXXXXXXXX 🛬 available 24*7 cash payment book now pay ...
@Girls @Call Chennai 🛬 XXXXXXXXXX 🛬 available 24*7 cash payment book now pay ...
 
Female Service Girls Call Delhi 9873940964 Provide Best And Top Girl Service ...
Female Service Girls Call Delhi 9873940964 Provide Best And Top Girl Service ...Female Service Girls Call Delhi 9873940964 Provide Best And Top Girl Service ...
Female Service Girls Call Delhi 9873940964 Provide Best And Top Girl Service ...
 

KubeCon & CloudNative Con 2024 Artificial Intelligent

  • 1. About Me Kişisel Bilgilerim <<12 Yıllık Tecrübeye Sahip Yazılım Mühendisiyim. Siemens AŞ de Cloud Mimarı >> Eğitim Bilgilerim <<Kocaeli Üniversitesi Bilgisayar Mühendisliği – 2011 >> İletişim Bilgilerim <<emre.gundogdu@siemens.com - gundogdu.emre@yahoo.com>> Hobilerim <<Basketbol Oynamak , Takip Etmek>> Skills Language <<Software - English>>
  • 4. The Cloud Native Computing Foundation (CNCF) 185 Projects 239K Contributors 16.2M Contributions 190 Countries
  • 5. Projects SANDBOX The CNCF Sandbox is the entry point for early stage projects INCUBATING Graduated and incubating projects are considered stable and are used successfully in production environments GRADUATED Graduated and incubating projects are considered stable and are used successfully in production environments
  • 7. Recap of KubeCon 2024 - Paris ATTENDEE OVER 12,000 PEOPLE ARTIFICIAL INTELLIGENT PLATFORM ENGINEERING GREENER COMPUTING
  • 8. Key Components of an AI Model Data The foundation of AI models. Models are trained on large datasets to learn patterns and relationships Algorithms The set of rules or procedures the model uses to process data and make decisions. Training The process of feeding data into the model and adjusting its parameters to minimize errors and improve accuracy Inference The phase where the trained model makes predictions or decisions based on new, unseen data.
  • 9. What We Talk on Today ? Training an AI Model •Gather and preprocess data relevant to the task Data Collection •Choose the appropriate algorithm or architecture for the problem Model Selection •Feed the data into the model, adjust parameters using optimization techniques (e.g., gradient descent) Training •Assess the model’s performance using metrics such as accuracy, precision, recall, F1 score Evaluation •Implement the trained model in a real-world application for inference Deployment
  • 10. CloudNative Artificial Intelligence (CNAI) • Cloud Native • Cloud Native technologies empower organizations to build and run scalable applications in modern, dynamic environments such as public, private, and hybrid clouds. Containers, service meshes, microservices, immutable infrastructure, and declarative APIs exemplify this approach. These techniques enable loosely coupled systems that are resilient, manageable, and observable. Combined with robust automation, they allow engineers to make high-impact changes frequently and predictably with minimal toil. • Cloud Native Artificial Intelligence (CNAI) Refers to approaches and patterns for building and deploying AI applications and workloads using the principles of Cloud Native. Enabling repeatable and scalable AI-focused workflows allows AI practitioners to focus on their domain
  • 11. Evolution of Artificial Intelligence Discriminative AI Generative AI Convolutional Neural Networks Transformers
  • 12. Large Language Model arge anguage odel – LLM is just a larger version of a language model • Why LLM ? : Number of Parameters, billion parameters • : Self-supervised learning Language Model Large Language Model
  • 13. Level of LLMs Prompt Engineering Model Fine Tuning Build Your Own LLM
  • 14. Step for Building LLM Application • Identifiy Problem to Solve PLAN • Choose the LLM • Customize the LLM • Setup the Application Architecture BUILD • Implement evalution and feedback RUN
  • 15. One Way to Deploy your LLM in CloudNative Model Definition Model Consumption (Local or API) Package LLM Containerize Serve Multiple Model
  • 16. Model Definition Define Your Problem Conversational Chatbot • Text Summarization • Classification • Question Answering Pick Your Model Strategy • Foundational Model (General Knowledge)- 70B/7B • Fine tune a model (Context knowledge) Own Data • Retrival Augumented Generation - RAG Find Tools • Hugging Face • LangChain
  • 18. Package LLMs Business needs a unified way to interact with models Business needs different types of LLMs Each model has different compute/storage requirements Each model has a different way to interact
  • 19. Exposing LLMs: LangChain LangChain: A framework for building apps powered by LLMs • Python and JS/Typescript library • Native support for 80+ LLMs, open source models supported by templates • Supports RAG pipelines, 75+ vector stores • LangServe: Deploy LangChain chains as REST API • LangSmith: Developer platform
  • 20. LLMLogic: Local Tell me about K8S Pompt Template CHAIN Model (Llama2 ~26GB) Pipeline
  • 21. LLMLogic: Local Optimized Tell me about K8S Pompt Template CHAIN Pipeline Llama Optimized Model (~7GB) chain = prompt|pipeline question = "Tell me about K8s" result = chain.invoke({"query": question})
  • 22. LLMLogic: External Tell me about K8S Pompt Template CHAIN External LLM API Model Client chain = prompt|pipeline question = "Tell me about K8s" result = chain.invoke({"query": question})
  • 23. Integrating Multiple Models Users UI LLM PROXY LLAMA2 Finance Fine Tunned Model Other Local LLAMA2 Legal Fine Tuned LLAMA2_Optimi zed Private General Knowledge Model (Small/Big) External LLM API General Knowledge Model
  • 25. Recap of Demo: Pre - Local Model Downloaded at each container launch UI (Frontend) send messages to the LLM Proxy LLM_proxy send messages to the selected LLM (INVOKE) After processed the answer send back to the UI
  • 26. DEMO
  • 27. Merging of Cloud Native and Artificial Intelligence
  • 28. Predictive and generative AI needs across computing, networking, and storage Challenges/Need Generative AI Predictive AI Computational Power Extremely high. Requires specialized hardware. Moderate to high. General-purpose hardware can suffice. Data Volume and Diversity Massive, diverse datasets for training. Specific historical data for prediction Model Training and Fine- tuning Complex, iterative training with specialized compute. Moderate training. Scalability and Elasticity Highly scalable and elastic infrastructure (variable and intensive computational demands) Scalability is necessary but lower elasticity demands. Batch processing or event-driven tasks Storage and Throughput High-performance storage with excellent throughput. Diverse data types. Requires high throughput and lowlatency access to data. Efficient storage with moderate throughput. It focuses more on data analysis and less on data generation; data is mostly structured. Networking High bandwidth and low latency for data transfer and model synchronization (e.g., during distributed training). Consistent and reliable connectivity for data access.
  • 29. Enabling Tools and Techniques
  • 30. LinuxFoundation AI Landscape Distributed Training Kubeflow Training Operator Pytorch DDP Torchx Tensorflow Distributed So on … General Orchestratio n Kubernetes Volcana Armada Kuberay Nvidia Nemo Yunikorn Kueue So on.. ML Serving Kserve Seldon VLLM So on.. CI/CD KubeFlow Pipeline ML FLow TFX BentoML MLRun so on ... Data Science Juypter Kubeflow Notebook Pytorch TensorFlow Apache Zeeplin so on .. Workload Observabilit y Prometheus Graphana InfluxDB Open Telemetery so on ... Auto ML Hyperopt Optuna Kubeflow Katib NNI so on ... Governance & Policy Kyverno Optuna OPA/ Gateway so on ... Data Architecture Clickhouse Apache Pinot Apache Druid Cassandra Hadoop HDFS Apache HBASE Apache Spark Apache Flink Apache Pulsar Vektor Database Milvus Chroma Quadrant Pinecone LLM Observabilit y Trulens Langfuse OpenLLMetry Distributed Training Kubeflow Training Operator Pytorch DDP Torchx Tensorflow Distributed OpenMPI Deepspeed Megatron
  • 31. CHALLENGES FOR CLOUD NATIVE ARTIFICIAL INTELLIGENCE The typical ML pipeline is comprised of: • Data Preparation (collection, cleaning/pre-processing, feature engineering) • Model Training (model selection, architecture, hyperparameter tuning) • CI/CD, Model Registry (storage) • Model Serving • Observability (usage load, model drift, security)
  • 32. Benefits of Kubernetes for ML REPEATABILITY PIPELINE PORTABILITY SCALING
  • 33. Right Tools for ML/AI Jobs GPU Huge amount of core Good for Lightweight task Design for graphics computation task CPU Small amount of Core Good for Heavy Task Design for Common Computation
  • 36. How GPUs are Actually Used Training environment Notebooks MLOps pipelines Data processing Tests Inference environment ML model serving Online operations Data pre-processing DPUs
  • 37. AI Landscape & Ecosystem Modeling Deployment Versioning Orchestration Compute Data How Infrastructure is Needed How Much Data Scientist Care
  • 38. AI Landscape & Ecosystem
  • 40. Summary What is LF , CNCF and Kubecon2024 Paris Cloud Native Artifical Intelligence What is LLM ? Multi Model LLM Demo on Kubernetes Cloud Native Production Ready AI Platform Components
  • 42. Q & A Thank you