Submit Search
Upload
Productionizing Predictive Analytics using the Rendezvous Architecture - for Architects
•
0 likes
•
101 views
D
danielschulz2005
Follow
Productionizing Predictive Analytics using the Rendezvous Architecture - for Architects
Read less
Read more
Data & Analytics
Report
Share
Report
Share
1 of 26
Download now
Download to read offline
Recommended
David Long Keynote on Beyond MBSE Looking Towards the Next Evolution in Syste...
David Long Keynote on Beyond MBSE Looking Towards the Next Evolution in Syste...
EnergyTech2015
Introduction to Modern Software Architecture
Introduction to Modern Software Architecture
Jérôme Kehrli
Continuous Engineering with IBM Rational RELM
Continuous Engineering with IBM Rational RELM
gjuljo
From catalogues to models: transitioning from existing requirements technique...
From catalogues to models: transitioning from existing requirements technique...
James Towers
Oslc case study (poc results) v1.1
Oslc case study (poc results) v1.1
Joseph Lopez, M.ISM
Architecture vs. Design vs. Agile: What’s the Answer?
Architecture vs. Design vs. Agile: What’s the Answer?
TechWell
Modern software architect post the agile wave
Modern software architect post the agile wave
Niels Bech Nielsen
Ch01 SE
Ch01 SE
mahirazainab
Recommended
David Long Keynote on Beyond MBSE Looking Towards the Next Evolution in Syste...
David Long Keynote on Beyond MBSE Looking Towards the Next Evolution in Syste...
EnergyTech2015
Introduction to Modern Software Architecture
Introduction to Modern Software Architecture
Jérôme Kehrli
Continuous Engineering with IBM Rational RELM
Continuous Engineering with IBM Rational RELM
gjuljo
From catalogues to models: transitioning from existing requirements technique...
From catalogues to models: transitioning from existing requirements technique...
James Towers
Oslc case study (poc results) v1.1
Oslc case study (poc results) v1.1
Joseph Lopez, M.ISM
Architecture vs. Design vs. Agile: What’s the Answer?
Architecture vs. Design vs. Agile: What’s the Answer?
TechWell
Modern software architect post the agile wave
Modern software architect post the agile wave
Niels Bech Nielsen
Ch01 SE
Ch01 SE
mahirazainab
Cp Repton
Cp Repton
guestea711d0
Future Facilities 6SigmaDCX Overview HD
Future Facilities 6SigmaDCX Overview HD
Robert Schmidt
Designing for the Cloud Tutorial - QCon SF 2009
Designing for the Cloud Tutorial - QCon SF 2009
Stuart Charlton
Avoiding Cloud Computing Planning & Implementation Failure
Avoiding Cloud Computing Planning & Implementation Failure
Nathaniel Payne
Successfully Integrating MBSE Data Without Replication Using OSLC
Successfully Integrating MBSE Data Without Replication Using OSLC
Joseph Lopez, M.ISM
A Software Engineering Perspective on SDN Programmability
A Software Engineering Perspective on SDN Programmability
Felipe Alencar
Migrating to Cloud: Inhouse Hadoop to Databricks (3)
Migrating to Cloud: Inhouse Hadoop to Databricks (3)
Knoldus Inc.
Modeling and Forecasting – Effective Baselines for Capacity Management
Modeling and Forecasting – Effective Baselines for Capacity Management
Precisely
Modern Agile Software Architecture
Modern Agile Software Architecture
Kannan Durairaj
A Model-Driven Approach to Support Cloud Migration Process- A Language Infras...
A Model-Driven Approach to Support Cloud Migration Process- A Language Infras...
Mahdi_Fahmideh
Show Me the Money: Connecting Performance Engineering to Real Business Results
Show Me the Money: Connecting Performance Engineering to Real Business Results
Correlsense
PROSTEP Api for the Digital Thread
PROSTEP Api for the Digital Thread
Joseph Lopez, M.ISM
Adam boczek 2015 agile architecture in 10 steps v1.0
Adam boczek 2015 agile architecture in 10 steps v1.0
iasaglobal
Monolith to serverless service based architectures in the enterprise
Monolith to serverless service based architectures in the enterprise
Sameh Deabes
Productionizing Predictive Analytics using the Rendezvous Architecture - for ...
Productionizing Predictive Analytics using the Rendezvous Architecture - for ...
danielschulz2005
Multi-core Real-time Simulation of High-Fidelity Vehicle Models using Open St...
Multi-core Real-time Simulation of High-Fidelity Vehicle Models using Open St...
Modelon
Production machine learning: Managing models, workflows and risk at scale
Production machine learning: Managing models, workflows and risk at scale
Alex Housley
Machine Learning Models: From Research to Production 6.13.18
Machine Learning Models: From Research to Production 6.13.18
Cloudera, Inc.
[DSC Europe 22] Reproducibility and Versioning of ML Systems - Spela Poklukar
[DSC Europe 22] Reproducibility and Versioning of ML Systems - Spela Poklukar
DataScienceConferenc1
Confluent Partner Tech Talk with QLIK
Confluent Partner Tech Talk with QLIK
confluent
Victor Chang: Cloud computing business framework
Victor Chang: Cloud computing business framework
CBOD ANR project U-PSUD
Machine Learning Model Deployment: Strategy to Implementation
Machine Learning Model Deployment: Strategy to Implementation
DataWorks Summit
More Related Content
What's hot
Cp Repton
Cp Repton
guestea711d0
Future Facilities 6SigmaDCX Overview HD
Future Facilities 6SigmaDCX Overview HD
Robert Schmidt
Designing for the Cloud Tutorial - QCon SF 2009
Designing for the Cloud Tutorial - QCon SF 2009
Stuart Charlton
Avoiding Cloud Computing Planning & Implementation Failure
Avoiding Cloud Computing Planning & Implementation Failure
Nathaniel Payne
Successfully Integrating MBSE Data Without Replication Using OSLC
Successfully Integrating MBSE Data Without Replication Using OSLC
Joseph Lopez, M.ISM
A Software Engineering Perspective on SDN Programmability
A Software Engineering Perspective on SDN Programmability
Felipe Alencar
Migrating to Cloud: Inhouse Hadoop to Databricks (3)
Migrating to Cloud: Inhouse Hadoop to Databricks (3)
Knoldus Inc.
Modeling and Forecasting – Effective Baselines for Capacity Management
Modeling and Forecasting – Effective Baselines for Capacity Management
Precisely
Modern Agile Software Architecture
Modern Agile Software Architecture
Kannan Durairaj
A Model-Driven Approach to Support Cloud Migration Process- A Language Infras...
A Model-Driven Approach to Support Cloud Migration Process- A Language Infras...
Mahdi_Fahmideh
Show Me the Money: Connecting Performance Engineering to Real Business Results
Show Me the Money: Connecting Performance Engineering to Real Business Results
Correlsense
PROSTEP Api for the Digital Thread
PROSTEP Api for the Digital Thread
Joseph Lopez, M.ISM
Adam boczek 2015 agile architecture in 10 steps v1.0
Adam boczek 2015 agile architecture in 10 steps v1.0
iasaglobal
Monolith to serverless service based architectures in the enterprise
Monolith to serverless service based architectures in the enterprise
Sameh Deabes
What's hot
(14)
Cp Repton
Cp Repton
Future Facilities 6SigmaDCX Overview HD
Future Facilities 6SigmaDCX Overview HD
Designing for the Cloud Tutorial - QCon SF 2009
Designing for the Cloud Tutorial - QCon SF 2009
Avoiding Cloud Computing Planning & Implementation Failure
Avoiding Cloud Computing Planning & Implementation Failure
Successfully Integrating MBSE Data Without Replication Using OSLC
Successfully Integrating MBSE Data Without Replication Using OSLC
A Software Engineering Perspective on SDN Programmability
A Software Engineering Perspective on SDN Programmability
Migrating to Cloud: Inhouse Hadoop to Databricks (3)
Migrating to Cloud: Inhouse Hadoop to Databricks (3)
Modeling and Forecasting – Effective Baselines for Capacity Management
Modeling and Forecasting – Effective Baselines for Capacity Management
Modern Agile Software Architecture
Modern Agile Software Architecture
A Model-Driven Approach to Support Cloud Migration Process- A Language Infras...
A Model-Driven Approach to Support Cloud Migration Process- A Language Infras...
Show Me the Money: Connecting Performance Engineering to Real Business Results
Show Me the Money: Connecting Performance Engineering to Real Business Results
PROSTEP Api for the Digital Thread
PROSTEP Api for the Digital Thread
Adam boczek 2015 agile architecture in 10 steps v1.0
Adam boczek 2015 agile architecture in 10 steps v1.0
Monolith to serverless service based architectures in the enterprise
Monolith to serverless service based architectures in the enterprise
Similar to Productionizing Predictive Analytics using the Rendezvous Architecture - for Architects
Productionizing Predictive Analytics using the Rendezvous Architecture - for ...
Productionizing Predictive Analytics using the Rendezvous Architecture - for ...
danielschulz2005
Multi-core Real-time Simulation of High-Fidelity Vehicle Models using Open St...
Multi-core Real-time Simulation of High-Fidelity Vehicle Models using Open St...
Modelon
Production machine learning: Managing models, workflows and risk at scale
Production machine learning: Managing models, workflows and risk at scale
Alex Housley
Machine Learning Models: From Research to Production 6.13.18
Machine Learning Models: From Research to Production 6.13.18
Cloudera, Inc.
[DSC Europe 22] Reproducibility and Versioning of ML Systems - Spela Poklukar
[DSC Europe 22] Reproducibility and Versioning of ML Systems - Spela Poklukar
DataScienceConferenc1
Confluent Partner Tech Talk with QLIK
Confluent Partner Tech Talk with QLIK
confluent
Victor Chang: Cloud computing business framework
Victor Chang: Cloud computing business framework
CBOD ANR project U-PSUD
Machine Learning Model Deployment: Strategy to Implementation
Machine Learning Model Deployment: Strategy to Implementation
DataWorks Summit
World Artificial Intelligence Conference Shanghai 2018
World Artificial Intelligence Conference Shanghai 2018
Adam Gibson
Practical machine learning
Practical machine learning
Faizan Javed
Best practices for application migration to public clouds interop presentation
Best practices for application migration to public clouds interop presentation
esebeus
Building a Scalable and reliable open source ML Platform with MLFlow
Building a Scalable and reliable open source ML Platform with MLFlow
GoDataDriven
SDLC MODEL
SDLC MODEL
KOMAL DAHERIYA
Application Darwinism - Why Most Enterprise Apps Will Evolve to the Cloud
Application Darwinism - Why Most Enterprise Apps Will Evolve to the Cloud
Skytap Cloud
Cloud forum-lessons-learned-20110405c-final
Cloud forum-lessons-learned-20110405c-final
Mauricio Godoy
How to Transform Corporate IT into the Driver for Digital Transformation
How to Transform Corporate IT into the Driver for Digital Transformation
Enterprise Management Associates
[DPE Summit] How Improving the Testing Experience Goes Beyond Quality: A Deve...
[DPE Summit] How Improving the Testing Experience Goes Beyond Quality: A Deve...
Roberto Pérez Alcolea
Confluent Partner Tech Talk with SVA
Confluent Partner Tech Talk with SVA
confluent
Apache ® Spark™ MLlib 2.x: How to Productionize your Machine Learning Models
Apache ® Spark™ MLlib 2.x: How to Productionize your Machine Learning Models
Anyscale
Why do the majority of Data Science projects never make it to production?
Why do the majority of Data Science projects never make it to production?
Itai Yaffe
Similar to Productionizing Predictive Analytics using the Rendezvous Architecture - for Architects
(20)
Productionizing Predictive Analytics using the Rendezvous Architecture - for ...
Productionizing Predictive Analytics using the Rendezvous Architecture - for ...
Multi-core Real-time Simulation of High-Fidelity Vehicle Models using Open St...
Multi-core Real-time Simulation of High-Fidelity Vehicle Models using Open St...
Production machine learning: Managing models, workflows and risk at scale
Production machine learning: Managing models, workflows and risk at scale
Machine Learning Models: From Research to Production 6.13.18
Machine Learning Models: From Research to Production 6.13.18
[DSC Europe 22] Reproducibility and Versioning of ML Systems - Spela Poklukar
[DSC Europe 22] Reproducibility and Versioning of ML Systems - Spela Poklukar
Confluent Partner Tech Talk with QLIK
Confluent Partner Tech Talk with QLIK
Victor Chang: Cloud computing business framework
Victor Chang: Cloud computing business framework
Machine Learning Model Deployment: Strategy to Implementation
Machine Learning Model Deployment: Strategy to Implementation
World Artificial Intelligence Conference Shanghai 2018
World Artificial Intelligence Conference Shanghai 2018
Practical machine learning
Practical machine learning
Best practices for application migration to public clouds interop presentation
Best practices for application migration to public clouds interop presentation
Building a Scalable and reliable open source ML Platform with MLFlow
Building a Scalable and reliable open source ML Platform with MLFlow
SDLC MODEL
SDLC MODEL
Application Darwinism - Why Most Enterprise Apps Will Evolve to the Cloud
Application Darwinism - Why Most Enterprise Apps Will Evolve to the Cloud
Cloud forum-lessons-learned-20110405c-final
Cloud forum-lessons-learned-20110405c-final
How to Transform Corporate IT into the Driver for Digital Transformation
How to Transform Corporate IT into the Driver for Digital Transformation
[DPE Summit] How Improving the Testing Experience Goes Beyond Quality: A Deve...
[DPE Summit] How Improving the Testing Experience Goes Beyond Quality: A Deve...
Confluent Partner Tech Talk with SVA
Confluent Partner Tech Talk with SVA
Apache ® Spark™ MLlib 2.x: How to Productionize your Machine Learning Models
Apache ® Spark™ MLlib 2.x: How to Productionize your Machine Learning Models
Why do the majority of Data Science projects never make it to production?
Why do the majority of Data Science projects never make it to production?
Recently uploaded
Call Girls In Dwarka 9654467111 Escorts Service
Call Girls In Dwarka 9654467111 Escorts Service
Sapana Sha
PKS-TGC-1084-630 - Stage 1 Proposal.pptx
PKS-TGC-1084-630 - Stage 1 Proposal.pptx
Pramod Kumar Srivastava
办理学位证中佛罗里达大学毕业证,UCF成绩单原版一比一
办理学位证中佛罗里达大学毕业证,UCF成绩单原版一比一
F sss
办理学位证纽约大学毕业证(NYU毕业证书)原版一比一
办理学位证纽约大学毕业证(NYU毕业证书)原版一比一
fhwihughh
Amazon TQM (2) Amazon TQM (2)Amazon TQM (2).pptx
Amazon TQM (2) Amazon TQM (2)Amazon TQM (2).pptx
Abdelrhman abooda
Brighton SEO | April 2024 | Data Storytelling
Brighton SEO | April 2024 | Data Storytelling
Neil Barnes
Building on a FAIRly Strong Foundation to Connect Academic Research to Transl...
Building on a FAIRly Strong Foundation to Connect Academic Research to Transl...
Jack DiGiovanna
How we prevented account sharing with MFA
How we prevented account sharing with MFA
Andrei Kaleshka
RadioAdProWritingCinderellabyButleri.pdf
RadioAdProWritingCinderellabyButleri.pdf
gstagge
VIP Call Girls Service Charbagh { Lucknow Call Girls Service 9548273370 } Boo...
VIP Call Girls Service Charbagh { Lucknow Call Girls Service 9548273370 } Boo...
Call Girls In Delhi Whatsup 9873940964 Enjoy Unlimited Pleasure
代办国外大学文凭《原版美国UCLA文凭证书》加州大学洛杉矶分校毕业证制作成绩单修改
代办国外大学文凭《原版美国UCLA文凭证书》加州大学洛杉矶分校毕业证制作成绩单修改
atducpo
Decoding Loan Approval: Predictive Modeling in Action
Decoding Loan Approval: Predictive Modeling in Action
Boston Institute of Analytics
Schema on read is obsolete. Welcome metaprogramming..pdf
Schema on read is obsolete. Welcome metaprogramming..pdf
Lars Albertsson
Beautiful Sapna Vip Call Girls Hauz Khas 9711199012 Call /Whatsapps
Beautiful Sapna Vip Call Girls Hauz Khas 9711199012 Call /Whatsapps
sapnasaifi408
From idea to production in a day – Leveraging Azure ML and Streamlit to build...
From idea to production in a day – Leveraging Azure ML and Streamlit to build...
Florian Roscheck
1:1定制(UQ毕业证)昆士兰大学毕业证成绩单修改留信学历认证原版一模一样
1:1定制(UQ毕业证)昆士兰大学毕业证成绩单修改留信学历认证原版一模一样
vhwb25kk
20240419 - Measurecamp Amsterdam - SAM.pdf
20240419 - Measurecamp Amsterdam - SAM.pdf
Human37
9654467111 Call Girls In Munirka Hotel And Home Service
9654467111 Call Girls In Munirka Hotel And Home Service
Sapana Sha
Deep Generative Learning for All - The Gen AI Hype (Spring 2024)
Deep Generative Learning for All - The Gen AI Hype (Spring 2024)
Universitat Politècnica de Catalunya
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptx
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptx
thyngster
Recently uploaded
(20)
Call Girls In Dwarka 9654467111 Escorts Service
Call Girls In Dwarka 9654467111 Escorts Service
PKS-TGC-1084-630 - Stage 1 Proposal.pptx
PKS-TGC-1084-630 - Stage 1 Proposal.pptx
办理学位证中佛罗里达大学毕业证,UCF成绩单原版一比一
办理学位证中佛罗里达大学毕业证,UCF成绩单原版一比一
办理学位证纽约大学毕业证(NYU毕业证书)原版一比一
办理学位证纽约大学毕业证(NYU毕业证书)原版一比一
Amazon TQM (2) Amazon TQM (2)Amazon TQM (2).pptx
Amazon TQM (2) Amazon TQM (2)Amazon TQM (2).pptx
Brighton SEO | April 2024 | Data Storytelling
Brighton SEO | April 2024 | Data Storytelling
Building on a FAIRly Strong Foundation to Connect Academic Research to Transl...
Building on a FAIRly Strong Foundation to Connect Academic Research to Transl...
How we prevented account sharing with MFA
How we prevented account sharing with MFA
RadioAdProWritingCinderellabyButleri.pdf
RadioAdProWritingCinderellabyButleri.pdf
VIP Call Girls Service Charbagh { Lucknow Call Girls Service 9548273370 } Boo...
VIP Call Girls Service Charbagh { Lucknow Call Girls Service 9548273370 } Boo...
代办国外大学文凭《原版美国UCLA文凭证书》加州大学洛杉矶分校毕业证制作成绩单修改
代办国外大学文凭《原版美国UCLA文凭证书》加州大学洛杉矶分校毕业证制作成绩单修改
Decoding Loan Approval: Predictive Modeling in Action
Decoding Loan Approval: Predictive Modeling in Action
Schema on read is obsolete. Welcome metaprogramming..pdf
Schema on read is obsolete. Welcome metaprogramming..pdf
Beautiful Sapna Vip Call Girls Hauz Khas 9711199012 Call /Whatsapps
Beautiful Sapna Vip Call Girls Hauz Khas 9711199012 Call /Whatsapps
From idea to production in a day – Leveraging Azure ML and Streamlit to build...
From idea to production in a day – Leveraging Azure ML and Streamlit to build...
1:1定制(UQ毕业证)昆士兰大学毕业证成绩单修改留信学历认证原版一模一样
1:1定制(UQ毕业证)昆士兰大学毕业证成绩单修改留信学历认证原版一模一样
20240419 - Measurecamp Amsterdam - SAM.pdf
20240419 - Measurecamp Amsterdam - SAM.pdf
9654467111 Call Girls In Munirka Hotel And Home Service
9654467111 Call Girls In Munirka Hotel And Home Service
Deep Generative Learning for All - The Gen AI Hype (Spring 2024)
Deep Generative Learning for All - The Gen AI Hype (Spring 2024)
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptx
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptx
Productionizing Predictive Analytics using the Rendezvous Architecture - for Architects
1.
Delivery Excellence: Love is
in the Air – Rendezvous Architecture for Analytics in Production Munich, October 25, 2019, Daniel Schulz
2.
AAC_RendezvousArchitecture.pptx Love is in
the Air What is Rendezvous Architecture and why is it Important?
3.
AAC_RendezvousArchitecture.pptx Why is Rendezvous
Architecture Important? • An Event-driven Architecture (EDA) • Applies Event stream processing style for Supervised Learning tasks • Solves common obstacles in productionizing Machine Learning Models for both • Initial Go-Live & • Updates in production • Analogous to the Ivory Tower Effect in Enterprise Architecture • Like the architect needs feedback from production systems to improve this & future architectures • Data Scientists need production feedback from their ML Models in order to improve the upcoming versions of it © 2019 Daniel Schulz. All rights reserved. 3 Taking Machine Learning Models to Production is Challenging
4.
AAC_RendezvousArchitecture.pptx Traditional Green/Blue Deployments
– a Load Balancer Forwards any Request to Exactly One Machine Learning Model © 2019 Daniel Schulz. All rights reserved. 4 The Reverse Proxy Enables Green/Blue Deployments Only Source & image courtesy: [MLL]
5.
AAC_RendezvousArchitecture.pptx Load Balanced, Parallelized Models Enable
Multiple ML Models at Once
6.
AAC_RendezvousArchitecture.pptx Multiple Machine Learning
Models in Parallel – Adding a Message Queue Enables Concurrency of Models © 2019 Daniel Schulz. All rights reserved. 6 But which Prediction to Choose in the End? Source & image courtesy: [MLL]
7.
AAC_RendezvousArchitecture.pptx Multiple Machine Learning
Models in Parallel – Adding a Message Queue Enables Concurrency of Models • Shortcomings: • Somewhat challenging various models for reference against one another • No comparing many model’s accuracies against one another • Continuous improvement in Data Science resp. DataOps team unlikely due to lack of feedback • Need to add return address (as an URI) and boolean flag to whether to return anything at all in the queue – for the consumer to respond to the original (HTTP) request, which might be completed then • Persistent Message Queues tend to be slower than pure in-memory ones; Spark Streaming working in micro-batches might have higher latencies as well © 2019 Daniel Schulz. All rights reserved. 7 But which Prediction to Choose in the End?
8.
AAC_RendezvousArchitecture.pptx Rendezvous Architecture Stop-back Request Guarantees, Parallelized,
Comparable, Rapid Model Updates, QoS Guarantees, etc.
9.
AAC_RendezvousArchitecture.pptx The ML Model’s
Rendezvous – a Scoring Stream Collects Various Predictions & Returns them to the Original Request’s Client © 2019 Daniel Schulz. All rights reserved. 9 Many Models – One Final Prediction Source & image courtesy: [MLL]
10.
AAC_RendezvousArchitecture.pptx Stateless Models are
Easier to Replicate – but Data Augmentation Might be Helpful or Necessary © 2019 Daniel Schulz. All rights reserved. 10 Any Additional Piece of or State Information Helpful to our Models Shall be Added in One Central Place Source & image courtesy: [MLL]
11.
AAC_RendezvousArchitecture.pptx The Decoy Model
– Collect Production Data for Debugging, Optimizations and Reproducibility © 2019 Daniel Schulz. All rights reserved. 11 The “Unit Tests” of AI Models in Production is Real-World Data Source & image courtesy: [MLL]
12.
AAC_RendezvousArchitecture.pptx Discussion when to
Add External Information © 2019 Daniel Schulz. All rights reserved. 12 No Augmentation ▪ Models only receive request information ▪ More reliable for technical failure ▪ More overhead when many models fetch the same data – use caches here
13.
AAC_RendezvousArchitecture.pptx Discussion when to
Add External Information © 2019 Daniel Schulz. All rights reserved. 13 No Augmentation ▪ Models only receive request information ▪ More reliable for technical failure ▪ More overhead when many models fetch the same data – use caches here All Augmentation in One Central Place ▪ Models receive all complete information ▪ Ideal case for reproducibility – as the “Decoy Model” stores all information for debugging and to explain predictions later-on ▪ Usually faster due to smaller overhead → Best Practice by Ellen Friedman & Ted Dunning, Chapter 3, sub-section “Stateful Models” in [MLL]
14.
AAC_RendezvousArchitecture.pptx Add Metrics &
the Canary Model for Monitoring & Optimizations © 2019 Daniel Schulz. All rights reserved. 14 Metrics are Crucial to Compare Many Models Against One Another & Hence Supports Accuracy Improvements Source & image courtesy: [MLL]
15.
AAC_RendezvousArchitecture.pptx Add Metrics &
the Canary Model for Monitoring & Optimizations © 2019 Daniel Schulz. All rights reserved. 15 Metrics ▪ Metrics monitor all model’s predictions and compares their performances ▪ Hence, we are able to judge challenging model’s performances compared to the incumbent model’s ones ▪ Metrics are helpful to detect outliers in predictions – • e.g. Adversarial Images, where models might predict obscure classifications • e.g. detect swayed models, shift in input data’s distributions, etc. ▪ Both technological SLA, timing & AI metrics • SLA metrics: latency, throughput, etc. • Timing metrics: computation time in threads, time for requests by source/endpoint, etc. • AI metrics: accuracy, error metrics, AUC, F-statistics, etc.
16.
AAC_RendezvousArchitecture.pptx Add Metrics &
the Canary Model for Monitoring & Optimizations © 2019 Daniel Schulz. All rights reserved. 16 Metrics ▪ Metrics monitor all model’s predictions and compares their performances ▪ Hence, we are able to judge challenging model’s performances compared to the incumbent model’s ones ▪ Metrics are helpful to detect outliers in predictions – • e.g. Adversarial Images, where models might predict obscure classifications • e.g. detect swayed models, shift in input data’s distributions, etc. ▪ Both technological SLA, timing & AI metrics • SLA metrics: latency, throughput, etc. • Timing metrics: computation time in threads, time for requests by source/endpoint, etc. • AI metrics: accuracy, error metrics, AUC, F-statistics, etc. The Canary Model ▪ Also a Best Practice by [MLL] for finding anomalies ▪ Is a rather dated model that keeps predicting to compare the newer models’ predictions with it – to help detect how production-ready they really are ▪ The difference in Canary Model and later ones is proof of progress – or lack thereof for the Data Scientists
17.
AAC_RendezvousArchitecture.pptx Rendezvous Architecture –
a Mixture of Models in Harmony • Advantages: • Model “warm-up” in production-like environments • Switch models in an instant – un-deploy & deploy AI models swiftly • Introduce time guarantees: all models work in parallel like Cassandra queries • Mix simple, technically robust (not failing) models along w/ more sophisticated ones, which might break suddenly • Incumbent vs challenging – collect raw data and performance metrics for various models (e.g. XGBoost vs Random Forests; e.g. Linear Model vs SVM; e.g. PCA vs T-SNE) and differing versions in model streams (version 0.1, 0.2, …) • Backstop: when taking too long, a simpler, less robust model may answer as a backstop for more complex, more sophisticated models; the same applies when longer-term performance metrics might indicate another model would perform better © 2019 Daniel Schulz. All rights reserved. 17 My Suggestion for Reliable AI Systems Due to…
18.
AAC_RendezvousArchitecture.pptx Résumé Resilient, Extendable & Production-ready
Architecture for Predictive Analytics
19.
AAC_RendezvousArchitecture.pptx Resumé on Rendezvous
Architecture © 2019 Daniel Schulz. All rights reserved. 19 Limitations ▪ Is no silver bullet – does not solve all logistical obstacles in AI projects ▪ Focusses on production-side architecture – development, test, QA and LTE environments might benefit from it ▪ Focus on technological, software architecture – does not cover ML Metrics, Hyperparameter Tuning, etc. ▪ Latencies increased a bit by Message-Queue-dependency ▪ I am not aware of any implementation as of now – neither OSS, nor to purchase, nor Cloud-based
20.
AAC_RendezvousArchitecture.pptx Resumé on Rendezvous
Architecture © 2019 Daniel Schulz. All rights reserved. 20 Major Advantages ▪ Manage multiple models in production and alike environments ▪ Test-drive incumbent & challenging models against one another ▪ Reproducibility & transparency of model’s predictions ▪ Solves common obstacles in productionizing Machine Learning Models for both • Initial Go-Live & • Updates in production
21.
AAC_RendezvousArchitecture.pptx Resumé on Rendezvous
Architecture © 2019 Daniel Schulz. All rights reserved. 21 Major Advantages ▪ Manage multiple models in production and alike environments ▪ Test-drive incumbent & challenging models against one another ▪ Reproducibility & transparency of model’s predictions ▪ Solves common obstacles in productionizing Machine Learning Models for both • Initial Go-Live & • Updates in production Minor Benefits ▪ Collect real-world data for future development ▪ Establish baseline performance values for Predictive Analytics ▪ Rapid development due to default, fallback models ▪ Latency guarantees for model predictions resilient, robust Predictive models in modern Agile & DevOps projects
22.
AAC_RendezvousArchitecture.pptx Rendezvous Architecture is the
Modern Bedrock of Robust Predictive models for Today’s Agile & DevOps Projects
23.
AAC_RendezvousArchitecture.pptx Thank You for
Your Attention © 2019 Daniel Schulz. All rights reserved. 23 Please Feel Free to Ask any Open Questions, Suggestions or Voice Your Opinion…
24.
A global leader
in consulting, technology services and digital transformation, Capgemini is at the forefront of innovation to address the entire breadth of clients’ opportunities in the evolving world of cloud, digital and platforms. Building on its strong 50-year heritage and deep industry-specific expertise, Capgemini enables organizations to realize their business ambitions through an array of services from strategy to operations. Capgemini is driven by the conviction that the business value of technology comes from and through people. It is a multicultural company of over 200,000 team members in more than 40 countries. The Group reported 2018 global revenues of EUR 13.2 billion. About Capgemini Learn more about us at www.capgemini.com This presentation contains information that may be privileged or confidential and is the property of the Capgemini Group. Copyright © 2019 Daniel Schulz. All rights reserved. People matter, results count.
25.
AAC_RendezvousArchitecture.pptx Source & Image
Courtesy from Book “Machine Learning Logistics” © 2019 Daniel Schulz. All rights reserved. 25 “Machine Learning Logistics” by Ellen Friedman & Ted Dunning ▪ Authors: Ellen Friedman & Ted Dunning ▪ Publisher: O'Reilly Media, Inc. ▪ Release Date: October 2017 ▪ ISBN: 9 7814 9199 7611 ▪ Picture source ID: MLL
26.
AAC_RendezvousArchitecture.pptx Source from Book
“Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition” © 2019 Daniel Schulz. All rights reserved. 26 “Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow” by Aurélien Géron ▪ Author: Aurélien Géron ▪ Publisher: O'Reilly Media, Inc. ▪ Release Date: October 2019 ▪ ISBN: 9 7814 9203 2649
Download now