SlideShare a Scribd company logo
1 of 11
Download to read offline
Env2Vec: Accelerating VNF
Testing with Deep Learning
Guangyuan Piao, Pat Nicholson, Diego Lugones
Nokia Bell Labs, Dublin, Ireland
The 15th European Conference on Computer Systems, 30/04/2020
© 2019 Nokia2
Introduction
Current challenges for network engineers adopting modern DevOps:
• Manage hundreds of distributed testbeds each running the new software build
• Test new virtual network function (VNF) builds in shorter development cycles
• Assure carrier-grade quality
Objective:
• Use deep learning (DL) techniques to accelerate VNF testing
• Automate the diagnosis of software defects and anomalous builds
• Point out testing engineers to problematic metrics
Introduction → Problem → Proposal → Evaluation → Conclusions
© 2019 Nokia3
Problem: Environments must be Considered!
Introduction → Problem → Proposal → Evaluation → Conclusions
125 different models
Varying accuracy
© 2019 Nokia4
Proposed Approach: Env2Vec Anomaly Detection
1. Learn a universal resource characterization model
2. Execute the model in real time
• to detect deviations between the inferred value (𝒚 𝒑
′
) and the actual observed RU of the VNF
• Flag anomaly if there is a significant deviation (larger than γ x standard deviations)
𝒚 𝒑
′
GRUs
FNNs
Lookup Tables
(Embeddings)
characterization
model with DL
Ruhistory (Resource Usage)
CF (contextual features)
• Workload metrics (WM)
• Performance metrics (PM)
EM (environment metadata)
• Testbed, build type, test cases etc.
Inferred resource
usage (RU) (e.g., CPU)
Introduction → Problem → Proposal → Evaluation → Conclusions
© 2019 Nokia5
System Overview
1. Testbed data collection
2. Model training (daily)
3. Prediction pipeline
4. Raising alarms
5. Updating the model
Introduction → Problem → Proposal → Evaluation → Conclusions
© 2019 Nokia6
Evaluation: Carrier-grade VNF Testing
Dataset
• 600+ real-world testing environments
• 125 build chains (testbed, build type, SUT, and test case)
• 400,000 timesteps/data points measured at 15 minute intervals
Evaluation metrics
• Number of generated alarms
• True and false alarm rate ( AT and AF )
• System with a higher number of generated alarms and higher AT is better
Introduction → Problem → Proposal → Evaluation → Conclusions
© 2019 Nokia7
Evaluation: Automating Anomaly Detection in New Builds
Compared approaches
• HTM-AD: AD based on RU time series only
• Ridge: linear regression trained on each environment
• Ridge(ts): considers RU history as features
• RFNNall: Env2Vec without using embeddings
Observations
• Either the priority is raising (1) more performance
problems automatically or (2) highly accurate alarms,
• Env2Vec provides the best performance with γ = 1or 2
Introduction → Problem → Proposal → Evaluation → Conclusions
© 2019 Nokia8
Evaluation: Reuse Embeddings in Unseen Environment
Env2Vec
• Reuses learned embeddings (capturing similarity of environments)
• Construct environment embedding for a new unseen environment
• Can detect performance problems better in unseen test executions
lack of data for each unseen env.
Introduction → Problem → Proposal → Evaluation → Conclusions
© 2019 Nokia9
Conclusions
• We proposed Env2Vec, which creates a universal easy-to-maintain model
• Env2Vec automatically detects defects and bugs in new software builds by identifying
performance degradation and informing the engineer
• Can be extrapolated to previously unseen environments
• Achieves accuracy between 86.2%-100%, while reducing the false alarm rate by 20.9%-
38.1% compared to other approaches
Introduction → Problem → Proposal → Evaluation → Conclusions
Q & A
Contact information:
Guangyuan Piao: guangyuan.piao@nokia-bell-labs.com
Pat Nicholson: pat.nicholson@nokia-bell-labs.com
Diego Lugones: diego.lugones@nokia-bell-labs.com
Env2Vec: Accelerating VNF Testing with Deep Learning

More Related Content

What's hot

Use Models for Extending IEEE 1687 to Analog Test
Use Models for Extending IEEE 1687 to Analog TestUse Models for Extending IEEE 1687 to Analog Test
Use Models for Extending IEEE 1687 to Analog TestPete Sarson, PH.D
 
LifeGuard: Precision Projectile Tracking by Randy S. Roberts, LLNL Electrical...
LifeGuard: Precision Projectile Tracking by Randy S. Roberts, LLNL Electrical...LifeGuard: Precision Projectile Tracking by Randy S. Roberts, LLNL Electrical...
LifeGuard: Precision Projectile Tracking by Randy S. Roberts, LLNL Electrical...Industrial Partnerships Office
 
Surveillance scene classification using machine learning
Surveillance scene classification using machine learningSurveillance scene classification using machine learning
Surveillance scene classification using machine learningUtkarsh Contractor
 
SuMo-SS: Submodular Optimization Sensor Scattering for Deploying Sensor Netwo...
SuMo-SS: Submodular Optimization Sensor Scattering for Deploying Sensor Netwo...SuMo-SS: Submodular Optimization Sensor Scattering for Deploying Sensor Netwo...
SuMo-SS: Submodular Optimization Sensor Scattering for Deploying Sensor Netwo...Komei Sugiura
 
Detection of fire through Camera
Detection of fire through CameraDetection of fire through Camera
Detection of fire through CameraShivam Singhal
 
FIRE DETECTION USING VIDEO ANALYTICS
FIRE DETECTION USING VIDEO ANALYTICSFIRE DETECTION USING VIDEO ANALYTICS
FIRE DETECTION USING VIDEO ANALYTICSGaurav Deshmukh
 
2022_03_28 "Raspberry Pi Applications in Electronics and Control Laboratories"
2022_03_28 "Raspberry Pi Applications in Electronics and Control Laboratories"2022_03_28 "Raspberry Pi Applications in Electronics and Control Laboratories"
2022_03_28 "Raspberry Pi Applications in Electronics and Control Laboratories"eMadrid network
 
Combining out - of - band monitoring with AI and big data for datacenter aut...
Combining out - of - band monitoring with AI and big data  for datacenter aut...Combining out - of - band monitoring with AI and big data  for datacenter aut...
Combining out - of - band monitoring with AI and big data for datacenter aut...Ganesan Narayanasamy
 
Container orchestration in geo-distributed cloud computing platforms
Container orchestration in geo-distributed cloud computing platformsContainer orchestration in geo-distributed cloud computing platforms
Container orchestration in geo-distributed cloud computing platformsFogGuru MSCA Project
 
AnupVMathur
AnupVMathurAnupVMathur
AnupVMathuranupmath
 

What's hot (11)

Use Models for Extending IEEE 1687 to Analog Test
Use Models for Extending IEEE 1687 to Analog TestUse Models for Extending IEEE 1687 to Analog Test
Use Models for Extending IEEE 1687 to Analog Test
 
LifeGuard: Precision Projectile Tracking by Randy S. Roberts, LLNL Electrical...
LifeGuard: Precision Projectile Tracking by Randy S. Roberts, LLNL Electrical...LifeGuard: Precision Projectile Tracking by Randy S. Roberts, LLNL Electrical...
LifeGuard: Precision Projectile Tracking by Randy S. Roberts, LLNL Electrical...
 
Surveillance scene classification using machine learning
Surveillance scene classification using machine learningSurveillance scene classification using machine learning
Surveillance scene classification using machine learning
 
SuMo-SS: Submodular Optimization Sensor Scattering for Deploying Sensor Netwo...
SuMo-SS: Submodular Optimization Sensor Scattering for Deploying Sensor Netwo...SuMo-SS: Submodular Optimization Sensor Scattering for Deploying Sensor Netwo...
SuMo-SS: Submodular Optimization Sensor Scattering for Deploying Sensor Netwo...
 
EMPhASIS - Implementation
EMPhASIS - ImplementationEMPhASIS - Implementation
EMPhASIS - Implementation
 
Detection of fire through Camera
Detection of fire through CameraDetection of fire through Camera
Detection of fire through Camera
 
FIRE DETECTION USING VIDEO ANALYTICS
FIRE DETECTION USING VIDEO ANALYTICSFIRE DETECTION USING VIDEO ANALYTICS
FIRE DETECTION USING VIDEO ANALYTICS
 
2022_03_28 "Raspberry Pi Applications in Electronics and Control Laboratories"
2022_03_28 "Raspberry Pi Applications in Electronics and Control Laboratories"2022_03_28 "Raspberry Pi Applications in Electronics and Control Laboratories"
2022_03_28 "Raspberry Pi Applications in Electronics and Control Laboratories"
 
Combining out - of - band monitoring with AI and big data for datacenter aut...
Combining out - of - band monitoring with AI and big data  for datacenter aut...Combining out - of - band monitoring with AI and big data  for datacenter aut...
Combining out - of - band monitoring with AI and big data for datacenter aut...
 
Container orchestration in geo-distributed cloud computing platforms
Container orchestration in geo-distributed cloud computing platformsContainer orchestration in geo-distributed cloud computing platforms
Container orchestration in geo-distributed cloud computing platforms
 
AnupVMathur
AnupVMathurAnupVMathur
AnupVMathur
 

Similar to Env2Vec: Accelerating VNF Testing with Deep Learning

Distributed Checkpointing on an Enterprise Desktop Grid
Distributed Checkpointing on an Enterprise Desktop GridDistributed Checkpointing on an Enterprise Desktop Grid
Distributed Checkpointing on an Enterprise Desktop Gridbrent.wilson
 
Lessons learned so far in operationalizing NFV
Lessons learned so far in operationalizing NFVLessons learned so far in operationalizing NFV
Lessons learned so far in operationalizing NFVJames Crawshaw
 
Swimming upstream: OPNFV Doctor project case study
Swimming upstream: OPNFV Doctor project case studySwimming upstream: OPNFV Doctor project case study
Swimming upstream: OPNFV Doctor project case studyOPNFV
 
Challenges in Practicing High Frequency Releases in Cloud Environments
Challenges in Practicing High Frequency Releases in Cloud Environments Challenges in Practicing High Frequency Releases in Cloud Environments
Challenges in Practicing High Frequency Releases in Cloud Environments Liming Zhu
 
SAND: A Fault-Tolerant Streaming Architecture for Network Traffic Analytics
SAND: A Fault-Tolerant Streaming Architecture for Network Traffic AnalyticsSAND: A Fault-Tolerant Streaming Architecture for Network Traffic Analytics
SAND: A Fault-Tolerant Streaming Architecture for Network Traffic AnalyticsQin Liu
 
Benchmarking and Performance on AWS - AWS India Summit 2012
Benchmarking and Performance on AWS - AWS India Summit 2012Benchmarking and Performance on AWS - AWS India Summit 2012
Benchmarking and Performance on AWS - AWS India Summit 2012Amazon Web Services
 
Mobile Reliability Challenges
Mobile Reliability ChallengesMobile Reliability Challenges
Mobile Reliability ChallengesBob Binder
 
Ensemble Launches Major Upgrade to NFV Platform
Ensemble Launches Major Upgrade to NFV PlatformEnsemble Launches Major Upgrade to NFV Platform
Ensemble Launches Major Upgrade to NFV PlatformADVA
 
Application of machine learning and cognitive computing in intrusion detectio...
Application of machine learning and cognitive computing in intrusion detectio...Application of machine learning and cognitive computing in intrusion detectio...
Application of machine learning and cognitive computing in intrusion detectio...Mahdi Hosseini Moghaddam
 
Improving performance and efficiency with Network Virtualization Overlays
Improving performance and efficiency with Network Virtualization OverlaysImproving performance and efficiency with Network Virtualization Overlays
Improving performance and efficiency with Network Virtualization OverlaysAdam Johnson
 
Software defined networking(sdn) vahid sadri
Software defined networking(sdn) vahid sadriSoftware defined networking(sdn) vahid sadri
Software defined networking(sdn) vahid sadriVahid Sadri
 
Networking Concepts Lesson 13 - Troubleshooting - Eric Vanderburg
Networking Concepts Lesson 13 - Troubleshooting - Eric VanderburgNetworking Concepts Lesson 13 - Troubleshooting - Eric Vanderburg
Networking Concepts Lesson 13 - Troubleshooting - Eric VanderburgEric Vanderburg
 
UVM_Full_Print_n.pptx
UVM_Full_Print_n.pptxUVM_Full_Print_n.pptx
UVM_Full_Print_n.pptxnikitha992646
 
Future Internet: Managing Innovation and Testbed
Future Internet: Managing Innovation and TestbedFuture Internet: Managing Innovation and Testbed
Future Internet: Managing Innovation and TestbedShinji Shimojo
 
How to Operate Kubernetes CI/CD Pipelines at Scale
How to Operate Kubernetes CI/CD Pipelines at ScaleHow to Operate Kubernetes CI/CD Pipelines at Scale
How to Operate Kubernetes CI/CD Pipelines at ScaleDevOps.com
 
Addressing the top 10 challenges of lte epc testing
Addressing the top 10 challenges of lte epc testingAddressing the top 10 challenges of lte epc testing
Addressing the top 10 challenges of lte epc testingAricent
 

Similar to Env2Vec: Accelerating VNF Testing with Deep Learning (20)

01-06 OCRE Test Suite - Fernandes.pdf
01-06 OCRE Test Suite - Fernandes.pdf01-06 OCRE Test Suite - Fernandes.pdf
01-06 OCRE Test Suite - Fernandes.pdf
 
Distributed Checkpointing on an Enterprise Desktop Grid
Distributed Checkpointing on an Enterprise Desktop GridDistributed Checkpointing on an Enterprise Desktop Grid
Distributed Checkpointing on an Enterprise Desktop Grid
 
Lessons learned so far in operationalizing NFV
Lessons learned so far in operationalizing NFVLessons learned so far in operationalizing NFV
Lessons learned so far in operationalizing NFV
 
Swimming upstream: OPNFV Doctor project case study
Swimming upstream: OPNFV Doctor project case studySwimming upstream: OPNFV Doctor project case study
Swimming upstream: OPNFV Doctor project case study
 
Challenges in Practicing High Frequency Releases in Cloud Environments
Challenges in Practicing High Frequency Releases in Cloud Environments Challenges in Practicing High Frequency Releases in Cloud Environments
Challenges in Practicing High Frequency Releases in Cloud Environments
 
First Steps to DevOps
First Steps to DevOpsFirst Steps to DevOps
First Steps to DevOps
 
SAND: A Fault-Tolerant Streaming Architecture for Network Traffic Analytics
SAND: A Fault-Tolerant Streaming Architecture for Network Traffic AnalyticsSAND: A Fault-Tolerant Streaming Architecture for Network Traffic Analytics
SAND: A Fault-Tolerant Streaming Architecture for Network Traffic Analytics
 
Benchmarking and Performance on AWS - AWS India Summit 2012
Benchmarking and Performance on AWS - AWS India Summit 2012Benchmarking and Performance on AWS - AWS India Summit 2012
Benchmarking and Performance on AWS - AWS India Summit 2012
 
Mobile Reliability Challenges
Mobile Reliability ChallengesMobile Reliability Challenges
Mobile Reliability Challenges
 
Ensemble Launches Major Upgrade to NFV Platform
Ensemble Launches Major Upgrade to NFV PlatformEnsemble Launches Major Upgrade to NFV Platform
Ensemble Launches Major Upgrade to NFV Platform
 
Computer Engineer Master Project
Computer Engineer Master ProjectComputer Engineer Master Project
Computer Engineer Master Project
 
Application of machine learning and cognitive computing in intrusion detectio...
Application of machine learning and cognitive computing in intrusion detectio...Application of machine learning and cognitive computing in intrusion detectio...
Application of machine learning and cognitive computing in intrusion detectio...
 
Improving performance and efficiency with Network Virtualization Overlays
Improving performance and efficiency with Network Virtualization OverlaysImproving performance and efficiency with Network Virtualization Overlays
Improving performance and efficiency with Network Virtualization Overlays
 
Software defined networking(sdn) vahid sadri
Software defined networking(sdn) vahid sadriSoftware defined networking(sdn) vahid sadri
Software defined networking(sdn) vahid sadri
 
Networking Concepts Lesson 13 - Troubleshooting - Eric Vanderburg
Networking Concepts Lesson 13 - Troubleshooting - Eric VanderburgNetworking Concepts Lesson 13 - Troubleshooting - Eric Vanderburg
Networking Concepts Lesson 13 - Troubleshooting - Eric Vanderburg
 
UVM_Full_Print_n.pptx
UVM_Full_Print_n.pptxUVM_Full_Print_n.pptx
UVM_Full_Print_n.pptx
 
Future Internet: Managing Innovation and Testbed
Future Internet: Managing Innovation and TestbedFuture Internet: Managing Innovation and Testbed
Future Internet: Managing Innovation and Testbed
 
How to Operate Kubernetes CI/CD Pipelines at Scale
How to Operate Kubernetes CI/CD Pipelines at ScaleHow to Operate Kubernetes CI/CD Pipelines at Scale
How to Operate Kubernetes CI/CD Pipelines at Scale
 
Addressing the top 10 challenges of lte epc testing
Addressing the top 10 challenges of lte epc testingAddressing the top 10 challenges of lte epc testing
Addressing the top 10 challenges of lte epc testing
 
Vinay Singh
Vinay SinghVinay Singh
Vinay Singh
 

More from GUANGYUAN PIAO

Domain-Aware Sentiment Classification with GRUs and CNNs
Domain-Aware Sentiment Classification with GRUs and CNNsDomain-Aware Sentiment Classification with GRUs and CNNs
Domain-Aware Sentiment Classification with GRUs and CNNsGUANGYUAN PIAO
 
A Study of the Similarities of Entity Embeddings Learned from Different Aspec...
A Study of the Similarities of Entity Embeddings Learned from Different Aspec...A Study of the Similarities of Entity Embeddings Learned from Different Aspec...
A Study of the Similarities of Entity Embeddings Learned from Different Aspec...GUANGYUAN PIAO
 
Retweet Prediction with Attention-based Deep Neural Network
Retweet Prediction with Attention-based Deep Neural NetworkRetweet Prediction with Attention-based Deep Neural Network
Retweet Prediction with Attention-based Deep Neural NetworkGUANGYUAN PIAO
 
WISE2017 - Factorization Machines Leveraging Lightweight Linked Open Data-ena...
WISE2017 - Factorization Machines Leveraging Lightweight Linked Open Data-ena...WISE2017 - Factorization Machines Leveraging Lightweight Linked Open Data-ena...
WISE2017 - Factorization Machines Leveraging Lightweight Linked Open Data-ena...GUANGYUAN PIAO
 
Hypertext2017-Leveraging Followee List Memberships for Inferring User Interes...
Hypertext2017-Leveraging Followee List Memberships for Inferring User Interes...Hypertext2017-Leveraging Followee List Memberships for Inferring User Interes...
Hypertext2017-Leveraging Followee List Memberships for Inferring User Interes...GUANGYUAN PIAO
 
ECIR2017-Inferring User Interests for Passive Users on Twitter by Leveraging ...
ECIR2017-Inferring User Interests for Passive Users on Twitter by Leveraging ...ECIR2017-Inferring User Interests for Passive Users on Twitter by Leveraging ...
ECIR2017-Inferring User Interests for Passive Users on Twitter by Leveraging ...GUANGYUAN PIAO
 
EKAW2016 - Interest Representation, Enrichment, Dynamics, and Propagation: A ...
EKAW2016 - Interest Representation, Enrichment, Dynamics, and Propagation: A ...EKAW2016 - Interest Representation, Enrichment, Dynamics, and Propagation: A ...
EKAW2016 - Interest Representation, Enrichment, Dynamics, and Propagation: A ...GUANGYUAN PIAO
 
SEMANTiCS2016 - Exploring Dynamics and Semantics of User Interests for User ...
SEMANTiCS2016 - Exploring Dynamics and Semantics of User Interests for User ...SEMANTiCS2016 - Exploring Dynamics and Semantics of User Interests for User ...
SEMANTiCS2016 - Exploring Dynamics and Semantics of User Interests for User ...GUANGYUAN PIAO
 
UMAP2016EA - Analyzing MOOC Entries of Professionals on LinkedIn for User Mod...
UMAP2016EA - Analyzing MOOC Entries of Professionals on LinkedIn for User Mod...UMAP2016EA - Analyzing MOOC Entries of Professionals on LinkedIn for User Mod...
UMAP2016EA - Analyzing MOOC Entries of Professionals on LinkedIn for User Mod...GUANGYUAN PIAO
 
UMAP2016 - Analyzing Aggregated Semantics-enabled User Modeling on Google+ an...
UMAP2016 - Analyzing Aggregated Semantics-enabled User Modeling on Google+ an...UMAP2016 - Analyzing Aggregated Semantics-enabled User Modeling on Google+ an...
UMAP2016 - Analyzing Aggregated Semantics-enabled User Modeling on Google+ an...GUANGYUAN PIAO
 
SAC2016-Measuring Semantic Distance for Linked Open Data-enabled Recommender ...
SAC2016-Measuring Semantic Distance for Linked Open Data-enabled Recommender ...SAC2016-Measuring Semantic Distance for Linked Open Data-enabled Recommender ...
SAC2016-Measuring Semantic Distance for Linked Open Data-enabled Recommender ...GUANGYUAN PIAO
 
JIST2015-Computing the Semantic Similarity of Resources in DBpedia for Recomm...
JIST2015-Computing the Semantic Similarity of Resources in DBpedia for Recomm...JIST2015-Computing the Semantic Similarity of Resources in DBpedia for Recomm...
JIST2015-Computing the Semantic Similarity of Resources in DBpedia for Recomm...GUANGYUAN PIAO
 
JIST2015-data challenge
JIST2015-data challengeJIST2015-data challenge
JIST2015-data challengeGUANGYUAN PIAO
 
Analyzing User Modeling on Twitter for Personalized News Recommendations
Analyzing User Modeling on Twitter for Personalized News RecommendationsAnalyzing User Modeling on Twitter for Personalized News Recommendations
Analyzing User Modeling on Twitter for Personalized News RecommendationsGUANGYUAN PIAO
 

More from GUANGYUAN PIAO (19)

Domain-Aware Sentiment Classification with GRUs and CNNs
Domain-Aware Sentiment Classification with GRUs and CNNsDomain-Aware Sentiment Classification with GRUs and CNNs
Domain-Aware Sentiment Classification with GRUs and CNNs
 
A Study of the Similarities of Entity Embeddings Learned from Different Aspec...
A Study of the Similarities of Entity Embeddings Learned from Different Aspec...A Study of the Similarities of Entity Embeddings Learned from Different Aspec...
A Study of the Similarities of Entity Embeddings Learned from Different Aspec...
 
Retweet Prediction with Attention-based Deep Neural Network
Retweet Prediction with Attention-based Deep Neural NetworkRetweet Prediction with Attention-based Deep Neural Network
Retweet Prediction with Attention-based Deep Neural Network
 
WISE2017 - Factorization Machines Leveraging Lightweight Linked Open Data-ena...
WISE2017 - Factorization Machines Leveraging Lightweight Linked Open Data-ena...WISE2017 - Factorization Machines Leveraging Lightweight Linked Open Data-ena...
WISE2017 - Factorization Machines Leveraging Lightweight Linked Open Data-ena...
 
Hypertext2017-Leveraging Followee List Memberships for Inferring User Interes...
Hypertext2017-Leveraging Followee List Memberships for Inferring User Interes...Hypertext2017-Leveraging Followee List Memberships for Inferring User Interes...
Hypertext2017-Leveraging Followee List Memberships for Inferring User Interes...
 
ECIR2017-Inferring User Interests for Passive Users on Twitter by Leveraging ...
ECIR2017-Inferring User Interests for Passive Users on Twitter by Leveraging ...ECIR2017-Inferring User Interests for Passive Users on Twitter by Leveraging ...
ECIR2017-Inferring User Interests for Passive Users on Twitter by Leveraging ...
 
EKAW2016 - Interest Representation, Enrichment, Dynamics, and Propagation: A ...
EKAW2016 - Interest Representation, Enrichment, Dynamics, and Propagation: A ...EKAW2016 - Interest Representation, Enrichment, Dynamics, and Propagation: A ...
EKAW2016 - Interest Representation, Enrichment, Dynamics, and Propagation: A ...
 
SEMANTiCS2016 - Exploring Dynamics and Semantics of User Interests for User ...
SEMANTiCS2016 - Exploring Dynamics and Semantics of User Interests for User ...SEMANTiCS2016 - Exploring Dynamics and Semantics of User Interests for User ...
SEMANTiCS2016 - Exploring Dynamics and Semantics of User Interests for User ...
 
UMAP2016EA - Analyzing MOOC Entries of Professionals on LinkedIn for User Mod...
UMAP2016EA - Analyzing MOOC Entries of Professionals on LinkedIn for User Mod...UMAP2016EA - Analyzing MOOC Entries of Professionals on LinkedIn for User Mod...
UMAP2016EA - Analyzing MOOC Entries of Professionals on LinkedIn for User Mod...
 
UMAP2016 - Analyzing Aggregated Semantics-enabled User Modeling on Google+ an...
UMAP2016 - Analyzing Aggregated Semantics-enabled User Modeling on Google+ an...UMAP2016 - Analyzing Aggregated Semantics-enabled User Modeling on Google+ an...
UMAP2016 - Analyzing Aggregated Semantics-enabled User Modeling on Google+ an...
 
SAC2016-Measuring Semantic Distance for Linked Open Data-enabled Recommender ...
SAC2016-Measuring Semantic Distance for Linked Open Data-enabled Recommender ...SAC2016-Measuring Semantic Distance for Linked Open Data-enabled Recommender ...
SAC2016-Measuring Semantic Distance for Linked Open Data-enabled Recommender ...
 
JIST2015-Computing the Semantic Similarity of Resources in DBpedia for Recomm...
JIST2015-Computing the Semantic Similarity of Resources in DBpedia for Recomm...JIST2015-Computing the Semantic Similarity of Resources in DBpedia for Recomm...
JIST2015-Computing the Semantic Similarity of Resources in DBpedia for Recomm...
 
JIST2015-data challenge
JIST2015-data challengeJIST2015-data challenge
JIST2015-data challenge
 
Analyzing User Modeling on Twitter for Personalized News Recommendations
Analyzing User Modeling on Twitter for Personalized News RecommendationsAnalyzing User Modeling on Twitter for Personalized News Recommendations
Analyzing User Modeling on Twitter for Personalized News Recommendations
 
RDFa Basics
RDFa BasicsRDFa Basics
RDFa Basics
 
Owl 2.0 Overview
Owl 2.0 OverviewOwl 2.0 Overview
Owl 2.0 Overview
 
OWL 2.0 Primer Part01
OWL 2.0 Primer Part01OWL 2.0 Primer Part01
OWL 2.0 Primer Part01
 
OWL2.0 Primer Part02
OWL2.0 Primer Part02OWL2.0 Primer Part02
OWL2.0 Primer Part02
 
Hdd industry
Hdd industryHdd industry
Hdd industry
 

Recently uploaded

MATERI MANAJEMEN OF PENYAKIT TETANUS.ppt
MATERI  MANAJEMEN OF PENYAKIT TETANUS.pptMATERI  MANAJEMEN OF PENYAKIT TETANUS.ppt
MATERI MANAJEMEN OF PENYAKIT TETANUS.pptRachmaGhifari
 
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...yulianti213969
 
obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...
obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...
obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...yulianti213969
 
如何办理(Dalhousie毕业证书)达尔豪斯大学毕业证成绩单留信学历认证
如何办理(Dalhousie毕业证书)达尔豪斯大学毕业证成绩单留信学历认证如何办理(Dalhousie毕业证书)达尔豪斯大学毕业证成绩单留信学历认证
如何办理(Dalhousie毕业证书)达尔豪斯大学毕业证成绩单留信学历认证zifhagzkk
 
Credit Card Fraud Detection: Safeguarding Transactions in the Digital Age
Credit Card Fraud Detection: Safeguarding Transactions in the Digital AgeCredit Card Fraud Detection: Safeguarding Transactions in the Digital Age
Credit Card Fraud Detection: Safeguarding Transactions in the Digital AgeBoston Institute of Analytics
 
如何办理加州大学伯克利分校毕业证(UCB毕业证)成绩单留信学历认证
如何办理加州大学伯克利分校毕业证(UCB毕业证)成绩单留信学历认证如何办理加州大学伯克利分校毕业证(UCB毕业证)成绩单留信学历认证
如何办理加州大学伯克利分校毕业证(UCB毕业证)成绩单留信学历认证a8om7o51
 
Displacement, Velocity, Acceleration, and Second Derivatives
Displacement, Velocity, Acceleration, and Second DerivativesDisplacement, Velocity, Acceleration, and Second Derivatives
Displacement, Velocity, Acceleration, and Second Derivatives23050636
 
Jual Obat Aborsi Bandung (Asli No.1) Wa 082134680322 Klinik Obat Penggugur Ka...
Jual Obat Aborsi Bandung (Asli No.1) Wa 082134680322 Klinik Obat Penggugur Ka...Jual Obat Aborsi Bandung (Asli No.1) Wa 082134680322 Klinik Obat Penggugur Ka...
Jual Obat Aborsi Bandung (Asli No.1) Wa 082134680322 Klinik Obat Penggugur Ka...Klinik Aborsi
 
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证ju0dztxtn
 
如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样
如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样
如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样jk0tkvfv
 
Seven tools of quality control.slideshare
Seven tools of quality control.slideshareSeven tools of quality control.slideshare
Seven tools of quality control.slideshareraiaryan448
 
Bios of leading Astrologers & Researchers
Bios of leading Astrologers & ResearchersBios of leading Astrologers & Researchers
Bios of leading Astrologers & Researchersdarmandersingh4580
 
Predictive Precipitation: Advanced Rain Forecasting Techniques
Predictive Precipitation: Advanced Rain Forecasting TechniquesPredictive Precipitation: Advanced Rain Forecasting Techniques
Predictive Precipitation: Advanced Rain Forecasting TechniquesBoston Institute of Analytics
 
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证pwgnohujw
 
Genuine love spell caster )! ,+27834335081) Ex lover back permanently in At...
Genuine love spell caster )! ,+27834335081)   Ex lover back permanently in At...Genuine love spell caster )! ,+27834335081)   Ex lover back permanently in At...
Genuine love spell caster )! ,+27834335081) Ex lover back permanently in At...BabaJohn3
 
如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证acoha1
 
Formulas dax para power bI de microsoft.pdf
Formulas dax para power bI de microsoft.pdfFormulas dax para power bI de microsoft.pdf
Formulas dax para power bI de microsoft.pdfRobertoOcampo24
 
Northern New England Tableau User Group (TUG) May 2024
Northern New England Tableau User Group (TUG) May 2024Northern New England Tableau User Group (TUG) May 2024
Northern New England Tableau User Group (TUG) May 2024patrickdtherriault
 
1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证
1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证
1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证dq9vz1isj
 
How to Transform Clinical Trial Management with Advanced Data Analytics
How to Transform Clinical Trial Management with Advanced Data AnalyticsHow to Transform Clinical Trial Management with Advanced Data Analytics
How to Transform Clinical Trial Management with Advanced Data AnalyticsBrainSell Technologies
 

Recently uploaded (20)

MATERI MANAJEMEN OF PENYAKIT TETANUS.ppt
MATERI  MANAJEMEN OF PENYAKIT TETANUS.pptMATERI  MANAJEMEN OF PENYAKIT TETANUS.ppt
MATERI MANAJEMEN OF PENYAKIT TETANUS.ppt
 
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
 
obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...
obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...
obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...
 
如何办理(Dalhousie毕业证书)达尔豪斯大学毕业证成绩单留信学历认证
如何办理(Dalhousie毕业证书)达尔豪斯大学毕业证成绩单留信学历认证如何办理(Dalhousie毕业证书)达尔豪斯大学毕业证成绩单留信学历认证
如何办理(Dalhousie毕业证书)达尔豪斯大学毕业证成绩单留信学历认证
 
Credit Card Fraud Detection: Safeguarding Transactions in the Digital Age
Credit Card Fraud Detection: Safeguarding Transactions in the Digital AgeCredit Card Fraud Detection: Safeguarding Transactions in the Digital Age
Credit Card Fraud Detection: Safeguarding Transactions in the Digital Age
 
如何办理加州大学伯克利分校毕业证(UCB毕业证)成绩单留信学历认证
如何办理加州大学伯克利分校毕业证(UCB毕业证)成绩单留信学历认证如何办理加州大学伯克利分校毕业证(UCB毕业证)成绩单留信学历认证
如何办理加州大学伯克利分校毕业证(UCB毕业证)成绩单留信学历认证
 
Displacement, Velocity, Acceleration, and Second Derivatives
Displacement, Velocity, Acceleration, and Second DerivativesDisplacement, Velocity, Acceleration, and Second Derivatives
Displacement, Velocity, Acceleration, and Second Derivatives
 
Jual Obat Aborsi Bandung (Asli No.1) Wa 082134680322 Klinik Obat Penggugur Ka...
Jual Obat Aborsi Bandung (Asli No.1) Wa 082134680322 Klinik Obat Penggugur Ka...Jual Obat Aborsi Bandung (Asli No.1) Wa 082134680322 Klinik Obat Penggugur Ka...
Jual Obat Aborsi Bandung (Asli No.1) Wa 082134680322 Klinik Obat Penggugur Ka...
 
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
 
如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样
如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样
如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样
 
Seven tools of quality control.slideshare
Seven tools of quality control.slideshareSeven tools of quality control.slideshare
Seven tools of quality control.slideshare
 
Bios of leading Astrologers & Researchers
Bios of leading Astrologers & ResearchersBios of leading Astrologers & Researchers
Bios of leading Astrologers & Researchers
 
Predictive Precipitation: Advanced Rain Forecasting Techniques
Predictive Precipitation: Advanced Rain Forecasting TechniquesPredictive Precipitation: Advanced Rain Forecasting Techniques
Predictive Precipitation: Advanced Rain Forecasting Techniques
 
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
 
Genuine love spell caster )! ,+27834335081) Ex lover back permanently in At...
Genuine love spell caster )! ,+27834335081)   Ex lover back permanently in At...Genuine love spell caster )! ,+27834335081)   Ex lover back permanently in At...
Genuine love spell caster )! ,+27834335081) Ex lover back permanently in At...
 
如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证
 
Formulas dax para power bI de microsoft.pdf
Formulas dax para power bI de microsoft.pdfFormulas dax para power bI de microsoft.pdf
Formulas dax para power bI de microsoft.pdf
 
Northern New England Tableau User Group (TUG) May 2024
Northern New England Tableau User Group (TUG) May 2024Northern New England Tableau User Group (TUG) May 2024
Northern New England Tableau User Group (TUG) May 2024
 
1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证
1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证
1:1原版定制伦敦政治经济学院毕业证(LSE毕业证)成绩单学位证书留信学历认证
 
How to Transform Clinical Trial Management with Advanced Data Analytics
How to Transform Clinical Trial Management with Advanced Data AnalyticsHow to Transform Clinical Trial Management with Advanced Data Analytics
How to Transform Clinical Trial Management with Advanced Data Analytics
 

Env2Vec: Accelerating VNF Testing with Deep Learning

  • 1. Env2Vec: Accelerating VNF Testing with Deep Learning Guangyuan Piao, Pat Nicholson, Diego Lugones Nokia Bell Labs, Dublin, Ireland The 15th European Conference on Computer Systems, 30/04/2020
  • 2. © 2019 Nokia2 Introduction Current challenges for network engineers adopting modern DevOps: • Manage hundreds of distributed testbeds each running the new software build • Test new virtual network function (VNF) builds in shorter development cycles • Assure carrier-grade quality Objective: • Use deep learning (DL) techniques to accelerate VNF testing • Automate the diagnosis of software defects and anomalous builds • Point out testing engineers to problematic metrics Introduction → Problem → Proposal → Evaluation → Conclusions
  • 3. © 2019 Nokia3 Problem: Environments must be Considered! Introduction → Problem → Proposal → Evaluation → Conclusions 125 different models Varying accuracy
  • 4. © 2019 Nokia4 Proposed Approach: Env2Vec Anomaly Detection 1. Learn a universal resource characterization model 2. Execute the model in real time • to detect deviations between the inferred value (𝒚 𝒑 ′ ) and the actual observed RU of the VNF • Flag anomaly if there is a significant deviation (larger than γ x standard deviations) 𝒚 𝒑 ′ GRUs FNNs Lookup Tables (Embeddings) characterization model with DL Ruhistory (Resource Usage) CF (contextual features) • Workload metrics (WM) • Performance metrics (PM) EM (environment metadata) • Testbed, build type, test cases etc. Inferred resource usage (RU) (e.g., CPU) Introduction → Problem → Proposal → Evaluation → Conclusions
  • 5. © 2019 Nokia5 System Overview 1. Testbed data collection 2. Model training (daily) 3. Prediction pipeline 4. Raising alarms 5. Updating the model Introduction → Problem → Proposal → Evaluation → Conclusions
  • 6. © 2019 Nokia6 Evaluation: Carrier-grade VNF Testing Dataset • 600+ real-world testing environments • 125 build chains (testbed, build type, SUT, and test case) • 400,000 timesteps/data points measured at 15 minute intervals Evaluation metrics • Number of generated alarms • True and false alarm rate ( AT and AF ) • System with a higher number of generated alarms and higher AT is better Introduction → Problem → Proposal → Evaluation → Conclusions
  • 7. © 2019 Nokia7 Evaluation: Automating Anomaly Detection in New Builds Compared approaches • HTM-AD: AD based on RU time series only • Ridge: linear regression trained on each environment • Ridge(ts): considers RU history as features • RFNNall: Env2Vec without using embeddings Observations • Either the priority is raising (1) more performance problems automatically or (2) highly accurate alarms, • Env2Vec provides the best performance with γ = 1or 2 Introduction → Problem → Proposal → Evaluation → Conclusions
  • 8. © 2019 Nokia8 Evaluation: Reuse Embeddings in Unseen Environment Env2Vec • Reuses learned embeddings (capturing similarity of environments) • Construct environment embedding for a new unseen environment • Can detect performance problems better in unseen test executions lack of data for each unseen env. Introduction → Problem → Proposal → Evaluation → Conclusions
  • 9. © 2019 Nokia9 Conclusions • We proposed Env2Vec, which creates a universal easy-to-maintain model • Env2Vec automatically detects defects and bugs in new software builds by identifying performance degradation and informing the engineer • Can be extrapolated to previously unseen environments • Achieves accuracy between 86.2%-100%, while reducing the false alarm rate by 20.9%- 38.1% compared to other approaches Introduction → Problem → Proposal → Evaluation → Conclusions
  • 10. Q & A Contact information: Guangyuan Piao: guangyuan.piao@nokia-bell-labs.com Pat Nicholson: pat.nicholson@nokia-bell-labs.com Diego Lugones: diego.lugones@nokia-bell-labs.com