SlideShare a Scribd company logo
Env2Vec: Accelerating VNF
Testing with Deep Learning
Guangyuan Piao, Pat Nicholson, Diego Lugones
Nokia Bell Labs, Dublin, Ireland
The 15th European Conference on Computer Systems, 30/04/2020
© 2019 Nokia2
Introduction
Current challenges for network engineers adopting modern DevOps:
• Manage hundreds of distributed testbeds each running the new software build
• Test new virtual network function (VNF) builds in shorter development cycles
• Assure carrier-grade quality
Objective:
• Use deep learning (DL) techniques to accelerate VNF testing
• Automate the diagnosis of software defects and anomalous builds
• Point out testing engineers to problematic metrics
Introduction → Problem → Proposal → Evaluation → Conclusions
© 2019 Nokia3
Problem: Environments must be Considered!
Introduction → Problem → Proposal → Evaluation → Conclusions
125 different models
Varying accuracy
© 2019 Nokia4
Proposed Approach: Env2Vec Anomaly Detection
1. Learn a universal resource characterization model
2. Execute the model in real time
• to detect deviations between the inferred value (𝒚 𝒑
′
) and the actual observed RU of the VNF
• Flag anomaly if there is a significant deviation (larger than γ x standard deviations)
𝒚 𝒑
′
GRUs
FNNs
Lookup Tables
(Embeddings)
characterization
model with DL
Ruhistory (Resource Usage)
CF (contextual features)
• Workload metrics (WM)
• Performance metrics (PM)
EM (environment metadata)
• Testbed, build type, test cases etc.
Inferred resource
usage (RU) (e.g., CPU)
Introduction → Problem → Proposal → Evaluation → Conclusions
© 2019 Nokia5
System Overview
1. Testbed data collection
2. Model training (daily)
3. Prediction pipeline
4. Raising alarms
5. Updating the model
Introduction → Problem → Proposal → Evaluation → Conclusions
© 2019 Nokia6
Evaluation: Carrier-grade VNF Testing
Dataset
• 600+ real-world testing environments
• 125 build chains (testbed, build type, SUT, and test case)
• 400,000 timesteps/data points measured at 15 minute intervals
Evaluation metrics
• Number of generated alarms
• True and false alarm rate ( AT and AF )
• System with a higher number of generated alarms and higher AT is better
Introduction → Problem → Proposal → Evaluation → Conclusions
© 2019 Nokia7
Evaluation: Automating Anomaly Detection in New Builds
Compared approaches
• HTM-AD: AD based on RU time series only
• Ridge: linear regression trained on each environment
• Ridge(ts): considers RU history as features
• RFNNall: Env2Vec without using embeddings
Observations
• Either the priority is raising (1) more performance
problems automatically or (2) highly accurate alarms,
• Env2Vec provides the best performance with γ = 1or 2
Introduction → Problem → Proposal → Evaluation → Conclusions
© 2019 Nokia8
Evaluation: Reuse Embeddings in Unseen Environment
Env2Vec
• Reuses learned embeddings (capturing similarity of environments)
• Construct environment embedding for a new unseen environment
• Can detect performance problems better in unseen test executions
lack of data for each unseen env.
Introduction → Problem → Proposal → Evaluation → Conclusions
© 2019 Nokia9
Conclusions
• We proposed Env2Vec, which creates a universal easy-to-maintain model
• Env2Vec automatically detects defects and bugs in new software builds by identifying
performance degradation and informing the engineer
• Can be extrapolated to previously unseen environments
• Achieves accuracy between 86.2%-100%, while reducing the false alarm rate by 20.9%-
38.1% compared to other approaches
Introduction → Problem → Proposal → Evaluation → Conclusions
Q & A
Contact information:
Guangyuan Piao: guangyuan.piao@nokia-bell-labs.com
Pat Nicholson: pat.nicholson@nokia-bell-labs.com
Diego Lugones: diego.lugones@nokia-bell-labs.com
Env2Vec: Accelerating VNF Testing with Deep Learning

More Related Content

What's hot

Use Models for Extending IEEE 1687 to Analog Test
Use Models for Extending IEEE 1687 to Analog TestUse Models for Extending IEEE 1687 to Analog Test
Use Models for Extending IEEE 1687 to Analog Test
Pete Sarson, PH.D
 
LifeGuard: Precision Projectile Tracking by Randy S. Roberts, LLNL Electrical...
LifeGuard: Precision Projectile Tracking by Randy S. Roberts, LLNL Electrical...LifeGuard: Precision Projectile Tracking by Randy S. Roberts, LLNL Electrical...
LifeGuard: Precision Projectile Tracking by Randy S. Roberts, LLNL Electrical...
Industrial Partnerships Office
 
Surveillance scene classification using machine learning
Surveillance scene classification using machine learningSurveillance scene classification using machine learning
Surveillance scene classification using machine learning
Utkarsh Contractor
 
SuMo-SS: Submodular Optimization Sensor Scattering for Deploying Sensor Netwo...
SuMo-SS: Submodular Optimization Sensor Scattering for Deploying Sensor Netwo...SuMo-SS: Submodular Optimization Sensor Scattering for Deploying Sensor Netwo...
SuMo-SS: Submodular Optimization Sensor Scattering for Deploying Sensor Netwo...
Komei Sugiura
 
EMPhASIS - Implementation
EMPhASIS - ImplementationEMPhASIS - Implementation
EMPhASIS - Implementation
NECST Lab @ Politecnico di Milano
 
Detection of fire through Camera
Detection of fire through CameraDetection of fire through Camera
Detection of fire through Camera
Shivam Singhal
 
FIRE DETECTION USING VIDEO ANALYTICS
FIRE DETECTION USING VIDEO ANALYTICSFIRE DETECTION USING VIDEO ANALYTICS
FIRE DETECTION USING VIDEO ANALYTICS
Gaurav Deshmukh
 
2022_03_28 "Raspberry Pi Applications in Electronics and Control Laboratories"
2022_03_28 "Raspberry Pi Applications in Electronics and Control Laboratories"2022_03_28 "Raspberry Pi Applications in Electronics and Control Laboratories"
2022_03_28 "Raspberry Pi Applications in Electronics and Control Laboratories"
eMadrid network
 
Combining out - of - band monitoring with AI and big data for datacenter aut...
Combining out - of - band monitoring with AI and big data  for datacenter aut...Combining out - of - band monitoring with AI and big data  for datacenter aut...
Combining out - of - band monitoring with AI and big data for datacenter aut...
Ganesan Narayanasamy
 
Container orchestration in geo-distributed cloud computing platforms
Container orchestration in geo-distributed cloud computing platformsContainer orchestration in geo-distributed cloud computing platforms
Container orchestration in geo-distributed cloud computing platforms
FogGuru MSCA Project
 
AnupVMathur
AnupVMathurAnupVMathur
AnupVMathur
anupmath
 

What's hot (11)

Use Models for Extending IEEE 1687 to Analog Test
Use Models for Extending IEEE 1687 to Analog TestUse Models for Extending IEEE 1687 to Analog Test
Use Models for Extending IEEE 1687 to Analog Test
 
LifeGuard: Precision Projectile Tracking by Randy S. Roberts, LLNL Electrical...
LifeGuard: Precision Projectile Tracking by Randy S. Roberts, LLNL Electrical...LifeGuard: Precision Projectile Tracking by Randy S. Roberts, LLNL Electrical...
LifeGuard: Precision Projectile Tracking by Randy S. Roberts, LLNL Electrical...
 
Surveillance scene classification using machine learning
Surveillance scene classification using machine learningSurveillance scene classification using machine learning
Surveillance scene classification using machine learning
 
SuMo-SS: Submodular Optimization Sensor Scattering for Deploying Sensor Netwo...
SuMo-SS: Submodular Optimization Sensor Scattering for Deploying Sensor Netwo...SuMo-SS: Submodular Optimization Sensor Scattering for Deploying Sensor Netwo...
SuMo-SS: Submodular Optimization Sensor Scattering for Deploying Sensor Netwo...
 
EMPhASIS - Implementation
EMPhASIS - ImplementationEMPhASIS - Implementation
EMPhASIS - Implementation
 
Detection of fire through Camera
Detection of fire through CameraDetection of fire through Camera
Detection of fire through Camera
 
FIRE DETECTION USING VIDEO ANALYTICS
FIRE DETECTION USING VIDEO ANALYTICSFIRE DETECTION USING VIDEO ANALYTICS
FIRE DETECTION USING VIDEO ANALYTICS
 
2022_03_28 "Raspberry Pi Applications in Electronics and Control Laboratories"
2022_03_28 "Raspberry Pi Applications in Electronics and Control Laboratories"2022_03_28 "Raspberry Pi Applications in Electronics and Control Laboratories"
2022_03_28 "Raspberry Pi Applications in Electronics and Control Laboratories"
 
Combining out - of - band monitoring with AI and big data for datacenter aut...
Combining out - of - band monitoring with AI and big data  for datacenter aut...Combining out - of - band monitoring with AI and big data  for datacenter aut...
Combining out - of - band monitoring with AI and big data for datacenter aut...
 
Container orchestration in geo-distributed cloud computing platforms
Container orchestration in geo-distributed cloud computing platformsContainer orchestration in geo-distributed cloud computing platforms
Container orchestration in geo-distributed cloud computing platforms
 
AnupVMathur
AnupVMathurAnupVMathur
AnupVMathur
 

Similar to Env2Vec: Accelerating VNF Testing with Deep Learning

01-06 OCRE Test Suite - Fernandes.pdf
01-06 OCRE Test Suite - Fernandes.pdf01-06 OCRE Test Suite - Fernandes.pdf
01-06 OCRE Test Suite - Fernandes.pdf
OCRE | Open Clouds for Research Environments
 
Distributed Checkpointing on an Enterprise Desktop Grid
Distributed Checkpointing on an Enterprise Desktop GridDistributed Checkpointing on an Enterprise Desktop Grid
Distributed Checkpointing on an Enterprise Desktop Grid
brent.wilson
 
Lessons learned so far in operationalizing NFV
Lessons learned so far in operationalizing NFVLessons learned so far in operationalizing NFV
Lessons learned so far in operationalizing NFV
James Crawshaw
 
Swimming upstream: OPNFV Doctor project case study
Swimming upstream: OPNFV Doctor project case studySwimming upstream: OPNFV Doctor project case study
Swimming upstream: OPNFV Doctor project case study
OPNFV
 
Challenges in Practicing High Frequency Releases in Cloud Environments
Challenges in Practicing High Frequency Releases in Cloud Environments Challenges in Practicing High Frequency Releases in Cloud Environments
Challenges in Practicing High Frequency Releases in Cloud Environments
Liming Zhu
 
First Steps to DevOps
First Steps to DevOpsFirst Steps to DevOps
First Steps to DevOps
Inductive Automation
 
SAND: A Fault-Tolerant Streaming Architecture for Network Traffic Analytics
SAND: A Fault-Tolerant Streaming Architecture for Network Traffic AnalyticsSAND: A Fault-Tolerant Streaming Architecture for Network Traffic Analytics
SAND: A Fault-Tolerant Streaming Architecture for Network Traffic Analytics
Qin Liu
 
Benchmarking and Performance on AWS - AWS India Summit 2012
Benchmarking and Performance on AWS - AWS India Summit 2012Benchmarking and Performance on AWS - AWS India Summit 2012
Benchmarking and Performance on AWS - AWS India Summit 2012
Amazon Web Services
 
Mobile Reliability Challenges
Mobile Reliability ChallengesMobile Reliability Challenges
Mobile Reliability Challenges
Bob Binder
 
Ensemble Launches Major Upgrade to NFV Platform
Ensemble Launches Major Upgrade to NFV PlatformEnsemble Launches Major Upgrade to NFV Platform
Ensemble Launches Major Upgrade to NFV Platform
ADVA
 
Computer Engineer Master Project
Computer Engineer Master ProjectComputer Engineer Master Project
Computer Engineer Master Project
Jordi Muntada Gómez
 
Application of machine learning and cognitive computing in intrusion detectio...
Application of machine learning and cognitive computing in intrusion detectio...Application of machine learning and cognitive computing in intrusion detectio...
Application of machine learning and cognitive computing in intrusion detectio...
Mahdi Hosseini Moghaddam
 
Improving performance and efficiency with Network Virtualization Overlays
Improving performance and efficiency with Network Virtualization OverlaysImproving performance and efficiency with Network Virtualization Overlays
Improving performance and efficiency with Network Virtualization Overlays
Adam Johnson
 
Software defined networking(sdn) vahid sadri
Software defined networking(sdn) vahid sadriSoftware defined networking(sdn) vahid sadri
Software defined networking(sdn) vahid sadri
Vahid Sadri
 
Networking Concepts Lesson 13 - Troubleshooting - Eric Vanderburg
Networking Concepts Lesson 13 - Troubleshooting - Eric VanderburgNetworking Concepts Lesson 13 - Troubleshooting - Eric Vanderburg
Networking Concepts Lesson 13 - Troubleshooting - Eric Vanderburg
Eric Vanderburg
 
UVM_Full_Print_n.pptx
UVM_Full_Print_n.pptxUVM_Full_Print_n.pptx
UVM_Full_Print_n.pptx
nikitha992646
 
Future Internet: Managing Innovation and Testbed
Future Internet: Managing Innovation and TestbedFuture Internet: Managing Innovation and Testbed
Future Internet: Managing Innovation and Testbed
Shinji Shimojo
 
How to Operate Kubernetes CI/CD Pipelines at Scale
How to Operate Kubernetes CI/CD Pipelines at ScaleHow to Operate Kubernetes CI/CD Pipelines at Scale
How to Operate Kubernetes CI/CD Pipelines at Scale
DevOps.com
 
Addressing the top 10 challenges of lte epc testing
Addressing the top 10 challenges of lte epc testingAddressing the top 10 challenges of lte epc testing
Addressing the top 10 challenges of lte epc testing
Aricent
 
Vinay Singh
Vinay SinghVinay Singh
Vinay Singh
Vinay Singh
 

Similar to Env2Vec: Accelerating VNF Testing with Deep Learning (20)

01-06 OCRE Test Suite - Fernandes.pdf
01-06 OCRE Test Suite - Fernandes.pdf01-06 OCRE Test Suite - Fernandes.pdf
01-06 OCRE Test Suite - Fernandes.pdf
 
Distributed Checkpointing on an Enterprise Desktop Grid
Distributed Checkpointing on an Enterprise Desktop GridDistributed Checkpointing on an Enterprise Desktop Grid
Distributed Checkpointing on an Enterprise Desktop Grid
 
Lessons learned so far in operationalizing NFV
Lessons learned so far in operationalizing NFVLessons learned so far in operationalizing NFV
Lessons learned so far in operationalizing NFV
 
Swimming upstream: OPNFV Doctor project case study
Swimming upstream: OPNFV Doctor project case studySwimming upstream: OPNFV Doctor project case study
Swimming upstream: OPNFV Doctor project case study
 
Challenges in Practicing High Frequency Releases in Cloud Environments
Challenges in Practicing High Frequency Releases in Cloud Environments Challenges in Practicing High Frequency Releases in Cloud Environments
Challenges in Practicing High Frequency Releases in Cloud Environments
 
First Steps to DevOps
First Steps to DevOpsFirst Steps to DevOps
First Steps to DevOps
 
SAND: A Fault-Tolerant Streaming Architecture for Network Traffic Analytics
SAND: A Fault-Tolerant Streaming Architecture for Network Traffic AnalyticsSAND: A Fault-Tolerant Streaming Architecture for Network Traffic Analytics
SAND: A Fault-Tolerant Streaming Architecture for Network Traffic Analytics
 
Benchmarking and Performance on AWS - AWS India Summit 2012
Benchmarking and Performance on AWS - AWS India Summit 2012Benchmarking and Performance on AWS - AWS India Summit 2012
Benchmarking and Performance on AWS - AWS India Summit 2012
 
Mobile Reliability Challenges
Mobile Reliability ChallengesMobile Reliability Challenges
Mobile Reliability Challenges
 
Ensemble Launches Major Upgrade to NFV Platform
Ensemble Launches Major Upgrade to NFV PlatformEnsemble Launches Major Upgrade to NFV Platform
Ensemble Launches Major Upgrade to NFV Platform
 
Computer Engineer Master Project
Computer Engineer Master ProjectComputer Engineer Master Project
Computer Engineer Master Project
 
Application of machine learning and cognitive computing in intrusion detectio...
Application of machine learning and cognitive computing in intrusion detectio...Application of machine learning and cognitive computing in intrusion detectio...
Application of machine learning and cognitive computing in intrusion detectio...
 
Improving performance and efficiency with Network Virtualization Overlays
Improving performance and efficiency with Network Virtualization OverlaysImproving performance and efficiency with Network Virtualization Overlays
Improving performance and efficiency with Network Virtualization Overlays
 
Software defined networking(sdn) vahid sadri
Software defined networking(sdn) vahid sadriSoftware defined networking(sdn) vahid sadri
Software defined networking(sdn) vahid sadri
 
Networking Concepts Lesson 13 - Troubleshooting - Eric Vanderburg
Networking Concepts Lesson 13 - Troubleshooting - Eric VanderburgNetworking Concepts Lesson 13 - Troubleshooting - Eric Vanderburg
Networking Concepts Lesson 13 - Troubleshooting - Eric Vanderburg
 
UVM_Full_Print_n.pptx
UVM_Full_Print_n.pptxUVM_Full_Print_n.pptx
UVM_Full_Print_n.pptx
 
Future Internet: Managing Innovation and Testbed
Future Internet: Managing Innovation and TestbedFuture Internet: Managing Innovation and Testbed
Future Internet: Managing Innovation and Testbed
 
How to Operate Kubernetes CI/CD Pipelines at Scale
How to Operate Kubernetes CI/CD Pipelines at ScaleHow to Operate Kubernetes CI/CD Pipelines at Scale
How to Operate Kubernetes CI/CD Pipelines at Scale
 
Addressing the top 10 challenges of lte epc testing
Addressing the top 10 challenges of lte epc testingAddressing the top 10 challenges of lte epc testing
Addressing the top 10 challenges of lte epc testing
 
Vinay Singh
Vinay SinghVinay Singh
Vinay Singh
 

More from GUANGYUAN PIAO

Domain-Aware Sentiment Classification with GRUs and CNNs
Domain-Aware Sentiment Classification with GRUs and CNNsDomain-Aware Sentiment Classification with GRUs and CNNs
Domain-Aware Sentiment Classification with GRUs and CNNs
GUANGYUAN PIAO
 
A Study of the Similarities of Entity Embeddings Learned from Different Aspec...
A Study of the Similarities of Entity Embeddings Learned from Different Aspec...A Study of the Similarities of Entity Embeddings Learned from Different Aspec...
A Study of the Similarities of Entity Embeddings Learned from Different Aspec...
GUANGYUAN PIAO
 
Retweet Prediction with Attention-based Deep Neural Network
Retweet Prediction with Attention-based Deep Neural NetworkRetweet Prediction with Attention-based Deep Neural Network
Retweet Prediction with Attention-based Deep Neural Network
GUANGYUAN PIAO
 
WISE2017 - Factorization Machines Leveraging Lightweight Linked Open Data-ena...
WISE2017 - Factorization Machines Leveraging Lightweight Linked Open Data-ena...WISE2017 - Factorization Machines Leveraging Lightweight Linked Open Data-ena...
WISE2017 - Factorization Machines Leveraging Lightweight Linked Open Data-ena...
GUANGYUAN PIAO
 
Hypertext2017-Leveraging Followee List Memberships for Inferring User Interes...
Hypertext2017-Leveraging Followee List Memberships for Inferring User Interes...Hypertext2017-Leveraging Followee List Memberships for Inferring User Interes...
Hypertext2017-Leveraging Followee List Memberships for Inferring User Interes...
GUANGYUAN PIAO
 
ECIR2017-Inferring User Interests for Passive Users on Twitter by Leveraging ...
ECIR2017-Inferring User Interests for Passive Users on Twitter by Leveraging ...ECIR2017-Inferring User Interests for Passive Users on Twitter by Leveraging ...
ECIR2017-Inferring User Interests for Passive Users on Twitter by Leveraging ...
GUANGYUAN PIAO
 
EKAW2016 - Interest Representation, Enrichment, Dynamics, and Propagation: A ...
EKAW2016 - Interest Representation, Enrichment, Dynamics, and Propagation: A ...EKAW2016 - Interest Representation, Enrichment, Dynamics, and Propagation: A ...
EKAW2016 - Interest Representation, Enrichment, Dynamics, and Propagation: A ...
GUANGYUAN PIAO
 
SEMANTiCS2016 - Exploring Dynamics and Semantics of User Interests for User ...
SEMANTiCS2016 - Exploring Dynamics and Semantics of User Interests for User ...SEMANTiCS2016 - Exploring Dynamics and Semantics of User Interests for User ...
SEMANTiCS2016 - Exploring Dynamics and Semantics of User Interests for User ...
GUANGYUAN PIAO
 
UMAP2016EA - Analyzing MOOC Entries of Professionals on LinkedIn for User Mod...
UMAP2016EA - Analyzing MOOC Entries of Professionals on LinkedIn for User Mod...UMAP2016EA - Analyzing MOOC Entries of Professionals on LinkedIn for User Mod...
UMAP2016EA - Analyzing MOOC Entries of Professionals on LinkedIn for User Mod...
GUANGYUAN PIAO
 
UMAP2016 - Analyzing Aggregated Semantics-enabled User Modeling on Google+ an...
UMAP2016 - Analyzing Aggregated Semantics-enabled User Modeling on Google+ an...UMAP2016 - Analyzing Aggregated Semantics-enabled User Modeling on Google+ an...
UMAP2016 - Analyzing Aggregated Semantics-enabled User Modeling on Google+ an...
GUANGYUAN PIAO
 
SAC2016-Measuring Semantic Distance for Linked Open Data-enabled Recommender ...
SAC2016-Measuring Semantic Distance for Linked Open Data-enabled Recommender ...SAC2016-Measuring Semantic Distance for Linked Open Data-enabled Recommender ...
SAC2016-Measuring Semantic Distance for Linked Open Data-enabled Recommender ...
GUANGYUAN PIAO
 
JIST2015-Computing the Semantic Similarity of Resources in DBpedia for Recomm...
JIST2015-Computing the Semantic Similarity of Resources in DBpedia for Recomm...JIST2015-Computing the Semantic Similarity of Resources in DBpedia for Recomm...
JIST2015-Computing the Semantic Similarity of Resources in DBpedia for Recomm...
GUANGYUAN PIAO
 
JIST2015-data challenge
JIST2015-data challengeJIST2015-data challenge
JIST2015-data challenge
GUANGYUAN PIAO
 
Analyzing User Modeling on Twitter for Personalized News Recommendations
Analyzing User Modeling on Twitter for Personalized News RecommendationsAnalyzing User Modeling on Twitter for Personalized News Recommendations
Analyzing User Modeling on Twitter for Personalized News Recommendations
GUANGYUAN PIAO
 
RDFa Basics
RDFa BasicsRDFa Basics
RDFa Basics
GUANGYUAN PIAO
 
Owl 2.0 Overview
Owl 2.0 OverviewOwl 2.0 Overview
Owl 2.0 Overview
GUANGYUAN PIAO
 
OWL 2.0 Primer Part01
OWL 2.0 Primer Part01OWL 2.0 Primer Part01
OWL 2.0 Primer Part01
GUANGYUAN PIAO
 
OWL2.0 Primer Part02
OWL2.0 Primer Part02OWL2.0 Primer Part02
OWL2.0 Primer Part02
GUANGYUAN PIAO
 
Hdd industry
Hdd industryHdd industry
Hdd industry
GUANGYUAN PIAO
 

More from GUANGYUAN PIAO (19)

Domain-Aware Sentiment Classification with GRUs and CNNs
Domain-Aware Sentiment Classification with GRUs and CNNsDomain-Aware Sentiment Classification with GRUs and CNNs
Domain-Aware Sentiment Classification with GRUs and CNNs
 
A Study of the Similarities of Entity Embeddings Learned from Different Aspec...
A Study of the Similarities of Entity Embeddings Learned from Different Aspec...A Study of the Similarities of Entity Embeddings Learned from Different Aspec...
A Study of the Similarities of Entity Embeddings Learned from Different Aspec...
 
Retweet Prediction with Attention-based Deep Neural Network
Retweet Prediction with Attention-based Deep Neural NetworkRetweet Prediction with Attention-based Deep Neural Network
Retweet Prediction with Attention-based Deep Neural Network
 
WISE2017 - Factorization Machines Leveraging Lightweight Linked Open Data-ena...
WISE2017 - Factorization Machines Leveraging Lightweight Linked Open Data-ena...WISE2017 - Factorization Machines Leveraging Lightweight Linked Open Data-ena...
WISE2017 - Factorization Machines Leveraging Lightweight Linked Open Data-ena...
 
Hypertext2017-Leveraging Followee List Memberships for Inferring User Interes...
Hypertext2017-Leveraging Followee List Memberships for Inferring User Interes...Hypertext2017-Leveraging Followee List Memberships for Inferring User Interes...
Hypertext2017-Leveraging Followee List Memberships for Inferring User Interes...
 
ECIR2017-Inferring User Interests for Passive Users on Twitter by Leveraging ...
ECIR2017-Inferring User Interests for Passive Users on Twitter by Leveraging ...ECIR2017-Inferring User Interests for Passive Users on Twitter by Leveraging ...
ECIR2017-Inferring User Interests for Passive Users on Twitter by Leveraging ...
 
EKAW2016 - Interest Representation, Enrichment, Dynamics, and Propagation: A ...
EKAW2016 - Interest Representation, Enrichment, Dynamics, and Propagation: A ...EKAW2016 - Interest Representation, Enrichment, Dynamics, and Propagation: A ...
EKAW2016 - Interest Representation, Enrichment, Dynamics, and Propagation: A ...
 
SEMANTiCS2016 - Exploring Dynamics and Semantics of User Interests for User ...
SEMANTiCS2016 - Exploring Dynamics and Semantics of User Interests for User ...SEMANTiCS2016 - Exploring Dynamics and Semantics of User Interests for User ...
SEMANTiCS2016 - Exploring Dynamics and Semantics of User Interests for User ...
 
UMAP2016EA - Analyzing MOOC Entries of Professionals on LinkedIn for User Mod...
UMAP2016EA - Analyzing MOOC Entries of Professionals on LinkedIn for User Mod...UMAP2016EA - Analyzing MOOC Entries of Professionals on LinkedIn for User Mod...
UMAP2016EA - Analyzing MOOC Entries of Professionals on LinkedIn for User Mod...
 
UMAP2016 - Analyzing Aggregated Semantics-enabled User Modeling on Google+ an...
UMAP2016 - Analyzing Aggregated Semantics-enabled User Modeling on Google+ an...UMAP2016 - Analyzing Aggregated Semantics-enabled User Modeling on Google+ an...
UMAP2016 - Analyzing Aggregated Semantics-enabled User Modeling on Google+ an...
 
SAC2016-Measuring Semantic Distance for Linked Open Data-enabled Recommender ...
SAC2016-Measuring Semantic Distance for Linked Open Data-enabled Recommender ...SAC2016-Measuring Semantic Distance for Linked Open Data-enabled Recommender ...
SAC2016-Measuring Semantic Distance for Linked Open Data-enabled Recommender ...
 
JIST2015-Computing the Semantic Similarity of Resources in DBpedia for Recomm...
JIST2015-Computing the Semantic Similarity of Resources in DBpedia for Recomm...JIST2015-Computing the Semantic Similarity of Resources in DBpedia for Recomm...
JIST2015-Computing the Semantic Similarity of Resources in DBpedia for Recomm...
 
JIST2015-data challenge
JIST2015-data challengeJIST2015-data challenge
JIST2015-data challenge
 
Analyzing User Modeling on Twitter for Personalized News Recommendations
Analyzing User Modeling on Twitter for Personalized News RecommendationsAnalyzing User Modeling on Twitter for Personalized News Recommendations
Analyzing User Modeling on Twitter for Personalized News Recommendations
 
RDFa Basics
RDFa BasicsRDFa Basics
RDFa Basics
 
Owl 2.0 Overview
Owl 2.0 OverviewOwl 2.0 Overview
Owl 2.0 Overview
 
OWL 2.0 Primer Part01
OWL 2.0 Primer Part01OWL 2.0 Primer Part01
OWL 2.0 Primer Part01
 
OWL2.0 Primer Part02
OWL2.0 Primer Part02OWL2.0 Primer Part02
OWL2.0 Primer Part02
 
Hdd industry
Hdd industryHdd industry
Hdd industry
 

Recently uploaded

原版一比一爱尔兰都柏林大学毕业证(UCD毕业证书)如何办理
原版一比一爱尔兰都柏林大学毕业证(UCD毕业证书)如何办理 原版一比一爱尔兰都柏林大学毕业证(UCD毕业证书)如何办理
原版一比一爱尔兰都柏林大学毕业证(UCD毕业证书)如何办理
tzu5xla
 
一比一原版莱斯大学毕业证(rice毕业证)如何办理
一比一原版莱斯大学毕业证(rice毕业证)如何办理一比一原版莱斯大学毕业证(rice毕业证)如何办理
一比一原版莱斯大学毕业证(rice毕业证)如何办理
zsafxbf
 
一比一原版南昆士兰大学毕业证如何办理
一比一原版南昆士兰大学毕业证如何办理一比一原版南昆士兰大学毕业证如何办理
一比一原版南昆士兰大学毕业证如何办理
ugydym
 
一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理
一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理
一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理
aguty
 
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
hqfek
 
Econ3060_Screen Time and Success_ final_GroupProject.pdf
Econ3060_Screen Time and Success_ final_GroupProject.pdfEcon3060_Screen Time and Success_ final_GroupProject.pdf
Econ3060_Screen Time and Success_ final_GroupProject.pdf
blueshagoo1
 
Sid Sigma educational and problem solving power point- Six Sigma.ppt
Sid Sigma educational and problem solving power point- Six Sigma.pptSid Sigma educational and problem solving power point- Six Sigma.ppt
Sid Sigma educational and problem solving power point- Six Sigma.ppt
ArshadAyub49
 
Cell The Unit of Life for NEET Multiple Choice Questions.docx
Cell The Unit of Life for NEET Multiple Choice Questions.docxCell The Unit of Life for NEET Multiple Choice Questions.docx
Cell The Unit of Life for NEET Multiple Choice Questions.docx
vasanthatpuram
 
Data Scientist Machine Learning Profiles .pdf
Data Scientist Machine Learning  Profiles .pdfData Scientist Machine Learning  Profiles .pdf
Data Scientist Machine Learning Profiles .pdf
Vineet
 
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
eudsoh
 
Open Source Contributions to Postgres: The Basics POSETTE 2024
Open Source Contributions to Postgres: The Basics POSETTE 2024Open Source Contributions to Postgres: The Basics POSETTE 2024
Open Source Contributions to Postgres: The Basics POSETTE 2024
ElizabethGarrettChri
 
Telemetry Solution for Gaming (AWS Summit'24)
Telemetry Solution for Gaming (AWS Summit'24)Telemetry Solution for Gaming (AWS Summit'24)
Telemetry Solution for Gaming (AWS Summit'24)
GeorgiiSteshenko
 
Drownings spike from May to August in children
Drownings spike from May to August in childrenDrownings spike from May to August in children
Drownings spike from May to August in children
Bisnar Chase Personal Injury Attorneys
 
Namma-Kalvi-11th-Physics-Study-Material-Unit-1-EM-221086.pdf
Namma-Kalvi-11th-Physics-Study-Material-Unit-1-EM-221086.pdfNamma-Kalvi-11th-Physics-Study-Material-Unit-1-EM-221086.pdf
Namma-Kalvi-11th-Physics-Study-Material-Unit-1-EM-221086.pdf
22ad0301
 
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
ihavuls
 
一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理
一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理
一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理
osoyvvf
 
一比一原版雷丁大学毕业证(UoR毕业证书)学历如何办理
一比一原版雷丁大学毕业证(UoR毕业证书)学历如何办理一比一原版雷丁大学毕业证(UoR毕业证书)学历如何办理
一比一原版雷丁大学毕业证(UoR毕业证书)学历如何办理
mbawufebxi
 
Discovering Digital Process Twins for What-if Analysis: a Process Mining Appr...
Discovering Digital Process Twins for What-if Analysis: a Process Mining Appr...Discovering Digital Process Twins for What-if Analysis: a Process Mining Appr...
Discovering Digital Process Twins for What-if Analysis: a Process Mining Appr...
Marlon Dumas
 
一比一原版英国赫特福德大学毕业证(hertfordshire毕业证书)如何办理
一比一原版英国赫特福德大学毕业证(hertfordshire毕业证书)如何办理一比一原版英国赫特福德大学毕业证(hertfordshire毕业证书)如何办理
一比一原版英国赫特福德大学毕业证(hertfordshire毕业证书)如何办理
nyvan3
 
8 things to know before you start to code in 2024
8 things to know before you start to code in 20248 things to know before you start to code in 2024
8 things to know before you start to code in 2024
ArianaRamos54
 

Recently uploaded (20)

原版一比一爱尔兰都柏林大学毕业证(UCD毕业证书)如何办理
原版一比一爱尔兰都柏林大学毕业证(UCD毕业证书)如何办理 原版一比一爱尔兰都柏林大学毕业证(UCD毕业证书)如何办理
原版一比一爱尔兰都柏林大学毕业证(UCD毕业证书)如何办理
 
一比一原版莱斯大学毕业证(rice毕业证)如何办理
一比一原版莱斯大学毕业证(rice毕业证)如何办理一比一原版莱斯大学毕业证(rice毕业证)如何办理
一比一原版莱斯大学毕业证(rice毕业证)如何办理
 
一比一原版南昆士兰大学毕业证如何办理
一比一原版南昆士兰大学毕业证如何办理一比一原版南昆士兰大学毕业证如何办理
一比一原版南昆士兰大学毕业证如何办理
 
一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理
一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理
一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理
 
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
 
Econ3060_Screen Time and Success_ final_GroupProject.pdf
Econ3060_Screen Time and Success_ final_GroupProject.pdfEcon3060_Screen Time and Success_ final_GroupProject.pdf
Econ3060_Screen Time and Success_ final_GroupProject.pdf
 
Sid Sigma educational and problem solving power point- Six Sigma.ppt
Sid Sigma educational and problem solving power point- Six Sigma.pptSid Sigma educational and problem solving power point- Six Sigma.ppt
Sid Sigma educational and problem solving power point- Six Sigma.ppt
 
Cell The Unit of Life for NEET Multiple Choice Questions.docx
Cell The Unit of Life for NEET Multiple Choice Questions.docxCell The Unit of Life for NEET Multiple Choice Questions.docx
Cell The Unit of Life for NEET Multiple Choice Questions.docx
 
Data Scientist Machine Learning Profiles .pdf
Data Scientist Machine Learning  Profiles .pdfData Scientist Machine Learning  Profiles .pdf
Data Scientist Machine Learning Profiles .pdf
 
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
 
Open Source Contributions to Postgres: The Basics POSETTE 2024
Open Source Contributions to Postgres: The Basics POSETTE 2024Open Source Contributions to Postgres: The Basics POSETTE 2024
Open Source Contributions to Postgres: The Basics POSETTE 2024
 
Telemetry Solution for Gaming (AWS Summit'24)
Telemetry Solution for Gaming (AWS Summit'24)Telemetry Solution for Gaming (AWS Summit'24)
Telemetry Solution for Gaming (AWS Summit'24)
 
Drownings spike from May to August in children
Drownings spike from May to August in childrenDrownings spike from May to August in children
Drownings spike from May to August in children
 
Namma-Kalvi-11th-Physics-Study-Material-Unit-1-EM-221086.pdf
Namma-Kalvi-11th-Physics-Study-Material-Unit-1-EM-221086.pdfNamma-Kalvi-11th-Physics-Study-Material-Unit-1-EM-221086.pdf
Namma-Kalvi-11th-Physics-Study-Material-Unit-1-EM-221086.pdf
 
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
 
一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理
一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理
一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理
 
一比一原版雷丁大学毕业证(UoR毕业证书)学历如何办理
一比一原版雷丁大学毕业证(UoR毕业证书)学历如何办理一比一原版雷丁大学毕业证(UoR毕业证书)学历如何办理
一比一原版雷丁大学毕业证(UoR毕业证书)学历如何办理
 
Discovering Digital Process Twins for What-if Analysis: a Process Mining Appr...
Discovering Digital Process Twins for What-if Analysis: a Process Mining Appr...Discovering Digital Process Twins for What-if Analysis: a Process Mining Appr...
Discovering Digital Process Twins for What-if Analysis: a Process Mining Appr...
 
一比一原版英国赫特福德大学毕业证(hertfordshire毕业证书)如何办理
一比一原版英国赫特福德大学毕业证(hertfordshire毕业证书)如何办理一比一原版英国赫特福德大学毕业证(hertfordshire毕业证书)如何办理
一比一原版英国赫特福德大学毕业证(hertfordshire毕业证书)如何办理
 
8 things to know before you start to code in 2024
8 things to know before you start to code in 20248 things to know before you start to code in 2024
8 things to know before you start to code in 2024
 

Env2Vec: Accelerating VNF Testing with Deep Learning

  • 1. Env2Vec: Accelerating VNF Testing with Deep Learning Guangyuan Piao, Pat Nicholson, Diego Lugones Nokia Bell Labs, Dublin, Ireland The 15th European Conference on Computer Systems, 30/04/2020
  • 2. © 2019 Nokia2 Introduction Current challenges for network engineers adopting modern DevOps: • Manage hundreds of distributed testbeds each running the new software build • Test new virtual network function (VNF) builds in shorter development cycles • Assure carrier-grade quality Objective: • Use deep learning (DL) techniques to accelerate VNF testing • Automate the diagnosis of software defects and anomalous builds • Point out testing engineers to problematic metrics Introduction → Problem → Proposal → Evaluation → Conclusions
  • 3. © 2019 Nokia3 Problem: Environments must be Considered! Introduction → Problem → Proposal → Evaluation → Conclusions 125 different models Varying accuracy
  • 4. © 2019 Nokia4 Proposed Approach: Env2Vec Anomaly Detection 1. Learn a universal resource characterization model 2. Execute the model in real time • to detect deviations between the inferred value (𝒚 𝒑 ′ ) and the actual observed RU of the VNF • Flag anomaly if there is a significant deviation (larger than γ x standard deviations) 𝒚 𝒑 ′ GRUs FNNs Lookup Tables (Embeddings) characterization model with DL Ruhistory (Resource Usage) CF (contextual features) • Workload metrics (WM) • Performance metrics (PM) EM (environment metadata) • Testbed, build type, test cases etc. Inferred resource usage (RU) (e.g., CPU) Introduction → Problem → Proposal → Evaluation → Conclusions
  • 5. © 2019 Nokia5 System Overview 1. Testbed data collection 2. Model training (daily) 3. Prediction pipeline 4. Raising alarms 5. Updating the model Introduction → Problem → Proposal → Evaluation → Conclusions
  • 6. © 2019 Nokia6 Evaluation: Carrier-grade VNF Testing Dataset • 600+ real-world testing environments • 125 build chains (testbed, build type, SUT, and test case) • 400,000 timesteps/data points measured at 15 minute intervals Evaluation metrics • Number of generated alarms • True and false alarm rate ( AT and AF ) • System with a higher number of generated alarms and higher AT is better Introduction → Problem → Proposal → Evaluation → Conclusions
  • 7. © 2019 Nokia7 Evaluation: Automating Anomaly Detection in New Builds Compared approaches • HTM-AD: AD based on RU time series only • Ridge: linear regression trained on each environment • Ridge(ts): considers RU history as features • RFNNall: Env2Vec without using embeddings Observations • Either the priority is raising (1) more performance problems automatically or (2) highly accurate alarms, • Env2Vec provides the best performance with γ = 1or 2 Introduction → Problem → Proposal → Evaluation → Conclusions
  • 8. © 2019 Nokia8 Evaluation: Reuse Embeddings in Unseen Environment Env2Vec • Reuses learned embeddings (capturing similarity of environments) • Construct environment embedding for a new unseen environment • Can detect performance problems better in unseen test executions lack of data for each unseen env. Introduction → Problem → Proposal → Evaluation → Conclusions
  • 9. © 2019 Nokia9 Conclusions • We proposed Env2Vec, which creates a universal easy-to-maintain model • Env2Vec automatically detects defects and bugs in new software builds by identifying performance degradation and informing the engineer • Can be extrapolated to previously unseen environments • Achieves accuracy between 86.2%-100%, while reducing the false alarm rate by 20.9%- 38.1% compared to other approaches Introduction → Problem → Proposal → Evaluation → Conclusions
  • 10. Q & A Contact information: Guangyuan Piao: guangyuan.piao@nokia-bell-labs.com Pat Nicholson: pat.nicholson@nokia-bell-labs.com Diego Lugones: diego.lugones@nokia-bell-labs.com