Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
A Sentient
Network
How High-velocity Data and Machine Learning
will Shape the Future of the Communication Services
OPNFV S...
A paradigm shift
NFV/SDN is empowering
Towards … ?
“The more real-time and
granular we can get, the more
responsive, and more
competitive, we can be.”
Peter Levine | Andrees...
New Paradigm Shift in Infrastructure: NFV/SDN
Domain specialized software on standard
hardware, delivered from the cloud
-...
High-velocity Cloud Empowers Business Transformation
Mobile' Infrastructure
Content' Distribution
Edge Computing
High'Velo...
High-velocity Data with Machine Learning
Telemetry,
IoT
sensors,
System
logs,
Monitors,
Mobile
devices …
Transmi
ssion of
...
A Closer Look: Data Analytics Velocity
“Meta Dimensionality” of Data
Gigabyte, Terabyte, Petabyte, Exebyte, Zettabyte, Yottabyte
uSec,mSec,Sec,min,days,months,ye...
Let’s look at some
examples in networking…
Learn to
optimize
resource
management
Automatically Adjust Resources to Maintain SLA
Systems can respond to usage spikes in real-time,toreallocate resources and...
Continuous Resource Optimization by Reinforcement
Learning
! Modeled as a Markov
Decision Process
! Learning probability
d...
Learn to
defeat
intruders
Classification by Concept Adapting Decision Tree
! Rules programmingis
labor intensive,error
prone, static
! Let algorithm...
Uncovering Unusual Hidden Activity by Monitoring Entropy
! Entropy in a moving
time window captures
the normal hummingof
t...
Clustering Users based on Behavior Patterns
! Non-parametric model
can be used for latent
features,overlapping
clusters an...
Learn to
better service
customers
! Mining telco CDR’s to
evaluate risks from
customer churn
! Combining locationand
real-time system infoto
pinpoint qualit...
Collaborative Learning by Sensing User Mood
Facial expressions
Pulse rate
Skin conductivity
Brain computer
interface (BCI)...
20
“How is Seamless Mobilitypowered by High
Velocity Cloud?”
Seamless Mobility by Contextual Learning
Live machine learnin...
Learn to
protect
privacy
Differential Privacy in Big Data and Machine Learning
! Anonymizationis not
enough
! Differential Privacy(!-
DP) provides ...
Computing on and Learning from Encrypted Data
transformed+
queryplain+query+
under+passive+attack
Application
decrypted+
r...
So, Any Takeaways for OPNFV ?
• Collect data
• Put data in an open format
• Consider privacy and security on day one
• Don...
“The future is already here – it’s just not very evenly distributed.”
William Gibson
A sentient network - How High-velocity Data and Machine Learning will Shape the Future of Communication Services
Upcoming SlideShare
Loading in …5
×

A sentient network - How High-velocity Data and Machine Learning will Shape the Future of Communication Services

281 views

Published on

Dell's Distinguished Engineer Wenjing Chu discusses innovations in applying Machine Learning to solve challenges in Telco/Communication Services, and predicts that the future is a Sentient Network powered by Machine Learning that can handle real-time high-velocity data.

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

A sentient network - How High-velocity Data and Machine Learning will Shape the Future of Communication Services

  1. 1. A Sentient Network How High-velocity Data and Machine Learning will Shape the Future of the Communication Services OPNFV Summit, Burlingame CA, November 9-12, 2015 Wenjing Chu Distinguished Engineer Dell Research Member of the Board and TSC of OPNFV
  2. 2. A paradigm shift NFV/SDN is empowering Towards … ?
  3. 3. “The more real-time and granular we can get, the more responsive, and more competitive, we can be.” Peter Levine | Andreessen Horowitz A Sentient Network 1 Elastic on-demand capacity Open software architecture promises flexible elastic capacity that can be rapidly provisioned and dynamically managed Data-driven operation automation Virtualization unleashes the latent value in the real-time data to optimize resource allocation and assure SLA Scalable infrastructure Standard open architecture infrastructure delivers capacity, cost efficiency, and right-sized reliability 2 3 Self-learning security and privacy Self-learning algorithms from real-time data delivers ultimate security and privacy at the same time 4 Machine intelligent user services Advances in Machine Learning promise continuous improvements in user experience 5
  4. 4. New Paradigm Shift in Infrastructure: NFV/SDN Domain specialized software on standard hardware, delivered from the cloud - Dramatically cuts CapEx & OpEx - Enhances service velocity - Enables Big Data driven business model
  5. 5. High-velocity Cloud Empowers Business Transformation Mobile' Infrastructure Content' Distribution Edge Computing High'Velocity'Cloud Packet Velocity • 100X moreperformance • 50X morecustomers Service Velocity • Deploy services in minutes vs months • Empower new, innovative business models Data Analytics Velocity • Sub-second real-time streaming analytics • Sentient intelligence
  6. 6. High-velocity Data with Machine Learning Telemetry, IoT sensors, System logs, Monitors, Mobile devices … Transmi ssion of data in streams Transfo rmation Learning in real- time Action on intellige nce
  7. 7. A Closer Look: Data Analytics Velocity
  8. 8. “Meta Dimensionality” of Data Gigabyte, Terabyte, Petabyte, Exebyte, Zettabyte, Yottabyte uSec,mSec,Sec,min,days,months,years… SizeTime
  9. 9. Let’s look at some examples in networking…
  10. 10. Learn to optimize resource management
  11. 11. Automatically Adjust Resources to Maintain SLA Systems can respond to usage spikes in real-time,toreallocate resources and maintainSLAs.
  12. 12. Continuous Resource Optimization by Reinforcement Learning ! Modeled as a Markov Decision Process ! Learning probability distributionby Bayesian inference ! Q-Learning, Deep Q- Network ! Consensus optimization Wikipedia: MDP
  13. 13. Learn to defeat intruders
  14. 14. Classification by Concept Adapting Decision Tree ! Rules programmingis labor intensive,error prone, static ! Let algorithm learns a DT (or a forest) on its own ! Concept adaptability: incorporate new, forget old Packets > 10 yes no Protocol=http Packets > 10 no yes Bytes > 60k yes no Protocol=ftp Data stream Data stream
  15. 15. Uncovering Unusual Hidden Activity by Monitoring Entropy ! Entropy in a moving time window captures the normal hummingof the system ! Out of ordinarymove of entropy plus context suggest attack vs. flash crowd
  16. 16. Clustering Users based on Behavior Patterns ! Non-parametric model can be used for latent features,overlapping clusters and infinite data ! Eg Dirichlet process, Gaussian process ! A cluster of ‘users’of abnormal behavior are suspects
  17. 17. Learn to better service customers
  18. 18. ! Mining telco CDR’s to evaluate risks from customer churn ! Combining locationand real-time system infoto pinpoint qualityissues ! Machine learning algorithm offers more precision Proactive Customer Support and Retention The peaks indicate areas of highest risk with more precision than traditional linear regression (the dotted line). Creative commons http://scicomp.stackexchange.com/
  19. 19. Collaborative Learning by Sensing User Mood Facial expressions Pulse rate Skin conductivity Brain computer interface (BCI) Voice pitch Remote UX metrics Media audience response Improve MOOC, CBT VR/AR style UI
  20. 20. 20 “How is Seamless Mobilitypowered by High Velocity Cloud?” Seamless Mobility by Contextual Learning Live machine learning algorithms ensure quality, security and seamless mobility. High-velocity Cloud High-velocity Analytics
  21. 21. Learn to protect privacy
  22. 22. Differential Privacy in Big Data and Machine Learning ! Anonymizationis not enough ! Differential Privacy(!- DP) provides a formal guarantee & a mechanism for tradeoff ! DP may also help avoid False Discovery Dr.Katrina Ligget, CalTech
  23. 23. Computing on and Learning from Encrypted Data transformed+ queryplain+query+ under+passive+attack Application decrypted+ results encrypted+ results DB+server encrypted+DB Proxy Secret Secret computation+on+ encrypted+data+≈+ regular+computation ! Stores+schema++ and+master+key ! No+query+execution trusted+client?side ! Data loss is prevalent everywhere you look ! Data privacy responsibilityis unclear ! Practical system can be deployed with strong encryptionwithout the risk of key disclosure ! Different algorithm for different computation Dr. Laruca Popa, UC Berkeley
  24. 24. So, Any Takeaways for OPNFV ? • Collect data • Put data in an open format • Consider privacy and security on day one • Don’t tie data to a specific implementation of a specific design • Must consider the time dimension of data, e.g. TSDB, streaming
  25. 25. “The future is already here – it’s just not very evenly distributed.” William Gibson

×