Spring Boot vs Quarkus the ultimate battle - DevoxxUK
Deep learning at nmc devin jones
1. Deep Learning at NMC
Devin Jones
Director Machine Learning Lab, Nielsen
2. Devin Jones
● Machine Learning & Statistics
○ Research
■ Classification
■ Inference
■ Time Series
○ Application
■ Large scale
■ Streaming
Introduction
● Columbia University
○ CS/ML
● Rutgers University
○ Statistics
○ Econ
○ Operations
Research
● Ad Tech (7 years)
3. ML at NMC
Intro to Deep Learning
Deep Learning Research at NMC
Agenda
5. “
”
Used to build larger audiences from smaller audience
segments to create reach for advertisers.
In theory, they reflect similar characteristics to a benchmark
set of characteristics the original audience segment
represents, such as in-market kitchen-appliance shoppers.
adage.com
The ML Challenge at NMC
Look Alike Modeling
9. Supervised What?
Machine Learning has two main categories:
Supervised Learning: Inferences on Labeled Data
Unsupervised Learning: Inferences on Unlabeled Data
10. Supervised vs Unsupervised Learning
Supervised:
Spam or Ham?
Unsupervised:
Clustering Wikipedia
Articles
12. The quality of data for a model will influence the model’s success
At NMC, we have access to high dimensional, sparse data:
The Feature Set & Scale
Models are trained in batches of 100,000 to 100,000,000 users depending on the purpose
~4,000 Segments ~200 Publishers User Agent
Geographic Info
(zip code)
+ + +
Resulting in over 100k features to choose from
13. To date, we have implemented these algorithms in our real time scoring engine:
We score billions of events per day using these models and our ML infrastructure
ML Algorithms at NMC
Binary Linear Model
kNN
Multinomial Linear Models
Online Learning for Linear
Models
Random Forest
And of course…
Deep Learning
17. 2
1 Recent Success in Deep Learning
NMC data is similar to Natural Language Processing (NLP) data
Certain ad targeting problems can be framed as expressive,
hierarchical relationships
MOTIVATION
3
18. Deep Learning: Recent Success
▪ AlphaGo defeats all world top
professional Go players
▪ Image and Speech recognition
exceed human abilities
▪ AI in consumer products:
Amazon Echo
Google Home
Autonomous Driving
All of these recent AI breakthroughs are
based on Deep Neural Networks!
19. NMC Data & NLP Data
NLP data:
Observation: [‘This’, ‘is’, ‘a’,
‘tokenized’, ‘feature’, ‘vector’,
‘used’, ‘for’, ‘machine’, ‘learning’,
‘in’, ‘NLP’]
NMC data:
User: [ ‘segment: Likes Outdoors’,
‘segment: Male 25-35’, ‘location:
New York, NY’]
35. Figure 2. Convergence of neural network model with
forward shortcut (Residual Net)
Figure 1. Convergence of neural network model without
forward shortcut(regular net)
Residual Network Convergence
37. Category Segment
City Prosperity
World-Class Health
Uptown Elite
Penthouse Chic
Metro High-Flyers
Prestige Positions
Premium Fortunes
Diamond Days
Alpha Families
Bank of Mum and Dad
Empty-Nest Adventure
Multi-level Hierarchical Classification
48. WEAK CONNECTIONS
most connections in deep neural network are
very weak and can be removed
TRIMMING
LOW ACCURACY IMPACT
the trimming has very little impact on the
accuracy
COMPRESSED DATA
the trimming models can be described by
sparse matrices, and thus the data in models
are highly compressed
51. Model Model File
Size (MB)
Trimming
Threshold
Accuracy Scoring
Time (ms)
Not trimmed 108 0.0 13.29 10.0
Trimmed 2.7 0.001 13.30 0.22
Trimming: Space, Time & Performance
inference improvement, in CPU time and storage
50x
52. Key Takeaways
Architecture:
● Residual Networks saved the day
● Leverage expressive power of DNN for your data
Inference:
● You might not need a GPU for Deep Learning
● Improvements can be made on Sparse Matrix Algebra
libraries
● Use trimming