My slides of the webinar between Anodot and Cloudera about real time anomaly detection in time series

1. 1
Webinar Anodot-
Cloudera
Meir TOLEDANO, Data science team leader

2. 2
in few words...

3. 3
Our customers use Anodot to
identify revenue-critical issues.
Our platform leverages AI to constantly
monitor and correlate business performance,
providing real-time alerts and forecasts.

4. What we do?
Partner
Monitoring
Revenue and Cost
Monitoring
Customer Experience
Monitoring
DAU
MAU
Retention
Usage Flows
Funnels
APIs
Partners
Affiliates
Customers (b2b)
3rd party services
Purchase/sales funnels
Price & promo glitches
Payment gateways
Cloud costs
Ad costs

5. 55
FROM BILLIONS TO ONES

6. 66
THE COMPREHENSIVE SOLUTION
Automatically analyzes all data continuously, interacts with us only when relevant
CORRELATE ACT
● All the data
● Cross silo
● Continuous learning of 100%
● Adjusts to changes
● Root cause guidance
● Multiple correlation algorithms
● Realtime
● Significance learning
ANALYZECOLLECT
USER
PRODUCT
SYS. HEALTH
INFRA

7. 77
THE COMPREHENSIVE SOLUTION
Automatically analyzes all data continuously, interacts with us only when relevant
CORRELATE ACT
● All the data
● Cross silo
● Continuous learning of 100%
● Adjusts to changes
● Root cause guidance
● Multiple correlation algorithms
● Realtime
● Significance learning
ANALYZECOLLECT
USER
PRODUCT
SYS. HEALTH
INFRA

8. 88
REAL TIME ANOMALY DETECTION STRATEGY
Anomaly =
Too far from the
expected behavior
Prediction range

9. 99
Business problem Mathematical / algorithms problem
REAL – TIME ANOMALY
DETECTION PROBLEM
Learning: Time series
modeling
Inference: Real time
prediction calculation
REAL TIME ANOMALY DETECTION STRATEGY
CONDITIONAL PREDICTION PROBLEM

10. 1010
CHALLENGES FOR LEARNING “NORMAL BEHAVIOR”
ADAPTATIONSEASONAL PATTERNSIGNAL DISTRIBUTIONSIGNAL TYPE
● Stationary / non
stationary
● Regularly Irregular
sampling
● Discrete/Real value
● Single/Mixture models
● Symmetric/non-symmetric
● Continuous/discrete
● Seasonal/non seasonal
● Single/multiple seasonal
patterns
● Additive/Convolutional
multi-seasonal patterns
● Optimal adaptation
during normal times
● Optimal adaptation
during anomalies
● Optimal adaptation
following anomalies

11. 1111
PATENT US10061632B2 PATENT US10061677B2 PATENT US20160210556A1 PATENT PENDING
ANOMALY SCORE SEASONALITY LEADING DIMENSIONS HD BASELINE AT SCALE
API requests for service 123
Drop in Play for app123 , source-promo
Traffic for partnerAccount, partnerName
Login Errors

12. 1212
• The system find the daily pattern (and not the 3h sub-pattern)
• The conditional prediction begins then to be very accurate.
• We are now able to catch even the smallest anomaly.
SEASON IS A KEY COMPONENT

13. 1313
METHODS TO DETECT SEASONALITY
Finding maximum(s) in
Fourier transform of signal
• Challenging to
detect low
frequency seasons
• Challenging to
discover multiple
seasons
• Sensitive to
missing data
Exhaustive search based
on cost function
• Computationally
expensive
• Robust to gaps
• Challenging to
discover
multiple
seasons
Finding Maximums in the
Auto Correlation Function
(ACF)
• Accurate detection
of single season –
challenging for
multi-season
• Computationally
expensive
• Robust to gaps
Finding maximums in
Auto-correlation of
signal
Computationally
expensive
More robust to
gaps
The Vivaldi Algorithm:
Fast ACF approach
• Smart sampling of the
ACF coefficients –
O(NlogN) computation
• Filter bank for detection
of multiple seasonal
patterns
• Scales to millions of
time series

14. 1414
ANODOT’S SCALE
5.8 BILLION
DATA POINTS PER DAY
120 MILLION
UNIQUE METRICS
240 MILLION
MODELS
500 MILLION
CORRELATIONS
14 MILLION
SEASONAL METRICS
30 TYPES
OF LEARNING ALGORITHMS

15. 15
THANK YOU
https://www.linkedin.com/in/meirtoledano/