Time-series data is more than just numbers. Identifying patterns and trends is important, but monitoring and detecting abnormalities in your data are relevant tasks that helps businesses to adapt and solve problems. Anomaly Detector is an Azure AI service that helps you foresee problems before they occur. Through an API, this service ingests time-series data and selects the best-fitting detection model for your data to ensure high accuracy. In this session, the Anomaly Detector service will be described, including terms, algorithms, parameters, costs, and best practices. A mobile app that analyzes time-series data and finds anomalies will be used to demonstrate the service.

1. #GlobalAzure
Boost your business reliability
with Azure Anomaly Detector
Luis Beltrán

2. #GlobalAzure
Luis Beltrán
• Researcher at Tomas Bata University in Zlín, Czech Republic.
• Lecturer at Tecnológico Nacional de México en Celaya,
Mexico.
• Passionate about Xamarin, Azure & AI
@darkicebeam
luis@luisbeltran.mx

3. #GlobalAzure
Agenda
• Anomaly Detection Introduction
• Azure Cognitive Services
• What is Anomaly Detector service?
• Demo
• Final thoughts
• Q & A
Download the slides: https://bit.ly/LuisGAMunich

4. #GlobalAzure

5. #GlobalAzure
Anomaly Detection
• Anomaly Detection is the
process of finding outliers,
unexpected, rare items, or events
from time series.
• Anomaly Detection recognizes
unusual data behavior patterns
that are not consistent with the
expected values.
Value
Time

6. #GlobalAzure
Anomaly Detection assumes that:
• Anomalies occur very rarely in the data.
• There is a significant difference between the features of anomalies
and normal data.
In short, anomaly detection identifies data points that do not fit well
with the rest of the data.

7. #GlobalAzure
The importance of Anomaly Detection
Anomalous data can be
connected to problems such
as:
• bank fraud
• medical problems
• structural defects,
• malfunctioning equipment
• …
It is important to determine which
data points can be considered
outliers, because identifying these
events is relevant for data owners
and businesses.

8. #GlobalAzure
Azure Cognitive Services
Perception Comprehension
Vision Speech Language Knowledge
Computer Vision
Face/Emotion Recognition
OCR/Handwriting
Custom Vision
Video Indexer
Text-to-Speech
Speech-to-Text
Translator
Custom Speech
Language Understanding
Text Translator
Text Analytics
QnA Maker
Bing Custom Search
Bing Autosuggest
Bing Image Search
Bing News Search
microsoft.com/cognitive

9. #GlobalAzure
Anomaly Detector
• An AI service that assists you into foreseeing problems before they
occur.
• By ingesting time-series data, it selects the best-fitting detection
model for the data to ensure high accuracy.
• It determines boundaries, expected values and which data points are
anomalies
Try the service: https://aka.ms/adDemo (requires a key)

10. #GlobalAzure
• Anomaly Detector provides a RESTful API and SDKs for several
technologies.
• Currently in preview
• It can be deployed to the edge with Docker containers.
• No other cloud provider offers anomaly detection as an AI service.

11. #GlobalAzure
Anomaly Detector API
Currently, the API offers three
functionalities:
• Find anomalies for the entire
series in batch
• Detect anomaly status of the
latest point in time series
• Find trend change point for the
entire series in batch
API reference:
https://westus2.dev.cognitive.microsoft.com/docs/services/AnomalyDetector/operations/
post-timeseries-entire-detect

12. #GlobalAzure
Input data
{
"granularity": "monthly",
"sensitivity": 95,
"series": [
{
"timestamp": "2018-03-01T00:00:00Z",
"value": 32858923
},
…
{
"timestamp": "2018-04-16T00:00:00Z",
"value": 35250105
}
]
}

13. #GlobalAzure
Find anomalies for the entire series in batch.
This operation generates a model using an entire series, each point is detected with the same
model. Points before and after a certain point are used to determine whether it is an anomaly.
The entire detection can give the user an overall status of the time series.
Request URL
https://{endpoint}/anomalydetector/v1.0/timeseries/entire/detect
{
"period": 7,
"expectedValues": [ 32894418.961561, 29707932.244719, … ],
"isAnomaly": [ false, false, … ],
"isNegativeAnomaly": [ false, false, … ],
"isPositiveAnomaly": [ false, false, … ],
"upperMargins": [ 1644720.948078, 1485396.612235, … ],
"lowerMargins": [ 1644720.948078, 1485396.612235, … ],
}
Content Response

14. #GlobalAzure
Find trend change point for the entire series in batch.
This operation generates a model using an entire series, each point is detected with the same
model. With this method, points before and after a certain point are used to determine
whether it is a trend change point. The entire detection can detect all trend change points of
the time series.
Request URL
https://{endpoint}/anomalydetector/v1.0/timeseries/changepoint/detect
{
"period": 4,
"confidenceScores": [ 0, 0.0018, 0.3281 … ],
"isChangePoint": [ false, false, true, … ],
}
Content Response

15. #GlobalAzure
Detect anomaly status of the latest point in time series.
This operation generates a model using points before the latest one. With this method, only
historical points are used to determine whether the target point is an anomaly. The latest point
detecting matches the scenario of real-time monitoring of business metrics.
Request URL
https://{endpoint}/anomalydetector/v1.0/timeseries/last/detect
{
"isAnomaly": false,
"isPositiveAnomaly": false,
"isNegativeAnomaly": false,
"period": 12,
"expectedValue": 809.2328084659704,
"upperMargin": 40.46164042329852,
"lowerMargin": 40.46164042329852,
"suggestedWindow": 49
}
Content Response

16. #GlobalAzure
Sensitivity
The API sensitivity value has a direct effect on the boundaries created
for anomaly detection.

17. #GlobalAzure
Sensitivity = 1 (99 %)

18. #GlobalAzure
Sensitivity = 3 (97 %)

19. #GlobalAzure
Sensitivity = 5 (95 %)

20. #GlobalAzure
Creating an Anomaly Detector resource in the
Azure Portal

21. #GlobalAzure
Get the key and endpoint

22. #GlobalAzure
Demo
Open source project available on GitHub:
https://github.com/icebeam7/PriceAnomalyDetection

23. #GlobalAzure
Pricing

24. #GlobalAzure
Recommendations
• Data points should be separated by the same interval.
• Missing less than 10% of the expected number of points in your data
shouldn’t impact negatively the anomaly detection process.
• Include at least 12 data points if your data doesn't have a clear
seasonal pattern (max. is 8640 points).
• Or include at least 4 pattern occurrences if your data does have a
clear seasonal pattern.

25. #GlobalAzure
Technical Details
Microsoft’s innovation provides a
generic framework to plug in
different algorithm ensembles to
handle a wide spectrum of different
time series. Following algorithms
have been used:
• Fourier Transformation
• STL Decomposition
• Dynamic Threshold
• Extreme Studentized Deviate (ESD)
• Z-score detector
• SR-CNN

26. #GlobalAzure
Anomaly Detection Scenarios

27. #GlobalAzure
Call to Action
Cognitive Services Lab In Action (Build 2018)
https://channel9.msdn.com/Events/Build/2018/THR3107
Anomaly Detector
https://azure.microsoft.com/en-us/services/cognitive-
services/anomaly-detector/
AI Show – Anomaly Detector videos
https://channel9.msdn.com/Tags/anomaly-detector

28. #GlobalAzure
Call to Action
Build an Enterprise-ready Anomaly Detection Solution
https://info.microsoft.com/ww-ondemand-rapidly-build-an-
anomaly-detection-solution.html
https://docs.microsoft.com/en-us/azure/cognitive-
services/anomaly-detector/anomaly-detector-container-howto
Install and run Anomaly Detector containers

29. #GlobalAzure

30. #GlobalAzure
Thank you for
your attention!
Luis Beltrán
Tomás Bata University in Zlín
Tecnológico Nacional de México en Celaya
luis@luisbeltran.mx luisbeltran.mx @darkicebeam
GitHub:
https://github.com/icebeam7
LinkedIn:
https://linkedin.com/in/luisantoniobeltran
SlideShare:
https://slideshare.net/icebeam
YouTube:
https://youtube.com/user/darkicebeam
About Me:
https://about.me/luis-beltran