AI is a scoring machine to automate data processing to get an inference. Quality of the data defines the quality of the AI Quality of feature engineering either does No domain knowledge - no AI Data Scientist works on data and models, to build an application Engineers and DevOps are required In the real world a model will encounter a case it was never trained for. Continuous Retraining is a must.

1. AI / ML 101
Rustem Zakiev

2. Credits
AI/ML Operations Automation

3. What is AI?
Karakuri Archer
?
=>
3

4. Artificial Intelligence (AI) and Machine Learning (ML)
4
AI = Task + [ Data ] + Algorithm
Magic
Model
Domain
Knowledge

5. Robots - Informed Path Search
Dijkstra’s Algorithm A* Algorithm
5
Animations Source: Wikipedia.org

6. Diagnostics - Falling Rule List
6https://arxiv.org/pdf/1411.5899.pdf

7. 7
00101010
AI = Task + [ Data ] + Algorithm
Artificial Intelligence (AI) and Machine Learning (ML)

8. Machine Learning: Features Engineering
8
https://developers.google.com/machine-learning/crash-course/representation/feature-enginee
ring

9. Machine Learning: Where are the Features?
9
“Life”
[ 0.02, 11, 345.43, …, 100500 ] [ 42 ]

10. Tasks
10
● Predicting the demand [for coffee, upon the weather forecasts]
● Advertising budgets allocation
● Predicting the malfunction of a drill rod
[Multi]Linear/Logistic Regression

11. [Multi]Linear regression
11https://ml-cheatsheet.readthedocs.io/en/latest/linear_regression.html
yi = wxi + b

12. Logistic regression
12
https://en.wikipedia.org/wiki/Logistic_regression
https://hackernoon.com/introduction-to-machine-learning-algorithms-logistic-regression-cbdd82d8
1a36

13. Tasks
13
● Market segmentation
● Recommendations service
● Car sharing demand prediction
● Ads targeting
Clustering

14. Clustering: K-means
14https://www.kdnuggets.com/2018/06/5-clustering-algorithms-data-scientists-need-know.html

15. Clustering: Agglomerative Hierarchy
15https://www.kdnuggets.com/2018/06/5-clustering-algorithms-data-scientists-need-know.html

16. Clustering: DBSCAN
16https://www.kdnuggets.com/2018/06/5-clustering-algorithms-data-scientists-need-know.html

17. ● Image recognition
● Sound recognition
● Data generation/compressing/processing
Tasks
17
Neural Networks

18. Neural Networks
18https://towardsdatascience.com/https-medium-com-piotr-skalski92-deep-dive-into-deep-netwo
rks-math-17660bc376ba

19. Neural Networks: error back propagation
19https://towardsdatascience.com/https-medium-com-piotr-skalski92-deep-dive-into-deep-netwo
rks-math-17660bc376ba
Y = WX + B
X → → Y

20. - Anomaly detection (e.g. Fraud detection)
- Facial recognition
- Self-driving Vehicles
- Health/Disease Diagnosis
- Schedule optimisation upon points state monitoring
- Machine Translation
- Natural Language Processing (chat bots, documents
categorisation, texts summarisation etc.)
- ...
Complex tasks
20

21. Model Training:
Basic [supervised] cycle
21
Validation

22. Model Training: Quality of the Data => Quality of the Model
22
https://www.youtube.com/watch?v=-i7HMPpxB-Y

23. Model Training: Active Learning
23

24. Neural Networks: Convolution
24https://blog.goodaudience.com/convolutional-neural-net-in-tensorflow-e15e43129d7d

25. 25https://blog.goodaudience.com/convolutional-neural-net-in-tensorflow-e15e43129d7d
Neural Networks: Convolution

26. 26https://blog.goodaudience.com/convolutional-neural-net-in-tensorflow-e15e43129d7d
Neural Networks: Convolution

27. Artificial Intelligence: Pipeline
27
https://www.youtube.com/watch?v=-i7HMPpxB-Y

28. 28
Pre-
processing
Bounding
boxes
Classification
Post-
processing
Application
Data Control
Artificial Intelligence: Pipeline

29. AI and ML team
29
AI = Task + [ Data ] + Algorithm
Domain
Expert
Data
Scientist
Application
[Production]
Data
Predictions to the
consequent
operations
DevOps
+
Software Engineer
Runtime Infrastructure

30. Neural Networks: Autoencoder
30
https://en.wikipedia.org/wiki/Autoencoder#/media/File:Autoencoder_structure.png

31. Neural Networks: Generative Adversarial Networks (GAN)
31
https://towardsdatascience.com/generative-adversarial-networks-explained-34472718707a

32. AE + GAN: Faces Photo Restoration
32
https://www.cc.gatech.edu/~hays/7476/projects/Avery_Wenchen/

33. 33
https://www.cc.gatech.edu/~hays/7476/projects/Avery_Wenchen/
AE + GAN: Faces Photo Restoration

34. 34
https://www.cc.gatech.edu/~hays/7476/projects/Avery_Wenchen/
AE + GAN: Faces Photo Restoration

35. Image Restoration: NVIDIA Research
35
https://news.developer.nvidia.com/new-ai-imaging-technique-reconstructs-photos-with-reali
stic-results/?ncid=nv-twi-37107

36. Challenges: Data Anomaly Detection
36

37. Challenges: Concept Drift
37

38. Data Anomalies and Concepts Drifts
38

39. 39
Data Anomalies and Concepts Drifts

40. 40
Data Anomalies and Concepts Drifts

41. Takeaways
41
● AI is a scoring machine to automate data processing to get an
inference.
● Quality of the data defines the quality of the AI
● Quality of feature engineering either does
● No domain knowledge - no AI
● Data Scientist works on data and models, to build an application
Engineers and DevOps are required
● In the real world a model will encounter a case it was never trained
for. Continuous Retraining is a must.

42. Thank you!
Rustem Zakiev
rzakiev@hydrosphere.io
hydrosphere.io
https://github.com/Hydrospheredata
twitter.com/hydrospheredata
facebook.com/hydrosphere.io