This document discusses challenges in building operational AI systems and provides guidance on how to approach AI development. It notes that while AI will significantly impact many aspects of life, building reliable operational AI systems faces hurdles such as a lack of high-quality data, problems being underspecified, tools not being designed for real-world use, and proof-of-concept systems failing to scale. The document offers design guidelines for AI such as ensuring ecological validity, being data-driven, making all aspects immutable and version-controlled, considering scaling from the start, and avoiding strong dependencies. It also provides advice on getting started with AI through self-study, using existing solutions, implementing non-critical test cases, and growing knowledge and processes

1. Challenges in building operational AI
And why you should not be afraid of them daniel@peltarion.com

2.
AI will impact every facet of life,
just like the industrial revolutions did

3. Introduction to AI01/

4. Peltarion - the operational AI platform
Technological disruptions
Internet
Cost of
Distributing Information
AI
1950 1960 1970 1980 1990 2000 2010
Transistor
Cost of
Arithmetic
Cost of
Prediction

5. Peltarion - the operational AI platform
History of Artificial Intelligence
Use data to automatically
learn to make predictions
Learn to both represent data
and make predictions
1950 1960 1970 1980 1990 2000 2010
Artificial
Intelligence
Science and engineering
of building intelligent machines
Machine
Learning
Deep
Learning

6. Sundar Pichai / CEO Google
“AI is one of the most important
things humanity is working on.
It’s more profound than, I don’t
know, electricity or fire”

7. Author: Owen Williams, The Kasparov Agency
Licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license
“I could feel – I could smell –
a new kind of intelligence across
the table.”
Garry Kasparov

8. 3
1
Neural network
classifying digits
46 lines of code

9. Artificial Neural Networks
Can be fed raw data, adapt, learn to interpret and conclude
Peltarion - The Operational AI Platform

10. A developer02/
perspective on
Deep Learning

11. “We need to talk
about CSV-files”

12. Detect the data types in CSV files with errors?
... ... ... ... ...
2012-04-23T18:25:43.511Z 12 54.4 283 Jennings Lane, Montgomery Village, MD 20886 Tree
2013-05-22T11:15:23.331Z Dog 98012.2345 9341 553 45
2015-01-03T 95 23.1 8529 West Pin Oak St. Copperas Cove, TX 76522 Sidewalk
2014-12-01T09:42:55.513Z 14 23 822 West Orange Dr. Long Branch, NJ 07740 Person
... ... ... ... ...

13. Where are the cats and dogs in this image?

14. Indicate the areas with brain tumors

15. How much power will these
windmills produce tomorrow?

16. What do these problems have in common?
Hard or “unsolvable”
1
Deep Learning can solve them
2

17.
What makes Deep Learning so
versatile and powerful?

18. Modelling flexibility
Multiple data types in same model
Re-use parts of models

19. Modelling possible for non-experts
Support vector machine solution Deep learning solution
Example: Retinal disease diagnosis system
VS

20. Training time can be long and demanding
But lookups can still be done really fast*
Ready for real-world applications
* < 200ms even for advanced models

21. Real world ready or not?
Every technological shift has two phases

22. Steam power
150 years

23. Electricity
100 years

24. Digitisation
40 years

25. AI is becoming operational as we speak
$1,000bn$26.2bn$524bn
$121bn$28bn$108bn $35bn
$860bn

26. Why many fail03/

27. POC & Research
New tech
Experimentation
Machine Learning
Product development
Reliability & performance
Maintain & refactor
Software engineering
Different cultures

28. High risk, high reward
Sometimes it is easy,
sometimes there is no
solution at all
Illustration by Tony Oliver.
Licensed under Creative Commons Attribution-ShareAlike 2.0 Generic

29. Not enough data or
lack of quality
Underspecified problem
Photo by: Austin Mills
Licensed under Creative Commons Attribution-ShareAlike 2.0 Generic

30. Existing tooling is not
made for solving
operational AI problems
Mostly tilted towards research,
not development and solutions

31. Glue code &
dependency hell
pip install tensorflow
pip install keras
pip install jupyter
pip install pandas
pip install sklearn

32. Proof of concept != Operational AI
Just because it works in the lab does not mean it works in the real world

33. ?!?!
Back to the drawing board...
Uhm...A bit rounded… Perhaps…
Heh.. Let’s move on...
8 8 8

35. AI proof of concept

36. Operational AI system
Adapted from Hidden Technical Debt in Machine Learning Systems

37. AI design
guidelines
04/

38. Take nothing for granted - until you’ve tried it
Experiment & learn
1
Ecological validity
2
Be prepared to fail
3

39. Be data and use-case driven
Clear problem statement
1
No data - no fun
2
Background illustrations: User by Justiconnic & Data by priyanka from the Noun Project

40. Immutability - it is just easier that way
for any distributed system
1
Foundation for versioning
2
Background illustration: Protected Edit by Mun May Tee from the Noun Project

41. Version everything
Background illustration: Connection by Jason D. Rowley from the Noun Project
Tuning of deep learning parameters is not self-explanatory: Document changes!
Model history
1
Model used in serving
2
Version of data used
3

42. Think of scaling from the beginning
Training can take a long time
1
Experimentation - parallel jobs
2
Engineering debt of trying to make a non-scalable system scalable can be enormous

43. Know your GDPR
Photo by: Dennis van der Heijden
Licensed under Creative Commons Attribution-ShareAlike 2.0 Generic

44. Costs matter… sometimes a lot
Massive scaling on expensive GPU:s may quickly put an end to serious AI effort
Add monitoring
1
Hardware requirements?
2
Early stopping
3

45. Avoid glue code and strong dependencies
Use libraries only if you must
1
Wrap vital libraries
2
Semantic versioning
4
Create your own formats
3
Background illustration: Glue by David Marioni from the Noun Project

46. How do I get05/
started with AI?

47. AI - from A-Z
Learn & have fun
Devote time for self-studies
Use existing solutions
Build demos
01
Get ready
Data & use case inventory
Spread knowledge
Start collecting missing data
02
Implement
One non-critical case with
business value
Existing vs your own tech?
Guidelines
03
Grow & improve
Spread AI-knowledge in
organisation
Processes & workflows
Recruit
04

48. Do not reinvent the
wheel each time. Think
in terms of a
framework early on.
Illustration by Loco Steve.
Licensed under Creative Commons Attribution-ShareAlike 2.0 Generic

49. Minimalistic framework for modelling

50. Minimalistic framework for serving

51. Consider existing solutions
Building a framework is not a small task

52. Take nothing for granted, until you’ve tried it

53. Help us with data* to
train an AI beer model
Photo by: Darien Graham-Smith
* visit us in our booth and drink beer
daniel@peltarion.com // bit.ly/jfokus-ai
Thank you!