Slides to a talk from @Chris_Betz, (https://data42.de) on AI, artificial intelligence, machine learning. What's driving AI hype and what's behind it. Understand general concepts and dig deep into explainability, debugging, verification, testing of machine learning solutions.

2. Debugging AI
Dr. Christian Betz

3. 3Photo by Gratisography from Pexels
„Debugging AI“ is deliberately
ambiguous.

4. What drives the current AI hype?
4

5. 5
Compute Power
https://www.youtube.com/watch?v=0ibVhtuQkZA

6. 6
More data
Photo by Negative Space from Pexels

7. 7
New business opportunities
(plus attention)
By Dllu - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=63450446

8. 8
and language processing.
28 Chapter 2 – Why is AI important?
AI is important because, for the first time, traditionally human
capabilities can be undertaken in software inexpensively
and at scale. AI can be applied to every sector to enable
new possibilities and efficiencies.
40 Chapter 3 – Why has AI come of age?
Specialised hardware, availability of training data, new
algorithms and increased investment, among other factors,
have enabled an inflection point in AI capability. After seven
false dawns since the 1950s, AI technology has come of age.
constraints of human experience
82 Chapter 6 – The war
While demand for AI professiona
winners and losers are emerging
Part 3: The AI Disrup
96 Chapter 7 – Europe’s
The landscape for entrepreneurs
AI startups are maturing, bringing
industries, and navigating new op
While the UK is the powerhouse o
France may extend their influence
https://www.mmcventures.com/wp-content/uploads/2019/02/The-State-of-AI-2019-Divergence.pdf
Speed of development

9. https://www.mmcventures.com/wp-content/uploads/2019/02/The-State-of-AI-2019-Divergence.pdf 9
years (Fig. 23), to an estimated $15bn in 2018 (CB Insights,
MMC Ventures).
Today’s leading technology companies – including Apple,
Amazon, Facebook, Google, IBM, Microsoft and Salesforce
– are also spending heavily on research and personnel to
develop and deploy AI. Internal corporate investment on AI,
among just the top 35 high tech and advanced manufacturing
companies investing in AI, may be 2.0x to 4.5x greater than the
capital invested by venture capital firms, private equity firms
and other sources of external funding combined (McKinsey),
further catalysing progress.
have increased fifteen-fold
in five years, to an estimated
$15bn in 2018.
(CB Insights, MMC Ventures)
Source: CB Insights, MMC Ventures
Fig 23. Venture capital investment in AI has increased 15-fold in five years
0
200
400
600
800
1000
1200
0
2
4
6
8
10
12
14
16
20132012 2014 2015 2016 2017 2018E
Fig. X: Venture capital investment in AI has increased 15-fold in five years
AIdeals
Disclosed Funding (right axis)
Number (left axis)
AIdealinvestment($billion)
High valuation

10. 10
Fear
Photo by samer daboul from Pexels

11. It’s important to understand…
11
that these factors impact your AI project.

12. What is AI?
Strong AI vs. Weak AI
AI is used as a generic term for a set of tools to cope with a certain set of
problems.
Machine Learning is a subset of this AI-toolset: „Programming by example“.
Other subsets are knowledge representation, planning, reasoning.
AI uses probabilistic logic instead of boolean logic.
12

13. 13
„The brown quick fox jumps over the lazy dog“
https://www.mcohen.io/2017/machine-learning-explained-in-three-easy-steps/
brown quick

14. Properties of AI problems
Hard to code „by hand“
• Requires non-formalized knowledge (experiential knowledge)
• Or even not yet existing knowledge
Afflicted with uncertainty (or missing information)
Changes rapidly (making it unreasonable to adopt software
manually)
14

15. Playing Chess, playing Go
15
f(s)
f(s)
f(s)
f(s)
f(s)
f(s)
f(s)
f(s)
1
0
0
1

16. Another typical problem: Computer vision
16

17. This is typically solved using „Deep neural
networks“
Neural networks are simple (sort of):
17
⨁
Input 1
Input 2
Input 3
Output
Weight 1
Weight 2
Weight 3

18. Deep Networks combine these simple
elements.
18
https://courses.cs.ut.ee/MTAT.03.291/2015_spring/uploads/Main/Artificial%20neural%20network%20for%20image%20classification.pdf
complex patterns in the data as it is propagated throught the network. For example, if the network is
given a task to recognize a face, the first hidden layer might act as a line detector, the second hidden
takes these lines as input and puts them together to form a nose, the third hidden layer takes the
nose and matches it with an eye and so on, until finally the whole face is constructed. This hierachy
enables the network to eventually recognize very complex objects.3
As stated before, the network is able to accurately approximate an arbitrary function by altering its
weights in a systematic way. Initially, the weights are given random values and the network must
be trained in order to find the weigth parameters that produce the desired effect. In order to achieve
Figure 2. An example of neural network layers as feature detectors.4
bles the network to eventually recognize very complex objects.

19. 19Photo by Gratisography from Pexels
digging deeper
into „debugging“

20. Recap: Properties of AI problems
Hard to code „by hand“
• Requires non-formalized knowledge (experiential knowledge)
• Or even not yet existing knowledge
Afflicted with uncertainty (or missing information)
Changes rapidly (making it unreasonable to adopt software
manually)
20

21. 21
These properties make verification
hard „by design“
Recap: Properties of AI problems
Hard to code „by hand“
• Requires non-formalized knowledge (experiential knowledge)
• Or even not yet existing knowledge

22. 22Photo by Pixabay from Pexels
Problems with unknown truth
For example: Medical classification problem
What is the correct diagnosis?
What is the correct therapy?
You won’t know (maybe until your patient either recovers or dies?)
Same is true for customer support systems. Is you customer satisfied?
You’ll probably only know by loosing him/her as a customer.

23. 23https://towardsdatascience.com/gender-bias-word-embeddings-76d9806a0e17, Photo by rawpixel.com from Pexels
ML replicates bias in the data
Example: Conceptual similarities from word embeddings
With words typically collocated, you can ask you model for
conceptual similarities:
king - man + woman ⇾ queen
Depending on your input corpus, your model will give you
doctor - man + woman ⇾ nurse
Do you really run a sexist, racist chatbot on your website?

24. AI systems break fundamental patterns we
developed as an industry
• No (or very little) isolation. You need to verify and retrain the
whole system.
• Higher dimension of failure space
• Time intense training cycles (instead immediate feedback cycles)
• Non-stationary nature of ML systems
24http://ai.stanford.edu/~zayd/why-is-machine-learning-hard.html

25. 25https://www.youtube.com/watch?v=piYnd_wYlT8
AI is prone to hacking. So know your tools!

26. We need …
26
… new quality management approaches

27. Understand model
requirements
Build the right thing...
27

28. Model requirements
Check whether the product / project requirements can be met with
the model choices.
Example:
Classify documents
Distinguish Cards (ID, Member Cards), structured documents or
free text? All of them? Or even mixed?
Are you also trying to extract information from these documents?
28

29. Quality management
on input data
Machine learning is
programming by example.
Make sure your examples
are legit.
29

30. Use BI, data visualization, statistics, Excel
Implement quality management for your input data.
Visualize, use statistical metrics, use data science on the input data, e.g. using
Tableau, Looker, Excel, whatever suits you)
E.g., to identify bias in the input data, check whether data is balanced, etc.
Use version control on input data (and processing steps) (see dvc.org, that's
not perfect, but what is?). Make sure data is versioned alongside code and
model.
If you create the data from another technical system, manage quality of the
generating code.
Try to document / check for documentation: Which distribution, what features.
30

31. With supervised learning, you need
supervisors.
Establish panel of judges both on input data labelling and on
outcomes. Due to the non-stationary nature this is is an ongoing
task, not a closed project.
Crowdsource your data labelling or let clickworkers do their job.
QM labelling, again using data visualization, fraud detection (outlier
detection, etc.)
31

32. Make sure you know the data
Use generated test data with known patterns, because otherwise
you won’t know if you miss whole categories.
32

33. Cope with
probabilistic logic
The world is not black and
white, it's full of color.
Make sure you see it!
33

34. Work with test sets
To test for non-binary outcomes (i.e., results with confidence level),
you need to handle test set as opposed to sets of single test
outcomes: For example test for outcome confidence distribution.
For non-stationary systems: establish test monitoring to accept
regression. „Accept new model if result is at least 95% of last
model.“
Do not only test outcomes, but implement inspection tools. E.g., in
RoboCup Simulation map agent movement paths.
34

35. Work with specific
tools
These depend on your
model requirements
35

36. These techniques are not generally applicable
Alter real data to fabricate test data with known properties.
Modify input variables and observe model behaviour. Usually small
changes should have small effects.
Sometimes, you even know the direction, amount or characteristic
of effects by applying common sense.
Measure at the right granularity (so the system won't utilize
loopholes)
36

37. Add a safety net
AI is a tool like any other. It
is created by humans, and
we all make mistakes.
37

38. Add a safety net
Implement multi-layer security fallbacks for subsets of your
problem (also to be used while testing), like „emergency break
systems“. Test these. Use for testing: If you need your security
fallback too often, your model may be bad.
Implement rules (i.e., code) for parts of your problem.
Just in research: Add explainability, local properties by black-box
tests on the model to verify the „anchor rules" (https://
homes.cs.washington.edu/~marcotcr/aaai18.pdf)
38

39. © data42 GmbH
Speed / cost benefits of Machine
Learning often only apply if you accept
a non-zero failure rate
Think twice before implementing high stakes AI applications
(E.g., do NOT use AI to screen applications in HR)
39
26.02.18

40. © data42 GmbH
Derive new knowledge from AI
… to solve new problems.
40
26.02.18

41. Thank you
- and be curious!
Dr. Christian Betz, data42