1. The ROC Curve
(Receiver Operating Characteristic)
Luis Serrano

2. A tale of two models

3. A tale of two models
Positive
Negative
False
positive
False
negative

4. A tale of two models

5. A tale of two models

6. A tale of two models
Medical model Spam model
Spam (+)
Ham (-)
Sick (+)
Healthy (-)
False positive:
Healthy person
sent for more tests
False negative:
Spam email
sent to inbox
False negative:
Sick person
not treated
False positive:
Good email
sent to spam folder

7. Drawing the ROC Curve

8. Bluecorrect
0
1
2
3
4
5
Red correct
0 1 2 3 4 5
Step Correct Correct
0 0 5
1 1 5
2 2 5
3 2 4
4 3 4
5 4 4
6 4 3
7 5 3
8 5 2
9 5 1
10 5 0
AUC = 21
(area under curve)
AUC = 0.84
(area under curve)

9. Making decisions

10. Bluecorrect
0
1
2
3
4
5
Red correct
0 1 2 3 4 5
Medical model
(No false negatives)
Spam model
(No false positives)
Ideally

11. 0 1

12. Bluecorrect
0
20
40
60
80
100
Red correct
0 20 40 60 80 100
Ideal
Threshold
Ideal
Threshold = 0.65
0 1

13. Comparing

14. Bluecorrect
0
1
2
3
4
5
Red correct
0 1 2 3 4 5
Bluecorrect
0
1
2
3
4
5
Red correct
0 1 2 3 4 5
AUC = 0.52
AUC = 0.8 AUC = 1Bluecorrect
0
1
2
3
4
5
Red correct
0 1 2 3 4 5

15. AUC = 1Bluecorrect
0
1
2
3
4
5
Red correct
0 1 2 3 4 5
Terrible model?
AUC = 0

16. Thank you!

17. https://www.manning.com/books/grokking-machine-learning
Discount code: serranoyt
Grokking Machine
Learning
By Luis G. Serrano

18. Thank you!
@luis_likes_math
Subscribe, like,
share, comment!
youtube.com/c/LuisSerrano
http://serrano.academy