The document discusses unsupervised learning in machine learning, focusing on clustering methods such as hierarchical clustering and k-means clustering. It details processes for grouping data points into clusters based on their similarities, and outlines steps for algorithms, including the selection of cluster centers and convergence criteria. Additionally, it mentions references for further learning about machine learning concepts and techniques.

1. Introduction to Machine
Learning
(Unsupervised learning)
Dmytro Fishman (dmytro@ut.ee)

2. Unsupervised Learning
Tumour size
Age

3. 2
0
4
6
5
0
1
3
3
3
1
5
8
3
8
6
7
2
4
2
3
9
1
3
8
0
8
2
4
1
3
1
8
2
0
4
6
5
4
7
8
2
?
We were given
annotated data

4. 2
0
4
6
5
0
1
3
3
3
1
5
8
3
8
6
7
2
4
2
3
9
1
3
8
0
8
2
4
1
3
1
8
2
0
4
6
5
4
7
8
2
?
We were given
annotated data
Based on which we
could predict a class
of novel data point

5. 2
0
4
6
5
0
1
3
3
3
1
5
8
3
8
6
7
2
4
2
3
9
1
3
8
0
8
2
4
1
3
1
8
2
0
4
6
5
4
7
8
2
?
We were given
annotated data
Based on which we
could predict a class
of novel data point

6. 2
0
4
6
5
0
1
3
3
3
1
5
8
3
8
6
7
2
4
2
3
9
1
3
8
0
8
2
4
1
3
1
8
2
0
4
6
5
4
7
8
2
3
We were given
annotated data
Based on which we
could predict a class
of novel data point

7. ?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
?
We were given
annotated data
Based on which we
could predict a class
of novel data point
Now, imagine that they
have taken away all
these good things from
us.

8. Clustering

10. Customer 1

11. Customer 1
Customer 2

12. Customer 1
Customer 2
Customer 3

13. Customer 1
Customer 2
Customer 3
Customer 4

14. Customer 1
Customer 2
Customer 3
Customer 4
Which segments of
customers should the
store target?

15. Grouping data points so that
semantically similar points would
be clustered together

16. Jack Vilo et. al Data Mining: https://courses.cs.ut.ee/MTAT.03.183/2017_spring/uploads/Main/DM_05_Clustering.pdf
Grouping data points so that
semantically similar points would
be clustered together

17. Hierarchical clustering

18. Tumour size
Age

19. Tumour size
Age
1.Let’s first assume that
all instances are
individual clusters

20. Tumour size
Age
1.Let’s first assume that
all instances are
individual clusters
2.Find two most similar
instances and merge
them into one cluster

21. Remember NN?
Tumour size
Age
1.Let’s first assume that
all instances are
individual clusters
2.Find two most similar
instances and merge
them into one cluster

22. Tumour size
Age
Usually similarity is defined by
euclidean or any other
distance measure1.Let’s first assume that
all instances are
individual clusters
2.Find two most similar
instances and merge
them into one cluster

23. Tumour size
Age
1.Let’s first assume that
all instances are
individual clusters
2.Find two most similar
instances and merge
them into one cluster

24. Tumour size
Age
1.Let’s first assume that
all instances are
individual clusters
2.Find two most similar
instances and merge
them into one cluster
3.Repeat 2 until all
cluster merge into one

25. Tumour size
Age
1.Let’s first assume that
all instances are
individual clusters
2.Find two most similar
instances and merge
them into one cluster
3.Repeat 2 until all
cluster merge into one

26. Tumour size
Age
1.Let’s first assume that
all instances are
individual clusters
2.Find two most similar
instances and merge
them into one cluster
3.Repeat 2 until all
cluster merge into one

27. Tumour size
Age
Distance between clusters can be
estimated with three strategies:
1.Let’s first assume that
all instances are
individual clusters
2.Find two most similar
instances and merge
them into one cluster
3.Repeat 2 until all
cluster merge into one

28. Tumour size
Age
Distance between clusters can be
estimated with three strategies:
1. Single linkage (min)1.Let’s first assume that
all instances are
individual clusters
2.Find two most similar
instances and merge
them into one cluster
3.Repeat 2 until all
cluster merge into one

29. Tumour size
Age
Distance between clusters can be
estimated with three strategies:
1. Single linkage (min)
2. Complete linkage (max)
1.Let’s first assume that
all instances are
individual clusters
2.Find two most similar
instances and merge
them into one cluster
3.Repeat 2 until all
cluster merge into one

30. Tumour size
Age
Distance between clusters can be
estimated with three strategies:
1. Single linkage (min)
2. Complete linkage (max)
3. Average linkage (avg)
1.Let’s first assume that
all instances are
individual clusters
2.Find two most similar
instances and merge
them into one cluster
3.Repeat 2 until all
cluster merge into one

31. Tumour size
Age
1.Let’s first assume that
all instances are
individual clusters
2.Find two most similar
instances and merge
them into one cluster
3.Repeat 2 until all
cluster merge into one

32. Tumour size
Age
1.Let’s first assume that
all instances are
individual clusters
2.Find two most similar
instances and merge
them into one cluster
3.Repeat 2 until all
cluster merge into one

33. Tumour size
Age
K = 2
1.Let’s first assume that
all instances are
individual clusters
2.Find two most similar
instances and merge
them into one cluster
3.Repeat 2 until all
cluster merge into one

34. Tumour size
Age
K = 1
1.Let’s first assume that
all instances are
individual clusters
2.Find two most similar
instances and merge
them into one cluster
3.Repeat 2 until all
cluster merge into one

35. Tumour size
1.Let’s first assume that
all instances are
individual clusters
Age
2.Find two most similar
instances and merge
them into one cluster
Dendrogram
4.Hierarchical clustering
is usually visualised
using dendrogram
3.Repeat 2 until all
cluster merge into one

36. Tumour size
1.Let’s first assume that
all instances are
individual clusters
Age
2.Find two most similar
instances and merge
them into one cluster
Dendrogram
4.Hierarchical clustering
is usually visualised
using dendrogram
3.Repeat 2 until all
cluster merge into one
K = 2

37. Tumour size
1.Let’s first assume that
all instances are
individual clusters
Age
2.Find two most similar
instances and merge
them into one cluster
Dendrogram
4.Hierarchical clustering
is usually visualised
using dendrogram
3.Repeat 2 until all
cluster merge into one
K = 3

39. K = 2

40. K = 2

41. K-means clustering

42. K-means clustering
*although they do have something in
common with K-nearest neighbour, but
they are not the same.

43. Tumour size
Age
1.Choose K, the number
of potential clusters

44. Tumour size
Age
1.Choose K, the number
of potential clusters
Let K be 2

45. Tumour size
Age
1.Choose K, the number
of potential clusters
Let K be 2
2.Initialise cluster centers
randomly within the data

46. Tumour size
Age
1.Choose K, the number
of potential clusters
Let K be 2
2.Initialise cluster centers
randomly within the data
3.Instances are clustered
to the nearest cluster
centre

47. Tumour size
Age
1.Choose K, the number
of potential clusters
Let K be 2
2.Initialise cluster centers
randomly within the data
3.Instances are clustered
to the nearest cluster
centre

48. Tumour size
Age
1.Choose K, the number
of potential clusters
Let K be 2
2.Initialise cluster centers
randomly within the data
3.Instances are clustered
to the nearest cluster
centre
4.Centroids of each of
the K clusters become
new cluster centers

49. Tumour size
Age
1.Choose K, the number
of potential clusters
Let K be 2
2.Initialise cluster centers
randomly within the data
3.Instances are clustered
to the nearest cluster
centre
4.Centroids of each of
the K clusters become
new cluster centers

50. Tumour size
Age
1.Choose K, the number
of potential clusters
Let K be 2
2.Initialise cluster centers
randomly within the data
3.Instances are clustered
to the nearest cluster
centre
4.Centroids of each of
the K clusters become
new cluster centers

51. Tumour size
Age
1.Choose K, the number
of potential clusters
Let K be 2
2.Initialise cluster centers
randomly within the data
3.Instances are clustered
to the nearest cluster
centre
4.Centroids of each of
the K clusters become
new cluster centers
5.Steps 3/4 are repeated
until convergence

52. Tumour size
Age
1.Choose K, the number
of potential clusters
Let K be 2
2.Initialise cluster centers
randomly within the data
3.Instances are clustered
to the nearest cluster
centre
4.Centroids of each of
the K clusters become
new cluster centers
5.Steps 3/4 are repeated
until convergence

53. Hierarchical K-means
Tumour size
Age
Tumour size
Age

54. Hierarchical K-means
Tumour size
Age
Tumour size
Age

55. 😈👼
Tumour size
Age
Tumour size
Age
Hierarchical K-means
Slow for modern size datasets
Hard to predict K
Good for visualisation purposes

56. to predict KTwo body parts methods

57. to predict KTwo body parts methods
The rule of thumb is to
choose as K
p
n/2

58. to predict KTwo body parts methods
The rule of thumb is to
choose as K
p
n/2
Elbow method: increase
K until it does not help to
describe data better

59. References
• Machine Learning by Andrew Ng (https://www.coursera.org/learn/machine-
learning)
• Introduction to Machine Learning by Pascal Vincent given at Deep Learning
Summer School, Montreal 2015 (http://videolectures.net/
deeplearning2015_vincent_machine_learning/)
• Welcome to Machine Learning by Konstantin Tretyakov delivered at AACIMP
Summer School 2015 (http://kt.era.ee/lectures/aacimp2015/1-intro.pdf)
• Stanford CS class: Convolutional Neural Networks for Visual Recognition by
Andrej Karpathy (http://cs231n.github.io/)
• Data Mining Course by Jaak Vilo at University of Tartu (https://courses.cs.ut.ee/
MTAT.03.183/2017_spring/uploads/Main/DM_05_Clustering.pdf)
• Machine Learning Essential Conepts by Ilya Kuzovkin (https://
www.slideshare.net/iljakuzovkin)
• From the brain to deep learning and back by Raul Vicente Zafra and Ilya
Kuzovkin (http://www.uttv.ee/naita?id=23585&keel=eng)

60. www.biit.cs.ut.ee www.ut.ee www.quretec.ee