1. www.SunilOS.com 1
Unsupervised Machine Learning
www.sunilos.com
www.raystec.com

2. Unsupervised Learning
❑ unsupervised machine learning is a type of machine
learning where we have only data points but no labels.
❑ We will make a group based on the similarity among data
points.
o For example, in real life we are arranging our bookshelves. In bookshelves we
have different kinds of books. We will make groups of books based on their
subjects. So, in unsupervised learning we iterate through data and group them
together based on similar characteristics.
❑ Unsupervised learning also known as clustering.
www.SunilOS.com 2

3. Types of clustering :
❑Partition Based
o partition-based clustering methods include K-Means, K-
Medoids, CLARANS, etc.
❑Hierarchical Based
o hierarchical clustering methods include BIRCH and
Chameleon.
❑Density Based Learning
o DBSCAN, OPTICS are the most popular density-based
clustering methods.
www.SunilOS.com 3

4. K-means clustering algorithm
❑ K-means is based on a partition based clustering method.
❑ K-means, it is one of the simplest unsupervised learning
algorithms that will solve the most well-known clustering
problem.
❑ The procedure can be grouped as the one which follows a
simple and very easy way to classify a given data set with
the help of a certain number of clusters (assume k clusters).
❑
www.SunilOS.com 4

5. How K-Means Clustering Works:
❑The K Means algorithm is iterative based, it repeatedly
calculates the cluster centroids, refining the values until
they do not change much.
❑The k-means algorithm takes a dataset of ‘n’ points as
input, together with an integer parameter ‘k’ specifying
how many clusters to create(supplied by the
programmer).
❑The output is a set of ‘k’ cluster centroids and a labeling
of the dataset that maps each of the data points to a
unique cluster.
❑
www.SunilOS.com 5

6. Steps of K-means clustering:
❑Choose the number of clusters k
❑Select k random points from the data as centroids
❑Assign all the points to the closest cluster centroid
❑Recompute the centroids of newly formed clusters
❑Repeat step 3 and 4
www.SunilOS.com 6

7. When to stop Iterating?
❑Centroids of newly formed clusters do not
change
❑Points remain in the same cluster
❑Maximum number of iterations are reached
www.SunilOS.com 7

8. Working of K-means
www.SunilOS.com 8
❑Sample Dataset:
Objects X Y Z
OB-1 1 4 1
OB-2 1 2 2
OB-3 1 4 2
OB-4 2 1 2
OB-5 1 1 1
OB-6 2 4 2
OB-7 1 1 2
OB-8 2 1 1

9. ❑ We have total 8 data points. We will divide these points into 2
clusters. K=2 in k-means.
❑ Taking any two centroids or data points (as you took 2 as K
hence the number of centroids also 2) in its account initially.
❑ After choosing the centroids, (say C1 and C2) the data points
(coordinates here) are assigned to any of the Clusters
❑ Assume that the algorithm chose OB-2 (1,2,2) and OB-6 (2,4,2)
as centroids and cluster 1 and cluster 2 as well.
❑ For measuring the distances, you take the following distance
measurement function (also termed as similarity measurement
function):
❑ d=|x2–x1|+|y2–y1|+|z2–z1|
www.SunilOS.com 9

10. calculation of distances
Objects X Y Z
Distance from
C1(1,2,2)
Distance from
C2(2,4,2)
OB-1 1 4 1 3 2
OB-2 1 2 2 0 3
OB-3 1 4 2 2 1
OB-4 2 1 2 2 3
OB-5 1 1 1 2 5
OB-6 2 4 2 3 0
OB-7 1 1 2 1 4
OB-8 2 1 1 3 4
www.SunilOS.com 10

11. Cluster formation
❑ After the initial pass of clustering, the clustered objects will
look something like the following:
❑
www.SunilOS.com 11
Cluster 1
OB-2
OB-4
OB-5
OB-7
OB-8
Cluster 2
OB-1
OB-3
OB-6

12. ❑
www.SunilOS.com 12

13. Distance from new Centroids
Objects X Y Z
Distance from
C1(1.4,1.2,1.6)
Distance from C2(1.33, 4,
1.66)
OB-1 1 4 1 3.8 1
OB-2 1 2 2 1.6 2.66
OB-3 1 4 2 3.6 0.66
OB-4 2 1 2 1.2 4
OB-5 1 1 1 1.2 4
OB-6 2 4 2 3.8 1
OB-7 1 1 2 1 3.66
OB-8 2 1 1 1.4 4.33
www.SunilOS.com 13

14. Updated Clusters
❑The new assignments of the objects with respect
to the updated clusters will be:
❑Algorithm will End here because no changes in
groups.
❑
www.SunilOS.com 14
Cluster 1
OB-2
OB-4
OB-5
OB-7
OB-8
Cluster 2
OB-1
OB-3
OB-6

15. Code Implementation of K-means
❑ import matplotlib.pyplot as plt
from matplotlib import style
style.use('ggplot')
import numpy as np
X = np.array([[1, 2],
[1.5, 1.8],
[5, 8 ],
[8, 8],
[1, 0.6],
[9,11]])
plt.scatter(X[:,0], X[:,1], s=150)
plt.show()
www.SunilOS.com 15

16. Code Implementation of K-means (cont.)
❑ from sklearn.cluster import Kmeans
❑ # You want cluster the records into 2
kmeans = KMeans(n_clusters=2)
❑ #train Model
kmeans.fit(X)
❑ #test Model
labels = kmeans.predict([[20,8]])
print(labels)
centroids = kmeans.cluster_centers_
print(centroids)
www.SunilOS.com 16

17. www.SunilOS.com 17
Reinforcement Learning
www.sunilos.com
www.raystec.com

18. Reinforcement Learning
❑We first learn by interacting with the environment.
❑Whether we are learning to drive a car or learning to walk,
the learning is based on the interaction with the
environment.
❑Learning from interaction is the foundational underlying
concept for all theories of learning and intelligence.
❑Reinforcement Learning – a goal-oriented learning based on
interaction with the environment. Reinforcement Learning
is said to be the hope of true artificial intelligence.
www.SunilOS.com 18

19. Problem Statement
❑How a child learn to walk?
www.SunilOS.com 19

20. Formalized the Problem?
❑ The child is an agent trying to manipulate the environment
(which is the surface on which it walks) by taking actions
(walking) and he/she tries to go from one state (each step
he/she takes) to another.
❑ The child gets a reward (let’s say chocolate) when he/she
accomplishes a sub module of the task (taking a couple of
steps) and will not receive any chocolate (negative reward)
when he/she is not able to walk.
❑ This is a simplified description of a reinforcement learning
problem.
www.SunilOS.com 20

21. Basis of Reinforcement Learning
www.SunilOS.com 21

22. Difference Between Different Kind of Machine Learning:
Supervised Unsupervised Reinforcement
Definition Learns by labeled data Learns by unlabelled
data
Learns by interacting
with environment by
actions and discovers
errors and rewards
Types of Problems Regression and
classification
Association and
clustering
Reward based
Data Labeled Unlabeled No predefined data
Training External supervision No supervision No supervision
Approach Map labeled input to
known output
Search patterns and
discover output
Follow trail and error
method
Algorithms SVM, KNN, Linear
Regression,
K-means, C-means Q-Learning, SARSA
etc.
www.SunilOS.com 22

23. Terminology of reinforcement Learning:
❑ Agent: An entity (computer program) that learns from the environment
based on the feedback.
❑ Action: Actions are steps taken by agent according to the
situation(Environment).
❑ Environment: The surrounding in which agent is present to act.
Environment is always random in nature.
❑ State: state is returned by the environment after each act of the agent
❑ Reward: It is a feedback which can be positive or negative based on
the action of the agent.
❑ Policy: This is an approach applied by agents for the next step based
on the current situation.
❑ Value: It is long term result opposite to the short term reward
❑ Q-value: same as value but with additional parameter as a current
action.
www.SunilOS.com 23

24. Key Points of Reinforcement Learning:
❑It is Based on try and error method
❑In this Learning agent is not guided about the
environment, and which next step to be taken.
❑Agent takes the next action based on the previous
feedback.
❑Agents will also get the delayed penalty.
❑The environment for the agent to interact is always a
random one, and the agent has to reach the destination
and get the maximum reward points.
www.SunilOS.com 24

25. How to implement RL in Machine Learning:
❑There are three approaches to implement RL
❑Model based learning
o In this approach a prototype is created for the environment and agents will
explore this model. For each situation a different model is created.
❑Policy based Learning
o This approach is based on finding the optimal strategy to get the maximum
future points without relying on any value function. There can be two
types of policy:
❑Value based learning
o In this approach agents try to get maximum value at any state under any
policy.
❑
www.SunilOS.com 25

26. When Not to Use RL?
❑Enough Data for training the model
❑It is a time consuming process
www.SunilOS.com 26

27. Why use Reinforcement Learning?
❑For a reward based system to learn.
❑When agents want to learn from the action.
❑Helps you to discover which action yields the
highest reward over the longer period.
❑When we want to find best method for obtaining
large rewards.
www.SunilOS.com 27

28. Learning Models of Reinforcement
❑Markov Decision Process
❑Q learning
❑SARSA (State Action Reward State Action)
❑Deep Q Neural Network (DQN)
www.SunilOS.com 28

29. Q-Learning
❑In Q learning , Q stands for quality.
❑It is a value based learning.
❑In this approach a value is given to the agent to inform
which action is best to take.
❑To perform any action, the agent will get a reward R(s,
a), and also he will end up on a certain state, so the Q -
value equation will be:
www.SunilOS.com 29

30. Q-Learning process
www.SunilOS.com 30

31. Application of RL
www.SunilOS.com 31

32. Gym environment for Reinforcement Learning:
❑Gym is the python library for developing
reinforcement learning algorithms:
❑We can install gym using following command:
❑pip install gym
www.SunilOS.com 32

33. env object contains the following main functions:
❑ The step() function takes an action object as an argument and
returns four objects:
❑ observation: An object implemented by the environment,
representing the observation of the environment.
❑ reward: A signed float value indicating the gain (or loss) from
the previous action.
❑ done: A Boolean value representing if the scenario is finished.
❑ The render() function creates a visual representation of the
environment.
❑ The reset() function resets the environment to the original state.
❑
www.SunilOS.com 33

34. Implementation
❑ Most Popular game is cart pole.
❑ In this game a pole is attached with a cart and we have to balance
it.
❑ If the pole tilts more than 15 degree or the cart moves more than
2.4 meter from center the pole will fall.
❑ This is the very simplest environment to learn the basics.
❑ The game has only four observations and two actions.
o The actions are to move a cart by applying a force of +1 or -
1.
o The observations are the position of the cart, the velocity of
the cart, the angle of the pole, and the rotation rate of the pole.
www.SunilOS.com 34

35. Getting environment Of Cartpole
❑ import gym
env = gym.make('CartPole-v0')
for i_episode in range(20):
observation = env.reset()
for t in range(1000):
env.render()
print(observation)
action = env.action_space.sample()
observation, reward, done, info =
env.step(action)
if done:
print("Episode finished after {}
timesteps".format(t+1))
break
env.close()
www.SunilOS.com 35

36. www.SunilOS.com 36
Data Preprocessing
www.sunilos.com
www.raystec.com

37. Why data Preprocessing
❑Data in real world is not perfect for learning.
❑It is noisy, dirty and incomplete.
❑No quality data no quality results.
www.SunilOS.com 37
Processing

38. Types of Data:
www.SunilOS.com 38

39. Types of Data(cont.)
❑ Nominal Data: categorical values without any order. For ex. Color of
cars: black, white, red, blue.
❑ Ordinal Data: Categorical Data with a natural order. For ex. Size of
clothes: small, medium, large, extra large. But the scale of difference is
not allowed. For example large-medium=small
❑ Interval Data: Numeric values with defined unit of measurement. For ex.
Temperature, dates.
❑ Ratio: numeric variables with a defined unit of measurement but both
differences and ratio is meaningful count, age , mass length.
❑ Time Series data: A time series is a series of data points indexed in time
order. Most commonly, a time series is a sequence taken at successive
equally spaced points in time. Ex. weather forecasting.
❑ Text Data: This is unstructured data. Text data usually consists of
documents which can represent words, sentences or even paragraphs of
free flowing text.
www.SunilOS.com 39

40. Data Processing Steps:
❑Dataset is viewed as a collection of data objects.
❑Data objects contain many features.
❑Features means characteristics of a data object. For
example color, speed, mileage of a car.
❑These are the basic steps in data processing
o Data Quality Assessment
o Feature Aggregation
o Feature Sampling
o Dimensionality Reduction
o Feature Encoding
www.SunilOS.com 40

41. Data Quality assessment:
❑Collected Data may be incomplete and noisy.
❑We cannot completely rely on data acquiring tools.
❑There may be flaws in the data collection process.
❑Raw data contains missing values, duplicate values, and
inconsistent values.
❑We have to tackle all these limitations before going for
machine learning.
www.SunilOS.com 41

42. Feature aggregation
❑After Collecting data from different sources.
❑Now aggregate data to single unit.
❑Reduce memory consumption.
❑For example we are collecting daily sales records of a
store from multiple places. We can aggregate these data
into monthly sales or yearly sales.
www.SunilOS.com 42

43. Feature Sampling:
❑ Large Dataset from different sources.
❑ Take a subset from it for machine learning model.
❑ Choose a sampling algorithm which properly divide the dataset
into working subset of data.
❑ Take care of imbalanced dataset classes.
❑ Some sampling algorithms:
o Simple random sampling.
o Systematic sampling.
o Stratified sampling.
o Clustered sampling.
o Convenience sampling.
o Quota sampling.
o Judgement (or Purposive) Sampling..
o Snowball sampling.
www.SunilOS.com 43

44. Dimensionality Reduction:
❑ Datasets are represented in Higher dimensions (3D graphs).
❑ We can not easily visualize the data in higher dimensions.
❑ Reduce the dimensions of datasets.
❑ Map Higher dimensions space (n dimensions) to the lower
dimensional space (2D plots).
❑ Lower dimension space is easy to process and visualize.
www.SunilOS.com 44

45. Feature Encoding:
❑ Machines cannot understand the data as humans.
❑ We have to convert the dataset into machine readable form.
❑ Feature encoding techniques are different for different kinds of
data.
www.SunilOS.com 45

46. Data Pre Processing Libraries
❑ # used for handling numbers
❑ import numpy as np
❑ # used for handling the dataset
❑ import pandas as pd
❑ # used for handling missing data
❑ from sklearn.impute import SimpleImputer
❑ # used for encoding categorical data
❑ from sklearn.preprocessing import LabelEncoder,
OneHotEncoder
❑ # used for splitting training and testing data
❑ from sklearn.model_selection import train_test_split
❑ # used for feature scaling
❑ from sklearn.preprocessing import StandardScaler
www.SunilOS.com 46

47. Label Encoder for the Categorical data:
❑ # Categorical Feature
❑ weather=['Sunny','Sunny','Overcast','Rainy','Ra
iny','Rainy','Overcast','Sunny','Sunny','Rainy'
,'Sunny','Overcast','Overcast','Rainy']
❑ # Import LabelEncoder
❑ from sklearn import preprocessing
❑ #creating labelEncoder
❑ le = preprocessing.LabelEncoder()
❑ # Converting string labels into numbers.
❑ weather_encoded=le.fit_transform(weather)
❑ print(weather_encoded)
www.SunilOS.com 47

48. Dealing with Missing value
❑ import pandas as pd
❑ import numpy as np
❑ df=pd.DataFrame({"Age":[23,70,56,24,np.nan],
"Salary":[30000,30000,50000,np.nan,40000]})
❑ print(df)
❑ from sklearn.impute import SimpleImputer
❑ imp = SimpleImputer(missing_values=np.nan,
❑ strategy="most_frequent")
❑ X = imp.fit_transform(df)
❑ df1=pd.DataFrame(X, columns=["Age","Salary"])
❑ print(df1)
www.SunilOS.com 48

49. Scaling Data
❑ from sklearn.preprocessing import
❑ StandardScaler
❑ sc = StandardScaler(with_mean=True)
❑ X = sc.fit_transform(df1)
❑ X_scaled=pd.DataFrame(X, columns=["Age","Salary
❑ ])
❑ print(X_scaled)
www.SunilOS.com 49

50. Disclaimer
❑This is an educational presentation to enhance the
skill of computer science students.
❑This presentation is available for free to computer
science students.
❑Some internet images from different URLs are used
in this presentation to simplify technical examples
and correlate examples with the real world.
❑We are grateful to owners of these URLs and
pictures.
www.SunilOS.com 50

51. Thank You!
www.SunilOS.com 51
www.SunilOS.com