Machine Learning lecture material about introduction

1. 1
ML-Chapter One
Introduction to Machine Learning
Belay E., Asst. Prof.
e-mail: belayenyew@gmail.com
Mobile: 0946235206
University of Gondar
College of Informatics
Department of Information Technology

2. 2
The Way Forward
Topics Areas covered
Introduction to
Machine
Learning
Definition of Machine Learning, a brief history of Machine
Learning, Fundamentals of Machine Learning, Application of
Machine Learning
Data Preprocessing
Data Cleaning, data integration, Data reduction, Data transformation,
and data discretization
Supervised
Learning
Algorithms
Concept of Supervised Learning, Decision Trees, Naïve
Bayesian Classification, the k-Nearest Neighbors Classifiers,
Ensemble, Linear Discriminant Analysis, Support Vector
Machine, Time-Series Forecasting, Sequential Pattern Analysis
Evaluation
techniques
Metrics, cross-validation, statistics, Addressing the multiple
comparisons problem.
Unsupervised
Learning
Algorithms
Concept of unsupervised learning, k-Means Clustering,
Hierarchical Clustering, Gaussian Mixture Model, Hidden Markov
Model, Principal Component Analysis
Reinforcement
Learning
Introduction to Reinforcement Learning, Markov Decision
Process, Monte Carlo Methods for Prediction & Control
Deep learning
Concept of deep learning, Regularization , convolutional neural
networks, recurrent neural networks etc.

3. 3
Reference
1. Ethem ALPAYDIN. Introduction to Machine Learning, Third Edition, 2014
2. Mohssen Mohammed, Muhammad Badruddin Khan, Eihab Bashier,
Mohammed Bashier Machine Learning: Algorithms and Applications,2017
3. Ian Goodfellow Yoshua Bengio Aaron Courville, Deep Learning, 2016
Evaluation:
-Assignment-10%
-Article Review-10%
-project-15%
-Test -20%
-Final Exam-45%

4. 4
Tools and Dataset
Tools:
• Python
• R/Rstudio
• MatLab
Dataset:
• UCI Machine Learning Repository:
http://archive.ics.uci.edu/ml/
• www.Kaggle.com

5. 5
Introduction
Definition of Machine Learning(ML):
▶ It is the field of study that gives a computer the ability to learn
without being explicitly programmed (Arthur Samuel, 1959)
▶ A computer program is said to learn from experience E with
respect to some class of tasks T and performance measure P, if
its performance at tasks in T, as measured by P, improves with
experience E (Tom Mitchell, 1999).
▪ Task T: classifying handwritten digits from images
▪ Performance measure P : percentage of digits classified
correctly
▪ Training experience E: dataset of digits given classifications
▶ Learning is Generalization: the ability to perform a task in a
situation which has never been encountered before

6. 6
Introduction..
Why Machine Learning?
▶ Necessity: many things we want to do cannot be done by “programming”.
▶ Recent progress in algorithms and theory
▶ Growing flood of online data
▶ Computational power is available
▶ Growth of Industry etc..
When Do We Need Machine Learning?
▶ Tasks That Are Too Complex to Program:
▪ Tasks beyond Human Capabilities-analysis of very large and complex data
sets like medical archives, weather prediction and electronic commerce etc.
▪ Develop systems that are too difficult or impossible to construct manually
▶ Adaptivity:
▪ Programmed tools is rigidity
▪ Develop systems that can automatically adapt and customize themselves to
the needs of the individual user through experience
▪ Machine learning tools –adapts to their input data – adaptive to changes in
the environment they interact with.

7. 7
Introduction..
AI vs ML vs DL
Artificial
Intelligence(AI)
Machine
Learning(ML)
Deep Learning
(DL)
A technique which enables a
machine to mimic human
behaviors
A subset of AI technique which uses statistical
methods to enable machine to improve with
experiences
A subset of ML which makes the computation
of multilayers neural network feasible

8. 8
Introduction..
Basic Paradigm of ML
▶ Observe set of examples: Training data
▶ Infer something about process that generated data
▶ Use inference to make predictions about previously unseen data: test data.

9. 9
Introduction..
Brief History of Machine Learning
▶From the 1950s to now, machine learning has significantly developed.
THE TURING TEST 1950
▶ Alan Turing created the “Turing Test” to determine whether or not a computer
was capable of real intelligence.
▶ Turing explored the idea of how to determine whether machines can think
THE FIRST COMPUTER PROGRAM 1952
▶ Arthur Samuel created the first implementation of machine learning, the game
of checkers.
▶ In winning strategies, and incorporating those strategies into the game.

10. 10
Introduction..
Brief History of Machine Learning..
NEURAL NETWORKS FOR COMPUTERS 1957
▶ Frank Rosenblatt designed the first neural network which is called perceptron
for computers in 1957, which was meant to simulate the thought process of a
human brain.
“NEAREST NEIGHBOR” ALGORITHM 1967
▶ The “nearest neighbor” algorithm was written in 1967, allowing computers to
begin recognizing basic patterns. This could be used as a mapping route for
traveling salesmen.
EXPLANATION BASED LEARNING 1981
▶ EBL, or Explanation Based Learning, was created in 1981 by Gerald Dejong.
This concept allowed a computer to analyze training data and create a
general rule it can follow by discarding unimportant data.

11. 11
Introduction..
Brief History of Machine Learning..
NetTalk program 1985
 Terry Sejnowski invented the NetTalk program that could learn to
like a baby does during the process of language acquisition.
 The artificial neural network aimed to reconstruct a simplified
the complexity of learning human-level cognitive tasks

12. 12
Introduction..
Brief History of Machine Learning..
Machine Learning in 1990s
 the work in machine learning shifted from the knowledge-driven
driven approach.
 Scientists and researchers created programs for computers that
amounts of data and draw conclusions from the results.
 This led to the development of the IBM Deep Blue computer,
world’s chess champion Garry Kasparov in 1997

13. 13
Introduction..
Brief History of Machine Learning
Deep Learning 2006
▶ This is the year when the term “deep learning” was coined by
▶ He used the term to explain a brand-new type of algorithms that
and distinguish objects or text in images or videos

14. 14
Introduction..
Brief History of Machine Learning
GOOGLE AND FACEBOOK UTILIZE MACHINE LEARNING 2014
▶ In 2014, Google and Facebook made machine learning the pivotal
technology of their businesses.
MACHINE LEARNING AND CUSTOMER CARE 2015
▶ In 2015, Interactions acquired AT&T’s Watson and the AT&T speech and
language research team. Combined with their award winning Adaptive
Understanding™ technology, Interactions delivers unprecedented accuracy
in understanding that helps enterprises revolutionize their customer care
experience.

15. 15
Introduction..
Brief History of Machine Learning
MACHINE LEARNING AND SOCIAL MEDIA 2017
▶ Acquired by Interactions in 2017, Digital Roots provides companies with AI-
based social media.
▶ Its technology allows brands to quickly filter, respond, and interact with
followers on social media.

16. 16
Introduction..
The three different types of machine learning

17. 17
Introduction..
Supervised Learning(SL)
▶ The main goal in SL is to learn a model from labeled training data
that allows us to make predictions about unseen or future data.
▪ Here, the term supervised refers to a set of samples where the desired
output signals (labels) are already known.
▶ SL is where you have input variables(x) and an output variable(y)
and you use an algorithm to learn the mapping function from the
input to the output.
▶ It inference a function from labeled training data consisting of a set
of training examples. Y=f(X)
Where y is a target and x is input values

18. 18
Introduction..
Supervised Learning(SL)
▶ Example, we can predict the market value of a used car by analyzing other cars
and the relationship between car attributes (X) such as year of make, car brand,
mileage, etc., and the selling price of the car (y).
▶ SL can be:
▪ Classification: e-mail as Spam or Non-Spam, Bank customers as Fraud or default,
Patient as Diabetic or Non- diabetic etc.
▪ Regression/continuous value: predict electric power consumption of a city in
Kilo-watt, Predict house price based on square meter, Predict sale amount
based on advertisement etc.

19. 19
Introduction..
Unsupervised Learning
▶ In SL, we know the right answer beforehand when we train our model, and in
unsupervised learning, however, we are dealing with unlabeled data or data
of unknown structure
▶ Using unsupervised learning techniques, we are able to explore the structure
of our data to extract meaningful information without the guidance of a
known outcome variable.
Finding subgroups with clustering:
▶ Clustering is an exploratory data analysis technique that allows us to organize a
pile of information into meaningful subgroups (clusters) without having any
prior knowledge of their group memberships
▶ Each cluster defines a group of objects that share a certain degree of similarity
but are more dissimilar to objects in other clusters, which is why clustering is
also sometimes called "unsupervised classification”

20. 20
Introduction..
Unsupervised Learning..
▶ Clustering is a great technique for structuring information and deriving
meaningful relationships among data.
▪ For example, it allows marketers to discover customer groups based on their
interests in order to develop distinct marketing programs.
▶ The figure below illustrates how clustering can be applied to organizing
unlabeled data into three distinct groups based on the similarity of their
features x1 and x2 :

21. 21
Introduction..
Reinforcement Learning(RL):
▶ Reinforcement is the process of learning from rewards while performing a
series of actions.
▪ the goal is to develop a system (agent) that improves its performance based on
interactions with the environment
▶ In reinforcement learning, we do not tell the learner or agent, for example,
a (robot), which action to take but merely assign a reward to each action
and/or the overall outcome.
• Agent: takes actions
• Environment: the world in which agent exist
and operate
• Action: a move the agent can make in the
environment
• Action Space: the set of possible actions an
agent can make in the environment.
• Reward: feedback that measures the success
or failure of agent’s actions

22. 22
Introduction..
Reinforcement learning(RL)..
▶ RL is learning by interacting with a space or an environment
▶ An RL learns from the consequences of its actions, rather than from being
taught explicit.
▪ It selects its actions on basis of its past experience and also by new choices
▶ The machine learning program should be able to assess the goodness of
policies and learn from past good action sequences to be able to generate a
policy i.e. RL
▶ Example: Game playing
▪ A popular example of RL is a chess engine.
▪ Here, the agent decides upon a series of moves depending on the state of the board
(the environment), and the reward can be defined as win or lose at the end of the
game
▪ a single move by itself is not that important
▪ It is the sequence of right moves that is good.
▪ A move is good if it is part of a good game playing policy

23. 23
Introduction..
Example of Machine Learning Application
▶ Learning Associations: Basket analysis/association discovery
▪ In the case of retail-finding associations between products bought by customers:
If people who buy X typically also buy Y
▪ Going over our data and calculate that P(chips/beer) = 0.7.
▪ The rule can be defined: 70 percent of customers who buy beer also buy chips.
▶ Classification: Credit scoring
▪ The bank calculates the risk of giving the amount of credit for a customer based
on customer information.
▪ Customer Information:-income, savings, collaterals, profession, age, past financial
history, and so forth
▪ record of past loan- the loan was paid back or not
▪ The aim is to infer a general rule coding the association between a customer’s
attributes and his risk. A classification rule learned may be of the form:
Discriminant rule: IF income > θ1 AND savings > θ2
THEN low-risk
ELSE high-risk

24. 24
Introduction..
Application of Machine Learning..
▶ Pattern recognition: recognizing character codes from their images
▪ Handwritten—for example, to read zip codes on envelopes or amounts
on checks.
▪ People have different handwriting styles; characters may be written
small or large, slanted, with a pen or pencil, and there are many possible
images corresponding to the same character

25. 25
Introduction..
Example of Machine Learning Application..
▶ Face recognition
▪ The input is an image, the classes are people to be recognized, and the
learning program should learn to associate the face images to identities.
▪ Difficulty in lighting, glasses may hide the eyes and eyebrows, and a
beard may hide the chin.
▶ In medical diagnosis
▪ The inputs are the relevant information we have about the patient and
the classes are the illnesses.
▪ The inputs contain the patient’s age, gender, past medical history, and
current symptoms
▶ Biometrics is recognition
▪ Recognition or authentication of people using their physiological and/or
behavioral characteristics
▪ Examples of physiological characteristics are images of the face,
fingerprint, iris, and palm; examples of behavioral characteristics are
dynamics of signature, voice, gait, and key stroke

26. 26
Introduction..
Example of Machine Learning Application..
▶ Regression: Predict the price of a used car
▪ Inputs are the car attributes—brand, year, engine capacity, mileage, and
other information—that we believe affect a car’s worth.
▪ The output is the price of the car. Such problems where the output is a
number are regression problems
▶ Document clustering
▪ the aim is to group similar documents
▪ For example, news reports can be subdivided as those related to politics,
sports, fashion, arts, and so on
▪ Documents are then grouped depending on the number of shared
words.

27. 27
Introduction
Machine Learning Workflow..

28. 28
Introduction
Machine Learning Workflow..
Data Preparation
▶ The preprocessing of the data is one of the most crucial steps in any
machine learning application
▶ Data collected from the real world is transformed to a clean dataset.
▶ Raw data may contain missing values, inconsistent values, duplicate
instances etc.
▶ So, raw data cannot be directly used for building a model.
▶ Different methods of cleaning the dataset are-
▪ Handling missing values
▪ Removing duplicate instances from the dataset.
▪ Normalizing the data in the dataset.
▪ Extract meaningful features
▪ Dimensionality reduction

29. 29
Introduction
Machine Learning Workflow..
Choosing Learning Algorithm-
▶ The best performing learning algorithm is researched.
▶ It depends upon the type of problem that needs to solved
and the type of data we have.
▶ If the problem is to classify and the data is labeled,
classification algorithms are used.
▶ If the problem is to perform a regression task and the data
is labeled, regression algorithms are used.
▶ If the problem is to create clusters and the data is
unlabeled, clustering algorithms are used.

30. 30
Introduction
Machine Learning Workflow..
Training Model-
▶ The model is trained to improve its ability.
▶ The dataset is divided into training dataset and testing dataset.
▶ The training and testing split is order of 80/20 or 70/30.
▶ It also depends upon the size of the dataset.
▶ Training dataset is used for training purpose.
▶ Testing dataset is used for the testing purpose.
▶ Training dataset is fed to the learning algorithm.
▶ The learning algorithm finds a mapping between the input and the
output and generates the model.

31. 31
Introduction
Machine Learning Workflow..
Evaluating Model-
▶ The model is evaluated to test if the model is any good.
▶ The model is evaluated using the kept-aside testing dataset.
▶ It allows to test the model against data that has never been used before for
training.
▶ Metrics such as accuracy, precision, recall etc. are used to test the
performance.
▶ If the model does not perform well, the model is re-built using different hyper
parameters.
▶ The accuracy may be further improved by tuning the hyper parameters.

32. 32
Introduction
Machine Learning Workflow..
Predictions
▶ The build system is finally used o do something useful in the real world
▶ Here, the true value of machine learning is realized

33. 33
Introduction
Limitation of Machine Learning
▶ Are not useful while working with high dimensional data, that is where we
have large number of inputs and outputs.
▶ Cannot solve crucial AI problem like NLP, image recognition etc.
▶ One of the big challenges with traditional machine learning model is a
process called feature extraction
▪ Such as object recognition, handwriting recognition huge challenges .

34. 34
Understand and install the following python packages
▶ NumPy
▶ SciPy
▶ scikit-learn
▶ Matplotlib
▶ pandas