1. The document introduces machine learning and provides an overview of key concepts, including the goal of machine learning, the meaning of learning, components of a learning problem, and types of learning feedback and algorithms. 2. Machine learning aims to develop methods for building intelligent learning machines that can improve performance on tasks using examples without being reprogrammed. Learning is defined as improved performance on a task over time based on experience. 3. A learning problem specifies the task, data, and performance measures, while a learning algorithm aims to improve performance through inductive or deductive reasoning from examples or known facts.

1. Machine Learning:
Introduction
Dr Valentina Plekhanova
University of Sunderland, UK
The Field of Machine Learning
The goal of machine learning is to develop methods,
techniques and tools for building intelligent learning
machines, that can solve the problem in combination with
machines,
examples.
an available data set of training examples.
Note: A learning machine or an algorithm does not solve
the problem directly.
Valentina Plekhanova Machine Learning: Introduction 2
The Meaning of Learning
When a learning machine improves its performance at a
given task over time, without reprogramming, it can be said
to have learned something.
Learning: improvement of performance with experience at a given task
improve over task T
with respect to performance measure P
based on experience E
Example from Machine/Computer Vision field:
learn to recognise objects from a visual scene or an image
T: identify all objects
P: accuracy (e.g. a number of objects correctly recognised)
E: a database of objects recorded
Valentina Plekhanova Machine Learning: Introduction 3
1

2. Components of a Learning Problem
Task: the behaviour or task that’s being improved, e.g.
classification, object recognition, acting in an environment.
Data: the experiences that are being used to improve
performance in the task.
Measure of improvements: How can the improvement
be measured? Examples:
Provide more accurate solutions (e.g. increasing the accuracy in
prediction)
Cover a wider range of problems
Obtain answers more economically (e.g. improved speed)
Simplify codified knowledge
New skills that were not presented initially
Valentina Plekhanova Machine Learning: Introduction 4
Notes on
Learning Problem & Learning Tasks
Learning Problem: Precise model that describes:
Problem
what is to be learned
how it is done
what measures are to be used in analysing and comparing the
performance of different solutions.
Learning tasks are formal definitions of “what we want to
“what
learn”.
learn”.
Valentina Plekhanova Machine Learning: Introduction 5
Learning Algorithm
In Machine learning: Learning algorithm - Learning Agent.
Note: A learning agent is not concerned with constructing
an exact answer (e.g. an exact concept definition).
Valentina Plekhanova Machine Learning: Introduction 6
2

3. Learning Feedback
Learning feedback can be provided by the system
environment or the agents themselves.
Supervised learning: specifies the desired activities/objectives of
learning – feedback from a teacher
Unsupervised learning: no explicit feedback is provided and the
objective is to find out useful and desired activities on the basis of trial-and-error
and self-organisation processes – a passive observer
Reinforcement learning: specifies the utility of the actual activity of
the learner and the objectives is to maximise this utility – feedback from a critic
[ Sen & Weiss, 1999 ]
Valentina Plekhanova Machine Learning: Introduction 7
Ways of Learning
Rote learning, i.e. learning from memory; in a mechanical way
Learning from examples and by practice
Learning from instructions/advice/explanations
Learning by analogy
Learning by discovery
…
Valentina Plekhanova Machine Learning: Introduction 8
Inductive & Deductive Learning
Inductive Learning: Reasoning from a set of examples
Learning
to produce a general rules. The rules should be
applicable to new examples, but there is no guarantee
that the result will be correct.
Deductive Learning: Reasoning from a set of known
Learning
facts and rules to produce additional rules that are
guaranteed to be true.
Valentina Plekhanova Machine Learning: Introduction 9
3

4. Assessment of Learning Algorithms
The most common criteria for learning algorithms
assessments are:
Accuracy (e.g. percentages of correctly classified +’s and –’s)
Efficiency (e.g. examples needed, computational tractability)
Robustness (e.g. against noise, against incompleteness)
Special requirements (e.g. incrementality, concept drift)
incrementality,
Concept complexity (e.g. representational issues – examples & BK)
Transparency (e.g. comprehensibility for the human user)
Valentina Plekhanova Machine Learning: Introduction 10
Some Theoretical Settings
Inductive Logic Programming (ILP)
Probably Approximately Correct (PAC) Learning
Learning as Optimisation (Tasks: Reinforcement Learning)
Bayesian Learning
…
Valentina Plekhanova Machine Learning: Introduction 11
“Each task can be related to one or more methods”
Concept Learning: an Example
Methods: Lazy Learning; Incremental Decision Tree
Methods:
Learning; Boosting; Support Vector Machines (SVM).
Theories: e.g. Inductive Logic Programming (ILP);
Theories:
Probably Approximately Correct (PAC) Learning;
Bayesian Learning; Statistical Learning.
Valentina Plekhanova Machine Learning: Introduction 12
4

5. Summary
A Model of Learning: Key Aspects
Learner: who or what is doing the learning, e.g. an algorithm, a
computer program.
Domain: what is being learned, e.g. a function, a concept.
Goal: why the learning is done.
Representation: the way the objects to be learned are represented.
Algorithmic Technology: the algorithmic framework to be used, e.g.
decision trees, lazy learning, artificial neural networks, support vector
machines. [Sen & Weiss, 1999 ]
Valentina Plekhanova Machine Learning: Introduction 13
Summary
A Model of Learning: Key Aspects
Information Source: the information/training data the program uses for
learning, e.g. positive/negative examples, feedback from actions.
Training/Learning Scenario: the description of the learning process, e.g.
interactive, supervised / unsupervised, etc.
Prior Knowledge: what is known in advance about the domain, e.g.
about specific properties of the concepts to be learned.
Success Criteria: the criteria for successful learning.
Performance: e.g. the amount of time, space and computational power
needed in order to learn a certain task.
[Sen & Weiss, 1999 ]
Valentina Plekhanova Machine Learning: Introduction 14
5