1. Draft 4/26/2010
Main single agent machine learning algorithms
Most of the machine learning methods described below are in more detail in [1].
Algorithm Description Potential for Multi-agent System
Decision Trees Decision tree learning is a method for approximating discrete function by a decision tree. In There are papers appearing about Multi-agent
the nodes of trees are attributes and in the leaves are values of discrete function. The decision tree learning problem, e.g. [2]. It mentioned
tree can be rewritten in a set of if-then rules. Trees learning methods are popular inductive that the new algorithm “is being applied to
inference algorithms, mostly used for variety of classification tasks (for example for several knowledge discovery problems in
diagnosing medical cases). For tree generation there is often used entropy as information gain molecular biology and network-based
measure of the attribute. The best-known methods are ID3, C4.5, etc. intrusion detection.
Neural Networks Neural networks learning methods provide a robust approach to approximating real-valued, There are papers which try to apply NN in
discrete-valued and vector-valued functions. The well-known algorithm, Back-Propagation, MAS. For instance, in [3], “a neural network
uses gradient descent to tune network parameters to best fit to training set with input-output based multi-agent, especially hierarchically
pair. This method is inspired by neurobiology. It imitates function of brain, where many organized, information retrieval system” was
neurons are inter-connected. The instances are represented by many input-output pairs. NN presented. This multi-agent approach was
learning is robust to errors in training data and has been successfully applied to problems motivated both analytically and
such as speech recognition, face recognition, etc. experimentally.
Bayesian Bayesian reasoning provides a probabilistic approach to inference. Bayesian reasoning The papers appeared in this area started
Methods provides the basis for learning algorithms that directly manipulate with probabilities, as well from 1993, e.g. [4-6]. These three papers
as a framework for analyzing the operation of other algorithms. Bayesian learning algorithm have the same first author. So, there is
that calculates explicit probabilities for hypothesis, such us the naive Bayes, are among the potential of succeed of this algorithm in
most practical approaches to certain type of learning problems. Bayes classifier is
MAS, but not clear evidence yet.
competitive with other ML algorithms in many cases. For example for learning to classify
text documents, the naive Bayes classifier is one of the most effective classifiers.
Reinforcement Reinforcement learning solves the task -- how the agent (that can sense and act in There was a paper [7], in which the
Learning environment) can learn to choose optimal actions to reach its goal. Each time the agent authors adopt general-sum stochastic
performs an action in its environment, a trainer may provide a reward or penalty to indicate games as a framework for multiagent
the conveniency of the resulting state. For example, when agent is trained to play a game reinforcement learning. Their work
then trainer might provide a positive reward when the game is won, negative reward when it
extends previous work by Littman on
is lost, and zero reward in other states. The task of agent is to learn from this delayed reward,
zero-sum stochastic games to a broader

2. Draft 4/26/2010
to choose sequences of actions that produce the greatest cumulative reward. An algorithm framework. They design a multiagent Q-
that can acquire optimal control strategies from delayed reward is called Q-learning. This learning method under this framework,
method can solve the problems like learning to control mobile robot, learning to optimize and prove that it converges to a Nash
operations in factories, learning to plan therapeutic procedures, etc. equilibrium under specified conditions.
(This paper was cited for 41 times.)
Inductive Logic Inductive logic programming has its roots in concept learning from examples, a relatively In recent years, a number of logic
Programming straightforward form of induction. The aim of concept learning is to discover, from a given programming-based proposals to
set of pre-classified examples, a set of classification rules with high predictive power. The
theory of ILP is based on proof theory and model theory for the first order predicate calculus.
agents have been put forward [8]. In
Inductive hypothesis formation is characterized by techniques including inverse resolution, [8], a comprehensive survey of
relative least general generalisations, inverse implication, and inverse entailment. This computational logic-based agents and
method can be used for creating logical programs from training data set. The final program multi-agent systems was provided.
should be able to generate that data back. The creating logical programs is very dependent on (This survey was cited for 21 times.)
task complexity. In many cases this method is not usable without many restrictions posed on
the final program. With success ILP is mostly used in Data Mining for finding rules in huge
databases.
Case-Based Case-Based Reasoning (CBR) is a lazy learning algorithm that classifies new query The Multi-agent Systems Lab (Dep. of
Reasoning instance by analyzing similar instances while ignoring instances that are very different from computer science at the Univ. of
the query. This method holds all previous instances in case memory. The instances/cases can Massachusetts at Amherst) has a project –
be represented by values, symbols, trees, various hierarchical structures or other structures. It “CBR in a MAS” [9]. They have now
is non-generalization approach. The CBR works in the cycle: case retrieval -reuse - solution
started investigating case-based learning
testing - learning. This method is inspired by biology, concretely by human reasoning using
knowledge from old similar situations. This learning method is also known as Learning by
and have implemeted a Distributed Case-
Analogy. CBR paradigm covers a range of different methods. Widely used is Instance-Based Based Learning System for multi-agent
Reasoning (IBR) algorithm that differs from general CBR mainly in representing instances. path planning and are in process of
The representation of the instances is simple, usually it is vector of numeric or symbolic evaluating it.
values. Instance-based learning includes k-Nearest Neighbors (k-NN) and Locally Weighted
Regression (LWR) methods.
Support Vector Support Vector Machines (SVM) has become very popular method for classification and Based on the search on web, there is not
Machines optimization at the recent time. SVMs were introduced by Vapnik et al. in 1992. This any evidence that SVM has been applied
method combines two main ideas. The first one is concept of an optimum linear margin to MAS. SVMs were successfully applied
classifier, which constructs a separating hyperplane that maximizes distances to the training in classification and regression problems.
point. The second one is concept of a kernel. In its simplest form, the kernel is a function
So there are two possible ways: 1)
which calculates the dot product of two training vectors. Kernels calculate these dot product

3. Draft 4/26/2010
in feature space, often without explicitly calculating the feature vectors, operating directly on describe MAS into several classification
the input vectors instead. When we use feature transformation, which reformulates input and regression problems; 2) apply kernel
vector into new features, the dot product is calculated in feature space, even if the new and large margin concept into MAS
feature space has higher dimensionality. So the linear classifier is unaffected. Margin condition.
maximization provides a useful trade off with classification accuracy, which can easily lead
to overfitting of the training data. SVM are well applicable to solve learning tasks where the
number of attributes is large with respect to the number of training examples.
Genetic Genetic algorithms provide a learning method motivated by an analogy to biological There are research groups undertakes
Algorithms evolution. The search for an appropriate hypothesis begins with a population of initial hy- research in applying GA in MAS:
pothesis. Members of the current population give rise to the next generation population by 1. The Evolutionary Computation
operations such as selection, crossover and mutation. At each step, a collection of hypothesis Research Group in the Department of
called the current population is updated by replacing some fraction of the population by off-
Computer Studies at Napier University
springs of the most fit current hypothesis. Genetic algorithms have been applied successfully
to a variety of learning tasks and optimization problems. For example, Genetic algorithms
undertakes research in evolutionary
can be used in other ML methods, such as Neural Network or Instance-Based Reasoning for computation and its applications. Its
optimal parameters setting. emphasis is on evolving multi- agent
systems and self-adaptation in genetic
algorithms with applications in
timetabling, logic minimisation and
control systems engineering.
2. The Evolutionary Computing Group at
UWE, Bristol, undertakes applied
research in collaboration with other
organisations, and performs research in
evolutionary computing, artificial life, and
multi-agent systems.
There are also papers about the research
in this area, e.g. [10] (2 citations).