STOCHASTIC PEDESTRIAN MOTION MODEL

A Stochastic Pedestrian Motion Model
Semester Prodject
Marko Tanaskovic
Department of Information Technology and Electrical Engineering
Advisor: Stefano Pellegrini
Supervisor: Prof. Dr. Luc van Gool
February 22, 2010

Abstract
A good motion model is an important part of any tracking algorithm. In this work we propose two stochastic
versions of already existing deterministic pedestrian motion model. The deterministic model we started
from shows significant improvements compared to other dynamic models when used for tracking tasks in
moderately crowded scenarios. In order to further extend this model we propose two stochastic variants.
The first approach is based on particle filter algorithm. Since this approach has several characteristics that
make it hard to use in practice, we propose one more algorithm that is based on approximating probability
density function with the mixture of Gaussians. Both algorithms were tested on real world data sets that
were available. Experiments show that in many situations in which the deterministic model has significant
prediction errors, our stochastic models yield acceptable results.

Contents
1 Introduction 1
2 Overview of the deterministic Linear Trajectory Avoidance model 3
3 Stochastic Linear Trajectory Avoidance model 7
3.1 Solution based on particle ﬁlter algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.2 Solution based on mixture of Gaussians . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
4 Experiments and Results 19
5 Conclusion 27
I

List of Figures
2.1 Illustration of the prediction obtained by using the deterministic LTA model . . . . . . . . . 5
3.1 Original energy function and corresponding probability density functions for different values
of parameter α . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.2 Illustration for the method of approximation of parameters of the mixture of Gaussians . . . 13
3.3 Comparison of the real probability density function and its approximation based on the
energy function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.4 Plot for mean velocity change over a time interval and its standard deviation . . . . . . . . . 16
3.5 Illustration of the local minima points of energy function search method . . . . . . . . . . . 17
4.1 Comparison of the deterministic and the stochastic model based on the particle ﬁlter algorithm 20
4.2 Illustration of the simulation results obtained using the model on particle ﬁlter on real-world
data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
4.3 Mean simulation errors obtained from the experiment on real-world data when using differ-
ent values for parameter α. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
4.4 Error variances obtained from the experiment on the real-world data when using different
values for parameter α. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
4.5 Comparison of error histograms for the model based on the mixture of Gaussians and ap-
proximation with a single Gaussian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
4.6 Results of the stochastic pedestrian motion model based on the mixture of Gaussians . . . . 25
III

Chapter 1
Introduction
A motion model is a very important component of any tracking system. In their recent work Pellegrini et al.
[1] proposed a deterministic pedestrian motion model for walking people with short-term prediction in mind,
termed Linear Trajectory Avoidance (LTA) model. This model has proved to be superior to other motion
models in terms of prediction as well as effective in tracking experiments. Furthermore, this dynamic model
can be very useful tool for tracking tasks in busy scenarios with frequent short-lasting occlusions. One of
its main advantages is that it operates in real world coordinates and can therefore be used by any tracker.
The model was tested with simple patch-based tracker as well as with tracking with state of the art moving
observer and it significantly improved these tracking algorithms in terms of reducing the number of ID
switches.
The aim of this work is to extend the LTA model by introducing two stochastic variants. When predicting
pedestrian velocity and position, the deterministic LTA model takes into account social interactions between
pedestrians as well as their orientation towards a goal. However, it is very hard to model all the factors
that actually influence pedestrian motion. For example, there may be some scene information that is not
considered, but might influence pedestrian motion significantly. Furthermore, often there are situations
in which there is more than one obvious direction and velocity that can be chosen by the pedestrian (for
example pedestrian can go around an obstacle by its two sides). In these situations, deterministic LTA
model can give only one of those possibilities which, if the actual choice of the subject was different, can
result in significant prediction error. The natural way of solving those problems is to introduce a stochastic
motion model based on LTA. This model will, instead of giving only one exact velocity that could be taken
by the pedestrian, give a probabilistic estimate of the next chosen velocity. Therefore, stochastic motion
model gives a probability density functions for pedestrian velocity and position, which can cover different
factors that we cannot precisely take into consideration when building our model as well as different choices
that are equally likely to be chosen by the pedestrian.
In this work we propose two different approaches for building stochastic pedestrian motion model based
on LTA. First we examined an approach that is based on particle filter algorithm. This approach was the
logical starting point for building a stochastic model, as the particle filter algorithm is widely used and
has been thoroughly researched in the past years. Although this approach gives satisfying results, it is
quite computationally expensive. Moreover, due to limitations that this algorithm has performing parallel
simulation for multiple subjects simultaneously becomes extremely computationally expensive. Therefore,
we examined another approach for building a stochastic variant of the motion model. This approach is
1

CHAPTER 1. INTRODUCTION
based on approximating the pedestrian position and velocity probability density function with the mixture of
Gaussians. The parameters in the mixture are approximated by using energy function given by deterministic
LTA model. This approach is much faster and can be used in practical applications. Both models were
implemented in Matlab and tested on real world data.
In this work we start by giving a short overview of the Linear Trajectory Avoidance model as it is the
base for both approaches presented in this work. We also show what was the main motivation for proposing
the stochastic version of the model. Then we describe the two algorithms used for building our stochastic
motion model. At the end we analyze the results of these two models and compare those results with the
results obtained by deterministic LTA model in order to illustrate improvements introduced by the stochastic
model.
2

Chapter 2
Overview of the deterministic Linear
Trajectory Avoidance model
Since the main goal of this work is to propose a stochastic variant of the Linear Trajectory Avoidance (LTA)
model, in the following chapter we will briefly describe this model and give its mathematical background.
Also, we will point out the weaknesses of the deterministic LTA model that were the main motivation for
suggesting a stochastic version as an extension to the existing model.
The LTA model takes into account three factors that influence human motion. The first one is that in
normal situations pedestrians would not bump into each other or into static obstacles in the scene, but rather
keep on moving while trying to evade the obstacles. Furthermore, it presumes that each pedestrian will try
to keep his velocity close to the desired average velocity of the movement. Thirdly, it assumes that every
pedestrian in the scene has a destination point and that, while avoiding the obstacles, he will keep on moving
towards this destination point. These three assumptions are mathematically modeled through the formation
of an energy function.
If we adopt the following notation for vectors of pedestrian velocity: v = vx vy and position:
p = px py in the world coordinates and if we denote the desired velocity of the subject (the velocity
with which the subject would like to proceed in the next time step) as v∗, square distance between the subject
in question and some other subject in the scene at the time t is given by:
dj(t, v∗
) = p + tv∗
− pj − tvj
2
(2.1)
Where pj and vj are the position and velocity of that other subject respectively.
It is assumed that positions and velocities of all the other subjects in the scene are known to the subject
in question. Another assumption is that the subject in question will adopt his new velocity v∗ so that the
distance between him and the subject j is always greater than the value that is considered to be comfortable.
The fact that the first derivative of dj(t, v∗) in time at the minima point is equal to zero, is used to eliminate
time from the equations and hence the minima value of the squared distance is:
d∗
j (v∗
) = kj −
kjqT
j
qj
2 qT
j
2
(2.2)
3

CHAPTER 2. OVERVIEW OF THE DETERMINISTIC LINEAR TRAJECTORY AVOIDANCE MODEL
Where, for the sake of bravity, the following notation was adopted:
kj = p − pj
qj = v∗ − vj
(2.3)
Based on this value an energy function is formed as:
Ej(v∗
) = e
−
d∗
j (v∗)
2σd
2
(2.4)
Where the coefficient σd determines the comfortable approach distance between the subjects.
All the subjects in the scene will contribute to this energy function, but not in the same way. Subjects that
are closer to the subject in question will have more influence, and this influence will also depend on whether
the other subject is in line of sight of the subject in question. Therefore, the overall energy component
influenced by other subjects in the scene will be given as a weighted sum:
I(v∗
) =
j
wd
j wΦ
j Ej(v∗
) (2.5)
Where the sum coefficients determine the contributions of each subject to this energy function and are given
by:
wd
j = e
−
kj
2
2σω2
wΦ
j = (1+cos(Φ)
2 )
β
(2.6)
Where the coefficient σω determines the radius of influence of other subjects, Φ is angular displacement
between subject in question and the subject j and coefficient β determines the perkiness of the field-of-view.
The Subject’s desire to keep the velocity close to a desired average velocity is modeled through an
energy potential:
S(v∗
) = (u− v∗
)2
(2.7)
Where u is the desired average velocity. Also, the desire for moving towards a destination point is modeled
by another energy potential:
D(v∗
) = −
(z − p)v∗
z − p v∗
(2.8)
Where z is the desired destination point.
The Overall energy function is obtained as a weighted sum of these three components:
E(v∗
) = I(v∗
) + λ1S(v∗
) + λ2D(v∗
) (2.9)
The desired velocity of the subject in question is the one that minimizes this energy function. Since the
minima point of the energy function cannot be found in closed form, the gradient descend algorithm is used.
This is the weakness of the deterministic model we will try to overcome by proposing the stochastic LTA
model. The weakness is that the energy function could have more than one local minima point, which, in
other words, means that the subject in question might have more than one possibility on how to proceed
moving toward the destination. If the gradient descend algorithm used by the deterministic LTA model finds
a local minima of the energy function that does not correspond to the choice of the subject, the prediction
obtained by the LTA model will have a noticeable error. These situations typically occur when there is an
4

Figure 2.1: Illustrates the case in which the deterministic LTA model makes a mistake. The image shows
six frames of a video that compares estimations of the deterministic LTA model with the real data. The
estimated pedestrian position is given by the green square and the real position is given by the red circle.
In the upper row, we can see that the prediction is quite good. However, in the lover row we see that the
deterministic LTA model makes a significant mistake. The reason is that the subject uses different side than
the one chosen by the LTA model to evade an incoming pedestrian
obstacle or another pedestrian that can be passed on by either side with equal or close to equal probability.
Figure 2.1 shows such a situation, where the prediction obtained by the LTA model has a significant error.
The velocity vector obtained by minimization of the energy function is the desired velocity of the pedes-
trian. However, it is assumed that because of the inertial constraints the subject has to undertake a transition
from his current velocity to that desired velocity, and therefore the velocity of the subject in the next time
step is given through filtering the desired velocity:
vt+1 = (1 − αf )v∗
+ αf vt (2.10)
Where αf is the filtering coefficient. Therefore, LTA model predicts the next position of the subject as:
pt+1 = pt + ∆tv (2.11)
Where ∆t is simulation time increment.
All the free design parameters in the deterministic LTA model were learned by using two real-world
5

sequences of moderately crowded walking scenes. In our work we keep the same values for those param-
eters and use the same real-world data for testing our stochastic models. Static obstacles are modeled as
pedestrians with zero velocity and at each time step the point of the obstacle closest to the subject is taken
to be the obstacle position.
The deterministic LTA model described here is the starting point of our work. In the following chapters
we will explore ways to further extend this model by proposing its stochastic version.
6

Chapter 3
Stochastic Linear Trajectory Avoidance
model
The deterministic pedestrian motion model gives exactly one possible value for pedestrian position and
velocity at each time step. In the stochastic model, on the other hand, the probability density function of
pedestrian position and velocity is calculated at each time step. We will explore two different approaches to
building a stochastic pedestrian motion model. The first approach is based on the particle filter algorithm and
the other one is based on approximating probability density function with the mixture of Gaussians. In both
approaches the energy function discussed in the previous chapter is transformed into a probability density
function. This probability density function will become the key tool for building our stochastic models.
Instead of calculating pedestrian desired velocity through minimization of the energy function as in the
deterministic LTA model, in our stochastic models this velocity will be given through a probability density
function obtained from the energy function. We use the idea that an energy function can be transformed into
a probability density function by the following formula:
p(v) =
1
Z
e−αE(v)
(3.1)
Where α is an arbitrary constant that will influence the final shape of the probability density function and
Z is the normalizing factor which insures that we obtain a proper probability density function. Since the
obtained probability density function has to satisfy the following property:
∞
−∞
∞
−∞ p(v)dvxdvy = 1 , the
normalizing factor is given by the expression:
z =
∞
−∞
∞
−∞
e−αE(v)
dvxdvy (3.2)
Equation 3.1 is intuitively very justifiable. The probability density function will have high values when
the energy function has small values. In limit cases we see that if the energy function tends to infinity, the
probability density function will tend to zero, and as the energy function tends to zero, the probability density
function will tend to one. Therefore, the local minima of the energy function will become local maxima of
the probability density function. This means that high probabilities will be attributed to velocity values at
the minima points of the energy function. Furthermore, significant probabilities will be attributed to all the
values that lay close to the minima points of the energy function. Figure 3.1 shows an energy function that
7

CHAPTER 3. STOCHASTIC LINEAR TRAJECTORY AVOIDANCE MODEL
has two minima points and its corresponding probability density functions obtained for different values of
parameter α .
−5
0
5
−5
0
5
−20
0
20
vx
Pedestrian energy function
vy
E(v)
−5
0
5
−5
0
5
0
0.05
0.1
vx
Probability density function for alfa=0.5
vy
p(v)
−5
0
5
−5
0
5
0
0.2
0.4
vx
Probability density function for alfa=1
vy
p(v)
−5
0
5
−5
0
5
0
0.5
1
vx
Probability density function for alfa=3
vy
p(v)
Figure 3.1: Upper left figure shows an energy function obtained from the real-world data by using the
deterministic LTA model. Other plots show the corresponding probability density functions obtained for
different values of parameter α. The probability density functions have two maxima points in place of the
minima points of the energy function
3.1 Solution based on particle filter algorithm
In the approach based on particle filter algorithm we will use a sample set to represent the probability density
function of pedestrian velocity and position over time. Instead of propagating the desired pedestrian velocity
deterministically, as described in the previous chapter, we will use a stochastic process for the propagation.
For the transition probability density function of the desired pedestrian velocity, we take the probability
density function given by the expression 3.1:
f(v∗
t |vt−1) =
1
Z
e−αE(v∗
t )
(3.3)
For making use of the particle filter algorithm, we first note that the stochastic process of estimating pedes-
trian’s next velocity and position given by this transition probability density function can be considered
8

Markovian. This means that the estimated position and velocity of the pedestrian at the current time mo-
ment depend only on the estimated position and velocity in the previous time step. Since we calculate
pedestrian position and velocity at the current time step according to the equations 2.10 and 2.11, it is clear
that they depend on the estimated position and velocity in the previous time step and the estimation of the
pedestrians desired velocity. Furthermore, since the energy function depends only on the velocity and posi-
tion in the previous time step, the transition probability density function for pedestrians desired velocity can
be considered Markovian. Hence, we can consider the whole propagation process to be Markovian.
In order to calculate position and velocity probability density functions at each time step, the samples
that represent them have to be propagated in time. We would like to do this by sampling the desired pedes-
trian velocities from the transition probability density function given by the equation 3.3. However, since
this probability density function is quite complicated, we do not sample from it directly, but we use an-
other distribution for sampling (this distribution is often called instrumental or importance distribution in
the literature). As the instrumental distribution we will use Normal distribution with the mean value that
corresponds to the sample velocity vector in the previous time step and with a variance that should be big
enough to ensure that the instrumental distribution completely covers the original distribution that we are try-
ing to estimate (instrumental distribution has a support that is big enough). We will denote the instrumental
distribution as follows:
q(v∗
t |vt−1) = N(vt−1, σ) (3.4)
The original distribution is approximated by weighting samples from the instrumental distribution. Each
of the samples (often called particles in the literature) i is assigned a positive weight. These weights are
proportional to the ratio of the original and the instrumental probability density functions.
ri
t =
f(v∗i
t |vi
t−1)
q(v∗i
t |vi
t−1)
(3.5)
Final weights are obtained by normalization. This means that for each particle we calculate the weight as
follows:
ωi
t =
ri
t
N
k=1 rk
t
(3.6)
Where N is the number of particles in the set.
The second step in the approximation is resampling. This is done by randomly drawing particles from the
particle set by using the normalized weights as the selection probabilities while keeping the same number of
samples. During the process of resampling those particles that have high weights are likely to be replicated
while those with low weights will probably be eliminated. Resampling is a very important step in our
algorithm. If we just propagated particles in time by multiplying the weights at each time step, after some
time they would become very small and this could cause numeric problems. Therefore, in order to avoid
these problems, resampling is introduced.
At each time step we have N particles that represent the probability density functions of pedestrian
position and velocity. In order to obtain the estimates of position and velocity in the next time step, each
particle is propagated by sampling from its instrumental distribution. Also, an energy function is formed for
each particle, where we regard each particle as a pedestrian, and the value of the normalizing parameter Z
that is needed for the transformation of the energy into probability is calculated. For this calculation we use
off-the-shelf Matlab function dblquad that performs numeric integration of functions with two variables.
Since particles that are identical have the same energy functions and hence the same values for Z, we ﬁrst
identify all the particles that are identical. The value of Z is calculated only once for each group of identical
particles, which signiﬁcantly reduces the computational effort. Having the value for Z, we transform the
9

energy function into the probability density function and calculate unnormalized weights for each particle.
In addition, the velocity is filtered and particle position vector is calculated. At the end all the weights
are normalized and the particle set is resampled. The particles that are left after resampling will represent
the probability density function of pedestrian velocity and position at the next time step. These probability
density functions could be obtained by taking the histogram of particle velocities and position. The whole
algorithm is initialized by taking the real values for pedestrian position and velocity. At the time step t = 0
we create N identical particles initializing their velocities and positions with the real velocity and position
of the subject. Since all the particles are identical, they are assigned weights that are equal to 1
N .
The algorithm with weight calculation and resampling described above is very similar to what is known
in the literature as the particle filter algorithm [3].The only difference is that our model is used purely for
prediction and we do not have observations that are present in the regular particle filter algorithm. This
algorithm is very computationally expensive. Furthermore, if we would like to make stochastic simulation
based on this algorithm not just for one subject in the scene, but for many of them simultaneously, we would
need to represent velocities and positions of all the pedestrians with one probability density function. This
is to say that if a motion of N subject is simulated, then the probability density functions of pedestrian
position and velocity will have the dimension of 2N. The problem here is that in order to correctly represent
a multidimensional probability density function with samples, we need a large sample set. In fact, it could
be shown that for the particle filter algorithm to work properly when the order of the filter is increased,
the number of particles has to increase exponentially with the order of the filter. This phenomenon is
called the curse-of-dimensionality of the particle filter [4, 5]. Since the simulation of one subject motion is
already quite expensive, this exponential growth makes the computation effort extremely big. Furthermore,
the experiments show that during resampling some of the paths represented by particles could become
meaningless which further makes this approach less useful in practice. Therefore, in order to overcome
these drawbacks we propose another algorithm.
3.2 Solution based on mixture of Gaussians
The main idea of this approach is to estimate the probability density function of pedestrian position and
velocity with the mixture of Gaussians. This means that at each time step, the probability density function of
pedestrian position and velocity is given by a weighted sum of Gaussian functions with different mean values
and covariance matrices. Therefore, the task of estimating this function becomes the task of estimating
mean values, covariance matrices and weights of the elements in the mixture at each time step. For those
estimations we will again assume that we can model the propagation of the pedestrian desired velocity by a
stochastic process, where the transition probability density function of the desired velocity is modeled by the
expression 3.1. The idea is to approximate this probability density function with the mixture of Gaussians:
p(v∗
) ≈
N
i=1
wi
N(v∗
, mi
, Σi
) (3.7)
Where N is the number of elements in the mixture, wi are positive weights that have to sum up to one, mi
are mean values and Σi are covariance matrices of the elements in the mixture.
Since the maximum of a single 2-D Gaussian probability density function is reached at its mean value,
we could find the mean of the Gaussian function simply by finding the point for which it attains its maximum.
Furthermore, if we denote a 2-D Normal distribution as: N(v, m, Σ) = 1
2π
√
|Σ|
e−1
2
ρ(v)
, where we have
10

ρ(v) = (v −m)Σ−1(v −m)T , it can easily be shown that the covariance matrix of the distribution is equal
the hessian matrix of ρ(v calculated at the mean value, where the hessian matrix is given by:
Hess(ρ(v)) =


∂2ρ(v)
∂v2
x
∂2ρ(v)
∂vx∂vy
∂2ρ(v)
∂vy∂vx
∂2ρ(v)
∂v2
y


Therefore, the natural way of approximating the probability density function given by the equation 3.1 with
the mixture of Gaussians would be to find all the local maxima points of this function. These points would
then correspond to the mean values of the elements in the mixture. Furthermore, the covariance matrices
of the mixture elements could be obtained by inverting the hessian matrices of the expression αE(v) at the
local maxima points. However, the local maxima of the probability density function correspond to the local
minima of the energy function. Moreover, since Hess(αE(v)) = αHess(E(v)), we can approximate the
covariance matrices of the mixture elements as:
Σ = (αHess(E(v))|v=argmin(E(v)))−1
(3.8)
For approximating mixture weights, we can look at the values of our target function at the found maxima
points (or the minima points of the energy function). If we assume that the function we are trying to
approximate is really a mixture of Gaussians, we could estimate the weights by solving the following system
of equations:
1
Z e−αE(mini) = wi
2π
√
|Σi|
+
j=i
wj
2π |Σj|
e−1
2
(mini−minj)Σ−1
j (mini−minj)T
i = 1, . . . , N; j = 1, . . . , N
(3.9)
Where wi are the weights we are looking for, mini are the minima points of the energy function, and Σi
are the approximations of the covariance matrices. For solving this system, we do not have to calculate
the value of Z, as we can use the constraint that all the weights should sum up to 1. However, since the
probability density function we are trying to estimate with the mixture of Gaussians is not really a mixture
of Gaussians, solving the system of equations above does not guaranty that all the weights will be positive.
Therefore, we will use an approximation that is more crude, but which guaranties that the weights we obtain
are positive. For this approximation we assume that the elements in the mixture are sufficiently separated so
as not to overlap. This means that we can consider:
e−1
2
(mini−minj)Σ−1
j (mini−minj)T
≈ 0 ∀i = j (3.10)
With this simplification our system of equations becomes:
1
Z e−αE(mini) = wi
2π
√
|Σi|
i = 1, . . . , N
N
i=1
wi = 1
(3.11)
And therefore we can calculate each weight as:
wi =
e−αE(mini)
|Σi|
N
j=1
e−αE(minj)
|Σj|
(3.12)
11

In order to approximate the probability density function given by the equation 3.1 with the mixture of
Gaussians we only need the energy function. We need to find all the local minima of the energy function
and the hessian matrices at those minima points, and then, using the equations 3.8 and 3.12, we can calculate
the covariance matrices and weights of the elements in the mixture.
We can test the accuracy of this approximation by performing a simple experiment. In this experiment
we take a real the mixture of Gaussians, and transform it into an energy function by using the formula:
E(v) = −
ln p(v) + ln Z
α
(3.13)
Where p(v) is the original probability density function and Z is an arbitrary constant. Then we approximate
the parameters of the mixture elements using the described procedure. This experiment shows that the ap-
proximation result is very close to the original probability density function, even in the cases where there is
a significant overlapping of the elements in the mixture. Figure 3.2 compares the original probability density
function given by the mixture of Gaussians and its approximation for the 1-D and 2-D case. It also shows the
corresponding energy functions. In the case when the probability density function is not really the mixture
of Gaussians, but a function obtained from the energy function, our approximation is not so accurate. Figure
3.3 compares the probability density function obtained from the energy function (the same one as in the
figure 3.1) and its approximation with the mixture of Gaussians. We can see that the approximation is quite
crude in this case. Instead of the approximation that we adopted, we could have used another algorithm, such
as expectation-maximization algorithm. Nevertheless, such an algorithm would be more computationally
expensive. Here we try to make a good balance between the accuracy and the computational expensiveness
and the approximation we adopted is quite fast and has acceptable accuracy.
The desired pedestrian velocity, denoted by v∗, has the distribution in the form of the mixture of Gaus-
sians with the parameters that are estimated as explained before. However, the pedestrian velocity is obtained
by filtering the desired velocity (the equation 2.10). Knowing the properties of the Normal distribution, the
probability density function of the filtered velocity will also be a mixture of Gaussians. The weights in this
mixture will be the same as the weights in the mixture of the desired velocity, and the mean values and the
covariance matrices of the mixture elements will be given by the following formulas:
vfilt = (1 − αf )v∗ + αf vold
Σfilt = (1 − αf )2Σ
(3.14)
Where vfilt and vold are the mean values of the filtered velocity and the pedestrian velocfity in the previous
time step, and Σfilt is the covariance matrix of the element in the the mixture of Gausssians that represents
the filtered pedestrian velocity.
In our stochastic pedestrian motion model the probability density function of pedestrian position and
velocity has to be calculated at each time step. In order to get these calculations, the pedestrian velocity and
position probability density function is propagated in time. In this propagation we assume a linear model.
That is to say that the propagation of the position and velocity can be represented by the following formula:
pt+1
vt+1
=
1 ∆t
0 1
pt
vfilt
+ q (3.15)
Where q is a random vector with the following distribution:
q ∼
N(0, ∆tΣfilt)
N(0, Σfilt)
(3.16)
12

Figure 3.2: The upper left figure shows the original probability density function and its approximation based
on our algorithm in 1-D. The original probability density function is shown in red and the approximation in
blue. The corresponding energy function is plotted in the upper right figure. In the lower row the same thing
is shown for the 2-D case. Here the original probability density function is illustrated by red x
Now that we have this linear model, we can define a 4X4 matrix P that we will call the matrix of uncertainty.
This matrix can be propagated in the following way:
Pt+1 = FPtFT
+ Q (3.17)
Where the matrices F and Q have the following form:
F =




1 0 ∆t 0
0 1 0 ∆t
0 0 1 0
0 0 0 1



 Q =




∆tΣfil
0 0
0 0
0 0
0 0
Σfil



 (3.18)
The pedestrian motion model based on the mixture of Gaussians is initialized by creating a single Gaus-
sian function as the probability density function of pedestrian position and velocity. This Gaussian function
is attributed the weight w0 = 1 and it has the mean value that corresponds to the real pedestrian position
and velocity at the starting moment of the simulation. Matrix P is initialized as a matrix of all zeros. This is
done because at the time step t = 0 we know the exact position and velocity of the subject and, therefore,
the uncertainty is equal to zero. This probability density function is propagated in time. Let us assume that
at time step t this probability density function is given by a the mixture of Gaussians with Mt elements,
13

Figure 3.3: Shows the original probability density function obtained from the energy function shown in
the figure 3.1 and its approximation with the mixture of Gaussians. The lower two figures show those two
probability density functions together. In order to illustrate their difference better, in the left figure the
approximation is plotted as transparent (blue) and in the right the original probability density function is
plotted as transparent (red)
where each element has the mean value pi
t and vi
t, weight wi
t and uncertainty matrix Pi
t , where the index
i = 1, . . . , Mt represents different elements in the mixture. In order to estimate the parameters of the mix-
ture elements in the next time step, the energy function is formed for each mixture element. The formation
of the energy function is done as in the deterministic model, where we use the expected element position
and velocity as deterministic values to form the energy function. Therefore, for each element in the mixture
we have an energy function denoted by Ei(v). For each mixture element, all the minima points of the cor-
responding energy function are found and based on those minima points, weights and covariance matrices
are estimated by using the algorithm described before. If we assume that each energy function Ei(v) has
Ni minima points, then we can denote them as mini
j, we also denote the corresponding weights as wi
j, and
the covariance matrices as Σi
j, where for each i, j = 1, . . . , Ni. In the next time step the mixture will have
Mt+1 = Mt
i=1 Ni elements and for each element in the mixture we calculate its mean value as:
vi,j
t+1 = (1 − αf )mini
j + αf vi
t
pi,j
t+1 = pi
t + ∆tvi,j
t+1
(3.19)
Furthermore, the weights of the mixture elements are calculated as:
wi,j
t+1 = wi
twi
j (3.20)
14

By this we make sure that at each time step the weights of all the elements in the mixture sum up to one.
Also, the matrix P in the next time step is formed by using the equation 3.17:
Pi,j
t+1 = FPi
t FT
+




∆t(1 − αf )2Σi
j
0 0
0 0
0 0
0 0
(1 − αf )2Σi
j



 (3.21)
The issue of finding the local minima of the energy function is not trivial as we need to find all the
local minima. For this purpose we will use the gradient descend algorithm. Starting from an initial point
this algorithm will, after a couple of iterations, give the local minima. Therefore, in order to make sure
we will find all the local minima points of the function, we need to create a discretized search area and
perform the gradient descend algorithm from each point in this area. The open question here is what this
area should be. Since the new pedestrian velocity is never too different from the velocity in the previous
time step, it is intuitively very justifiable to take the square area with the expected mixture element velocity
in the previous time step as its center. However, the question of how big this area should be and how fine its
discretization should be is not an easy one. Taking a very big area and a fine discretization guaranties that
we will find all the local minima points. However this would mean that we have a lot of points for which
we perform the gradient descend algorithm which makes the procedure of finding the minima points very
computationally expensive. Therefore, we will optimize this procedure by trying to find all the local minima
of the energy function, but at the same time reducing the number of points from which we need to start the
gradient descend algorithm as much as possible.
For deciding on the size of the search area, we look at the recorded sequences of the real-world crowded
scenes that are available and that were used for training the deterministic pedestrian motion model. We are
interested in how much the pedestrian velocity can change over time. We look at the mean velocity change
and its variance for all the pedestrians in the scene and over different time intervals. Figure 3.4 shows the
mean velocity change and its standard deviation for different time intervals. Therefore, depending on the
time interval of the simulation, in order to make sure that our search area is big enough to cover all the
possible minima points, we take the side of the search area to be:
a = 2(mean(∆t) + 3σ(∆t)) (3.22)
Where mean(∆t) is the mean velocity change for the given time increment of the simulation and σ(∆t) is
the corresponding standard deviation of the velocity change. For the discretization we try to find a solution
that would give us a satisfying precision for a small number of points that have to be visited. The experiments
on the real-world data show that the optimal distance between the points in the search area should be between
0.15 and 0.25. For our algorithm we take this value to be 0.2.
With such a discretization area there are typically around 60 points that should be visited in order to
find all the energy function minima. However, we can additionally reduce this number without impeding
the accuracy. We have divided our search area into a finite number of squares and took only one point to
represent the whole square. If we start the gradient descend algorithm from one point and if starting from
that point we find a minima point, we assume that there is no need to start the algorithm from the points that
represent squares lying on the straight line between the point from which the descend algorithm was started
and the found minima point. In order to use this approximation fully, we note that in most cases we will have
one or several minima points that are located near the center of the discretized search area. Therefore, in our
search algorithm we should first visit the points that lie on the edges of the search area as this would give
very long lines which cross many cells in the search area. In our implementation we start the search from
15

1 2 3 4 5 6 7 8 9 10
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
∆ t=0.4
∆v
Figure 3.4: Mean velocity change for different time intervals is shown in red. The corresponding standard
deviations are illustrated as blue bars
the far left edge of the search area and perform the search (for all the points for which it is needed) going
column by column from the left to the right until the central column is reached, then we perform the same
procedure for the right side of the search area (we start the search from the border right column of the area
and perform the search going column by column from the right to the center). This procedure additionally
reduces the number of points that need to be visited to the number between 20 and 30. Figure 3.5 illustrates
this procedure.
For the gradient descend algorithm we use off-the-shelf Matlab function fminunc. This function can
be very fast if we provide it with the gradient of the function for which we are trying to ﬁnd the minima.
Since the gradient descend was used in the deterministic model, and the gradient of the energy function
was calculated, we use these results to providefminunc the gradient values of the energy function. Using
this off-the-shelf function brings an additional advantage. Since, in addition to coordinates of the minima
point, this function returns the hessian matrix at the minima point, there is no need to perform a separate
calculation in order to obtain this matrix that we need in our algorithm.
16

Figure 3.5: Illustrates the local minima points of energy function search method. The red dot represents the
minima point. All the cells that are colored in green do not have to be visited, since they lie on the straight
line connecting the starting point (the blue X) and the minima point. The algorithm ﬁrst searches the left
side of the search area and then the right side. The arrows illustrate how the search is done column by
column going from the left to the center and from the right to the center of the search area
17

18

Chapter 4
Experiments and Results
In this section we present the results obtained using the two stochastic pedestrian motion models. We also
compare these two models with the deterministic pedestrian motion model. In building the code for our two
stochastic models, we used the existing code for the deterministic LTA model. In all the simulations we
keep the same values of coefficients that are used for the formation of the energy function.
As we have seen, the approach based on particle filter algorithm is quite computationally expensive and
therefore, this approach is not very useful in practice. Nevertheless, it uses an algorithm that is very well
known and often used in many practical problems and therefore it was our first approach to creating the
stochastic pedestrian motion model. Though it has a limited use in practice, the stochastic pedestrian mo-
tion model based on the particle filter algorithm has certain advantages over the deterministic LTA model.
In order to illustrate these advantages, we consider an artificial, but a very simple setting. We have only one
subject, with the known starting position and the desired position as well as the desired velocity. The scene
consists of only one obstacle that is symmetric and normal to the line connecting the pedestrian starting
and desired positions. It is clear that in terms of the energy function formation and minimization there are
two equally probable paths that the subject could take. Which path it would be depends on the pedestrian
decision which may be influenced by many different factors that are hard to model precisely. Nevertheless,
the deterministic motion model gives only one of these two possible paths, completely ignoring the other
one. The stochastic pedestrian motion model based on the particle filter algorithm, on the other hand, gives
probabilities for different pedestrian positions in the form of a particle cloud which will include both paths
that can be taken. Figure 4.1 shows the simulation results obtained for the described scenario by the deter-
ministic model as well as the stochastic model based on particle filter algorithm.
The stochastic model based on the particle filter algorithm was also tested on the real-world data avail-
able from the two video sequences that were used to learn parameters of the deterministic LTA model. Figure
4.2 shows a couple of frames from one of those two videos with plots of the simulation results obtained by
the stochastic model based on the particle filter algorithm. Similarly to the previous experiment, we can see
that this model gives two particle clouds representing the ways the subject in question can walk around an
incoming pedestrian. However, during those simulations on the real-world data we noticed another disad-
vantage of this algorithm. When propagating the particles that are divided into two or more clouds, a whole
cloud might be lost during the process of resampling in order to favor another cloud that is more likely. In
this simulation setting all the possible trajectories represented by the particle cloud that was lost are mean-
ingless. This is yet another significant disadvantage of this approach that makes it less useful in practice.
19

CHAPTER 4. EXPERIMENTS AND RESULTS
−10 −5 0 5 10
−4
−2
0
2
4
6
8
Starting position of the pedestrian
Desired destination point
Obstacle in the way
Pedestrian path given by the deterministic LTA model
Another possible path that is equaly probable
−10 −5 0 5 10
−4
−2
0
2
4
6
8
Starting position of the pedestrian
Desired destination point
Obstacle in the way
The paths that LTA combined with particle filter gives
Figure 4.1: Compares the deterministic and the stochastic model based on the particle filter algorithm. The
pedestrian starting position is shown by the black square and its desired destination by the green square,
the pedestrian desired velocity was set to 1.2. The obstacle is symmetric and is given by the red rhombs. In
the left figure, the blue line shows the path that the deterministic LTA gives in this case; on the same figure
with the magenta line we show another path that is equally probable. This path was generated using the one
given by the deterministic LTA and using the symmetry of the scene. In the right figure the cloud of particles
shows the possible paths that can be taken according to the stochastic motion model based on the particle
filter algorithm
In the approach based on the mixture of Gaussians, coefficient α is a free design parameter. We can
learn this parameter from the real-world data. For this experiment we used the available data sequence that
is different from the sequences used to train the deterministic LTA model. In deciding what the best value
for the parameter α should be, we look at the error of our stochastic model for different values of α and
chose the one which yields the smallest error. Since in our stochastic model we are dealing with the prob-
abilities of pedestrian position and velocity, we need to express the error in terms of probability. Since at
each time step the simulation gives the probability density function of pedestrian position and velocity, we
can calculate the error as:
error = −log(f(realpos) (4.1)
Where f() presents the probability density function of the pedestrian position that we get by our model
and realpos the real pedestrian position. We will perform a stochastic simulation using our model based on
the mixture of Gaussians for each pedestrian in the scene and calculate its error as the mean value of the
errors at each time step of the simulation. The overall simulation error is calculated as the mean error for all
the simulated subjects (we will call this error the mean simulation error). In order to see the advantage of
approximating the pedestrian position and velocity probability density function with the mixture rather than
20

Figure 4.2: Shows frames from a video sequence of the real-world walking scene that was used for estimating
pedestrian motion using the approach based on the particle ﬁlter algorithm. The results of the simulation are
presented in the form of a particle cloud. The algorithm yields two clouds that present the two possibilities
for the subject to go around an incoming pedestrian
with a single Gaussian function, we perform another experiment in which we approximate the position and
velocity probability density function with a single Gaussian function equal to the mixture element with the
highest weight. The mean simulation error for this experiment is calculated in the same way as in the case
of the simulation with the algorithm based on the mixture of Gaussians. Estimating the pedestrian position
and velocity probability density function with a single Gaussian function comes closer to the logic of the
deterministic model, but is still different as it is dealing with the position and velocity probabilities and not
their exact values. Figure 4.3 compares the mean simulation error obtained by our model and the one mode
approximation for the different values of parameter α. If we look once again at the equations that describe
our stochastic model based on the mixture of Gaussians, it is easy to realize that for high values of α we
are approaching the deterministic model. If we look at the equation 3.8 we see that the high values of α
give the covariance matrix that has all the elements close to zero, and therefore the elements in the mixture
of Gaussians will be close to the Dirac pulses and not proper Gaussian functions. On the other hand, for
values of α that are close to zero, the covariance matrix will have big elements and therefore the mixture of
Gaussians will be close to a uniform distribution. We see that in both cases the simulation error is very high.
In fact, the smallest error is obtained for α = 1 and therefore, we conclude that this is the optimal value to
be used by our model.
If we compare the results obtained by using our mixture model and the approximation with a single
Gaussian, we see that the mean simulation error for our model is only slightly smaller. The reason for this
is that our model actually reduces the error in cases where there is more than one possible path. Since the
21

0 1 2 3 4 5 6 7 8
1
2
3
4
5
6
7
8
9
α
meansimulationerror
algorithm based on mixture of Gaussians
aproximation with single Gaussian
Figure 4.3: Mean simulation errors obtained from the experiments on the real-world data when using dif-
ferent values for parameter α. The blue line shows the mean error value obtained by our algorithm based
on the mixture of Gaussians and the red line shows the result obtained by the one mode approximation
number of such situations is generally not very big in an ordinary walking scene, the mean simulation errors
are not very different. On the other hand, in the cases where there are more possible paths, our model gives
significantly smaller error. This becomes obvious if we look at the simulation variance. The simulation
variance is the variance of the mean errors for all the subjects in the data sequence. Figure 4.4 shows the
variances obtained by simulating with our model and the approximation with a single Gaussian function for
different values of parameter α. It is clear that our model results in a significantly smaller variance. This
means that using our pedestrian motion model based on the mixture of Gaussians yields smaller error risk
than using the model which approximates the pedestrian probability density function with a single Gaussian.
In order to further illustrate this fact, we calculate the histogram of mean subject errors in these two cases.
Figure 4.5 shows histograms of error values for all the subjects in the data set for the chosen optimal value
for α. The histogram for our model based on the mixture of Gaussians has a shorter tail representing big
errors than the model based on the single mode. This confirms that the risk of having big errors is lower for
our model.
In order to compare the stochastic pedestrian motion model based on the mixture of Gaussians with the
deterministic motion model, we will illustrate the results of the stochastic model on the real-world data. We
chose a situation in which the deterministic model makes a mistake and gives results that are far from the
ground-thought. In the figure 4.6 we show a couple of frames from the recorded sequence of the real scene
with plots of the prediction obtained using the stochastic model based on the mixture of Gaussians and the
deterministic model. The example that we chose here is a typical situation in which the deterministic LTA
model makes a significant mistake due to its limitations. In those cases our stochastic model based on the
mixture of Gaussians gives much better results as the real pedestrian position is usually covered by one of
the elements in the mixture of Gaussians.
22

0.5 1 1.5 2
0
5
10
15
20
25
30
α
simulationerrorvariance
Figure 4.4: Error variances obtained from the experiment on the real-world data sequence when using the
different values for parameter α. The blue line shows the error variance value obtained by our algorithm
based on the mixture of Gaussians and the red line shows the results obtained by the one mode approximation
In couple of similar simulations we have noticed that when the subject chooses one of the possible paths
and after some time deviates from this path it tends to change the path to correspond to the path that was the
other possible choice before. The reasons for the deviation are scene elements that are not taken as static
obstacles in the deterministic LTA model such as snow on the pathway. This behavior could suggest that
the subject in question somehow remembers the second optimal path and tries to stick to it once it decides
to change the path that was originally selected. However, this is just a hypothesis and we cannot claim that
this is true in general.
23

0 5 10 15
0
5
10
15
20
25
30
0 5 10 15
0
5
10
15
20
25
30
Figure 4.5: The left ﬁgure shows the histogram of mean subject errors for the simulation when using the
model based on the mixture of Gaussians. The ﬁgure on the right shows the histogram for the model based
on a single mode approximation. In both simulations α was set to 1
24

Figure 4.6: Illustrates the result obtained by using the stochastic pedestrian motion model based on the
mixture of Gaussians and compares this result with the result obtained by the deterministic model and the
ground truth. The image shows frames from a video sequence that was used for simulation. In each frame
the mixture of Gaussians that represents the probability density function of pedestrian position is illustrated
by the red markers at mean points of the elements in the mixture. The size of the markers corresponds to
the weights that the mixture elements have. The covariance matrices are illustrated by the blue ellipsoids
around the mean value points. Results that the deterministic LTA model gives are plotted with the green
square
25

26

Chapter 5
Conclusion
In this semester project we have proposed two different stochastic pedestrian motion models. Both models
were programmed in Matlab and tested on the real-world data. These results were used to learn the optimal
value of the free design parameter for the stochastic model based on the mixture of Gaussians and to com-
pare the performance of the stochastic models and the deterministic one.
The approach based on particle filter algorithm was a logical starting point in the development of our
stochastic model. Although this model has certain advantages over the deterministic LTA model, we have
seen that it is not very useful in practice. It is quite computationally expensive and, consequently, the overall
algorithm is slow. Furthermore, the fact that during simulation some particle clouds might disappear, thus
making all the paths represented by them meaningless, adds to the practical limitations of this approach.
Therefore, we explored the approach based on the mixture of Gaussians in order to keep all the advan-
tages of the previous algorithm, and cure its weaknesses. In order to make a fast algorithm we applied quite
crude approximation of the probability density function obtained from the energy function with the mixture
of Gaussians. This approximation works quite fine in our case. With the aim to make the algorithm as fast
as possible we used the off-the-shelf Matlab function for finding the local minima of the energy function.
Also, we tried to reduce the search area and the number of the points that need to be visited in the search.
As a result, we got an algorithm that is less computationally expensive and more useful in practice than the
one based on the particle filter algorithm.
Our goal was to propose a stochastic model in order to further extend the existing deterministic LTA
model. As shown in the previous chapter, the two stochastic models improve the prediction in the cases
where there is more than one path that could be taken by the pedestrian. Unlike the deterministic model,
which can only predict one of those paths, our stochastic models can give predictions for all the paths with
corresponding probabilities. Our experiments on the real-world data show that approximating the velocity
and position probability density function with the mixture of Gaussians rather than with a single Gaussian
function can significantly reduce the simulation error variance and hence make the prediction more reliable.
Furthermore, by creating the stochastic version of the LTA model, we take into account that the determin-
istic model could not model all the factors that can influence human motion in crowded scenes. However,
even the stochastic model can make significant errors in prediction. The stochastic models presented in this
work do not work well in cases when the pedestrians are wondering aimlessly or just standing still. This
limitation is inherited from the deterministic model, as its main idea is that the subjects are moving towards
a clear target position and with a clear target velocity. Another limitation of the model that has partially been
27

CHAPTER 5. CONCLUSION
remedied by its stochastic version is that the model takes into account only the solid physical obstacles and
not some other scene elements that could influence human motion. A good example for this is one of the two
recorded crowded scenes that was used to train the deterministic model and which we used to illustrate the
results of our stochastic models. Some of the frames from this scene are shown in figures 2.1 and 4.4 where
we see that part of the path along which the people are walking is covered with snow. What we could see in
the sequence is that, while walking, pedestrians often tried to avoid walking through the snow, which is the
factor that the deterministic model does not take into consideration. If the deviations of the path caused by
these factors are not too big, our stochastic model works fine and improves the prediction capabilities of the.
However, if those deviations are significant, even our stochastic model has a significant prediction error.
Even with those limitations our stochastic pedestrian motion model is a very powerful tool in simulating
pedestrian motion in moderately crowded scenes. It has all the advantages of the deterministic model, such
as that it operates in the real-world coordinates and can be used by any tracker. Moreover, it reduces the
prediction error in cases when there is more than one possible path that could be taken by the subject, and
therefore improves the prediction capabilities of the model. However, there are still ways to further improve
the LTA model. One of them is reconsidering static obstacle modeling, where it would be interesting to in-
vestigate whether taking the obstacle point in the direction of the pedestrian motion instead of simply taking
the closest point to represent the obstacle position would improve the algorithm. Furthermore, our algorithm
based on the mixture of Gaussians could be further enhanced by designing a way of merging elements in
the mixture that are very close to each other in terms of expected values as this would make the algorithm
faster.
28

Bibliography
[1] S. Pellegrini, A.Ess, K.Schindler and L. van Gool: You’ll Never Walk Alone: Modeling Social Behavior
for Multy-target Tracking. In ECCV, 2009.
[2] D. Helbing and P. Molnár: Social force model for pedestrian dynamics. Physical Review E, 51(5):4282-
4286, 1995.
[3] Oliver Cappé, Simon J. Godsill and Eric Moulines: An overview of existing methods and recent ad-
vances in sequential Monte Carlo. Proceedings of the IEEE ISSN 0018-9219, vol. 95, 2007
[4] Thomas Bengtsson, Peter Bickel and Bo Li: Curse-of-dimensionality revisited: Colapse of particle filter
in very large scale systems. IMS Colections, Probability and Statistics: Essays in Honor of David A.
Freedman Vol. 2 (2008) 316-334
[5] F. Daum, J. Huang: Curse of dimensionality and particle filters. IEEEAC paper 1018 December 2002
29

STOCHASTIC PEDESTRIAN MOTION MODEL

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Similar to STOCHASTIC PEDESTRIAN MOTION MODEL

Similar to STOCHASTIC PEDESTRIAN MOTION MODEL (20)

More from Marko Tanaskovic

More from Marko Tanaskovic (7)

STOCHASTIC PEDESTRIAN MOTION MODEL