Stochastic Hydrology Lecture 1: Introduction

Stochastic HydrologyStochastic Hydrology
byby
Dr.Dr. Amro ElfekiAmro Elfeki

ObjectivesObjectives
•• Introducing the fundamental aspects of stochastic modelling andIntroducing the fundamental aspects of stochastic modelling and
geogeo--statistics: Mean, Variance, Covariance, Correlation,statistics: Mean, Variance, Covariance, Correlation,
VariogramVariogram,, KrigingKriging, Scale effects., Scale effects.
•• How to perform stochastic modelling for porous media studies:How to perform stochastic modelling for porous media studies:
generation of heterogeneous media (generation of heterogeneous media (GaussianGaussian,, MarkovianMarkovian,,
Hybrid fields and point processes etc.).Hybrid fields and point processes etc.).
•• Error propagation analyses: how uncertainty in the inputError propagation analyses: how uncertainty in the input
parameters propagates into the results, stochastic techniquesparameters propagates into the results, stochastic techniques
(Analytical and Monte(Analytical and Monte--Carlo methods).Carlo methods).
•• UpUp--scaling: replaces the small scale stochastic input fields byscaling: replaces the small scale stochastic input fields by
effective parameters on the large scale.effective parameters on the large scale.
•• Reducing uncertainty in model predictions: ConditionalReducing uncertainty in model predictions: Conditional
simulation).simulation).
•• Performing some computer exercises on PCs.Performing some computer exercises on PCs.

Time TableTime Table
Day Time Topic Location Lecturer
21-04-2004 9:00-11:30 Introduction to probability theory and
statistics of single and multi-variates
4.96 Elfeki
28-04-2004 14:00-16:30 Representation of Stochastic Process
in Real and Spectral Domains and
Monte-Carlo Sampling.
4.96 Elfeki
05-05-2004 9:00-11:30 Stochastic Models for Site
Characterization (Theory).
4.96 Elfeki
12-05-2004 9:00-11:30 Stochastic Models for Site
Characterization (Computer
Exercises).
4.96 Elfeki
19-05-2004 9:00-11:30 Stochastic Differential Equations and
Methods of Solution (Theory and
Computer Exercises).
4.96 Elfeki
26-05-2004 9:00-11:30 Kriging and Conditional Simulations
(Theory and Computer Exercises).
4.96 Elfeki
09-06-2004 9:00-11:30 Oral Examination 4.96 Elfeki

Lecture (1)Lecture (1)
IntroductionIntroduction
to Probabilityto Probability
Theory and Statistics ofTheory and Statistics of
Single and MultiSingle and Multi--VariatesVariates

Time SeriesTime Series
0
20
40
60
80
100
120
140
160
1-1-1978
1-3-1978
1-5-1978
1-7-1978
1-9-1978
1-11-1978
1-1-1979
1-3-1979
1-5-1979
1-7-1979
1-9-1979
1-11-1979
Series1

Space SeriesSpace Series
Mount Simon Sand Stone Aquifer, USA

Field MeasurementsField Measurements

Outcrop (1) A Layered StructureOutcrop (1) A Layered Structure

Outcrop (2) TwoOutcrop (2) Two--Scale StructureScale Structure

A Trench (Fine Scale Heterogeneity)A Trench (Fine Scale Heterogeneity)

Site CharacterizationSite Characterization
•• Deterministic Approach.Deterministic Approach.
•• Stochastic Approach.Stochastic Approach.

Deterministic ApproachDeterministic Approach
From boreholes, geologists construct the geological cross-section utilizing
geological experience and technical background from practitioners.
The produced image is considered as a deterministic one.
The final user of that image may rely on it as a subjectively certain picture
of the subsurface.
0 200 400 600 800 1000 1200 1400 1600
-15
-10
-5
0

Stochastic ApproachStochastic Approach
•• The word stochastic has its origin in the Greek adjectiveThe word stochastic has its origin in the Greek adjective
στστooχαστικχαστικooςς which means skilful at aiming or guessing .which means skilful at aiming or guessing .
•• The stochastic approach is used to solve differentialThe stochastic approach is used to solve differential
equations with stochastic parameters.equations with stochastic parameters.
•• It is a tool to evaluate the effects of spatial variability ofIt is a tool to evaluate the effects of spatial variability of
thethe hydrogeologicalhydrogeological parameters on flow and transportparameters on flow and transport
characteristics in porous formations.characteristics in porous formations.

Definition of Stochastic ProcessDefinition of Stochastic Process
•• A stochastic process may be defined, Bartlett [1960]:A stochastic process may be defined, Bartlett [1960]:
" a physical process in the real world, that has some random or" a physical process in the real world, that has some random or
stochastic element involved in its structures"stochastic element involved in its structures"
If a process is operating through time or space: it is considereIf a process is operating through time or space: it is considered asd as
system comprising a particular set of states:system comprising a particular set of states:
•• In a classical deterministic model:In a classical deterministic model: the state of the system inthe state of the system in
time or space can be exactly predicted from knowledge of thetime or space can be exactly predicted from knowledge of the
functional relation specified by the governing differentialfunctional relation specified by the governing differential
equations of the system (deterministic regularity).equations of the system (deterministic regularity).
•• In a stochastic model:In a stochastic model: the state of the system at any time orthe state of the system at any time or
space is characterized by the underling fixed probabilities of tspace is characterized by the underling fixed probabilities of thehe
states in the system (statistical regularity).states in the system (statistical regularity).

Why do we need the Stochastic Approach?Why do we need the Stochastic Approach?
•• The erratic nature of theThe erratic nature of the hydrogeologicalhydrogeological
parameters observed at field data.parameters observed at field data.
•• The uncertainty due to the lack of informationThe uncertainty due to the lack of information
about the subsurface structure which isabout the subsurface structure which is
known only at sparse sampled locations.known only at sparse sampled locations.

Concept of Stochastic SimulationConcept of Stochastic Simulation
•• Generation of alternativeGeneration of alternative equiprobableequiprobable images of spatialimages of spatial
distributions of objects or nodal values.distributions of objects or nodal values.
•• Each alternative distribution is called aEach alternative distribution is called a stochasticstochastic
simulation.simulation.
0 5 10 15 20 25
-25
-20
-15
-10
-5
0
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Single Realization ln (K)
0 5 10 15 20 25
-25
-20
-15
-10
-5
0
0 5 10 15 20 25
-25
-20
-15
-10
-5
0

Stochastic Simulation (1)Stochastic Simulation (1)
GaussianGaussian Random FieldRandom Field
0 20 40 60 80 100 120 140 160 180 200
-40
-20
0
-3.3 -2.3 -1.3 -0.3 0.7 1.7 2.7
Y=Log (K)

ObjectObject--based modelbased model

CMCCMC--methodmethod
0 5 0 1 0 0 1 5 0 2 0 0 2 5 0
-1 0
-5
0
0 5 0 1 0 0 1 5 0 2 0 0 2 5 0
- 1 0
-5
0
1
2
3
4
5
0 5 0 1 0 0 1 5 0 2 0 0 2 5 0
-1 0
-5
0
0 5 0 1 0 0 1 5 0 2 0 0 2 5 0
- 1 0
-5
0
0 5 0 1 0 0 1 5 0 2 0 0 2 5 0
-1 0
-5
0
0 5 0 1 0 0 1 5 0 2 0 0 2 5 0
-1 0
-5
0
0 5 0 1 0 0 1 5 0 2 0 0 2 5 0
- 1 0
-5
0

Merging MethodsMerging Methods
0 100 200
-200
-100
0
0 100 200
-200
-100
0
0 100 200
-200
-100
0
0 100 200
-200
-100
0
0 100 200
-200
-100
0
0 100 200
-200
-100
0
0 100 200
-200
-100
0
0 100 200
-200
-100
0
0 100 200
-200
-100
0
0 100 200
-200
-100
0
0 100 200
-200
-100
0
0 100 200
-200
-100
0

0 100 200
-200
-100
0
0 100 200
-200
-100
0
0 100 200
-200
-100
0
0 100 200
-200
-100
0
0 100 200
-200
-100
0
0 100 200
-200
-100
0
0 100 200
-200
-100
0
0 100 200
-200
-100
0
0 100 200
-200
-100
0
0 100 200
-200
-100
0
0 100 200
-200
-100
0
0 100 200
-200
-100
0

Handling MultiHandling Multi--Scale DataScale Data
TwoTwo--step Procedurestep Procedure
Multi-Scale and Multi-Resolution Stochastic Modelling
of
Subsurface Heterogeneity
0 50 100 150 200 250
Horizontal Distance between Wells (m)
-50
0
Depth(m)
Well 1 Well 2
Micro-structure of the white layer
(from core analysis)
Micro-structure of the black layer

StepStep--1:Delineation of Large Scale1:Delineation of Large Scale
StructureStructure
0 50 100 150 200 250
-50
0
Depth(m)
Well 1 Well 2
Well 1 Well 2
0 50 100 150 200 250
1) Large scale geological structure
(drawn by geological expertise, interpolation methods etc.)
-50
0
Depth(m)

0 50 100 150 200 250
-50
0
0 50 100 150 200 250
-50
0
Depth(m)
Well 1 Well 2
Tree-indexed Markov Chain

TwoTwo--scalescale structuesstructues
0 100 200
-200
-100
0
0 100 200
-200
-100
0
0 100 200
-200
-100
0
0 100 200
-200
-100
0
0 100 200
-200
-100
0
0 100 200
-200
-100
0
0 100 200
-200
-100
0
0 100 200
-200
-100
0
0 100 200
-200
-100
0
Large Scale
Structure
Small Scale
Structure
Two Scales
Structure
Synthetic Data Simulation
Example (1)
Example (2)
Example (3)

TreeTree--indexed MCindexed MC--methodmethod
0 50 100 150 200 250
-250
-200
-150
-100
-50
0

Random walk MethodRandom walk Method
0
0.1
1
10
100
0 50 100 150 200 250
-10
-5
0
1
2
3
4
5
0 50 100 150 200 250
-10
-5
0
0 50 100 150 200 250
-10
-5
0
49 days
0 50 100 150 200 250
-10
-5
0
279 days
0 50 100 150 200 250
-10
-5
0
594 days

Historical BackgroundHistorical Background
•• Warren and Price [1961]: Stochastic model ofWarren and Price [1961]: Stochastic model of
macroscopic flow.macroscopic flow.
•• Warren and Price [1964]: Stochastic model ofWarren and Price [1964]: Stochastic model of
transport.transport.
•• Freeze [1975]: OneFreeze [1975]: One--dimensional groundwater flow.dimensional groundwater flow.
•• Freeze [1977]: OneFreeze [1977]: One--dimensional consolidationdimensional consolidation
problems.problems.
•• Revolution in stochastic subsurface heterogeneity.Revolution in stochastic subsurface heterogeneity.

Scales of Natural Variability (HierarchicalScales of Natural Variability (Hierarchical
Structure)Structure)

Types of Stochastic Models for SubsurfaceTypes of Stochastic Models for Subsurface
CharacterizationCharacterization
Discrete Models Continuous Models Hybrid Models
Stochastic Models

Definition of The Stochastic ProcessDefinition of The Stochastic Process
A stochastic process can be defined as:
“a collection of random variables”.
In a mathematical form:
the set {[x, Z(x,ζi)], x ∈ Rn }, i = 1,2,3...,m.
Z(x,ζ) is stochastic process, (random function),
x is the coordinates of a point in n-dimensional space,
ζ is a state variable (the model parameter),
Z(x,ζi) represents one single realization of the stochastic process, i= 1,2,...,m
(i: realization no. of the stochastic process Z),
Z(x0,ζ) = random variable, i.e., the ensemble of the realizations of the
stochastic process Z at x0, and
Z(x0,ζi)= single value of Z at x0.
For simplification the variable ζ is generally omitted and the notation of this
stochastic process is Z(x).

Realizations of A Stochastic ProcessRealizations of A Stochastic Process

UniUni--dimensional Stochastic Processdimensional Stochastic Process
A stochastic process in which the variation of a property of a physical
phenomenon is represented in one coordinate dimension is called uni-
dimensional stochastic process. The coordinate dimension can be time as in
time series, or space as in space series.
0 10 20 30 40
Space or Time Scale
-2.0
0.0
2.0
Parameter

Spatial Random FieldsSpatial Random Fields
Random fields are multi-dimensional stochastic processes.
0 20 40 60 80 100 120 140 160 180 200
-40
-20
0
-3.3 -2.3 -1.3 -0.3 0.7 1.7 2.7
Y=Log (K)

Comparison in Terminology betweenComparison in Terminology between
Statistical and Stochastic TheoriesStatistical and Stochastic Theories
Statistical Theory Stochastic Theory
Sample Realization
Population Ensemble

Probabilistic Description of StochasticProbabilistic Description of Stochastic
ProcessesProcesses
•• Single Random Variable (Single Random Variable (UnivariateUnivariate).).
•• Multi Random Variables (Multivariate):Multi Random Variables (Multivariate):
Random VectorRandom Vector

Probability Distribution FunctionProbability Distribution Function
(Cumulative Distribution Function)(Cumulative Distribution Function)
Single random variable
}{Pr)( zZ=zP ≤
where
Pr{A} is a probability of occurrence of an event A, and
P(A) is a cumulative distribution function of the event A,
z is a value in a deterministic sense.
The distribution function is monotonically nondecreasing.
1)(
0)(
=+P
=P
∞
−∞

Probability Density Function (PDF)Probability Density Function (PDF)
The density function, p(z), of random variable Z is defined by,
dz
zdP
=
z
z+zZ<z
=zp
z
)(}Pr{
lim)(
0 ∆
∆≤
→∆
Inversely, the distribution function can be expressed in terms of
the density function as follows
∫∞
z
-
dzzp=zP ')'()(
p(z) is not a probability, but must be multiplied by a certain region ∆z to obtain a
probability (i.e.: p(z).∆z).
P(z) is dimensionless, but p(z) is not. It has dimension of [z-1].

Graphical Representation ofGraphical Representation of pdfpdf andand cdfcdf

Uniform DistributionUniform Distribution
f(x) or F(x)
x
a b
1.0
0
F(x)
f(x)
(a-b)
1
1
( )f x
b a
=
−
[ ]
1 1
( )
x
a
F x dt x a
b a b a
= = −
− −∫
• Two parameter distribution
• Useful when only range is known

GaussianGaussian (Normal) Distribution(Normal) Distribution
PDF
CDF
2
22
1 ( )
( ; , ) exp
22
x
f x
µ
σ µ
σπσ
⎧ ⎫−
= −⎨ ⎬
⎩ ⎭
2
22
1 ( )
( ; , ) exp
22
x
t
F x dt
µ
µ σ
σπσ −∞
⎧ ⎫−
= −⎨ ⎬
⎩ ⎭
∫
0
0.2
0.4
0.6
0.8
1
-3 -2 -1 0 1 2 3
x
F(x)
f(x)
F(x) or f(x)
µ
2σ
• σ2 and µ are variance and mean
• X ~ N(mean, variance) = N(µ , σ2)

NaturalNatural--LogLog--normal Distributionnormal Distribution
2
ln( )
ln( )
2
ln( )
ln( )
ln( )
2
ln( )
ln( )
2 ln( )
0 ln( ) ln( )
1
( ) ......., 0
2
ln( )1 1
( ) 1
22 2
x
x
x
x
x
x
t
x
x
x x
f x e x
x
x
F x e dt erf
t
µ
σ
µ
σ
πσ
µ
πσ σ
⎡ ⎤−
⎢ ⎥−
⎢ ⎥⎣ ⎦
⎡ ⎤−
⎢ ⎥−
⎢ ⎥⎣ ⎦
= >
⎡ ⎤⎛ ⎞−
⎢ ⎥= = + ⎜ ⎟
⎜ ⎟⎢ ⎥⎝ ⎠⎣ ⎦
∫

Exponential DistributionExponential Distribution
0
1
0
F(x)orf(x)
x
F(x)
f(x)
exp( ), 0
( )
0 otherwise
x x
f x
λ −λ ≥⎧
= ⎨
⎩
[ ]
0
( ) exp( ) 1 exp( )
x
F x t dt x= λ −λ = − −λ∫
•• One parameter modelOne parameter model
•• Useful for fracture spacingUseful for fracture spacing

Derivation of aDerivation of a pdfpdf from a Time Seriesfrom a Time Series
Z(t)
t
z
z+dz
dt1 dt2 dt3
T
⎭
⎬
⎫
⎩
⎨
⎧
∆
∆⎭
⎬
⎫
⎩
⎨
⎧
∆
∆≤
⎭
⎬
⎫
⎩
⎨
⎧
∆∆≤
∑
∑
=∞→
→∆→∆
=
∞→
3
1
3
1
.
1
lim
}Pr{
lim)(
1
lim}Pr{
i
i
T
0z0z
i
i
T
t
Tz
=
z
z+zZ<z
=zp
t
T
=z+zZ<z
p(z).∆z = probability that Z(t) lies between z and z+∆z.

Joint Probability Density FunctionJoint Probability Density Function
,
,
i
j
X
Y
0
Z
Z
sij
1
p
,
,
,
2 3
,
01
.
.
0
1 1 1 1
1 2 1
Pr { ,..., } ( )
( ) lim
... ...z
zn
n
n n n n
n n
Pz Z z z z Z z zp
z z z z z
∆ →
∆ →
< ≤ + ∆ < ≤ + ∆ ∂
= =
∆ ∆ ∆ ∂ ∂
z
z
Here, z without index is a vector.
Inversely, the joint distribution function can be expressed in terms of the joint
density as follows,
1
1
- - -
( ) ( ') ' ... ( ') '... '
nzz
nP p d p dz dz
∞ ∞ ∞
= =∫ ∫ ∫
z
z z z z

BivariateBivariate NormalNormal pdfpdf
( ) ⎥
⎥
⎦
⎤
⎢
⎢
⎣
⎡
⎭
⎬
⎫
⎩
⎨
⎧ −
+
−−
−
−
−
2
2
2
22
21
2211
2
1
2
11
22
21
21
)())((2)(
12
1
exp
12
1
),(
σ
µ
σσ
µµρ
σ
µ
ρρσσ
ZZZZ
-
-
π
=ZZf

Joint Probability Distribution FunctionJoint Probability Distribution Function
Consider Z as a random vector defined in a vector form as {Z1,Z2,…,Zn}T,
where, Z1, Z2,… and Zn are single random variables. z is described by the joint
distribution function of Z as,
)(Pr).( zZ,...,zZ,zZ=z.,,.z,zP nn2211n21 ≤≤≤
1)(
0)(
=...,+,+,+,+P
=...,-,-,-,-P
∞∞∞∞
∞∞∞∞

Graphical Representation ofGraphical Representation of jpdfjpdf andand jcdfjcdf

Derivation ofDerivation of jpdfjpdf of twoof two--seriesseries
Z(t)
t
z
z+dz
dt1 dt2 dt3
T
Y(t)
t
y
y+dy
dt1 dt2 dt3
T
⎭
⎬
⎫
⎩
⎨
⎧
∆
∆∆⎭
⎬
⎫
⎩
⎨
⎧
∆∆
∆≤∆≤
⎭
⎬
⎫
⎩
⎨
⎧
∆∆≤∆≤
∑∑
∑∑
==
∞→
∞→
→∆
→∆
→∆
→∆
==∞→
∞→
yz
yz
n
i
i
n
j
T
T
0y
0z
0y
0z
n
i
i
n
jT
T
t
TTyz
=
yz
y+yY<yz+zZ<z
=yzp
t
TT
=y+yY<yz+zZ<z
1121
1121
.
1
lim
},Pr{
lim),(
1
lim},Pr{
2
1
2
1

Marginal Probability Density FunctionMarginal Probability Density Function
The marginal probability density function is defined as follows,
1
1 2 -1 1( ) ... ( ) ... ...
n
i i i np p dz dz dz dz dzz
−
∞ ∞
+
−∞ −∞
= ∫ ∫ z
∫
∫
∞
∞
∞
∞
-
-
dzzzp=zp
dzzzp=zp
1212
2211
),()(
),()(
In case of bivariate pdf:

Marginal Probability Distribution FunctionMarginal Probability Distribution Function
The marginal probability distribution function of a component Z1 of the random
vector Z is obtained from the joint density function by the integration,
1
1 1 1 2
2 1
( ) Pr( ,- ,...,- )
... ( ) ... '
n
z
n
P z Z z Z Z
p dz dz dz
∞ ∞
−∞ −∞ −∞
= < ∞ < < ∞ ∞ < < ∞
⎡ ⎤
= ⎢ ⎥
⎣ ⎦
∫ ∫ ∫ z
The term between brackets is the marginal probability density function of the
component Z1, and
P(z1) is called the marginal distribution function of the component Z1 of the
random vector Z.

Conditional Prob. Distribution FunctionConditional Prob. Distribution Function
The conditional distribution function of one component Zn of a random vector Z
given that the random components at n-1, n-2,…,1 have specified values is
defined by,
}∆∆Pr{
).(
11111111
121
z+zZ<z,...,z+zZ<z|zZ
=z..,,z,z|zP
n-n-n-n-nn
n-n-n
≤≤≤
where, the definition Prob{A⎮B} is the conditional probability of event A given
that, event B has occurred and is defined by,
}Pr{
}{Pr
}{Pr
B
BA
=B|A
∩
where, A∩B is the conjunctive event of A and B.

Conditional Probability Density FunctionConditional Probability Density Function
The conditional density function of component Zn of the random vector Z given
that the random components at n-1, n-2,…,1 have specified values is defined
by
-1 1
-1 1
( | ,..., )
( | ,..., ) n n
n n
n
P z z zp z z z
z
∂
=
∂
the function p(zn ⎮ zn-1,…,z1) can be expressed in a more convenient form as
follows,
-1 -2 1
1 2 -1
( )
( | , ,..., )
( , ,..., )
n n n
n
p
p z z z z
p z z z
=
z
where, p(z), is the joint density function of all the components of the vector Z. It
can also be written,
1 2 3( ) ( , , ,..., )np p z z z z=z

Conditional Probability DistributionConditional Probability Distribution
Function (Cont.)Function (Cont.)
1 2 3
-1 -2 1
1 2 -1
( , , ,..., )
( | , ,... )
( , ,..., )
n
n n n
n
p z z z zp z z z z
p z z z
=
If Z1, Z2,..., and Zn are independent random variables then the joint density
function is the multiplication of the marginal density function of the individual
random components. This can be expressed as follows,
1 2 3 1 2( , , ,..., ) ( ). ( )... ( )n np p p pz z z z z z z=
So, in conclusion, for independent random variables the following holds,
-1 -2 1( | , ,... ) ( )n n n np pz z z z z=

Statistical Properties of StochasticStatistical Properties of Stochastic
ProcessesProcesses
•• Spatial or TemporalSpatial or Temporal
Properties.Properties.
MeanMean
VarianceVariance
CovarianceCovariance
Higher order momentsHigher order moments
•• Ensemble Properties.Ensemble Properties.
MeanMean
VarianceVariance
CovarianceCovariance
Higher order momentsHigher order moments

Spatial Average (Mean)Spatial Average (Mean)
( )
1
( )
| |
i i
v x
dZ Z
v ′
= ∫ x x
where, v(x′) is the specified length, area or volume (for one, two or three
dimensional space respectively) centred at x′ of measure ⎮v⎮ and index i is the
i-th realization.
1
1
( )
n
i i j
j
Z Z
n =
= ∑ x
where, n is the number of discrete points discretizing the volume v, index j is
the j-th point on volume v.
Z
x
v
Z
x
Z1
Z2 Zi Zn

Spatial Mean Square ValueSpatial Mean Square Value
22
( )
1
( )
| |
ii
v x
dZZ
v ′
= ∫ x x
i-th realization.
2 2
1
1
( )
n
i i j
j
Z Z
n =
= ∑ x
Z
x
v
Z
x
Z1
Z2 Zi Zn

Spatial VarianceSpatial Variance
i-th realization.
2
2
2
( )
[ ] ( ) -
1
( ) -
| |
i
i i iZ
i i
v x
Var xZ Z ZS
x dxZ Z
v ′
⎡ ⎤= = ⎣ ⎦
⎡ ⎤= ⎣ ⎦∫
2
2
1
1
( ) -
-1i
n
i ijZ
j
xZ ZS
n =
⎡ ⎤= ⎣ ⎦∑
222
- ( )i i iZ
ZZS =
Z
x
v
Z
x
Z1-Z
Z2-Z Zi-Z
Zn-Z

SpatialSpatial AutoCovarianceAutoCovariance
( )
1
( ( ), ( )) ( )- ( ) ( )- ( )
| |
i i i i i i
v x
Cov dZ Z Z Z Z Z
v ′
⎡ ⎤ ⎡ ⎤= ⎣ ⎦ ⎣ ⎦∫x +s x x +s x +s x x x
( )
1
1
( ( ), ( )) ( ) - ( ) ( ) - ( )
( )
n
i i i j i j i j i j
j
Cov Z Z Z Z Z Z
n =
⎡ ⎤ ⎡ ⎤= + +⎣ ⎦ ⎣ ⎦∑
s
x +s x s sx x x x
s
where, n(s) is the number of points with lag s.

SpatialSpatial CrossCovarianceCrossCovariance
The covariance is a measure of the mutual variability of a pair of realizations;
or in other words, it is the joint variation of two variables about their common
mean.
( )
1
( ( ), ( )) ( )- ( ) ( ) - ( )
| |
i i i i i i
v x
Cov dZ Y Z Z Y Y
v ′
⎡ ⎤ ⎡ ⎤= ⎣ ⎦ ⎣ ⎦∫x +s x x +s x +s x x x
( )
1
1
( ( ), ( )) ( ) - ( ) ( ) - ( )
( )
n
i i i j i j i j i j
j
Cov Z Y Z Z Y Y
n =
⎡ ⎤ ⎡ ⎤= + +⎣ ⎦ ⎣ ⎦∑
s
x +s x s sx x x x
s
where, n(s) is the number of points with lag s.

Ensemble Statistical PropertiesEnsemble Statistical Properties
(Mathematical Expectation)(Mathematical Expectation)
The average of statistical properties over
all possible realizations of the process
at a given point on the process axis.

Ensemble Average (Mean)Ensemble Average (Mean)
{ ( )} ( ) ( )E Z z p z dz
+∞
−∞
= ∫o ox x
xo is the coordinate of a point on the space axis,
p(z) is pdf of the process Z(x) at location xo, and
E{.} is the expected value operator.
1
1
{ ( )} ( )
m
i
i
E Z Z
m =
≈ ∑o ox x
m is the number of realizations.

Ensemble Mean Square ValueEnsemble Mean Square Value
2 2
{ ( )} ( ) ( )E p z dzZ z
+∞
−∞
= ∫o ox x
2 2
1
1
{ ( )} ( )
m
i
i
E Z Z
m =
= ∑o ox x

Ensemble VarianceEnsemble Variance
[ ]{ } ( )
[ ]
( )
2 2
2
( )
2
2
( )
1
222
( )
( ) - { ( )} ( )( ) - { ( )}
1
( ) - { ( )}
{ ( )}- { ( )}
Z
m
iZ
i
Z
E Z E Z p z dzZ E Z
E ZZ
m
E E ZZ
σ
σ
σ
∞
−∞
=
= =
≈
=
∫
∑
o
o
o
o oo ox
o ox
o ox
x xx x
x x
x x

EnsembleEnsemble AutoCovarianceAutoCovariance
( ( ), ( ))
( ( ), ( )) ( ) ( )
Cov Z Z
p z z dz dz
∞ ∞
−∞ −∞
+ =
+ +∫ ∫
x s x
x s x x s x
p(z(x+s),y(x)) is the joint probability
density function of the process Z(x) at
locations x+s and x.
1
1
( ( ), ( )) ( ) - { ( )} . ( ) - { ( )}
m
j j j j
j
Cov Z Z E EZ Z Z Z
m =
+ ≈ + +⎡ ⎤ ⎡ ⎤⎣ ⎦ ⎣ ⎦∑x s x x s x s x x

EnsembleEnsemble CrossCovarianceCrossCovariance
( ( ), ( ))
( ( ), ( )) ( ) ( )
Cov Z Y
p z y dz dy
∞ ∞
−∞ −∞
+ =
+ +∫ ∫
x s x
x s x x s x
p(z(x+s),y(x)) is the joint pdf of the
process Z(x) and Y(x) at locations
x+s and x.
1
1
( ( ), ( )) ( ) - { ( )} . ( ) - { ( )}
m
j j jj
j
Cov Z Y E Z EZ Y Y
m =
⎡ ⎤+ ≈ ⎡ ⎤⎣ ⎦⎣ ⎦∑x s x x +s x +s x x

Some TerminologySome Terminology
•• StationarityStationarity (Statistical Homogeneity).(Statistical Homogeneity).
•• NonNon--stationaritystationarity..
•• Intrinsic Hypothesis.Intrinsic Hypothesis.
•• ErgodicityErgodicity..

StationarityStationarity
,
,
i
j
X
Y
0
Z
Z
sij
1
p
,
,
,
2 3
,
The stochastic process is said to be second-order stationary (weak sense) if:
1) The mean value is constant at all points in the field, i.e., the mean
does not depend on the position.
{ ( )} ZE Z µ=x
2) The covariance depends only on the difference between the
position vectors of two points (xi-xj)= sij the separation vector, and does
not depend on the position vectors xi and xj themselves.
[ ]{ }( ( ), ( )) ( )- { ( )} ( ) - { ( )} ( )i j i i j jCov Z Z E Z E Z Z E Z Cov= =⎡ ⎤⎣ ⎦ sx x x x x x
[ ] 2
( ) (0) ZVar Z Cov σ= =x

NonNon--StationarityStationarity (time or space series)(time or space series)
A stochastic process is called non-stationary, if
the moments of the process are variant in space, i.e., from one position to
another.

NonNon--stationaritystationarity (random fields)(random fields)
0 200 400 600 800 1000 1200 1400 1600 1800 2000
-400
-200
0
-10 -8 -6 -4 -2 0 2 4 -5.0 -3.0 -1.0 1.0 3.0
0 200 400 600 800 1000 1200 1400 1600 1800 2000
-400
-200
0
0.0 0.8 1.5 2.3 3.0
0 200 400 600 800 1000 1200 1400 1600 1800 2000
-400
-200
0
-8 -6 -4 -2 0 2 4
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Horizontal Distance (m)
-400
-200
0
Depth(m)
1 2 3 4
Log (Hydraulic Conductivity m/day) Log (Hydraulic Conductivity m/day)
Log (Hydraulic Conductivity m/day)
Log (Hydraulic Conductivity m/day)
(a) Non-Stationarity
in The Mean.
(b) Non-Stationarity
in The Variance.
(c) Non-Stationarity
in Correlation Lengths.
(d) Global Non-
Stationarity.
Geological Structure.
0 200 400 600 800 1000 1200 1400 1600 1800 2000
-400
-200
0

Intrinsic HypothesisIntrinsic Hypothesis
The intrinsic hypothesis assumes that even if the variance of Z(x) is not finite,
the variance of the first-order increments of Z(x) is finite and these increments
are themselves second-order stationary.
This hypothesis postulates that:
(1) the mean is the same everywhere in the field; and
(2) for all distances, s, the variance of the increments, {Z(x+s)-Z(x)} is a
unique function of s so independent of x.
A stochastic process that satisfies the stationarity of order two also satisfies the
intrinsic hypothesis, but the converse is not true.
[ ]{ }2
{ ( )- ( )} 0
( )- ( ) 2 ( )
E Z Z
E Z Z γ
+ =
+ =
x s x
x s x s
γ(s) is called the semi-variogram,

Intrinsic Hypothesis (cont.)Intrinsic Hypothesis (cont.)
From practical point of view:
1. The intrinsic hypothesis is appealing, because it allows the determination of
the statistical structure, without demanding the prior estimation of the mean.
2. For a stationary random process, where both a covariance and a sime-
variogram are exist, it is easy to show the relationship between them as,
0 5 10 15 20 250 5 10 15 20 25
)()0()( ss - Cov= Covγ

Comparison between Intrinsic HypothesisComparison between Intrinsic Hypothesis
and Secondand Second--orderorder StationarityStationarity
Intrinsic Hypothesis Second-order
Stationarity
Less strict than 2nd
order stationarity
More strict than
Intrinsic hypothesis
Variogram Correlogram
If the phenomenon does not have a finite
variance, the variogram will never have a
horizontal asymptotic value.
{ ( )- ( )} 0E Z Z =x +s x { ( )} ZE Z µ=x
[ ]{ }21
( ) ( )- ( )
2
E Z Zγ =s x +s x [ ][ ]{ }( ) ( ) . ( )Z ZCov E Z Zµ µ= − −s x +s x
finitebemustσ Z
2

ErgodicityErgodicity
Ergodicity is a statistical property which implies that:
(spatial statistics) are equivalent to (ensemble statistics).
{ ( )}i E ZZ ≈ ox
2 2
( )i ZZS σ≈ ox
This equivalence is achieved when the size of the space domain is sufficiently
large or tends to infinity.
It is theoretical defined, but practically impossible.

Stochastic Hydrology Lecture 1: Introduction

More Related Content

What's hot

Similar to Stochastic Hydrology Lecture 1: Introduction

More from Amro Elfeki

Recently uploaded

Stochastic Hydrology Lecture 1: Introduction