2. Outline
• Definition of Random Variable (RV)
• Conditions on Random Variables
• Types of RV
• Cumulative Probability Distribution Function (CDF)
• Probability Density Function (PDF)
• Gaussian Random Variable
• Other random variables:
– Discrete: Binomial, Poisson.
– Continuous: Uniform, Exponential, Rayleigh …. (more example
in Appendix F)
• Conditional density & distribution
2
3. Chapter 2: The Random Variable
The outcome of a random experiment may not be a number,
for example tossing a coin or selecting a color ball from a box.
However we are usually interested not in the outcome itself, but
rather in some measurement or numerical attribute of the
outcome.
Examples
Number of heads and not in the specific order in which
heads and tails occur.
Weight of the student.
In each of these examples, a numerical value is assigned to the
outcome.
3
4. Definition of RV
4
Example: Consider the experiment of tossing a coin 3 times and
observing the number of “Heads” (which is a numeric value).
Example: 𝐻 ← 0, 𝑇 ← 1
A real random variable X
is a function that assigns
a real number X(s) to
each outcome s in the
sample space.
5. Additional Random Variable Example
Let X be a random variable that maps the “head”
outcome to the positive number which correspond
to the dots on the die, and “tail” outcome map to
the negative number that are equal in magnitude
to twice the number which appear on the die.
Example : The experiment of rolling a die and tossing a coin
5
Example: 𝑠 is the outcome of the wheel of chance.
𝑋 𝑠 = 𝑠2
• Maps to the numbers from 0 to 144
• Not necessary that the sample space maps
uniquely
𝑋 𝑠 = 𝑠 or |𝑠2|
6. Conditions for a Function to be a Random Variable
1. RV can not be multivalued.
The random variable function
can map more than one point in
S into same point on the real
line.
2. 𝑷 𝑿 ≤ 𝒙 = sum of the
probability of all elementary
event s that satisfies the
condition.
3. 𝑷 𝑿 = −∞ = 𝟎, 𝑷 𝑿
RVs are functions that satisfies the followings:
6
7. Example: Tossing a coin 3 times and observing the number of heads
{ X 1 } { HTT, THT, TTH, TTT }
Equivalent
P{ X 1 } P(HTT) P(THT) P(T+ TH) P(T+ + TT)
Equality
7
8. Types of Random Variables
Discrete
Flipping a coin 3 times and counting the number of heads.
Selecting a number from the positive integers.
Number of cars arriving at gas station A.
Continuous
Selecting a number between { 0 and 6 }
{ 0 ≤ X ≤ 6 }
8
Mixed RV
Wheel of chance 0 < 𝑠 ≤ 6 → −1 , 6 < 𝑠 ≤ 12 → 𝑠
The type is based on the event not the Sample Space. Example:
Wheel of chance 0 < 𝑠 ≤ 6 → −1 , 6 < 𝑠 ≤ 12 → +1
In this example S is continuous and the event is discrete
9. Distribution Function
If you have a random variable X which is numeric by mapping a
random experiment outcomes to the real line
Example: Flipping a coin 3 times and counting the
number of heads
We can describe the distribution of the R.V X using the probability
9
10. We will define two more distributions of the random variable
which will help us finally to calculate probability.
Distribution Function
We define the cumulative probability distribution function (CDF)
( )F x P X x
X
where ,
In our flipping the coin 3 times and counting the number of heads
(2) 2F P X
X
(2) 2 { 0} { 1} { 2}F P X P X P X P X
X
1 3 3 7
8 8 8 8
10
11. 1
: Let {0,1,2,3} with ( 0) ( 3)
8
3
( 1) ( 2)
Exa
8
mple X P X P X
P X P X
1
(0) ( 0)
8
XF P X
1 3 1
(1) ( 1) ( 0) ( 1)
8 8 2
XF P X P X P X
1 3 3 7
(2) ( 2) ( 0) ( 1) ( 2)
8 8 8 8
XF P X P X P X P X
1 3 3 1
(3) ( 3) ( 0) ( 1) ( 2) ( 3) 1
8 8 8 8
XF P X P X P X P X P X
11
12. Distribution Function Properties
1. ( ) 0XF
2. ( ) 1XF
3. 0 ( ) 1XF x
1 2 1 24. ( ) ( )X XF x F x if x x
1 2 2 15. ( ) ( )X XP x X x F x F x
6. ( ) ( )X XF x F x
12
• From property 4: 𝐹𝑋(𝑥) is a non decreasing function
• Properties 1,2,4, & 6 can be used as a test for the
validity of distribution functions.
A right-continuous function
13. 1
: Let {0,1,2,3} with ( 0) ( 3)
8
3
( 1) ( 2)
Exa
8
mple X P X P X
P X P X
1
(0) ( 0)
8
XF P X
1 3 1
(1) ( 1) ( 0) ( 1)
8 8 2
XF P X P X P X
1 3 3 7
(2) ( 2) ( 0) ( 1) ( 2)
8 8 8 8
XF P X P X P X P X
1 3 3 1
(3) ( 3) ( 0) ( 1) ( 2) ( 3) 1
8 8 8 8
XF P X P X P X P X P X
13
14. Consider the experiment of tossing the coin 3 times and
observing the number of heads
The probabilities and the distribution function are shown below
The stair type distribution function
can be written as
( ) ( 0) ( 1) ( 2)( ) ( 1) ( 2)
() 3)( 3
step function step function step function
step fun on
X
cti
u x u x uF x P X P X P X x
u xP X
14
𝑋 discrete => 𝐹𝑋(𝑥) stair steps,
in general 𝑭 𝑿 𝒙 = 𝒊=𝟏
𝑵
𝑷 𝑿 = 𝒙𝒊 𝒖(𝒙 − 𝒙𝒊)
were 𝑁 can be infinite. We may simply 𝑃 𝑋 = 𝑥𝑖 = 𝑃(𝑥𝑖)
15. Density Function
Probability density function (pdf) 𝑓 𝑋(𝑥):the derivative of the
distribution function 𝐹 𝑋(𝑥) as the
• In our discrete R.V since
1
( ) ( ) ( )
N
X i i
i
F x P X x u x x
( )
( ) X
X
dF x
f x
dx
1
( ) ( ) ( )
N
X i i
i
f x P x x x
1
( ) ( ) ( )
N
X i i
i
d
f x P X x u x x
dx
1
( ) ( )
N
i i
i
d
P X x u x x
dx
1
( ) ( )
N
i i
i
P X x x x
15
17. Properties of Density Functions
17
1. 𝑓𝑋 𝑥 ≥ 0, for all 𝑥
(Density is non-negative derivative of non-decreasing function)
2. −∞
+∞
𝑓𝑋 𝑥 𝑑𝑥 = 1
3. 𝐹𝑋 𝑥 = −∞
𝑥
𝑓𝑋 𝑧 𝑑𝑧
From (3), 𝐹𝑋 ∞ = 1
4. 𝑃 𝑥1 < 𝑋 ≤ 𝑥2 = 𝑥1
𝑥2
𝑓𝑋 𝑥 𝑑𝑥
𝑃 𝑥1 < 𝑋 ≤ 𝑥2 = 𝐹 𝑥2 − 𝐹 𝑥1 = −∞
𝑥2
𝑓𝑋 𝑥 𝑑𝑥 − −∞
𝑥1
𝑓𝑋 𝑥 𝑑𝑥
• Properties (1) and (2) are used to prove if a certain function can
be a valid density function.
18. Additional Examples: Example 1
• A random current is described by the sample space 𝑆 = −4 ≤ 𝑖 ≤ 12}. A
random variable is defined to by
𝑋 𝑖 =
−2 𝑖 ≤ −2
𝑖 −2 < 𝑖 ≤ 1
1 1 < 𝑖 ≤ 4
6 4 < 𝑖
a. Show the mapping of the random variable.
b. What type of random variable is this?
18
19. Example 2: PDF & CDF
The sample space of an experiment is 𝑆 = 0,1,2.5,6}. The random
variable is defined to be 𝑋 = 2𝑠. Sketch the density and distribution
functions of the random variable 𝑋.
19
20. Example 3: Properties of pdf
Determine the real constant 𝑎, for arbitrary real constants 𝑚 and 𝑏 > 0, such
that
𝑓𝑋 𝑥 = 𝑎𝑒−
𝑥−𝑚
𝑏
is a valid density function (called Laplace density)
• The required tests are :
1. 𝑓𝑋 𝑥 ≥ 0, for all 𝑥
(Density is non-negative derivative of non-decreasing function)
2. −∞
+∞
𝑓𝑋 𝑥 𝑑𝑥 = 1
• From a sketch, 𝑓𝑋 𝑥 ≥ 0 𝑖𝑓 𝑎 > 0, 𝑎𝑛𝑑 𝑏 > 0 is necessary for non-infinite
area under 𝑓𝑋 𝑥 .
• For this assumed true
−∞
+∞
𝑓𝑋 𝑥 𝑑𝑥 = 2 𝑚
∞
𝑎𝑒−
𝑥−𝑚
𝑏 𝑑𝑥 = 2𝑎𝑏 0
+∞
𝑒−𝑧 𝑑𝑧 = 2𝑎𝑏 −𝑒−𝑧 |0
+∞
= 2𝑎𝑏
• We did change of variable z =
𝑥−𝑚
𝑏
• Thus, we require 𝒃 > 𝟎, 𝒂 = 𝟏/𝟐𝒃 20
21. The Gaussian (Normal) Random Variable
2
0X Xwhere and a
A random variable X is called Gaussian if its density function
has the form
22
2
2
( )1
( )
2
X xa
x
x
Xf ex
are real constants (we will see their
significance when we discuss the
mean and variance later).
The “spread” about the point
x = ax is related to σx
21
What is the value of the pdf at
𝑥 = 𝑎 𝑥, 𝑥 = 𝑎 𝑥 − 𝜎 𝑥, 𝑥 = 𝑎 𝑥 + 𝜎 𝑥?
22. The Gaussian density is the most important of all densities.
It accurately describes many practical and significant real-world quantities
such as noise, voltage, current, electrons, and quantities which are results
from many small independent random effects
The distribution function is found from
𝐹𝑋 𝑥 = −∞
𝑥
𝑓𝑥 𝜁 𝑑𝜁
2 2
( ) 2
2
1
( )
2
X xa
x
x
XF x e d
The integral has no known closed-form solution and must be evaluated by
numerical or approximation method (tables “Appendix B”, Matlab)
What is the value of the CDF at 𝑥 = 𝑎 𝑥?
22
23. However to evaluate numerically for a given 𝑥
2 2
( ) 2
2
1
( )
2
X xa
x
x
XF x e d
We need 𝜎 𝑥
2 and 𝑎 𝑥
Example: Let 𝜎 𝑥
2 = 3 and 𝑎 𝑥 = 5, then
𝐹𝑋 𝑥 =
1
2𝜋3 −∞
𝑥
𝑒
−
𝜁−5 2
2 3 𝑑𝜁
We then can construct the Table for various values of 𝑥.
220
( 5 2(3))1
( 20)
2 3
XF e d
26
( 5 2() 3)1
(6)
2 3
XF e d
Evaluate
Numerically
Evaluate
Numerically
23
24. Finally we will get a Table for various values of 𝑥.
However there is a problem!
The Table will only work for Gaussian distribution with
𝜎 𝑥
2 = 3 and 𝑎 𝑥 = 5 .
We know that not all Gaussian distributions have 𝜎 𝑥
2 = 3 and 𝑎 𝑥 = 5 .
Since the combinations of 𝑎 𝑥 and 𝜎 𝑥
2 are infinite (uncountable infinite)
Uncountable infinite tables to be constructed
Unpractical method
24
25. 2 2
( ) 2
2
1
( )
2
X xa
x
x
XF x e d
we make the variable change = ( ) in ( )X x Xu a F x
2( )
21
( )
2
X xx a
u
XF x e du
We will show that the general distribution function 𝐹 𝑋(𝑥)
2
21
( ) 0 , 1
2
x
X xF x e d a
can be found in terms of the normalize distribution 𝐹(𝑥)
25
= 𝐹
𝑥 − 𝑎 𝑥
𝜎 𝑋
𝐹𝑋(𝑥) = 𝐹
𝑥 − 𝑎 𝑥
𝜎 𝑋
31. Other Distribution and Density
Examples
• Discrete: Binomial, Poisson.
• Continuous: Uniform, Exponential, Rayleigh ….
(more example in Appendix F)
31
32. Other Distribution and Density Examples
0
(( ) (1 ) )
N
k
k
X
N k
p pf k
N
k
xx
is called the binomial density function.
isthebinomialcoefficie
!
!(
t
)!
n
N N
k k N k
The density can be applied to the
- Bernoulli trial experiment. (only two outcomes)
- Games of chance (win or lose).
Binomial
Let 0 < 𝑝 < 1, N = 1, 2,..., then the function
- Detection problems in radar and sonar. 32
𝑁 = 6, 𝑝 = 0.25
33. It applies to many experiments that have only two possible
outcomes ({H,T}, {0,1}, {Target, No Target}) on any given
trial (N).
It applies when you have N trials of the experiment of only
outcomes and you ask what is the probability of k-successes out
of these N trials.
0
(( ) (1 ) )
N
k
k
X
N k
p pF k
N
k
u xx
Binomial distribution
N = 6 p = 0.25
33
34. Matlab Example: Binomial
• % Dr. Ali Muqaibel
• clear all
• close all
• % EE315 HW 3
• N=6;
• P=0.25;
• x=0:N;
• y=binopdf(x,N,P)
• y1=cdf('bino',x,N,P)
• figure (1)
• stem(x,y)
• xlabel('Power delivered by the Master Station')
• ylabel ('pdf')
• % axis ([-.1 0.9 0 0.6])
• figure (2)
• stairs(x,y1)
•
• xlabel('Power delivered by the Master Station')
• ylabel ('CDF')
• % axis ([0 0.8 0 1.2])
34
0 1 2 3 4 5 6 7
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
Power delivered by the Master Station
pdf
0 1 2 3 4 5 6 7
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Power delivered by the Master Station
CDF
The following figure illustrates the binomial density and
distribution functions for 𝑁 = 6 and 𝑝 = 0.25.
36. (consta
0
nt)
N
N b
p
p
0
( ) ( )
!
b
k
X
k
f x xe
k
b
k
0
( ) ( )
!
b
k
X
k
F x u x
k
e k
b
where b > 0 is a real constant.
Binomial Poisson
The Poisson RV X has a density and distribution
Poisson
Developed by the French mathematician Simeon Denis Poisson in 1837
N 10
N 20
N 1000
36
37. The Poisson RV applies to a wide variety of counting-type
applications:
- The number of defective units in a production line.
- The number of telephone calls made during a period of time.
- The number of electrons emitted from a small section of a
cathode in a given time interval.
37
Counting type problems:
Time interval of interest 𝑇
Average arrival rate 𝜆
How many arrival 𝑥 ?
𝑏 = 𝜆𝑇
0
( ) ( )
!
b
k
X
k
f x xe
k
b
k
0
( ) ( )
!
b
k
X
k
F x u x
k
e k
b
38. Example: Poisson Distribution
• At a gasoline station, cars arrive according to Poisson
distribution at a rate of 50 per hour. There is 1 gasoline
pump and it takes 1 min to obtain fuel.
• What is the probability that a waiting line will occur.
• Waiting occurs if we have more than 1 customer =1 − 𝐹𝑋(1)
• 𝜆 =
50
60
car per min
• 𝑇 = 1 𝑚𝑖𝑛
• 𝑏 = 𝜆𝑇 =
5
6
• Probability of waiting= 1 − 𝐹𝑋 1 = 1 − 𝑒−
5
6 1 +
5
6
= 0.2032
• Probability of waiting in a line=20.32%
38
0
( ) ( )
!
b
k
X
k
F x u x
k
e k
b
39. Practice: Poisson Dist.
Packets arrive at an internet router at a rate of
20 𝑝𝑎𝑐𝑘𝑒𝑡𝑠 𝑝𝑒𝑟 𝑚𝑖𝑛𝑢𝑡𝑒𝑠 and the router needs
3 𝑠𝑒𝑐𝑜𝑛𝑑𝑠 to serve each packet. If the router can
𝑠𝑒𝑟𝑣𝑒 4 𝑝𝑎𝑐𝑘𝑒𝑡𝑠 𝑎𝑡 𝑎 𝑡𝑖𝑚𝑒. Find the probability that
new incoming calls will not be served.
39
Ans. 3.5 × 10−3
0
( ) ( )
!
b
k
X
k
f x xe
k
b
k
0
( ) ( )
!
b
k
X
k
F x u x
k
e k
b
41. Exponential Distribution
• 𝑓𝑋 𝑥 =
1
𝑏
𝑒−
𝑥−𝑎
𝑏 𝑥 > 𝑎
0 𝑥 < 𝑎
• 𝐹𝑋 𝑥 = 1 − 𝑒−
𝑥−𝑎
𝑏 𝑥 > 𝑎
0 𝑥 < 𝑎
• where 𝑏 > 0 , −∞ < 𝑎 < ∞
• Used to represent the fluctuation of
signal strength in “radar”… rain drop
size
41
42. Example: Exponential
Distribution
• 𝑃 is the received by a radar from aircraft
𝑓𝑃 𝑝 =
1
𝑝0
𝑒
−
𝑝
𝑝0 𝑝 > 0
0 𝑝 < 0
• 𝑝0 is the average amount of received power 𝑃.
• What is the probability that the received power is above the
average?
• Exponential with 𝑎 = 0, 𝑏 = 𝑝0, 𝑓𝑋 𝑥 =
1
𝑏
𝑒−
𝑥−𝑎
𝑏 𝑥 > 𝑎
0 𝑥 < 𝑎
• 𝑃 𝑃 > 𝑃0 = 1 − 𝑃 𝑃 ≤ 𝑝0 = 1 − 𝐹𝑃 𝑝0 = 1 − 1 − 𝑒
−
𝑝0
𝑝0 = 𝑒−1
≈ 0.368
42
43. Rayleigh
The Rayleigh density and distribution functions are
2
( )2
( )
( )
0
x a b
X
x a e x a
f x b
x a
2
( )
1
( )
0
x a b
X
e x a
F x
x a
for real constants and 0a b
43
Important in wireless
applications, and
measurements errors
44. Median & Example for Rayleigh
• Median: It is the value of 𝑋 for which the
probability is 0.5 that the values of 𝑋 does not
exceed the median.
• The value 𝑥0 such that 𝑃 𝑋 ≤ 𝑥0 = 𝑃 𝑥0 < 𝑋 ,
this 𝑥 = 𝑥0 is called the median.
• Find the median for Rayleigh distribution?
• 𝑃 𝑋 ≤ 𝑥0 = 𝐹𝑋 𝑥0 = 0.5
• 𝐹𝑋 𝑥0 = 1 − 𝑒
−
𝑥0−𝑎 2
𝑏 = 0.5
• 𝑥0 = 𝑎 + 𝑏𝑙𝑛 2 0.5
44
𝑚𝑒𝑑𝑖𝑎𝑛 ≠ 𝑎𝑣𝑒𝑟𝑎𝑔𝑒
Find out the difference
45. Conditional Distribution and Density Functions
For two events 𝐴 and 𝐵 the conditional probability of event
𝐴 given event 𝐵 had occurred was defined as
𝑃 𝐴 𝐵 =
𝑃 𝐴 ∩ 𝐵
𝑃 𝐵
We extend the concept of conditional probability to include
random variables
45
46. Conditional Distribution
Let X be a random variable and define the event 𝐴
𝐴 = 𝑋 ≤ 𝑥}
We define the conditional distribution function 𝐹𝑋(𝑥|𝐵)
{X x
X
} B
A
P{X x B}
F (x|B) = P{X x|B} =
P(B)
46
𝑃 𝐴 𝐵 =
𝑃 𝐴 ∩ 𝐵
𝑃 𝐵
47. Properties of Conditional Distribution
X
X
(1) F ( |B) = 0
F ( |B) = P X |B
P X B 0
= 0
P(B) P(B)
proof
X
X
(2) F ( |B) = 1
F ( |B) = P X |B
P X B P(B)
Pr
= 1
P(B) P(B)
oof
X(3) 0 F (x|B) 1
1 2 X 2 X 1(5) P x < X x |B = F (x |B) F (x |B)
X 1 X 2 1 2(4) F (x |B) F (x |B) if x < x
+
X X(6) F (x |B) = F (x|B)
None Decreasing
Right continuous 47
48. Conditional Density Functions
We define the Conditional Density Function of the random
variable X as the derivative of the conditional distribution
function
X
X
dF (x|B)
f (x|B) =
dx
If contain step discontinuities as when X is discrete or
mixed (continues and discrete ) then will contain
impulse functions.
XF (x|B)
Xf (x|B)
48
49. Properties of Conditional Density
X(1) f (x|B) 0
X(2) f (x|B) dx = 1
x
X X(3) F (x|B) = f (ξ|B)dξ
2
1
x
1 2 Xx
(4) P x < X x |B = f (x|B)dx
49
50. Example: Conditional density &
distribution
• Two mutually exclusive events:
𝐵1 “select the smaller box”, 𝑃 𝐵1 =
2
10
𝐵2 “select the larger box”, 𝑃 𝐵2 =
8
10
A random variable is defined to be 𝑋
𝑃 𝑋 = 1 𝐵 = 𝐵1 =
5
100
𝑃 𝑋 = 2 𝐵 = 𝐵1 =
35
100
𝑃 𝑋 = 3 𝐵 = 𝐵1 =
60
100
𝑓𝑋 𝑥 𝐵1 =
5
100
𝛿(𝑥 − 1)+
35
100
𝛿(𝑥 − 2)+
60
100
𝛿(𝑥 − 3)
𝐹𝑋 𝑥 𝐵1 =
5
100
𝑢(𝑥 − 1)+
35
100
𝑢(𝑥 − 2)+
60
100
𝑢(𝑥 − 3)
50
𝒙 𝟏 Ball
Color
Small
box (𝐵1)
Larger
box (𝐵 𝟐)
Total
1 Red 5 80 85
2 Green 35 60 95
3 Blue 60 10 70
Total 100 150 250
52. Next we define the event were b is a real number B = X b
< b <
X
P X x X b
F (x|X b) = P X x|X b =
P X b
P X b 0
Cas xe 1 b
{X b} {X x }
{X x} {X b} = {X b}
X
P X x X b P X b
F (x|X b) = = 1
P X b P X b
x
X
b
52
53. Cas xe 2 b
{X x} {X b }
{X x} {X b} = {X x}
X
X
X
P X x X b P X x F (x)
F (x|X b) = =
P X b P X b F (b)
By combining the two expressions we get
X
XX
F (x)
x < b
F (b)F (x|X b) =
1 b x
X
x b
53
54. Then the conditional distribution FX(x|X ≤ b) is never smaller
then the ordinary distribution FX(x)
X XF (x|X b) F (x)
X
XX
F (x)
x < b
F (b)F (x|X b) =
1 b x
54
55. The conditional density function derives from the derivative
X
X
dF (x|X b)
f (x|X b) =
dx
X Xf (x|X b) f (x)
X X
b
X
X
f (x) f (x)
= x < b
F (b) f (x)dx=
0 x b
Similarly for the conditional density function
55
56. Example 8 Let X be a random variable with an exponential
probability density function given as
0
0 0
( )X
xe x
x
f x
2
( )
( )
Since 2( | )
2
0 2
X
X
X
f x
f x dx
f x X
x
x
Find the probability P( X < 1 | X ≤ 2 )
( | )2X
Xf x
( )X
f x
56
=
𝑒−𝑥
/(1 − 𝑒−2
) 0 < 𝑥 ≤ 2
0 𝑥 < 0
0 𝑥 > 2
57. 1
0
( 1| )2 2| )(X
X XP f x dX x
1
0
2
1
xe dx
e
1
2
1
1
e
e
0.7310
1
0
2
1
xe dx
e
57
