lecture2.ppt

Postacademic Course on
Telecommunications
20/4/00
p. 1
Module-3 Transmission Marc Moonen
Lecture-2 Limits of Communication K.U.Leuven-ESAT/SISTA
Lecture-2: Limits of Communication
• Problem Statement:
Given a communication channel (bandwidth B),
and an amount of transmit power, what is the
maximum achievable transmission bit-rate
(bits/sec), for which the bit-error-rate is (can be)
sufficiently (infinitely) small ?
- Shannon theory (1948)
- Recent topic: MIMO-transmission
(e.g. V-BLAST 1998, see also Lecture-1)

Telecommunications
20/4/00
p. 2
Overview
• `Just enough information about entropy’
(Lee & Messerschmitt 1994)
self-information, entropy, mutual information,…
• Channel Capacity (frequency-flat channel)
• Channel Capacity (frequency-selective channel)
example: multicarrier transmission
• MIMO Channel Capacity
example: wireless MIMO

Telecommunications
20/4/00
p. 3
`Just enough information about entropy’(I)
• Consider a random variable X with sample space
(`alphabet’)
• Self-information in an outcome is defined as
where is probability for (Hartley 1928)
• `rare events (low probability) carry more information
than common events’
`self-information is the amount of uncertainty
removed after observing .’
 
M



 ,...,
,
, 3
2
1
K

)
(
log
)
( 2 K
X
K p
h 
 

K

)
( K
X
p 
K


Telecommunications
20/4/00
p. 4
`Just enough information about entropy’(II)
• Consider a random variable X with sample space
(`alphabet’)
• Average information or entropy in X is defined as
because of the log, information is measured in bits
 
M



 ,...,
,
, 3
2
1



K
K
X
K
X p
p
X
H


 )
(
log
).
(
)
( 2

Telecommunications
20/4/00
p. 5
`Just enough information about entropy’ (III)
• Example: sample space (`alphabet’) is {0,1} with
entropy=1 bit if q=1/2 (`equiprobable symbols’)
entropy=0 bit if q=0 or q=1 (`no info in certain events’)
q
p
q
p X
X 

 1
)
0
(
,
)
1
(
q
H(X)
1
1
0

Telecommunications
20/4/00
p. 6
`Just enough information about entropy’ (IV)
• `Bits’ being a measure for entropy is slightly
confusing (e.g. H(X)=0.456 bits??), but the
theory leads to results, agreeing with our intuition
(and with a `bit’ again being something that is
either a `0’ or a `1’), and a spectacular theorem
• Example:
alphabet with M=2^n equiprobable symbols :
-> entropy = n bits
i.e. every symbol carries n bits

Telecommunications
20/4/00
p. 7
`Just enough information about entropy’ (V)
• Consider a second random variable Y with sample
space (`alphabet’)
• Y is viewed as a `channel output’, when X is
the `channel input’.
• Observing Y, tells something about X:
is the probability for after
observing
 
N



 ,...,
,
, 3
2
1
)
,
(
| K
K
Y
X
p 
 K

K


Telecommunications
20/4/00
p. 8
`Just enough information about entropy’ (VI)
• Example-1 :
• Example-2 : (infinitely large alphabet size for Y)
+
noise decision
device
X Y
00
01
10
11
+
noise
X Y
00
01
10
11
00
01
10
11

Telecommunications
20/4/00
p. 9
`Just enough information about entropy’(VII)
• Average-information or entropy in X is defined as
• Conditional entropy in X is defined as
Conditional entropy is a measure of the average uncertainty
about the channel input X after observing the output Y



K
K
X
K
X p
p
X
H


 )
(
log
).
(
)
( 2
 


K K
K
K
Y
X
K
K
Y
X
K
Y p
p
p
Y
X
H
 




 )
|
(
log
).
|
(
)
(
)
|
( |
2
|

Telecommunications
20/4/00
p. 10
`Just enough information about entropy’(VIII)
• Average information or entropy in X is defined as
• Conditional entropy in X is defined as
• Average mutual information is defined as
I(X|Y) is uncertainty about X that is removed by observing Y



K
K
X
K
X p
p
X
H


 )
(
log
).
(
)
( 2
 


K K
K
K
Y
X
K
K
Y
X
K
Y p
p
p
Y
X
H
 




 )
|
(
log
).
|
(
)
(
)
|
( |
2
|
)
|
(
)
(
)
|
( Y
X
H
X
H
Y
X
I 


Telecommunications
20/4/00
p. 11
Channel Capacity (I)
• Average mutual information is defined by
-the channel, i.e. transition probabilities
-but also by the input probabilities
• Channel capacity (`per symbol’ or `per channel
use’) is defined as the maximum I(X|Y) for all
possible choices of
• A remarkably simple result: For a real-valued
additive Gaussian noise channel, and infinitely
large alphabet for X (and Y), channel capacity is
)
,
(
| K
K
Y
X
p 

)
( K
X
p 
)
( K
X
p 
)
1
(
log
.
2
1
2
2
2
n
x



signal
(noise)
variances

Telecommunications
20/4/00
p. 12
Channel Capacity (II)
• A remarkable theorem (Shannon 1948):
With R channel uses per second, and channel
capacity C, a bit stream with bit-rate C*R
(=capacity in bits/sec) can be transmitted with
arbitrarily low probability of error
= Upper bound for system performance !

Telecommunications
20/4/00
p. 13
Channel Capacity (II)
• For a real-valued additive Gaussian noise
channel, and infinitely large alphabet for X (and
Y), the channel capacity is
• For a complex-valued additive Gaussian noise
channel, and infinitely large alphabet for X (and
Y), the channel capacity is
)
1
(
log 2
2
2
n
x



)
1
(
log
.
2
1
2
2
2
n
x




Telecommunications
20/4/00
p. 14
Channel Capacity (III)
Information I(X|Y) conveyed by a real-valued channel with
additive white Gaussian noise, for different input alphabets,
with all symbols in the alphabet equally likely
(Ungerboeck 1982)
2
2
n
x
SNR




Telecommunications
20/4/00
p. 15
Channel Capacity (IV)
Information I(X|Y) conveyed by a complex-valued channel
with additive white Gaussian noise, for different input
alphabets, with all symbols in the alphabet equally likely
(Ungerboeck 1982)

Telecommunications
20/4/00
p. 16
Channel Capacity (V)
This shows that, as long as the alphabet is
sufficiently large, there is no significant loss in
capacity by choosing a discrete input alphabet,
hence justifies the usage of such alphabets !
The higher the SNR, the larger the required
alphabet to approximate channel capacity

Telecommunications
20/4/00
p. 17
Channel Capacity (frequency-flat channels)
• Up till now we considered capacity `per symbol’ or
`per channel use’
• A continuous-time channel with bandwidth B (Hz)
allows 2B (per second) channel uses (*), i.e. 2B
symbols being transmitted per second, hence
capacity is
(*) This is Nyquist criterion `upside-down’ (see also Lecture-3)
second
bits
)
1
(
log
2
1
.
2 2
2
2
n
x
B





received signal (noise) power

Telecommunications
20/4/00
p. 18
• Example: AWGN baseband channel
(additive white Gaussian noise channel)
n(t)
+
channel
s(t) r(t)=Ho.s(t)+n(t)
Ho
f
H(f)
B
-B
Ho
second
bits
)
1
(
log
. 2
2
2
0
2
n
x
H
B



B
N
B
Es
n
x
.
2
.
0
2
2




Telecommunications
20/4/00
p. 19
• Example: AWGN passband channel
passband channel with bandwidth B accommodates
complex baseband signal with bandwidth B/2 (see Lecture-3)
n(t)
+
channel
s(t) r(t)=Ho.s(t)+n(t)
Ho
f
H(f)
x
Ho
second
bits
)
1
(
log
2
.
2 2
2
2
0
2
n
x
H
B





x+B

Telecommunications
20/4/00
p. 20
Channel Capacity (frequency-selective channels)
n(t)
+
channel
s(t) R(f)=H(f).S(f)+N(f)
H(f)
• Example: frequency-selective AWGN-channel
received SNR is frequency-dependent!
f
H(f)
B
-B

Telecommunications
20/4/00
p. 21
• Divide bandwidth into small bins of width df,
such that H(f) is approx. constant over df
• Capacity is
optimal transmit power spectrum?
f
H(f)
B
-B
second
bits
).
)
(
)
(
)
(
1
(
log 2
2
2
2
  df
f
f
f
H
n
x


0
B

Telecommunications
20/4/00
p. 22
Maximize
subject to
solution is
`Water-pouring spectrum’
  df
f
f
f
H
n
x
).
)
(
)
(
)
(
1
(
log 2
2
2
2


Available Power 
 df
f
x
x ).
(
2
2


)
)
(
)
(
,
0
max(
)
( 2
2
2
f
H
f
L
f n
x

 

B
L
)
(
)
(
2
2
f
H
f
n

area 2
x


Telecommunications
20/4/00
p. 23
Example : multicarrier modulation
available bandwidth is split up into different `tones’, every
tone has a QAM-modulated carrier
(modulation/demodulation by means of IFFT/FFT).
In ADSL, e.g., every tone is given (+/-) the same power,
such that an upper bound for capacity is (white noise case)
(see Lecture-7/8)
second
bits
).
).
(
1
(
log 2
2
2
2
  df
f
H
n
x



Telecommunications
20/4/00
p. 24
MIMO Channel Capacity (I)
• SISO =`single-input/single output’
• MIMO=`multiple-inputs/multiple-outputs’
• Question:
we usually think of channels with one transmitter
and one receiver. Could there be any advantage
in using multiple transmitters and/or receivers
(e.g. multiple transmit/receive antennas in a
wireless setting) ???
• Answer: You bet..

Telecommunications
20/4/00
p. 25
MIMO Channel Capacity (II)
• 2-input/2-output example
A
B
C
D
+
+
X1
X2 Y2
Y1
N1
N2


























2
1
2
1
.
2
1
N
N
X
X
D
C
B
A
Y
Y

Telecommunications
20/4/00
p. 26
MIMO Channel Capacity (III)
Rules of the game:
• P transmitters means that the same total power
is distributed over the available transmitters
(no cheating)
• Q receivers means every receive signal is
corrupted by the same amount of noise
(no cheating)
Noises on different receivers are often assumed to be
uncorrelated (`spatially white’), for simplicity
...
)
(
)
( 2
2
2
2
1


 
 df
f
df
f X
X
X 


...
2
2
2
2
1


 N
N
N 



Telecommunications
20/4/00
p. 27
MIMO Channel Capacity (IV)
2-in/2-out example, frequency-flat channels
Ho
0
0
Ho
+
+
X1
X2 Y2
Y1
N1
N2


























2
1
2
1
.
0
0
2
1
N
N
X
X
Ho
Ho
Y
Y
first example/attempt

Telecommunications
20/4/00
p. 28
MIMO Channel Capacity (V)
• corresponds to two separate channels, each
with input power and additive noise
• total capacity is
• room for improvement...


























2
1
2
1
.
0
0
2
1
N
N
X
X
Ho
Ho
Y
Y
2
2
X
 2
N

second
bits
)
.
2
1
(
log
.
.
2 2
2
2
0
2
n
x
H
B




Telecommunications
20/4/00
p. 29
MIMO Channel Capacity (VI)
Ho
Ho
-Ho
Ho
+
+
X1
X2 Y2
Y1
N1
N2



























2
1
2
1
.
2
1
N
N
X
X
Ho
Ho
Ho
Ho
Y
Y
second example/attempt

Telecommunications
20/4/00
p. 30
MIMO Channel Capacity (VII)
A little linear algebra…..



























2
1
2
1
.
2
1
N
N
X
X
Ho
Ho
Ho
Ho
Y
Y

































2
1
2
1
.
2
1
2
1
2
1
2
1
.
.
2
0
0
.
2
N
N
X
X
Ho
Ho
Matrix V’

Telecommunications
20/4/00
p. 31
MIMO Channel Capacity (VIII)
A little linear algebra…. (continued)
• Matrix V is `orthogonal’ (V’.V=I) which means that it
represents a transformation that conserves energy/power
• Use as a transmitter pre-transformation
• then (use V’.V=I) ...


























2
1
2
1
'.
.
.
2
0
0
.
2
2
1
N
N
X
X
V
Ho
Ho
Y
Y













2
ˆ
1
ˆ
.
2
1
X
X
V
X
X

Telecommunications
20/4/00
p. 32
MIMO Channel Capacity (IX)
• Then…
+
+ Y2
Y1
N1
N2
+
+
X^1
X^2
X2
X1
transmitter
A
B
C
D
V11
V12
V21
V22
channel receiver

Telecommunications
20/4/00
p. 33
MIMO Channel Capacity (X)
• corresponds to two separate channels, each
with input power , output power
and additive noise
• total capacity is
2
2
X

2
N

second
bits
)
1
(
log
.
.
2 2
2
2
0
2
n
x
H
B





























2
1
2
ˆ
1
ˆ
.
.
2
0
0
.
2
2
1
N
N
X
X
Ho
Ho
Y
Y
2
)
.
2
(
2
2 X
Ho

2x SISO-capacity!

Telecommunications
20/4/00
p. 34
MIMO Channel Capacity (XI)
• Conclusion: in general, with P transmitters and
P receivers, capacity can be increased with a
factor up to P (!)
• But: have to be `lucky’ with the channel (cfr.
the two `attempts/examples’)
• Example : V-BLAST (Lucent 1998)
up to 40 bits/sec/Hz in a `rich scattering
environment’ (reflectors, …)

Telecommunications
20/4/00
p. 35
MIMO Channel Capacity (XII)
• General I/O-model is :
• every H may be decomposed into
this is called a `singular value decompostion’, and works for
every matrix (check your MatLab manuals)
 





















P
QxP
Q X
X
H
Y
Y
:
.
:
1
1
'
.
. V
S
U
H 
diagonal matrix
orthogonal matrix V’.V=I
orthogonal matrix U’.U=I

Telecommunications
20/4/00
p. 36
MIMO Channel Capacity (XIII)
With H=U.S.V’,
• V is used as transmitter pre-tranformation
(preserves transmit energy) and
• U’ is used as a receiver transformation
(preserves noise energy on every channel)
• S=diagonal matrix, represents resulting,
effectively `decoupled’ (SISO) channels
• Overall capacity is sum of SISO-capacities
• Power allocation over SISO-channels (and as a
function of frequency) : water pouring

Telecommunications
20/4/00
p. 37
MIMO Channel Capacity (XIV)
Reference:
G.G. Rayleigh & J.M. Cioffi
`Spatio-temporal coding for wireless communications’
IEEE Trans. On Communications, March 1998

Telecommunications
20/4/00
p. 38
Assignment 1 (I)
• 1. Self-study material
Dig up your favorite (?) signal processing
textbook & refresh your knowledge on
-discrete-time & continuous time signals & systems
-signal transforms (s- and z-transforms, Fourier)
-convolution, correlation
-digital filters
...will need this in next lectures

Telecommunications
20/4/00
p. 39
Assignment 1 (II)
• 2. Exercise (MIMO channel capacity)
Investigate channel capacity for…
-SIMO-system with 1 transmitter, Q receivers
-MISO-system with P transmitters, 1 receiver
-MIMO-system with P transmitters, Q receivers
P=Q (see Lecture 2)
P>Q
P<Q

lecture2.ppt

Recommended

Recommended

More Related Content

Similar to lecture2.ppt

Similar to lecture2.ppt (20)

Recently uploaded

Recently uploaded (20)

lecture2.ppt