VTU CBCS E&C 5th sem Information theory and coding(15EC54) Module -3 notes
•
4 likes•1,233 views
VTU CBCS E&C 5th sem Information theory and coding(15EC54) Module -3 notes
![FORMULAS FOR REFERENCE
MODULE –3 ( INFORMATION CHANNELS)
The entropy of input symbol is given by
H (A) = 𝑷(𝒂𝒊)𝐥𝐨𝐠(
𝟏
𝑷 (𝒂 𝒊)
𝒓
𝒊=𝟏 ) bits/ message-symbol
The entropy of output symbol is given by
H (B) = 𝑷(𝒃𝒋)𝐥𝐨𝐠(
𝟏
𝑷(𝒃 𝒋)
𝒔
𝒋=𝟏 ) bits/ message-symbol
The joint entropy is given by
H (A, B) = 𝑷( 𝒂𝒊, 𝒃𝒋)𝐥𝐨𝐠(
𝟏
𝑷( 𝒂 𝒊,𝒃 𝒋)
)𝒔
𝒋=𝟏
𝒓
𝒊=𝟏 bits/ message-symbol
The equivocation H(A/B) is given by
H (A / B) = 𝑷( 𝒂𝒊, 𝒃𝒋)𝐥𝐨𝐠(
𝟏
𝑷 ( 𝒂 𝒊/𝒃 𝒋)
)𝒔
𝒋=𝟏
𝒓
𝒊=𝟏 bits/ message-symbol
Since P( 𝒂𝒊/𝒃𝒋) =
𝑷( 𝒂 𝒊,𝒃 𝒋)
𝑷 (𝒃 𝒋
)
And another formula
H (A/B) = H (A, B) – H (B) bits/ message-symbol
The equivocation H(B/A) is given by
H (B / A) = 𝑷( 𝒂𝒊, 𝒃𝒋)𝐥𝐨𝐠(
𝟏
𝑷(𝒃 𝒋/ 𝒂 𝒊)
)𝒔
𝒋=𝟏
𝒓
𝒊=𝟏 bits/ message-symbol
Since P(𝒃𝒋/ 𝒂𝒊) =
𝑷( 𝒂 𝒊,𝒃 𝒋)
𝑷 (𝒂 𝒊)
And another formula
H (B/A) = H (A, B) – H (A) bits/ message-symbol
The mutual information is given by
I (A, B) = H (A) – H (A/B) bits/ message-symbol
The channel capacity is given by
C = (log s – h) 𝒓 𝒔 bits/ sec
Since h= 𝐇
𝐁
𝐀
= 𝑷𝒋 𝐥𝐨𝐠(
𝟏
𝑷 𝒋
𝒔
𝒋=𝟏 ) bits/ message-symbol
The capacity of the channel is given by
C=B 𝒍𝒐𝒈 𝟐 [1 + S/N]
Source efficiency 𝜼 𝑺 is given by
𝜼 𝑺 =
𝑯(𝑿)
𝑯 𝑿 𝒎𝒂𝒙
x 100%](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to VTU CBCS E&C 5th sem Information theory and coding(15EC54) Module -3 notes
Similar to VTU CBCS E&C 5th sem Information theory and coding(15EC54) Module -3 notes (12)
More from Jayanth Dwijesh H P
More from Jayanth Dwijesh H P (10)
Recently uploaded
Recently uploaded (20)
VTU CBCS E&C 5th sem Information theory and coding(15EC54) Module -3 notes
- 1. INFORMATION THEORY AND CODING 5th SEM E&C JAYANTHDWIJESH H P M.tech (DECS) Assistant Professor – Dept of E&C B.G.S INSTITUTE OF TECHNOLOGY (B.G.S.I.T) B.G Nagara, Nagamangala Tq, Mandya District- 571448
- 2. FORMULAS FOR REFERENCE MODULE –3 ( INFORMATION CHANNELS) The entropy of input symbol is given by H (A) = 𝑷(𝒂𝒊)𝐥𝐨𝐠( 𝟏 𝑷 (𝒂 𝒊) 𝒓 𝒊=𝟏 ) bits/ message-symbol The entropy of output symbol is given by H (B) = 𝑷(𝒃𝒋)𝐥𝐨𝐠( 𝟏 𝑷(𝒃 𝒋) 𝒔 𝒋=𝟏 ) bits/ message-symbol The joint entropy is given by H (A, B) = 𝑷( 𝒂𝒊, 𝒃𝒋)𝐥𝐨𝐠( 𝟏 𝑷( 𝒂 𝒊,𝒃 𝒋) )𝒔 𝒋=𝟏 𝒓 𝒊=𝟏 bits/ message-symbol The equivocation H(A/B) is given by H (A / B) = 𝑷( 𝒂𝒊, 𝒃𝒋)𝐥𝐨𝐠( 𝟏 𝑷 ( 𝒂 𝒊/𝒃 𝒋) )𝒔 𝒋=𝟏 𝒓 𝒊=𝟏 bits/ message-symbol Since P( 𝒂𝒊/𝒃𝒋) = 𝑷( 𝒂 𝒊,𝒃 𝒋) 𝑷 (𝒃 𝒋 ) And another formula H (A/B) = H (A, B) – H (B) bits/ message-symbol The equivocation H(B/A) is given by H (B / A) = 𝑷( 𝒂𝒊, 𝒃𝒋)𝐥𝐨𝐠( 𝟏 𝑷(𝒃 𝒋/ 𝒂 𝒊) )𝒔 𝒋=𝟏 𝒓 𝒊=𝟏 bits/ message-symbol Since P(𝒃𝒋/ 𝒂𝒊) = 𝑷( 𝒂 𝒊,𝒃 𝒋) 𝑷 (𝒂 𝒊) And another formula H (B/A) = H (A, B) – H (A) bits/ message-symbol The mutual information is given by I (A, B) = H (A) – H (A/B) bits/ message-symbol The channel capacity is given by C = (log s – h) 𝒓 𝒔 bits/ sec Since h= 𝐇 𝐁 𝐀 = 𝑷𝒋 𝐥𝐨𝐠( 𝟏 𝑷 𝒋 𝒔 𝒋=𝟏 ) bits/ message-symbol The capacity of the channel is given by C=B 𝒍𝒐𝒈 𝟐 [1 + S/N] Source efficiency 𝜼 𝑺 is given by 𝜼 𝑺 = 𝑯(𝑿) 𝑯 𝑿 𝒎𝒂𝒙 x 100%
- 3. Source efficiency 𝑹 𝜼 𝑺 is given by 𝑹 𝜼 𝑺 = 𝟏 − 𝜼 𝑺 Channel efficiency is given by 𝜼 𝒄𝒉 = 𝐈(𝐗,𝐘) 𝐂 x 100% Channel redundancy is given by Channel redundancy = (1- 𝜼 𝒄𝒉) x 100% Estimation of channel capacity by MUROGA’S method