Phd Defence

910 views

Published on

Construction and Analysis of Non systematic Codes on Graph for Redundant Data.

  • Be the first to comment

  • Be the first to like this

Phd Defence

  1. 1. Construction and Analysis of Non Systematic Codes on Graphs for Redundant Data Amira A LLOUM Ecole Nationale Sup´ rieure des T´ l´ communications e ee Telecom Paris Tech September 5th, 2008
  2. 2. Presentation Outline Introduction and Motivations Part I: Non Systematic LDPC Codes Constructions Part II: Density Evolution Analysis for Split-LDPC Codes Part III: Exit Chart Analysis for Split-LDPC Codes Part IV : EM for Joint Source-Channel Estimation Conclusions and Future Work
  3. 3. Introduction The Non Uniform Assumption 0.5 0 1 P (s = 1) = 0.5 Source Encoder Channel Encoder Channel Channel Decoder Source Decoder Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 1 / 35
  4. 4. Introduction The Non Uniform Assumption Source Encoder Source Decoder 0.9 0 Channel 0.1 Channel Encoder Channel Decoder 1 P (s = 1) = µ The Uniform Assumption is not valid anymore When 1 It is not worth to compress (bad channel conditions) Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 1 / 35
  5. 5. Introduction The Non Uniform Assumption Source Encoder Source Decoder 0.9 0 Channel 0.1 Channel Encoder Channel Decoder 1 P (s = 1) = µ Source Encoder Source Decoder The Uniform Assumption is not valid anymore When 1 It is not worth to compress (bad channel conditions) 2 Using sub-optimal compression (highly redundant sources) Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 1 / 35
  6. 6. Introduction Channel Coding Strategies for Non Uniform Sources 0.9 P (s = 1) = µ 0 0.1 1 Source Channel Encoder Channel Channel Decoder Sink Shannon has intuited in his 1948 Paradigm Any Redundancy in the source will usually help if it is utilized at the receiveing ...This redundancy will help to combat noise Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 2 / 35
  7. 7. Introduction Channel Coding Strategies for Non Uniform Sources 0.9 P (s = 1) = µ 0 0.1 1 Source Channel Encoder Channel Channel Decoder Sink SCCD µ Channel Coding for Redundant data follows the following strategies 1 Source Controlled Channel Coding (Hagenauer 1995). Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 2 / 35
  8. 8. Introduction Channel Coding Strategies for Non Uniform Sources 0.9 P (s = 1) = µ Systematic Coding 0 0.1 1 Info Parity Source Channel Encoder Channel Channel Decoder Sink Parity µ Non−Systematic Coding µ Best Constructions Channel Coding for Redundant data follows the following strategies 1 Source Controlled Channel Coding (Hagenauer 1995). 2 Non Systematic Encoding Structures (Shamai and Verdu 1997) Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 2 / 35
  9. 9. Introduction Information Theoretical Limits wih Redandancy: AWGN Channel Capacity limit Versus Source Entropy, Coding Rate = 0.5, AWGN Channel 2 0 Minimum Achievable Eb/N0 (dB) -2 -4 -6 -8 -10 Systematic code, BPSK input Non-Systematic code, BPSK input Gaussian input -12 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Source Entropy (bits) Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 3 / 35
  10. 10. Introduction Information Theoretical Limits wih Redandancy: AWGN Channel Capacity limit Versus Coding Rate , Source Entropy= 0.5, AWGN Channel 6 Systematic codes, BPSK input 5 Non-Systematic code, BPSK input Gaussian input 4 3 Minimum Achievable Eb/N0 (dB) 2 1 0 -1 -2 -3 -4 -5 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Coding Rate Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 4 / 35
  11. 11. Introduction Achieving the Theoretical Limits 1 In the presence of Redundancy The Theoretical Limits of Information Theory are mooving to better regions. Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 5 / 35
  12. 12. Introduction Achieving the Theoretical Limits 1 In the presence of Redundancy The Theoretical Limits of Information Theory are mooving to better regions. To Attain these Challenging limits : 1 Building Non Systematic Capacity Achieving Encoding Structures In the Codes On graphs Family. 2 Using Source Controlled Channel Decoding with Iterative Algorithms In the Sum-Product Algorithms Family. Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 5 / 35
  13. 13. Introduction Related Work and Motivation Non−Systematic Codes On Graph Turbo Codes LDPC Non−Systematic MN Codes for BSC Alajaji Codes et.al. LDPC Codes Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 6 / 35
  14. 14. Introduction Main System Assumptions 0.9 P (s = 1) = µ 0 0.1 1 Source Channel Encoder Channel Channel Decoder Sink Lossless source coding or no source coding. Binary i.i.d. source with entropy Hs = H2 (µ), where µ = P (si = 1). Source sequence s = (s1 , s2 , ..., sK ) encoded by a binary channel code of rate Rc = K/N , dimension K, and length N . x = (x1 , ..., xN ) denotes the codeword (channel input). Transmitted information rate R = Hs × Rc bits per channel use. Any symmetric binary-input channel can be considered, mainly BEC, BSC, and AWGN. Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 7 / 35
  15. 15. Part I Non Systematic LDPC Constructions
  16. 16. Encoding Structures Decoding Strategy Simulation Results Scrambling-LDPC Encoding Structure s u c(u,v) Cs G Scramble LDPC c = G × Cs × s v (1−R).N β (1−R).N v db LDPC Cs : sparse matrix of dimension β K × K. In the regular case row and u db R.N dc u column weight are ds u: systematic bits for the inner LDPC. s α ds v: parity bits for the inner LDPC. R.N s Cs α R.N ds s: source bits. s α Scrambler Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 8 / 35
  17. 17. Encoding Structures Decoding Strategy Simulation Results Splitting-LDPC Encoding Structure s u c(u,v) C−1 s G Splitter LDPC −1 × s c = G × Cs v (1−R).N β (1−R).N v LDPC dc db Cs : full rank sparse matrix of u β dimension K × K. In the regular R.N case row and column weight are ds u Cs u: systematic bits for the inner ds ds Splitter LDPC. s α v: parity bits for the inner LDPC. s α R.N R.N s: source bits. s α Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 9 / 35
  18. 18. Encoding Structures Decoding Strategy Simulation Results Source Controlled Sum-Product Decoding The bitnode Rule: v β v LLRci →pcei =LLRtype + LLRpcej →ci LLR0+ Extrinsic Information ∗ pcej ∈Sc i β u LLR0 if bitnode ∈ {u, ϑ} LLRtype = u LLRs if bitnode ∈ {s} s α LLR0 is the channel observation LLR. 1−µ s α LLRs = log( ) is the source LLR . µ LLRs+ Extrinsic Information s α Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 10 / 35
  19. 19. Encoding Structures Decoding Strategy Simulation Results Source Controlled Sum-Product Decoding The bitnode Rule: v β LLRci →pcei =LLRtype + LLRpcej →ci ∗ v pcej ∈Sc i LLR0 if bitnode ∈ {u, ϑ} β LLRtype = u Extrinsic Information LLRs if bitnode ∈ {s} u The checknode Rule: s α LLRcj →pcei LLRpcei →ci = 2 tanh−1 tanh( ) s α ∗ 2 cj ∈Spce i s α Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 10 / 35
  20. 20. Encoding Structures Decoding Strategy Simulation Results Source Controlled Sum-Product Decoding The bitnode Rule: v β LLRci →pcei =LLRtype + LLRpcej →ci ∗ v pcej ∈Sc LLR0+ Extrinsic Information i LLR0 if bitnode ∈ {u, ϑ} β LLRtype = u LLRs if bitnode ∈ {s} u Extrinsic Information The checknode Rule: s α LLRcj →pcei LLRpcei →ci = 2 tanh−1 tanh( ) s α ∗ 2 LLRs+ Extrinsic Information cj ∈Spce i s α Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 10 / 35
  21. 21. Encoding Structures Decoding Strategy Simulation Results Scrambling or Splitting: Information Theoretical Comparaison Mutual information vs. Eb /N0 for Hs = 0.5 and coding rates Rc = 0.5 (AWGN Channel). 1.8 1.6 Gaussian input 1.4 Non-Systematic code, BPSK input Scrambled ds=5 Scrambled ds=3 1.2 Systematic codes, BPSK input Mutual information 1 0.8 0.6 0.4 0.2 0 -10 -5 0 5 10 Eb/N0(dB) Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 11 / 35
  22. 22. Encoding Structures Decoding Strategy Simulation Results Finite-length Performance 1E-01 1E+00 (3,6) systematic LDPC without SCCD (3,6) systematic LDPC without SCCD (3,6) systematic LDPC with SCCD (3,6) systematic LDPC with SCCD scrambler ds=3 scrambler ds=3 MN code db=3 MN code db=3 splitter ds=4 splitter ds=4 1E-02 1E-01 Frame Error Rate Bit Error Rate 1E-03 1E-02 1E-04 1E-03 1E-05 1E-04 -2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5 -2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5 SNR SNR Bit (left) and word (right) error probabilities vs. signal to noise ratio for codes with rate Rc = 1/2 length N = 2000 and non uniform source distribution µ = 0.1 : systematic (3,6) LDPC with and without SCCD, split-LDPC with ds = 4, scramble-LDPC ds = 3 and MN Codes db = 3. Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 12 / 35
  23. 23. Encoding Structures Decoding Strategy Simulation Results Finite-length Performance (2/2) 1E+00 (3,30) systematic LDPC with SCCD scrambler ds=5 splitter ds=7 1E-01 Frame Error Rate 1E-02 1E-03 1E-04 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5 4 SNR Word error probabilities vs. signal to noise ratio Eb /N0 for codes with rate Rc = 0.9 length N = 2000 and non uniform source distribution µ = 0.1 : systematic (3,30) LDPC with SCCD, scramble-LDPC with ds = 5 and split-LDPC with ds = 7. Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 13 / 35
  24. 24. Part II DE Analysis for Split-LDPC Codes
  25. 25. DE Motivation and Assumption DE statement Simulation Results Stability Analysis Motivation and Contribution The problem: How close do Split-LDPC structures approach the Challenging asymptotical limits ? Exploring the Split-LDPC asymptotical convergence behaviour. Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 14 / 35
  26. 26. DE Motivation and Assumption DE statement Simulation Results Stability Analysis Motivation and Contribution The problem: How close do Split-LDPC structures approach the Challenging asymptotical limits ? Exploring the Split-LDPC asymptotical convergence behaviour. The proposal: Density Evolution Analysis 1 Deriving a Density Evolution algorithm for Split-LDPC codes. 2 Investigating the stability issues related to the decoder convergence. Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 14 / 35
  27. 27. DE Motivation and Assumption DE statement Simulation Results Stability Analysis Density Evolution Assumptions 1 Concentration and the local tree Assumption. Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 15 / 35
  28. 28. DE Motivation and Assumption DE statement Simulation Results Stability Analysis Density Evolution Assumptions 1 Concentration and the local tree Assumption. 3 types of message distribution =⇒ Local tree assumption over 3 types of trees Message oriented Density Evolution. db dc p3(x) ϑ β ϑ β 3 dc db p2(x) u ds u 2 ds p1(x) s s 1 1 1 α α Messages Distributions Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 15 / 35
  29. 29. DE Motivation and Assumption DE statement Simulation Results Stability Analysis Density Evolution Assumptions 1 Concentration and the local tree Assumption. 2 Symmetry Conditions (Channel, variable nodes, Checknodes). Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 15 / 35
  30. 30. DE Motivation and Assumption DE statement Simulation Results Stability Analysis Density Evolution Assumptions 1 Concentration and the local tree Assumption. 2 Symmetry Conditions (Channel, variable nodes, Checknodes). 3 The all-zero codeword restriction. Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 15 / 35
  31. 31. DE Motivation and Assumption DE statement Simulation Results Stability Analysis Density Evolution Assumptions 1 Concentration and the local tree Assumption. 2 Symmetry Conditions (Channel, variable nodes, Checknodes). 3 The all-zero codeword restriction. 0 0 0 0 0 / ∈B A Typical Set C[A]=2k B C(B)=2K×Hs Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 15 / 35
  32. 32. DE Motivation and Assumption DE statement Simulation Results Stability Analysis The DE statement: the Checknode level u 0 0 0 0 { 0 s 0 s Averaging p1(x) α 1 u 1 0 0 0 { ds − 1 0 u u pm (x) = Rc ps (x), (1 − µ) q1 (x) + µ q1 (−x) α m m where: 1−µ ps (x) = δ(x − log ) = δ(x − s) µ q1 (x) = ρα (pm (x)) m 1 Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 16 / 35
  33. 33. DE Motivation and Assumption DE statement Simulation Results Stability Analysis The DE statement: the Checknode level β R p2(x) + (1 − R) p3(x) dc − 1 1 2 u + 1ϑ 2 1 2 u + 1ϑ 2 pm (x) = ρ Rc pm (x) + (1 − Rc ) pm (x) β 2 3 Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 16 / 35
  34. 34. DE Motivation and Assumption DE statement Simulation Results Stability Analysis The DE statement: the Checknode level s β α p1(x) R p2(x) + (1 − R) p3(x) ds − 1 dc − 1 u u 1 u + 1ϑ 1 u + 1ϑ 2 2 2 2 pm (x) = Rc ps (x), (1 − µ) q1 (x) + µ q1 (−x) α m m pm (x) = ρ Rc pm (x) + (1 − Rc ) pm (x) β 2 3 Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 16 / 35
  35. 35. DE Motivation and Assumption DE statement Simulation Results Stability Analysis The DE statement: the variable-node level p1 α u ds − 1 db η(x) xλ(x) α α β β p1 R p2 + (1 − R) p3 ds − 1 dc − 1 1 1 1 u u 2 u + 2 ϑ 2 u + 1ϑ 2 p1 (x) at the (m + 1)th iteration pm+1 (x) = p0 (x) ⊗ λ1α pm (x) 1 α ⊗ λ1 pm (x) β Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 17 / 35
  36. 36. DE Motivation and Assumption DE statement Simulation Results Stability Analysis The DE statement: the variable-node level β p2 u ds db − 1 xη(x) λ(x) α α β β p1 R p2 + (1 − R)p3 ds − 1 dc − 1 u u 1 + 1 1 + 1ϑ 2u 2ϑ 2u 2 p2 (x) at the (m + 1)th iteration pm+1 (x) = p0 (x) ⊗ λ2α pm (x) 2 α ⊗ λ pm (x) β Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 17 / 35
  37. 37. DE Motivation and Assumption DE statement Simulation Results Stability Analysis The DE statement: the variable-node level β p3 ϑ db − 1 R p2 + (1 − R)p3 β β β dc − 1 1 1 1 2u + 2ϑ 2u + 1ϑ 2 p3 (x) at the (m + 1)th iteration pm+1 (x) = p0 (x) ⊗ λ pm (x) 3 β Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 17 / 35
  38. 38. DE Motivation and Assumption DE statement Simulation Results Stability Analysis 2 lambda(x)=0.32660x+0.11960x^2+0.18393x^3+0.36988x^4 rho(x)=0.78555x^5+0.21445x^6 0 Minimum Achievable Eb/N0 (dB) -2 -4 -6 -8 Systematic code, BPSK input -10 Non-Systematic code, BPSK input Gaussian input split-LDPC code, DE thresholds -12 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Source Entropy (bits) Minimum achievable Eb /N0 versus source entropy Hs for a regular ds = 3 splitter concatenated to an irregular LDPC code of rate Rc = 1/2 over a BIAWGN channel. Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 18 / 35
  39. 39. DE Motivation and Assumption DE statement Simulation Results Stability Analysis The asymptotical behaviour of the decoder in the neighborhood of δ∞ 1−R ϑ ϑ β CHANNEL P0 u u Pu0 R s BSC (µ δ−s + (1 − µ) δs )⊗ ds µ ds s BSC µ s BSC µ Pu0 = p0 (x) ⊗ (µ δ−s + (1 − µ) δs )⊗ dS Proposition 1 In the neighborhood of δ∞ , the message density given by the splitter to the core LDPC (from node α to node u) is equivalent to the initial message density of ds parallel concatenated BSC with a crossover probability µ. Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 19 / 35
  40. 40. DE Motivation and Assumption DE statement Simulation Results Stability Analysis The asymptotical behaviour of the decoder in the neighborhood of δ∞ 1−R ϑ ϑ β CHANNEL P0 u u Pu0 R s BSC (µ δ−s + (1 − µ) δs )⊗ ds µ ds s BSC µ s BSC µ Pu0 = p0 (x) ⊗ (µ δ−s + (1 − µ) δs )⊗ dS Proposition 2 For non-uniform sources,type-1 message distribution (from node u to node α) shows a permanent stability around the fixed point δ∞ . Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 19 / 35
  41. 41. DE Motivation and Assumption DE statement Simulation Results Stability Analysis The asymptotical behaviour of the decoder in the neighborhood of δ∞ 1−R ϑ ϑ β CHANNEL P0 u u Pu0 R s BSC (µ δ−s + (1 − µ) δs )⊗ ds µ ds s BSC µ s BSC µ Pu0 = p0 (x) ⊗ (µ δ−s + (1 − µ) δs )⊗ dS Proposition 3 When close to zero error rate, stability of the LDPC constituent is not disturbed by the splitter. Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 19 / 35
  42. 42. DE Motivation and Assumption DE statement Simulation Results Stability Analysis The asymptotical behaviour of the decoder in the neighborhood of δ∞ 1−R ϑ ϑ β CHANNEL P0 u u Pu0 R s BSC (µ δ−s + (1 − µ) δs )⊗ ds µ ds s BSC µ s BSC µ Pu0 = p0 (x) ⊗ (µ δ−s + (1 − µ) δs )⊗ dS If the embedded LDPC is stable the Split-LDPC would be so. The inverse is not true ! Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 19 / 35
  43. 43. DE Motivation and Assumption DE statement Simulation Results Stability Analysis Stability Condition for Split-LDPC CHANNEL Systematic LDPC p0(x) The General Stability Condition for a systematicLDPC: B(p0 )λ (0) ρ (1) ≤ 1 +∞ where B(p0 ) = p0 (x)e−x/2 dx is the Bhattacharyya constant of the channel. −∞ Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 20 / 35
  44. 44. DE Motivation and Assumption DE statement Simulation Results Stability Analysis Stability Condition for Split-LDPC p0 (x) 1−R ϑ CHANNEL ϑ β Averaging EQUIVALENT u u CHANNEL R ds Pu0 = p0(x) ⊗ (µ δ−s + (1 − µ) δs )⊗ GLOBAL EQUIVALENT CHANNEL Systematic CORE−LDPC Peq = R pu0 + (1 − R) p0(x) The General Stability Condition for Split-LDPC: B(Peq )λ (0) ρ (1) ≤ 1 where Peq = Rc Puo (x) + (1 − Rc ) p0 (x) is the initial message density of the global equivalent channel; and B(Peq ) is the Bhattacharyya constant of the global equivalent channel. Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 20 / 35
  45. 45. DE Motivation and Assumption DE statement Simulation Results Stability Analysis The Splitter Asymptotical Properties: Proposition 4 For uniform sources, the threshold and the stability condition of the split-LDPC code are the same for the CORE-LDPC Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 21 / 35
  46. 46. DE Motivation and Assumption DE statement Simulation Results Stability Analysis The Splitter Asymptotical Properties: Proposition 4 For uniform sources, the threshold and the stability condition of the split-LDPC code are the same for the CORE-LDPC Example Split-LDPC Stability Condition for BEC Channel: 1 1 λ (0) ρ (1) < × ε [(1 − Rc ) + Rc (2 µ(1 − µ))ds ] Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 21 / 35
  47. 47. DE Motivation and Assumption DE statement Simulation Results Stability Analysis The Splitter Asymptotical Properties: Proposition 4 For uniform sources, the threshold and the stability condition of the split-LDPC code are the same for the CORE-LDPC Example Split-LDPC Stability Condition for BSC Channel: 1 1 λ (0) ρ (1) < × 2 λ(1 − λ) [(1 − Rc ) + Rc (2 µ(1 − µ))ds ] Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 21 / 35
  48. 48. DE Motivation and Assumption DE statement Simulation Results Stability Analysis The Splitter Asymptotical Properties: Proposition 4 For uniform sources, the threshold and the stability condition of the split-LDPC code are the same for the CORE-LDPC Example Split-LDPC Stability Condition for AWGN Channel:: 1 1 λ (0) ρ (1) < e 2σ2 × [(1 − Rc ) + Rc (2 µ(1 − µ))ds ] Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 21 / 35
  49. 49. Part III EXIT Chart Analysis for Split-LDPC Codes
  50. 50. Motivation and Assumptions Split-LDPC EXIT Chart Statement Design of Irregular Split-LDPC Results Motivation and Proposal The problem High computational complexity for Density Evolution Low Complexity Approaches based on Gaussian Approximation Lower Complexity and one-dimensional More Insightful Less Accurate How to build capacity achieving Split-LDPC ? The Proposal: Message oriented Bi-dimensional low complexity approach based on a more accurate Exit Chart method. Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 22 / 35
  51. 51. Motivation and Assumptions Split-LDPC EXIT Chart Statement Design of Irregular Split-LDPC Results Assumptions and Notations: Message-oriented local tree assumption. p1 α u ds − 1 db η(x) xλ(x) α α β β p1 R p2 + (1 − R) p3 ds − 1 dc − 1 1 1 1 u u 2 u + 2ϑ 2 u + 1ϑ 2 Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 23 / 35
  52. 52. Motivation and Assumptions Split-LDPC EXIT Chart Statement Design of Irregular Split-LDPC Results Assumptions and Notations: Message-oriented local tree assumption. Consistency Condition is realized on all types of message distribution. f (x) = f (−x) ex for all x ∈ R+ Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 23 / 35
  53. 53. Motivation and Assumptions Split-LDPC EXIT Chart Statement Design of Irregular Split-LDPC Results Assumptions and Notations: Message-oriented local tree assumption. Consistency Condition is realized on all types of message distribution. Gaussian approximation is applied at the bitnodes output based on equal mutual information. Uout U0 Uout f f −1 Uin U0 Uin Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 23 / 35
  54. 54. Motivation and Assumptions Split-LDPC EXIT Chart Statement Design of Irregular Split-LDPC Results Assumptions and Notations: Message-oriented local tree assumption. Consistency Condition is realized on all types of message distribution. Gaussian approximation is applied at the bitnodes output based on equal mutual information. We display the error probability as a measure of knowledge U . Uout U0 +∞ σ 1 t2 Perr = Q( ) = √ e− 2 dt 2 2π σ/2 Uout f f −1 Uin U0 Uin Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 23 / 35
  55. 55. Motivation and Assumptions Split-LDPC EXIT Chart Statement Design of Irregular Split-LDPC Results The EXIT Chart Statement for Split-LDPC : Message Combining β ∗ P2 = (1 − R)P3 + RP2 β P2 P3 u ϑ ds db − 1 db − 1 xη(x) λ(x) Pin2 ∗ β β β α α β β Pin1 ∗ dc − 1 ds − 1 dc − 1 1 1 1 u u 1 u + 1ϑ 1 u + 1ϑ 2u + 2ϑ 2u + 1ϑ 2 2 2 2 2 ∗ ∗ ∗ Pout2 = G(Pin1 , Pin2 ) Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 24 / 35
  56. 56. Motivation and Assumptions Split-LDPC EXIT Chart Statement Design of Irregular Split-LDPC Results The EXIT Chart Statement for Split-LDPC : Message Combining β ∗ P2 = (1 − R)P3 + RP2 β P2 P3 u ϑ ds db − 1 db − 1 xη(x) λ(x) Pin2 ∗ β β β α α β β Pin1 ∗ dc − 1 ds − 1 dc − 1 1 1 1 u u 1 u + 1ϑ 1 u + 1ϑ 2u + 2ϑ 2u + 1ϑ 2 2 2 2 2 db db G(x, y) = Rc × λj fds +1,j (x, y) + (1 − Rc ) × λj gj (y) j=2 j=2 Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 24 / 35
  57. 57. Motivation and Assumptions Split-LDPC EXIT Chart Statement Design of Irregular Split-LDPC Results The EXIT Chart Statement for Split-LDPC : Message Combining ∗ α P1 u ds − 1 db η(x) xλ(x) α α β β Pin2 ∗ Pin1 ∗ ds − 1 dc − 1 u u 1 u + 1ϑ 1 u + 1ϑ 2 2 2 2 ∗ ∗ ∗ Pout1 = F (Pin1 , Pin2 ) Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 24 / 35
  58. 58. Motivation and Assumptions Split-LDPC EXIT Chart Statement Design of Irregular Split-LDPC Results The EXIT Chart Statement for Split-LDPC : Message Combining ∗ α P1 u ds − 1 db η(x) xλ(x) α α β β Pin2 ∗ Pin1 ∗ ds − 1 dc − 1 u u 1 u + 1ϑ 1 u + 1ϑ 2 2 2 2 db F (x, y) = λj fds ,j+1 (x, y) j=2 Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 24 / 35
  59. 59. Motivation and Assumptions Split-LDPC EXIT Chart Statement Design of Irregular Split-LDPC Results The validity Range for the Bidimensional Dynamical System ∗ ∗ The mixtures of Gaussian densities associated to inputs Pin1 and Pin2 must satisfy the two equalities: db ∗ 1 1 Pin1 = λj erf c m0 + (ds − 1)mα + jmβ j=2 2 2 db ∗ 1 1 Pin2 = Rc × λj erf c m0 + ds mα + (j − 1)mβ j=2 2 2 db 1 1 + (1 − Rc ) × λj erf c m0 + (j − 1)mβ j=2 2 2 m0 : Mean of messages from the channel. mα : Mean of messages from the splitter checknodes α. mβ : Mean of messages from the LDPC checknodes β. Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 25 / 35
  60. 60. Motivation and Assumptions Split-LDPC EXIT Chart Statement Design of Irregular Split-LDPC Results Trajectory of error probability near the code threshold Es Right to the threshold: = −5.58dB, Threshold=−5.68dB. Final fixed point is 0. N0 Transfer Function F(x,y) Trajectory of Pout1 Pout1 0.20 0.25 0.15 0.20 0.10 0.15 0.10 0.05 0.05 0.00 0.00 0.20 0.25 0.15 0.20 0.15 0.10 Pin1 0.10 0.05 Pin2 0.05 0.00 Illustration for an irregular Split-LDPC code. Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 26 / 35
  61. 61. Motivation and Assumptions Split-LDPC EXIT Chart Statement Design of Irregular Split-LDPC Results Trajectory of error probability beyond the code threshold (2) Es Left to the threshold: = −5.78dB, Threshold=−5.68dB. Final fixed point is non-zero. N0 Transfer Function F(x,y) Trajectory of Pout1 Pout1 0.20 0.15 0.25 0.20 0.10 0.15 0.05 0.10 0.05 0.00 0.00 0.20 0.25 0.15 0.20 0.10 Pin1 0.15 0.10 0.05 Pin2 0.05 0.00 Illustration for an irregular Split-LDPC code. Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 27 / 35
  62. 62. Motivation and Assumptions Split-LDPC EXIT Chart Statement Design of Irregular Split-LDPC Results Open Tunnel for the Bi-dimensional EXIT Chart Open tunnel obtained by plotting the trajectory of error probability and its z = x plane Es reflection at = −5.58dB . N0 Transfer Function F(x,y) Pout1 Trajectory of Pout1 Inverse of Trajectory of Pout1 Plan z=x 0.18 0.25 0.16 0.20 0.14 0.12 0.15 0.10 0.08 0.10 0.06 0.04 0.05 0.02 0.00 0.00 0.25 0.20 0.15 0.10 Pin2 0.00 0.05 0.10 Pin1 0.15 0.20 0.05 Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 28 / 35
  63. 63. Motivation and Assumptions Split-LDPC EXIT Chart Statement Design of Irregular Split-LDPC Results Design Irregular Split-LDPC codes The Design linear Program: 1 Maximize λi /i i≥2 2 subject to : λi ≥ 0, λi = 1 i≥2 ∗ ∗ 3 and ∀(P1in , P2in ) ∈ T (S) db ∗ ∗ j=2 λj fds ,j+1 (P1in , P2in∗ ) < P1in db ∗ ∗ ∗ ∗ j=2 λj [Rc × fds +1,j (P1in , P2in ) + (1 − Rc ) × gj (P2in )] < P2in Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 29 / 35
  64. 64. Motivation and Assumptions Split-LDPC EXIT Chart Statement Design of Irregular Split-LDPC Results Design Irregular Split-LDPC codes The Design Process: Select a regular code with the desired coding rate. ∗ ∗ Use the regular code degree sequence for mapping (P1in , P2in ) to the appropriate input Gaussian mixture densities. Find the EXIT charts surfaces for different variable degrees. Find a linear combination with an open EXIT chart that maximizes the rate and meets all the required design criterion. Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 29 / 35
  65. 65. Motivation and Assumptions Split-LDPC EXIT Chart Statement Design of Irregular Split-LDPC Results Design Irregular Split-LDPC codes The Design Process: Select a regular code with the desired coding rate. ∗ ∗ Use the regular code degree sequence for mapping (P1in , P2in ) to the appropriate input Gaussian mixture densities. Find the EXIT charts surfaces for different variable degrees. Find a linear combination with an open EXIT chart that maximizes the rate and meets all the required design criterion. Best Approach to Shannon limits is within 0.1 dB ! Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 29 / 35
  66. 66. Motivation and Assumptions Split-LDPC EXIT Chart Statement Design of Irregular Split-LDPC Results Bi-dimensional EXIT Chart Accuracy Error in approximation of threshold (dB) using EXIT chart analysis for various split-LDPC codes of Rate one-half with ds = 3 , Hs = 0.5. ∆ is the log-ratio quantization step. Eb /N0 ∗ (dB) Eb /N0 ∗ (dB) Error ∆Eb /N0 ∗ db dc Rate DE ∆ = 0.005 EC ∆ = 0.005 for ∆ = 0.005 3 6 0.5 −2.22 −2.199 0.020 4 8 0.5 −1.12 −1.129 0.009 5 10 0.5 −0.32 −0.369 0.049 Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 30 / 35
  67. 67. Part IV EM Source-Channel Estimation
  68. 68. Motivation and Proposal Expectation-Maximization Complex BIAWGN Channel Simultations Results Motivation and Proposal Motivation Full utilization of systematic and non-systematic LDPC codes requires: 1 Knowledge of the source probability distribution (SSI) at the decoder side. 2 Knowledge of channel parameters (CSI) at the decoder side. Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 31 / 35
  69. 69. Motivation and Proposal Expectation-Maximization Complex BIAWGN Channel Simultations Results Motivation and Proposal Motivation Full utilization of systematic and non-systematic LDPC codes requires: 1 Knowledge of the source probability distribution (SSI) at the decoder side. 2 Knowledge of channel parameters (CSI) at the decoder side. Proposal Joint Source-Channel Iterative Estimation and Decoding for non-uniform sources based on the Expectation Maximization Algorithm (EM) → No performance loss → Negligible estimation complexity Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 31 / 35
  70. 70. Motivation and Proposal Expectation-Maximization Complex BIAWGN Channel Simultations Results Brief statement of the EM algorithm κ : complete data, κ = (x, y) 0.9 x:missing data y: incomplete data (observed) p(s = 1) = µ 0.1 0 1 x y Source Channel Encoder Channel Channel Decoder Sink SSI = µ CSI Θ=SSI+CSI set of parameters to be estimated, SSI+CSI ˆ Θ E-step: Compute the Auxiliary function Q: Q(θ|θi ) = E[log p(x, y|θ)|y, θi ] = log[P (y|x, θ) P (x|θ)] AP Pi (x) x Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 32 / 35
  71. 71. Motivation and Proposal Expectation-Maximization Complex BIAWGN Channel Simultations Results Brief statement of the EM algorithm κ : complete data, κ = (x, y) 0.9 x:missing data y: incomplete data (observed) p(s = 1) = µ 0.1 0 1 x y Source Channel Encoder Channel Channel Decoder Sink SSI = µ CSI Θ=SSI+CSI set of parameters to be estimated, SSI+CSI ˆ Θ E-step: Compute the Auxiliary function Q: Q(θ|θi ) = E[log p(x, y|θ)|y, θi ] = log[P (y|x, θ) P (x|θ)] AP Pi (x) x The SSI part in the auxiliary function is: P (x|θ) ≡ P (s|θ) = µωH (s) (1 − µ)K−ωH (s) Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 32 / 35
  72. 72. Motivation and Proposal Expectation-Maximization Complex BIAWGN Channel Simultations Results Brief statement of the EM algorithm κ : complete data, κ = (x, y) 0.9 x:missing data y: incomplete data (observed) p(s = 1) = µ 0.1 0 1 x y Source Channel Encoder Channel Channel Decoder Sink SSI = µ CSI Θ=SSI+CSI set of parameters to be estimated, SSI+CSI ˆ Θ E-step: Compute the Auxiliary function Q: Q(θ|θi ) = E[log p(x, y|θ)|y, θi ] = log[P (y|x, θ) P (x|θ)] AP Pi (x) x M-step: θ i+1 = arg max Q(θ|θ i ) θ Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 32 / 35
  73. 73. Motivation and Proposal Expectation-Maximization Complex BIAWGN Channel Simultations Results Joint Source-Channel Estimation on Complex BIAWGN The Complex BIAWGN channel is defined as φ √ yi = Ae xi + ηi with = −1 where the three CSI parameters are : The amplitude A , which is real positive The phase ambiguity φ, which is uniformly distributed between 0 and 2π. The Gaussian noise variance σ 2 . Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 33 / 35
  74. 74. Motivation and Proposal Expectation-Maximization Complex BIAWGN Channel Simultations Results Joint Source-Channel Estimation on Complex BIAWGN The Complex BIAWGN channel is defined as φ √ yi = Ae xi + ηi with = −1 E-step: Auxiliary Function µ Q(θ|θi ) = log[ ]˜ + K log[(1 − µ)] − N log[2πσ 2 ] s 1−µ N N N 1 2 A2 2 A − yj − xj + R xj ∗ e− ˜ φ yj 2σ 2 j=1 2σ 2 j=1 σ2 j=1 Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 33 / 35
  75. 75. Motivation and Proposal Expectation-Maximization Complex BIAWGN Channel Simultations Results Joint Source-Channel Estimation on Complex BIAWGN The Complex BIAWGN channel is defined as φ √ yi = Ae xi + ηi with = −1 M-step: Maximizing the auxiliary function for SSI K i+1 j=1 sj ˜ µ = K Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 33 / 35
  76. 76. Motivation and Proposal Expectation-Maximization Complex BIAWGN Channel Simultations Results Joint Source-Channel Estimation on Complex BIAWGN The Complex BIAWGN channel is defined as φ √ yi = Ae xi + ηi with = −1 M-step: Maximizing the auxiliary function for CSI N φi j=1 ˜ ∗ R xj yj e Ai+1 = N 2 j=1 xj AP Pi (xj ) xj N 2 (i+1) 1 φi x 2 σ = yj − Ai e j 2N j=1 N φi+1 = − Arg ˜ ∗ xj yj j=1 Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 33 / 35
  77. 77. Motivation and Proposal Expectation-Maximization Complex BIAWGN Channel Simultations Results 100 Systematic + Uniform Systematic + Non Uniform (Perfect) Systematic + Non Uniform (EM observation) Systematic + Non Uniform (EM APP) Non Systematic + Non Uniform (Perfect) 10-1 Non Systematic + Non Uniform (EM APP) Frame Error Rate 10-2 10-3 10-4 -2.4 -2 -1.6 -1.2 -0.8 -0.4 0 0.4 0.8 1.2 1.6 2 2.4 Eb/N0 (dB) Systematic and non-systematic LDPC codes on AWGN, Rc = 1/2. Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 34 / 35
  78. 78. Conclusions and Open Issues Conclusions The theoretical limits for redundant data are mooving to better regions. Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 35 / 35
  79. 79. Conclusions and Open Issues Conclusions The theoretical limits for redundant data are mooving to better regions. Split-LDPC codes are the best non systematic LDPC constructions. Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 35 / 35
  80. 80. Conclusions and Open Issues Conclusions The theoretical limits for redundant data are mooving to better regions. Split-LDPC codes are the best non systematic LDPC constructions. Split-LDPC codes have good asymptotical performance and properties. Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 35 / 35
  81. 81. Conclusions and Open Issues Conclusions The theoretical limits for redundant data are mooving to better regions. Split-LDPC codes are the best non systematic LDPC constructions. Split-LDPC codes have good asymptotical performance and properties. Designed irregular Split-LDPC codes are capacity achieving. Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 35 / 35
  82. 82. Conclusions and Open Issues Conclusions The theoretical limits for redundant data are mooving to better regions. Split-LDPC codes are the best non systematic LDPC constructions. Split-LDPC codes have good asymptotical performance and properties. Designed irregular Split-LDPC codes are capacity achieving. The 2-D EXIT Chart formalism is lower complexity and as accurate as the DE. Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 35 / 35
  83. 83. Conclusions and Open Issues Conclusions The theoretical limits for redundant data are mooving to better regions. Split-LDPC codes are the best non systematic LDPC constructions. Split-LDPC codes have good asymptotical performance and properties. Designed irregular Split-LDPC codes are capacity achieving. The 2-D EXIT Chart formalism is lower complexity and as accurate as the DE. The performance using EM is as good as the perfect knowledge case. Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 35 / 35
  84. 84. Conclusions and Open Issues Open Issues Investigating more complex source and channel models. Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 36 / 35
  85. 85. Conclusions and Open Issues Open Issues Investigating more complex source and channel models. Extending our study to irregular splitting and scrambling LDPC. Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 36 / 35
  86. 86. Conclusions and Open Issues Open Issues Investigating more complex source and channel models. Extending our study to irregular splitting and scrambling LDPC. Performing Complexity and Finite-length Optimization. Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 36 / 35
  87. 87. Conclusions and Open Issues Open Issues Investigating more complex source and channel models. Extending our study to irregular splitting and scrambling LDPC. Performing Complexity and Finite-length Optimization. Consider lossy sources assumption . Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 36 / 35
  88. 88. Conclusions and Open Issues Open Issues Investigating more complex source and channel models. Extending our study to irregular splitting and scrambling LDPC. Performing Complexity and Finite-length Optimization. Consider lossy sources assumption . Studying and incorporating Split-LDPC in practical communication systems. Amira A LLOUM Construction and Analysis of Non-Systematic Codes on Graphs for Redundant Data September 5th, 2008 36 / 35

×