Optimized Codes for the Binary Coded Side-Information Problem Anne Savard, Claudio Weidmann ETIS / ENSEA - Université de Cergy-Pontoise - CNRS UMR 8051 F-95000 Cergy-Pontoise Cedex, France
Outline 1 Introduction 2 Proposed method 3 Code optimization 4 Conclusion Optimized Codes for the Binary Coded Side-Information Problem A. Savard 19 August 2014 2 / 26
Introduction Coded side information Outline 1 Introduction Coded side information Standard decoder setup using LDPC codes 2 Proposed method 3 Code optimization 4 Conclusion Optimized Codes for the Binary Coded Side-Information Problem A. Savard 19 August 2014 3 / 26
Introduction Coded side information Coded side information problem: binary case 2 correlated discrete binary sources X and Y, separately encoded by E X and E Y at rates R X and R Y D X tries to reconstruct X losslessly Symmetric correlation channel Binary quantization with Hamming distortion is sufficient to achieve the rate region 1 1 W. Gu, R. Koetter, M. Effros and T. Ho, "On source coding with coded side information for a binary source with binary side information", ISIT 2007, Nice, France, June 24 - June 29, pp. 1456-1460, 2007 Rate region X E X S D X ˆX R X h(p + D 2pD) R Y 1 h(d) BSC-p Y BSC-D Ŷ Optimized Codes for the Binary Coded Side-Information Problem A. Savard 19 August 2014 4 / 26
Introduction Coded side information Coded side information problem: binary case 2 correlated discrete binary sources X and Y, separately encoded by E X and E Y at rates R X and R Y D X tries to reconstruct X losslessly Symmetric correlation channel Binary quantization with Hamming distortion is sufficient to achieve the rate region 1 1 W. Gu, R. Koetter, M. Effros and T. Ho, "On source coding with coded side information for a binary source with binary side information", ISIT 2007, Nice, France, June 24 - June 29, pp. 1456-1460, 2007 Rate region X E X S D X ˆX R X h(p + D 2pD) R Y 1 h(d) BSC-p Y BSC-D Ŷ Can be reformulated as Slepian-Wolf coding of X with Ŷ as side information Optimized Codes for the Binary Coded Side-Information Problem A. Savard 19 August 2014 4 / 26
Introduction Standard decoder setup using LDPC codes Outline 1 Introduction Coded side information Standard decoder setup using LDPC codes 2 Proposed method 3 Code optimization 4 Conclusion Optimized Codes for the Binary Coded Side-Information Problem A. Savard 19 August 2014 5 / 26
Introduction Standard decoder setup using LDPC codes Standard setup using LDPC codes Compression of Y n : Trellis-coded quantizer Compute quantization index W Using Viterbi algorithm on convolutional code trellis Reconstruction of W : codeword Ŷ n (W ) Slepian-Wolf with LDPC codes as proposed by Liveris et al. 1 Compression of X n = (X 1, X 2,..., X n) Computation of the syndrome S n k = X n H T H parity check matrix of an LDPC code 1 A. D. Liveris, Z. Xiong and C. N.Georghiades, "Compression of binary sources with side information at the decoder using LDPC codes", IEEE Communications Letters, vol. : 6, pp. 440-442, 2002 Optimized Codes for the Binary Coded Side-Information Problem A. Savard 19 August 2014 6 / 26
Proposed method Principle of the proposed method Outline 1 Introduction 2 Proposed method Principle of the proposed method Characterization of Voronoi cells Decoder Voronoi decoder: BCJR 3 Code optimization 4 Conclusion Optimized Codes for the Binary Coded Side-Information Problem A. Savard 19 August 2014 7 / 26
Proposed method Principle of the proposed method Principle of the proposed method Many sequences are mapped onto the same quantization index w: they form the Voronoi cell V w x Ŷ (w) y V w Geometrical intuition Optimized Codes for the Binary Coded Side-Information Problem A. Savard 19 August 2014 8 / 26
Proposed method Principle of the proposed method Principle of the proposed method Many sequences are mapped onto the same quantization index w: they form the Voronoi cell V w x Ŷ (w) y V w ˆx (t) Geometrical intuition Optimized Codes for the Binary Coded Side-Information Problem A. Savard 19 August 2014 8 / 26
Proposed method Principle of the proposed method Principle of the proposed method Many sequences are mapped onto the same quantization index w: they form the Voronoi cell V w Projection of an intermediate solution ˆx (t) on V w : Ŷ (w) (t+1) x Ŷ (w) Ŷ (w) (t+1) y ˆx (t) V w Geometrical intuition Optimized Codes for the Binary Coded Side-Information Problem A. Savard 19 August 2014 8 / 26
Proposed method Principle of the proposed method Principle of the proposed method Many sequences are mapped onto the same quantization index w: they form the Voronoi cell V w Projection of an intermediate solution ˆx (t) on V w : Ŷ (w) (t+1) Use to modify LLRs before continuing LDPC decoder iterations Goal: accelerate decoding of x x Ŷ (w) Ŷ (w) (t+1) y ˆx (t) V w Geometrical intuition Presented at ITW 2013 (A. Savard and C. Weidmann, Improved decoding for binary source coding with coded side information, ITW 2013) Optimized Codes for the Binary Coded Side-Information Problem A. Savard 19 August 2014 8 / 26
Proposed method Principle of the proposed method Principle of the proposed method Many sequences are mapped onto the same quantization index w: they form the Voronoi cell V w Projection of an intermediate solution ˆx (t) on V w : Ŷ (w) (t+1) Use to modify LLRs before continuing LDPC decoder iterations Goal: accelerate decoding of x x Ŷ (w) Ŷ (w) (t+1) y ˆx (t) V w Geometrical intuition Presented at ITW 2013 (A. Savard and C. Weidmann, Improved decoding for binary source coding with coded side information, ITW 2013) Characterization of Voronoi cells Properties of linear codes: V w = V 0 Ŷ (w) V 0 is characterized by a modified decoder state machine 1 1 A. R. Calderbank, P. C. Fishburn and A. Rabinovich, "Covering properties of convolutional codes and associated lattices", IEEE Trans. on Information Theory, vol. : 41, pp. 732-746, 1995 Optimized Codes for the Binary Coded Side-Information Problem A. Savard 19 August 2014 8 / 26
Proposed method Characterization of Voronoi cells Outline 1 Introduction 2 Proposed method Principle of the proposed method Characterization of Voronoi cells Decoder Voronoi decoder: BCJR 3 Code optimization 4 Conclusion Optimized Codes for the Binary Coded Side-Information Problem A. Savard 19 August 2014 9 / 26
Proposed method Example Characterization of Voronoi cells 0 11 00 0 1 01 10 1 Figure: Convolutional code (1, 1 + D) with 2 states Future evolution of Viterbi decoder depends only on the metric differences Define a metric state [m 0 m, m 1 m,..., m 2 ν m], where m = min{m i } For Hamming metric, the number of metric states is finite Optimized Codes for the Binary Coded Side-Information Problem A. Savard 19 August 2014 10 / 26
Proposed method Characterization of Voronoi cells 10 Decoder Markov chain [2, 0] 11 10 00,11 00,11 11 00,01 Stochastic automaton (Markov chain) describing decoder trajectories [1, 0] 10,11 [0, 0] [0, 1] 01,10 Yields average probability of error (channel decoding) and average distortion (source coding) 01 01,10 00 01 Edges have n-bit labels (n: output width of convolutional encoder) [0, 2] 00 NB. non-uniform branch probabilities in general Optimized Codes for the Binary Coded Side-Information Problem A. Savard 19 August 2014 11 / 26
Proposed method Characterization of Voronoi cells Graph of Markov chain for Voronoi cell V 0 Necessary condition The edge (state 0) (state 0) labeled with the all-zero word must have a winning metric (on the Viterbi decoder trellis). 00 [0,2] 00 [0,1] 01 10 10 [0,0] 01 00 10 00 [1,0] Optimized Codes for the Binary Coded Side-Information Problem A. Savard 19 August 2014 12 / 26
Proposed method Decoder Outline 1 Introduction 2 Proposed method Principle of the proposed method Characterization of Voronoi cells Decoder Voronoi decoder: BCJR 3 Code optimization 4 Conclusion Optimized Codes for the Binary Coded Side-Information Problem A. Savard 19 August 2014 13 / 26
Proposed method Decoder Principle of the proposed decoder E X : syndrome s, E Y : index w Codeword Ŷ (w) T BP decoding iterations In case of failure : Run Voronoi decoder (projection onto V 0 ) Modification of the LLRs t additional LDPC decoder iterations S 1 + S 2 +... S n k 1 + S n k +... X 1 X 2 X n 1 X n Voronoi decoder Ŷ 1 (w) Ŷ 2 (w)... Ŷ n 1 (w) Ŷn(w) Proposed decoder graph Optimized Codes for the Binary Coded Side-Information Problem A. Savard 19 August 2014 14 / 26
Proposed method Voronoi decoder based on the BCJR algorithm Outline 1 Introduction 2 Proposed method Principle of the proposed method Characterization of Voronoi cells Decoder Voronoi decoder: BCJR 3 Code optimization 4 Conclusion Optimized Codes for the Binary Coded Side-Information Problem A. Savard 19 August 2014 15 / 26
Proposed method Voronoi decoder based on the BCJR algorithm Principle Soft input, soft output decoder 1 Inputs : { zi LLRi BCJR if = Ŷi(w) = 0 z i if Ŷi(w) = 1 z: scaled version of the extrinsic from the LDPC decoder z i = The extrinsic must be weakened according to the reliability of ˆx (T ) c j N (v i ) LLR for the next iterations : ( ( ) ( LLR i = 2 tanh (tanh 1 extri log 1 p ))) p tanh 2 2 m c v j,i 1 L. R. Bahl, J. Cocke, F. Jelinek and J. Raviv, "Optimal decoding of linear codes for minimizing symbol error rate", IEEE Trans. on Information Theory, vol. : IT-20, pp. 284-287, 1974 Optimized Codes for the Binary Coded Side-Information Problem A. Savard 19 August 2014 16 / 26
Code optimization Density evolution for our improved decoder Outline 1 Introduction 2 Proposed method 3 Code optimization Density evolution for our improved decoder Algorithm used Results 4 Conclusion Optimized Codes for the Binary Coded Side-Information Problem A. Savard 19 August 2014 17 / 26
Code optimization Density evolution for our improved decoder Principle Numerical density evolution along the lines of the approach by Kavcic et al. 1 1 A. Kavcic, X. Ma and M. Mitzenmacher, "Binary intersymbol interference channel: Gallager codes, density evolution, and code performance bounds", IEEE Trans. on Information Theory, vol. : 49, pp. 1636-1652, 2003 Notations Input: degree distributions λ and ρ f (l) v : pdf of message from a VN to a CN at l-th iteration f (l) c : pdf of message from a CN to a VN at l-th iteration f (l) o : pdf of a priori LLR at the l-th iteration f (l) e : pdf of extrinsic given to the BCJR at l-th iteration Optimized Codes for the Binary Coded Side-Information Problem A. Savard 19 August 2014 18 / 26
Code optimization Density evolution for our improved decoder Computation of f (l) o Initialize with BSC-ɛ model from X to Ŷ (ɛ = p + D 2pD) f (1) o = ɛδ ( x + log ( ) ) 1 ɛ + (1 ɛ)δ( x log ɛ { (l) f f (l+1) o, if iteration index l kt o = ɛ trellis (f (l) e, p), else ɛ trellis : symbolic notation for trellis evolution No closed-form expression for this evolution Computed numerically using Monte-Carlo techniques ( ) ) 1 ɛ ɛ Optimized Codes for the Binary Coded Side-Information Problem A. Savard 19 August 2014 19 / 26
Code optimization Algorithm used Outline 1 Introduction 2 Proposed method 3 Code optimization Density evolution for our improved decoder Algorithm used Results 4 Conclusion Optimized Codes for the Binary Coded Side-Information Problem A. Savard 19 August 2014 20 / 26
Code optimization Algorithm used Density evolution with BCJR Voronoi decoder Algorithm 1 Density evolution with BCJR Voronoi decoder 1: Initialization 2: Density evolution: Phase 1 iter T 3: for i proj do 4: Trellis evolution: f (l+1) o = ɛ trellis (f (l) e, p) 5: Density evolution: Phase 2 iter t 6: end for Trellis evolution Need to simulate general inputs: all-zero codeword is not sufficient due to nonlinearity of the decoder state machine Need to take into account the reliability of the estimate ˆx t Optimized Codes for the Binary Coded Side-Information Problem A. Savard 19 August 2014 21 / 26
Code optimization Results Outline 1 Introduction 2 Proposed method 3 Code optimization Density evolution for our improved decoder Algorithm used Results 4 Conclusion Optimized Codes for the Binary Coded Side-Information Problem A. Savard 19 August 2014 22 / 26
Code optimization Results Example setup Quantizer built with a rate-5/6 convolutional code from Tang et al. 1 Rate-1/2 LDPC code ensemble with variable degree distribution λ(x) = 0.094167x 2 + 0.7275x 3 + 0.0125x 5 + 0.045x 6 + 0.00417x 10 + 0.0317x 14 + 0.0233x 15 + 0.000833x 16 + 0.04583x 19 + 0.015x 20 and concentrated check-node degrees obtained with differential evolution Decoding threshold: p = 0.06 n = 1200 10000 samples 1 H.-H. Tang and M.-C. Lin, On (n, n-1) convolutional codes with low trellis complexity, IEEE Trans. Commun., vol. 50, pp. 37 47, 2002 Optimized Codes for the Binary Coded Side-Information Problem A. Savard 19 August 2014 23 / 26
Code optimization Results 10000 9000 standard method without optimization proposed method without optimization standard method with optimization proposed method with optimization 8000 7000 Number of successes 6000 5000 4000 3000 2000 1000 0 0.02 0.03 0.04 0.05 0.06 0.07 0.08 Crossover probability p Number of decoding successes as a function of the crossover probability p. Optimized Codes for the Binary Coded Side-Information Problem A. Savard 19 August 2014 24 / 26
Conclusion Conclusion Knowledge of the Voronoi cell V 0 helps coded side information decoder LDPC code optimization of the improved iterative decoder presented at ITW 2013 with density evolution using Monte Carlo simulations Optimized codes beat standard codes for the BSC, since they are better adapted to the quantizer characteristics Optimized code performance still limited by approximations in decoder (accounting for reliability of ˆx) Optimized Codes for the Binary Coded Side-Information Problem A. Savard 19 August 2014 25 / 26
Conclusion Thank you for your attention. Questions? Optimized Codes for the Binary Coded Side-Information Problem A. Savard 19 August 2014 26 / 26