Decoding Turbo Codes and LDPC Codes via Linear Programming

Transcription

1 Decoding Turbo Codes and LDPC Codes via Linear Programming Jon Feldman David Karger MIT LCS Martin Wainwright UC Berkeley MIT LCS J Feldman, Decoding Turbo Codes and LDPC Codes via Linear Programming p1/22

2 nts Binary Error-Correcting Code information word Encoder code word Noisy Channel decoded info decoded code word corrupt code word Decoder Binary Symmetric Channel (BSC): each bit flipped independently with probability (small constant) J Feldman, Decoding Turbo Codes and LDPC Codes via Linear Programming p2/22

3 Turbo Codes + LDPC Codes Low-Density Parity-Check (LDPC) codes [Gal 62] Turbo Codes introduced [BGT 93], unprecedented error-correcting performance Ensuing LDPC Renaissance [SS 94, MN 95, Wib 96, MMC 98, Yed 02, ] Simple encoder, belief-propagation decoder Theoretical understanding of good performance: - Threshold as [LMSS 01, RU 01]; - Decoder unpredictable with cycles Finite-length analysis: combinatorial error conditions known only for the binary erasure channel [DPRTU 02] J Feldman, Decoding Turbo Codes and LDPC Codes via Linear Programming p3/22

4 Our contributions [FK, FOCS 02] [FKW, Allerton 02] [FKW, CISS 03] Poly-time decoder using LP relaxation Decodes: binary linear codes turbo codes LDPC codes Pseudocodewords: exact characterization of error patterns causing failure Fractional distance : - LP decoding corrects up to errors - Computable efficiently for turbo, LDPC codes Error rate bounds based on high-girth graphs Closely related to iterative approaches, other notions of pseudocodewords J Feldman, Decoding Turbo Codes and LDPC Codes via Linear Programming p4/22

5 Outline Error correcting codes Using LP relaxation for decoding Details of LP relaxation for binary linear codes Pseudocodewords Fractional Distance Girth-based bounds J Feldman, Decoding Turbo Codes and LDPC Codes via Linear Programming p5/22

6 Maximum-Likelihood Decoding Code Cost function BSC: Other channels: : negative log-likelihood ratio of if Given: Corrupt code word Find: such that, if takes on arbitrary soft values is minimized Linear Programming formulation: - Variables for each code bit, - Linear Program: J Feldman, Decoding Turbo Codes and LDPC Codes via Linear Programming p6/22

7 Linear Programming Relaxation Polytope : relaxation, Decoder: Solve LP using simplex/ellipsoid If, output, else output error ML certificate property: all outputs ML codewords Want low word error rate (WER) := ents no noise 011 noisy 000 No noise: [ : optimal Noise: perturbation of objective function ] Design code, relaxation accordingly J Feldman, Decoding Turbo Codes and LDPC Codes via Linear Programming p7/22

8 ents Tanner Graph The Tanner Graph of a linear code is a bipartite graph modeling the parity check matrix of the code Variable nodes Check Nodes : n hood of check Code words: st: Codewords: , , , etc J Feldman, Decoding Turbo Codes and LDPC Codes via Linear Programming p8/22

9 IP/LP Formulation of ML Decoding for each code bit Variables LP: IP: = valid configurations of, For check bit, for each check node Variables ents LP: IP: Vars:,,,,,,, J Feldman, Decoding Turbo Codes and LDPC Codes via Linear Programming p9/22

10 IP/LP Formulation of ML Decoding Minimize, subject to: Let be the relaxed polytope IP: formulation of ML decoding What do fractional solutions look like? J Feldman, Decoding Turbo Codes and LDPC Codes via Linear Programming p10/22

11 Fractional Solutions Suppose: ML codeword: ML codeword cost: Frac sol: ents Frac sol cost: J Feldman, Decoding Turbo Codes and LDPC Codes via Linear Programming p11/22

12 LP Decoding Success Conditions Pr[ Decoding Success ] = Pr[ is the unique OPT ] Assume - Common asssumption for linear codes - OK in this case due to symmetry of polytope Pr[ is the unique OPT ] = Pr[ All other solutionss have cost > 0] Theorem [FKW, CISS 03]: Assume the allzeros codeword was sent Then, the LP decodes correctly i ff all non-zero points in P have positive cost J Feldman, Decoding Turbo Codes and LDPC Codes via Linear Programming p12/22

13 Pseudocodewords Pseudocodewords are scaled points in Previous example: Scaled to integers: Natural combinatorial definition of pseudocodeword (independent of LP relaxation) Theorem [FKW, CISS 03]: LP decodes correctly i ff all pseudocodewords have cost J Feldman, Decoding Turbo Codes and LDPC Codes via Linear Programming p13/22

14 Fractional Distance Classical distance: - = min Hamming dist of codewords in Adversarial performance bound: - ML decoding can correct errors Another way to define minimum distance: - = min ( ) dist between two integral verts of Fractional distance: - = min ( ) dist between an integral and a fractional vertex of - = min wt fractional vertex of - Lower bound on classical distance: - LP Decoding can correct errors J Feldman, Decoding Turbo Codes and LDPC Codes via Linear Programming p14/22

15 errors LP Decoding corrects errors occur Suppose fewer than be a vertex of, Let flipped, +1 ow; So, if, When Since Therefore J Feldman, Decoding Turbo Codes and LDPC Codes via Linear Programming p15/22

16 Computing the Fractional Distance Computing for linear/ldpc codes is NP-hard If the polytope has small size (LDPC), the fractional distance is easily computed - More general problem: Given an LP, find the two best vertices - Algorithm: * Find * Guess the facet on which sits but does not * Set facet to equality, obtaining * Minimize over Good approximation to the classical distance? Good prediction of relative classical distance? J Feldman, Decoding Turbo Codes and LDPC Codes via Linear Programming p16/22

17 Using Girth for Error Bounds For rate-1/2 RA (cycle) codes: If has large girth, neg-cost pseudocodewords (promenades) are rare Erdös (or [BMMS 02]): Hamiltonian 3-regular graph with girth Theorem [FK, FOCS 02]: For any long as Arbitrary, girth, WER, all var nodes have degree, as : Theorem [FKW, CISS 03]: Can achieve Stronger graph properties (expansion?) are needed for stronger results J Feldman, Decoding Turbo Codes and LDPC Codes via Linear Programming p17/22

18 Other pseudocodewords BEC: Iterative decoding successful iff no zero-cost stopping sets [DPRTU 02] - In the BEC, pseudocodewords = stopping sets - Iterative/LP decoding: same performance in BEC Tail-Biting trellisses (TBT): Iterative decoding successful iff dominant pseudocodeword has negative cost [FKMT 98] - TBT: need LP along lines of [FK, FOCS 02] - Iterative/LP decoding: same performance on TBT Min-sum decoding successful iff no neg-cost deviation sets in the computation tree [Wib 96] - Pseudocodewords are natural closed analog of deviation sets J Feldman, Decoding Turbo Codes and LDPC Codes via Linear Programming p18/22

19 Other Results For high-density binary linear codes, need representation of without exponential dependence on check node degree - Use parity polytope of Yannakakis [ 91] - Orig representation: - Using parity polytopes: New iterative methods [FKW, Allerton 02]: - Iterative tree-reweighted max-product [WJW 02] tries to solve dual of our LP - Subgradient method for solving LP gives provably convergent iterative algorithm Experiments on performance, distance bounds J Feldman, Decoding Turbo Codes and LDPC Codes via Linear Programming p19/22

20 Performance Comparison Random rate-1/2 (3,6) LDPC Code Word Error Rate Min-Sum Decoder (100 iterations) LP Decoder Both Error BSC Crossover Probability Length 200, left degree, right degree J Feldman, Decoding Turbo Codes and LDPC Codes via Linear Programming p20/22

21 Growth of Average Fractional Distance 10 Rate 1/4 Gallager Ensemble Fractional Distance Average Fractional Distance Code Length Gallager distribution, left degree, right degree J Feldman, Decoding Turbo Codes and LDPC Codes via Linear Programming p21/22

22 Future Work New WER, fractional distance bounds: - Lower rate turbo codes (rate-1/3 RA) - Other LDPC codes, including * Expander codes, irregular LDPC codes, other constructible families - Random LDPC, linear codes? ML Decoding using IP, branch-and-bound? Using generic lifting procedures to tighten relaxation? Deeper connections to sum-product belief-propagation? LP decoding of other code families, channel models? J Feldman, Decoding Turbo Codes and LDPC Codes via Linear Programming p22/22