Status of Knowledge on Non-Binary LDPC Decoders

Status of Knowledge on Non-Binary LDPC Decoders Part III: Why Non-Binary Codes/Decoders? D. Declercq ETIS - UMR85 ENSEA/Cergy-University/CNRS France IEEE SSC SCV Tutorial, Santa Clara, October 2st, 2 D. Declercq (ETIS - UMR85) / 4

Outline Advantages of Non-Binary LDPC Codes 2 Coding for high Spectral Efficiency 3 Coding for Very Low Rates 4 Decoding General Linear Block Codes 5 Concept of NB-decoder Diversity 6 Conclusion D. Declercq (ETIS - UMR85) 2 / 4

Increased Interest from the Scientific Community since 24 Theoretical Aspects: [Hu,24] X.Y. HU AND E. ELEFTHERIOU, BINARY REPRESENTATION OF CYCLE TANNER-GRAPH GF(2 q ) CODES, IEEE Int. Conf. on Commun., PARIS, FRANCE, JUNE 24 [Rathi,26] V. RATHI AND R.L. URBANKE, DENSITY EVOLUTION THRESHOLDS AND THE STABILITY CONDITIONS FOR NON-BINARY LDPC CODES, IEEE Transactions on communication, VOL. 52(6), DEC. 25 [Bennatan,26] A. BENNATAN AND D. BURSHTEIN, DESIGN AND ANALYSIS OF NON-BINARY LDPC CODES FOR ARBITRARY DISCRETE MEMORYLESS CHANNELS, IEEE Transactions on Information Theory, VOL. 52, PP. 549 583, FEB. 26. [Li,23] G. LI, I. FAIR AND W. KRZYMIEN, ANALYSIS OF NONBINARY LDPC CODES USING GAUSSIAN APPROXIMATION, Proceedings of ISIT, KANAGAWA, JAPAN, JULY 23. [Byers,25] G. BYERS AND F. TAKAWIRA, EXIT CHARTS FOR NON-BINARY LDPC CODES, Proceedings of ICC, SEOUL, KOREA, MAY 25. [Sassatelli,2] L. SASSATELLI AND D. DECLERCQ, NON-BINARY HYBRID LDPC CODES, IEEE Transactions on Information Theory, VOL. 56(), PP 534-5334, OCT. 2 [Kasai,28] K. KASAI, C. POULLIAT, D. DECLERCQ, T. SHIBUYA AND K. SAKANIWA, WEIGHT DISTRIBUTION OF NON-BINARY LDPC CODES, in the proc. of ISITA 8, AUCKLAND, NEW ZEALAND, DEC. 28 D. Declercq (ETIS - UMR85) 4 / 4

Increased Interest from the Scientific Community since 24 Theoretical Aspects: [Goupil,27] A. GOUPIL, M. COLAS, G. GELLE AND D. DECLERCQ, FFT-BASED DECODING OF GENERAL LDPC CODES OVER ABELIAN GROUPS, IEEE Transactions on communication, VOL. 55(4), APRIL 27. [Declercq,2] D. DECLERCQ, NON-BINARY DECODER DIVERSITY FOR DENSE OR LOCALLY-DENSE PARITY-CHECK CODES, to appear in IEEE Transactions on communication, 2 Practical Finite Length Design: [Hu,24] X.Y. HU AND E. ELEFTHERIOU, BINARY REPRESENTATION OF CYCLE TANNER-GRAPH GF(2 q ) CODES, IEEE Int. Conf. on Commun., PARIS, FRANCE, JUNE 24 [Song,26] S. SONG, L. ZENG AND K. ABDEL-GHAFFAR, ALGEBRIC CONSTRUCTION OF NON-BINARY QUASI-CYCLIC LDPC CODES, Proceedings of ISIT, SEATTLE, USA, JULY 26. [Zeng,28] L. ZENG, L. LAN, Y. YU TAI, B. ZHOU, SHU LIN, A. KHALED AND A.S. ABDEL-GHAFFAR, CONSTRUCTION OF NONBINARY CYCLIC, QUASI-CYCLIC AND REGULAR LDPC CODES: A FINITE GEOMETRY APPROACH, IEEE transactions on Communications, VOL. 56, PP. 788-793, MARCH 28. [Venkiah,28] A. VENKIAH, D. DECLERCQ AND C. POULLIAT, DESIGN OF CAGES WITH A RANDOMIZED PROGRESSIVE EDGE GROWTH ALGORITHM, IEEE Communication letters, VOL. 2, PP. 3-33, APRIL, 28. [Poulliat,28] C. POULLIAT, M. FOSSORIER AND D. DECLERCQ, DESIGN OF REGULAR (2,DC)-LDPC CODES OVER GF(Q) USING THEIR BINARY IMAGES, IEEE Transactions on communication, VOL. 56(), PP. 626-635, OCT. 28 D. Declercq (ETIS - UMR85) 5 / 4

Increased Interest from the Scientific Community since 24 Advantages for High Rates Application: [Ma,26] L. MA, L. WANG AND J. ZHANG, PERFORMANCE ADVANTAGE OF NON-BINARY LDPC CODES AT HIGH CODE RATE UNDER AWGN CHANNEL, Proceedings of ICCT 6, GUILIN, CHINA, NOV. 26. [Zeng,28] L. ZENG, L. LAN, Y. YU TAI, B. ZHOU, SHU LIN, A. KHALED AND A.S. ABDEL-GHAFFAR, CONSTRUCTION OF NONBINARY CYCLIC, QUASI-CYCLIC AND REGULAR LDPC CODES: A FINITE GEOMETRY APPROACH, IEEE transactions on Communications, VOL. 56, PP. 788-793, MARCH 28. Advantages for Channels with Memory/Burst Error Correction: [Morinoni,28] A. MORINONI, P. SAVAZZI AND S. VALLE, EFFICIENT DESIGN OF NON-BINARY LDPC CODES FOR MAGNETIC RECORDING CHANNELS, ROBUST TO ERROR BURSTS, Proceedings of IEEE ISTC, LAUSANNE, SWITZERLAND, SEPT. 28. [Chen,25] J. CHEN, L. WANG AND Y. LI, PERFORMANCE COMPARISON BETWEEN NON-BINARY LDPC CODES AND REED-SOLOMON CODES OVER NOISE BURST CHANNELS, Proceedings of IEEE ICCS 5, CHINA, MAY 25. Advantages for Low Rates Applications: [Sassatelli,28] L. SASSATELLI, D. DECLERCQ AND C. POULLIAT, LOW-RATE NONBINARY HYBRID LDPC CODES, in the proc. of IEEE Turbo-Coding Symposium, LAUSANNE, SWITZERLAND, SEPT. 28 [Kasai,2] K. KASAI, D. DECLERCQ, C. POULLIAT, K. SAKANIWA, RATE-COMPATIBLE NON-BINARY LDPC CODES CONCATENATED WITH MULTIPLICATIVE REPETITION CODES, in the proc. of ISIT, AUSTIN, TEXAS, USA, JUNE 2. D. Declercq (ETIS - UMR85) 6 / 4

Increased Interest from the Scientific Community since 24 Advantages for High Throughput Vector Channels: [Declercq,24] D. DECLERCQ, M. COLAS AND G. GELLE, REGULAR GF(2 q ) LDPC CODED MODULATIONS FOR HIGHER ORDER QAM-AWGN CHANNELS, Proceedings of IEEE ISITA, PARMA, ITALY, OCT. 24. [Peng,26] R.-H. PENG AND R.-R. CHEN, DESIGN OF NON-BINARY LDPC CODES OVER GF(q) FOR MULTIPLE-ANTENNA TRANSMISSION, Proceedings of MILCOM, WASHINGTON DC, USA, OCT. 26. [Jiand,29] X. JIAND, Y. YAN, X.-G. XIA AND M.H. LEE, APPLICATION OF NON-BINARY LDPC CODES BASED ON EUCLIDEAN GEOMETRIES TO MIMO SYSTEMS, Proceedings of Intern. conference on wireless comm. and Signal processing, NANJING, CHINA, NOV. 29. [Byers,24] G.J. BYERS AND F.TAKAWIRA, NON-BINARY AND CONCATENATED LDPC CODES FOR MULTIPLE-ANTENNA SYSTEMS, Proceedings of AFRICON, GABORONE, BOTSWANA, SEPT. 24. [Pfletschinger,2] S. PFLETSCHINGER AND D. DECLERCQ, GETTING CLOSER TO MIMO CAPACITY WITH NON-BINARY CODES AND SPATIAL MULTIPLEXING, in the proc. of Globecom, MIAMI, FLORIDA, USA, DECEMBER 2. D. Declercq (ETIS - UMR85) 7 / 4

Different Research Directions Small Order Fields Slightly Irregular LDPC codes in small order Fields GF (8) or GF (6). for such low Field order, the issue of the decoder complexity is less crucial, and all decoders mentioned in this presentation work nicely. (FFT-logBP, EMS small n m, Min-Max, etc) Large Order Fields Stricly regular d v = 2 LDPC codes in high order fields GF (q), q 64. Very Large Order Fields Is there a need for decoders of NB-LDPC codes in GF (496) or higher? D. Declercq (ETIS - UMR85) 8 / 4

Why ultra-sparse d v = 2 Tanner Graphs Amazing girths can be obtained for small to moderate codeword lengths d c = 3 d c = 4 d c = 6 d c = 8 N s = 5 g=22 g=8 g=4 g=8 N s = 3 g=28 g=2 g=6 g=2 Number of independant decoding iterations = g/4. If g, message passing decoder is closer to MLD, [Hu,24] X.Y. HU AND E. ELEFTHERIOU, BINARY REPRESENTATION OF CYCLE TANNER-GRAPH GF(2 q ) CODES, IEEE Int. Conf. on Commun., PARIS, FRANCE, JUNE 24 [Venkiah,28] A. VENKIAH, D. DECLERCQ AND C. POULLIAT, DESIGN OF CAGES WITH A RANDOMIZED PROGRESSIVE EDGE GROWTH ALGORITHM, IEEE Communication letters, VOL. 2, PP. 3-33, APRIL, 28. D. Declercq (ETIS - UMR85) 9 / 4

Link between d v = 2 NB-LDPC protograph and Parallel Turbo-Codes reconciliate Turbo-Codes and LDPC codes i Π r r 2 Π Formally, a (d v = 2, d c = 3) NB-LDPC code with protograph description is equivalent to a parallel Turbo-Code, with i Bloc codes instead of Conv. codes Symbol-wise interleaver Information Symbols Same kind of limitations : D min log 2 (N) log 2 (q) Π Non Binary Parity Check Non Binary Parity Check r r 2 D. Declercq (ETIS - UMR85) / 4

Non-Binary d v = 2 LDPC Codes for Short Block-Lengths Performance Comparison, K=24 information bits 2 R=4/5 Frame Error Rate 3 4 5 R=/2 6 7 JPL codes Optimized GF(256) codes R=2/3 8.5.5 2 2.5 3 3.5 4 4.5 E b /N (in db) Is it Really Worth the Additionnal Complexity? [Poulliat,28] C. POULLIAT, M. FOSSORIER AND D. DECLERCQ, DESIGN OF REGULAR (2,DC)-LDPC CODES OVER GF(Q) USING THEIR BINARY IMAGES, IEEE Transactions on communication, VOL. 56(), PP. 626-635, OCT. 28 D. Declercq (ETIS - UMR85) / 4

Non-binary Codes for Multiple Antennas Two Possible Schemes (a) q-ary information processing: plug the non-binary coded symbols directly on the modulated-mimo channel in a Coded Modulation (CM), (b) binary information processing: classical scheme using Bit-Interleaved Coded modulation (BICM) Assumptions: spatial multiplexing (no Space Time code), uncorrelated i.i.d. Rayleigh fading, no CSIT, perfect CSIR. D. Declercq (ETIS - UMR85) 3 / 4

Capacities associated with the Two schemes Capacity [bits per channel use] 8 7 6 5 4 4.4 4.2 4 3.8 Zoom: 6 QAM 6 QAM, CM.56 db 3.6 6 7 8 9 2 2 MIMO, i.i.d. Rayleigh fading 6 QAM, BICM Zoom: 8 QAM, 8 PSK 3.2 3 Gaussian input 3 2 8 QAM, CM.35 db 8 QAM, BICM 2.8 8 PSK, BICM 6 QAM, CM 2.6 6 QAM, BICM 4 5 6 7 5 5 5 2 SNR [db] 8 QAM, CM 8 PSK, BICM 8 QAM, BICM D. Declercq (ETIS - UMR85) 4 / 4

Why NB-Codes act as natural Space-Time codes? The key is to assign one and only one code symbol to each channel use n T log 2 (M) = log 2 (q) As a result of this constraint, the L-values at the input of the decoder: - are uncorrelated from one code-symbol to another: we do not loose information due to correlation at the demapper output. - are sufficient statistics with respect to the vector-channel (QAM+MIMO), so the code aims at reaching the vector-channel capacity. The following cases have been tested - GF(64) LDPC code, R = /2, n T = 2 antennas, 8-QAM/8-PSK, - GF(256) LDPC code, R = /2, n T = 2 antennas, 6-QAM. Advantage at the receiver: Mapping of one code symbol to one channel use leads to very simple APP demapper (no need for reduced-complexity MIMO detector i.e. sphere decoder, MMSE, etc.) [Pfletschinger,2] S. PFLETSCHINGER AND D. DECLERCQ, GETTING CLOSER TO MIMO CAPACITY WITH NON-BINARY CODES AND SPATIAL MULTIPLEXING, in the proc. of Globecom, MIAMI, FLORIDA, USA, DECEMBER 2. D. Declercq (ETIS - UMR85) 5 / 4

Finite Length Simulation Results () 2 2 MIMO, 8 QAM/PSK 8 QAM, q = 64 8 PSK, binary BER 2 3 4 limit for Alamouti scheme BICM Shannon limit CM Shannon limit 3.5 4 4.5 5 5.5 6 6.5 7 SNR = n E / N [db] T S D. Declercq (ETIS - UMR85) 6 / 4

Finite Length Simulation Results (2) 2 2 MIMO, 6 QAM q = 256 binary 2 BICM Shannon limit BER 3 4 5 CM Shannon limit limit for Alamouti scheme 6 6.5 7 7.5 8 8.5 9 9.5 SNR = n E / N [db] T S D. Declercq (ETIS - UMR85) 7 / 4

Existing Solutions for Low Rates codes Good solutions at N = + LDPC-Hadamard codes (special kind of Generalized Low Density codes), Binary multi-edges types (MET-LDPC) or Protographs, Both solutions have decoding threshold between.2db and.4db from the Shannon Limits. Nothing works at small to moderate lengths LDPC-Hadamard: NO results reported in litterature for lengths smaller than K = 4 bits, Binary MET-LDPC codes / Protographs: NO results reported for rates below R = /6, The only existing solution at small length is the Turbo-Hadamard or Zigzag-Hadamard, but suffer for very high error floors 4. [Kasai,2] K. KASAI, D. DECLERCQ, C. POULLIAT, K. SAKANIWA, RATE-COMPATIBLE NON-BINARY LDPC CODES CONCATENATED WITH MULTIPLICATIVE REPETITION CODES, in the proc. of ISIT, AUSTIN, TEXAS, USA, JUNE 2. D. Declercq (ETIS - UMR85) 9 / 4

Proposed scheme based on NB repetition nodes Start from an Ultra-Sparse R = /3 NB-LDPC Code Optimized Interleaver for Very High Girth h3 h h2 h2 h3 h h h2 h3 h3 h h2 D. Declercq (ETIS - UMR85) 2 / 4

Proposed scheme based on NB repetition nodes Repeat all symbols with extra non-zero values h4 h4 h4 h4 h4 h4 Optimized Interleaver for Very High Girth h3 h h2 h2 h3 h h h2 h3 h3 h h2 D. Declercq (ETIS - UMR85) 2 / 4

Proposed scheme based on NB repetition nodes Continue repetition... h5 h6 h5 h6 h5 h6 h5 h6 h5 h6 h5 h6 h4 h4 h 4 h4 h4 h4 Optimized Interleaver for Very High Girth h3 h h2 h2 h3 h h h2 h3 h3 h h2 D. Declercq (ETIS - UMR85) 22 / 4

Performance Results GF(496) LDPC code vs. MET/Protograph Binary code Multi Edge type LDPC, K=24 bits, R=/6 GF(496), K=8 bits, R=/6 2 Frame Error Rate 3 4 5 6 7.5.5.5 (E /N ) in db b D. Declercq (ETIS - UMR85) 23 / 4

Performance Results GF(496) LDPC code vs. GF(256) LDPC codes, K = 8 info. bits R=/3 2 Frame Error Rate 3 4 R=/8 R=/6 5 6 GF(256) Codes GF(496) Codes SP59 Lower Bound: R=/8 7.5.5.5 (E b /N ) in db D. Declercq (ETIS - UMR85) 24 / 4

Binary Representation of Non-Binary Codes Binary Maps and Companion Matrices h ji GF (q) c i GF(q) h ji.c i GF(q) Let us replace h ji with a binary matrix representation A(h ji ): Companion matrix of hji Binary map of hji*ci Binary map of ci D. Declercq (ETIS - UMR85) 26 / 4

Binary Representation of Non-Binary Codes non-binary codes are locally dense codes 6 4 2 8 Non Binary Parity Check Matrix in GF(256) 6 4 2 8 6 4 2 5 5 2 25 3 Binary Representation of the Parity Check Matrix 2 8 6 4 2 5 5 2 25 3 35 4 6 4 2 8 6 4 2 5 5 2 25 3 Random Permutation of Rows and Columns of H D. Declercq (ETIS - UMR85) 27 / 4

Background Let start with H b, a binary (M b N b ) parity check matrix of a dense block code, One can obtain a non-binary Tanner graph of H b by bit-clustering of order p, - by combining p adjacent columns to form a non-binary symbol of order q = 2 p, - by combining p adjacent rows to form a NB-parity check node in the group F p 2 the obtained Tanner graph has M = M b /p generalized parity checks and N = N b /p symbol nodes, if M b and N b are not integer multiples of p: complete with all-zeros columns (pruned bits), complete with redundant rows. a decoding instance is obtained by several iterations of generalized BP decoder on the non-binary Tanner graph [Goupil,27], [Goupil,27] A. GOUPIL, M. COLAS, G. GELLE AND D. DECLERCQ, FFT-BASED DECODING OF GENERAL LDPC CODES OVER ABELIAN GROUPS, IEEE Transactions on communication, VOL. 55(4), APRIL 27. D. Declercq (ETIS - UMR85) 28 / 4

Generalized parity-check Equations Generalized Parity Check Equation Xd c h ij (x j ) = in F(2 p ) j= where non-zero values are now function h ij (.) : F(2 p ) F(2 p ) Or in Vector Form Xd c H ij.x j = p j= in (F 2 ) p H ij is a binary matrix (p p) called Binary Cluster D. Declercq (ETIS - UMR85) 29 / 4

Functions Associated to Binary Clusters Nonzero Cluster Associated function Nonzero Cluster Associated function b i b j b i b j Hij= (a) Hij= (b) Full Rank Case Rank Deficient Case D. Declercq (ETIS - UMR85) 3 / 4

Decoding Results Case of the extended quadratic-residue linear code (N = 48, K = 24, Dmin = 2) 2 3 Frame Error Rate 4 5 6 7 Binary Min Sum Non Binary F 6 8 Non Binary F 256 Non Binary F 496 Lower Bound 9 2 3 4 5 6 7 8 9 (E b /N ) in db D. Declercq (ETIS - UMR85) 3 / 4

A small example () Diversity of Tanner Graph Representations Good and Bad Tanner Graphs after clustering from the same H b : etc... More Sparse! Good and Bad? D. Declercq (ETIS - UMR85) 33 / 4

A small example (2) Diversity of Decoding behaviors test case: noise vectors at Channel Error Probability α =. 59 cases where both decoders converge to the right codeword, 384 cases where both decoders fail to converge after 5 iterations, 589 cases where both decoders converge to a wrong codeword, 23 cases where one decoder fails to converge, and the other one converges to the right codeword, 4 cases where one decoder fails to converge, and the other one converges to a wrong codeword, 754 cases where one decoder converges to a wrong codeword, and the other one converges to the right codeword. Good cases Bad cases D. Declercq (ETIS - UMR85) 34 / 4

Capitalizing on the Decoding Diversity Diversity Sets Definition of Diversity Set: set of d distinct non-binary Tanner graphs of the same code: n o G () p,..., G (i) p i,..., G (d) p d H (i) b 2 G (i) p i = P (i) (H b ) = A (i).h b.π (i) is called pre-processing, is the Tanner graph obtained by clustering H (i) with an order p b i. [Declercq,2] D. DECLERCQ, NON-BINARY DECODER DIVERSITY FOR DENSE OR LOCALLY-DENSE PARITY-CHECK CODES, to appear in IEEE Transactions on communication, 2 D. Declercq (ETIS - UMR85) 35 / 4

Capitalizing on the Decoding Diversity Merging Strategies Serial merging: Use the decoders sequentially, i.e. switch to another decoder only when the current decoder failed to converge to a codeword, O(Serial) = ( + ε) O(p.2 p.n.n it ) Parallel merging: Run all d decoders in parallel and choose appropriately an estimated codeword (majority voting, maximum likelihood, etc) O(Parallel) = d O(p.2 p.n.n it ) Other merging strategies: O(Serial) Comp O(Parallel), Lower bound: Is there at least graph among the d candidates such that the decoder converges to the right codeword? (we do not address the PB of finding the good one among the d) D. Declercq (ETIS - UMR85) 36 / 4

BCH codes on the BSC Channel Diversity set and parameters NO information about Error Location is available Constant clustering order in the diversity sets p, Considered Preprocessing: Build H b from low weight codewords of the BCH dual code H b is as sparse as possible, 2 consider random row permutations and random column permutations: H (i) b = Π (i).h b.π (i) 2 D. Declercq (ETIS - UMR85) 37 / 4

BCH codes on the BSC Channel BCH(N = 27, K = 7, D min = 9) BCH (N=27,K=7,T=9) over the BSC channel 2 Frame Error Rate 3 4 5 6 GDD (p=8,d=) GDD (p=8,d=2) lower bound GDD (p=8,d=2) lower bound Bounded Distance Decoding 7 3 3.5 4 4.5 5 5.5 6 6.5 7 E /N in db b D. Declercq (ETIS - UMR85) 38 / 4

Turbo-Codes from the DVB-RCS standard Simulation results. Diversity decoder performance for a (R =.5, N = 848) duobinary TC Turbo decoder 5 group decoders: serial merging 5 group decoders: lower bound 2 Frame Error Rate 3 4 5 6 7.5.5 2 2.5 3 3.5 E b /N in db D. Declercq (ETIS - UMR85) 39 / 4

Conclusions People are generally happy with the modern error correcting performance of LDPC and Turbo-Codes. They are right: binary modern codes have excellent performance. For binary-input memoryless channels, NB-LDPC codes have a slight advantage in the waterfall and a more important advantage in the error floor. Does the increased decoding complexity justify those small gains? For more complex systems, like MIMO and M-QAM channels, or channels with memory where errors appear in burst, there is no doubt that NB-LDPC codes have an advantage: the mutual information loss due to marginalization in the binary case does not apply for the non-binary case. anyhow, the advances in NB-LDPC decoding have shown that the decoding complexity of GF(q) codes is not q times larger than binary decoders, which is promising for future an emerging non-binary applications. D. Declercq (ETIS - UMR85) 4 / 4