VLSI Design for High-Speed Sparse Parity-Check Matrix Decoders

Transcription

1 VLSI Design for High-Speed Sparse Parity-Check Matrix Decoders Mohammad M. Mansour Department of Electrical and Computer Engineering American University of Beirut Beirut, Lebanon mmansour@aub.edu.lb Abstract In this paper, the design of high-speed iterative decoders for sparse parity-check matrix (SPCM) codes such as LDPC, repeataccumulate and turbo-like codes is addressed. The random nature of the underlying Tanner graph associated with these codes is problematic for a high-speed decoder implementation. This issue is addressed by designing structured SPCM codes tailored for low-complexity scalable decoders using the turbo-decoding message-passing (TDMP) algorithm. Analysis using EXIT charts shows that a better performance/complexity tradeoff is achieved when the number of decoding iterations is small which is attractive for high-speed applications. A scalable decoder architecture for structures SPCM codes employing the TDMP algorithm is presented. I. INTRODUCTION Sparse-parity check matrix (SPCM) codes have gained widespread attention in advanced communications systems after the introduction of turbo codes by Berrou et al. in 993 []. SPCM codes are linear block codes defined by a sparse parity-check matrix, and are typically known as low-density parity-check (LDPC) codes [2] or turbo-like codes [3]. LDPC codes in general have quadratic encoding complexity with respect to code length, while turbo-like codes are in general systematic codes with linear encoding complexity. The principle of decoding SPCM codes dates back to Gallager s seminal work on LDPC codes in 963 [2] and Tanner s work on codes on graphs in 98 [4]. It is based on iterative suboptimal decoding by message-passing as an alternative to one-shot maximum likelihood decoding. Codeword symbols are processed iteratively in an attempt to improve their reliability based on the code constraints. Output reliability measures of the decoded symbols at the end of each decoding iteration are used as input prior information for the next iteration. Inter-iteration correlation between input and output messages is minimized by using extrinsic outputs which do not depend on the inputs used to generate them. Extrinsic output messages are typically produced by subtracting or excluding the corresponding input prior messages from the computations. Decoding iterations proceed until a stopping criterion is satisfied. Many variations of SPCM codes have been proposed in the literature, each achieving error performance within a fraction of db from channel capacity (e.g., [5], [6]). As such, these codes have been adopted in many advanced communications including next generation digital video broadcasting (DVB-S2) via satellite [7], wireless LAN using MIMO systems (IEEE 82.n) [8], and Gbit/s ethernet over twisted-pair wire (IEEE 82.3an) [9]. One major challenge in the decoder design for long SPCM codes is the random structure of the parity-check matrix or equivalently the edge permuter of the associated Tanner graph [4]. To achieve full throughput for a large number of iterations using the inherently parallel standard message-passing algorithm, these permuters are typically implemented using a physical interconnect of wires that matches exactly the topology of the code s (random) Tanner graph. This network quickly becomes an implementation bottleneck and prohibits a parallel decoder realization for long codes. The alternative would be to serialize the decoding operations and implement the permuter using memory blocks, which degrades decoding throughput and increases memory requirements. Although parallel SPCM decoders achieve the highest possible throughput, they are not practical to implement for code lengths larger than 248 bits for two main reasons. First, the number of function units to be integrated on-chip for message computation grows linearly with the code length n as (2 R)n, where R is the code rate. This number becomes substantial for large n which complicates block placement and routing of the physical layout. Second, the interconnect complexity grows as 2b P n j= rj, where b is the bitprecision and r j is the number of ones in the ith column of the parity-check matrix (or equivalently, the bit-node degree). Again this number becomes substantial even for n = 24 (e.g., the number of wires to be routed for the 24-bit parallel decoder in [] is 26,624). This results in an interconnect-dominant chip with massive wiring congestion, excessive buffering to cope with long wires, and increased power consumption due to the switching activity of this large number of wires. This paper addresses VLSI design issues of high-speed decoders for long SPCM decoders. First, the message-computation units should be both compact and accurate to allow for efficient integration of a large number of units for high-speed decoders of long codes as described in Section II. This requires modification of the standard message-computation algorithm which uses highly non-linear functions. Second, message-communication should be done using a programmable interconnection network rather than via pure physical wires as in fully-parallel decoders. This requires modification of both the parity-check matrix of the SPCM code and the decoding algorithm as discussed in Section III. These desirable properties are achieved by adopting what is called architecture-aware SPCM codes having a structured parity-check matrix, employing the parallel turbo-decoding message-passing algorithm, and using the efficient -function for message computation. In the following, we assume that an SPCM code is defined by a sparse parity-check matrix H m n with m parity-check equations on n codeword bits. The number ones in row i is c i and the number of ones in column j is r j. Its associated Tanner graph has m check nodes and n bit nodes, with node degrees c i and r j respectively. If all check nodes have uniform degrees, we say that the code is checkregular with degree c. Similarly, we say the code is bit-regular if all bit nodes have uniform degree r. II. MESSAGE COMPUTATION KERNELS In the standard decoding algorithm [2], a check message µ vi u from a check node v i to a neighboring bit node u is computed as /5/$2. 25 IEEE 78

2 follows: µ vi u = ψ X s N (v i ) s u ψ(µ s vi ), all u N(v i), () where N (v i) is the set of neighbors of node v i. The non-linear function ψ(x) and its inverse ψ (x) are defined as ψ(x) = sgn(x), ψ(x) = sgn(x), log tanh x, (2) 2 ψ (α, x) = α log tanh x 2, (3) where sgn(x) = if x<, and + otherwise, and ψ(x) = log tanh x. The sum ψ(x)+ψ(y) is defined as ψ(x)+ψ(y) 2 sgn(xy), ψ(x) + ψ(y). These functions have a wide dynamic range for small values of x ( x 2), and flatten out for large values of x ( x > 2). They are typically implemented using lookup tables with fine bit resolution for enhanced accuracy and wider range. In addition, when quantized, the two functions must remain inverses of one another. These requirements lead to inaccuracies and require large function units and wide data-paths, which limit the code length for which practical decoders can be implemented. An alternative approach to compute messages is by using the simple -function (x, y) = max(x, y)+max(5/8 x y /4, ) max(x + y, ) max(5/8 x + y /4, ) (4) log(e x + e y ) log(e x+y +). which approximates the difference between two logsums of two exponentials. It is easy to verify that the following properties hold for (x, y): (i) (x, y) =(y, x) (ii) ( x, y) =(x, y) (iii) (x,) = (,y)=(, ) = (iv) (x, y) =( x, y) = (x, y) (v) ( x, y ) (vi) (x, y) =`sgn(x) x, sgn(y) y = sgn(xy) ` x, y These properties considerably simplify the implementation of (x, y). The relationship between (x, y) and ψ(x) in (2) is given by the following properties: (vii) sgn(xy) ψ `ψ( x )+ψ( y ) = (x, y) (viii) sgn(xyz) ψ `ψ( x )+ψ( y )+ψ( z ) = `(x, y),z (ix) sgn(π c i=x i) ψ `P c ψ( xi ) i= = ( (((x,x 2),x 3), ),x c) For convenience, we define the following notation: (x,,x c) ( (((x,x 2),x 3), ),x c) [i] (x,,x i,x i,x i+,,x c) (x,,x i,x i+,,x c) The subscript [i] in the second notation denotes the index of the variable to be left-out of the computation. Property (ix) can be applied to compute messages recursively (c.f. eq. ()). Consider computing messages pertaining to a parity-check equation of weight c with input messages λ,λ 2,,λ c: Λ j = [j] (λ,,λ j,,λ c), j =,,c. (5) The message Λ j is determined by first computing (λ,λ 2) = α 2, then ((λ,λ 2),λ 3) = (α 2,λ 3) = α 3, then (((λ,λ 2),λ 3),λ 4) = (α 3,λ 4) = α 4, up Algorithm Message Computation Algorithm: Λ = SISO(λ) // Input: λ =[λ,λ 2,,λ c] // Output: Λ =[Λ, Λ 2,, Λ c] // Initialization α λ β c λ c // Forward-backward recursion for i =to c 2 do j = c (i ) α i+ (α i,λ i+)... (7) β j (λ j,β j)... (8) if c/2 i c 2 then Λ i+ (α i,β i+2)... (9) Λ j (α j 2,β j) end if end for Λ β 2 Λ c α c to (α j 2,λ j ) = α j at position j, as well as reversely computing (λ n,λ n) = β n, then (λ n 2,(λ n,λ n)) = (λ n 2,β n ) = β n 2, then (λ n 3,(λ n 2,(λ n,λ n))) = (λ n 3,β n 2) = β n 3, down to (λ j+,β j+2) = β j+ at position j +, and finally combining the results as Λ j = (α j,β j+). The pseudo-code of the message computation algorithm using the -function in (4) is listed as Algorithm assuming c is even. The forward and backward recursions are initialized with λ and λ c, respectively. The loop computes forward and backward updates until half way through the recursions, at which point output messages are produced in pairs. The leftmost and rightmost messages Λ and Λ c are computed directly from the β 2 and α c, respectively. The case when c is odd is handled with a slight modification. Note that unlike (), computing messages using (5) does not involve inverses of non-linear functions which are susceptible to quantization noise. The accuracy of the -function approximation is evaluated in Figure which compares the BER performance achieved using the -function approximation with the ideal case when decoding a rate-/2 24-bit regular LDPC code. Also shown in the figure is the BER plot using the constant correction term approximation of []. The -function approximation incurs negligible loss in coding gain compared to the ideal BER, and is more accurate than the approximation of []. Figure 2 shows the logic schematic implementation of the - function using simple gates. Note that the max operations eliminate the need for lookup tables to approximate functions. Figure 3(a) shows a parallel data-flow graph of the message computation algorithm for the case c =6. Intermediate messages α i and β j are computed starting from the left and the right until half the inputs have been processed, after which output messages are generated in pairs. The number of -function blocks is 3(c 2). Figure 3(b) shows a serial data-flow graph with four -function blocks that uses a stack for storing the intermediate messages accumulated in the forward and backward computations. Messages are pushed onto the stack until half the inputs have been processed, and popped afterwards to generate output messages using a pair of -function blocks. The message computation kernels are also referred to soft-input soft-output (SISO) decoders. 79

3 2 Ideal Using proposed (x,y) Using q c (x,y), c=.5 λ λ 2 λ 3 λ 4 λ 5 α α 2 α 3 α 4 α 5 Λ 6 3 Λ 2 Λ 3 Λ 4 Λ 5 BER Λ β 2 β 3 β 4 β 5 β 6 4 λ 2 λ 3 λ 4 λ 5 Λ j = [j] (λ,λ 2,λ 3,λ 4,λ 5, ), j =,, 6 (a) E b /N [db] Fig.. Comparison of BER performance using the -function with the ideal case and the constant correction term method of []. α i λ,,λ 5,,λ 2 β j x[b-:] y[b-:] Fig. 2. 5/8 S[b:] >> 2-5/8 D[b:] >> 2 xy[b-:] xy[b-:] xy[b-:] Logic schematic of the -function. S[b] [b-:] stack Λ 3, Λ 2, Λ Λ 4, Λ 5, Λ 6 stack Λ j = [j] (λ,,λ j,,), j =,, 6 (b) Fig. 3. (a) Parallel, and (b) serial data-flow graphs of the message computation algorithm (SISO decoders) for computing messages Λ j = [j] (λ,,λ j,, ), j =,, 6. III. DECODER ARCHITECTURES In the standard two-phase algorithm [2], a decoding iteration is divided into two rounds of computations, one pertaining to variable node computations and one pertaining to check node computations. The scheduling of operations is done irrespective of the structure of the Tanner graph (e.g., if it contains trellises). In each round, all nodes are activated in parallel, sending new outgoing messages to all their neighbors using messages received in the previous round. This decoding schedule operates using two types of messages and requires saving all intermediate messages between both rounds at every iteration. Moreover, newly computed messages in a round of computations do not participate in further message computations until the decoding iteration is over. If updated messages are used directly within an iteration to compute new messages, then refined estimates will spread faster among neighboring nodes in the graph, speeding up the convergence behavior of the algorithm. Further, new check messages become directly variable messages during an iteration, hence both variable and check messages collapse into a single type of messages leading to significant memory savings. This decoding algorithm is called the turbo-decoding message-passing (TDMP) algorithm. The TDMP algorithm is described with the help of the parity-check matrix H shown in Fig. 4, which has six parity-check equations corresponding to a code of length 2. The algorithm is based on decoding the rows (parity-check equations) of H sequentially. Extrinsic messages generated from decoding earlier rows are used as input prior messages to decode subsequent rows. To each row i in H, we associate the vector λ i =[λ i,,λ i c i ] of extrinsic messages corresponding to the non-zero entries in that row. The number of nonzeros c i in a row is called the weight. Let I i denote the set of indices of the non-zero entries of row i in H. For example, in Fig. 4, the weight of row 3 is c 3 =6and its index set is I 3 = {, 3, 6, 8,, 2}. Hence, λ 3 corresponds to bit, λ 3 2 corresponds to bit 3,, λ 3 6 corresponds to bit 2. The messages in vector λ i are related to the likelihood with which the information and parity bits corresponding to the non-zero entries in row i satisfy the ith parity-check equation, given extrinsic estimates about the bits from all other parity-check equations in which these bits participate. In addition, we maintain the vector γ =[γ,,γ n] of n posterior messages that stores the sum of all messages generated by the rows in which each bit participates (i.e., the jth entry of γ, γ j, corresponds to the sum of all messages for the jth bit generated by decoding the rows in H where the jth entry is non-zero). For example, n =2in Fig. 4, and the 6th entry of γ is γ 6 = λ λ λ 5 2. The posterior messages of row i are indexed as γ(i i). Hard decisions are made by slicing the vector γ. Decoding the ith parity-check row involves the following four steps. The steps are illustrated in Fig. 4. () Read: The extrinsic messages λ i and the posterior messages γ(i i) for row i are read. (2) Subtract: λ i are subtracted from γ(i i) to generate prior messages ρ =[ρ,,ρ ci ]=γ(i i) λ i. This step ensures the extrinsic principle in that messages generated earlier from decoding a particular row are not used as inputs in decoding the same row in a later iteration. (3) Decode: Decode row i using eq. () or a soft-input soft-output algorithm (e.g. BCJR algorithm) with ρ as input. Let Λ i = [Λ i,, Λ i c i ] denote the output messages. For example, the jth message in row i is computed using eq. (5) as Λ i j = [j] (ρ,,ρ j,,ρ ci ) for i =,,n, and j =,,c i. (4) Write back: The vector Λ i replaces the old extrinsic messages λ i, and the posterior messages for the bits located at positions indicated by I i are updated by adding Λ i to ρ: γ(i i)=ρ+λ i. Steps ()-(4) constitute a decoding sub-iteration. A decoding 7

4 λ λ 2 λ 3 λ 4 λ 5 λ 3 λ 3 (extrinsic messages) ρ (total sum of vertical messages) γ(i 3) γ(i 3) γ inputs: λ 3 decode parity-check equation outputs: Λ 3 (new extrinsic messages) Λ 3 (old difference messages) ρ (updated total sum of messages) γ(i 3) λ λ 2 λ 3 λ 4 λ 5 (a) Read extrinsic messages and sum of all vertical messages. I3 corresponds to the grey shaded rectangles. (b) Subtract extrinsic messages from total sum. The difference ρ is saved and used as intrinsic messages. (c) Decode parity-check equation using new prior messages as input. (d) Decoded messages are saved as extrinsic messages. The total sum of vertical messages is updated by adding these messages to the old saved difference. Fig. 4. Message-passing under the TDMP algorithm. The figure illustrates the four steps of the algorithm in decoding the third row of the parity-check matrix: (a) Read extrinsic and posterior messages, (b) subtract extrinsic messages from posterior messages to generate prior messages, (c) decode the parity-check equation using these messages as inputs, and (d) write back the output messages as new extrinsic messages and update the posterior messages. iteration comprises multiple sub-iterations corresponding to the rows of H. A round of sub-iterations over all rows of H constitutes a decoding iteration. Compared to the standard two-phase algorithm, the TDMP algorithm uses the most recent messages in decoding the bits of a row in H. Moreover, the check and variable message rounds of computations are merged into a single step of computation where simultaneously new check messages are computed using the most recent variable messages, and variable messages are updated with the newly computed check messages. The TDMP algorithm and can be applied in general to decode any code defined by a sparse parity-check matrix such as (irregular) LDPC, (irregular) repeat-accumulate [5], and more general turbolike codes [3], [2]. It accelerates the convergence behavior of the standard iterative decoding algorithm by roughly a factor of two in terms of decoding iterations, and attains an order of magnitude improvement in bit error rate (BER) at high signal-to-noise ratios (SNRs). Analysis using EXIT charts shows that the TDMP algorithm offers algorithm offers a better performance/complexity tradeoff when the number of decoding iterations is small, which is attractive for high speed applications. In terms of storage, the two-phase algorithm requires memory storage for M LDPC = nx c i i= check messages + X n+k r j j= variable messages + (n + k) channel values messages, assuming H has row weights {c i} and column weights {r j} and the code is systematic. The TDMP algorithm on the other hand requires storage for only M TDMP = nx c i i= extrinsic messages + (n + k) posterior messages messages, resulting in a memory savings of P n+k j= rj 2 Pn+k %. j= rj +(n + k) (Note that the posterior messages include the channel values). For uniform column degrees r, this amounts to r(n + k)/(2r(n + k)+ (n + k)) = r/(2r +) %. For example, the savings in memory is about 43.75% for the rate-/2 648-bit irregular LDPC (RA) code specified in the DVB-S2 standard [7]. A. Parallel TDMP Decoder Architecture The TDMP algorithm can be parallelized by embedding some structure into H that allows parallel processing of multiple rows without communicating messages between the nodes corresponding to these rows. The resulting H is composed of several bands of non-overlapping rows. Further, if these bands are divided into either permutation matrices or all-zero matrices as shown in Fig. 5, then messages can be communicated between bands using a programmable network according to the corresponding permutations. SPCM codes having this structure are referred to as architecture-aware SPCM (AA-SPCM) codes, and the parallelized TDMP as P-TDMP algorithm. H (D S) (B S) = Fig. 5. I π I π2 I π3 I π4 I π5 I π6 I π7 I π8 I π9 I π I π I π2 I π3 I π4 I π5 I π6 I π7 I π8 I π9 I π2 Architecture-aware SPCM matrix. I π2 I π22 I π23 I π24 In what follows, assume that an AA-SPCM code is defined by a parity-check matrix H composed of S S sub-matrices. The parameter S is a parallelism factor that scales throughput by a factor of S. Each block row has B sub-matrices and each block column has D sub-matrices. The S rows of H can be processed in parallel using S SISO decoders. Decoder s processes row s in each row of submatrices, for a total of D rows, and maintains extrinsic messages denoted by λ d,s, d =,,D, in a local memory. Posterior messages are stored in a global memory of size n and passed in parallel (S messages at a time) to the decoders using a network that implements the factored edge permuter. 7

5 Fig. 6. Parallel TDMP decoder architecture with S =64. An immediate advantage of the architecture-aware structure is that it enables an efficient implementation of the permuter using a programmable multi-stage interconnection network (e.g., Benes network [3]). The network is programmed to route the various permutations when processing the rows of sub-matrices in H. A Benes network that routes any permutation of size S is composed of (2 log S ) stages where each stage contains S/22 2 switches. For a regular code with uniform check node degree c, the total number of control bits is c S D S/2(2 log S ). The actual VLSI implementation of a Benes network is relatively easy because of its regular structure and simple switch elements that can be implemented using only transistors. For example, a Benes network with S =64 can be implemented using 4,8 transistors and requires 3,68 control and data wires, assuming 4-bit messages. Fig. 6 shows the architecture of a P-TDMP decoder operating on an AA-SPCM code with parameters S =64, D =6, B =32, c =6, and code length n = 248. Messages are represented using b =4bits. The decoder contains 64 constituent SISO decoders each having its own local λ-memory. Four decoders are lumped in each of the UAD-SISO blocks shown in the figure. Each λ-memory stores 6 6 extrinsic messages pertaining to a single row in each of the 6 block rows in H. The posterior messages are stored in an γ-memory pertaining to the 32 block columns of sub-matrices in H. The network shown in the figure is an Ω-network composed of six stages of programmable switches that route four bit messages to and from these decoders and γ-memory. The control bits of the network are stored in a bit π-sram (it stores 6 6=96 permutations, each permutation is on 64 elements; each stage requires 32 bits). Each decoder reads a pair of extrinsic messages from its local λ- Memory. In addition, the network reads two rows from γ-memory (2 64 messages) and routes the corresponding pair of posterior messages to the each of the 64 decoders. After a latency of three time units (c/2), the decoders start to generate new extrinsic messages in pairs which are stored in the local extrinsic memories. In addition, the decoders generate updated pairs of posterior messages as well which are routed back using the network to update γ-memory. The same operations are repeated for the remaining three block rows. The decoder processes n = BS messages in cd time units. Assuming the decoder operates at a frequency f, performs T decoding iterations, and messages are represented using b bits, the throughput attained by the decoder is Θ= b f B c T D S bits/sec. REFERENCES [] C. Berrou, A. Glavieux, and P. Thitimajshima, Near Shannon limit error-correcting coding and decoding: Turbo codes, in IEEE Int. Conf. on Comm., 993, pp [2] R. G. Gallager, Low-Density Parity-Check Codes, MIT Press, Cambridge, MA, 963. [3] D. Divsalar, H. Jin, and R. J. McEliece, Coding theorems for turbolike codes, in Proc. of the 36th Allerton Conf. on Comm., Control, and Computing, Sep. 998, pp [4] R. M. Tanner, A recursive approach to low complexity codes, IEEE Trans. on Info. Theory, vol. IT-27, pp , Sep. 98. [5] T. Richardson, M. Shokrollahi, and R. Urbanke, Design of capacityapproaching irregular low-density parity-check codes, IEEE Trans. on Info. Theory, vol. 47, no. 2, pp , Feb. 2. [6] S.-Y. Chung et al., On the design of low-density parity-check codes within.45 db of the Shannon limits, IEEE Comm. Let., vol. 5, no. 2, pp. 58 6, Feb. 2. [7] European Standard Telecommunications Series, Digital Video Broadcasting (DVB-S2) via Satellite standard, available at [8] The Working Group for Wireless LANs Standards, IEEE 82.n wireless mimo standard, available at [9] The Working Group for Wireless LANs Standards, IEEE 82.3 GBase-T task force, available at [] A. Blanksby and C. Howland, A 69-mW -Gb/s 24-b, rate-/2 lowdensity parity-check decoder, IEEE Journal of Solid-State Circuits, vol. 37, no. 3, pp , Mar. 22. [] X. Y. Hu et al., Efficient implementations of the sum-product algorithm for decoding LDPC codes, in Proc. IEEE Global Telecomm. Conf. 2 (GLOBECOM ), San Antonio, Texas, Nov. 2, pp E. [2] H. Jin, A. Khandekar, and R. J. McEliece, Irregular repeat-accumulate codes, in Proc. of 2nd Int. Conf. on Turbo Codes and Related topics, Brest, France, Sep. 2, pp. 8. [3] V. E. Benes, Optimal rearrangeable multistage connecting networks, Bell system technical journal, vol. 43, pp ,