T efficient coded modulation scheme [ 13, has found

Transcription

1 874 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 41, NO. 6, JUNE 1993 A Modified Design of Trellis-Coded MPSK for the Fading Channel Shalini S. Periyalwar, Member, IEEE, and Solomon M. Fleisher, Senior Member, IEEE Abstruct-Multiple trellis-coded modulation (MTCM) of MPSK employs separate designs, which are optimum at high, for the Rayleigh fading channel and the AWGN channel, in order to meet the different design criteria for the two channels. The separate designs are no longer optimum if the channel conditions are interchanged, if the channel is Rician, or if the s are moderate. In this paper, a modified design MTCM (MD) is considered, wherein it is shown that the classical AWGN channel design can be retained for the Rician fading channel (which is Rayleigh and AWGN at the extremes) with any degree of fading, by choosing an appropriate signal set of size Mmin. The signal set size Mmin depends on the throughput rate and the number of parallel transitions between a given pair of states. For a given throughput rate and signal diversity, the proposed MTCM (MD) schemes achieve a higher value of &(free) and the same or higher value of branch distance product when compared to existing MTCM schemes designed for the Rayleigh fading channel. The improved performance ensuing on the Rician fading channel is demonstrated by asymptotic coding gains with respect to existing MTCM schemes and simulation results for some of the codes. The effect of phase errors due to fading on the MTCM (MD) schemes which have increased signal set size in some cases is also studied by simulation. I. INTRODUCTION RELLIS-CODED modulation (TCM), a bandwidth- T efficient coded modulation scheme [ 13, has found practical application in high-speed data transmission modems for the band-limited telephone channel. Recent interest has focused on the design of TCM for the slow fading mobile satellite communication channel [2]-[6]. M-ary PSK, with its radially symmetric decision regions which are unaffected by the fading depth, is the modulation choice for these channels. The most extensive work to date on MPSK trellis codes for fading channels has been that of Divsalar and Simon. They have evaluated the performance of trellis-coded MPSK schemes on the fading channel [2]-[5], and have demonstrated that multiple TCM (MTCM) schemes transmitting k 2 2 MPSK signals per trellis branch provide performance gains on the fading channel [4], [5]. In [4], it has been shown Paper approved by the Editor for Data Communications and Modulation of the IEEE Communications Society. Manuscript received July 29, 1990; revised June 3, 1991 and November 8, This work was supported in part by the National Science and Engineering Research Council of Canada (NSERC) under Grant NSERC The authors are with the Department of Electrical Engineering, Technical University of Nova Scotia, Halifax, N.C. B3J 2x4. IEEE Log Number that the signal diversity and the product of branch distances play a major role in the performance of TCM schemes on the fading channel. In the conventional TCM scheme [l], where the output bits from a rate m/m + 1 encoder are assigned a signal from a 2m+1-ary signal set, the signal diversity cannot exceed one when parallel transitions exist. In the MTCM scheme [8], the output bits from a rate m/m + q encoder are assigned signal k-tuples from MPSK signal sets of size Mi, with m + q = ~f=llog2mi. When MTCM is employed, the signal diversity along parallel transitions can be increased to equal k. In the MTCM scheme with set partitioning optimum for the AWGN channel (MTCM (SPA)) [9], the signal diversity in the trellis with parallel transitions is less than k. The set partitioning proposed [4] for the fading channel (MTCM (SPF)) maximizes the signal diversity along the parallel transitions to k, but reduces the attainable d2(free). In the sequel, it will be shown to what extent the reduced d2(free) impairs the attainable performance when the fading is not Rician channel with strong direct signal component). In this paper, it will be shown that the classical AWGN channel design can be retained for the fading channel with any degree of fading, if carried out with the appropriate signal set size M = Mmin. The size of the signal set Mmin is shown to depend upon the number of parallel transitions between a given pair of states and the throughput rate. In the proposed MTCM scheme with a modified design (MTCM (MD)), the output bits from a rate m/m + 1 trellis encoder are assigned multiple signals from an Mm;,-PSK signal set. When compared to the MTCM (SPF) design, the MTCM (MD) scheme: 1) provides the same signal diversity and the same or increased values of the branch distance product, and 2) always leads to increased value of d2(free). As a result, the performance is always improved on the Rician (excluding Rayleigh) fading channel due to the increased d2(free), and is improved on any (including Rayleigh) fading channel when an increase in branch distance product P takes place. When compared to the MTCM (SPA) design, the MTCM (MD) scheme, under certain conditions, provides the same value of d2(free) as the MTCM (SPA) design, and is then asymptotically optimum for both the AWGN channel and the fading channels. In Section I1 of this paper, the design criteria for trellis codes on the AWGN channel and fading channels are reviewed, and the differences between the MTCM (SPA) and MTCM (SPF) schemes are discussed. The design and performance of /93$ IEEE I r-

2 PERIYALWAR AND FLEISHER: MODIFIED DESIGN OF TRELLIS-CODED MPSK FOR FADING CHANNEL 875 Fig. 1. A TCM system for coherent detection. 0 Channel the ITCM (MD) schemes are presented in Section 111, ani Section IV contains the discussion. 11. PERFORMANCE OF TRELLIS-CODED SCHEMES ON THE AWGN AND FADING CHANNELS The general structure of a TCM system for coherent detection is shown in Fig. 1. At the transmitter, the m + q(q 2 1) bits out of the trellis encoder are assigned k 2 1 signals from an MPSK signal set, where k is referred to as the multiplicity of the TCM scheme. The throughput T of the scheme is mlk b/s/hz, and for a given throughput, the number of parallel transitions in the trellis increases with k. The MPSK signal set mapping is carried out to meet the design criteria of interest. At the receiver, the MPSK signals disturbed by the channel are demodulated and Viterbi decoded. On the fading channel, interleaving and deinterleaving are employed at the transmitter and receiver, respectively. For a coded MPSK signal sequence x (x E C, where C is the set of allowable codeword sequences) and the corresponding received signal sequence y, the Viterbi decoder finds the signal sequence that is closest to the sequence of received signals in terms of the minimization of a distance metric m,(x, y). The decoder makes an error by selecting some sequence li: # x if the metric m(2,y) 5 m(x,y). The metric m(x,y) is some nonnegative function of x given y, and from the point of view of computation, it is desirable that the metric have an additive property [2] (for example, the maximum likelihood metric). The selection of the sequence 2 # x results in an error event. The average error event probability of the trellis-coded scheme is upper bounded by the union bound (41 x?ec where p(x) denotes the a priori probability of transmitting the sequence x, and P(x + 2) denotes the error event probability, i.e., the probability that the sequence x # x is the decoded sequence given that x is the transmitted sequence. On the AWGN channel, the error event probability is given as where E,/No is the M-ary signal, with E, being the symbol energy and NO the one-sided power spectral density of the input noise, L is the length of the error sequence x # x, and x1 and i l are signals in the sequences of x and 2, respectively. The error event probability on the fading channel with ideal interleavinddeinterleaving (assuming perfect recovery of frequency and phase) is conditioned on the fading amplitude vector p, which affects the received signal as y = ptx + n, and is given in [4] as L P(z ) ---f kip) 5 exp -- p;lxl - ill2. (3) ( 4No 1=1 Equation (3) applies to the case of ideal channel state information (CSI) being available at the receiver. The fading amplitude pl is an r.v. which is assumed to be constant over at least one symbol interval, and may be defined by the Rician probability density function (pdf) where y is the ratio of signal energies in the direct and scattered signal components. Here, y = 00 corresponds to the AWGN channel, and y = 0 indicates the presence of only the diffuse multipath signal and corresponds to Rayleigh fading. Divsalar and Simon have derived the error event probability bound for trellis-coded signals on the Rician fading channel for coherent and differentially coherent demodulation with and without CSI. Under the conditions of ideal interleavinddeinterleaving, coherent detection with ideal CSI and a Gaussian decoding metric, the error event probability has been derived as [4] L 1+Y P(x + 2) I n - l=1 1 + y + & 1x1 - & I2 where E, is the symbol energy averaged over the fading amplitude. By applying a simplification to (5) in terms of a high condition - n the error event probability on the fading channel is shown to be independent of d2(free) [4, eq. (S)] (accurate for the Rayleigh fading channel only). The expression (5) can be used to analyze the error events on the Rayleigh and Rician fading channels. Using (2) and (5), the following may be determined that [4]. 1) On the AWGN channel, the performance is guided by the minimum squared Euclidean distance of the error events [in particular, d2(free)]. 2) On the fading channel with small values of the fading parameter y and under high conditions (6), the performance is guided by the error event path with the smallest number of symbols at nonzero Euclidean distance (denoted as Lmin) and by the product P of these nonzero branch distances. 3) On the fading channel with values of y corresponding to the presence of a strong direct signal component orland under With respect to the signals on the correct path.

3 876 IEEE TRANSACI'IONS ON COMMUNICATIONS, VOL. 41, NO. 6, JUNE 1993 (a) A= 2.6 B= c= 3.5 D= 5, ,7.5. DI2-0, , ' J d'vnc)=2.34. L,,=2, (b) ' A= 3.7 B= , ' c= 5.7 o= Fig. 2. Two-state trellis for the rate 4/6 MTCM scheme with set partitioning for the AWGN channel and the fading channel. moderate conditions, the performance is guided by all of the conditions mentioned in statements (1) and (2) above. Thus, the design criterion for trellis-coded schemes on the AWGN channel is d2(free), while for the Rayleigh fading channel, the design criteria are the quantities Lmin and P. To satisfy the respective design criteria for MTCM schemes on the AWGN channel and on the fading channel [under the condition (6)], Divsalar and Simon have proposed two different set partitioning procedures for MTCM schemes. The signal assignments resulting from the two partitioning procedures are illustrated for the two-state rate 4/6 (k = 2) MTCM scheme with 8PSK signals [5] in Fig. 2. The set partitioning procedure for the AWGN channel design (SPA) begins with the division of the original MPSK signal constellation into two subconstellations, comprising the odd- and even-numbered signal points. Multiple (or k-tuple) signals from each of these subconstellations are assigned to alternate trellis states. For trellises with parallel transitions, this partitioning results in Lmin < k because the number (four) of MPSK signals in the subconstellation is less than the number (eight) of parallel transitions. To optimize the design for the fading channel [5] where Lmin = k is desired, the second set partitioning procedure (SPF) employs k-tuple signals from the unpartitioned signal constellation. The SPF design results in reduced d2(free) (= d2(a, B) + d ( A, C) = 4 x d2(0, 1) = 2.34) compared to that obtained by using the SPA design (d2(free) = d2(a, B) + d2(a, C) = d2(0, 2) + 2 x d2(0, 1) = 3.17) (refer to Fig. 2). For small values of y and for high for which this scheme is designed, it works very well because d2(free) has a negligible effect on the performance. For larger values of y, indicating fading with a strong direct signal component, or/and under moderate conditions, all three quantities influence the performance (see statement 3) above), and the reduced d2(free) does have an effect. A pictorial analysis of the factors that influence the performance is given in Fig. 3 for the two-state rate 4/6 MTCM scheme (Fig. 2). The upper bound to the error event probability (5) is plotted for the SPF design and the SPA design with various values of y. For each design, the error event probability P(z + 2) is calculated for the one-branch (L = Lmin, d2 > d2(free)) error event2 and the two-branch (L > Lmin, d2 = d2(free)) error event. These error events for *L is the number of symbols at nonzero Euclidean distance and d' is the sum of squared Euclidean distances along any error event path. = ). - ij lo' g : 10-2 P tl ( f 10-3 a f 104 I t 10.~ o-~ 10" (C) ( r > a 3 io8 a - P W t 10.~ 10.~ (b) (4 0 Fig. 3. Error event probability versus (Es/N~) for various values of y for the two-state rate 4/6 MTCM scheme (fading channel design). The curves are for the Lmin and &(free) error events. The worst case error event probability for the AWGN channel design is also illustrated. the SPF design may be identified in Fig. 2 as [(0, O,), (1,5)] and [(A, B), (A, C)], respectively (assuming that the all-zero path is the correct path). The results are given as two separate curves. Similarly, the error events for the SPA design in Fig. 2 are [(O,O), (0,4)] and [(A, B), (A, C)], respectively. For this design, a single curve is shown, with the points marked on this curve corresponding to the largest (predominant) error event probability due to either the two-branch error event (hence, d2(free)) (circled) or the one-branch error event (hence, Lmin). It is well known that as y + m, the d2(free) error event dominates the performance. For the SPF design, it can be seen that for y = 0 [Fig. 3(a)], the performance is dominated by the error event path with Lmin(= 2). As y increases, the d2(free) error event path begins to dominate the performance [Fig. 3(b)-(f)]. The at which the Lmin error event dominates the performance increases with increasing values of y. The SPA design with Lmin = 1 is the worst performer for y = 0; 10 [Fig. 3(a), (b)]. As y increases above these values, it can be seen that the SPA design performs better than the SPF design due to its larger value of d2(free). Fig. 3 therefore indicates that for small values of y and high 's, Lmin is clearly the design criterion; but for performance on fading channels with strong direct signal

4 PERIYALWAR AND FLEISHER: MODIFIED DESIGN OF TRELLIS-CODED MPSK FOR FADING CHANNEL 811 components or with moderate conditions, it is essential that d2(free) and Lmin be maximized simultaneously. These conclusions leave some room for possible improvement of the MTCM (SPF) design. It must be noted that in cases where MTCM trellises have parallel transitions and conventional TCM trellises have none, the conventional TCM trellis may provide a higher value of d2(free), L,;,, and P than the MTCM trellis. Compare the eight-state rate 2/3 conventional TCM code (d2(free) = 4.586, Lmin = 3, P = 8) to the eight-state rate 4/6 MTCM (SPA) scheme (d2(free) = 3.17, Lmin = 2, P = 4) both employing 8PSK signals. MTCM schemes are found to be superior to TCM schemes on the AWGN channel and on the fading channel when the number of states is very small, or when the number of states is large enough so that the MTCM trellis has no parallel transitions. In the next section, the proposed modified design of MTCM (MTCM (MD)) is discussed. The MTCM (MD) schemes are designed to achieve Lmin 2 k (by the appropriate choice of the signal set size Adm;,) when the set partitioning is carried out as for the AWGN channel design. A. The Signal Set Size 111. CODE DESIGN AND PERFORMANCE From each state in the trellis of a rate m/m + q trellis encoder, there are 2 transitions, each of which represents one of the 2 unique combinations of the m binary input bits. When the number of transitions is more than the number of states, there are parallel transitions in the trellis (the parallel transitions are accounted for by the uncoded bits in the trellis encoder). In general, the number of parallel transitions between a given pair of states for an S-state trellis shown in Fig. 4 can be given as where C is the trellis connectivity (C = 1/2 for a halfconnected trellis, C = 1 for a fully connected trellis). For example, the two-state rate 4/6 MTCM trellis code referred to earlier has Nparallel = 8. When they exist, the parallel transitions usually account for the L,;, error events. While designing the scheme for the fading channel, it is therefore important that the signal diversity is maximized for the parallel transitions, i.e., each of the signal k-tuples assigned to the parallel transitions have all distinct elements. For the MTCM (MD) scheme, the AWGN channel set partitioning is preserved, and the original MPSK signal constellation is first divided into odd and even subconstellations of size M/2 from which the k-tuple signals assigned to the trellis branches are constructed. Since the total number of k-tuples constructed from a given subconstellation should be at least equal to the 2 transitions from any given state, we require (7) Fig. 4. S-state trellis. (a) Fully connected (C = 1). (b) Half-connected (C = 1/2). This is, in fact, Ungerboeck s expanded set which is optimum for the AWGN channel, generalized into an MTCM scheme. From the requirement Lmin = k on the parallel transitions, it follows that the size of the subconstellation should be at least equal to the number of parallel transitions in order to avoid common elements in the signal k-tuples assigned to those transitions. Thus, we require M 2 2m+l -2- or M>- 2 cs cs. The minimum value of M that satisfies both (9) and (8) is denoted as Mmin. Mmin is determined by (8), i.e., equals the Ungerboeck s expanded set, when (9) log, CS 2 m - r. (10) In this case, the MTCM (MD) scheme provides the same value of d2(free) as the SPA design, along with the same value of Lmin > k and a higher (or equal) value of P as the SPF design. Hence, when (10) holds, the MTCM (MD) scheme is optimum for both the AWGN channel and the fading channel with any degree of fading. Interestingly, MTCM (MD) schemes sometimes provide higher values of P when compared with MTCM (SPF) (Table 111). This happens when the signal set size for the MTCM (SPF) design is not chosen in accordance with (8) (refer to the T = 2.5, S = 2, rate 5/6 code in Table 111). It must be noted that, in MTCM schemes (SPA and SPF designs), the number of states does not affect the signal set size. When Mmin is determined by (9), i.e., (10) does not hold, the signal set size required is larger than Ungerboeck s expanded set. This leads to reduced d2(free) when compared to the MTCM (SPA) design, and an increased sensitivity to phase jitter. However, the loss in d2(free) is less than with the MTCM (SPF) design, while Lmin and P are the same. Summarizing, we conclude that the proposed MTCM (MD) scheme, when compared to MTCM (SPF) with the same throughput, always achieves better performance on the fading channel with y > 0 due to increased d2(free). When the value of P is also higher than in the MTCM (SPF) design, the MTCM (MD) scheme achieves better performance on any fading channel. If (10) holds, the MTCM (MD) scheme does

5 878 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 41, NO. 6, JUNE 1993 not require an increase in signal set size with respect to the MTCM (SPA) or the MTCM (SPF) schemes, and consequently provides a single design optimum for both the AWGN and the fading channels. If (10) does not hold, Adm;" is double the size of the Ungerboeck expanded signal set, which leads to an increased phase error sensitivity. However, the increased value of d2(free) with respect to the MTCM (SPF) design may overcompensate for the deterioration in performance due to phase errors (as shown in the simulation example). These conclusions are clarified below by a comparative analysis of the proposed MTCM (MD) schemes and existing MTCM (SPF) schemes. B. MTCM (MD) Schemes for the Fading Channel MTCM (MD) schemes employ a rate m/m + 1 trellis encoder, with each of the 2m+1 unique combinations of output bits being assigned a phase-modulated signal k-tuple, with the signal set size Mmin determined by (8) and (9). The generation of subsets is shown in Table I for the (two-state) rate 314 and 415 MTCM (MD) schemes with multiplicity IC = 2, with the addition being modulo-mmin. In Table I, the Mmin/2-PSK signal subconstellations of even-numbered signals from the original Mm;,-PSK signal set are illustrated. Subsets A and B are constituted of these signals. Subsets C and D are constituted of the subconstellation of odd-numbered signals. These subsets are assigned to trellis transitions in the two-state trellis such that d2(free) is maximized. The generation of subsets for the (two-state) rate 415 MTCM (MD) scheme with multiplicity k = 4 is shown in Table I1 with the addition being modulo-mmin. Table 111 lists d2(free), L,;,, and P for the different schemes. It also provides a comparison to existing MTCM schemes on the basis of the same throughput rate r and &,in. The value of d2(free) has been calculated by using the generalized transfer function technique proposed by Biglieri in [7] for conventional TCM systems. Fig. 5 illustrates the trellis structures used for some of the MTCM (MD) schemes which are discussed below. 1) Two-State Rate 213 Code (C = 1, r = 1 b/s/hz, k = 2): For this code, employing signals from a QPSK signal set, set construction on the lines of Table I is quite straightforward. The signal assignment to the two-state trellis is as follows: A = [(00),(22}]; B = A + (02); C = A + { 13); D = B + (1 3). The trellis assignments and the performance of this code are the same as that for the rate 214 MTCM code of Divsalar and Simon [8]. 2) Four-State Rate 314 Code (C = 1, r = 1.5 b/s/hz, and k = 2): The signals are assigned to the trellis branches from an 8PSK signal set. Since Mmin is determined by (8) and is the same for S = 4 as for S = 2, subsets for this code may be derived by an odd-even split of the elements in the subsets given for the two-state trellis in Table I as A = [{00},{44)]; B = [(22},(66)]; C = A + (04); D = B + (04); E = A + (15); F = B + (15); G = C + { 15); H = D + { 15). For this code, the minimum Euclidean distance for an error event along the parallel path is equal to 2 x d2(0, 4) = 2 x 4sin2(7r/2) = 8. The minimum TABLE I GENERATION OF SUBSETS FOR THE TWO-STATE RATE 3/4, 4/5, AND 5/6 MTCM (MD) COnES WITH k = I I I rate 3/4 rate 4/5 I Subset A: 1. All pairs of points on axes All points on sub-axes with diagonal counterparts 3. All remaining points at 2700, I@, d;, (A Subset B: Add to subset A d;, (AB) Subset C, D: Add to subsets A, B dim (A.C ) dim (BSJ) TABLE I1 GENERATION OF SUBSETS FOR THE TWO-STATE RATE 4/5 AND 5/6 MTCM (MD) CODES WITH k = 4 1. All pairs of points on axes All points on sub-axes with diagonal counterparts All remaining points at 270, 1800,900 d:, (A Subset 8: Add to subset A squared Euclidean distance for the two-branch error event path is also equal to 2 x d2(0, 2) + d2(0,4) = 8sin2(7r/4) + 4sin2(7r/2) = 8, and therefore d2(free) = 8.0. The parallel paths have Lmin = 2 and P = 4sin2(7r/2) x 4sin2(7r/2) = r-

6 PERIYALWAR AND FLEISHER: MODIFIED DESIGN OF TRELLIS-CODED MPSK FOR FADING CHANNEL 879 TABLE I11 PERFORMANCE OF MTCM (MD) SCHEMES IN TERMS OF #(free) L,,, AND P 1 E-01 MTCM (MD) MTCM r S Code M k 4;- L,,, D Code M k 4;- L,,, D Design [5]Fading 1 2o a /M $ : 1: 1171 I TCM (conventional) is better I I I I I 2 5/ I 2 5 ; l5/ I I [5lFading I E CD DB X % I / / [5lFading / ABCD EFGH BADC FEHG A-H. b I.* - D n 1 E-02! 1E-03 F - m 1 E-04 1 MDIlOdB) -4-1E-05 I I I I d (db) Fig. 6. Bit error probability versus (Eb/No) for various values of y for the eight-state rate 3/4 (IC = 2, 8PSK) MTCM (MD) scheme and the eight-state rate 3/6 (IC = 2, 8PSK) MTCM scheme. I (4 (b) (C) Fig. 5. Trellis structures for: (a) two-state code, (b) four-state code, etc., MTCM (MD) scheme. 1E-01 L 16. This code achieves the same d2(free), L,i,, and P as the four-state rate 6/12 MTCM (SPA) scheme with 8PSK signals [9]. Note that the latter scheme with k = 4 has a higher multiplicity which may result in a larger error coefficient. 3) Eight-State Rate 314 Code (C = 1, r = 1.5 b/s/hz, k = 2): The 16 signal subsets for this code are derived by further splitting the eight signal subsets listed in 2) (because Mmin is the same as for the four-state code above). There are no parallel transitions. This code achieves d2(free) = d2(0,4) + d2(0, 1) + d2(0,3) = 4sin2(r/2) + 4sin2(r/8) + 4sin2(3r/8) = 8.0, Lmin = 3, and P = 8. The rate 316 (k = 2) MTCM (SPF) scheme with 8PSK signals [5] has d2(free) = 5.17, Lmin = 3, and P = 8. Although both schemes employ SPSK signals, a higher value of d2(free) is obtained with the MTCM (MD) code, giving an asymptotic coding gain of 1.9 db on the AWGN channel. The simulation results in Fig. 6 for this code are discussed in Section 111-C. 4) Two-State Rate 415 Code (C = 1, r = 2 b/s/hz, k = 2): The subsets for this code (Mmin = 16) are given in Table I. With subsets A and C assigned to state 0, and subsets D and B assigned to state 1, this code has a minimum squared Euclidean distance along the parallel path = 4sin2(r/8) + 4sin2(5=/8) = 4.0, Lmin = 2, and P = (4sin2(r/8) x 4sin2(5=/8)) = 2. The two-branch error event achieves d2(free) = 2 x (4sin2(r/16) + 4sin2(3r/16)) = The two-state rate 4/6 (k = 2) MTCM (SPF) scheme [5] with 8PSK signals has d2(free) = 2.34, Lmin = 2, and P = 2. For the same values of Lmin and P, the MTCM (MD) scheme has a higher value of d2(free). 5) Four-State Rate 4/5 Code (C = 1, r = 2 b/s/hz, k = 2): For this code (Mmin = 8), the elements of the eight 1E-05 I I I I I (db) Fig. 7. Bit error probability versus (Eb/No) for various values of y for the four-state rate 4/5 (IC = 2, SPSK) MTCM (MD) scheme and the four-state rate 4/5 (IC = 2, 4/8PSK) MTCM scheme. signal subsets are as follows: A = [ { 0 0}, { 2 2}, { 4 4), { 6 6)]; B = A + (04); C = A + (02); D = A + (06): E = A + (15); F = B + (15); G = C + (15); and H = D + { 1 5). Subsets A-D and subsets E-H are assigned to alternate trellis states, giving d2(free) = d2(0, 2) + 2d2(0, 1) = 3.17, Lmin = 2, and P = 4.0. The four-state rate 415 (IC = 2) MTCM scheme with a hybrid QPSK/8PSK signal set [5] has d2(free) = 2.586, Lmin = 2, and P = The larger values of d2(free) and P result in improved performance. The simulation results presented for this code in Fig. 7 are discussed in Section 111-C. 6) Four-State Rate 516 Code (C = 1, r = 2.5 b/s/hz, IC = 2): For this code (Mmin = 16), the signal sub-

7 880 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 41, NO. 6, JUNE E-02 t (db) Fig. 8. Bit error probability versus (E*/No) for various values of y for the four-state rate 5/6 (k = 2, 8PSK) MTCM (MD) scheme and the four-state rate 5/6 (k = 2, 8PSK) MTCM scheme. MD (phase error) f I =.nc I- "" (db) Fig. 9. Bit error probability versus (Eb/>Vo) for various values of y for the four-state rate 5/6 (k = 2, 8PSK) MTCM (MD) scheme and the four-state rate 5/6 (k = 2, BPSK) MTCM scheme in the presence of phase jitter. sets are as follows: A = [(00), (2 lo), {44), (6 14}, (88), {102},(1212},{146}]; B = A + (08); C = A + (04); D = A + (012); E = A + (15); F = B + (15); G = C + (1 5); and H = D + (1 5). This code achieves d2(free) = 4 x d2(0,2) = 2.34, Lmin = 2, and P = 2. The four-state rate 516 (IC = 2) MTCM (SPF) scheme with 8PSK signals [5] gives d2(free) = 1.757, Lmin = 2, and P = The simulation results presented for this code in Fig. 8 are discussed in Section 111-C. 7) Two-State Rate 415 Code (C = 1, r = 1 b/.s/hz, k = 4): This code with Mmin = 16 achieves d2(free) = 2(d2(0, 2) + d2(0, 10)) = 8.0 and P = 4.0 for the error event along a parallel transition. The two-state rate 4/12 MTCM (SPF) scheme with 8PSK signals [5] achieves d2(free) = 4.68 and P = 4.0. Again, a larger value of d2(free) is obtained by the MTCM (MD) code. C. Performance On the Rayleigh fading channel, for a given value of r and L,;,, the performance of the MTCM (MD) schemes is equivalent to the MTCM (SPF) schemes (if and when both codes have the same branch distance product P). The advantages of the MTCM (MD) schemes are greatest on the fading channel with y > 0, or under moderate conditions, when the performance is influenced by all three parameters d2(free), L,;,, and P (as discussed in Section 11). For given values of r and L,i,, the gains offered by the increased d2(free) (and P) in MTCM (MD) schemes as compared to MTCM schemes are illustrated for some of these codes in Figs. 6-9 by simulation results for the bit error probability P,, versus = Eb/No, where E,, is the unfaded bit energy. The simulation programs used to obtain the plots in Figs. 6-9 are adapted from the software developed for the shadowed Rician fading model in the Canadian Mobile Satellite Communications (MSAT) Program [lo]. A convolutional interleaver-deinterleaver [ 111 is employed, with the degree of interleaving equal to 4 and the depth of interleaving equal to 1. Coherent demodulation of the MPSK signals is implied. It is assumed that the phase of the received signal is unaffected by the fading process. The Viterbi algorithm with the minimum distance metric is used to decode the trellis-coded symbols. Amplitude CSI is used in the decoding process. The bandwidth B of the fading process, normalized with respect to the symbol rate, is assumed to be 0.3. The effect of phase jitter due to the fading process is included only in Fig. 9, where the two codes compared have different signal set sizes. Simulation results are presented for different values of y, including y = cc (AWGN channel). The curves plotted in Fig. 6 are for the eight-state rate 314 (IC = 2, 8PSK) MTCM (MD) scheme (L,in = 3, d2(free) = 8.0, P = 8) discussed in Section III-B3), and for the eight-state rate 316 (IC = 2, 8PSK) MTCM (SPF) scheme [5] (L,in = 3, d2(free) = 5.17, P = 8). Since the values of Lmin and P are the same of both schemes, it can be expected that both schemes have similar performance on the Rayleigh fading channel (y = 0), as confirmed by the simulation results. Elsewhere in Fig. 6, the MTCM (MD) scheme has improved performance due to the higher value of d2(free). The improvement is already obvious for y = 10 db, and increases with y. While the coding gains are small over the range of 's plotted, the asymptotic (high ) coding gain due to d2(free) is 1.9 db. Fig. 7 illustrates the bit error probability for the fourstate rate 415 (IC = 2, 8PSK) MTCM (MD) scheme with Lmin = 2, d2(free) = 3.17, and P = 4 [see Section III- BS)], and for the four-state rate 415 (IC = 2, 8PSK) MTCM (SPF) scheme [5] with Lmin = 2, d2(free) = 2.586, and P = In this case, the gains in both P and &(free) contribute to the improved performance of the MTCM (MD) scheme at all values of y. At y = 0, the

8 PERIYALWAR AND FLEISHER: MODIFIED DESIGN OF TRELLIS-CODED MPSK FOR FADING CHANNEL 88 1 performance gain is due to the larger value of P and is asymp- presented in Figs. 6-9 for some illustrative cases indicate totically equal to 10 loglo(p~~tc~~ (~~D)/PMTCM)~I~~~~ = that the MTCM (MD) schemes provide performance gains 1010g,~(4/1.17)~/~ = 2.67 db. On the AWGN channel, the over MTCM (SPF) schemes, with no penalty in bandwidth or performance is improved due to the coding gain of 0.9 db in complexity. The gains are small for the range of values d2(free). The smaller gain in d2(free) as compared to the gain plotted, but they increase to provide the asymptotic values at in P results in a smaller improvement in performance on the high 's. AWGN channel as compared to the Rayleigh fading channel. The MTCM (MD) schemes differ from MTCM (SPA) For y = 10 db, the performance improvement is due to a schemes in that an additional constraint Lmin 2 k is imposed. mixture of the gains in d2(free) and P. This constraint sometimes limits the achievable value of The curves in Fig. 8 are for the four-state rate 5/6 (k = 2, d2(free) to values that are lower than that obtained when 16PSK) MTCM (MD) scheme with Lmin = 2, d2(free) the schemes are designed to maximize d2(free) alone (e.g., = , and P = 2 [see Section III-B6)], and the four-state compare the two-state codes for the rate 4/6 MTCM [(SPF) rate 5/6 (k = 2,8PSK) MTCM (SPF) scheme [5] with Lmin = and (SPA)] schemes and the rate 4/5 MTCM (MD) scheme]. 2, d2(free) = and P = The asymptotic gain in However, when the value of Mmjn is determined by (S), which P of 10l0g,,(2/1.17)~/~ = 1.2 db, and in d2(free) (also is the redundant signal set size of Ungerboeck, the MTCM 1.2 db), results in improved performance on the Rayleigh, (MD) design proposed for the fading channel is optimum for Rician, and AWGN fading channels. Again, the nature of the AWGN channel as well (the eight-state rate 3/4 MTCM the performance gains on these channels is similar to that (MD) scheme serves as an example). described in the last paragraph. For this MTCM (MD) scheme, The size of the signal set Mmin depends on the signal which has an increased signal set size, it is of interest to multiplicity (throughput rate) and the number of parallel examine if the performance gains will prevail when the effect transitions in the trellis. In contrast, the signal set in MTCM of phase noise due to fading is included. In Fig. 9, the results (usually 8PSK or lower) is independent of the number of are plotted for the y = 10 db Rician fading channel, with parallel transitions in the trellis. This is, in fact, the basic a fading bandwidth of 0.3. For this channel, the phase error difference between the MTCM (MD) design and the MTCM standard deviation was observed to be 18.62'. The probability (SPA) design. It must be observed that the restriction of distribution of the phase error on the Rician fading channel signal set size to 8PSK (or lower) implies that it will not is given in [12]. The sensitivity of the schemes to phase be possible to construct MTCM schemes with more than error was studied by attenuating [lo] the phase error by an eight parallel transitions to achieve Lmin = IC. Thus, for arbitrary factor of 4 (assuming that the phase recovery circuits will compensate for some of the distortion introduced by the channel). Again, the MTCM (MD) scheme demonstrates performance gains over the MTCM (SPF) scheme, although the latter scheme demonstrates less sensitivity to phase error than the former. IV. DISCUSSION Under the conditions (6) of high and fading characteristics close to the Rayleigh model (y = 0), trellis code design for the fading channel has been based [4] on maximizing the minimum signal diversity Lmin and P. However, under moderate conditions, or in the case of fading with strong direct signal reception (y > 0), the AWGN channel design criterion d2(free) also plays a role, along with the quantities Lmin and P. This observation makes it possible to improve the code design and the attainable performance on the Rician fading channel with y > 0. The curves of error event probability versus presented in Section I1 illustrate the potential gains that may be obtained. One way to realize an improvement in the performance on the fading channel is the proposed modified design of trellis-coded MPSK, MTCM (MD), based on set partitioning optimum for the AWGN channel, but with a signal set of size Mmin chosen to meet the fading channel design criteria. Table I11 contains the values of d2(free), Lmi,, and P for the MTCM (MD) schemes along with results for the MTCM (SPA and SPF) schemes for comparison. The simulation results a rate 5/6 code with k = 2, the two-state trellis with 16 parallel transitions will only have L = 1 (< k). On the other hand, the larger signal set size (16PSK) used for some of the codes in this paper is prone to larger errors due to phase noise. This issue is addressed in the simulation results presented in Fig. 9 for the four-state rate 5/6 MTCM (MD) scheme (16PSK signals) in comparison with the four-state rate 5/6 MTCM (SPF) scheme (SPSK signals). It is observed that the performance gains offered by the MTCM (MD) scheme prevail, although the MTCM (MD) scheme is clearly more sensitive to phase jitter than the MTCM (SPF) scheme. Under severe phase error conditions, the performance of the MTCM (MD) schemes with larger signal set sizes (e.g., the rate 5/6 code discussed above) may be worse than their MTCM (SPF) counterparts. Note, however, that the size of the signal subset assigned to transitions between a given pair of states is the same for both schemes since the MTCM (MD) scheme contains only even-numbered or only odd-numbered signals in the subset, while the MTCM (SPF) scheme contains all the signals in the subset. Therefore, with respect to subset decoding, both schemes are affected by phase errors to the same extent. The encoder-modulator for multiple trellis-coded schemes can be described by the Calderbank-Mazo representation [ 131. Fig. 10 is an implementation of the encoder-modulator for the four-state rate 3/4 (k = 2, SPSK) MTCM (MD) scheme. For the three input bits bl, b2, and b3, the output signal two-tuples are x1 = 4b3 + b4 + 2blb5 and 22 = b4 + 2blb5 + 4b2b3b4, with b4 and b5 being the delayed bits. The maximum likelihood

9 882 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 41, NO. 6, JUNE 1993 b2 b, B [7] 4 E. Biglieri, High-level modulation and coding for nonlinear satellite channels, IEEE Trans. Commun., vol. COM-32, pp , May [8] D. Divsalar and M. K. Simon, Multiple trellis coded modulation, IEEE Trans. Commun., vol. 36, pp , Apr x2 (91 -, Generalized multiple trellis coded modulation (MTCM), in IEEE Global Telecommun. Conf Rec., Tokyo, Japan, Nov [lo] P.J. McLane, P.H. Wittke, P. K.-M. Ho, and C. Loo, PSK and DPSK trellis codes for fast fading shadowed mobile satellite communication channels, IEEE Trans. Commun., vol. 36, pp , Nov [ll] A. Lee and P. J. McLane, Convolutionally interleaved PSK and DPSK trellis codes for shadowed fast fading mobile satellite communication channels, IEEE Trans. Vehic. Technol., Feb [12] K. A. Norton, E. L. Shultz, and H. Yarbrough, The probability distribution of the phase of the resultant vector sum of a constant vector plus a Rayleigh-distributed vector, J. Appl. Phys., vol. 23, pp , Jan [13] A. R. Calderbank and N. J. A. Sloane, New trellis codes based on lattices and cosets, IEEE Trans. Inform. Theory, vol. IT-33, pp , 2 Mar Fig. 10. Calderbank- Mazo representation of the encoder-modulator for the eight-state rate 3/4 MTCM (MD) scheme. decoder for trellis-coded schemes is the well-known Viterbi decoder. ACKNOWLEDGMENT The authors wish to thank the anonymous reviewers for their valuable comments and suggestions, and the Telecommunications Research Institute of Ontario for the simulation software. Shalini S. Periyalwar (S 88-M 90) received the B.Eng. degree in electrical engineering from Bangalore University, India. Since 1985, she has been with the Department of Electrical Engineering, Technical University of Nova Scotia, Halifax, Canada, where she is currently an Assistant Professor under the Burchill Telecommunications Research Program. Her current research interest is in the area of digital communications. REFERENCES [31 [41 [51 G. Ungerboeck, Channel coding with multilevel/phase signals, IEEE Trans. Inform. Theory, vol. IT-28, pp , Jan D. Divsalar and M.K. Simon, Trellis coded modulation for 4800 to 9600 bps transmission over a fading satellite channel, IEEE J. Select. Areas Commun., vol. SAC-5, pp , Feb M.K. Simon and D. Divsalar, The performance of trellis coded multilevel DPSK on a fading mobile satellite channel, IEEE Trans. Vehic. Technol., vol. 37, May D. Divsalar and M.K. Simon, The design of trellis coded MPSK for fading channels: Performance criteria, IEEE Trans. Commun., vol. 36, pp , Sept , The design of trellis coded MPSK for fading channels: Set partitioning for optimum code design, IEEE Trans. Commun., vol. 36, pp , Sept C. Schlegel and D. J. Costello, Bandwidth efficient coding for fading channels: Code construction and performance analysis, IEEE J. Select. Areas Commun., vol. 7, pp , Dec Solomon M. Fleisher (M 72-SM 80) received the M.S. and Ph.D. degrees from the Institute of Communications Engineering, St. Petersburg, Russia, in 1954 and 1965, respectively. From 1954 to 1962 he worked on the design of broadcast receivers in Riga, Latvia. In 1962 he joined the Institute of Communications Engineering, St. Petershurg, where he was (from 1968) an Associate Professor. In 1972 he joined the Ben-Gurion University, Beersheva, Israel, and in 1975 the Halon Institute of Technology, Tel-Aviv University. Since 1982 he has been a Professor of Electrical Engineering at the Technical University of Nova Scotia, Halifax, Canada. His current research interests include digital communications and communication theory.