Principles of Advanced Digital Communications and Coding

Transcription

1 现代数字通信与编码理论 讲义 Principles of Advanced Digital Communications and Coding 白宝明 西安电子科技大学 综合业务网国家重点实验室 010 年 7 月 1

2 Notice This is a draft. The notes are work in progress. Comments will be much appreciated; please send them to me at bmbai@mail.xidian.edu.cn

3 Course Information for 019 现代数字通信与编码理论 (Principles of advanced digital communications and coding) The goal of this class is to introduce the information transmission techniques used in modern communication systems, with emphasis on information-theoretic and advanced coding aspects. This is done by understanding the following course contents: 各种信道模型 ( 包括功率受限 带宽受限 ISI 衰落 多天线等 ) 及其 Shannon 容量的计算 ; 最新的可达容量限的信道码的编译码原理 ; 现代编码通信系统的性能分析技术 Prerequisite: Principles of communications Error control coding (preferable but not necessary) Instructor: Prof. Baoming BAI Assistant: Time and place: Monday 8:30 10:05 a.m. and Wednesday 3:35 5:10 p.m. in Classroom J-04 Grading: 50% Homework 50% Project (The project will involve in writing a report as well as an oral presentation) Class WWW page: Outline Preliminaries - Phase splitter and analytic signal - Complex baseband representation of passband signals - Signal space representations - Circularly symmetric Gaussian processes - Some facts from information theory Digital Transmission of Information over Ideal AWGN Channels (10 hours) - Discrete-time AWGN channel model - Signal constellation - PAM and QAM transmission systems - Capacity for M-PAM and M-QAM signaling - The gap between uncoded performance and the Shannon limit - Performance analysis of small signal constellations - Design of signal constellations Performance Analysis of Coded Communication Systems - Approaching capacity with coding 3

4 - Techniques for performance analysis of coded communication systems - Bhattacharyya bound and Gallager bound Introduction to Modern Coding Theory (8 hours) - Trellis representation of codes and decoding on a trellis (linear block codes, VA, BCJR) - Turbo codes and the iterative decoding principles - Performance analysis - Codes defined on graphs and the sum-product algorithm - LDPC codes Bandwidth-Efficient Coded-Modulation Techniques (for Ideal Band-limited Channels) (8 hours) - Lattice constellations - Shaping gain - TCM principles and performance analysis techniques - Multi-dimentional TCM and multiple TCM - Turbo-TCM codes - Multilevel coding and multistage decoding - Bit-interleaved coded modulation using Turbo codes and LDPC codes (Gallager mapping) - Constellation shaping techniques Transmission over Linear Gaussian Channels (6 hours) - Linear Gaussian channels - Equivalent discrete-time model - Principles of water pouring and evaluation of the channel capacity - Optimal receiver in the presence of both ISI and AWGN - Optimal detection: MAP, ML sequence detection - Symbol-by-symbol equalization methods: MMSE-LE, ZF-LE and MMSE-DFE - Tomlinson-Harishima precoding - Coding for ISI channels - Principles of Turbo equalizations - Approaching capacity with parallel transmission: COFDM Communications over Fading Channels (5 hours) - Wireless channel models - Capacity of wireless channels - Diversity techniques - Coding for fading channels (including adaptive coding & modulation) - Bound on the probability of decoding error - Information-theoretic aspects of spread-spectrum communications MIMO Wireless Communications (5 hours) - Multi-antenna (MIMO) channel models - Capacity of MIMO wireless channels - Diversity and spatial multiplexing - Approaching capacity with space-time coding - Performance analysis and design criteria for space-time codes on fading channels - Various space-time coding schemes 4

5 References [1] C. E. Shannon, A mathematical theory of communication, Bell Syst. Tech. J., vol. 7, pp , , July-Oct. 1948; Reprinted in C. E. Shannon and W. Weaver, The Mathematical Theory of Communication. Urbana, IL: Univ. Illinois Press, [] R. G. Gallager, Claude E. Shannon: A Retrospective on His Life,Work, and Impact, IEEE Trans. Inform. Theory, vol.47, no.7, pp , Nov [3] G. D. Forney, Jr. and G. Ungerboeck, Modulation and coding for linear Gaussian channels, IEEE Trans. Inform. Theory, vol.44, no.6, pp , Oct [4] E. Biglieri, J. Proakis, and S. Shamai (Shitz), Fading channels: Information-theoretic and communication aspects, IEEE Trans. Inform. Theory, vol.44, pp , Oct [5] D. J. Costello, J. Hagenauer, H. Imai, and S. B. Wicker, Applications of error-control coding, IEEE Trans. Inform. Theory, vol.44, no.6, pp , Oct [6] A. R. Calderbank, The art of signaling: Fifty years of coding theory, IEEE Trans. Inform. Theory, vol.44, no.6, pp , Oct [7] J. G. Proakis, Digital Communications. 4 rd ed. New York: McGraw-Hill, 000. [8] E. A. Lee and D. G. Messerschmitt, Digital Communication, nd ed. Kluwer Academic Publishers, Boston, [9] G. D. Forney and R. Gallager, Principles of Digital Communications. Course notes. MIT. [10] R. G. Gallager, Information Theory and Reliable Communication. New York: John Wiley and Sons, [11] T. M. Cover and J. A. Thomas, Elements of Information Theory. Wiley, [1] J. L. Massey, Applied Digital Information Theory. Course notes. ETH. [13] S. G. Wilson, Digital Modulation and Coding. Prentice-Hall, [14] E. Biglierli, D. Divsalar, P. J. McLane, and M. K. Simon, Introduction to Trellis-Coded Modulation with Applications. New York: MacMillan, [15] D. N. C. Tse and P. Viswanath, Fundamentals of Wireless Communication. Cambridge University Press, 005. [16] A. Goldsmith, Wireless Communications. Cambridge University Press, 005. [17] T. J. Richardson and R. L. Urbanke, Modern Coding Theory. Course notes. EPFL. [18] C. Schlegel and L. Perez, Trellis and Turbo Coding. IEEE Press, 004. [19] Proceedings of The IEEE, Special issue on wireless commun., vol.9, no., Feb [0] IEEE Signal Processing Magazine, Jan [1] IEEE Communication Magazine, Aug [] M. Medard and R. G. Gallager, Bandwidth scaling for fading multipath channels, IEEE Trans. Inform. Theory, vol.48, no.4, pp , April 00. [3] I. C. Abou-Faycal, M. D. Trott, and S. Shamai (Shitz), The capacity of discrete-time memoryless Rayleigh-fading channels, IEEE Trans. Inform. Theory, vol.47, no.4, pp , May 001. [4] E. Biglieri, G. Caire, and G. Taricco, Limiting performance of block-fading channels with multiple antennas, IEEE Trans. Inform. Theory, vol.47, no.4, pp , May 001. [5] G. J. Foschini and M. J. Gans, On limits of wireless communications in a fading environment when using multiple antennas, Wireless Personal Communications, vol.6, no.3, pp , Mar [6] I. E. Telatar, Capacity of multi-antenna Gaussian channels, European Trans. Telecomm., vol.10, no.6, pp , Nov.-Dec

6 [7] C. Berrou and A.Glavieux, Near optimum error correcting coding and decoding: Turbo-codes, IEEE Trans. Commun., vol. 44, no.10, pp , Oct [8] C. Heegard and S. B. Wicker, Turbo Coding, Norwell, MA: Kluwer, [9] B. Vucetic and Jinhong Yuan, Space-Time Coding, Wiley, 003. [30] Shu Lin and D. J. Costello, Jr. Error Control Coding: Fundamentals and Applications. nd ed. Prentice-Hall, 004. [31] William E. Ryan and Shu Lin, Channel Codes: Classical and Modern. Cambridge University Press, 009. [3] Proceedings of The IEEE, Special issue on Turbo-information processing: Algorithms, implementations and applications, vol.95, no.6, June

7 019 现代数字通信与编码理论 September 1, 010 XDU, Fall 010 Lecture Notes Introduction Digital communication is a field in which theoretical ideas have had an unusually powerful impact on actual system design. The basis of the theory was developed 58 years ago by Claude Shannon, and is called information theory. The goal of this course is to get acquainted with some of these ideas and to gain deep understanding on how to efficiently and reliably communicate through a channel, especially to better understand the advanced techniques for signal transmission and coding used in modern digital communication systems. We will focus on point-to-point systems consisting of a single transmitter, a channel and a receiver. A. Block diagram of a digital communication system In 1948, Claude E. Shannon of the Bell Telephone Laboratories published one of the most remarkable papers in the history of engineering. This paper ( A Mathematical Theory of Communication", Bell System Tech. Journal, Vol. 7, July and October 1948, pp and pp ) laid the groundwork of an entirely new scientific discipline, information theory", in which Shannon first introduced the following figure to model a digital communication system. Channel encoder 信源信源编码器 ECC 编码器数字调制器 Channel decoder 信宿信源译码器 ECC 译码器数字解调器 信道 n(t) 干扰 调制信道 bits symbols waveform The source encoder involves the efficient representation of source signals. It has the function of converting the input from its original form, e.g., speech waveforms, image waveforms, and text, into a sequence of bits. The objective of doing this is as efficiently as possible. i.e., transmitting as few bits as possible, subject to the need to reconstruct the input adequately at the output. In this case source encoding is often called data compression. Shannon showed that the ultimate data compression is the entropy of the source. The channel encoder box in the figure above has the function of mapping the binary sequence at the source/channel interface into channel inputs. The channel inputs might be waveforms, or might be discrete sequences. The general objective here is to map binary inputs at the maximum bit rate possible into waveforms or sequences such that the channel decoder can recreate the original bits with low probability of error. One simple approach to performing this is called modulation and demodulation. From the geometric signal-space viewpoint, the modulation process may be thought of a two-step process: first mapping 7

8 binary digits into signals (e.g. signal levels) and then signals into waveforms. Since high error probability is frequently incurred with simple modulation and demodulation in the presence of noise, the error-correcting codes was introduced and the channel coder is separated into two layers, first an error-correcting encoder, and then a simple modulator. Shannon showed that, with appropriate coding schemes, arbitrarily low error probabilities can be achieved at any data rate below a certain data rate called the channel capacity. By the 1980 s, channel coding usually involved a two layer system similar to that above, where an error-correcting code is followed by a modulator. At the receiver, the waveform is first demodulated, and then the error correction code is decoded. Since the Ungerboeck s work in 198, it has been recognized that coding and modulation should be considered as a unit, resulting in the schemes called coded modulation. In such schemes, the lower error probability can be achieved without sacrificing bandwidth efficiency. The purpose of the modulation system is to create a good discrete channel from the modulator input to the demodulator output, and the purpose of the coding system is to transmit the information bits reliably through this discrete channel at the highest practicable rate. -- Massey In this course, we will study the concepts and fundamental principles involved in advanced digital communication systems. We will focus on the channel coding component in the above figure. As we will see later, many advanced techniques used in modern digital communication systems (including mobile communication systems) are developed using information-theoretic ideas. This course will attempt to reflect these new evolutions. Some of exposition has benefited from the excellent notes written by Gallager and Forney for the MIT courses and We will present the material in such a unified way that the channel model and the corresponding channel capacity are introduced first, and then the coding and signal process techniques for approaching these optimal performance limits are presented, and followed by the discussion on the performance of the actual systems with these channel coding schemes. B. Relevant results from information theory The communications problem can be broken down without loss of reliability or efficiency into the separate components shown in the above diagram.reliable communication can be achieved at any rate below the capacity of the communications channel. We add controlled redundancy to data transmitted over the channel. This redundancy lowers the raw data rate, but reduces the error rate after using the redundancy to correct errors. (distance gain) The net effect is to increase the rate at which clean data is delivered C. Historical notes Hamming codes: 1950 Convolutional codes: 1955 (by Elias) BCH, Reed-Solomon codes: 1960 LDPC codes: 196 (by Gallager) (rediscoved in late 1990 s) Concatenated codes: 1966 (by Forney) Viterbi algorithm: 1967 TCM: 198 (by Ungerboeck) 8

9 Turbo codes: 1993 (by Claude Berrou) Space-time codes: 1998 (by V. Tarokh) Dirty-paper coding, Cooperation via distributed coding and network coding: 000- Most of important achievements in digital communications are based on the results of information theory and coding. D. Giants in the field of digital communications Harry Nyquist (198) Analog signals of bandwidth W can be represented by W samples/s Channels of bandwidth W support transmission of W symbols/s Claude Shannon (1948) 4/30/1916 /4/001 His information theory addressed all the big questions in a single stroke. He thought of both information sources and channels as random and used probability models for them. Most modern communication systems are designed according to the principles laid down by Shannon. We conclude this section, which should have provided some motivation for the use of coding, with an adage from R. E. Blahut: To build a communication channel as good as we can is a waste of money use coding instead! 9

10 019 现代数字通信与编码理论 September 1, 010 XDU, Fall 010 Lecture Notes Chapter 1 Preliminaries In this chapter we will briefly review some basic concepts and principles, which will be used as the basis of discussions later. A. Phase splitter and analytic signal If x(t) is a real-valued signal, then its Fourier transform X(f) satisfies the symmetry property * X ( f ) = X ( f) where X * (f) is the complex conjugate of X(f). The symmetry property says that knowing X(f) for f 0 is sufficient to entirely describe X(f) and thus to describe x(t). A phase splitter (also known as Hilbert filter) is a complex filter with impulse response h + (t) and transfer function H + (f), where 1, f 0 H + ( f ) = 0, f < 0 x(t) h + (t) x A (t) x(t) j x A (t) 1 πt xˆ( t) Figure 1.0 A Hilbert filter If the real-valued input to a phase splitter is x(t), then the output is 1 x () [ () ˆ A t = x t + jx()] t, or X ( ) ( ) ( ) A f = X f H+ f where xˆ( t ) is the Hilbert transform of x(t). We introduce the factor so that x(t) and x A (t) have the same energy (or power). Notice that x A (t) is a complex-valued signal. A signal with only nonnegative frequency components is called an analytic signal. x(t) can be recovered from x A (t) by x() t = R x () t [ ] A 10

11 B. Complex baseband representation of passband signals Suppose that x(t) is a real-valued passband signal with a spectrum centered at f = f c. The complex baseband equivalent signal (sometimes also called complex envelope) of x(t) can be represented as 1 jπ fct x ˆ b() t = [ x() t + jx()] t e (1.1) analytic passband signal x A (t) In terms of Fourier transforms X( f + fc), f + fc 0 Xb( f) = X A( f + fc) = 0, f + fc < 0 The original passband signal can be recovered from x b (t) by j fct xt () = R xb () te π (1.) The relationship between x( t), xa( t) and xb( t ) is shown in Fig. 1.1 in terms of their spectrum. X(f) W -f c 0 f c X A (f) X b (f) 0 f c 0 W/ Figure 1.1 Fourier transform of a passband signal x(t) and the transform of the corresponding complex baseband signal. Baseband to passband and go back 11

12 x b (t) x A (t) x(t) y(t) R[ ] channel h ( t) + x b (t) 0 e π j ft c 0 f c -f c 0 f 0 f c 0 c j π ft c e alternative y(t) e jπ ft c -f c 0 LPF x b (t) f ( t) Figure 1. Baseband equivalent channel (at carrier frequency f c ) 1 h () [ () ˆ A t = h t + jh()] t jπ fct h () t = h () t e b A An alternative representation of a real signal is derivative of the complex envelope representation. The real and imaginary parts of the complex envelope xb ( t ) are referred to as the in-phase and quadrature components of x(t), respectively, and are denoted by xi( t) = R { xb( t)} and xq() t = I { xb()} t. From (1.), we have the in-phase and quadrature representation of a real signal x(t) given by j fct xt () = R xb () te π = R x ( t) cos( π f t) I x ( t) sin( π f t) [ ] [ ] b c b c = xi( t)cos( π fct) xq( t)sin( π fct) (1.3) A quadrature modulator performing upconversion and a quadrature demodulator performing downconversion are shown in Fig. 1.3, respectively. 1

13 Figure 1.3 Quadrature modulation and demodulation C. A complete system diagram The next step in implementing a digital communication system is to convert the discrete-time signal sequence into a baseband waveform (such as a PAM or QAM modulated waveform), and vice versa. This is performed via baseband modulation and demodulation. For example, with QAM transmission, the baseband complex waveform can be expressed as x () t = x p( t nt) = ( x + jx ) p( t nt) b n n n n n Z where {x n } is a discrete-time sequence of complex symbols to be transmitted, T is the symbol interval/duration, and x n = R { xn} and x n = I { xn}. The real waveform p(t) is a basic modulation pulse. At the receiver, the sequence {x n } can be retrieved from the sampled outputs yn = yb( nt). Figure 1.4 shows a complete system diagram with p() t = sinc() t which is defined as sin( πt) sinc( t)= π t Figure 1.4 A complete system diagram from the baseband transmitted symbol to the baseband received symbol. D. Signal space representations A signal space is a linear space (or vector space) in which vectors represent signals. n In an n-dimensional complex vector space C, the inner product of two vectors u = ( u1,..., u n ) and v = ( v1,..., v n ) is defined as n * uv, = uv i i i= 1 A vector space equipped with an inner product is called an inner product space. A special notation is used for <u, v>, n ui i= 1 uu, u = = where u is called the norm of vector u and geometrically is the length of the vector. Two vectors u, v are said to be orthogonal if <u, v> = 0. Schwarz inequality: Let u and v be vectors in an inner product space (either on R or C). Then 13

14 uv, u v Orthonormal bases: In an inner product space, a set of vectors φ 1, φ,... is orthonormal if 1, for j = k φj, φ k = δ jk = 0, for j k One dimensional projections: The vector u can be viewed as the sum of two vectors u= u v + u v where u v is collinear with v, and u v is orthogonal to v. The vector u v is called the projection of u onto v. Finite dimensional projections: If S is a subspace of an inner product space V, and u V, the projection of u on S is defined to be a vector u S S such that u u S, v = 0 for every vector v S. u S u S Figure 1.5 Projection theorem: Let S be an n-dimensional subspace of an inner product space V and assume that φ1, φ,..., φ n is an orthonormal basis for S. Then any u V may be decomposed as u= u S + u S, where u S S and u S, v = 0 for all v S. Futhermore, u S is uniquely determined by u n = u, φ φ S j j j= 1 A consequence of projection theorem is that the projection u S is the unique closest vector in S to u; that is, for all v S, u u S u v with equality iff v= u S. See figure 1.5. Gram-Schmidt orthogonalization procedure: It produces an orthonormal basis {φ j } for an arbitrary n-dimensional subspace S with the original basis s 1,, s n. See, e.g., [Proakis 000, ch4] for details. E. Circularly symmetric Gaussian processes A vector X with M jointly Gaussian real-valued random variables has the p.d.f. 1 1 T 1 px( x) = exp ( ) K ( ) M / x mx X x m X ( π ) det( K ) X T where K = E[( x m )( x m ) ] is the covariance matrix, and m = E[ x ] is X X X X 14

15 the vector mean. A complex-valued Gaussian random process consists of two jointly Gaussian real-valued processes, a real part and an imaginary part. By jointly Gaussian, we mean that any arbitrary set of samples of real and imaginary parts is a jointly Gaussian set of random variables. Let Z(t) be a zero mean complex-valued Gaussian process. Let R(t) = R[Z(t)] and I(t) = I[Z(t)]. By definition, both R(t) and I(t) are zero mean real Gaussian processes. Thus, R(t) and I(t) are fully characterized by their nd order statistics, R ( ) [ ( ) ( )] R τ = E Rt+ τ Rt, RI ( τ ) = E [ It ( + τ ) It ( )], RRI ( τ ) = E [ Rt ( + τ ) It ( )] The complex-valued process Z(t) is strictly stationary if R(t) and I(t) are jointly wise-sense stationary, and hence jointly strictly stationary. By definition, the complex-valued process Z(t) is wise-sense stationary if the autocorrelation function * R ( ) [ ( ) ( )] Z τ = E Zt+ τ Z t is independent of t. Notice that this is not the same as saying that the real and imaginary parts are jointly wide-sense stationary, since RZ ( τ ) could not by itself contain information equivalent to RR( τ ), RI ( τ ) and RRI ( τ ). Thus, we require more than RZ ( τ ) to fully specify the statistics of Z(t). In addition to RZ ( τ ), it suffices to know the complementary autocorrelation function defined as R ( ) [ ( ) ( )] Z τ = E Zt+ τ Zt Using the relations R(t) = Z(t) + Z * (t) and ji(t) = Z(t) - Z * (t), it is easy to show that RR( τ ) = Re{ RZ( τ)} + Re{ R Z( τ)} RI( τ ) = Re{ RZ( τ)} Re{ R Z( τ)} RRI ( τ ) = Im{ R Z ( τ)} Im{ RZ ( τ)} With these equations, we can see that if Z(t) is wise-sense stationary, and in addition R ( ) Z τ is not a function of t, then R(t) and I(t) are jointly wise-sense stationary, and Z(t) strictly stationary. Circularly symmetric Gaussian random variables: Let Z = R + ji be a zero mean Gaussian variable. Z will be called circularly symmetric if E[ Z ] = E[ R ] E[ I ] + je [ RI] = 0 Note that R and I are i.i.d. iff E [ Z ] = 0. j The source of the terminology: e φ Z has the same distribution as Z. It is 1 z pz ( z) = exp πσ σ A complex-valued zero mean Gaussian process is circularly symmetric if E [ Zt ( + τ ) Zt ( )] = 0 for all t and τ Property: - A circularly symmetric Gaussian process is strictly stationary iff it is wide-sense stationary. - For a wide-sense stationary circularly symmetric Gaussian process, 15

16 1 1 RR( τ ) = RI( τ) = Re{ RZ( τ)}, RRI ( τ ) = Im{ RZ ( τ )} - Circularly symmetric processes with a real-valued RZ ( τ ) have a real and imaginary part that are independent at all time, since R RI (τ) = 0. - Circularly symmetry is preserved by linear (time-invariant or time-varying) systems. A white complex-valued Gaussian process has an autocorrelation function ( ) ( ) RZ τ = N0δτ, RZ ( k) = σ δ k for continuous and discrete time, respectively. For a circularly symmetric white Gaussian process, the real and imaginary parts are identically distributed, and are independent of each other. D. Basics of information theory Entropy and mutual information For a discrete random variable X with sample space Ω X, its entropy is defined as H( X) = E [ log PX( x)] = PX( x)log PX( x) x Ω X The mutual information between two random variables X and Y are given by I( X, Y) = H( X) H( X Y) = H( Y) H( Y X) p( yx) I( X; Y) = p( x) p( y x)log p y x y ( ) p( y x) I( XY ; ) = px ( ) pyx ( )log dy p y x ( ) ( ) I( XY ; ) = pxpyx ( ) ( )log p y x dxdy p y ( ) Channel capacity and the coding theorem (Operational) Channel capacity: Maximum rate R for which reliable communication can be achieved. Information channel capacity: Maximum of mutual information over all possible input statistics P(X) C max I( X; Y) = max [ H( Y) H( Y X)] PX PX Suggested Reading [1] R. G. Gallager, Principles of Digital Communication. Cambridge University Press, 009. ( 影印版, 人民邮电出版社,010) 16