6.02 Fall 2013 Lecture #7

Transcription

1 6. Fall Lecture #7 Viterbi decoding of convoluonal codes 6. Fall Lecture 7, Slide #

2 Convolutional Coding Shift Register View + mod p [n] x[n] x[n-] x[n-] The values in the registers define the state of the encoder + mod p [n] Message bit in, parity bits out Input bits arrive one- at- a- me from the le< The parity bits are computed from the incoming and K- previous message bits At the end of the bit interval, the saved message bits are shi%ed right by one, the oldest bit drops out, and the incoming bit moves into the le< posion 6. Fall Lecture 7, Slide #

3 State-Transition Diagram x[n-]x[n-] Trellis time K- states STARTING STATE / / / / / / Example: K=, rate- ½ convoluonal code g = : p [n] = *x[n] + *x[n- ] + *x[n- ] g = : p [n] = *x[n] + *x[n- ] + *x[n- ] States labeled with x[n- ] x[n- ] Arcs labeled with x[n] / p p or just p p 6. Fall Lecture 7, Slide #

4 Trellis View at Transmitter Message x[n] Codeword / / / / / / / / / / / / / / / / / / x[n-]x[n-] time Generates a LINEAR code! Minimum weight of nonzero word = 5 6. Fall Lecture 7, Slide #4

5 Decoding: Finding Max Likelihood Path Received voltage samples.,..4,..,.99.7,.5.,.5.8,.4,,? Given received voltage samples, find most-likely path through trellis, i.e., minimize distance between the received values and those produced along the trellis path. Hard decoding 6. Fall Lecture 7, Slide #5

6 Simple-minded Decoding Enumerate all paths that start at the zero state, find the one that corresponds to a parity bit sequence that s closest in Hamming distance to the received (hard- decoded) sequence. With L message bits, there are ^L possible paths intractable for the kinds of large L that one would like to use ( s to s). 6. Fall Lecture 7, Slide #6

7 The Viterbi Algorithm The Viterbi algorithm (VA) is a recursive opmal soluon to the problem of esmang the state sequence of a discrete- me finite- state Markov process observed in memoryless noise. Many problems in areas such as digital communicaons can be cast in this form. This paper gives a tutorial exposion of the algorithm and of how it is implemented and analyzed. Applicaons to date are reviewed. Increasing use of the algorithm in a widening variety of areas is foreseen. (Abstract of Forney s paper, Proc. IEEE, 97) 6. Fall Lecture 7, Slide #7

8 Applications today: The algorithm has found universal applicaon in decoding the convoluonal codes used in both CDMA and GSM digital cellular, dial- up modems, satellite, deep- space communicaons, and 8. wireless LANs. It is now also commonly used in speech recognion, speech synthesis, keyword spoing, computaonal linguiscs, and bioinformacs. For example, in speech- to- text (speech recognion), the acousc signal is treated as the observed sequence of events, and a string of text is considered to be the "hidden cause" of the acousc signal. The Viterbi algorithm finds the most likely string of text given the acousc signal. (hkp://en.wikipedia.org/wiki/viterbi_algorithm) 6. Fall Lecture 7, Slide #8

9 Viterbi Algorithm Dynamic programming algorithm, computes most likely message sequence leading up to every intermediate state, & associated cost Branch metric: BM(xmit,rcvd) for each branch of the trellis proporonal to negave log likelihood, i.e. negave log probability that we receive rcvd, given that xmit was sent Hard decision : use digized bits, compute Hamming distance between xmit and rcvd (assumes p < ½ on BSC channel) So< decision : use funcon of received voltages directly (e.g., sum of squares for iid Gaussian noise) Path metric: PM[s,i] for each state s of the K- transmiker states and bit me i, where i < L- PM[s,i] = smallest sum of BM(xmit, rcvd), minimized over all message sequences that place transmiker in state s at me i PM[s,i+] computed from PM[s,i] and the BM for outgoing branches at s,i 6. Fall Lecture 7, Slide #9

10 Hard Decisions As we receive each bit it is immediately digized to or by comparing it against a threshold voltage We lose the informaon about how good the bit is: a at.9999v is treated the same as a at.5v The branch metric used in the Viterbi decoder under hard- decision decoding is the Hamming distance between the digized received voltages and the expected parity bits Throwing away informaon is (almost) never a good idea when making decisions Can we come up with a beker branch metric that uses more informaon about the received voltages? 6. Fall Lecture 7, Slide #

11 Soft-Decision Decoding In pracce, the receiver gets a voltage level, V, for each received parity bit Sender sends V or V volts; V in (-, ) assuming addive Gaussian noise Idea: Pass received voltages to decoder before digizing Define a so< branch metric as the square of the Euclidian distance between received voltages and expected voltages.,..,. Soft metric when expected parity bits V p,v p are, V p + V p.,..,. So<- decision decoder chooses path that minimizes sum of the squares of the Euclidean distances between received and expected voltages Different BM & PM values, but otherwise the same algorithm 6. Fall Lecture 7, Slide #

12 Complexity of VA At any given me there are K- most- likely messages we re tracking me complexity of algorithm grows exponenally with constraint length K, but only linearly with message length L (as opposed to exponenally in L for simple enumeraon) (We saw K=5, for example, on Cassini spaceprobe) 6. Fall Lecture 7, Slide #

13 Hard-decision Branch Metric BM = Hamming distance between expected parity bits and received parity bits Compute BM for each transion arc in trellis Example: received parity = BM(,) = BM(,) = BM(,) = BM(,) = Will be used in compung PM[s,i+] from PM[s,i]. State Time: i i+ / / / 6. Fall Lecture 7, Slide #

14 Computing PM[s,i+] Starng point: All PM[s,i] known, label in trellis box for each state at me i. Example: PM[,i] = means bit error detected when comparing received parity bits to what would have been transmiked when sending the most likely message, considering all messages that place the transmiker in state at me i. Q: What s the most likely state s given measurement up to me i? A: state (smallest PM[s,i]) State Time: i i+ / / / 6. Fall Lecture 7, Slide #4

15 Computing PM[s,i+] cont d. Q: If the trellis is in state s at me i+, what states could it have been in at me i? A: For each state s, there are two predecessor states α and β in the trellis diagram Example: for state, α= and β=. Any message sequence that leaves the trellis in state s at me i+ must have le< the trellis in state α or state β at me i. State Time: i i+ / / / 6. Fall Lecture 7, Slide #5

16 Computing PM[s,i+] cont d. e.g., Which is the more likely path into state at me i+? Time: i i+ PM[,i+] = min{pm[,i] +, PM[,i] + } = min(+, +) = State / / /? 6. Fall Lecture 7, Slide #6

17 Computing PM[s,i+] cont d. Formalizing the computaon: Time: i i+ PM[s,i+] = min(pm[α,i] + BM[α s], PM[β,i] + BM[β s]) Remember which arc was min; saved arcs will generate path through trellis If both arcs have same sum, break e arbitrarily (e.g., when compung PM[,i+]) State / / / 6. Fall Lecture 7, Slide #7

18 Post-decoding BER v. BSC error prob. BER (log scale) - All codes except (7,4) Hamming code are rate-/ (so don t assume it s bad; it actually is better than (8,4) rect parity and one of the conv. codes). Bottom curves: good conv codes Pink curve: bad conv code What makes a code good? p for BSC (linear scale) 6. Fall Lecture 7, Slide #8

19 Spot Quiz Time. What are the path metrics for the empty boxes?. What is the most- likely state a<er me step 6?. If the decoder had stopped a<er me step and returned the most- likely message, what would the bits of the message be (careful about order!)? 6. Fall Lecture 7, Slide #9