A LOW-COMPLEXITY RLS-DCD ALGORITHM FOR VOLTERRA SYSTEM IDENTIFICATION

Transcription

1 A LOW-COMPLEXITY -DCD ALGORITHM FOR VOLTERRA SYSTEM IDENTIFICATION Raffaello Claser and Vítor H. Nascimento Univ. of São Paulo, Brazil Yuriy V. Zakharov University of York, UK ABSTRACT Adaptive filters for Volterra system identification must deal with two difficulties: large filter length M (resulting in high computational complexity and low convergence rate) and high correlation in the input sequence. The second problem is minimized by using the recursive least-squares algorithm (), however, its large computation complexity (O(M )) might be prohibitive in some applications. We propose here a low-complexity algorithm, based on the dichotomous coordinate descent algorithm (DCD), showing that in some situations the computational complexity is reduced to O(M). The new algorithm is compared to the standard, normalized least-mean squares () and affine projections (AP) algorithms. I. INTRODUCTION Linear models and methods have played a key role in engineering and signal processing because of their inherent simplicity. However, there are numerous practical situations in which nonlinear processing is needed, either because the nonlinearities in the system under study are too important to be disregarded, or because the desired behavior cannot be achieved with a linear system []. In the case of system identification, an important class of nonlinear models are linear-in-the-parameters nonlinear models, in which the input-output relation is nonlinear, but the estimation problem is essentially linear. Popular examples are polynomial filters, and in particular Volterra filters []. The Volterra system model is similar to a Taylor series, but with the ility to capture memory effects. In [], the LMS second-order adaptive Volterra filter has been introduced using truncated Volterra series expansions. Truncated Volterra series models have become popular in nonlinear adaptive filtering applications, such as echo cancellation [], [3], channel equalization [4], system identification, detection and estimation and physiological system modeling []. Several adaptive filters using Volterra models have been proposed, based on the least-mean squares (LMS), recursive least-squares (), affine projections (AP) algorithms, among others [], [3] []. A problem related to Volterra system identification is the large number of parameters to be estimated, which results in The work of Y.V. Zakharov and V.H. Nascimento was partly supported by York-FAPESP grant no. 4/76-6. In addition, V.H. Nascimento also received support from FAPESP grant 4/46- and CNPq grant 3668/4-. Contact: rclaser@gmail.com, vitor@lps.usp.br. large computational complexity. In fact, a full second-order Volterra kernel with memory depth of N samples contains N(N + )/ terms [4], see Section II. This means that adaptive filters for identification of Volterra models should have low computational complexity, preferly linear on the filter length M (the number of parameters to be estimated). This is the case of the LMS and normalized LMS () algorithms, but their convergence rate is slow, due to the correlation in the Volterra kernel [4]. The algorithm can solve this problem, but at the cost of a computational complexity that grows quadratically with M. Fast O(M) versions of the algorithm, such as lattice [6] cannot be used in this case, since they require time-shifted regressors, which is not the case for Volterra models. Fast multichannel QRD- filters are good options, but require a large number of divisions and square-roots [7], [8]. Another option is the AP algorithm (APA) [4], [6], whose computational complexity grows linearly with M, with a convergence rate between those of the LMS and algorithms. In this paper we describe a new alternative, a lowcomplexity version of the algorithm suitle for Volterra system identification, extending the -DCD algorithm, a low-complexity version of, based on the dichotomous coordinate descent (DCD) algorithm. - DCD was originally proposed in [9], and later was generalized to deal with widely-linear models in []. The - DCD algorithm of [9], [] has complexity O(M) when the regressor has a time-shift structure. The algorithm in [] extends this result to widely-linear estimation, in which the regressor is composed of two separate delay lines. We show how a generalized version of [] can be applied to lower the complexity of Volterra system identification, organizing the Volterra kernel in a number of separate delay lines. The reduction in complexity results from exploring some blockmatrix symmetries in the regressor autocorrelation matrix R(i). For a full second-order Volterra kernel, the resulting complexity is O(M 3/ ), but if the number of cross-terms is limited, as suggested in [4], the complexity becomes linear in the filter length M. For simplicity, this paper concentrates on the second-order Volterra kernel, but the idea can easily be extended to higher-order kernels. This paper is organized as follows: in Section II we give a brief description of Volterra series. Section III describes the DCD and -DCD algorithms and how to reduce the computational cost of updating the matrix R(i) in /6/$3. 6 IEEE 6

2 -DCD for second-order Volterra kernels. Section IV exemplifies the performance of our new technique under different conditions, and compares the new algorithm with standard, and affine projection algorithms. Lastly, Section V concludes the paper. II. THE TRUNCATED VOLTERRA SERIES For simplicity of presentation, we restrict our discussion to truncated second-order Volterra system models. However, the concepts discussed in this section are valid for higherorder Volterra systems with finite memory []. As we Fig.. Block-diagram of an Adaptive Truncated Volterra Filter. can see in () and Figure, the adaptive filter tries to approximate the desired signal d(i) using a first- and second-order truncated Volterra series expansion in the N most recent samples, as ˆd(i) = m = w (m ; i)x(i m ) + w (m,m ; i)x(i m )x(i m ), m = m =m where w (m ; i) and w (m,m ; i) are linear and quadratic coefficients, respectively, of the adaptive filter at time i (a bias term could also be included). The adaptive filter iteratively updates its coefficients at each timestep so as to minimize the mean squared error E{e (i)}, where E{ } is the expectation operator and e(i) is the error signal () e(i) =d(i) ˆd(i). () In this paper, the regressor vector s(i) of the Volterra filter is organized in sets of delay lines defined as: s L (i) (Linear delay line of length N), s Q (i) (Quadratic delay line of length N) and s Ck (i) (delay-k Cross-Product delay line of length N k) i.e., s(i) = s L (i) s Q (i) s C (i)... s CK (i) T, (3) where K N denotes the maximum delay included in the cross-product terms. The tap-delay lines are given by s L (i) = x(i) x(i )... x L (i N + ) T (4) s Q (i) = x (i) x (i )... x (i N + ) T () s Ck (i) = x(i)x(i k) x(i )x(i k)... T, (6) where the length of the last delay line is N k. This particular ordering of the regressor vector s(i) is used in the next section to derive a low-complexity -DCD algorithm for Volterra filters. III. THE -DCD ALGORITHM The algorithm computes a weight vector w(i) by iteratively solving the normal equations [6], [] where R(i) = jx i= R(i)w(i) =p(i) (7) i j s(j)s T (j) = R(i ) + s(i)s T (i) (8) is the M M autocorrelation matrix, and is the forgetting factor. The cross-correlation vector p(i) is ix i p(i) = j d(j)s(j) = p(i ) + d(i)s(i) (9) j= For a regressor vector such as in (3), the standard algorithm uses the matrix inversion lemma to find a recursion to R (i), and thus solves (7). This requires O(M ) multiplications and additions at each time instant [6], []. Low-complexity (O(M)) versions of, such as the fast transversal filter (FTF) [6], lattice [6], [3] and -DCD [9], [] require that the regressor vector be composed of a single delay line and thus cannot be applied. This constraint was eased in [], in which a modified version of the -DCD algorithm was developed for the case of widely-linear estimation, in which the regressor is composed of two delay lines. Here we generalize the result of [] to the case of multiple delay lines, and show how this can be used to obtain a low-complexity -DCD algorithm suitle for Volterra system identification. Contrary to standard, the -DCD algorithm uses a low-cost iterative algorithm to find an approximate solution w(i) to the modified problem R(i) w(i) = (i), () where (i) is computed through the recursion ˆd(i) =s T (i)ŵ(i ), (i) = r(i ) + e(i)s(i), ŵ (i) =ŵ (i ) + ŵ(i) r(i) = (i) R(i) w(i), and r(i) is the residue at time i. Since in an adaptive filter the weight update w(i) is expected to be small, a sufficiently good approximation to () can be obtained through an iterative algorithm, using just a few iterations. We follow [9], [4] and choose the DCD algorithm to solve (), obtaining an O(M) algorithm. DCD is an iterative method for solving least-squares problems, designed to avoid multiplications and divisions (which are replaced by bit-shifts) [], []. We use here the leading DCD algorithm, summarized in Tle I. M b represents the precision in the solution (it corresponds to the number of bits in the solution if 7

3 the algorithm is implemented using fixed-point arithmetic). The constant H should be chosen as a power of two, in which case all multiplications and divisions reduce to bit shifts, thus allowing simple implementations in hardware [4]. N u is the maximum number of vector operations in the algorithm, which for adaptive filtering applications can be chosen as N u M. The complexity of the leading DCD algorithm is upper limited by (M + )N u + M b additions. This corresponds to a worst case scenario when the algorithm makes use of all N u updates and the condition at step 3 in Tle I is never satisfied [9]. Tle I. Leading DCD algorithm step Equation Initialization: w(i) =, r =, h = H, m = for i =,...,N u n = arg max p=,...,m { r p } m = m +, h = h/ 3 if m>m b, algorithm stops 4 if r n (h/)r n,n, then go to step w(i) = w(i)+sign(r n)h 6 r(i) =r(i) sign(r n)hr (:M,) The DCD algorithm thus allows us to solve () in O(M) operations. The difficulty remains the update of R(i): if we use (8) directly, the number of operations required is O(M ). When the regressor vector s(i) consists of a single delay line, [9] was le to use the structure in R(i) to reduce the number of operations to O(M); later [] extended this result to the case of s(i) consisting of two delay lines. In the case of Volterra system identification, s(i) in (3) consists of K + delay lines, where K N is a limit to the number of cross-terms considered. With a memory depth of N, we have N elements for each s L (i) and s Q (i), and N k elements for s Ck (i), so that the total number of coefficients M is (N K)K M =N +, K N. () Taking into account the structure of s(i), R(i) can be partitioned as 3 R L (i) R LQ (i)... R LCK (i) R QL (i) R Q (i)... R QCK (i) R(i) = R CL(i) R CQ(i)... R CC K (i) () R CK L(i) R CK Q(i)... R CK (i) where for each block R (i), a, b {L, Q, C,...,C K } we have a recursion (note that we write R aa (ii) as R a (i) to simplify notation) R (i) = R (i ) + s a (i)s T b (i). (3) For each block on the main diagonal, instead of (3), we can use the low-cost update proposed in [9], ra (i) T a (i) R a (i) = (:,: ) a (i) R a (i ), (4) (:,: ) where the notation R a (i ) stands for a matrix with the elements of R a (i ) from rows through and from columns through, and is an integer equal to the dimension of s a (i). r a (i) is a real number and a (i) is an ( ) vector. From (4), we note that we only need to update the first column of R a (i), since we already (:,: ) have R a (i ) from the previous iteration and R a (i) is symmetric. This is done as follows R (:,) ra (i) a (i) = = R (:,) a (i) a (i ) + s () a (i)s a (i), () where s () a (i) represents the first element of vector s a (i). Similarly, for the off-diagonal blocks we can use the recursion [] r (i) T ba R (i) = (i) (:,: ) (i) R (i ), (6) where a 6= b, and and are the lengths of s a (i) and s b (i), respectively. In general, R is not symmetric, so 6= ba : R (:,) r (i) (i) = = R (:,) (i) (i ) + s () a (i)s b (i), (7) ba (i) = ba (i ) + s () b (i)s (:,) a (i). (8) These recursions reduce the total number of operations in the update of the R(i) matrix, so that the total computational cost for the -DCD algorithm applied to Volterra system identification reduces to the values given in Tle II (we present the number of multiplications for two cases, when < < is any value, or when = m for an integer m>). Tle III summarizes the algorithm. The costs for, and APA (order ) are from [6]. Tle II. Computational cost (,, APA from [6]) Algorithm + M +3M M +M + 3M 3M + APA of order ( + )M ( + )M DCD (M from ()) -DCD ( = m ) (N +)K K 3 / K/+4M+ (M +)N u + M b (N +)K K 3 K +M+ (M +)N u + M b (N +)K K 3 K +4M (N +)K K 3 / /K +3M Note from Tle II and () that if we increase M by increasing the value of N, but keeping fixed the value of K, then the computational cost of DCD- grows linearly with M. This situation is important, since in practice the value of K is kept small to avoid having a filter length that is too large, as explained in [4]. On the other hand, for a full second-order Volterra kernel, such that K = N, () implies that M N / for large N, and from Tle II we conclude that the computational complexity of -DCD is O(M 3/ ), still much lower than the O(M ) of standard

4 Tle III. -DCD algorithm for Volterra system identification step Equation Initialization: ŵ () = M, r() = M, Each block-matrix of main diagonal of R() = I, Each block-matrix of upper diagonal of R() = for i =,,... for a = L, Q, C,... R (:,) a (i) = R (:,) a (i ) + s a(i)s a(i) end for = LQ, LC,LC,... (upper diagonal of R(i)) R (:,) (i) = R (:,) (i ) + s a(i)s b (i) 3 R (:,) (i) = R (:,) (i ) + s b (i)s (:,) a (i) end 4 R L, R Q, R C,...) R(i) y(i) =s T (i)ŵ(i ) 6 e(i) =d(i) y(i) 7 (i) = r(i ) + e(i)s(i) 8 R(i) w(i) = (i) ) ŵ(i), r(i) 9 ŵ (i) =ŵ (i ) + ŵ(i) DCD (Nu=4) -DCD (Nu=8) - Fig.. MSD and tracking performance comparison between and -DCD when estimating a Volterra model, using M b = 6, N =, K =4and M =. IV. RESULTS In this section, we compare the new -DCD algorithm with, and APA for the identification of a Volterra model. The true plant in our simulation has a linear part plus a nd-order term, N = and K =4. The input signal x(i) is white Gaussian noise with unit variance, independent of the measurement noise v(i). The desired signal is given by d(i) =w T opts(i)+v(i) (9) where w opt is the optimum coefficient vector (chosen randomly from a standard Gaussian distribution), and v(i) is white Gaussian noise with variance. The forgetting factor is = 6 and R() = (/ ) I, where I is the identity matrix of appropriate size. For the -DCD algorithm, we use the same value of, H = 64 and M b = 6. The simulation in Fig. compares the mean square deviation (MSD = E{kw w opt k }) of the and -DCD with the same values of N = and K =4, so from (), M =. We choose N u to be 4 or 8, and include a change in the system coefficients at the middle of the simulation to compare the tracking performance of the algorithms. We use simulations to obtain the ensemble-average learning curves. We see from Fig. that the performance of the -DCD algorithm is already close to that of the standard, even when using N u as small as 4. Fig. 3(a) and 3(b) compare the number of additions necessary for,, APA with = 4and = 3 and -DCD. For this comparison we use N u = 4, M b = 6 as in Fig., but vary the memory depth N for fixed K =4in Fig. 3(a), while in Fig. 3(b) we fix N = and vary the maximum cross-term delay K. As we can see, the number of additions is in most cases smaller than that obtained with APA with =4, and much smaller than the number of additions needed by and APA with = 3. The number of multiplications behaves similarly. Another simulation was performed by comparing the performance between, -DCD, and APA. For this simulation we used the same parameters as in Fig. for and -DCD, and a step-size µ =.8for, µ =.3 and µ =.3 for APA with projection orders =4and = 3, respectively. As we can see in Fig. 4(a) (APA, =4) and Fig. 4(b) (APA, = 3), both and APA have slower convergence when compared to the and -DCD algorithms. V. CONCLUSION In this paper, we propose a DCD-based Volterra algorithm with reduced computational complexity, exemplifying the method using second-order Volterra kernels. When the maximum delay K in the cross-terms of the Volterra kernel is kept small, the computational complexity of the new algorithm grows linearly with the filter length and is comparle with the complexity of a low order APA. We compared the new algorithm with standard, and APA, showing that the same steady-state MSD from can be achieved using the -DCD algorithm, but with fastest convergence when compared to and APA (even for an APA order as high as = 3). VI. REFERENCES [] V. J. Mathews, Adaptive polynomial filters, IEEE Signal Process. Mag., vol. 8, pp. 6, 99. [] A. Stenger and R. Renstein, Adaptive Volterra filters for nonlinear acoustic echo cancellation. in NSIP, 999, pp [3] A. Fermo, A. Carini, and G. L. Sicuranza, Lowcomplexity nonlinear adaptive filters for acoustic echo cancellation in GSM handset receivers, European transactions on telecommunications, vol. 4, no., pp. 6 69, 3. 9

5 N. of additions APA, γ=4 APA, γ=3 -DCD N (a) (a) APA (γ=4) -DCD (Nu=4) N. of additions 6 APA, γ=4 APA, γ=3 -DCD K (b) Fig. 3. Comparison of the number of additions between,, APA ( =4), APA ( = 3) and -DCD for N u =4. (a) K =4; (b) N = APA (γ=3) -DCD (Nu=4) (b) Fig. 4. MSD and tracking performance comparison between,, -DCD and APA using (a) = 4, (b) = 3, M b = 6, N u =4, K =4, N = and M =. [4] G. L. Sicuranza and A. Carini, A multichannel hierarchical approach to adaptive Volterra filters employing filtered-x affine projection algorithms, IEEE Trans. Signal Process., vol. 3, no. 4, pp ,. [] L. Yao and C.-C. Lin, Identification of nonlinear systems by the genetic programming-based Volterra filter, Signal Processing, IET, vol. 3, no., pp. 93, 9. [6] A. H. Sayed, Adaptive filters. John Wiley & Sons,. [7] A. Ramos, J. Apolinário, and S. Werner, Multichannel fast qrd-rls adaptive filtering: Block-channel and sequential-channel algorithms based on updating backward prediction errors, Signal Processing, vol. 87, no. 7, pp , Jan. 7. [8] M. Shoaib, S. Werner, and J. A. Apolinário, Multichannel fast QR-decomposition algorithms: Weight extraction method and its applications, IEEE Trans. Signal Process., vol. 8, no., pp. 7 88,. [9] Y. Zakharov, G. White, and J. Liu, Low-complexity algorithms using dichotomous coordinate descent iterations, IEEE Trans. Signal Process., vol. 6, no. 7, pp. 3 36, 8. [] F. G. Almeida Neto, V. H. Nascimento, and Y. V. Zakharov, Low-complexity widely linear filter using DCD iterations, in XXX SBrT (Brazilian Symposium on Telecommunications),. [] V. J. Mathews and G. L. Sicuranza, Polynomial Signal Processing. New York: Wiley-Interscience,. [] V. H. Nascimento and M. T. M. Silva, Adaptive filters, in Academic Press Library in Signal Processing:, R. Chellappa and S. Theodoridis, Eds. Chennai: Academic Press, 4, vol., Signal Processing Theory and Machine Learning, pp [3] M. D. Miranda, M. Gerken, and M. T. M. Silva, Efficient implementation of error-feedback LSL algorithm, Electronics Letters, vol. 3, no. 6, pp , 999. [4] J. Liu, Y. Zakharov, and B. Weaver, Architecture and FPGA design of dichotomous coordinate descent algorithms, IEEE Trans. Circuits Syst. I, vol. 6, no., pp , 9. [] Y. Zakharov and T. Tozer, Multiplication-free iterative algorithm for LS problem, Electronics Letters, vol. 4, no. 9, pp , 4.