ABSOLUTE AVERAGE ERROR BASED ADJUSTED STEP SIZE LMS ALGORITHM FOR ADAPTIVE NOISE CANCELLER Thamer M.Jamel 1, and Haider Abd Al-Latif Mohamed 2 1: Universirty of Technology/ Department of Electrical and Electronic Enginering Baghdad Iraq. 2. Universirty of Technology/ Department of Electrical and Electronic Enginering Baghdad Iraq. ABSTRACT In this paper, an Absolute Average Error-based Adjusted Step Size LMS (AAE-ASSLMS) algorithm is proposed, which overcome and avoid one of the drawbacks of the standard LMS algorithm. This drawback is convergence speed misadjustment trade off problem. In this proposed algorithm an appropriate time varying value of the step size is calculated based on gradually decreasing maximum step size to the minimum value. This time varying step size is based on the absolute average value of the current and the previous estimation errors. The transition from larger step size into smaller one is taking place smoothly, as will be seen from the results. The proposed algorithm shows through a computer simulation result fast convergence time and low level of misadjustment compared with tradtional LMS and another Variable Step Size LMS algorithm (VSSLMS) for adaptive noise cancellation system. The proposed algorithm shows through the computer simulation results higher attenuation performance and good estimation weight coefficients. I. INTRODUCTION Adaptive noise canceller was widely used in digital communication systems in order to reduce or eliminate an unwanted noise from a received signal. The simple structure for adaptive noise canceller was the Finite Impulse Response filter (FIR) which can be trained using Least Mean Square adaptive algorithm (LMS) which was first proposed by Widrow and Hoff at Stanford University, Stanford, CA in 1960 [1]. LMS algorithm is one type of the Gradient Search algorithms and is regarded as one of the most popular algorithms in adaptive signal processing due to the simplicity in the number of calculations required for its update. Furthermore, it does not require matrix inversion, nor does it require measurements of the pertinent correlation functions [1]. However, this algorithm suffers from a slow convergence adaptation process since the convergence time of LMS algorithm is inversely proportional to the step size [2]. The step size influences two important parameters, namely the low level of misadjustment or Minimum Mean Square Error (MMSE) (steady state behavior) and the convergence speed (transient behavior). The step size is directly proportional to the convergence speed and inversely proportional to the MMSE which makes the compromise process difficult [3]. To overcome this problem, one can start with large step size, to enhance the convergence speed, and gradually reduce it to attain its minimum value, to achieve desirable MMSE (i.e. low level of misadjustment) [3]. Therefore, a lot of modifications of the time varying step size LMS algorithm has been reported [3, 4, 5, 6,7,8,9,10,11,12,13,14,15, and 16]. In this paper time varying step size is chosen due to its powerful effect on the performance of the system also the structure of the adaptive noise canceller will not be changed, and this technique requires fewer overheads in computations, which are an important factor for hardware implementation. The proposed algorithm in this paper is called Absolute Average Error based adjusted step size LMS algorithm (AAE-ASSLMS) algorithm. The value of the time varying step size in this proposed algorithm is adjusted according to the absolute average value of the current and the previous estimation errors. The new proposed algorithm shows good performance in terms of fast convergence and low level of misadjustment compared with LMS algorithm. II. BASIC CONCEPT OF ADAPTIVE NOISE CANCELLER (ANC) As the name implies, ANC is a technique used to remove an unwanted noise from a received signal, the operation is controlled in an adaptive way in order to obtain an improved signal-to-noise ratio (SNR) [17]. As shown in figure (1), the ANC has two inputs called primary and reference inputs. The reference input is filtered and subtracted from a primary input which is containing both signal and noise. As a result the primary noise is attenuated or eliminated by cancellation. This filtering and subtraction must be controlled by an appropriate process in order to obtain a noise reduction with little risk of distorting the signal or increasing the noise level [2]. As shown in figure (1), a signal is transmitted over a noise path channel (H (z)) to a sensor that receives the signal (s) plus an uncorrelated noise. The combined signal and noise (s+ ) from the primary input to the system. A second sensor receives a noise which is uncorrelated with the signal but correlated with the noise. This sensor provides the reference input to the system. The noise is filtered to produce an output y that is a close replica of. This output is subtracted from the primary input (s+ ) to produce the system output (s+ -y) [2].
step size parameter controls the influence of the updating factor. Selection of a suitable value for μ is imperative to the performance of the LMS algorithm, if the value is too small the time the adaptive filter takes to converge on the optimal solution will be too long; if μ is too large the adaptive filter becomes unstable and its output diverges. Figure (1) : ANC Model This system output signal is called an error signal (e):- e=s+ -y (1) This error signal (e) is used to update (adjust) the adaptive filter's impulse response (weight coefficients) using a suitable adaptive algorithm such that the error signal is progressively minimized. Squaring equation (1), one obtains:- 2 (2) Taking the expectation E [.] of both sides of equation (2) and realizing that s is uncorrelated with and with y, yields [2]. 2 = + (3) The signal power will be unaffected as the filter is adjusted to minimum output power (MMSE) is:- (4) When the filter is adjusted so that is minimized, is therefore also minimized. The output filter is then a best least-squares estimate of the primary noise [2]. III. LMS ALGORITHM The Least Mean Square (LMS) algorithm was first developed by Widrow and Hoff in 1960 through their studies of pattern recognition [17]. From there it has become one of the most widely used algorithms in adaptive filtering. The LMS algorithm is a type of adaptive algorithm known as stochastic gradient-based algorithms as it utilizes the gradient vector of the filter tap weights to converge on the optimal wiener solution. It is well known and widely used due to its computational simplicity. The filter weights of the adaptive filter (figure (2)) are updated in each iteration according to the following formula [1]. 1 2µ (5) Here (n) is the input vector of reference noise, such that (n) = [ (n) (n-1) (n-2).. (n-l+1), where n is time index. The vector = [ (n) (n) (n)... (n) represents the coefficients (weights) of the adaptive FIR filter tap vector.the parameter μ is known as the step size parameter and is a small positive constant. This Figure (2) : Structure of FIR filter NEW PROPOSED ALGORITHM (AAE- ASSLMS) As explained previously this paper proposes an algorithm which is called Absolute Average Error Adjust Step Size LMS (AAE-ASSLMS) algorithm. AAE-ASSLMS regards as modified version of the standard LMS algorithm. AAE- ASSLMS algorithm used variable step size that will be adjusted according to absolute average value of the current and the previous estimator errors as follows:- µ (6) 0< <1 Or (7) The way in which µ is changing (equation (6)) depends on previous value of step size and also on the absolute average estimation error. As shown in equation (6) one can start with large step size, to enhance the convergence speed, and gradually reduce it to attain its minimum value, to achieve a low level of misadjustment. To achieve best performance the step size should decrease to the next, smaller step in smoothing transient manner. Therefore, in this new proposed algorithm the subtraction process is used to make the next step size ( )) always smaller than the current step size ( ). Furthermore, in this algorithm the absolute value is used to get a decrease in step size in order to arrive at the minimum step size with the lower number of iterations. In this algorithm, the average error is used to increase the ability of adaptive filter for tracking the weight coefficients of the noise path. One should note that the step size should vary within the stability boundaries as shown in equation (7). The way in
which the step size is varied is very important. The ANC converges smoothly and fast to the minimum mean square error value. Actually, the gradient will be close to zero as the learning curve starts converging to the steady state values. Moreover, the learning curve is decreasing function of time, and it fluctuates highly, due to the stochastic nature of the adaptive filter [3]. To overcome this problem, we used the absolute average instantaneous and prior estimation errors to adjust step size in each iteration for smooth transition. To ensure stability, the variable step size µ is constrained (equation (7)) to the pre-determined maximum and minimum step size values, such that is set to or when it falls below or above these lower and upper bounds, respectively [18]. The constant is normally selected near the point of instability of the conventional LMS to provide the maximum possible convergence speed. The value of is chosen as a compromise between the desired level of steady state missadjustment and the required tracking capabilities of the algorithm. The parameter controls the convergence time as well as the level of misadjustment of the algorithm at a steady state. The constant (β) is very important in the process of tracking the weight coefficients of the noise path transfer function. If the value of the constant (β) is small, the speed of arrival to the minimum step size ( ) is slow. The small value of (β) is an important in the environments where the noise level is low because the step size remains large for more iterations before arrives at ( ), this makes the convergence time fast. On the other hand, the large value of (β) is an important in the environments where the noise level is high in order to make the step size arrive at ( ) with faster time, this makes the misadjustment low at a steady state. Then the update equation for the weight vector for AAE- ASSLMS algorithm will be:- IV. SIMULATION RESULTS The performance of proposed, VSSLMS and traditional LMS algorithms are validated by simulations of an adaptive noise as shown in figure 1. In this simulation we evaluate the performance of the standard LMS, VSSLMS and the AAE-ASSLMS in terms of the attenuation factor, error function, learning curve (MSE), and estimation filter coefficients. The ANC parameters are chosen as following, given that all the values of these parameters for LMS, VSSLMS and AAE-ASSLMS algorithms were chosen to achieve better performance in terms of fast convergence time and low level misadjustment in order to make fairly comparison between these algorithms. 1. The order of FIR adaptive filter (L) for all simulation was eight taps (in order to implement the proposed algorithm on real time hardware which repersents our next research). Moreover, the low order of adaptive FIR filter reduces the misadjustment at steady state [2]. 2. The signal to noise ratio at the primary input for all simulation is alternated between [(-20, -15, -10, -5, 0) db]. 3. The source noise used for all simulation was white Gaussian noise with zero mean and different variance. 4. The optimum step size for the standard LMS was chosen by trial and error to be 0.05. 5. The optimum value for the AAE-ASSLMS of was chosen to be 0.05 and 0.0005 respectively and the constant β=0.00002. 6. The optimum value for the VSSLMS of was chosen to be 0.05 and 0.0005 respectively and the constants (α, γ) to be (0.99, 0.9) respectively. 7. The impulse response of the noise path was randomly chosen as ([0.2,-0.15, 1.0, 0.21, 0.03, 0]). 8. Original speech s(n) with number of samples (number of iterations) is shown in figure (3). 1 2µ (8) In this paper in addition to LMS, a comparison between the performance of the AAE-ASSLMS and another Variable Step Size (VSSLMS) [18] algorithm are introduced. The steps required by the VSSLMS algorithm are shown below: µ (9) 0< α < 1 γ > 0 Figure (3): Original speech Signal 9. Corrupted speech signal for different signal to noise ratios is shown in figure (4). Then Then Otherwise (10)
high noise; therefore, it has the lower attenuation performance. Figure (4): Corrupted speech Signal A. Attenuation Factor The attenuation factor was measured in (db), and it represents the ratio between the original signal s(n) to the difference between original signal s(n) and the error signal (e(n)).more negative value of attenuation means the best performance. The average attenuation factor can be calculated as follows: 2010 (11) Where R represents the window length, long window length takes more running time in computer but makes the attenuation factor performance smoother. In this paper, the window length equal to (4000). Figure (5), Figure (6) and Figure (7) show the performance of attenuation factor for the LMS, VSSLMS and proposed algorithms at different signal to noise ratios respectively. Figures ((5), (6), (7)) show that the proposed algorithm has a higher attenuation performance for different signal to noise ratio and lower misadjustment at the steady state compared with the LMS and VSSLMS algorithms. Average attenuation can be calculated using Matlab-simulation using equation (11) (for example, from figure (5). The average attenuation factor for LMS algorithm is (-7dB) at SNR equal to (-20 db)). As shown in these figures, the proposed algorithm gives a higher (more negative) level of attenuation than the LMS and VSSLMS when the signal to noise ratio at the primary input is increased. The average attenuation factor for proposed algorithm at a signal to noise ratio equal to (0, -5, -10, -15, -20) db is (-32, -30, -24, -23, - 19) db respectively. Furthermore, as shown in these figures, the proposed algorithm has a constant difference (8 db) with respect to the LMS algorithm when the signal to noise ratio at the primary input equals to (0, -5, -10) db. While this difference is equal to (10, and 12) db when the signal to noise ratio equals to (-15, -20) db respectively. The drawbacks of the VSSLMS algorithm are very sensitive to Figure (5) Attenuation performance for LMS algorithm. Figure (6): Attenuation performance for VSSLMS algorithm. Figure (7): Attenuation performance for AAE-ASSLMS algorithm. B. Error Function This function represents the difference between the original signal s(n) and the error signal e(n),this function explains clearly the enhancement of the AAE-ASSLMS compared with other algorithms. (12) Figure (8) shows the error function performance for LMS, VSSLMS and proposed algorithm for signal to noise ratio at the primary input equal to (-20 db). This Figure consists of two parts. The part on the left hand represents the number of samples (number of iterations) from the (0-2500) and the
part on the right hand represents the number of sample from the (2501-30000). These two parts are required to explain the tradeoff between the convergence time and the misadjustment at the steady state. From this figure, one can investigate that the proposed algorithm gives faster convergence time and lower misadjustment at the steady state. Figure (8) Error function performance using different algorithms C. Learning Curve (MSE) In ANC application the adaptive filter tries to minimize the total output power in order to make the output error to be a best least-squares estimate (LSE) of the original signal. Figure (9) shows that the proposed algorithm has the best ability for estimation the original signal when a signal to noise ratio equal to (-20 db). The ability of LMS algorithm is better than the VSSLMS algorithm due that the VSSLMS algorithm is sensitive to high noise, and it requiring optimization of parameters. (γ, α). Figure (9): Learning Curve (MSE) performance D. Filter Coefficients Estimation The ability of algorithms for tracking the noise path coefficients (i.e. transfer function of the noise path ) is investigated. Figure (10) shows the weight coefficients estimation performance for LMS, VSSLMS and AAE- ASSLMS for signal to noise ratio at the primary input equals (-20 db). As shown in figure (10), the Adaptive FIR filter (AFIR) coefficients of ANC using AAE-ASSLMS converges fast to the noise path coefficients and has good estimation coefficients than the standard LMS and VSSLMS.. Figure (10): Weight coefficients Estimation for different algorithms at SNR (-20 db). V.. CONCLUSIONS This paper focused on enhancement performance of the standard LMS using new proposed algorithm (AAE- ASSLMS).This proposed algorithm used an appropriate time varying step size that is calculated based up on the absolute average value of the current and the previous estimator errors. The motivation of the proposed algorithm is starting from the maximum step size to achieve fast convergence time and decreasing to the minimum step size to get a low level of misadjustment. The performance of the proposed algorithm was illustrated by simulations of ANC. Through simulation results, the proposed algorithms show fast and low level of missadjustment compared with LMS and VSSLMS algorithms. Furthermore, the attenuation factor of ANC was enhanced using proposed algorithm compared with other algorithms. As shown by the analysis and simulation results, the proposed algorithm gives high robustness to high variance noise signals compared with other algorithms. REFERENCES [1] B.Farhang Boranjrncy, 1999, Adaptive Filters John Wileys & Sonc, New York.
[2] B. Widrow and S. Stearns, 1985 Adaptive Signal Processing Prentic-Hall, Inc. [3] Khaled F. Abusalem, and Yu Gong, Variable Step LMS Algorithm Using the Accumulated Instantaneous Error Concept Proceedings of the World Congress on Engineering 2008 Vol I WCE 2008, July 2-4, 2008, London, U.K. [4] Long Le, Ozgu Ozun, and Phiipp Steurer, 2002, Adaptive Channel Estimation for Sparse Channels, submitted for the degree of Bachelor of Eng. (Honors) in the Division of Elect. And Electronic Eng. university of Queensland,.. [5] Bozo K.,Zdravko U., and Ljubisa S., April 2003, Adaptive Equalizer with Zero-Noise Constrained LMS algorithm, In Elec. Eng. of FACTA University, vol.16,, 1-xx. [6] Wang GuangHui, Luo XiaoWu, and Zhang-Min, A New Variable Step CP-LMS Algorithm for Beamforming 2004 4th' International Conference on Microwave and Millimeter Wave Technology Proceedings. [7] Wenceslao Novotny, Hilda N.Ferrao. Claudia F., Modified LMS Algorithm for Fast Identification of Digital Channels Proceedings the 15th International Conference on Electronics Communications, and Computers (CONIELECOMP2005). [8] U Irusta, at el, A Variable Step Size LMS Algorithm for the Suppression of the CPR Artefact from a VF Signal Computers in Cardiology 2005;32:179 182. [9] Shuying Xie, Chengjin Zhang, Variable Learning Rate LMS Based Linear Adaptive Inverse Control Journal of Information and Computing Science Vol. 1, No. 3, 2006, pp. 139-148 England, UK. [10] R.W. Wies, A. Balasubramanian, J. W. Pierre, "Using Adaptive Step-Size Least Mean Squares ( ASLMS) for Estimating Low- Frequency Electromechanical Modes in Power Systems ", In 9 th International Conference on Probabilistic Methods Applied to Power Systems KTH, Stockholm, Sweden June 11-15, 2006. [11] Pedro Ramos, Roberto Torrubia, Ana L opez, Ana Salinas, and Enrique Masgrau, Step Size Bound of the Sequential Partial Update LMS Algorithm with Periodic Input Signals Hindawi Publishing Corporation EURASIP Journal on Audio, Speech, and Music Processing Volume 2007, Article ID 10231, 15 pages. [12] Li Yan and Wang Xinan, A Modiefied VS LMS Algorithm Feb. 12-14, 2007 ICACT2007. [13] Yonggang Zhang, Ning Li, Jonathon A. Chambers, and Yanling Hao, 2008, "New Gradient- Based Variable Step Size LMS Algorithms" Research Article in Hindawi Publishing Corporation EURASIP Journal on Advances in Signal Processing Volume, Article ID 529480, 9 pages doi: 10.1155/ 2008/ 529480. [14] Junghsi Lee, Jia-Wei Chen, and Hsu-Chang Huang, Performance Comparison of Variable Step-Size NLMS Algorithms Proceedings of the World Congress on Engineering and Computer Science 2009 Vol I WCECS 2009, October 20-22, 2009, San Francisco, USA. [15] Zou kun and Zhao Xiubing, A New Modified Robust Variable Step Size LMS algorithm ICIEA 2009. [16] Wen He, at el, A Variable Step-Size LMS Algorithm Based on Cloud Model with Application to Multiuser Interference Cancellation RSKT 2010, LNAI 6401, pp. 632 639, 2010. Springer-Verlag Berlin Heidelberg 2010. [17] Haykin, Simon. 1991, Adaptive Filter Theory. 2nd Edition. Prentice-Hall Inc., New Jersey. [18] R.H.Kang, E.W.Johnstone, July 1992, A variable Step Size LMS algorithm IEEE Trans. On Signal Processing, Vol.40, No.7, pp.1633-1642,.