Approximated fast estimator for the shape parameter of generalized Gaussian distribution for a small sample size

Transcription

1 BULLETIN OF THE POLISH ACADEMY OF SCIENCES TECHNICAL SCIENCES, Vol. 63, No. 2, 2015 DOI: /basts Aroximated fast estimator for the shae arameter of generalized Gaussian distribution for a small samle size R. KRUPIŃSKI West-Pomeranian University of Technology in Szczecin, Chair of Signal Processing and Multimedia Engineering, Kwietnia St., Szczecin, Poland Abstract. Most estimators of the shae arameter of generalized Gaussian distribution (GGD) assume asymtotic case when there is available infinite number of observations, but in the real case, there is only available a set of limited size. The most oular estimator for the shae arameter, i.e., the maximum likelihood (ML) method, has a larger variance with a decreasing samle size. A very high value of variance for a very small samle size makes this estimation method very inaccurate. A new fast aroximated method based on the standardized moment to overcome this limitation is introduced in the article. The relative mean square error (RMSE) was lotted for the range of the shae arameter for comarison with other methods. The method does not require any root finding, any long look-u table or multi ste aroach, therefore it is suitable for real-time data rocessing. Key words: estimation, generalized Gaussian distribution, standardized moment, aroximated fast estimator. 1. Introduction Most of the estimation methods for the shae arameter of GGD assume that the samle size is large. The estimation of GGD arameters may be carried out by the use of ML method [1], the moment method (MM) [2], entroy matching [3]. For all these methods the existence and the uniqueness of the arameters are based on asymtotic behavior, i.e., the samle size is sufficiently large. In [4], it was shown that the comutation of GGD arameters on small samles is not the same as on larger ones. The authors resented a necessary and sufficient condition for the existence of the arameters in a maximum likelihood framework. GGD was observed to aear in many signal and image rocessing alications. A large samle size very often is not available. Therefore, the relaying method for the estimation of GGD shae arameter is necessary. The ML method used for the estimation of the shae arameter is comlex and time consuming. The comlexity can be reduced by the alication of the One Moment (OM) method [5]. Instead of the ML method the Mallat s method is often used for estimation, even though the method is not accurate for the whole range of the shae arameter [6]. In [7], the scale-invariant fourth moment is used as an accurate initial value to the Newton Rahson iteration in the estimation arameters of comlex GGD. The method [6] based on the aroximation of the moment method in four intervals allows fast estimation of GGD shae arameter for real-time alications and requires storing only twelve coefficients. The authors resented the method which aroximates the estimation of GGD shae arameter in the range A review of the different aroaches to shae arameter estimation roblems can be found in [8]. The authors stated that the estimators (the Mallat s generalized Gaussian ratio method (MRM), the kurtosis generalized Gaussian ratio method (KRM)) were still not satisfactory in the case of short data. The negentroy matching (NM) method can still accurately estimate the arameters for small samle size, but for the shae arameter < 1. The eaky distributed signals can be observed in many signal rocessing alications [9, 10]. In Sec. 2 the estimation methods of GGD shae arameter and an aroximation model for a small samle size are discussed. In Sec. 3 the numerical results are resented. The methods based on the root finding may not have a real root for a small samle size created from simulated observations. In Sec. 3 it can be observed that this situation aeared for the ML method for N < 60 = 0.4, N < 85 = 1 and N < 120 = Material and methods The robability density function (PDF) of the continuous random variable of GGD [11] takes the form where Γ(z) = λ(, σ) f(x) = ( )e [λ(,σ) x µ ], (1) 1 2 Γ 0 t z 1 e t dt, z > 0, is the Gamma function [12], denotes the shae arameter, µ is the location rkruinski@w.l 405

2 arameter and λ relates to the variance of the distribution and can be calculated from ( ) 1/2 3 λ(, σ) = 1 Γ σ ( ) 1, (2) Γ where σ denotes the standard deviation. The secial case of GGD can be observed when the shae arameter equals = 1 and = 2, which corresonds to Lalacian and Gaussian distributions resectively. For 0, f(x) aroaches an imulse function, and for, f(x) aroaches a uniform distribution. For µ = 0, the robability density function is centered around zero. From the definition of absolute moment E [ X m ] = x m f(x)dx, for GGD the following is obtained [13] ( ) m + 1 Γ E m = ( ), (3) 1 λ m Γ where m can be a real number. The E m estimated value of the moment can be acquired from the equation Ê m = 1 N N x i m (4) and where N denotes the number of observed variables, and x 1, x 2,...,x N } is the collection of N i.i.d zero-mean random variables. The resented model [6] for the aroximated estimation of GGD shae arameter is derived from two moments method and is calculated for the selected intervals related to the shae arameter where ( ) 1/c log(g) a =, (5) b G = E m 1 (E m2 ) m 1 m 2 (6) and denotes the estimated value of the shae arameter, E m1 and E m2 are the estimated values of the moments, which can be found from Eq. (4). The arameters a, b and c are set indeendently for each interval and both their values and the rocedure for their selection are discussed in [6]. Du [1] described the estimation of the shae arameter derived from the maximum likelihood method R. Kruiński where ( Ψ ) + log() ( ) log 1 N x i N N x i log( x i ) = 0, N x i Ψ(τ) = γ (7) (1 t τ 1 )(1 t) 1 dt (8) and γ = denotes the Euler constant. The root of Eq. (7) gives the ML estimate. Equation (3) for two different moment values m 1 and m 2 and eliminating λ leads to: ( ) ( ) m m 2 m 1 1 Γ Γ g = ( ) = E m 2, (9) m1 + 1 m 2 m 1 (E m1 ) m 2 m 1 Γ which is the method for the estimation of the shae arameter based on two moments. The inverse function to the g function, Eq. (9), deends on the moment values m 2 and m 1 and can be aroximated as follows: m 1 = 0.25 and m 2 = = 55 g , for g < g , for g 1.079, (10) g < g , for g m 1 = 0.5 and m 2 = 1 26 g , for g < = 5.5 g , for g 1.27, (11) m 1 = 1 and m 2 = 2 29 g , for g < 2 ˆ 2 = 5 g , for g 2, (12) m 1 = 2 and m 2 = 3 77 g , for g < 1.6 ˆ 3 = 16.3 g , for g 1.6, (13) m 1 = 2 and m 2 = 4 12 g , for g < 6 ˆ 4 = 6 g , for g 6, (14) 04 = 0.5 ( ). (15) 406 Bull. Pol. Ac.: Tech. 63(2) 2015

3 Aroximated fast estimator for the shae arameter of generalized Gaussian distribution for a small samle size Equation (14) for two moments m 1 = 2 and m 2 = 4 corresonds to kurtosis. The inverse function of Eq. (9) for two moments m 1 = 2 and m 2 = 4 is deicted in Fig. 1. RMSE = 1 M M ( ) 2 2, (17) where is a value estimated by the model and is a real value of a shae arameter. M denotes the number of reetitions and was set to M = 10 4 for all exeriments. A small samle size may lead to difficulties with the root finding of the ML method (Eq. (7)), therefore, the sto condition was set to tolx = 1e 4 and toly = 1e 5, where tolx and toly are the absolute errors. The RMSE increase with decreasing samle size for the ML method is deicted in Fig. 2. The curves are lotted for two selected fixed values = 1 and = 2 in the GGD generator. Before the estimation with ML, the samle size was centered twofold: ML med a median value is subtracted from a samle; ML mean a mean value is subtracted from a samle. Fig. 1. Inversion function of Eq. (9) for two moments m 1 = 2 and m 2 = 4 and two comonents from Eq. (14) Additionally, in the same figure two comonents aroximating this function from Eq. (14) are overlaid with a threshold value g = 6. It can be noticed that the inverse function is better aroximated by one comonent over the threshold value g = 6 and by another comonent below this value. It should be ointed out that the model used for the aroximation of inverse function to the g function (Eq. (9)) in this article is different than the model introduced in [6] (Eq. (5)). Equations (10) (15) were obtained by fitting the inverse function of Eq. (9) to the model a g b + c. Authors also examined other sets for the moment orders: m 1 = 0.1 and m 2 = 0.5; m 1 = and m 2 = Nevertheless, Eqs. (10) (15) became the most usable in the final estimation, i.e, led to the smallest error. The final estimation S can be found from the following relation: 4, for , for < 1.86 S =, (16) 1, for < , for 1 < 0.57 where 0, 1, 04 and 4 are defined by Eqs. (10), (11), (15) and (14) resectively. 1 has been chosen as a reference value in (16), because it has the smallest RMSE and is constant over the widest interval of the considered range from the considered aroximations. 3. Calculation The equations from the article were validated with the GGD generator [14] with fixed variance to unity and varying shae arameter in the range 0.3, 3. The values were selected to cover the range of the most tyical values in the signal rocessing alications. RMSE was alied for the comarison of the estimators outut. RMSE was calculated from the equation Fig. 2. Comarison of RMSE for the ML method of the estimation of the shae arameter of GGD with a varying samle size N and for the selected fixed values = 1 and = 2 in the GGD generator. ML med a median value is subtracted from a samle before estimation. ML mean a mean value is subtracted from a samle before estimation It can be noticed that with a small samle size the ML method resulted in the very high value of error in terms of RMSE (Fig. 2). In the first ste, the observation set is centered. It can be done simly by subtracting either the mean value of the set or the median value of the set. It turns out to have influence on the final estimation error. Figure 2 shows that the smaller RMSE is assured by subtracting the median value. The similar behavior for the aroximated method (Arox, Eq. (5)) is observed, i.e., the subtraction of a mean value resulted in the higher value of RMSE. In the following, when the results for the Arox and ML methods will be demonstrated, they will be based on the centering using a median. First, the location arameter µ has to be determined after collecting a samle of GGD random variable. The estimation of the µ arameter can be conducted twofold by: the mean or median. Then it is made centering by subtracting the estimation of the µ arameter. Figure 3 deicts three curves with a median centering for the location arameters in the GGD Bull. Pol. Ac.: Tech. 63(2)

4 generator: µ = 1, µ = 0 and µ = 1 for 0 (Eq. (10)). It can be noticed that these curves overla. R. Kruiński Fig. 3. Comarison of RMSE for the 0 (Eq. (10)) comonent of the estimation of the shae arameter of GGD with a varying samle size N and for the selected fixed value = 2 for three location arameters µ = 1, µ = 0 and µ = 1 in the GGD generator and with a median centering Fig. 5. Comarison of RMSE for the 0 (Eq. (10)) and 1 (Eq. (11)) comonents of the estimation of the shae arameter of GGD with a varying samle size N and for the selected fixed value = 2 in the GGD generator. med and mean a median and mean values are subtracted from a samle before estimation resectively The similar curves with a mean centering for 1 (Eq. (11)) for the location arameters in the GGD generator: µ = 1, µ = 0 and µ = 1 were lotted in Fig. 4. In this case the curves also overla. Fig. 6. Comarison of RMSE for both all comonents (Eq. (10) (15)) and the final equation ( S, Eq. (16)) of the estimation of the shae arameter of GGD with a varying shae value and a selected fixed samle size N = 31 Fig. 4. Comarison of RMSE for the 1 (Eq. (11)) comonent of the estimation of the shae arameter of GGD with a varying samle size N and for the selected fixed value = 2 for three location arameters µ = 1, µ = 0 and µ = 1 in the GGD generator and with a mean centering Figure 5 shows the influence of the median and mean subtraction for two selected comonents 0 (Eq. (10)) and 1 (Eq. (11)) used in the final equation (Eq. (16)). The median subtraction resulted in the smaller value of RMSE for both 0 (Eq. (10)) and 1 (Eq. (11)) comaring to the mean subtraction. RMSE for the final equation ( S, Eq. (16)) is deicted in Fig. 6. The curve can be comared to all comonents of the equation (Eq. (10) (15)) for the small samle size N = 31. A median value is subtracted from a samle before estimation. The method is designed to kee a constant RMSE (at least not increasing) over the considered range of. A figure for the comarison of RMSE for the S (Eq. (16)), ML, Arox methods of the estimation of the shae arameter of GGD with a varying shae value and a selected fixed samle size N = 31 is not resented in the article due to an excessive error of ML. The smallest N that would be reresentative to comare the S (Eq. (16)), ML, Arox methods was selected N = 71. In Fig. 7, it is observed that the Arox method lot is not comleted. Uon closer examination the curve discontinuities can be noticed. Such a case denotes when the Arox estimator [6] outut has at least one comlex value. Thus, RMSE was inalicable and incomarable. For the ML case, it can be noticed that with increasing values of in the GGD generator, RMSE increase raidly with ringing. A median value is subtracted from a samle before estimation. The global convergence method (GCM) [8] requires to find the root of Z n () = 0, which for a small samle size may not have a real root. 408 Bull. Pol. Ac.: Tech. 63(2) 2015

5 Aroximated fast estimator for the shae arameter of generalized Gaussian distribution for a small samle size From Figs. 9 and 10 it can be read that S is stable with the small values of N whereas RMSE raidly grows for ML or get comlex for Arox. For these both simulations, the reetition count in Eq. (17) was increased to M = A median value is subtracted from a samle before estimation. Fig. 7. Comarison of RMSE for S (Eq. (16)), ML, Arox methods of the estimation of the shae arameter of GGD with a varying shae value and a selected fixed samle size N = 71 The similar lot for N = 121 is deicted in Fig. 8, where as exected the RMSE for ML has been reduced. In both cases the introduced estimator (Eq. (16)) behaves stable and the RMSE is the smallest or comarable. Fig. 8. Comarison of RMSE for S (Eq. (16)), ML, Arox methods of the estimation of the shae arameter of GGD with a varying shae value and a selected fixed samle size N = 121 Fig. 10. Comarison of RMSE for S (Eq. (16)), ML, Arox methods of the estimation of the shae arameter of GGD with a varying samle size N and for the selected fixed values = 2 and = 3 in the GGD generator In Fig. 11, RMSE for S when a mean ( S mean ) and median ( S med ) values are subtracted from a samle before estimation is lotted. It can be noted that with the higher values of the GGD generator subtracting a mean resulted in a smaller value of RMSE. The mean value of these two values S med mean = 0.5 ( S mean + S med ) can give an estimation that will roduce the lowest RMSE in short interval around = 2. These curves suggest the combination of them, for instance, S join = [if ( S med < 2) then ( S med ) else ( S med mean )], but the combined curve S join did not result in better RMSE erformance, because of the decision oint S med being biased with an error. Fig. 9. Comarison of RMSE for S (Eq. (16)), ML, Arox methods of the estimation of the shae arameter of GGD with a varying samle size N and for the selected fixed values = 0.4 and = 1 in the GGD generator Fig. 11. Comarison of RMSE for the S (Eq. (16)) method of the estimation of the shae arameter of GGD with a varying shae value and a selected fixed samle size N = 31. S med a median value is subtracted from a samle before estimation. S mean a mean value is subtracted from a samle before estimation. S med mean a mean value from estimation S med and S mean. S join a combination of S med and S med mean Bull. Pol. Ac.: Tech. 63(2)

6 As a limit case it has been considered N = 1000 and N = 2000 and the results are deicted in Figs. 12 and 13. In this simulation and the following ones, a median value is subtracted from a samle before estimation. RMSE of the ML and Arox methods is stable in the range 0.3, 3 whereas S fluctuates. R. Kruiński Fig. 14. Comarison of RMSE for the S (Eq. (16)), L (Eq. (18)), ML, Arox methods of the estimation of the shae arameter of GGD with a varying samle size N and for the selected fixed values = 0.4 in the GGD generator Fig. 12. Comarison of RMSE for the S (Eq. (16)), L (Eq. (18)), ML, Arox methods of the estimation of the shae arameter of GGD with a varying shae value and a selected fixed samle size N = 1000 Fig. 15. Comarison of RMSE for the S (Eq. (16)), L (Eq. (18)), ML, Arox methods of the estimation of the shae arameter of GGD with a varying samle size N and for the selected fixed values = 1 in the GGD generator Fig. 13. Comarison of RMSE for the S (Eq. (16)), L (Eq. (18)), ML, Arox methods of the estimation of the shae arameter of GGD with a varying shae value and a selected fixed samle size N = 2000 The final estimation L with a correction for the larger values of N and can be found from the following relation: 3, for , for < 1.86 L =, (18) 1, for < , for 1 < 0.57 where 0, 1, 2 and 3 are defined by Eqs. (10) (12) and (13) resectively. For the larger values N, the fluctuation of S in the range 2, 3 can be corrected with the alication of L (Fig. 13). Fig. 16. Comarison of RMSE for the S (Eq. (16)), L (Eq. (18)), ML, Arox methods of the estimation of the shae arameter of GGD with a varying samle size N and for the selected fixed values = 2 in the GGD generator 410 Bull. Pol. Ac.: Tech. 63(2) 2015

7 Aroximated fast estimator for the shae arameter of generalized Gaussian distribution for a small samle size For N (450, 2000) and = 0.4, the S method gives the comarable results to L (Fig. 14). Other lots S and L for N (450, 2000) and = 1, = 2, = 3 are deicted in Figs. 15, 16 and 17. It is advised to use L instead of S for the larger values of N and. Fig. 17. Comarison of RMSE for the S (Eq. (16)), L (Eq. (18)), ML, Arox methods of the estimation of the shae arameter of GGD with a varying samle size N and for the selected fixed values = 3 in the GGD generator 4. Conclusions The article focuses on the estimation of the shae arameter of GGD when a small size is only available. Most estimation methods for the shae arameter of GGD assume that there is available a set of an unlimited size. In real situations only a set of observations limited in size is usually available. Therefore, the lower the samle size, the higher variance of the estimated value. This also leads to a situation when the already known estimation methods do not have a real root. The method introduced in the article allows to estimate the shae arameter of GGD in the range 0.3, 3 for a small samle size. Moreover, the method does not require any root finding, any long look-u table or multi ste aroach, thus, it is simle, fast and relatively efficient. The resented method kees relatively small RMSE in the range 0.3, 3 for a small samle size as it was confirmed by simulations whereas other methods had an excessive error in the art of the range, for instance N = 71, the ML method for > 0.75, the aroximated method for > 1.6. The simulations also exhibited slowly increasing RMSE with the decreasing samle size to N = 31 for a new method, where other methods had a jum in RMSE for some threshold samle size. It was observed for the simulations for the shae arameter = 0.4, = 1, = 2 and = 3. REFERENCES [1] Y. Du, Ein shärisch invariantes Verbunddichtemodell für Bildsignale, Archiv für Elektronik und Übertragungstechnik, AEÜ-45 (3), (1991). [2] S.G. Mallat, A theory of multiresolution signal decomosition: the wavelet reresentation, Trans. Pattern Anal. Mach. Intell., IEEE 11 (7), (1989). [3] B. Aiazzi, L. Alarone, and S. Baronti, Estimation based on entroy matching for generalized Gaussian PDF modeling, Signal Process. Lett., IEEE 6 (6), (1999). [4] S. Meignen and H. Meignen, On the modeling of small samle distributions with generalized Gaussian density in a maximum likelihood framework, Image Processing, IEEE Trans. 15 (6), (2006). [5] R. Kruiński and J. Purczyński, Modeling the distribution of dct coefficients for jeg reconstruction, Image Commun. 22 (5), , DOI: /j.image (2007). [6] R. Kruiński and J. Purczyński, Aroximated fast estimator for the shae arameter of generalized Gaussian distribution, Signal Process. 86 (2), , DOI: /j.sigro (2006). [7] M. Novey, T. Adali, and A. Roy, A comlex generalized Gaussian distribution characterization, generation, and estimation, Signal Processing, IEEE Trans. 58 (3), , March (2010). [8] S. Yu, A. Zhang, and H. Li, A review of estimating the shae arameter of generalized Gaussian distribution, J. Comut. Information Systems 8 (21), (2012), [Online] available: htt:// name= df [9] T. Marciniak, R. Weychan, A. Stankiewicz, and A. Dąbrowski, Biometric seech signal rocessing in a system with digital signal rocessor, Bull. Pol. Ac.: Tech. 62 (2), , DOI: /basts (2014). [10] M. Mazur, J. Domaradzki, and D. Wojcieszak, Otical and electrical roerties of (Ti-V)O x thin film as n-tye Transarent Oxide Semiconductor, Bull. Pol. Ac.: Tech. 62 (3), , DOI: /basts (2014). [11] G. Box and G.C. Tiao, Bayesian Inference in Statistical Analysis, Addison Wesley, New York, [12] F.W.J. Olver, Asymtotics and Secial Functions, Academic Press, New York, [13] M.K. Varanasi and B. Aazhang, Parametric generalized Gaussian density estimation, J. Acoust. Soc. Amer. 86 (4), (1989). [14] K. Kokkinakis and A.K. Nandi, Exonent arameter estimation for generalized Gaussian robability density functions with alication to seech modeling, Signal Process. 85 (9), , DOI: /j.sigro (2005). Bull. Pol. Ac.: Tech. 63(2)