Glottal inverse filtering based on quadratic programming

Transcription

1 INTERSPEECH 25 Glottal inverse filtering based on quadratic programming Manu Airaksinen, Tom Bäckström 2, Paavo Alku Department of Signal Processing and Acoustics, Aalto University, Finland 2 International Audio Laboratories Erlangen, Friedrich-Alexander University (FAU), Germany manu.airaksinen@aalto.fi, tom.backstrom@audiolabs-erlangen.de Abstract This study presents a novel quadratic programming based approach to glottal inverse filtering. The proposed method aims to jointly model the effect of the vocal tract and lip radiation with a single filter whose coefficients are optimized using the quadratic programming framework. This allows the proposed method to directly estimate the glottal flow of speech, which mitigates the problem of non-flat closed phases in inverse filtering estimates. The proposed method was objectively evaluated using a synthetic Liljencrants-Fant model based test set of sustained vowels containing a wide variety of phonation types and fundamental periods. The results indicate that the proposed method is robust to changes in f and state-of-the-art quality results were obtained for high pitch voices, when f is in the range 33 to 45Hz. Index Terms: glottal inverse filtering, GIF, quadratic programming, voice source. Introduction The glottal volume velocity waveform, or the glottal flow, is the main source of excitation for voiced human speech production. Obtaining knowledge about the excitation is important in fundamental research of speech, but also in medicine (e.g. occupational voice or speech pathology), phonetics (e.g. prosody), and neuroscience (e.g. brain responses evoked by speech). In addition, estimation of the glottal excitation has recently gained momentum in speech technology, especially in speech synthesis []. Glottal inverse filtering (GIF) is a computational method for estimating the glottal flow from a recorded microphone signal. This approach assumes the so-called source-filter model of speech production, which is most commonly presented as a cascade of three processes: () a time-domain input that represents the glottal flow, (2) a digital filter representing the vocal tract transfer function, and (3) a differentiator that represents the lip radiation effect. GIF is performed by blindly applying antiresonances to the recorded acoustic pressure signal so that the effects of the vocal tract and lip radiation are cancelled, ideally leaving the glottal flow intact. The practice is effective and non-invasive which is key for automated solutions. Given these properties, methods utilizing various forms of linear prediction (LP) (e.g. conventional linear prediction [2] or discrete all-pole modeling [3]) for modeling of the vocal tract have become a popular basis to achieve relatively simple and computationally efficient GIF algorithms. In modeling of the lip radiation effect, GIF algorithms typically utilize a fixed first-order differentiator [4]. In this study, the use of quadratic programming [5] in GIF is introduced to obtain a more thorough mathematical optimization model that is inspired by the principles of the Closed Phase Covariance (CP) GIF method [6]. The CP method is based on computing the vocal tract model from samples that are located in the closed phase of the glottal cycle. This principle was developed further in the recent Quasi Closed Phase (QCP) method [7] by using temporally weighted linear prediction (WLP) as a vocal tract modelling technique. Instead of using a few samples located in the (true) closed pahse, the QCP method takes advantage of all samples of the analysis frame and computes a WLP-based vocal tract model in which the contribution of samples located in the closed phase becomes emphasized in comparison to those that occur in the open phase. Another key point of the current study is the unification of the lip radiation effect and the vocal tract transfer function within the optimization model. This is used to obtain a better estimation of a horizontal, near-zero closed phase for the glottal flow, which might be problematic in most state-of-the-art GIF methods [8, 9]. The principles of GIF within the source-filter model are discussed in Section 2, and its application to quadratic programming is presented in Section 3. Particular attention is paid to the selection of the optimization criterion of quadratic programming in Section 3., as well as to the practical implementation of the proposed method in Section Principles of GIF The source-filter model of speech production is defined in the z-domain as: S(z) = G(z)V (z)l(z), () where S(z) is the speech signal, G(z) is the glottal excitation (depicted in the time domain in Fig. ), V (z) is the vocal tract transfer function, and L(z) is the lip radiation effect that converts the air volume velocity waveform at lips into an acoustic pressure waveform outside the lips. In most GIF methods L(z) is modeled as a fixed first-order differentiator [4]: L(z) = αz, α. (2) When V (z) and L(z) are known, the glottal flow is obtained as: G(z) = S(z) V (z)l(z). (3) Using the fixed lip radiation model of Eq. 2, however, might lead to a low-frequency distrotion that creates an ascending or descending component in the closed phase of the estimated flow pulses (seen e.g. in the female IAIF examples of Fig. 3). This phenomenon was analyzed in [9] and it was suggested to be due to following factors. First, the assumption of an ideal flow-topressure conversion might not be precise, e.g., in cases where the recording microphone is not sufficiently in the far field []. Copyright 25 ISCA 2342 September 6-, 25, Dresden, Germany

2 Second, estimation methods of the vocal tract, such as LP or discrete all-pole modeling, focus mainly on obtaining good modeling performance in formants paying less attention to low frequencies. Small errors in the very-low frequencies of the vocal tract model, however, can greatly affect the shape of the integrated waveform, especially with α-values of Eq. 2 that are very close to. In [9] a method was developed to automatically determine an α-value that aims to compensate the error produced by the above mentioned effects, but in the present study the approach taken is to merge V (z) and L(z) into a unified linear model A(z) = V (z)ˆl(z), (4) where ˆL(z) is an all-pole approximation of L(z) as proposed in [4], and A(z) is a linear FIR model. This leads to the following GIF model: G(z) = S(z)A(z) (5) An important thing to note about the linear model A(z) is that because it includes the FIR approximation of the integrator, its length must be within the range of a single fundamental period of the corresponding speech signal. This assures that L(z) the approximated integrator does not leak information within a single period. The increased length of the filter enables estimating the zeroes of the vocal tract transfer function, which are, for example, produced by the piriform fossa [] and the nasal cavity. 3. Speech production model for quadratic programming The speech production model presented in Eq. 5 can be represented in matrix notation as Ŝâ = g, (6) where Ŝ is a Toeplitz convolution matrix where colums represent the input signal s at consecutive delays, g is the glottal flow, and â is the linear speech production model. To be specific, if â is assumed to correspond to the conventional LP model, the first coefficient of â must be unity, whereby Ŝâ = s + Sa = g, (7) where vector a contains all coefficients of â except the first one, s is the first column of Ŝ, and S has the remaining columns of Ŝ. Now it can be observed that given a, the calculation of g is possible, and vice versa. Since the focus of our interest is the glottal flow g, the effect of a can be canceled by analytic methods. With that objective and by defining the null-space of S as S whereby S S =, Eq. 7 can be rewritten as: S Sa = = S (g s ) (8) The latter equation, = S (g s ), does not contain a, whereby this fulfills the objective of presenting g without a, and this equation can thus be used equivalently with Eq. 7. As a second part of the quadratic speech production model, the glottal flow g will be constrained to be non-negative (g ). This is justified by the fact that speech is produced by exhaling, which produces a positive flow. The flow is zero during glottal closed phase and otherwise positive. However, s is a pressure waveform and our objective is to obtain the flow waveform g. As explained in Section 2, pressure is the derivative of flow, Amplitude Amplitude.5 Glottal flow waveform with w CP glot w CP Glottal flow derivative waveform with w AME d/dt glot w AME Figure : Two cycles of a glottal flow waveform (top) and the corresponding glottal flow derivative waveform (bottom) superposed with the weighting functions W CP and W AME. which means that the zero level of g is ambiguous. Therefore an adaptive zero level δ must be applied, so that g δ. The model is thus so far { S (g S ) = (9) g + δu, where u is a vector of ones. Note that Eq. 9 represents constraints in the sense that they define the feasible space of g. That is, all those glottal flows g which fulfill Eq. 9 could have originated from the defined speech production model. 3.. Optimization criterion Quadratic programming is the problem of optimizing a quadratic function subject to linear constraints on these variables, i.e.: min( x 2 xt Hx + f T x) () { Ax = b s.t. () Ex d It can be seen that the constraints of the speech production model of Eq. 9 are linear and can be applied to quadratic programming, but the optimization criterion that points to the best fitting glottal flow waveform within these constraints is still missing. The optimization criterion is a quadratic function that contains a combined contribution of the norm- and norm-2. The optimization criterion can be formulated in numerous ways. GIF methods utilizing LP-based methods most commonly minimize the residual energy (only the norm-2) of the pre-emphasized speech signal, which yields a reasonable estimate of the vocal tract transfer function because the preemphasis approximately inverts the spectral tilt of the glottal flow [4]. Pre-emphasis is commonly performed as the first time derivative of the input signal (which is a pressure signal), which brings it to the domain of the second derivative of the glottal flow g (which is a volume velocity signal). Furthermore, our recent studies on WLP show that the Attenuated Main Excitation (AME) weighting greatly increases the estimation accuracy both in formant estimation [2] and GIF [7]. The AME weighting function is a temporal waveform that downgrades the contribution of speech samples that are located in the vicinity of the main excitation of the vocal tract near the glottal closure instants (GCIs). This suggests that a good optimization criterion could 2343

3 Error (%) NAQ All 8<f<2 2<f<33 33<f<45 Error (%) QOQ Error (db) HH2 Error (%).5.5 HRF Figure 2: Average error measures of the glottal source parameters for the tested GIF methods. include the AME weighted norm-2 of the second derivative of g: min g,δ (γ (WAME CCg) γ 2δ 2 ) (2) where W AME = Diag(w AME), and w AME is the AME weighting function shown in Fig., C is a convolution matrix that approximates the time derivative, and γ and γ 2 are optimization coefficients. As discussed in Section 2, the modeling of the lip radiation effect has its most prominent effects in the closed phase of the glottal flow estimate. Furthermore, the results obtained in [9] indicate that the norm- is robust for finding a suitable lip radiation model for conventional GIF methods. Thus, the minimization of the norm- of the exact closed phase of g would seem like a valuable addition to the optimization criterion. The physical interpretation of this is that when the vocal folds are closed, the air flow coming from the lungs is ideally zero. In the optimization model this can be expressed as: min g,δ (γ WAMECCg γ 2δ 2 + γ 3 W CP g ) (3) where W CP = Diag(w CP), and w CP is a weighting function with the same size as g ( for the samples of g located in the closed phase, and zero for the samples in the open phase), and γ 3 is an optimization coefficient. It is important to note that the accurate determination of the closed phase is key for accurate estimation of the glottal flow. For example, the SEDREAMS algorithm [3] provides good estimates of the glottal closure and opening instants (GCIs and GOIs, respectively) Practical application of the model The quadratic speech production model is applied in practice using the constraints and the optimization criterion to the standard formula of quadratic programming (Eqs. and ) by selecting: [ g x = δ] [ ] γ3w f = CP H = E = [ I u ] d = [ ] K T K γ 2 K = γ W AMECC A = [ S ] b = S s The output of this model is the vector x that contains the estimated glottal flow g, and the zero-level term δ as its last coefficient so that g final = g + δu; The weighting functions W CP and W AME can be constructed as the AME function if the glottal closure (and opening for W CP) instants are known. In the present study, W CP was constructed as determined by the SEDREAMS algorithm, and W AME was constructed by using the AME parameters [7] PQ =.5, DQ =.85, N ramp = 7. The coefficient values used for γ 3 were γ = 4 (2 for real speech), γ 2 = 5, and γ 3 = 5. The filter order m was selected as the nearest even integer to.85 N f, where N f is the length of the fundamental period in samples. It follows from this high-order filter requirement that the length of the analysis window needed for the proposed method is somewhat longer than for traditional LP applications. In the present study, 5 ms windows of 8kHz samples were used. 4. Experiments The objective evaluation of GIF methods is problematic because it is not possible to measure the real glottal flow signal from natural speech. To overcome this problem, the use of synthetic vowels, e.g. sustained vowels created according to the sourcefilter model, is a common method of obtaining test data for inverse filtering experiments. In the present study, the proposed quadratic programming based GIF method (denoted by QPR) was objectively compared to existing state-of-the-art GIF methods. The evaluation was done using a database of sustained synthetic vowels. The vowels were inverse filtered using the selected GIF methods, and the obtained glottal flow estimates were automatically parameterized into selected glottal flow parameterizations. The average errors on these parameters were used as the objective measures. The used parameters were the Normalized Amplitude Quotient (NAQ) [4], Quasi Open Quotient (QOQ) [5], Harmonic Richness Factor [6] (HRF), and H-H2 [7]. NAQ measures the relative length of the glottal closing phase, QOQ measures the approximate length of the glottal open quotient, and HRF and HH2 are measures that reflect on the spectral decay of the waveform. 4.. Test data The test set was created by using the source-filter model of speech production, where the source was modeled according to the Liljencrants-Fant (LF) model [8], and the vocal tract was modeled as an 8th-order all-pole filter with four formants. Lip radiation was modeled by an ideal differentiator. The LF parameters used to form the excitations were interpolated between breathy and creaky phonations taken from [9] to form a total of 625 phonations. f was varied between 8 Hz to 45 Hz in -Hz increments, and the vocal tract was constructed as in [2], modeling the three vowels [a], [e], and [i]. In total, the test set contained 7 25 test sounds. Because of the long analysis filter of the proposed method (see Section 2), and the relatively low number of parameters in the vocal tract (8th order IIR filter) and in the excitation 2344

4 Breathy Modal Pressed Female Male IAIF QPR DEGG Figure 3: Representative real speech examples of glottal flows estimated with the proposed method (QPR) and the IAIF method for a male and a female speaker with various phonations. Differentiated electroglottography signals shown as reference. (4 parameter LF-model), ideal synthetic vowels (i.e. sustained phonation, no additive noise) were found to be easily overlearned by the method, meaning that the provided glottal flow estimates were easily transformed into waveforms resembling a Dirac delta function. This problem was mostly mitigated by adding a small Gaussian noise component to the synthetic vowels, yielding a SNR of 6dB, but it was still observed that the overall waveforms produced by the proposed method on synthetic data were not of as high quality as they were with real speech (e.g. in Fig. 3) Evaluation methods The proposed QPR method was compared to four existing GIF methods: Quasi Closed Phase Analysis (QCP) [7], Closed Phase Covariance Analysis (CP) [6], Iterative Adaptive Inverse Filtering (IAIF) [2], and Complex Cepstral Decomposition [22]. All GIF methods used the sampling rate of 8kHz, and the order of the vocal tract model was set to p = in QCP, CP, and IAIF. The analysis frame duration was set to 3ms, with a varying length of additional buffer samples on both sides of the frame determined by the respective methods. The QPR method used a ms frame of of extra samples on both sides of the analysis window, whereas the other methods used a frame of 2.5ms. The CP method was implemented utilizing the covariance criterion in LP analysis, by using two pitch-period analysis for frames with F 2Hz. The QCP method utilized the fixed AME parameters of P Q =.5, DQ =.7, and N ramp = 7. IAIF utilized the secondary prediction order of m = 4. The CCD method was obtained from the GLOAT toolbox [23]. GCI (and GOI) information for the QCP, CP, and CCD methods were obtained straight without errors from the reference signals. 5. Results The results for the objective test are shown in Fig. 2. The results are divided into overall and f specific categories. The f specific categories were divided into low (8 2Hz), mid (2 33Hz), and high (33 45Hz) categories. It can be seen that for the tested synthetic vowels, the proposed method is inferior for the low f to most of the compared methods. However, as the f range incrases, the performance of the proposed method remains relatively constant, whereas other methods (excluding CCD) show significant deterioration in their performance. For the high f range the proposed method produces results that improve the state-of-the-art performance for the NAQ, HH2 and HRF parameters. Representative real speech examples are presented in Fig. 3, where sustained vowels of a male and a female speaker with varying phonation types have been inverse filtered with QPR, and IAIF. The differentiated electroglottography (DEGG) signal is also shown as a reference for the glottal closing and opening instants. It is conventionally interpreted that the negative peaks of the DEGG signal correspond to GCIs, and the positive peaks correspond to GOIs [3]. It can be seen that for the proposed QPR method, the closed phases of the estimated glottal flows correspond remarkably well with the DEGG reference. For IAIF the closed phase is commonly distorted as discussed in Section Discussion This study introduced a new method of glottal inverse filtering based on quadratic programming. The novelty of the proposed method is particularly in modeling the vocal tract and lip radiation effect within a single filter whose coefficients are optimized using the quadratic programming framework. The proposed method directly estimates the glottal flow, as opposed to the more conventional approach of first estimating the glottal flow derivative waveform, after which the lip radiation is separately compensated. This ensures that the estimated glottal flow waveforms can be optimized to show more ideal behavior during the glottal closed phase, where the glottal airflow is assumed to be zero. The objective results obtained with a synthetic LF-model based test set suggest that the proposed method is robust in performance regarding the f of speech, and it can produce stateof-the-art quality results for very high f s. It is important to note that the synthetic vowels of the test set have been created with the speech production model that is the basis of the QCP, IAIF, and CP methods. In contrast, the proposed method and the CCD method have significantly different modeling approaches which might bias the results in favor of the more similar methods. The modeling of the lip radiation effect of conventional methods is also ideal for the synthetic vowels, which produces an additional bias against the proposed method in the performed evaluation. 7. Acknowledgements The research leading to these results has received funding from the Academy of Finland (project no , 28467). 2345

5 8. References [] T. Raitio, A. Suni, J. Yamagishi, H. Pulakka, J. Nurminen, M. Vainio, and P. Alku, HMM-based speech synthesis utilizing glottal inverse filtering, Audio, Speech, and Language Processing, IEEE Transactions on, vol. 9, no., pp , 2. [2] J. D. Markel and A. H. Gray Jr., Linear Prediction of Speech. Springer-Verlag, Berlin, 976. [3] A. El-Jaroudi and J. Makhoul, Discrete all-pole modeling, Signal Processing, IEEE Transactions on, vol. 39, no. 2, pp , 99. [4] L. Rabiner and R. Schafer, Digital Processing of Speech Signals, ser. Prentice-Hall signal processing series. Prentice-Hall, 978. [5] J. Nocedal and S. Wright, Numerical Optimization, ser. Springer Series in Operations Research and Financial Engineering. Springer New York, 26. [6] D. Wong, J. Markel, and A. Gray Jr., Least squares glottal inverse filtering from the acoustic speech waveform, Acoustics, Speech and Signal Processing, IEEE Transactions on, vol. 27, no. 4, pp , 979. [7] M. Airaksinen, T. Raitio, B. Story, and P. Alku, Quasi closed phase glottal inverse filtering analysis with weighted linear prediction, Audio, Speech, and Language Processing, IEEE/ACM Transactions on, vol. 22, no. 3, pp , 24. [8] T. Koc, Post-processing method for removing low-frequency bias in glottal inverse filtering, Electronic Letters, vol. 5, no., pp. 2, 25. [9] M. Airaksinen, T. Bäckström, and P. Alku, Automatic estimation of the lip radiation effect in glottal inverse filtering, in Proc. Interspeech, 24. [] J. J. R. Deller, J. H. L. Hansen, and J. G. Proakis, Discrete-Time Processing of Speech Signals, 2nd ed. Wiley-IEEE Press, 999. [] J. Dang and K. Honda, Acoustic characteristics of the piriform fossa in models and humans, The Journal of the Acoustical Society of America, vol., no., pp , 997. [2] P. Alku, J. Pohjalainen, M. Vainio, A.-M. Laukkanen, and B. H. Story, Formant frequency estimation of high-pitched vowels using weighted linear prediction, The Journal of the Acoustical Society of America, vol. 34, no. 2, pp , 23. [3] T. Drugman, M. Thomas, J. Gudnason, P. Naylor, and T. Dutoit, Detection of glottal closure instants from speech signals: A quantitative review, Audio, Speech, and Language Processing, IEEE Transactions on, vol. 2, no. 3, pp , 22. [4] P. Alku, T. Bäckström, and E. Vilkman, Normalized amplitude quotient for parametrization of the glottal flow, The Journal of the Acoustical Society of America, vol. 2, no. 2, pp. 7 7, 22. [5] T. Hacki, Klassizierung von glottisdysfunktionen mit hilfe der elektroglottographie, Folia phoniatrica, vol. 4, no., pp , 989. [6] D. G. Childers and C. K. Lee, Vocal quality factors: Analysis, synthesis, and perception, The Journal of the Acoustical Society of America, vol. 9, no. 5, pp , 99. [7] G. Fant, The LF-model revisited. Transformations and frequency domain analysis, STL-QPSR, vol. 36, no. 2-3, pp. 9 56, 995. [8] G. Fant, J. Liljencrants, and Q. Lin, A four-parameter model of glottal flow, STL-QPSR, vol. 26, no. 4, pp. 3, 985. [9] C. Gobl, The voice source in speech communication - production and perception experiments involving inverse filtering and synthesis, Ph.D. dissertation, KTH, Speech Transmission and Music Acoustics, 23. [2] B. Gold and L. Rabiner, Analysis of digital and analog formant synthesizers, Audio and Electroacoustics, IEEE Transactions on, vol. 6, no., pp. 8 94, 968. [2] P. Alku, Glottal wave analysis with pitch synchronous iterative adaptive inverse filtering, Speech Communication, vol., no. 23, pp. 9 8, 992. [22] T. Drugman, B. Bozkurt, and T. Dutoit, A comparative study of glottal source estimation techniques, Computer Speech & Language, vol. 26, no., pp. 2 34, 22. [23] T. Drugman, GLOttal Analysis Toolbox (GLOAT), 22, downloaded November 22. [Online]. Available: drugman/toolbox/ 2346