An Array of First Order Differential Microphone Strategies for Enhancement of Speech Signals

Transcription

1 Master Thesis Electrical engineering Thesis no: MSE-20YY-NN MM YYYY An Array of First Order Differential Microphone Strategies for Enhancement of Speech Signals Naresh Reddy. NagiReddy Arun Kumar. Korva School of Engineering Blekinge Institute of Technology SE Karlskrona Sweden

2 This thesis is submitted to the School of Engineering at Blekinge Institute of Technology in partial fulfillment of the requirements for the degree of Master of Science in Electrical Engineering with emphasis on Telecommunications. Contact Information: Authors: Naresh Reddy. NagiReddy Arun Kumar. Korva University advisor: Dr. Nedelko Grbic School of Engineering (ING) Phone: School of Engineering Blekinge Institute of Technology Internet : SE Karlskrona Phone : Sweden Fax :

3 Abstract The quality and intelligibility of speech is degraded with the presence of background noise in the speech signal, which affects the listeners ability to understand the speech clearly. Speech enhancement is the process with which the background noise can be suppressed to improve the quality and intelligibility of the speech signal. With the development of speech based human computer interfaces, the demand for speech enhancement is growing. There are many applications like hand free mobile communication, teleconferencing, automatic speech recognition, hearing aids etc where there is a requirement for speech enhancement due to the noise interruption in the speech signal. Among all the applications mentioned, hearing aids are the ones which drew attention. This motivated us to go in depth with the research on hearing aids using different speech enhancement techniques and algorithms to enhance the quality of the speech at the end user. There are many algorithms and techniques that can be used to enhance the speech signals quality and intelligibility. Several beamforming techniques using multimicrophone arrays are widely used at present in the field of speech enhancement. In this thesis, Elko and Wiener beamforming algorithms with first order differential microphone arrays are being used to enhance the speech signal in an application, especially like hearing aids. The main reason for using Elko algorithm is: it tracks and attenuates the background noise or interference present in the back half plane of the microphone array. The Wiener beamformer is used as; it is a minimum mean square error beamformer which has the ability to nullify all the interference signals and sustains a high level of performance by getting signals from the desired direction. In this thesis, the assemblage of Elko and Wiener beamformers is also implemented as the Elko-Wiener Beamformer. These algorithms were implemented in a computer simulated anechoic chamber using MATLAB R2008. Recorded male and female speech signals sampled at 16 KHz were used as inputs to the system, where female speech is the target signal vocalizing from forward direction to the microphone array and male speech is the interference signal impinging from the backward direction to the microphone array. The performance metrics used to measure the quality of the speech signal are signalto-noise ratio increment (SNRI), speech and noise distortions and ITU-T recommended PESQ MOS values. The simulation results show that the Elko-Wiener Beamformer has the advantages of individual Elko and Wiener Beamformers giving 29dB SNRI. The Elko and the Wiener Beamformers has 10 db and 17 db SNRI respectively. Hence the Elko-Wiener joint Beamformer outperforms its individual beamformers in its class. Key words: speech enhancement, beamforming, microphone array, elko algorithm, wiener beamformer, differential microphone.

4 Acknowledgments We would like to dedicate our thesis work to our family members. We have our great respect and sincere gratitude for their support, love and encouragement. It is a great honor and privilege to thank our honorable thesis supervisor Dr. Nedelko Grbic at, ING School of Engineering, Blekinge Institute of Technology, who guided our work with his scholarly advice. Without his guidance and encouragement, this task would have been unachievable. We express our heartfelt gratitude to him. We are thankful to our friend LeelaKrishna.G who has been helping and guiding us in every aspect of our thesis work. We would like to thank our friends who supported us during our master s thesis. ii

5 To Our Parents iii

6 List of Figures 1.1 Path of speech from position P to L in a closed room Cardioid Super-cardioid Hyper-cardioid Bidirectional Shotgun Omnidirectional Working of beamformer First order sensor containing two zero-order sensors and a delay Directional responses for the two-element microphone array (a)t=0, (b)t=(d/c)/2 (c)t=(d/c) First order differential microphone using back-to-back cardioid system First order sensor containing two zero-order sensors and a delay Matrices for wiener beamformer (required and interferenced) [13] Diagrammatic representation of process Wiener beamformer structure Simulation set-up in anechoic chamber Elko Beamformer Simulation Set-up Signal plots representing degraded and enhanced Signals of Elko Beamformer Graph representing input and output SNRs of Elko Beamformer Graph representing input and output PESQ MOS values of Elko Beamformer Graph representing the Speech Distortion curves of Elko Beamformer at 0dB input SNR Graph representing noise distortion curves of Elko Beamformer at 0dB input SNR The wiener beamformer simulation set-up iv

7 7.9 Signal plots representing degraded and enhanced Signals of Wiener Beamformer Graph representing input and output SNRs of wiener beamformer Graph representing speech distortion curves of the wiener beamformer at 0dB input SNR Graph representing noise distortion curves of wiener beamformer at 0dB input SNR Graph representing input and output PESQ MOS values of wiener Beamformer Elko Wiener Beamformer simulation setup Elko-Wiener Signal Plots representing degraded and enhanced signals Graph representing input and output SNRs of Elko-Wiener Beamformer Graph representing the Speech Distortion curves of Elko-Wiener Beamformer at 0dB input SNR Graph representing the Noise Distortion curves of Elko-Wiener Beamformer at 0dB input SNR Graph representing input and output PESQ MOS values of Elko- Wiener Beamformer Input vs. output SNRs comparison plot SNR 3D comparison plot PESQ MOS comparison plot v

8 List of Tables 6.1 Quality option score used in PESQ Table showing results of elko beamformer under different input SNR levels Table showing results of wiener beamformer under different input SNR levels Table showing results of elko-wiener beamformer under different input SNR levels vi

9 Contents Abstract Acknowledgments i ii 1 Introduction Objectives of study Structure of report Differential microphones Microphones characteristics Microphone arrays Microphone polar pattern Beamformer Beamformer introduction and classifications Fractional Delays Elko beamformer Elko Derivation Wiener beamformer Wiener beamformer Wiener beamformer (WBF) Evaluation metrics Signal to Noise ratio Speech distortion normalized Noise distortion normalized Perceptual evaluation of speech quality (PESQ) Evaluation Procedure Results The Elko Beamformer The Wiener Beamformer vii

10 7.3 Elko Wiener Beamformer Results Comparison Plots Summary and Conclusion 38 9 Future Work 40 viii

11 Chapter 1 Introduction Research shows that 1.3 million persons have slight hearing impairment (age group of 18 and above). In addition to these 495,000 have moderate hearing impairment and 120,000 have a very severe hearing impairment [14]. When the environment becomes too noisy or reverberant using monaural hearing aids makes difficult for impaired listeners to understand speech easily [18]. Since hearing impairment reduces both binaural, monaural phenomenon, using hearing aids on both the ears to simulate cocktail-party effect (phenomenon of hearing with two-ears) fails to normalize listening among impaired listeners. However, hearing aids help to reduce difficulty in hearing but they are not a complete substitute and not everyone is benefited from using hearing aids. In general terms hearing aids is necessary for severely impaired listeners, in case of persons with moderate hearing impairment, hearing aids can improve speech intelligibility. The purpose of this study is to improve the efficiency of existing hearing aids used by impaired listeners. There are various properties and constraints, which influence working of hearing aids. Following are few important entities that contribute to the working of hearing aids: Properties of sound in an environment: To understand the properties of sound let us consider these properties of sound in a closed environment such as a room. When a person speaks in a closed environment such as a room, speech reaches listener directly. This speech does not only travel directly but the surrounding walls can also reflect it. To understand this concept further, take a look at the fig 1.1. This figure shows different paths that speech takes from a position P (Speaker) to L (Listener). The first bar shows a direct path and other paths are reflections. These reflections take longer distance to reach listener, than direct paths. Based on the path, direction and speed of these speech impulses, there is some variation in how speech arrives listener s ears. Based on this figure and the arrival time following graph can be drawn. This graph shows variation in arrival time at listener s ears. Path length and coefficient of absorption (reflecting efficiency of surrounding walls) collectively help to calculate the amplitude of a particular reflection. This coefficient of absorption varies from 0 to 1. Here 1 represents total absorption and 0 total reflection [18]. 1

12 Introduction Figure 1.1: Path of speech from position P to L in a closed room Any reflection that reaches the listener after 40 milliseconds of receiving the sound from direct path it might be heard as a distinct echo. Reflections that arrive 20 to 40 milliseconds of receiving direct path sound can create confusion and cause difficulty in understanding the speech. Acceptable levels sound: Human ears are sensitive and can respond to remarkable range of sounds. But sounds that are between 130 db to 140 db are painful to human ears [10]. Differences in sounds loudness are observed from 3 db at the lower threshold and 0.5 db for loud sounds. In addition to this a sound is considered to be high or low in a logarithmic pattern, where a ten fold increase in sound power is described as twice as loud. 1.1 Objectives of study From the above discussion it can be understood that human ears are sensitive to loud noises and at the same time there are some criteria, which should be taken into consideration while designing a hearing aid. It can also be made clear that simulating a normal human ear using hearing aids has not yet been successful. This thesis further aims to find out possibilities in designing a better hearing aid using an array of microphones and beamformer algorithms (Wiener and Gary- Elko). 2

13 Introduction 1.2 Structure of report Chapter 1: Introduction, on need for hearing aids, environment of hearing aids and objective of this study. Chapter 2: Differential microphones Chapter 3: Beamformer and fraction delays Chapter 4: Gary-Elko beamformer Chapter 5: Wiener beamformer Chapter 6: Evaluation metrics Chapter 7: Results, outcomes of the experiment conducted under controlled circumstances Chapter 8: Summary and Conclusion Chapter 9: Future work 3

14 Chapter 2 Differential microphones Hearing aid is combination of microphones arranged in a specific pattern and also a combination of few algorithms, which can analyze the signal received at the microphones. The properties and types of microphones are explained in detail under this chapter. 2.1 Microphones characteristics Microphones are an acoustic-to-electric transducer or sensor, which converts speech into electrical impulses. Microphones use electro magnetic induction, capacitance change, piezoelectric generation, or light modulation to convert mechanical vibration (i.e., pressure exerted by speech) into electrical signals, which are further processed. There are different varieties of microphones based on transducer principle such as condenser, dynamic etc. There are various other properties that contribute to selection of right set of microphones. They are as follows: Sensitivity: The microphone converts pressure exerted by speech into electrical signals. The extent to which a microphone can convert this pressure into intense electrical signals can be termed as sensitivity. Microphones sensitivity is important because, if a microphone is less sensitive then its output cannot be processed by certain applications. Similarly if the microphone is too sensitive it might produce lot of distortion in the output sound. Over load characteristics: Microphones when given loud sounds to process, then such sounds overload the input electronics and create distortions. When a microphone is continuously exposed to loud sound constantly, it can cause damage to the microphone diaphragm. If the diaphragm is damaged the microphone might lose sensitivity to process the normal sounds also. 4

15 Differential microphones Linearity or distortion: Noise: The distortion characteristic in a microphone varies in the way their diaphragm is designed and assembled. Premium quality headphones are those, which produce accurate electrical impulses with minimum distortion rate. The difference in model numbers is due to variation in quality of output these microphones produce. Microphones should be built in such a way that every part in it is carefully aligned. Since electrical impulse from microphones are very small, even if there is a slight disturbance that can make a huge difference to the sound when it is amplified. So slightest detail in building a microphone must be kept into consideration while selecting it for use. 2.2 Microphone arrays Microphones are positioned in an array to capture spatial information easily. When these microphones are used in arrays in signal processing, it can help in estimating some parameters or extraction of some signals. The extraction of signals depend on the application and spatio-temporal information available at output of microphone [6]. Depending on the type of application, microphones are arranged. If this arrangement is done application specific it can help in formulation of processing algorithms. Microphone array processing algorithms are many, some of them are generated and some others are borrowed from narrow band array processing. The advantage of borrowing algorithms available with antenna arrays is that these can be extended without much effort [16]. But the issue is that these algorithms are not made to work in the real acoustic environments. These microphone arrays when used properly help to solve many problems such as: noise reduction, echo reduction, dereverberation, localization of a single source, estimation of the number of sources, localization of multiple sources, source separation, and 5

16 Differential microphones cocktail party. When sounds received at microphones are passed through filters the above problems can be reduced. For example in a hands-free devices, signals received are not only those received from a direct path, but there are some other signals that are received from delayed replicas of original signal. These delayed replicas are received due to reflections from boundaries and objects in a room. This kind of reflected signals that are received at microphones end are due to reflections and diversion in speech signals and are termed as reverberations. To improve the intelligibility of speech signals de-reverberation is required. This improvement is possible when reverberation can be identified. In acoustic environments and applications such as automatic camera tracking, beamformer steering for reducing noise and reverberations, identifying source locations can be helpful. The problem of identifying source is often called as source-localization. Positioning microphone arrays in 2 or 3 dimensions can help in finding arrival angle or cartesian coordinates of a source. In addition to this microphone arrays can also help separate signals coming from same or different directions and same or different source. In some situations using multiple arrays of microphones or multiple microphones in an array can increase possibility of solving this estimation problem [5]. 2.3 Microphone polar pattern Microphones have sensitivity to receiving speech signals, based on the direction of this sensitivity to speech signals microphones directionality or polar patterns is decided. Following are some of the polar patterns [3]: Omnidirectional: This type of microphones response is a perfect sphere in 3 dimensions. The polar pattern of an omnidirectional microphone is function of frequency. Microphones with smallest diameter exhibit omnidirectional characteristics at high frequency. Since these microphones do not employ resonant cavities as delays, these microphones can be called the purest microphones. Bi-directional: Sound reaches equally at front and back in this kind of microphones. Sounds received from sides do not influence these microphones, because these sounds reach both front and rear side of microphone at same time, which causes no gradient. These microphones respond to gradient along the axes normal to the plane of diaphragm. Shotgun: These microphones are highly directional. Sensitivity of this microphone is concentrated towards front and it has small areas of sensitivity on 6

17 Differential microphones Figure 2.1: Cardioid Figure 2.2: Supercardioid Figure 2.3: Hypercardioid Figure 2.4: Bidirectional Figure 2.5: Shotgun Figure 2.6: Omnidirectional rear, left and right. These microphones are commonly used in television and film sets and for field recording in wildlife. Unidirectional: A unidirectional microphone is sensitive to sounds from only one direction. The most common unidirectional microphone is Cardioid microphone. Cardioid: This microphone is named cardioid because its sensitivity pattern is heart shaped. There are two other types under this category. They are, hyper-cardioid and super-cardioid. The hyper-cardioid microphone has a smaller rear sensitivity and tighter front sensitivity, but the difference in case of super-cardioid microphone is that there is slightly lesser rear sensitivity and slightly more sensitivity in forward direction. These three are generally used as speech microphones. 7

18 Chapter 3 Beamformer 3.1 Beamformer introduction and classifications There is occurrence of noise and interfering signals in spatially propagating signals. If the actual signal and interference signals occupy same temporal frequency, then temporal filtering cannot help separating actual signal from the interferers. But in general the interferer signal and original signal originate from a different spatial locations. This spatial separation can be used to identify direct signal from interferers signal using a beamformer [2]. Beamformer has an array of sensors arranged in a specific pattern. Each of these sensors is fixed with filters and the final filtered output signal is the resultant obtained after summation of each of these individual sensors output. These sensors can help synthesize large aperture when a low frequency signal is used, which is not quite possible with a single physical antenna. Another advantage is that, using array of sensors spatial filtering versatility is offered by discrete sampling [1]. Working of beamformer As described above beamformer is an arrangement of sensors and each of these sensors are multiplied with certain weight and summed up to get the final output signal. A diagrammatical representation of this process is given in fig 3.1 In the above figure sound (speech) is sensed on left side of arrangement and then calculated based on weights to give the output signal on the right side. This representation is a more generalized form of a beamformer. The weights can be Figure 3.1: Working of beamformer 8

19 Beamformer fixed weights or adaptive weights. If weights dont vary they are fixed weights. But in some cases if the weights vary based on the position of sensors and various other conditions then they are adaptive weights. Based on how weights are chosen beamformers are further categorized as: Statistically independent Data independent Statistically optimum beamformer In statistically optimum beamformer weights are chosen based on the statistics of array data and in case of data independent beamformer selection of weights does not depend on array data. Statistics of this data keeps varying over time so adaptive algorithms are used to calculate weights. Adaptive algorithms are designed so that beamformer responses come to a statistically optimum solution. Data independent beamformer In a data independent beamformer, weights are arranged such that beamformer can estimate desired response without depending on array data or data statistics. An example to this data independent beamformer is Delay and sum beamformer. When weights are chosen statistically from the data received at the arrays then that is called as statistically optimum beamformer. The output of through this beamformer consists of very few noise and signals that are arriving from other than the desired directions. An example of this statistically optimum beamformer is Frost beamformer. Application of beamformer is in SONAR, RADAR, communications, imaging, geophysical exploration, Biomedical and also in acoustic source localization [19]. 3.2 Fractional Delays Fractional delay (FD) filters help in fine-tuning the sampling instances. The application of this is in many areas such as audio, communication, music technology, speech coding and synthesis, etc. [24]. This FD filters are used in some of these applications because actual samplings instances are also important in addition to the sampling frequencies. For any sample to represent the continuous original signal, it is required that the sample rate satisfy Nyquist criterion. In addition to satisfying Nyquist criterion, it is required that the sampling instances should also be properly selected. The examples where this selection criterion is important are in case of digital communications and in solving problems with modeling of musical instruments. Fractional delay is defined as assuming uniform sampling, a delay that is a non-integer multiple of the sample interval [24, 15, 23, 9, 25]. 9

20 Beamformer There are two known types of filters for approximating a fractional delay value they are designed in a non-recursive and recursive fashion. FIR come under nonrecursive category and IIR, allpass filters come under recursive. This thesis uses Thiran allpass filter. Allpass filter is used as its magnitude of response is exactly unity at all frequencies and is well suited for fractional delay approximations. Other known allpass filters are as follows [15]: 1. Least squaes (LS) phase approximation; 2. LS phase delay approximation; 3. Maximally-flat group delay approx. (thiran allpass filter) 4. Iterative WLS phase error design (enables almost equiripple phase approximation 5. Iterative WLS phase error design (enables almost equiripple phase-delay approximation) Allpass filters are used in this context as they have some advantages over FIR filters. A comparison between allpass filter and FIR show these advantages [22]. Comparison in terms of frequency response error (FRE) magnitude shows that the order of allpass filter is 5 and length of approximations in case of FIR is 10 (these results are based on a wideband specification where 80% Nyquist limit is the passband of approximation). When a window function with 35-dB ripple level was selected for sinc windowing and a low pass (FIR FD) filter was considered the coefficients had to be scaled to obtain best approximation, but when allpass filters were considered they automatically scaled by the algorithms that were designed. There is difference in the range of delay D between FIR and allpass filters too. Error curves are symmetric in allpass filters and in case of thiran interpolation there is a lower limit for the delay to be approximated and is stable only for D > N 1. The Thiran allpass filter gives low performance in terms of peak FRE but it is easy to design and better suits for a narrow-band approximations [? ]. 10

21 Beamformer The Thiran all-pole filter is used to maximally flat allpass FD filter. It can also be noticed that even when the order is increased, there is not much decrease in error identified. The sample MATLAB code used in this thesis is as follows: function [A, B] = thiran(d,n) % [A, B] = thiran(d,n) % returns the order N Thiran allpass interpolation filter % for delay D (samples). A = zeros(1, N + 1); for k = 0:N Ak = 1; for n = 0 : N Ak = Ak (D N + n)/(d N + k + n); end A(k + 1) = ( 1) k nchoosek(n, k) Ak; end B = A(N+1:-1:1); 11

22 Chapter 4 Elko beamformer 4.1 Elko One of the basic problem with acoustic transduction is detrimental effect of background noise and as these communication devices are starting to become more portable, the acoustic pickup by microphones need to be designed such that they include a combination of mini transducers and signal-processing which can allow high quality communication. Reverberation is another issue that can have a negative impact on reception quality in hands-free applications. To solve these issues directional microphones can be used and in case of teleconferencing and personal communications differential microphone array is more suited. The differential array of microphones uses sensors, which are placed very closely when compared to acoustic wavelength. Since these microphones are closely spaced and arranged in a alternating sign fashion, this can help to realize directionality. As a result this arrangement gives a scope to produce increased directivity, which is higher than the summed output of uniformly arranged sensor elements. Using this Elko suggested a different method for adaptive directional microphones [8, 7, 20, 21]. This suggested method includes implementation and design of adaptive first-order differential microphone that reduces microphone output power (constraint is first-order microphone null is located in rear-half plane). When this proposed method is used in some acoustic fields, there is improvement observed in signal-to-noise ratio. The idea behind this design is to propose an adaptive microphone system that adjusts its directivity based on speech signals and maximizes signal-to-noise ratio. This adaptive differential microphone is combination of two omnidirectional elements to form back-to-back cardioid directional microphones. Weighted subtraction of these outputs helps to realize first-order array [8] Derivation Assume a sound wave of s(t) with a spectrum S(ω) is incident at an angle θ on to a two-element microphone array, which are spaced at a distance d. Since there is an angle of incidence to the two-element microphone array, sound will reach 12

23 Elko beamformer Figure 4.1: First order sensor containing two zero-order sensors and a delay both these microphones with a time delay of τ. And this τ can be denoted as τ = a c = dcosθ c (4.1) In the above equation (4.1) c is the speed with which sound propagates. To achieve the final sound signal y(t), output of sound signal at microphone which receives the sound signal earlier to that of the other needs to be subtracted. y(t) = s(t τ) s(t T ) = s(t cosθ ) s(t T ) (4.2) c Frequency domain form to the equation (4.2) is Y (ω, θ) = (e jωdcosθ c e jωt ) (4.3) the directional response for arrangement shown in fig above (fig.4.1) is shown below figure (fig.4.2). Here we can observe that when delay is varied among different values of T (i.e. 0 and d ) null location can be changed or steered from c 90 o to 0 o One way to work out in achieving adaptive directional microphone is to implement a changing time delay T on the above discussed array of microphones. For this to happen is feasibility to generate 0 T d and calculations for these c approximations in real-time are going to be numerous. From the figure it can be identified that when a fixed time delay of one sample and when a time period is maintained at d. This requires less computational cost even. So based on this a c back-to-back cardioid system can also be implemented, which can be done by the setup described in fig.4.3 The output to this can be written as C F (t) = s(t) s(t T τ) = s(t) s(t d c dcosθ ) c (4.4) C B (t) = s(t τ) s(t T ) = s(t dcosθ ) s(t d c c ) (4.5) y(t) = C F (t) β C B (t) (4.6) 13

24 Elko beamformer Figure 4.2: Directional responses for the two-element microphone array (a)t=0, (b)t=(d/c)/2 (c)t=(d/c) Figure 4.3: First order differential microphone using back-to-back cardioid system Frequency domain form to equation (4.6) is Y (ω, θ) = C F (ω, θ) β C B (ω, θ) (4.7) Y (ω, θ) = S(ω) 1 e jω d c (1+cosθ) βe jω d c cosθ + e jω d c (4.8) in the above equation time delay T is fixed and null can be varied between 180o and 90o by changing β from 0 to 1. The time delay maintained T=1 and by changing values of β the directional responses shown in fig 4.4 are observed in the system. To make the system adaptive further, LMS algorithm is used with back-to-back cardioid first order differential array. Differentiating and squaring equation (4.6) results in dy 2 (t) = 2y(t)C B (t) (4.9) dβ The resultant LMS version of above equation is β t+1 = β t + 2µy(t)C B (t) (4.10) 14

25 Elko beamformer Figure 4.4: First order sensor containing two zero-order sensors and a delay Normalizing the above equation gives us, β t+1 = β t + 2µy(t) C B(t) C b 2 (t) (4.11) Here, µ is the step size and C b 2 (t) is the time to normalize µ. 15

26 Chapter 5 Wiener beamformer 5.1 Wiener beamformer Wiener beamformer is also called as minimum mean square error beamformer [11]. A detailed discussion on wiener beamformer is described in articles [11, 26, 12, 13] W opt = argmin w E{y[n] s r [n] 2 }r [1, 2,...N] (5.1) Single reference mic observation when only required signal is selected as input is denoted by S r [n]. In the above equation y[n] is the output and it is the optimum output signal that is inferred from optimal weights. y[n] = N L 1w i [j]x[n j] (5.2) i=0 j=0 The above equation is inferred from beamformer. Here L-1 is order of filter and w[j]. Where j varies from 0 to L-1 and these valuse are the filter taps for I th mic. x i (n) is the i th mic observation and N is microphone number.given below is the equation to maximize mean square error between output and reference signal. W opt = [R ss + R nn ] 1 r s (5.3) In the above equation auto-correlation matrices for signal of interest and noise are denoted by R ss and R nn respectively. R s1s1 R s1s2... R s1sn R s2s1 R s2s2... R s2sn R ss =. (5.4)..... R nn = R sns1 R sns2... R snsn R n1n1 R n1n2... R n1nn R n2n1 R n2n2... R N2nN (5.5) R nnn1 R nnn2... R nnnn 16

27 Wiener beamformer The cross correlation vector r s is denoted as And each element in this is represented as, r s = [r 1 r 2...r n ] (5.6) r i [k] = Es i [n]s r [n + k]i = 1, 2, 3,...N, r [1, 2,...N], K = 0, 1, 2,..., L 1 (5.7) The autocorrelation matrix of the required signal R ss is designed around cross correlation vector r s. The optimal weights (w) are arranged as, w = [w T 1 w T 2... w T n ] T (5.8) In the next chapter wiener beamformer is evaluated and results are discussed Wiener beamformer (WBF) To get microphone response at each microphone, simulation procedure for wiener beamforming is initiated before adding required and interference signal. For this interference and desired signal are considered separately (at each microphone) and represented in a matrix form. The order of required and interference matrices would be N x S, when S is assumed to be the total number of speech samples and interference signals at each microphone. Microphones response to desired/interference is represented by each row in the matrix. Figure 5.1: Matrices for wiener beamformer (required and interferenced) [13] As discussed earlier section, equation 13 represents the optimum weights that maximizes mean square error between reference signal and output. This requires auto correlation matrices (for interference and required signals), correlation vectors. The method used to infer these matrices with the help of M S and M I is as follows [13]: 17

28 Wiener beamformer 1. Assuming the order of wiener filter to be 64, create an empty matrix with order 64*N x K as X. Here, K is the integer part S/ In the next step pick first 64 columns from matrix M S and make these selected elements into a N x 64 matrix into a single column vector and name it as x 1. Now, substitute the first column of X with x Similarly pick next 64 columns (65 to 128) from M S, convert it into column vector, name it as x 2 and substitute it in the second column of X. continuing this process till end of M S and this will give matrix X. in the same way matrix Y can be obtained from M I. Diagrammatic representation of this process is shown in Figure 5.2: Diagrammatic representation of process From these auto-correlation matrices X and Y interference signal I(n) and required signal s(n) can be calculated through the following equations: required (Rss) and 18

29 Wiener beamformer interference signal (Rnn) can be represented in auto-correlation forms as follows: R ss = 1 = 1 k (x i.x T i )R nn = 1 = 1 k (y i.yi T ) (5.9) K K i In the equation Eqn. (5.3) The cross-correlation vector term (r s ) is inferred from centre column of auto-correlation matrix R ss. To concentrate on interference signal alone the cross-correlation vector needs to be selected as center column of auto-correlation matrix (R nn ). The optimum weight vector W opt can be calculated, as we have required entities. For this example considered above order of optimum weight vector W opt is N*64 x 1. Following steps can be executed to obtain the wiener beamformer response s (n): 1. Pick first 64 elements of W opt, flip upside down and name it as w N. 2. Pick the next 64 elements and flip them upside down and name it as w N Continue this procedure till the end of W opt i.e. till w 1 is obtained. Using these individual weight vectors filter N microphone responses in the array. Finally summing up all these outputs can retrieve desired output and process is described in the figure shown below. i Figure 5.3: Wiener beamformer structure 19

30 Chapter 6 Evaluation metrics These metrics are used to find quality of speech produced when a speech enhancement system is used. A speech communication system is expected to reproduce sound to the listener, the same way it is received by the speech communication system. This system is also expected to suppress interference or noise in the environment and enhance the desired sound. There are different metrics that help us evaluate speech communication system and in this thesis I used following metrics: Signal to noise ratio: Speech distortion normalized: Noise distortion normalized: Perceptual evaluation of speech quality: 6.1 Signal to Noise ratio The signal to noise ratio can said as ratio between meaningful/ desired signal to background noise. In this experiment since there is no background noise introduced but the interference is considered as noise. SNR = P s P n. (6.1) (where P s is average power of signal and P n is average power of noise) Since most of the signals have varied dynamic range, logarithmic notation is used to express SNRs decibel scale [4]. SNR db = 10 log 10 P s P n (6.2) Hence, signal to noise ratio improvement (SNRI) is given by, SNRI = SNR O UT - SNR I N. 20

31 Evaluation metrics 6.2 Speech distortion normalized The deviation of power spectral density of clear speech signal at an assumed reference microphone to the output signal received after processing speech is normalized speech distortion. A power level reference is obtained from enhanced output signal by normalizing target speech signal. The speech distortion is denoted as: abs(zs 1 (Ω) S(Ω)) SD normalized = 10 log 10 (6.3) abs(s(ω)) Where Z 1 S (Ω) is, Z 1 S(Ω) = N S Z S (Ω) (6.4) Here N s is normalization constant given as, N S = mean(s(ω)) mean(z S (Ω)) (6.5) Where S(Ω) and Z S (Ω) are power spectral density of S(n), Z(n) respectively. 6.3 Noise distortion normalized The deviation of power spectral density of clear interference signal at an assumed reference microphone to the output signal received after processing speech is termed as normalized noise distortion. A power level reference is obtained from output signal by normalizing target noise signal. The speech distortion is denoted as, abs(zi 1 (Ω) I(Ω)) SD normalized = 10 log 10 (6.6) abs(i(ω)) Where Z 1 s (Ω) is, Here N I is normalization constant given as, Z 1 I (Ω) = N I Z I (Ω) (6.7) N I = mean(i(ω)) mean(z I (Ω)) (6.8) Where I(Ω) and Z I (Ω) are power spectral density of I(n), Z(n) respectively. 6.4 Perceptual evaluation of speech quality (PESQ) This is a family of standards with a test strategy to automatically assess the speech quality as experienced by a user of hands free device, which is standardized by ITU-T recommendation P.862. The PESQ has a sensory model that 21

32 Evaluation metrics compares the original signal to the degraded signal obtained after processing. This comparison results in a Mean Option Score (MOS), which is designed after a sequence of tests conducted (MOS shown in Table 6.1) [17]. The PESQ takes into consideration various errors, coding distortions, packet loss, delay and variable delay, and filtering in analogue network components. The interface is designed such a way that recorded speech files or analogue connections can be easily accessed. Which can be used in many fields of business, education and on any channels. Table 6.1: Quality option score used in PESQ Quality of speech score Excellent 5 Good 4 Fair 3 Poor 2 Bad Evaluation Procedure The evaluation routine that we followed through out the thesis consists of the following two steps: First, the beamformer is allowed to converge and all the associated weights in the structure of the beamformer are saved. Now, the desired speech and the interference speech signals are filtered solely using the above saved weights, in order to calculate the mentioned performance metrics. 22

33 Chapter 7 Results This chapter deals with the evaluation of the implemented beamformers. A standardised environment is considered for all the beamforming algorithms in order to compare their individual performances. We used a six element linear differential microphone array throughout the thesis work. The intra-element distance between the microphones is cms. A female speaker recording and a male speaker recording were used as input signals to the microphone array. The female speech is the target signal impinging from the forward path of the array and the direction of arrival (DOA) was set at 45 o to the centre of the array. The male speech is the interference signal impinging from the backward path of the array and its DOA was set at 135 o to the centre of the array. These two input signals were sampled at 16 khz frequency. Fractional delays have been considered. The beamforming algorithms were executed at different input SNR levels and their performances are compared using the metrics described in chapter 6. A vivid description of the simulation set-up is as shown in Fig Figure 7.1: Simulation set-up in anechoic chamber 23

34 Results 7.1 The Elko Beamformer The Elko beamformer simulation set-up is as shown in the Fig Figure 7.2: Elko Beamformer Simulation Set-up. Normalized Least Mean Square (NLMS) Algorithm has been used to incur the weight parameter while performing Elko beamforming. The Elko algorithm has the ability to nullify one particular background noise/interference at a time giving signal-to-noise ratio increment (SNRI) of about 10 db. It clearly enhances the distorted signal as shown in Fig 7.3. Figure 7.3: Signal plots representing degraded and enhanced Signals of Elko Beamformer. 24

35 Results The table below gives the SNRI, speech and noise distortion values under different input SNR levels. Table 7.1: Table showing results of elko beamformer under different input SNR levels Input SNR SNRI Speech Distortion Noise Distortion 0 db db db db db Below figures are the observations while performing Elko Beamforming. Figure 7.4: Graph representing input and output SNRs of Elko Beamformer 25

36 Results Figure 7.5: Graph representing input and output PESQ MOS values of Elko Beamformer Figure 7.6: Graph representing the Speech Distortion curves of Elko Beamformer at 0dB input SNR 26

37 Results Figure 7.7: Graph representing noise distortion curves of Elko Beamformer at 0dB input SNR From the above results, we can state that the performance of the Elko Beamformer is good with an average SNRI of about 10 db and increment in the PESQ MOS values are also observed. The suppression of the backward interference is the significant observation while implementing Elko beamformer. 27

38 Results 7.2 The Wiener Beamformer The simulation set-up of the wiener beamformer is as shown in Fig 7.8 Figure 7.8: The wiener beamformer simulation set-up The wiener beamformer provides promising results and clearly enhances the input distorted signal as indicated in the below Fig. 7.9 Figure 7.9: Signal plots representing degraded and enhanced Signals of Wiener Beamformer 28

39 Results The table below gives the SNRI, speech and noise distortion values under different input SNR levels. Table 7.2: Table showing results of wiener beamformer under different input SNR levels Input SNR SNRI Speech Distortion Noise Distortion 0 db db db db db Below figures are the observations while performing Wiener Beamforming. Figure 7.10: Graph representing input and output SNRs of wiener beamformer 29

40 Results Figure 7.11: Graph representing speech distortion curves of the wiener beamformer at 0dB input SNR Figure 7.12: Graph representing noise distortion curves of wiener beamformer at 0dB input SNR 30

41 Results Figure 7.13: Beamformer Graph representing input and output PESQ MOS values of wiener From the above results, we can state that the performance of the wiener beamformer is quite promising with an average SNRI of about 15 db and the increment in the PESQ MOS values representing a good quality speech signal. The wiener beamformer is then used to get signals from the desired direction. 31

42 Results 7.3 Elko Wiener Beamformer Figure 7.14: Elko Wiener Beamformer simulation setup Elko-Wiener Beamformer has the advantages of both the Elko and Wiener Beamformer. SNRI values of Elko-Wiener Beamformer is significantly appreciable than the Elko and Weiner Beamformers individual SNRI values. Also PESQ MOS values of Elko-Wiener Beamformer values represent a high quality speech signal. Elko-Wiener Beamformer clearly enhances the distorted signal as shown in the figure. Figure 7.15: Elko-Wiener Signal Plots representing degraded and enhanced signals 32

43 Results The table below gives the SNRI, speech and noise distortion values under different input SNR levels. Table 7.3: SNR levels Table showing results of elko-wiener beamformer under different input Input SNR SNRI Speech Distortion Noise Distortion 0 db db db db db Below figures are the observations while performing Elko-Wiener Beamforming. Figure 7.16: Graph representing input and output SNRs of Elko-Wiener Beamformer 33

44 Results Figure 7.17: Graph representing the Speech Distortion curves of Elko-Wiener Beamformer at 0dB input SNR Figure 7.18: Graph representing the Noise Distortion curves of Elko-Wiener Beamformer at 0dB input SNR 34

45 Results Figure 7.19: Graph representing input and output PESQ MOS values of Elko-Wiener Beamformer From the above results, we can state that the Elko-Wiener Beamformer provides quite promising results and sustains a high level of performance with an average SNRI of about 28dB. The rise in the PESQ MOS values representing a high quality speech signal. Hence, The Elko-Wiener beamformer could be used for applications in which the user wants to nullify the backward signals and preferred to get signals from the desired direction. 35

46 Results 7.4 Results Comparison Plots In the following plots all the three implemented beamformers results are compared. Figure 7.20: Input vs. output SNRs comparison plot 36

47 Results Figure 7.21: SNR 3D comparison plot Figure 7.22: PESQ MOS comparison plot 37

48 Chapter 8 Summary and Conclusion Enhancement of noisy speech signals has been investigated with three different beamforming techniques applied to linear array of differential microphones. These are: a well-known Elko beamforming technique, Wiener beamformer and a new Elko-Wiener beamformer. One of the major contributions of this thesis is the accomplished description of these techniques given in chapter 5 and 4. The Elko beamformer minimizes the first order differential microphone output power by locating its null in the rear-half plane. The Wiener beamformer minimizes the mean square difference between the beamformer output and a single microphone output. WBF is typically implemented using omnidirectional microphones. However, in the present thesis work it has been developed using back-to-back cardioid directional microphones. The assemblage of these two beamformers has been implemented as the Elko-Wiener beamformer, which is now proved to be advantageous than the individual beamformers. Concerning the simulation of these beamformers, two speech recordings sampled at 16 KHz have been used as inputs to the six element linear array of differential microphones. The DOA s of the target and interference signals was set at 45 o and 135 o to the centre of the array. These algorithms were implemented in a computer simulated anechoic chamber and a similar test environment has been considered in order to compare their performances. The Elko algorithm was successful in cancelling backward interference from any position. The Wiener beamformer was quite promising in extracting the target signal from the desired direction. Hence, the Elko-Wiener beamformer has the ability to nullify the backward signals and preferred to get the signals from the desired direction. Taking these constraints into account, the performance of the linear differential microphone array using the above mentioned beamformers has been simulated and evaluated by means of three measures, SNRI, distortion measures and the ITU-T recommended PESQ MOS values. The Elko algorithm has a SNR improvement of 10dB and posses a rapid adaptation with less computational complexity. The Wiener beamformer has a SNR improvement of 17dB with moderate complexity. The Elko-Wiener beamformer outperforms the individual Elko and Wiener beamformer and possesses a SNR improvement of 28dB. The cost of this high level performance is its computational complexity. The simulation time is 38

49 Summary and Conclusion very high compared to the individual two beamformers. Finally, it must be pointed out that the Wiener beamformer plays active role in getting the desired results. If the target and interference both are in the same direction of the plane then the Wiener beamformer itself will enhance the noisy speech with minimal target speech distortion. 39

50 Chapter 9 Future Work The fact that the proposed beamformer concedes promising results could encourage us to work for the improvement of such beamformer. The following suggestions for future work are given: During the course of work, only interference has been considered in the rear half plane of the array. The Elko algorithm significantly removes the interference from any position in the rear half plane of the linear array. However, if multiple interferers are used in the rear half plane then just by using series of Elko algorithms would be very convenient to remove all the unwanted sources in rear half plane. It would be very useful and challenging to find such results. Although the Elko-Wiener beamformer presents better performance than the individual Elko and Wiener beamformers, the computational complexity and simulation time are still rather coarse and hence it is suggested to take necessary steps in avoiding such circumstances. While linear microphone arrays delivering proficient results, beamformers should imply different approach by using other array geometries like circular arrays, semi circular arrays, etc. 40