Array processing for echo cancellation in the measurement of Head-Related Transfer Functions

Transcription

1 Array processing for echo cancellation in the measurement of Head-Related Transfer Functions Jose J. Lopez, Sergio Martinez-Sanchez and Pablo Gutierrez-Parera ITEAM Institute, Universitat Politècnica de València, 46022, Valencia, Spain. Summary The use of Head-Related Transfer Functions (HRTF) is necessary for individualizing binaural sound for each subject so as to provide a better experience. For measuring HRTF, motorized positioning systems and anechoic chambers are usually employed, which implies the utilization of expensive and complex facilities that are not always available. Measurement in normal rooms would be desirable, but reflections which are not part of a HRTF would be introduced. In this paper, a method to cancel these echoes by means of signal processing techniques allowing to measure in non-anechoic rooms is proposed. The sound field is captured by using a spherical array of microphones, placed where the listener will be situated, and obtaining the room impulse response (RIR) between each loudspeaker position and each microphone of the array. The use of array techniques as Plane Wave Decomposition (PWD) allow to discriminate direction of reflections. The proposed system is a sub-band method, where the higher frequency part of the HRTF is obtained by cropping the first segment of the RIR before the first reflection and the lower one by using array techniques. Two spherical microphones of different diameter will be compared for this task. PACS no Pn, Fg 1. Introduction The future of spatial sound is bound up with binaural audio, as headphone market has experienced a remarkable expansion [1] and more and more audiovisual content is being consumed through headphone listening. Virtual and augmented reality, and immersive technologies are also using headphones to reproduce 3D audio content with the binaural approach. Binaural sound employs Head-Related Transfer Functions (HRTF) to generate the spatialization of sounds to listen with headphones. The HRTF captures the effects that a source in free-field experiences to the ear canals of a subject. The contribution of the head and torso and significantly the outer ear, are registered in the HRTF [2]. Due to the strong influence of the anthropometric characteristics of the listener (size of the head, position of the ears, shape of the pinna, etc.) the HRTFs present tailored features that make them specific for each subject. Then, for a better experience, the binaural sound must be individualized for each subject through the use of their personal HRTF. Individualized HRTF provide a better immersive and natural listening experience [3]. It is possible to accurately simulate virtual sound sources in any position in the space just convolving the source signal with the individualized HRTF. There are different techniques that try to obtain personalized HRTFs: measuring directly the subject s HRTF in an anechoic chamber, through anthropometric data (including synthesis calculation with numerical methods), based on subjective perception, etc. [4]. The traditional method for measuring HRTFs was described in the Blauert in [5]. It uses a single loudspeaker mounted in a positioning system that measures the acoustic transfer path between the loudspeaker and the two microphones inserted in the subject ears. Using the positioning system, the loudspeaker is moved to different points from a virtual sphere around the subject, and the acoustic paths from these locations are measured. Despite the minimum audible angle is around 1-2 for frontal sources [6], the HRTF set resolution should be around 5 in the horizontal plane and 10 in the vertical plane for common applications. According to this rule, the number of HRTF measurement points should be bigger than However, different interpolations techniques have been proposed, where using less points, produce successfully results in practice. In any case, because of the large number of measurement points (from hundreds to thousands) the total Copyright 2018 EAA HELINA ISSN: All rights reserved

2 Euronoise Conference Proceedings time for a personal HRTF recording session could extend to even more than one hour with the traditional method where only one loudspeaker or just a few are employed. This causes fatigue and discomfort in subjects because they should not be moved to avoid measurement errors. To reduce the measurement time, the most obvious method is to install multiple loudspeakers in multiple positions to save the time required for positioning system movements. However, one loudspeaker in each measuring position is impractical, because it would require about a thousand units. Therefore, hybrid combinations have been used, using multiple loudspeakers to cover a polar angle and a single-axis positioning system to cover the other. The most common hybrid method uses as many loudspeakers as there are elevations to be measured, installed in an arc around the listener. In this way, the single-axis positioning system rotates the arc structure around the listener or rotates the listener seated on a turntable [7, 8]. There are also methods for simultaneous measurement of different HRTFs using multiple loudspeakers. A multiple exponential sweep method (MESM) was proposed in [8] and refined in [9], which allows the simultaneous playing of sweep signals through various loudspeakers, saving even more time. The HRTF refers to a free field measurement, so the measurement inside a room will introduce reflections that are not part of the HRTF. Anechoic conditions are, then, generally necessary for the HRTF measurement environment. The measurement systems described above should be installed in an anechoic chamber, setting up a complex, very expensive and not everywhere available installation, restricting these measurements to research laboratories and keeping these technologies away from the general public. These setbacks led to the appearance of new and innovative methods for individualizing HRTF based on the use of databases with anthropometric data and numerical methods. However, other obstacles related to matching physical human body parameters were needed to overcome, despite getting rid of anechoic chambers and positioning systems. Consequently, the possibility of employing a non-anechoic room with multiple loudspeaker arrays in different planes for HRTF measurement is considered as an attempt of bringing spatial sound closer to the general public. Additional processing, e.g. Spatial Decomposition Method (SDM) or Plane Wave Decomposition (PWD), is needed in order to detect room reflections and cancel them to obtain precise and clear HRTF. In this paper, the deployment of a complete system of loudspeaker arrays in a non-anechoic room is presented, together with an echo cancellation system. Therefore, in section 2 the aims of the project and hardware set-up of the proposed system are explained. The description of the employed microphone arrays, as well as the software environment based on Plane Wave Decomposition for discriminating the direction of reflections, are detailed in section 3. Section 4 covers the methodology followed to cancel the echoes in different measurement scenarios, describing the subband processing which will allow to establish a fullband method. The results after processing and the discussion are performed in section 5, highlighting the key points and problems that could arise from the method. Finally, section 6 summarizes the main idea of this paper and future lines to be addressed in order to improve the system are also proposed. 2. Objective and Set-Up In the previous section, the main drawbacks of traditional HRTF measurement methods have been noted. Also, synthesis of individualized HRTF has been commented as possible method. Anyway, until the last methods are precise enough, measuring the HRTF would be the best method for individualization. Moreover, in order to continue working into intelligent methods for HRTF individualization based on deeplearning techniques [10, 11], large collections of HRTF would be needed for training the systems. In order to develop a feasible, easy accessible and comfortable HRTF measuring installation for intensive use, a normal room instead of an anechoic would be desirable. In this paper a set-up and methodology for HRTF measuring in a non-anechoic room are presented. The method encompasses two important points: The use of different loudspeaker arrays in different planes in order to avoid the complex positioning system. The development of an echo cancellation system to remove undesired reflections inside the room. In this manner, it is possible to get rid of anechoic chambers and provide alternatives to the mechanical positioning system. Both elements contribute to build a set-up which allows to save measurement time, hardware complexity and costs. The set-up previously described in [12] has been used as a basis for an evolved system. It consists of a circular 2-meters array of 72 loudspeakers, composed by self-amplified monitors M-Audio model BX5 D2, providing a resolution of 5 in azimuth. This array provides just information for the horizontal plane. As reflected in Figure 1, the set-up evolved to include elevation. A circular array of 1 meter of radius has been deployed on the floor, concentric to the array of 72 loudspeakers. In this case, 8 loudspeakers have been used, placed each 45 in azimuth, and leaning at 45 with respect to the floor axis, in order to point to the listener position. Another circular array

3 Euronoise Conference Proceedings Figure 1. 3D model of the room with the complete system. of the same radius, also composed of 8 loudspeakers, has been suspended from the ceiling, being each loudspeaker at an angle of 45 to the ceiling axis. Finally, a single loudspeaker is placed at an elevation of 90 (zenithal position) in the center of all the arrays, oriented to the listener position. The presented configuration allows to measure four different elevation levels. The selected positions and radius of arrays coincide with the reflection point on the floor and ceiling and, as it is described in section 4, they will be useful in order to achieve the elimination of those echoes. Nevertheless, it is not possible to completely describe the whole evolution of the HRTF in the elevation plane with these four levels. Thus, new proposals for future set-ups come into mind in the sense of adding several loudspeakers in intermediate positions in elevation plane. For instance, it would be interesting to set additional loudspeakers in the elevation between 0 and 45. On the other hand, intelligent interpolation systems based on neural networks and deep learning are being developed in the literature and it would be profitable to take advantage of the data these systems provide so as to guarantee an interpolation of the aforementioned intermediate points, leaving the system untouched as it has been described or adding few loudspeakers. In fact, these techniques would cause a reduction in hardware equipment, since it would not be necessary to place a loudspeaker each 5, just for improving interpolation. For the moment, the current system is composed by 89 loudspeakers. We expect that just adding a few more intermediate elevation loudspeakers in an improved prototype, and employing the interpolation techniques described above, it is possible to accomplish an accurate system for fast HRTF measurement with less than 100 loudspeakers. 3. Spherical microphone arrays and Plane-Wave Decomposition To compensate the echoes inside the room in the HRTF measurement, it is necessary to separate the reflections from the direct sound at the listener s position for each of the loudspeakers. In this project, the sound field is analyzed using spherical microphone arrays placed at the position where the listener would be, which allows us to register the echoes for further processing

4 Euronoise Conference Proceedings Figure 2. Deployment of the proposed system inside the room Microphone arrays In the sound field analysis process, the sound field is captured by using spatially distributed microphone arrays, either on a real or imaginary surface. As stated in [13], several systems have been designed during last years with different configurations, like the scanning/sequential array designed at the Cologne University of Applied Sciences (VariSphear), which allows to capture many sampling positions as desired with a single microphone. Moreover, other configurations like spherical arrangements of microphones over open or rigid physical spheres were examined, providing a limited set of measurements (equal to the number of sensors distributed along the body) but saving time in the process (simultaneous capturing) and taking profit of the rotational symmetry, suitable for measuring 3D sound fields. individual electret microphone capsules inserted in a rigid sphere of 4.2 cm radius, providing better resolution in results at higher frequencies (up to a certain frequency, known as limit frequency of operation). A second array with bigger size (20 cm of radius) and in open sphere configuration was built and customized (Figure 3, left part) and is being rehearsed for this study. This system is custom-made, employing 3Dprinted pieces for the vertices or joints between edges and working as support for the different microphones, having the same sensor density as the em32 Eigenmike but assuming it will provide better resolution at lower frequencies. Both arrays have a microphone disposition which follows the shape of a pentakis dodecahedron Plane Wave Decomposition Software The selection of a microphone array comes along with the process of finding the best trade-off among the different factors or characteristics, depending on the application. For the purpose of our study, two arrays of different diameter will be used in the measurement process to compare the results in terms of frequency and resolution, as will be stated in section 5. Techniques to decompose the sound field in the different possible directions are necessary to help determining the origin of the echoes or reflections inside the room. Plane Wave Decomposition (PWD) method has been selected for studying direction-dependent echo cancellation, as it offers an extraction of the plane waves that compose the sound field by virtue of the spherical harmonics domain [15]. The em32 Eigenmike microphone array (Figure 3, right part) from mh acoustics [14] is composed by 32 A sound field analysis toolbox for MATLAB, called SOFiA [16], available as open source, has been em

5 Euronoise Conference Proceedings Figure 4. MATLAB interface for detection and management of echoes inside the room with measured sound field by spherical arrays. Figure 3. Spherical microphone arrays. On the left, the custom-made 20-cm microphone array in open sphere; on the right, the em32 Eigenmike microphone array. ployed in the project. The toolbox is structured in different modules which constitute each a step in the PWD process. Varying the parameters involved in the method, it is possible to adjust it for each microphone array and obtain information with the impinging direction of sound in azimuth and elevation, as well as the RIR in each possible direction with a resolution of 1 in each plane, giving possible plane waves. To take the maximum profit of this tool, a GUI interface for MATLAB has been developed. The program, as it can be observed in Figure 4, allows to study the echoes through the processing of the sound field captured by the microphone arrays. It incorporates the option of selecting and loading the measures dataset corresponding to different scenarios and then choose the loudspeaker in the direction being studied. In that way, the corresponding group of Room Impulse Responses is loaded and plotted. In the graph, two cursors are available to select the portion of RIR to process and obtain the direction of arrival of that sound by using SOFiA toolbox. Moreover, two modes of processing are available: for a specified single frequency or integration for the whole frequency range. A color globe sphere is represented with the intensity and reflecting the distribution of the arrival of reflections to the array. 4. Methodology and case studies The underlying idea in this project is to obtain HRTF measures free from echoes inside the non-anechoic room, mainly those with origin in the floor or walls. By cropping the first part of the impulse response before the first reflection, it is possible to keep only the direct path. However, the obtained windowed impulse response is usually too short to provide enough resolution at low frequencies Sub-band processing Since the system it is being exposed is expected to work properly for the whole frequency range and due to the resolution problem at low frequencies by using the cropping method, in this paper a sub-band technique has been chosen for the HRTF computation, where the high part of the spectrum is calculated by cropping, whereas the low part is subjected to array processing in order to suppress the echoes. The impulse responses are measured using the SineSweep technique, in which an exponential timegrowing frequency signal is reproduced and, at the same time, is being deconvolved in order to obtain the linear impulse response of the room [17]. In Figure 5, a typical result for an impulse response in the measurement process can be observed. Since the direct sound arrives, there exists a moment in time where an echo appears, which corresponds with a specific sample of the recorded data. Following Equation (1), it is possible to obtain the number of samples between direct sound and the arrival of the first echo, as well as the corresponding cut-off frequency up to which the array processing is needed to be employed. fcut of f = 1 Hz, tdelay,1st s echo (1) where, tdelay,1st is the delay of the first echo. echo Note that, when applying some kind of soft window to cut the RIR and to have a safety margin, a frequency between the one calculated in Equation (1) and twice that value would be the most suitable option. Figure 6 describes the process of sub-band processing proposed in this paper. From the HRTF measured, in the upper branch the HRTF is processed for low frequencies. Then, a low-pass filter is applied and the echo cancellation is carried out based on the room

6 Euronoise Conference Proceedings Figure 5. Room Impulse Response example and echoes detected. Figure 7. Set-up scheme for case study I. Translating that difference in Equation (3) in time interval by dividing by sound velocity: Figure 6. Sub-band processing diagram flow. acoustics information that the microphone array provides and PWD techniques. In the lower branch, the signal is processed for high frequencies and, using the cropping method, it is windowed before first reflection. Both treated signals are summed, giving as a result an HRTF that should be equal to other measured in anechoic conditions Case I: Floor reflection In order to describe the sub-band processing, the operation is illustrated through an example. The set-up is reflected in Figure 7. In the position where the listener would be placed (center of the circular array), a microphone array is set, and the Room Impulse Responses (RIR) are captured by the 32 sensors when loudspeaker emits. It is also reflected the path that follows the direct sound to the array (noted as H 1, which corresponds to a distance of 2 meters) and the reflection on the floor (named H 2 ). The distance in meters the wave traverses until it reaches the floor (first part of the reflection in H 2, segment AB) is: (2.005 ) 2 AB = + (1.44) 2 2 = m. (2) The same distance computed in Equation (2) is equal to the path the wave travels from point B (reflection on the floor) until point C (arrival to the array). Then, multiplying by 2 and subtracting the distance of H 1, the difference in space between direct sound and reflection is obtained: diff H1,H 2 = ( ) 2 = m. (3) ( ) m t H1,H 2 = 343 m = s. (4) s Then, the time interval obtained in Equation (4) is equal to the term t delay,1 st in Equation (1). So, applying that and expressing it in number of samples by echo multiplying by a sampling frequency, f s, of 48 khz: f cut off = Hz. (5) s N H1,H 2 = s f s = 211 samples. (6) After the results obtained in Equations (5) and (6), a frequency of 227 Hz is set as the minimum limit to necessarily apply array processing techniques, being the interval of 211 samples the distance in samples between the arrival of direct sound and the first reflection from the floor. Continuing with the method, a study of the sound field in the listener position is carried out. With the microphone array in the listener position, it can be considered that the acoustic channel between the loudspeaker and the listener position, H, is a mixture of the different contributions from all directions. Considering there is only one reflection, H is composed by the direct sound path (H 1 ) and reflection from the floor (H 2 ) at 45. H(φ, θ 1, f) = H 1 (φ, θ 1, f) + H 2 (φ, θ 2, f), (7) where φ is the azimuth angle and θ the one in elevation, being the subscripts 1 and 2 indicators of the direction of direct sound and floor reflection, respectively, in this last plane

7 Euronoise Conference Proceedings Due to the use of the spherical microphone arrays, as it has been explained in section 3, it is possible to separate H 1 and H 2 together with the application of PWD techniques. Then, if a dummy head is placed instead, the measured HRTF would be unprocessed and mixed with the two aforementioned signals. It could be expressed and decomposed as follows: HRT F (φ, θ 1, f) = H 1 (φ, θ 1, f) HRT F (φ, θ 1, f) + H 2 (φ, θ 2, f) HRT F (φ, θ 2, f). (8) The term HRT F (φ, θ 1, f) in Equation (8) is the interest object here. Solving and rearranging: HRT F (φ, θ 1, f) = HRT F (φ, θ 1, f) H 1 (φ, θ 1, f) H 2(φ, θ 2, f) HRT F (φ, θ 2, f).(9) H 1 (φ, θ 1, f) As it can be observed from Equation (9), in order to obtain HRT F (φ, θ 1, f) it is necessary to get HRT F (φ, θ 2, f). Nevertheless, it can be obtained in an easier manner, since there will not be floor reflection in that term from downwards to upwards or, in the case it contains any, it could be eliminated with other techniques. On the other hand, either H 1 or H 2, apart from having the acoustic channel, they also contain the frequency response of the loudspeaker, H loud (f), which can be compensated as well, since a measure of the loudspeaker in free-field conditions is available. Figure 8. Set-up scheme for case study II. 5. Results and discussion Results for the previous case studies are presented here, comparing them with the use of measurements of the two microphone arrays employed. The echo detection stage will be the only result illustrated here (for case I), since the composition and cancellation of them will be presented in future works. The RIRs are processed for a window which includes the direct sound and the first reflection, as observed in Figure 9. In Figure 10 and 11 are represented the reconstructed IRs after PWD processing with the MAT- LAB interface, by employing measures from Eigenmike and the custom-made 20-cm array, respectively Case II: Wall reflection In the previous case, H 1 only contains the loudspeaker frequency response, H loud (f). However, in this second case, as reflected in Figure 8, there also exists a reflection from the wall behind the loudspeaker, which arrives in the same direction as the direct sound, named as H 3, and goes from point A (loudspeaker), reflects in the wall and arrives to point D (microphone array). H 3 can be included inside the expression of H 1 as: H 1 (φ, θ 1, f) = H loud (f) + H 3 (φ, θ 1, f). (10) The solution will be, therefore, to correct the loudspeaker response as before, together with the cancellation of the wall reflection by inverting the signal. Moreover, the procedure to cancel the floor reflection is also needed, as it appears in the process. Figure 9. Interface results for case study I. The direction of the loudspeaker corresponds, in the coordinate system of the toolbox, with 0 AZ and 90 EL, and the results of the PWD method prove the same direction. From Figures 10 and 11 it can be seen the separation of the direct sound from the floor reflection, with the corresponding time interval in between. Results have been processed until a frequency of 300 Hz (closer to the cut-off frequency), providing a better resolution in result with Eigenmike measurements

8 Euronoise Conference Proceedings 0.8 H 1 & H 2 after Plane Wave Decomposition (PWD) References H 1 (0º AZ, 90º EL) Amplitude Time (s) H 2 (0º AZ, 135º EL) Figure 10. H 1 and H 2 separation for case study I (measures from Eigenmike array). Amplitude H 1 & H 2 after Plane Wave Decomposition (PWD) H 1 (0º AZ, 90º EL) H 2 (0º AZ, 135º EL) Time (s) Figure 11. H 1 and H 2 separation for case study I (measures from custom-made 20-cm array). 6. Conclusions The deployment of a complete system based on loudspeaker arrays in different elevations for measuring HRTF in non-anechoic rooms has been presented. In addition to this, an echo cancellation system has been proposed in order to detect and suppress reflections of the room. To this effect, Plane Wave Decomposition techniques have been used for detection, together with the developed sub-band processing method, which provides a full-band system. The results have proved the possibility of a clear detection of the echoes with the aforementioned techniques, which will allow to apply the cancellation of all of them with the described methods. The final aim of this project is to present this system as an advantageous alternative to traditional anechoic environments. As future work, all the methodology illustrated along the sections must be completely verified, specially the part of the echo cancellation and, then, check if the result would be a HRTF comparable to one measured in anechoic conditions. Moreover, intermediate loudspeakers and the use of interpolation techniques will be subject of future papers. Acknowledgement [1] Global Market Insights Inc., Earphones And Headphones Market Size By Technology (Wired, Wireless), By Application (Consumer, Call Center, Industrial, Aviation, Construction, Public Safety), Industry Analysis Report, Regional Outlook, USA, [2] H. Møller, M. F. Sorensen, D. Hammershoi, and C. B. Jensen. Head-Related Transfer-Functions of Human- Subjects, J. Audio Eng. Soc., vol. 43, no. 5, pp , [3] R. Nicol. Binaural Technology. New York: Audio Engineering Society Inc., 2010, ISBN [4] K. Sunder, J. He, E. Tan, W. S. Gan, and E. Tan. Natural Sound Rendering for Headphones, IEEE Signal Process. Mag., March, pp. 1-37, 2015, doi: /msp [5] J. Blauert. Spatial Hearing: The psychophysics of human sound localization, MIT Press Cambridge, 2nd edition, [6] J. C. Makous and J. C. Middlebrooks. Two- Dimensional Sound Localization by Human Listeners, J. Acoust. Soc. Am., vol. 87, pp , [7] J.G. Richter and J. Fels. Evaluation of Localization Accuracy of Static Sources Using HRTFs from a Fast Measurement System, Acta Acustica united with Acustica, vol. 102(4), pp , [8] P. Majdak, P. Balazs, and B. Laback. Multiple exponential sweep method for fast measurement of headrelated transfer functions, Journal Audio Eng. Soc, vol. 55(7/8), pp , [9] P. Dietrich, B. Masiero, and M. Vorlander. On the optimization of the multiple exponential sweep method, J. Audio Eng. Soc, vol. 61(3), pp , [10] X. Zhong: Interpolation of Head-related Transfer Functions Using Neural Network. Fifth International Conference on Intelligent Human-Machine Systems and Cybernetics, [11] M. Xu, Z. Wang and Y. Gao: Interpolation of Minimum-Phase HRIRs Using RBF Artificial Neural Network. Fuzzy Systems and Data Mining III, [12] J.J. Lopez, P. Gutierrez-Parera: Equipment for fast measurement of Head-Related Transfer Functions. Audio Engineering Society, Berlin, May [13] B. Bernschutz, P. Stade, M. Ruhl: Sound Field Analysis in Room Acoustics. 27th Tonmeistertagung - VDT International Convention, [14] mh acoustics: em32 Eigenmike microphone array release notes (v18.0). USA, June 2014 [15] V. Pulkki, S. Delikaris-Manias, A. Politis: Parametric Time-Frequency Domain Spatial Audio. Wiley, pp , Finland, [16] B. Bernschutz, C. Porschmann, S. Spors, S. Weinzierl: SOFiA Sound Field Analysis Toolbox, In: Proceedings of the International Conference on Spatial Audio. ICSA 2011, Detmold, Germany. [17] S. Guy-Bart, E. Jean-Jacques, A. Dominique: Comparison of different impulse responses measurement techniques. University of Liege, Belgium, The Spanish Ministry of Economy, Industry and Competitiveness supported this work under the project TEC R