Ivan Tashev Microsoft Research

Transcription

1

2 Hannes Gamper Microsoft Research David Johnston Microsoft Research Ivan Tashev Microsoft Research Mark R. P. Thomas Dolby Laboratories Jens Ahrens Chalmers University, Sweden

3

4 Augmented and virtual reality, telecommunication, navigation, tour guiding, ubiquitous computing, monitoring, entertainment, social networking,...

5 Mixed reality Virtual surround sound

6

7 Head-related transfer function (HRTF)

8 Real sound source

9 x h L,R = y L,R Real sound source Virtual sound source

10

11 HRTF measurement setup Example head-related impulse responses (HRIRs) y L,R = x h L,R

12 Input audio stream Spatial rendering engine x audio frame Dynamic cues Interaural and spectral cues Reverb y Orientation and Position tracking Head models and Measurements (HRTFs) Room models and measurements

13 Input audio stream Spatial rendering engine x audio frame Dynamic cues Interaural and spectral cues Reverb y Orientation and Position tracking Head models and Measurements Room models and measurements

14 Input audio stream Spatial rendering engine audio frame Dynamic cues Interaural and spectral cues Reverb Orientation and Position tracking Head models and Measurements Room models and measurements

15 Input audio stream Spatial rendering engine audio frame Dynamic cues Interaural and spectral cues Reverb Orientation and Position tracking Head models and Measurements Room models and measurements

16 Input audio stream Spatial rendering engine audio frame Dynamic cues Interaural and spectral cues Reverb Orientation and Position tracking Head models and Measurements Room models and measurements

17

18 [1] R. S. Woodworth and G. Schlosberg, Experimental Psychology. NY: Holt, Rinehard and Winston, [2] Duda, R. O.; Avendano, C. & Algazi, V. R. An adaptable ellipsoidal head model for the interaural time difference, Proc. IEEE ICASSP, 1999, [3] Gamper, H.; Thomas, M. R. P. & Tashev, I. J. Anthropometric parameterisation of a spherical scatterer ITD model with arbitrary ear angles Proc. IEEE WASPAA, 2015 [4] Algazi, V. R.; Duda, R. O.; Duraiswami, R.; Gumerov, N. A. & Tang, Z. Approximating the head-related transfer function using simple geometric models of the head and torso, J. Acoust. Soc. Am., ASA, 2002, 112,

19 [1] Satarzadeh, P.; Algazi, V. R. & Duda, R. O. Physical and Filter Pinna Models Based on Anthropometry, Proc. 122nd AES Conv., 2007 [2] Kan, A.; Jin, C. & van Schaik, A. A psychophysical evaluation of near-field head-related transfer functions synthesized using a distance variation function, J. Acoust. Soc. Am., ASA, 2009, 125, [3] Greff, R. & Katz, B. F. G. Round Robin Comparison of HRTF Simulation Results: Preliminary Results, in Proc. 123 rd AES Conv., 2007, New York, NY, USA.

20

21

22

23 HRTF measurement rig Measurement locations

24 Frequency (Hz) z Direction (deg) x y HRTF in horizontal plane, right ear HRTF at 1000 Hz, right ear HRTF pair for 90 azimuth

25 HRTF measurements 3-D head scans Anthropometric / demographic data

26 y t = x t h t Y f = X f H f ejρ f Head-related impulse responses (HRIRs) Head-related transfer functions (HRTFs)

27 Personalisation Rendering Listener Anthropometry? Demographics? Sensory ability? HRTFs / BRTFs Measurements? Models? Interpolation / extrapolation? Spatial sound Hardware? Context? Mixing?

28 Direct estimation Anthropometry-based personalisation Model-based personalisation

29 Gamper, H.; Thomas, M. & Tashev, I. (2015). Estimation of multipath propagation delays and interaural time differences from 3-D head scans. Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing (ICASSP).

30 P. Bilinski, J. Ahrens, M. R. P. Thomas, I. J. Tashev, J. C. Platt, HRTF magnitude synthesis via sparse representation of anthropometric features, ICASSP, 2014.

31 Average face fitted to depth image. Gamper, H.; Johnston, D. & Tashev, I. J. (2017). Interaural time delay personalisation using incomplete head scans, Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing (ICASSP).

32

33 Pinna scans.

34

35

36

37