The Human Auditory System

Transcription

1 medial geniculate nucleus primary auditory cortex inferior colliculus cochlea superior olivary complex The Human Auditory System Prominent Features of Binaural Hearing Localization Formation of positions of the auditory events (i.e. azimuth, elevation & distance) Spatial extent of auditory events Suppression of Directional information coming from reflections (e.g., Precedence effect, summing localization) Reverberation, coloration and noise Identification & Segregation of Auditory streams (e.g., concurrent musical instruments, cocktail-party effect,warning signals) 1

2 Binaural Models and their Technological Application Jens Blauert, D-Bochum Application Area # 1 as Identified by AABBA Spatial Scanning and Mapping of Auditory Scenes Estimation of the position and the spatial extents of the of auditory events that form an auditory scene be it a natural scene as in room acoustics WW or virtual scene as in virtual-reality applications, or at the play-back end of audio systems including spatially diffuse auditory events, often perceived as components of reverberance The AABBA grouping, 2009 What is AABB ABBA? An international grouping of 14 laboratories (since 2009), dealing with Aural Assessment By Means of Binaural Algorithms 2

3 Application Area # 2 as Identified by AABBA Analysis of Auditory Scenes with the Aim of Deriving Parametric Representations at the Signal Level Estimates of these parameters may be used, for instance, For coding and/or re-synthesis of auditory scenes For speech-enhancement in complex acoustic environments, incl. hearing aids For systems to enhance the spatial perception in sound fields, such as better localization and/or a better sense of envelopment; further, decolouration and de-reverberation For the identification of perceptual invariances of auditory scenes The AABBA grouping, 2009 Application Area # 3 as Identified by AABBA Analysis of Auditory Scenes with the Aim of Deriving Parametric Representations at the Symbolic Level For example, Identification of determinants of meaning contained in binaural-activity maps Assignment of meaningful symbols to the output of binaural models The AABBA grouping,

4 Application Area # 4 as Identified by AABBA Evaluation of Auditory Scenes in Terms of Quality Where quality will strictly be judged from the user s point of view, for example, Quality of the acoustics of spaces for musical performances Quality of systems for holophonic representation of auditory scenes, such as auditory displays and virtual-reality generators Spatial quality of audio-systems (for recording, transmission and play-back), incl. systems that employ perceptual coding Quality of speech-enhancement systems, incl. hearing aids The AABBA grouping, 2009 Binaural Models and their Technological Application Introduction Prominent Features of Binaural Hearing Generic Application Areas of Binaural Models Model Architectures The Periphery & Bottom-up Processing ILDs, ITDs, interaural coherence Binaural-activity Mapping & Interpretation Cocktail-party processing, sound-quality assessment Higher-order Processes Precedence effect, auditory-scene analysis Franssen effect Interactive Listening Architecture for a comprehensive model of binaural hearing Conclusion and Outlook 4

5 The Periphery & Bottom-up Processing ILDs, ITDs, interaural coherence Binaural-activity Mapping & Interpretation Cocktail-party processing, quality assessment Higher-order Processes Precedence effect, auditory-scene analysis Franssen effect Interactive Listening Architecture for a comprehensive model of binaural hearing binaural-activity map interaural-level-difference analysis 2006 interaural-time-difference analysis pure bottom-up processing, signal driven! Block Diagram of the Binaural-Analysis System of IKA Bochum 5

6 Ear-Adequate Band-Pass-Filter Bank spectral decomposition envelope extraction compressive A/D conversion compressive! Probabilistic Spike Generator A Simplified Functional Model of the Hair Cells 6

7 The Jeffress Processor Estimates Interaural Cross Correlation Ψ y (τ) = 1/(t 1 - t 0 ) t 1 Σt=t 0 y l (t) y r (t +τ) similarity to Jeffress coincidence model: τ k τ k+1 τ k+2 τ k+3 τ k+4 τ k+5 τ k+6 τ k+7 τ k+8 Braasch 2002 contralaterally-inhibited IACC IACC Contral-Lateral Inhibition after Lindemann 7

8 Cross-Correlation vs. Lindemann impulsive input signals with no ITD, but with an ILD, looked at in an auditory critical band about 800 Hz schematic plot Gaik 1988 Natural Combinations of ITDs and ILDs 8

9 Gaik s Extension: How to learn the ears of its owner? Weighted Contralateral Inhibition Gaik 1988 center frequency of critical band left < lateral deviation > right Output of the Jeffress-Lindemann-Gaik Model frontal broad-band sound source 9

10 A Figurative Plot of the Output of the Gaik Processor presented angle estimated angle natural hearing Blumlein stereophony loudspeakers 60 apart loudspeakers 45 apart Binaural-Model Estimation of ITDs for Natural Hearing and Amplitude-Difference Stereophony (Blumlein) Blauert & Braasch

11 The Periphery & Bottom-up Processing (ILDs, ITDs, interaural coherence) Binaural-activity Mapping & Interpretation Cocktail-party processing, quality assessment Higher-order Processes Precedence effect, auditory-scene analysis Franssen effect Interactive Listening An architecture for a comprehensive model of binaural hearing recognition, interpretation and judgement binaural activity mapping Schematic Plot of a Typical Binaural Model 11

12 2 concurrent talkers) dry talker reverberant talker de-reverbed talker Examples for Binaural Activity Maps of Speech direction finder weight assigment Binaural Model Control Unit source selection Wiener Filter spectral decomposition speech cleansing Architecture of a Cocktail-Party Processor, Based on Binaural Modelling 12

13 one noise source two concurrent talkers A human listener would not localize the reflection! two moving talkers talker plus wall reflection Source Tracking with the Binaural Model Bodden 1993 Spatial Selectivity of a Model of Binaural Hearing Bodden

14 taken at an instant t = t o from a running correlogram Sample Output of the Binaural Model one frontal sound source, sending out a musical chord Impulse Response and Running Interaural Cross-correlation in a Room with Reflecting Walls 14

15 prediction: left < center > right Model Output with Incoherent Ear-Input Signals parameter: degree of interaural coherence, k pink noise, 12 subjects after Blauert & Lindemann, 1985 (plot enhanced for contrast) Area Covered by Auditory Events as a Function of the Amount of Interaural Coherence, k 15

16 critical band about 700 Hz reflected sounds direct sound left lateral position right Binaural-Activity Map of the Impulse Response of a Concert Hall rendered by the model of binaural signal processing of Lindemann & Gaik 1986,1990 plotted by Okabe 1997 critical band at 500 Hz 2 noises from Φ = 30 & snapshots across bands 90% correlated Binaural Model Fast PC Version Hess

17 Application to Sound Quality The Layer Model Higher Aural-communication Quality sound of quality communication sciences ideas, concepts, functionalities, content, sound as a sign carrier Abstraction Acoustic Quality quality of realization physics physical attributes & properties, absence of acoustic distortions, physical form Aural-scene Quality quality of presentation perceptual psychology aural gestalt, authenticity, aural perspective, enhancement, sensory consistency, perceptual form Lower Auditive Quality quality of sound classical psychoacoustics auditive attributes & properties Blauert & Jekosch 2007, 2012 brain of the system top-down processing The brain of the system contains explicit knowledge: e.g., data bases, rule system, semantic networks, transition probabilities, domain models, the history of the situation object building Gestalt rules Blauert 1999, 2005 binaural-activity mapping Intelligent Evaluation of Binaural-Activity Maps 17

18 The Periphery & Bottom-up Processing ILDs, ITDs, interaural coherence Binaural-activity Mapping & Interpretation Cocktail-party processing, quality assessment Higher-order Processes Precedence effect, auditory-scene analysis, Franssen effect Interactive Listening An architecture for a comprehensive model of binaural hearing lead summing localization region localization dominance region region with two auditory events lag inter-stimulus interval, ISI Blauert: Spatial Hearing, 1983 The Precedence Effect Do not mix up with the Haas Effect! 18

19 Output of the Binaural Model for a Frontal Sound Plus One Lateral Reflection left panel: cross correlation only right panel: cross correlation plus contralateral inhibition Short Clicks! Psycho-Acoustic Results no localization dominance band-pass-noise bursts 200-ms duration, 500-Hz center frequency Ongoing Sounds! 19

20 lag about 0.7 khz 100-ms noise ISI...10 ms estimate of the interaural cross-correlation function Output of the Jeffress Coincidence Processor Lag Frontally-Fixed, Lead Moving About Braasch 2005 lag about 0.7 khz 100-ms noise ISI...10 ms estimate of the inhibited interaural cross-correlation function Output of Lindemann s Processor with Lateral Inhibition Lag Frontally-Fixed, Lead Moving About Braasch

21 broad band 100-ms noise ISI ms estimate of the interaural cross-correlation function Output of the Jeffress Coincidence Processor Lag Frontally Fixed, Lead Moving About Braasch 2005 broad band 100-ms noise ISI = 10 ms estimate of the inhibited interaural cross-correlation function Output of Lindemann s Coincidence Processor with Lateral Inhibition Braasch

22 Modified Lindemann Model inhibited IACC x type I + type II level diff Dizon s experimental paradigm left-ear signal right-ear signal observation window Dizon 2001, Dizon & Colburn

23 Time Course unusual approach using a monaural lead/lag pair lead stronger lag weaker monaural! Autocorrelation Evaluation of Filtered Signal lead only lead & lag lead, after lag removal error monaural! 23

24 current model: matched filter prior models: simple substraction Echo Canceller for a Simple Lead-Lag Pair Braasch 2008 Cross-correlation Analysis for Binaural Signals lead only lead & lag recovered error binaural! 24

25 model data lead position lag position psychoacoustic data very narrow band Model Data with Echo Canceller adaptive filters take time! Binaural-Model Architecture with Monaural Echo Cancellers 25

26 Fast & Slow Melodies Trumpet Fo time Echo Thresholds Fast & Slow Melodies ET in ms F FM S SM 0 F fast, L1 L2 L3 L4 FM fast in mixture, S slow, SM slow in mixture 26

27 Wolf s Experiment II alternative plot The Clifton Effect Clifton

28 Sound Sources Alternating in Space Blauert & Col 1989 Virtual Test Room 28

29 Test-Room Switching The Franssen Effect Franssen

30 The Periphery & Bottom-up Processing ILDs, ITDs, interaural coherence Binaural-activity Mapping & Interpretation Quality assessment, cocktail-party processing Higher-order Processes Precedence effect, Haas effect, auditory-scene analysis Franssen effect Interactive Listening An architecture for a comprehensive model of binaural hearing Feed-back Paths to be Considered Feedback from the binaural-mapping stage, that is, the output of auditory signal processing, to head-position control for instance, the so-called turn-to reflex Feedback from the cognitive stage to head-position control for exploratory head movements Feedback from the segmentation stage to the signal-processing stage to solve ambiguities by activating additional pre-processing routines, for example, cocktail-party effect and/or precedence effect processing Feedback from the cognitive stage to the signal-processing stage, to model efferent/reafferent effects of attention, such as by modifying filter characteristics and/or concentrating on dominant spectral regions Feedback from the cognitive stage to the segmentation stage, for instance, to request task-specific and/or action-specific information on particular features 30

31 Blauert 1999, 2011 Blauert & Obermaier 2012 symbols signals An architecture for a Model of Interactive Binaural Listening indicating possible feedback loops, cross-modal-date input and various expert modules NB.: This is the architecture as currently discussed in AABBA Potential Applications for Binaural Algorithms looking back in time Blauert 1988! 31

32 AABBA projects as of 2012 [1] Berlin Project (Raake): Application of binaural models QUALITY to multi-channel-reproduction assessment [2] Bochum Project (Kolossa): Understanding of aural SCENE ANALYSIS scenes with a multi-feedback system and graphical MEANING models [3] Boston Project (Colburn): A binaural pre-processor ENHANCEMENT for speech recognition in complex environments [4] Cardiff/Lyon Project (Culling): Predicting binaural ENHANCEMENT intelligibility for architectural acoustics [5] Copenhagen Project (Dau): Binaural analysis of SCENE ANALYSIS complex listening environments [6] Dresden Project (Jekosch/Altinsoy): Assessment of MULTIMODAL human machine interfaces employing binaural/tactile QUALITY interaction AABBA [7] Eindhoven Project (Kohlrausch): Binaural analysis of ENHANCEMENT acoustic scenes [8] Helsinki Project (Pulkki): Assessment of parametric QUALITY spatial-audio coding with of binaural models [9] Munich Project (Hemmert): Evaluation of binaural cues QUALITY in electric hearing" [10] Oldenburg Project (Hohmann): Analysis of aural scenes with SCENE ANALYSIS multi-dimensional statistical filters [11] Paris/Toulouse Project (Gas): Binaural audition in robotics ROBOT AUDITION [12] Patras Project (Mourjopoulos): Assessment and reduction QUALITY of reverberance and colouration using binaural models ENHANCEMENT [13] Troy NY Project (Braasch): Binaural analysis of SCENE ANALYSIS telematic music Further: A MATLAB Toolbox for Binaural Modeling will be provided AABBA 32

33 What does the sound mean to me? recognition interpretation FUNCTION FORM Where and how is the sound? detection perception Audition and Cognition Come in Couples Thank you! Questions? 33

34 Copyright note: This material is not in the public domain. The author(s) claim(s) all applicable rights. However, permission to copy it is granted under the condition that proper reference is given to the author(s). Corresponding author: Jens Blauert, Emeritus Professor of Acoustics Institute of Communication Acoustics Ruhr-Universitaet Bochum D Bochum, Germany Tel.: (direct: 3480) Fax: