HEAD acoustics Factors impacting the speech quality in Vo scenarios and how to assess them Dr.-Ing. H.W. Gierlich HEAD acoustics GmbH Ebertstraße 30a D-52134 Herzogenrath, Germany Tel: +49 2407/577 0! Fax: +49 2407/577 99 http://www.head-acoustics.de www.head-acoustics.de 12-Feb-03 #1 Overview " Speech Quality in Vo " Parameters influencing speech quality measurement procedures " Single talk - listening " Single talk talking " Double talk " Background noise " Summary www.head-acoustics.de #2 1
Speech Quality Parameters from the user s perspective talking situation sidetone echo delay speech quality listening situation sound quality naturalness intelligibility conversational situation delay echo (during double talk) duplex capability www.head-acoustics.de #3 Typical Signal Processing... in terminals and gateways noise reduction EC level switching VAD coder microphone NR NLP VAD - Coder Buffer, Packetization H(f ) packet loss, delay, delay jitter + Decoder Buffer, PLC loudspeaker EC CN comfort noise decoder jitter buffers, PLC www.head-acoustics.de #4 2
Typical Scenarios PABX E1/T1 Gateway Gateway E1/T1 PABX - " connection between two phones " all kinds of signal processing in the terminal www.head-acoustics.de #5 Typical -Scenarios PABX E1/T1 Gateway Gateway E1/T1 PABX PSTN - - " connection between the PSTN and one phone " all kinds of signal processing in gateway and terminal www.head-acoustics.de #6 3
Typical -Scenarios PABX E1/T1 Gateway Gateway E1/T1 PABX PSTN PSTN over PSTN - - " connection between two PSTN subscribers over " all kinds of signal processing in gateways www.head-acoustics.de #7 Overview " Speech Quality in Vo " Parameters influencing speech quality measurement procedures " Single talk - listening " Single talk talking " Double talk " Background noise " Summary www.head-acoustics.de #8 4
Speech Quality Parameters Listening Situation terminal A terminal B Sound quality, intelligibility, naturalness: " voice activity detection " speech coders " packet loss " packet loss concealment " noise reduction "... www.head-acoustics.de #9 Testing Techniques Listening Situation terminal A terminal B feeding recording and analysis " Analysis methods: traditional analysis & perceptual speech quality models www.head-acoustics.de #10 5
Testing Techniques The traditional numbers - to be determined under realistic use conditions " loudness ratings (SLR, RLR) " frequency responses " listener sidetone (LSTR) " listener echo www.head-acoustics.de #11 Testing Techniques Instrumental Measures based on Hearing Models: Modeling the Results of Auditory Tests by Comparison of Reference Speech Signal with Processed Speech Signal Typical Processing Steps (Schematic): Results of Listening Tests! Speech Signal Test Object Adaptation processed signal reference signal Hearing Model Hearing Model Comparison, Reference, Reduction, Signal Value S index Q PSQM P.861 PESQ P.862 PSQM99 KPN PAMS BT TOSQA T-Systems PACE Ascom VQI Ericsson The ITU standard for electrical access www.head-acoustics.de #12 6
Typical Results 2 nd ETSI Vo 5.0 PESQ Electrical - Electrical: PESQ reference G.711 4.0 reference G.729 3.0 2.0 1.0 0ms / 0% / 0ms 0ms / 1% / 0ms 0ms / 2% / 0ms 0ms / 3% / 0ms 0ms / 5% / 0ms 50ms / 1% / 20ms 0ms / 0% / 0ms 0ms / 1% / 0ms 0ms / 2% / 0ms 0ms / 3% / 0ms 0ms / 5% / 0ms 50ms / 1% / 20ms PABX G.711 G.729 G.723@6.3 G.723@5.3 references / VAD off G.711 / 10&20ms / VAD on/off / PLC on G.729 / 10&20ms / VAD on/off / PLC on www.head-acoustics.de #13 Test Signals and Analysis Methods Analysis of packet loss and PLC implementation (example) Occurrence of packet loss (20 ms packet length) Packet loss concealment (typical implementation), signal discontinuities www.head-acoustics.de #14 7
Relative Approach " Approach: forward estimation based on signal history, comparison with actual signal value Hearing model Extrapolation in the time domain Interpolation between critical bands Display of estimation error = audible degradation # Relative Approach " Relative Approach takes into account the sensitivity of the human ear on instantaneous signal variation in time on dominant spectral structures Relative Approach needs no reference signal www.head-acoustics.de #15 Relative Approach Q = f ( N, S ) + f ( 24 F ( i 1) F F (, i n+ ) w (, i F ())) i G 1 2 ( i) w ( if ( i)) + G G 1 G i= 1 n= 1 G ] T F ( i, n) G Basic principle of the Relative Approach: Comparison between short term and long term averaging of signal energies in critical bands (app. 2 s vs. 2 ms) based on a hearing model [Sottek] www.head-acoustics.de #16 8
Packet Loss and Concealment The Reasons Behind the Scores...? Transmitted time signal (5s) ITUITU-T Recommendation P.501 periodical repetition of a voiced sound pitch frequency approx. approx. 330 Hz Relative Approach hearing model based forward estimation comparison to actual signal displays estimation error error Cross correlation analysis vs. time analysis of signal phase periodical pattern (approx. approx. 3 ms) www.head-acoustics.de #17 Packet Loss and Concealment 5.0 Electrical - Electrical: PESQ ref erenc e G.711 PESQ 4.0 reference G.729 3.0 Electrical - Electrical: T MOS 5.0 TMOS G.723@5.3 G.723@6.3 PABX G.729 0ms / 5% / 0ms 0ms / 3% / 0ms 50ms / 1% / 20ms 2.0 G.711 / 10&20ms / VAD on/off / PLC on 0ms / 2% / 0ms 3.0 0ms / 1% / 0ms 0ms / 0% / 0ms 0ms / 5% / 0ms 0ms / 3% / 0ms referenc e G.711 4.0 50ms / 1% / 20ms 0ms / 2% / 0ms 0ms / 1% / 0ms 0ms / 0% / 0ms 1.0 G.711 2.0 reference G.729 referenc es / VAD off G.729 / 10&20ms / VAD on/off / PLC on G.711 / 10&20ms / VAD on/off / PLC on www.head-acoustics.de G.723@5.3 G.723@6.3 G.729 G.711 PABX 50ms / 1% / 20ms 0ms / 5% / 0ms 0ms / 3% / 0ms 0ms / 2% / 0ms 0ms / 1% / 0ms 0ms / 0% / 0ms 0ms / 5% / 0ms 50ms / 1% / 20ms 0ms / 3% / 0ms 0ms / 2% / 0ms 0ms / 1% / 0ms 0ms / 0% / 0ms 1.0 references / VAD off G.729 / 10&20ms / VAD on/off / PLC on #18 9
Testing Techniques Background noise transmission - Influence of VAD Example: Gateway - sim.- Gateway " G.711 codecs " no packet loss, no jitter " no additional delay red: original test signal " noise sequence " Hoth spectrum (P.800) " increasing level vs. time yellow, magenta, cyan: different implementations adaptive comfort noise VAD sensitivity threshold, no comfort noise www.head-acoustics.de #19 Overview " Speech Quality in Vo " Parameters influencing speech quality measurement procedures " Single talk - listening " Single talk talking " Double talk " Background noise " Summary www.head-acoustics.de #20 10
Speech Quality Parameters Talking Situation...echo? terminal A terminal B Delay and echo also with background noise " round trip delay " echo level and echo characteristic " implementation of speech echo cancellers " quality of background noise transmission www.head-acoustics.de #21 Testing Techniques...echo? Talking Situation terminal A terminal B feeding, recording and analysis " Analysis methods: echo measurements based on Composite Source Signals (CSS) under single and double talk conditions www.head-acoustics.de #22 11
Testing Techniques Echo under single talk conditions - ITU-T G.131 Requirement on echo loss depends on transmission delay SLR JLR JLR RLR 7dB 0dB 0dB 3dB TELR a(echo) a(echo) TELR RLR JLR JLR SLR 3dB 0dB 0dB 7dB OLR = 10dB www.head-acoustics.de #23 Echo Tests according to G.168 " Convergence and steady state residual and returned echo level tests " Convergence test in the presence of background noise " Leak rate test " Infinite return loss convergence test " Non-divergence on narrow-band signals www.head-acoustics.de #24 12
Overview " Speech Quality in Vo " Parameters influencing speech quality measurement procedures " Single talk - listening " Single talk talking " Double talk " Background noise " Summary www.head-acoustics.de #25 Speech Quality Parameters Conversational Situation terminal A terminal B " propagation delay - conversation dynamics impairments " double talk detection (EC implementation) " echo during double talk " level variations during double talk www.head-acoustics.de #26 13
Testing Techniques Conversational Situation terminal A feeding feeding, recording and analysis terminal B " Analysis methods: double talk measurements based on two uncorrelated Composite Source Signals www.head-acoustics.de #27 Testing Techniques Requirements on echo and switching during double talk: MOS 4.0 4.0-3.5 3.5-3.0 3.0-2.5 2.5-2.0 2.0 TELR DT [db] 37 33 27 21 13 <13 a Hsdt [db] 3 6 9 12 15 >15 a Hrdt [db] 3 5 8 10 12 >12 TELR DT : talker echo loudness rating during double talk a Hsdt : attenuation range sending during double talk a Hrdt : attenuation range receiving during double talk www.head-acoustics.de #28 14
Testing Techniques Double talk evaluation: Test result 1 Note strong echo components which occurred only during double talk but not under single talk conditions strong echo green: measured signal red: original test signal yellow: overlap www.head-acoustics.de #29 Testing Techniques Double talk evaluation: Test result 2 Note clipping at beginning and end of most bursts and comfort noise during pauses clipping green: measured signal red: original test signal yellow: overlap comfort noise www.head-acoustics.de #30 15
Testing Techniques Analysis of echo during double talk terminal A feeding terminal B feeding, recording and analysis " Analysis method: Separation of echo and near end signal by appropriate filtering www.head-acoustics.de #31 Testing Techniques Analysis of echo during double talk Original test signal in receiving direction Near end signal Echo components Example: Measured power density spectra between 400 Hz and 1.2 khz www.head-acoustics.de #32 16
Background Noise Double Talk Double talk signal Gateway NISTNet Gateway ISDN ISDN Simulator Simulator PBX PBX Packet Loss Delay PBX PBX ISDN ISDN Simulator Simulator Background noise In Out In Out RCV SND SND RCV MFE VI USB ERL 6 db, ERL 40 db and infinite Testsystem ACQUA www.head-acoustics.de #33 The work in ETSI STQ WI 011 Mouth to Ear Speech Transmission Quality Including Terminals Advanced Measurement measurement of standard procedures, parameters taking into account the conversational situation frequency responses (SLR, RLR, STMR, LSTR) Loudness Quality of Ratings background noise transmission D-Factor Double talk performance Delay Switching characteristics Terminal Level Adjustments coupling loss by Companding or AGC Distortion Additional & Echo out of disturbances band signals Speech Sound Quality Loudness and Noise www.head-acoustics.de #34 17
Automated Testing with ACQUA www.head-acoustics.de #35 ACQUA with MFE VI MFE VI: USB-frontend for acoustical and electrical access including echo path simulation www.head-acoustics.de #36 18
Acoustical Access www.head-acoustics.de #37 Summary " Listening situation " Overview speech quality tests => P.862 PESQ or TOSQA with acoustical components " Detailed investigations => Specific test signals and analysis procedures e.g. Relative Approach " Talking situation " G. 168 Tests " Additional background noise tests " Double talk/conversational tests " Delay tests " Double talk echo tests using specific test signals and analysis procedurs " Switching tests using specific test signals and analysis procedurs " Background noise tests during double talk " To do: " Tests methods for noise reduction, single number for speech quality,.. www.head-acoustics.de #38 19