5pSC20: EM sensor measurements of glottal. structure versus time. 1st Pan-American/Iberian Meeting on Acoustics. Cancun, Mexico. Dec.

Size: px

Start display at page:

Download "5pSC20: EM sensor measurements of glottal. structure versus time. 1st Pan-American/Iberian Meeting on Acoustics. Cancun, Mexico. Dec."

Richard Heath
5 years ago
Views:

1 5pSC20: EM sensor measurements of glottal structure versus time 1st Pan-American/Iberian Meeting on Acoustics Dec. 1-6, 2002 Cancun, Mexico John F. Holzrichter*, Lawrence C. Ng, and Gerald J. Burke Lawrence Livermore National Laboratory, Livermore, CA *Also UC Davis, Dept. of Applied Physics James B. Kobler and John J. Rosowski Massachusetts Eye and Ear Infirmary, Boston, MA Work supported by U.S. DOE, Darpa, and NSF see web site at

2 I. The Good News is: GHz Electro-Magnetic waves easily penetrate human tissue: They reflect from all of the Speech articulator tissues EM radar-like sensors are important because they... provide real time information, unaffected by Acoustic noise, use very low power EM waves, < 1mW, can be very low in cost, < $5 each, in quantity are very small. JFH:8/2/01cd 2

3 II. The Not so Good News is: EM waves reflect, refract, scatter, and are partly absorbed by all dielectric and conductivity interfaces in their path Interferometric sensor signals may be ambiguous regarding the origins of their reflected signals due to Longitudinal location ambiguity, and Product of (Target Area x Movement ) can be ambiguous Sensor movement relative to the targeted tissues Transverse resolution is typically 2-3 cm, but longitudinal resolution is a few microns JFH:8/2/01cd 3

4 III. Good News: EM sensors robustly sense Voiced speech signals Direct measurements of vocal fold movements--with and without contact: < 1mm to 1 cm movements Pressure induced trachea wall movements (primarily the anterior wall): micron movements. Pressure induced vocal resonator wall and surface motions-- e.g., cheek, tongue, lips, pharynx, sinus surfaces: e.g., 5-10 micron movements (Macro-movements of jaw, tongue, lips, soft palate, etc. are being quantified) JFH:8/2/01cd 4

5 What are the Sources of Radar-like Sensor Signals? Experiment configurations-glottis, trachea walls, cheek wall Laryngeal Prominence Location Data Vocal Tract Wall Data Conclusions JFH:8/2/01cd 5

6 Laser doppler and EM sensor signals from a subject having a stoma in the neck are compared Gems on cheek Laser Doppler from Inner Cheek Wall Laser Doppler from Rear Trachea Wall Gems on sub-glottal region 4 cm below vocal folds JFH:8/2/01cd 6

7 What are the Sources of Radar-like Sensor Signals? Experiment configurations Laryngeal Prominence Location Data Vocal Tract Wall Data Conclusions JFH:8/2/01cd 7

8 An EM sensor signal reflected from the glottal area, is correlated to an EM sensor signal using high speed video (3 per ms, 0.03 ms exposure)* * G.C.Burnett Thesis UC Davis JFH:8/2/01cd 8

9 The EM sensor laryngeal-prominence signal is from vocal fold opening and closing*, typ. 1 volt signals Onset of Vocal Fold closure Typical EM sensor signals* Laser velocity Rear wall position Sub-glottal Pressure IEGG vocal fold contact Microphone acoustics * Subject was treated for laryngeal paresis following thyroplasty with silicon prosthesis implant, and implant of #6 Montgomery speaking valve. JFH:8/2/01cd 9

10 Vocal Fold Membrane with Circular Glottal Opening Simulation mesh of trachea and vocal fold membrane Typical observation point of reflected waves Incoming Plane Wave 2.3 GHz JFH:8/2/01cd cm The neck model is an infinite dielectric ( ε = 25 ), with an internal soup can air space divided by a 4 mm membrane Example of parabolic slot in membrane

11 JFH:8/2/01cd x y Abs@ED Ey Scat., 1.5 cm cyl., 3 mm parabolic slot HclippedL Extensive EM simulations were employed to obtain amplitude and phase of Reflecting EM waves

Simulated EM wave wave reflections from the glottal opening show origin of strong glottal signals a c Sagittal slice through 1.5 cm dia air tube, with 4mm vocal fold membrane & circular hole.

12 Simulated EM wave wave reflections from the glottal opening show origin of strong glottal signals a c Sagittal slice through 1.5 cm dia air tube, with 4mm vocal fold membrane & circular hole. A) open tube, wave reflects from first surface b d B) fold adducting, carried wave into tube C) nominal opening in folds, showing reflection locations D) closed glottis shows little reflection, wave passes through JFH:8/2/01cd 12

13 JFH:8/2/01cd X (M) x HmL x (m) E y HVmL 100 Measurement Location, 1.75 cm Solid Open AR= 0.25 AR = 0.5 Trachea Tube 1.5 cm ID EM Ey Amplitude V/M 175 Glottis Condition- Larynx, Abs@EyDscattered, Circular Glottal dia=1.5 Opening cm, q=90 deg., const. vol. From the simulations we obtain reflected EM wave Amplitudes and Φs, for an open tube, a solid membrane, circular and slot openings in the membrane

14 Both phase and amplitude contribute to the EM sensor signal, A cos Φ, between configurations. A phase and amplitude jump occurs upon initial fold separation x (m) JFH:8/2/01cd 14 phase Ey (deg) Phase changes versus slot increase Parabolic slot width: open: black, dot-dash 3 mm: red, solid 2 mm: grn, short dash 1 mm: blue, med. dash 0.5 mm: violet, long dash 0. mm: black, solid Slot width Phase jump upon slot formation 1.5cm dia

15 An air flow function can be generated by associating the EM sensor signal with glottal area ( In Progress) Glottal area ratio Signal Level parabolic Parabolic slot shows rapid initial rise in reflectivity due to slot formation circular nominal region of vocal fold opening and closing 40 JFH:8/2/01cd 15

16 What are the Sources of Radar-like Sensor Signals? Experiment configurations Laryngeal Prominence Location Data Vocal Tract Wall Data Conclusions JFH:8/2/01cd 16

17 Vocal tract wall experiments,e.g., trachea and cheek, show 5-10 micron motions versus 5-10 cm H 2 O pressure cycles Employed an EM sensor, laser, pressure sensor, EGG, and microphone. Gems on cheek Laser Doppler from Inner Cheek Wall, 15 micron movement due to 5 cm H 2 O -pressure Laser Doppler from Rear Trachea Wall, 14 micron movement due to 7 cm H 2 O -pressure Gems on sub-glottal region 4 cm below vocal folds, 15 microns movement of anterior sub-glottal wall (from 30 mv sensor signal) JFH:8/2/01cd 17

18 JFH:8/2/01cd x Posterior Wall Reflection, est. 4 mv z 0 Incident EM wave 0.01 Sub-glottal pressure changes of 5-10 cm H 2 O Anterior Wall reflection, typ. 40 mv Posterior wall movement Anterior wall movement EM sensor signal amplitude and shape can be estimated by simulating changes in position of posterior and anterior surfaces of the tracheal tube

19 EM sensed subglottal trachea wall signals show anterior wall ballooning versus pressure Vocal fold signals, 1 V Onset of Fold closure Rear tracheal wall position versus time, from laser doppler velocity signal Sub-glottal anterior wall movement EM sensor measured trachea wall motion, -4cm down from larynx, giving 40mV signals JFH:8/2/01cd 19

20 What are the Sources of Radar-like Sensor Signals? Experiment configurations Laryngeal Prominence Location Data Vocal Tract Wall Data Conclusions JFH:8/2/01cd 20

21 Conclusion: Low power, interferometric EM wave sensor signals are understood and very useful for real time speech processing We can measure Vocal-Tract tissue interface motions as small as =+ 1 µm using < 0.5 mw interferometric sensors: Two types of Glottal related signals ( Hz) are obtained from the neck and head region: Direct measurement of vocal fold cycle, x = cm Air pressure induced vocal tract wall movement, x = 5-15 µm Low power EM sensor signal data are enabling tremendous improvements in human speech characterization for many applications Low Bandwidth Vocoding, < 300 Hz Denoising Speaker Verification Speech Recognition JFH:8/2/01cd 21

22 0 JFH:8/2/01cd Frequency Normalized Amplitude ω 0.7 TF(ω) = A(ω) / E(ω) EM Sensor Excitation spectral Transfer output function or Transfer function For /a/ Acoustic EM sensor jaw tongue lips pharynx Pressure Reservoir (lungs) subglottus subglottis E A microphone vocal folds 4 soft palate Horizontal vocal tract with 4 resonator chambers The human vocal tract can be well characterized if a sufficiently good excitation function can be obtained

23 2 EM sensor measurements for butter, glottis and jaw Jaw open Jaw closed JFH:8/2/01cd 23

24 Glottal-synchronous transfer functions for butter using EM sensor excitation & ARMA u t e...r JFH:8/2/01cd 24

25 Additional information is available at the following: See web sites for additional information: (many reports and papers) (LLNL licensee: commercial applications and demonstrations) Recent working draft, available as preprint, describes details of EM wave interaction with glottal and tracheal structures (in review for publication). Titled: EM Wave Measurements of Glottal Structure Dynamics Holzrichter, Ng, Burke,Champaign, Kallmann, Sharpe --LLNL Kobler, Rosowski, and Hillman -- Mass. Eye and Ear Infirmary available as Livermore Report number: UCRL -JC and above llnl.gov web site. JFH:8/2/01cd 25

26 Major collaborators have been Prof. Robert Hillman - Harvard and Mass. Eye & Ear Infirmary Profs. Neville Luhman and Richard Freeman UC Davis Drs. Greg Burnett and Todd Gable Former students: Lawrence Livermore and UC Davis/ LLNL campus Prof. John Ohala UCB Dr. Rebecca Leonard UCD Hospital Prof. Ingo Titze U of Iowa Dr. Wayne Lea Speech Sciences Institute JFH:8/2/01cd 26

Source-Filter Theory 1

Source-Filter Theory 1 Vocal tract as sound production device Sound production by the vocal tract can be understood by analogy to a wind or brass instrument. sound generation sound shaping (or filtering)