Active and Passive Acoustic Detection, Classification and Recognition with the Hopkins Acoustic Surveillance Unit (HASU) Andreas G. Andreou Electrical and Computer Engineering and Center for Language and Speech Processing Johns Hopkins University andreou@jhu.edu
What is the Problem? Is there something interesting in the environment? detection in a specific class of objects Where is it? localization What is it? identification/recognition it is all about a few bits in the right place at the right time!
Computational Sensors and Memories Conventional Processing in Sensor Networks: Sensor Sensor Encoder and Transmitter Encoder and Transmitter Channel Receiver and Decoder Information Extraction Distributed Information Extraction (JHU Approach) Sensor Information Extraction Encoder and Transmitter Sensor Information Extraction Encoder and Transmitter Channel Receiver and Decoder reduced bandwidth requirements reduced power dissipation and form factor trade-off between complexity/power of integrated processing
JHU Acoustic Surveillance Unit (JHU-ASU) Localization display Unattended ground sensor Battery powered Small form factor (3 in diameter) Target detection and localization Micropower ASICs developed at Hopkins Wake-up detector Cross-correlation localizer Gradient flow localizer Wireless communication interface COTS Mica2 programmable interface (Xbow technologies) Wake-up activated for low power Mica2 wireless interface JHU-ASU
Passive Acoustic Sensing
Smart Microphone Integrated Prototype 4 chamber acoustic horn Cross-Correlation ASIC Gradient Flow ASIC Auto-Correlation Wake-Up ASIC Analog Sensor Data (test) & RS-232 Contact Reports Power Strobe Circuit 4 MEMs Microphones
Integrated Optoelectronic Readout Interface for Acoustic and Vibration Sensors Chip-scale Fabry-Perot Interferometer
Smart Microphone Power Budget Power Budget to Meet 9.5 Month Endurance on 4 AA Batteries Component Power (mw) @ 100% Duty Cycle Notes Budget Actual included in integrated prototype Microphone & Signal Cond 1.2 1.44/3.84 Power Strobe Micro-controller 6.0 22.5 ->9* *New controller Auto-Corr Wake-Up.015.006 Pk Periodicity Wake Up.015 N/A.025-25 mw on test board Gradient Flow 3.7.04 Cross-Correlator 3.7.6 Cochlea Bearing Est 3.7 N/A 3.7 meas on test board Clocks/Bias/Additional Amps Totals 0 33 -> 8* *Low power clocks and redo biases 10.9 (cochlea only) 58/60 -> 21.4 21.4 = 6.5 month endurance (2.0 mw)
ACF Wake-Up Detector Architecture Even select (20 bits) Even ACF register (20x10 bits) ACC ACC ACC ACC [0] [2] [36] [38] Even ACF value (10 bits) XNOR XNOR XNOR XNOR Audio input (1 bit) INP [0] INP [2] INP [36] INP [38] INP [40] INP [50] Input shift register (52x1 bits) INP [1] INP [3] INP [37] INP [39] INP [41] INP [51] XNOR XNOR XNOR XNOR Odd ACF register (20x10 bits) Odd select (20 bits) ACC ACC ACC ACC [1] [3] [37] [39] Even ACF value (10 bits) Odd ACF value (10 bits) ABS DIFF Periodicity measure ACC (16 bits) COMP Threshold (16 bits) Odd ACF value (10 bits) Detect? (1 bit)
ACF Wake-Up Detector Algorithm A signal with periodic content has a bumpy autocorrelation function (ACF). Compute ACF over a range of time lags corresponding to the frequency range of interest. Compute the power of the derivative of the ACF---a bumpiness measure. Use the ACF derived from a 1-bit version of the signal : Replace multiplication with XNOR. Eliminate normalization, because the 1-bit signal is insensitive to amplitude. ~ R ~~ xx K 1 k 0 n ~ x n ~ x k n For the power of the derivative, replace squaring with absolute value: PM N max n N min ~ R ~~ xx ~ n 1 R~~ n xx
2 Gradient Flow Localization s( t ) t 1 2 10 01 00 s (t) t 1 2 1 s(t) Gradient flow bearing resolution is fundamentally independent of aperture Resolution is determined by sensitivity of gradient acquisition Mechanical differential coupling (Miles et al.) Optical differential coupling (Degertekin) Analog VLSI differential coupling d dt + + + + - + + - 00 10 01 s ( t) s ( t) 1 s ( t) 2
Gradient Flow System Diagram Analog inputs x -10 x 01 x 10 Average, temporal derivative, and spatial gradient estimates Spatial gradients with suppressed common-mode Digital estimated delays x 0-1.
Broadband Bearing Estimation Architecture Maximum detector 1 bit A/D Mic2 u 2 (t) 0/1 z -1 z -1 Mic1 u 1 (t) 0/1 z -1 z -1 104 stages α 0 104 stages α 0
ACF Wake-Up Detector ASIC Fully functional Core power consumption: 0.8 uw Total power consumption: 6.3 uw
ACF Detector and BBB Estimator Chip 2.9 x 2.4 mm Die TSMC 0.35 4M2P 165,000 Transistors ACF Detector (1.25 x 0.6) mm BBB Estimator (2 x 2) mm XTAL Oscillator + Preamplifier (0.6 x 0.3) mm ACF Detector BroadBand Bearing Estimator
ACF Wake-Up Detector Simulation GV2A1078 GV3C1090 Ambient 20 20 20 Lag (samples) 30 40 Lag (samples) 30 40 Lag (samples) 30 40 50 50 100 150 200 250 Time (sec) 50 50 100 150 200 Time (sec) 50 50 100 150 200 Time (sec) 10 4 10 4 10 4 Periodicity measure 10 3 Periodicity measure 10 3 Periodicity measure 10 3 10 2 50 100 150 200 250 Time (sec) 10 2 50 100 150 200 Time (sec) 10 2 50 100 150 200 Time (sec)
ACF Wake-Up Detector Field Results Ambient noise Stimulus amb stim 300 300 250 250 200 200 Frequency 150 Frequency 150 100 100 50 50 0 0 5 10 15 20 Tim e No detection Detection 0 0 1 2 3 4 5 6 7 Time No detection SNR: 25 db SNR: 12 db Minimum SNR for Detection: 13-16 db (1 Hz Band) Pre-whitening filter needed for increased sensitivity
Acoustic Surveillance Unit Bearing Accuracy Field Test (Severna Park, MD) 10 2 GadF NB Bias GradF NB StdDev XCor NB Bias XCor NB StdDev Chip Avg Bias Avg Std Dev Error (Deg) 10 1 GradFlow 5.7º 1.2º X-Corr 4.0º.79º 10 0 10-1 -100-50 0 50 100 Angle of Arrival (Deg) Bias likely due to: - ambient noise, - electrical phase distortion, - non-uniform placement of phase centers
Spesutie Island Oval (Aberdeen Proving Grounds, MD) 108m Node V1 (N) M60 mid speed 662m V1 97m 132m206mV3 ASU #2, UGS AWS Node V2 (S) 88m V2 133m 80m ASU#3, AWS BS HEMET mid speed ASU#1 AWS, PC Base Station tent Network Fusion PC w/ three B vs T and Geoplot Display Colors Blue - Target Ground Truth Bearing Green - Analog bearings Red GF bearings Light Blue XC bearings
Ft. Devens field test (August 2005) Organized by BAE/Honeywell under MASINT Opportunity to field test JHU-HASU hardware and collect data for further experiments
Chevrolet Malibu 40 mph model data
Active Acoustic Sensing
Acoustic u-doppler sonar
Acoustic Attenuation in Air Ax ( ) Aexp( x) o 2 air : 1.6 10 f db m 10 1 H.E. Bass et al, Atmospheric absorption of sound: Update, JASA, 1990.
COTS Transducers Frequency (khz) Maximum Detection (m)* 0% Humidity 50% Humidity Manufacturer 20 1250 152 Murata 30 555 85 Murata 40 312 60 Panasonic 90 62 27 ITC 150 22 14 ITC 200 12 9 ITC 312 5 4 ITC *Maximum detection is an approximate number based on a simple S/N calculation.
Sensing Principle V fo f 60 cm 30 cm 2v fo 2 f f o Transducer
Volunteer 1: Walking Towards CWAR (Hz) (s) December 2005
Volunteer 2: Walking Away From CWAR
Miniature Pinscher, 9.5lbs
American Pitbull
Horse of unknown breed
Classification preliminary results- Model: Gaussian Mixture Model Feature vectors: Cepstral vectors + differential cepstral vectors Decision rule: Maximum likelihood Input: 20 signatures from 8 people 50% for training, 50% for classification
Results (Direct Sampling)
Results (Bandpass Sampling)
Results: 14bit vs 8bit in Bandpass Sampling
In this project we will develop a cognitive acoustic scene analysis system that is able to synthesize composite representations of animate entities and their behaviour by integrating information from active and passive sound signatures; i.e. from actively self-generated (sonar) sounds and from passively received sounds emitted by those entities.