Design of FPGA-Based Rapid Prototype Spectral Subtraction for Hands-free Speech Applications

Transcription

1 Design of FPGA-Bsed Rpid Prototype Spectrl Subtrction for Hnds-free Speech Applictions Sryut Amornwongpeeti *, Nobutk Ono *, nd Mongkol Ekpnypong * Principles of Informtics Reserch Division, Ntionl Institute of Informtics, Tokyo, Jpn E-mil: sryut.mornwongpeeti@it.c.th, onono@nii.c.jp Tel: Deprtment of Microelectronics nd Embedded Systems, Asin Institute of Technology, Pthum Thni, Thilnd E-mil: mongkol@it.c.th Tel: Abstrct In this pper, the design of FPGA-bsed rpid prototype for Short-Time Fourier Trnsform (STFT) bsed spectrl subtrction for hnds-free speech pplictions using Xilinx System Genertor (XSG) tools without trditionl HDL hnd coding is presented. Initilly, the concept of dul-chnnel short-time spectrl subtrction lgorithm for removing the widebnd bckground noise in speech signl is introduced. The studied lgorithm is developed in the system-level modeling simultor using MATLAB Simulink environment. For the digitl hrdwre design, simple hrdwre rchitectures for dt frming nd overlpping lgorithms re proposed by utilizing the bsic DSP blocksets of the XSG librry, nd replced in the simultion model. Finlly, complete simultion model of the FPGA-bsed short-time spectrl subtrction lgorithm using XSG is presented. The comprtive performnce evlution bsed on simultion results nd the summry of resource utiliztion re confirmed the implementtion fesibility of the rel-time FPGA-bsed short-time spectrl subtrction lgorithm for hnds-free speech pplictions. I. INTRODUCTION For the ppliction domin in the re of udio nd coustic signl processing, it is inevitble tht the input speech signl cptured by the microphone sensor is lwys mixed with the bckground noise in rel-world environments. Depending on the level of the mbient noise, it cn significntly degrde the desired speech signl in term of the speech qulity nd intelligibility. However, the presence of dditive coustic noise in the interested speech cn be reduced nd llevited by using brod rnge of mny existing speech enhncement techniques, such s spectrl subtrction [,4], signl-subspce embedding [], time-domin itertive pproches [3]. The min objective of these speech enhncements is to remove the bckground noise while mintin the perceptul spects ner the level of the noise-free input speech. Spectrl subtrction is simple nd efficient single-chnnel speech enhncement technique, which ws originlly proposed in the STFT domin by []. Lter, the spectrl subtrction pproch with brod clss of estimtors including the mgnitude nd the power subtrction hs been studied in [4]. The concept of this method is to suppress the dditive coustic noise from the noisy speech by subtrcting either the mgnitude or the power spectrum between the observed signl nd the estimted noise spectrum. For the hnds-free speech pplictions, multiple input microphones of the multi-chnnel speech system cn be plced fr wy from the speker of interest [5]. In this pper, the system is considered s dul-chnnel speech enhncement, where two signl inputs re vilble mking the system possible to use the uxiliry chnnel s the noise reference s shown in Fig.. In ddition, it is ssumed tht the bckground noise source is locted fr from these two microphone sensors so tht the non-sttionry noise vrince of two chnnels cn be considered in equl except the noise wveform in time domin. By this wy, dptive noise cncelltion cn be esily chieved only by plcing the reference microphone fr wy from the speker of interest. Nowdys, the hrdwre-bsed FPGA pltform hs been recognized s promising technology for the high performnce rel-time digitl signl processing system. FPGAs offer promising fetures over thn the trditionl DSPs including, fst computtion time due to the highly internl prllel processing, customizble pltform with design flexibility, nd high relibility. By using hrdwre description lnguges, such s VHDL nd Verilog HDL, complete digitl logic systems cn be simply designed in hierrchicl modules nd synthesized into the FPGA. Xilinx System Genertor (XSG) is high-level grphicl progrmming tool used for designing high performnce DSP system with MATLAB Simulink trgeting for the FPGA. XSG tool llows the hrdwre designer to build systemlevel model in Simulink for lgorithm verifiction, such s dptive noise cnceling [6,7], STFT bsed pttern recognition [8], spectrl subtrction [9-]. In ddition, the XSG code genertor lso enbles the user to utomticlly generte the synthesizble HDL code mpping into the highly pre-optimized IP cores, which drmticlly reduced the design time nd quickly evlute new lgorithms in hrdwre compring with trditionl HDL hnd coding. According to the literture, lthough the power spectrl subtrction rchitecture using high-level XSG tool hs been presented in [9], the bsence of hrdwre design for the DFT frming-windowing lgorithm s well s the inverse DFT overlp-dd lgorithm leds to the lgorithm limittion for the rel FPGA hrdwre implementtion. In [,], complete APSIPA APSIPA 4

2 The short-time Fourier trnsform (STFT) is twodimensionl representtion in the meningful time nd frequency domins. The STFT of signl x(t), evluted t time index m nd frequency index n is defined s [3]: Fig. Overll system of dul-chnnel short-time spectrl subtrction for hnds-free speech pplictions. FPGA-bsed system of the mgnitude spectrl subtrction with the XSG design pproch hs been successfully implemented. However, the use of only initil dt frmes for estimting the entire noise mgnitude spectrum mke the system less ttrctive for hnds-free speech pplictions, which often involves the non-sttionry noise in the rel environment. The initil frme must be contined only the noise component in order to estimte the verge constnt noise floor. Although the dt frming nd the overlp-dd opertions were implemented with the bsic Xilinx ddressble shift register block, this design is limited with the mximum memory depth of 4 resulting in the limittion for the lrge dt buffer size. A FPGA-bsed mgnitude spectrl subtrction using the trditionl HDL coder-bsed pproch hs been developed in []. However, this pproch is not suitble for the rpid prototype development due to lrge time consuming during the design process. Thus, the im of this pper is to present the design of FPGA-bsed rpid prototype short-time spectrl subtrction for hnds-free speech pplictions using XSG tools nd lso propose simple hrdwre rchitectures for FPGA implementtion of the dt frming nd overlpping lgorithms, which cn be expndble for the lrge dt buffer size. The outline of this pper is orgnized s follows. The introduction is briefly discussed in Section. In Section, the concept of short-time spectrl subtrction for reducing the mbient coustic noise is described. In Section 3, the systemlevel simultion model nd the XSG-bsed rchitecture model of the studied lgorithm is developed nd presented. In Section 4, simple hrdwre rchitectures utilizing the bsic XSG blocksets for dt frming nd overlpping lgorithms is proposed. Finlly, simultion results nd conclusions re covered in Section 5 nd Section 6, respectively. II. THE CONCEPT OF SHORT-TIME SPECTRAL SUBTRACTION The principles of short-time spectrl subtrction is considered bsed on the trnsform domin pproch, which consists of the short-time nlysis-modify-synthesis steps in the frme-bsed processing. Bsed on the ssumption tht the noise is n dditive zero-men nd uncorrelted with the clen speech in the wide-sense sttionry rndom process. Thus, the observed noisy signl x(t) is sum of the clen speech signl s(t) nd the dditive white noise d(t), which commonly given s: A. Short-time Fourier Trnsform xt () = st () + dt () () mr+ N j π ( t mr) n/ N Xmn (, ) = wt ( mrxte ) ( ) t= mr () wheres the inverse STFT of the processed signl Y(m,n) is: + N j π ( t mr) n/ N yt () = wt ( mr) Ymne (, ) N m= n= (3) where N is the size of frme length, R is the frme shift fctor, w(t) is the windowing function. For perfect reconstruction, + wt ( mr) = (4) m= should be stisfied. B. Short-time Power Spectrl Subtrction By considering the stereo-chnnel speech enhncement system, let us denote X (m,n) s the STFT representtion of the first chnnel contining the trget speech nd the bckground noise, while X (m,n) is the STFT representtion of chnnel two contining only the bckground noise. By using the STFT nlysis into (), the mthemticl expression of mixed model in STFT domin cn be written s: X ( m, n) = S( m, n) + D( m, n) (5) Thus, the STFT mgnitude squred for power spectrl subtrction method cn be expressed s [4]: (, ) (, ) (, ) (, ) X m n = S m n + D m n + S m n D ( m, n ) + S ( m, n) D( m, n) Due to the uncorrelted noise with the clen speech signl, the cross term between these two signls cn be sttisticlly set to zero, which cn be simplified nd rerrnged s: Smn ˆ(, ) X ( mn, ) Dmn (, ) (6) = (7) where Smn ˆ(, ) nd D( mn, ) re the estimted speech nd noise power spectrum, respectively. In [3], the rel positive gin G(m,n) is defined s: Smn ˆ(, ) Gmn (, ) (8) X( m, n) By substituting in (7), the defined gin ( < G(m,n) < ) cn be rewritten s: Gmn (, ) X( m, n) D( m, n) X( m, n) Qmn (, ) (9)

3 where the reltive signl level Q(m,n) cn be defined s: Qmn (, ) = X( m, n) Dmn (, ) () For voiding the spectrl subtrction estimtes negtive spectrl mgnitudes, the vlue of Q(m,n) is replced by one if it is less thn one. Finlly, the processed signl Y(m,n) cn be expressed s function of the rel positive gin G(m,n) nd obtined s: Ymn (, ) Gmn (, ) X( mn, ) = () C. Noise Estimtion In generl, the noise estimtion cn be determined by either the ssumed known properties of the bckground noise or the ctul mesurement during the intervls of the input speech bsence [4]. In this pper, for the simplicity it is ssumed tht the coustic mbient noise is ccessible on seprte microphone chnnel such s in hnds-free speech pplictions. Thus, the effect of dditive bckground noise on the signl spectrum cn be esily estimted by tking the dt from the ccessible uxiliry microphone. For estimting noise power spectrum D( mn, ), we cn use the other observtion chnnel. Let X ( m, n) be the STFT representtion of the other chnnel. If we cn ssume tht this chnnel cptures only noise, we cn estimte s: D( mn, ) β X( mn, ) = () where β is control prmeter to compenste the sensitivity mismtch between two chnnels. By this wy, the ctul noise vrince with different vritions cn be evluted ccurtely nd dptively suppressed in the rel-time processing. Fig. shows the bsic structure of dul-chnnel short-time spectrl subtrction for hnds-free speech pplictions. Fig. Bsic structure of dul-chnnel short-time spectrl subtrction. III. SIMULATION MODEL OF SHORT-TIME SPECTRAL SUBTRACTION A. System-level MATLAB-bsed Model Initilly, for lgorithm verifiction the short-time spectrl subtrction lgorithm is design nd simulted t the systemlevel model with the floting-point MATLAB implementtion in Simulink environment. The MATLAB-bsed model minly consists of block digrms of the STFT, the gin estimtor, nd the inverse STFT s shown in Fig. 3, which developed bsed on the mthemticl equtions from () to (). B. System-level XSG-bsed Model As mentioned in the previous section, the XSG toolbox does not only offer high-level grphicl progrmming tool tht cn be simulted nd then configured to the FPGA trget but lso n efficient solution for FPGA-bsed rpid prototype development. In contrst to HDL coder pproch, the system verifiction cn be esily mde nd explored in severl lgorithm vritions with XSG model-bsed pproch. However, the one-to-one conversion between the Simulink functionl blocks nd the XSG blocksets hs to be mde mnully. In this pper, the bsic XSG blocksets for DSP re From Wve File speech.wv Out (5Hz/Ch/6b) Speech Noise source Noise source hmming Win Window Function System Genertor Unbuffer Unbuffer x[n] w[n] X[nL,k] Short-Time FFT x[n] w[n] X[nL,k] Short-Time FFT In Gtewy In Gtewy In Noise Estimttion In Gin Estimtor Multiply XSG-bsed dul-chnnel Spectrl Subtrction Unbuffer3 Unbuffer4 X[nL,k] x[n] w[n] Inverse Unbuffer5 Short-Time FFT Out Gtewy Out Scope Fig. 3 Top-level simultion digrm showing MATLAB-bsed model nd XSG-bsed model of short-time spectrl subtrction. In Out Out Dely Buffer nd Overlp ++ cst Convert ddr Counter ROM Out Out Dely Buffer nd Overlp b b Mult const b b Mult Out In Out3 FFT Out In In In3 Gin Estimtor b b Mult3 b b Mult4 Out In Submtrix Out In3 Inverse FFT x xs b + b AddSub Normliztion In Out Conctente Submtrix const In Out3 FFT Noise Estimtion Fig. 4 XSG-bsed hrdwre rchitecture of dul-chnnel short-time spectrl subtrction.

4 used for the hrdwre design nd replced in the MATLABbsed simultion model. Severl XSG librry blocksets, including wide rnge of primitive bsic blocksets, Xilinx FFT block, CORDIC divider, nd CORDIC SQRT, re utilized for the XSG-bsed fixed-point implementtion model for high performnce DSP system of the short-time spectrl subtrction lgorithm s shown in Fig. 4. IV. XSG DESIGN AND FPGA IMPLEMENTATION A. Short-time Fourier Trnsform Module The STFT module consists of Hmming window, buffer with overlp, nd FFT subsystem. The Hmming window is implemented with the Xilinx single port ROM with 5 depths nd n ddress count-limited counter. Since the provided XSG librry is still limited to some bsic functionl blocksets, which cnnot be chieved for the sophisticted signl processing functions nd complex opertions, such s frming nd overlp-dd functions. In this pper, the dt frming is implemented with the proposed rchitecture including selected multiplexer, 8 identicl dely lines ( cscded multiplexer with unit delys), nd control logic circuit s shown in Fig. 5. The buffer size nd overlp re set to 5 nd 56, respectively. The Xilinx FFT block is used to perform the FFT function with 5 trnsform length, which provides two output signls of the rel nd imginry prts. For continuously computtion of STFT, the FFT pipelined streming I/O option is selected for hrdwre implementtion. B. Gin Estimtor Module In the short-time Fourier trnsform domin, the gin is determined individully t ech frequency region nd pply to ech frequency bin of the input noisy speech. In this pper, the gin estimtor module consists of the power spectrl clcultion nd the mplitude limiter, which cn be esily implemented with the pre-optimized IP cores. Fig. 6 shows the XSG-bsed hrdwre rchitecture of gin estimtor. C. Inverse Short-time Fourier Trnsform Module The inverse ISTFT module consists of inverse FFT, overlp nd dd, nd normliztion subsystem. The inverse FFT is implemented with the Xilinx FFT block with the setting of 5 trnsform length. However, the two input signls of the inverse FFT block (the rel nd imginry prts) hve to be converted crefully to the unity scle due to the input normlize requirements. In this pper, the FFT nd the inverse FFT re implemented with the 3-multiplier structures for the complex number multipliction for the purpose of resource optimiztion. Since the overlp-dd lgorithm utilizes mtrix conctente block nd submtrix block from the Simulink blocksets, this pper presents simple hrdwre rchitectures hving two dely chins nd control unit for the mtrix conctente nd the submtrix s shown in Fig. 7 nd Fig. 8, respectively. By this wy, the proposed hrdwre rchitecture, which cn be expndble for the lrge buffer size nd comprising of only multiple cscded units of the dely line, cn be esily chieved by utilizing simple DSP blocksets of the provided XSG librry. The long dely with lrge dt Fig. 5 The simplified rchitecture of buffer nd overlp function. Fig. 6 XSG-bsed hrdwre rchitecture of gin estimtor. Fig. 7 The simplified rchitecture of mtrix conctention function.

5 TABLE I SIMULATION PARAMETERS OF SHORT-TIME SPECTRAL SUBTRACTION Resolutions Noise Type Input Speech Signl Smpling 6 bits Frequency Noise Source Gussin with zero men Gin Estimtor Vrin Control prmeter (β) 5 STFT nd Inverse STFT (Hmming window).5 s Window length 5 Overlp 56 FFT length 5 Smples per frme 5 bus widths re implemented with dul port RAM ssocited with two ddress counters insted of using simple dely block provided by the XSG librry for minimizing the usge of FPGA hrdwre resources. Fig. 8 The simplified rchitecture of submtrix function. V. SIMULATION RESULTS In this section, the simultion prmeters of the MATLABbsed model nd the XSG-bsed model of dul-chnnel shorttime spectrl subtrction in Simulink environment re summrized in the Tble I. In order to verify the vlidity of () () (b).5.5 (b).5 Amplitude (V) (c) (d) Frequency (Hz) (c).5.5 (d) Time (s) Fig. 9 Simulted speech wveforms: () Originl speech; (b) Noisy speech; (c) Enhnced speech with MATLAB implementtion; (d) Enhnced speech with XSG implementtion..5.5 Time (s) Fig. Spectrogrms: () Originl speech; (b) Noisy speech; (c) Enhnced speech with MATLAB implementtion; (d) Enhnced speech with XSG implementtion.

6 TABLE II FPGA RESOURCE UTILIZATION FOR SHORT-TIME SPECTRAL SUBTRACTION ON XILINX VIRTEX-5 XC5VLXT CHIP. Resource Type Avilble Used Utiliztion Number of Flip Flops 69, 4,73 58% Number of LUTs 69, 33,4 48% Number of Occupied Slices 7,8 3,44 77% Number of Bonded IOBs % Number of Block RAM/FIFO 48 3% Number of DSP48Es % Mximum Operting Freq..86 MHz spectrl subtrction lgorithm for hnds-free speech pplictions, the simultion results re obtined with three different noisy environments strting from the condition (t = s) with the speech signl contining the dditive white Gussin noise (vrince of.3). After t =.5 s, the power spectrum noise is incresed with the vrince of.7. At t =.5 s, the noise vrince is decresed to.5. It should be noted tht the dditive white noise for ech microphone chnnel is generted seprtely from different noise sources except the sme setting vlue of the noise vrince. It cn be observed tht the short-time spectrl subtrction lgorithm hs effectively ttenuted the coustic mbient noise nd hence significntly enhnced the intelligibility of the trget speech. The enhnced speech is dptively reduced vrying dditive bckground noise levels while contins the informtion very similr to the originl speech. In ddition, in order to verify the lgorithm correctness of the hrdwre design incorporting with proposed rchitectures for the FPGA implementtion, the simultion results hve been compred between the floting-point MATLAB implementtion nd the fixed-point XSG implementtion. It cn be concluded tht the fixed-point XSG implementtion grees with the flotingpoint MATLAB implementtion nd performs consistently the sme performnce in term of clcultion ccurcy with the mximum error less thn.965 %. Fig 9 nd Fig. show the simulted speech wveforms nd spectrogrms compring mong the originl speech, noisy speech, enhnced speech (MATLAB nd XSG implementtion), respectively. Furthermore, the entire system design of dul-chnnel shorttime spectrl subtrction lgorithm hs been relized nd fully integrted into single trget FPGA chip. The fixed-point XSG-bsed hrdwre design utilizes pproximtely with 58% of flip-flops, 48% of LUTs, nd 77% of occupied slices vilble in totl of FPGA resources with the mximum operting frequency of.86 MHz on the Virtex-5 XC5VLXT FPGA chip (see Tble II), while the design in [9] chieved with the mximum operting frequency of 6.56 MHz on the Virtex-6 XC6VLX4T FPGA pltform. The summry of totl FPGA resource utiliztions confirms tht the entire complete system incorporting with proposed rchitectures is perfectly fitted nd possible configured into the trget FPGA, which mde it is suitble for the rpid prototype development system. VI. CONCLUSIONS In this pper, the design of FPGA-bsed rpid prototype short-time spectrl subtrction lgorithm using the XSG tool hs been presented. A dul-chnnel speech enhncement bsed on the power spectrl subtrction for hnds-free speech pplictions ws designed in system-level simultion model using MATLAB Simulink. The XSG-bsed system model ws developed for the FPGA hrdwre implementtion. The hrdwre rchitectures for dt frming nd overlpping lgorithms were designed nd proposed, which do not provide by the XSG librry. The fixed-point lgorithm design ws verified by compring results with the floting-point MATLAB implementtion, nd ws successfully relized into Xilinx Virtex-5 FPGA. This design chievement confirms the implementtion fesibility with less design time of the FPGA-bsed dul chnnel spectrl subtrction lgorithm in rel-time processing for hnds-free speech pplictions. REFERENCES [] S. Boll, Suppression of coustic noise in speech using spectrl subtrction, IEEE Trns. Acoustics, Speech nd Signl Processing, vol. 7, no., pp. 3-, Apr [] Y. Ephrim nd H. L. Vn Trees, A signl subspce pproch for speech enhncement, IEEE Trns. Speech nd Audio Processing, vol. 3, no. 4, pp. 5-66, Jul [3] S. Roucos nd A. Wilgus, High qulity time-scle modifiction for speech, in Proc. ICASSP, pp , Apr [4] J. S. Lim nd A. V. Oppenheim, Enhncement nd bndwidth compression of noisy speech, Proceedings of the IEEE, vol. 67, no., pp , Dec [5] S. Gungji, P. Arbi, nd J. Hui, Phse-Bsed Dul- Microphone Speech Enhncement Using A Prior Speech Model, IEEE Trns. Audio, Speech, nd Lnguge Processing, vol. 5, no., pp. 9-8, Jn. 7. [6] B. Hndong, Z. Zhiguo, H. Jing, nd L. Zhiwen, FPGA implementtion of ICA lgorithm for dptive noise cnceling, in Proc. icast, pp , Sep.. [7] M. Bhour nd H. Ezzidi, FPGA-implementtion of sequentil dptive noise cnceller using Xilinx System Genertor, in Proc. ICM, pp. 3-6, Dec. 9. [8] M. Bhour nd H. Ezzidi, FPGA implementtion of feture extrction technique bsed on Fourier trnsform, in Proc. ICM, pp. -4, Dec.. [9] M. Bhour nd H. Ezzidi, Implementtion of spectrl subtrction method on FPGA using high-level progrmming tool, in Proc. ICM, pp. -4, Dec.. [] J. Whittington, K. Deo, T. Kleinschmidt, nd M. Mson, FPGA implementtion of spectrl subtrction for in-cr speech enhncement nd recognition, in Proc. ICSPCS, pp. -8, Dec. 8. [] J. Whittington, K. Deo, T. Kleinschmidt, nd M. Mson, FPGA implementtion of spectrl subtrction for utomotive speech recognition, in Proc. CIVVS, pp. 7-79, Mr. 9. [] U. Mhbub, T. Rhmn, nd A. B. M. H. Rshid, FPGA implementtion of Rel Time coustic noise suppression by Spectrl Subtrction using Dynmic Moving Averge Method, in Proc. ISIEA, pp , Oct. 9. [3] O. Cppe nd J. Lroche, Evlution of short-time spectrl ttenution techniques for the restortion of musicl recordings, IEEE Trns. Speech nd Audio Processing, vol. 3, no., pp , Jn. 995.