Fisher-Information-Based Data Compression for Estimation Using Two Sensors

Transcription

1 Fisher-Iormatio-Based Data Compressio or Estimatio Usig Two Sesors Mar L. Fowler * ad Mo Che State Uiversity o New Yor at Bighamto Departmet o Electrical ad Computer Egieerig P. O. Box 6 Bighamto, NY 39-6 Phoe: Fax: {mowler, mche}@bighamto.edu Abstract: A trasorm codig data compressio method is developed or the case o maig estimates rom data collected usig two sesors. We ocus o the traser o data rom oe sesor to aother sesor, where the shared data is the used with the local data to estimate a parameter. Particular attetio is paid to the case where either sesor aloe ca estimate the parameter rom its data. The method uses a operatioal rate-distortio viewpoit together with a distortio measure based o the Fisher iormatio o the estimatio problem. Explicit meas o usig the trasormed data to compute operatioal measures o the Fisher iormatio are give. A iteger optimizatio versio o the Lagrage multiplier method is used to eicietly determie the optimal operatig poit o the trasorm compressio algorithm. The advatages o the method lie i its ability to use trasorm codig to eectively capture the impact o compressio o estimatio accuracy i a way that leds itsel to eiciet optimizatio withi the operatioal rate-distortio viewpoit. The applicability ad eectiveess o the method are demostrated or two illustrative examples: i estimatio o time-dierece-oarrival TDOA, ad ii estimatio o reuecy-dierece-o-arrival FDOA. I these two cases it is show that the Fisher-iormatio-based method outperorms the stadard MSE approach. * Correspodig Author

2 I. Itroductio Ote a sigle sesor ca t estimate a desired sigal parameter e.g., [], []; eve whe it ca, usig data rom other sesors ca improve the estimate. We ocus o compressio o data at oe sesor S, which is the trasmitted to a secod ode S where it is used with the local data to estimate a sigle parameter; extesio to multiple iereces is o-goig [3]. A ey to such compressio problems is to use a ierece-cetric distortio measure such as Fisher iormatio [4]. We assume a sigal i AWGN model where the oises at the two sesors are idepedet a simpliyig but oetheless useul sceario. I particular, wor o compressio or time/reuecy-dierece o arrival TDOA/FDOA systems goes bac over two decades with limited success [5]-[7], ad a better solutio, eve uder simpliyig assumptios, is useul. Compressio methods ca be developed either uder a classical rate-distortio viewpoit [8] or a operatioal rate-distortio viewpoit [9]. The ormer strives to develop methods that are optimal o average. The latter speciies a ramewor ad optimizes its operatig poit or the particular sigal at had. Because a sesor system would liely operate i dierig sigal eviromets, we ocus o the operatioal viewpoit, which uses umerically-computed allocatios o bits see [9],[3] rather tha classical closed orms such as reverse water-illig see [8]. We model the sesor data as a determiistic sigal with determiistic parameter i AWGN. We compress data collected at S usig o more tha a budgeted R bits while maig the estimate at S with the lowest possible mea-suare estimatio error. We assume that the estimatio processig ad compressio processig are ot joitly desiged this is motivated by our belie that a sesor may liely be called o to provide data to other sesors or systems that are idepedetly desiged. Compressio or distributed estimatio has bee cosidered i [] [], which ocus either o optimizig compressio or i estimatig the source sigal [],[],[7],[8] or o ii estimatig source parameters [3]-[6],[9],[]. May ocus o oly scalar uatizatio [] -

3 [3],[5],[6],[]. Others ocus o more geeral compressio structures [4],[7],[8],[9], although, most o these [4],[8],[9] ocus o establishig iormatio theoretic results. The closest to our approach are [5] ad [6], which use FI but limit their desigs to scalar uatizers. The result i [] also limits its ocus to scalar uatizers, but taes a uite dieret ierececetric metric speciic to the TDOA case. I cotrast, our approach cosiders trasorm-based compressio with FI ad our results show that proper choice o trasorm allows better exploitatio o the FI structure. II. Algorithm Developmet Let the real or complex data vector x have a probability desity uctio PDF px; that is parameterized by, which is to be estimated. The FI the is [] x; l p J ; x = E, where the otatioal depedece o x is icluded to show the data set or which the FI is computed. Lossy compressio chages the PDF ad thus chages the FI. We see to retai the maximum FI i the compressed xˆ while satisyig a bit budget R. Whe the FI depeds o, a uestio arises as to how to precisely characterize this desire to maximize the FI. Thus, i geeral, we propose a miimax approach. Let J ; x ad J ; xˆ be the FI o the origial data x ad the compressed data xˆ, respectively, ad compress to R bits such that we satisy [ ; x J ; xˆ ] mi max J, where the miimizatio is over all xˆ that meet bit budget R. x ˆ Clearly, whe the FI does ot deped o this becomes max J xˆ, o which we will ocus. At sesor ode S we model the received sigal vectors x as x ˆ x = s + w, =,,

4 3 where s is a uow determiistic vector depedet o the uow determiistic scalar, ad w is a zero-mea, white Gaussia oise vector with w ad w idepedet. We assume the variace σ o w is ow or estimated. For trasorm codig we use a orthoormal ON basis N { } = φ with χ as the coeiciets or x, ad ON expasio coserves FI, so J χ = J x. Oly those χ with sigiicat cotributio to the FI should be selected ad uatized. Let Ω {,, K, N} be a set o selectio idices; let B = { b Ω} be a set o bit allocatios; let { ˆ χ Ω} be the selected/uatized coeiciets usig allocatio B. The compressed sigal is Ω x = χˆ φ ˆ ; groupig these coeiciets ito vector orm gives χˆ = ξ + ω = ξ + ν + ε, 3 where ξ holds the selected oisy sigal coeiciets ξ, ω holds the correspodig oise coeiciets ω, ad ε is the uatizatio oise vector. By idepedece J xˆ, x = J xˆ + J so we eed oly maximize J xˆ. We use a stadard model or the x uatizatio oise: white, uiormly distributed, zero mea, ad idepedet o the sesor-oise vector ω ; idepedece is valid due to ditherig by the sesor oise []. The variace o the sum o these two oises is the sum o their two variaces. But what PDF or ˆχ should be used i? For uatizatio to bit we use the ideas o [] to get a explicit result; or multi-bit uatizatio, the derivatio o the FI seems itractable [] we will motivate the use o a Gaussia approximatio. Cosider that r i χˆ = χˆ + jχˆ is a elemet i ˆχ that has bee uatized to oe bit. Usig the ideas o [] gives that the FI o this uatized coeiciet ca be computed accordig to

5 4 ± = + = ˆ ˆ ˆ ˆ ; ; ; ; ˆ χ χ χ χ χ J i i r r 4 where the superscript idicates that this is or the -bit case ad where dx x dx x χ χ i r = = ˆ ˆ } Im{ exp ; } Re{ exp ; σ ξ π σ ξ π 5 with ξ beig the oise-ree part o the uuatized coeiciet χ. For the multi-bit case, i } var{ } var{ ε ω <<, ν i 3 is approximately Gaussia, which ollows rom the relatioship betwee their characteristic uctios: C C C ε ω ν =. Namely, i } var{ } var{ ε ω <<, the sic uctio C ε is approximately lat where the Gaussia C ω is sigiicat. Numerical results give i the Appedix idicate that the approximatio is good at low SNR values eve or bits but gets worse at higher SNR values; ote that our approximatio is best at low SNR, where every ouce couts. The results or the FI or the complex Gaussia case see 5.5 i [] give ˆ m χ J + = σ ξ. 6 where m idicates multi-bit ad is the variace o the th elemet o the uatizatio oise. Establishig the exact orm or is geerally ot possible ad it is commo to use a approximate model [9]. A widely used model is: b X Var C =, 7 where C is a costat ote determied heuristically that depeds o the PDF o the radom variable X [4]. We use small blocs o coeiciets to estimate VarX.

6 5 I the operatioal settig we evaluate 4 ad 6 usig the oisy coeiciets i ˆχ. Thus the distortio uctio we use o the th coeiciet uatized to b bits is Jˆ, = J J m ˆ χ, ˆ χ, i b i b i b = =, 8 ad we see a bit allocatio set B = { b,, K, N} that solves N N max Jˆ subject to b R. 9 B = = Due to their eiciecy, Lagragia optimizatio methods are commoly used to determie the bit allocatio i operatioal rate-distortio methods [9], [3] with a additive objective uctio lie that i 9. We use the algorithm developed i [3]. A advatage o our approach is that oce the uctioal orm o the FI is oud, it geerally provides isight ito the choice o a the trasorm. For example, as see i Sectio III whe the parameter to be estimated is the delay betwee two sigals, the FI depeds o a speciic badwidth measure, so a reuecy domai trasorm maes sese i that case. We are particularly iterested i problems where data must be shared betwee sesors because either sesor ca estimate the parameter by itsel. We call these types o problems dualsesor-critical problems. Such problems ote arise i passive systems due to lac o owledge about a trasmitted sigal that has bee perturbed by some parameter. It is importat to eep i mid that the real essece o FI is that it captures the sesitivity o the data to a chage i the parameter; this is due to the derivative i the deiitio o FI i. Thus, i a passive sesor settig the data at oe sesor may be very sesitive to the parameter i.e., the FI is o-zero yet you may still be uable to estimate the value o the parameter. But with two sets o data each havig a dieret parameter value you ca estimate the dierece i the two parameter values. For example, the data ca be sesitive to a chage i time delay but without a reerece it is impossible

7 6 to estimate the delay. What maes a active sesor problem dieret is that you have a ow reerece sigal: estimatio o the parameter s chage rom the reerece sigal leads to estimatio o the actual value. I a passive sesor sceario you ote eed the data at the other sesor to play the role o the reerece. Thus, to mae our approach wor we oly eed that the data at a sesor is sesitive to the parameter, ad that is what the Fisher iormatio i 8 assesses. III. Example Applicatios We choose two examples TDOA ad FDOA estimatio [] ad illustrate i the eectiveess o our method relative to MSE-drive methods, ad ii how the orm o the FI drives the choice o the trasorm. We use cross correlatio to estimate TDOA ad FDOA [4]. The sigal we use is a complex basebad FM sigal with a pseudo-radom modulatig sigal; a sample spectrum is show i Figure. For each SNR ad compressio ratio CR o iterest, we evaluated the estimatio error over 4 Mote Carlo rus. We heuristically chose the value o C i 7 to be C = π 3, which gives good results i our simulatios. To ocus o the capabilities o the trasorm codig we perormed o etropy codig, which would liely provide urther improvemet i the CR with o urther accuracy degradatio. Figure : The spectrum o a typical FM sigal used i the simulatios.

8 7 A. Compressio or TDOA Estimatio It is well ow [4] that or TDOA estimatio, the FI is proportioal to S d, where S is the Fourier trasorm o the source sigal. This view drives us to choose the DFT as our trasorm to allow discardig less useul reuecy compoets. The cotiuous-time sigal model or two passively-received complex basebad sigals havig a uow TDOA o is give by x t = s t t x t = s t t + / + w t / + w t where t is also a uow parameter that ca ot be estimated, ad w i t is complex badlimited white Gaussia oise. I the reuecy domai this model becomes X ω = S ω e X ω = S ω e jω t + / jω t / + W ω + W ω Now cosider samples o such that the oise samples give discrete-time white oise with i variaces σ ad the aliasig o sigal s t t + / is egligible. Taig the DFT o these samples taig care to reduce leaage errors leads to the reuecy domai model π X[ ] S[ ]exp[ j t + / ] + W[ ] = N /, N / +, K, N /, N where the S[] are the DFT coeiciets or egative ad positive reuecies o the samples o sigal st ad W [] are the DFT coeiciets o the oise. This is clearly a dual-sesor-critical problem. Each data set is sesitive to chages i the time-o-arrival t + / but either sesor by itsel ca estimate t + /, t aloe, or aloe. I act, eve usig both data sets it is impossible to estimate the uisace parameter t. Al-

9 8 though this appears to be a two-parameter problem that might reuire a -D FI matrix, there really is oly oe thig that matters: sesitivity o the irst sesor s data to or a ixed t. Note that because the DFT is a orthogoal, but ot orthoormal, trasorm the DFT oise variace is N σ. Usig 6, the FI ater multi-bit uatizatio becomes Jˆ m π X [ ] =, = N /, N / +, K, N / Nσ + X [ ]. 3 This shows that the DFT coeiciets get uadratically weighted by reuecy. For the -bit result applicatio o 4 ad 5 alog the lies o those i [] gives Jˆ X [ ] 4 = πσ Im { X [ ]}exp Re { [ ]}} X Nσ er Re { X[ ]}exp Nσ Im { X[ ]}} er Im{ X [ ]}/ Re{ X [ ]}/ Nσ + Nσ 4 Usig 3 ad 4 i 9 with the Lagrage optimizatio method produces our results. Figure ad Figure 3 show or CRs o 4: ad 8:, respectively the TDOA accuracy perormace o our method labeled Fisher vs. the perormace o stadard MSE-optimum DFT-based trasorm compressio labeled MSE as the SNR at sesor S is varied; the value o SNR at sesor S is ixed at 4 db. We have icluded the case where just the sigal at S is compressed labeled S as well as the case where the sigals at S ad S were both compressed labeled S&S. The perormace with o compressio is labeled w/o comp. Figure 4 ad Figure 5 show results or the case o both SNR s chagig but set eual to each other. I all cases show, our method provides better TDOA accuracy tha the MSE-optimized method; at moderately high SNR our method is early the same as whe o compressio is used eve whe the CR is 8:. Also ote that whe both sesor s sigals have bee compressed at 8:, our method s perormace is oly degraded a small amout where as the perormace o the MSE-based method is severely degraded.

10 9 RMS TDOA Error s TDOA Perormace SNR =4 db CR=4: w/o comp MSE S MSE S&S Fisher S Fisher S&S SNR db Figure : TDOA accuracy vs SNR o pre-compressed sesor S sigal or a CR o 4:; the SNR o the sesor S sigal was 4 db. 4 TDOA Perormace SNR = 4 db CR=8: RMS TDOA Error s w/o comp MSE S MSE S&S Fisher S Fisher S&S SNR db Figure 3: TDOA accuracy vs SNR o pre-compressed sesor S sigal or a CR o 8:; the SNR o the sesor S sigal was 4 db.

11 RMS TDOA Error s TDOA Perormace SNR = SNR CR=4: w/o comp MSE S MSE S&S Fisher S Fisher S&S SNR, SNR db Figure 4: TDOA accuracy vs SNR o pre-compressed sesor S sigal or a CR o 4:; the SNR o the sesor S sigal was set eual to SNR. 5 TDOA Perormace SNR = SNR CR=8: 5 RMS TDOA Error s 75 5 w/o comp MSE S MSE S&S Fisher S Fisher S&S SNR, SNR db Figure 5: TDOA accuracy vs SNR o pre-compressed sesor S sigal or a CR o 8:; the SNR o the sesor S sigal was set eual to SNR

12 B. Compressio or FDOA Estimatio It is well ow [4] that or FDOA estimatio, the FI is proportioal to t s t dt. This view drives us to choose the idetity trasorm which is a ON trasorm to provide the ability to discard time compoets that cotribute little to the FI. Thus we will directly uatize the complex-valued sigal samples, usig idividual uatizers or the real ad imagiary parts. The model or two passively-received sigals havig a uow FDOA o is give by x [ ] = s[ ] e x [ ] = s[ ] e j ν + / j ν / + w [ ] + w [ ] = N = N /, /, N N / / +, K, N +, K, N / / 5 where v is a uow uisace parameter that ca ot be estimated, ad w i [] is complex Gaus- sia oise with variace o σ i, with σ assumed ow. The model i 5 is mathematically idetical to the TDOA model or the DFT trasorm i ad thereore we ca use the previous results to immediately state that ater multi-bit uatizatio the per-sample FI becomes Jˆ m π x [ ] x [ ] =, = N /, N / +, K, N / σ + For the -bit result applicatio o 4 ad 5 alog the lies o those i [] gives. 6 Jˆ x [ ] 4 = πσ Im { x [ ]}exp Re { [ ]}} x σ er Re { x[ ]}exp σ Im { x[ ]}} er + Re{ x [ ]}/ σ Im{ x [ ]}/ σ 7 Usig these results i 9 with the Lagrage optimizatio method produces the results show i Figure 6 ad Figure 7, which show results or the FDOA case. I all cases show, our method provides better FDOA accuracy tha the MSE-optimized method.

13 RMS FDOA Error mhz FDOA Perormace SNR = 4 db CR=4: w/o comp MSE S MSE S&S Fisher S Fisher S&S SNR db Figure 6: FDOA accuracy vs SNR o pre-compressed sesor S sigal or a CR o 4:; the SNR o the sesor S sigal was 4 db TDOA Perormace SNR = 4 db CR=8: RMS FDOA Error mhz 5 5 w/o comp MSE S MSE S&S Fisher S Fisher S&S SNR db Figure 7: FDOA accuracy vs SNR o pre-compressed sesor S sigal or a CR o 8:; the SNR o the sesor S sigal was 4 db

14 3 IV. Cocludig Remars As demostrated i the example applicatios, the use o a distortio measure desiged speciically or a speciic estimatio problem ca lead to compressio methods that outperorm those usig MSE-based distortio measures, especially whe both sesor sigals eed to be compressed at high CRs. While MSE distortio accurately captures the eect o the compressio o the compressed sigal s SNR, it ails to capture the true impact o compressio o the estimatio accuracy. This is similar to the sceario i image ad audio compressio, where MSE distortio ails to capture the impact o compressio perceptual uality o the compressed data. I those areas researchers have proposed eective distortio measures based o the psychology o perceptio. O course, others have used such ierece-cetric distortio measures beore or uatizer desig; but here we see the power o combiig this with trasorm codig, which leads to some ew isights: i the structure o the FI provides isight ito the proper choice o trasorm choice, ii the choice o trasorm ad the optimal bit allocatio ca be i colict or dieret parameter estimatios this is importat as we exted to the case o multiple estimatios, ad iii it is possible to optimize FI-based measures withi a speciied operatioal compressio ramewor. There are some directios or which urther wor is eeded: i extesio to the case o multiple estimates ad decisios although we have some prelimiary results i this area [3], ii examiatio o the computatioal ad implemetatio aspects; particularly, better models or the multi-bit post-compressio FI, ad iii geeralizatio to the case whe the FI depeds o the parameter ; at the ed o Sectio III-A we have proposed what we thi is the correct approach but have ot yet ully explored its applicatio.

15 4 Appedix: Numerical Results or Gaussia Approximatio We assume that we have a set o oise-ree coeiciets that lie i the rage ±A. A oisy versio o the coeiciets havig additive Gaussia oise o variace σ is uatized to b bits usig a mid-step uiorm uatizer with uatizatio cell size give by = + σ BA. / The true PDF ca be umerically oud via covolutio o a Gaussia PDF with a uiorm PDF ad the plotted or various values o the pea SNR PSNR to be PSNR = A /σ. Two such plots are give i Fig. A- or the case o bit uatizatio at two values o PSNR. The results provide motivatio that at least or low PSNR the approximatio seems to be valid eve dow to the lowest umber o bits or which it is applied. For higher PSNR the approximatios at bits will be poorer; oetheless, we use the approximatio. For sesor problems the iterest geerally lies at low SNR ad it is good that we have a better approximatio i that rage. True PDF Approx PDF.8 True PDF Approx PDF.6.8 PSNR = db.4. PSNR = db p v x.6.4 p v x x x Figure A - Numerical results or the true PDF compared to the approximatio PDF or the case o bit uatizatio or PSNR o db ad db.

16 5 Reereces [] D. Torrieri, Statistical Theory o Passive Locatio Systems, IEEE Tras. o Aerospace ad Electroic Systems, pp , March 984. [] S. Tila, N. B. Abu-Ghazaleh, ad W. Heizelma, A taxoomy o wireless microsesor etwor models, ACM Mobile Computig ad Commuicatios Review MCR, vol. 6, pp. 8, April. [3] M. Che ad M. L. Fowler, Geometry-Adaptive Data Compressio For TDOA/FDOA Locatio, IEEE ICASSP 5, Philadelphia, PA, pp. IV69 IV7, March 8 3, 5. [4] S. Stei, Dieretial Delay/Doppler ML Estimatio with Uow Sigals, IEEE Tras. o Sigal Processig, pp , August 993. [5] D. J. Matthiese ad G. D. Miller, Data traser miimizatio or coheret passive locatio systems, Report No. ESD-TR-8-9, Air Force Project No. 4, Jue 98. [6] G. Desjardis, TDOA/FDOA techiue or locatig a trasmitter, US Patet #5,57,99 issued Oct. 9, 996, Locheed Marti Federal Systems. [7] M. L. Fowler, Coarse uatizatio or data compressio i coheret locatio systems, IEEE Tras. Aero. ad Electr. Systems, vol. 36, o. 4, pp , Oct.. [8] T. M. Cover ad J. A. Thomas, Elemets o Iormatio Theory. New Yor: Wiley, 99. [9] A. Ortega ad K. Ramchadra, Rate distortio methods or image ad video compressio, IEEE Sigal Processig Magazie, vol. 5, Nov. 998, pp [] S. Kay, Fudametals o Statistical Sigal Processig: Estimatio Theory, Eglewood Clis, NJ: Pretice Hall, 993. [] M. Di Bisceglie ad M. Logo, Decetralized ecodig o a remote source, Sigal Processig, , pp [] T. J. Fly ad R. M. Gray, Ecodig o correlated observatios, IEEE Trasactios o Iormatio Theory, vol. IT-33, o. 6, Nov. 987, pp [3] J. A. Guber, Distributed estimatio ad uatizatio, IEEE Trasactios o Iormatio Theory, vol. 39, o. 4, July 993, pp [4] T. S. Ha ad S. Amari, Parameter estimatio with multitermial data compressio, IEEE Trasactios o Iormatio Theory, vol. 4, o. 6, Nov. 995, pp [5] H. V. Poor, Fie Quatizatio i Sigal Detectio ad Estimatio, IEEE Trasactios o Iormatio Theory, vol. 34, o. 5, Sept. 988, pp

17 6 [6] W. Lam ad A. R. Reibma, Desig o uatizers or decetralized estimatio systems, IEEE Trasactios o Commuicatios, vol. 4, o., Nov. 993, pp [7] S. S. Pradha, J. Kusuma, ad K. Ramchadra, Distributed compressio i a dese microsesor etwor, IEEE Sigal Processig Magazie, pp. 5 6, March. [8] A. Scaglioe ad S. Servetto, O the iterdepedece o routig ad data compressio i multi-hop sesor etwors, MOBICOM, Sept. 3 6,, Atlata, Ga. [9] Z. Zhag ad T. Berger, Estimatio via compressed iormatio, IEEE Trasactios o Iormatio Theory, vol. 34, o., March 988, pp. 98. [] L. Vasudeva, A. Ortega, U. Mitra, Applicatio speciic compressio or time delay estimatio i sesor etwors, Proceedigs o the First ACM Iteratioal Coerece o Embedded Networed Sesor Systems, Nov. 3, Los Ageles, CA, pp [] R. M. Gray ad T. G. Stocham, Jr., Dithered Quatizers, IEEE Tras. o Iorm. Theory, vol. 39, pp. 85-8, May 993. [] A. Høst-Madse ad P. Hädel, Eects o Samplig ad Quatizatio o Sigle-Toe Freuecy Estimatio, IEEE Tras. Sigal Processig, Vol. 48, No. 3, pp , March. [3] Y. Shoham ad A. Gersho, Eiciet bit allocatio or a arbitrary set o uatizers, IEEE Tras. o Acoustics, Speech, ad Sigal Processig, 369: , 988. [4] R. Gray ad D. Neuho, Quatizatio, IEEE Tras. Iormatio Theory, Vol. 44, No. 6, pp. 63, October 998.