Gas Plume Detection and Tracking in Hyperspectral Video Sequences using Binary Partition Trees

Transcription

1 Gas Plume Deecion and Tracking in Hyperspecral Video Sequences using Binary Pariion Trees Guillaume Tochon, Jocelyn Chanusso, Jérôme Gilles, Mauro Dalla Mura, Jen-Mei Chang, Andrea Berozzi To cie his version: Guillaume Tochon, Jocelyn Chanusso, Jérôme Gilles, Mauro Dalla Mura, Jen-Mei Chang, e al.. Gas Plume Deecion and Tracking in Hyperspecral Video Sequences using Binary Pariion Trees. IEEE Workshop on Hyperspecral Image and Signal Processing: Evoluion in Remoe Sensing (WHISPERS 2014), Jun 2014, Lausanne, Swizerland. <hal > HAL Id: hal hp://hal.univ-grenoble-alpes.fr/hal Submied on 28 Jul 2014 HAL is a muli-disciplinary open access archive for he deposi and disseminaion of scienific research documens, wheher hey are published or no. The documens may come from eaching and research insiuions in France or abroad, or from public or privae research ceners. L archive ouvere pluridisciplinaire HAL, es desinée au dépô e à la diffusion de documens scienifiques de niveau recherche, publiés ou non, émanan des éablissemens d enseignemen e de recherche français ou érangers, des laboraoires publics ou privés.

2 GAS PLUME DETECTION AND TRACKING IN HYPERSPECTRAL VIDEO SEQUENCES USING BINARY PARTITION TREES G. Tochon 1, J. Chanusso 1,4, J. Gilles 2, M. Dalla Mura 1, J.-M. Chang 3, A. L. Berozzi 2 1 GIPSA-lab, Grenoble Insiue of Technology, Sain Marin d Hères, France 2 Deparmen of Mahemaics, Universiy of California Los Angeles, Los Angeles, USA 3 Deparmen of Mahemaics and Saisics, California Sae Universiy, Long Beach, USA 4 Deparmen of Elecrical and Compuer Engeneering, Universiy of Iceland, Reykjavik, Iceland ABSTRACT Thanks o he fas developmen of sensors, i is now possible o acquire sequences of hyperspecral images. Those hyperspecral video sequences are paricularly suied for he deecion and racking of chemical gas plumes. However, he processing of his new ype of video sequences wih he addiional specral diversiy, is challenging and requires he design of advanced image processing algorihms. In his paper, we presen a novel mehod for he segmenaion and racking of a chemical gas plume diffusing in he amosphere, recorded in a hyperspecral video sequence. In he proposed framework, he posiion of he plume is firs esimaed, using he emporal redundancy of wo consecuive frames. Second, a Binary Pariion Tree is buil and pruned according o he previous esimae, in order o rerieve he real locaion and exen of he plume in he frame. The proposed mehod is validaed on a real hyperspecral video sequence and compared wih a sae-of-he-ar mehod. Index Terms segmenaion, racking, Binary Pariion Tree, chemical gas plume, hyperspecral video sequence 1. INTRODUCTION The deecion and racking of chemical gas plumes in he amosphere is of grea ineres for several domains [1, 2]. In he defense and securiy area for example, such analysis could be employed in order o deec he use of chemical gas weapons. In he environmenal proecion field, he deecion and racking of gas plumes could be also of use o idenify and repair gas leaks in order o minimize heir impac on he environmen and he poenial harm hey could cause on human populaions. However, his ask sill remains an open research opic as mos gases do no appear in he visible specrum and hence remain invisible o human inspecion or radiional color imaging sysems. As a maer of fac, heir specral signaure significanly responds only in a resrained porion of he infrared (IR) domain [3, 4], hence he need for a fine sampling of he elecromagneic specrum. Addiionally, he emporal dimensionaliy inheren o video sequences requires appropriae processings. Hyperspecral video sensors combine he abiliy o precisely describe specral properies of he capured scene and o record is evoluion over ime, bu a he cos of an imporan amoun of daa o process [5 8]. In his paper, we propose a new mehod o process a hyperspecral video sequence for he deecion and racking of a chemical gas plume diffusing in he amosphere. This mehod relies boh on specral properies of he hyperspecral scene and on he emporal redundancy of This maerial is based upon work suppored by he Naional Science Foundaion under gran no. DMS and no. DMS he video sequence. A rough esimae of he posiion of he plume in he frame is firs compued. The acual posiion and exen of he plume is hen rerieved using a Binary Pariion Tree. The remainder of he paper is organized as follows: secion 2 inroduces he Binary Pariion Tree (BPT) algorihm ha consiues he core of he proposed mehod. Secion 3 furher deails he proposed racking and segmenaion algorihm. Secion 4 displays some resuls obained on a real sequence and feaures some comparisons. Conclusions are given in secion BINARY PARTITION TREE (BPT) The BPT is a hierarchical region-based represenaion of an image sored in a ree srucure [9,10]. Saring from an iniial pariion of he image, regions are ieraively merged unil only one region remains, corresponding o he whole image suppor. The merging sequence is sored in a ree srucure T. In his represenaion, regions from he iniial pariion form he leaf nodes, he whole image represens he roo, and each node inbeween corresponds o a region resuling from he merging of is wo children. There are wo noions of primary imporance when building a BPT. The region model M R describes how regions are represened mahemaically and how o model he merging of wo regions. The merging crierion O(R i,r j) is a measure assessing he similariy beween wo neighboring regionsr i andr j by measuring he disance beween heir region models. The merging crierion hus deermines he sequence in which he regions are merged. The pruning sep follows he consrucion of he BPT. I aims a cuing off some branches in he BPT so he leaves of he pruned ree correspond o regions achieving he bes segmenaion wih respec o he desired ask. Unlike he consrucion of he BPT, which is generic up o he definiion of he region model and merging crierion, he pruning sep is applicaion dependen, and differen pruning sraegies applied on he same BPT generally leads o differen segmenaion resuls [11]. 3. PROPOSED METHOD 3.1. Daa se and pre-processing The daa se used in his sudy was acquired and provided by he John Hopkins Applied Physics Laboraory. The specral radiance of he scene was recorded by a long wave IR specromeer, abou 2 kilomeers away from he gas release, producing a hyperspecral video sequence a a frame rae of 0.2 Hz. Each frame of he sequence is herefore a hyperspecral image of size pixels and

3 {I } N =1 Pre-processing { } N =1 { } N =1 I 1 1 Deecion ˆP Esimaion P Fig. 1: General framework of he proposed mehod. RGB GRAY I diff comprising 129 specral bands corresponding o wavelenghs evenly disribued beween 7830 nm and nm. Le {I } N =1 denoe he hyperspecral video sequence, N being he oal number of frames. In his sudy, N = 23. An iniial pre-processing is applied o he whole sequence. I comprises a Principal Componen Analysis (PCA) done on each frame, where he hree firs Principal Componens (PCs) are reained, followed by a Midway equalizaion [12] o ease he visualizaion of he daa, as deailed in [5]. The oupu of he preprocessing sep is he false color represenaion sequence labeled { } N =1. Figures 3a and 3b display wo consecuive frames of he false color represenaion sequence. The proposed mehod is organized in wo seps: - The esimaion of he posiion of he plume in he curren frame by aking advanage of he emporal redundancy inheren o he video sequence. - The validaion and refinemen of he previous esimae using he BPT. Figure 1 illusraes he proposed workflow. Please noe ha he preprocessing is only used o provide a rough esimae of he posiion of he plume. The acual segmenaion is performed using he iniial full hyperspecral frame. For each inpu frame I, he oupu of he proposed algorihm is he binary mask P feauring he posiion of he plume in he curren frame Esimaion sep The goal of he esimaion sep is o produce a reliable esimae of he posiion of he plume and use his esimae as a priori informaion when pruning he BPT. Noe ha only he false color represenaion video sequence is considered a his sage of he algorihm. The whole esimaion process is feaured by he workflow in figure 2, and is based on he emporal redundancy beween consecuive frames. More specifically, i is assumed ha only he plume is moving beween wo consecuive frames, and he background does no change. Consequenly, he image difference beween wo consecuive frames is expeced o conain low values in areas ha do no feaure any change beween he wo frames, and higher values when significan change occurs. The firs sep of he esimaion sage is o idenify hose areas. This operaion is illusraed in figure 2. More precisely: 1. Inpu frames are iniially very noisy as i can be seen in figures 3a and 3b. Therefore, a preliminary denoising is applied on boh images. This is achieved using he Block Maching 3D algorihm () [13]. k-means C ˆP = P 1 C Fig. 2: Workflow of he esimaion sep. 2. The image difference is hen compued, denoised again o remove residual noise, and convered in grayscale o produce he image labeledi diff (see figure 3c). The underlying idea is o consider I diff as a opographical image where peaks (brigh regions) correspond o areas ha are significanly changing beween he wo consecuive frames. Those areas correspond o regions ha are eiher invaded or lef by he plume. 3. Peaks are exraced by hresholding he opographical image wih an auomaically se hreshold value defined by a woclass K-means algorihm, producing he binary image C displayed in figure 3d. The las sep of he esimaion sage is o produce he esimae posiion of he plume. I can be done combining he posiion of he plume deeced in he previous frame P 1 and he curren change map since he new posiion corresponds o he previous one plus he region ha have been invaded, minus hose ha have been lef. This can be mahemaically formulaed ˆP = P 1 C (1) where denoes he binary XOR operaion, hus producing he esimae ˆP Deecion sep The second sep of he proposed mehod is he acual deecion of he plume P in he curren frame, using he previously compued esimae ˆP as some a priori knowledge. The deecion is handled hrough he consrucion and pruning of a BPT, as shown by he workflow feaured on figure 4.

4 (a) (b) where q sands for he number of specral band in he frame, R is he number of pixels in region R, and p (i) is he value in hei-h band a pixel locaionp. - The merging crierion beween wo neighboring regions was defined as he Specral Angle beween heir region models: ( ) ri, r j O(R i,r j) = arccos. (4) r i 2 r j 2 (c) Fig. 3: Illusraion of he esimaion process: (a,b) wo consecuive noisy frames along wih (c) heir image difference I diff and (d) he resuling binary map. {I } N =1 I BPT consrucion T BPT pruning P M R (d) segmap O(R i,r j ) Fig. 4: Workflow of he deecion sep Consrucion of he BPT As menionned in secion 2, he consrucion of he BPT asks for hree inpu parameers: - The definiion of he iniial leaves: saring from he pixel level generaes = leaves. This considerably impacs he compuaional load and leads o a high number of non-signfican nodes in he BPT. On he conrary, saring wih leaves corresponding o he regions of a preliminary rough segmenaion significanly reduces he number of final nodes in he ree and hence he compuaional load, while no impacing he final segmenaion resuls. The only required condiion is o sar from an over-segmenaion, as iniial regions will no be allowed o spli in he furher seps. This is easily achieved by using a waershed segmenaion. The exension of he waershed o hyperspecral images presened in [14] has been seleced. - The region model, which describes how regions are mahemaically represened. For his work, he mean specrum region model has been implemened: [ M R = r = r (1),..., r (q)] (2) wih ˆP r (i) = 1 p (i), (3) R p R Pruning of he BPT As he plume is a hin layer overlaying he background, he specral response of pixels belonging o he plume only differs slighly from pixels behind he plume (be i ground or sky). However, he BPT is able o capure he plume as one single region while i has no oo much diffused ye, or wo differen regions (one being he boom half of he plume which overlays wih he ground, and he oher one being he op half superimposed on he sky). The goal of he pruning sep is o idenify in he ree srucure which node or se of nodes corresponds o he plume. Therefore, he implemened pruning sraegy is based on he esimaed posiion ˆP and seeks he bes node or se of nodes maching his esimae. The maching crierion is defined as follows: a se of nodes{n 1,...,N m} and is corresponding regions {R 1,...,R m} is said o mach he esimae ˆP if he area covered by all he regions overlaps wih a leas γ% of ˆP, and if each region independenly has a leas half of is pixels belonging o ˆP. Mahemaically,{R 1,...,R m} maches ˆP if: m ( R i) ˆP γ ˆP (5) i=1 R i ˆP R i\ˆp i = 1,...,m (6) The parameer γ represens he confidence in he esimae and was empirically se o 70%. This value achieves a rade-off beween rus and misrus in he esimae. To reconsruc he plume wih as few regions as possible (ideally only one), he reained pruning is he one leading o he ree wih he smalles number of nodes. If several regions were found, hey are fused ogeher in a las sep o obain he final binary maskp. 4. RESULTS Figures 5a o 5d display segmenaion resuls obained by he presened mehod for he second, sixh, enh and foureenh frames afer he plume appearance,respecively. Figures 5e o 5h exhibi segmenaion resuls on he same frames obained by he sae-of-ar mehod [5], based on he Merriman-Bence-Osher (MBO) semi-supervised clusering scheme [15]. I is worh menioning ha he MBO uilizes only he firs five principal componens while our mehod uses he whole hyperspecral daa. I can be seen how boh mehods accuraely deec and segmen he plume from he background, and rack i along he frames. However, he MBO mehod produces some small false deecion areas a he inerface beween he ground and he sky. This issue does no arise in our proposed mehod. Moreover, a small cloud of dus, riggered by he explosive release of he plume, can be seen near he boom lef par of he plume in he wo middle frames of figure 5. Our proposed mehod is able o correcly differeniae i from he gas plume while i is included wihin he plume region for he MBO resuls. Noe ha [5] uilizes he oupu of a background subsracion sep o iniialize he MBO scheme while he esimaion process of our mehod only sars when he plume appears in he video sequence. In boh cases, he exac appearance ime of he plume in he video sequence mus be known in order o rigger he deecion process.

5 (a) (b) (c) (d) (e) (f) (g) (h) Fig. 5: Top row: segmenaion resuls obained by he presened mehod for four frames of he video sequence. Boom row: segmenaion resuls obained by [5] for he same frames. Resuls are displayed on he false color represenaion video sequence. 5. CONCLUSION In his aricle, we presened a novel algorihm for he deecion and racking of chemical gas plume in a hyperspecral video sequence. The proposed mehod is organized in wo sages being he esimaion of he posiion of he plume in he curren frame and he deecion of he real plume, which relies on he previous esimae. While he firs sep is based on he emporal redundancy inheren o video sequences, he second one involves he consrucion and pruning of a Binary Pariion Tree. The proposed algorihm gives saisfacory visual resuls for he presened video sequence. Fuure work includes he design of a mehod o quaniaively assess he qualiy of he obained segmenaion and racking despie he lack of ground-ruh daa. The use of anomaly deecion echnics o blindly deec he release insan of he plume will also be inversigaed in order o make he proposed mehod fully unsupervised. 6. REFERENCES [1] V. Farley, A. Vallires, A. Villemaire, M. Chamberland, P. Lagueux, and J. Giroux, Chemical agen deecion and idenificaion wih a hyperspecral imaging infrared sensor, in SPIE Defense, Securiy, and Sensing, 2007, vol. 6739, pp [2] M. Hinnrichs, Imaging specromeer for fugiive gas leak deecion, in SPIE Defense, Securiy, and Sensing, 1999, vol. 3853, pp [3] J. B. Broadwaer, T. S. Spisz, and A. K. Carr, Deecion of gas plumes in cluered environmens using long-wave infrared hyperspecral sensors, in SPIE Defense and Securiy Symposium. Inernaional Sociey for Opics and Phoonics, 2008, pp R 69540R. [4] A. Vallires, A. Villemaire, M. Chamberland, L. Belhumeur, V. Farley, J. Giroux, and J-F. Legaul, Algorihms for chemical deecion, idenificaion and quanificaion for hermal hyperspecral imagers, in SPIE Defense, Securiy, and Sensing, 2005, vol. 5995, pp G 59950G 11. [5] T. Gerhar, J. Sunu, L. Lieu, E. Merkurjev, J-M. Chang, J. Gilles, and A. L. Berozzi, Deecion and racking of gas plumes in LWIR hyperspecral video sequence daa, in SPIE Defense, Securiy, and Sensing. Inernaional Sociey for Opics and Phoonics, 2013, pp J 87430J. [6] A. Banerjee, P. Burlina, and J. B. Broadwaer, Hyperspecral video for illuminaion-invarian racking, in Hyperspecral Image and Signal Processing: Evoluion in Remoe Sensing, WHISPERS 09. Firs Workshop on, Augus 2009, pp [7] H. Van Nguyen, A. Banerjee, and R. Chellappa, Tracking via objec reflecance using a hyperspecral video camera, in Compuer Vision and Paern Recogniion Workshops (CVPRW), 2010 IEEE Compuer Sociey Conference on. IEEE, 2010, pp [8] C. S. Gran, T. K. Moon, J. H. Gunher, M. R. Sies, and G. P. Williams, Deecion of amorphously shaped objecs using spaial informaion deecion enhancemen (SIDE), Seleced Topics in Applied Earh Observaions and Remoe Sensing, IEEE Journal of, vol. 5, no. 2, pp , April [9] P. Salembier and L. Garrido, Binary pariion ree as an efficien represenaion for image processing, segmenaion, and informaion rerieval, Image Processing, IEEE Transacions on, vol. 9, no. 4, pp , [10] S. Valero, P. Salembier, and J. Chanusso, Hyperspecral image represenaion and processing wih binary pariion rees, Image Processing, IEEE Transacions on, vol. 22, no. 4, pp , [11] S. Valero, P. Salembier, and J. Chanusso, Comparison of merging orders and pruning sraegies for binary pariion ree in hyperspecral daa, in Image Processing (ICIP), h IEEE Inernaional Conference on. IEEE, 2010, pp [12] J. Delon, Midway image equalizaion, Journal of Mahemaical Imaging and Vision, vol. 21, no. 2, pp , [13] K. Dabov, A. Foi, V. Kakovnik, K. Egiazarian, e al., Image denoising wih block-maching and 3D filering, in Proceedings of SPIE, 2006, vol. 6064, pp [14] Y. Tarabalka, J. Chanusso, and J. A. Benediksson, Segmenaion and classificaion of hyperspecral images using waershed ransformaion, Paern Recogniion, vol. 43, no. 7, pp , [15] E. Merkurjev, T. Kosic, and A. L. Berozzi, An MBO scheme on graphs for segmenaion and image processing, UCLA CAM Repor, pp , 2012.