Efficient Virtual Slide Telepathology Systems with JPEG2000

Transcription

1 Efficient Vitual Slie Telepathology Systems with JPEG2000 J.P. Otiz, V.G. Ruiz, an I. Gacia Univesity of Almeia, Spain ABSTRACT In the pesent pape we analyze the JPEG2000 stana an how to exploit all the offee poweful featues to buil efficient telepathology systems base on vitual slies. We also popose a fast metho fo stitching in the wavelet omain. Stitching is a pocess necessay in many vitual slie systems to geneate only one compose image of a slie with a high esolution. The popose metho pofits fom the stuctue of the JPEG2000 images to cay out the pocess with the minimum memoy consumption an computational loa, obtaining a smoothe union without losing any etail. INTRODUCTION Telepathology is the pactice of the pathology at istance. A pathologist can make iagnostics by obseving though a communication link the meical slies captue by a emote micoscope. This way the localization of the pathologist an the meical slies ae unboun. Fo example, iagnostics of pathologists fom iffeent centes can be compae an shae. Thee ae iffeent kins of telepathology systems, all of them commente in []. The fist geneation of these systems consists of the acquisition of the micoscope image, eithe in eal-time moe o in stoe-an-fowa moe, an its sening to the pathologist. This can emotely manipulate the micoscope to select the slie egion to visualize, equiing a new acquisition wheneve this egion changes. In these systems the opeato is always boun to the micoscope, so any session equies using in an exclusive way this evice. The use of the evice is clealy inefficient an eunant. The secon geneation of telepathology systems intouces the concept of vitual slie. Vitual slies ae igitizations of complete slies, that ae stoe in a seve fo its late emotely inspection. A client/seve achitectue allows pathologists to navigate emotely though the vitual slies, simulating the woking of a micoscope. Diffeent egions of the vitual slie can be obseve, allowing to incease o ecease the zoom, without having to ownloa the entie emote image. The seve sens only the equie ata to see. Pathologists have in this way the sensation of woking with a vitual emote micoscope, with the iffeence that, in this case, the igitization is mae in a pevious step, of the whole slie. It is not necessay to euse the micoscope fo the same slie. In [2] it is popose a possible implementation of a system with these chaacteistics. To obtain a igitization of the entie slie with the enough quality fo a coect iagnostic, it must have, among othe featues, the appopiate esolution. Complying with this pemise is not always possible with the available acquisition evice. In many cases its esolution is not enough to cove the entie slie in oe to igitize without losing etail. In these cases the slie is ivie in contiguous egions in oe that each egion is igitize with the maximum esolution of the evice. In a posteio step all the esulting images ae joine in a stitching pocess, geneating only one big image with the necessay etail. The JPEG2000 still image compession stana is being impose in the context of imaging meical systems, ue to its poweful featues. Telepathology systems ae being favoue with the chaacteistics of this stana. Fo example, in [3] a new metho fo emosaicking micoscope images base on this stana is popose, showing how to integate it within a teleiagnosis system. The quality of the image compesse with JPEG2000 fo the same bit-ate is quite highe than that offee by its peecesso, the JPEG stana, as it can be obseve in [4]. This featue is essential in systems base on vitual slies. JPEG2000 offes a high scalability, allowing a anom access to the compesse bit-steam of the image, as well as its tansmission by means of a quality pogession. These featues ae specially inteesting in the vitual slies context, allowing the seve to extact egions of the images without ecoing, an sening them aapting the communication to the available banwith. The JPIP potocol is efine within the same stana, esigne to exploit at the maximum the featues of this compession stana in emote bowsing applications, as fo example, telepathology vitual slie systems. In the pesent pape the main featues of the JPEG2000 stana ae analyze, as well as the JPIP potocol, showing how they can be use to buil efficient telepathology vitual slie systems. It is also popose a fast an simple metho to constuct vitual slies base on the stitching in the omain of the wavelet tansfom. With this metho the necessay time an memoy fo the pocess ae consieably euce. Moeove, the image bounaies of the final montage appea quite smoothe without any aitional pocessing o quality lost.

2 JPEG2000 FOR VIRTUAL SLIDES JPEG2000 is base on the yaic DWT (Discete Wavelet Tansfom). Figue shows how the DWT ecomposes an image in + 3D spatial fequency subbans, whee D is the numbe of analysis tansfom stages. The stage geneates fou subbans LL, HL, LH an HH. L stans fo the low-pass filte an H fo the high-pass filte, an its position is associate to the iection of the filte (left when hoizontally an ight when vetically). Each stage is applie to the LL subban, o the entie image when = 0. As it can be obseve, applying D analysis stages a multiesolution epesentation of the image is geneate, with D + iffeent levels. Using the subbans LL, HL, LH an HH, the esolution level can be econstucte. The compession pocess geneates a list of concatenate packets. A packet inclues an incemental contibution of all the coe-blocks of a pecinct. Fo each pecinct thee ae as many packets as quality layes efine when compessing. The numbe of quality layes etemines the scalability in quality. The stana is oganize in seveal pats. Pat [5] escibes the compession system an how to buil the simplest coe-steam. This coe-steam contains the packets geneate by the image compession pocess an aitional infomation. The simplest JPEG2000 image file fomat contains a coe-steam, without any othe aition. In this Pat of the stana a moe complex image file fomat is also efine, allowing to inclue any use metaata infomation. This is specially exploitable in telepathology systems. Fo example, in [6] it is popose to inclue patient XML ata within the same image file of the igitalize slie. Pat 9 [7] coves the efinition of a new potocol, JPIP (JPeg2000 Inteactive Potocol), as well as a famewok to evelop systems fo emote bowsing of JPEG2000 images. Figue 2 shows a epesentation of the client/seve stuctue popose fo this new potocol. LL HL Client LH HH HL 0 WOI equest ata JPIP Client WOI status Bowse WOI image Client cache ata Decompesso Seve LH HH 0 0 JPIP Seve Figue. 2 DWT stages (3 esolution levels) applie to the Lena image, patitione by coe-blocks. Each subban is ivie into ectangula blocks calle coe-blocks which ae inepenently coe an compesse. Coe-blocks ae collecte in lage ectangula goups calle pecincts. The size of the coe-blocks an the pecincts etemines the ganulaity fo a late access to the compesse steam. This ata oganization is mae fo each image component. The coe-blocks ae elate to ectangula egions within each DWT subban but, the pecincts ae elate to ectangula egions within each esolution level, in the image omain. Fo example, a pecinct of the esolution level woul contain all the coe-blocks belonging to the same spatial egion, within the subbans HL, LH an HH (o LL if = D ). Client cache moel JPEG2000 images Figue 2. Client/seve stuctue popose fo the JPIP potocol. On the client sie, the use specifies a WOI (Winow Of Inteest) using the application bowse. The WOI efines the egion of the emote image that the use wants to see. This WOI is passe to the JPIP client moule an the ecompesso moule. The fist one manages the communication with the seve, elaboating the coesponing equest an eaing the esponse fom the seve. When the infomation is ea, it is stoe in the intenal cache. The ecompesso moule will gathe the equie cache infomation to econstuct the WOI image, which will be shown to the use by the bowse moule. When the JPIP seve eceives a client equest fo a specific WOI, it eas the associate image extacting the appopiate infomation which must be

3 sent to the client in oe to econstuct the equeste WOI. Thanks to the high scalability offee by JPEG2000, this extaction can be one with a minimum of computational loa. The seve optionally maintains a moel of the client cache that is use to avoi esening eunant infomation to the client. All the commente featues offee by the JPEG2000 stana allow to buil highly efficient telepathology systems base on vitual slies. All the images woul be compesse with this stana. The image files can be use inepenently, without being inclue within any othe moe complex file. As avantage it is possible to inclue any associate metaa, as it is popose in [6], an etieve it with any common JPIP client, inepenently of the metaata fomat. The compession of the images woul be one in oe to obtain a goo scalability, configuing a lage numbe of quality layes an a small size fo pecincts. In this way the seve can extact moe exactly only the equie ata an sen it aapting the communication to the existent banwith. The pathologist can inspect emotely the images using a JPIP client, so the achitectue simulates a pefect vitual micoscope. STITCHING IN THE DWT DOMAIN In the pevious Section it has been possible to obseve that the JPEG2000 stana offes all the necessay featues to ceate telepathology systems base on vitual slies, an futhemoe these systems can achieve a high efficiency. Thee is also othe aspect in the context of the vitual slies that can be consieably impove using JPEG2000: the stitching of images. Vitual slies ae ceate as a composition of a set of micoscope images that contain infomation about contiguous aeas of an entie slie. This composition o stitching consists basically of thee global steps: i) unification, ii) seach of the matching egions, an iii) union. In the fist step the possible geometic istotion of the images is coecte in oe to be able to join all the images in the same plane. The possible iffeences of illumination between the images, like fo example the vignetting, ae also coecte. In the secon step the existent common egions between the images ae seache, etemining how to join them. Finally, in the last step, all the images ae stitche, incluing a blening pocess too, smoothing the boes of the unions. Afte 2 secons Afte 8 secons The moe heteogeneous ae the images to join the moe complex ae these thee steps. In the concete case of the images acquie with a micoscope, in geneal, thee is quite homogeneity between them. The geometic istotion is not common, an if it appeas it can be ientifie an coecte inepenently in each image. The possible illumination iffeences o vignetting can be also coecte inepenently. The seach of the matching egions consists of fining simple ovelappings between the images, as tanslation isplacements. The stitching pocess is applie using ectilinea boes. Figue 3. Tissue econstuction at two iffeent instants of time. With the aim to give a little example of the pefomance of JPEG2000, we have use a JPIP client/seve achitectue to exploe a emote image though a communication channel with a banwith limite to 4 KB/s (like an ol moem). The image, about a meical tissue, has a esolution of an a compesse size bigge than 2 MB. In Figue 3 two econstuctions of a WOI of , at two iffeent instants of time, have been shown. As it can be obseve, afte only 2 secons the use can ecie if the WOI is the equie one o not. An, afte 6 secons moe, the use can obseve a high quality econstuction. Thee ae iffeent JPEG2000 softwae packages, with suppot fo JPIP, commecial as well as non-commecial, like fo example Kakau [8] o 2KAN [9]. The fist one has been use in this wok to evelop the examples an evaluation expeiments. These softwae packages allow to evelop quickly JPIP client/seve achitectues that, with some aitional moifications, coul be fine vitual slie telepathology systems. The stitching pocess fo geneating vitual slies has less complexity than the pocess fo geneating othe kins of mosaic images, like fo examples panoamas. In [0] the high complexity necessay of an automatic stitching metho fo panoamic images can be obseve. The geneation of vitual slies, as an image composition, oes not equie to take into account all the paametes an possibilities assume in this wok. This chaacteistic allows to evelop efficient an fast automatic stitching methos. Next a new metho that exploits the featues of the JPEG2000 stana is popose. Hee in afte we assume that the acquie micoscope images ae initially pocesse, uing the unification step, an afte that they ae compesse with JPEG2000, all of them with the same compession paametes (esolution levels, quality layes, etc.). It is necessay to configue the compesso so that the pecincts have the same spatial influence in each esolution level, as it is popose in []. A cetain pecinct size fo the esolution level 0 is chosen, an this is ivie by 2 successively fo the followings levels.

4 Thus, fo each esolution level, the pecinct woul have a size of p x, (with) an p y, (height) so that: p p p = p = 2 2 x, x, y, y, This allows to access to the coe-steam of the images with a fine ganulaity an to mege them without ecoing when it was possible. Due to the multiesolution epesentation of the compesse content of the JPEG2000 images, the secon step of the stitching pocess can be mae in two iffeent hieachical substeps, initial seach an efinement, without having to ecompess the entie contents. A gaphic example of the seach of the matching egions between two images can be obseve in Figue 4. Fo the initial seach, it is necessay to ecompess only the smallest esolution level. Afte fining the matching egions, they ae ecompesse, using the biggest esolution level, to efine the pecision. In this way a consieable amount of memoy an computational loa is save. fom the subbans { 0... D } x 2, y 2, w 2, h 2 ( + ) ( + ) ( + ) ( + ) LL, HL, LH an HH fo =. Then the egions associate to the same subban, fom each image, ae joine. The final global esult woul be conceptually the same that the esult of a DWT tansfom of the iect union of the images. Notice that the numbe of esolution levels of the esult is the same that the numbe of esolution levels chosen when compessing the images. + seach (esolution D+) Figue 5. Example of the stitching pocess in the DWT omain. efinement (esolution 0) Figue 4. Scheme of the popose two-steps seach metho. In JPEG2000 the colou images ae not usually compesse using the classic RGB components, but a tansfom is pefome in oe to geneate othe colou epesentations that allow bigge compession atios. In these colou epesentations thee is a component with the luminance content of the image. This component can be use to cay out the seaching pocess, without having to ecompess the est of the image components. Once the matching aeas ae foun pecisely, the joining pocess is mae in the DWT omain. This implies to make a multiesolution stitching, as it appeas in the example of Figue 5. Fom each subban fom each esolution level the egions to join ae extacte. Fo example, if the egion to extact of an image is efine by ( x, y, w, h ), hoizontal an vetical position an with an height, espectively, it woul be necessay to extact the egions: Thee ae two main avantages when the stitching pocess is mae in the wavelet omain: i) the computation time is euce an ii) the union boes ae smoothe. The fist avantage is given since the invese DWT of each image to join an the DWT of the compose esult is avoie. Due to the popeties of the fequential omain in which the stitching is mae, the high fequencies of the union boes ae euce. This means that these boes ae natually smoothe in the image econstuction. The moe esolution levels ae use, the moe smoothe become the boes. The smoothness is even moe efficient than othe methos that ae pefome in the image omain, since these ones usually affect to the entie image, losing infomation. Caying out the stitching pocess as hee is popose, only the union boes ae smoothe, without affecting the est of the content. Since a smooth postpocessing is usually necessay when the vitual slie is compose, with the popose stitching metho woul avoi this associate computation loa. The egions ae joine in the wavelet omain but by means of the image pecincts, since they ae the basic oganization unit when accessing to the coe-steam content. Pofiting that each pecinct is compesse inepenently of the est, we have not to ecoe all of them when joining the images. It is necessay to ecoe

5 only those pecincts unaligne with the global pecinct gi of the final compose esult. Notice that the final esult, that is, the vitual slie compose afte stitching all the images, has a global pecinct oganization that efines a gi fo each esolution level. Evey egion to join has a specific position an size within this gi. The pecincts of the egions that ae not aligne with this gi nee to be ecoe in oe to comply with the global oganization. If the position of the egion is not aligne, then all of thei pecincts will nee to be ecoe. If the position is aligne, pobably only the pecincts of the boes woul nee to be ecoe. Figue 6 shows a little example of how to join two iffeent egions within a specific esolution level. As it can be obseve the esult image has a specific gi oganization. The top egion is completely aligne with this gi, so all thei pecincts can be copie without ecoing at all. Nevetheless, the bottom egion is not aligne, so all thei pecincts nee to be ecoe, that is, they ae ecompesse an thei contents ae aligne with the global gi an compesse again. Joining the egions in this way allows to save a consieable amount of computation loa an memoy consumption, epening of the ovelapping egee of the images EVALUATION pecincts copie without ecoing pecincts copie with ecoing Figue 6. Example of how to join two egions within a esolution level. In oe to evaluate the popose stitching metho, we have obtaine the necessay time to stitch each pai of images of two iffeent sets. Neithe the algoithm to select the pai of images no the associate time is evaluate in this pape, since this is aleay well covee in othe woks, like [2] o [3]. The main goal of the expeiments is to evaluate the pefomance of the stitching metho, exploiting the featues of the JPEG2000. This time is compae to the obtaine one using a classical metho that consists of seaching an joining in the image omain. Each pai of images woul be completely ecompesse, pocesse, an compesse again. The use seach algoithm, fo the two methos to compae, is base on fining the phase coelation between each pai of images. We have assume that the images ae homogeneous an geometically compatibles. Evey image can be expesse as a isplacement of anothe. Fo the popose metho the phase coelation has been calculate fo the lowest esolution level an has been efine in the highest one. As it is use in [2] as well in [4] the Nomalize Coss Coelation coefficient has been use to obtain the phase coelation. This coefficient is calculate with: NCC = ( u u )( v v ) ( u u ) ( v v ) 2 2 whee u an v ae the images of the pai to stitch, an u an v thei espective means. This calculus can be pefome faste by means of the Fast Fouie Tansfom, an theeby the phase coelation woul consist of looking fo the peaks of: F * U V * U V whee U = F { u} an V F { v} =, F is the Fouie tansfom, an the asteisk stans fo the conjugate. The fist image set (A) contains 36 images with a size of 268 X 787. The othe image set (B) contains 25 images with a size of 3272 X Each image set is associate to a complete vitual slie, obtaine fom a micoscope. All the images have been compesse with JPEG2000 using these compession paametes: Resolution levels: 6 Quality layes: 0 Coe-block size: RESULTS In Table we have shown the aveage time pecentage of the popose stitching metho in elation to the aveage time equie fo the classical metho. It is clea that exploiting the JPEG2000 featues it is possible to euce significantly the computation time of the stitching pocess. The vitual slie constuction can

6 be pefome quickly an with the minimum esouce equiements. In Figue 7 a etail of a union boe is shown, compaing the esult of the popose metho to the esult of a simple metho (simply join without smooth postpocessing). CONCLUSIONS In this pape the main featues of the JPEG2000 stana have been analyze in oe to exploit them to buil highly efficient telepathology systems base on vitual slies. A new automatic stitching metho has been also popose, base specially on the stuctue of this stana. This metho euces at the maximum the equiements of computation loa an memoy. Moeove, using this stitching metho, the union boes ae automatically smoothe without losing infomation. ACKNOWLEDGMENTS Thanks to D. Olga Fee Roca, of the Univesity of La Laguna, Teneife (Spain), fo the micoscope meical images. REFERENCES Simple stitching Popose stitching Figue 7. Compaison of the simple stitching an the popose stitching. The popose stitching metho can impove consieably the esult, smoothing the union boe without losing infomation. This boe is noticeable ue to, although the images ae quite homogeneous, the illumination is not completely pefect. Image set Numbe of images A 36 B 25 Image size Time pecentage.05% 0.38% Table. Aveage time pecentage fo the popose stitching metho in elation to the simple stitching metho.. R.S. Weinstein, M.R. Descou, Chen Liang, A.K. Bhattachayya, A.R. Gaham, J.R. Davis, et al., Telepathology oveview: Fom concept to implementation'', Human Pathology, Decembe 200, vol. 32, iss. 2, pp Ümit Çatalyüek, M.D. Beynon, Chialin Chang, Tahsin Kuc, Alan Sussman, an Joel Saltz, The Vitual Micoscope, IEEE Tansactions on Infomation Technology in Biomeicie, Decembe 2003, vol. 7, no. 4, pp B. Paein, M. Tain, an P. Hoain, Demosaicking an JPEG2000 compession of micoscopy images, IEEE Intenational Confeence on Image Pocessing, Octobe 2004, vol., pp N.R. Wanigasekaa, Shenghao Ding, Zhuangzhi Yan, an Yongqin Zeng, Quality Evaluation fo JPEG 2000 base Meical Image Compession, Secon Joint EMBS/BMES Confeence, Octobe 2002, pp ISO/IEC 5444, Infomation Technology - JPEG2000 Image Coing System - Pat : Coe coing system, L. Montesinos an J. Puentes, Specialize Telepathology Electonic Patient Reco Base On JPEG 2000, IEEE Confeence on Infomation Technology Applications in Biomeicine, Apil 2003, pp ISO/IEC 5444, Infomation Technology - JPEG2000 Image Coing System - Pat 9: Inteactive tools, APIs an potocols, Kakau Softwae, A Compehensive Famewok fo JPEG2000, 9. The eseach an evelopment poject 2KAN, 0. M. Bown an D.G. Lowe, Recognising Panoamas, Intenational Confeence on Compute Vision, 2003, pp

7 . R. Ronsenbaum an D. Taubman, Meging Images in the JPEG2000 omain, Visualization, Imaging an Image Pocessing, Septembe 2003, vol., pp B. Appleton, A.P. Baley, an M. Wilemoth, Towas Optimal Image Stitching fo Vitual Micoscopy, Digital Image Computing: Techniques an Applications, Decembe 2005, pp P. Thévenaz, D. Lambiel, an M.A. Unse, A stategy base on maximum spanning tees to stitch togethe micoscope images, SPIE Meical Imaging, 2006, vol M. Capek, R. Wegenkittl an P. Felkel, A Fully Automatic Stitching of 2D Meical Data Sets, EURASIP BIOSIGNAL, June 2002, pp