86 F. ROSSANT, F. AMIEL, T. EA, C. MARAVER MARTINEZ, M. PAQUES, ENHANCEMENT OF OPTICAL COHERENCE... Enhancement o Optical Coherence Tomography Images o the Retina by Normalization and Fusion Florence ROSSANT, Frederic AMIEL, Thomas EA, C. MARAVER MARTINEZ, Michel PÂQUES 2 Institut Supérieur d Electronique de Paris, ISEP, 2 rue d Assas 75006 Paris, France 2 Centre d'investigation Clinique - Centre Hospitalier National d'ophtalmologie des Quinze-Vingts, 28 rue de Charenton 7502 Paris, France lorence.rossant rederic.amiel thomas.ea @isep.r, michel.paques@gmail.com Abstract. This paper describes an image processing method applied to Optical Coherence Tomography (OCT) images o the retina. The aim is to achieve improved OCT images rom the usion o sequential OCT scans obtained at identical retinal locations. The method is based on the normalization o the acquired images and their usion. As a result, a noise reduction and an image enhancement are reached. Thans to the resulting improvement in retinal imaging, clinical specialists are able to evaluate more eiciently eyes pathologies and anomalies. This paper presents the proposed method and gives some evaluation results. Keywords Optical coherence tomography, denoising, iltering, correlation, usion.. Introduction Optical coherence tomography (OCT) is a cross-sectional imaging technique allowing micrometric-scale resolution o retinal structures. It enables reliable demonstration o changes in overall retinal thicness, detection o luid in and behind the neurosensory retina, and identiication o the retinal nerve iber and photoreceptor layers. It is routinely used or diagnosis o retinal diseases. The OCT principle is to obtain a histology image o the retina by measuring the relection intensity o a low coherence inra-red light beam on the retina. The ovea o the eye, region where the analysis is done on, has a length between 0.5 and mm. The cut taes place on a dimension o 3 to 0 mm and on 52 points. The best resolution is, thereore, obtained or a length o 3 mm. The depth o 20 μm is measured by the tomograph on 024 points []. The obtained relection coeicients are displayed as an image (52x024 pixels) in alse colors. Although these images give an excellent idea about the retina, the excessive presence o noise maes the intervening layers o the neurosensory retina be only vaguely discernible. A primary goal o our study was to obtain a better characterization o the outer retina, that is, the interace between the retinal pigment epithelium and the photoreceptors. With ultra high resolution instruments, based on titanium sapphire lasers [3], urther improvements would be possible. However, this instrument is not commercialized yet. This is the reason why the research o a method that increases the signal to noise ratio and enhance the image has become an issue to study [4]. Image usion generally results in a better quality, since it leads to an increased signal-to-noise ratio. In OCT imaging o the retina, a series o scans (I I N ) are acquired rom identical undus locations. A irst pre-processing step is applied to each one, in order to normalize the images. This step is called alignment" in what ollows. It allows a better readability o each image taen separately, and is necessary to achieve the usion. The usion step itsel is based on an average, but other unctions could be considered. The bloc diagram o the global system is shown in Fig.. I Max intensity I 2 relexion Normalisation A 2... layer process :... I N detection alignment A N Fig.. Global system. A Fusion I used We have explored this idea using data rom the conventional StratusOCT instrument [], [2]. We have wored directly with the relection coeicients instead o the processed alse color image. Indeed, the relection coeicients provide much more inormation (about 500 dierent values), whereas the given output images are quantized on about 245 dierent values. So, the inormation obtained by the StratusOCT tomography, to be processed by our method, is a matrix o relection coeicients (024 rows per 52 columns). The developed algorithm treats this matrix as an image, where each value corresponds to a pixel. The paper is organized as ollows. Sections 2 and 3 present the normalization method, which is based on the detection o the maximal intensity relection layer (Section
RADIOENGINEERING, VOL. 7, NO. 4, DECEMBER 2008 87 2), ollowed by an image transormation, that results in the alignment o this layer (Section 3). Section 4 describes the usion method. Finally, some evaluation results are analyzed in Section 5. The last section presents a conclusion and introduces uture wor. 2. Maximum Intensity Relection Layer Detection 2. Region o Interest Fig. 2 shows two scans o retinal OCT images, I and I 2, captured consecutively rom the same patient. The images are displayed in alse colors. They have been also cropped and resized or a better readability o the igure. Our coordinate system is deined by the origin at the upper let corner o the image, the vertical x-axis and the horizontal y-axis. It is obviously not possible to use directly these images: some distortions can be observed between both, since they do not exactly represent the same retina cut (because o ocular movements). So, an essential preprocessing step consists in detecting a common region o interest, the maximal intensity relection layer (Fig. 2), and aligning all images, so that the pixels belonging to this layer are located at the same vertical position (Fig. 5). Thus, the retinal structure will be at the same position in all images, and the usion can be achieved. Another important goal o this transormation is that it allows a better clinical interpretation o the retina OCT images. (c) Fig. 2. Two scans taen rom the same patient (I, I 2 ), (c) a zoom o igure. The maximal intensity relection layer (in red) is the common region o interest used as reerence to achieve the normalization. The major diiculty or this detection is the high noise presence (Fig. 2(c)). The dominating noise source in OCT images is usually specle noise arising rom intererence between coherent waves bacscattered rom nearby scatters in the measuring volume o the retina. Some o the classic methods used or denoising are averaging each pixel with its neighbors (spatial low-pass iltering), applying a median iltering or a low pass iltering in the requency domain [5]. These three techniques are used in dierent steps all along the method that is described below. 2.2 Detection Method The aim is to ind an internal line on the retinal layer. In the OCT image, this layer corresponds to a roughly horizontal set o pixels taing the highest values (displayed in red). Because o the low signal-to-noise ratio, such simple methods lie looing or the maximal pixel value column by column, do not wor. In addition, nown algorithms about boundary detection, as or example active contours [6], do not wor properly either, as the noisy pixels bloc the evolution o the contour. Consequently, more complex treatments are proposed. Firstly, a pixel belonging to the region o interest, called "internal point", is ound with certainty. This pixel is then used to initialize an iterative algorithm that allows to deduce the median line o the maximal intensity layer. This algorithm uses correlation inormation between adjacent pixel columns, in order to reach a more robust and representative result. It is also applied on images that have been beorehand low-pass iltered, in order to reduce the noise. The bloc diagram o the detection method, applied to each source image I n ( n N ), is shown in Fig. 3. I n Internal Point Detection LP FIR Filter J n (x 0,y 0 ) Median Line Detection ( ( x,, 0 y < j ) opt 52 Fig. 3. Maximal relection intensity layer detection method. Searching or an internal point: The image I n is irst smoothed, using a large convolution mas (0*40 coeicients equal to /400). The pixel reaching the highest value, at the (x 0,y 0 ) coordinates, is certain to belong to the maximal intensity relection layer, and consequently, it is taen as the internal point. In our experimentations, it is proved that a rectangular mas leads to better results, since it is more representative o the globally horizontal structure we want to detect. Low-pass iltering: The initial image I n is iltered in the requency domain, using a FIR ilter o order 6, with a normalized cut-o requency equal to 0.05. As a result, the signal to noise ratio is increased while the boundaries o the region o interest are not signiicantly delocalized. The output image, denoted by J n, is passed to the median line detection algorithm. Median line detection: Starting rom the detected internal point (x 0,y 0 ), the algorithm deduces iteratively, column by column, the median line o the maximal intensity relection layer. Let consider the next column at the horizontal coordinate y=y 0 +. The pixels o this column are low-pass iltered, using a convolution mas o K = 2 coeicients, all equal to /K: S ( = K K / 2 J n x +, = K / 2 (. ()
88 F. ROSSANT, F. AMIEL, T. EA, C. MARAVER MARTINEZ, M. PAQUES, ENHANCEMENT OF OPTICAL COHERENCE... The highest outputs o this ilter are located inside the maximal intensity relection layer. In order to avoid alse detections and smooth the searched median line, a recursive low-pass ilter is also applied to the S( coeicients, along the y coordinate. Let denote by C( the output o this second ilter (Eq. 2). The decision at column y is taen by retaining the x ( opt vertical coordinate corresponding to the maximum output value (Eq. 3): C( = ( α ) S( + αc( y ), (2) max x ( { C( } C( x, =. (3) opt In this method, the S( coeicients are continuously integrated to provide the decision at the column y. The parameter α o this recursive ilter expresses the relative importance between the current local results, and the previous ilter outputs. In our experiments, we use α=0.8. This value results rom a compromise: with a greater value, the median line is not accurately ollowed; with a smaller value, the algorithm is too sensitive to bright noise pixels. The same algorithm is applied in the other direction, or decreasing y coordinates, starting also rom the irst detected internal point (x 0,y 0 ). The result obtained ater applying the method to the image I (Fig. 2) is presented below, superimposed on the iltered image J. The detected median line (in dar blue) ollows accurately the maximal intensity relection layer. estimated by averaging the pixels on a small area around the median line. Fig. 5 shows the superior boundary ound with this method. Fig. 5. Detection o the superior boundary o the maximum relection intensity layer, and alignment against this line. Finally, the alignment is carried out. It consists on a column by column simple shit (Fig. 5). The resulting images are denoted by A n in what ollows. Fig. 4. Maximal intensity relection layer detection. The dar line represents the median position o this layer. 3. Alignment The aim o the alignment is to normalize the dierent source images, so that they can be used, and also to improve their readability or the clinical diagnostic. The alignment is not carried out against the median internal line ound by the method previously described, but against the superior boundary o the maximum intensity relection layer. This boundary can be deduced through a region growing algorithm, whose seed is the internal median line. In order to suppress the noise, a median ilter is applied beorehand column per column. Its size is equal to the typical width o the maximum intensity relection layer. The region growing stops when a pixel value is below a threshold T that is dynamically set to the hal o the mean pixel value M o the region o interest (T = 0.5M). M is 4. Fusion The aim o this section is to obtain an enhanced image in order to satisy the medical interest. The idea is to generate rom several aligned images a new one, with a higher signal-to-noise ratio. Beore using, the correct superposition between the images has to be ound. Translation, rotation and homothety transormation could be considered. But only translation is studied, because a rotation will give a dierent cut and thus a dierent image. Homothety transormation is not signiicant because the patient does not move his head during snapshots. The images are correlated two by two, in order to ind the optimal superposition between all couples. The higher is the correlation score, the higher is the similarity between both images. So, the maximum correlation value corresponds to the optimal translation shit. The maximum correlation scores obtained over all the aligned images allow also to choose a reerence image, rom which all the translations will be achieved, and to reject images that are not enough similar to it. The reerence image A R is chosen as ollows: it is the one that leads, in average, to the two highest correlation scores with the other images. Images that get a correlation score below S=0.5 (threshold experimen-
RADIOENGINEERING, VOL. 7, NO. 4, DECEMBER 2008 89 tally ound) are considered to be erroneous ones, and will not be included in the usion. A A 2... A N Inter correlation Reerence Image A R Other accepted images A Rejected C( AR, A ) > S images A C ( A,A ) S R A R A v u Fig. 6. Fusion process. Fusion Dierent usion unctions can be envisaged. In our irst experiments, we use a simple average unction that applies spatially on each retained image A and also between all these images: A ( x = = i= j= I used K, A ( x + u + i, y + v + j) (4) 9K where K (K N), the number o accepted images, indexed by, including the reerence image A R, (u,v ) the horizontal and vertical shits between the image A and the reerence image A R. The results can be still improved, by excluding rom the average the pixels that are liely to be noise pixels. A ( represents the mean pixel value, calculated over the K retained images, on a small 3x3 neighborhood around (. In the same way we compute an estimation o the standard deviation σ (. The pixels o the neighborhood that dier too much rom the mean value A ( are excluded rom the averaging (5). The resulting image I used is consequently improved, compared to the mean image A. A ( A ( x + u + i, y + v + j) > 0.5σ ( x, A ( x + u + i, y + v j) excluded rom the usion. (5) + and the variance calculated on the used image (6) provides an estimation o the signal to noise ratio improvement. This study has been conducted only on images o healthy retinas, otherwise the three regions cannot be all correctly deined. Tab. indicates the number o images used or the usion and the gains (in db) obtained or each region. G ( i) σ 2 K = ( i) ( i 0log, σ m = σ ( i) 2 K = σ m ( i) Number o images (K) Tab.. The signal to noise ratio gain measured on three layers, or 7 cases o healthy eye. The signal to noise ratio is increased in all cases, with a gain between 3 and 2 db. The greater is the number o used images, the greater is the SNR gain. The Fovea and the ONL regions are clearly denoised (gain over 6 db) while the layer has been signiicantly smoothed. ONL G () (db) G (2) (db) ) G (3) (db) ONL 3 8.5 3.5 6.3 3 9.4 2.7 5.2 4 8.4 3.3 5.4 6 7.5 2.9 6. 9.9 8.0 8.4 0 4 7.0 8.4 5 0.3 6.2 7.5 (6) An example o usion is shown in Fig. 8. Four images A were used. This process results in a higher signal to noise ratio and an enhanced contrast (see next section). 5. Results Our study has been done over 0 patients, healthy or presenting dierent pathologies. Three to iteen images were captured per patient. The proposed method has been evaluated and quantiied based on the ollowing procedure. We have considered three homogenous areas where the grey levels should be almost constant without the presence o noise: the ovea (), the ganglion cell layer and the inner plexiorm layer (), and the outer nuclear layer (ONL) (Fig. 7). We have estimated the noise power by calculating the image variance on these regions (manually delimitated). Let us denote by σ (i) the standard deviation o the region i in the image, and by σ (i) the corresponding measure in the used image. Then, the ratio between the mean variance calculated on the original images ONL Fig. 7. Fusion o 0 images: one o the original images, the result image. Images are represented in grey levels and normalized between 0 and. The results were also submitted to clinical specialists rom the Quinze-Vingts hospital (Paris). Doctors assessed in all cases that the normalization and usion method leads to an enhanced image that maes the clinical interpretation easier and more accurate. It is clearly apparent rom Fig. 7 and 8 that the contrast between the retinal layers is improved. In particular, the relective layer attributed to the external
90 F. ROSSANT, F. AMIEL, T. EA, C. MARAVER MARTINEZ, M. PAQUES, ENHANCEMENT OF OPTICAL COHERENCE... limiting membrane, that is, the rontier between the inner and outer segment o the photoreceptors, becomes clear, while it was slightly apparent in non-processed images. (c) Fig. 8. Fusion o 4 aligned images (c)(d) to provide the enhanced output image (e). In the example shown in Fig. 9, the interruption o the photoreceptor line is evident. Alterations o the intraretinal structure are also observed, consisting o collection o oedema within the retina. (c) (d) (e) (d) Fig. 9. Fusion o 6 aligned images (three o them are represented in igures (c)) to provide the enhanced output image (d). 6. Conclusion This paper describes a method o optical coherence tomography imaging enhancement o the retinal area o the eye. The methods and algorithms proposed in this article, i.e. relection layer detection, image alignment and usion, have proved to increase considerably the quality and the signal to noise ratio. In act, usion o a collection o image OCT scans rom the same retinal area enhances enough the quality o imaging to reveal new details o the retina. To date, only total retinal thicness was considered in most studies o OCT; alterations o retinal layering, which is an indicator o retinal diseases, could not be reliably deined. Averaging OCT scans thus provides additional inormation about the intraretinal structures. This is in agreement with the results presented by [4]. The higher deinition o the retinal layering oers the opportunity to better deine the alterations o retinal structures. Such improved precision contributes to a better diagnosis o retinal alterations and hence to the cause o visual impairment. Future wor will include investigation o new usion techniques, in order to improve the processing speed and the quality o the result. The automatic detection o the dierent physiological layers o the retina, in order to calculate some measures useul or the clinical diagnostic, is also a subject o urther study. Reerences [] StratusOCT tomography, Zeiss documentation, http://www.zeiss.com/c25679e005c774/contents- Frame/2BF7095D5578B4D882572430063F9AD. [2] Carl Zeiss Meditec AG, Goeschwitzer Str 5-52, 07745 Jena, Germany, http://www.zeiss.com/. [3] KENT, C. Taing OCT technology to the next level. Review o Ophthalmology, 2006, vol. 3:06, Issue 6/7/2006. [4] SANDER, B., LARSEN, M., THRANE, L., HOUGAARD, J. L., JØRGENSEN, T. M. Enhanced optical coherence tomography imaging by multiple scan averaging. Br. J. Ophthalmol. 2005; vol. 89; p. 207-22 doi:0.36/bjo.2004.045989, December 2005. [5] GONZALES, R. C., WOODS, R. E. Digital Image Processing. New Jersey: Prentice Hall, 2002. [6] SETHIAN, J. A. Level Set Method and Fast Matching Methods. Cambridge University Press, 999.