Fully Automated Quantification of Leaf Venation Structure

Transcription

1 Fully Automated Quantification of Leaf Venation Structure J. Mounsef 1, and L. Karam 2 1 School of Electrical, Computer & Energy Engineering, Arizona State University, Tempe, Arizona, USA 2 School of Electrical, Computer & Energy Engineering, Arizona State University, Tempe, Arizona, USA Abstract - Recently, there has been a surge of diverse approaches to investigate leaf vein patterning, covering genetic analyses, pharmacological approaches and theoretical modeling. Genetic and pharmacological approaches remain at this stage insufficient for analyzing the formation of vascular patterns since the molecular mechanisms involved are still unclear. Similarly, theoretical models are not sufficiently constrained, and thus difficult to validate or disprove. Only few exceptions attempted to provide a link between experimental and theoretical studies by implementing different imaging techniques. Visual imaging methods have been lately extensively used in applications that are targeted to understand and analyze physical biological patterns, specifically to classify different leaf species and quantify leaf venation patterns. There is a rich literature on imaging applications in the above field and various techniques have been developed. However, current methods that aimed to provide high precision results, failed to avoid manual intervention and user assistance for the developed software tools. In this paper, we introduce a fully automated imaging approach for extracting spatial vein pattern data from leaf images, such as vein densities but also vein reticulation (loops) sizes and shapes. We applied this method to quantify leaf venation patterns of the first rosette leaf of Arabidopsis thaliana throughout a series of developmental stages. In particular, we characterized the size and shape of vein network reticulations, which enlarge and get split by new veins as a leaf develops. For this purpose, the approach uses imaging techniques in a fully automatic way that enables the user to batch process a high throughput of data without any manual intervention, yet giving highly accurate results. Keywords: fully automation; leaf venation pattern; feature extraction; feature quantification; adaptive thresholding 1 Introduction Over the past few years, there has been a resurgence of interest in the leaf vein patterning. Traditional strategies include genetic analyses, pharmacological approaches, and theoretical modeling. Most recently, quantitative studies attempted to provide a link between theoretical and experimental methods. Many quantitative studies of leaf venations patterns have resurged with the sole aim of quantifying the response of plants to changing environments and to breed plants that can respond to such changes. Unfortunately, most of the suggested empirical analyses have failed to provide a quantitative study of the complete leaf vascular network [1],[2]. One major recent approach successfully provided a quantitative study of the complete leaf venation network [3]. The aforementioned approach proposed to quantify vein patterns spatially at the tissue level throughout leaf development. The venation forms a complex network composed of veins of different orders within the leaf tissue. Higher-order veins differentiate within loops that enlarge with the leaf growth. Therefore, quantifying vein patterns spatially at the tissue level in terms of loop shapes and sizes, in addition to vein densities, has brought new insights into patterning mechanisms. It has provided the necessary spatial resolution that was lacking in traditional methods. The procedure developed to extract venation patterns from leaf images consisted of measuring the leaf vascular network parameters by digitizing the vein segments using a user interface: points along each segment were recorded manually leading to a matrix representing all vein segments on the leaf. From the extracted series of vein segments, the topology of the network was calculated allowing the identification of each vein segment and loop. The number and position of branching points and free ending points were also recorded. Finally, the complete venation pattern and leaf outline for each sample was stored in matrix form. Such an approach proved to be efficient on the resolution side. Nevertheless, it is time consuming and painstaking especially when the throughput is high. The approach [3] tried to solve this issue by automatically digitizing veins using unpublished software. Nevertheless, the software was not sufficiently accurate to replace the manual technique described in [3]. Another recent work developed a user-assisted software tool (LEAF GUI) that extracts macroscopic vein structures directly from leaf images [4]. The software tool takes an image of a leaf and, following a series of interactive steps to clean up the image, returns information on the structure of that leaf's vein networks. Structural measurements include the dimensions, position and connectivity of all network veins, in addition to the dimensions, shape and position of all non-vein areas, called areoles. Although the tool is meant to enable users to automatically quantify the geometry of entire leaf networks, it unquestionably needs user assistance and intervention at

2 several steps of the images preprocessing, and cleaning. Many fully automated image-based algorithms have been in vain proposed for analyzing the leaf venation patterns. For instance, two approaches, one involving scale-space analysis and another including a combination of edge detectors [5], have been implemented to assess the digitization results of 5 test leaf images compared to the results of a manual technique that uses Adobe Photoshop. The main encountered problems were the presence of too many artefacts in the resulting processed images due to inappropriate smoothing and edge detection. Moreover, the scale-space technique was criticized for it resulted in broken venation structures. Here, we propose a fully automatic approach based on the work described in [3]. Our approach is able to quantify the vein patterns of the first rosette leaf of Arabidopsis thaliana by processing 135 images provided by [3], without the need of any user assistance. It also stores the leaf venation pattern for every sample in a matrix, yet giving the same spatial accuracy as with the manual method, while significantly speeding the analysis process. The algorithm is developed in MATLAB and is based on different imaging analysis techniques including noise cleaning, adaptive thresholding, edge detection, and skeletonization. 2 Image analysis and pattern extraction The image dataset collected for [3] was kindly provided to us by the authors. The available leaves images correspond to patterns of differentiated xylem elements. Fifteen leaves images were captured each day for nine days: from eight to sixteen days after sowing (DAS). The images were stored as jpg into 9 different folders corresponding to each DAS day. However, the algorithm can process any other image format (tif, gif, png, bmp). Custom programs were developed to obtain quantitative pattern data from the first rosette leaf of Arabidopsis thaliana images using MATLAB. The main goal of our algorithm was speeding up the process of leaf venation pattern extraction while keeping the same spatial accuracy of the manual method [3]. To achieve a practical method that enables high throughput image analysis, it is essential to fully automate the software by avoiding any manual intervention. The main challenge remains the accuracy of the results without requiring any user assistance. Our task was made harder with the presence of noisy images with low contrast and uneven varying illumination resulting from the fact that the images were captured at different harvesting times (Fig. 1a). To resolve these issues, we incorporated different image processing techniques into the algorithm. The main steps performed by our algorithm are described as follows for a given input image: (1) Converting the rgb image into a grayscale image, (2) Performing homomorphic filtering of grayscale image, (3) Denoising and enhancing the grayscale image, (4) Removing border, (5) Thresholding the image, (6) Skeletonizing and processing the image, and (7) Extracting and quantifying the venation pattern features. All the steps do not require any user intervention since they are fully automated. Next we describe in details the overall process that our algorithm performs to process a leaf image and measure characteristic features of the corresponding venation structure. Step 1: Converting the rgb image into grayscale image When converting from an rgb image to a grayscale image, the value of each pixel in the resulting image is a single sample that only represents the intensity information. The range of intensities varies from absolute black (0) to absolute white (255). Step 2: Homomorphic filtering of the image To better improve the illumination and contrast of the image, homomorphic filtering is applied [6]. An image is a function that is expressed as the product of reflectance and illumination F(x,y) = I(x,y).R(x,y). In order to separate reflectance and illumination, the function can be expressed as a logarithmic function wherein the product of the Fourier transform can be represented as the sum of the illumination and reflectance components ln(x,y) = ln(i(x,y)) + ln(r(x,y)). The Fourier transform of the previous equation is Z(u,v) = F I (u,v) + F R (u,v). A filter function H(u,v) is applied to the Fourier transformed signal where 2 cd( u, v) / Do 2 H(u,v) = ( H L )[1 e ] L (1) The constant c controls the sharpness of the slope of the filter function as it transitions between H and L, D 0 is the cutoff frequency and D(u,v) is the distance between (u,v) and the frequency origin. H(u,v) is designed such that it tends to decrease the contribution made by low frequencies (illumination,γ L <1) and amplify the contribution made by high frequencies (reflectance, γ H >1). Next, the resulting function is inverse Fourier transformed. Finally, the inverse exponential operation yields an enhanced image. Step 3: Denoising and enhancing the image It is essential to attenuate the presence of background corresponding to the leaf blade and the rest of the image with respect to the foreground represented by the veins. Therefore, applying a denoising technique that cleans the background, which is considered as the unwanted part of the image, is crucial. A non-orthogonal wavelet denoising technique is used here [7]. One of its main characteristics is that it preserves the phase information, which is fundamental to the human visual perception. Also, this technique is capable of determining denoising threshold levels automatically. Nevertheless, the denoising step affects the venation structure for it attenuates the pixels intensities, including the vein pixels. Therefore, enhancing the image contrast is mandatory at this stage of the image processing chain. For this purpose, histogram equalization is applied, where a gray-level transform is used to flatten the histogram of the denoised image. Then, linear contrast stretching is applied to the enhanced

3 a. Original Image b. Grayscale Image d. Denoised Image e. Contrast Enhanced Image c. Homomorphic Transformed Image f. Contour Eroded Image final image in code 2 g. Binarized Image h. Skeletonized Image i. Final Venation Pattern (Cleaned, Gap Filled and Pruned Spurs Image) Figure 1. Arabidopsis leaf harvested at DAS 16: Processed images corresponding to the 7 automated steps of the leaf image processing enhanced image where the value in the low end of the resulting histogram is assigned to extreme black (0) and the value at the high end to extreme white (1). Step 4: Removing border In order to remove the leaf perimeter, we need first to identify the leaf lamina from the rest of the image. This is achieved by applying global thresholding where pixels with intensities above a certain thereshold are set to 1 (leaf pixels) and pixels with values lower than the threshold are set to 0 (background pixels). The resulting image shows the entire leaf blade in white while the remaining image is black. Next, cropping is applied to the white contour corresponding to the leaf blade perimeter, by applying morphological erosion that uses a disk of n pixel diameter as a structural element. Finally, all the pixels of the eroded white area are preserved in the original image, while the values of the remaining pixels corresponding to the black area are set to 0. Step 5: Thresholding the image Thresholding is applied to extract the vein segments of the leaf from the rest of the image. Since leaves harvested on the same day showed uneven illumination in the same image, it was difficult to find a standard value for the threshold value that could be applicable to all images. Thus, adaptive thresholding is used by automatically calculating the

4 discriminative threshold from the corresponding image histogram. The histogram of any leaf image shows three main gray scale level ranges: a lower one that corresponds to the background of the image, a middle one that corresponds to the leaf blade, and a higher one that corresponds to the leaf veins. Therefore, the higher gray scale level region is of interest for our thresholding. Since the histogram showed additional peaks corresponding to the small variations resulting from high frequency noise, histogram smoothing is performed by applying a weighted averaging filter that acts as a low-pass filter for the histogram. As a result, the smoothed histogram shows three peaks that are separated by two main valley points. The value of the lowest valley point is computed by finding the gray scale level that corresponds to the first local minimum of the histogram function. After excluding the gray scale level values that are lower than the computed valley point, the resulting histogram (consisting now of two peaks) is used to find the optimal threshold separating the gray level values of the leaf blade from the gray level values of the leaf veins. Global thresholding using Otsu s method [8] is thus applied to the new histogram. The final thresholded image represents the venation pattern of the leaf. Step 6: Skeletonizing and processing the image Thinning is then performed by skeletonizing the resulting leaf venation structure. Skeletons represent a powerful tool for qualitative shape matching as they can effectively represent both the object shape and its topology. A thinning process is used in our approach to guarantee the condition of obtaining one-pixel thick and connected skeletons. If the skeleton is considered as a connected graph, each vertex can then be labeled as an end point or a branching point. A pixel is defined as an end point if it has a single connected pixel among its 8 neighborhood pixels. A pixel is defined as a branching point if it has more than two connected pixels among its 8 neighborhood pixels. A segment connecting two adjacent vertices is defined as a branch. Skeletonization is followed by image cleaning, which consists of removing unwanted connected blogs below a certain size threshold. For instance, all connected blog regions smaller than 10 pixels are removed. In order to reinforce our algorithm as a fully automated technique, we need to avoid any broken contour that might result from thresholding and any extraneous spurs that might occurr after skeletonization. A gap filling function was developed to join all the disconnected edges that are supposed to be connected. The function consists of considering all end points of the skeleton and check if they were connected in the original image of the leaf before thresholding by calculating the average intensity of the pixels joining the two end points in the prethresholded image. If the computed value is lower than a preset threshold, the two end point pixels are considered not initially connected and the gap is not filled. If the computed value is higher than the threshold, then the gap between the two end point pixels is filled. This step is followed by cleaning all connected objects that are smaller than the largest connected object (venation structure). This guarantees to clean any other concomitant unwanted leaves or intruders present with the main leaf in the image, which in turn avoids the time-wasting step of manually removing any extraneous features. On the other hand, extraneous spurs are pruned by considering all branch segments having an end point and a branching point as its vertices. If the branch segment length is smaller than a preset threshold, it is removed. Step 7: Extracting and quantifying the venation pattern features Using the known coordinates of the branching points and end points along each vein segment, we could record segment parameters. We could also automatically record the number and position of branching points and free-vein endings. Loops could also be identified by automatically counting the number of closed elements. The latter correspond to elements starting at one node and ending at the same visited node. The area of the blade is computed by finding the area of the white blog resulting from the aforementioned Step 4. As for the length of the venation structure, it is computed by summing the number of pixels of the final processed skeleton. 3 Results and discussion Software was developed in the lab to extract venation patterns from leaf images. Fig. 1 shows the different images resulting from the processing steps described above. The case of a leaf harvested at DAS 16 was selected on purpose to highlight the efficiency of our algorithm when applied to a complex venation structure. Fig. 2 displays a comparison of the results between the manual and the proposed techniques. The first rosette leaves of Arabidopsis from the [3] data set grew throughout the period analyzed (from 8 to 16 days after sowing), with a decrease in growth rate after 14 days from sowing for both non-automated and automated methods (Fig. 2a). The vein density for each leaf was calculated as the total vein length across the leaf divided by the whole leaf area. The average leaf vein density is shown in Fig. 2b and shows that venation decreases over the period analyzed. The vein density has an obvious decreasing rate starting 12 days after sowing, which is consistent with the results of the non-automated approach. The mean loop number per leaf and the mean segment number per leaf for each day are shown in Figs. 2c, d. Both techniques showed similar plots where loop numbers from day 12 till day 16 and segment numbers from day 13 till day 16 were not significantly different. Free vein ending and branching point densities were calculated as number of free vein endings and branching points per unit leaf area (mm -2 ) (Figs. 2e,f). Free vein endings showed significant similarities in mean densities of free vein endings between the plots of the non-automated and automated methods. Both techniques showed that free vein ending densities varied considerably between days. We observed comparable results for the

5 branching point densities where the observed increase in branching point densities between day 8 and day 10 in the automated plot (Fig. 2f) was consistent with the nonautomated results. d a e b c f Figure 2. Plots of automated and non-automated temporal analysis of veins patterns in first rosette leaf of Arabidopsis. 4 Conclusions We developed a method to automatically process and quantify leaf vein patterns using a series of image processing techniques and applied it to the analysis of vein pattern formation during leaf development in the first rosette leaf of Arabidopsis. Our work has the advantage of fully automating the extraction and analysis of leaf venation patterns without any user assistance. Therefore, it speeds up the analysis and allows large scale comparative analyses of vein patterning. To our knowledge, there is no available software that allows for an efficient extraction and analysis of the leaf venation

6 pattern using fully automated programs. Therefore, our algorithm might be considered a reliable tool that can be used by plant biologists to assess their manual results rapidly and under various conditions. 5 Acknowledgments We would like to thank Prof. Anne-Gaëlle Rolland- Lagan for kindly providing us with the dataset of the Arabidopsis leaves and for her helpful comments on the draft of this paper. 6 References [1] Boyce CK, Brodribb TJ, Feild TS, Zwieniecki MA (2009) Angiosperm leaf vein evolution was physiologically and environmentally transformative. Proc Biol Sci 276: [2] Brodribb TJ, Feild TS (2010) Leaf hydraulic evolution led a surge in leaf photosynthetic capacity during early angiosperm diversification. EcolLett 13: [3] Rolland-Lagan A-G, Amin M, and Pakulska M (2009) Quantifying leaf venation patterns: two-dimensional maps, The Plant Journal, vol. 57, no.1, pp [4] Price CA, Symonova O, Mileyko Y (2011) Leaf Extraction and Analysis Framework Graphical User Interface: Segmenting and Analyzing the Structure of Leaf Veins and Areoles, Plant Physiology, vol. 155, no. 1, [5] Clarke J, Barman S, Remagnino P, Bailey K, Kirkup D, Mayo S, Wilkin P (2006) Venation pattern analysis of leaf images, Lecture Notes In Computer Science, vol. 4292, pp [6] Gonzalez R, Woods R (2001) Digital Image Processing, Second Edition, Prentice Hall [7] Kovesi P (1999) Phase Preserving Denoising of Images, The Australian Pattern Recognition Society Conference: DICTA'99. Perth WA. pp [8] Otsu N (1979) A threshold selection method from gray level histograms, IEEE Transactions on Systems, Man and Cybernetics, vol.9, no.1, pp.62-66