Tool for Automated Image Based Grain Sizing. Richard D. Adams

Transcription

1 Tool for Automated Image Based Grain Sizing Richard D. Adams A dissertation/thesis submitted to the faculty of Brigham Young University in partial fulfillment of the requirements for the degree of Master of Science Rollin H. Hotchkiss, Chair Jim Nelson Department of Civil and Environmental Engineering Brigham Young University December 2013 Copyright 2013 Richard D. Adams All Rights Reserved

2

3 ABSTRACT Tool for Automated Image Based Grain Sizing to Standard Pebble Counts Richard D. Adams Department of Civil and Environmental Engineering, BYU Master of Science Automated Image Based Grain Sizing is a family of methods for determining the grainsize distribution (GSD) of objects contained within a digital image. Application of these methods to images of channel beds produces pebble counts that can be used for further analysis and classification of the channel. The primary task in this project was the creation of a tool to perform Automated Grain Sizing (AGS) as done after Graham et al. (2005) which performs image segmentation based on particle boundaries. Programming of the tool was done in the C++ programing language, using OpenCV (Bradski 2000) as the image processing library. The tool was developed at the request of the Federal Highway Administration (FHWA). Keywords: pebble count, grain-size distribution, automated grain sizing

4

5 ACKNOWLEDGEMENTS I would like to thank my family for all of their support and encouragement. My father for his knowledge, example and faith in me and my mother for her foresight, humor and perseverance. I would also thank the faculty at BYU for all of their teaching and support. Specifically Jolene Johnson for all of her support and patience. Also, the seemingly limitless patience of my advisor Jim Nelson. Last but not least, I would like to thank Aquaveo and their employees for this opportunity and all of the support that I received throughout. Specifically Chris Smemoe and Eric Jones for all of their knowledge and patience with my endless questions, and Alan Zundel for providing this opportunity and his mentorship in my development as a software engineer.

6

7 TABLE OF CONTENTS LIST OF FIGURES... ix 1 INTRODUCTION Basis Traditional Grain Sizing Techniques Automated Grain Sizing Technique Federal Highway Administration Contract METHODS Image Capture Hydraulic Toolbox Project Setup Image Pre-Processing Measure Scale Crop to Desired Area Image Segmentation Median Filter Subtract Background Threshold Morphologic Close Watershed Measure Grains / Fit Ellipse Grain-Size Data Processing Correction Factor Result Visualization DISCUSSION v

8 3.1 Difficulties with Automated Grain Sizing Over Segmentation of Large Grains Effects of Lighting on Results User Input Benefits of Automated Grain Sizing Speed Documentation SUMMARY Effectiveness Conclusion REFERENCES vi

9 LIST OF TABLES Table 4-1: Comparison of Wolman vs AGS GSDs...24 vii

10 viii

11 LIST OF FIGURES Figure 2-1: Flowchart of AGS Process (Strom, Kuhns and Lucas 2010)...4 Figure 2-2: Illustration of image capture (Graham, Rice and Reid 2005)...5 Figure 2-3: Hydraulic Toolbox Main Window...6 Figure 2-4: Project with Rock/Sediment Gradation Analysis...6 Figure 2-5: Main Rock/Sediment Gradaition Analysis Window...7 Figure 2-6: Rock/Sediment Gradation Analysis with Image Gradation Added...8 Figure 2-7: Digital Gradation Analysis Definition Screen...9 Figure 2-8: Scale Selection Screen...10 Figure 2-9: Setting Cropping Extents...11 Figure 2-10: Digital Gradation Tool with All Settings Available...12 Figure 2-11: Before (left) and after (right) Median Filter...13 Figure 2-12: Image before (left) and after (right) background subtraction...14 Figure 2-13: Thresholded Pixel Overlay...15 Figure 2-14: Watershed Result Overlay...16 Figure 2-15: Polygon (left) and Ellipse (right) Image Overlays...17 Figure 2-16: Completed Digital Gradation Analysis...18 Figure 2-17: Plot All Gradations Plot Window...19 Figure 2-18: Plot Combined Gradation Plot Window...20 Figure 4-1: Wolman Count Test Images 1 (top left) 2 (top right) 3(bottom left) and 4(bottom right)...23 Figure 4-2: Comparison of Image GSDs...24 Figure 4-3: Comparison of AGS Gradation to Traditional Wolman Count with AGS Correction Factor Figure 4-4: Comparison of AGS Gradation to Traditional Wolman Count with AGS Correction Factor ix

12 x

13 1 INTRODUCTION 1.1 Basis All of my work is based on Strom et al. (2010). Within his paper he compared the methods shown by Graham et al. (2005) with the more traditional methods documented by Wolman (1954). The work done was per contract from the Federal Highway Administration (FHWA), awarded to Aquaveo LLC as additional functionality to be added to their Hydraulic Toolbox (Aquaveo LLC 2013). 1.2 Traditional Grain Sizing Techniques Wolman formalized the practice of generating a grain-size distribution (GSD) in his article in the Transactions of the American Geophysical Union (Wolman 1954). The method Wolman proposed in his paper is industry standard. For the Wolman method, grains are selected along visually identified parallel lines. The spacing for each of the Wolman samples is approximately one operator step. After taking a step, the operator bends down and touches the ground at the point of their toe, without looking. Whatever grain they touch when they touch is then measured (long, intermediate and short axis) and recorded for use in the Wolman count. 1

14 1.3 Automated Grain Sizing Technique The technique used in the tool I developed was originally presented by Graham et al. (2005) in his paper A transferable method for the automated grain sizing of river gravels. In the original development of this technique Graham used Matlab. In their paper, Strom Kuhns, and Lucas (2010) used ImageJ (Gasband ) an application developed in Java for scientific analysis of images. My implementation uses OpenCV (Bradski 2000) to do the image processing required for extraction of the grain sizes for the images. Processing of the grain sizes and generation of graphs and reports is then handled by the gradation portion of the Hydraulic Toolbox (Aquaveo LLC 2013). The method presented by Grahm et al. (Graham, Rice and Reid 2005) uses a digital image of a bed, coupled with image processing techniques to quickly produce a GSD of the surface sediment (Strom, Kuhns and Lucas 2010). The process can be broken down into four major steps: 1. Image collection: Images are gathered from the site from visually similar sections of the sediment bed. 2. Image preprocessing: Cropping and orthorectification of images as well as conversion to greyscale. 3. Image processing and analysis: Run the images through a set of image filters and computer vision algorithms to extract grain sizes. 2

15 4. Numerical sieving: Develop the GSD curve from the measured grain sizes in the images. 1.4 Federal Highway Administration Contract The tool that I developed was commissioned by the Federal Highway Administration (FHWA) via contract with Aquaveo LLC. The FHWA desired to have a tool that could simplify the process presented by Strom et al. (2010) and provide them with documented and reliable GSDs in a shorter amount of time. 2 METHODS The Images to be used in determining a grain size distribution are processed through computer vision techniques to extract the grain sizes from them. Figure 2-1 presents the process as shown by Strom et al. (2010). 3

16 Figure 2-1: Flowchart of AGS Process (Strom, Kuhns and Lucas 2010) 2.1 Image Capture All images captured for use in the AGS system discussed in the paper must be taken orthogonal to the surface of the bed, or another utility must be used to orthorectify them prior to use in this AGS utility. Also, each image must contain two points with a known distance from each other. The scale of the images must be such that the smallest grain of interest has a minor axis larger than 23 pixels (Graham, Rice and Reid 2005). Images can be collected with a standard consumer grade camera, however, care should be taken that the images are as vertical as possible, especially if the user is not going to employ an 4

17 othorectification technique. It is important that the image contain at least two points with known distance from each other, so that a conversion from pixel size to real world size is possible. Figure 2-2: Illustration of image capture (Graham, Rice and Reid 2005) 2.2 Hydraulic Toolbox Project Setup Hydraulic Toolbox (Aquaveo LLC 2013) houses all of the algorithms and user interfaces for the AGS utility. Windows and example images shown in this document were taken from Hydraulic Toolbox, and this document will show the steps required to perform a digital gradation. Upon launching Hydraulic Toolbox the user is presented with the window shown in Figure

18 Figure 2-3: Hydraulic Toolbox Main Window Clicking the toolbar button indicated creates a new rock/sediment gradation analysis and adds it to the current project as shown in Figure 2-4. Figure 2-4: Project with Rock/Sediment Gradation Analysis 6

19 Double clicking the Rock/Sediment Gradation analysis will open it for editing. The main Rock/Sediment Gradaition Analysis window is shown in Figure 2-5. Figure 2-5: Main Rock/Sediment Gradaition Analysis Window Each image gradation added to the project will have its own GSD based on multiple images that the user supplies. The images should be from the same site and represent a region of the bed with similar grains. After an Image Gradation has been added to the project, click the Define Image Gradations button as shown in Figure

20 Figure 2-6: Rock/Sediment Gradation Analysis with Image Gradation Added When the Define Image Gradations button has been pressed, the user is presented with the screen shown in Figure

21 Figure 2-7: Digital Gradation Analysis Definition Screen In the definition screen, the number of photos to be used for the analysis is entered and if the user wants to fine tune their results then the Advanced Controls radio button can be toggled on. Hitting the Browse button next to each of the entries allows the user to locate each image file on disk and assign it to one of the entries. The View button can be used to verify the image content. Once all of the images are added, each one must be set up by pressing the Setup button next to each one of their entries. The process of setting up images is discussed in further detail in sections 2.3 and

22 2.3 Image Pre-Processing Prior to grain size extraction, images must be prepared by selecting the area of the image to use and determining the pixel size in terms of real world units. The result of the preprocessing is a greyscale image of only the region of interest to the GSD Measure Scale An object with known length must be identified within the picture so that the pixels lengths can be converted to real world units. By clicking out two points with known distance and specifying the length between them, a real world pixel size can be established. The pixels are assumed to be uniform squares, and orthorectification based on more than two points is something that could be added in future versions. Figure 2-8: Scale Selection Screen 10

23 2.3.2 Crop to Desired Area All light areas within the image are considered to be part of a grain. If anything that is not a particle is included within the picture then it must be cropped out so that it isn t counted as a grain. During the cropping step, the cropping box should be resized so that only the area to be used for the AGS is within it, as shown in Figure 2-9. Figure 2-9: Setting Cropping Extents If the advanced controls are not turned on then there is no further user interaction required for this image. Clicking on Results within the Views: list box will compute the results for the image and display them in tabular form. If all of the images have their scale line 11

24 and cropping box set, then the Recompute button in the Digital Gradation Analysis Definition screen (Figure 2-7) will run analysis on all images at once. 2.4 Image Segmentation All segmentation operations are performed on a greyscale image that comes from the preprocessing discussed in section 2.3. The result of the image segmentation process is a set of ellipses approximating the grains visible within the image. It is not necessary to set the parameters associated with the image segmentation, however it may be desired for fine tuning of the segmentation results. If the user wants to set these parameters, they may be accessed by toggling Advanced Controls on in the Digital Gradation Analysis Definition screen (Figure 2-7). Once Advanced Controls has been enabled, the user can choose to further fine tune their results by toggling on Advanced Settings in the User Defined Variables as shown in Figure Figure 2-10: Digital Gradation Tool with All Settings Available 12

25 2.4.1 Median Filter Median filters blur the image by assigning the median value of all of the pixels within a given radius to the current pixel. The overall effect removes noise and makes the greyscale rocks appear much smoother. For example, this would make the flecks in a piece of granite blur into the rest of the rock. If the advanced controls are turned on you are given the option to select the radius used in the median filter. The larger the radius is, the more blurred the image will be. Figure 2-11: Before (left) and after (right) Median Filter Subtract Background A rolling ball filter is used to remove gradients and other background information from the image. I programmed this based on the rolling ball background subtraction in ImageJ (Gasband ). The Rolling Ball Background Subtraction algorithm is originally from Stanley Sternberg s article Biomedical Image Processing (Sternberg 1983). 13

26 If the advanced controls are turned on then you have the option of inputting the size of the rolling ball filter radius: the larger the rolling ball is, the more definition is removed from the image. Figure 2-12: Image before (left) and after (right) background subtraction Threshold Determining of the background and foreground (void space and rocks) is determined by a threshold. Pixels with values greater than the threshold are considered to be part of a grain, and all other pixels are considered to be edges. Within the program, the pixels with values above the threshold are displayed as a blue layer on top of the greyscale image. 14

27 Figure 2-13: Thresholded Pixel Overlay The Threshold is typically selected via Otsu s method (Otsu 1979). However, if the advanced controls are turned on it is possible, though not recommended, to manually enter the threshold value. The image created by thresholding is binary, with every pixel that is considered to be part of a rock being on and all void space being off Morphologic Close A morphologic close is the morphologic erosion of a morphologic dilation of the pixels that are on in the binary threshold image. The result is the removal of small holes in the rocks that can result from color abnormalities not removed by median filtering and background subtraction. If the advanced controls are turned on then the number of morphologic iterations can be changed. This is not recommended, but more morphologic iterations means that larger void spaces will be filled in. 15

28 2.4.5 Watershed A topological watershed is applied on a Euclidian distance map of each of the on thresholded pixels to the nearest off pixel. Because of numerical error it is possible to have false maxima in the Euclidian distance map that should not be used to represent the center of a rock. Filtering out the false maxima is done using a flood depth that defaults to 0.9 but is settable if the advanced controls are turned on. The flood depth numerically fills every maxima by the amount specified to see if any further downhill pixels can be found at that depth. If further downhill pixels are found, then the maxima is eliminated and all pixels contributing to that maxima are reassigned to the further downhill pixels. Figure 2-14: Watershed Result Overlay Measure Grains / Fit Ellipse The user has no control over the polygon and ellipse generation. It is done automatically based on the watershed areas. The overlays are provided to display the segmentation results to the user. This helps them to see if the algorithms are finding correctly sized grains in the image and how well those grains correspond to the original image. 16

29 Figure 2-15: Polygon (left) and Ellipse (right) Image Overlays 2.5 Grain-Size Data Processing After a set of ellipses representing the grains is extracted from the image segmentation process discussed in section 2.4, they are numerically sieved based on the small axis of each ellipse. To correct for the inherent differences between AGS and Wolman methods, a correction factor is also applied to each of the sieve bins Correction Factor To compare AGS methods to traditional Wolman Counts, the fact that larger grains tend to be picked up during Wolman Counts (Kellerhals and Bray 1971) must be accounted for. Strom et al. (Strom, Kuhns and Lucas 2010) achieved this by applying a correction factor to the GSD. By turning on the advanced controls in the Digital Gradation Analysis screen it is possible to change the factor. However, by default the correction factor is set to 2.5. Strom et al. (2010) recommends a correction factor of

30 2.6 Result Visualization Once all images have been inputted, setup, and computed the results can be viewed in the Digital Gradation Analysis screen. Particle counts, the diameter-percent-smaller-than values, and the Semi-Log gradation plot are all displayed as shown in Figure Figure 2-16: Completed Digital Gradation Analysis The Plot All Gradations button can be used to see Semi-log gradation plots for each of the images side-by-side as shown in Figure This is useful for seeing if any of the images 18

31 results vary greatly from the other images. Because all of the images should be of a similar bed, their Semi-log plots should be fairly similar. Figure 2-17: Plot All Gradations Plot Window The plot of the combined AGS results can be viewed in the bottom right hand side of the Digital Gradation Analysis screen as shown in Figure 2-16 or by clicking the Plot Combined Gradation button in the bottom left corner of the Digital Gradation Analysis screen. 19

32 Figure 2-18: Plot Combined Gradation Plot Window 3 DISCUSSION 3.1 Difficulties with Automated Grain Sizing Over Segmentation of Large Grains Large and strangely formed grains can be over segmented due to the Euclidian Distance Map forming valleys in the middle of these grains. Also, the watershedding algorithm developed as part of the AGS tool may not be as effective as other implementations available. 20

33 3.1.2 Effects of Lighting on Results Rolling Ball filter attempts to remove gradients from image but is not always successful. My rolling ball algorithm attempts to achieve the same results as ImageJ (Gasband ), but appears to have slightly different results. More work should be done on this algorithm or we could experiment with adaptive thresholding User Input Median Filter Radius, and Rolling Ball Radius can be inputted by the user and should be adjusted based on the number of pixels per grain. If these filters are too large for the image then the filter will blur out the edge of the grain, too small and over segmentation will occur. We attempt to address this issue by automatically resizing the input image to a lower resolution that is still acceptable for AGS. Not only does this make the default filter radii acceptable for the image, it also reduces computation time. 3.2 Benefits of Automated Grain Sizing Speed Using AGS rather than traditional grab techniques can cut the time to create a GSD from hours to minutes (Strom, Kuhns and Lucas 2010) Documentation Because the GSD is based on images, the images can be saved with the GSD and reviewed later if someone wants a more in depth idea of how the GSD was formed and the 21

34 channel conditions. The AGS can also be rerun at a later time if it is felt that the initial run was flawed in some way. 4 SUMMARY 4.1 Effectiveness In 2012 a limited test of the effectiveness of the AGS system outlined in this paper was compared with a typical Wolman Count by the FHWA. A series of four images shown in Figure 4-1 were taken at the same site that a traditional Wolman Count had been performed. The flags in the images are 60 inches apart and were used to establish the pixel to real world unit conversion. 22

35 Figure 4-1: Wolman Count Test Images 1 (top left) 2 (top right) 3(bottom left) and 4(bottom right) Default values were used for the analysis and a correction factor of 2.0 was used per Graham et al. (2005). A graphical comparison of the semi-log gradation plots can be seen in Figure

36 Figure 4-2: Comparison of Image GSDs A comparison of the traditional Wolman Count against the AGS results can be seen in Figure 4-3. The results for this test give AGS output with grains that are about an inch larger than those that were measured by traditional Wolman methods. Wolman Count AGS Method D D D D D Table 4-1: Comparison of Wolman vs AGS GSDs 24

37 Figure 4-3: Comparison of AGS Gradation to Traditional Wolman Count with AGS Correction Factor 2.0 It is possible to get a better correlation between the Wolman and AGS methods if a smaller correction factor is used in the AGS method. However, more analysis and test would be required to see if a smaller correction factor is required generally. The comparison of the Wolman Count against the AGS method with a correction factor of 1.5 can be seen in Figure

38 Figure 4-4: Comparison of AGS Gradation to Traditional Wolman Count with AGS Correction Factor Conclusion The accuracy of the AGS tool outlined in this paper needs further verification before being used as the sole source for a GSD. It could currently be used alongside a traditional Wolman Count to add additional documentation and show its applicability for the situation. It is possible that this AGS utility simply needs to use a smaller correction factor of 1.5. It is also possible that the background subtraction and watershedding algorithms need more work before they can see widespread use in the field. 26

39 REFERENCES Aquaveo LLC. Hydraulic Toolbox. Provo, Utah, Bradski, G. "opencv_library." Dr. Dobb's Journal of Software Tools, Gasband, W S. ImageJ. Bethesda, Maryland, Graham, D. J., S. P. Rice, and I. Reid. "A transferable method for the automated grain sizing of river gravels." Water Resources Research, no. W07020 (2005): 41. Kellerhals, Rolf, and Dale L. Bray. "Sampling Procedures for Coarse Fluvial Sediments." Journal of the Hydrauilcs Division 97, no. 8 (August 1971): Otsu, Nobuyuki. "A Threshold Selection Method from Gray-Level Histograms." IEEE Transactions 9, no. 1 (January 1979): Sternberg, Stanley. "Biomedical Image Processing." IEEE Computer, January Strom, K. B., R. D. Kuhns, and H. J. Lucas. "Comparison of Automated Image-Based Grain Sizing to Standard Peble-Count Methods." Journal of Hydraulic Engineering, August 2010: Wolman, M Gordon. "A Method of Sampling Coarse River-Bed Material." Transactions of the American Geophysical Union 35, no. 6 (1954):

40

41 29