Counting Sugar Crystals using Image Processing Techniques Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Risuna Nkolele, Precious Makganoto, David Merand, Andrew Paskaramoorthy, Nouralden, Lucky Daniel January 22, 2019 ill Seota, Netshiunda Emmanuel, GodsGift Uzor, Counting Risuna Sugar Nkolele, Crystals Precious using Makganoto, Image Processing DavidTechniquesJanuary Merand, Andrew Paskaramoorthy, 22, 2019 1 Noural / 23
Contents Introduction Objectives Data Methodology Results Conclusion Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Counting Risuna Sugar Nkolele, Crystals Precious using Makganoto, Image Processing DavidTechniquesJanuary Merand, Andrew Paskaramoorthy, 22, 2019 2 Noural / 23
Background Commercial sugar crystal analysis consists of: Crystal Habit: A description of the crystals shape. Includes Elongation Ratio, Coefficient Variation, Mean Aperture Sugar Counting: Reflects efficiency of sugar refinement process Figure 1: A microscope image of unrefined sugar from Illovo Sugar in Swaziland Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Counting Risuna Sugar Nkolele, Crystals Precious using Makganoto, Image Processing DavidTechniquesJanuary Merand, Andrew Paskaramoorthy, 22, 2019 3 Noural / 23
Objectives The objectives of the project are: 1 to count the number of sugar crystals per unit of area 2 to determine the length-to-width ratio of each crystal (the Elongation Ratio) 3 to comment on the D shape occurrence We design and implement an image processing pipeline to address the first two objectives Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Counting Risuna Sugar Nkolele, Crystals Precious using Makganoto, Image Processing DavidTechniquesJanuary Merand, Andrew Paskaramoorthy, 22, 2019 4 Noural / 23
Processing Pipeline Our Processing Pipeline consists of the following steps: Pre-processing Segmentation Object Detection Object Classification We compare actual in-sample results with algorithmic results to determine optimal parameters. Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Counting Risuna Sugar Nkolele, Crystals Precious using Makganoto, Image Processing DavidTechniquesJanuary Merand, Andrew Paskaramoorthy, 22, 2019 5 Noural / 23
Data Data consists of: 1300 Microscope images: may differ by resolution, and level of magnification Unrefined Sugar: vary by company, location of the sugar mill, week of the year Refined Sugar: year, the run, the testing point, and duration of heating Spreadsheets of image details: crystal count, crystal size distribution Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Counting Risuna Sugar Nkolele, Crystals Precious using Makganoto, Image Processing DavidTechniquesJanuary Merand, Andrew Paskaramoorthy, 22, 2019 6 Noural / 23
Data: Some Examples a: A Well-behaved Image b: Large Overlap c: Varying Magnification d: High transparency Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Counting Risuna Sugar Nkolele, Crystals Precious using Makganoto, Image Processing DavidTechniquesJanuary Merand, Andrew Paskaramoorthy, 22, 2019 7 Noural / 23
Methodology: Implemented Pipeline Our results are generated from the following pipeline: 1 Conversion to Gray Scale 2 Filtering (Gaussian Blurring) 3 Contrast Stretching 4 Image Dilation 5 Segmentation (Otsu s Method) 6 Object Detection (Countours) 7 Object Classification Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Counting Risuna Sugar Nkolele, Crystals Precious using Makganoto, Image Processing DavidTechniquesJanuary Merand, Andrew Paskaramoorthy, 22, 2019 8 Noural / 23
Methodology: Grayscale Conversion Grayscale conversion changes image from a 3-D array to a 2-D array, simplifying subsequent manipulation. Figure 2: A BW image represented as a 2D matrix. Numbers in the matrix are pixel intensity. An RGB image is represented by three 2D matrices. Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Counting Risuna Sugar Nkolele, Crystals Precious using Makganoto, Image Processing DavidTechniquesJanuary Merand, Andrew Paskaramoorthy, 22, 2019 9 Noural / 23
Methodology: Spatial Filtering The Filtering Transformation: Figure 3: Spatial Filtering. A linear filter equally weights elements in the input mask Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Counting Risuna Sugar Nkolele, Crystals Precious using Makganoto, Image Processing DavidTechniquesJanuary Merand, Andrew Paskaramoorthy, 22, 2019 10 Noural / 23
Methodology: Contrast Enhancement We consider two methods of Contrast Enhancement: Histogram Equalisation Transforms the histogram of pixel intensities to have a uniform empirical Cumulative Distribution Function. Contrast Stretching Enhance contrast using the following function: b (x, y) = ( ) 2 N a (x, y) min (a) 1 max (a) min (a) where a is the original image, b is the transformed image, and 2 N 1 is the dynamic range. Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Counting Risuna Sugar Nkolele, Crystals Precious using Makganoto, Image Processing DavidTechniquesJanuary Merand, Andrew Paskaramoorthy, 22, 2019 11 Noural / 23
Methodology: Contrast Enhancement Figure 4: An example of contrast enhancement applied to an image of sugar crystals. The increase in contrast is represented by the widening of the histogram. Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Counting Risuna Sugar Nkolele, Crystals Precious using Makganoto, Image Processing DavidTechniquesJanuary Merand, Andrew Paskaramoorthy, 22, 2019 12 Noural / 23
Methodology: Dilation and Segmentation Dilation is a morphologocial operation that expands objects and increases connectivity between them. Thresholding is a binary function that sets all pixels above the threshold to the max value (white), and all values below the threshold to min value (black) Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Counting Risuna Sugar Nkolele, Crystals Precious using Makganoto, Image Processing DavidTechniquesJanuary Merand, Andrew Paskaramoorthy, 22, 2019 13 Noural / 23
Methodology: Dilation Figure 5: Dilation has the effect of making the background (light areas) wider and dark areas smaller. The net result is improved noise reduction Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Counting Risuna Sugar Nkolele, Crystals Precious using Makganoto, Image Processing DavidTechniquesJanuary Merand, Andrew Paskaramoorthy, 22, 2019 14 Noural / 23
Methodology: Segmentation Figure 6: A series of automatic thresholding procedures to seperate foreground objects from the background Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Counting Risuna Sugar Nkolele, Crystals Precious using Makganoto, Image Processing DavidTechniquesJanuary Merand, Andrew Paskaramoorthy, 22, 2019 15 Noural / 23
Methodology: Summary Figure 7: Preprocessing: Conversion to Grayscale, Blurring, Contrast Enhancement, Dilation, Segmentation Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Counting Risuna Sugar Nkolele, Crystals Precious using Makganoto, Image Processing DavidTechniques Merand, January Andrew Paskaramoorthy, 22, 2019 16 Noural / 23
Methodology: Summary Figure 8: Preprocessing: Conversion to Grayscale, Blurring, Contrast Enhancement, Dilation, Segmentation Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Counting Risuna Sugar Nkolele, Crystals Precious using Makganoto, Image Processing DavidTechniques Merand, January Andrew Paskaramoorthy, 22, 2019 17 Noural / 23
Methodology: Crystal Detection We want to identify the crystals in the binarised image: Gradients of pixel intensities are calculated to determine the edges of objects. A boundary is drawn where there is a large change in gradient. The boundary is stored in memory as an array of pixels. The length of the array proxies the size of the crystal. 1. A threshold array length was chosen to separate crystals from noise, impurities, and other objects 1 An equivalent micro-meter unit of measurement can be calculated Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Counting Risuna Sugar Nkolele, Crystals Precious using Makganoto, Image Processing DavidTechniquesJanuary Merand, Andrew Paskaramoorthy, 22, 2019 18 Noural / 23
Methodology: Crystal Detection Figure 9: Original and labelled images Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Counting Risuna Sugar Nkolele, Crystals Precious using Makganoto, Image Processing DavidTechniquesJanuary Merand, Andrew Paskaramoorthy, 22, 2019 19 Noural / 23
Results: Crystal Counting Table 1: Actual vs predicted number of crystals in out-of-sample images Image Actual Predicted Error 1 15 20 5 2 21 20-1 3 16 19 3 4 13 15 2 5 17 20 3 6 13 21 8 7 16 26 0 8 16 14-2 9 24 24 0 10 15 18 3 11 16 21 5 12 17 17 0 Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Counting Risuna Sugar Nkolele, Crystals Precious using Makganoto, Image Processing DavidTechniquesJanuary Merand, Andrew Paskaramoorthy, 22, 2019 20 Noural / 23
Methodology: Width-to-Length To calculate the length-to-width ratio: We applied the Hough transform on the segmented image The result is coordinates for a set of lines on the boundary of each object Using co-ordinates, we aimed to approximate a rectangle around each object, but did not complete this step Figure 10: Probabilistic Hough Transformation to provide co-ordinates of boundary lines Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Counting Risuna Sugar Nkolele, Crystals Precious using Makganoto, Image Processing DavidTechniquesJanuary Merand, Andrew Paskaramoorthy, 22, 2019 21 Noural / 23
Challenges Faced IMAGES: Improved quality of images (for instance, images with less blur) could lighten amount of work done on the image processing, since it proved difficult to detect objects that have soft or blurred edges. OVERLAPPING CRYSTALS: The crystals were immersed in a liquid of low density. This made the image segmentation an issue. To gain a depth perception of sorts, one could use different thresholding parameters for the binarisation of the images during the pre-processing phase. TOUCHING CRYSTALS: If two distinct crystals touch, the threshold used in the preprocessing phase was unable to distinguish the two as separate objects. To combat this issue, one could use image erosion on the binarised image to increase the sharpness of edges in the image. Fatigue Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Counting Risuna Sugar Nkolele, Crystals Precious using Makganoto, Image Processing DavidTechniquesJanuary Merand, Andrew Paskaramoorthy, 22, 2019 22 Noural / 23
Conclusion To recap: We developed an automated procedure to count sugar crystals, assuming they are well-separated, with strong out-of-sample performance We were unable to complete the calculation of the length-to-width ratio Future Work and Considerations: Much of the difficulty lies with correctly separating overlapping crystals and identifying transparent crystals Spreadsheets with data labels may allow for supervised learning procedures Bill Seota, Netshiunda Emmanuel, GodsGift Uzor, Counting Risuna Sugar Nkolele, Crystals Precious using Makganoto, Image Processing DavidTechniquesJanuary Merand, Andrew Paskaramoorthy, 22, 2019 23 Noural / 23