CT336/CT404 Graphics & Image Processing Section 9 Morphological Techniques
Morphological Image Processing The term 'morphology' refers to shape Morphological image processing assumes that an image consists of structures that may be handled by mathematical set theory Normally applied to binary (B&W) images e.g. after applying thresholding A 'set' is a group of pixels
Set Theory a) Pixel x is an element of the set A b) Sets A and B (overlapping) c) The shaded set C is a subset of A d) The union of A and B e) The complement of A f) The intersection of A and B
Set Morphology Morphological image processing operates at a low level (small regions of pixels) The three black objects A,B,C are very similar in size (i.e. number of pixels) but different in morphology/shape We can describe morphological differences is in terms of intersections with 'test' sets e.g. several possible translations of D will fit into A, but none will fit into B D as a test for compactness? Reference pixel Test sets, or 'structuring elements such as D allow analysis of shape
Basic Morphological Operations Morphological operations transform an image => changing a pixel from black to white (or vice versa) if a defined structuring element 'fits' at that point These operations can: extract information regarding the components contained in a binary image transform an image to remove or retain objects satisfying some criteria regarding their structure The shape+size of the structuring element directly affects the morphological operation and the information about the image that its obtains
Erosion If D is a structuring element and A is a set of black pixels then the erosion of A by D is defined as the set of all pixel locations for which D placed at those locations is a subset of A Here, the previous image is eroded by D If D is placed with its reference pixel at (i, j), then D consists of the 4 pixels (i, j), (i+1, j), (i, j+1) and (i+1, j+1) If and only if all of these pixels are black will pixel (i, j) remain black in the eroded image
Dilation, Opening, Closing Dilation is normally defined as the complement (opposite) to erosion, i.e. the dilation of a set of black pixels on a white background is equivalent to the erosion of the white background Opening is an erosion followed by a dilation Closing is a dilation followed by an erosion If a noisy image is segmented by thresholding, the resulting boundaries will often be ragged, the objects will have false holes in them, and the background will be peppered with small noise objects: successive openings and closings can often improve this without distorting larger shapes => excellent for image cleanup prior to further processing
Examples a) original set b) structuring element c) erosion d) dilation e) opening (=erosion then dilation) f) closing (=dilation then erosion) Note that sequences of opening and closing are often applied with structuring elements of increasing size, since the earlier operations remove small speckle features that would otherwise interfere with the later operations
Successive Erosion+Dilation Here erosion then dilation isolates the larger squares
Image cleanup with basic morphological operators
Canvas code example A series of erosions and dilations will allow us to isolate this white rectangle from the noisy background, while removing the noise on the rectangle itself The exact size of the rectangle will be preserved by using the same number of erosions as dilations 05_Morphological_1.html
<html> <head> <script src="imageprocessingfunctions.js"></script> <style> input { width:25px; </style> <script> // these globals are set in readimage() var imgwidth, imgheight; var ctx; function loadimage() { var textbox = document.getelementbyid("filename"); var filename = textbox.value; var canvas = document.getelementbyid("canvas"); readimage(canvas,filename); function processimage() { var imagedata = ctx.getimagedata(0, 0, imgwidth, imgheight); greyscale(imagedata); threshold(imagedata, 127); erode(imagedata, 13); dilate(imagedata, 13); dilate(imagedata, 13); erode(imagedata, 13); ctx.putimagedata(imagedata, 0, 0); </script> </head> <body> <canvas id="canvas" style="border-width:1px; border-style:solid;" width="500" height="500"></canvas><br> <input type="text" id="filename" style="width:600px;" value="images/white_square_noisy.jpg"> <input type="button" style="width:100px;" onclick="loadimage();" value="load Image"><br> <input type="button" style="width:180px;" value="process Image" onclick="processimage();"> </body> </html>
Extra functions in imageprocessingfunctions.js function erode(imagedata,kernelsize) { // assumes image has been thresholded (i.e. colours are all 0 or 255) var halfkernelsize = Math.floor(kernelSize/2); // create a temporary array for output var outputdata = new Array(); var sz = imagedata.data.length; for (i=0;i<sz;i++) outputdata[i] = imagedata.data[i]; var wid = imagedata.width; var hgt = imagedata.height; for (x=1;x<wid-halfkernelsize;x++) { for (y=1;y<hgt-halfkernelsize;y++) { var index = (x + y * wid) * 4; if (imagedata.data[index]==255) { // this pixel is white so needs to be processed erodeouterloop: for (xx=-halfkernelsize;xx<=halfkernelsize;xx++) { for (yy=-halfkernelsize;yy<=halfkernelsize;yy++) { var index2 = (x+xx + (y+yy)*wid) * 4; if (imagedata.data[index2]==0) { // pixel at x,y needs to be removed outputdata[index] = outputdata[index+1] = outputdata[index+2] = 0; break erodeouterloop; // exit xx and yy loops // copy data from output array to original array for (i=0;i<sz;i++) imagedata.data[i] = outputdata[i];
Extra functions in imageprocessingfunctions.js function dilate(imagedata,kernelsize) { // assumes image has been thresholded (i.e. colours are all 0 or 255) var halfkernelsize = Math.floor(kernelSize/2); // create a temporary array for output var outputdata = new Array(); var sz = imagedata.data.length; for (i=0;i<sz;i++) outputdata[i] = imagedata.data[i]; var wid = imagedata.width; var hgt = imagedata.height; for (x=1;x<wid-halfkernelsize;x++) { for (y=1;y<hgt-halfkernelsize;y++) { var index = (x + y * wid) * 4; if (imagedata.data[index]==0) { // this pixel is black so needs to be processed erodeouterloop: for (xx=-halfkernelsize;xx<=halfkernelsize;xx++) { for (yy=-halfkernelsize;yy<=halfkernelsize;yy++) { var index2 = (x+xx + (y+yy)*wid) * 4; if (imagedata.data[index2]==255) { // pixel at x,y needs to be added outputdata[index] = outputdata[index+1] = outputdata[index+2] = 255; break erodeouterloop; // exit xx and yy loops // copy data from output array to original array for (i=0;i<sz;i++) imagedata.data[i] = outputdata[i];
Another example: accurate estimation of bubble size Count white pixels! 05_Morphological_2.html
function processimage() { var imagedata = ctx.getimagedata(0, 0, imgwidth, imgheight); greyscale(imagedata); // threshold at 52 for initial segmentation threshold(imagedata, 52); // invert the pixels so that foreground colour is 255, background is 0 invert(imagedata); // erosion with 8x8 kernel removes nearly all noise erode(imagedata, 8); // dilation with kernel size 8x8 // original object is almost back to where it was in terms of size/shape // (what I have performed with erosion then dilation is actually opening) dilate(imagedata, 8); // closing with kernel size 30x30 fills in the gap caused by the shine, without affecting it // very much in any other way dilate(imagedata, 30); erode(imagedata, 30); ctx.putimagedata(imagedata, 0, 0); var whitepixels = countpixels(imagedata,255); window.alert("bubble size: "+whitepixels+" pixels");
function invert(imagedata) { // assumes image is greyscale (i.e. RGB all the same) // so we will only read the Red channel var wid = imagedata.width; var hgt = imagedata.height; for (x=0;x<wid;x++) { for (y=0;y<hgt;y++) { var index = (x + y * wid) * 4; var grey = imagedata.data[index]; if (grey==255) grey=0; else grey=255; imagedata.data[index] = imagedata.data[index+1] = imagedata.data[index+2] = grey; function countpixels(imagedata,matchvalue) { // assumes image is greyscale (i.e. RGB all the same) // so we will only read the Red channel var count=0; var wid = imagedata.width; var hgt = imagedata.height; for (x=0;x<wid;x++) { for (y=0;y<hgt;y++) { var index = (x + y * wid) * 4; if (imagedata.data[index]==matchvalue) count++; return count; Extra functions in imageprocessingfunctions.js
The Distance Transform Replaces pixels of one value (black or white) in a binary image with their distance to the nearest pixel of opposite value (white or black) Useful for granulometry (=studying the size distribution of objects) this allows the study of groups of objects: How many? What size? How are they distributed spatially? The assumption is that a local maximum is the centre of a distinct object: use non-maximal suppression to remove all other pixels
Operations Preserving Structural Connectivity Morphological operations that do not fragment objects Thinning: reduces objects to a thickness of 1 pixel Uses 8 structuring elements that remove pixels from the outside edges of an object The structuring elements are defined to identify border pixels but to reject 'critical pixels' whose removal would fragment an object. => preserves connectivity, unlike erosion thinning
Thinning kernels In these depictions, a white pixel is one that must be part of an object, a black pixel is one that must not be part of an object, and a grey pixel is one that does not matter. If any one or more of the kernels does not fit, then the pixel in question is removed These kernels are applied by passing through the image repeatedly until an entire pass removes no pixels
Thinning Thinning is a connectivity operation - it preserves connectivity and allows topological (structural) properties such as the following to be determined: number of branches (a branch junction is a pixel with exactly 3 neighbours) central axis of object orientation total length (by counting the remaining pixels)
Thinning example
Additional function in imageprocessingfunctions.js
Thinning example 05_Morphological_3.html Also try with images/blob_for_thinning.jpg
Pruning Often applied after thinning in order to remove spurs The technique is to look for pixels with only one neighbour that are close to a branch junction. ('Close to' means less than a predetermined distance e.g. 15 pixels). Spurs can be removed by a series of operations to remove all endpoints (thereby shortening all branches), followed by a series of operations to grow branches back by the same amount (by cross-referencing with the original image you could obtain undistorted re-growth) A complete pruning (removing all end points) preserves only closed loops
Pruning function: 1 of 2
Pruning function: 2 of 2
Thinning+Pruning example 05_Morphological_4.html Also try with images/blob_for_thinning.jpg
Example: PCB Defect Analysis
PCB analysis: actions Pads: Threshold at 127, then try 150, then try 175. 150 seems best as it doesn t merge together any separate objects Invert it (so that foreground is white/255 and background is black/0) Clean up using Closing (=dilation then erosion) with template size 3x3 Erosion with template size 6 to try to find pads. Then try size 10: this robustly finds pads Traces: Back to thresholded image: then do thinning. Note the problem in that 2 traces were connected incorrectly by the thresholding. Go back to original image, threshold at 140. Invert, and apply thinning. Prune at length 25. It s still not quite finished. You d need to crossreference with the position of extracted pads, and then identify any endpoints not due to them as being trace-breaks.
Exercise! Implement the above algorithmic steps using the code provided in imageprocessingfunctions.js Downloadable on course webpage