Image Processing for Mechatronics Engineering For senior undergraduate students Academic Year 2017/2018, Winter Semester

Image Processing for Mechatronics Engineering For senior undergraduate students Academic Year 2017/2018, Winter Semester Lecture 2: Elementary Image Operations 16.09.2017 Dr. Mohammed Abdel-Megeed Salem Media Engineering Technology, German University in Cairo

Course Info - Contents 1. Introduction 2. Elementary Image Information and Operations 3. Fundamentals of Signal and Image Processing 1. Definition, 2. Basic Signals 3. Sampling and Quantization 4. Image Acqusition and Digitization 5. Sensing and Perception (HVS) and the Color Image Processing 6. Image Operations 1. Point Image Operations 2. Local Image Operations and Filters 3. Global Image Operation and Transforms Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 2

Report 1 Based on your own understanding of what is meant by image, image Processing, image analysis, Computer Vision and Computer Graphics, draw a new Venn diagram similar to the diagram of Castleman. You may include new terms in your diagram, or you may exclude some of the above mentioned terms for simplicity. Image Processing or Computer Vision have many applications in the field of mechatronics, such as vision-based quality control, robot vision, or autonomous vehicles. Discuss one application to show the details and importance of the role of image processing to achieve the applications goals. Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 3

Course Info - Contents 1. Introduction 2. Elementary Image Information and Operations 3. Fundamentals of Signal and Image Processing 1. Definition, 2. Basic Signals 3. Sampling and Quantization 4. Image Acqusition and Digitization 5. Sensing and Perception (HVS) and the Color Image Processing 6. Image Operations 1. Point Image Operations 2. Local Image Operations and Filters 3. Global Image Operation and Transforms Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 4

Objective The objective of this lecture is to gain a touchable feeling about the basic information of images and basic operation may be operated on images. This understanding will help us in the coming lecture to study the fundamentals of Signal and Image Processing. For you to achieve this objective you should try to implement these operations on images created by your own or, when this is not available, on standard set of images. Ex: http://imagedatabase.cs.washington.edu/groundtruth/ Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 5

Simple Example irgb = imread('fabric.png'); [Rows, Cols, Clrs] = size(irgb); for (r = 1:Rows) for (c = 1:Cols) igray(r,c) = (irgb(r,c,1) + irgb(r,c,2) + irgb(r,c,3) )/3; end end Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 6

Report 2-1 Use the last image RGB to gray example to convert a gray level image to binary image. Use one of your own images. Write down your observations/ comments. What about converting a gray level image into colored image? Is it possible to restore the original RGB image? Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 7

Image Statistics An empirical quantities is a measurable real characteristics of a sample (difference to the characteristics of a probability function) Fg: empirical cumulative distribution function Pi: frequency distribution Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 8

Image Statistics m 1 = 127, σ 2 = 0 m 1 = 127, σ 2 = 127 x 127 σ 2 =? m 1 =? Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 9

Image Statistics Image Size: total number of pixels. Minimum gray level, Maximum gray level, Range, the difference between the maximum and the minimum. Entropy: a measure of randomness used to characterize the texture of the image. Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 10

Image Statistics Image Size: 768x1024 = 786432 Minimum gray level: 0 Maxmum gray level: 255 Range: 255 Mean gray level: 142,9 Mode gray level: 160 Median gray level: 151 Entropy: 7.7554 Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 12

Report 2.2 Compute the mean and standard deviation of the given image: Are the discussed statistical features enough to identify certain image from a collection of images? Support your answer with examples. Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 13

Image Histogram Histogram is a graph that shows frequency of anything. Histograms usually have bars that represent frequency of occurring of data. Histogram has two axis: x and y. The x axis contains the event whose frequency we count. The y axis contains frequency. Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 14

Image Histogram Provides information on grey scale distribution Represents the Discrete Probability Function for the colour distribution h i = x y I(x, y) Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 15

Image Histogram Most used: normalized histogram with relative frequencies h g of grey values For M pixel with G grey values (g = 0, 1, G-1) and a g as number of pixels with grey value g holds: and and Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 16

Image Histogram Interpretation of typical histograms Homogeneous image: one bar of length 1 Two level image: two bars Dark image with low contrast: there are more values of h g for lower g Bright image with low contrast: there are more values of h g for larger g High contrast image: all h g equal (e.g. = 1 / 256) Typical foreground / background image: bimodal histogram Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 17

Image Histogram Marsa Matrouh - Dark Marsa Matrouh Marsa Matrouh - Light Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 18

Image Histogram Marsa Matrouh High Contrast Marsa Matrouh Marsa Matrouh Low Contrast Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 19

Report 2.3 Apply the operations of increasing and decreasing the brightness and contrast on one of your own images. Show the histogram of the output images in comparison to the input image and comment the results. Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 20

Image Histogram Nearly bimodal histogram Meffert and Salem Image Processing for Mechatronics Engineering Lecture 21 21

Image Histogram Properties of histograms Spatial information is lost Every image has a special histogram BUT: every histogram is not assigned to a single image Example: if objects in an image were rearranged then the histogram is not changed The structure of an image can not derived from the histogram Histogram tells the user if an image is scaled well Meffert and Salem Image Processing for Mechatronics Engineering Lecture 21 22

Image Histogram Two different images and their (identical) histogram 10 Meffert and Salem Image Processing for Mechatronics Engineering Lecture 21 23

Image Histogram N-dimensional histogram Useful for a lot of joint processes In image processing (e.g.): multi-channel images Definition with number of corresponding pixel pairs Example: number of pixels having the value g μ in the matrix for red and g ν in the matrix for blue Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 24

Image Statistics - Histogram red Two-dimensional histogram number of pixels with the two defined grey values blue Meffert and Salem Image Processing for Mechatronics Engineering Lecture 21 25

Image Statistics - Histogram The baboons colour image BUNT Meffert and Salem Image Processing for Mechatronics Engineering Lecture 21 26

Image Statistics - Histogram G B G R B Color R BUNT Meffert and Salem Image Processing for Mechatronics Engineering Lecture 21 27

Image Statistics - Histogram BUNT R G 5 red 5 green hr labe l 512 hg labe l 512 0 100 200 300 label 0 100 200 300 label Meffert and Salem Image Processing for Mechatronics Engineering Lecture 21 28 H1

Image Statistics - Histogram BUNT R B 5 red 5 blue hr labe l 512 hb labe l 512 0 100 200 300 label 0 100 200 300 label Meffert and Salem Image Processing for Mechatronics Engineering Lecture 21 29 H4

Image Statistics - Histogram BUNT G 5 green B 5 blue hg labe l 512 hb labe l 512 0 100 200 300 label 0 100 200 300 label Meffert and Salem Image Processing for Mechatronics Engineering Lecture 21 30 H4

Wikipedia: Mandrillus is the genus of the mandrill and its close relative the drill. These two species are closely related to the baboons, and until recently were lumped together as a single species of baboon. Both Mandrillus species have long furrows on either side of their elongated snouts. The adult male mandrill's furrows are blue, while the furrows of the drill are black. Both species are terrestrial, living on the ground of the rainforest and occasionally grasslands of Central Africa. Foto: Haremsmännchen Meffert and Salem Image Processing for Mechatronics Engineering Lecture 21 31

Report 2.4 Find and discuss an application of the 2D Histogram for land use classification in satellite images. Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 32

Image Statistics - Histogram Cumulative Histogram The cumulative histogram H(i) at gray level i of an image I is given as: H i = This encodes the fraction of pixels with an intensity that is equal to or less than a specific value. i h(i) Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 33

Image Statistics - Histogram Cumulative Histogram Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 34

Histogram Equalization k n k j sk T( rk ) n j 0 j 0 p r ( r j ) Histogram equalization (HE) results are similar to contrast stretching but offer the advantage of full automation, since HE automatically determines a transformation function to produce a new image with a uniform histogram.

Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 36

Basic Image Processing Operations Image-Processing operations may be divided into 3 classes based on information required to perform the transformation. 1) Point Operations process according to the pixel s value alone (single pixel). 2) Neighborhood processing process the pixel in a small neighborhood of pixels around the given pixel. 3)Transforms process entire image as one large block Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 37

Point Operations The gray value of each pixel in the output image g(x, y) depends on the gray value of only the corresponding pixel of the input image f (x,y). Every pixel of f(x, y) with the same gray level maps to a single (usually different) gray value in the output image. Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 38

Image Arithmetic Image arithmetic is the implementation of standard arithmetic operations. Addition Subtraction Multiplication Division Complement Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 39

Image Arithmetic Let x is the old gray value, y is the new gray value, c is a positive constant. Addition: y = x + c Subtraction: y = x - c Multiplication: y = cx Division: y = x/c Complement: y= 255 - x Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 40

Image Arithmetic Image arithmetic has many uses in image processing for example, image subtraction. Image Arithmetic Saturation Rules: The results of integer arithmetic can easily overflow the data type for storage. Ex: unit8 [0-255] Result Class Truncated Value 300 uint8 255-45 uint8 0 10.5 uint8 11 Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 41

Report 3.1 Discuss different techniques to re-range the new image values of the output images after applying the arithmetic operations, back into the range of [0,255]. Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 42

Image Arithmetic - Addition Lighten the image Some details may be lost + 128 = Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 43

Image Arithmetic - Subtraction Darken the image Some details may be lost - 128 = Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 44

Image Arithmetic - Multiplication Lighten the image Some details may be lost but less than addition * 2 = Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 45

Image Arithmetic - Division Darken the image Some details may be lost but less than subtraction / 2 = Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 46

Image Arithmetic Addition vs Multiplication Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 47

Image Arithmetic Subtraction vs Division Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 48

Image Arithmetic Complement Create the negative image Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 49

Image Arithmetic Image arithmetic functions can be used in combination to perform a series of operations. I = imread('rice.png'); I2 = imread('cameraman.tif'); K = imdivide(imadd(i,i2), 2); % not recommended Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 52

Image Arithmetic imlincomb performs all the arithmetic operations in the linear combination in double precision and only rounds and saturates the final result. K = imlincomb(.5,i,.5,i2); % recommended Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 53

Report 3.2 Apply the image arithmetic operations on one of your own images or on MarsaMatrouhGrey image. Show the histograms and comment the results. 3.3 Use the function imlincomb to turn a new or recent (personal) photo into an old photo. Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 54

Image Arithmetic Denoising/Sharping Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 55

Image Arithmetic Fault Detection Where is the defect? Image g(x,y) (no defect) Image f (x,y) (with defect) Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 56

Report 3.4 Consider the previous images of circuits without and with defects. Apply manual alignment to align both images to each other and then apply the subtraction to find the defect. Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 57

Image Arithmetic Moving Object Detection Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 58

Image Arithmetic Moving Object Detection Identify the active cells? Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 59

Thresholding Segmentation can be achieved simply by thresholding the image at a particular intensity level. The technique that s most frequently employed for determining thresholds involves the histogram of the image. Problems: ignores spatial context, selection of the threshold parameter Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 60

Thresholding Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 61

Thresholding Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 62

Thresholding Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 63

Thresholding Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 64

Thresholding Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 65

Double Thresholding A larger threshold T2 yields seeds of the segmentation, which are accepted in all cases. They grow in all directions where grey values can be found that exceed some smaller threshold T1. method does take into account spatial context Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 66

Report 3.5 Program a small image processing code for fault detection for textile or glass industry. For example your program should be able to detect broken/ splitted/ cracked glass. Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 67

Reading M. A.-M. Salem, Multiresolution Image Segmentation, Humboldt-Universität zu Berlin, 2008, [Chapter 2] Rafael G. Gonzalaz and Richard E. Woods, Digital Image Processing, 3 rd Edition, Pearson Edu., 2008. [Chapter 1] E.R. Davies, Machine Vision: Theory, Algorithms and Practicalities, Third Edition, Morgan Kaufmann Publishers, 2004. [Chapter 1] Bovik, A.C.: Handbook of Image and Video Processing. Academic Press, May 2000. Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 68

Contacts Image Processing for Mechatronics Engineering, for senior students, Winter Semester 2017 Dr. Mohammed Abdel-Megeed M. Salem Media Engineering Technology, German University in Cairo Office: C7.311 Tel.: +2 011 1727 1050 Email: mohammed.salem@guc.edu.eg Salem, Image Processing for Mechatronics Engineering, Winter Semester 2017 Lecture 2 69