Digital images Digital Image Processing Fundamentals Dr Edmund Lam Department of Electrical and Electronic Engineering The University of Hong Kong (a) Natural image (b) Document image ELEC4245: Digital Image Processing (Second Semester, 2017 18) http://wwweeehkuhk/ elec4245 (c) Synthetic image (d) Satellite image E Lam (The University of Hong Kong) ELEC4245 Jan Apr, 2018 1 / 19 E Lam (The University of Hong Kong) ELEC4245 Jan Apr, 2018 2 / 19 Digital images Varieties of digital images What and how to process depends on many factors: Image type (grayscale vs color) Image size Mechanism for image formation (eg, infra-red, visible light, magnetic signal, etc) Image content Viewer (human vs machine; professional vs amateur) Applications (e) MRI (magnetic strength) (f) Schlieren photography (air density) 1 1 photo credit: wwwmakezinecom E Lam (The University of Hong Kong) ELEC4245 Jan Apr, 2018 3 / 19 E Lam (The University of Hong Kong) ELEC4245 Jan Apr, 2018 4 / 19
Varieties of processing Applications of image processing Traditional topics: Enhancement Restoration Compression Color More recent topics: Super-resolution Digital watermarking Vision Segmentation Pattern recognition Computational imaging and photography Augmented reality Consumer electronics: digital camera, printer, scanner, Medical and biological uses: microscopy, radiology, surgery, Industrial uses: inspection, metrology, defect detection, Smart world: Intelligent transportation system, smart home, Military uses: target identification, surveillance, Scientific uses: hyperspectral imaging, remote sensing, astronomy, We won t have time for all of them But we ll select from both categories E Lam (The University of Hong Kong) ELEC4245 Jan Apr, 2018 5 / 19 E Lam (The University of Hong Kong) ELEC4245 Jan Apr, 2018 6 / 19 EEE signal processing image processing Image processing is also deeply rooted in engineering How we acquire an image is an engineering process Images are signals The processing of an image is also an engineering process Image representation That is what we will do for the entire semester: process signals E Lam (The University of Hong Kong) ELEC4245 Jan Apr, 2018 7 / 19 E Lam (The University of Hong Kong) ELEC4245 Jan Apr, 2018 8 / 19
Image representation Image representation In many cases, we treat images as a two-dimensional signal: g(x, y) or g[x, y] Image is usually rectangular (so it is essentially a matrix) Each value represents a pixel: picture element (In 3D, a pixel becomes a voxel: volume element ) An image of size M N means M pixels horizontally, N pixels vertically Commonly, x = horizontal and y = vertical g(0, N 1) g(1, N 1) g(m 1, N 1) g(0, N 2) g(1, N 2) g(m 1, N 2) g(x, y) = g(0, 0) g(1, 0) g(m 1, 0) BUT there are other conventions, such as Start counting with 1 instead of 0 First index is row (vertical), second index is column (horizontal): the normal way to denote a matrix E Lam (The University of Hong Kong) ELEC4245 Jan Apr, 2018 9 / 19 E Lam (The University of Hong Kong) ELEC4245 Jan Apr, 2018 10 / 19 Obtaining a digital image g(x, y) has discrete and finite indices and discrete and finite values How do we obtain it from a general 2D function? eg, using a digital sensor to capture continuous light distributions To do so, we need sampling and quantization An image usually has a certain size, beyond which it does not exist (not defined) ie g(x, y) 0 only for X min < x < X max and Y min < y < Y max This is also called the finite support of the image Sampling is usually regular, eg x apart in the horizontal and y apart in the vertical directions The digital image therefore has X max X min x Y max Y min y pixels Image as matrix Image as a matrix of numbers: intuitive or revolutionary? Your mind does not process images as a matrix of numbers There is a growing interest in the image processing community to seek alternative representations of images that are closer to how our mind considers, eg, with objects and shapes On the other hand, some argues that certain applications demand a higher dimensional matrix this is known as tensor Multispectral images: each spatial location has a vector of intensities corresponding to different wavelengths Video: each spatial location has intensity values that change with time E Lam (The University of Hong Kong) ELEC4245 Jan Apr, 2018 11 / 19 E Lam (The University of Hong Kong) ELEC4245 Jan Apr, 2018 12 / 19
Resolution Pixels and resolution are related Pixel count is (commonly, but not so accurately) called the resolution of the image Pixel has no size = a digital picture has no (physical) size If we need to print out a digital picture on a certain physical object (eg 4 6 photo paper), we can calculate the effective size of the pixel It may not be square We can color the entire effective size of the pixel to be of the same color (or do something smarter) Often, more pixels mean better image quality or resolution But not always Resolution Signal processing says: We can artificially add more pixels without improving the image quality, by interpolation This is often the case for digital zoom (a) 36 50 (b) 360 500 E Lam (The University of Hong Kong) ELEC4245 Jan Apr, 2018 13 / 19 E Lam (The University of Hong Kong) ELEC4245 Jan Apr, 2018 14 / 19 Simple operations Simple operations Image translation: g (x, y) = g(x 1, y 1) Boundary problem: eg, what is g (0, 0)? Translation by a fraction of a pixel? (Sub-pixel shift) Image rotation: g (x, y) = g(y, x) How to do image stretching, shearing, distortion, etc? 1 Coordinate transformation: figuring out how a pixel in the new image is related to (a group of) pixels in the original image 2 Fill in missing values, if needed Rotation by arbitrary angle? Boundary problem? E Lam (The University of Hong Kong) ELEC4245 Jan Apr, 2018 15 / 19 E Lam (The University of Hong Kong) ELEC4245 Jan Apr, 2018 16 / 19
Resolution Imaging system representation So how do we characterize resolution? Intuitively, related to sharpness More a property of how the images are formed (the camera) than a property of the digital images themselves (the number of pixels) Quantifiable The way images are formed is called imaging Often, we treat the imaging system as linear, time-invariant (LTI) Convolution is extended to two dimensions 1D: output = input channel 2D: image = object filter/blur Fourier transform is also extended to two dimensions 1D: F {output} = F {input} F {channel} 2D: F {image} = F {object} F {filter/blur} It is assumed that you understand these well from Signals and Linear Systems We will have some review, but only very briefly E Lam (The University of Hong Kong) ELEC4245 Jan Apr, 2018 17 / 19 E Lam (The University of Hong Kong) ELEC4245 Jan Apr, 2018 18 / 19 Summary Consider the difference between an object and an image Object Three-dimensional Continuous in time Continuous in space Continuous in spectrum Image g(x, y) Two-dimensional A single snapshot in time Discrete in space Several color components Recurrent questions: Is g(x, y) a faithful representation of the object? What did we lose? Is it possible to retain and reconstruct more information about the object? E Lam (The University of Hong Kong) ELEC4245 Jan Apr, 2018 19 / 19