ENGG1015 Digital Images - PDF Free Download

ENGG1015 Digital Images 1 st Semester, 2011 Dr Edmund Lam Department of Electrical and Electronic Engineering The content in this lecture is based substan1ally on last year s from Dr Hayden So, but all errors should be blamed on me

Back to top- level High Level This week Applications Systems Digital Logic Image & Video Processing Computer & Embedded Systems Computer Network Mobile Network Combinational Logic Boolean Algebra Circuits Basic Circuit Theory Low Level Electrical Signals Voltage, Current Power & Energy 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 2

Digital Images Note: The three parts are mostly independent (even with different language ), but they do intersect. Representation Hardware Processing 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 4

1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 5

Representing Images R bitmap G B pixel An image is broken down into small regions called picture elements (pixels) Digital image (bitmap): A pixel-by-pixel representation of an image. Implications? 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 6

Image Dimensions Image Size The number of pixel in X-Y direction Sometimes quoted using the total number of pixels in a picture (N megapixels) 14 15 Image Resolution The density of pixels Measured by pixel-per-inch (PPI) NOT the number of pixels 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 7

Representing Pixels Each pixel is represented by one or more values Black & white images (binary images): Each pixel is represented by exactly 1 value (B or W) 1 bit is enough to represent 2 possible values Grayscale images: Each pixel is usually a byte (8 bits), keeping the brightness or gray levels Color images: Each pixel represented a group of color components of that location often three colors Different color systems: RGB, CMYK, YCbCr, etc Hyperspectral images: Many values per pixel location, corresponding to different frequencies 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 8

Binary and Grayscale Images B&W B&W (w/ dither) Grayscale Binary Image Each pixel is 1 bit, either 0 or 1 Dithering is used to produce (fake) different intensities Grayscale Image Each pixel is usually a byte (8-bit), keeping the brightness or gray levels 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 9

Color Images 16 colors 256 colors 16M colors indexed color image # of color support depends on the # of bits for each pixel 4 bits 2 4 =16 colors 8 bits 2 8 =256 colors Color Look-Up Tables (LUTs) Color palette 24-bit color image Each pixel is represented by 3 bytes using a certain color model Supports 256x256x256 colors 16 million colors 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 10

RGB Color Model Additive color model Primary colors: Red, Green, and Blue Secondary colors obtained by additive mixing of primary colors: Cyan, Magenta, Yellow Commission Internationale d'eclairage (CIE) in 1931 specifies red to be 700nm, green to be 546.1nm and blue to be 435.8nm Used in media that transmit light (e.g. TV) 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 11

CMY Color Model Subtractive color model Subtractive primaries: Cyan, magenta, and yellow A subtractive primary absorbs a primary color and reflects the other two E.g. Cyan absorbs red and reflect blues and green Used in printing device 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 12

Colors that Can be Reproduced 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 13

Printing an Image Print Size Depends on the mapping between printer s resolution, image resolution & image size A Printer s printing resolution is usually higher than an image s resolution because multiple dots of ink are needed to created color of an image pixel Color Space On screen display: (additive) Printing devices: (subtractive) Color Production Each pixel may have different color Each ink drop has only on-off (one bit!) 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 14

Dithering Create the illusion of new colors and shades by varying the pattern of dots. E.g. Newspaper photographs are dithered. If you look closely, you can see that different shades of gray are produced by varying the patterns of black and white dots. There are no gray dots at all. 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 15

Dither, Halftone, Grayscale original dither halftone 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 16

RGB Color Space The RGB model describes the formation of color by linearly mixing different portion of Red, Blue and Green light. Color is represented by a triplet {r,g,b}, which indicates the weighting coefficients We often normalize the coefficients to be between 0 and 1 (inclusive), or integers between 0 and 255 (8-bit). 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 17

More Color Models Both RGB and CMY(K) model specify linear combinations of the primaries But they have little resemblance to how human beings reason about colors E.g. How do you get the RGB values of the pale orange color on the right? [R G B] = [204 131 42] [R G B] = [???] [248 215 152] 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 18

HS(B/V), HSL, HSI Color Model The family of HSx models describe colors similar to how human perceives colors Also similar to how painters create colors HSB: Hue Saturation Brightness HSV: Hue Saturation Value HSL: Hue Saturation Lightness HSI: Hue Saturation Intensity Similar, but often comes with confusing (or even contradicting) definitions 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 19

Cylindrical- Coordination Hue: The dominant color The angle away from red Saturation The amount away from the center How full the color is Lightness/ Brightness/Value The amount of white/black added 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 20

Luminance- chrominance Another common alternative: a luminancechrominance representation One value for luminance (Y): the brightness, or achromatic image Y = 0.2126 R + 0.7152 G + 0.0722 B Need two more numbers for the chrominance YUV and YCbCr Why? TV broadcast and digital picture compression 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 21

More Image Representations? Raster image (bitmap image) - Raster graphics uses pixel values to describe an image. The file size is independent of the image complexity. For higher resolution, the file size increases dramatically Vector graphics (draw graphics) - An alternate approach is to use only instructions for drawing lines, circles, ellipses, curves, and other shapes. 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 22

Vector Graphics Vector-based images are composed of key points and paths which define shapes, and coloring instructions, such as line and fill colors. Example: 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 23

Vector Graphics Advantages Vector graphics can be scaled up and down easily and quickly while retaining the quality of the picture. Raster images scale poorly and display poorly at resolutions other than that for which the image was originally created. Vector graphics require less bandwidth and can be accessed and viewed faster than raster graphics. Vector graphics can be edited and manipulated far easier than raster images. 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 24

(Partial) Summary Many decisions to make No universally best options Depends on the physical system, e.g. monitor vs printer Depends on the requirement, e.g. color vs grayscale Intersects with other fields e.g. Psychology (visual science) How about our video chats problem? 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 25

1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 26

Image Processing Used in digital camera, TV, cell phones Used in all kinds of photo editing SW e.g. Photoshop, GIMP 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 27

Image Processing - Examples 1st semester, 2011 Original Grayscale Blur Edge Detection Digital Images - ENGG1015 - Dr. E. Lam 28

RGB to Grayscale Conversion Each pixel of a grayscale image has only one intensity value, V High V: white, Low V: black Easiest conversion: V = R + G + B 3 Produce better result if you weight G and R more than B Human eyes are more sensitive to green and red 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 29

Basic Filtering: Windowing Filters are building blocks of image processing systems One of the most basic filtering method is by windowing y[r,c] = i, j 1 h[i, j] r 1 1 i= 1 j = 1 h[i, j]x[r + i,c + j] c 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 30

Windowing in Action X H Y 12 8 27 26 54 48 14 9 16 8 29 9 3 11 10 15 50 60 8 12 34 2 29 52 17 2 44 35 56 72 1 2 1 2 4 2 1 2 1 14 19 34 22 39 43 34 63 77 1 16 89 + 2 29 816 + 1 29 98 2 10 15 11 + 4 15 50 10 + 2 50 60 15 1 34 12 2 + 2 292 34 + 1 29 522 + + = 14 19 34 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 31 16

Gaussian Blur A simple but effective way to blur a picture Each pixel is replaced with a weighted sum of the values of its surrounding pixels The weighting factors have a Gaussian distribution, thereby the name Intuitively: each pixel is mixed to certain extent with its neighbors 2 4 5 4 2 4 9 12 9 4 5 12 15 12 5 4 9 12 9 4 2 4 5 4 2 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 32

Generalizing Windowing So far we are only doing (weighted) average of pixel values within a window A linear technique How about a nonlinear technique, e.g. taking median? Actually that s very useful Which is faster, mean or median? People often flip the filters by 180 degrees: y[r,c] = i, j 1 h[i, j] 1 i= 1 j = 1 h[i, j]x[r i,c j] We cheated, because our filters were symmetric Why? Because it links us to a signal processing technique called convolution Extensive body of knowledge, allowing us to know and compare the effects of these filters for different weights 1 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 33

Edge Detection Useful in understanding an image For robot, face recognition, medical imaging etc In a smooth contour, the pixel values usually do not change rapidly However, the pixel exhibit sudden jump in values near an edge E.g. jump from 1 to 130 Sobel edge detection is one of the simplest algorithms that makes use of this observation to find edges Compares values of the neighbors of pixel 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 34

Sobel Tilter +1 0-1 +1 +2 +1 +2 0-2 0 0 0 H x +1 0-1 H y -1-2 -1 A Sobel filter combines the results of the two weight matrices Each filter kernel estimates gradient in x and y direction from the input image. Magnitude of the resulting pixel in matrix D is: D[r,c] = D x 2 [r,c] + D y 2 [r,c] From convolving with H x From convolving with H y 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 35

Sobel Filter Example x dir flipped S G x D 3 3 3 39 39 39 39 3 3 3 40 40 40 40 3 3 3 41 41 41 41 3 3 3 42 42 42 42 3 3 3 41 41 41 41-1 0 +1-2 0 +2-1 0 +1 0 152 152 0 3 3 3 40 40 40 40 3 3 3 39 39 39 39-1 40 3 + 0 40 3 + 1 40 3-2 41 3 + 0 41 3 + 2 41 3-1 42 3 + 0 42 3 + 1 42 3 + + = 152 0 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 36

Sobel Filter Example x dir flipped S G x D 3 3 3 39 39 39 39 3 3 3 40 40 40 40 3 3 3 41 41 41 41 3 3 3 42 42 42 42 3 3 3 41 41 41 41 3 3 3 40 40 40 40 3 3 3 39 39 39 39-1 0 +1-2 0 +2-1 0 +1 0 0 152 152 0 0 0 0 0 152 152 0 0 0 0 0 152 152 0 0 0 0 0 152 152 0 0 0 0 0 152 152 0 0 0 0 0 152 152 0 0 0 0 0 152 152 0 0 0 Result in D shows a clear line at the edge Note that G x is a flipped version of H x Some more normalization has to be done in actual implementation 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 37

Image Processing Summary Image processing is the task of manipulating the image by mathematical means to achieve high level requirements Common operations: filtering Many other operations: E.g. Image forensic, Lithography, medical imaging, automatic image diagnosis, robot control, etc What s the (computational) cost of various image processing algorithms? What sort of image processing operations do we need in our video chats? 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 38

1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 39

Digital Cameras Resolution measured in pixels H x V Image sensing: charge coupled device (CCD) or complementary metal-oxide semiconductor (CMOS) Megapixels is used to denote the total max pixels in the image E.g. 5 Megapixel - in the 2520 by 1890 and higher pixel range. Photo quality 11 x 14 prints from this class of camera. Comparing film cameras to digital cameras is difficult since resolution is measured differently 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 40

Taking Pictures 1. Image focused by lens 2. Image captured on CCD 3. CCD generates analog representation of image 4. Analog signal converts to digital 5. Digital signal processing (DSP) adjust quality, etc Step 1 Step 2 Step 3 Step 4 Step 5 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 41

Marketing Caveats Q: For digital cameras, higher megapixel value always produce better photos? A: Not really. If you will only look at the photos on websites, or will only print them on 3R papers, you don t need all the pixels from a 10M pixels camera. 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 42

Area You Ready? 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 43

Flat Panel TVs and Monitors Pictures displayed as matrix of pixels on screen Two major technologies for generating picture Plasma Liquid Crystal Display (LCD) Plasma Neon-Xenon gas trapped between two glasses When electrically charged, each pixel display red, blue or green color. LCD Liquid crystal between glasses pass/block light depending on electrical signal Pass corresponding backlight 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 44

LED TVs? Misleading term Proper name: LED-backlight LCD TVs Use the same LCD display technology as all other LCD displays. Most other standard LCD displays use cold cathode fluorescent light (CCFL) for backlight 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 45

3 Characteristic Dimensions Panel Size The physical dimension of the panel A 42 panel has a diagonal measurement of 42 Display Resolution The number of picture-elements (pixels) along each X-Y direction Dot Pitch The distance between two pixel of the screen Panel Size = Display Resolution * Dot Pitch 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 46

Standard Display Resolutions 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 47

Marketing Caveats Q: For flat panel TVs, a bigger screen always produce better display than a smaller screen? A: Not really. It depends on the distance you will be watching the TV and the TV source signal. 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 48

More Pixel = Good? Human eye can identify 120 pixels per degree of visual arc i.e. if 2 dots are closer than 1/120 degree, then our eyes cannot tell the difference At a distance of 2m (normal distance to a TV) our eyes cannot differentiate 2 dots 0.4mm apart. Closer to TV => easier to differentiate pixels Far away => cannot tell the difference screen Minimum: 2 arc minute 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 49

Image courtesy of www.carltonbale.com 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 50

True LED displays Each pixel is a LED Used mostly in outdoor, largescale displays Source: http://www.diamond-vision.com /quad_dot_pattern.asp 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 51

Hong Kong Shatin Racecourse 70.4m x 8m World s Longest TV screen Dallas Cowboys Stadium Sideline Display 48.64m x 21.76m Pixel Pitch: 20mm Displays World s Largest High -Definition Video Display 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 52

In Conclusion Digital signal processing is a very broad field within EEE The processing of digital image is a good example of high-level applications that run on digital signal processing systems. To display and process digital images correctly, you need the right combination of image representation, hardware, and processing power. 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 53

Homework 1 Homework 1 is out Due 14 Oct, 2011 5pm Turn in physical copy of your answer Homework boxes near Room 712, Chow Yei Ching Building Individual homework Good way to study for final Zero tolerance on plagiarism 1st semester, 2011 Digital Images - ENGG1015 - Dr. E. Lam 54