ENGG1015 Digital Images

Transcription

1 ENGG1015 Digital Images 1 st Semester, 2011 Dr Edmund Lam Department of Electrical and Electronic Engineering The content in this lecture is based substan1ally on last year s from Dr Hayden So, but all errors should be blamed on me

2 Back to top- level High Level This week Applications Systems Digital Logic Image & Video Processing Computer & Embedded Systems Computer Network Mobile Network Combinational Logic Boolean Algebra Circuits Basic Circuit Theory Low Level Electrical Signals Voltage, Current Power & Energy 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 2

3 Back to top- level High Level This week Applications Systems Digital Logic Image & Video Processing Computer & Embedded Systems Computer Network Mobile Network Combinational Logic Boolean Algebra Circuits Basic Circuit Theory Low Level Electrical Signals Voltage, Current Power & Energy 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 3

4 Digital Images Note: The three parts are mostly independent (even with different language ), but they do intersect. Representation Hardware Processing 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 4

5 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 5

6 Representing Images R bitmap G B pixel An image is broken down into small regions called picture elements (pixels) Digital image (bitmap): A pixel-by-pixel representation of an image. Implications? 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 6

7 Image Dimensions Image Size The number of pixel in X-Y direction Sometimes quoted using the total number of pixels in a picture (N megapixels) Image Resolution The density of pixels Measured by pixel-per-inch (PPI) NOT the number of pixels 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 7

8 Representing Pixels Each pixel is represented by one or more values Black & white images (binary images): Each pixel is represented by exactly 1 value (B or W) 1 bit is enough to represent 2 possible values Grayscale images: Each pixel is usually a byte (8 bits), keeping the brightness or gray levels Color images: Each pixel represented a group of color components of that location often three colors Different color systems: RGB, CMYK, YCbCr, etc Hyperspectral images: Many values per pixel location, corresponding to different frequencies 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 8

9 Binary and Grayscale Images B&W B&W (w/ dither) Grayscale Binary Image Each pixel is 1 bit, either 0 or 1 Dithering is used to produce (fake) different intensities Grayscale Image Each pixel is usually a byte (8-bit), keeping the brightness or gray levels 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 9

10 Color Images 16 colors 256 colors 16M colors indexed color image # of color support depends on the # of bits for each pixel 4 bits 2 4 =16 colors 8 bits 2 8 =256 colors Color Look-Up Tables (LUTs) Color palette 24-bit color image Each pixel is represented by 3 bytes using a certain color model Supports 256x256x256 colors 16 million colors 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 10

11 RGB Color Model Additive color model Primary colors: Red, Green, and Blue Secondary colors obtained by additive mixing of primary colors: Cyan, Magenta, Yellow Commission Internationale d'eclairage (CIE) in 1931 specifies red to be 700nm, green to be 546.1nm and blue to be 435.8nm Used in media that transmit light (e.g. TV) 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 11

12 CMY Color Model Subtractive color model Subtractive primaries: Cyan, magenta, and yellow A subtractive primary absorbs a primary color and reflects the other two E.g. Cyan absorbs red and reflect blues and green Used in printing device 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 12

13 Colors that Can be Reproduced 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 13

14 Printing an Image Print Size Depends on the mapping between printer s resolution, image resolution & image size A Printer s printing resolution is usually higher than an image s resolution because multiple dots of ink are needed to created color of an image pixel Color Space On screen display: (additive) Printing devices: (subtractive) Color Production Each pixel may have different color Each ink drop has only on-off (one bit!) 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 14

15 Dithering Create the illusion of new colors and shades by varying the pattern of dots. E.g. Newspaper photographs are dithered. If you look closely, you can see that different shades of gray are produced by varying the patterns of black and white dots. There are no gray dots at all. 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 15

16 Dither, Halftone, Grayscale original dither halftone 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 16

17 RGB Color Space The RGB model describes the formation of color by linearly mixing different portion of Red, Blue and Green light. Color is represented by a triplet {r,g,b}, which indicates the weighting coefficients We often normalize the coefficients to be between 0 and 1 (inclusive), or integers between 0 and 255 (8-bit). 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 17

18 More Color Models Both RGB and CMY(K) model specify linear combinations of the primaries But they have little resemblance to how human beings reason about colors E.g. How do you get the RGB values of the pale orange color on the right? [R G B] = [ ] [R G B] = [???] [ ] 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 18

19 HS(B/V), HSL, HSI Color Model The family of HSx models describe colors similar to how human perceives colors Also similar to how painters create colors HSB: Hue Saturation Brightness HSV: Hue Saturation Value HSL: Hue Saturation Lightness HSI: Hue Saturation Intensity Similar, but often comes with confusing (or even contradicting) definitions 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 19

20 Cylindrical- Coordination Hue: The dominant color The angle away from red Saturation The amount away from the center How full the color is Lightness/ Brightness/Value The amount of white/black added 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 20

21 Luminance- chrominance Another common alternative: a luminancechrominance representation One value for luminance (Y): the brightness, or achromatic image Y = R G B Need two more numbers for the chrominance YUV and YCbCr Why? TV broadcast and digital picture compression 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 21

22 More Image Representations? Raster image (bitmap image) - Raster graphics uses pixel values to describe an image. The file size is independent of the image complexity. For higher resolution, the file size increases dramatically Vector graphics (draw graphics) - An alternate approach is to use only instructions for drawing lines, circles, ellipses, curves, and other shapes. 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 22

23 Vector Graphics Vector-based images are composed of key points and paths which define shapes, and coloring instructions, such as line and fill colors. Example: 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 23

24 Vector Graphics Advantages Vector graphics can be scaled up and down easily and quickly while retaining the quality of the picture. Raster images scale poorly and display poorly at resolutions other than that for which the image was originally created. Vector graphics require less bandwidth and can be accessed and viewed faster than raster graphics. Vector graphics can be edited and manipulated far easier than raster images. 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 24

25 (Partial) Summary Many decisions to make No universally best options Depends on the physical system, e.g. monitor vs printer Depends on the requirement, e.g. color vs grayscale Intersects with other fields e.g. Psychology (visual science) How about our video chats problem? 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 25

26 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 26

27 Image Processing Used in digital camera, TV, cell phones Used in all kinds of photo editing SW e.g. Photoshop, GIMP 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 27

28 Image Processing - Examples 1st semester, 2011 Original Grayscale Blur Edge Detection Digital Images - ENGG Dr. E. Lam 28

29 RGB to Grayscale Conversion Each pixel of a grayscale image has only one intensity value, V High V: white, Low V: black Easiest conversion: V = R + G + B 3 Produce better result if you weight G and R more than B Human eyes are more sensitive to green and red 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 29

30 Basic Filtering: Windowing Filters are building blocks of image processing systems One of the most basic filtering method is by windowing y[r,c] = i, j 1 h[i, j] r 1 1 i= 1 j = 1 h[i, j]x[r + i,c + j] c 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 30

31 Windowing in Action X H Y = st semester, 2011 Digital Images - ENGG Dr. E. Lam 31 16

32 Gaussian Blur A simple but effective way to blur a picture Each pixel is replaced with a weighted sum of the values of its surrounding pixels The weighting factors have a Gaussian distribution, thereby the name Intuitively: each pixel is mixed to certain extent with its neighbors st semester, 2011 Digital Images - ENGG Dr. E. Lam 32

33 Generalizing Windowing So far we are only doing (weighted) average of pixel values within a window A linear technique How about a nonlinear technique, e.g. taking median? Actually that s very useful Which is faster, mean or median? People often flip the filters by 180 degrees: y[r,c] = i, j 1 h[i, j] 1 i= 1 j = 1 h[i, j]x[r i,c j] We cheated, because our filters were symmetric Why? Because it links us to a signal processing technique called convolution Extensive body of knowledge, allowing us to know and compare the effects of these filters for different weights 1 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 33

34 Edge Detection Useful in understanding an image For robot, face recognition, medical imaging etc In a smooth contour, the pixel values usually do not change rapidly However, the pixel exhibit sudden jump in values near an edge E.g. jump from 1 to 130 Sobel edge detection is one of the simplest algorithms that makes use of this observation to find edges Compares values of the neighbors of pixel 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 34

35 Sobel Tilter H x H y A Sobel filter combines the results of the two weight matrices Each filter kernel estimates gradient in x and y direction from the input image. Magnitude of the resulting pixel in matrix D is: D[r,c] = D x 2 [r,c] + D y 2 [r,c] From convolving with H x From convolving with H y 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 35

36 Sobel Filter Example x dir flipped S G x D = st semester, 2011 Digital Images - ENGG Dr. E. Lam 36

37 Sobel Filter Example x dir flipped S G x D Result in D shows a clear line at the edge Note that G x is a flipped version of H x Some more normalization has to be done in actual implementation 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 37

38 Image Processing Summary Image processing is the task of manipulating the image by mathematical means to achieve high level requirements Common operations: filtering Many other operations: E.g. Image forensic, Lithography, medical imaging, automatic image diagnosis, robot control, etc What s the (computational) cost of various image processing algorithms? What sort of image processing operations do we need in our video chats? 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 38

39 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 39

40 Digital Cameras Resolution measured in pixels H x V Image sensing: charge coupled device (CCD) or complementary metal-oxide semiconductor (CMOS) Megapixels is used to denote the total max pixels in the image E.g. 5 Megapixel - in the 2520 by 1890 and higher pixel range. Photo quality 11 x 14 prints from this class of camera. Comparing film cameras to digital cameras is difficult since resolution is measured differently 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 40

41 Taking Pictures 1. Image focused by lens 2. Image captured on CCD 3. CCD generates analog representation of image 4. Analog signal converts to digital 5. Digital signal processing (DSP) adjust quality, etc Step 1 Step 2 Step 3 Step 4 Step 5 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 41

42 Marketing Caveats Q: For digital cameras, higher megapixel value always produce better photos? A: Not really. If you will only look at the photos on websites, or will only print them on 3R papers, you don t need all the pixels from a 10M pixels camera. 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 42

43 Area You Ready? 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 43

44 Flat Panel TVs and Monitors Pictures displayed as matrix of pixels on screen Two major technologies for generating picture Plasma Liquid Crystal Display (LCD) Plasma Neon-Xenon gas trapped between two glasses When electrically charged, each pixel display red, blue or green color. LCD Liquid crystal between glasses pass/block light depending on electrical signal Pass corresponding backlight 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 44

45 LED TVs? Misleading term Proper name: LED-backlight LCD TVs Use the same LCD display technology as all other LCD displays. Most other standard LCD displays use cold cathode fluorescent light (CCFL) for backlight 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 45

46 3 Characteristic Dimensions Panel Size The physical dimension of the panel A 42 panel has a diagonal measurement of 42 Display Resolution The number of picture-elements (pixels) along each X-Y direction Dot Pitch The distance between two pixel of the screen Panel Size = Display Resolution * Dot Pitch 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 46

47 Standard Display Resolutions 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 47

48 Marketing Caveats Q: For flat panel TVs, a bigger screen always produce better display than a smaller screen? A: Not really. It depends on the distance you will be watching the TV and the TV source signal. 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 48

49 More Pixel = Good? Human eye can identify 120 pixels per degree of visual arc i.e. if 2 dots are closer than 1/120 degree, then our eyes cannot tell the difference At a distance of 2m (normal distance to a TV) our eyes cannot differentiate 2 dots 0.4mm apart. Closer to TV => easier to differentiate pixels Far away => cannot tell the difference screen Minimum: 2 arc minute 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 49

50 Image courtesy of 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 50

51 True LED displays Each pixel is a LED Used mostly in outdoor, largescale displays Source: /quad_dot_pattern.asp 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 51

52 Hong Kong Shatin Racecourse 70.4m x 8m World s Longest TV screen Dallas Cowboys Stadium Sideline Display 48.64m x 21.76m Pixel Pitch: 20mm Displays World s Largest High -Definition Video Display 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 52

53 In Conclusion Digital signal processing is a very broad field within EEE The processing of digital image is a good example of high-level applications that run on digital signal processing systems. To display and process digital images correctly, you need the right combination of image representation, hardware, and processing power. 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 53

54 Homework 1 Homework 1 is out Due 14 Oct, pm Turn in physical copy of your answer Homework boxes near Room 712, Chow Yei Ching Building Individual homework Good way to study for final Zero tolerance on plagiarism 1st semester, 2011 Digital Images - ENGG Dr. E. Lam 54