Introduction to Computer Vision and image processing

Transcription

1 Introduction to Computer Vision and image processing 1.1 Overview: Computer Imaging 1.2 Computer Vision 1.3 Image Processing 1.4 Computer Imaging System 1.6 Human Visual Perception 1.7 Image Representation 1.8 Digital Image File Format 1.1

2 Overview: computer imaging Computer imaging the acquisition and processing of visual information by computers. The computer representation of an image requires the equivalent of many thousand of words of data. One picture is worth a thousand words. The receiver of the visual information: the human visual system and the computer Computer vision application the processed (output) images are for use by a computer Image processing applications the output images are for human consumption. 1.2

3 Overview: computer imaging (cont.) Computer Imaging Computer Vision Image Processing 1.3

4 Computer vision In computer imaging, the images are examined and acted upon by a computer. Image analysis the examination of the image data to facilitate solving a vision problem, involving Feature extraction the process of acquiring higher-level image information, such as shape or color information Pattern classification the act of taking this higher-level information and identifying objects within the image 1.4

5 Computer vision (cont.) Applications of computer vision: Recognition face, fingerprint, optical character Trademark retrieval, image retrieval Medical imaging diagnose skin tumors, aid neurosurgeons Law enforcement and security identification of fingerprint, DNA analysis, retinal scans, facial scans, veins in the hand Autonomous vehicles, target tracking and identification Weather prediction, planet changing 1.5

6 Image processing In image processing, the images are examined and acted upon by people, including Image restoration the process of taking an image with some known, or estimated, degradation, and restoring it to its original appearance. (see Fig ) Image enhancement taking an image and improving it visually, typically taking advantage of the human visual system s responses. (see Fig ) Image compression reducing the massive amount of data needed to represent an image by taking advantage of the redundancy inherent in most images. 1.6

7 Image processing (cont.) Applications of image processing: Medical imaging medical image diagnostic, such as PET (Positron Emission Tomography), CT (Computerized Tomography), MRI (Magnetic Resonance Imaging) Entertainment special effects, image editing, creating artificial scenes and beings (computer animation) Virtual reality Multimedia communications 1.7

8 Computer imaging systems The computer imaging system consists of: Hardware» Image acquisition subsystem camera, scanner, video player» Computer system o Frame grabber hardware that translate standard analog video signal into digital images, this process is called digitization (see Fig ) Sampling translating continuous signal into digital form at a fixed rate Quantization translating the voltage of the signal into the brightness of the image (see Fig )» Image display subsystem monitor, printer, film, video recorder Image processing software (see Fig ) 1.8

9 Computer imaging systems (cont.) The image brightness at a point depends on both the intrinsic properties of the object and the lighting conditions in the scene. An image can be regarded as a two-dimensional array of data with each data point referred to as a pixel (picture element) The digital image can be represented as I(r, c) = the brightness of the image at the point (r, c) where r = row and c = column 1.9

10 Human visual perception Why study visual perception? Compression compact the data as much as possible but still retain all the necessary visual information Enhancement improving the image visually achieved by understanding how the visual information is perceived 1.10

11 Human visual system The human visual system has two primary components the eye and the brain, which are connected by the optic nerve (see Fig ) Eye the image receiving sensor Brain the information processing unit The visible light energy corresponds to an electromagnetic wave that falls into the wavelength of about 380 to 825 nanometers (see Fig for the electromagnetic spectrum). In imaging system, the spectrum is often divided into various spectral bands, with each band defined by a range on the wavelengths (or frequency). For example, blue (400 ~ 500 nm), green (500 ~ 600 nm), red (600 ~ 700 nm) 1.11

12 Human visual system (cont.) The eye has two primary type of sensors rods and cones, which are distributed across the retina (see Fig ) Cones» primary used for daylight vision» sensitive to color» concentrated in the central region of the eye» have a high resolution capability Rods» primary used in night vision» see only brightness (not color)» distributed across the retina» have medium to low level resolution Blind spot a place on the retina with no sensors exist 1.12

13 Human visual system (cont.) The rods see only a spectral band (brightness), and cannot distinguish color (see Fig (a) for the response of rods) There are three types of cones, each responding to different wavelengths of light energy called the tristimulus curves because all the colors that we perceive are the combined result of the response to these three sensors (see Fig (b)). About 65% of the cones responds most to wavelengths corresponding to red and yellow, 33% to green, and 2% to blue. 1.13

14 Spatial frequency resolution Resolution the ability to separate two adjacent pixels; if we can see two adjacent pixels as being separate, then we say that we can resolve the two. Spatial frequency how rapidly the signal is changing in space, and the signal has two values for brightness 0 and Maximum (see Fig ). The spatial frequency concept must includes distance from the viewer to the object as part of the definition. The spatial frequency is defined in terms of cycles per degree which provides us a relative measure to eliminate the necessity to include distance (see Fig ). 1.14

15 Spatial frequency resolution (cont.) The physical limitations that affect the spatial frequency response of the visual system are both optical and neural: The spatial resolution are are limited by the size of the sensors (rods and cones), we cannot resolve things smaller than the individual sensors. The primary optical limitation are caused by the lens which limit the amount of light and typically contains imperfections that cause distortion in our visual perception. The spatial resolution is affected by the average (background) brightness of the display (see Fig ). 1.15

16 Brightness adaptation The response of the human visual system actually varies based on the average brightness observed and is limited by the dark threshold and the glare limit. Subjective brightness is a logarithmic function of the light intensity incident on the eye (see Fig ). Experimentally, we can detect only about 20 changes in brightness in a small area within a complex image. For an entire image, due to the brightness adaptation that our vision system exhibits, about 100 different gray levels are necessary to create a realistic image. 1.16

17 Brightness adaptation (cont.) False contouring (bogus lines) if fewer gray levels are used, the gradually changing light intensity will not being accurately represented, as seen in Fig Mach band effect when there is a sudden change in intensity, our vision system responds overshoots the edge, thus creating a scalloped effect. accentuates the edges and helps us to distinguish, and separate objects within an image (see Fig ). 1.17

18 Temporal resolution The temporal resolution deals with how we respond to visual information as a function of time. See Fig the temporal contrast sensitivity to the frequency. Flicker sensitivity our ability to observe a flicker( 閃爍 ) in a video signal displayed on a monitor. The human visual system has a temporal cutoff frequency of about 50 hertz (cycles per seconds) 1.18

19 Image representation The digital image I(r, c) is represented as a two-dimensional array of data, where each pixel value corresponds to the brightness of the image at the point (r, c). In linear algebra form, the two-dimensional array like I(r, c) is referred to as a matrix, and one row (or column) is called a vector. There are multiband images (color, multispectral), and they can be modeled by a different I(r, c) function corresponding to each separate band of brightness information. 1.19

20 Binary image representation A binary image takes on two values, typically black and white, or 0 and 1. A binary image is referred to as 1 bit/pixel image because it takes only one binary digit to represent each pixel. Applications fields of binary images (see Fig ): Optical character recognition (OCR) Robot Facsimile (FAX) image Binary images are often created from gray-scale image via a threshold operation every pixel above the threshold value is turned white ( 1 ) and those below it are turned black ( 0 ) 1.20

21 Gray-scale image representation Gray-scale images are referred to as monochrome, or one-color images they contain only brightness, no color, information. A typical image contains 8 bit/pixel data, which can represent 256 (0-255) different brightness (gray) levels Provides a noise margin by allowing for approximately twice an many gray levels the human visual system required 8-bit represent a byte, which is the standard small storage unit of digital computers In certain applications, such as medical imaging or astronomy, 12 or 16 bit/pixel representation are used. 1.21

22 Color-image representation Color images can be modeled as three-band monochrome image data, where each band of data corresponds to a different color. Typically, color images are represented as red, green, and blue, or RGB images 24 bit/pixel, 8 bit for each of the three color band A single pixel s red, green, and blue values can be referred to as a color pixel vector (R, G, B), see Fig In many applications, RGB color information is transformed into a mathematical space that decouples the brightness information from the color information Create a more people-oriented way of describing the colors 1.22

23 Color-image representation (cont.) Additive color system (color of light) Primary colors- red (R), green (G), blue (B) Secondary colors- magenta (R+B) 紫紅, cyan (G+B) 青藍, yellow (R+G) 黃色 1.23

24 Color-image representation (cont.) RGB color space: G Green Yellow Cyan White Black Red R B Blue Magenta 1.24

25 Color-image representation (cont.) Subtractive color system (color of pigments, colorants) Primary colors- magenta, cyan, yellow Secondary colors- red, green, blue 1.25

26 1.26 Color-image representation (cont.) C Y M Black Blue White Yellow Magenta Cyan Red Green C M Y R G B = CMY color space:

27 Color-image representation (cont.) Characteristics of Colors brightness - intensity chromaticity - hue and saturation hue - dominant wavelength (color) perceived by human eyes saturation - relative purity or the amount of white light mixed with a hue. tristimulus values - the amounts of red, green, blue (denoted as X, Y, and Z) needed to form any particular color. trichromatic coefficients - x, y, z with x = X /(X + Y + Z), y = Y/(X + Y + Z), z = Z/(X + Y + Z). 1.27

28 Color-image representation (cont.) HSL (hue/saturation/lightness) color space Lightness the brightness of the color Hue the color (e.g., green, blue, orange) Saturation how much white is in the color (e.g., pink, pure red) See Fig for HSL color space. 1.28

29 Color-image representation (cont.) The spherical coordinate transform (SCT) decouples the brightness information from the color information The transform equations are as follows: L = A = B = R 2 cos cos + G B L R Lsin( A) : the length of See Fig for SCT transform. + B 2 the RGB vector : the angle form the B - axis to the RG plane : the angle between the R - and G - axis 1.29

30 Color-image representation (cont.) One problem with the color spaces previously described is that they are not perceptually uniform two different colors in one part of the color space will not exhibit the same degree of perceptually difference as two colors in another part of the color space, even though they have the same distance apart (see Fig ) We cannot define a metric to tell us how close, or far apart, two colors are in terms of human perception. 1.30

31 Color-image representation (cont.) The CIE (Commission Internationale de I Eclairage) has defined internationally recognized standards. For the RGB space, chromaticity coordinates are defined as: R r = R + G + B G g = R + G + B B b = R + G + B The CIE has defined the standard CIE XYZ color space and the perceptually uniform L*u*v*, L*a*b* color spaces. 1.31

32 Color-image representation (cont.) CIE chromaticity diagram: 1.32

33 Color-image representation (cont.) Color spaces: RGB model - for hardware (e.g., monitors, video cameras) CMY model- for color printers YUV (luminance, hue, saturation) British PAL television standard YIQ (luminance, inphase, quadrature) NTSC TV broadcasting YDrDb French SECAM broadcast system YCrCb CCIR-601 video standard HSI (hue, saturation, intensity) - color image manipulation HSV (hue, saturation, value) - color image manipulation 1.33

34 YUV Model Y luminance or total illumination U, V color difference values Y = 0.299R G B U = 0.493(B Y) V = 0.877(R Y) RGB YUV YUV RGB Y = U V R G B R 1 = G 1 B Y U V 1.34

35 YIQ Model The luminance component (Y) and color information (I, Q) are decoupled. Useful in color TV broadcasting - for transmission efficiency and maintaining compatibility with monochrome TV. Human visual system is more sensitive to changes in luminance than to changes in hue or saturation. 1.35

36 YIQ Model (cont.) Y luminance Y = 0.299R G B I in-phase I = Vcos33 0 Usin33 0 Q quadrature Q = Vsin33 0 Ucos33 0 RGB YIQ YIQ RGB Y R I = G Q B R Y G = I B Q 1.36

37 YDrDb Model Y luminance Dr difference in red Dr = (R Y) Db difference in blue Db = 1.505(B Y) RGB YDrDb Y R Dr = G Db B 1.37

38 YCrCb Model Same as YUV without the coefficient for U and V plus a minor modification to allow integer math. Y = 0.299R G B Cr = 0.713(R Y) Cb = 0.564(B Y) RGB YCrCb YCrCb RGB Y Cr Cb = R G B R G B 1 = Y Cr Cb 1.38

39 HSV Model HSV - hue, saturation, and value H : specific angles around the vertical axis S : the radial distance from the vertical axis V : along the central axis 1.39

40 HSV Model (cont.) V = Y/256 S = ( Cr 128) + ( Cb 128) H = sin -1 ((Cr-128)/(128*s)) 2 2 /

41 HSI Model H : the angle of the vector w.r.t. the red axis S : the distance from p to the center of the triangle I : measured w.r.t. a line perpendicular to the triangle and passing through its center 1.41

42 1.42 HSI Model (cont.) RGB HSI + + = + + = + + = )] )( ( ) [( )] ( ) [( 2 1 cos ) ( )],, [min( B G B R G R B R G R H B G R B G R S B G R I Note: 1. if B > G then H = 360 -H, and let H = H/ If S = 0, H undefined 3. S is undefined if I = 0

43 HSI Model (cont.) HSI RGB (1)RG sector (0 < H 120 ) b = 1 3 (1 S) 1 S cos r = cos(60 g = 1 ( r + b) H H ) R = 3Ir, G = 3Ig, B = 3Ib 1.43

44 1.44 HSI Model (cont.) (2)GB sector (120 <H 240 ) (3)GB sector (240 <H 360 ) R = 3Ir, G = 3Ig, B = 3Ib R = 3Ir, G = 3Ig, B = 3Ib ) ( 1 ) cos(60 cos ) ( g r b H H S g S r H H + = + = = = ) ( 1 ) cos(60 cos ) ( b g r H H S b S g H H + = + = = =

45 Multispectral image representation Multispectral images contain information outside the normal human perceptual range, including infrared, ultraviolet, X-ray, acoustic, or radar data. Multispectral image systems include satellite system, underwater sonar systems, airborne radar, infrared imaging systems, medical diagnostic imaging systems. 1.45

46 Digital image file format Image data is divided into two primary categories: Bitmap images (raster images)» can be represented by the image model I(r, c)» the brightness value of each pixel data is stored in some file format» e.g., BIN, PPM, TIFF, GIF (see Fig ) Vector images» representing lines, curves, and shapes by storing only the key points» the key points are sufficient to define the shapes» rendering the process of turning the key points into an image 1.46

47 Digital image file format (cont.) Data visualization the process of representing data as an image Remapping the process of taking the original data and defining an equation to translate the original data to the output data range typically 0 to 255 for 8-bit display (see Fig ) Linear mapping Logarithmic mapping 1.47