1 Department of Computer Engineering, g, Faculty of Engineering King Mongkut s Institute of Technology Ladkrabang 01076531 MULTIMEDIA SYSTEMS Pakorn Watanachaturaporn, Ph.D. pakorn@live.kmitl.ac.th, pwatanac@gmail.com Chapter 2 Digital Image Representation 2
Introduction Digital images are created by three basic methods Bitmapping (also called pixmaps or raster graphics) Created with a pixel-by-pixel specification of points of color Vector graphics Use object specification and mathematical equations to describe shapes to which color are applied Procedural modeling (also called algorithmic art) A computer program uses some combination of mathematics, logic, control structures, and recursion to determine the color of pixels and thereby the content of the overall picture 3 Bitmaps Digitization A bitmap is two-dimensional array of pixels describing a digital image Three main ways to create a bitmap 1. Software 2. Reproductions of scenes and objects a snapshot from a traditional camera, then scan the photograph with a digital image 3. Shoot the image with a digital camera 4
Bitmap 5 Bitmaps Digitization A matter of the color model used and the corresponding bit depth Both sampling and quantization can introduce error Not enough samples, the image will lack clarity A low bit depth can result in patchiness of color 6
Bitmap Pixel Dimensions, Resolution, and Image Size Logical pixel a bitmap Physical pixel a computer display Pixel dimensions Defined as the number of pixel horizontally and vertically denoted w h ; e.g., 1600 1200 Resolution Defined as the number of pixels in an image file per unit of spatial measure E.g., pixels per inch, ppi Resolution of a printer dots per inch, dpi 7 Bitmap Pixel Dimensions, Resolution, and Image Size Image size Defined as the physical dimensions of an image when it is printed out or displayed on a computer For an image with resolution r and pixel dimensions w h where w is the width and h is the height, the printed image size a b is given by a w = and b= r h r 8
Bitmap Pixel Dimensions, Resolution, and Image Size Cropping Cutting off part of the picture, discarding the unwanted pixels Resampling Changing the number of pixels in an image Involving some kind of interpolation, averaging, or estimating Cannot improve the quality of an image Upsampling Increase the pixel dimensions Downsampling Decrease the pixel dimensions 9 Frequency in Digital Images Considering an image as a function that can be represented in a graph such that y = f ( x) x = the position of one point of color f = a function over the spatial domain y = the color value at position x 10
Frequency in Digital Images In the realm of digital imaging, frequency refers to the rate at which color values change 11 Frequency in Digital Images In a digital image we need to discretize them 12
Frequency in Digital Images 13 Discrete Cosine Transform Fourier theory Any complex periodic waveform can be equated to an infinite sum of simple sinusoidal waves of varying frequencies and amplitudes f ( x) = an cos( nω x) n= 0 f(x) is a continuous periodic function over the spatial domain, ω is angular frequency where ω = 2πf, f is the fundamental frequency of the wave, a n is the amplitude for the n th cosine frequency component 14
Discrete Cosine Transform 15 Discrete Cosine Transform Consider a single line of pixels across a digital image, which values are [0, 0, 0, 153, 255, 255, 220, 220] F( u) ( ) ( ) F ( ) ( + ) M 1 2 C u 2 1 x u cos r u π f = for 0 r < M u= 0 M 2M 2 where C ( u ) = if u = 0 otherwise C ( u ) = 1 2 is one-dimensional array of coefficients. Each function cos ( 2r+ 1) 2M uπ is called a basis. 16
Discrete Cosine Transform ( ) 2r+ 1 uπ Each function cos is called a basis. 2M Can think of each function as a frequency component ( ) The coefficients in F u tell how much each frequency component is weighted in the sum that produces the pixel values think as how much each frequency component contributes to the image 17 Discrete Cosine Transform 18
Discrete Cosine Transform 19 Discrete Cosine Transform 20
Discrete Cosine Transform 21 Discrete Cosine Transform DCT is stated as follows: ( ) F ( ) f ( ) ( + ) M 1 2C u 2r 1 uπ u = f r cos for 0 u < M r= 0 M 2M 2 where C( u) = if u = 0 otherwise C( u) = 1 2 The equation tells how to transform an image from the spatial domain (grayscale values) to the frequency domain (which gives coefficients by which the frequency components should be multiplied) 22
Discrete Cosine Transform 23 Discrete Cosine Transform The first element F(0) ( ) is called the DC component All the other components F(1) through F(M-1) are called AC components 24
Discrete Cosine Transform The 2D discrete cosine transform F ( uv) ( ) ( ) ( + ) π ( + ) M 1N 1 2C u C v 2r 1 u 2s 1 vπ, = f( rs, ) cos cos r= 0 s= 0 MN 2 M 2 N 2 where C( δ) = if δ = 0 otherwise C( δ) = 1 2 The 2D inverse discrete cosine transform ( ) ( ) ( + ) π ( + ) M 1N 12C u C v 2r 1 u 2s 1 vπ f ( r, s) = F( u, v) cos cos r= 0 s= 0 MN 2M 2N 2 where C( δ) = if δ = 0 otherwise C( δ) = 1 2 25 Discrete Cosine Transform Rather than being applied to a full M N image, the DCT is generally applied to 8 8 pixel subblocks 255 255 255 255 255 255 159 159 255 0 0 0 255 255 159 159 255 0 0 0 255 255 255 255 255 0 0 0 255 255 255 255 255 255 255 255 255 255 100 255 255 255 255 255 255 255 100 255 255 255 255 255 255 255 100 255 255 255 255 255 255 255 100 255 26
Discrete Cosine Transform 1628-61 39 234 173-128 171 22-205 -163 74 222 1 74-30 111 81 150-95 -82-42 -11-6 -53 188 231-135 -188-53 -36-2 -103 96 71-42 -78-32 2-17 -32 25-42 25 3-14 27-26 19 70 15-6 -51-17 7-16 -13 94 72-38 -87-18 -14-4 -40 Amplitudes of Frequency Components 27 Discrete Cosine Transform 28
Aliasing Blurriness and Blockiness Consider Figure 2.1 & 2.2 If the one color changes, the two colors cannot be represented by the sample Imply the image reconstructed from the sample will not be a perfect reproduction of the original scene Mathematically, the spatial frequencies of the original scene will be aliased to lower frequencies in the digital photograph Virtually, when all the colors are averaged to one color, the reconstructed image looks blocky and the edges of objects are jagged 29 Aliasing Blurriness and Blockiness One would need a very high sampling rate to capture a real-world scene with complete fidelity However, the human eye is not going to notice a little lose of detail 30
Aliasing Moiré Patterns or Moiré Effect 31 Aliasing Moiré Patterns or Moiré Effect Can result when a digital photograph is taken and when a picture is scanned in to create a digital image 32
Aliasing Moiré Patterns or Moiré Effect 33 Aliasing Moiré Patterns or Moiré Effect Occur in digital photography because it is based on discrete samples An alias of the original pattern results if the samples are taken off beat from a detailed pattern in the subject being photographed Sometimes aliasing in digital images manifests itself as small areas of incorrect colors or artificial auras around objects be referred as color aliasing, moiré fringes, false coloration, or phantom colors. 34
Aliasing Moiré Patterns or Moiré Effect Many current digital cameras use charge-coupled device (CCD) technology to sense light and thereby color 35 Aliasing Moiré Patterns or Moiré Effect Bayer color filter array, or a Bayer filter There twice as many green sensors as blue or red The interpolation algorithm for deriving the two missing color channels at each photosite is called demosaicing. 36
Aliasing Moiré Patterns or Moiré Effect A nearest neighbor algorithm determines a missing color c for a photosite based on the colors of the nearest neighbors that have the color c. G R G B G B B G B G R G G R G B G B Determining R or B from the center G photosite entails an average of two neighboring sites Determining B from the center G photosite entails an average of two neighboring sites 37 Aliasing Jagged Edges Sometime, the term aliasing used to describe the jagged edges along lines or edges that are drawn at an angle across a computer screen Occur during rendering rather than sampling and results from the finite resolution of computer displays 38
Aliasing Jagged Edges Anti-aliasing a technique for reducing the jaggedness of lines or edges caused by aliasing 39 Aliasing Jagged Edges Bitmap vs vector graphics 40
Color Color Perception and Representation Composed of electromagnetic waves These waves fall upon the color receptors of the eyes the human brain translates the interaction between the waves and the eyes as color perception The colors human see are almost produced d by a combination of wavelengths Possible to represent a color by mean of spectral density graph 41 Color Color Perception and Representation The colors human see are almost produced by a combination of wavelengths 42
Color Color Perception and Representation Hue color s dominant wavelength Saturation color purity Luminance the area beneath the curve L= ( d a) e+ ( f e)( c b) S = ( f e )( c b ) L 43 Color C = rr+ gg+ bb RGB Color Model Varying combinations of three primary colors RGB color component (or color channel) The origin i (0, 0, 0) corresponds to black Grayscale values fall along the RGB cube s diagonal from (0, 0, 0) to (1, 1, 1) 44
Color CMY Color Model Divide a color into three primaries Using subtractive color creation process The origin of the cube is white (rather than black) The value for each component indicates how much red, green, and blue are subtracted out C = 1 R M = 1 G Y = 1 B 45 Color CMY Color Model Used in professional four-color printed processes by adding a fourth component (pure black) ( C M Y) K = min,, Cnew = C K M new = M K Y = Y K new 46
Color 47 Color HSV and HLS Color Models Speak of a color in terms of its hue (essential color), its lightness (or value or luminance), and its saturation (the purity of the color) Geometrically, HSV color space is a distortion of the RGB space into a kind of three-dimensional diamond called a hexacone Hue a position of a point in degrees, from 0 to 360, with red conventionally set at 0 Saturation a function of the color s distance from the central axis. The farther a color is from this axis, the more saturated the color Value axis lies from the black point of the hexacone ranging from 0 for black to 1 for white 48
Color HSV and HLS Color Models The distortion of the RGB color space to either HSV or HLS is a non-linear transformation 49 Color 50
Color 51 Color Luminance and Chrominance Color Models Capture all the luminance information in one value and put the color (chrominance) information in the other two values; e.g., YIQ model YIQ model is a simple translation of the RGB model More efficient i for television i broadcasting Consolidate all of the black and white information (luminance) in one of the three components and capture all the color information in the other two 52
Color Luminance and Chrominance Color Models YIQ model Y 0.299 0.587 0.114 R I 0.596 0.275 0.321 G = Q 0.212 0.523 0.311 B Y is the luminance component, and I and Q are chrominance The coefficients in the matrix are based on primary colors of red, green, and blue that are appropriate for the standard National Television System Committee (NTSC) RGB phosphor p 53 Color Luminance and Chrominance Color Models YUV Originally used in the European PAL analog video standard Based upon luminance and chrominance YCbCr Closely related to the YUV with its chrominance values scaled and shifted Used in JPEG and MPEG compression 54
Color CIE XYZ and Color Gamuts The obvious way to generate all possible colors is to combine all possible intensities of red, green, and blue light 256 256 256 = 16,777,216 colors There exists colors outside the range of those we can create An experiment called color matching Human subjects are asked to compare pure colors projected onto one side of a screen to composite colors projected beside them The pure colors are created by single wavelength light The amount of the three components are called the tristimulus i values 55 Color CIE XYZ and Color Gamuts Experimentally, no three visible primaries can be linearly combined to produce all colors in the visible spectrum The range of colors that a given monitor can display is called its color gamut There will be colors that t you can represent on your computer monitor but you cannot print, and vice versa 56
Color CIE XYZ and Color Gamuts Need of a mathematical model that captures all visible colors CIE XYZ or CIE color model First step in the direction of a standard color model that represents all visible colors Devised in 1931 by the Commission Internationale de l Echlairage 57 Color CIE XYZ and Color Gamuts 58
Color CIE XYZ and Color Gamuts The amount of red light energy needed to create the perceived pure spectral red at wavelength λ is a function of the wavelength, given by r(λ), and similarly for green g(λ) and blue b(λ) Let C(λ) be the color the average observer perceives at wavelength λ ( λ ) = ( λ ) + ( λ ) + ( λ ) C r R g G b B R refers to pure spectral red light at a fixed wavelength, and similarly for G and B 59 Color CIE XYZ and Color Gamuts The CIE model is based on the observation that No three visible primary colors that can be combined in positive amounts to create all colors in the visible spectrum Possible to use three virtual primaries to do so These primaries i called X, Y, and Z are theoretical rather than physical entities. Do not correspond to wavelengths of visible light Provide a mathematical way to describe colors that exists in the visible spectrum 60
Color CIE XYZ and Color Gamuts X, Y, and Z are chosen so that all three functions remain positive over the wavelengths of the visible spectrum ( λ ) = ( λ) + ( λ) + ( λ) C x X y Y z Z 61 Color CIE XYZ and Color Gamuts 62
Color CIE XYZ and Color Gamuts 63 Color CIE XYZ and Color Gamuts 64
Color CIE L*a*b, CIE L*U*V, and Perceptual Uniformity The CIE XYZ model has three main advantages Device-independent Provide a way to represent all colors visible to humans The representation is based upon spectrophotometric measurements of color RGB and CMYK color models are not device-independent Different computer monitors or printers can use different values for R, G, and B Their gamuts are not necessarily identical 65 Color CIE L*a*b, CIE L*U*V, and Perceptual Uniformity The CIE XYZ model has a disadvantage that it is not perceptually uniform In a perceptually uniform color space, the distance between two points is directly proportional to the perceived difference between the two colors The Commission International de l Eclairage refined its color model and produced the CIE L*a*b and CIE L*U*V models 66
Color Color Management Systems The colors one choose might not be exactly the colors that others see when the picture is placed on the web or printed in hard copy RGB monitor is not identical to the gamut printable in a CMYK color processing system A color management system communicates the assumptions about color spaces, setting for primary colors, and the mapping from color values to physical representations in pixels and ink from one device to another 67 Color Color Management Systems Involve five steps Calibrating your monitor Characterizing your monitor s color profile Creating an individual image s color profile that includes choices for color model and rendering intent Saving the color profile with the image Reproducing the image s color on another device or application program on the basis of the source and destination profiles 68
Vector Graphics Geometric Objects in Vector Graphics Drawn object by object File format:.fh Freehand.ai Adobe Illustrator.wmf Windows metafile.eps Encapsulated Postscript Contain the parameters to mathematical ti formulas defining how shapes are drawn 69 Vector Graphics Specifying Curves with Polynomials and Parametric Equations Parametric cubic polynomial functions An n th degree polynomial at + a t + a t +... + at + a n n 1 n 2 n n 1 n 2 1 0 where a n 0 and a 0, a 1, a 2,, a n are the coefficients of the polynomial Cubic polynomial (i.e., 3 rd degree polynomials, where the highest power is 3) 70
Algorithmic Art and Procedural Modeling Algorithmic art or procedural modeling Creating a digital image by writing a computer program based on some mathematical computation or unique type of algorithm "Octopod" by Mikael Hvidtfeldt Christensen. An example of algorithmic art produced with the software Structure Synth. 71 Algorithmic Art and Procedural Modeling Algorithmic art or procedural modeling Fractal Generation A graphical image characterized by a recursively repeated structure Jon Zander (Digon3)" 72
Algorithmic Art and Procedural Modeling 73 Algorithmic Art and Procedural Modeling 74
Algorithmic Art and Procedural Modeling The Mandelbrot set 75