1 Department of Computer Engineering, Faculty of Engineering King Mongkut s Institute of Technology Ladkrabang 01076531 MULTIMEDIA SYSTEMS Pk Pakorn Watanachaturaporn, Wt ht Ph.D. PhD pakorn@live.kmitl.ac.th, pwatanac@gmail.com Chapter 3 Digital Image Processing 2
Tools for Digital Image Processing Digital cameras Point-and-shoot Consumer level less expensive, easy to use, compact Prosumer Lie between the consumer and professional levels l in quality and expense More options, o file types, compression o levels, settings One view is offset from the lens view Professional Usually single-lens reflex cameras (SLR) one look through the view finder and see exactly what the lens sees High quality, detachable lenses 3 Tools for Digital Image Processing Digital cameras USB or IEEE1394 (Firewire) Memory card Pixel dimensions vs. storage Aspect ratio 4:3 4
Tools for Digital Image Processing Scanner Resolution DPI (dots per inch) Software A paint program a raster graphic editor Create e bitmap images; e.g., Microsoft Paint A drawing program Create vector graphics; e.g., Illustrator, Freehand An image processing programs Have tools for painting and drawing; e.g., Photoshop, GIMP 5 Digital Image File Types Bitmap Images File Suffix Abbreviation File Type.bmp BMP Windows bitmap.gif GIF Graphics Interchange Format.jpeg or.jpg JPEG Joint Photographic Experts Group.png PNG Portable Network Graphics.psd PSD Adobe Photoshop.raw Photoshop.tif or.tiff TIFF Tagged Image File Format 6
Digital Image File Types Vector Images File Suffix Abbreviation File Type.ai AI Adobe Illustrator t.swf SWF Shockwave Flash.cdr CDR Corel Draw.dxf DXF AutoCAD ASCII Drawing Interchange Format Metafiles File Suffix Abbreviation File Type.cgm CGM Computer Graphics Metafile.emf,.wmf EMF, WMF Enhanced metafile and Windows metafile.eps EPS Encapsulated Postscript.pdf PDF Portable Document Format 7 Digital Image File Type File format categories Bitmap image Vector image A hybrid of the two called metafiles 8
Digital Image File Type Bitmap file Considering Color model; e.g., RGB, CMYK, indexed color Bit depth; e.g., 1, 4, 8, 16, 24, 32, 48, or 64 bits Compression type; e.g., LZW, RLE, JPEG OS, browsers, application software that support it 9 Digital Image File Type Bitmap file GIF and JPEG are widely used on the web Good for continuous tone photographic images JPEG Compress with JPEG compression algorithm Level of compression The file format is actually called JFIF for JPEG File Interchange Format GIF Optional interlaced format - allow progressive download 10
Digital Image File Type Bitmap file RAW format Unprocessed image data without any color interpolation, white balancing, or contrast adjustments Photoshop.raw format RAW format from a digital camera Depend on the camera s engineering Proprietary to the camera Need special software to read the file when one port the file to a computer 11 Digital Image File Type Vector graphic file The size of a vector graphic file is proportional to the number of graphical objects in it Store image data in terms of geometric objects The objects are specified by parameters like line styles, side lengths, radius, color, gradients, etc. LineType 1; LineWidth 2.0; LineColr 1; Line (200, 400) (200, 600); Circle (500, 600), 430; 12
Digital Image File Type Metafile Combine vector graphics and bitmap images The most widely used is PDF Portable Document Format Can be used on all major operating systems Contain text, bitmap images, vector graphics, hyperlinks Microsoft Windows Metafile Format (WMF) The revised version of WMF is Enhanced Metafile Format (EMF) 13 Indexed Color One want to reduce the number of colors used in an image A picture doesn t use a large number of colors Slight differences in color may not be important May have constraints on the file size The process of reducing the number of color is called color quantization In image processing programs, the color mode associated with color quantization is called indexed color 14
Indexed Color Color quantization Begin with an image file stored with a bit depth of n Reduce to bit depth to b Involve three steps Determine the actual range and number of color used in a picture Choosing 2 b colors to represent those that actually appear in the picture Map the colors in the original picture to the colors chosen for the reduced bit-depth picture 15 Indexed Color Popularity algorithm 2 b colors that appear most often are chosen for the reduced-bit depth picture Map one of the original colors to the more limited palette by finding the color that is most similar using the minimum mean squared distance 2 b colors in the reduced d palette be given by their RGB color components such that the i-th color has components r i, g i, and b i for 0 i 2 b Find the color at index i that minimizes ( R r ) + ( G g ) + ( B b ) 2 2 2 1 i i 16
Indexed Color Uniform partitioning algorithm Divide the subspace containing the existing colors into 2 b blocks of equal size If the quantization is perfectly uniform, the slices in each dimension are equally spaced However, the size of a slice in one dimension does not have to equal the size of a slice in another dimension The slices must be made such that they partition the color space into no more than 256 blocks; e.g., 16 x 4 x 4, 8 x 8 x 4 If we don t mind sacrificing i some color, fewer than 256 can be used; e.g., 6 x 6 x 6 17 Indexed Color Uniform partitioning algorithm 18
Indexed Color Uniform partitioning algorithm The disadvantage Does not account for the fact that the equal-sized partitions of the color space may not be equally populated there may be many colors in one partition, and only few in another 19 Indexed Color Combination of the popularity and the uniform partitioning algorithm n = 24, b = 8, and k = 12 For each pixel, consider only the first k/3 = 4 bits in each of the R, G, B for the total of 12 bits = 4,096 categories Run through the original image file and count how many of the pixels fall into each of the 4,096 categories Take the 2 b = 256 most frequently occurring of these categories and use them in the final indexed color table for your image Take each of the original i pixels and map it to the closest of the colors in the color table based on minimum mean squared distance 20
Dithering A technique for simulating colors that are unavailable in a palette by using available colors that are blended by the eye so that they look like the desired colors 21 Dithering Original Photo Original image using the web-safe color palette with no dithering applied 22
Dithering Depth is reduced to a 16-color optimized palette in this image, with no dithering This image also uses the 16-color optimized palette, but the use of dithering helps to reduce banding 23 Dithering Noise dithering (a.k.a. random dithering) Adding high frequency noise speckles of black and white 24
Dithering Random Halftone Michelangelo's David Threshold Bayer (ordered) Floyd Steinberg 25 Dithering Jarvis, Judice & Ninke Stucki Michelangelo's David Threshold Burkes Scolorq 26
Channels, Layers, and Masks Digital image processing tools Break images into parts that can be treated separately RGB can be broken down to Red, green, blue color components An additional alpha channel store the opacity level for each pixel Alpha channel 27 Channels, Layers, and Masks Alpha blending A mathematical procedure for putting together multiple images or layers with varying levels of opacity Let a foreground pixel be given by F = ( f,, ) r fg fb Let a background pixel be given by B = ( br, bg, bb ) The result composite pixel color C = ( cr, cg, cb ) The opacity level α where 0 α 1 f ( 1 α f ) ( 1 α ) ( 1 α ) c = α f + b r f r f r c = α f + b g f g f g c = α f + b b f b f b f 28
Blending Modes Layers have blending modes associated with them in additional to opacity setting Blending modes create a variety of effects in making a composite of a foreground and background image 29 Blending Modes Top layer Bottom layer 30
Blending Modes Top layer Bottom layer Multiply blend mode Screen blend mode 31 Blending Modes 32
Pixel Point Processing Categorize image transforms into two types Pixel point processing a pixel value is changed based only on its original value, without reference to surrounding pixels Spatial filtering changes a pixel s value based on the values of neighboring pixels. 33 Pixel Point Processing Histograms a discrete function that describes frequency distribution Histogram function ( ) = for min max hi v i i Mode the value that occurs most frequently Mean average 1 n xi n i = 1 x= Median a value of x such that at most half of the values in the sample population are less than x and at most half are greater 34
Pixel Point Processing 35 Pixel Point Processing Transform functions and Curves A transform is a function that changes pixel values g ( ) ( xy, ) = T f( x, y) 36
Pixel Point Processing Transform functions and Curves 37 Pixel Point Processing Transform functions and Curves 38
Spatial Filtering A filter is an operation performed on digital image data to sharpen, smooth, or enhance some feature Filtering in the spatial domain Perform on image data in the form of the pixel s color values Filtering in the frequency domain Perform on image data that is represented in terms of its frequency components 39 Spatial Filtering Convolutions Each output pixel is computed as a weighted sum of neighboring input pixels Based on a matrix of coefficients called a convolution mask or a filter 40
Spatial Filtering 41 Spatial Filtering 42
Spatial Filtering Filters are sometimes used for smoothing or blurring an image by means of an averaging convolution mask Help removing image noise unwanted speckles Sometime be referred to as low-pass filters because their effect is to remove high-frequency components of an image 43 44
Gaussian blur an alternative ti for smoothing The coefficients in the convolution mask get smaller as you move away from the center of the mask 45 Pixel Point Processing Filters in digital image processing programs Digital image processing such as Photoshop / GIMP Have an array of filters to choose from Provide custom mask Custom filter from Photoshop and GIMP 46
Pixel Point Processing Filters in digital image processing programs In order to preserve the brightness balance of an image, n w i s should equal 1 1 1 1 i = 1 0 0 0 An edge detection filter -1-1 -1 47 Pixel Point Processing Filters in digital image processing programs Unsharp mask (USM) This filter sharpens images The name is derived from the way the mask is constructed First, a blurred version is constructed The pixel values in the original image are doubled d The blurred version of the image is substracted from this 48
Pixel Point Processing Filters in digital image processing programs Unsharp mask (USM) 49 Resampling and Interpolation Resampling a process of changing the total number of pixels in a digital image Resampling is required whenever the number of pixels in a digital image is changed The resolution is not changed, but the print size is The print size is not changed, but the resolution is 50
Resampling and Interpolation Resampling Downsampling A 300 ppi of 8 10 inches image = 24,000 pixels Want a 200 ppi of 8 10 inches image = 16,000 pixels Upsampling A 72 ppi of 4 5 inches image = 288 360 pixels Want to display on 90 ppi monitor Zoom in / Zoom out Upsampling or downsampling only for display purposes 51 Resampling and Interpolation Resampling Replication -simplest method for upsampling Inserting pixels and giving them the color value of a neighboring preexisting gpixel Work only if you are enlarging an image by an integer factor Row-column deleting simplest method for downsampling 52
Resampling and Interpolation Resampling Quality of an image Row-column deletion throws away information about the image, one obviously lose detail Replication makes a guess about the colors that might have been sampled between existing samples Gain pixels, but does not get any sharper Usually, the image lose quality 53 Resampling and Interpolation Resampling The only true information is the information one get when the image is created by taking a digital photograph or scanning in a picture Any information one generate after that is only an approximation or guess about what the original image look like. 54
Resampling and Interpolation Interpolation Give better results than simple replication or discarding of pixels A process of estimating the color of a pixel based on the colors of neighboring g pixels Define Scaling an affine transformation aso ao of digital dgaimage agedaa data that changes the total number of pixels in the image Scale factor s If s is greater than 1, the scaled image will increase in size If s is less than 1, the image will decrease in size 55 Resampling and Interpolation 56
Resampling and Interpolation 57 Resampling and Interpolation Nearest neighbor interpolation Rounds down to find one close pixel whose value is used for fs(i, j) When s is an integer greater than 1, it is effectively equivalent to pixel replication Work with noninteger scale factors 58
Resampling and Interpolation 59 Resampling and Interpolation Bilinear interpolation Use four neighbors and makes fs(i, j) a weighted sum of their color values The contribution of each pixel toward the color of fs(i, j) is a function of how close the pixel s coordinates are to (a, b) 60
Resampling and Interpolation 61 Resampling and Interpolation Bicubic interpolation Use a neighborhood of sixteen pixels to determine the value of fs(i, j) The neighborhood of (a, b) extends from x 1 to x + 2 and from y 1 to y + 2 62
Resampling and Interpolation 63 Digital Image Compression LZW Compression Lempel-Ziv-Welvh Applicable to both text and image compression Based on the observation that sequences of color in an image file are often repeated Use a sliding expandable window to identify successively longer repeated sequences Put into a code table as the file is processed for compression 64
Digital Image Compression LZW Compression Encoding With a first pass over the image file, the code table is initialized to contain all the individual colors that exist in the image file these colors are encoded in consecutive integers After initialization, the sliding expandable window moves across these pixels Begin with a width of one pixel if the pixel sequence is already in the code table, the window is successively expanded by one pixel until finally a color sequence not in the table is under the window (n pixels long). The code for the sequence that is n 1 pixels long is output into the compressed file, and the n-pixel-long l sequence is put into the code table 65 Digital Image Compression 66
Digital Image Compression 67 Digital Image Compression LZW Compression Decoding Require one a table initialized with the colors in the image The remaining codes are recaptured as the decoding progresses 68
Digital Image Compression 69 Digital Image Compression Huffman Encoding A variable-length encoding scheme not all color codes use the same number of bits Colors that appear more frequently in the image are encoded with fewer bits Require two passes Determining the codes for the colors Compressing the image file by replacing each color with its code 70
Digital Image Compression Huffman Encoding White Black Red Green Blue 70 50 130 234 245 71 Digital Image Compression Huffman Encoding 120 White Black Red Green Blue 70 50 130 234 245 72
Digital Image Compression Huffman Encoding 250 120 White Black Red Green Blue 70 50 130 234 245 73 Digital Image Compression Huffman Encoding 250 120 479 White Black Red Green Blue 70 50 130 234 245 74
Digital Image Compression Huffman Encoding 749 250 120 479 White Black Red Green Blue 70 50 130 234 245 75 Digital Image Compression Huffman Encoding 0 749 1 0 250 0 120 1 1 479 0 1 White Black Red Green Blue 70 50 130 234 245 76
Digital Image Compression Huffman Encoding White 000 Black 001 Red 01 Green 10 Blue 11 77 Digital Image Compression JPEG Compression An acronym for Joint Photographic Experts Group Commonly use JPEG compression to refer to a compression algorithm A standardized JPEG file format with the name JFIF (JPEG File Interchange Format) was introduced by C-Cube Microsystems and become the de facto standard d An alternative file format designed by C-Cube is called TIFF/JPEG Possible to apply JPEG compression to images saved in TIFF, PICT, EPS 78
Digital Image Compression JPEG Compression Effective because of the following three observations Image data usually changes slowly across an image, especially within an 8x8 block Therefore images contain much redundancy Experiments indicate that humans are not very sensitive to the high frequency data images Therefore we can remove much of this data using transform coding Humans are much more sensitive to brightness (luminance) information than to color (chrominance) 79 Digital Image Compression JPEG Compression A lossy compression method The information that is lost is not very important to how the picture looks The algorithm removes closely spaced changes in color that are not easily perceived by the human eye Allow one to choose the JPEG compression rate Able to specify how important the image size is versus the image s fidelity to the original i subject 80
Digital Image Compression JPEG Compression Algorithm jpeg /* Input: A bitmap image in RGB mode. Output: The same image, compressed */ { Divide image into 8 8 pixel blocks Convert image to a luminance/chrominance model such as YCbCr (optional) Shift pixel value by subtracting 128 Use discrete cosine transform to transform the pixel data from the spatial domain to the frequency domain Quantize frequency values Store DC value (upper left corner) as the difference between current DC value and DC from previous block Arrange the block in a zigzag order Do run-length encoding Do entropy encoding (e.g., Huffman) } 81 Digital Image Compression JPEG Compression 82
Digital Image Compression JPEG Compression Step 1: Divide the image into 8 8 pixel blocks and convert RGB to a luminance/chrominance color model Manageable block size Human eye is more sensitive to changes in light (i.e., luminance) than in color (i.e., chrominance) need less detailed information with regard to chrominance Chrominance subsampling (also called chrominance downsampling) a process of throwing away some of the bits used to represent pixels 83 Digital Image Compression JPEG Compression Original without color subsampling Image after color subsampling 84
Digital Image Compression JPEG Compression 85 Digital Image Compression JPEG Compression 86
Digital Image Compression JPEG Compression Step 2: Shift values by -128 and transform from the spatial to the frequency domain 87 Digital Image Compression JPEG Compression Step 2: Shift values by -128 and transform from the spatial to the frequency domain 88
Digital Image Compression JPEG Compression Step 3: Quantize the frequency values A typical quantization matrix, as specified in the original i JPEG Standard B is the quantized DCT coefficients 89 Digital Image Compression JPEG Compression Step 4: Apply DPCM to the block DPCM Differential Pulse Code Modulation In this context, DPCM is simply storing the difference between the first value in the previous 8 8 block and the first value in the current block 90
Digital Image Compression JPEG Compression Step 5: Arrange the values in a zigzag order and do run-length encoding 91 Digital Image Compression JPEG Compression Step 6: Do entropy encoding Additional compression can be achieved with some kind of entropy encoding 92
Digital Image Compression JPEG 93