Chapter 3 Part 2 Color image processing Motivation Color fundamentals Color models Pseudocolor image processing Full-color image processing: Component-wise Vector-based Recent and current work Spring 2002 Video Processing - Dr. Marques 32
Motivation 2 main factors: Color is a powerful descriptor. Humans can discern thousands of color shades and intensities, compared to about only a few dozen shades of gray. It is still a fairly new area (with many open research topics): Only in the past 10 years the hardware became inexpensive and widely available Popularization of the Internet Extension of grayscale methods to color is not straightforward Spring 2002 Video Processing - Dr. Marques 33
Color fundamentals (1) Color perception by humans is a physiopsychological phenomenon that is not yet fully understood. During the 1960s, physiological results confirmed that two mechanisms operate simultaneously: Trichromatic theory Opponent-process theory On the other hand, the physical nature of color can be expressed on a formal basis supported by experimental and theoretical results (color spectrum, wavelengths, etc.). Spring 2002 Video Processing - Dr. Marques 34
Color fundamentals (2) Trichromatic theory: color vision depends on three receptor mechanisms (5-7 million cones), each with different spectral sensitivities: 65% are sensitive to Red light 33% are sensitive to Green light 2% are sensitive to Blue light Opponent-process theory: humans distinguish between a red-green axis of color and an independent blue-yellow axis. Spring 2002 Video Processing - Dr. Marques 35
Color fundamentals (3) Color wheel Spring 2002 Video Processing - Dr. Marques 36
Color fundamentals (4) Physical properties of light and color Newton s prism 1666 Color spectrum (400 nm to 700 nm) Quantities used to describe the quality of a chromatic light source: Radiance: total amount of energy that flows from the light source (W) Luminance: measure of the amount of energy an observer perceives from a light source (lm) Brightness: subjective descriptor that embodies the achromatic notion of intensity and is practically impossible to measure. Spring 2002 Video Processing - Dr. Marques 37
Color fundamentals (5) Colors can be seen as variable combinations of the socalled primary colors of light (also known as additive primaries) (R, G, and B). The primary colors of pigments (also known as subtractive primaries) are: magenta (Red + Blue), cyan (Green + Blue), and yellow (Red + Green). Spring 2002 Video Processing - Dr. Marques 38
Color fundamentals (6) Common misconception 1: The interpretation that the three standard primaries, when mixed in various intensity proportions, can produce all visible colors. This is not correct, unless the wavelength also is allowed to vary. Common misconception 2: Even though it is correct to state that any wavelength of visible light has a perceived color, it is incorrect to associate each perceived color with a particular wavelength. Spring 2002 Video Processing - Dr. Marques 39
Color fundamentals (6) Color properties: Brightness: embodies the achromatic notion of intensity Hue: attribute associated with the dominant wavelength (~ color name) Saturation: relative purity or the amount of white light mixed with a hue. Hue and saturation together are usually called chromaticity. Spring 2002 Video Processing - Dr. Marques 40
Color fundamentals (7) CIE chromaticity diagram (x = red, y = green) Note that a triangle with vertices at any three fixed colors cannot enclose the entire (tongue-shaped) color region Triangle can be the color gamut (typical range of colors) of a device (e.g., monitor) Spring 2002 Video Processing - Dr. Marques 41
Color models (1) A color model (also called color space or color system) is a specification of a coordinate system and a subspace within that system where each color is represented by a single point. Hardware-oriented models: RGB (cameras and monitors) CMY and CMYK (color printers) User-oriented models: HIS (Hue-Saturation-Intensity) L*a*b* (CIELAB) (many others) Spring 2002 Video Processing - Dr. Marques 42
RGB color model (1) Based on a Cartesian coordinate system. Unit cube: all values of R, G, and B are normalized to the range [0,1]. Pixel depth: number of bits used to represent each pixel (typical value: 24 bits = 3 image planes * 8 bits per plane) Spring 2002 Video Processing - Dr. Marques 43
RGB color model (2) The RGB safe-color cube Safe colors are the 216 colors that are likely to be well reproduced in any system. Spring 2002 Video Processing - Dr. Marques 44
Spring 2002 Video Processing - Dr. Marques 45 CMY and CMYK color models Used for color printers Subtractive primaries (plus black) Conversion from RGB to CMY is straightforward: C M Y R G B = 1 1 1
The HSI color model (1) Spring 2002 Video Processing - Dr. Marques 46
The HSI color model (2) Conversion from RGB to HSI: S = 3 1 R+ G+ B min R G B [ (,, )] 1 I = R+ G+ B 3 ( ) H = cos 1 undefined 1 2 360 H if [( R G) + ( R B) ] 2 [( R G) + ( R B)( G B) ] if B / V S > = 0 G / V 1/ 2 Spring 2002 Video Processing - Dr. Marques 47
The HSI color model (3) Conversion from HSI to RGB: For 0º < H 120º: 1 b= S 3 ( 1 ) r 1 S H = + 3 1 cos cos( 60 H) g = 1 ( r + b) For 120º < H 240º: 1 r = S 3 ( 1 ) g H = H 120 1 S H = + 3 1 cos cos( 60 H) b = 1 ( r+ g) For 240º < H 360º: 1 g = S 3 ( 1 ) b H = H 240 1 S H = + 3 1 cos cos( 60 H) r = 1 ( b+ g) Spring 2002 Video Processing - Dr. Marques 48
The HSI color model (4) Advantages: It matches the human way of describing colors It allows for independent control over hue, saturation, and intensity Limitations: Discontinuity in numeric values of Hue around red Expensive conversion to/from RGB Hue is undefined for a saturation of 0 Spring 2002 Video Processing - Dr. Marques 49
Perceptually uniform color models Chief characteristic: unit distance in the space corresponds to a uniform perceived difference in color. MacAdam ellipses and Just Noticeable Difference (JND) in color. If the distance in in coordinate space < fixed threshold: humans cannot perceive the difference. Examples: CIELAB: for reflected light CIELUV: for emitted light Spring 2002 Video Processing - Dr. Marques 50
Pseudocolor image processing Basic idea: assigning colors to gray values based on a specified criterion. Simplest and best-known technique: intensity (or density) slicing. Viewing the monochrome image as a 2-D intensity function, this method defines slicing planes that slice the original image in different points above the xy plane. Each side of the plane will be assigned a different color. The result will be a two-color image whose color distribution can be controlled by moving the slicing plane up or down. The idea can be easily extended to M planes and M+1 intervals. Spring 2002 Video Processing - Dr. Marques 51
Intensity slicing example Spring 2002 Video Processing - Dr. Marques 52
Spring 2002 Video Processing - Dr. Marques 53 Full-color image processing Two major categories: Component-wise: each component image is processed individually and then form a composite processed image Vector methods: color pixels are treated as vectors: = = ), ( ), ( ), ( ), ( ), ( ), ( ), ( y x B y x G y x R y x c y x c y x c y x B G R c
Full-color image processing The two methods are equivalent if and only if: The process is applicable to both vectors and scalars. The operation on each component of a vector is independent of the other components. Spring 2002 Video Processing - Dr. Marques 54
Full-color image processing techniques Color transformations Color complements Color slicing Tone and color corrections Histogram processing Smoothing and sharpening Noise removal Segmentation Edge detection Spring 2002 Video Processing - Dr. Marques 55
Color transformations It is possible to extend the concept of grayscale transformations to color images. Example: reducing the intensity of an image to 70% of its original value 2 options: Apply transfer function to R, G, and B individually. Apply transfer function to I, leaving H and S unchanged. Spring 2002 Video Processing - Dr. Marques 56
Color complements Equivalent of grayscale negative for color images. 2 options: Apply trivial transfer function to R, G, and B individually. Apply a trivial transfer function to I, a nontrivial transfer function to H and leave S unchanged. Spring 2002 Video Processing - Dr. Marques 57
Color slicing Mapping process by which: All colors outside a range of interest are mapped to a neutral color. All colors of interest remain unchanged. Region of interest is typically viewed as a hypercube or hypersphere centered at a prototypical reference color. Spring 2002 Video Processing - Dr. Marques 58
Tone and color corrections Used for photo enhancement and improvements on color reproduction. Particularly important in professional publishing, where consistency between monitors and output devices (e.g., color printers) must be enforced. Very often an interactive process that we consider part of the image manipulation category. Spring 2002 Video Processing - Dr. Marques 59
Histogram processing The concept of histogram can be extended to color images: 3 histograms with N (4 N 256) bins each. Histogram equalization can be applied to the Intensity component of a color image, leaving H and S unchanged. Spring 2002 Video Processing - Dr. Marques 60
Smoothing and sharpening Linear smoothing and sharpening techniques can be extended to color images under the component-wise paradigm. Surprisingly enough, both RGB and HSI models can be used with little difference in results (if masks are small, e.g., 3 x 3). Examples: Mean (neighborhood averaging) filter and Laplacian. Spring 2002 Video Processing - Dr. Marques 61
Noise removal The impact of noise on color images is strongly dependent on the color model used. Even when only one of the R,G, or B channels is affected by noise, conversion to HSI spreads to all HSI components. Spring 2002 Video Processing - Dr. Marques 62
Noise removal Lenna (RGB) Spring 2002 Video Processing - Dr. Marques 63
Noise removal Lenna (HSI) Spring 2002 Video Processing - Dr. Marques 64
Noise removal Noisy Lenna (RGB) Spring 2002 Video Processing - Dr. Marques 65
Noise removal Noisy Lenna (HSI) Spring 2002 Video Processing - Dr. Marques 66
Noise removal Linear techniques (such as mean filter) can be applied on each R,G, and B component. Non-linear techniques (such as median filter) cannot be extended just as easily. The vector equivalent of the median filter was first published in 1990 (!) and the extension of linear image processing algorithms to vectors started only in 1998 (!!) (see required reading for pointers). Spring 2002 Video Processing - Dr. Marques 67
Segmentation Segmentation is the process of partitioning an image into regions. Color segmentation can be done both in HSI (more intuitive) as well as RGB spaces, with RGB providing better results. Spring 2002 Video Processing - Dr. Marques 68
Edge detection Conventional gradient calculations can be applied to a color image on a percomponent basis. Best results are obtained using an extension of the gradient approach to vector quantities proposed by Di Zenzo (1986). Spring 2002 Video Processing - Dr. Marques 69