IMAGE PROCESSING >COLOR SPACES UTRECHT UNIVERSITY RONALD POPPE
OUTLINE Human visual system Color images Color quantization Colorimetric color spaces
HUMAN VISUAL SYSTEM
HUMAN VISUAL SYSTEM
HUMAN VISUAL SYSTEM 2 Why cover it here? Difference between perceived and displayed image Luminance is actual physical brightness Brightness is how we perceive it (e.g., in the dark, we distinguish between dark and light ) We often try to emulate the human visual system Our eyes/brain can do tricks that we like to mimic
HUMAN VISUAL SYSTEM 3 To the brain
HUMAN VISUAL SYSTEM 4 Intermezzo: blind spot Close your left eye and fixate your right eye on the cross Now move closer to the screen At one point, the circle should disappear
HUMAN VISUAL SYSTEM 5 The eye and part of the brain are tasked with visual processing The eye converts light rays to electric pulses that are processed in the brain Light can be considered a wave and has: An amplitude: the intensity A frequency: the color
HUMAN VISUAL SYSTEM 6 Light rays with a sufficient strength and within the right range of wavelengths generate an electric pulse in the brain Cornea protects the eye and refracts the beams Pupil and cornea regulate the total amount of light Lens further refracts the light, to focus on certain distances The retina converts light into electric pulses using rods and cones
HUMAN VISUAL SYSTEM 7 Rods Cones 100 million 6-7 million Evenly spread More responsive grey Night vision (scotopic) Especially around fovea Three types color Day vision (photopic)
HUMAN VISUAL SYSTEM 8 Luminous efficiency: How do things appear colored?
HUMAN VISUAL SYSTEM 9 Color blindness explained:
QUESTIONS?
COLOR IMAGES
COLOR IMAGES We talked about grey level intensity values Single number per pixel: intensity Colors can be described in two different ways: True color: pixel represented as multiple dimensions/channels Indexed: pixel value refers to a row in a color array
COLOR IMAGES 2 Indexed color uses a palette or look-up array P of N colors: With array size of N, N is the number of possible colors Each pixel value k (0 k < N) is an index to this array With color channels red, green, blue: P(k) = (r k, g k, b k )
COLOR IMAGES 3 Two ways of implementing/ordering true color images: 1. Component (planar) ordering Each color channel has its own array/layer 2. Packed ordering Each pixel comprises of a part per channel Example for 24-bits colors: Component: 3 arrays of 8-bit colors Packed: 1 array of 24-bit colors
COLOR IMAGES 4 A color space describes how color dimensions make up a color Additive color spaces start with black and adding primary colors Color is mixed as beams of light Subtractive color spaces start with white and each color reduces the intensity of the reflected light Typical in print
RGB Most common additive color space is red, green blue (RGB) Typical in light emitters (e.g. TV screens) Color described as a combination of values for red, green and blue Each channel is limited in the number of values (2 n ) Number of colors = 2 n*3 For n=8: 24-bits color
RGB 2 In RGB color space, assuming a range [0,1]: Black is (0,0,0) (no light) White is (1,1,1) (combination of all wavelengths) Gray colors have equal values for red, green and blue
RGB 3 Values are indicative of the intensity/luminance of the pixel Conversion to grayscale: Y = w R *R + w G *G + w B *B w R, w G and w B depend on application w G usually higher because green dominates our perception of light intensity
RGB 4 Saturation is the amount of color or the deviation from gray of the same intensity Desaturation is the process of uniform reduction of the saturation in a continuous manner Factor s COL ϵ [0,1] is used to control the amount of color Y is the intensity value For s COL = 0, same as grayscale conversion For values s COL > 1, exaggeration of the color
RGB 5 Advantages of RGB color space: Easy to understand Fits well with the way TV screens work (three types of LEDs) Disadvantages: Strong correlations between color channels Brightness (dark-light) affects all channels Does not correlate well with human vision
HSV AND HLS The Hue, Saturation, Value (HSV) space solves some drawbacks of RGB Independent channel for intensity (value, V) Independent channel for saturation (S) Independent channel for color tone (hue, H) Often visualized as a cone/pyramid Implemented as cylinder Hue and saturation are irrelevant for black
HSV AND HLS 2 Hue typically seen as angle Interpolation from red-green-blue-red HLS (Hue, Luminance, Saturation) is very similar to HSV Motivated by observation that white is brighter than e.g. red Double pyramid (HLS) instead of single (HSV) Implementation is again a cylinder
HSV AND HLS 2 Conversion from RGB to HSV Calculate saturation Calculate luminance (value) Calculate hue: When R=G=B, the color is gray and hue is undefined First find dominant channel (red, green or blue) Determine offset from other channels Convert to angle/unit range
HSV AND HLS 3 Example conversion RGB to HSV: RGB = (250, 150, 50) orange Calculate high, low and range: C high = max(r, G, B) = 250 C low = min(r, G, B) = 50 C rng = C high C low = 250 50 = 200 Calculate saturation: S HSV = 200/250 = 0.8 Calculate value: V HSV = 250/255 = 0.98
HSV AND HLS 4 Continued conversion RGB to HSV: RGB = (250, 150, 50) orange Normalize color components: Determine hue based on dominant channel: Normalize hue value: In our example, R =0, G =0.5, B =1, H = 1 0.5 = 0.5, H HSV = 0.08 (30 )
HSV AND HLS 5 See the book for other conversions Comparison of HSV and HLS: Hue channel is identical Differences in value (HSV)/luminance (HLS) Differences in saturation Differences especially for brighter colors
HSV AND HLS 6 Distribution of colors in RGB, HSV and HLS Starting from uniform distribution in RGB Bright colors more densely projected to HSV RGB allows for more differentiation in light colors But less in darker colors! Opposite effect for HSV
TV COLOR SPACES YUV and YIQ are color spaces of NTSC and PAL YCbCr is color space used in digital television All have an intensity channel (Y) and two chromatic channels (blue and red) Intensity typically encoded with more bandwidth Instead of encoding color value, they encode color difference More suitable to guarantee compatibility with legacy systems
COLOR SPACES FOR PRINTING Print uses subtractive color spaces: Reflection of light rather than mixing of light Example: Blue paint absorbs all colors but blue When shining a red light on blue paint, it appears black
COLOR SPACES FOR PRINTING 2 Most common model: Cyan, Magenta, Yellow (CMY) Y does not stand for intensity! C = 1 R, M = 1 G, Y = 1 - B CMYK also has black (K) channel Black difficult to mix from C, M and Y (true black cannot be mixed) Black ink is much cheaper
COLOR IMAGE STATISTICS Counting the number of unique colors: Keeping an array with the counts of all possible colors: 2 24 cells! Alternatively: sort pixels based on some criteria (e.g. concatenate RGB bytes) Criteria not really important, but must be unique Non-zero transitions for different colors Count transitions between subsequent pixels (and add 1)
COLOR IMAGE STATISTICS 2 We can make a histogram of color values of all pixels: Histogram per channel does not provide full story 3D histogram is too large (2 24 bins) Alternative: use two (or three) 2D histograms E.g., RG, RB and GB
QUESTIONS?
COLOR QUANTIZATION
COLOR QUANTIZATION When we move from true colors to palette colors, we need to reduce the total number of distinct colors Common when converting to GIF or PNG Color quantization: process of replacing the original colors with colors from a smaller set Challenge is to define a palette that faithfully represents the original colors
COLOR QUANTIZATION 2 Scalar/uniform quantization ignores color counts Simple and fast Convert value c i per channel i: with n and m the original and new number of colors, respectively Same as integer division and truncation Drop least significant bits/bits
COLOR QUANTIZATION 3 Scalar quantization is only optimal for uniformly distributed colors Never happens in practice When we include colors in our palette that do not occur in the image Waste of colors Underrepresentation of colors that do occur Scalar quantization also does not consider dependencies between color channels
COLOR QUANTIZATION 4 Vector quantization looks at color values and how often they occur: 1. Find a set of n representative color vectors (the palette) 2. Replace each original color with a color from the palette Aim is to minimize the difference between original and new image Requires distance measure such as Euclidian distance Combinatorial optimization problem so exact solution takes time Typically a local solution is used
COLOR QUANTIZATION 5 Populosity algorithm selects n most frequent colors as palette Sometimes preceded by scalar quantization Median cut algorithm first builds a 3D histogram of all colors Select the box with the largest number of pixels Split the box across the longest axis, at the median point Each split replaces one color by two new ones (center of box) Repeat until the desired number (n) of colors is obtained Colors mostly sampled in high-density areas
COLOR QUANTIZATION 6 Octree algorithm builds a hierarchy of colors from scratch: Each box split in up to 8 subboxes Maximum number of K color values stored in tree Allows for quick determination of closest color from palette More intuitive intro: https://www.youtube.com/watch?v=lqst9mitkrw
COLOR QUANTIZATION 7 Comparison median-cut and octree More uniform for median Lower error for octree Octree has more spikes for incidental colors
COLOR QUANTIZATION 8 Typically, a small sample (e.g., 10%) of the pixels suffices to build the palette Other techniques to determine the palette include clustering K-means is an algorithm that quickly converges to a local solution
COLORIMETRIC COLOR SPACES
COLORIMETRIC COLOR SPACES Device or medium typically determines to a large extent how colors appear Need for a representation that is independent of device and medium Colorimetric or calibrated color spaces do exactly this Motivated by physical properties of light or human visual system
COLORIMETRIC COLOR SPACES 2 CIE XYZ describes color as linear combination of (imaginary) primary colors Visual colors lie on a cone-shaped region in this space Y component encodes lightness/luminosity Black is the origin of the space (0,0,0) RGB colors are embedded in CIE XYZ space Linear transformation between RGB and CIE XYZ
COLORIMETRIC COLOR SPACES 3 Hue and saturation described using chromaticity channels x, y, z Channels sum up to 1, so one channel can be omitted x,y plane describes hue and saturation Visible colors only within the area Boundary corresponds to pure colors (except for connection) Saturation decreases towards point E x, y plane does not denote luminosity
COLORIMETRIC COLOR SPACES 4 Standard illuminants are common light settings that define how colors should appear in a particular setting D50: direct sunlight, a color temperature of 5000 K D65: indirect sunlight (cloudy day), color temperature of 6500 K Each illuminant defines specific values of X, Y and Z for the reference white
COLORIMETRIC COLOR SPACES 5 Gamut is the region of possible colors in a color space 3D space in CIE XYZ or a 2D region in the chromaticity plane Important to realize that a color space is limited Limitation of CIE XYZ is that is it not perceptually motivated Similar differences in (x,y) values are perceived differently CIE YUV, CIE YC b C r and CIE L*a*b* address this
COLORIMETRIC COLOR SPACES 6 Distances in CIE L*a*b* correspond with those perceived by humans Leads to more intuitive color space L* is luminosity channel a* and b* are the color components red-green and blue-yellow Colors are relative to a reference white point Color differences are measured as Euclidian distance
COLORIMETRIC COLOR SPACES 7 CIE L*a*b* corresponds to human perception Not optimally efficient in digital processing srgb designed to be more efficient Used in JPEG and PNG As in CIE XYZ, srgb uses primary colors, reference white, ambient lighting conditions and gamma values Allows for translation between CIE XYZ and srgb Transformation consists of nonlinear (gamma) part and linear part Adobe RGB is similar to srgb but covers a larger gamut
COLORIMETRIC COLOR SPACES 8 Humans can perceive the same color under different lighting conditions as the same Conversely: lighting conditions affect how we perceive colors Describing colors relative to a reference white can compensate for this Chromatic adaptation is the process of translating between reference whites XYZ scaling adapts the colors per color channel Bradford adaptation first translates to an RGB space, then performs the adaptation and then transforms back to CIE XYZ space
COLORIMETRIC COLOR SPACES 9 Choosing ideal color space depends on: Input and output device Importance of relation to visual perception Importance of efficient calculation There is no best color space! But HSV is pretty handy for most image processing tasks
QUESTIONS?
NEXT LECTURE
NEXT LECTURE Next lecture is about: Filters for Color Images (Book II, Chapter 3) Edge Detection for Color Images (Book II, Chapter 4) Wednesday October 31, 15:15-17:00 (RUPPERT-PAARS) Walk-in session: Friday November 2, 9:00-10:45 Deadline Sunday November 11, 23:00
EXAM Q&A Exam Q&A on Friday November 2, 11:00-12:45 Send me questions/topics so I can cover them No later than Wednesday October 31
CONTENTS OF THIS LECTURE
CONTENTS OF THIS LECTURE Fundamental Techniques (book I) Chapter 8: Color Images (not 8.1.2, no Java implementations) Core Algorithms (book II) Chapter 5: Color Quantization Chapter 6: Colorimetric Color Spaces (not 6.6) Human Visual System These slides