IMAGE PROCESSING >COLOR SPACES UTRECHT UNIVERSITY RONALD POPPE

Transcription

1 IMAGE PROCESSING >COLOR SPACES UTRECHT UNIVERSITY RONALD POPPE

2 OUTLINE Human visual system Color images Color quantization Colorimetric color spaces

3 HUMAN VISUAL SYSTEM

4 HUMAN VISUAL SYSTEM

5 HUMAN VISUAL SYSTEM 2 Why cover it here? Difference between perceived and displayed image Luminance is actual physical brightness Brightness is how we perceive it (e.g., in the dark, we distinguish between dark and light ) We often try to emulate the human visual system Our eyes/brain can do tricks that we like to mimic

6 HUMAN VISUAL SYSTEM 3 To the brain

7 HUMAN VISUAL SYSTEM 4 Intermezzo: blind spot Close your left eye and fixate your right eye on the cross Now move closer to the screen At one point, the circle should disappear

8 HUMAN VISUAL SYSTEM 5 The eye and part of the brain are tasked with visual processing The eye converts light rays to electric pulses that are processed in the brain Light can be considered a wave and has: An amplitude: the intensity A frequency: the color

9 HUMAN VISUAL SYSTEM 6 Light rays with a sufficient strength and within the right range of wavelengths generate an electric pulse in the brain Cornea protects the eye and refracts the beams Pupil and cornea regulate the total amount of light Lens further refracts the light, to focus on certain distances The retina converts light into electric pulses using rods and cones

10 HUMAN VISUAL SYSTEM 7 Rods Cones 100 million 6-7 million Evenly spread More responsive grey Night vision (scotopic) Especially around fovea Three types color Day vision (photopic)

11 HUMAN VISUAL SYSTEM 8 Luminous efficiency: How do things appear colored?

12 HUMAN VISUAL SYSTEM 9 Color blindness explained:

13 QUESTIONS?

14 COLOR IMAGES

15 COLOR IMAGES We talked about grey level intensity values Single number per pixel: intensity Colors can be described in two different ways: True color: pixel represented as multiple dimensions/channels Indexed: pixel value refers to a row in a color array

16 COLOR IMAGES 2 Indexed color uses a palette or look-up array P of N colors: With array size of N, N is the number of possible colors Each pixel value k (0 k < N) is an index to this array With color channels red, green, blue: P(k) = (r k, g k, b k )

17 COLOR IMAGES 3 Two ways of implementing/ordering true color images: 1. Component (planar) ordering Each color channel has its own array/layer 2. Packed ordering Each pixel comprises of a part per channel Example for 24-bits colors: Component: 3 arrays of 8-bit colors Packed: 1 array of 24-bit colors

18 COLOR IMAGES 4 A color space describes how color dimensions make up a color Additive color spaces start with black and adding primary colors Color is mixed as beams of light Subtractive color spaces start with white and each color reduces the intensity of the reflected light Typical in print

19 RGB Most common additive color space is red, green blue (RGB) Typical in light emitters (e.g. TV screens) Color described as a combination of values for red, green and blue Each channel is limited in the number of values (2 n ) Number of colors = 2 n*3 For n=8: 24-bits color

20 RGB 2 In RGB color space, assuming a range [0,1]: Black is (0,0,0) (no light) White is (1,1,1) (combination of all wavelengths) Gray colors have equal values for red, green and blue

21 RGB 3 Values are indicative of the intensity/luminance of the pixel Conversion to grayscale: Y = w R *R + w G *G + w B *B w R, w G and w B depend on application w G usually higher because green dominates our perception of light intensity

22 RGB 4 Saturation is the amount of color or the deviation from gray of the same intensity Desaturation is the process of uniform reduction of the saturation in a continuous manner Factor s COL ϵ [0,1] is used to control the amount of color Y is the intensity value For s COL = 0, same as grayscale conversion For values s COL > 1, exaggeration of the color

23 RGB 5 Advantages of RGB color space: Easy to understand Fits well with the way TV screens work (three types of LEDs) Disadvantages: Strong correlations between color channels Brightness (dark-light) affects all channels Does not correlate well with human vision

24 HSV AND HLS The Hue, Saturation, Value (HSV) space solves some drawbacks of RGB Independent channel for intensity (value, V) Independent channel for saturation (S) Independent channel for color tone (hue, H) Often visualized as a cone/pyramid Implemented as cylinder Hue and saturation are irrelevant for black

25 HSV AND HLS 2 Hue typically seen as angle Interpolation from red-green-blue-red HLS (Hue, Luminance, Saturation) is very similar to HSV Motivated by observation that white is brighter than e.g. red Double pyramid (HLS) instead of single (HSV) Implementation is again a cylinder

26 HSV AND HLS 2 Conversion from RGB to HSV Calculate saturation Calculate luminance (value) Calculate hue: When R=G=B, the color is gray and hue is undefined First find dominant channel (red, green or blue) Determine offset from other channels Convert to angle/unit range

27 HSV AND HLS 3 Example conversion RGB to HSV: RGB = (250, 150, 50) orange Calculate high, low and range: C high = max(r, G, B) = 250 C low = min(r, G, B) = 50 C rng = C high C low = = 200 Calculate saturation: S HSV = 200/250 = 0.8 Calculate value: V HSV = 250/255 = 0.98

28 HSV AND HLS 4 Continued conversion RGB to HSV: RGB = (250, 150, 50) orange Normalize color components: Determine hue based on dominant channel: Normalize hue value: In our example, R =0, G =0.5, B =1, H = = 0.5, H HSV = 0.08 (30 )

29 HSV AND HLS 5 See the book for other conversions Comparison of HSV and HLS: Hue channel is identical Differences in value (HSV)/luminance (HLS) Differences in saturation Differences especially for brighter colors

30 HSV AND HLS 6 Distribution of colors in RGB, HSV and HLS Starting from uniform distribution in RGB Bright colors more densely projected to HSV RGB allows for more differentiation in light colors But less in darker colors! Opposite effect for HSV

31 TV COLOR SPACES YUV and YIQ are color spaces of NTSC and PAL YCbCr is color space used in digital television All have an intensity channel (Y) and two chromatic channels (blue and red) Intensity typically encoded with more bandwidth Instead of encoding color value, they encode color difference More suitable to guarantee compatibility with legacy systems

32 COLOR SPACES FOR PRINTING Print uses subtractive color spaces: Reflection of light rather than mixing of light Example: Blue paint absorbs all colors but blue When shining a red light on blue paint, it appears black

33 COLOR SPACES FOR PRINTING 2 Most common model: Cyan, Magenta, Yellow (CMY) Y does not stand for intensity! C = 1 R, M = 1 G, Y = 1 - B CMYK also has black (K) channel Black difficult to mix from C, M and Y (true black cannot be mixed) Black ink is much cheaper

34 COLOR IMAGE STATISTICS Counting the number of unique colors: Keeping an array with the counts of all possible colors: 2 24 cells! Alternatively: sort pixels based on some criteria (e.g. concatenate RGB bytes) Criteria not really important, but must be unique Non-zero transitions for different colors Count transitions between subsequent pixels (and add 1)

35 COLOR IMAGE STATISTICS 2 We can make a histogram of color values of all pixels: Histogram per channel does not provide full story 3D histogram is too large (2 24 bins) Alternative: use two (or three) 2D histograms E.g., RG, RB and GB

36 QUESTIONS?

37 COLOR QUANTIZATION

38 COLOR QUANTIZATION When we move from true colors to palette colors, we need to reduce the total number of distinct colors Common when converting to GIF or PNG Color quantization: process of replacing the original colors with colors from a smaller set Challenge is to define a palette that faithfully represents the original colors

39 COLOR QUANTIZATION 2 Scalar/uniform quantization ignores color counts Simple and fast Convert value c i per channel i: with n and m the original and new number of colors, respectively Same as integer division and truncation Drop least significant bits/bits

40 COLOR QUANTIZATION 3 Scalar quantization is only optimal for uniformly distributed colors Never happens in practice When we include colors in our palette that do not occur in the image Waste of colors Underrepresentation of colors that do occur Scalar quantization also does not consider dependencies between color channels

41 COLOR QUANTIZATION 4 Vector quantization looks at color values and how often they occur: 1. Find a set of n representative color vectors (the palette) 2. Replace each original color with a color from the palette Aim is to minimize the difference between original and new image Requires distance measure such as Euclidian distance Combinatorial optimization problem so exact solution takes time Typically a local solution is used

42 COLOR QUANTIZATION 5 Populosity algorithm selects n most frequent colors as palette Sometimes preceded by scalar quantization Median cut algorithm first builds a 3D histogram of all colors Select the box with the largest number of pixels Split the box across the longest axis, at the median point Each split replaces one color by two new ones (center of box) Repeat until the desired number (n) of colors is obtained Colors mostly sampled in high-density areas

43 COLOR QUANTIZATION 6 Octree algorithm builds a hierarchy of colors from scratch: Each box split in up to 8 subboxes Maximum number of K color values stored in tree Allows for quick determination of closest color from palette More intuitive intro:

44 COLOR QUANTIZATION 7 Comparison median-cut and octree More uniform for median Lower error for octree Octree has more spikes for incidental colors

45 COLOR QUANTIZATION 8 Typically, a small sample (e.g., 10%) of the pixels suffices to build the palette Other techniques to determine the palette include clustering K-means is an algorithm that quickly converges to a local solution

46 COLORIMETRIC COLOR SPACES

47 COLORIMETRIC COLOR SPACES Device or medium typically determines to a large extent how colors appear Need for a representation that is independent of device and medium Colorimetric or calibrated color spaces do exactly this Motivated by physical properties of light or human visual system

48 COLORIMETRIC COLOR SPACES 2 CIE XYZ describes color as linear combination of (imaginary) primary colors Visual colors lie on a cone-shaped region in this space Y component encodes lightness/luminosity Black is the origin of the space (0,0,0) RGB colors are embedded in CIE XYZ space Linear transformation between RGB and CIE XYZ

49 COLORIMETRIC COLOR SPACES 3 Hue and saturation described using chromaticity channels x, y, z Channels sum up to 1, so one channel can be omitted x,y plane describes hue and saturation Visible colors only within the area Boundary corresponds to pure colors (except for connection) Saturation decreases towards point E x, y plane does not denote luminosity

50 COLORIMETRIC COLOR SPACES 4 Standard illuminants are common light settings that define how colors should appear in a particular setting D50: direct sunlight, a color temperature of 5000 K D65: indirect sunlight (cloudy day), color temperature of 6500 K Each illuminant defines specific values of X, Y and Z for the reference white

51 COLORIMETRIC COLOR SPACES 5 Gamut is the region of possible colors in a color space 3D space in CIE XYZ or a 2D region in the chromaticity plane Important to realize that a color space is limited Limitation of CIE XYZ is that is it not perceptually motivated Similar differences in (x,y) values are perceived differently CIE YUV, CIE YC b C r and CIE L*a*b* address this

52 COLORIMETRIC COLOR SPACES 6 Distances in CIE L*a*b* correspond with those perceived by humans Leads to more intuitive color space L* is luminosity channel a* and b* are the color components red-green and blue-yellow Colors are relative to a reference white point Color differences are measured as Euclidian distance

53 COLORIMETRIC COLOR SPACES 7 CIE L*a*b* corresponds to human perception Not optimally efficient in digital processing srgb designed to be more efficient Used in JPEG and PNG As in CIE XYZ, srgb uses primary colors, reference white, ambient lighting conditions and gamma values Allows for translation between CIE XYZ and srgb Transformation consists of nonlinear (gamma) part and linear part Adobe RGB is similar to srgb but covers a larger gamut

54 COLORIMETRIC COLOR SPACES 8 Humans can perceive the same color under different lighting conditions as the same Conversely: lighting conditions affect how we perceive colors Describing colors relative to a reference white can compensate for this Chromatic adaptation is the process of translating between reference whites XYZ scaling adapts the colors per color channel Bradford adaptation first translates to an RGB space, then performs the adaptation and then transforms back to CIE XYZ space

55 COLORIMETRIC COLOR SPACES 9 Choosing ideal color space depends on: Input and output device Importance of relation to visual perception Importance of efficient calculation There is no best color space! But HSV is pretty handy for most image processing tasks

56 QUESTIONS?

57 NEXT LECTURE

58 NEXT LECTURE Next lecture is about: Filters for Color Images (Book II, Chapter 3) Edge Detection for Color Images (Book II, Chapter 4) Wednesday October 31, 15:15-17:00 (RUPPERT-PAARS) Walk-in session: Friday November 2, 9:00-10:45 Deadline Sunday November 11, 23:00

59 EXAM Q&A Exam Q&A on Friday November 2, 11:00-12:45 Send me questions/topics so I can cover them No later than Wednesday October 31

60 CONTENTS OF THIS LECTURE

61 CONTENTS OF THIS LECTURE Fundamental Techniques (book I) Chapter 8: Color Images (not 8.1.2, no Java implementations) Core Algorithms (book II) Chapter 5: Color Quantization Chapter 6: Colorimetric Color Spaces (not 6.6) Human Visual System These slides