Visual Perception Spring 06 Instructor: K. J. Ray Liu ECE Department, Univ. of Maryland, College Park Overview Last Class Introduction to DIP/DVP applications and examples Image as a function Concepts of sampling and quantization Today Human visual perception: monochrome vision and color vision ENEE631 Digital Image Processing (Spring'06) ENEE631 Digital Image Processing (Spring'06) Lec2 HVS [2] Information Processing by Human Observer The Eye image Visual perception Concerns how an image is perceived by a human observer eye perceived image preliminary processing by eye this lecture further processing by brains understanding of content Important for developing image fidelity measures needed for design and evaluate DIP/DVP algorithms & systems Cross section illustration Figure is from slides at Gonzalez/ Woods DIP book website (Chapter 2) ENEE631 Digital Image Processing (Spring'06) Lec2 HVS [3] ENEE631 Digital Image Processing (Spring'06) Lec2 HVS [4] 1
UMCP ENEE631 Slides (created by M.Wu 2001/2004) Two Types of Photoreceptors at Retina Rods Long and thin Large quantity (~ 100 million) Provide scotopic vision (i.e., dim light vision or at low illumination) Only extract luminance information and provide a general overall picture Cones Short and thick, densely packed in fovea (center of retina) Much fewer (~ 6.5 million) and less sensitive to light than rods Provide photopic vision (i.e., bright light vision or at high illumination) Help resolve fine details as each cone is connected to its own nerve end Responsible for color vision our interest (well-lighted display) Mesopic vision provided at intermediate illumination by both rod and cones ENEE631 Digital Image Processing (Spring'06) Lec2 HVS [5] Light Figure is from slides at Gonzalez/ Woods DIP book website (Chapter 2) Light is an electromagnetic wave with wavelength of 350nm to 780nm stimulating human visual response Expressed as spectral energy distribution I(λ) The range of light intensity levels that human visual system can adapt is huge: ~ on 10 orders of magnitude (10 10 ) but not simultaneously Brightness adaptation: small intensity range to discriminate simultaneously ENEE631 Digital Image Processing (Spring'06) Lec2 HVS [6] Luminance vs. Brightness Luminance vs. Brightness (cont d) UMCP ENEE631 Slides (created by M.Wu 2001/2004) Same lum. Different brightness Luminance (or intensity) Independent of the luminance of surroundings I(x,y,λ) -- spatial light distribution V(λ) -- relative luminous efficiency func. of visual system ~ bell shape (different for scotopic vs. photopic vision; highest for green wavelength, second for red, and least for blue ) Brightness Perceived luminance Depends on surrounding luminance Different lum. Similar brightness Example: visible digital watermark How to make the watermark appears the same graylevel all over the image? from IBM Watson web page Vatican Digital Library ENEE631 Digital Image Processing (Spring'06) Lec2 HVS [7] ENEE631 Digital Image Processing (Spring'06) Lec2 HVS [8] 2
Look into Simultaneous Contrast Phenomenon Mach Bands UMCP ENEE631 Slides (created by M.Wu 2001/2004) Human perception more sensitive to luminance contrast than absolute luminance Weber s Law: L s L 0 / L 0 = const Luminance of an object (L 0 ) is set to be just noticeable from luminance of surround (L s ) For just-noticeable luminance difference ΔL: ΔL / L d( log L ) 0.02 (const) equal increments in log luminance are perceived as equally different Empirical luminance-to-contrast models Assume L [1, 100], and c [0, 100] c = 50 log 10 L (logarithmic law, widely used) c = 21.9 L 1/3 (cubic root law) Figure is from slides at Gonzalez/ Woods DIP book website (Chapter 2) Visual system tends to undershoot or overshoot around the boundary of regions of different intensities Demonstrates the perceived brightness is not a simple function of light intensity ENEE631 Digital Image Processing (Spring'06) Lec2 HVS [9] ENEE631 Digital Image Processing (Spring'06) Lec2 HVS [10] Visual Angle and Spatial Frequency Visual angle matters more than absolute distance Smaller but closer object vs. larger but farther object Eyes can distinguish about 25-30 lines per degree in bright illumination Spatial Frequency Measures the extent of spatial transition in unit of cycles per visual degree dot dot Visibility Threshold at Various Spatial Frequency 2-D linear (spatial) invariant system can be characterized by Point Spread Function (PSF) (i.e. impulse response) and the 2-D transfer function The magnitude of the (normalized) transfer function is called the Modulation Transfer Function (MTF) We ll study 2-D systems and transforms in detail in 2 lectures Visibility threshold at different spatial frequency Eyes are most sensitive to mid frequencies, and least sensitive to high frequencies Most sensitive to horizontal and vertical ones than other orientations ENEE631 Digital Image Processing (Spring'06) Lec2 HVS [11] ENEE631 Digital Image Processing (Spring'06) Lec2 HVS [12] 3
Monochrome Vision Models See Jain s s Fig.3.7 (pp55) Fig.3.9 (pp57) Image Fidelity Criteria MTF of Human Visual System (HVS) Modulation Transfer Func. (MTF) from experiment with sinusoidal grating of varying contrast similar to a band-pass filter most sensitive to mid frequencies least sensitive to high frequencies also depends on the orientation of grating most sensitive to horizontal and vertical ones Overall monochrome vision models How light is transformed by eye to brightness information Subjective measures Examination by human viewers Goodness scale: excellent, good, fair, poor, unsatisfactory Impairment scale: unnoticeable, just noticeable, Comparative measures with another image or among a group of images Objective (Quantitative) measures Mean square error and variations Pro: Simple, less dependent on human subjects, & easy to handle mathematically Con: Not always reflect human perception ENEE631 Digital Image Processing (Spring'06) Lec2 HVS [13] ENEE631 Digital Image Processing (Spring'06) Lec2 HVS [14] Mean-square Criterion Average (or sum) of squared difference of pixel luminance between two images Signal-to-noise ratio (SNR) SNR = 10 log 10 ( σ s2 / σ e2 ) in unit of decibel (db) σ s2 image variance, σ e2 variance of noise or error PSNR = 10 log 10 ( A 2 / σ e2 ) with A being peak-to-peak value PSNR is about 12-15 db higher than SNR Color of Light Perceived color depends on spectral content (wavelength composition) e.g., 700nm ~ red. spectral color A light with very narrow bandwidth Spectrum from http://www.physics.sfasu.edu/astro/color.html A light with equal energy in all visible bands appears white ENEE631 Digital Image Processing (Spring'06) Lec2 HVS [15] ENEE631 Digital Image Processing (Spring'06) Lec2 HVS [16] 4
Perceptual Attributes of Color Absorption of Light by R/G/B Cones Value of Brightness (perceived luminance) Chrominance Hue specify color tone (redness, greenness, etc.) depend on peak wavelength Saturation describe how pure the color is depend on the spread (bandwidth) of light spectrum reflect how much white light is added HSV circular cone is from online documentation of Matlab image processing toolbox Figure is from slides at Gonzalez/ Woods DIP book website (Chapter 6) RGB HSV Conversion ~ nonlinear http://www.mathworks.com/acces s/helpdesk/help/toolbox/images/co lor10.shtml ENEE631 Digital Image Processing (Spring'06) Lec2 HVS [17] ENEE631 Digital Image Processing (Spring'06) Lec2 HVS [18] UMCP ENEE631 Slides (created by M.Wu 2001/2004) Representation by Three Primary Colors Any color can be reproduced by mixing an appropriate set of three primary colors (Thomas Young, 1802) Three types of cones in human retina Absorption response S i (λ) has peaks around 450nm (blue), 550nm (green), 620nm (yellow-green) Color sensation depends on the spectral response {α 1 (C), α 2 (C), α 3 (C) } rather than the complete light spectrum C(λ) C(λ) color light S 1 (λ) C(λ) d λ S 2 (λ) C(λ) d λ S 3 (λ) C(λ) d λ α 1 (C) α 2 (C) α 3 (C) Identically perceived colors if α i (C 1 ) = α i (C 2 ) ENEE631 Digital Image Processing (Spring'06) Lec2 HVS [19] UMCP ENEE408G/631 Slides (created by M.Wu & R.Liu 2002/2004) Example: Seeing Yellow Without Yellow 570nm 520nm mix green and red light to obtain perception of yellow, without shining a single yellow photon 630nm Seeing Yellow figure is from B.Liu ELE330 S 01 lecture notes @ Princeton; R/G/B cone response is from slides at Gonzalez/ Woods DIP book website ENEE631 Digital Image Processing (Spring'06) Lec2 HVS [20] = 5
Color Matching and Reproduction Mixture of three primaries: C = Sum(β k P k (λ) ) To match a given color C 1 adjust β k such that α i (C 1 ) = α i (C), i = 1,2,3. Tristimulus values T k (C) T k (C) = β k /w k w k the amount of k th primary to match the reference white Chromaticity t k = T k / (T 1 +T 2 +T 3 ) t 1 +t 2 +t 3 = 1 visualize (t 1, t 2 ) to obtain chromaticity diagram Chromaticity Diagram Chromaticity Diagram Color gamut Reference white Line of purples ENEE631 Digital Image Processing (Spring'06) Lec2 HVS [21] ENEE631 Digital Image Processing (Spring'06) Lec2 HVS [22] UMCP ENEE631/408G Slides (created by M.Wu & R.Liu 2001/2002) CIE Color Coordinates (cont d) CIE XYZ system hypothetical primary sources to yield all-positive spectral tristimulus values Y ~ luminance Color gamut of 3 primaries Colors on line C1 and C2 can be produced by linear mixture of the two Colors inside the triangle gamut can be reproduced by three primaries From http://www.cs.rit.edu/~ncs/color/t_chroma.html ENEE631 Digital Image Processing (Spring'06) Lec2 HVS [23] RGB Primaries and Color Representation Use red, green, blue light to represent a large number of visible colors The contribution from each primary is normalized to [0, 1] Color-cube figures: left figure is from B.Liu ELE330 S 01 lecture notes @ Princeton, right figure is from slides at Gonzalez/ Woods DIP book website ENEE631 Digital Image Processing (Spring'06) Lec2 HVS [24] 6
UMCP ENEE408G/631 Slides (created by M.Wu & R.Liu 2002/2004) Color Coordinate for Printing Primary colors for pigment Defined as one that subtracts/absorbs a primary color of light & reflects the other two CMY Cyan, Magenta, Yellow Complementary to RGB Proper mix of them produces black Figure is from slides at Gonzalez/ Woods DIP book website (Chapter 6) Examples RGB HSV YUV ENEE631 Digital Image Processing (Spring'06) Lec2 HVS [25] ENEE631 Digital Image Processing (Spring'06) Lec2 HVS [26] Color Coordinates Used in TV Transmission Facilitate sending color video via 6MHz mono TV channel YIQ for NTSC (National Television Systems Committee) transmission system Use receiver primary system (R N, G N, B N ) as TV receivers standard Transmission system use (Y, I, Q) color coordinate Y ~ luminance, I & Q ~ chrominance I & Q are transmitted in through orthogonal carriers at the same freq. YUV (YCbCr) for PAL and digital video Y ~ luminance, Cb and Cr ~ chrominance Color Coordinates RGB of CIE XYZ of CIE RGB of NTSC YIQ of NTSC YUV (YCbCr) CMY ENEE631 Digital Image Processing (Spring'06) Lec2 HVS [27] ENEE631 Digital Image Processing (Spring'06) Lec2 HVS [28] 7
Summary Monochrome human vision visual properties: luminance vs. brightness, etc. image fidelity criteria Color Color representations and three primary colors Color coordinates Next time: Pixel-based operation Reading Assignment: Chapter 2.1-2.2; 6.1-6.2 Of Gonzalez s book (or) Chapter 3 of Jain s book; Sec.1.1 of Wang s book ENEE631 Digital Image Processing (Spring'06) Lec2 HVS [29] 8