Contours, Saliency & Tone Mapping. Donald P. Greenberg Visual Imaging in the Electronic Age Lecture 21 November 3, 2016

Transcription

1 Contours, Saliency & Tone Mapping Donald P. Greenberg Visual Imaging in the Electronic Age Lecture 21 November 3, 2016

2 Foveal Resolution

3 Resolution Limit for Reading at 18" The triangle subtended by a 30 second angle Sine of 30 sec = sine of 1/120 deg Thus 18"sin(30 sec) = sin ( ) = = "

4 Time Dependent Responses

5 Time Response of Cones and Rods

6 The Significance of Movement-Sensitive Cells A picture is viewed by an observer while we monitor eye position and hence direction gaze. The eye jumps, comes to rest momentarily (producing a small dot on the record), then jumps to a new locus of interest. - David H. Hubel. EYE, BRAIN, AND VISION, 1988 Scientific American Books, Inc. p. 80.

7 The Significance of Movement-Sensitive Cells - David H. Hubel. EYE, BRAIN, AND VISION, 1988 Scientific American Books, Inc. p. 80.

8 The Significance of Movement-Sensitive Cells - David H. Hubel. EYE, BRAIN, AND VISION, 1988 Scientific American Books, Inc. p. 80.

9 Saliency Where do we look? Where should we have high resolution?

10

11 Vuillard, Self Portrait (1888)

12 Sunset at Sea, Renoir (1879)

13 Guido Reni, Saint Sebastian (1616)

14 Rene Magritte, The Human Condition 1933

15 Simultaneous Contrast

16

17 Receptive Fields urse%20materials/physiology%2 0101/Chapter%20Notes/Fall% /figure_10_39_labeled.jpg

18 Three identical disks pasted onto the photograph appear as different shades in different locations. Alan Gilchrist. Seeing in Black & White Scientific American vol. 18, no. 2, 2008.

19 Perception Edge Effects

20 The Eye Sees Contours Contours are so dominant in our visual perception that when we draw an object, it is almost instinctive for us to begin by sketching its outlines.

21

22 The Eye Sees Contours Contours are so dominant in our visual perception that when we draw an object, it is almost instinctive for us to begin by sketching its outlines. We see contours when there is a contrast, or difference, in the brightness or color between adjacent areas.

23

24 Contour and Contrast Rate of Discharge of nerve impulses produced by steady illumination of a single receptor, A, in the eye of the horseshoe crab Limulus is directly related to the intensity of the light. The nerve fibers from the receptor are separated by microdissection and connected to an electrode from an amplifier and a recorder. - Floyd Ratliff. Contour and Contrast, Scientific American June 1972, p. 93.

25 Contour and Contrast Inhibition of receptor, A, steadily exposed to moderate illumination is produced when neighboring receptors, B, are also illuminated. - Floyd Ratliff. Contour and Contrast, Scientific American June 1972, p. 93.

26 Contour and Contrast Lateral Inhibition in the eye of the horseshoe crab is strongest between receptors a short distance apart and grows weaker as the distance between receptors increases. - Floyd Ratliff. Contour and Contrast, Scientific American June 1972, p. 94.

27 Contour and Contrast Observed Brightness Curve obtained by psychophysical measurements (black line) has two sharp flections, one corresponding to the bright band and the other to the dark band. Measurement of actual luminance (colored line) across a halfshadow region reveals that the effect lies in the beholder and is not an objective phenomenon. - Floyd Ratliff. Contour and Contrast, Scientific American June 1972, p. 92.

28 Contour and Contrast Rapid Rotation of this disk will create the Cornsweet illusion. The white spur creates a local variation near the contour between the two zones. What will Happen? - Floyd Ratliff. Contour and Contrast, Scientific American June 1972, p. 96.

29 Contour and Contrast Rapid Rotation of this disk will create the Cornsweet illusion. The white spur creates a local variation near the contour between the two zones that causes the apparent brightness of the inner zone to increase. In the same way the dark spur creates a local variation that causes the outer zone to appear darker. Except in the spur region the objective luminance of the disk when it is rotating is the same in both the inner and the outer region. - Floyd Ratliff. Contour and Contrast, Scientific American June 1972, p. 96.

30 Contour and Contrast Source of Craik-O Brien Effect can be demonstrated by covering the contour with a wire or string. When this is done, the inner and outer regions appear equally bright. - Floyd Ratliff. Contour and Contrast, Scientific American June 1972, p. 97.

31 Contour and Contrast Luminance on both sides of the Craik-O Brien contour is the same but the inside appears brighter. The human visual system may extrapolate (colored curve) from the maximum and minimum produced by inhibitory processes (black curve at right). - Floyd Ratliff. Contour and Contrast, Scientific American June 1972, p. 98.

32

33

34 Le petit déjeuner Paul Signac ( ) This is a portion of his Le petit dejeuner ( ). Note how the shadow is darker near the unshaded tablecloth and lighter next to the dark matchbox. - Floyd Ratliff. Contour and Contrast, Scientific American June 1972, p. 90.

35 Contour and Contrast - Floyd Ratliff. Contour and Contrast, Scientific American June 1972, p. 90.

36 Contour and Contrast Korean Vase (18 th century) The moon appears to be brighter than the sky directly below it. The actual luminance is just the reverse. If only a portion of the moon and an equal portion of the sky about one moon diameter below it are viewed through two identical small holes in a paper so that the dark contour is masked, the moon appears darker than the sky. - Floyd Ratliff. Contour and Contrast, Scientific American June 1972, p. 99.

37 Contour and Contrast - Floyd Ratliff. Contour and Contrast, Scientific American June 1972, p. 97.

38

39 Our eyes are so good at seeing edges that we even see contours which don t exist. - Uncle Don

40 Subjective Contours

41

42 Subjective Triangles - Gaetano Kanizsa. Subjective Contours, Scientific American, April 1976, pp

43 Geometric Regularity - Gaetano Kanizsa. Subjective Contours, Scientific American, April 1976, pp

44 A. Dalla Costa

45 TONE MAPPING

46 Visual Adaptation luminance (log cd/m 2 ) displays starlight moonlight indoor lighting sunlight range of illumination scotopic mesopic photopic poor contrast no color low acuity good contrast good color high acuity

47 The tone reproduction problem the absolute range of environmental illumination is vast (~10-12 log units) the dynamic range of scenes can be large (~3-4 log units) the absolute and dynamic ranges of display devices are small (~ log units)

48 Tone Reproduction What is the appropriate mapping from simulated scene intensities to display intensities to produce a better image?

49 Threshold visibility Weber s Law DL L L+DL DL = kl Weber s law L

50 Tone Mapping Look-up Tables

51 Tone Mapping Look-up Tables A 3-Bit Image A 3-Bit with Pixel Values

52 Tone Mapping Look-up Tables Address Value

53 Tone Mapping Look-up Tables Address Value

54

55

56 Original Image Snake River

57 Modified Image

58 Modified Image

59 Modified Image

60 Burning A technique for selective lightening of an area of a print by giving it additional exposure. Usually accomplished by blocking the projected image, letting light fall only on the selected region.

61 Burning-in (Ansel Adams) Ansel Adams. THE PRINT, 1983, Trustees of the Ansel Adams Publishing rights Trust.

62 Dodging Holding back the light which reaches an area of an image during exposure to darken it. Usually done with the hands or a dodging tool.

63 Dodging (Ansel Adams) Ansel Adams. THE PRINT, 1983, Trustees of the Ansel Adams Publishing rights Trust.

64 Output Linear display response 0% 6.7% 13.3% 20% % 33.3% 40% 46.7% % 60% 66.7% 73.3% % 86.7% 93.3% 100% Input

65 Output Gamma 2.2 display response 0% 0.3% 1.2% 2.9% % 8.9% 13.3% 18.7% % 32.5% 41.0% 50.5% % 73.0% 85.9% 100% Input

66 Gamma Curve er/file:gammafunctiongraph.svg

67 Displaying high dynamic range scenes 10,000:1 dynamic range before linear mapping after visual mapping

68 Digital Image Manipulation Filter Types Box (Blurring) Triangle (Blurring) Gaussian (Blurring) Hat (Edge Sharpening)

69 Compute the weighted average of the selected points. Eliot Feibush, Marc Levoy, Robert L. Cook. Synthetic Tecturing Using Digital Filters, acm Siggraph, Computer Graphics, July 1980.

70

71

72

73

74

75

76

77 End...