Optics, perception, cognition. Multimedia Retrieval: Perception. Human visual system. Human visual system

Transcription

1 Multimedia Retrieval: Perception Remco Veltkamp Optics, perception, cognition Be aware of human visual system, perception, and cognition Human visual system Human visual system Optics: Rods for b/w Cones for color Rods have higher spatial resolution and are more light sensitive

2 Human visual system Optical truth perceptual truth Complex working One example: lateral inhibition solid: real dashed: perceived Mach bands, caused by lateral inhibition Optical truth perceptual truth Optical truth perceptual truth Human visual system favors generic interpretation over nongeneric With dots: generic interpretation is blocked Without dots: generic blue square Hermann grid: count the black dots

3 Color perception Bright, saturated colors emphasize Different objects of same color interpreted the same HVS cannot distinguish color of small objects Common color deficiency: red = green at low saturation and luminance Red area perceived as larger than same green area Red area perceived closer than blue area Color cognition Color has meaning e.g. red is danger Cannot instantly remember more than 7 meanings Size perception Perceived size depends on: Visual angle: all other things being equal, the object with largest visual angle seems larger Phenomenon of size constancy: objects of kown size seem to have constant size regardless small changes in distance Perspective: objects of same size in strong perspective appear larger at greater distance The Ames room Size illusion

4 Size illusion Size illusion The Ames room The Ames room Ponzo illusion: horizontal segments have same size Size illusion Table tops have same size and shape (print, cut, and lay left table top over right one) Size illusions Müller-Lyer illusion Both illusions caused by perspective animated

5 Distance perception Overlap (overlapping object closer) Color, intensity (fainter object further, atmospherical aberration) Size constancy Visual angle Texture gradient Perspective In real 3D: convergence; not on 2D screen Projection moves across retina Motion perception Eyes are moved, projection not Head is moved, projection not Motion illusion Phi phenomenon: two objects flashing on and off perceived as a single moving object Motion illusion 30 times per second perceived as smooth motion: film

6 Motion illusion Motion illusion Ouchi motion illusion: relative motion of inner w.r.t. outer part Ouchi motion illusion: relative motion of inner w.r.t. outer part Symmetry in human perception, a variant is more similar to its prototype, than the prototype is similar to its variant variant A: prototype B: Triangle inequality d(a,b)+d(b,c) d(a,c) However, in human perception, A can be similar to B, and B similar to C, but A need not be similar to C A: B: C: d(a,b) < d(b,a)

7 Gestalt Theory Initiated by Wertheimer The whole is more than the sum of the parts Goal: explain relation between patterns and their perceptual organization Gestalt Principles/Laws 1 Figure and ground: elements are separated based on contrast 2 Similarity: similar elements seen as group Gestalt Principles/Laws 3 Proximity/contiguity: elements close together seen as group 4 Continuity/continuation: preference for good continuation Gestalt Principles/Laws 6 Area: larger of two overlapping objects is seen as ground, smaller as figure 5 Closure: tend to see complete figures (Kanizsa triangle)

8 Gestalt Principles/Laws No strenght ordering among different Gestalt laws: no unambiguous perceptual organization Koffka introduced law of Prägnanz: conveying the essence of something See a shape pattern as being as regular, simple, or symmetrical as possible Not part of the theory: measure for Prägnanz Gestalt Principles/Laws 7 Prägnanz/Simplicity/Symmetry/Singularity: regions bound by symmetrical borders seen as coherent Perceptual Grouping Perceptual grouping For example on the basis of Gestalt principles: original logo: alternative human segmentations:

9 Perceptual Matching Hering illusion Red lines are staight and parallel Gestalt Illusions Wundt illlusion Blue lines are straight and parallel two geometrical partial matches: confusingly similar not confusingly similar Gestalt Illusion Gestalt Illusion Poggendorf illusion Zöllner illusion Poggendorf illusion Interactive: poggendorff_illusions.html

10 Ambiguity Ambiguity Young or old? Satellite image Ambiguity Ambiguity Satellite image or cow? Satellite image or cow?

11 Ambiguity Ambiguity Image Image or text? Ambiguity in art Dali: Slave Market Ambiguity in art Sandro del Prete: The Window Opposite Figure and ground Figure and ground

12 Ambiguity in art Escher: Ascending, descending; Waterfall Ambiguity in trademarks distance, size, perspective Perceptual features of sound events Musical note: pitch low-high: c. 90 categories duration long-short: multiples of 2 and 3 quantizing into categories loudness soft-loud; non-categorical timbre, tone quality categorical? (voice and instrument recognition) Perceptual grouping Sound events are organised in groups successive sounds form melodies simultaneous sounds form chords or harmonies Music with one sound event at a time is called monophonic Music with more than one sound event at a time is called polyphonic usually perceived as melody + chords less frequent: 2 or more melodies

13 Gestalt principles Low level principles: proximity rhythmic pitch similarity duration timbre continuity melodic These produce closure of wholes High-level principles paralellism intensification Example: Beethoven, 5 th symphony Bob Snyder, Music and Memory (2000) Melodic continuity In vision: a cross is interpreted as two straight lines In music, we tend not to hear crossings instead, pitch proximity principle dominates overcome by timbral differentiation Example: Tchaikovsky, 6 th Symphony first violin second violin together whole orchestra Image Retrieval Research: Profi Perceptually-relevant Retrieval Of Figurative Images Music Retrieval Research: Muugle Musical Utrecht University Global Lookup Engine original logo: alternative human segmentations: