Humans used a web interface to say same person or different person for a large set of faces. Several computer programs made the same comparisons

Transcription

1 OPTO 6124 Perception Scott Stevenson Image Segmentation What is really behind so many perception demos? Perception demos show us that our visual understanding of the world involves a lot of filling in of information in order to reach knowledge of objects and their relationships. Often, visual information is sparse or ambiguous. The demos are surprising to us because we don t often realize how much guesswork or top down processes are involved. The process of finding objects in an image is called Image Segmentation. Perception is a hard problem to solve! In 1966, Artificial Intelligence pioneer Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to spend the summer linking a camera to a computer and getting the computer to describe what it saw. (Szeliski 2009, Computer Vision) This turns out to be one of the hardest problems in AI. As an example, compare the ability of humans and computers to match faces across age. Andy Adler of Carleton University in Ottawa CA ran an experiment on humans and machines to see which did better at matching faces. Humans used a web interface to say same person or different person for a large set of faces. Several computer programs made the same comparisons This plot shows the error rates for humans and machines. Lower numbers mean better performance. Humans (data shown by the dots) generally made few errors Each solid curve is a different program, and the newest programs make fewer errors. Software is just now catching up with humans at this basic, everyday task.

2 Image Segmentation. Image segmentation refers to the problem of sorting raw image data into distinct objects. Seeing things instead of just areas of color and orientation. In computer vision, this is recognized as an extremely difficult problem. For humans, it is almost effortless. Consider this image randomly pulled from the Internet using the search phrase kitchen drawer clutter If I ask you to reach for the slotted spatula, after a few moments of scanning you will identify the object, then orient your hand to grab the handle. Robots on assembly lines have to solve this problem when reaching for a part from a bin full of jumbled items. It is an extremely difficult problem for artificial intelligence.

3 Perception doesn t always get it right: The following images demonstrate how your own visual system sometimes makes errors or has conflicting answers or has difficulty with this problem. Sometimes, we see familiar shapes in random configurations, a phenomenon called pareidolia meaning wrong image. The Rorschach Test uses pareidolia to probe the psyche of patients in psychoanalysis. Rorschach s original image 3 Meow! (from Dr. Bedell, source unknown) Google image search, Jesus in a tortilla

4 We see organization in repeating patterns. These can shift and reorganize dynamically. Source unknown

5 We see shapes and ignore backgrounds, even if those backgrounds are shapes. Zoom in on the image, look between the pillars. This is a set of actual pillars at the Exploratorium in San Francisco

6 In situations with very poor information, it can take a long time to reach an organized perception. Do you see the animal? Dalmation from Marr 1982

7 Sometimes we fill in missing information, like edges that are probably there even though they have no contrast in the image. These are called illusory contours. Many people see a solid white triangle with distinct edges, lying in front of the outlined triangle and discs. The solid white triangle looks a little brighter than the background. Kanizsa Triangle by Gaetano Kanizsa, 1955

8 The inferences made about an image have a strong influence on how missing information gets filled in. Is the wire frame cube in front of black discs? Or is it behind a set of holes? Kanizsa Necker Cube, source unknown.

9 Impossible figures, like those made famous by MC Escher, pit local solutions to image segmentation against global solutions. In this figure, the missing information is shading, stereoscopic 3D, and motion parallax. Without these, the brain makes a best guess at the three dimensional shapes based on edges. The artist has cleverly created ambiguity from one end to another of the object. Source unknown. From a google image search on Multistable Perception